The perils and pitfalls of forensic science

Both the New York Times and TalkLeft took issue yesterday with the oft-unacknowledged imperfections in forensic science.

The New York Times picks up on the National Academy of Sciences report that we've mentioned before, and how a cross-section of scientists and government officials back the recommendations that the NAS put forward.

Barry Fisher, a past president of the American Academy of Forensic Sciences and a former director of the crime laboratory at the Los Angeles County Sheriff’s Department, said he and others had been pushing for this kind of independent assessment for years. “There needs to be a demonstration that this stuff is reliable,” he said.

It’s not that there hasn’t been any research in forensic science. But over the years much of it has been done in crime labs themselves. “It hasn’t gotten to the level where they can state findings in a rigorous scientific way,” said Constantine Gatsonis, director of the Center for Statistical Sciences at Brown University and co-chairman of the National Academy of Sciences committee. And rather than being teased out in academic papers and debated at scientific conferences, “a lot of this forensic stuff is being argued in the courtroom,” Mr. Fisher said. “That’s not the place to validate any kind of scientific information.”

Meanwhile, TalkLeft discussed the unreliability of fingerprint evidence when analysts are given a "context" for a print. For example, scientists might be given a latent print, and then given a print for comparison and told it was taken from a suspect. Human beings are suggestible, and because this analyst is a human being, they're statistically more inclined to find that the prints match, independent of whether they actually do.

He has conducted studies that show that when working on an identification, fingerprint examiners can be influenced by what else they know about a case. In one experiment, he found that the same examiner can come to different conclusions about the same fingerprint, if the context is changed over time.

The same kinds of contextual biases arise with other decision-makers, said Dr. Dror, who works with the military and with financial and medical professionals. He thinks one reason forensic examiners often do not acknowledge that they make errors is that in these other fields, the mistakes are obvious. “In forensics, they don’t really see it,” he said. “People go to jail.”

Finally, TChris at TalkLeft had a good analogy:

Just as police officers conducting lineups should not be told whether the suspect is or isn't among those who are lining up, forensic scientists who are asked to match a fingerprint shouldn't be told whether the print is suspected to have been left by a particular person. Any other standard raises concerns about the objectivity of the analysis.

Houston man, victim of prosecutorial misconduct, could be freed on bond

An article in the Houston Chronicle a few days ago told the story of Gary Alvin Richard, who was convicted of a rape and robbery in 1987. Richard has spent 22 years behind bars for what is now clearly a crime he did not commit. New blood-typing tests and recently-unearthed (withheld evidence, in this case) prosecutorial misconduct solidify that conclusion.

Both sides are asking a judge to overturn his conviction.

A jury convicted Gary Alvin Richard in a 1987 attack on a nursing student in a trial based largely on blood-typing evidence from the Houston Police Department crime lab. But, prosecutors and the defense attorney agree, new tests completed Friday show that an [Houston Police Department] analyst misled jurors at Richard’s trial and failed to report evidence that may have helped him.

Based on the new tests, both sides will ask a judge next week to release Richard on bond while they sort out what happened in his case...

Richard’s case abounds with issues common to wrongful convictions. Among them:

The victim identified him some seven months after the attack. HPD crime lab analysts came to conflicting conclusions about the evidence, but reported only the results favorable to the case. Physical evidence collected in what is known as a “rape kit” has been destroyed, a victim of poor evidence preservation practices, leaving nothing for DNA testing now.

Richard's case is of many that have come to light since the Houston Police Department initiated a review of past cases in October of 2007. That review was spurred "days after DNA evidence cleared Ronald Taylor of sexual assault in a case where HPD analysts performed faulty tests on body-fluid evidence." Kudos to Houston for reviewing its past cases with some genuinely desire for justice, but this episode also serves as a reminder of the importance of getting things right the first time.

Visit IPF's Website by clicking here; sign up to volunteer by clicking here; contribute to our work by clicking here.

Texas bill allows defendants to challenge their convictions based on junk science

Per the Innocence Project's blog today, "A bill passed by the Texas Senate this week would provide an avenue for prisoners to challenge convictions based on discredited forensic science." They reference a story in the Marshall News Messenger that begins,

Criminals who were sent to prison — or sentenced to death — based on discredited scientific evidence would be given a new way to challenge their convictions under a bill passed this morning by the Texas Senate.

In recent years, an increasing number of arson and gunshot convictions in Texas have triggered alarm as new technology proved earlier evidence wrong, and convictions were cast into doubt — including at least one case in which the prisoner was executed.

The measure by state Sen. John Whitmire, D-Houston, would allow discredited scientific evidence that figured in a criminal conviction to be considered by an appeals court in order to establish the innocence of a defendant...

Advancements in forensic testing — DNA, ballistics and arson — have led to new evidence being uncovered in several cases in Texas. Whitmire said that led him to file the bill, which clarifies how discredited scientific evidence can be used in court appeals.

In December 2008, we worked to overturn Jimmy Ates' conviction, based largely on fraudulent FBI bullet lead analysis. Since then, the National Academy of Sciences has issued a scathing report, decrying the sorry state of forensic science labs around the country.

We know well how junk science can contribute to a wrongful conviction – indeed, the Innocence Project in New York says that junk science contributed to over half of the nation's first 225 DNA exonerations. We applaud steps like these being taken in Texas, and hope that a new incredulity toward and accountability regarding forensic science will spread to other states and jurisdictions.

The Coverdell Grant Program for forensic science

The Innocence Project in New York recently released a report titled, "Investigating Forensic Problems in the United States: How the Government Can Strengthen Oversight through the Coverdell Grant Program." From the executive summary:

In 2004, Congress established an oversight mechanism within the Paul Coverdell Forensic Science Improvement Grant Program, which provides federal funds to help improve the quality and efficiency of state and local crime labs and other forensic facilities.

[...] Nearly five years after Congress passed legislation to help ensure that forensic negligence or misconduct is properly investigated, extensive independent reviews show that the law is largely being ignored and, as a result, serious problems in crime labs and other forensic facilities have not been remedied. In short, the U.S. Department of Justice’s Office of Justice Programs (OJP), which is responsible for the program, has failed to make sure that even the law’s most basic requirements are followed.

Yesterday, the blog for TheHill.com paraphrased some of the results of this study, and reiterated the Innocence Project's call for increased oversight or, rather, they called for the Obama administration to increasingly take advantage of the grant program that Congress created five years ago. One particularly egregious fact they quote is this: only 13% of designated oversight entities meet the federal law’s forensic oversight requirements. If you were a defendant, would you want to take a 1-in-8 chance that the forensic lab that processed the evidence in your trial was subject to proper oversight?

Finally, "Under new leadership, the Department of Justice can – and should – make sure crime lab problems are properly addressed, which will enhance the public safety and help prevent wrongful convictions." Remember, working to correct problems in order to preclude wrongful convictions is cheaper than housing wrongfully incarcerated individuals.


No sooner had The Hill run this post than Grits for Breakfast published some presentations from the public meetings held by the National Academy of Sciences, meetings held to address the problems plaguing forensic science labs around the country. They link to this presentation in particular that calls for forensic tests "to be as blind as possible, for as long as possible," and which contains the shocking graphic on common error rates linked above.

You'll notice that firearms and fingerprints, while among the most reliable forensic testing methods, still yield erroneous conclusions around 1-5% of the time. Some toolmark and bitemark tests, meanwhile, are reliable less than half of the time. That report also refers to several studies that found that, for example, when a scientist was provided with "context" for certain samples – context such as, "The suspect has already confessed, here's his hair and a hair from the crime scene" – that error rates were much higher. Those who conducted the psychological studies could induce false positives by giving false context, leading the forensic scientist to believe certain conclusions before they came to them independently.

All of these scientific studies point to the sad state of the crime labs in this country. Scientists might think so, but they are not immune to psychological tendencies – such as suggestibility – that afflict every human being. Independent oversight and common-sense reforms are the necessary solution to the problem.

Visit IPF's Website by clicking here; sign up to volunteer by clicking here; contribute to our work by clicking here.

UPDATE II: CBLA Amicus Brief

On Tuesday, IPF received the good news that the Florida Supreme Court sent the case of Thomas Wyatt, a death-row inmate, back down to the trial court for a new hearing on the Comparative Bullet Lead Analysis issue and the jailhouse snitch issue in Mr. Wyatt's case.

This was particularly important because they instructed the lower court to have a hearing on whether the CBLA testimony given at trial against Mr. Wyatt sufficiently prejudiced his trial given the fact that the FBI issued a letter recently saying that its testimony was false and not supported by science.

This ruling preserves the case-specific FBI letters as the paramount newly discovered evidence in these CBLA cases.

You can learn more about CBLA by reading this Washington Post article from 2007 or by reading our amicus brief by clicking here.

Visit IPF's Website by clicking here; sign up to volunteer by clicking here; contribute to our work by clicking here.

Update: Wyatt v. State Amicus brief

On Thursday, April 2nd, the Innocence Project of Florida, along with The Innocence Network, filed an amicus curiae brief with the Florida Supreme Court in the case of Wyatt v. State of Florida. IPF is urging the court to allow inmates, including Mr. Wyatt, the opportunity to present to the trial court recently received letters from the Federal Bureau of Investigation indicating that its agents gave false and misleading trial testimony regarding Comparative Bullet Lead Analysis (CBLA). The press release we posted earlier is now available for download by clicking here, or you can read the brief we filed by clicking here.

Visit IPF's Website by clicking here; sign up to volunteer by clicking here; contribute to our work by clicking here.

What if DNA evidence from a crime scene could tell us what the perpetrator looked like?

That question is the taken from a new post over at the Change.org Criminal Justice blog. The post is inspired by an article in the Wall Street Journal, that explains how researchers at Pennsylvania State University performed a study attempting to link specific genes to "phenotypes," or their physical expressions. What this means, then, is that scientists might be able to tell, roughly speaking, a person's physical characteristics by examining their DNA.

As Matt Kelley, the author of the post points out, there is much reason for alarm here. As he says, and as we have noted before, there is cause for concern over "the propensity of criminal justice agencies to use scientific methods before they're ready." He adds, "The U.K. and the Netherlands are already using some form of these tests, as are some U.S. states. Germany has outlawed the practice, along with Indiana, Wyoming and Rhode Island."

Lastly,

The WSJ reports that researchers are able to predict eye color 70-90% of the time and skin color 46% of the time. These numbers aren't strong enough to rely on. Wrongful convictions happen - and real perpetrators get away - when faulty or limited science puts police on the wrong track in those crucial first few days after a crime. I agree that these tests could be helpful in some cases to confirm other evidence, but can we guarantee that genetic composite sketches won't be become simply another form of unreliable forensic science?

The question is when and whether this will become more accurate than other methods of identification, and whether this technique can be used without any illusions of its accuracy. The worry is that juries might hear DNA evidence and assume the practice is close to 100% accurate, rather than the more pitiful, but more truthful, measure of its accuracy.

DNA clears men in Virginia and Washington

Two reminders today about why the arrival of reliable DNA testing is quite possibly the most important advent in the history of criminal justice. The Richmond Times-Dispatch out of Virginia is reporting that a man found guilty of a rape in 1984 was wrongfully convicted and incarcerated. DNA testing showed the true perpetrator to be one of the area's most notorious rapists, Leon Davis. Thomas Haynesworth was the man wrongfully convicted.

Reached last week by telephone at the Greensville Correctional Center, Haynesworth said of Davis, "I knew all along he was the man. I told my lawyer. I told [police]. He lived right down the street from me."

"I told them: 'This man fit the description.' But nobody ever listened to me," he complained. "Everybody said we looked alike. Only difference between me and him, he is taller and weighed more," said Haynesworth.

And cheers to the Virginia Department of Forensic Science:

The recent DNA testing was performed as part of the Virginia Department of Forensic Science's post-conviction project aimed at clearing people who may have been wrongly convicted from 1973 through 1988, before forensic DNA testing was widely available.

Similar programs should be in place in every jurisdiction throughout the country, as citizens anywhere can be victims of wrongful incarceration.

Secondly, in Washington state, a man who was arrested and held for 17 days for the rape of an 11-year-old girl in 2007 has filed a federal lawsuit against the local police department alleging wrongful imprisonment. David Lynch was released from prison, his charges dropped, after DNA tests from the rape excluded him as a possible rapist.

There are always celebrations fanfare when a person is released after serving X years behind bars for a crime they didn't commit. Sometimes we might forget that the real reason we work hard to shed light on these abuses and reform the system is to prevent wrongful convictions from happening in the first place. Though spending 17 days wrongfully in jail is certainly a harrowing experience, it is far and away a better thing than spending 27 years behind bars for a crime you didn't commit.

Update: David Lynch's story is similar to two others that were reported in Florida very recently. The first, out of Tampa, is of Kenneth Robinson, who was arrested for a burglary he did not commit and released after DNA testing proved as much. The second is of Wade Jackson in Brevard county, Florida, who was released after spending 17 months in prison as a suspect for a murder he did not commit.

Visit IPF's Website here; sign up to volunteer here; contribute to our work here.

Arson convictions "feeling the heat"

The recent report from the National Academy of Sciences tore into traditional forensic techniques used to analyze cases of arson, among other sciences. But it is not the first time that fire science has met resistance from experts in the field, or scientists outside it. An article in Miller-McCune magazine has a lengthy article detailing arson science's somewhat troubled history.

Even though much of the science has been discredited, Phoenix Attorney Larry Hammond points out the difficulties of getting a conviction overturned.

"Our legal system is designed to foreclose post-conviction review," [Hammond] noted, "and it does frustrate, and has frustrated, many of these cases." Attorney Walter Reaves, who works with the Innocence Project of Texas, agreed: "You have to convince a court that it is (newly discovered evidence), and then you have to convince them to actually listen and hear it."

DNA is the undisputed gold standard for exonerations, a virtually unassailable magic bullet. But arson convictions are a new frontier for exoneration work, and they are qualitatively different. If you find a bullet or knife in a dead man's back, no one disputes that a crime has taken place. Fires, however, are not so simple.

I found this following quote especially egregious:

Arson is the only crime for which someone can receive the death penalty based on the testimony of an expert witness whose education ended with high school.

And the article discusses possible suggestions for fixing the problem.

Common themes emerge when discussing reforms. One hot topic is the routine dependence on negative corpus evidence —simply put, investigators rule out electrical faults and exploding coffee pots, for example, rather than rule in evidence of how a fire did in fact start. So rather than a more accurate description of "cause undetermined," fires are often called arsons based on investigation by exclusion.

To veteran investigator Patrick Kennedy, that practice is unethical and immoral. "I don't know what it is, so it must be arson?" he said. "That is a pretty poor reason to kill somebody."

Reactions to the NAS Report: Roundup

We blogged earlier about the National Academy of Sciences report eviscerating the nation's forensic laboratories and the way scientific testimony is used and abused at trial. The report has received a good deal of attention in the papers, even Scientific American chimes in. This is a good sign; the national press is giving the report the attention it deserves.

The Innocence Project in New York has been doing a good job of cataloguing all of the press. Here are some of the best snippets.

Grits for Breakfast excerpts this, which is a good summary of the conclusions and highlights:

Rigorous and mandatory certification programs for forensic scientists are currently lacking, the report says, as are strong standards and protocols for analyzing and reporting on evidence. And there is a dearth of peer-reviewed, published studies establishing the scientific bases and reliability of many forensic methods. Moreover, many forensic science labs are underfunded, understaffed, and have no effective oversight.

Forensic evidence is often offered in criminal prosecutions and civil litigation to support conclusions about individualization -- in other words, to "match" a piece of evidence to a particular person, weapon, or other source. But with the exception of nuclear DNA analysis, the report says, no forensic method has been rigorously shown able to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source.

...there has been little rigorous research to investigate how accurately and reliably many forensic science disciplines can do what they purport to be able to do. In terms of a scientific basis, the disciplines based on biological or chemical analysis, such as toxicology and fiber analysis, generally hold an edge over fields based on subjective interpretation by experts, such as fingerprint and toolmark analysis. And there are variations within the latter group; for example, there is more available research and protocols for fingerprint analysis than for bitemarks.

Nuclear DNA analysis enjoys a pre-eminent position not only because the chances of a false positive are minuscule, but also because the likelihood of such errors is quantifiable, the report notes. Studies have been conducted on the amount of genetic variation among individuals, so an examiner can state in numerical terms the chances that a declared match is wrong. In contrast, for many other forensic disciplines -- such as fingerprint and toolmark analysis -- no studies have been conducted of large populations to determine how many sources might share the same or similar features. For every forensic science method, results should indicate the level of uncertainty in the measurements made, and studies should be conducted that enable these values to be estimated, the report says. [emphasis mine]

And here's Simple Justice, with a good – although cynical – take on the report and its possible impact on society.

So the cat is out of the bag. Decades of reliance on forensics are now in doubt. But this isn't the first time, and won't be the last. As scientific knowledge and tools developed, we kept getting the "new improved" version of whatever tools existed to prove facts. Of course, if a scientific tool was so reliable that it could put a man in prison for life, what did it say when a new tool came along that was that much more reliable? Courts embraced the new tool, confident that it was the latest and greatest in scientific proof, and lost no sleep over the demise of yesterday's absolutely certain scientific method. No one got hot and bothered by the conundrum.

So the NAS rips existing forensic practice to shreds and offers a roadmap to correcting junk science going forward. In a rational world, somebody would ask, "What do we do with all those people who have been convicted on science that we now know to be so flawed as to be unreliable, or at best cannot be sufficiently certain is reliable to admit as evidence in court?" By somebody, I mean a judge.

The answer is that the prison doors will not be thrown open, with tens of thousand, hundreds of thousand, told that they are free to leave.

I think GenPop also has a good "So what?" that fits well with my general belief about the holism of common prosecutorial tactics. Courtney says that the report's conclusions,

combined with the fact that eyewitness statements are often faulty, and jailhouse snitches are just doing it for the sentencing break, pretty much leaves us with DNA testing. And yet, people are continually convicted based on these types of fallible evidence. We are going to be setting free innocent people until the end of time, it seems.

We can hope President Obama takes the report's recommendations to hear, including creating a National Institute for Forensic Science.

National Academy of Sciences Blasts Nation's Crime Labs

A report being released this month by the National Academy of Sciences is expected to be "a sweeping critique of many forensic methods that the police and prosecutors rely on," according to The New York Times last week.

People who have seen it say it is a sweeping critique of many forensic methods that the police and prosecutors rely on, including fingerprinting, firearms identification and analysis of bite marks, blood spatter, hair and handwriting.

The report says such analyses are often handled by poorly trained technicians who then exaggerate the accuracy of their methods in court.

Its authors recommend the creation of a federal agency "to finance research and training and promote universal standards in forensic science, a discipline that spans anthropology, biology, chemistry, physics, medicine and law." Oh, and it also "calls for tougher regulation of crime laboratories." Hmmm....is it a surprise then to find out that the report's publication was delayed by opposition from law enforcement?

What is it with these guys? The reason for using science in investigations is to find the truth -- not to exaggerate it, distort it or create it, but to find it. Why do so many law enforcement personnel and prosecutors react so negatively to attempts to discover the truth about a crime? In the words of The King, it never ceases to amaze me that we continue to get resistance from certain State Attorneys to DNA testing, and even AFTER we have exclusionary DNA results in a case, they continue to resist. I mean, what exactly is it that THEY are seeking? It's certainly not the truth, or they would welcome testing if the evidence is available.

And what really got my ticker going was learning that

the National Institute of Justice, a research arm of the Justice Department, tried to derail the forensic study by refusing to finance it and demanding to review the findings before publication. A bipartisan vote in Congress in 2005 broke the impasse with a $1.5 million appropriation.

That's our National Institute of Justice, a misnomer if I've ever heard one. There is so much wrong with this that words fail me. I'm just sitting here shaking my head.

It's hard to say how much of an impact the report will have. The National Academy certainly shook things up in 2004 when they issued a report discrediting the practice of "matching the chemical signature of lead in bullets at a crime scene to similar bullets possessed by a suspect." The F.B.I. ended up having to contact hundreds of people who might have been wrongfully convicted due to this bogus science. One of them, Jimmy Ates, is here in Florida. Jimmy is now out on bond and waiting to see if the State will re-try him.

"Everyone interviewed for this article agreed that the report would be a force of change in the forensics field." We can only hope.

Read the entire article here.

Visit IPF's Website here; sign up to volunteer here; contribute to our work here.