The perils and pitfalls of forensic science

Both the New York Times and TalkLeft took issue yesterday with the oft-unacknowledged imperfections in forensic science.

The New York Times picks up on the National Academy of Sciences report that we've mentioned before, and how a cross-section of scientists and government officials back the recommendations that the NAS put forward.

Barry Fisher, a past president of the American Academy of Forensic Sciences and a former director of the crime laboratory at the Los Angeles County Sheriff’s Department, said he and others had been pushing for this kind of independent assessment for years. “There needs to be a demonstration that this stuff is reliable,” he said.

It’s not that there hasn’t been any research in forensic science. But over the years much of it has been done in crime labs themselves. “It hasn’t gotten to the level where they can state findings in a rigorous scientific way,” said Constantine Gatsonis, director of the Center for Statistical Sciences at Brown University and co-chairman of the National Academy of Sciences committee. And rather than being teased out in academic papers and debated at scientific conferences, “a lot of this forensic stuff is being argued in the courtroom,” Mr. Fisher said. “That’s not the place to validate any kind of scientific information.”

Meanwhile, TalkLeft discussed the unreliability of fingerprint evidence when analysts are given a "context" for a print. For example, scientists might be given a latent print, and then given a print for comparison and told it was taken from a suspect. Human beings are suggestible, and because this analyst is a human being, they're statistically more inclined to find that the prints match, independent of whether they actually do.

He has conducted studies that show that when working on an identification, fingerprint examiners can be influenced by what else they know about a case. In one experiment, he found that the same examiner can come to different conclusions about the same fingerprint, if the context is changed over time.

The same kinds of contextual biases arise with other decision-makers, said Dr. Dror, who works with the military and with financial and medical professionals. He thinks one reason forensic examiners often do not acknowledge that they make errors is that in these other fields, the mistakes are obvious. “In forensics, they don’t really see it,” he said. “People go to jail.”

Finally, TChris at TalkLeft had a good analogy:

Just as police officers conducting lineups should not be told whether the suspect is or isn't among those who are lining up, forensic scientists who are asked to match a fingerprint shouldn't be told whether the print is suspected to have been left by a particular person. Any other standard raises concerns about the objectivity of the analysis.