Science in the Courtroom

Tuesday, January 24th, 2012

I’m currently working on a piece for Huffington Post on the latest prescription painkiller hysteria. In researching the piece, I found this 2010 Time piece by Maia Szalavitz on how post-mortem overdose diagnoses may be overstated.

The problem is that it’s difficult to isolated a particular drug as cause of death. So the rise in opioid-related overdose deaths that the CDC and numerous media outlets have been screaming about for the last few months could be the result of lots of people ODing on painkillers, or it could merely be that because more people are taking painkillers, more people are likely to have painkillers in their systems when they die. Hence, the use of the term “opioid-related” to describe these deaths. That allows panic-sowing without the need to establish any causal connection. (It’s similar to the way the government calculates “marijuana-related emergency room incidents.)

But the problem gets more urgent when we start using these diagnoses in court, as the government has done in the trials of doctors accused of contributing to a patient’s overdose death.

It’s here that the opinions of one of  Szalavitz’s sources seem particularly troubling.

Given the state of the science, then, should it be used in court? Ed Cheng, a professor of law at Brooklyn Law School and expert on scientific testimony, says, yes, noting that more research is still needed. “If we were to require studies and statistical assessment on every assertion, almost nothing would be able to be used in court. My view on this is that the question here is not throwing the baby out with the bathwater,” says Cheng. “It’s clear that the forensic sciences do not have as much of an empirical basis as we would like them to have. The question becomes how do we motivate them sufficiently to come up with the empirical basis that we want?”

In the Schneider case, which entered jury deliberations on Wednesday, the defense team sought and failed to prevent the jury from hearing testimony that it believed did not have sufficient scientific foundations. But according to Cheng, it may be preferable to let the jury hear both sides of the scientific dispute and make up their own minds. “I myself have floated between the poles on this,” he says. “I’m currently more on the ‘Let the jury hear it’ side. I’m not convinced that good science and bad science is always cut and dried.”

“Let the jury hear it” sounds great on its face. But there’s more to it than that. If the science linking a particular drug to a particular overdose isn’t established–if the scientific community is split over whether you can make that connection–then the jury shouldn’t hear it. (If nothing else, that would seem to establish reasonable doubt.)

Yes, we do have an adversarial judicial system. But lay juries aren’t trained scientists. Most people don’t know what to look for  when evaluating the veracity of some science-based claim. Get two scientific-sounding witnesses pitching the jury competing or mutually-exclusive theories, and the winner will more often be not who advocated the best science, but who was a better expert witness. Or more bluntly, who was a better salesman.

We’ve seen this over and over again with bite mark testimony. Frauds like Michael West have sold crap science to juries for years, sometimes unopposed, but often opposed by more credible experts. Even now, with a solid consensus in the forensics community that you can’t “match” bite marks in skin to one person to the exclusion of everyone else, we still see appeals courts shoot down post-conviction petitions on the grounds that the defense already challenged the state’s expert at trial, and the jury found the prosecution’s witness more convincing. It doesn’t seem to matter that we now know the prosecution’s witness was spewing pseudo-science hokum.

I think you could make a strong case that West was able to persuade juries because he didn’t sound scientific. I’ve read more than a few trial transcripts where West and the prosecutor would actually use an opposing expert’s credentials against him, contrasting him as a fancy out-of-town hired gun with a bunch of letters after his name with West, the local dentist just trying to do the right thing, helping put bad guys away with intuition, common sense, and some self-taught expertise. The scary thing is that when you see West in action, he sounds convincing, even when you know he’s a fraud.

Of course, West is only one example (although he is one of the most egregious). I don’t know the best way to determine what science has reached enough of a consensus to be used in a courtroom, but leaving the decision to individual juries on a case-by-case basis seems like a bad idea. In the federal courts, and in much of the country, challenges to scientific evidence are currently resolved by the judge in what’s called a Daubert hearing. From my understanding, while those hearings have done a decent (but far from perfect) job keeping junk science out of civil cases, the process has been less successful at keeping it out of criminal cases.

Skeptical as I am of blue ribbon commissions, this may be one area where we’re best off having an established, accredited panel of specialists set policy.

Digg it |  reddit |  del.icio.us |  Fark

17 Responses to “Science in the Courtroom”

  1. #1 |  B | 

    Skeptical as I am of blue ribbon commissions, this may be one area where we’re best off having an established, accredited panel of specialists set policy.

    I actually don’t even think getting a bunch of scientists together to set a broad policy or set of standards for what constitutes “good” and “bad” science would solve any problems. The very nature of (real) science is that the facts themselves are fluid: or more precisely, the “facts” are really “our best possible interpretation of data collected to date”. Setting a broad policy based on science is very difficult, because science is itself undergoing continuous revision. Policy would have to be continually revised as well to keep up. Our political and justice systems are poorly suited to this.

    As a scientist, I sometimes find myself arguing against points I made myself just a few years ago. That’s how fast it can move.

    An alternative might be to set up a mechanism of scientific review of scientific evidence on a case-by-case basis, probably something that would be a sort of hybrid between a grand jury and an NIH study section. This would have to be completely independent of the legal review for admissibility to be truly scientific.

    I don’t know the law well enough to say how feasible that sort of thing really would be, but I imagine that there would be significant constitutional issues with that, both in terms of the independence of the judiciary and the integrity of the jury system.

  2. #2 |  Burgers Allday | 

    A couple years ago I had my expert challenged in a patent case on Daubert grounds. The expert was trying to testify that if you had two similar parts, made of similar material that the one with the smaller cross-section would break more easily and with less applied force. Expert was highly credentialed, long time Mech E professor, etc., etc. Court wouldn’t take his testimony because it did not believe that the expert could tell that the smaller version would break more easily just by eyeballing. It was not clear to me whether the court didn’t think he could tell which piece was smaller (it was easy to see) or if the court didn’t think he could make the predicative statement that the smaller one would break easier because it was smaller. We won without the expert, so I am not too broken up about it.

    I tell this story bcs, at the time I was researching to try to keep the expert’s statement in, it was hilarious to read the Daubert progeny cases that involved law enforcement experts. The precedents (primarily Daubert) were the same, but the law enforecement expert’s testimony always, always got in. “Scientific method” means a completely different when the science is law enforcement related. I always think of the Mr. Show sketch with the hunchback and Bob Odenkirk as the wizard.

  3. #3 |  B | 

    #2’s story reminds me to add: scientific review of scientific evidence would also have to be COMPLETELY independent of law enforcement and the DA. Though I suppose that nearly goes without saying among this crowd.

  4. #4 |  Dave Krueger | 

    Lung cancer statistics are distorted the same way as the drug stats. It makes the record keeping easier and the stats help support the latest health crusade. It works for drunk driving accidents, too. If you were drunk and had an accident, then the accident must have been caused by your being drunk.

  5. #5 |  C. S. P. Schofield | 

    I think that, by and large, the ‘let the jury hear everything’ attitude is marginally preferable to the ‘the jury pool is too full of morons, so we need to have science vetted by experts’ attitude. Marginally. But my opinion is colored by a cynical feeling that the vetting of science by ‘experts’ will be run by the State, and in consequence all kinds of prosecution- friendly frauds will get a pass.

  6. #6 |  Burgers Allday | 

    Off topic: There was a bad Supreme court decision yesterday:

    http://www.supremecourt.gov/opinions/11pdf/11-208.pdf

  7. #7 |  BBCC | 

    Whenever I hear about legislators trying to “fix” the painkiller problem, I always wish that they could experience what being in pain – every day, all the time – is like. I’m thinking their tune would change quickly.

  8. #8 |  BamBam | 

    “we still see appeals courts shoot down post-conviction petitions on the grounds that the defense already challenged the state’s expert at trial, and the jury found the prosecution’s witness more convincing. It doesn’t seem to matter that we now know the prosecution’s witness was spewing pseudo-science hokum”

    Procedure was followed — that is all that matters. NEXT!

  9. #9 |  Michael Chaney | 

    Frauds like Michael West…

    Preach it, brother!

  10. #10 |  BamBam | 

    But my opinion is colored by a cynical feeling that the vetting of science by ‘experts’ will be run by the State, and in consequence all kinds of prosecution- friendly frauds will get a pass.

    It already happens. There is essentially a process for becoming “licensed” to appear as an expert witness (as determined by The State, with requisite fees). These witnesses are known to lawyers, and there is a “expert witness circuit” (for lack of better sarcastic words) that they all travel. One could even half-jokingly state that there is an “expert witness” ranking system, full of scouts, draft ratings, 40 yd dash scores, etc.

  11. #11 |  Irving Washington | 

    If it makes anyone feel any better, Cheng is way, way outside the mainstream of legal theory. One of the threshold questions that makes expert testimony admissible is whether the jury needs expert analysis of the evidence. If the expert can’t say that the science, reliably and repeatedly, makes a fact more or less likely, his testimony should be excluded.

  12. #12 |  Burgers Allday | 

    search warrants sealed in Matthew Stewart case:

    http://www.sltrib.com/sltrib/news/53366697-78/search-police-warrants-court.html.csp

  13. #13 |  James D | 

    This issue is a real problem. My wife worked with setting up clinical trials for years and said it was so hard to find good candidates because they wanted someone who was as healthy as possible and took little (or no) other medication. Because once a “side effect” or “death” is attached to a drug, then that drug will forever be linked with that, when more than likely some other medication or the patients general lack of health is the real culprit. So you could have drug XYZ that could cure cancer, but if test patients A, B, and C all have existing bad conditions (such as, I don’t know, stage 4 CANCER) and have a side-effect or die, then that drug with be unfairly linked.
    Like the first post points out, science(/medicine) is more fluid than people realize. People think science helps set things in stone (A causes B, X = Y), but in many cases it’s more like “for what can see and prove at this moment in time, A appears likely to be a cause for B” and “X, under the current testing environment, seems to be equivalent to Y”.

  14. #14 |  Jay | 

    If it floats, its a witch! Expert testimony has proven that witches are made of wood.

    That doctors evil pills turned me into a newt! [I got better]

  15. #15 |  H man | 

    The scary thing is that when you see West in action, he sounds convincing, even when you know he’s a fraud.

    Is this the sign of a psychopath or a sociopath? I don’t really know what the distinction is between the two.

  16. #16 |  Goober | 

    #7 FTW. I’ve lived in pain. Since I’ve done it, I can say without any reservations that if you haven’t been in debilitating pain (defined as pain so bad it is all that you can think about) for more than 6 consecutive months in your life, then you have no freaking idea what it is like. I sure didn’t, but I thought I did. Trust me, you have no idea. If you haven’t, then you have no effing right whatsoever to even begin to talk about regulating pain killers. Period.

    These people want to pretend that they are all compassionate and save lives, but none of them really understand what it means to live a life that isn’t worth saving – to lay on the floor in your living room, unable to pray to God to kill you, please, just let me die! because it hurts too bad to actually put those thoughts together. None of them understand that lot of these painkiller overdoses are actually acetominophen overdoses because they put liver poison into the pain killers to keep people from abusing them. They would rather have you dead than high, you see.

    I wouldn’t wish it upon any of them, but i do wish that they could understand so that they stop what they are doing and back off a little.

  17. #17 |  Goober | 

    @ 15 – there is no difference. They are the same thing – interchangeable terms to describe the same medical condition.

Leave a Reply