Another Crime Lab Under Fire

Tuesday, June 21st, 2011

This time in Indiana:

Nearly one of every three cocaine tests conducted for criminal cases by the Indiana Department of Toxicology from 2007-2009 were bad, according to audit results released today by Indiana University.

The findings follow the release in April of audit results that revealed a 10 percent error rate in marijuana tests during the same period.

The lab has been operated by Indiana University since 1957, but will become a standalone state agency later this year. The move comes as a result of legislation passed this year by the General Assembly in the wake of growing concerns about problems at the lab, including long waits for results and questions about the accuracy of results . . .

MacIntyre said the bulk of the bad tests results were from 2007 and 2008, with “most of the problems resolved in 2009.” That would coincide with the arrival of a new lab director, Michael Wagner, who was the first forensic toxicologist to head the lab in more than a decade. Wagner resigned — under pressure, he contends, from IU officials — as concerns about the lab mounted.

The audit will now move to alcohol tests results from 2007-2009, which MacIntyre said will include the largest number of tests to be reviewed.

But the Indianapolis Star reported last month that the problems go back even further:

The Indianapolis Star reported Tuesday that about 2,000 emails it obtained from the lab show the agency was beset by incorrect test results from 2004 to 2006. A current audit of the lab’s work is covering only 2007 to 2009.

The lab tests blood and urine samples for evidence in criminal cases. The emails obtained by The Star show inadequate staffing and funding produced an environment in the lab ripe for errors, including the kind that could lead to people being denied justice, or escaping it.

The emails are correspondence to and from Peter Method, who served as the acting director of the department from 2003 to 2008. They suggest benign neglect on the part of the Indiana University School of Medicine, which did not authorize an audit of test results until 2008, at least four years after the first testing errors were reported by email.

One of the most telling notes was written by the lab’s supervisor in November 2006 to Method and Method’s supervisor, Michael Vasko, chair of the medical school’s Department of Pharmacology & Toxicology.

“I never had this (happen), error after error. … I guess if this is acceptable to you and the department, then I don’t have to worry. If an error occurs again in the future, I won’t bother you anymore,” the supervisor wrote.

The audit has already shown the lab sent out, on average, a flawed marijuana result every 3.28 days and a false positive marijuana result once every 18 days.

Among the more important revelations from the email correspondence are IU allowed the lab to languish under Method from 2003 to 2008, even though he acknowledged he was underqualified for the job.

They also show that while Method reported to Vasko, IU said Vasko’s job was merely broad “administrative oversight” — leaving the lab to Method as problems mounted.

Based on the emails alone, The Star found documentation of 26 bad test results from 2004 to 2006 that were reported to law enforcement.

Of those, 12 were false positives — findings that might have compromised the rights of Indiana residents. The other 14 were false negatives that might have prevented law enforcement from charging guilty people.

The error rates were seeing in these stories are jaw-dropping, especially when you consider the potential consequences of those errors. These stories ought to be a huge national scandal.

Note that this particular lab was affiliated with a university; it didn’t fall under the auspices of a law enforcement agency. The latter is especially problematic, but you don’t correct these problems simply by giving a lab more independence. The underlying problem is that we’re producing evidence that’s presented in court as “science” under conditions that fall well short of that definition.

This is also another data point in the case for a system of rivalrous redundancy. I think it’s safe to say that you’re considerably less likely to fall into a rut of “benign neglect” if you know your results will be regularly double-checked by another lab.

Digg it |  reddit |  del.icio.us |  Fark

22 Responses to “Another Crime Lab Under Fire”

  1. #1 |  Ken | 

    Or, to use government employee terms, “meets expectations.”

  2. #2 |  Jay | 

    Toxicology is pretty well founded scientifically, and generally uses a lot of the same equipment and processes you find in analytical chemistry research. These articles don’t seem to say anything about what sort of mistakes where made (but the fact that they could be figured out years later, after the samples would’ve been discarded or degraded) suggests they were paperwork/analysis problems. That’s not evidence the science is bad, just that the people were incompetent.

    So, yes, it argues very strongly for the rivalrous redundancy. It would’ve been picked up a lot sooner even if the defense lawyers in Indiana regularly had the testing confirmed privately.

    By the way, you could add Colorado Springs to the list of cities which have had problems: http://www.gazette.com/articles/report-97354-police-discuss.html

  3. #3 |  SJE | 

    If Indiana University is semi-independent from the state, it should be easier to deny these administrators sovereign immunity.

    If a private lab was so unconcerned about pathology results, and someone died as a result, there would be hell to pay.

  4. #4 |  Leon Wolfeson | 

    Ken: No, they just got what they paid for from private enterprise.

    See, I can toss slogans around as well. Again, taking these things OUT of local control, and having standards enforced from the top, with testing, is the only reasonable thing to do when NO mistakes are tolerable.

  5. #5 |  SJE | 

    If you read some of the emails, you see
    (1) pressure from prosecutors (e.g. your test came back negative, but we are sure that the driver was drunk or on drugs)
    (2) That the politicians etc seemed concerned about turn around times etc (shouldnt accuracy be more important?)
    (3) There were insufficient quality controls to check employees.
    (4) the director was trying to get on top of things but there were multiple considerations and CYA.

  6. #6 |  Andrew S. | 

    Obviously, if the state had just given the crime lab more money, this never would have happened!

    /Huffington Post’ed

  7. #7 |  Radley Balko | 

    Again, taking these things OUT of local control, and having standards enforced from the top, with testing, is the only reasonable thing to do when NO mistakes are tolerable.

    Huh? This was a university lab, that got tests from all over the state. There were no local control issues.

    Nor were the local control issues at the numerous state crime labs that have had problems in recent yeras.

  8. #8 |  Mattocracy | 

    Leon,

    What makes you think there weren’t standards before hand? There were and they obviously were ignored. You can have all the rules in the world but that’s just more guidelines for personell ignore. Cause ya know, you can’t just make people do something just because you have fucking rules and standards.

    You know how you get people to do shit right? You fire people for fucking up. If it sends innocent people to prison, you get charged with a crime. That’s how you get people to be accountable.

    And where were private labs ever mentioned?

  9. #9 |  SJE | 

    Leon: if you look at the emails, you can see that there WAS a lot of “top down” forcing of standards. Unfortunately, those standards include (a) doing more with less (b) not making the politicians look bad and (c) being helpful to the prosecutors when a result came back negative.

  10. #10 |  AnonymousCoward | 

    I work as a psychologist at a facility for the chronically mentally ill, we end up dropping every new employee, every new resident, every resident we suspect of relapse, and periodic drops on every resident in early sobriety. We don’t use a state lab at all, yet we’ve switched services several times in the last few years. Why? Because we’ve yet to find a service that doesn’t yield a 10% error rate. The science behind toxicology is strong, but the actual labs doing the work tend to be terrible.

  11. #11 |  Rifleman | 

    Assuming their website is accurate, here’s a staff listing: http://isdt.iusm.iu.edu/staff/

    Two things strike me:

    1) More administrative staff than analysts?
    2) Only seven analysts for samples from all over the state? Without knowing all the context, it seems on the surface that the ‘insufficient staffing’ argument may be legitimate. Not that it makes the current situation any better.

  12. #12 |  Leon Wolfeson | 

    Radley; They’re local labs. Not ones which are part of a consistent, national network. The *problem* is that you have these independent labs scattered around!

    Mattocracy; There may have been “standards”. However, we’re meaning totally different things. You’re talking about bits of paper. I’m meaning a prescriptive set of testing, as used in industries where failure *cannot be tolerated*.

    This means a consistent service, with identical procedures. And checks and guards at every level to enforce them. Testing for consistency within the lab. Testing for consistency between labs. Audits…

    This is totally different from suffering LOCAL political interference, which of course interferes with the proper proceedings of crime labs. That’s not “top down”, it’s “local meddling” with issues which are identical nationally! A single, nationwide set of standards set by a board of professional leaders can resist that kind of meddling.

    AC: Yep, and outside the centralised crime lab in the UK (which the Tories are shutting), *exactly* the same thing is seen here.

  13. #13 |  Leon Wolfeson | 

    Look – Where the Government has a mandate to do something (and yes, I think there are a shitload of things it should get out of), and criminal justice is undeniably one of those, it should do it *properly*.

    (Oh, and no, sorry – I have no time for subdivisions within a state wanting special powers where there isn’t a good argument for them)

    The concept that paying a profit margin for lab testing, where the important criteria is consistency (state labs should only be doing established tests, if *defendants* want to pay for cutting edge testing, that’s another matter entirely)…I am not seeing any reason for it. There is steady, non-creative work where a bureaucracy is indeed best for ensuring consistent results. That means a centrally-run network.

  14. #14 |  dave smith | 

    Ah, well. Kill ‘em all and let God sort ‘em out. Their guilty of something or they would not have their stuff in a crime lab to begin with.

  15. #15 |  mattt | 

    Seems to me that if you have multiple independent crime labs competing with each other, but competing mostly for state contracts, the incentives would not be for better accuracy but for 1) lower cost, and 2) confirmation of prosecutors’ accusations.

  16. #16 |  plutosdad | 

    Leon Wolfeson, it’s not the science that needs standards. It’s the conduct. Standards we need are : 1. blind analysis must be done (which means comparing many sample of random people, along with the suspect sample, and the scientist must not know which one is the suspect), multiple labs must be sent samples in 1 out of x cases (x could be 1 (every),2,3).

    Those are the standards necessary, by doing the second, confirming with other labs, you don’t need central control over the lab.

    That’s how science has worked for hundreds of years: external confirmation. You don’t throw that away for “central control and standards”, then you’re not practicing science anymore.

  17. #17 |  On the intellectual arrogance of our justice system | The Thinker | 

    […] The Agitator passes along news of error rates at an Indiana crime lab: Nearly one of every three cocaine tests conducted for criminal cases by the Indiana Department of Toxicology from 2007-2009 were bad, according to audit results released today by Indiana University. […]

  18. #18 |  Leon Wolfeson | 

    plutosdad, they *both* need standards. A standard set of tests, and a standard way of administering them.

    “External” confirmation is only part of a robust system of checking. But you can’t do it properly unless you’re ensuring that identical procedures have been carried out. Anything else leads to doubt, which is toxic to the legal system.

    The best way to do it, without paying large unnecessary sums, is a nationalwide network of identikit labs, where testing is carried out blind on samples (they could be local, they could be from elsewhere…).

  19. #19 |  fwb | 

    I run a testing lab at a university. My folks have old equipment no state or federal support and yet manage to receive 97% acceptance on double blind quarterly QC testing for the past 3 yrs running. It’s called having an ISO 17025 accredited QA/QC plan in action. We utilize an external body of scientists from around the world to perform the evaluation of the lab.

    If I was this guy I’da shot myself for letting garbage get through on my watch.

    But then I value honor above every thing.

  20. #20 |  fwb | 

    I advocate taking all forensic sample analysis out of the hands of the state and placing it into the hands of internationally accredited labs. AND that once the analyses are complete, both the prosecution and the defense receive ALL the information.

    I believe the state has a corner on things when it comes to dealing with so-called criminals. The state is all powerful with deep pockets. If the state wants you the state will get you.

    I also advocate eliminating all DAs, elected or appointed, and using a computerized random selection process for prosecutor and defender in EVERY case.

  21. #21 |  Indiana May Halt Crime Lab Investigation | The Agitator | 

    […] remember how Indiana was launching a big investigation into errors at the state crime lab? Remember how hundreds of convictions could have been called […]

  22. #22 |  Indiana Halts Audit of Crime Lab | 

    […] This is unfortunate, especially in light of the findings so far: […]

Leave a Reply