More Junk Science in the Courtroom

Tuesday, March 6th, 2012

A Texas man is currently 13 1/2 years into a 20-year sentence for molesting his two young cousins. The allegations from the children were corroborated by a bit of pseudo-science quackery worthy of a spot in the forensics hall of shame.

Michael Arena was summoned to a psychologist’s office to measure his sexual attraction to children.

The test given required the 16-year-old to click through images of swimsuit-clad people of various ages while the computer secretly measured how long he viewed each photo. The results, according to the prosecution-hired psychologist who administered the test, showed Arena to be a pedophile who was a “high risk” to strike again.

Bell County prosecutors hammered the finding during Arena’s 1999 trial, urging jurors to choose prison over probation to protect children from a teen “diagnosed as a pedophile by an expert.” The jury responded with a 20-year sentence.

The cousins have since recanted their testimony, and have said they were pressured by their mother to say they’d been molested. She was fighting a custody battle at the time. As you might imagine, the swimsuit photo test has since been shown to be nonsense.

The test, defense lawyers say, had an unacceptably high 35 percent error rate that was not disclosed to Arena’s judge and jury. It was never intended to be used to identify pedophiles, they claim, and a university study found that its results were little better “than chance” when trying to distinguish pedophiles from non-pedophiles.

In addition, the psychologist who examined Arena inflated the test’s effectiveness and scientific support when he testified at Arena’s trial, leading to a reprimand from a state regulatory agency four years later, court records show.

That was the evidence against Arena. The allegations from his two cousins and the test. The test has been shown to be a fraud. The cousins have retracted their allegations. Yet Arena is still in prison. Two judges have upheld his conviction since all of this has come out, and of course the prosecutors aren’t relenting.

The good news is that by the Statesman account, the Texas Court of Criminal Appeals at least appeared skeptical of Arena’s questioning during oral arguments.

One other item worth noting. The test, called the Abel Test, is run by a for-profit company. The owner of the rights to the test (eventually) said it was never intended to diagnose pedophiles, but to aid in the treatment of people already diagnosed. But he also refuses to release the test’s methodology, even for criminal cases, claiming that the information is proprietary.

We’ve seen this with breath test machines in DWI cases as well. I just don’t see how this can be acceptable. If you’re going to allow your technology to be used to put people in prison, it seems to me that anything and everything about how the technology works and how the results are interpreted has to be subject to cross-examination.

Any criminal law Agitatortots out there know if the Supreme Court has heard a Confrontation Clause case on the use of proprietary technology in criminal cases? I don’t recall coming across one in my reporting.

(Thanks to Ted Frank for the tip.)

 

Digg it |  reddit |  del.icio.us |  Fark

45 Responses to “More Junk Science in the Courtroom”

  1. #1 |  Mo | 

    If I was Arena or anyone else falsely convicted due to that test, I would sue the hell out of that company.

  2. #2 |  Nando | 

    It makes sense that you should be able to cross-examine any piece of evidence used to convict you. However, the state will argue that it’s harder for them to get a conviction if they grant access to proprietary info (either because it will be found to be flawed or because some will not want their proprietary information becoming public record), so most judges will probably rule against the defendent in order to not set precedent.

  3. #3 |  Peter Ramins | 

    But it’s not the state’s proprietary info. It’s just a methodology cloaked in secrecy, held by a private party, that the state relies on to get (sometimes fraudulent) convictions.

  4. #4 |  Windypundit | 

    “If you’re going to allow your technology to be used to put people in prison, it seems to me that anything and everything about how the technology works and how the results are interpreted has to be subject to cross-examination.”

    I have some sympathy for the technology companies here. On the one hand, the details of how a breath test machine works could be important in a DUI case. On the other hand, if a police department uses Photoshop to enhance a witness’s mobile-phone shot of a crime in order to identify a license plate, should Adobe — who is not a party to the case — be forced to turn over source code worth hundreds of millions of dollars?

  5. #5 |  primus | 

    I guess all of we who are regular readers are ‘agitatortots’. Is a lawyer who reads this blog an ‘agitatortort’?

  6. #6 |  Discord | 

    Next time they should just see if the defendant weighs more than a duck, and dispense with the air of scientific rigor.

  7. #7 |  contrarian | 

    “I have some sympathy for the technology companies here. On the one hand, the details of how a breath test machine works could be important in a DUI case. On the other hand, if a police department uses Photoshop to enhance a witness’s mobile-phone shot of a crime in order to identify a license plate, should Adobe — who is not a party to the case — be forced to turn over source code worth hundreds of millions of dollars?”

    There’s a distinction you’re missing, which is between the police using a tool, and being able to introduce the results as evidence at a trial. What happened in the Breathalyzer case was that Breathalyzer was selling its equipment for the purpose of creating an evidentiary record. The sales contracts they had with police departments, and state law governing contracts with government agencies, required them to submit to discovery and eventually turn over their source code to defense attorneys (and it turned out to be riddled with errors, mostly just guesswork really.)

    Software companies are free not to sell to police departments, and they are free to limit the terms of use to investigation and not evidence gathering. So Adobe could say that photoshop might be used to provide leads in an investigation, but that those leads would not be admissible in court. If the companies choose to sell into the evidentiary market then it is only reasonable that their products be subject to scrutiny.

  8. #8 |  Mike T | 

    I don’t understand why proprietary tests aren’t banned by statute by every legislature. They’re just a waste of time for the courts because proprietary methodology precludes independent analysis of the means by which the jury examined the evidence and arrived at their decision. It’s all but axiomatic that it precludes the Bill of Rights’ and 14th amendment’s guarantees of due process.

  9. #9 |  C. S. P. Schofield | 

    Not that I’m not interested in the proprietary information issue, but what I want to know is, what happened to the mother? Is she in ANY kind of trouble?

  10. #10 |  Highway | 

    What would have to change to make ‘retraction / invalidation of key evidence’ and even ‘provable innocence’ an immediate mandatory trigger for overturning a conviction, or at least a retrial on the remaining evidence only? It is absolutely morally bankrupt that after ‘temporary’ (recanted testimony, ‘scientific’ methods later proven false, undisclosed alibis) evidence is used to convict someone, the applicability of that evidence seems to be made irrelevant and any appeals are then based solely on procedure.

  11. #11 |  Goober | 

    On the other hand, if a police department uses Photoshop to enhance a witness’s mobile-phone shot of a crime in order to identify a license plate, should Adobe — who is not a party to the case — be forced to turn over source code worth hundreds of millions of dollars?

    If the photo being ‘enhanced’ is being used to convict a persona nd put them in prison, and there is a concern that the program, itself, may have had some hand in producing the results given, or the program is needed to determine if the police expert made sure the results produced what was needed to secure a conviction, then hell yes.

  12. #12 |  Bernard | 

    Anyone who has read this site will know from long experience that this stuff goes on in the justice system.

    With that in mind the most alarming aspect of this case for me is that a mother would pressure her sons into falsifying testimony to send their cousin to jail for 20 years on horrific charges just to win a custody battle.

    Seriously, what the hell?

    Given that systems are ultimately just groups of people surely this woman is as culpable as the justice system for railroading this guy. I can’t fathom the mentality that would do this under any circumstance. If people like her are raising charges and being listened to in the first place then it’s no wonder things are so screwed up.

  13. #13 |  JSL | 

    Bah, the photoshop analogy doesn’t work. There are already multiple image editors/”enhancers” out there and many experts who could look at the enhanced image and/or the source image to determine if there is a fault in the police evidence.

    Bernard, welcome to family/divorce court. Lawyers and the courts promote this insanity to gain power and money. They’re fearful of being shamed by women’s rights groups on top of it. This is the modern day version of witch trials in action.

  14. #14 |  Mattocracy | 

    Mike T, you have a great point. Have never thought of it in that manner before.

  15. #15 |  Bob Mc | 

    From the “you can beat the rap, but you can’t beat the ride” file:

    http://blog.simplejustice.us/2012/03/06/the-price-of-winning.aspx

  16. #16 |  Yizmo Gizmo | 

    Reminds me of the Probation people that were showing
    a Sex Offender photos of a perky-breasted 14-year old in a bikini
    and wanted to re-incarcerate him based on his subsequent woody.
    Behold the Penile Plethysmograph.
    http://en.wikipedia.org/wiki/Penile_plethysmograph

  17. #17 |  sheenyglass | 

    There’s a difference between source code and methodology. A software developer could reveal the methodology by stating what factors are measured and how conclusions are drawn from those factors. “Scientific” principles cannot be copyrighted or patented, therefore the underlying methodology which is implemented by the source code is not proprietary, unless its a trade secret. I’m not a criminal attorney, but it seems to me that if its a trade secret and the methodology can’t be reviewed, then by definition use of it at trial cannot meet the Frye or Daubert standards for expert testimony admission because it is known only to the developer (and cannot have widespread acceptance among the relevant scientific community) or, for Daubert, the Judge cannot evaluate whether the methodology was reliably applied to the facts.

    Additionally, in determining what evidence is admissible the judge serves a gatekeeper function. He/she could order in camera review (i.e., in the judge’s chambers and non-public) of any proprietary aspect of the software necessary to determine its reliability.

    The problem a defendant faces in this situation is that usually appellate courts review decisions on admissibility using the abuse of discretion standard, which is kind of a joke. I’m assuming that’s why Radley was asking about confrontation clause issues, as the confrontation clause violations are treated much more strictly.

  18. #18 |  el coronado | 

    So for all of us, especially the parents out there, here’s yet another example of a lesson your kids need to learn, and learn well: ***DO NOT talk to/answer questions from/interact with/take “harmless tests” with/from ANY adult who claims/projects/denies “authority” without calling mom & dad first, or without a lawyer present.***

    Emphasize that mom & dad won’t get mad; emphasize the self-protection aspects of this doctrine; make ‘em watch the law prof’s “Don’t ever talk to cops” video until they can recite it from memory. Hell, have it tattooed on their forearms. Tell ‘em it looks badass. They’re teenagers: they’ll fall for it.

    They pulled that bullshit test on the guy when he was 16. Would any of you want to be held legally responsible for thoughts you had at that age? Say it with me, son: “Smile and say ‘NO’ until your lips bleed; and keep doing it till mom or I get there.”

  19. #19 |  GeneralGarbage | 

    @ #10 – While I generally agree, it makes a lot of sense that recanted testimony doesn’t carry a lot of weight. If it did, it would open witnesses up to all sorts of post conviction bribery and coercion.

  20. #20 |  Boyd Durkin | 

    While #6 is awesome, #17 wins. I used to develop games with all kinds of cheats (and claims of sophisticated calc). Force me to disclose the science or don’t use the product.

  21. #21 |  (B)oscoH | 

    @Windypundit: You asked: On the other hand, if a police department uses Photoshop to enhance a witness’s mobile-phone shot of a crime in order to identify a license plate, should Adobe — who is not a party to the case — be forced to turn over source code worth hundreds of millions of dollars?

    No. In that case, the tool/technology was not used for a specific, advertised use. Additionally, there is basic, very settled information science for which any computer science graduate student could provide expert testimony, i.e. you can’t pull 4 megapixels of information out of a 1 megapixel photo.

    The technology in question in the case at hand was marketed precisely to solve the prosecution’s problem, detecting pedos. Just as a breathalyzer is marketed precisely to determine the BAC of your drunk uncle when he gets behind the wheel. Photoshop, no. That enhance thing is a misuse of the technology.

  22. #22 |  FTP | 

    Except that it doesn’t seem to be marketed to detect them, but rather as a tool for treatment (at least according to Abel). It seems the State of Texas was misapplying the technology, so why was it even allowed as evidence in the first place?

  23. #23 |  Jack | 

    The mother of those two kids should be the one in jail.

  24. #24 |  Ben | 

    How is his man’s conviction upheld when all evidence against him has been proven either nonsense or recanted testimony? On what grounds was he kept in prison?

  25. #25 |  Mike T | 

    There’s a difference between source code and methodology. A software developer could reveal the methodology by stating what factors are measured and how conclusions are drawn from those factors.

    That’s not really correct. The source code implements the methodology. When you are talking about the “business logic,” you are talking about the potentially lucrative, non-trivial aspects of the code. 90% of Adobe Photoshop is virtually indistinguishable from its competitors; they would gain nothing from having it opened. They would, however, probably be quite interested in the “factors involved” in how Adobe’s JPEG resize algorithm does such a realistic extrapolation from 1MP to 2MP.

  26. #26 |  ClubMedSux | 

    I have to agree with Mike T that I don’t see it so much as a Confrontation Clause issue as a Daubert/Frye admissibility of expert testimony issue. That evidence is coming in SOMEHOW, and I don’t know how you can prove to a judge that the methodology is generally accepted if the methodology is proprietary.

  27. #27 |  Peter Ramins | 

    WindyPundit – Adobe could either testify/affidavit that they do not endorse or intend the use of photoshop for these purposes, or turn the source code over. It’s pretty clearly a case of having your cake and eating it too.

    Either you are willing to allow your product to be scrutinized and challenged, or you’re not. The reason for ‘not’ doesn’t matter. If you’re not willing, it shouldn’t be admissible, or should be challenged quite easily. (and then excluded based on an easy challenge, via confrontation as Radley suggest.)

    Corporate profit should not trump an accused’s right to a competent and fair defense/trial.

  28. #28 |  Mannie | 

    #17 | sheenyglass | March 6th, 2012 at 1:52 pm

    There’s a difference between source code and methodology. A software developer could reveal the methodology by stating what factors are measured and how conclusions are drawn from those factors.

    How can the credibility of the in-house expert be independently challenged. Without access to the source code, he could say anything he wants to. It’s not like he has any incentive to tell the truth and possibly discredit his product, and jeopardize his job by doing so.

  29. #29 |  awp | 

    For this particular case it is more a problem with “expert” witnesses than secret technology.

    #6 | Discord | March 6th, 2012 at 11:58 am
    Next time they should just see if the defendant weighs more than a duck, and dispense with the air of scientific rigor.

    That is exactly what this/some “expert” witnessing boils down too. I told this kid to look at pictures of girls and he spent “too long” on some of them. Who the hell would weigh that as any type of evidence, “expert” or no.

    I hope the original conviction really came down to the cousins’ coerced testimony.

  30. #30 |  awp | 

    As a friend to some “expert”(economics) witnesses, it really boils down to who’s “expert” is prettier or (I hope) more convincing. If economics “experts” are being used in a case the real issue of contention is going to be the discount rate. The average discount rate across the population has a pretty wide acceptable range and the two economists pick high or low based on who is paying them. Although, often, one party(generally the defense) will not call their “expert” hoping the final value of the payout (calculated using the discount rate) put forth by the other party was so ridiculous that the jury just rejects it out of hand.

    So anyway I would have to conclude that the defense in this trial either couldn’t afford an expert witness to point out how stupid the test was. Or, again that the original conviction and sentencing really came down to the cousins’ coerced testimony.

  31. #31 |  contrarian | 

    “#17 | sheenyglass | March 6th, 2012 at 1:52 pm
    There’s a difference between source code and methodology. A software developer could reveal the methodology by stating what factors are measured and how conclusions are drawn from those factors. “Scientific” principles cannot be copyrighted or patented, therefore the underlying methodology which is implemented by the source code is not proprietary, unless its a trade secret. ”

    If you look at the Breathalyzer case you’ll see why this argument doesn’t work. The problem with Breathalyzer weren’t with the methodology, but in the implementation. Two specifics that I remember:

    * The methodology was to take a certain number of measurements at certain intervals and average them. The programmer assigned this task didn’t know how to do it properly. Rather than adding up all of the measurements and dividing by the number of observations, he kept a running tally. Each time a new measurement arrived he added it to the tally and divided by two.

    * The sensors used had a range of values for which their accuracy was valid. If a value was encountered outside of the range it was changed to the closest valid value.

    Deposing the programmer who wrote this code would not have revealed these errors, as the programmer did not believe that they were errors. Only inspection of the source code by a defense expert revealed them.

  32. #32 |  Cornellian | 

    I don’t understand why so many self-identified conservatives are so outraged by junk science in civil cases but so indifferent to the use of junk science in criminal cases.

  33. #33 |  pim FEE | 

    It is because of great stories like this that I come to this site everyday. You make it look easy Mr. Balko. But please don’t become a Bill Maher type libertarian. Don’t make the move from religious critic to religion hater like so many on the left have. God Bless!

  34. #34 |  hamburglar007 | 

    We call it voight-kampff for short.

  35. #35 |  Goober | 

    @ 32 – Bill Maher is not a libertarian. i can’t even imagine a person who is less libertarian than he is. He is all about big government intruding into people’s lives. That is NOT libertarian.

  36. #36 |  Radley Balko | 

    On what grounds was he kept in prison?

    Finality!

  37. #37 |  Aresen | 

    Couldn’t they use dowsing rods and save themselves the costs of the expert witnesses?

  38. #38 |  Mike T | 

    Bill Maher is not a libertarian. i can’t even imagine a person who is less libertarian than he is. He is all about big government intruding into people’s lives.

    Bill Maher is an idiot who heard someone make a snide comment that libertarians are just libertines and actually took them seriously.

  39. #39 |  JOR | 

    To be fair, Bill Maher used to be more libertarian than he is now, and as far as I know he now denies being a libertarian.

    Even my contrarian devil’s advocacy does not extend so far as to defend him against accusations of idiocy.

  40. #40 |  Discord | 

    #32 | Cornellian | March 6th, 2012 at 5:51 pm
    I don’t understand why so many self-identified conservatives are so outraged by junk science in civil cases but so indifferent to the use of junk science in criminal cases.

    It’s because hypocrisy is the biggest problem in our political system, and it’s prevalent on both sides of the isle. “Freedom for me, but not for thee.” IMO calling yourself a “democrat” or “republican” is being intellectually dishonest in some way.

  41. #41 |  forensicoutreach | 

    Why was this “test” not subjected to a Daubert hearing? Surely it would have flat-out failed and been thrown out!

  42. #42 |  William E Kruse, Esq. | 

    Daubert v. Merrell Dow Pharmaceuticals was decided in 1993, and only applied to the Federal Courts. Texas didn’t start applying the Daubert Standard until after this trial. That is why. See Mack Trucks, Inc. v. Tamez, 206 S.W.3d 572, 577 (Tex. 2006).

  43. #43 |  highnumber | 

    Am I missing something with the Adobe analogy? The source code isn’t secret, it’s just patented. Adobe has control over its use through their licensing but they aren’t hiding what it is.

  44. #44 |  Matt | 

    @highnumber

    The adobe source code most certainly is secret, although you could get a (very ugly) copy by decompiling it I suppose. I’m sure it is also copyrighted (not patented, although I’m sure they have patented some processes or algorithms they used in it). Proprietary software companies generally don’t turn over their source code.

  45. #45 |  bacchys | 

    He’s also still in prison because his attorneys aren’t perfect. That’s a shame, because no attorneys are perfect (IOW, I’m not bashing them for not being perfect).

    The recent case of Bullcoming v. New Mexico solidified the confrontation clause right to cross examine the actual lab technician that performed the tests. It doesn’t seem much of a stretch to me to that right includes the ability to inspect the software and hardware used by an “expert” witness to arrive at his conclusions. Bullcoming is well into the future after this man’s conviction, but the biggest hardass on the USSC when it comes to the confrontation clause is Scalia- who was on the Court when he was convicted. It seems more likely that his lawyers didn’t pursue the confrontation clause aspects as vigorously as they could have. Again, I’m not blaming them. It’s not reasonable to expect perfection, and thinking of every possible legal strategy- or having the funds to pursue them all- is at least bordering on perfection.

    It’s an indictment of our system that the wheels of justice move so slowly when it comes to recognizing errors, let alone correcting them. This prosecution has ruined this man’s life, and it’s probable that no consequences will accrue to those responsible.

Leave a Reply