Science-reporting smell test of the week -

Science-reporting smell test of the week

Colby Cosh on how a study linking abortion and depression was grossly misinterpreted


Here’s the lede of a science story from Saturday’s Winnipeg Free Press:

WINNIPEG — Depression and substance abuse plague about half of American women who reported having an abortion, according to a new University of Manitoba study.

The study, published in the current issue of the Canadian Journal of Psychology, suggests there’s an association between mental disorders and abortion…

Eager to investigate this shocking headline claim—the Edmonton Journal, picking up the story, literally gave it the headline “Depression or drug abuse found in half of women who aborted”—I set out to find the study. This presented something of a problem, since there has not been a “Canadian Journal of Psychology” since 1993. I spent a little while rifling through Canadian Psychology and the Canadian Journal of Experimental Psychology until a helpful reader on Twitter clued me in. Yes, you guessed it: it can be found in the Canadian Journal of Psychiatry. First place I should have looked, really.

That’s an understandable mistake. It’s a bit more of a problem that the first sentence of the article—an article that includes a warning from the lead author to the effect that it is “important the study is not misinterpreted”—is totally false. Because of, y’know, misinterpretation.

The paper, entitled “Associations Between Abortion, Mental Disorders, and Suicidal Behaviour in a Nationally Representative Sample”, does what it says on the tin: the data are taken from interviews with a demographically representative subset of the U.S.’s National Comorbidity Survey Replication project. It is hard to know what numbers the reporter added or multiplied or pulled out of a hat to reach the conclusion that “Depression and substance abuse plague about half of American women who reported having an abortion.” (I spoke to the lead author of the study, and she can’t figure it out either.) But a good guess would be that she looked at this section from the article’s main chart—

—and simply added together the estimated lifetime incidence of depression among women who had had an abortion (29.3%) and the lifetime incidence of substance-use disorders (24.6%). It will probably have occurred to you that there might be some overlap there between depression and substance abuse, which go together like poached eggs and hollandaise. You don’t need a Ph.D. to know that the depression group is likely to contain almost all of the women in the substance-abuse group.

And this naïve math (which is hardly attributable to a failure to grasp hyper-advanced statistics) is compounded by the wording of the offending sentence, which doesn’t say that “some percentage of abortion recipients have, at some point before or after getting an abortion, experienced depression or substance abuse or both.” It uses present tense, unjustifiably implying that all the women in question are plagued by both problems now.

This mess is already being picked up, “carelessly” garbled even further, and circulated around the globe by pro-lifers, despite the personal entreaties of the scientist who helped the newspaper with its reporting and the many, many methodological and interpretive caveats in the original study. This kind of thing is exactly why a lot of scientists hate talking to reporters. Nor does it make sincere research into therapeutic abortion any easier. The UM study can’t be used to attribute psychiatric morbidity to abortion, but it could be used by fair-minded pro-lifers (let’s assume for the sake of argument that there were some) to raise questions about abortion’s place in our society and argue for a research program.

Oh, I know: we’re a hundred years away from that kind of discussion being possible. But the inadvertent propagation of urban legends only pushes that day further into the future.