Two weeks ago, I posted in my Research Worth Reading series about an article that found that heterosexuals harboring ill-will towards gays lived shorter lives. It seemed like a methodologically sound article, but one thing nagged at the back of my brain. The un-adjusted results were huge, and after controlling for a few sensible factors, the adjusted results were still impressive, but much smaller.
That always makes me worry about uncontrolled (or poorly controlled) confounding, and I figured I'd look into it. There were a bunch of analytic choices I would have made differently, but none of them seemed like they'd be a big deal.
I got excited by their analysis and writeup, and wanted to play with the same data myself, try out a few different things, maybe look at different sub-groups, that sort of thing. I also thought it was a great approach, looking at the degree to which people harboring hatred may lead shorter lives.
So, I downloaded the same GSS files the authors used and fiddled around with it myself.
The results I got were not quite as impressive as theirs, and suggest that nearly all the main effects can be explained easily by routine confounding factors. Rather than starting out with a 187% increased death rate that is reduced to 25% after adjustment, my analyses showed a 70% increased death rate that was reduced to 8% after adjusting for similar factors.
There are a few important differences between their approach and mine, but it would be a lot more re-assuring to see similar results despite slightly different approaches, and I'm tempted to put this finding on hold until some replication in another dataset comes forward.
Thursday, January 2, 2014
Mark L. Hatzenbuehler, Anna Bellatorre, Peter Muennig. (2014). Anti-gay prejudice and all-cause mortality among heterosexuals in the United States. American Journal of Public Health. Published online ahead of print, Dec 12. 2013.
I'm so glad someone has finally done this study!
We all know that homophobia is bad for your health. It could be as direct as gay-bashing, or societal disapproval leading to depression, and less directly by causing high blood pressure and that sort of thing.
But what about the haters? What are the ill effects on people who are themselves homophobic?
In this paper, the authors used 20,226 people who answered the General Social Survey to figure out how much anti-gay prejudice people feel, specifically heterosexuals, then followed them for 5 to 20 years after the survey to see whether straight people who harbor anti-gay prejudices die sooner than those who don't.
They found that heterosexuals with a high degree of anti-gay prejudice were much more likely to die, dying at a rate nearly 3 times as fast as heterosexuals with a lower degree of anti-gay prejudice. That may seem implausibly high, and it is. People who harbor anti-gay prejudice tend to have less formal education, and tend to be older, and both of those factors strongly predict mortality.
But even after adjusting for age and educational attainment (and a few other things), they found that heterosexuals with a high degree of anti-gay prejudice died about 25% faster than heterosexuals with lower anti-gay beliefs. That's more reasonable, but still higher than I'd expect. I suspect that at least some of that difference is due to the fact that the General Social Survey is so long and tedious for respondents that there's a fairly high rate of non-sensical responses in there.
Promising work, but when you see over 80% of the apparent effect (an excess hazard ratio of 187% dropping to 25%) after being "explained" by control factors, what's left has to be treated very skeptically.
I'll be eager to see how this line of inquiry pans out in other datasets, although this is clearly the best dataset to start with, and it may be challenging to find another than could produce comparable results for quite some time.
Well worth reading: the language is pretty accessible even if you're not steeped in the public health world. The methods are a bit challenging, but you can skip the most confusing parts because they don't really make much difference anyway.
This study is actually really well done, much better than most public health research these days. Despite the inherent flukiness of the GSS, the authors used methods that should be pretty robust despite the relatively high rates of non-sense that you find in the GSS.
Having given high praise overall, I'll move on to the relatively minor things I'd quibble with... First, the measure of whether a person has a high degree of anti-gay prejudice is based on some questions that are horribly out of date, and were horribly out of date when they were asked, from 1988 to 2002. The items are taken from a series of questions designed to assess general social attitudes about communists, atheists, homosexuals and other "undesireables", so the questions can sound a bit strange to us today, especially the first three, which are probably more about civil liberties than prejudice:
- "If some people in your community suggested that a book in favor of homosexuality should be taken out of your public library, would you favor removing this book, or not?"
- "Should a man who admits that he is a homosexual be allowed to teach in a college or university, or not?"
- "Suppose a man who admits that he is a homosexual wanted to make a speech in your community. Should he be allowed to speak, or not?"
- "Do you think that sexual relations between two adults of the same sex is always wrong, almost always wrong, wrong only sometimes, or not wrong at all?"
If I were doing the study, I'd probably ignore the first three as anachronistic and focus just on the fourth one. But what they did is collapsed the fourth one into a yes/no of "not wrong at all" vs. any of the other responses, and then (as best as I can tell), said that a "yes" to any of the four indicated a high degree of prejudice. It's possible that someone had to say "yes" to any two or more to make it into the high prejudice category. At any rate, it would have been re-assuring to show some kind of dose-response curve from lower endorsement to higher endorsement, and also a check to see if the pattern held when just looking at the fourth item, which is most clearly related to prejudice.
Of course, it would also be nice to have some sort of response codes indicating a positive inclination towards us, rather than assuming that our words and deeds have only the potential to offend.
In terms of potential confounders adjusted for, they used pretty much the same list I would have, but I would have modeled some of them a bit differently. Rather than treating age and education as continuous, I'd want to look at them in categories first to make sure that a linear trend makes a logical fit. And I wouldn't use household income itself, but adjust it first to the size and composition of the household relative to poverty. $20,000 for a single person in 1988 would be a lot more comfortable than $20,000 for a household of four in 2002, and log-transforming the household income doesn't help with those issues at all.
Most importantly, I'd want to explore the year of the survey in a bit more detail. The surveys were conducted from 1988 to 2002, and the follow-up for death ended in 2008, so someone from the early part of the survey could be followed for up to 20 years, while someone interviewed in 2002 might be followed up for as few as five years. They used Cox proportional hazards, which should account for these differences in the length of follow-up, but the fact that anti-gay prejudicial attitudes have shifted rapidly over the same time period makes me less confident that the model did what it was supposed to do. You can probably think of someone who would answer those questions differently in 2002 than they would have in 1988. But the model assumes that they would have answered the same way at both points in time, or at the very least that someone giving a certain answer in 1988 had the same level of prejudice as someone giving the same answer in 2002, despite the fact that it became much less acceptable to express anti-gay attitudes over this time period. It might screw up the model a bit to add year of interview in as a potential confounder, but I'd give it a try anyway, because it's quite possible that what we're seeing is just an artifact of the fact that as the population has developed fewer anti-gay attitudes, they've also been followed for a shorter period of time, and are thus less likely to be seen dying, despite the beauty of the Cox proportional hazards approach in dealing with censored data.