Two weeks ago, I posted in my Research Worth Reading series about an article that found that heterosexuals harboring ill-will towards gays lived shorter lives. It seemed like a methodologically sound article, but one thing nagged at the back of my brain. The un-adjusted results were huge, and after controlling for a few sensible factors, the adjusted results were still impressive, but much smaller.
That always makes me worry about uncontrolled (or poorly controlled) confounding, and I figured I'd look into it. There were a bunch of analytic choices I would have made differently, but none of them seemed like they'd be a big deal.
I got excited by their analysis and writeup, and wanted to play with the same data myself, try out a few different things, maybe look at different sub-groups, that sort of thing. I also thought it was a great approach, looking at the degree to which people harboring hatred may lead shorter lives.
So, I downloaded the same GSS files the authors used and fiddled around with it myself.
The results I got were not quite as impressive as theirs, and suggest that nearly all the main effects can be explained easily by routine confounding factors. Rather than starting out with a 187% increased death rate that is reduced to 25% after adjustment, my analyses showed a 70% increased death rate that was reduced to 8% after adjusting for similar factors.
There are a few important differences between their approach and mine, but it would be a lot more re-assuring to see similar results despite slightly different approaches, and I'm tempted to put this finding on hold until some replication in another dataset comes forward.
No comments:
Post a Comment