Data Unicorns

How many unicorns are in your data? Sounds like a silly question. But there can be some major problems when we don't think to ask it. Because every dataset has what appear to be unicorns in it - impossible combinations of data made possible because of infrequent errors.

Rob Kelly, Blackout Tattoo Studio, Hong Kong

Usually it's not a problem because the unicorns make up a really small proportion of your sample. And if the data combination is in fact impossible, or makes up a tiny proportion of what you're really interested in, you can just ignore them, or even try to "correct" them if you have additional information. But when you're interested in a rare phenomenon, it can be hard to tell the difference between unicorns and the real cases you're interested in.

Gay Blood Donors

Take, for instance, a paper I've been working on for years about estimating how many gay blood donors there are.

If the American Red Cross's procedures were followed to the letter, there shouldn't be any because any man who has "had sex with a man, even once, since 1978" is supposed to be excluded. In other words, any apparent gay blood donors should be unicorns –impossible data combinations.

We know that there are some, because every once in a while, someone tests positive during the blood donation screening process, and when they go back to interview the donor, some donors admit to "having sex with a man, even once, since 1978". But we have no idea how many HIV- gay blood donors there are, how many men who are giving on a regular basis without incident, despite the ban.
So, I've been looking at various datasets trying to get a rough idea of how many gay blood donors there are, trying to make the point that the ban on gay male donors isn't just discriminatory, it's also ineffective. And if we could talk with the men who are giving blood regularly without incident, maybe we could develop new exclusion criteria based on what they are doing.

It sounds simple enough, look up how many gay men there are in these datasets, and count how many of them are giving blood. But here's the problem. There are errors in counting who's a gay man, and also errors in counting who gives blood. So, any heterosexual male blood donor who is inaccurately coded as gay or bisexual will appear to be a gay/bi blood donor. As will any gay/bisexual non-donor who is accidentially coded as a blood donor. Let's start out with some plausible (but made up) numbers to illustrate...

Let's give ourselves a decent-sized dataset, with 100,000 men in it. Suppose that 95% of the male population has not "had sex with a man since 1978", and 5% of them have given blood. That's 4,750 straight men who are blood donors.
In the 1970's the Census did a big study where they interviewed people twice, and found that in about 0.2% of the cases, the two interviews resulted in a different sex for the respondent - about one in 500. So, what if 0.2% of these 4,750 guys who are giving blood without bending the rules at all get mis-coded as gay or bisexual - that's about 9 cases of what appear to be excludable blood donors.
Let's just make a guess that instead of 5% of heterosexual men giving blood, that 0.5% of gay/bisexual men do. Then we've got 100,000 x 5% x 0.5% = 25 cases of gay/bi men who are giving blood despite the ban.
So, all told, it looks like there are 34 gay/bi blood donors, but only 74% of them really are gay/bi blood donors.
But what if 0.06% of gay/bi men are really giving blood? Then there would be 3 real gay/bi blood donors, but there would appear to be 12, and only 25% of them would really be gay/bi blood donors. Most of the time, we'd be looking at unicorns.
What's frustrating is that I can't tell the difference between these two scenarios. I can't tell if my unicorn ratio is only 24%, or if it's 75%.

There's another problem, too - with the blood donation questions. Sometimes, people want to inflate their sense of altruism, and they'll say they gave blood in the last year even if it was closer to two years ago. That I can live with, but an even bigger problem is that people get confused by the wording of the question, and they say they've given blood even if all they did was have a blood test at the doctor's office. So, there are some surveys where the blood donation rate appears to be upwards of 25%.
Let's assume that 5% of the population (gay or straight) who haven't given blood say that they have because they mis-understood the question (or that the interviewer was inattentive and hit the wrong button).
Then the number of straight men who say they've given blood would be 10%, not 5%, or 9,500. And if 0.2% of them were mis-classified as gay/bisexual, that would be 19 men who appear to be gay/bisexual blood donors. Then, if we take 5% of the gay/bisexual men as being mis-classified as being blood donors, that would be another 250 men who really aren't blood donors, but appear to be. In that case, if there are really 25 gay/bisexual blood donors, they would make up only 9% of the 294 men who appear to be gay/bisexual blood donors, and if there were really only 3 gay/bisexual blood donors, they would be 1% of the 272 who appear to be blood donors, or in other words, 99% unicorns.
And just to underscore the point, that's coming from errors of 0.2% and 5%.

There is a way to sort through this mess. You'd just need to call the men who appear to be gay/bi blood donors and ask them to clarify on a second interview. The number who would be inaccurately coded twice would be really small, because the relevant error rates are small (0.2% and 5%). But it is unlikely that anyone will do that kind of call-back.

Unicorns Ahead

There are a number of other contexts where we should expect to see unicorns in LGBT health research.
One is transgender health. There are a number of States that have been asking BRFSS respondents if they are transgender, and it looks like about 1 in 500 say that they are. But we need to be very careful in researching this population, because if the 1970's Census estimates hold, it's probably not unreasonable to think that 0.2% of the population will inadvertently be coded as being transgender, and that could easily be most of the people identified as transgender in these surveys. Again, the easiest solution is to call people back to verify. But in the absence of a call-back survey, we won't know whether 70% of the people identified as trans are actually trans, or if only 7% are.
Another group heavily influenced by unicorns is married same-sex couples. Before 2004, almost all people identified as married same-sex couples in the United States were unicorns, because it wasn't a legal status available to anyone. Another analysis I'm working on shows that the proportion of people identified in surveys as married same-sex couples who are really married same-sex couples can be as low as 10%, and rarely gets above 50%, but it's getting better in states where marriage is legal.

Research Directions

    Hey there blogfriends, I'm super excited because I'm going to have a first-author paper coming out in a few days - about the racial distribution of trees and pavement across the US - and exploring a few reasons that may explain it, like segregation (yes) and poverty (no). It looks like there's going to be some press on it, so keep an eye out.
    And my next first-author paper is getting really close to submission - so it's probably six months to a year from publication. That one's about the influence of living in more segregated cities on the probability of experiencing racial discrimination. That one's pretty interesting - lots of studies within one particular city or another have found that experiences of racial discrimination tend to be less common among Blacks who live in predominantly Black neighborhoods, and more common among Blacks who live in predominantly White neighborhoods. As far as I can tell, ours is the first to look at the degree to which the overall segregated character of the city (and her suburbs) affects reporting of racial discrimination experiences. We're seeing pretty dramatic results in that more segregation results in more experiences of racial discrimination, for Blacks, Hispanics, Whites and Asians.

    But what I'm stymied with at the moment is where to go after my most recent first-author paper - showing that gay men are more likely to be in excellent health than straight men... I'd love to get another paper on TBLG health out there, relatively soon, but it's challenging, because I have to do the work on my own dime and my own time. So here's some ideas, and I'd love to hear your thoughts on what would be most helpful (helpful in any sense - informing policy, improving science, satisfying curiosity - whatever greases your gears).

ONE: Improving Identification of Same-Sex Couples in Large Probability Datasets
    I know. Boring title. But here's why this has been floating my boat lately. When I was working on gay men in excellent health, I looked at the biggest dataset I could lay my hands on, the BRFSS. There were a fair number of same-sex married couples, even before same-sex marriage was legal anywhere in the US, which struck me as odd. Another thing that was odd is that their demographics (how old they were, how many kids they have, whether they served in the military, etc.) were a lot like heterosexually married people. I figured that what was most likely happening was that a small number of heterosexually-married people were accidentally mis-coded - and ended up being counted as same-sex couples. So, I threw them out of the analysis.
    BRFSS is especially vulnerable to this kind of error, but the problem is ubiquitous in any of the large probability samples that get used for research on same-sex couples - and rarely acknowledged.
    So what this project would be about is systematically going through the major datasets and trying to estimate how many of the same-sex couples identified are really same-sex couples, and how many are mis-coded heterosexually-coupled people.
    The main reason that it's important to do this project is that there are a lot of publications out there claiming that same-sex married couples are "just like" heterosexually-married couples. That may be a comforting message, and there's probably something to it, but a likely explanation that is almost never discussed is that a lot of those same-sex married couples are in fact heterosexuals. If we want an accurate picture, we need actual same-sex couples.

TWO: BLG health in relation to voting on marriage restrictions
    OK, so my thesis (never was able to get it published) was about the occurrence of suicide in relation to heteronormativity - the more heteronormative an area is, the higher the suicide rate there - especially for young men. I measured heteronormativity in three ways: the legal status of employment discrimination; how people voted on restricting marriage; how many same-sex couples the Census counted in an area.
     Given that nobody seems to care about employment discrimination any more these days, I figure that I should focus on the voting thing. The way I see it, how people in an area vote on restricting marriage to "one man and one woman" is a pretty good heteronormativity thermometer. There are some complications in that the wording is different from State to State, and the change in public attitudes is so rapid that a 60% endorsement rate today probably corresponds to an 80% endorsement rate in 2004. But assuming I can figure out a way to handle that, the other part is finding a dataset that has good BLG health measures in it.
    For my thesis, I used the overall suicide rate, and I didn't particularly care whether the people who died of self-inflicted injuries were "gay" or not. In fact, I suspect that the highest suicide risk associated with being gay or bisexual is before one declares openly to anyone else, and even before having sex, so it would be kind of silly to try to figure out who's who after they're dead. But I think that's one of the reasons I had trouble getting anyone interested in publishing it - it seems like people want to know how BLG people are affected by homophobia. Well, I'm interested in how heterosexuals are affected also. I very much doubt that it's a zero-sum game where heterosexuals gain some advantage while BLG people pay the price. I suspect it's much more likely that heterosexuals, too, are harmed by heteronormativity. And since there are a lot more of them, it should be even easier to pin that down. But I digress.
    So, I need a dataset that A) is a probability (random) sample of the US, B) has a large sample size (ideally in the 10's of millions, but I'll have to settle for less), C) identifies who is gay, lesbian, bisexual, and heterosexual, D) has a high degree of spatial resolution so I can figure out what the local homophobia "temperature" is, and E) has decent temporal resolution so I can figure out when people were sampled relative to important dates, and F) has decent measures of health in it.
    There are some datasets that come close to fitting the bill, but it's a challenge.

THREE: Transgender health from large population datasets
    There's only one publication out there about transgender health based on a probability sample - from the Massachusetts BRFSS. But there's the potential to do so much more. There are seven States that have asked about transgender identity on BRFSS. I'd love to collect the data from all seven, compare the basic demographics of transgender-identified people across the different question wordings & hypothesize about which questions work best. And then get into the health outcomes, much like the Massachusetts study did, but with much more data. I suspect that all of the question wordings are going to have a significant problem much like the same-sex married people identified in large population datasets - that is, even a very small number of errors in the coding of cisgender people is going to be a major headache. There's really only one way to handle that that I can think of - call them back to verify it - but I really can't see that happening anytime soon.

FOUR: The Real Blood Donors of Gaytown, USA
    There are just so many things wrong with banning gay blood donors. It made sense in 1985 (and frankly, it would have made even more sense earlier). But it doesn't make sense now, and everyone knows it. Including lots of gay men who donate blood anyway, and increasing numbers of young straight people who won't donate because they don't feel right about the discrimination. I'd love to be part of qualitative research on gay men who give blood. Why do they do it? How does it make them feel? What 'rules' about donating have they made for themselves to decide when they should and should not donate?
    There's a lot of interesting policy angles to wrangle through on this issue, but I think getting to know these guys would be really interesting - and informative in coming up with better deferral guidelines.

FIVE: Wage Gap and Death
    Strangely enough, there are only a handful of studies out there measuring how sexism affects health at a population level. Most of them use some sort of complicated mash of different ideas into an "index", and I hate indices - you never know what's really going on in there. So I took a simpler approach, just looking at the wage gap between men and women. It varies a lot - there are some parts of the country where women make almost as much as men, and some parts where men make about twice as much as women. What I expected to see was that women's mortality would be higher in areas where men make more. But I saw something completely different: where men make more relative to women, they live longer, but women's mortality is unrelated to the wage gap. I basically put this project on ice because I can't figure out a narrative that makes sense. But I could go back to it if y'all have fresh ideas.

So let me know, what do you think I should work on? And if you're feeling especially generous, for only $62,000, you get to decide.

Research Worth Reading (4) - trans health in Massachusetts

Gunner Scott, Sewart Landers and pals have served up a very interesting paper in January's AJPH - the first time anyone anywhere has published anything peer-reviewed on a population-representative sample of transgender people.

In Massachusetts, the Behavioral Risk Factor Surveillance Study (BRFSS) has asked the adults it interviews "Do you consider yourself to be transgender?", and a whole lot of demographic and health-related questions.

Many studies in the past have sought out a transgender population to try to say something about the health of the group, but this is the first one to rely on a "random" sample, meaning calling people up at random; and that's probably the best way to be sure that you've got a study population that is fairly representative (at least of people with phones).

In addition to addressing trans health from a population perspective for the first time, this study is also the first to report simple basic demographics of the transgender population in the US as well, including the most basic one - how many transgender people are there?

The answer, in this study, is about 1 in 200 in Massachusetts, about 1 in 110 in a similar study from Vermont, and 1 in 170 in Boston. It is likely (for reasons I've discussed before) that these are overestimates, meaning that the true proportion is probably somewhat lower than that, but how much lower? That's hard to guess, it depends on how many nontrans people answer the wrong way because they are distracted or misunderstood the question. The only way to figure that out is to call back the people who said they were trans and ask them again.

The study is very interesting in that it validates some things trans health activists have known for years, but there isn't strong evidence to support all the health disparities that have been identified from "convenience" samples. Members of the trans population in this Massachusetts study were less likely to be employed, and more likely to be living in poverty than the nontrans population. The study also documented that 36% of the trans population were smokers, compared to 17% of the nontrans population. But markers of access to health care were not particularly different. The trans population was less quite a bit less likely to have health insurance (86%) than the nontrans population (94%), but this did not translate into not having a regular health care provider or not seeing a doctor because they couldn't afford it, and the trans population was even more likely to have had a checkup in the last 12 months (85%) than the nontrans population (75%).

Mental health measures did show some substantive differences: 70% of the trans population reported usually or always getting needed emotional support, but this was quite a bit lower than the 90% of nontrans people who got their emotional needs met; and 14% of the trans population reported being dissatisfied with their life, but only 6% of the nontrans population did.

The authors were very thoughtful about ways that these results might be misleading - for instance that trans people are probably less likely to be stably housed and have a telephone, so these figures may well present a rosier picture than a fully representative sample of trans people would be. And also, not knowing how many cis-gender (nontransgender) folks inadvertently classified themselves as trans, it is hard to know the degree to which true differences between the trans and cis populations are diluted by these inaccurately coded folks. Another possible source of bias might be people who have transitioned, but no longer consider themselves to be transgendered, although I suspect this is pretty unlikely to be a substantial part of the population, because the way the question in Massachusetts was asked, they made it clear that they meant "experience(d) a different gender identity from their sex at birth. For example, a person born into a male body, but who feels female or lives as a woman". But excluding people who no longer consider themselves to be trans would, probably, make the differences seem larger than they actually are. A similar bias would arise from trans people not feeling comfortable describing themselves as trans to a stranger on the phone.

Thanks Gunner & Stewart!