Interview Completion: Are Sexual and Gender Minority People More or Less Likely to Engage in Research?

This post is part of a series about engagement of sexual and gender minority populations in survey research. For an overview, start here.

One of the most direct measures of research participation I'm looking at is interview completion, getting to the end of the survey. Certainly there are many reasons for cutting an interview short; some of these are more closely related to a lack of interest in engaging in the research effort than others, but on the whole, the more a respondent is engaged, the more likley they are to get to the end of the survey.

Alas, out of the 25 surveys I have tabulated so far, only 3 report whether the interview was completed (the National Health Interview Survey (NHIS), the Health Information National Trends Survey (HINTS), and the American National Election Survey (ANES). I also created  mwasures of interview completion for 2 more surveys (the Behavioral Risk Factor Surveillance System (BRFSS) and the Household Pulse Survey (HPS)) by looking at patterns of missing values―if all values sequentially after a given point in the interview are missing, then I code that as an interview termination. Developing those measures is time-consuming, and I doubt I'll do it for any others.

So, here's what I found in these 5 surveys, from most to fewest nummber of respondents:

Household Pulse Survey (waves 34-55, 2021-2023, Internet survey)

Overall, LGBT interview completion was a bit lower than among cisgender heterosexuals, lower among transgender respondents, perhaps higher for gay cismen, and lower for cismen of "another" sexual orientation. These are raw (weighted) percents, so to get an estimate of relative likelihood of interview completion after adjusting for respondent age, state or residence, and time trend, I estimated a logistic model to get adjusted odds ratios fro interview completion:

After adjustment, interview completion was actually higher for LGBT people averaged together (on this chart, 1.0 means equally likely as the comparison group), for cisgender sexual minority women, cisgender gay and bisexual men, and about the same as cisgender heterosexuals  for transgender respondents.

Behavioral Risk Factor Surveillance Survey (2014-2021, Telephone survey, restricted to states where SOGI items were asked in the demographics section)

In the crude rates, LGBT respondents were just about as likely to complete the interview as cisgender heterosexuals. Transgender people somewhat less likely, and as in HPS, gay and bisexual cismen more likely to complete the interview, while cismen of another seuxal orientation were less likely to complete the interview.

Again, I did a logistic model to adjust for respondent age and state of residence, and time trend.

After adjustment, LGBT people were slightly more likely to complete interviews, transgender people less likely to, gay and bi cismen more likely, and cismen of another seuxal orientation were less likely to complete interviews.

National Health Interview Survey (2014-2021, Face-to-face interviews)

Although I was able to combine several years of data here, the sample size of NHIS is considerably smaller than HPS or BRFSS, so the comparison between sexual minority adults and heterosexuals is robust, getting into some of the subgroups gets harder to interpret. NHIS did not collect gender identity, but they did identify people who said they weren't sure about their seuxal orientation.

Overall, sexual minority respondents were about as likely to complete interviews, and it looks like the questioning groups may have been less likely to complete the interview.

Again, a logistic model, adjusted for respondent age, region of residence, and time trend:

Overall, the relative likelihood of interview completion for sexual minority respondents was slightly higher, in the same range as the two larger surveys above. Subgroups are too small to interpret here.

Health Information for National Trends Survey (2017-2020, Internet & Mail)

HINTS is a very rich survey, lots of in-depth information about experience with cancer and beliefs about cancer prevention. However, with about 15,000 respondents after pooling 4 annual surveys, the sample is just too small to say anything confidently about sexual minority respondents relative to heterosexuals. Also, the reported interview completion rate is very high, which is great, but it also means there's not a lot of variation to look at from a statistical perspective.

I'm hesitant even to show model results because of this, but for the sake of completeness, here they are:

American National Election Survey (2016, 2020, mostly Internet, some face-to-face, televideo, and telephone interviews)

Really nothing to say about this survey, given that it is shy of 10,000 respondents, and again with such a high interview completion rate that there isn't much statistical variation to play with.

All 5 Surveys Together

The main value of looking at these 5 surveys, with different methodologies, covering different subject matter, and over (somewhat) different time frames is being able to look at them all together. Here are the results of the five logistic models for LGB(T) populations compared to cisgender heterosexuals, all on the same scale:

With only 5 surveys, it doesn't make sense to do a formal meta-analysis, especially given that they surveys are really quite different from one another. Nonetheless, it is reassuring to see that the three largest studies have relative completion rates that are compatible with one another (the 2 smaller studies are also compatible with these, but also compatible with such a wide range of alternate possibilities that they are simply not informative).

It may come as a surprise to some readers that LGBT people are, at least in terms of interview completion, more likely to engage in research, and thus perhaps slightly over-represented in research datasets.