Let's look at the latest data from the Americas Barometer, a frequent survey of most countries in the Americas. I've used it before to examine state-level homophobia and transphobia in prior posts. Since my last losts, there is new data from 2023, and some new questions. So, just jumping right in, here is the mean of an 11-point scale of how strongly respondents approve (10) or disapprove (0) of same sex marriage (averaged from 2010, 2012, 2014, 2016, 2018 & 2023), and grouped by region. Higher values indicate higher average support for same sex marriage.
Bill and Tuna
I'm Bill. These are my observations on queer health, and other things I care about for one reason or another. Tuna was my adorable dog, a companion of 16 years.
Saturday, January 20, 2024
Support for Same Sex Marriage in the Americas
Let's look at the latest data from the Americas Barometer, a frequent survey of most countries in the Americas. I've used it before to examine state-level homophobia and transphobia in prior posts. Since my last losts, there is new data from 2023, and some new questions. So, just jumping right in, here is the mean of an 11-point scale of how strongly respondents approve (10) or disapprove (0) of same sex marriage (averaged from 2010, 2012, 2014, 2016, 2018 & 2023), and grouped by region. Higher values indicate higher average support for same sex marriage.
Tuesday, March 28, 2023
Interview Completion: Are Sexual and Gender Minority People More or Less Likely to Engage in Research?
In the crude rates, LGBT respondents were just about as likely to complete the interview as cisgender heterosexuals. Transgender people somewhat less likely, and as in HPS, gay and bisexual cismen more likely to complete the interview, while cismen of another seuxal orientation were less likely to complete the interview.
HINTS is a very rich survey, lots of in-depth information about experience with cancer and beliefs about cancer prevention. However, with about 15,000 respondents after pooling 4 annual surveys, the sample is just too small to say anything confidently about sexual minority respondents relative to heterosexuals. Also, the reported interview completion rate is very high, which is great, but it also means there's not a lot of variation to look at from a statistical perspective.
American National Election Survey (2016, 2020, mostly Internet, some face-to-face, televideo, and telephone interviews)
Really nothing to say about this survey, given that it is shy of 10,000 respondents, and again with such a high interview completion rate that there isn't much statistical variation to play with.
Wednesday, February 22, 2023
Proportion LGBT in 20 Probability Samples
Thursday, February 2, 2023
Queer and Trans Representation in Research
WHAT IS REPRESENTATION?
A common refrain in research on LGBT populations is that we are underrepresented in research. In many ways, that is undoubtedly true. Many data collection systems do not include items on sexual orientation, and even fewer include gender identity. And many of those that do are small enough that there is not enough of a queer/trans population to provide reliable estimates. Arguably funding for research on (not often enough with) sexual and gender minority populations is decades overdue and falls short of the mark. And publications about sexual and gender minority comprise a tiny fraction of the published scientific literature.
And yet. The number of research datasets with reliable information on sexual orientation and gender identity has expanded rapidly, first in large survey datasets, more recently in infectious disease tracking systems, and coming soon in medical records datasets and large administrative databases. Funding has increased dramatically in recent years, and there are now established journals dedicated to sexual and gender minority population research.
In a more limited sense of 'representation', we really don't know the degree to which sexual and gender minority populations are represented in these datasets - in other words, how much more or less likely are these populations to be included in research? With age, race/ethnicity, and geography, we can use Census records to compare the distribution of people included in a study with respect to their expected distribution in the population, and "sex" in a broad sense, although this breaks down when considering gender identity. When we know these distributions, we can re-weight the analytic dataset to reflect the population at large.
But, with sexual and gender minority populations, there is no Census standard - in fact, these surveys themselves are the closest thing we have to a standard. But there is considerable variability from one survey to another in terms of the proportion of people identifying as sexual and gender minorites, as well as variation in the questions asked - and responses options offered.
DATA SOURCES
In a series of posts I plan to explore here, I'll be looking at representation in this narrow sense (likelihood of responding to an invitation to engage in survey research) across a wide range of large probability surveys in the US, namely:
Behavioral Risk Factor Surveillance System (2014-2021)
Household Pulse Survey (2021-2023)
National Health Interview Survey (2013-2021)
National Health and Nutrition Examination Survey (1999-2019)
National Survey of Drug Use and Health (2015-2020)
National Survey of Family Growth (2011-2019)
National Adult Tobacco Survey (2012-2014)
Population Assessment of Tobacco and Health (2011, 2016)
California Heath Interview Survey (2021)
New York City Community Health Survey (2003-2020)
National Crime Victimization Survey (2017-2021)
Health Reform Monitoring Surveys (2013-2020)
Well-Being and Basic Needs Survey (2017-2020)
General Social Survey (2008, 2010, 2012, 2014, 2016, 2018, 2021)
American National Election Surveys (2016, 2020)
Collaborative Multiracial Post-election Surveys (2012, 2016, 2017)
Associated Press VoteCast (2018)
These 17 large surveys with public use data reflect a broad range of sampling strategies (random telephone dial, internet recruitment from Census lists, panels recruited by established survey firms, quite a few in-person interviews based on physical addresses, and one using televideo interviews), on a variety of topics (puplic opinion polling, health surveys, crime), using a variety of question wording and response options. They are heavily weighted towards recent years, but there are some going back decades. Is there a dataset you think I've overlooked? Let me know!
MEASURING REPRESENTATION
If there are no Census data (or other standard) for the distribution of sexual and gender identity data, how do I propose to look at relative representation of these groups in these research datasets? Indirectly. I plan to use measures that are fairly inuitive correlates of research participation, as determined in prior research.
One of the most intuitive is how many attempts it took to get a successful interview. Presumably, people who answer the call to participate immediately are "easy" interviews, and those who take 20-50 attempts to connect with are less eager to participate. So, we can look at the distribution of how many contact attmepts were made to connect for an interview as a proxy for eagerness to participate. Alas, this measure is only reported publically in 2 of the above studies.
Another fairly intuitive proxy is how likely a respondent is to complete the interview once started. Presumably people who hang on to the end of an interview are more invested in the research endeavor than those who break off after a short period. This measure is available for 3 of the above surveys - many of these surveys only report out complete interviews (or impute values for missing data) so that there are no "short" interviews to compare to. For others, the sexual orientation and/or gender identity items are asked late enough in the interview that there is not information on these items among those who cut the interview short.
A measure that seems to make sense (but may be less useful than it appears) is the weight assigned to a respondent. If the weighting system the survey is using works well enough, respondents who are harder to reach will have a higher weight, and those easy to reach will have a lower weight. The factors that go into these weights typically include sex, race/ethnicity, age, geography, how many phones the respondent uses and how many people could answer the phone, and interactions betweeen these factors. So, to the degree that how much more or less likely sexual and gender minority people are to respond because of these factors, the relative weighting could be informative. But, to the extent that sexual and gender minority respondents elect to engage with researchers is related to being LGTBQ over and above those delineated factors that go into the weighting, the relative weights will fail to reflect participation.
A less intuitive measure is called the "fraction of missing information", or how many items the respondent used a "don't know" or "not sure" response, or declined to answer. Presumably, a person who declines to answer a larger proportion of questions is less invested in the research than a person who answers everything. Of course, there are many reasons to leave questions blank or say "don't know" that have nothing to do with eagerness to participate. And I have to be careful to distinguish between questions left blank because the respondent heard it and didn't answer vs. the question was skipped on purpose, or skipped because the interview already ended. Another difficulty with this measure is that an awful lot of people answer every question, even when they truly don't know or aren't sure, so the median number of blank items is 0, a rough distribution to work with from a statistical perspective. On the plus side, these measures are available for all the studies above, except one that imputes missing and don't know values to "known" values before releasing the public use datatset.
I've gone ahead and split these missing information measures into two categories: one based on demographic items (race/ethnicity, marital status, educational attainment, employment status, income, household composition, citizenship, language), and another based on all other items on the survey, which I've called 'substantive items' for lack of a better generic term for "everything else", whether that be a history of cancer to presidential candidate preference.
STATEMENT OF EXPECTED HYPOTHESES
What do I expect to see in all this? It's hard to say for sure, which is what makes it especially interesting to me. I do expect to see heterogeneity. I expect to see greater participation from sexual and gender minority populations from Internet-based recruiting than telephone, for instance. I expect to see greater participation from gay men than lesbian women, and greater still than bisexual men and women; from cisgender people than transgender. Overall, I think that LGBT people will probably be somewhat more likley to participate in research, but if I had to guess, I'd say the difference is probably pretty small, compared to differences in participation related to age, race/ethnicity and sex. I suspect that the variation between sexual and gender minority groups will be greater than the difference between LGBT people as a whole and cisgender heterosexual adults.
I would say I don't have a strong expectation about participation between transfeminine and transmasculine people. I don't have as solid a foundation of experience to draw from. I'm also not sure about what to expect about younger or older LGBT people relative to younger or older cisgender heterosexuals, or about LGBT people belonging to minoritized racial/ethnic groups relative to cisgender heterosexual non-Hispanic Whites.
WHAT YOU SHOUOLD EXPECT
Over the next weeks to months, I plan to post a variety of analyses here related to this topic. Expect to see analyses based on one survey at a time. Expect to see an analysis of the same quetion or proxy outcome across multiple surveys. Expect to see analyses of missing data due to particular items across multiple surveys and populations. Expect to see analyses looking at trends over time, differences across survey methodologies, differences with respect to survey topics (drug use, general health, crime victimization, politics). In other words, this topic is too big (at least in my mind) for synthesis into a single paper for publication. I want to explore it with you and figure out along the way what the paper(s) within the topic are to pursue for publication in a more formal setting.
Sunday, July 31, 2022
How Transphobic is my State (Part I)
I think it's safe to say that the social climate in every state can be characterized as transphobic. It also depends on who you ask, and what you ask them.
But if some states are more transphobic than others, is there a way to measure those differences in degree?
To date, most researchers have used measures based on legislation and policy to describe the climate of each state. The Movement Advancement Project has the most comprehensive listing of policies affecting transgender, genderqueer, non-binary & agender people in the United States.
Public opinion is also a promising way to measure the transphobic climate of the states, and a number of polling firms have reported public opinion on items related to transphobia. A few recent examples include: NPR reporting that a majority of Americans oppose allowing girls and women who are trans to participate in womens' sports; Pew reports that fewer than half of Americans favor requiring health insurance to cover gender affirming therapies; and Gallup reports that support for transgender people being able to serve openly in the military has declined since 2019. For a comprehensive assessment of Americans' views, check out this poll conducted by IPSOS on behalf of the WIlliams Insititue. Unfortunartely, none of these polls report state-level results in a way that researchers like me can use them to develop measures of state-level climate, as I was able to do for measures of homophobia derived from the AmericasBarometer and the American National Election Survey.
For today's blog, I have to thank Maggi Price (@MaggiPrice) and colleagues for bringing to my attention (through a terrific preprint, don't know if I can link to it) that Project Implicit publishes a dataset ideal for assessing state-level transphobic attitudes. It's a bonanza and I've been working on for a couple weeks & am finally ready to share findings with you-all!
OK, here are the first set of findings, methodology below:
The diamonds indicate the average score for each state, after weighting to the state population (more details below). The states are ranked from lowest mean transphobia as assessed on a 9-item scale (Vermont) to highest (Guam, or Mississippi if you're just interested in states) on this particular measure. The vertical whiskers are 95% confidence intervals indicating the degree of uncertainty in the state-specific measures. A long line incidates a low level of certainty in the state's score, and a short line indicates a hgiher degree of certainty - but it is worth noting that none of these confidence intervals are short enough to have a high degree of confidence the the exact ranking of each state. Also, these confidence intervals are based on sampling variation only, and are thus far smaller than they should be if I had accurately accounted for additional sources of uncertainty - a decent rule of thumb is that the confidence intervals from a sample like this should be about doubled in size. The absolute values on the y-axis (15-50) have no simple direct interpretation, it is the sum of 9 items-see details below to how they are calculated.
Methodology
A couple generic methodologic notes that apply to all measures I've developed from this Project Implicit dataset. The survey is designed to measure an individual's implicit bias against or for a given group (usually a minoritized or marginalized group), based on how quickly they associate "good" and "bad" words with that group compared to another group (usually the socially dominant group). The survey also asks a number of questions about explicit bias. For my purposes, I am not terribly interested in how individuals respond, but the overall tenor of a state (or in the future, metro area or smaller geography, as sample size allows), and I am interested in both the implicit and explicit measures.
This is a "convenience" sample, meaning that the questionnaire was filled out by whoever elected to do it, not from a systematic sample of Americans. Thus it is important to pay attention to who elected to fill it out, especially since I am trying to estimate these measures for the state as a whole, not for the people who elected to take the survey. Some people took it as part of a school project, or because they were encouraged to (perhaps even required to) by their employer. Some people took it because they heard about it in the news, on social media, or from a friend. Presumably, those required to take it may be more representative of the state climate (thanks to Jarvis Chen for this insight), because they would be closer to a representative population, but I've decided (for now anyway) to include respondents regardless of their reason for taking the test. In any event, a large number of people have done so (about 200,000 over 2020-2021 for the Transgender Implicit Associations Test), and I have constructed weights based on people's state of residence, sex (admittedly an imperfect variable in this case!), age group, and race/ethnicity, to attempt to make the sample more closely resemble the general population of each state according to the Census's July 2020 population estimates (or as close as I could get to that for the teritories of American Samoa, Guam, Puerto Rico and the US Virgin Islands). (In the future, watch for variations on this weighting theme, such as weighting for educational attainment, gender identity, policital ideology, county-level geography, etc. Jarvis gave me many great ideas about trying out different weighting schemes).
I'd also like to explore, in future variations, restricting to respondents aged 18-64, and/or those who are cisgender, to reflect the attitudes in the socially dominant population, but for now, I have included all respondents aged 10 to 85+, and of all gender identities.
There is some evidence of occasional "goofball" respondents (like people who say that they are all races, and that they are all genders (cismale, cisfemale, transmasculine and transfeminine), etc.), but I have not yet done anything about excluding or assigning low weights to the goofballs. It can be a bit of a tightrope walk defining who is a goofball and who is not - there is a risk of assigning someone with multiply marginalized positions as a goofball by mistake.
Eager to hear any suggestions people may have for further refinements!
Now, on to the specific measures:
Nine-Item Transphobia Scale
This is a measure of explicit bias against transgender people, measured using a 9-item scale, with responses ranging from strongly disagree (1) to strongly agree (7) on a 7-point scale. Thus, the scale can range from 7 to 63. Here are the items:
- I don't like it when someone is flirting with me, and I can't tell if they are a man or a woman.
- I think there is something wrong with a person who says that they are neither a man nor a woman.
- I would be upset, if someone I'd known a long time revealed to me that they used to be another gender.
- I avoid people on the street whose gender is unclear to me.
- When I meet someone, it is important for me to be able to identify them as a man or a woman.
- I believe that the male/female dichotomy is natural.
- I am uncomfortable around people who don't conform to traditional gender roles, e.g., aggressive women or emotional men.
- I believe that a person can never change their gender.
- A person's genitalia define what gender they are, e.g., a penis defines a person as being a man, a vagina defines a person as a woman.
I required answers to all 9 items, a future version may include multiple imputation to be able to include respondents who missed a couple items - often people skip items for good reasons (like the wording doesn't make sense, or they just don't know how they feel), and it is a shame to exclude people. I also didn't do anything fancy like factor analysis or trying to account for the relative importance of each item-I just added them together.
This scale was asked of about half of the people taking the test. I have done nothing special to account for that fact - notably the weights are the same as those for the whole dataset, not in any way adjusted to reflect the subsample that was given this scale.
I think it's worth noting that these items don't talk about transgender people explicity. Arguably, these items are more about a form of sexism I call "stereosexism" - the notion that there are but two genders and that these are permanent attributes. However, whatever that concept is, it is arguably very closely linked to transphobia.
Single-Item Preference for Transgender People
This next measure is based on a single item, with 7 possible responses:
- I strongly prefer transgender people to cisgender people.
- I moderately prefer transgender people to cisgender people.
- I slightly prefer transgender people to cisgender people.
- I like cisgender people and transgendper people equally.
- I slightly prefer cisgender people and transgendper people.
- I moderately prefer cisgender people and transgendper people.
- I strongly prefer cisgender people and transgendper people.
You may well wonder why I didn't lead with the implicit bias measure, especially since this is the whole purpose for the data collection by Project Implicit in the first place!
The main reason I put this third is that it is the measure I have the least familiarity with, and thus the least confidence I am aware of potential measurement issues inherent to it. To be completely honest with you, I have not delved into the methodology deeply enough to even be able to describe how this measure is calculated, let alone what the scale (y-axis) means.
As before, I have ranked the states and territories from the least implicit bias against transgender people (Guam, or Utah if you're only counting states) to the most implicit bias (North Dakota), with the diamonds representing my method's best guess as to the average level of anti-trans implicit bias in the state, and the vertical whiskers indicating 95% confidence intervals - with the wider ranges meaning less confidence and the narrower ranges higher confidence.
Although this measure was asked of all respondents, it had more missing-ness than the single-item preference for transgender people measure above - and I have not looked into why these are missing - could be that people got tired out before the test was finished, could be that there is a data-cleaning step to remove measures if people were too slow or too quick to answer, or some other factors I haven't thought of.
And one more measure - this one is about contact with transgender people. It is based on 4 items, each with a yes/no response. I have simply added these up, resulting in a scale from 4 (no contact) to 8 (contact in each category):
- Do you have a family member who is transgender?
- Do you have a friend who is transgender?
- Do you have friendly interactions with transgender people on a regular basis?
- Have you even met a transgender person?
- about the conceptual idea of measuring state-level transphobia - does this even make sense to you? Does this kind of quantification seem viable to you?
- what the different measures are telling you about the contours os between-state variation in transphobia - are these 4 measures all measuring the same underlying construct or are the differences between contact, implicit bias, and the 2 explicit measures of transphobia speaking to you about something more than there being a single "thing" to measure?
- how could we use these measures to examine the causal role of state-level climate on mood disorders? Gender euphoria? The impact of inhibiting gender expression on stunting the development of cisgender, as well as transgender, people?
Tuesday, February 1, 2022
How Homophobic is my State (Part II)
This is the second post in a series on using public opinion polling to assess structural heteronormativity. The first (based on the LatinoBarometer surveys) can be found here.
For today's installment, I collated responses to a "feeling thermometer" from the American National Election Survey, a survey of Americans of voting age conducted to assess a wide variety of attitudes and voting pattern data. For this measure, I averaged responses from surveys conducted in advance of major elections in 2008, 2012, and 2016. Please note that these averages DO NOT use the appropriate weighting supplied with the data, and are not adjusted for anything other than which election wave the data were collected in. Thus, the means may be off what they should be (either higher or lower, impossible to predict), and the confidence intervals are probably narrower than they should be.
The question wording for the "feeling thermometer" is reproduced below, after which a number of groups are asked about - the order of the groups presented is randomized from one respondent to another.
"We’d also like to get your feelings about some groups in American society. When I read the name of a group, we’d like you to rate it with what we call a feeling thermometer. Ratings between 50 degrees-100 degrees mean that you feel favorably and warm toward the group; ratings between 0 and 50 degrees mean that you don’t feel favorably towards the group and that you don’t care too much for that group. If you don’t feel particularly warm or cold toward a group you would rate them at 50 degrees. If we come to a group you don’t know much about, just tell me and we’ll move on to the next one."
"Gay Men and Lesbians"
As before, the states are ranked from least heteronormative rating to most heteronormative rating.