How Transphobic is my State (Part I)

 I think it's safe to say that the social climate in every state can be characterized as transphobic. It also depends on who you ask, and what you ask them.

But if some states are more transphobic than others, is there a way to measure those differences in degree?

To date, most researchers have used measures based on legislation and policy to describe the climate of each state. The Movement Advancement Project has the most comprehensive listing of policies affecting transgender, genderqueer, non-binary & agender people in the United States.

Public opinion is also a promising way to measure the transphobic climate of the states, and a number of polling firms have reported public opinion on items related to transphobia. A few recent examples include: NPR reporting that a majority of Americans oppose allowing girls and women who are trans to participate in womens' sports; Pew reports that fewer than half of Americans favor requiring health insurance to cover gender affirming therapies; and Gallup reports that support for transgender people being able to serve openly in the military has declined since 2019. For a comprehensive assessment of Americans' views, check out this poll conducted by IPSOS on behalf of the WIlliams Insititue. Unfortunartely, none of these polls report state-level results in a way that researchers like me can use them to develop measures of state-level climate, as I was able to do for measures of homophobia derived from the AmericasBarometer and the American National Election Survey.

For today's blog, I have to thank Maggi Price (@MaggiPrice) and colleagues for bringing to my attention (through a terrific preprint, don't know if I can link to it) that Project Implicit publishes a dataset ideal for assessing state-level transphobic attitudes. It's a bonanza and I've been working on for a couple weeks & am finally ready to share findings with you-all!

OK, here are the first set of findings, methodology below:

The diamonds indicate the average score for each state, after weighting to the state population (more details below). The states are ranked from lowest mean transphobia as assessed on a 9-item scale (Vermont) to highest (Guam, or Mississippi if you're just interested in states) on this particular measure. The vertical whiskers are 95% confidence intervals indicating the degree of uncertainty in the state-specific measures. A long line incidates a low level of certainty in the state's score, and a short line indicates a hgiher degree of certainty - but it is worth noting that none of these confidence intervals are short enough to have a high degree of confidence the the exact ranking of each state. Also, these confidence intervals are based on sampling variation only, and are thus far smaller than they should be if I had accurately accounted for additional sources of uncertainty - a decent rule of thumb is that the confidence intervals from a sample like this should be about doubled in size. The absolute values on the y-axis (15-50) have no simple direct interpretation, it is the sum of 9 items-see details below to how they are calculated.

Methodology

A couple generic methodologic notes that apply to all measures I've developed from this Project Implicit dataset. The survey is designed to measure an individual's implicit bias against or for a given group (usually a minoritized or marginalized group), based on how quickly they associate "good" and "bad" words with that group compared to another group (usually the socially dominant group). The survey also asks a number of questions about explicit bias. For my purposes, I am not terribly interested in how individuals respond, but the overall tenor of a state (or in the future, metro area or smaller geography, as sample size allows), and I am interested in both the implicit and explicit measures.

This is a "convenience" sample, meaning that the questionnaire was filled out by whoever elected to do it, not from a systematic sample of Americans. Thus it is important to pay attention to who elected to fill it out, especially since I am trying to estimate these measures for the state as a whole, not for the people who elected to take the survey. Some people took it as part of a school project, or because they were encouraged to (perhaps even required to) by their employer. Some people took it because they heard about it in the news, on social media, or from a friend. Presumably, those required to take it may be more representative of the state climate (thanks to Jarvis Chen for this insight), because they would be closer to a representative population, but I've decided (for now anyway) to include respondents regardless of their reason for taking the test. In any event, a large number of people have done so (about 200,000 over 2020-2021 for the Transgender Implicit Associations Test), and I have constructed weights based on people's state of residence, sex (admittedly an imperfect variable in this case!), age group, and race/ethnicity, to attempt to make the sample more closely resemble the general population of each state according to the Census's July 2020 population estimates (or as close as I could get to that for the teritories of American Samoa, Guam, Puerto Rico and the US Virgin Islands). (In the future, watch for variations on this weighting theme, such as weighting for educational attainment, gender identity, policital ideology, county-level geography, etc. Jarvis gave me many great ideas about trying out different weighting schemes).

I'd also like to explore, in future variations, restricting to respondents aged 18-64, and/or those who are cisgender, to reflect the attitudes in the socially dominant population, but for now, I have included all respondents aged 10 to 85+, and of all gender identities.

There is some evidence of occasional "goofball" respondents (like people who say that they are all races, and that they are all genders (cismale, cisfemale, transmasculine and transfeminine), etc.), but I have not yet done anything about excluding or assigning low weights to the goofballs. It can be a bit of a tightrope walk defining who is a goofball and who is not - there is a risk of assigning someone with multiply marginalized positions as a goofball by mistake.

Eager to hear any suggestions people may have for further refinements!

Now, on to the specific measures:

Nine-Item Transphobia Scale

This is a measure of explicit bias against transgender people, measured using a 9-item scale, with responses ranging from strongly disagree (1) to strongly agree (7) on a 7-point scale. Thus, the scale can range from 7 to 63. Here are the items:

1. I don't like it when someone is flirting with me, and I can't tell if they are a man or a woman.
2. I think there is something wrong with a person who says that they are neither a man nor a woman.
3. I would be upset, if someone I'd known a long time revealed to me that they used to be another gender.
4. I avoid people on the street whose gender is unclear to me.
5. When I meet someone, it is important for me to be able to identify them as a man or a woman.
6. I believe that the male/female dichotomy is natural.
7. I am uncomfortable around people who don't conform to traditional gender roles, e.g., aggressive women or emotional men.
8. I believe that a person can never change their gender.
9. A person's genitalia define what gender they are, e.g., a penis defines a person as being a man, a vagina defines a person as a woman.

I required answers to all 9 items, a future version may include multiple imputation to be able to include respondents who missed a couple items - often people skip items for good reasons (like the wording doesn't make sense, or they just don't know how they feel), and it is a shame to exclude people. I also didn't do anything fancy like factor analysis or trying to account for the relative importance of each item-I just added them together.

This scale was asked of about half of the people taking the test. I have done nothing special to account for that fact - notably the weights are the same as those for the whole dataset, not in any way adjusted to reflect the subsample that was given this scale.

I think it's worth noting that these items don't talk about transgender people explicity. Arguably, these items are more about a form of sexism I call "stereosexism" - the notion that there are but two genders and that these are permanent attributes. However, whatever that concept is, it is arguably very closely linked to transphobia.

Single-Item Preference for Transgender People

This next measure is based on a single item, with 7 possible responses:

1. I strongly prefer transgender people to cisgender people. 
2. I moderately prefer transgender people to cisgender people.
3. I slightly prefer transgender people to cisgender people.
4. I like cisgender people and transgendper people equally.
5. I slightly prefer cisgender people and transgendper people.
6. I moderately prefer cisgender people and transgendper people.
7. I strongly prefer cisgender people and transgendper people.

Note, I copied these responses from the data documentation, including the misspelling of "transgendper" instead of "transgender" for responses 4-7, and presumably a typo in items 5-7 "cisgender and" instead of "cisgender to". I assume these are errors in the documentation, not in the original survey, but buyer beware...

For the chart above, I've re-ranked the states and territories from the least likely to prefer cisgender people (Vermont) to the most likely (Arkansas), with the diamonds indicating the average response for the tates's residents, and the vertical whiskers indicating the 95% confidence interval based on sampling variability only (and again, these should be interpreted as though the confidence intervals are about twice as wide as they are calculated, given that this is not a random sample, and due to the influence of additional sources of variation not accounted for in the standard calculations).

This item was asked of everyone who took the test (but not answered by some). I have done nothing special to the weights to account for missing-ness on this item, just included people who actually answered.

Those of you with eagle eyes for this sort of thing may have noticed that, even though this measure has nearly double the sample as the transphobia scale above, the uncertainty around these state-level estimates are generally larger than for the 9-item scale (on a relative scale), meaning that the relative ranking of states and territories on this measure is even less stable than the previous measure. Could be a bunch of reasons for that - my leading hypothesis would be that the 9-item scale has lower variability between individuals (which is just a way of saying it is more stably measured, not that it is a closer measure of transphobia). I do like that this measure directly asks people about transgender and cisgender people, that seems much more of a direct linkage to me than the items in the scale above. On the other hand, many people (particularly the cisgender majority) do not have a strong internalized notion of what "cisgender" means, and may not be able to answer this item accurately, even though IAT defines the terms.

Implicit Bias

You may well wonder why I didn't lead with the implicit bias measure, especially since this is the whole purpose for the data collection by Project Implicit in the first place!
The main reason I put this third is that it is the measure I have the least familiarity with, and thus the least confidence I am aware of potential measurement issues inherent to it. To be completely honest with you, I have not delved into the methodology deeply enough to even be able to describe how this measure is calculated, let alone what the scale (y-axis) means.
As before, I have ranked the states and territories from the least implicit bias against transgender people (Guam, or Utah if you're only counting states) to the most implicit bias (North Dakota), with the diamonds representing my method's best guess as to the average level of anti-trans implicit bias in the state, and the vertical whiskers indicating 95% confidence intervals - with the wider ranges meaning less confidence and the narrower ranges higher confidence.
Although this measure was asked of all respondents, it had more missing-ness than the single-item preference for transgender people measure above - and I have not looked into why these are missing - could be that people got tired out before the test was finished, could be that there is a data-cleaning step to remove measures if people were too slow or too quick to answer, or some other factors I haven't thought of.

Another concern I have is that many respondents are pretty unfamiliar with the term "cisgender", and the implicit association test, which counts on a knee-jerk rapid response comparing "transgender" and "cisgender" may not adequately tap into this automatic level of thinking if people have to keep reminding  themselves what the terms mean, as opposed to "Black" and "White" which are concepts most Americans have deeply emplanted understandings of.

It also appears that the variability around each state's measure is higher still, relative to the variability between states, suggesting that the relative ranking between states is less stable than the other two measures. I'm worried about reading too much into that at this stage, but initally, it suggests to me that either implict bias is tough to measure precisely, or that the level of implicit bias is relatively uniform (and presumably high) across the country, whereas for the explicit measures, they may be tapping into how socially acceptable it is to express bias, which may well vary more from state-to-state.

Contact with Transgender People

And one more measure - this one is about contact with transgender people. It is based on 4 items, each with a yes/no response. I have simply added these up, resulting in a scale from 4 (no contact) to 8 (contact in each category):

1. Do you have a family member who is transgender?
2. Do you have a friend who is transgender?
3. Do you have friendly interactions with transgender people on a regular basis?
4. Have you even met a transgender person?

These items were asked of nearly half the people who took the test (with no overlap bewteen this half and the half who were asked the 9-item transphobia scale). I have only included people who answered all 4 items, and not done anything to account for those who are missing one or more, and as before, used the overall weighting, nothing special for this particular measure.

As with the other measures, I have ranked the states, this time from the least contact (The US Virgin Islands, or West Virginia) to the states with the highest levels of contact (Guam, or Vermont). Note that the ranking on this measure is conceptually in the opposite direction from the prior measures, but generally the states end up in nearly the same ranking (once you take the reversed ranking into account).

I think it's interesting that the variabilty between states is pretty low on this measure, with most states bunched in a pretty narrow range indicating an affirmative response to 1-2 of the items above.

This type of measure is often interpreted as a precursor to people's attitudes towards a group, under the "Contact Hypothesis" elaborated by Allport in the 1950's, fairly early on in America's fascination with sociology and efforts to understand how eugenics in World War II and racializing attitudes in post-war America came into the mainstream of public thought and political action.

However, I think it has a much more complicated relationship than simply reflecting prior contact as a driver of current attitudes. In this case, that people who are aware of contact with transgender people are more likely to have had some awareness and openness to that possibility before they meet (or become aware of) transgender people in their lives. Also, I would argue that the state climate is likely to be a strong determinant of how open people are to be about being transgender (and thus come to the conscious attention of others), and potentially even of recognizing that experiences around gender are compatible with a transgender identity. Thus, I think these contact measures are better considered as reflections of the state climate than causes of it.

Closing Queries

Hey, thanks for bearing with me through all this. I am super interested to hear your thoughts...

• about the conceptual idea of measuring state-level transphobia - does this even make sense to you? Does this kind of quantification seem viable to you?
• what the different measures are telling you about the contours os between-state variation in transphobia - are these 4 measures all measuring the same underlying construct or are the differences between contact, implicit bias, and the 2 explicit measures of transphobia speaking to you about something more than there being a single "thing" to measure?
• how could we use these measures to examine the causal role of state-level climate on mood disorders? Gender euphoria? The impact of inhibiting gender expression on stunting the development of cisgender, as well as transgender, people?

If you want to chat, but don't want to leave a public comment, go ahead and get to me through twitter @billandtuna. I look forward to hearing from you.