Month: January 2024

How the Washington Post/KFF poll represents underrepresented trans people

grvsmth
The Telephone Survey Support Center call center at NORC. At least fifty people are sitting in cubicles at computer workstations, holding pieces of paper.
The Telephone Survey Support Center call center at the University of Chicago National Opinion Reseearch Center.

I’ve been posting recently about the transgender survey released last year by the Washington Post and the Kaiser Family Foundation. Some of the things I write are critical, but I want to be clear how much I appreciate and value it as a real advance in understanding the situations and opinions of trans people in the United States. It’s a very big deal, because it really is the first representative survey of trans people in this country.

I’ve written extensively before about why representativeness matters in science, why transgender surveys are no exception, and how so many transgender surveys fail. So what does the Washington Post/KFF survey do that the other surveys don’t? As the researchers describe in their survey methodology report, they draw from representative panels.

Other studies have been unrepresentative because they relied on trans people responding to broadcast announcements (even in web forums), or surveyors reaching out to trans people they already know. These methods ignore several populations: people who might not be looking at those announcements, people who might not respond to a general announcement, and people who aren’t known to surveyors. That can exclude whole communities of trans people.

The Washington Post/KFF study draws on three existing panels that are built to randomly sample the population of the entire country. One, the AmeriSpeak panel from the National Opinion Research Center at the University of Chicago, uses something called “Address Based Sampling,” dividing the country into segments and sending letters to randomly selected addresses from each segment, asking them to participate in a panel. The others, the Gallup panel and the SSRS Opinion Panel, use a combination of Address Based Sampling and “Random Digit Dial,” where they call random phone numbers and ask people if they want to be on a panel.

The panels are groups of people, contacted through one of these methods, who’ve agreed to answer questionnaires. They’re paid a small amount, between $5 and $20, for every questionnaire they answer. In one questionnaire, the organization that runs the panel (NORC, Gallup or SSRS) asked them if they identified as trans; the Washington Post/KFF researchers don’t give the exact text of those questions. From these three panels, the researchers found 483 people who identified as trans, and then added 29 people from previous KFF surveys and three more, for a total of 515.

I have to note here that while you can pay people to answer questions like this, you can’t force them, in any ethical way. Similarly, you can reach people by mail and phone, but if people change their mailing addresses and phone numbers, they’re less likely to be able to join one of the panels.

These panels are not quite representative of the entire population of the country. Their answers can only be generalized to those people in the country who have stable postal addresses or phone numbers, and are willing to spend time answering surveys for NORC or Gallup or SSRS, for $5-20 each time. And the responses to this survey are not representative of all transgender people in the US, but of trans people with stable addresses or phone numbers who are willing to answer surveys.

It may sound complex, but I want to stress how much better this is than previous surveys. They didn’t post online or on a physical bulletin board and wait for people to respond. They didn’t buttonhole people who regularly visit LGBTQ service organizations. They asked people who had gotten literally random phone calls and letters from polling organizations and agreed to answer questions. They may well have reached trans populations that might be completely disconnected from the social networks that have answered previous surveys, and potentially less subject to peer pressure and groupthink.

I’ve already identified some conceptual problems with the survey, and I plan to discuss more, but I think this is a major advance, and it’s probably why the poll has some results that don’t fit with the typical picture painted by many trans advocates and previous surveys. I’ve already mentioned one finding: that 22% of respondents report that the transgender actions they’ve taken have made them at least somewhat less satisfied. I’ll talk about other findings that I think are worth highlighting. Honestly, I think it’s interesting that these haven’t been discussed more since this survey was released last March.

Previous posts about the 2022 Washington Post/Kaiser Family Foundation Transgender Survey:

What it means to be living

grvsmth
The author, wearing a dress with a white on blue floral pattern, a black sweater and gold-plated earrings and necklace, lifts a glass of prosecco to the camera
Is this living or what?

Recently I was talking to a friend about the question from last year’s Washington Post/Kaiser Family Foundation survey, “Has living as a gender that is different from the one assigned to you at birth made you (more satisfied) or (less satisfied) with your life?” She said, “but have you really lived as a woman?” Many trans people would bristle at the question, but for me it’s a valid question, and it gets to the heart of one of the conceptual problems at the heart of the survey. It also provides important context for interpreting the results.

The survey question is important because it fits into a debate: some people say they’re unsatisfied with the results of transgender actions they’ve taken. Some reactionaries have seized on this dissatisfaction as an excuse to impose restrictions on trans people. Some transgender advocates have responded by denying that anyone is ever really dissatisfied with transgender actions in any way worth discussing.

The issue is important to me personally, even though I would say that the transgender actions I’ve taken have overall increased my satisfaction with life. One reason I think I’ve been so satisfied is that I have paid attention to experiences of dissatisfaction that others have shared, and to my own dissatisfaction with the results of particular actions I’ve taken.

So have I lived as a woman? If you add up the days I’ve spent interacting with people as a woman over the course of my life, you might reach half a year. The hours I’ve spent as a woman in online communities, by message and now video, would probably add more than a year’s worth of interactions.

In that time I’ve had many of the negative experiences that women often point to. I’ve been catcalled and groped by men. I’ve walked in uncomfortable shoes, and had difficulty finding clothes that fit me well. I’ve avoided going places alone out of fear for my safety. I’ve also had good experiences, such as women welcoming me or showing they feel at ease around me.

But we’re talking about a survey, so how did the researchers at the Washington Post and the KFF define “living as a gender”? And this gets to one of the major conceptual faults with the survey. Immediately before the satisfaction question, they asked, “Currently, how much of the time do you physically present as a gender (such as a man, woman, or nonbinary) that is different from the one assigned to you at birth?” Here are the responses:

Option n %
None of the time 82 16 %
Some of the time 175 34 %
Most of the time 103 20 %
All of the time 155 30 %

The pollsters describe how they used this question to decide who to ask the following question about satisfaction with living as a different gender. 16% of respondents said that currently, they never physically present as a different gender, and the pollsters did not ask those 82 people the question about living as a different gender. I was assigned male at birth and currently I present myself as a woman some of the time, so if I were part of the panel I would have been asked about living as a woman.

Even more serious: the responses to the satisfaction question were used for the headline of the main article reporting the study: “Most trans adults say transitioning made them more satisfied with their lives.” So the researchers asked about “currently … physically present as a gender … some/most/all the time”, then presupposed that that meant “living as a gender” and reported it as “transitioning.”

I’ve written about the conceptual problems with “transition” before: there are some who argue for an expansive definition of transition, but in practice the concept is so overwhelmed by its prototype that when people hear the word, they tend to think of permanent, full-time, binary gender transition with major hard body modifications, including hormones and surgery. The Washington Post and KFF researchers did not ask respondents about that, they asked about current physical presentation, including “some of the time.”

The way the surveyors asked this question, it would be possible for a respondent to think they were saying that it felt good to wear on a leather jacket in their bedroom for ten minutes every few months, and that gets reported as “transitioning made them more satisfied with their lives.” That’s why in my earlier post on the topic I interpreted it as “satisfaction with transgender actions” rather than talking about transition.

I know people who manifest their gender in ways other than physical presentation. Those manifestations do have an effect on their social lives, and these people consider themselves to have transitioned, but they would not have been asked about living as a different gender.

Similarly, the use of the word “currently” is highly problematic. For most of three years (1999-2002) I rarely presented as a woman in public, because I lived in places where I judged the culture to be intolerant of fluid gender presentation. If I had been given this questionnaire then, I would not have been asked about my satisfaction with living as a different gender, because I was not “currently” presenting as a woman at all.

Did any of the 82 respondents who said they didn’t currently present as a different gender do so in the past? We don’t know, because the researchers didn’t ask them.

The issue with “currently” is a serious problem because there is a debate over people who physically present full time as a different gender, but then stop. This is often called “detransition.” Some people say they stopped presenting as a different gender out of dissatisfaction with the results of that action, others because they fear violence or discrimination based on their actions, and others say they simply decided to do something different.

The Washington Post/KFF poll could have shed some light on the motives of people who present as a different gender and then stop. But by adding “currently” to the question about physical presentation, they excluded anyone who might have presented as a different gender in the past from the satisfaction question.

Did the pollsters do these things intentionally? I doubt it. I think they had a particular idea in mind, the prototypical full-time, body-modifying transitioner, but they were aware that there are people who didn’t fit that mold, so they tried to make the questions flexible enough to cover those people.

I think the researchers underestimated how many people would turn out not to fit their preconceived idea of a transition. They discuss that in some of the articles they wrote about the survey results, but they realized that after they’d finalized the questions. I hope they get a chance to clean up those questions and run another survey soon.

Why people care about transgender satisfaction – and dissatisfaction

grvsmth

As I wrote recently, last March the Washington Post released a poll they conducted with the Kaiser Family Foundation. This was the first representative survey of transgender people in the United States, and it told us a bunch of interesting things about what it means to be trans in this country. In fact, the responses highlight some problems with the researchers’ concept of transgender, and suggest some ways the next survey could be better, but right now I want to focus on where this survey fits into the current political climate around transgender issues.

Over the past fifty years, people have acquired more rights to take transgender actions like wearing different clothes and changing official gender designations, and people and governments have shown more approval and respect for those actions. There has also been a negative reaction, and over the past ten years, right wing politicians have discovered that they can profit off of this reaction to bring in votes and donations. They have introduced a wide range of bills outlawing various transgender actions, and some of those bills have become law.

From one point of view, clothing, official gender designations and marriage are arbitrary, so why is this drawing such a strong reaction from some quarters? Much of it is the same reaction to any change in gender or sexuality, going back to Lot and Diogenes: visible non-conformity to rules of gender and sexuality are threats to the patriarchal power structure, and have the potential to reduce birth rates, which many nationalists see as a source of power.

There are also some parents who are terrified of their children’s growing autonomy. They panic when they see their kids taking on new names and pronouns, and maybe making new friends that they don’t know.

There are some women who find power and safety in what they think of as all-female spaces. They feel threatened by anything they perceive to be an intrusion by men or boys, or an abandonment of those spaces by people they see as women and girls.

There is a certain cost that people face in accommodating other people’s transgender actions. At a minimum there are changes in paperwork and getting used to different names and pronouns. Some transgender actions, like hormone treatment or surgical procedures, have a significant cost, and many trans people want those costs to be paid by the government, or by private health insurance offered through employers.

You might have noticed that the reactions I described above are largely from people who aren’t doing transgender things, feeling threatened by people doing trans things. As a trans person I can roll my eyes at them, but they’re important to acknowledge, because they’re often the real motivation, disguised with a false concern for trans people.

What about the risks to actual trans people? They fall into four broad categories: irreversible changes, financial, safety and opportunity costs.

The first is that many of these transgender actions – particularly “hard” body modifications like surgical procedures and hormone prescriptions, but even official gender designations or the public pronouncements someone might make about their gender and/or sexuality – are difficult or even impossible to reverse. Some of them, particularly surgical procedures, come with risks that are widely known, but that may be downplayed for various reasons.

There is simple monetary cost: the costs of hormones and surgery I described above, but also hair removal, hairstyling and clothing.

Then there is a meta-threat that is kind of difficult to articulate: it’s that if people are allowed to take these transgender actions then we’ll be subject to discrimination, harassment and violence. Some people seem to use that threat to justify subjecting others to discrimination, harassment and violence, which is an incoherence on the order of “the beatings will continue until morale improves.”

Finally, there is an opportunity cost. Some transgender actions can take a lot of time, energy and focus, which is not available for other activities like hobbies. It may even take away from exercise, relationships, and work.

Over the years, many trans people have reported that they’re dissatisfied with the results of certain actions they’ve taken, particularly ones that are difficult or impossible to reverse, like “hard” body modifications, but also about the financial and opportunity costs, or the cost of retaliation by others. Some people take an extreme position that it was all for nothing, but many people say that the benefits of these actions simply didn’t justify the costs.

Of course, this dissatisfaction and regret has been weaponized by the non-trans actors I described above: insecure parents, certain kinds of feminists, and right wing politicians and activists. Some may be genuinely concerned, but often it is clear that they are simply using these trans people to attack other trans people.

In reaction, many transgender advocates have staked out their own extreme positions, downplaying or even denying any dissatisfaction with transgender actions. Some advocates have attacked the character of individuals who have reported dissatisfaction. Others have pointed to surveys that report little to no dissatisfaction, but many of these surveys rely on unrepresentative samples, so there is no way to rule out the possibility of sampling bias.

As I understand it, this is a primary goal of the Washington Post/KFF poll: to shine some light on the issue by asking a representative sample of Americans who’ve taken transgender actions whether they’re satisfied. Here are the responses to the question “Has living as a gender that is different from the one assigned to you at birth made you (more satisfied) or (less satisfied) with your life?”

A lot more satisfied 45
Somewhat more satisfied 33
Somewhat less satisfied 17
A lot less satisfied 5

As I discussed in my last post, I would answer this question as “A lot more satisfied” as well. The Washington Post and KFF researchers consolidated the satisfied and unsatisfied responses to report the total as 78% satisfied, and used this for their headline, Most trans adults say transitioning made them more satisfied with their lives. Again, there are conceptual problems with this question, but I think it does show, broadly speaking, that the vast majority people who take transgender actions tend to be satisfied with the results of those actions.

But let’s take a minute and focus on the 17% (73 respondents out of 427) who said that living in a different gender from the one assigned to them at birth made them somewhat less satisfied with their lives, and the 5% (21 respondents) who said it mad them a lot less satisfied.

These are not the tiny dissatisfaction percentages trumpeted by many advocates. This is almost a quarter of the people who reported that they lived in a different gender from the one assigned to them at birth. It doesn’t mean people shouldn’t take transgender actions, but it does mean that we need to be careful about which actions we take, when we take them, and how.

In order to make good decisions, we need to know what can go wrong, and what can make for an unsatisfactory experience. That means that we need to listen to the people who are dissatisfied, and hear their stories. It’s not all about the backlash.

A big reason that my experience with transgender actions has been so satisfactory is that I heard some of those stories of dissatisfaction and regret before I ever seriously considered taking any actions. Informing myself and being careful about my actions has helped me avoid doing some things that I think I would have regretted.