Articles / Good research / Media / Washington Post/KFF Survey

How the Washington Post/KFF poll represents underrepresented trans people

The Telephone Survey Support Center call center at NORC. At least fifty people are sitting in cubicles at computer workstations, holding pieces of paper.
The Telephone Survey Support Center call center at the University of Chicago National Opinion Reseearch Center.

I’ve been posting recently about the transgender survey released last year by the Washington Post and the Kaiser Family Foundation. Some of the things I write are critical, but I want to be clear how much I appreciate and value it as a real advance in understanding the situations and opinions of trans people in the United States. It’s a very big deal, because it really is the first representative survey of trans people in this country.

I’ve written extensively before about why representativeness matters in science, why transgender surveys are no exception, and how so many transgender surveys fail. So what does the Washington Post/KFF survey do that the other surveys don’t? As the researchers describe in their survey methodology report, they draw from representative panels.

Other studies have been unrepresentative because they relied on trans people responding to broadcast announcements (even in web forums), or surveyors reaching out to trans people they already know. These methods ignore several populations: people who might not be looking at those announcements, people who might not respond to a general announcement, and people who aren’t known to surveyors. That can exclude whole communities of trans people.

The Washington Post/KFF study draws on three existing panels that are built to randomly sample the population of the entire country. One, the AmeriSpeak panel from the National Opinion Research Center at the University of Chicago, uses something called “Address Based Sampling,” dividing the country into segments and sending letters to randomly selected addresses from each segment, asking them to participate in a panel. The others, the Gallup panel and the SSRS Opinion Panel, use a combination of Address Based Sampling and “Random Digit Dial,” where they call random phone numbers and ask people if they want to be on a panel.

The panels are groups of people, contacted through one of these methods, who’ve agreed to answer questionnaires. They’re paid a small amount, between $5 and $20, for every questionnaire they answer. In one questionnaire, the organization that runs the panel (NORC, Gallup or SSRS) asked them if they identified as trans; the Washington Post/KFF researchers don’t give the exact text of those questions. From these three panels, the researchers found 483 people who identified as trans, and then added 29 people from previous KFF surveys and three more, for a total of 515.

I have to note here that while you can pay people to answer questions like this, you can’t force them, in any ethical way. Similarly, you can reach people by mail and phone, but if people change their mailing addresses and phone numbers, they’re less likely to be able to join one of the panels.

These panels are not quite representative of the entire population of the country. Their answers can only be generalized to those people in the country who have stable postal addresses or phone numbers, and are willing to spend time answering surveys for NORC or Gallup or SSRS, for $5-20 each time. And the responses to this survey are not representative of all transgender people in the US, but of trans people with stable addresses or phone numbers who are willing to answer surveys.

It may sound complex, but I want to stress how much better this is than previous surveys. They didn’t post online or on a physical bulletin board and wait for people to respond. They didn’t buttonhole people who regularly visit LGBTQ service organizations. They asked people who had gotten literally random phone calls and letters from polling organizations and agreed to answer questions. They may well have reached trans populations that might be completely disconnected from the social networks that have answered previous surveys, and potentially less subject to peer pressure and groupthink.

I’ve already identified some conceptual problems with the survey, and I plan to discuss more, but I think this is a major advance, and it’s probably why the poll has some results that don’t fit with the typical picture painted by many trans advocates and previous surveys. I’ve already mentioned one finding: that 22% of respondents report that the transgender actions they’ve taken have made them at least somewhat less satisfied. I’ll talk about other findings that I think are worth highlighting. Honestly, I think it’s interesting that these haven’t been discussed more since this survey was released last March.

Previous posts about the 2022 Washington Post/Kaiser Family Foundation Transgender Survey:

Leave a Reply

Your email address will not be published. Required fields are marked *