How the L.A. Times, KFF polled American immigrants

By David LauterSenior Editor

Sept. 17, 2023 4 AM PT

Seventy-five thousand, eight hundred surveys mailed, more than 13,000 hours of phone interviews, translations into nine languages in addition to English: Conducting an accurate survey of the nation’s hugely diverse immigrant population required resources far beyond what’s needed for a typical poll.

Earlier this year, KFF, formerly known as the Kaiser Family Foundation, and the Los Angeles Times carried out that survey along with several supplemental focus groups. Today, The Times begins publishing “Immigrant Dreams,” a series of articles, photos and videos based on the survey findings as well as work by Times reporters, photographers and graphic artists.

Drawing on an unprecedented poll, this series tells the stories of immigrant life in America today, putting their voices in the foreground.

Read the stories

The effort aims to fill gaps in knowledge about the foreign-born share of the American population, which is near a record high for the last century — adding up to roughly 1 in 6 adults in the U.S.

Immigration has played a dominant role in public debate for three decades, from California’s Proposition 187 in 1994 to Donald Trump’s 2016 presidential campaign and beyond. Much of that debate, however, involves other people talking about immigrants, often in ways that inaccurately describe their lives. The survey and the “Immigrant Dreams” project were designed to put the voices and experiences of immigrants at the forefront.

Fear of immigration and the changes it brings have driven a large part of the division in American society over the last generation, leading to a stalemate over immigration policy that has now lasted for three decades.

In recent years, the division over immigration has accelerated. Although he aimed his harshest rhetoric at unauthorized immigrants, Trump in 2016 became the first major-party presidential candidate since Calvin Coolidge to campaign in favor of reducing legal immigration, a policy he partially succeeded in carrying out. Polling repeatedly has shown that a desire to reduce immigration and concern over how immigrants may change American society are among the strongest factors binding together his “Make America Great Again” movement.

A lot of the immigration debate, however, focuses on chaotic and compelling scenes of people trying to get across the border, not on the lives led by the millions of immigrants already in the U.S.

The “Immigrant Dreams” project aims to widen the lens, to show the optimism and resilience of immigrant families, as well as the hardships and discrimination many of them face. That alone won’t reduce the country’s divisions over immigration, but it’s potentially a start.

“We wanted to tell the deeper story about the immigrant experience in America,” said Drew Altman, chief executive of KFF.

“It’s just really, really challenging to do.”

To capture the diversity of the immigrant population, the survey was conducted in Chinese, Korean, Vietnamese, Tagalog, Arabic, Spanish, Portuguese, French and Haitian Creole, in addition to English. It was designed to yield representative samples not only of the overall immigrant population, but also people from 11 individual national or regional origins, including Mexican, Central and South American, Chinese, Indian and sub-Saharan African immigrants.

While other researchers have polled individual immigrant groups, no previous survey has yielded data that provide a similarly in-depth look across different immigrant populations, said KFF vice president Samantha Artiga.

The poll also provides a representative sample of immigrants in California, which is home to more than 1 in 5 foreign-born U.S. residents.

Among immigrants, themselves, the state has a positive reputation. Seven in 10 immigrants in California say they feel welcome in the state, a share that’s significantly lower elsewhere in the country, the poll found.

Some findings may surprise people, including the high level of optimism that immigrants of all national origins express about their lives, despite the hardships that many face.

Even the nonsurprising results, however, can be “important ... in the current political context,” Altman said.

Accurate polling always has challenges, but “this is the far extreme.”

— Ashley Kirzinger, director of survey methodology, KFF

Planning for the survey began more than two years ago. The first question was whether an accurate poll of the immigrant population was possible.

Most polls start with a list: all registered voters in a particular state, for example, or all telephone numbers in a specified geographic region. A pollster takes the list, contacts a specified number of respondents and goes from there.

With immigrants, “there is no list,” said Ashley Kirzinger, KFF’s director of survey methodology.

In addition, “there are issues of trust,” she said. “This is a group that for good reason would be skeptical about efforts to track them.”

Many immigrants come from countries where “the government is the only one doing surveys, and sometimes that can be malevolent,” Kirzinger said. Others are living in the U.S. without legal authorization or have family members who are undocumented, making them leery about answering questions.

In late 2021 and early 2022, KFF and the national polling firm SSRS tested survey methods to see whether any would work.

Some approaches were discarded because they produced inaccurate samples, skewed too heavily toward immigrants who speak fluent English or who are relatively affluent, Kirzinger said.

By the spring of 2022, however, the pilot tests showed that several approaches would work to obtain an accurate sample. The pollsters settled on three primary methods: The largest group of potential survey takers received mailed invitations, often with multiple mailings. They could respond by doing the survey online, mailing back a paper copy or calling a toll-free number to be interviewed and could receive a gift card as a small incentive if they participated. Pollsters contacted a second group by calling a random selection of prepaid cellphones, which are often used by recent immigrants. KFF also made calls to numbers that had been flagged in previous surveys as belonging to non-English speakers.

With the methodology set, KFF and Times reporters and editors began work on drafting the survey questions. KFF also consulted outside groups that work with immigrants for help in framing questions and designing the survey.

With any survey, it’s tricky to write questions that elicit an accurate response and don’t push people in one direction or another. The task gets harder when translating the words into multiple languages, requiring pollsters to extensively pretest the questionnaire to make sure it means the same thing to a Chinese speaker, for example, that it means in Spanish and that both are the same as the original English.

“You have to do cognitive interviewing to make sure each question is measuring the same concept” in each language, Kirzinger said. “It takes months, months.”

Problems can arise not just from the words but from the cultural context of the survey respondents. A question about the status of “people in my household,” for example, had to be rewritten after the pretesting showed confusion among various immigrant groups about who “a member of my household” referred to, Kirzinger said.

Finding interviewers who could survey people in some of the languages posed another challenge, especially with languages like Haitian-Creole, which has many fewer speakers than Spanish or Chinese. Polling in Creole, however, allowed the survey to capture the experiences of a population in which fluent English speakers are rare.

“It’s just a monumental effort,” Kirzinger said. KFF has conducted a wide variety of polls of different groups in the U.S., and each poses different challenges, she said, but “this is the far extreme.”

Ten languages, thousands of phone calls: Accurately polling immigrants posed unprecedented challenges

More to Read

More From the Los Angeles Times

Most Read in World & Nation