Retraction of gay marriage study leaves L.A. canvassers feeling jilted
Laura Gardiner knew she was making a difference with her work.
As national mentoring coordinator at the Los Angeles LGBT Center’s Leadership Lab, she and her colleagues had toiled to train 1,000 volunteers who had fanned out across Los Angeles and beyond, lobbying voters in precincts that had cast ballots against gay rights.
The idea was to push back against prejudice, house by house — and over the years, the group’s internal evaluations indicated, the Leadership Lab had gotten quite good at changing voter minds.
When an independent study published in the prestigious journal Science confirmed the group’s success, Gardiner had been thrilled.
Then, last week, a report was issued raising significant doubts about the study’s validity.
“It felt like being cheated on in a relationship,†she said Thursday after the journal issued a formal retraction. “Breakup songs have been cathartic this week.â€
The study had excited readers well beyond Gardiner’s circle for its surprising conclusion that a single doorstep chat could prompt a skeptic to embrace marriage equality. It even reported a “spillover†effect that extended to household members who didn’t talk to canvassers.
Although the findings contradicted a body of research that said firmly held opinions weren’t easily swayed by lobbying and political advertising, they seemed to confirm an idea people were happy to embrace — that honest conversation and open minds could bring people together.
The study made headlines across the country and was featured on the public radio program “This American Life.†Its primary author, UCLA graduate student Michael LaCour, scored a job offer from Princeton University.
As LaCour prepared to decamp for New Jersey, he handed off the study to a team at Stanford and UC Berkeley.
That’s how things began to unravel.
The new researchers were the first to suspect that something wasn’t quite right with LaCour’s data. They produced a report that persuaded LaCour’s coauthor, Columbia University political scientist Donald Green, to request a retraction last week.
The editors of Science agreed, citing three reasons for retracting the study. They said LaCour lied about the way he recruited participants for his study and did not pay volunteers to complete online surveys, as he had claimed. They also said he lied about receiving research funding from the Williams Institute, the Ford Foundation and the Evelyn and Walter Haas Jr. Fund. LaCour’s attorney has acknowledged both of these deceptions.
Perhaps most significantly, the editors said, “LaCour has not produced the original survey data from which someone else could independently confirm the validity of the reported findings.â€
LaCour still maintains that his study is sound. He said he has been preparing a “definitive response†to his critics, which he plans to provide Friday.
“I appreciate your patience, as I gather evidence and relevant information,†he said Thursday in an email to The Times.
The whole tale began at the LGBT Center, said Leadership Lab director David Fleischer. He came up with the idea of the canvassing effort after voters approved Proposition 8, the 2008 ballot measure that repealed same-sex marriage in California.
“We wanted to understand the problems and what it would take to remedy them,†he said.
Canvassers started knocking on doors in the Los Angeles area in 2009, focusing on precincts that had voted in favor of the gay marriage ban by a margin of at least 2 to 1. By 2013, with more than 12,000 one-on-one conversations under their belts, group members were pretty sure they were changing voter minds.
Fleischer was eager to find out whether an independent analysis would prove they were right — and he knew just the person to call.
“I had admired Don’s work for years,†he said, praising Green’s careful studies that apply the rigorous methods used in medical research trials to the squishier realm of political science.
Green was intrigued, but he warned Fleischer that research would probably show the LGBT Center was having a smaller impact than it thought.
“I said OK — I just want to know,†Fleischer recalled.
Green made the introduction to LaCour. The men met in April 2013, Fleischer said, and soon started working together.
To assess whether the talks were changing minds, LaCour was charged with designing an online survey to gauge residents’ opinions on marriage equality. Then canvassers knocked on their doors. In 20-minute conversations, they discussed the benefits of marriage and explained that gays and lesbians wanted to experience them too.
The study also included a “placebo†group of residents who were lobbied about recycling and a control group that wasn’t lobbied about anything.
LaCour was supposed to administer follow-up surveys after the visits were complete.
The study results purported to show that after speaking with canvassers, people were more inclined to support same-sex marriage, an increase from 39% to 47%. One year later, support for gay marriage was 14 percentage points higher among people who were lobbied by a gay person and 3 percentage points higher among those who were canvassed by a straight person, the study said.
With LaCour wrapping things up at UCLA, the LGBT Center brought on David Broockman, a professor of political economy at Stanford, and Joshua Kalla, a political science graduate student at UC Berkeley, to carry on the research.
But as they made plans to track a forthcoming canvassing project the Leadership Lab is undertaking in Miami, they started noticing problems with the work. For instance, as they began their own pilot survey, they noticed that their response rate was “notably lower†than LaCour’s.
When they sought additional advice from the survey firm that LaCour had reportedly employed, they quickly realized something was amiss.
“The survey firm claimed they had no familiarity with the project and that they had never had an employee with the name of the staffer we were asking for,†the researchers wrote. “The firm also denied having the capabilities to perform many aspects of the recruitment procedures described.â€
Alarmed, Broockman and Kalla turned a skeptical eye toward LaCour’s data and began investigating further with the help of Yale political scientist Peter Aronow. They soon realized that some of the paper’s key data were identical to that of a different national survey conducted in 2012: the Cooperative Campaign Analysis Project. That discovery raised “suspicions that the data might have been lifted from CCAP,†the researchers wrote.
The researchers compiled their findings in a 26-page report and sent it to Green. When confronted with the findings, Green immediately sent a letter to Science requesting that the paper be retracted.
“I am deeply embarrassed by this turn of events and apologize to the editors, reviewers, and readers of Science,†Green wrote.
Fleischer and his staff learned of the investigations last week, he said. By Thursday, he had started informing volunteers too, who were busily preparing to head off for the Miami project.
Although the news had been “a punch in the gut,†he said it didn’t sway the group’s confidence in the work it was doing.
“We know we proceeded with integrity,†he said. “We’re eager to embrace the opportunity we now have to have our work measured, which is what we always wanted.â€
@LATerynbrown
@montemorin
