The lead author of a study claiming that short conversations can dramatically alter a person’s view on same-sex marriage has issued a retraction upon learning his co-author may have forged the data.
The study in question, titled “When contact changes minds: An experiment on transmission of support for gay equality,” was published in Science back in December 2014. It showed that gay political canvassers, when conversing one-on-one with constituents for as little as 20 minutes, could influence the vote in favor of same-sex marriage. What’s more, the study claimed that the effect could last upwards of a year, and that it was “contagious” within the voter’s household.
It was a major finding — and an observation allegedly backed by some rather compelling data. As political scientist Andrew Gelman wrote at the time:
What stunned me about these results was not just the effect itself—although I agree that it’s interesting in any case—but the size of the observed differences. They’re huge: an immediate effect of 0.4 on a five-point scale and, after nine months, an effect of 0.8.
A difference of 0.8 on a five-point scale . . . wow! You rarely see this sort of thing. Just do the math. On a 1-5 scale, the maximum theoretically possible change would be 4. But, considering that lots of people are already at “4” or “5” on the scale, it’s hard to imagine an average change of more than 2. And that would be massive. So we’re talking about a causal effect that’s a full 40% of what is pretty much the maximum change imaginable. Wow, indeed. And, judging by the small standard errors (again, see the graphs above), these effects are real, not obtained by capitalizing on chance or the statistical significance filter or anything like that.
But as RetractionWatch is reporting, all that “compelling” data should have raised some flags.
Accusations of wrongdoing first emerged after a trio of researchers — Stanford’s David Broockman, UC Berkeley’s Joshua Kalla, and Yale’s Peter Aronow — could not replicate the results. Their ensuing paper highlighted eight statistical irregularities in the data set — data the researchers claim could not have been “collected as described.”
The study’s lead author, Donald Green from Columbia University, issued a formal retraction in consideration of these accusations, and after questioning his co-author, Michael LaCour, a graduate assistant from UCLA.
“I am deeply embarrassed by this turn of events and apologize to the editors, reviewers, and readers of Science,” Green told RetractionWatch.
More from POLITICO:
In an email to POLITICO, Green said that he spoke with LaCour by phone on Tuesday, and that he “maintained that he did not fabricate the data but told me that he could not locate the Qualtrics source files for the surveys on the Qualtrics interface or on any of his drives.”
“I asked him to write a retraction, and he indicated he would do so, but when it did not appear last night, I sent off my own retraction,” Green wrote.
In another email to POLITICO, LaCour wrote that he has read the investigation and will respond: “I’m gathering evidence and relevant information so I can provide a single comprehensive response. I will do so at my earliest opportunity.”
What a mess. Science writer Ed Yong has expressed his thoughts on Twitter:
And credit where credit is due:
Note: An earlier version of this article incorrectly stated that Green was LaCour’s advisor during the study; and (via POLITICO) that Qualtrics was directly involved in the study. We received an email from Qualtrics with a clarification: “Qualtrics is a self-service platform on which our customers create and own their own survey content and response data. As such we can confirm that Qualtrics did not collaborate with Michael LaCour or any other party to author a study about public opinion surrounding same-sex marriage, as some media outlets erroneously reported.”
Top photo: Davidlud/CC