On the campaign trail leading up to last year’s presidential election, then-candidate Donald Trump kept up a refrain that proved popular with his supporters: The election, he claimed, was rigged by fraudulent voters, many of whom were illegal immigrants. Trump proclaimed he had peer-reviewed research to support his claim: a 2014 study that said enough noncitizen immigrants were registered to vote to swing an election. What Trump didn’t tell his base is that other scientists had debunked the study, identifying a major flaw in the data analysis.
Coverage of the 2014 study tended to focus on its mistakes. But FiveThirtyEight staff writer Maggie Koerth-Baker wanted to find out why those mistakes occurred. In her May 2017 feature “The Tangled Story behind Trump’s False Claims of Voter Fraud,” Koerth-Baker goes beyond the drumbeat of criticism in the press and chronicles how an honest but crucial error derailed the study—and what the study’s politicization meant for its authors. It was a major undertaking that involved tracking down statisticians, political scientists, voting experts, science-communication experts, and sociologists who study how science is used in politics.
Her perseverance paid off in surprising ways. When she talked to the study’s lead author, Koerth-Baker learned something about the work that no other reporter had uncovered: the project idea came from an undergraduate researcher who is the daughter of Pakistani immigrants and is one of the paper’s coauthors.
Here, Koerth-Baker tells TON editor-at-large Jeanne Erdmann how she unraveled complicated statistics and traced the study’s surprising backstory to produce a complex yet accessible piece that went beyond what other media organizations had written. (This interview has been edited for length and clarity.)
This story was so in depth—it must have been a bear to chase down. What motivated you to take it on?
I had actually been avoiding writing this story. I knew during the election that this paper was floating around; I also knew that many other people had debunked the study, so I kept ignoring it. One of the things I wanted to get away from was that culture of just writing stories to tell people how wrong they are. That’s the bread and butter of blogging and I didn’t feel like doing that anymore. And then in late January or early February of this year, this paper again got cited by the Trump administration. [Statistician and FiveThirtyEight editor-in-chief] Nate Silver and our politics editor came to me and said they really wanted to write something about this. We talked about how to write about the study [in a way] that wouldn’t make me hate myself in the morning.
How did you decide on an approach you could live with?
We centered on this “zombie” framework, of how something that is functionally wrong and has been debunked by scientists continues to have an afterlife by circulating through the media, and through social media. The study did contain a measurement error that is easy to make when analyzing what happens at low frequency in a large data set. We concluded that “wrong” wasn’t the right way to frame it, but rather that the study wasn’t doing a good job of doing what it wanted to do, but still kept getting spread around.
A lot of reporting went into this piece. What did you do first?
I sent a bunch of questions over to [lead author] Jesse Richman, and got an interview. He had been horrified over how [his study] had been used by Trump, and also by others claiming that immigrants could very easily swing elections based on his work, which he told me showed the opposite. He was also horrified his work was used as a xenophobic propagation of anti-immigrant ideas. Richman felt like even people who were supporting him were not understanding what the paper said, and I thought that was really interesting.
Very seldom do we go back and ask scientists what their thinking was about how their work has been presented in the public sphere.
I very quickly figured that it was not a thing where he had an agenda, where he was trying to make some conservative point; it’s not like this study originally set out to find evidence that supports Donald Trump. I saw how this was framed in a lot of those debunking stories, and that’s not really what was going on. It was a lot more complex than that. I wanted to help people understand how you get to a conclusion like this without it being a partisan set piece.
Richman really took the brunt of this. It’s something [he] hasn’t talked a lot about with me other than to set the record straight. He’s not some idiot who is ignoring a flaw that nobody else would ignore. Lots of other people have made this same mistake; lots of other people are making this same mistake.
Very seldom do we go back and ask scientists what their thinking was about how their work has been presented in the public sphere. With something like politics, that’s a super important thing to know, because it helps other scientists, and because it helps the public understand what scientists are thinking. I think it also makes for a more nuanced story.
A key thing you learned in that interview was that the idea for the study came from one of Richman’s undergraduates at the time, Gulshan Chattha, who is the daughter of Pakistani immigrants. But she left science after the paper came out. How did you track her down?
I asked Richman if he would connect me with Chattha, and he basically said, “I don’t usually connect her with journalists but I really like your questions, and I like that you are interested in getting something other than just saying, ‘Wrong, wrong, wrong, stupid.’”
How did discovering her involvement affect how you reported the story?
This really mattered in how I thought about the story, and also how I wrote about it. I had already planned to make this a story about how science gets misinterpreted through pop culture and politics, and spreads, but when there’s a student behind it that just adds to the narrative. You would reasonably expect a professional scientist to be prepared for the media attention this study brought, but you wouldn’t expect an undergrad to be prepared for that. [And] I think that the story became more about the human impact by having Gulshan be the daughter of immigrants herself.
Chattha is very like you and me. She was an average person thrust into a situation that is very normal for political science, but not very normal for average people. I thought it was important to have that view on what was happening, and it was interesting that it existed within the authorship of the paper.
Given how sensitive the subject of immigration is—sensitivities exacerbated by Trump’s use of her study to make false claims about illegal immigrants—was it difficult to broach the topic of her family background and get her to talk about how it led to this study?
[When] I reached out I told her that I would really like to talk about how her background influenced her interest in this topic initially, and how it influenced what happened after it was published. She was very willing to talk about that.
I wanted to have three things that if you walked away from this story, you could tell somebody in a cocktail party.
During our interviews, I never got the sense that she was uncomfortable. She hadn’t been in the news at all; she had watched it one step removed. From talking to her, I know that the reactions that she saw hurt her, but it’s different than if she were getting 5,000 emails a day personally. I think that mattered to why she was willing to talk to me. I think if she’d been someone who’d been attacked for a year, she might not have been. But it was really sad to me that her own experience built her up and made her want to be a scientist, and then she came away feeling like this wasn’t a place she wanted be.
Did you outline this story before sitting down to write it?
What I did was to go back through all of my notes and make a big list of important details and quotes and things that I thought were important. Then reading through that big list, I wanted to have three things that if you walked away from this story, you could tell somebody in a cocktail party. And then I framed everything around those three things. After that I banged out the first draft in two days. Other than shifting some stuff around, it ran fairly close to what it was when I first wrote it, which is crazy and insane.
What were the biggest reporting challenges you ran into?
Understanding the data analysis, because I don’t have a background in statistics. At first, I had sources explaining it to me, but it took Nate sitting down with me for it to make more sense. Then, I was able to write about it a little better. Also having Nate and others on staff who really know statistics gave us the authority to say what was actually wrong, and not just what [critics] say was wrong. I don’t think I could have done this story if I’d worked someplace else. I have really been loving being on staff, but this is also one of those moments where it’s just so apparent what benefits you have from really being part of a team, and the things you can do as part of a team that you can’t do when you are on your own.
You use a graphic to illustrate the statistical glitch in the research paper and how easily an error like that can happen. Can you talk about how you decided on a graphic to handle that explanation?
I think that the graphic turned out to be incredibly important. The data error that we’re talking about is really simple, but it is also really difficult to wrap your head around. I am a very visual thinker. I need visual images to make sense of things. Nate made the suggestion that we should make this an interactive. We did that with Christie Aschwanden’s story “Science Isn’t Broken.” I think it turned out really well because it made the statistical error tangible.
Did this story present any structural challenges?
My original draft had an analysis of what the paper got wrong further down in the story. We went back and forth and ended up deciding we wanted to have the numbers and data higher up because that’s what FiveThirtyEight does, and that’s what people expect from us. I [also] knew we needed the data piece higher up to make certain people understand what we were talking about, because the main thing we did that other people who had covered this story hadn’t done was explain this very well.
But by placing the data early, there’s still that risk that I made the story a takedown piece that focused on why this paper is wrong, which is what I didn’t want to make it. I wanted the story to be about a bigger picture of American society.
I also wanted to make sure that I did right by Gulshan and by Richmond. They aren’t stupid, and they aren’t bad people.
Given the subject matter, did you give any special attention to making sure you weren’t inadvertently doing anything that was culturally insensitive?
There are so many stories you read where some man has written a story about a woman, and it comes out and all of these women just face-palm—like, “If you’d just let one woman read this she could have told you what was wrong with it before it was published.” I kept that in mind with Gulshan; she’s Pakistani, and her parents are immigrants. As a white woman whose family has been here for years, I didn’t want to do the same thing.
I asked a friend, who’s also the child of Pakistani immigrants, to read it to see if anything was glaringly obvious that I was being culturally insensitive on and didn’t realize it.
This is the first time I’ve had a story where I really felt this big responsibility for talking about someone else’s experience and someone else’s life.
She didn’t see any gaffes. The biggest thing that she suggested was something that I’d also been thinking about. In the original draft that she read, Chattha’s story got lost a little bit at the end, and we hadn’t come back to her. This wasn’t a heritage-based critique, but my friend felt like we stopped caring about Chattha’s story, and we needed to finish it. During my edits, partly based on her feedback, I made certain the story came back to Chattha.
Do you have any regrets about any aspect of the story?
When I was writing this I had a miscommunication with the rebuttal-paper researchers. They had a three-person paper and I only talked to [Stephen] Ansolabehere and [Brian] Schaffner. I asked about the third colleague [Samantha Luks] and I got the impression that [she] didn’t talk to the press because she worked for a private company, and they didn’t want to draw attention. I never contacted her. I should have. After the story came out she was upset—not that we hadn’t talked to her, but that we hadn’t mentioned her, other than briefly in passing. On the original citation we mentioned that it had three authors but I kept referring to it the rest of the way through as the Ansolabehere-Schaffner paper, and that makes a problem, right? It came across in the way that there were only two authors, and they were male authors. We went back and revised it to show that the paper had three authors, not two.
I apologized later, and she was very kind. This is something she’s been dealing with her entire career, and has happened on this paper a lot. That was something that I got wrong for sure on this. I should have just called her—that’s what it boils down to.
We sat down and figured out how we’re going to deal with this in the future at FiveThirtyEight. We always have this issue of more people who are authors, and who deserve credit, than we can really list out in a story. When there are only three authors, we will make sure that all three are important all the way through.
What stuck with you after the story was published?
This is the first time I’ve had a story where I really felt this big responsibility for talking about someone else’s experience and someone else’s life. When you write a story, you necessarily leave some things out, and you put some things in. You only have so much space that frames a person. Somebody gives you this part of themselves and you might never know whether you did right by them, or whether you hurt them, and that’s hard. I don’t know if I did a good job of that.
That’s a responsibility, and it’s a risk, and maybe we don’t train journalists well enough to do it. It’s something that I think is going to stay in my work going forward. I think it made me more aware that as a journalist I have a lot of power over people’s identities. It has primed me to be more aware of my sources’ vulnerability and my power.
TON cofounder and editor-at-large Jeanne Erdmann is an award-winning freelance health-and-science writer based in Missouri. Her writing has been published in Nature Medicine, Nature, Women’s Health, Discover, The Washington Post, Slate, Aeon, and elsewhere. She is on the board of the Association of Health Care Journalists. Follow her on Twitter @jeanne_erdmann.
