How should journalists cover coronavirus preprint studies?

When a story in the Los Angeles Times recently claimed that the novel coronavirus behind COVID-19 had mutated into a more contagious version, it was quickly amplified by other outlets and stoked fears that the virus was becoming more dangerous.

The problem is, that wasn’t necessarily true. Scientists quickly took to Twitter to point out the research paper the story was based on was a preprint — a first draft of scientific findings. The research had flaws of its own, several scientists said, and was overstating its conclusions about the contagiousness of the virus.

Headlines that don’t stand up to scrutiny are among the perils of covering preprint scientific articles, which have yet to go through the academic rigor of peer review — a process in which experts in the field evaluate a study, critique it, and recommend new experiments or more cautious conclusions before clearing the study to be published. During the pandemic, preprints abound as researchers scrambling to understand the coronavirus try to share their preliminary results as quickly as possible.

But with the public hungry to get word of strange symptoms, new treatments or useful predictions, news coverage of preprints could unintentionally share bad information if journalists aren’t vigilant, says Sharon Dunwoody, emerita professor of journalism and mass communication at the University of Wisconsin–Madison.

“Most individuals invest little time in evaluating what they’re seeing. Even if they might know what to look for, they just don’t give much time to it,” she says. “So, the onus is on the journalist. They have to be extra careful.”

To properly evaluate and write about preprint studies, Dunwoody says journalists need to treat the research for what it is: unverified information. Many science journalists are used to covering preliminary information, for example from public conference presentations.

“What they’re doing in those instances is what they should be doing here, which is vetting the research with the help of other scientists,” says Dunwoody. “A lot of stories reporting on a preprint acknowledge that the work hasn’t made its way through peer review. I appreciate that. The next thing I expect to see in the story is other scientists reacting to the research that is being covered.”

The original version of the LA Times story gave prominence to quotes from the preprint’s author, taken from the researcher’s Facebook page. Proper evaluation, Dunwoody says, would require interviewing independent scientists who were willing to look at the study and critique it.

Normally, new scientific results aren’t made public until after they’ve been published in a peer-reviewed journal. But preprints are becoming much more common. These first drafts are sent to public, online preprint servers, where anyone can read the study. It’s a transparent way to communicate about science, quickly. And for scientists, these early results can be a huge boon.

“When it comes to issues such as the pandemic — where speed of discovery is essential — if you’re a scientist, your ability to get your hands on preprints is a crucial aspect of moving things along,” says Dunwoody.

With the public hungry to get word of strange symptoms, new treatments or useful predictions, news coverage of preprints could unintentionally share bad information if journalists aren’t vigilant.

Unfortunately for news consumers, our reading habits leave us vulnerable. Research suggests that we assign credibility to a news channel and then let down our guard on evaluating the stories in that channel.

“When we need to decide what information is credible, we tend to make judgments about the channel, not about the stories,” says Dunwoody, who counts The New Yorker and The New York Times among her trusted channels. Even scientists use the same rule of thumb. Nobody’s immune.

“That means if a story in one of those credible channels reflects flawed science, we’re not even going to notice,” says Dunwoody. “If it shows up in one of our trusted channels, as far as we are concerned it is true.”

So that means that as journalists sift through the deluge of preliminary information, they have to take a critical eye to these early reports. Question the researcher, talk to other scientists, and follow the old journalism mantra: If your mother says she loves you, check it out. If something about the research raises a red flag, cautionary prose may be ineffective; the reader may be better served by avoiding coverage of the story, or waiting for peer review.

Dunwoody says there are parallels between the rise of preprint scientific articles and another quick, raw, but chaotic new format: social media. Both are full of promise, but also peril, for processing information.

“In some ways preprints join the world of social media and all these other channels now that allow you to pick and choose information that you want to believe,” says Dunwoody. “Preprints have added to transparency — and to the messiness of the information environment we live in.”