A story about gender inequality in the scientific research industry. A deep dive into the daily rhythm of the immune system. A look at vaccine efficacy for COVID-19 variants. These are some examples of messages based on forms — Research studies that have not been officially reviewed by the scientific community.
Journalists have historically been discouraged from reporting on preprints because they feared the results might be exaggerated, inaccurate, or outright wrong. But our new research suggests that the COVID-19 pandemic may have changed things, pushing preprint-based journalism into the mainstream.
Read more: Preprints: How draft academic papers have become indispensable in the fight against COVID
While this new normal offers important benefits for journalists and their audiences, it also brings risks and challenges that deserve our attention.
Peer Review and the Pandemic
Traditionally, studies must be read and evaluated by at least two independent experts before they can be published in a scientific journal – a process known as “peer review”.
This is not the case with preprints, which are posted online almost immediately without formal verification. This immediacy has made preprints a valuable resource for scientists working on the COVID-19 pandemic.
The lack of formal verification makes preprints a faster way to communicate science, albeit a potentially riskier approach. While peer review isn’t perfect, it can help scientists identify flaws in data or communicate their findings more clearly.
Studies indicate that most preprints stand up to peer review. Nonetheless, in some cases, results can change significantly between the time a study is published as a preprint and the time it is published in a peer-reviewed journal, which can take more than 100 days on average .
A “paradigm shift” in science journalism
As researchers in journalism and science communication, we have been closely monitoring media coverage of preprints since the beginning of the pandemic. In a study, we found that a variety of media outlets reported on COVID-19 preprints, including major media outlets such as The New York Times and The guard.
Unfortunately, many of these outlets failed to mention that these studies were preprints, leaving listeners unaware that the science they were reading had not been peer-reviewed.
READ ALSO: In The Rush For Coronavirus Information, Unreviewed Scientific Papers Are Released
We took a closer look at how and why journalists use preprints. In in-depth interviews, we asked health and science journalists about the strategies they have used to find, verify, and communicate preprints and whether they plan to report on them post-COVID-19.
Our peer-reviewed published study found that preprints have become an important source of information for many journalists, and some plan to continue using them post-pandemic. Journalists reported actively searching for these unverified studies by visiting online servers (sites where scientists post preprints) or monitoring social media.
Although some journalists were unsure about continuing to use preprints, others said these studies had caused “a complete paradigm shift” in science journalism.
A careful equation
Journalists told us they value preprints because they are more up-to-date than peer-reviewed studies, which are often published months after scientists have conducted the research. As one freelancer we interviewed put it, “When people die, you have to get things moving a little bit.”
Journalists also appreciated that preprints are freely accessible and usable, while many peer-reviewed journal articles are not.
Journalists weigh these benefits against the potential risks to their audience. Many expressed high levels of skepticism about unverified studies and raised concerns about the possibility of spreading misinformation.
Some journalists provided examples of issues that had been “extremely clouded” by preprints, such as whether schools should remain open during the pandemic.
Many journalists found it important to label preprints as “preprints” in their articles or to mention that the research was not peer-reviewed. At the same time, they acknowledged that their audience would probably not understand what the words “preprint” or “peer review” mean.
Additionally, reviewing preprints seemed to be a real challenge for journalists, even those with advanced academic training. Many told us they relied heavily on interviews with experts to verify the findings, with some journalists organizing what they called their “own peer review”.
Other journalists simply relied on their intuition or “gut feeling,” especially when deadlines were looming or experts were unavailable.
Supporting journalists in communicating science
Recently, media organizations have started publishing resources and guides for reporting on preprints. While these resources are an important first step, our findings suggest that more needs to be done, especially if preprint-based journalism is truly going to be permanent.
Whether by providing specialized training, updating journalism school curricula, or revising existing professional guidelines, we must support journalists to review and communicate about preprints effectively and ethically. The quality of our news depends on it.