How to Approach Preprints for Quality Science Reporting?
How to report on preprints: verify credibility, highlight limitations, consult experts, and prioritise accurate, transparent science reporting over breaking news speed
Part of a journalist’s job is to find new stories to tell. And if you’re a science journalist, you likely want to be on the lookout for new research emerging in the fields you cover. These days, one of the channels researchers increasingly use to share their work early is preprint servers, where new studies can receive an initial round of comments, scrutiny, and input from peers and the broader community.
The Rise of Preprints During the COVID-19 pandemic
This practice has steadily grown in recent years; however, it experienced a dramatic surge, especially in the biological and medical fields, during the COVID-19 pandemic. This highlighted the pros and cons of making early findings accessible not only to academics but also to the non-specialised public, with a heightened risk of misunderstandings, miscommunication (sometimes intentional), and even social consequences (e.g., the spread of the false claim that the SARS-CoV-2 virus contained genetic sequences of the HIV, which originated from a preprint that was swiftly retracted).
Weighing the opportunities and risks of highlighting unpublished scientific findings, journalists should approach this topic with a critical mindset, paying particular attention to transparency. It is essential to clearly explain what a preprint is, its strengths and limitations, and – importantly – to follow up on the story whenever new developments emerge. Whether a preprint is peer-reviewed and published in an academic journal, undergoes substantial changes, or is even retracted, these changes should be thoroughly communicated to the audience, as part of responsible reporting.
To help academics, journalists, and the general public navigate the complex world of preprint publishing, a few years ago, the Dutch Research Council (NWO) produced a “Practical Guide to Preprints.” This document offers essential information on what a preprint is, how to recognise it (for instance, a non-peer-reviewed article usually states at the beginning that it is “unpublished” or a “working paper”), and how to assess whether its content is reliable, particularly when it appears especially newsworthy.
Some practical tips for reading preprints
- Check the references: Are most of the articles cited in the preprint published works? If so, in what type of journals?
- Check the authors’ academic record: a quick Google Scholar search to look at previous publications and citation rates (i.e., how often the authors’ works are cited by other scientists – a sign of good standing in the relevant scientific community) can do the trick. A note of caution: early-career researchers may not yet have many citations or publications. In such cases, check the track records of their co-authors (often senior researchers or supervisors) to estimate the likely quality of the work.
- Read critically and consult experts: seeking insights on the research from trusted contacts within your professional network can help you understand the relevance and reliability of the findings.
Another key recommendation from the Dutch Research Council concerns transparency. It is essential to clearly state, in a plain and accessible language, that the article presents preliminary findings and has not yet undergone the usual critical evaluation by scientific peers. While peer review is not a definitive quality check, as we pointed out in an earlier post, it is generally able to filter out flawed research.
However, one of the most common – and perhaps most important – pieces of advice from experienced science journalists is this: don’t rush to get the news out. Award-winning American reporter Christie Aschwanden wrote in Wired, during the height of the COVID-19 pandemic, that “bad data is worse than no data at all”. In other words, as she told Roxanne Khamsi for a piece in The Open Notebook, reporters should not sacrifice quality and rigour just to be the first to break a story. While it may be tempting, that approach is unlikely to yield long-term benefits. Instead, Aschwanden urges caution as the safer and most responsible path: “Reporters,” Khamsi writes, “might even benefit by assuming any given preprint is probably a false lead.”
More readings:
