Preprints and the pandemic

One of the most interesting things about the COVID-19 pandemic, at least from a meta-academic sort of way, is the way it has dramatically upended academic publishing. Suddenly, reams of new research about the SARS-CoV-2 virus and the COVID-19 disease is constantly being put out (interestingly, not just in fields like virology as we would expect, but even from economists, mathematicians, physicists, sociologists, etc.) and that research needs a place to go. While many biological and medical journals have tried to keep up with the volume of submissions and have indeed even managed to cut review times for submissions, another venue that these researchers have increasingly turned to is posting preprints.

Preprints are version of papers before the peer review stage. They are typically a precursor to publication in a peer-reviewed journal (although sometimes, like Perelman’s proof of the Poincare conjecture, for which he won – then denied – $1 million, they are a way of completely bypassing the traditional publication process). While they have long been a common medium of disseminating papers in some fields like economics, math, and physics on the arXiv preprint server, in bio and med fields (except for the subfield of quantitative biology, which has its own section on arXiv) preprints have not really been used. That began to change in 2013, when bioRxiv, a preprint server for the life sciences, and in 2019, when medRxiv, for clinical medicine preprints, were launched.

Those launches were not met without opposition. Especially in the medRxiv case, there was significant concern among some that medical research was inherently different than math or physics because medical studies tend to have a greater societal fallout – a quick sweep of social media or a news site will yield numerous discussion on medical topics like diet, exercise, weight loss, “cleanses”, and the like, but perhaps nowhere near as much hype on a topic like Ben Green and Terry Tao’s proof of arbitrarily long arithmetic progressions in the primes. As hurtful as this is to me to say as a fan of mathematics, it’s easy to see that a medical study falsely purporting a new cancer treatment could be far more damaging to people’s lives than a faulty proof that P=NP.

So, in a sense, a certain sensitivity needs to be accorded to medical research. As well, the Ingelfinger rule, named for the erstwhile editor of the New England Journal of Medicine (NEJM), prohibited NEJM from publishing results that had been presented elsewhere, either at a conference, in the news media, or in other journals. This rule was adopted in other journals as well, and effectively disallowed publishing of preprints in medical journals for a long time. This effectively kept the preprint idea from medicine for decades, while it began to thrive in math and physics.

Then came the pandemic. As I said before, it arguably upturned many of the old academic norms in bio and med. In a sense, it was a perfect storm. medRxiv had just been launched a year ago, but was seeing slow adoption. Some medical journals had already begun to relax the Ingelfinger rule to allow for preprint posting. These two facts, combined with the volume of research in the midst of COVID-19, precipitated a surge in preprint posting on bioRxiv and medRxiv. Now, some of the most important papers that form the base of our knowledge about COVID-19 were first posted as preprints. An example is Hoffman et. al.’s paper which was the first to describe how the virus enters cells; first posted on bioRxiv, it was later published in Cell, where it has, at the time of writing, been cited 1474 times (according to Google Scholar).

MedRxiv now receives hundreds of submissions a week, a huge surge since before COVID-19. Indeed, I mined medRxiv metadata for the first four months of 2020 and found that on many days, 100% of all submissions were related to COVID-19. The pandemic has arguably driven growth in both medRxiv and bioRxiv usage as the graph below shows.

This is helpful as a fast way of disseminating information in an evolving situation. Yet, the issue with the lack of peer review remains. While both servers do some basic level of screening for rigor and scientific content, this is not to the level of traditional peer review. To their credit, both websites have disclaimers on the top alerting readers that the results are not peer reviewed and that decisions about human health should not be made based on preprints.

Still, some fail to heed these warnings: infamously, the LA Times reported a grossly exaggerated story claiming SARS-CoV-2 was mutating to become more deadly, on the basis of a bioRxiv paper. Such reporting and “hyping” of COVID-19 research, especially ones that are not peer reviewed, only serve to increase panic – and possibly even cause panic fatigue, causing people to take the pandemic less seriously.

What’s the answer to these problems? It’s difficult to say. There are some possible solutions. I was immensely interested by an initiative by the Precision Immunology Institute at the Icahn School of Medicine in New York. This initiative, called the Sinai Immunology Review Project (SIRP), posts comments on bioRxiv and medRxiv papers in the field of immunology. Relevant papers are “reviewed” by trainees (ie. PhD students), and the reviews are validated by a faculty member. Their initiative is so interesting to me, especially as it harnesses the brainpower of trainees at a time when their research work may have been halted by the pandemic. You can read a wonderful description of their work here in Nature Reviews Immunology. I wonder if other initiatives like this have sprung up, similarly using students; I for one would love to be involved in writing reviews.

Ultimately, for the time frame in which COVID-19 emerged, I think that academics and publishers have been surprisingly fast to adapt. In any rapid transition, there are inevitably going to be some kinks. How the world of academic publishing will look like post-pandemic depends strongly on if and how those kinks are worked out – I have some ideas on that which I may explain in a future post.