Why did NEJM and Lancet retract two COVID-19 papers?
Academics and laypeople alike may have now heard of recent retractions of two COVID-19 related papers published in high-profile medical journals. To recap: “Cardiovascular Disease, Drug Therapy, and Mortality in Covid-19” published in the New England Journal of Medicine (NEJM) and “Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis” published in The Lancet – among the two most prestigious journals in medicine – were recently retracted mere hours apart.
Retraction is a process by which a journal essentially renounces a paper if significant issues that threaten the validity of its conclusion arise after publication; in this Internet Age, it means putting a big bold disclaimer on the website to make it clear that the study has been retracted, as you can see on both of the above journal’s sites. Retractions are not to be taken lightly, as they can be exceedingly embarrassing for a journal hoping to maintain respect for its peer review process.
Why were both of these articles retracted immediately one after the other? As one of my favorite blogs, Retraction Watch, which reports on retractions in the scientific literature, recounts, the answer is found in a common link between the two papers: Samir Desai, who co-authored both papers. Desai is the founder of a company named Surgisphere, which has the jargon-laden mission statement to “relentlessly pursue advancements in machine learning, artificial intelligence, and big data … [empowering] healthcare providers to make better, faster, and more accurate decisions.” Surgisphere provided the data for both studies. The other coauthors seem to be academics associated with various hospitals and research centers; it’s unclear what previous relationship they had with Desai prior to the collaboration.
However, Retraction Watch reports that some concern in academic circles began to percolate about the validity of the data (although I could not find specific criticisms; they are perhaps somewhere on the Twittersphere). Soon, both NEJM and Lancet grew concerned enough to ask for the raw data that the authors used. That’s where the story takes a turn: three of the four authors of the Lancet paper, barring Desai, wrote to the editor claiming that Desai and Surgisphere were not allowing them access to the original data. While their correspondence claims that Surgisphere cited some sort of legal agreement as the reason for why it could not share the data, they make it clear that they lost faith in the veracity of the data because of this refusal. The Lancet retracted the paper, and hours later, this time with the request of Desai (!), NEJM retracted the other paper.
The story is intriguing precisely because it sounds so mysterious, and even after browsing the web for more information, I was left with more questions than answers. The original correspondence that felled the Lancet paper was couched in professional terms and so it is a bit difficult to decipher what the underlying story is. What exactly did Desai say that led his coauthors to turn on him? Was Desai refusing to share the data because he doctored it, and hid behind some obscure legal justification? Without the data, it’s impossible to say to what extent it was doctored. But then, why did he request the NEJM retraction? Did he realize the writing was on the wall after the Lancet retraction? It’s an altogether puzzling story and stubbornly devoid of details.
Symptom of a larger issue?
In the days since the retractions, people have wondered: was this just a sole incident, or a symptom of a larger problem in science, especially during COVID-19? I think there are significant reasons to lean towards the latter. As I have written about before, numerous issues with academic publishing have arisen or at least exacerbated due to COVID-19. Journals have had to face a deluge of submissions, as researchers running the gamut of scientific specialties and sub-specialties try to cash in on the COVID-19 gold rush (both figuratively and literally, as governments and grant organizations have opened up huge sums of money to fund COVID-19 research). I can’t blame the researchers entirely though – they are perhaps simply reacting to the incentives and pressures present in the world of academia: doing more, publishing more, and obtaining more grants leads to citations and promotions.
On the other hand, journals aren’t entirely blameless either. Journals, especially at the very top end of the distribution like NEJM and Lancet, heavily curate their reputation by accepting only the most “high-impact” papers. This leads to greater readership of their journal, more subscriptions, more attention in the news media, and overall more prestige. It’s why NEJM and Lancet accept only (approximately) 5-10% of papers submitted to them. NEJM, for example, explicitly states on its website, that they look for “scientific accuracy, novelty, and importance”. A paper can be entirely scientifically correct, but if it is not deemed novel or important enough by the editors – a subjective measurement at best – it has no chance.
I would argue that such journals are seeking to curate their reputation during COVID-19 as well, and are most interested in those papers that can significantly impact our understanding of it; in medical journals, those papers that can significantly impact medical practice would be deemed important. For example, the retracted Lancet paper showed that hydroxychloroquine – a much talked about COVID-19 treatment – was unsafe to use on COVID-19 patients, and prompted the World Health Organization to stop a trial studying it’s efficacy (although it resumed the trial after the retraction). I would argue that such curation can inevitably lead to instances where certain papers are given favorable treatment by editors if they think it will be a “hot” paper.
Counterpoint, or, in defense
At the same time, I’m hesitant to deride all of science as being tainted because of the issues above. It’s understandable that researchers feel the need to publish something about COVID-19 and I applaud the three co-authors who stepped forward and told the Lancet that they do not have faith in the veracity of Desai’s data.
As well, perhaps the reaction to these retractions has been disproportionately large simply because NEJM and Lancet are two of the most prestigious and widely-read journals in medicine. I think it also sets up an interesting imbalance whereby articles published in the likes of those journals are more criticized and discussed precisely because more people are reading them. That is to say, there could very well be many multitudes of incorrect papers in low-end journals, but no one has noticed, because, well, no one is reading them.
And what of peer review? It is an imperfect tool in the best of times, and these are evidently not the best of times. Reviewers, who volunteer their time on top of their usual academic duties, do not – and cannot – redo the entire paper line-by-line in order to find inaccuracies. Inevitably, their job has been made harder by COVID-19-related interruptions to their regular duties and the increased volume of submissions. I do not envy them.
While evidently there are issues with scientific publishing, the Surgisphere story also demonstrates, to me at least, the good. Desai’s coauthors were willing to speak honestly with the editor in the interests of scientific integrity – indeed were willing to retract their own work for it. The errors in the papers were still found by other researchers, a sort of post-publication peer review. And really, two retractions among the numerous COVID-19 papers NEJM and Lancet have published is a surprisingly good track record, all things considered. The Surgisphere story, while unsettling, ultimately does not shake my trust in either of these two journals: they were proactive in dealing with the post-publication concerns and were quick to retract the papers after the issues were confirmed. While the Surgisphere story evidently exposes some undesirable aspects of science, one can’t ignore the people and institutions that worked to right the transgression.
Ultimately, science isn’t perfect, and perhaps never can be, but it works towards it.
Maaz's Notes 