A Study in Humans
What People Worried About the Quality of COVID-19 Studies Should Know
Recalled COVID-19 studies have raised valid questions about who or what to trust about the pandemic.
Whether we can trust science matters.
If it’s unreliable, that matters, but if the research is of high quality, we should condemn sensationalist headlines. Let’s look at the facts, then you decide.
Scientists have completed over 40,000 COVID-19 related studies since January 2020. When the pandemic began, many scientists asked how they could apply their knowledge to the problems facing us.
They successfully collaborated across culture, place, creed, and politics all around the world. Nothing requires them to do it, but that speaks to what science is and does. It is the pursuit of truth for the good of humankind.
Still, if the recent efforts had flaws, it’s important to recognize them and how it affects recommendation.
Total Number of COVID-19 Studies
- 40,127+ COVID-19 related studies published as of Aug 12, 2020
- 30 full retractions
- 1 temporary retraction
- 3 formal letters of concern
People often struggle to comprehend numbers at that scale, so here’s what that might look like.
We should look critically at all potential sources for guidance, but the standard for your judgment must reflect reality. Comparisons happen between the options on the table, not against perfection unless someone is offering it.
If someone could offer us perfection or even a better record, science would be the first field to say, “go with them.” As it stands, that’s not an option. Discrediting our best hope of navigating out of a crisis without supplying a better alternative, without offering solutions, renders us defenseless and harms society far more than any nefarious actor.
Do studies with serious problems exist?
Yes, so we must look at which studies have problems, why, and what it may mean.
Science already knows it can be wrong. That’s why retractions exist, why we seek those most able to criticize our work, why we adapt to new evidence, and why conclusions should not come from a single study.
The average time between publishing & retraction
The average time from publishing to retraction was 18 days. One study published and retracted the same day. It had already been previously published and was the publisher’s error. Here retraction did not reflect poor quality of the study.
The most common subject among retracted studies
The most common subject of retracted studies was hydroxychloroquine. The issues varied from questionable ethics to scientists conducting credible work, but whose data had been proprietary. The data supplier failed to provide sufficient transparency concerning the origins of the data.
Some retractions reflected an honest mistake. Often, temporary retractions reflect this error. The authors may correct and publish an updated study.
Science may be better with friends
The vast majority of studies involving multiple countries were less common on the list. The vast majority of studies involved authors from a single country, though not the same country each time. It’s possible this was merely a coincidence or correlation, but perspectives that differ from our own can bring insight and new ideas to projects.
Word Map of Countries with Retracted Studies
Retractions themselves don’t necessarily show poor quality of studies or a problem with the science in that country. Countries that do more studies will have more chances for an error. Concerning issues like intentionally doctoring or manipulating results have happened, but retractions happen for other reasons, too.
I hope this helps you better understand the state of research on COVID-19 and that the retraction rate is about 0.00084%.
Does Science have all the answers? Again no, but I strongly suspect the advice given by scientists has a better track record than most other groups. That’s the detail that matters.
Compare that to elected officials, for example.
Do they actively seek their most competent peers to criticize their work and accept the feedback? Do they actively seek more information to ensure their first impression was not mistaken, following with a change of opinion where necessary? Could anything broadly create a unique conflict of interest in their conduct, like needing the public to like them?
That’s the tricky thing about looking for the truth. Not everyone wants to hear it and still, we still have to tell it.
Next: The Truth About the Most Famous Retraction of All
References:
Chen, Qingyu, Alexis Allot, and Zhiyong Lu. 2020. “Keep up with the Latest Coronavirus Research.” Nature 579(7798): 193–193.
eliesbik, Author. 2020. “Thoughts on the Gautret et al. Paper about Hydroxychloroquine and Azithromycin Treatment of COVID-19 Infections.” Science Integrity Digest. https://scienceintegritydigest.com/2020/03/24/thoughts-on-the-gautret-et-al-paper-about-hydroxychloroquine-and-azithromycin-treatment-of-covid-19-infections/ (August 10, 2020).
Gautret, Philippe et al. 2020. “Hydroxychloroquine and Azithromycin as a Treatment of COVID-19: Results of an Open-Label Non-Randomized Clinical Trial.” International Journal of Antimicrobial Agents 56(1): 105949.
Lowe, Derek. 2020. “The Latest Hydroxychloroquine Data, As of April 11.” Science Magazine. https://blogs.sciencemag.org/pipeline/archives/2020/04/11/the-latest-hydroxychloroquine-data-as-of-april-11 (August 10, 2020).
Maisonneuve, Hervé. 2020. “COVID-19 : La Bagarre Pour Publier Vite n’est Pas Acceptable Car de Mauvais Articles Sont Diffusés Aux Journalistes et Patients.” Rédaction Médicale et Scientifique. https://www.redactionmedicale.fr/2020/03/covid-19-la-bagarre-pour-publier-vite-nest-pas-acceptable-car-de-mauvais-articles-sont-diffus%C3%A9s-aux-.html (August 10, 2020).
Marcus, Author Adam. 2020. “Elsevier Investigating Hydroxychloroquine-COVID-19 Paper.” Retraction Watch. https://retractionwatch.com/2020/04/12/elsevier-investigating-hydroxychloroquine-covid-19-paper/ (August 10, 2020).
Rosendaal, Frits R. 2020. “Review of: “Hydroxychloroquine and Azithromycin as a Treatment of COVID-19: Results of an Open-Label Non-Randomized Clinical Trial Gautret et al 2010, DOI:10.1016/j.Ijantimicag.2020.105949.” International Journal of Antimicrobial Agents 56(1): 106063.
