On 2nd June, medical journal The Lancet issued a statement of concern about one of its published scientific articles, the conclusions of which were made on the basis of data the authors had obtained from a small US company named Surgisphere. Surgisphere’s methodology and practices had been called into question following a recent investigation by journalists at The Guardian. By 4th June, The Lancet had fully retracted the article that had used Surgisphere data. And The Lancet wasn’t alone: The New England Journal of Medicine, an American medical journal, also issued a retraction of one paper (an article in a scientific journal) it had published that had used data from Surgisphere.
What Is Surgisphere, And Why Does It Matter?
Surgisphere is an American healthcare analytics company that claims to have access to medical data from at least 1,200 hospitals. Its data sets have been used for analysis that has formed the basis of papers in medical journals across multiple continents. Published findings in reputable scientific and medical journals can be used to guide medical treatment and to form public health policy. The current scandal concerns a number of papers, including the retracted Lancet article, some of which suggested that a medication named hydroxychloroquine worsened outcomes in COVID-19 patients. The papers in question ultimately led the World Health Organisation to halt trials into the use of the drug in COVID-19 patients. If it transpires that hydroxychloroquine actually improves outcomes in COVID-19 patients, then patients will have been denied an effective treatment as a result of these medical journal articles.
The influence of Surgisphere’s datasets aren’t limited to the drug hydroxychloroquine: one article written by the company’s Chief Executive indicated that an anti-parasite drug named Ivermectin reduced death rates in severely ill COVID-19 patients. After being published online without undergoing the usual peer review process, it prompted the Peruvian government to add the drug to its national COVID-19 treatment guidelines. Another article co-authored by the company’s Chief Executive, and which was published in the New England Journal of Medicine, found common blood pressure lowering medications known as angiotensin-converting–enzyme inhibitors and angiotensin-receptor blockers were not associated with a higher risk of harm in COVID-19 patients. This article was peer-reviewed.
The extensive investigation into Surgisphere by The Guardian found, amongst other things, that:
- Its dataset has clear errors, such as listing one Asian hospital as being in Australia;
- An employee listed as a science editor appears to be a science fiction author and fantasy artist whose professional profile suggests writing is her full-time job. This calls into question the appropriate expertise of Surgisphere’s employees;
- Prior to The Guardian’s investigation into the company, the “get in touch” link on Surgisphere’s homepage redirected to a WordPress template for a cryptocurrency website;
- Surgisphere’s Chief Executive Dr Sapan Desai has been named in three medical malpractice suits, unrelated to the Surgisphere database. In an interview with The Scientist publication, Desai previously described the allegations as “unfounded”;
- In 2008, Dr Desai launched a campaign on crowdfunding website Indiegogo, promoting a wearable “next generation human augmentation device that can help you achieve what you never thought was possible”. The device never came to fruition.
Surgisphere claims to employ 11 people, which some data analysts have claimed is too small a research base to produce verifiable and rigorous data analyses. After The Guardian’s investigation into the company was published, enquiries were made by The Lancet’s independent peer reviewers, after which the Lancet said that:
“Surgisphere would not transfer the full dataset, client contracts, and the full ISO audit report to their servers for analysis as such transfer would violate client agreements and confidentiality requirements. As such, our reviewers were not able to conduct an independent and private peer review and therefore notified us of their withdrawal from the peer-review process.”
Are Retractions From Scientific Journals Common?
No, primarily because scientitic journal articles usually undergo a process called peer review. Medical and scientific journal articles are also often viewed as part of ongoing discussions on a specific topic. When medical journals publish articles, their contents are not a final say on the matter, and the scientific community is encouraged to scrutinise them.
How Does The Peer Review Process Work, And How Do Articles End Up In Medical Journals?
Peer review is the process by which articles are accepted for publication in an academic journal. If a researcher submits an article to a journal, first the editor will consider the article. If the article does not fit the journal’s remit, it will be rejected. If it does, the editor will decide whether to reject the article, suggest amendments to the article, or accept it for peer review. After this point, the journal’s peer reviewers will analyse the paper and make a recommendation of whether to accept or reject the article. They may also sometimes suggest additional work that would need to be done before publication, or suggest minor amendments to the article. The editor will then make the final decision whether or not it should be published.
This system of peer review has existed almost entirely unchanged for several hundred years. Its strength lies in the acceptance of an article being in the hands of many learned people in the field of study, potentially reducing the scope for erroneous judgments prior to publication. It allows for scrutiny, though it isn’t foolproof. If there is a wide enough consensus among peer reviewers that supports an idea that is incorrect, then this will necessarily be reflected in journals’ publishing output. Examples of errors made by peer reviewers include the publication of an article by Andrew Wakefield in the 1990s that erroneously correlated use of the MMR vaccine with autism.
It could be argued that the problems with the Surgisphere dataset should have been spotted by peer reviewers, who review primary source material for any innacuracies. However, peer reviewers often take presented data in a paper at face value and, in any case, it ultimately transpired that Surgisphere would not release their original data. This scandal is part of a broader problem relating to the current COVID-19 pandemic: so many articles and information sets are being generated on the disease it may be difficult to spot trends through this noise. The vast amount of research being conducted by scientists is sometimes referred to an “infodemic” for this reason.
Whatever is the truth about Surgisphere and hydroxychloroquine, it is right to be concerned – as any disinformation can lead to negative outcomes in healthcare, and in COVID-19 treatment. The potential effects, therefore, of any dishonesty or incompetence in the handling of any data by any research group should not be underestimated.