In the last few weeks, there has been a common trend for media outlets to create diagrams and charts that display daily COVID-19 case numbers and mortality figures, often to facilitate comparison between countries. While these graphs are visually appealing, at Lit we have decided not to do this. Here we explain why.
Don’t compare apples and oranges – they aren’t the same
When we compare COVID-19 case numbers and mortality rates between countries, we often aren’t comparing like with like. Countries don’t all report COVID-19 cases the same way, and each country has its own, individual, testing regulations. Some countries lack the infrastructure to conduct widespread testing, and others have opted to test only a minority of the general population. In many countries, such as France and Iran, testing is now only conducted on those with severe COVID-19 symptoms due to a limited testing capacity. Since 12th March, the UK has been conducting COVID-19 tests exclusively on those ill enough to be hospitalised.
Other countries have conducted much more widespread testing, even of those without symptoms. In South Korea, over 450,000 tests have been carried out as of 5th April 2020. This widespread testing system was partly funded by the South Korean government, which operated a scheme where those requesting a test would be charged 170,000 won (around £112) but reimbursed if their test comes back positive, or if their test result is used in a scientific study.
Countries in East Asia are more likely to conduct widespread COVID-19 testing in part because many of those countries have emergency protocols in place that were created in the aftermath of the 2003 SARS outbreak. Countries such as Singapore had these guidelines in place in order to counter any future SARS or SARS-like virus; Singapore’s emergency protocol emphasises the importance of case identification and contact tracing. Click here to read the Singaporean public health protocol in the event of coronaviruses and severe influenza viruses (opens in new tab).
This diversity of testing protocols has a knock-on impact on apparent mortality rates in different countries. Those that have conducted widespread testing have naturally detected a greater number of mild and asymptomatic cases of COVID-19 than those that have not. The more mild cases that are detected, the lower a country’s mortality rate will appear. This fact was discussed by Prof Chris Whitty, the UK’s Chief Medical Officer, during a press conference on 19th March, just after the UK announced it would limit COVID-19 testing to those admitted to hospital. Whitty noted that this would artificially increase the UK’s apparent mortality rate.
Countries have differing testing regulations. But that’s only one piece of the puzzle.
Countries have different testing regulations, but there is another feature that has an impact on national comparisons: the role of politics in disease reporting.
On March 21st, 2020, the Washington Post reported that US officials had attempted to warn US President Trump in January that there was a distinct possibility the Chinese authorities had minimised the scale of the COVID-19 outbreak. Around the same time, a team of scientists at Imperial College London attempted to calculate independently the total number of COVID-19 cases that had occurred in Wuhan in the early stage of the outbreak, so they could compare their calculation to the official Chinese government figures. They calculated the likely number of cases by extrapolating from the number of exported COVID-19 cases that had been reported in countries other than China, in mid-January: they published their findings on January 16th. At this point, the Chinese authorities claimed the official number of COVID-19 cases was 41: the Imperial College London estimate put the number of cases at 1,723. This kind of analysis has a high margin of error, but the results seem significant.
The Chinese authorities have a history of downplaying or delaying international reporting of novel viruses: one prominent example is their reporting of the 2003 SARS outbreak, which originated in China. The first case of the 2003 SARS outbreak is thought to have occurred in November 2002, but Chinese officials did not report the novel respiratory syndrome to the World Health Organisation until February 2003, and only did so after repeated demands from the World Health Organisation that they disclose their disease data. Over 300 deaths had occurred in China before the World Health Organisation was informed of the outbreak.
China is not alone in publishing infection figures that have come under scrutiny. Questions have also been raised regarding Iran’s reporting of COVID-19 cases, as their reported mortality rate early in the outbreak at one point reached 16% – more than 10 percentage points greater than that for most other countries.
It should be noted that COVID-19 infection figures reported by the World Health Organisation are simply made up of any individual country’s official figures. So if one country’s officials have over- or under-stated their infection figures, the WHO’s official figures will reflect this. The WHO does have an interactive map that is updated daily, which displays all countries’ official COVID-19 infection figures: click here to access it (opens in new tab).
Not so fast, Sherlock!
Even if we take into account the problems with official COVID-19 infection figures and the differences between them, there’s still another problem – that of timescales.
Trends take time to emerge, and any public health policies will take time to be effective. This lag period between the enactment of social isolation policies and decreasing virus rates is calculable using arithmetic given the information we now have about the virus. COVID-19 has an incubation period (that’s the time it takes for a person to develop symptoms after being infected) of between 2 and 14 days, and on average it takes one to two weeks after onset of symptoms for a COVID-19 victim to require hospitalisation. COVID-19 cases that had been transmitted very shortly prior to the introduction of social distancing measures will therefore take anywhere from seven to 21 days, on average, to reach a hospital. And even then it may take as long as one or two weeks for any patients who are hospitalised to reach an intensive care unit, or to die. That time adds up.
This all means that it takes a considerable length of time for any trends to emerge, or to be discerned from changes in daily mortality figures. A moderate dip in mortality figures that lasts a few days, particularly early into social distancing measures, isn’t enough to indicate a downward trend, and we run the risk of disseminating fake news when we assume it is.
The UK government’s COVID-19 strategy papers make reference to the fact that extended quarantine measures, such as full lockdown or widespread school closures, will delay peak incidences of COVID-19 by two to three weeks. The wave of COVID-19 deaths in the first two to three weeks of lockdown are cases that were likely transmitted prior to lockdown, so it will take at least this period to see any meaningful change in mortality figures.
In our haste to understand the virus and the situation across the globe, narratives can be created that aren’t really evidenced. It’s just too early to tell.
Drawing conclusions this early into the pandemic leads to errors
Trying to force comparison between countries’ COVID-19 infection and mortality rates when available data are so limited is risky. It would be too easy to draw conclusions that turn out to be incorrect. Whilst there is some information, there is insufficient evidence at this point in the pandemic to make accurate comparisons. That’s why we won’t be producing graphs or diagrams based on minimal data, or comparing countries this early in the outbreak. The reality is, the scientists themselves don’t know a great deal about this virus or how to contain it – so neither do we.