Coronavirus: Can we trust the data?

Barely a day goes by in the lifecycle of the coronavirus pandemic without a new model or analytical tool aiming to chart the spread or predict the outcome.

But data for COVID-19 is far from perfect.

Each afternoon the Department of Health and Social Care, which has collected figures from the four nations, publishes UK totals for confirmed cases and deaths.

The health authorities in those nations also release their own figures, not necessarily covering the same period.

Sky News is using these figures to report on coronavirus in the UK, but we are using data from Johns Hopkins University Center for Systems Science and Engineering to compare countries.

These are just a few of the sources used by politicians and journalists worldwide talk about the number of cases and deaths, but the true totals are a mystery.

Here are a few reasons why.

The “unknown” number of tests

It is difficult to say which country is testing more. Currently, there is no centralised source with this information and many countries do not publish this on a regular basis but aggregate the total up to one particular day.

Testing is important to identify as many people infected as possible and isolate them to contain the spread of the virus.

Experts say that Germany’s testing capacity is one of the main reasons for keeping the number of deaths with COVID-19 much lower than other countries with similar number of infected. But there are several challenges in the testing process.

As COVID-19 is a new disease, there was no existing test and developing an accurate and reliable one takes time.

Early tests took a swab from a patient’s nose and throat and tried to match it to the DNA of the virus, but we cannot be sure of its accuracy.

It is also only able to detect a patient who currently has COVID-19, so it is impossible to know how many people have been infected and recovered. The UK is among the countries hoping to begin anti-body testing which aims to solve this problem.

Another challenge is testing enough people. It is well known there has been a shortage of tests, but different countries also have different approaches and that complicates comparisons.

For example, South Korea and Germany have tested extensively, whereas Japan used a targeted testing criteria.

In the UK the criteria for testing has altered over time and the ratio of positive results to tests has fluctuated accordingly.

Most recently, the UK has mainly carried out tests in hospitals. This means it is impossible to identify what proportion of the general population might have the virus and those who do test positive are often the sickest so most likely to die.

The under-reported cases

Chief Scientific Adviser Sir Patrick Vallance regularly points this out, saying the true ratio of deaths to cases is probably one death for every 1,000 cases so there are now likely tens, if not hundreds of thousands of cases in the UK that haven’t been identified.

As testing is expanded, we can expect to see more positive results, but the proportion might fall as we test more healthy people.

So, whilst identifying cases is crucial to understanding the scale of the pandemic, there is no way of knowing the true number of people infected with COVID-19 – currently or in the past. What we do know is how many people have received a positive result from a laboratory test.

This is what organisations like the World Health Organization, Johns Hopkins University or the European Centre for Disease Prevention and Control are recording as cases. And this is what we call “confirmed cases” and the number is likely to be a small proportion of the true total.

Most asymptomatic people and those with mild symptoms are unlikely to have been tested and will be excluded from the count.

Comparing the number of confirmed cases in different countries is also problematic because many use different criteria to define a “confirmed case” and have different testing regimes. So, comparing the number of cases in Italy and the UK, for example, is not necessarily comparing like with like.

Deaths “with” or “from”

Deaths is a more reliable measure, but it is not free of caveats.

Every day we are told the number of people who tested positive for COVID-19 and died but we do not know if this was also the cause of their death. That is why it is important the distinction whether someone died “with” COVID-19 or “from” it.

Around three quarters of people to die “with” the virus in the UK are aged over 70 and more than 95% of them have underlying health problems. It is reasonable to assume that many of them might have died from other conditions, despite being diagnosed with COVID-19.

Professor Neil Ferguson who leads the Imperial College London team has suggested this could inflate the number of total deaths three-fold.

Official figures from most of the countries normally include only people who die in hospital. This means those who die at home, or in care homes, are missing.

There is no way of knowing how many people this affects, not least because many of them might not have been tested for the virus, but in the UK the Office for National Statistics (ONS) has now released some data that gives us a glimpse of the disparity.

This inability to know the true number of deaths “from” COVID-19 plus the fact that the number of cases is underestimated makes it difficult to estimate the so-called “death rates” or case fatality ratios, that is the number of deaths divided by the number of cases.

The World Health Organization and scientists say they expect the death rate at the end of the pandemic to be around 1%. Until its over, there is really no way of knowing.

The illusion of time

Daily figures give us just a snapshot of the current situation. But it is important to remember there is a crucial delay between infection and outcome: recovery or death.

So, people who are dying with COVID-19 today, most likely contracted it around three weeks ago. The temptation is to look at record daily increases in deaths and conclude measures to contain the virus aren’t working. However, patience is required as the true impact of lockdown won’t be clear for at least three weeks from implementation.

In the UK, deaths announced on a particular day do not necessarily relate to people who have died in the previous 24-hours. Often there is a delay created partly by the need to contact relatives. When that data is published it can include deaths spanning a ten-day period. This makes it incredibly difficult to chart progress made in slowing the spread of the virus.

So, if the data aren’t perfect, then why use them?

The simple answer is: it is all we have and despite it being incomplete, and at times misleading, it still provides a crucial part of the picture.

However, it is the responsibility of those that do use it – scientists, politicians and journalists – to be clear about its shortcomings. That is why we have written this article and we hope it helps you make sense of the COVID-19 pandemic and its progress.

(c) Sky News 2020: Coronavirus: Can we trust the data?