Coronavirus: How did the UK’s COVID-19 testing data become such a mess?

How many people have been tested in the UK for COVID-19? It’s a pretty simple question, which most countries are capable of answering.

And it’s a pretty important one, since until you know how many people have been tested it’s very difficult to get a real early handle on the spread of the disease.

Will there be a second spike? Is it safe to loosen the restrictions on our lives? Are quarantines justified or not? If not, how do we ensure those coming to the country do not bring the disease with them?

Testing – doing it well, accounting for it well and using the results sensibly – is clearly a crucial part of the solution.

Yet here in Britain we don’t know how many people have been tested for the disease.

Indeed, while the government was up until recently producing numbers on people tested, it recently admitted those figures are suspect and will have to be revised.

This is only one of the issues the Statistics Authority has with the government, as covered in a highly critical letter from its head, Sir David Norgrove, to Health Secretary Matt Hancock.

One of Sir David’s other criticisms is one you’ll probably have picked up on in the past weeks: that the government is obsessed with getting the number of tests up as high as possible, but is lumping all sorts of very different kinds of things in under the headline of “tests”.

His other is that the data is enormously complicated and the government makes very little effort to try to explain it.

I can testify to this myself, having spent a good day going through a recently published spreadsheet provided by the Department for Health and Social Care (DHSC) of its testing programme.

Some bits are relatively straightforward: there is what the government calls “pillar 1” tests.

These are the main “gold standard” clinical tests carried out on those in hospitals and health workers. And the number of these tests being carried out is much the same as it was in early May – albeit up from the early stages of the crisis – roughly 33,000 on average a day.

Then there are the next pillars: pillar 2 which is other clinical tests, mostly on those with symptoms.

These include the drive-in tests that are becoming increasingly commonplace around the country. These tests – at least the in-person tests – have hovered around the 30,000 mark in recent weeks. I say in person because actually a lot of the pillar 2 tests are test kits which have been sent out to households.

These account for the biggest chunk of testing – an average of 47,000 a day in recent weeks.

There are two other pillars: pillar 3 is the serology antibody tests that are now starting.

These are an important test of immunity, though it’s worth emphasising they are rather different from the pillar 1 and 2 tests designed to show whether someone has the disease at that given moment rather than having had it in the past. The distinction is very important when it comes to the epidemiology here.

And finally there is pillar 4, which is tests carried out by the Office for National Statistics and others to try to get a sense of the prevalence of the disease.

Again, these are a category difference from the rest, since they are not really a clinical test but more of an academic exercise – important though such things are.

Oh and just to complicate things even further there is a fifth pillar – a government initiative to get mass testing done bigger and better in the future – though as far as anyone can tell this isn’t a solid pillar but a kind of aspirational pillar. Not the kind of pillar you could lean on or build anything on, anyway.

As you can see, among this maze of pillars there are some solid reliable kinds of tests and some less reliable tests and some tests that might not have happened at all. It’s a bit of a mess.

While it’s clear the government’s capacity to test has improved enormously from the early stages of the pandemic (and presumably that’s why it’s always going on about the big headline numbers), the public understanding of what those tests are telling us has, if anything, taken a step backwards.

The government has a decent handle on the results from the pillar 1 tests; they even have comprehensive daily numbers about how many people have been tested, which is an important distinction.

Since the tests are not 100% accurate, often people will need to be tested more than once, therefore the number of people tested is well below the number of tests.

For instance, while there have been 33,000 tests a day on average in the past fortnight, the average number of people tested under pillar 1 is just under 21,000.

But here’s where things get murky. We know anecdotally that there have been some problems with the pillar 2 tests.

In particular, while the government counted tests mailed out to people in their grand total, many of these tests were never carried out or completed, some aren’t entirely reliable and some people may have been tested under a variety of the schemes.

The upshot is that the bald number of tests performed (or in this case performed/sent) doesn’t tell you much of anything at all. So back to the question at the beginning: how many people have been tested?

While we know the answer for that gold standard pillar 1 category, we don’t know for the other constellation of tests.

A few weeks ago we thought we did: the government was producing numbers each day for how many people had been tested. Yet a couple of weeks ago it abruptly stopped publishing those numbers and withdrew any relevant data from its website.

It has since admitted that there was some double counting going on in a “small percentage of cases”.

Now, mistakes happen, but a glance back at the data we were getting at the time provides some evidence of really dodgy data keeping.

Consider the five-day period from 16 April 16 to 21 April. During that period there were, according to the government’s data, 21,730 tests carried out under pillars 2, 3 and 4. Yet according to the data the department was producing at the time, the number of people tested under those pillars (eg subtracting those tested under pillar 1) was 22,231.

Hmm. So the number of people being tested is more than the number of actual tests carried out? Odd and odder.

In the DHSC’s defence it has recognised the problems here and is trying to improve the data.

However the sheer messiness of the numbers is desperately depressing, and it is no wonder the Statistics Authority has sent as sternly-worded a letter as I can remember to a minister.

:: Listen to the Daily podcast on Apple Podcasts, Google Podcasts, Spotify, Spreaker

These data matter. It is no coincidence that many of the countries which have done very well in tracking and tracing and combating the disease are also the most open with their data.

In Hong Kong you can look through a spreadsheet at every case in the city, anonymised details of where people caught the disease, the seriousness of their symptoms and what happened to them next.

In the UK we have nothing of the sort – at least not publicly available.

In the grand scheme of things, this is not the most important aspect of Britain’s strategy with the virus. But knowing the scale of infection entails understanding the data. Understanding the data entails trusting the data. And right now Britain’s data on testing cannot be trusted.

Tonight and tomorrow night at 8pm, Dermot Murnaghan will be hosting After The Pandemic: Our New World – a special live programme about what our world will be like once the pandemic is over.

Thursday’s programme will include an exclusive interview with Prince Charles.

(c) Sky News 2020: Coronavirus: How did the UK’s COVID-19 testing data become such a mess?