With dozens of council areas now in some form of lockdown, we all want to know how well the authorities are managing local outbreaks – and which parts of the country might be next to face restrictions.

Just yesterday, the Labour leader Keir Starmer cited figures in the Commons that appeared to show infection rates had actually got worse since tighter rules were introduced in three Lancashire towns.

And the party’s own analysis finds the same pattern in 19 out of 20 areas of England that have been locked down for over two months. Today’s Daily Mirror says this demonstrates that local lockdowns “aren’t working”.

Meanwhile, a report in the Sunday Times last weekend went so far as to suggest that ministers were sparing some constituencies from stronger measures simply because they were held by Conservative MPs. The evidence behind it, provided by a local health official in Blackburn, was that places with a similar rate of confirmed cases were being treated differently.

But the metric that both sets of claims rely on – official Covid cases per 100,000 population – doesn’t tell us as much as we might think.

Here’s why.

Testing, testing

The government “targets” testing at different parts of the country when they become areas of concern. And the more tests you conduct, the more cases you’re likely to find. So it’s hardly surprising that locked-down areas are seeing their official tallies increase over time.

Indeed, when it announced new measures for Oldham, Pendle and Blackburn in August, the Department of Health pointed out that “the sharp rise in cases [in those areas] is in part due to a major increase in testing led by local councils”.

Public Health England (PHE) issues a similar caveat on its figures in its weekly Covid surveillance reports: “Case detections are limited by testing capacity”.

Any alternatives?

The limits of these figures don’t mean that the actual number of infections isn’t also rising. It’s just that the official caseloads are unhelpfully confounded by the uneven spread of tests around the country.

So is there a way to find out if infections have gone up for real?

PHE suggests “positivity rates provide a better indication of change in activity in some areas”. The Department of Health also alluded to this measure as revealing the true extent of the change back in August.

The “positivity rate” simply means the proportion of people tested who were told they had the virus (rather than the proportion of people living in the area).

The figure indicates whether your official case numbers are catching pretty much everyone in the community who has Covid (in which case, the positivity rate would be low), or whether your official stats are just the tip of the iceberg (which would show up as a high positivity rate).

We can get some insight from individual parts of the country. For example, Manchester City Council says 22.9 per cent of tests conducted there resulted in a positive diagnosis in the week ending 3 October – up from 14.2 per cent in the previous seven days.

And areas on the government’s “Contain Framework” watchlist also have their positivity rates reported in PHE’s weekly roundup.

But PHE has confirmed to FactCheck that it does not publish positivity rates broken down by every local authority. So we can’t make a full national comparison.

Professor Sheila Bird, who was formerly Programme Leader at the University of Cambridge’s MRC Biostatistics Unit, agreed that confirmed cases per 100,000 aren’t especially helpful when it comes to local comparisons.

She told FactCheck that we need to know several more pieces of information: the proportion of the population that are tested in Pillar 1 (hospital settings) and Pillar 2 (publicly available community tests), the positivity rates for each pillar, and the actual number of positive tests per 100,000 population.

And she says, when it comes to those Pillar 2 tests, we want even more detail about whether people getting tested had symptoms, whether they were in quarantine and whether they’d already been contacted by Test and Trace.

Why do some areas go into lockdown?

There was speculation at the weekend that some places were spared tougher restrictions because they were represented by Conservative MPs. Last month, local health officials complained that the government had changed the rules for some parts of the country without explaining why.

Of course, we can’t say for certain what’s going on behind closed doors.

But the Department of Health did shed some light on the factors that they use when deciding which areas go into lockdown.

They told FactCheck that, among other things, they consider “the number of patients in hospital with coronavirus, seroprevalence, modelling data, ONS and React surveillance data, positivity rates and NHS indicators such as GP attendance, calls to the NHS 111 service and hospital admissions.”

(“Seroprevalence” is when the government analyses blood samples from donors to calculate how many people might have had coronavirus without any symptoms. “React surveillance data” comes from a major study by Imperial).

But there’s a catch: as we understand it, the government only publishes one of these datasets (calls to NHS 111) broken down by local authority. Hospital admissions stats are available by NHS Trust, but the boundaries of these don’t match up with councils.

So while it might be true that the government uses a range of measures to decide which areas go into lockdown, there is still a lack of transparency about what the figures actually show at a local level.

A spokesperson for the Department of Health told FactCheck: “The incidence rate is only one of a set of considerations regarding when it is appropriate to impose and release restrictions, and decisions are made in close consultation with local leaders and public health experts, informed by the latest evidence from the JBC and NHS Test and Trace, PHE and the Chief Medical Officer for England.

“While we recognise how much of an imposition these measures are, they are based on the latest scientific evidence in order to suppress the virus, and protect us all while doing everything possible to support the economy”

FactCheck verdict

It’s tempting to use the number of recorded cases per 100,000 as a way to work out if local restrictions in England are working, and even to decide whether places have been unfairly locked down to begin with.

But this metric doesn’t actually tell us very much because it’s heavily affected by the amount of testing available in a particular place. It’s possible that the apparent rise in infections in locked-down areas is a result of the authorities detecting more cases.

There might be a better way to work out whether outbreaks are really on the rise – the proportion of people tested who get a positive result. But there’s a catch: the government doesn’t publish this data for every local authority in England.

There’s been some anger from local health officials and MPs about how and why lockdown decisions in England are taken. The Department of Health has shown FactCheck eight measures that form part of its calculation – but frustratingly, it seems only one of these is publicly available broken down by local authorities. So it’s very hard to make a proper comparison.

In short: we shouldn’t use official cases per 100,000 to make sweeping statements about local lockdowns, but there remains a lack of transparency around exactly how decisions are taken.