Deaths, deceptive data and the detail of COVID reporting

If, fate forbid, you died from COVID today, how long would it take for your statistic to register on the government’s ‘COVID-19 reporting’ page?

Probably longer than you think.

But the lack of clarity around this has tripped up Australian media – including this publication – science communications and health experts in recent days, and potentially more than that among the general public in the last 2 months.

Several publishers, including Cosmos yesterday, noted the federal government’s public-facing digest of COVID-19 data had reported weeks with zero COVID-related deaths, on a rolling 7-day average.

Good news for Australia, right?

As it turns out, it wasn’t quite the case, as was picked up by infectious disease modeller James Wood in comments he provided to media outlets through the Australian Science Media Centre on Thursday.

“While this is encouraging data, it’s almost certainly not correct that there were zero COVID deaths in that week,” Wood said.

“Instead, our near real-time reporting system for reporting of deaths has mostly wound down and we now need to rely on the slower ABS reporting which tends to occur at about a 2-month delay.”

The delay, is, however, poorly reflected on the government’s website.

Following a scheduled update today, it has been revealed that COVID-associated deaths from New South Wales are effectively not included in the federal graph. And those zero COVID weeks actually had several deaths recorded.

Context is king

Karen Cutter is chair of the Actuaries Institute’s mortality working group. Actuaries – for those unfamiliar with the profession – are effectively risk analysts who work across a range of fields providing deep statistical analysis to support risk management in business, government and not-for-profit sectors.

That committee has closely followed Australia’s experience of the pandemic in data, regularly issuing updates on the nation’s excess mortality.

But retrieving good data has been hard – not just for actuaries but across the scientific community – since state and federal governments began winding back their reporting requirements. Daily reporting has become weekly, fortnightly, monthly and now even every 2 months.

Cutter explains the issue with what appeared, at one point, to be a positive piece of news.

“[The graphs] are not inaccurate, but you need a good understanding of what the information is to be able to interpret what is shown,” she tells Cosmos.

“For the graph of deaths, it shows the number of deaths against the date that the death occurred.

“For the most recent month or so, the number of deaths will be understated because it takes time between when a death happens, and when a death is reported.”

If you departed this mortal coil due to COVID earlier this week, it’s highly unlikely your death would be reflected in the scheduled data update that was published today.

“It is almost impossible for a death to happen, for it to be registered and/or reported to the state-based health authority, and then reported through to the Federal authority, all within the space of a week,” Cutter says.

“Every time there is an update of the graph on the federal database, you will see that deaths for the most recent week are zero. There is simply not enough time for deaths to be reported through to the federal level.”

And she’s right: ‘No deaths’ again in the fresh update as at April 10 (but published online on April 12).

The quality of data is also in question. And it’s not sat easy with actuaries like Cutter for some time.

“The number of deaths reported had been looking inconsistent with the other information provided on hospitalisation levels since early January,” Cutter says. “They looked too low.”

She’s noted that the health department’s reporting of COVID deaths has been lower than similar data published by the Australian Bureau of Statistics.

“It turns out that New South Wales have not been reporting the majority of their COVID deaths since around December sometime,” she says.

That caveat is today reflected – for the first time – on the federal health website as a footnote beneath the COVID-19 associated deaths data:

 

“As there are delays between when a death happens and when it is reported, this means that the graph does not include most NSW deaths from about the middle of November. This is why it looks like deaths peaked in November before reducing. However the majority of that reduction, particularly for the months of December and January, is because the NSW data is just not included in that graph,” Cutter says.

Data, context and clarity

Is the data fit for public consumption?

“I think there are improvements that could be made,” Cutter says.

Cutter describes the data, as presented on the government’s public-facing website, as difficult to understand.

But how many people would realise this when paying a visit to a webpage that had been a valuable source of information relating to the state of the pandemic for the last 4 years? Until today, few would have known that COVID-associated deaths in the nation’s most populated state were barely included.

On a previous version of the government website, provisional data was (somewhat) clearly indicated with a dashed line. But no more.

“It seems to have disappeared,” Cutter says.

“There are better ways that the information could be presented so that it was more easily understood and the public had the information they needed,” Cutter says.

With a patchwork reporting system from the states and territories feeding into the federal government, that may be difficult.

Still, people in health-related fields who rely on transparent data, let alone the curious public, could probably do with more transparency on these published statistics. Simple graphs may be easy to understand, but the omission of vital context (like the absence of an entire state’s worth of data) should be stated.

Professor Adrian Esterman, chair of biostatistics at the University of South Australia, has been a reliable source of public comment on patterns related to emerging variants, infection numbers and other pandemic data since the first outbreak.

He would like to see a shift in the way governments communicate COVID data to the public.

“The thing is that it’s a dog’s breakfast,” Esterman tells Cosmos.

“So you’ve got the situation where, for example, they’ve got hospitalisation rates not split by state. If you want case numbers, they send you to a totally different website, which is difficult to use. And even the data they do publish like hospitalization data, death data, is out of date.”

Esterman poses the question: “How useful is it?”

Given what has emerged in today’s update, it’s a legitimate question.

For his part, Esterman is hopeful that Australia’s first Centre for Disease Control – a CDC – currently in development by the federal government, will help mop up the data mess that applies to a disease still having a disproportionate effect on the population, compared to other diseases.

“What they’re trying to do is they’re trying to make the point that COVID-19 is now no different to any other notifiable disease,” Esterman says.

“It is different. The main reason it’s different is because we have so many cases – there’s no other notifiable disease which has a constant 1000s of cases a week.

“We should be tracking it and we are not, and we’re not reporting it.

“I would absolutely like the CDC to focus on infectious diseases, at least in the first year.”

Cosmos approach the Department of Health for comment but received no response by deadline.