Tuesday, July 20, 2021

When No Number Is Right (COVID Edition)


Credit CDC -  Limits of Infectious Disease Surveillance & Reporting


Every day we are inundated by numbers which purport to tell us how many people are currently infected with COVID, how many have died, the effectiveness of vaccines against COVID infection (or COVID illness, hospitalization, or death), and forward-looking estimates of where these numbers will be a week, or a month, or a year from now. 

And while most (but certainly not all) of these numbers are the product of honest and ethical endeavors, they are invariably all going to be wrong.

During the 2009 H1N1 pandemic, I wrote repeatedly on our inability to accurate count – or even estimate – the number of H1N1 flu deaths in the United States and around the globe. Just a few of those posts include:

Dead Reckoning

The Tip Of The Iceberg

When No Number Is Right

Apples, Oranges, And Influenza Death Tolls

The CFR, or case fatality ratio is generally seen as the most important statistic in any pandemic, and yet it is often the hardest to quantify. This number is the percentage of people who, once infected, die (either directly or indirectly) as a result of that infection.

The CDC stopped counting H1N1 influenza deaths in the summer of 2009, realizing that their official tally was more misleading that helpful. The `official’ death toll, as reported by the World Health Organization was roughly 18,000 deaths globally, but the WHO offered this disclaimer: 

The reported number of fatal cases is an under representation of the actual numbers as many deaths are never tested or recognized as influenza related. – World Health Organization.

Still, the media, desperate for something tangible to report, latched on to this 18,000 deaths number and many reported this as `fact' (or without disclaimers), for months, giving the impression that the 2009 pandemic was overblown. 

In 2012, two years after the pandemic ended, The Lancet published a study that attempted to estimate the number of global H1N1 deaths during the first year of the pandemic, and it comes up with a number than runs between 15 and 30 times higher than reported to the WHO.

Estimated global mortality associated with the first 12 months of 2009 pandemic influenza A H1N1 virus circulation: a modelling study

Dr Fatimah S Dawood MD A Danielle Iuliano PhDCarrie Reed DSc Martin I Meltzer PhD David K Shay MDPo-Yung Cheng PhDDon Bandaranayake MBBS Robert F Breiman MD W Abdullah Brooks MD Philippe Buchy MD Daniel R Feikin MDKaren B Fowler DrPH Aubree Gordon PhD Nguyen Tran Hien MD Peter Horby MBBSlQ Sue Huang PhD Mark A Katz MD Anand Krishnan MBBS Renu Lal PhD Joel M Montgomery PhD Kåre Mølbak MDoRichard Pebody MBBSAnne M Presanis PhD Hugo Razuri MD Anneke Steens MSc Yeny O Tinoco DVM Jacco Wallinga PhD Hongjie Yu MDrSirenda Vong MD Joseph Bresee MD , Dr Marc-Alain Widdowson VetMB


We estimate that globally there were 201 200 respiratory deaths (range 105 700—395 600) with an additional 83 300 cardiovascular deaths (46 000—179 900) associated with 2009 pandemic influenza A H1N1. 80% of the respiratory and cardiovascular deaths were in people younger than 65 years and 59% occurred in southeast Asia and Africa.

While probably closer to reality, the true death toll from the H1N1 pandemic remains unknown. As are the death tolls from the 1968 H3N2 pandemic (est. 1 million), the 1957 H2N2 pandemic (Est. 1 - 4 million), and 1918 (est. 20 - 100 million).  

Over the past decade we've looked at dozens of studies on the reported incidence (and/or  number of deaths from) avian flu (H5N1, H7N9, H5N6, etc.), Swine Flu, MERS-CoV, Malaria, Dengue, Ebola, and many other infectious diseases - and they all pretty much tell the same story. 

Cases, and deaths, are always undercounted or underreported. Often by a factor of 10 or more.  

While we hear reassuringly precise numbers of how many people have been infected, or have died, from the MERS Coronavirus from the WHO (As of May 31st, 2021 a total of 2574 laboratory-confirmed cases , including 886 associated deaths.) we've seen studies suggesting that these are huge undercounts. 

The same is true for novel flu, whether it be China's H7N9 (see Lancet: Clinical Severity Of Human H7N9 Infection) or a limited outbreak of a swine variant flu in the United States (see CID Journal: Estimates Of Human Infection From H3N2v (Jul 2011-Apr 2012- which estimated that during a time when only 13 cases were reported by the CDC - that the actual number of infections was likely 200 times (or more) higher.
Results. We estimate that the median multiplier for children was 200 (90% range, 115–369) and for adults was 255 (90% range, 152–479) and that 2055 (90% range, 1187–3800) illnesses from H3N2v virus infections may have occurred from August 2011 to April 2012, suggesting that the new virus was more widespread than previously thought.

Of course, you can't really take any of these revised estimates to the bank either.  But you can assume, with some degree of confidence, that whatever numbers are quoted, they are probably undercounts. 

While many of these undercounts are due to the limits of surveillance and reporting, some statistics are undoubtedly highly censored by governments, many of which see no advantage in disclosing higher numbers.  

All of which brings us to a new report, published today by The Center For Global Development, which attempts to quantify the number of COVID deaths in India up to the end of June, 2021.  

While the official government death toll is roughly 400,000, we've seen numerous anecdotal reports - including stories of crematoriums working 24/7 - which suggests this is a gross underestimate. 

Although all of their methods have shortcomings, the three estimates that the authors of the following study come up with are at least 8 to 10 times greater than the `official' number. 

Three New Estimates of India’s All-Cause Excess Mortality during the COVID-19 Pandemic
Abhishek Anand , Justin Sandefur and Arvind Subramanian
JULY 20, 2021

India lacks an authoritative estimate of the death toll from the COVID-19 pandemic. We report excess mortality estimates from three different data sources from the pandemic’s start through June 2021. First, extrapolation of state-level civil registration from seven states suggests 3.4 million excess deaths.

Second, applying international estimates of age-specific infection fatality rates (IFR) to Indian seroprevalence data implies a higher toll of around 4 million. Third, our analysis of the Consumer Pyramid Household Survey, a longitudinal panel of over 800,000 individuals across all states, yields an estimate of 4.9 million excess deaths. 

Each of these estimates has shortcomings and they also diverge in the pattern of deaths between the two waves of the pandemic. Estimating COVID-deaths with statistical confidence may prove elusive.

But all estimates suggest that the death toll from the pandemic is likely to be an order of magnitude greater than the official count of 400,000; they also suggest that the first wave was more lethal than is believed. Understanding and engaging with the data-based estimates is necessary because in this horrific tragedy the counting—and the attendant accountability—will count for now but also the future.

While it is safe to say we'll never know the true number of COVID cases, or deaths, it is important that we get a better handle on this pandemic's impact.  Too many people used the low numbers from the last pandemic to justify doing little or nothing to prepare for this one. 

And that is an error in judgement we can't afford to make again.