# 6739
An excellent piece by Kelly Crowe of CBC News over the weekend questions the credibility of flu mortality numbers and reminds us, once again, that in public health easy answers are often the hardest to deliver.
First a link to the article, which I encourage you to read in its entirety.
Flu deaths reality check
Credibility of flu models disputed
By Kelly Crowe, CBC News
Posted: Nov 25, 2012 5:14 PM ET
Do thousands of Canadians really die every year from the flu? The flu folks keep saying so. I've already heard it repeated several times this year and flu season has just started. This is what the Public Health Agency of Canada said in a recent press release: "Every year, between 2,000 and 8,000 Canadians die of the flu and its complications."
Long time visitors to this blog will recall that we’ve trod this perilous path before - for both seasonal and pandemic flu - along with a variety of other diseases.
Quite understandably, the public and the media expect public health officials to have some kind of handle on the number of deaths caused by infectious diseases in our society.
Particularly with something as ubiquitous as flu.
But the truth is, no one really knows.
After more than a decade of promoting the `flu kills roughly 36,000 Americans each year’ meme, the CDC revised (and hopefully improved) their estimates in 2010 ( see MMWR: Estimates Of Yearly Seasonal Influenza Deaths)
For deaths with underlying pneumonia and influenza causes (the most narrow definition of flu-related fatalities used) the models estimated a yearly average of 6,309 (range: 961 in 1986--87 to 14,715 in 2003--04) influenza-associated deaths.
Using a broader criteria (underlying respiratory and circulatory causes including pneumonia and influenza causes) the models estimated an annual average of 23,607 (range: 3,349 in 1986--87 to 48,614 in 2003--04) influenza-associated deaths.
Despite the 12-fold difference in deaths between the 1986-87 and 2003-04 seasons, the operative word here remains `estimated’.
- Estimates are extrapolated based on a surveillance subset of the country, not the whole nation
- There are often co-circulating viruses that may influence overall mortality.
- Surveillance, testing, and reporting may change over time
- Different mathematical models can produce differing results
- There are varying opinions as to what constitutes an influenza-related fatality.
When combined with the inevitable variations in the severity of influenza seasons (H3 years are usually more severe than H1 years), this makes it impossible to derive a single number that `works’.
In an attempt not to compound a felony, I try to leave it as influenza `kills thousands each year’ or `is the cause of substantial mortality’ in this blog. Sometimes I’ll use the range (3,000-48,000) offered by the CDC, but most of the time I don’t.
The same holds true for just about any illness or disease you’d care to mention.
Ask the CDC how many people contracted West Nile Fever this summer, and they will tell you that (as of Nov. 20th) they had recorded 5,207 cases of West Nile virus disease in people, including 234 deaths, but that the real number may be 50 times higher.
Severe (neuroinvasive) cases are pretty easy to spot, but they estimate only 1%-3% of mild cases of West Nile Fever are diagnosed and reported.
If we do the math, and assume the 2654 non-neuroinvasive cases officially reported constitute between 1% and 3% of the total number of actual cases we get a range of between 250,000 and 85,000 infections.
The chart below illustrates the problem nicely.
Relying only on lab confirmed fatalities isn’t much of a solution, either. The `official’ death toll for the 2009 pandemic - as reported by the World Health Organization - was roughly 18,000 deaths globally.
The WHO offered this disclaimer:
The reported number of fatal cases is an under representation of the actual numbers as many deaths are never tested or recognized as influenza related. – World Health Organization.
Unfortunately, the mainstream media often reported the low official number of deaths without adequately explaining the acknowledged gaps in the data, leading many to believe that the 2009 pandemic was a damp squib.
In contrast, earlier this year, in Lancet: Estimating Global 2009 Pandemic Mortality, we saw a study who’s estimate found:
We estimate that globally there were 201 200 respiratory deaths (range 105 700—395 600) with an additional 83 300 cardiovascular deaths (46 000—179 900) associated with 2009 pandemic influenza A H1N1. 80% of the respiratory and cardiovascular deaths were in people younger than 65 years and 59% occurred in southeast Asia and Africa.
With no way to accurately count cases, analysts are reduced to creating mathematical models, fueled by both hard data and assumptions, in order to extrapolate the impact of diseases on the population.
The old adage (well, not that old, as it is attributed to George E. P. Box, Professor Emeritus of Statistics at the University of Wisconsin) is that:
“All models are wrong, but some models are useful.”
To that I would add, that while useful, mathematical models can be extraordinarily difficult to defend.
It requires agreement over assumptions. And among academics, that’s never easy to reach.
Going hand-in-hand with the estimates of flu deaths each year has been the purported effectiveness of the seasonal flu shot.
Up until about a year ago the CDC’s mantra has been for healthy adults under the age of 65, in years when the vaccine is a good match to circulating strains, effectiveness ranges from 70%-90%.
Despite these important qualifiers, the message often ended up being shortened in the media to the flu vaccine being `up to 90% effective’.
A little more than a year ago the CDC updated their FAQ on Flu Vaccine effectiveness, and as part of a much longer detailed posting, lowered their estimate of the inactivated flu shot’s effectiveness to read:
. . . recent RCTs of inactivated influenza vaccine among adults under 65 years of age have estimated 50-70% vaccine efficacy during seasons in which the vaccines' influenza A components were well matched to circulating influenza A viruses.
A number that pretty much matched CIDRAP’s finding (see A Comprehensive Flu Vaccine Effectiveness Meta-Analysis) which would be released a couple of weeks later. That analysis showed the trivalent inactivated vaccine (TIV) had a combined efficacy of 59% among healthy adults (aged 18–65 years).
So what are we left with?
Well, every time we get into statistics (admittedly not my strong suit) I’m reminded of the story of the statistician who drowned trying to ford a river that was, on average, only 3 feet deep.
Still I think we can safely draw a few conclusions.
Influenza-like-Illnesses (ILIs) obviously contribute to a good deal of morbidity and mortality each year.
In addition to influenza, these illnesses can be caused by the metapneumovirus, parainfluenzavirus, respiratory syncytial virus (RSV), adenoviruses, or any of the myriad Rhinoviruses (Common cold). Among others.
The percentage of these illnesses that are actually due to the influenza virus varies considerably from year to year, and so only a portion of these deaths are actually `vaccine preventable’.
Today’s influenza vaccine, whose effectiveness is described as just `moderate’ by CIDRAP’s recent 160-page Comprehensive Influenza Vaccine Initiative (CCIVI) report, undoubtedly saves lives and reduces hospitalization, but is not the panacea that many would hope for.
Hence the call for better vaccines.
Despite their limitations, I still get a flu vaccine each year, as I believe partial protection beats no protection any day of the week.
As far as the estimate of deaths from influenza are concerned, I doubt this debate can really be solved to everyone’s satisfaction. There is no single, `good’ answer when the parameters change as often as they do with influenza.
We live in a world driven by easily adopted memes, 10 second sound bytes, and 140 character tweets.
As a result, officials are often tempted to provide us with simplified, easy to digest, answers. While brevity may have many advantages, scientific precision is rarely one of them.
Of course, if someone comes up with a better way to measure the number of deaths from influenza each year, I’ll feature it in this blog. Until that happens, I’ll simply leave it as the cause of `substantial mortality’.
