UPDATED (see bottom)
# 3971
It sounds like it ought to be a simple request.
How many people died last week (or in the last month, or YTD) from the pandemic H1N1 virus in the United States?
But it is anything but simple to try to come up with a number. And the CDC, to whom most people look to for this sort of information, is obviously struggling to find some way to represent incomplete and inaccurate numbers responsibly.
Which is why the CDC stopped `counting’ H1N1 deaths months ago.
Instead, they’ve made an attempt to report P&I related hospitalizations and deaths from specific localities around the nation.
Some people have chosen to regard those as a kind of official H1N1 death toll, but as far as I know, it has never been represented as such by the CDC.
Yesterday, the CDC changed the way they report on P&I (pneumonia & Influenza) deaths on their pandemic surveillance pages. Something that has raised eyebrows, and some concerns, around Flublogia.
But before we look at the new representation, let’s look at the old one. On August 31st, they `reset the count’ to zero, and for the next two months reported two numbers.
Laboratory confirmed H1N1 deaths, and P&I Syndrome deaths.
P&I Syndrome hospitalizations and deaths can be based on syndromic, admission or discharge data, or a combination of data elements that could include laboratory-confirmed and influenza-like illness hospitalizations.
This is the chart from 2 weeks ago, where total of more than 2400 were listed as having died in the previous 6 weeks from P&I Syndrome.
In yesterday’s US update, we got the following chart showing 672 laboratory confirmed influenza deaths since August 31st, but the P&I Syndrome hospitalizations and deaths were missing.
This change in reporting was made without fanfare or explanation (see UPDATE at bottom), leaving some to wonder just what is going on? We’ve gone from well over 2,000 deaths on this page last week to under 700 this week.
Confusing matters further, CDC Director Frieden stated last week that there have been `more than 1,000 deaths’ from the H1N1 virus in the US.
This is no doubt true, but it is kind of like saying that Bill Gates has more than 1 million dollars. Technically true, but not very revealing.
So what is going on here?
Since I’m not privy to what goes on behind closed doors at the CDC, I can only speculate. But I think they’ve got themselves a first class dilemma, and they aren’t sure how to best handle it.
Consider: What do you do when you know all of your numbers are wrong?
The press and the American people expect some number, some quantification of the severity of this pandemic. And yet, the surveillance and reporting systems in this country (or any country) are inadequate to provide an accurate case or fatality count.
What do you do?
It’s the same with every disease, but we don’t worry about it as much when we don’t call it a pandemic.
Whether it’s Lyme Disease, West Nile Virus, Salmonella poisoning, Heart Attacks, or influenza . . . the CDC only has estimates of the number of cases, and deaths.
Even when it is a reportable disease – like Lyme – they understand that not every case gets diagnosed, and not everyone diagnosed gets reported.
Those cases that end up reported to the CDC usually represent only the tip of the iceberg – or in this case, a pyramid – of the actual number of cases.
When someone dies, there isn’t some sign on their chest that says “I died of an influenza-related complication”.
There isn’t even a good definition of what an influenza-related complication is.
If I get the flu and get dehydrated and weak, and then a few days from now get up out of my sick bed - and while walking to my mailbox - keel over with a heart attack . . . did the flu contribute to my death?
Maybe.
How do we count that? Does the flu body count go up a full point? Or should 55 years of fast food restaurants get half a point, and the flu the other half? What about a family history of coronary artery disease?
1/3rd . . .1/3rd . . . 1/3rd?
What if a week after I got the `flu’, instead of a massive coronary, I developed a bacterial pneumonia and ended dying after a few days in the hospital?
After a week, any influenza tests would come back negative.
How do you count that? Would I have caught pneumonia without having had the flu first? Probably not.
But . . . did I really have the flu?
It could have been some other respiratory virus. Perhaps metapneumovirus, parainfluenzavirus, respiratory syncytial virus (RSV), one of the myriad Rhinoviruses (Common cold), or adenovirus.
Even during the height of flu season, at least half of all ILI’s (influenza-like-illnesses) are probably caused by something other than influenza.
This chart (see ILI’s Aren’t Always The Flu) from early October show more than 70% of samples tested by the CDC came back negative for influenza.
There is no requirement to report adult influenza and pneumonia deaths in the United States, only pediatric deaths. And even all of those aren’t reported.
Mortality is monitored using the 122 MRS system, which tracks P&I related deaths in 122 cities around the nation . .. but that covers only about 25% to 30% of the nation.
And it most certainly doesn’t catch all P&I related deaths in those localities. It is a benchmark, something we can compare one year to the next, but not an accurate measurement of cases.
Here’s how the CDC describes it (highlights & reparagraphing mine):
Exact numbers of how many people died from flu this season cannot be determined.
Flu-associated deaths (which have laboratory confirmed influenza), are only a nationally notifiable condition among children; however not all pediatric influenza deaths may be detected and reported and there is no requirement to report adult deaths from influenza.
In addition, many people who die from flu complications are not tested, or they seek medical care later in their illness when flu can no longer be detected from respiratory samples.
However, CDC tracks pneumonia and influenza (P&I) deaths through the 122 Cities Mortality Reporting System. This system collects information each week on the total number of death certificates filed in each of the 122 participating cities and the number of death certificates with pneumonia or influenza listed as a cause of death.
The 122 Cities Mortality Reporting system helps gauge the severity of a flu season compared with other years. However, only a proportion of all P&I deaths are influenza-related and, as noted, most flu deaths are not lab confirmed.
Thus, this system does not allow for an estimation of the number of deaths, only the relative severity among different influenza seasons. For the 2007-08 season, the proportion of deaths due to pneumonia and influenza was higher than the previous two years, but was similar to the 2004-05 season.
I’ve only scratched the surface on the complexities involved here.
Every day, about 6,000 people die in the United States. Most never get a post mortem exam, and of those that do, only a very small percentage might be tested for the H1N1 virus.
Essentially, unless someone is under the age of 18 (we scrutinize child deaths pretty closely), or is an adult who dies while in the hospital and under treatment for the H1N1 virus, the odds are pretty good that their death will go uncounted.
That’s simply a limitation of our surveillance and reporting system.
And you will find huge differences in the scope and quality of surveillance across the country. There are many places that still use locally elected coroners – usually the local funeral home director – instead of a medical examiner system.
You’ll also find different communities (or at least their officials) have varying ideas over how this data should be collected and released. Some places seem intent on downplaying deaths and cases, out of what I view as misplaced concern over panic.
Crof over at Crofsblog has been covering some excellent reporting from the University of Southern California's Annenberg School of Communications and Journalism. L.A. County's Swine Flu Victims: Death Certificates Tell the Stories Of A Ravaging Virus shows how cases aren’t always identified or publicly acknowledged.
The 57 deaths attributed to H1N1 in the Los Angeles area are undoubtedly an undercount.
Yes, it is an imperfect system – one saddled by fifty years of underfunding and neglect.
So you can understand the quandary. What is the responsible number for the CDC to report?
Do you lump all P&I hospitalizations and deaths reported, and call that the right number? (Note: The CDC never said those were all H1N1 related)
Of course, those numbers don’t represent the whole country. And maybe half . . . or some weeks maybe even more, may have stemmed from something other than influenza.
Or do we go with the new system, that reports only laboratory confirmed influenza deaths?
It has the advantage of only counting confirmed influenza cases, but we know this is a gross undercounting of actual flu cases.
While there are other metrics they could use, there is really no good solution here. Any number you can come up with is going to be wrong – or at the very least, grossly incomplete.
And so I understand their dilemma.
What is the responsible number to report?
I don’t honestly know. Believe me, I’m frustrated by this, and want a number just as badly as you do.
I want to know just how bad this pandemic really is. I want to know if we are over-reacting, under-reacting, or acting just about right when it comes to the threat.
And I’d like some ammunition to use against the pandemic deniers who are using the CDC’s and WHO’s numbers to `prove’ their contention that this pandemic is a non-event.
A little over a week ago a study was published in the EID Journal (see Adjusting The Numbers) that attempts to estimate the impact of the H1N1 virus over the first 3 months. This was still at a time when the CDC was attempting to count cases.
Estimates of the Prevalence of Pandemic (H1N1) 2009, United States, April–July 2009
C. Reed et al. (364 KB, 7 pages)
Over a time period (April-July 23rd) when the official numbers were:
43,677 confirmed cases
5009 hospitalizations
302 deaths
The newly estimated numbers are:
1.8 – 5.7 Million cases
9,000-21,000 Hospitalizations
800 (range 550-1300) deaths
In other words, the best guess was that even at a time when they were trying to count all deaths, the most likely number was 250% higher, and might have been as much as 500% higher.
For months the CDC has been intentionally vague when asked about the number of deaths due to this virus, and while I don’t like that, I can’t really blame them.
It is impossible to calculate . . . and the moment you do attempt to put a number to it, some bright guy or gal with a sharp pencil is going to challenge your methodology and publically rip it to shreds.
Supposedly, the CDC is working on better `real time estimates’, that will hopefully give us a better idea of the impact of this pandemic. When those will ready for prime time, is hard to say.
But they won’t be `counts’, they will be estimates. Like the ones above. And frankly, that’s as good as we are ever likely to get.
Where the CDC has erred, in my opinion, is in not explaining the challenges of counting cases and fatalities to the press, and the public, better than they have.
They’ve also confused the issue by changing – without comment or explanation – the way they track P&I cases. If the reporting system needs to change (no quibble there), for goodness sakes, explain why. (Note: see the update at the bottom of this blog)
They need to understand that people are watching, and scrutinizing (and yes – even comparing week to week) the data they put out.
When there are changes in format, or inconsistencies in the numbers, people will notice.
Sure . . . I’d like to be able to jump up and down, scream, and vent over the lack of good, solid numbers . . . but I understand, at least a little, the scope of the problem. Even if I were in charge, and were given all the resources the CDC has, I don’t know how I’d go about coming up with an accurate count of pandemic flu deaths.
At least not without the aid of a magic wand.
* * * * * * * * * *
UPDATE: Buried near the bottom of another CDC webpage - Monitoring Influenza Activity, Including 2009 H1N1 dated September 11, 2009, 6:00 PM ET are 3 updates (which do not show up in the What’s New listing on the site) made sometime yesterday (Nov 6th).
How accurate a representation are these numbers?
Updated Nov 06
Laboratory-confirmed data is thought to be an underestimation of the true number of cases because most people will not be tested for influenza. However, influenza and pneumonia syndrome hospitalizations and deaths may be an overestimate of actual number of flu-related hospitalizations and deaths because that diagnostic category includes other illnesses. Influenza and pneumonia syndromic reports are less specific to influenza, but are helpful in following disease trends.
Will the old 2009 H1N1 counts prior to August 30, 2009 remain available?
Yes, the cumulative number of 2009 H1N1-related hospitalizations and deaths reported to CDC from April through August 2009 will be archived and available for future reference.
What hospitalizations and death information will CDC report publicly?
Added Nov 06
From August 30-October 24, 2009 CDC publicly posted reports of both laboratory-confirmed influenza for all types of influenza AND syndromic cases, i.e., cases of presumed influenza and/or pneumonia based on ICD-9 coded hospitalizations or death reports each week. From the week beginning October 25-31, 2009; however, CDC will only publicly report laboratory-confirmed influenza-associated hospitalizations and deaths. The reason for this change in public reporting is that the interpretation of syndromic data (most of which is not laboratory-confirmed) relative to laboratory-confirmed data has been challenging. CDC will continue to collect and analyze both laboratory-confirmed and syndromic data reported to CDC by states.
How many states are reporting laboratory-confirmed hospitalizations and deaths?
Added Nov 06
The number of states reporting laboroatory-confirmed hospitalizations and deaths varies each week, however, for the week of October 25-31, 2009, more than 30 U.S. states were continuing to report laboratory-confirmed data.
So it seems the CDC did offer some explanation for the change (Interpretation of the data was `challenging’ ), but managed to put it someplace where it wasn’t going to be terribly easy to find.
There is – btw – a link on the surveillance page where the new chart is located called Questions and Answers: Monitoring Influenza Activity, Including 2009 H1N1 – but as of this writing, it is a dead link.
The proper link is http://www.cdc.gov/h1n1flu/reportingqa.htm
My thanks to Cathy M on the Flu Wiki for finding and posting this notice.
* * * * * * * * * *
Note: If anyone from the CDC is reading this, and finds that I’ve gotten any of it wrong, let me know. I’d appreciate the opportunity to make any corrections.
