#15,919
The late George E. P. Box (18 October 1919 – 28 March 2013) - Professor Emeritus of Statistics at the University of Wisconsin - is often credited with coining the familiar adage:
“All models are wrong, but some models are useful.”
While imperfect, we use computer models every day to try to mathematically simulate real- world events; everything from climate change to Hurricane Forecasting to COVID hospitalization and death forecasts.
Anyone who's left their umbrella home due to an errant weather forecast, or evacuated for a hurricane that never arrived, knows that models aren't always 100% dependable. But we've seen dramatic improvement in weather forecasting (and hurricane) models over the past 20 years.
COVID forecasting is still very much in its infancy, and as such, is prone to generating a wide range of future `possibilities'. While we can't take individual COVID-19 forecasts literally yet, they can tell us - in close to real time - what the trends are, and suggest when and where adjustments can and should (or shouldn't) be made in order to change the course of the epidemic.
Over time, just as we have with hurricane forecasting models, we'll discover which datasets and assumptions best represent real-world conditions, and produce the most reliable results.
Although the CDC has been producing COVID hospitalization and death forecasts for more than a year, the ECDC unveiled their COVID-19 Forecast Hub only this week. As with the CDC forecasts, the ECDC uses dozens of different models, and can display them either individually, or melded into a EuroCOVIDhub-ensemble.
First some excerpts from yesterday's announcement, then a link to the new platform.
Forecasting COVID-19 cases and deaths in Europe - new hub will support European pandemic planningPress release
22 Apr 2021
Europe is at a critical juncture in its fight against COVID-19. Many countries are currently relaxing restrictions, some against a backdrop of increasing numbers of cases and with new variants emerging, while still rolling out nationwide vaccination programmes. Evidence-based near-term forecasts of COVID-19 cases and deaths can therefore be of very helpful for public health decision-making.
This is the catalyst for the new European COVID-19 Forecast Hub, an initiative of the European Centre for Disease Prevention and Control (ECDC), with the support of the Centre for Mathematical Modelling of Infectious Diseases (CMMID) at the London School of Hygiene & Tropical Medicine.
Based on similar projects in the US, Germany and Poland, the hub collates weekly short-term forecasts from modelling teams around the world, predicting the number of COVID-19 cases and deaths four weeks ahead in 32 countries: 30 EU/EEA Member States, the UK and Switzerland. These are combined into an ‘ensemble’ and displayed alongside the individual forecasts.
“We are excited about European modelling teams collaborating together with others from around the world to develop accurate and timely predictions of COVID-19 cases and deaths in Europe. We clearly foresee the potential impact on public health policy. This initiative yields insight now and also strengthens our capacity to respond to infectious disease threats in the future” says Helen Johnson, Mathematical Modeller, ECDC.
During this pandemic, many forecasts have been produced, some very accurate, others less so. Scientists and their models need to remain responsive and flexible, especially given the increasing complexity of the pandemic, including changes in behaviour, “pandemic fatigue”, vaccine roll-out and newly-emerging variants. Some models account for these evolving factors less than others, leading to potentially biased forecasts. Collating forecasts into a single ‘ensemble’ forecast makes it possible to counteract the biases from different models, providing more accurate and reliable forecasts than a single model. This approach has been used successfully during outbreaks of other diseases, including Ebola, dengue, and influenza.
According to Sebastian Funk, Professor of Infectious Disease Dynamics, London School of Hygiene & Tropical Medicine, UK: "By constructing an ‘ensemble’, we can leverage the contribution from each of the modelling teams, with the aim of creating the best possible forecast.”
As Dr. José L Aznarte, Associate Professor of Artificial Intelligence, UNED Madrid, Spain, puts it: "[...] the weekly aggregation of all our predictions into an ensemble model which consistently outperforms every single one, is the perfect example of scientific cooperation for social good.”
Every week on Mondays, the teams submit their forecasts of cases and deaths, predicting up to four weeks ahead. The hub's website displays current and past forecasts from each of the models, the ensemble forecast, and various evaluations of the submissions. The latest round of submissions, collated on 19 April 2021, contains 33 models representing a wide range of modelling methodologies.
According to Dr. Johannes Bracher, postdoctoral researcher in Statistics, Karlsruhe Institute of Technology, Germany: "Our experience from running a German and Polish COVID-19 Forecast Hub has been that this format provides a very useful feedback loop for modellers. It creates a track record of what models said at different points in time, allowing for apples-to-apples comparisons, systematic evaluation and, ultimately, improvement of models."
This is confirmed by Dr. Tyll Krüger, Professor at Wrocław University of Science and Technology, Poland: " I consider the European Forecast Hub to be an extremely valuable tool for policy makers, epidemiologists, public health professionals, modellers and the public. The Forecast Hub is the ideal platform to compare and ultimately improve forecasts from many different groups in a scientific and transparent way. It also provides a clear representation of scientists' certainties and uncertainties about the epidemic dynamics in the near future. The ‘ensemble’ forecast, which is composed of many individual forecasts, should receive broad public coverage in the media of every European country."
However, the European Forecast Hub provides more than forecasts. It creates a platform for the participating modellers to meet colleagues from across Europe and elsewhere. A weekly online meeting is held, during which the teams share ideas and thoughts, representing a collaborative project that will be of benefit for years to come. During these meet-ups, the most recent forecasts are discussed and teams have the opportunity to present and discuss their forecast models or related work. There are currently 28 teams participating, representing eight different countries: Poland, Spain, Germany, Czechia, Italy, UK, Australia, and the USA. The Hub is also looking forward to welcoming further teams in the future.
“I very much enjoy attending the weekly on-line meetings, which both myself and my PhD student, Barbara, have always participated in. [...] We also very much appreciate the spirit of transparent co-operation among the different working groups, which I find very stimulating” says Professor Paolo Giudici, Professor of Statistics at the University of Pavia, Italy.
As Dr. José L Aznarte explains: “The variety of approaches pursued by the different international teams represents a huge opportunity to learn from the expertise of other researchers, usually coming from different fields."
This project was enabled by ECDC, supported by the Centre for Mathematical Modelling of Infectious Diseases (CMMID) at the London School of Hygiene & Tropical Medicine, and supported through collaboration with the University of Massachusetts-Amherst, the Karlsruhe Institute of Technology, and the Robert-Koch Institute.
https://covid19forecasthub.eu/visualisation.html
