MicroEnsemble
Members
First name (team leader)
Jonathan
Last name
Weyn
Organisation name
Microsoft
Organisation type
Large Tech Company
Organisation location
United States of America
First name
Hannah
Last name
Guan
Organisation name
Harvard University
Organisation type
Academic (Student)
Organisation location
United States of America
First name
Soukayna
Last name
Mouatadid
Organisation name
University of Toronto
Organisation type
Research Organisation (Academic, Independent, etc.)
Organisation location
Canada
First name
Paulo
Last name
Orenstein
Organisation name
Instituto de Matemática Pura e Aplicada
Organisation type
Research Organisation (Academic, Independent, etc.)
Organisation location
Brazil
First name
Judah
Last name
Cohen
Organisation name
Massachusetts Institute of Technology
Organisation type
Research Organisation (Academic, Independent, etc.)
Organisation location
United States of America
First name
Lester
Last name
Mackey
Organisation name
Microsoft Research
Organisation type
Large Tech Company
Organisation location
United States of America
First name
Alex
Last name
Lu
Organisation name
Microsoft Research
Organisation type
Large Tech Company
Organisation location
United States of America
First name
Genevieve
Last name
Flaspohler
Organisation name
Rhiza Research
Organisation type
Small & Medium Enterprise or Startup
Organisation location
United States of America
First name
Zekun
Last name
Ni
Organisation name
Microsoft
Organisation type
Large Tech Company
Organisation location
United States of America
First name
Haiyu
Last name
Dong
Organisation name
Microsoft
Organisation type
Large Tech Company
Organisation location
United States of America
Models
Model name
Huracan
Number of individuals supporting model development:
1-5
Maximum number of Central Processing Units (CPUs) supporting model development or forecast production:
48-1,000
Maximum number of Graphics Processing Units (GPUs) supporting model development or forecast production:
64-256
How would you best classify the IT system used for model development or forecast production:
Cloud computing system
Model summary questionnaire for model Huracan
Please note that the list below shows all questionnaires submitted for this model.
They are displayed from the most recent to the earliest, covering each 13-week competition period in which the team competed with this model.
Which of the following descriptions best represent the overarching design of your forecasting model?
- Post-processing of numerical weather prediction (NWP) data.
- Hybrid model that integrates physical simulations with machine learning or statistical techniques.
- Ensemble-based model, aggregating multiple predictions to assess uncertainty and variability.
What techniques did you use to initialise your model? (For example: data sources and processing of initial conditions)
This model uses only initializations from the ECMWF extended range ensemble.
If any, what data does your model rely on for real-time forecasting purposes?
This model uses operational S2 class ECMWF extended ensemble forecasts.
What types of datasets were used for model training? (For example: observational datasets, reanalysis data, NWP outputs or satellite data)
ECMWF extended ensemble forecasts and hindcasts (EEFO, ENFO); ERA5 reanalysis.
Please provide an overview of your final ML/AI model architecture (For example: key design features, specific algorithms or frameworks used, and any pre- or post-processing steps)
The model is an ensemble transformer (PoET). Data from ECMWF S2S are converted to weekly averages. Hindcasts are used for training and forecasts for inference, resulting in a 100-member ensemble with member voting for quintiles.
Have you published or presented any work related to this forecasting model? If yes, could you share references or links?
The approach is based on PoET: https://journals.ametsoc.org/view/journals/aies/3/1/AIES-D-23-0027.1.xml
Before submitting your forecasts to the AI Weather Quest, did you validate your model against observational or independent datasets? If so, how?
We only evaluated against ERA5, the competition targets.
Did you face any challenges during model development, and how did you address them?
The single biggest challenge was data processing and I/O. Even at 1.5-degree resolution, the datasets are TBs large and processing train-ready data requires careful consideration to complete in a timely manner. The availability of the S2 class dataset from ECMWF is very helpful.
Are there any limitations to your current model that you aim to address in future iterations?
The model is trained on a relatively small dataset of hindcasts. Adding more data and doing more parameter tuning are needed to get better performance.
Are there any other AI/ML model components or innovations that you wish to highlight?
No.
Who contributed to the development of this model? Please list all individuals who contributed to this model, along with their specific roles (e.g., data preparation, model architecture, model validation, etc) to acknowledge individual contributions.
Jonathan Weyn - End-to-end development (data, model, evaluation)
Zekun Ni - Experiments with AI prediction models
Haiyu Dong - advisory role
Model name
StillLearning
Number of individuals supporting model development:
6-10
Maximum number of Central Processing Units (CPUs) supporting model development or forecast production:
8-48
Maximum number of Graphics Processing Units (GPUs) supporting model development or forecast production:
< 4
How would you best classify the IT system used for model development or forecast production:
Small cluster
Model summary questionnaire for model StillLearning
Please note that the list below shows all questionnaires submitted for this model.
They are displayed from the most recent to the earliest, covering each 13-week competition period in which the team competed with this model.
Which of the following descriptions best represent the overarching design of your forecasting model?
- Post-processing of numerical weather prediction (NWP) data.
- Statistical model focused on generating quintile probabilities.
- Hybrid model that integrates physical simulations with machine learning or statistical techniques.
What techniques did you use to initialise your model? (For example: data sources and processing of initial conditions)
We did not initialize our model and instead relied on dynamical model forecasts that were themselves initialized.
If any, what data does your model rely on for real-time forecasting purposes?
For real-time forecasting, our model relies on ECMWF forecasts for temperature, precipitation, and mean sea-level pressure.
What types of datasets were used for model training? (For example: observational datasets, reanalysis data, NWP outputs or satellite data)
Our model was trained using historical ECMWF forecasts and reforecasts and ERA5 reanalysis data.
Please provide an overview of your final ML/AI model architecture (For example: key design features, specific algorithms or frameworks used, and any pre- or post-processing steps)
Our model learns site- and date-specific corrections for ECMWF dynamical model forecasts by combining lagged observations, dynamical forecasts, and climatology to minimize forecasting error over adaptively-selected training periods.
Have you published or presented any work related to this forecasting model? If yes, could you share references or links?
This approach was inspired by Adaptive Bias Correction for Improved Subseasonal Forecasting (Mouatadid et al., Nature Communications, 2023, https://www.nature.com/articles/s41467-023-38874-y).
Before submitting your forecasts to the AI Weather Quest, did you validate your model against observational or independent datasets? If so, how?
Yes, we validated our forecasts by evaluating RPSS against ERA5 reanalysis data.
Did you face any challenges during model development, and how did you address them?
Some of our greatest challenges have concerned data quality and availability. For example,
1. As we do not have access to the mars service underlying the contest pipeline for generating evaluation data, we have been unable to replicate the evaluation data pipeline used in the competition. We attempted to do so using the cdsapi, but the resulting data always differs from the official evaluation data by a significant margin.
2. Our delayed access to ECMWF forecasts required us to make all contest forecasts on weekends rather than during the workweek.
3. Our regularly scheduled downloads of NetCDF forecast data via the ecmwfapi fail intermittently due to an apparent delay in the release of the NetCDF files.
Are there any limitations to your current model that you aim to address in future iterations?
In the future, we hope to make our training and prediction pipeline more robust to data corruption and missingness.
Are there any other AI/ML model components or innovations that you wish to highlight?
No
Who contributed to the development of this model? Please list all individuals who contributed to this model, along with their specific roles (e.g., data preparation, model architecture, model validation, etc) to acknowledge individual contributions.
Contributors include Hannah Guan (data preparation, model development, model validation), Soukayna Mouatadid (data preparation, model development, model validation), Paulo Orenstein (data preparation, model development, model validation), Genevieve Flaspohler (data preparation, model development), Judah Cohen (model development, model validation), Alex Lu (data preparation, model development, model validation), and Lester Mackey (data preparation, model development, model validation).
Model name
MicroDuet
Number of individuals supporting model development:
6-10
Maximum number of Central Processing Units (CPUs) supporting model development or forecast production:
48-1,000
Maximum number of Graphics Processing Units (GPUs) supporting model development or forecast production:
64-256
How would you best classify the IT system used for model development or forecast production:
Cloud computing system
Model summary questionnaire for model MicroDuet
Please note that the list below shows all questionnaires submitted for this model.
They are displayed from the most recent to the earliest, covering each 13-week competition period in which the team competed with this model.
Which of the following descriptions best represent the overarching design of your forecasting model?
- Post-processing of numerical weather prediction (NWP) data.
- Statistical model focused on generating quintile probabilities.
- Hybrid model that integrates physical simulations with machine learning or statistical techniques.
- Ensemble-based model, aggregating multiple predictions to assess uncertainty and variability.
What techniques did you use to initialise your model? (For example: data sources and processing of initial conditions)
This model uses ECMWF EEFO data.
If any, what data does your model rely on for real-time forecasting purposes?
This model uses ECMWF EEFO data.
What types of datasets were used for model training? (For example: observational datasets, reanalysis data, NWP outputs or satellite data)
This model uses ECMWF EEFO forecasts and hindcasts for training and inference. Targets are ERA5.
Please provide an overview of your final ML/AI model architecture (For example: key design features, specific algorithms or frameworks used, and any pre- or post-processing steps)
This model is an ensemble of Huracan and StillLearning designed to capture the strengths of each model.
Have you published or presented any work related to this forecasting model? If yes, could you share references or links?
No
Before submitting your forecasts to the AI Weather Quest, did you validate your model against observational or independent datasets? If so, how?
We evaluated against ERA5.
Did you face any challenges during model development, and how did you address them?
See responses for Huracan and StillLearning.
Are there any limitations to your current model that you aim to address in future iterations?
See responses for Huracan and StillLearning.
Are there any other AI/ML model components or innovations that you wish to highlight?
No
Who contributed to the development of this model? Please list all individuals who contributed to this model, along with their specific roles (e.g., data preparation, model architecture, model validation, etc) to acknowledge individual contributions.
Contributors include Hannah Guan (data preparation, model development, model validation), Soukayna Mouatadid (data preparation, model development, model validation), Paulo Orenstein (data preparation, model development, model validation), Genevieve Flaspohler (data preparation, model development), Judah Cohen (model development, model validation), Alex Lu (data preparation, model development, model validation), and Lester Mackey (data preparation, model development, model validation). Contributors for Huracan include Jonathan Weyn (end-to-end development), Zekun Ni (experiments), and Haiyu Dong (advisory).
Submitted forecast data in previous period(s)
Please note: Submitted forecast data is only publicly available once the evaluation of a full competitive period has been completed. See the competition's full detailed schedule with submitted data publication dates for each period here.
Access forecasts data