As data volumes grow, and the demands for energy-efficient computing become ever greater, machine learning has a key role to play in numerical weather prediction (NWP), and in Earth system modelling and prediction more generally.

Machine learning (ML) is one branch of artificial intelligence (AI) where computer algorithms can be trained to reproduce patterns within large amounts of data and so represent complex processes, such as those within the Earth system. With applications much wider than just NWP, AI is a field where new developments are emerging almost daily.

Recognising the opportunities, but also the challenges involved, ECMWF launched a ten-year Roadmap in 2021 which set out a path towards the integration of ML into NWP and environmental services. In 2022, we transitioned to a three-person ML coordination team, with members from across the organisation, to take activities forward.

A huge amount of progress has already been made in exploring and developing ML techniques that could play a role in the NWP workflow, such as the data assimilation process and the model physics.

In 2022 we moved closer to bringing ML systems into our operational forecasts. For example, a new ML-based technique which improves the identification of errors in observations will become operational in 2023.

While not yet operational, there were also further successes in using neural networks to emulate Earth system model components, by training them with input/output data from conventional physically-based code.

The radiation scheme from the Integrated Forecasting System (IFS) was successfully emulated as part of MAELSTROM, a European High-Performance Computing Joint Undertaking project. Using the graphical processing unit (GPU) nodes in ECMWF’s Atos HPC facility and running decoupled from the IFS, the emulator is around 60 times faster than the conventional solver run on central processing units (CPUs).

Work was then extended to emulate three-dimensional cloud effects in the radiation scheme – something that is currently too computationally expensive to incorporate within operational predictions.

Another pillar of the ML Roadmap is ECMWF’s leading role in helping to build the knowledge and skills needed within NWP communities in our Member and Co-operating States and more widely, through training, events and the sharing of relevant datasets.

2022 saw the development of a Massive Open Online Course (MOOC) on Machine Learning in Weather and Climate, produced with the International Foundation on Big Data and Artificial Intelligence for Human Development (IFAB), and with contributions from a wide range of experts.

Developments in ML are also key for our contribution to the EU Destination Earth Initiative (DestinE), in which ECMWF is working alongside EUMETSAT and ESA. The use of ML is being explored for the post-processing of the output of digital twin simulations and in particular for developing techniques to quantify the uncertainties of the kilometre-scale simulations.

Within the wider AI community, there were very significant developments. Making use of ECMWF’s publicly available ERA5 reanalysis dataset, groups at Google, NVIDIA and Huawei each produced pure machine-learning weather forecasting models which, for the first time, were comparable in quality to the IFS.

Their experiments did not include the data assimilation part of the NWP process, and forecasts were performed for only a small sample of years, but nonetheless, these were landmark results, and they will encourage further exploration of weather forecasting based entirely on machine learning models.

We are considering the implications and formulating recommendations for our machine learning plans going forward. 2023 promises to be a particularly exciting year.