Science and technology

The year saw much progress in science and technology. It brought, for example, scientific developments in a new cycle of the Integrated Forecasting System (IFS) implemented in June 2023. IFS Cycle 48r1 included a horizontal resolution increase in ensemble forecasts, a major upgrade of extended-range forecasts, the introduction of a multi-layer representation of snow, and a revised parametrization of microphysical processes. In parallel, the operational forecasting system for global atmospheric composition of the EU’s Copernicus Atmosphere Monitoring Service (CAMS), implemented by ECMWF, was substantially extended with the upgrade to IFS Cycle 48r1. We also developed an Artificial Intelligence Forecasting System (AIFS), which is the subject of a separate section.

Other advances included work towards IFS Cycle 49r1, to be introduced in 2024. This involved, for example, improvements to surface vegetation fields. Also, two-metre temperature measurements were to be used in 4D-Var data assimilation, the process that establishes the initial conditions of forecasts. In addition, we improved the prediction of clear-air turbulence (CAT), and we successfully predicted the 2023 El Niño event. We explored the real-time use of remote observations to improve fire danger forecasts, and we upgraded OpenIFS, a version of the IFS provided for research and education.

To follow changes in the Arctic climate with unprecedented detail and accuracy, scientists from the EU’s Copernicus Climate Change Service (C3S), implemented by ECMWF, developed the Copernicus Arctic Regional Reanalysis (CARRA). An ECMWF-coordinated project for a prototype system for a Copernicus CO2 service (CoCO2) to monitor anthropogenic CO2 emissions worldwide concluded its programme, and a new, decadally-varying climatology of tropospheric aerosol was developed. Data from new satellite missions were investigated or final preparations were made for receiving them.

In technology, we continued a multi-year effort to migrate daily operations data output from the file format GRIB edition 1 (GRIB1) to GRIB edition 2 (GRIB2). Time-critical forecasts ran well on the Atos high-performance computing facility (HPCF), and there were no delays in the production of forecasts due to issues with the system. The Data Handling System (DHS) provided a reliable service over the year, and a multi-purpose cloud infrastructure, called the Common Cloud Infrastructure (CCI), was installed in our data centre in Bologna (Italy). Finally, work was carried out to make the IFS ready for a hybrid CPU–GPU compute model.

IFS Cycle 48r1

The horizontal resolution of medium-range ensemble forecasts increased from 18 to 9 km. This led to big improvements in forecasts.

In June 2023, we upgraded our Integrated Forecasting System (IFS) to Cycle 48r1. This increased the horizontal resolution of medium-range ensemble forecasts (ENS) and substantially improved the skill of our weather predictions. The resolution increased from 18 to 9 km, the same resolution as the high-resolution forecast (HRES).

This led to big improvements in forecasts, for example of tropical cyclone tracks and intensity, but also of upper-air and surface variables. Extended-range forecasts now have 101 instead of 51 ensemble members and they run more frequently than before.

The horizontal resolution of medium-range ensemble forecasts increased from 18 to 9 km. This led to big improvements in forecasts.

Another major change in the forecast model was from a single-layer to a multi-layer representation of snow in the surface scheme. The multi-layer snow scheme markedly improved the realism of the snow pack in the model. The impact on two-metre temperatures in snow-prone regions included an improved daily cycle, and there were reduced snow depth forecast errors.

The parametrization of microphysical processes was also revised in Cycle 48r1. The aim was to allow supercooled drizzle drops to be formed and only to freeze if they come into contact with pre-existing ice or snow particles, or if they experience much colder temperatures. This made it possible for the IFS to predict high-impact freezing drizzle events, where supercooled rain/drizzle drops freeze on impact at the surface and form a glaze of ice.

Mean sea-level pressure (MSLP) in the centre of Tropical Cyclone Ilsa, forecast from 9 April 2023, 00 UTC, using the IFS Cycle 48r1 ensemble with a resolution of 9 km, and using the IFS Cycle 47r3 ensemble with a resolution of 18 km.

Tropical cyclone forecast

Mean sea-level pressure (MSLP) in the centre of Tropical Cyclone Ilsa, forecast from 9 April 2023, 00 UTC, using the IFS Cycle 48r1 ensemble with a resolution of 9 km, and using the IFS Cycle 47r3 ensemble with a resolution of 18 km.

Upgrading atmosphere monitoring

The operational forecasting system for global atmospheric composition of the EU’s Copernicus Atmosphere Monitoring Service (CAMS), implemented by ECMWF, was substantially extended with the upgrade to IFS Cycle 48r1. This included, for example, the introduction of the Belgian Assimilation System for Chemical ObsErvations (BASCOE) in CAMS. Together with the Carbon Bond Mechanism 5 (CB05) scheme, it became possible to fully represent chemistry throughout the atmosphere in the IFS for CAMS.

Other improvements were the addition of secondary organic aerosols and new tropospheric species; the improved treatment of emissions from specific sectors; and the assimilation of carbon monoxide retrievals from the Sentinel-5P satellite.

Error of the CAMS ozone forecast for day 1 and day 5 of Cycle 48r1 (blue) and Cycle 47r3 (red) against five ozone sondes over Antarctica in October 2022 during the ozone hole event. The ozone forecasts for day 1 are similar as they are initialised from the respective ozone analysis. The forecast errors for day 5 in the stratosphere (50–10 hPa) are much smaller in Cycle 48r1 than in Cycle 47r3. The horizontal lines show variability within +/–1 standard deviation.

CAMS ozone forecast errors

Error of the CAMS ozone forecast for day 1 and day 5 of Cycle 48r1 (blue) and Cycle 47r3 (red) against five ozone sondes over Antarctica in October 2022 during the ozone hole event. The ozone forecasts for day 1 are similar as they are initialised from the respective ozone analysis. The forecast errors for day 5 in the stratosphere (50–10 hPa) are much smaller in Cycle 48r1 than in Cycle 47r3. The horizontal lines show variability within +/–1 standard deviation.

Work towards IFS Cycle 49r1

Meanwhile, work continued on improving the IFS in readiness for Cycle 49r1, to be introduced in 2024. For example, physics changes were to include extensive improvements to surface vegetation fields and changes to make the interpolation of the temperature to two metres more realistic.

Also, for the first time, two-metre temperature measurements were to be used in 4D-Var data assimilation, the process that establishes the initial conditions of forecasts. This proved beneficial after extensive testing and tuning. In addition, there were improvements to snow data assimilation. Overall, better two-metre temperature forecasts could be achieved, especially in northern hemisphere winter.

Other changes to be implemented in Cycle 49r1 include the introduction of an urban scheme into the IFS to improve temperature and wind speed forecasts for urban areas. This is important because the temperature in cities is often elevated due to an effect known as the urban heat island.

The plot illustrates the improvement (negative) Cycle 49r1 brings for two-metre temperature forecasts verified against weather station observations (SYNOP). It shows the percentage change in root-mean-square error for December 2021 to February 2022 and the region 20° to 90°N. The grey rectangles show 95% confidence intervals.

Two-metre temperature improvements

The plot illustrates the improvement (negative) Cycle 49r1 brings for two-metre temperature forecasts verified against weather station observations (SYNOP). It shows the percentage change in root-mean-square error for December 2021 to February 2022 and the region 20° to 90°N. The grey rectangles show 95% confidence intervals.

Probabilistic clear-air turbulence

In 2023, we improved the prediction of clear-air turbulence (CAT), which is the main weather threat to civil aviation at cruising level in the lower stratosphere. CAT was originally made available from ECMWF’s high-resolution forecast in October 2021, and in 2023 it became a fully probabilistic product. This approach can help to judge the significance of this intermittent and rare process, and to obtain reasonably smooth fields. At the time of its introduction in June 2023, it was the highest-resolution probabilistic CAT product worldwide in terms of vertical and horizontal resolution.

The image shows an 18-hour forecast of the probability of CAT to exceed 0.1 m2/3s–1 at Flight level 300 (about 9,200 m). In ecCharts, a suite of web-based applications to inspect, explore and visualise ECMWF forecast data, different parameters can easily be overlayed, and any region globally can be considered at different zoom settings.

CAT in ecCharts

The image shows an 18-hour forecast of the probability of CAT to exceed 0.1 m2/3s–1 at Flight level 300 (about 9,200 m). In ecCharts, a suite of web-based applications to inspect, explore and visualise ECMWF forecast data, different parameters can easily be overlayed, and any region globally can be considered at different zoom settings.

Predicting the 2023 El Niño event

ECMWF forecasts, and forecasts provided by the EU-funded Copernicus Climate Change Service (C3S) implemented by ECMWF, were consistent throughout the first half of 2023 in indicating the development of an El Niño event. Such an event is a prolonged period of abnormally high sea-surface temperatures (SST) in the tropical Pacific Ocean. It goes hand in hand with changes in atmospheric conditions and can have strong repercussions on global weather patterns. As the year progressed, an El Niño event did in fact materialise. The figure shows a prediction made by ECMWF in the summer of how the El Niño might look towards the end of 2023. C3S forecasts from different centres generally agreed about the trend.

The figure shows a mean anomaly forecast. Uncertainty stems from unpredictable wind variations over the equatorial Pacific.

We account for this uncertainty by running ensemble forecasts – the details in each ensemble member evolve differently, and we obtain a ‘plume’ of forecast SST values.

The chart shows the SST ensemble mean anomaly forecast from July 2023 for October–November–December 2023, according to ECMWF’s seasonal forecasting system SEAS5. The NINO3.4 region is indicated by the box.

SST ensemble mean anomaly forecast

The chart shows the SST ensemble mean anomaly forecast from July 2023 for October–November–December 2023, according to ECMWF’s seasonal forecasting system SEAS5. The NINO3.4 region is indicated by the box.

Remote observations in fire forecasting

We explored the real-time use of remote observations to improve fire danger forecasts.

As part of a project funded by the European Space Agency (ESA), we explored the real-time use of remote observations to improve fire danger forecasts. The proposed fire occurrence probability index (FOPI) combines the most-used model of fire danger, the Canadian fire weather index (FWI), with remote observations of vegetation optical depth (VOD) as a proxy of fuel amount and moisture.

We explored the real-time use of remote observations to improve fire danger forecasts.

There are two innovative aspects in FOPI. The first is that it provides a framework to account for real-time fuel availability. FOPI thus limits unrealistically high values registered in desert areas, where fire activity is hindered. The second advantageous aspect of FOPI is that it expresses a probability of fire occurrence based on previous observations.

The maps show same-day forecasts for 4 January 2020. The left panel shows the FOPI prediction for that day, and the right panel shows the FWI prediction. FOPI and FWI are both dimensionless. Recorded burnt areas (BA) for the same day are provided as black circles. The size of the symbols is proportional to the burnt area expressed in hectares.

Forecasts for large fire events in 2020 in New South Wales, Australia

The maps show same-day forecasts for 4 January 2020. The left panel shows the FOPI prediction for that day, and the right panel shows the FWI prediction. FOPI and FWI are both dimensionless. Recorded burnt areas (BA) for the same day are provided as black circles. The size of the symbols is proportional to the burnt area expressed in hectares.

OpenIFS adopts new version

OpenIFS, a supported and easily accessible version of the IFS provided for research and education, was upgraded to the latest IFS model cycle. A beta release of OpenIFS 48r1 was made available for testing by some licensees in September 2023, and by the end of the year the new version was nearly ready for all users. It has the forecast capabilities of IFS Cycle 48r1, which became operational in June 2023.

The upgrade heralds the release of more frequent model releases, planned about every two years. Many infrastructure and code changes had to be implemented to move from OpenIFS 43r3 to 48r1. Users can now run the model in double or single precision, and they benefit from improvements such as moist physics upgrades and the new multi-layer snow scheme.

The graph shows the number of OpenIFS licences active any year since 2014.

OpenIFS licences

The graph shows the number of OpenIFS licences active any year since 2014.

New data show wetter and warmer Arctic

Scientists from C3S have observed major changes in Arctic precipitation, air temperature and sea-ice coverage over the most recent three decades. The changes have big impacts in the Arctic, but they also affect northern hemispheric weather and climate more generally. One trend is increasing precipitation in the Arctic, particularly rain at the expense of snow over ocean regions. To follow these changes with unprecedented detail and accuracy, C3S developed the Copernicus Arctic Regional Reanalysis (CARRA): a state-of-the-art, data-driven reanalysis that enables a new focus on the Arctic climate. The CARRA dataset is now updated every month adding one month of new data close to real time.

A map of change in annual rainfall from 1991 to 2021 shows a particularly marked increase along the southeast coastline of Greenland.

Increase in rainfall

A map of change in annual rainfall from 1991 to 2021 shows a particularly marked increase along the southeast coastline of Greenland.

Copernicus CO2 project concluded

A three-year, ECMWF-coordinated project for a prototype system for a Copernicus CO2 service (CoCO2) to monitor anthropogenic CO2 emissions worldwide concluded its ambitious programme successfully at the end of 2023. The purpose of this EU-funded initiative was to feed into a permanent anthropogenic greenhouse gas emissions Monitoring and Verification Support Capacity (CO2MVS). This capacity was going to provide continuous monitoring of carbon dioxide (CO2) and methane (CH4) emissions and fluxes using a combination of observations and Earth system modelling.

CO2MVS is being developed as part of CAMS. Among other things, CoCO2 provided an assessment of the current state of in-situ observation sites to monitor anthropogenic greenhouse gas emissions; prior knowledge of emissions and natural fluxes used as input to CO2MVS; the assimilation of observations into models; and the consideration of local as well as global scales.

This is a screenshot of a high-resolution ‘nature run’ of fluxes of CO2 produced by CoCO2 for the year 2021. It is intended to represent environmental conditions as accurately as possible.

CO2 fluxes

This is a screenshot of a high-resolution ‘nature run’ of fluxes of CO2 produced by CoCO2 for the year 2021. It is intended to represent environmental conditions as accurately as possible.

Preparation for SEAS6 and ERA6

Tropospheric aerosol has an important radiative impact on the atmosphere and wider Earth system. In configurations of the IFS for medium-range, extended-range and seasonal forecasting, it is represented as a fixed climatology. A new, decadally-varying climatology of tropospheric aerosol was developed to represent the large changes in anthropogenic aerosol that have occurred over recent decades. It was derived from and is compatible with the aerosol modelling used by CAMS. The target implementation was IFS Cycle 49r2, for use in ERA6 and the forthcoming SEAS6 seasonal forecast upgrade.

Clear benefits on forecasts as well as reanalysis were also seen as a result of using stronger ocean–atmosphere coupling in the data assimilation system, which establishes the initial conditions of forecasts. There is a target implementation in IFS Cycle 49r2, which was also going to be used to produce the ERA6 reanalysis.

Regional variation in the importance of aerosols can be demonstrated with an experiment in which all tropospheric aerosols are removed from the IFS. The figure shows the impact of doing so on the net surface solar radiation. It covers the period of June–July–August, calculated from 10 years of 10-member seasonal forecasts starting on 1 May.

Impact of removing tropospheric aerosols

Regional variation in the importance of aerosols can be demonstrated with an experiment in which all tropospheric aerosols are removed from the IFS. The figure shows the impact of doing so on the net surface solar radiation. It covers the period of June–July–August, calculated from 10 years of 10-member seasonal forecasts starting on 1 May.

New satellite missions

Following the successful launch of the first satellite in EUMETSAT’s Meteosat Third Generation series (MTG-I1) in December 2022, the first images from the new Flexible Combined Imager (FCI) became available to ECMWF. All of the advanced features of the new sensor were clearly demonstrated: more spectral channels, finer horizontal resolution and enhanced time sampling capability. As a preliminary step, ECMWF constructed a simulation of the first FCI images using high-resolution forecasts (2.8 km fields from a dedicated DestinE model run of the IFS). The realism of the model simulation compared to the real MTG image was quite striking. The Lightning Imager (LI) has also been successfully activated by EUMETSAT, and initial evaluations of the data have been performed by ECMWF.

We also made final-stage preparations for monitoring and assimilation of cloud radar and lidar observations from the EarthCARE satellite to be launched in 2024.

EarthCARE data will be available in near-real time, facilitating the potential operational assimilation of space-borne cloud radar and lidar observations into numerical weather prediction models for the first time. EarthCARE is a joint venture between the European Space Agency (ESA) and the Japan Aerospace Exploration Agency (JAXA).

True colour image constructed from the enhanced visible bands of the MTG-I1 FCI (left) and a visible radiative transfer simulation from the Integrated Forecasting System model run at 2.8 km.

MTG-I1 and Integrated Forecasting System

True colour image constructed from the enhanced visible bands of the MTG-I1 FCI (left) and a visible radiative transfer simulation from the Integrated Forecasting System model run at 2.8 km.

HPCF performance and archive status

Following the migration of computing from Reading to Bologna, in the autumn of 2022 ECMWF became settled into providing operational services from the new data centre as business as usual. The greater power and cooling capacity provided by the data centre compared to what was available in Reading made it possible to install the new Atos high-performance computing facility (HPCF). This provided sufficient computational performance to deliver the major scientific upgrade of IFS Cycle 48r1 in June 2023.

Time-critical forecasts and research workloads were running well and reliably on the Atos HPCF. However, as with all large and complex HPC systems, supporting the service was a constant challenge. In 2023, Atos/Eviden continued detailed technical investigations into the remaining stability issues and worked closely with ECMWF to maintain the overall system availability and reliability. The Atos work was expected to bear fruit in 2024 and result in further stability improvements.

The Data Handling System (DHS) is a large, tiered storage system that uses ten robotic tape libraries, more than 450 tape drives, almost 300 Linux servers, 34 petabytes of disk space, and about sixty thousand tapes. It provided a reliable service over the year. At the end of 2023, the data archive stood at 700 PB of primary data. The most important part of this data, amounting to 229 PB, is duplicated in different tape libraries for safety and recovery.

This chart shows the storage growth in ECMWF’s Meteorological Archival and Retrieval System (MARS) and in ECMWF’s File Storage (ECFS) system, which together amount to 700 PB of primary data. On top of that, a secondary data store duplicates some of the primary data.

Storage growth

This chart shows the storage growth in ECMWF’s Meteorological Archival and Retrieval System (MARS) and in ECMWF’s File Storage (ECFS) system, which together amount to 700 PB of primary data. On top of that, a secondary data store duplicates some of the primary data.

Migration from GRIB1 to GRIB2

During 2023, we continued a multi-year effort to migrate our model output data from the file format GRIB edition 1 (GRIB1) to GRIB edition 2 (GRIB2). This will enable us to exploit the much richer metadata available in GRIB2. It also responds to the call for global numerical weather prediction (NWP) at convection-permitting resolutions set out in ECMWF’s ten-year Strategy 2021–2030. Such resolutions require GRIB2 rather than GRIB1 data because of the limitations of GRIB1 grid definitions. We had already produced IFS output on vertical model levels using the GRIB2 format for several years. In 2023, we continued the transition to GRIB2 by applying it to the new ocean reanalysis, ORAS6, which was to become operational in 2024.

The chart shows some limitations of GRIB1 and some of the benefits GRIB2 will bring.

GRIB1 and GRIB2 characteristics

The chart shows some limitations of GRIB1 and some of the benefits GRIB2 will bring.

Start of operational cloud service from Bologna

A new cloud computing service entered operations in the summer of 2023.

Following a pilot project to gather use cases and requirements from Member States, a new cloud computing service was procured and entered operations in the summer of 2023.

The cloud service complements the existing HPC compute and DHS storage services by providing Member States with the flexibility to self-provision Virtual Machines (VMs) with the operating system and software stack of their choice, as well as object storage with a de-facto standard Amazon Simple Storage Service (S3) interface for interoperability.

A new cloud computing service entered operations in the summer of 2023.

The architecture is based on a Common Cloud Infrastructure (CCI) layer, on top of which the different services, such as the European Weather Cloud (EWC) and the Copernicus Data Stores (CDS), are built. This architecture also enables ECMWF to improve efficiency by starting to consolidate a number of user-facing and internal compute services into the same hardware platform.

The EWC service is the result of a distributed cloud computing project jointly developed with EUMETSAT to provide a community cloud for the European Meteorological Infrastructure. In 2023, it was running at ECMWF over 60 Member-State-driven use cases for research purposes and operational official duties. The service is also used for some collaborative projects driven by organisations such as the World Meteorological Organization (WMO) and EUMETNET.

This diagram shows the CCI and its support of the European Weather Cloud and the Copernicus Data Stores. IaaS, PaaS and SaaS stand for ‘infrastructure as a service’, ‘platform as a service’, and ‘software as a service’.

The Common Cloud Infrastructure

This diagram shows the CCI and its support of the European Weather Cloud and the Copernicus Data Stores. IaaS, PaaS and SaaS stand for ‘infrastructure as a service’, ‘platform as a service’, and ‘software as a service’.

Progress in Hybrid 2024

Hybrid 2024 is an internal ECMWF project to adapt the IFS so that it is ready for a hybrid CPU–GPU compute model. Increasing the forecast resolution relies on greater performance of the HPCF used, and powerful modern GPUs can potentially provide this if the application can be mapped to them effectively. Hybrid 2024 aims to incrementally adapt IFS model components to different accelerators and programming models, focussing both on developing GPU capabilities and a sustainable technical evolution of the model code itself.

The initial GPU adaptation work, which was expected to conclude in 2024, was carried out in close collaboration with ECMWF Member States. There are also strong synergies with the EU-funded Destination Earth initiative, in which ECMWF is one of the partners. This involvement provides ECMWF with access to EuroHPC exascale-class systems that feature large GPU partitions.

This chart shows how increasing the model resolution from 210 km in the 1980s to 9 km in 2023, for high-resolution forecasts (HRES) and ensemble forecasts (ENS), has involved ever more powerful high-performance computing facilities.

Supercomputers used at ECMWF

This chart shows how increasing the model resolution from 210 km in the 1980s to 9 km in 2023, for high-resolution forecasts (HRES) and ensemble forecasts (ENS), has involved ever more powerful high-performance computing facilities.