The world’s weather has been simulated in astounding detail with the first-ever 1 km scale seasonal simulation of the atmosphere, completed on the US Summit supercomputer. It represents a major contribution towards the development of a ‘digital twin’ of the Earth system.

ECMWF scientists were given the unprecedented opportunity to run two seasonal simulations (winter and autumn) of the global atmosphere at a resolution of 1.4 km on Summit, one of the world’s fastest supercomputers. This was part of the US INCITE1 (Innovative and Novel Computational Impact on Theory and Experiment) programme for 2020.

ECMWF’s operational medium-range weather forecasts are currently run at a horizontal resolution of 9 km, and coarser still in the long- and extended-range set up. Increasing that to nearly 1 km in a global simulation of several months represents a huge step both scientifically and computationally.

The work showed that at resolutions close to 1 km, it is possible to resolve the impact of deep convection on the circulation. Deep convection is a process that plays a key role in the re-distribution of energy and moisture in the tropics, driving the wider-scale circulation, and influencing mid-latitude weather. Typically, deep convection is parametrized using assumptions to represent convection as a subgrid process expressed through its bulk effect on the larger-scale flow. ECMWF has been very successful in the past 20 years in designing and improving convection parametrization.

In the 1.4 km experiment, the parametrization schemes for deep convection and gravity wave drag were switched off. For comparison, experiments were also run at 9 km resolution, both with and without parametrized deep convection and gravity wave drag. The simulations were uncoupled (no ocean or wave model), with 137 vertical levels.

Albeit only single realisations, the seasonal 1 km data provide a reference to evaluate the strengths and weaknesses of ECMWF’s operational numerical weather prediction forecasts and indeed to further improve the existing convection parametrization, when operating at horizontal resolutions affordable in the near future.

The 1.4 km simulation showed a remarkably realistic large-scale circulation, with improved representation of convective storm activity compared with simulations at 9 km. However, other aspects such as the magnitude of tropical rainfall and the Madden Julian Oscillation, an important source of predictability, did not improve.

Convective storm activityConvective storm activity over the Gulf of Mexico shows a realistic distribution in the 1.4 km simulation (c). At 9 km resolution, convective activity is less realistic – being much too weak with parametrized convection (a) and too strong when the parametrization is switched off (b). Convective activity is illustrated with updraft helicity (> 5 m2 s-2), a combined measure of updraft strength and cyclonic vorticity, which is dominated here by the impact of more realistic vertical velocities.
From Wedi, N. P., Polichtchouk, I., Dueben, P., Anantharaj, V. G., Bauer, P., Boussetta, S., et al. (2020). A baseline for global weather and climate simulations at 1 km resolution. Journal of Advances in Modeling Earth Systems, 12, e2020MS002192. https://doi.org/10.1029/2020MS002192.
Creative Commons Attribution Licence CC BY 4.0.

The simulations were conducted on only a fraction of Summit, at a speed of about one simulated week per day. The setup initially focused on efficient use of Summit’s central processing units (CPUs). First steps have also been taken to move parts of the model to graphics processing units (GPUs), which hold the promise of substantial further speed enhancements. This successful use of a hybrid computer architecture is a testament to the success of ECMWF’s Scalability Programme, and is key to realising km-scale simulations at speeds required for time-critical global numerical weather prediction.

The fidelity and realism of the 1.4 km simulations, with respect to the well-calibrated simulation at 9 km with parametrized deep convection, is remarkable, and achieved without adjustments to the remaining parametrizations in the model.

These ground-breaking simulations are a major step towards the development of digital twins of the Earth system and a core contribution to the European Destination Earth programme. Used in observing system simulation experiments, such high-resolution models could also support the planning of future satellite missions. Routine use of 1 km resolution simulations may become possible within the next decade.

Hurricane DorianThe 1 km simulation captures the intricate cloud structure of category 5 hurricane Dorian making landfall in the Bahamas on 1 September 2019.
Image in collaboration with NVIDIA, created by OMNIVERSE.
1 This research used resources of the Oak Ridge Leadership Computing Facility (OLCF), which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725.

Going down to 1 km is a quantum leap. It allows us to see how explicitly simulated deep convection interacts with and induces global changes in weather patterns. The results will help us to advance global numerical weather prediction as well as climate science.

Nils Wedi, Head of Earth System Modelling Section at ECMWF