Strong scaling for numerical weather prediction at petascale with the atmospheric model NUMA
Loading...
Authors
Müller, Andreas
Kopera, Michal A.
Marras, Simone
Wilcox, Lucas C.
Issac, Tobin
Giraldo, Francis X.
Subjects
Atmospheric modeling
numerical weather prediction
dynamical core
global circulation model
parallel scalability
spectral elements
Galerkin methods
petascale
numerical weather prediction
dynamical core
global circulation model
parallel scalability
spectral elements
Galerkin methods
petascale
Advisors
Date of Issue
2019
Date
Publisher
SAGE
Language
Abstract
Numerical weather prediction (NWP) has proven to be computationally challenging due to its inherent multiscale nature. Currently, the highest resolution global NWP models use a horizontal resolution of 9 km. At this resolution, many important processes in the atmosphere are not resolved. Needless to say, this introduces errors. In order to increase the resolution of NWP models, highly scalable atmospheric models are needed. The non-hydrostatic unified model of the atmosphere (NUMA), developed by the authors at the Naval Postgraduate School, was designed to achieve this purpose. NUMA is used by the Naval Research Laboratory, Monterey as the engine inside its next generation weather prediction system NEPTUNE. NUMA solves the fully compressible Navier–Stokes equations by means of high-order Galerkin methods (both spectral element as well as discontinuous Galerkin methods can be used). NUMA is capable of running middle and upper atmosphere simulations since it does not make use of the shallow-atmosphere approximation. This article presents the performance analysis and optimization of the spectral element version of NUMA. The performance at different optimization stages is analyzed using a theoretical performance model as well as measurements via hardware counters. Machine-independent optimization is compared to machine-specific optimization using Blue Gene (BG)/Q vector intrinsics. The best portable version of the main computations was found to be about two times slower than the best non-portable version. By using vector intrinsics, the main computations reach 1.2 PFlops on the entire IBM Blue Gene supercomputer Mira (12% of the theoretical peak performance). The article also presents scalability studies for two idealized test cases that are relevant for NWP applications. The atmospheric model NUMA delivers an excellent strong scaling efficiency of 99% on the entire supercomputer Mira using a mesh with 1.8 billion grid points. This allows running a global forecast of a baroclinic wave test case at a 3-km uniform horizontal resolution and double precision within the time frame required for operational weather prediction.
Type
Article
Description
The article of record as published may be found at https://doi.org/10.1177/1094342018763966
Series/Report No
Department
Applied Mathematics (MA)
Organization
Naval Postgraduate School (U.S.)
Identifiers
NPS Report Number
Sponsors
This work was supported by the Office of Naval Research (PE-0602435 N), the Air Force Office of Scien- tific Research (Computational Mathematics program), and the National Science Foundation (Division of Mathematical Sciences; 121670). This research used resources of the Argonne Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC02-06CH11357.
Funder
Format
Citation
Müller, Andreas, et al. "Strong scaling for numerical weather prediction at petascale with the atmospheric model NUMA." The International Journal of High Performance Computing Applications 33.2 (2019): 411-426.
Distribution Statement
Rights
This publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.