Strong scaling for numerical weather prediction at petascale with the atmospheric model NUMA
Loading...
Authors
Müller, Andreas
Kopera, Michal A.
Marras, Simone
Wilcox, Lucas C.
Issac, Tobin
Giraldo, Francis X.
Subjects
Advisors
Date of Issue
2015-11-05
Date
Publisher
American Mathematical Society
Language
Abstract
Numerical weather prediction (NWP) has proven to be computationally challenging due to its inherent multiscale nature. Currently, the highest resolution NWP models use a horizontal resolution of approximately 15 km. At the resolution many important processes in the atmosphere are not resolved. Needless to say this introduces errors. In order to increase the resolution of NWP models highly scalable atmospheric models are needed. The Non-hydrostatic Unified Model of the Atmosphere (NUMA), developed by the authors at the Naval Postgraduate School, was designed to achieve this purpose. NUMA is used by the Naval Research Laboratory, Monterey, as the engine inside its next generation weather prediction system NEPTUNE. NUMA solves the fully compressible Navier-Stokes equations be means of high-order Galerkin methods (both spectral element as well as discontinuous Galerkin methods can be used. Mesh generation is done using the p4est library. NUMA is capable of running middle and upper atmosphere simulations since it does not make use of the shallow-atmosphere approximation. This paper presents the performance analysis and optimization of the spectral element version of NUMA. The performance at different optimization stages is analyzed using hardware counters with the help of the Hardware Performance Monitor Toolkit as well as the PAPI library. Machine independent optimization is compared to machine specific optimization using BG/Q vector intrinsics. By using vector intrinsics the main computations reach 1.2 PFlops on the entire machine Mira. The paper also present scalability studies for two idealized test cases that are relevant for NWP applications. The atmospheric model NUMA delivers an excellent strong scaling efficiency of 99% on the entire supercomputer Mira using a mesh with 1.8 billion grid points. This allows us to run a global forecast of a baroclinic wave test case at 3 km uniform horizontal resolution and double precision within the time frame required for operational weather prediction.
Type
Article
Description
Series/Report No
Department
Organization
Identifiers
NPS Report Number
Sponsors
Financial support for the work presented in this paper was provided by the Office of Naval Research through Program Element PE-0602435N, The Air Force Office of Scientific Research through the Computation Mathematics program, and the National Science Foundation (Division of MAthematical Sciences) through program elelment 121760. AM, MK and SM are grateful to the National Research Council of the National Academies.
Funder
Format
Citation
arXiv:1511.01561v1 [cs.DC] 5 Nov 2015
Distribution Statement
Approved for public release; distribution is unlimited.
Rights
This publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.