Strong scaling for numerical weather prediction at petascale with the atmospheric model NUMA
Kopera, Michal A.
Wilcox, Lucas C.
Giraldo, Francis X.
MetadataShow full item record
Numerical weather prediction (NWP) has proven to be computationally challenging due to its inherent multiscale nature. Currently, the highest resolution NWP models use a horizontal resolution of approximately 15 km. At the resolution many important processes in the atmosphere are not resolved. Needless to say this introduces errors. In order to increase the resolution of NWP models highly scalable atmospheric models are needed. The Non-hydrostatic Unified Model of the Atmosphere (NUMA), developed by the authors at the Naval Postgraduate School, was designed to achieve this purpose. NUMA is used by the Naval Research Laboratory, Monterey, as the engine inside its next generation weather prediction system NEPTUNE. NUMA solves the fully compressible Navier-Stokes equations be means of high-order Galerkin methods (both spectral element as well as discontinuous Galerkin methods can be used. Mesh generation is done using the p4est library. NUMA is capable of running middle and upper atmosphere simulations since it does not make use of the shallow-atmosphere approximation. This paper presents the performance analysis and optimization of the spectral element version of NUMA. The performance at different optimization stages is analyzed using hardware counters with the help of the Hardware Performance Monitor Toolkit as well as the PAPI library. Machine independent optimization is compared to machine specific optimization using BG/Q vector intrinsics. By using vector intrinsics the main computations reach 1.2 PFlops on the entire machine Mira. The paper also present scalability studies for two idealized test cases that are relevant for NWP applications. The atmospheric model NUMA delivers an excellent strong scaling efficiency of 99% on the entire supercomputer Mira using a mesh with 1.8 billion grid points. This allows us to run a global forecast of a baroclinic wave test case at 3 km uniform horizontal resolution and double precision within the time frame required for operational weather prediction.
Approved for public release; distribution is unlimited
Showing items related by title, author, creator and subject.
Dotson, Michael E. (1987-06);Atmospheric refractive index gradients significantly modify the path of electromagnetic (EM) waves as they propagate through the atmosphere. Accordingly, the performance of U.S. Navy (EM) systems can be wither degraded ...
In Situ Raman Spectroscopy of the Nanodiamond-to- Carbon Onion Transformation During Thermal Annealing of Detonation Nanodiamond Powder Cebik, Jonathan E. (Monterey, California. Naval Postgraduate School, 2012-06);In this study, in situ Raman spectroscopy is utilized to investigate the onset of nanodiamond (ND) graphitization, the conversion of diamond (sp3) to graphitic (sp2) carbon, and the subsequent formation of carbon onions. ...
Hoover, Cloyes R. (Monterey, California. Naval Postgraduate School, 1991-12);Gradients in the index of refraction caused by temperature fluctuations create optical turbulence. This optical turbulence greatly degrades the performance of imaging and laser systems operating in the atmosphere. There ...