Representing Uncertainty of Hierarchical and Response Surface Models to Improve Design of Experiments
Loading...
Authors
Lucas, Thomas W.
Sanchez, Paul J.
McDonald, Mary
Upton, Steve
Pav, Russsell
Subjects
Advisors
Date of Issue
2015
Date
Period of Performance: 12/1/14 - 12/31/15
Publisher
Monterey, California. Naval Postgraduate School
Language
Abstract
The Office of the Chief of Naval Operations (OPNAV) uses a hierarchy of simulation models as part of scenario-based planning to help decide how the Navy should be equipped, organized, and employed. Simulation is used throughout the acquisition process, from platform design to force structure. In hierarchical combat modeling, the mean outputs of lower-level, higher-resolution models are typically used as inputs to higher-level, lower-resolution models. The objective of this process is to inform Navy leadership on how detailed design changes ultimately impact campaign effectiveness. Unfortunately, by ignoring variability in linkages between layers in the hierarchy, the results may bias campaign-level outcomes and/or dramatically understate the final variability (or risk) estimated by the campaign-level model. Consequently, the goal of this research was to design and run experiments to better understand the impacts on the hierarchical modeling process associated with error propagation methods and design of experiments techniques. To empirically explore a host of different error propagation approaches, this research conducted thousands of experiments using a two model hierarchical structure in an Anti- Submarine Warfare (ASW) setting. The results indicate that the sampling methodology has a significant impact on the mean probability Blue wins the campaign, as well as the mean number of losses Blue takes when it wins. In addition, the sampling methodology has a significant effect on the standard deviation for the probability Blue wins and the amount of losses Blue expects to take when it wins. Moreover, different metamodels were developed using different design of experiment techniques. While this initial effort involves just a single set of experiments, the results suggest that hierarchical combat models should adopt methods that include the entire distribution of lower-level model outcomes in order to better represent risk. More research is needed to determine how best to accomplish this.
Type
Report
Description
Naval Research Program, Executive Summary
Department
Organization
Naval Research Program
Identifiers
NPS Report Number
Sponsors
Naval Research Program
Prepared for: OPNAV N98, Mr. Victor Steinman and Mr. Chris March
Prepared for: OPNAV N98, Mr. Victor Steinman and Mr. Chris March
Funder
Format
Citation
Distribution Statement
Rights
This publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.