Utilizing Statistical Inference to Guide Expectations and Test Structuring During Operational Testing and Evaluation

Download
Author
Brathwaite, Joy
Wallace, Alton
Holcomb, Robert
Date
2011-04-30Metadata
Show full item recordAbstract
Comparative tests are commonly used during the operational testing phase to baseline the system under test (SUT) against the current status quo. Depending on the type of SUT, the comparative test may be costly and resource intensive. Thus, any insights which may be gleaned about the potential results of the test beforehand may provide guidance on (1) the potential benefits of conducting the test and (2) the structuring of the test. This paper offers a statistical approach to understanding the type of results which may emerge during comparative testing of the SUT. Specifically, we utilize the concept of statistical inference to determine the needed performance difference between the SUT and the baseline system. If performance differences are statistically different, there may be useful information to be gained from conducting the test as is. Performance differences, which are not statistically different, may indicate that the test should be restructured or postponed. In either case, the relevant decision-maker is provided with information about the potential results of the test beforehand in order to make an informed decision. We illustrate the method of statistical inference on a system which improves situational awareness on the battlefield.
Description
Proceedings Paper (for Acquisition Research Program)
Rights
This publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.NPS Report Number
NPS-AM-11-C8P10R03-044Related items
Showing items related by title, author, creator and subject.
-
A Statistical Multimodel Ensemble Approach to Improving Long-Range Forecasting in Pakistan
Gillies, Shane D. (Monterey, California. Naval Postgraduate School, 2012-03);We have designed, developed, and tested a method for generating long-range forecasting systems for predicting environmental conditions at intraseasonal to seasonal lead times (lead times of several weeks to several seasons). ... -
Subseasonal to Seasonal (S2S) Forecasting of Tropical Cyclones
Murphree, Tom (Monterey, California: Naval Postgraduate SchoolMonterey, California. Naval Postgraduate School., 2021);Tropical cyclones (TCs) can have major impacts on planning and executing Navy and USMC operations. For operational planning, it is important to forecast TCs at subseasonal to seasonal (S2S) lead times. The existing NPS ... -
Subseasonal to Seasonal (S2S) Forecasting of Tropical Cyclones
Murphree, Tom (Monterey, California: Naval Postgraduate SchoolMonterey, California. Naval Postgraduate School., 2021);Tropical cyclones (TCs) can have major impacts on planning and executing Navy and USMC operations. For operational planning, it is important to forecast TCs at subseasonal to seasonal (S2S) lead times. The existing NPS ...