Mathematical modeling to reduce the cost of complex system testing: Characterizing test coverage to assess and improve information return

Download
Author
Pfeiffer, Karl D.
Kanevsky, Valery A.
Housel, Thomas J.
Date
2011-09-21Metadata
Show full item recordAbstract
Effective, cost-efficient testing is critical to the long-term success of Open Architecture within the Navy''s Integrated Warfare System. In previous research we have developed a simple, effective framework to examine the testing of complex systems. This model and its prototype decision aid provide a rigorous yet tractable approach to improve system testing, and to better understand and document the system and component interdependencies across the enterprise. An integral part of this model is characterizing test coverages on modules. Using idealized simulations of complex systems, we investigate the sensitivity of test selection strategy to the precision with which these coverages are specified. Monte Carlo analysis indicates that best-test selection strategies are somewhat sensitive to the precision of test coverage specification, suggesting significant impact on testing under fixed-cost constraint. These results provide significant insight as we extend this work with further study of real-world systems by applying, and refining, the mathematical analysis and computer simulation within this framework. The current decision-aid software will be further developed using these operational test and evaluation data, improving the fidelity of the current modeling while making available to program managers and system designers a usable and relevant tool for test''retest decisions.
Description
Sponsored Report (for Acquisition Research Program)
Rights
This publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.NPS Report Number
NPS-TE-11-181Related items
Showing items related by title, author, creator and subject.
-
Implementation and Design of a Novel Student Developed Modular HTOL/HTRB System Using Thermoelectric Control
O’Neal, Nathaniel J.; Porter, Matthew A.; Martino, Christopher Adrian (American Society for Engineering Education, 2021);Addressing reliability issues is critical to the successful design and implementation of new semiconductor material systems proposed for next generation power electronic devices. For military systems, reliability is central ... -
Modeling Complex System Testing: Characterizing Test Coverage to Improve Information Return
Pfeiffer, Karl; Kanevsky, Valery; Housel, Thomas (Monterey, California. Naval Postgraduate School, 2011-04-30); NPS-AM-11-C8P10R01-042Effective, cost-efficient testing is critical to the long-term success of open architecture within the Navy''s Integrated Warfare System. In previous research we have developed a simple, effective framework to examine the ... -
Reducing the cost of risk-based testing: management of testing options to manage risk in test and evaluation
Pfeiffer, Karl D.; Kanevsky, Valery A.; Housel, Thomas J. (Monterey, California. Naval Postgraduate School, 2009); NPS-GSBPP-09-027In the acquisition or management of complex systems, testing is the means by which we trade budget or schedule for information about the likelihood our system will work correctly under operational load. Branch paths in ...