Two-Stage, Dynamic Data Envelopment Analysis of Technology Transition Performance in the U.S. Defense Sector
Loading...
Authors
Sueyoshi, Toshiyuki
Ryu, Youngbok
Subjects
Advisors
Date of Issue
2021-05-10
Date
05/10/21
Publisher
Monterey, California. Naval Postgraduate School
Language
Abstract
Both the commercial world and Department of Defense (DoD) are challenged with system safety issues when dealing with Machine Learned (ML)/Artificial Intelligence (AI) deployed products. DoD has a more severe issue when deploying weapons that could unintentionally harm groups of people and property. Commercial manufacturers are motivated by profit, while DoD is motivated by defense readiness. Both are in a race and can suffer the consequences from focusing too much on the finish line. Establishing formal oversight ensures safe algorithm performance. This paper presents a measurement approach that scrutinizes the quality and quantity of training data used when developing ML/AI algorithms. Measuring quality and quantity of training data increases confidence in how the algorithm will perform in a "realistic" operational environment. Combining modality with measurements determines: (1) how to curate data to support a realistic deployed environment; (2) what attributes take priority during training to ensure robust composition of the data; and (3) how attribute prioritization is reflected in size of the training set. The measurements provide a greater understanding of the operational environment, taking into account issues that result when missing and/or sparse data occur, as well as how data sources supply input to the algorithm during deployment.
Type
Presentation
Description
Series/Report No
Department
Organization
Identifiers
NPS Report Number
SYM-AM-21-091
Sponsors
Prepared for the Naval Postgraduate School, Monterey, CA 93943.
Naval Postgraduate School
Naval Postgraduate School
Funder
Format
Citation
Distribution Statement
Approved for public release; distribution is unlimited.
Approved for public release; distribution is unlimited.
Approved for public release; distribution is unlimited.
Rights
This publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.