Understanding, Assessing, and Mitigating Safety Risks in Artificial Intelligence Systems
Loading...
Authors
Kroll, Joshua A.
Berzins, Valdis
Subjects
Artificial Intelligence
Safety, Assessment
Evaluation
MIL-STD 882
Safety, Assessment
Evaluation
MIL-STD 882
Advisors
Date of Issue
2022-12
Date
December 2022
Publisher
Monterey, California. Naval Postgraduate School
Language
Abstract
Traditional software safety techniques rely on validating software against a deductively defined specification of how the software should behave in particular
situations. In the case of AI systems, specifications are often implicit or inductively defined. Data-driven methods are subject to sampling error since practical
datasets cannot provide exhaustive coverage of all possible events in a real physical environment. Traditional software verification and validation approaches may
not apply directly to these novel systems, complicating the operation of systems safety analysis (such as implemented in MIL-STD 882). However, AI offers
advanced capabilities, and it is desirable to ensure the safety of systems that rely on these capabilities. When AI tech is deployed in a weapon system, robot, or
planning system, unwanted events are possible. Several techniques can support the evaluation process for understanding the nature and likelihood of unwanted
events in AI systems and making risk decisions on naval employment. This research considers the state of the art, evaluating which ones are most likely to be
employable, usable, and correct. Techniques include software analysis, simulation environments, and mathematical determinations.
Type
Technical Report
Description
Prepared for: Naval Air Warfare Development Center (NAVAIR)
Series/Report No
Department
Computer Science (CS)
Organization
Naval Postgraduate School
Identifiers
NPS Report Number
NPS-CS-22-003
Sponsors
Naval Air Warfare Development Center
Funder
Naval Postgraduate School, Naval Research Program (PE 0605853N/2098)
Format
67 p.
Citation
Distribution Statement
Approved for public release. Distribution is unlimited
Rights
This publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.