MASTERING THE DIGITAL ART OF WAR: DEVELOPING INTELLIGENT COMBAT SIMULATION AGENTS FOR WARGAMING USING HIERARCHICAL REINFORCEMENT LEARNING
Loading...
Authors
Black, Scotty E.
Subjects
reinforcement learning
RL
hierarchical reinforcement learning
HRL
artificial intelligence
AI
combat simulation
abstraction
intelligent agents
RL
hierarchical reinforcement learning
HRL
artificial intelligence
AI
combat simulation
abstraction
intelligent agents
Advisors
Darken, Rudolph P.
MacKinnon, Douglas J.
Darken, Christian J.
Balogh, Imre L.
Wade, Brian, TRADOC Analysis Center Monterey
Date of Issue
2024-06
Date
Publisher
Monterey, CA; Naval Postgraduate School
Language
Abstract
In today’s rapidly evolving military landscape, advancing artificial intelligence (AI) in support of wargaming becomes essential. Despite reinforcement learning (RL) showing promise for developing intelligent agents, conventional RL faces limitations in handling the complexity inherent in combat simulations. This dissertation proposes a comprehensive approach, including targeted observation abstractions, multi-model integration, a hybrid AI framework, and an overarching hierarchical reinforcement learning (HRL) framework. Our localized observation abstraction using piecewise linear spatial decay simplifies the RL problem, enhancing computational efficiency and demonstrating superior efficacy over traditional global observation methods. Our multi-model framework combines various AI methodologies, optimizing performance while still enabling the use of diverse, specialized individual behavior models. Our hybrid AI framework synergizes RL with scripted agents, leveraging RL for high-level decisions and scripted agents for lower-level tasks, enhancing adaptability, reliability, and performance. Our HRL architecture and training framework decomposes complex problems into manageable subproblems, aligning with military decision-making structures. Although initial tests did not show improved performance, insights were gained to improve future iterations. This study underscores AI’s potential to revolutionize wargaming, emphasizing the need for continued research in this domain.
Type
Thesis
Description
Series/Report No
Department
Identifiers
NPS Report Number
Sponsors
Funder
Format
Citation
Distribution Statement
Distribution Statement A. Approved for public release: Distribution is unlimited.
Rights
This publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.