Solving reward-collecting problems with UAVs: a comparison of online optimization and Q-learning
Loading...
Authors
Liu, Yixuan
Vogiatzis, Chrysafis
Yoshida, Ruriko
Morman, Erich
Subjects
uncrewed autonomous vehicles
random adversaries
reinforcement learning
Deep Q-Learning
online optimization
random adversaries
reinforcement learning
Deep Q-Learning
online optimization
Advisors
Date of Issue
2021-11-30
Date
30 November 2021
Publisher
ArXiv
Language
Abstract
Uncrewed autonomous vehicles (UAVs) have made significant contributions to reconnaissance and surveillance missions in past US military campaigns. As the prevalence of UAVs increases, there has also been improvements in counter-UAV technology that makes it difficult for them to successfully obtain valuable intelligence within an area of interest. Hence, it has become important that modern UAVs can accomplish their missions while maximizing their chances of survival. In this work, we specifically study the problem of identifying a short path from a designated start to a goal, while collecting all rewards and avoiding adversaries that move randomly on the grid. We also provide a possible application of the framework in a military setting, that of autonomous casualty evacuation. We present a comparison of three methods to solve this problem: namely we implement a Deep Q-Learning model, an ε-greedy tabular Q-Learning model, and an online optimization framework. Our computational experiments, designed using simple grid-world environments with random adversaries showcase how these approaches work and compare them in terms of performance, accuracy, and computational time.
Type
Preprint
Description
Series/Report No
Department
Operations Research (OR)
Organization
Identifiers
NPS Report Number
Sponsors
R.Y. is partially supported by NSF DMS 1916037 and Consortium for Robotics and Unmanned
Systems Education and Research (CRUSER).
Funder
NSF DMS 1916037
Consortium for Robotics and Unmanned Systems Education and Research (CRUSER)
Consortium for Robotics and Unmanned Systems Education and Research (CRUSER)
Format
24 p.
Citation
Liu, Yixuan, et al. "Solving reward-collecting problems with UAVs: a comparison of online optimization and Q-learning." Journal of Intelligent & Robotic Systems 104.2 (2022): 1-14.
Distribution Statement
Rights
This publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.