COMPARISON OF DECISION POLICIES FOR OPTIMAL REPAIR-REPLACE IN INFRASTRUCTURE SYSTEMS

Abstract

Critical infrastructure systems are under threat: natural disasters, degradation, and enemy action form a recipe for disruption. Many systems rely on maladapted or incomplete models to forecast and respond to disruptions; these are especially vulnerable to the increasing severity of disruptions as well as over-reliance on artificial intelligence and machine learning (ML). Every Department of Defense installation has infrastructure systems that it deems critical, and the goal for this research is to support more adaptive, resource-aware policies for maintaining mission-critical infrastructure in times of peace and crisis. We formulate a network flow problem with uncertain edge failures and model repair-replace decisions as a Markov Decision Process (MDP). We then compare optimal decision policies derived from linear programming with approximated policies generated via Q-learning and a heuristic approach. We leverage stochastic simulation to determine the expected costs of following each policy and compare their performance. The primary output of this thesis is our demonstrated use of a method to directly compare ML and heuristic policy approximations to a known optimal solution. This technique helps elucidate when different decision models can be deployed in more complex systems where the optimal solution is computationally infeasible.