An analysis of learning algorithms in complex stochastic environments

Abstract

As the military continues to expand its use of intelligent agents in a variety of operational aspects, event prediction and learning algorithms are becoming more and more important. In this paper, we conduct a detailed analysis of two such algorithms: Variable Order Markov and Look-Up Table models. Each model employs different parameters for prediction, and this study attempts to determine which model is more accurate in its prediction and why. We find the models contrast in that the Variable Order Markov Model increases its average prediction probability, our primary performance measure, with increased maximum model order, while the Look-Up Table Model decreases average prediction probability with increased recency time threshold. In addition, statistical tests of results of each model indicate a consistency in each model's prediction capabilities, and most of the variation in the results could be explained by model parameters.