Validating computational human behavior models: consistency and accuracy issues

Abstract

As leaders of the Department of Defense (DoD) rely more on modeling and simulation (MandS) to provide information on which they base strategic and tactical decisions, the credibility of simulations becomes more important. This credibility is initially gained through the verification, validation, and accreditation process DoD models are required to undergo prior to their use in simulations. The process of validating behavioral models is not well defined, nor is the process extendable to meet requirements for validating the varied and complex behavioral models. Through a series of empirical studies, this research identifies subject matter expert (SME) biases and their effects on consistency and accuracy of results. This research concludes that a SME's bias has a statistically significant effect on subjective assessment of human performance of urban combat skills. To this end, the research demonstrates how the effects of the natural biases of SMEs can be mitigated based on the scale used to assess assessing human behavior representation (HBR) models, providing a more consistent and accurate means of validating HBR models. In doing so, it assists the DoD MandS Community by providing enhancements to face validation procedures for HBR model implementations for future use in DoD legacy and developmental combat models.