A FRAMEWORK FOR EVALUATING SIMULATION CONCEPTUAL MODEL DOCUMENTATION METHODS

Abstract

This research examines the nuances of simulation conceptual model (SCM) documentation and how it is used in the context of littoral kill webs by the Marine Corps. A brief survey of current SCM documentation methods and interviews from modeling and simulation subject matter experts were used to create a framework for evaluating the documentation of an SCM. The evaluation framework consisted of five evaluation criteria: accuracy, usability, modifiability, reusability, and interoperability. For a given SCM, the degree to which each criterion is exhibited within the documentation is evaluated on a scale from one to five, with one being little to no evidence of exhibition and a five indicating a great degree of presence within the SCM documentation. The framework was then applied to two SCMs within the littoral fires domain, each utilizing different documentation methods, in order to test the viability of the framework. The result of the application of the evaluation framework was a series of scores produced for each documentation method. The importance of these metrics is that they provide an element of increased objectivity for the modeler when deciding which documentation method to use for a given SCM. Additionally, the test cases highlighted some important gaps within the framework and provided suggestions for future work to address these shortfalls.