The use of statistical measures to validate system dynamics models

Abstract

System dynamics modelers have often been criticized for the informal methods of model validation and for not using more formal, quantifiable measures to lend confidence to the validation process. Numerous proponents of the system dynamics approach have highlighted this shortcoming, however, and have suggested a variety of appropriate statistical measures which could be used in the model validation process. The objective of this thesis is to complement earlier validation efforts of the Abdel-Hamid and Madnick System Dynamics Model of Software Development by submitting the model to a battery of appropriate statistical measures. The model is evaluated with statistics which have been used by others in the system dynamics field. The evaluation makes two different comparisons. First, an evaluative comparison is made between data generated by the model and actual data of two real software projects. Then, an evaluative comparison is made between model generated data and data obtained by direct experimentation for two different experiments, using the model's gaming interface. The two evaluations serve to promote confidence in the model.