Investigation of Leading Indicators for Systems Engineering Effectiveness in Model-Centric Programs

Abstract

This technical report summarizes the research conducted by Massachusetts Institute of Technology under contract award HQ0034-19-1-0002 during July 22, 2019 – August 31, 2021. Involved research team members include: Dr. Donna H. Rhodes, Principal Investigator; Dr. Eric Rebentisch, Research Associate; and Mr. Allen Moulton, Research Scientist. Systems engineering practice is evolving under the digital engineering paradigm, including use of model-based systems engineering and newer approaches such as agile. This drives a need to re-examine the existing use of metrics and leading indicators. Early engineering metrics were primarily lagging measures, whereas more recent leading indicators draw on trend information to provide more predictive analysis of technical and programmatic performance of the engineering effort. The existing systems engineering leading indicators were developed under the assumption of paper-based (traditional) systems engineering practice. This research investigates the model-based implications relevant to the existing leading indicators. It aims to support program leaders, transitioning to model-based engineering on their programs, in continued use of leading indicators. It provides guiding insights for how current leading indicators can be adapted for model-based engineering. The study elicited knowledge from subject matter experts and performed literature review in identifying these implications. An illustrative case was used to investigate how four leading indicators could be generated directly from a model-based toolset. Several recommendations for future research are proposed extending from the study. A companion research study (“phase 2”) under contract HQ0034-20-1-0008 provides insights for the art of the possible for future systems engineering leading indicators and their use in decision-making on model-centric programs. For completeness, selected background information and illustrative case are included in the technical reports in both studies. This research aims to provide insights for current practice within programs transforming to digital engineering, for continued use of systems engineering leading indicators. Several recommendations for future research are proposed extending from results of the study.