An inter-rater comparison of DoD Human Factors Analysis and Classification System (HFACS) and Human Factors Analysis and Classification System Maritime (HFACS-M)

Abstract

Human error has been identified as a factor in virtually every major maritime mishap over the past decade. The Department of Defense (DoD) currently employs the Human Factors Analysis and Classification System (HFACS) taxonomy to identify and quantify human error in major mishaps. HFACS divides errors into categories, sub-codes, and nano-codes. The generic nature of DoD HFACS raises the question of whether or not a domain-specific version for the surface Navy could be applied more consistently. Twenty-eight subjects (14 Surface Warfare Officers (SWOs) and 14 non-SWOs) employed either DoD HFACS or a developmental maritime domain specific version, HFACS-M, to classify findings in a National Transportation Safety Board (NTSB) maritime accident investigation. Fleiss Kappa was used to determine inter-rater reliability among subjects. The results of this study revealed that SWOs using HFACS-M had a higher inter-rater reliability (10.9%, 7.3%, and 6.5%) at every classification level than non-SWOs. HFACS-M itself was also shown to have a slightly higher overall inter-rater reliability (5.7%, 7.4%, and 3.6%) than DoD HFACS. The research concluded that although HFACS-M performed well, further testing is necessary to validate it.