Measurement and Analysis of Officer of the Deck Competency: New Findings for FY19-20

Abstract

In response to several high-profile ship collisions in 2017, the Surface Warfare Officers School (SWSCCOLCOM, commonly known as SWSC) implemented a program to assess the proficiency of first-tour U.S. Naval Officers of the Deck (OODs). The program has three components: a simulator exercise assessed by a post-command officer, written exams of rules of the road and seamanship knowledge, and a self-reported survey of OOD’s operational experience and background. In a continuation of our study of the first round of data collected in 2018, SWSC asked us to analyze the statistical relationship between proficiency, knowledge, and experience from data collected in 2019. They also asked us to make recommendations for how future assessment data can be collected and analyzed in order to inform optimal training and watchstanding policies. The 2019 data contains a random sample of 66 OODs who were assessed at the end of their first tour. The experience survey revealed large variation in OODs’ operational experience, partly stemming from significant variation in the time spent underway. For example, while the median first-tour OOD had 200 hours of experience, OODs in the 5th and 95th percentiles of the distribution had 18 and 855 hours of experience, respectively. 10% of experience was gained in a simulator, and most OODs had no watchstanding experience in the past 90 days at the time of assessment. Assessment scores were normally distributed around “average” proficiency and had an almost identical distribution as scores from the 2018 data collection round. Knowledge is positively correlated with assessed proficiency, but we found no correlation between experience and proficiency. Ultimately, the small sample size precludes our ability to make precise recommendations about optimal training policies. However, the Surface Community has developed a plan to assess all OODs at multiple points during their careers starting in 2021 and the resulting data, if consistently collected and stored correctly, will facilitate data-driven policy decisions concerning training, proficiency remediation, and officer detailing.