A text analysis of the Marine Corps fitness report

Abstract

The Marine Corps loses about half of its nearly two thousand officers at the end of their initial contracts for various reasons. In an effort to control talent retention, the Marine Corps is examining if the appropriate evaluation structure is in place to identify the top performers. This study is an analysis of textual information contained in fitness reports to determine the extent to which it informs promotion boards of the quality of a Marine officer. We examine 71,212 observed fitness reports from the 1996, 1997, 2006, and 2007 officer cohorts, which we observe from 2007 to 2016. We use text statistics, readability indicators, natural language processing, and a variety of statistical machine learning algorithms to predict the top and bottom performers. We find that fitness reports for the best-performing officers are well written, use simple words in longer sentences, and comment on future command opportunities. Remarks on performance, potential, billet assignment, and education do not contribute predictive power. The fitness report contributors often disagree and informative power is lost when the assigned marks do not conform to issued guidance. In isolation, the comment sections are inconclusive for predicting an officer's performance tier. We attain a correct classification rate of 67% when using an optimized ensemble of prediction models. We recommend that the Marine Corps provide word-picture guidance to distinguish talented Marines and promote conformity in issuing quantitative assessments of performance.