An evaluation of the construct validity of the command safety assessment survey

Abstract

The purpose of this study was to evaluate the construct validity of the 61-item command safety assessment survey (CSAS) using the responses of 110,014 U.S. Naval aircrew. Utilizing a combination of exploratory and confirmatory factor analysis, we were unable to identify a stable factor structure from the CSAS data. We believe that this finding was because of the effect of the nonconstant variance of the data, which was due to the large proportion of participants using a satsificing strategy (respondents interpret each question superficially and select what they believe to be a reasonable answer). Fortunately, since 2006, the amount of time taken by respondents to complete the survey was collected. This time to complete data was then used as a metric to identify the respondents utilizing an optimizing strategy (respondents generating the optimum answer). A total of 2,344 responses were retained for analysis. We also elected to discard the CSAS items that had low variability. Using the truncated dataset, we carried out an exploratory and confirmatory factor analysis and were able to establish a stable, 12-item, two-factor (named personnel leadership, and integration of safety and operations) model. Based on the analysis, recommendations for improving the CSAS were made.