The role of personality in determining variability in evaluation expertise

Abstract

This research investigated how different experts in a single domain chose their individual subjective evaluation criteria of a highly aggregate task based upon their individual differences. The Conning Officer Virtual Environment (COVE) was utilized to provide a domain of experts and a subjectively evaluated task. 116 expert ship-handlers were investigated to understand how their personality affects their evaluation of a novice performing an underway replenishment (UNREP). The experts were issued a survey that inventoried their personality, UNREP evaluation criteria, and ship handling style. In general, the participant experts were lower in Neuroticism and higher in Extraversion and Conscientiousness than the average adult. Extraversion appeared to be correlated with the expert's desire to use sensory input as a critical evaluation criterion (p=.18) while Openness was correlated with analytical input (p=.16) and UNREP style (p=.16) as critical evaluation factors. Also correlated with UNREP style was Agreeableness (p=.16). Finally, the expert's level of Conscientiousness correlated with the critical evaluation criteria of analytical input (p=.17) and sensory input (p=.39). Results from this research provide insight to the link between observed behavior and its subjective evaluation and will allow COVE's programmers to develop an Intelligent Tutoring System (ITS) that will customize the automated training process