Performance Evaluation Trait Validation
Loading...
Authors
Helzer, Erik
Subjects
performance evaluation system
performance assessment
traits
job performance
measurement validity
reporting senior’s cumulative average (RSCA)
performance assessment
traits
job performance
measurement validity
reporting senior’s cumulative average (RSCA)
Advisors
Date of Issue
2022-10-28
Date
Publisher
Monterey, California: Naval Postgraduate School
Language
Abstract
We present the results of a year-long study supporting the Navy’s efforts to transform its existing Performance Evaluation System (PES) with an evaluation tool that has been updated to reflect current Navy values and priorities. Our team validated and refined trait and value statements (TVS) developed for rating Sailors on a variety of dimensions, including leadership, teamwork, communication, resilience, and other character strengths. In addition, our team developed rating scales for assessing Sailors’ future potential for performance at the next paygrade, as well as a comparative assessment tool for evaluating Sailors against their peers. We then conducted a largescale prototyping study, which involved the development of a performance evaluation prototype instrument, recruitment of Sailors-reported-on and performance raters, and statistical analysis of the properties of the prototype instrument—in particular, the measurement validity of the TVS. Using data from 606 performance raters, we find that ratings of Sailors based on past performance and future potential predict comparative assessments, with some traits (e.g., leadership) better distinguishing the top from middle performers. Few of the trait ratings distinguished among bottom performers, although such performers were relatively rare in our sample. Ratings based on past performance and future potential were correlated with one another; analyses indicate they convey unique information about the Sailor-reported-on, but that both predicted workplace behaviors. Based on our research, we offer four concrete recommendations for action. First, we recommend amplifying the comparative assessment in future PES. Second, the Navy should consider adopting ratings of future potential for developmental/coaching purposes using actionable, concrete developmental feedback. Third, future evaluation instruments should focus on a subset of performance and/or future potential traits in assessing job performance to keep PES simple and useful. Finally, we recommend assessing the predictive validity of these new measures using multisource data.
Type
Report
Description
NPS NRP Executive Summary
Series/Report No
Department
Organization
Identifiers
NPS Report Number
Sponsors
N1 - Manpower, Personnel, Training & Education
Navy Personnel Command
Navy Personnel Command
Funder
This research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098).
Format
Citation
Distribution Statement
Approved for public release. Distribution is unlimited.
Rights
This publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.