Generalizability of Job Performance Measurements: Marine Corps Rifleman

One important indicator of the quality of a hands-on measure of job performance is its dependability. Due to the complexity of such measurements, two concerns arise: (a) Can test examiners reliably score hands-on job performance, and (b) can the complexities of such measurements be modeled statistically to evaluate the reliability of hands-on tests? Generalizability theory (G theory) was used to model the hands-on performance of U.S. Marine Corps infantry riflemen. Results show that (a) experienced, well- trained, and calibrated examiners can dependably score hands-on performance; (b) several ways of examining the dependability of examiners' scoring produce equivalent results, thus possibly simplifying analytic procedures; (c) the major source of measurement error in job performance measurements is the heterogeneity of tasks that comprise the job; and (d) despite the heterogeneity of job tasks, reliable generalizations from performance on a sample of tasks to performance on the job can be made.