Development and reliability of a job analysis technique

This paper discusses reliability issues associated with the development of a job analysis technique—the Job Structure Profile. Inter‐rater reliability, measured by intraclass correlation, and retest reliability are calculated from the job elements of a sample of nine jobs. There is no difference in inter‐rater reliability between supervisors and job incumbents which is approximately 0.9. The effect of different mixes of incumbents and supervisors on inter‐rater reliability was calculated and satisfactory levels are achieved with 4***5 raters. The average retest reliability for job elements is 0.76 with no difference in retest reliability between supervisors and incumbents. These findings are compared with those from other studies and practical issues are discussed.