Interrater Reliability of Manual Pavement Distress Evaluations

Many government agencies are faced with the challenge of pavement evaluation and maintenance as part of their pavement management systems. These agencies perform or contract manual or automated distress surveys over the pavement network to monitor the structure performance and obtain the necessary data to calculate pavement condition indexes. Although manual distress ratings are done according to well-defined criteria, a certain amount of subjectivity and the experience of the raters have an undoubted influence on the ratings. This study proposes a new approach to estimate the interrater or intercrew reliability for manual or semiautomated distress data collection. The proposed analysis acknowledges that a certain degree of variability in the visual distress ratings is likely to occur and, thus, minimum acceptable values of complete and partial agreements of the crews or raters are suggested. The statistical approach to validate the level of agreement between the ratings of two raters or crews is based on the use of the chi-square distribution to test hypotheses about multinomial experiments.