Statistical methods for assessing agreement for ordinal data

Evaluation of various methods in clinical practice is often based on interpretations by two or more observers. Such data need to be analysed with correct statistics, or the results and conclusions may be misleading. In this study, the use of measures of agreement for ordinal data in five international nursing journals is reviewed and various methods for measuring agreement are presented and discussed. Analyses of agreement did not seem to be very common in nursing research, but a great variation was found regarding the choice of method for analysing agreement. Both acceptable and unacceptable methods were found in the articles reviewed. When choosing among various methods for agreement the weighted kappa-coefficient is probably the most useful for ordinal data, but several issues of concern arise and need to be taken into consideration when using these types of analyses.