Intra‐ and Inter‐rater Agreement on Magnetic Resonance Imaging Evaluation of Rotator Cuff Integrity After Repair

PURPOSE: To investigate the intra- and inter-rater agreement of magnetic resonance imaging (MRI) evaluations of rotator cuff integrity at 6 and 24 months after arthroscopic rotator cuff repair (ARCR). METHODS: Three shoulder surgeons reviewed 68 MRI scans from 34 patients who had undergone ARCR and MRI examination at both 6 and 24 months after surgery. Postoperative rotator cuff integrity was investigated by using Owen, Sugaya, and Hayashida classifications to determine whether the rotator cuff was intact or whether there was a partial-thickness retear or full-thickness retear and Burks score to assess tendon appearance. Multirater kappa statistics were used to measure intra- and inter-rater agreement. Kappa values were interpreted according to guidelines adapted from the work of Landis and Koch. RESULTS: All classifications had similar intra- and inter-rater agreement (κ = 0.14 to 0.67, 0.23 to 0.60, respectively), but no intra- or inter-rater agreement scored "excellent." Inter-rater agreement after ARCR was higher at 24 months (κ = 0.31 to 0.60) than at 6 months (κ = 0.23 to 0.44) in all evaluations. Reviewers identified full-thickness retears with a moderate to good degree of inter-rater agreement in all evaluations, at both 6 months (κ = 0.42 to 0.73) and 24 months (κ = 0.61 to 0.80) after ARCR. However, poor inter-rater agreement (κ = 0.13 to 0.19) was found in the identification of partial-thickness retears in all evaluations at 6 months after ARCR. CONCLUSIONS: Shoulder surgeons showed better intra- and inter-rater agreement in predicting full-thickness tears compared with partial-thickness tears. The inter-rater agreement at 24 months after ARCR was superior to that at 6 months in predicting not only full-thickness retear but also partial-thickness retear. MRI evaluation of rotator cuff integrity at 6 months after ARCR may be less reliable, regardless of which classification system is used. LEVEL OF EVIDENCE: Level III, retrospective comparative study.