Aspects of rating communicative interaction

Communication partner training can be effective in improving communication in aphasia. However, further research is needed on how to measure the outcome of such interventions. In this paper we discuss the phenomenon of reliability in assessments in relation to the results of analyses using a rating scale designed to measure the ability to support a person with aphasia in natural conversational interaction. The scale was used by four assessors to rate 45 video recordings. Calculations of reliability and agreement produced varying results but were mostly satisfactory. However, the results highlight how interaction between factors such as complexity of assessments; design of the rating scale; factors inherent in the individual assessor; and the statistical measures used to analyse the outcome may result in a conflict between aspects of validity and reliability. Interpretations of outcome obtained with rating scales thus need to be based on knowledge about factors influencing the results.