Accurately measuring changes in core autism symptoms following early intervention is challenging. The Brief Observation of Social Communication Change (BOSCC) is a promising tool for assessing social interactions in preschoolers with autism spectrum disorder (ASD). However, data on its responsiveness remain limited, warranting validation to support its use in evaluating treatment-related changes. This study aimed to assess the reliability, validity, and sensitivity to change of the BOSCC based on international recommendations from the COSMIN expert group. The BOSCC was rated using 414 video observations from a large multicenter randomized controlled trial including 177 preschoolers with ASD between 19 and 36 months. Videos were coded using the original BOSCC protocol by trained, blinded raters. Analyses addressed reliability, structural and convergent validity, and responsiveness over a 2-year follow-up. Interrater, intrarater, and test–retest reliability were consistently high, with good internal consistency (Cronbach’s α = 0.88–0.98). Factor analysis supported a three-factor structure. Convergent validity was modest with Autism Diagnostic Observation Scale (ADOS) change scores (r = 0.05–0.20) but stronger with global ratings of improvement on the Clinical Global Impression–Improvement (CGI-I; r = 0.40–0.60, p < 0.001). ROC analyses confirmed acceptable to good responsiveness when anchored to the CGI-I, but poor discrimination relative to the ADOS. The BOSCC is a reliable and responsive measure of change in preschoolers with ASD. Its naturalistic format supports its use as an outcome measure in early intervention trials. Establishing thresholds for clinically meaningful change will be a critical next step for both research and clinical practice. Lay Abstract Early intervention may improve social interaction and communication in young children with autism spectrum disorder (ASD). However, it is often difficult to measure changes in core autism symptoms over time. Many commonly used assessment tools were developed for diagnosis and are not always sensitive to treatment-related change. The Brief Observation of Social Communication Change (BOSCC) was specifically designed to address this gap by using short, naturalistic observations of children’s social communication. This study examined how well the BOSCC works as a tool to capture change. We assessed its reliability (whether it gives consistent results), validity (whether it measures what it is intended to measure), and responsiveness (whether it can detect change over time). The study included 414 video observations from a large multicenter randomized controlled trial involving 177 preschool children with ASD, aged 19–36 months, followed up for over a 2-year period. Videos were rated by trained, independent observers using the original BOSCC coding system. Results showed that BOSCC is a highly reliable measure, with strong agreement between different raters and good consistency over time. The structure of the scale was supported, and BOSCC scores were meaningfully related to clinicians’ overall judgments of improvement. In contrast, changes in BOSCC scores were less closely related to changes measured by standard diagnostic tools. Overall, these findings support the BOSCC as a useful and sensitive outcome measure for evaluating change in early autism interventions. Future research should define thresholds for clinically meaningful change to strengthen its use in research, clinical practice, and service evaluation.
Jay et al. (Mon,) studied this question.