Development and validation of a vacant lot care condition instrument for longitudinal evaluation of place-based interventions

BACKGROUND: Remediating vacant lots is associated with reductions in neighborhood violence and other adverse health outcomes, potentially via increasing visible cues-to-care that signal stewardship. However, accurately measuring these cues over time remains challenging, limiting our ability to measure the sustained effects of lot remediation. METHODS: We developed a standardized audit protocol using subject-matter and community experts and assessed vacant lot cues-to-care from Google Street View imagery of vacant lots in Philadelphia from 2007-2023. We fit a two-parameter item response theory model to combine the observable cues-to-care measured consistently into a single latent score representing the lot care condition. We validated the latent score by testing its sensitivity to a randomized controlled greening trial and a local lot maintenance program. RESULTS: Five raters audited 3419 images (518 vacant lots; 6315 ratings) captured from July 2007-October 2023. We selected reliable and conceptually aligned items to fit an item response theory (IRT) model representing a latent score of lot care condition and evaluated validity of that score. Average pairwise Cohen's kappa for audited items was 0.31 (SD=0.14) across all raters, and 0.55 (SD=0.18) among our most concordant raters. Eight items with above-moderate reliability (K>0.40) were retained for the final IRT model, which demonstrated high internal reliability (0.96). This IRT-derived care score was sensitive to improvements in lot conditions following greening interventions and enrollment in a lot maintenance program. CONCLUSION: Our IRT-derived score can be used to measure vacant lot care condition accurately over time. This can support long-term evaluation of vacant lot remediation interventions.