Beyond Methods: An Empirical Study of File-Level Production-Test Co-evolution and Its Implications for Automation

The co-evolution of production and test code is a critical yet challenging software maintenance activity. However, a persistent paradigm gap exists in the literature. While the identification of co-evolution is advancing to a realistic file-level scope, the automation of test updates predominantly remains focused on an oversimplified method-level view. To bridge this gap, we conduct the first multi-project, empirically grounded taxonomy of file-level change propagation patterns that also quantifies the automation paradigm gap using representative tools. We construct a dataset of over 300,000 co-evolution pairs from 128 Java projects and perform a qualitative analysis on a statistically justified sample set. Our analysis yields a comprehensive, hierarchical taxonomy of four high-level Change Propagation Patterns that explains the mechanisms of file-level co-evolution. It reveals that simple one-to-one method changes—the primary target of current tools—constitute only a minority (2.15%) of cases. Our quantitative analysis then measures the profound impact of this gap. We show that the operational scope of state-of-the-art, method-level automation tools covers only a fraction of these real-world patterns, resulting in a performance mismatch of up to 69% when applied to the broader file-level context. Our work provides both a new theoretical lens and a challenging public benchmark to steer the community towards building the next generation of automation tools that address how co-evolution happens in practice.