Distinguishing programmatic change from data artifacts in hospital certification records: a change‑point and leakage-audit approach in Thailand

BACKGROUND: Routine Health Information Systems (RHIS) produce administrative data where apparent shifts may reflect recording practices rather than genuine program changes. Methods appearing to support such shifts can share hidden information with them, a reproducibility concern in machine‑learning research. OBJECTIVE: This study aims to illustrate a leakage pattern that can produce misleading corroboration in change‑point and clustering analyses of routine health data and to propose a protocol to reduce such circularity. This is a preliminary, proof-of-concept validation, not a full operating-characteristic evaluation. METHODS: Certification records from 86 hospitals in a regional environmental‑certification program in Thailand were analysed. Change‑point detection on a recorded outcome field was assessed against three checks: an alternative construct, an alternative definition, and clustering restricted to timing-unrelated features. The protocol was also tested on synthetic data with a known, non‑leaking discontinuity. RESULTS: A discontinuity was detected in the recorded outcome field, consistent across two change‑point algorithms. It was not supported by the alternative construct or definition and was not reproduced by leakage‑free clustering. An initial clustering analysis appeared to corroborate it but later shared timing information. All three checks were consistent with the known discontinuity in the synthetic dataset. CONCLUSION: The discontinuity is more consistent with a change in recording or definitional practice than with a genuine program effect, though the data cannot confirm the mechanism. This preliminary protocol may give analysts a practical, low‑cost safeguard against reporting such shifts as genuine change without further checks. It could be automated as a first-pass screen pending further validation.