From reproducible processing to valid inference: A role-specific framework across open ECG, BVP, and EDA datasets

Reproducible pipelines make heterogeneous datasets computationally comparable, but they can also conceal that the resulting outputs support fundamentally different types of inference. This study tested that tension on three PhysioNet datasets—Fantasia (ECG, n = 40 ), Non-EEG ( n = 20 ), and an Empatica E4 wearable dataset ( n = 36 )—and found that a unified validation model broke down at each step. In Fantasia, expected age-related HRV decline was recovered at the subject level (all p < 0.001 , Cohen’s d = 1.28 – 1.36 ), but window-level pooling collapsed p -values by more than sixty orders of magnitude, showing that reproducible computation does not guarantee valid evidence. In Non-EEG, subject-separated classification achieved a pooled AUC of 0.758 (bootstrap 95% CI [ 0.698 , 0.817 ] , permutation p = 0.002 ), but this reflected protocol-based state discrimination rather than physiological verification, because the dataset lacked both RR intervals and an external autonomic benchmark. In the wearable dataset, 733 samples yielded 12 extractable features, yet frequency-domain PRV estimates showed limited availability in short windows, and PRV-RMSSD values could not be treated as numerically interchangeable with ECG-HRV without paired calibration; extractability did not establish cross-device equivalence. An alternative-architecture analysis showed that collapsing the three roles reintroduced at least one explicitly defined inferential distortion. Validation is not conferred by running the same workflow across datasets; it is a claim-specific relation among signal source, analysis unit, benchmark, and intended inference. We establish a role-specific architecture separating physiological verification, protocol-based phenotyping, and wearable measurement characterization as distinct evidential operations.