Two ways to manufacture a finding: the room, and the corpus

A clinical finding in acoustic voice analysis can be produced without anybody doing anything wrong. This paper measures two routes, both on real recordings, and proposes one number that would close them. THE ROOM. Using an electroglottographic channel as an independent laryngeal truth -- it follows vocal fold contact through the neck and never passes through the air -- we built two groups of 346 recordings each, matched cell by cell on true jitter and on pitch, and split by nothing but the noise of the recording. The true difference between them is +0.023 percentage points and is not significant (p = 0.68). The reported difference is +0.267 percentage points at p = 1.4 x 10^-15, Cohen's d = 0.630, produced by a 7.4 dB gap in harmonics-to-noise ratio. The transmission coefficient is 0.0473 percentage points per decibel (95 % CI 0.0438-0.0508). AT A REALISTIC SAMPLE SIZE. Thirty per group -- an ordinary clinical study -- needs a 2.55 dB room difference to manufacture a significant result. The median disagreement between three defensible ways of computing harmonics-to-noise ratio on the SAME recordings is 4.36 dB. The gap required to separate two groups is smaller than the gap you can create by changing your mind about the software. THE SAME MECHANISM HIDES A REAL DIFFERENCE. Two groups differing genuinely in true jitter, matched on the recording, transmit 30 % of that difference to the printed number. The room inflates a difference that is not there; the estimator suppresses one that is; and the two faults cover for each other. THE CORPUS. In the open corpus this field uses most, the healthy speakers have a median age of 22.8 years and the pathological speakers 53.0. Age alone separates the groups at AUC 0.871 with no acoustic computation whatsoever. The recording date separates them at 0.762 -- the controls were collected first and the patients later, from 0 % pathological in 1997 to 96 % in 2004 -- and age and date together reach 0.908 under speaker-disjoint cross-validation. Sex, as a control, reaches 0.504. A model that has never heard a voice reaches 0.908. WHAT A CLASSIFIER LEARNS. Given the matched groups and a standard 52-feature set, a logistic classifier separates them at AUC 0.717 when the recording gap is 3.15 dB and 0.909 when it is 9.31 dB, while the true jitter gap is indistinguishable from zero throughout. Two controls fix the interpretation: with the labels shuffled the AUC is 0.500, and with the groups matched on the recording as well as the larynx it is 0.478. Everything the model found was the room. THE PROPOSAL. Every paper of this kind should report one number: the AUC of a classifier trained to predict the recording session, or the recording site, from the same feature set. If it is near chance the paper has answered its most serious objection before anybody raised it. A DATED PREDICTION is stated in section 6, testable by anybody: a machine learning result separating two recording environments for the same speakers will lose that separation, or most of it, when the ambient noise floor of each recording is entered as a covariate. We make it because we tested it on our own spaceflight data and withdrew a finding as a result. No group and no paper is named. The target is a practice that reasonable people arrive at by making reasonable decisions. Scripts and the per-recording derived tables are included; session metadata carries age in years rather than dates of birth. The recordings belong to the Saarbruecken Voice Database and are not redistributed.