From Benchmark Accuracy to Industrial Evidence: A Registered Systematic Map and Nested Experiment-Level Audit of Validation Credibility in AI-Based Rotating-Machinery Fault Diagnosis

Artificial intelligence (AI) methods for rotating-machinery diagnosis based on vibration and acoustic signals are often evaluated on benchmark splits that do not establish physical independence or industrial generalization. This registered study combined a PRISMA-aligned systematic map with a nested experiment-level audit of validation credibility. Searches identified 11,916 records, retained 7520 unique records, and placed 5159 reports in the full-text population. The operational 500-record Batch 005 block produced a non-probability 351-report full-text queue. A post-review eligibility-consistency check yielded 97 included studies, 13 exclusions, 241 unresolved reports, and 157 validation experiments. Within this access-influenced subset, 130 experiments (82.8%) remained at A0–A1 physical independence, 27 (17.2%) reached A2–A4, and 41 (26.1%) evaluated B2–B4 domain shifts. Under a narrower leakage definition, positive evidence supported High risk in 19 experiments (12.1%), whereas 93 (59.2%) were Unclear because reporting was insufficient. Seventy-seven experiments reported classification-like numeric headline scores, but metric types were heterogeneous and were stratified rather than pooled. Metadata diagnostics confirmed source-structured selection and access-related differences between resolved and unresolved reports. The Industrial Evidence Matrix provides a structured vocabulary for validation evidence; all proportions describe the nested audit, not the complete literature.