FA-ATF: Leakage-Safe Factorial-Aware Multi-Output Learning for Small Hierarchical Experimental Data

Machine learning on small designed experiments is prone to optimistic evaluation when several observations share one physical unit and are split row-wise between training and test sets. We study this problem on a small hierarchical factorial dataset—81 section-level observations from 27 specimens of hard-turned 51CrV4 steel—with two physically different targets: surface roughness (Ra) and post-process hardness (HV30). We propose (i) a leakage-safe evaluation protocol based on specimen-grouped nested cross-validation (nine outer, four inner folds) with fold-local preprocessing, and (ii) Factorial-Aware Asymmetric Target Fusion (FA-ATF), a multi-output learner that encodes the specimen–section hierarchy through a factorial line-profile backbone, uses target-specific backbones, and admits additional experts and residual correction only when inner-fold gains are stable. Under the same protocol, FA-ATF was compared with 17 models spanning regularized regression, mixed-effects, kernel, ensemble, boosting, Gaussian process, and neural approaches. FA-ATF yielded the best observed performance (mean target-wise normalized root-mean-square error (NRMSE) 0.3773; macro-R2 0.8542) and ranked first in 83% of specimen-cluster bootstrap resamples, and ablation attributed almost all of the gain to the factorial line-profile backbone, which is the core contribution, rather than to the additional components. Multiplicity-corrected paired tests did not establish statistically significant superiority over the strongest comparators, and several differences were too small to matter in practice, illustrating how grouped validation tempers performance claims on small data. Code, fold specifications, and out-of-fold predictions are openly available.