From Model Proliferation to Deployment Intelligence: A Risk-Traceable System-Aware Model Selection Framework

The rapid growth of machine-learning architectures has shifted an important engineering challenge from model development to selecting the most suitable trained candidate for a specific deployment context. This study proposes the System-Aware Model Selection (SMS) Framework, implemented through the Deployment-Aware Model Selection Algorithm (DAMSA), as a three-layer post-training decision architecture integrating risk-to-criterion mapping, candidate control, reference ranking, and robustness analysis. Across five engineering scenarios, DAMSA-SMS was compared with benchmark-oriented and conventional decision procedures under common candidate evidence and priority assumptions. Conventional multi-criteria procedures generally agreed on the nominal reference winner, but DAMSA-SMS revealed different levels of decision robustness. YOLOv7 was selected for traffic monitoring and pharmaceutical inspection with first-rank acceptabilities of 92.8% and 91.2%, respectively. In defense surveillance, YOLOv8s was preferred to the highest-mAP YOLOv11s because its Precision was 18.9 percentage points higher. YOLOv8s was Pareto-unique and preference-stable in microplastics detection. In CCPP monitoring, Random Forest was preferred under the Testing-Generalization profile, with the decision switching below a 12.99% Generalization Stability weight. Removing F1-score did not change any vision recommendation. These results show that nominal ranking agreement does not necessarily imply deployment robustness.