A quantitative evaluation method for the applicability domain of aerodynamic/aerothermal machine learning prediction models

With the continuous expansion of Machine Learning (ML) modeling methods in the aerospace field, ML prediction models face challenges in priori evaluation of prediction errors for new samples and quantitative characterization of their applicability domains. To address this, this paper proposes an adaptive Feature Space Distance measurement method based on parameter Optimization and sample Orthogonality metric (OO-FSDI), for quantitatively evaluating the applicability domain of ML prediction models. First, initial feature space distances are calculated using orthogonality and K-nearest neighbor distance between the predicted samples and training samples in a weighted feature space. Subsequently, the relationship between the feature-space distances and prediction errors is established using least squares fitting. Then, parameters such as feature weights are optimized to adapt the OO-FSDI to different models. Finally, based on this relationship, the threshold of feature-space distance indicating model applicability domain is determined according to the specified error upper limit. Validation was conducted using publicly available aerodynamic/aerothermal prediction models built with various ML algorithms. The results demonstrate that: OO-FSDI achieves quantitative delineation of model applicability domain, and it provides accurate estimation of prediction errors for samples without true values, with estimation errors under 10%. This method provides a reliable applicability assessment tool for the engineering application of ML-based aerodynamic / aerothermal prediction models.