Efficient Reliability Analysis: Debiasing Machine Learning with Partial Data

In safety-critical applications, ensuring high reliability is essential for operational success and regulatory compliance. Reliability analysis is crucial for evaluating system performance, yet precise event analysis remains a challenge due to multiple parameters and significant computational demands. To reduce these costs, researchers have increasingly utilized machine learning models as substitutes for expensive physical experiments. The accuracy of these models depends heavily on the quality of training samples. This paper focuses on the Gaussian process-based model, which has gained attention in reliability analysis and real-world applications. Gaussian process (GP) model has gained increasing attention in both reliability analysis and real-world applications. Constructing GP-based models usually requires fully observed data, but many engineering scenarios suffer from missing values due to unobserved inputs, unreliable sensors, and data collection errors. While integrating partially observed datasets could expand the dataset size, naive imputation may introduce bias. Though data augmentation techniques can help, bias is unavoidable, causing tradeoffs in prediction accuracy. To address these challenges, this paper introduces a hierarchical Bayesian Gaussian process model for partial data (HBGP-PD) framework designed to minimize bias. This framework leverages partially observed data for model training and quantifies the uncertainties arising from the heterogeneous partial data. This technique is particularly valuable for complex real-world problems where data comes from multiple sources and exhibits diverse formats and structures. We provide theoretical guarantees for our approach, enabling inference and construction of confidence intervals. This is crucial for efficient reliability estimation, offering insights into the precision and uncertainty of the estimate. Supported by solid theoretical proof, we evaluate HBGP-PD through numerical experiments, demonstrating its superior performance. In conclusion, the HBGP-PD model provides an effective solution for developing high- fidelity predictive models from partially observed datasets, while also enabling uncertainty propagation and quantification. It is particularly well-suited for high-dimensional, complex systems where fully observed data is limited, but low-fidelity or noisy data is more accessible. However, as the sparsity of partial data increases, the model's performance may decline due to the introduction of greater uncertainty. Therefore, the model's accuracy is best maintained when there is a balanced mix of fully observed and partial data.