Abstract Scientific regression tasks in materials science, chemistry, and thermodynamics often involve monotonic relationships arising from fundamental physical principles. While these relationships are known from domain theory, existing machine learning methods require manual specification or fail to leverage them effectively. We introduce Ordinality-Aware Gradient Boosting (OA-GBM), which automatically discovers and enforces such physics-informed constraints through false discovery rate-controlled statistical testing and continuous confidence weighting. Unlike binary constraint approaches in XGBoost and LightGBM, OA-GBM employs soft constraints scaled proportionally to statistical evidence, enabling graceful degradation for uncertain relationships. Our two-stage architecture separates discovery from enforcement: Stage 1 identifies significant monotonic features using Spearman correlation with Benjamini-Hochberg correction; Stage 2 integrates these constraints into gradient boosting via modified residual gradients that penalize violations proportionally to confidence. We establish theoretical properties including bounded weights, zero-penalty for monotonic models, and backward compatibility with standard gradient boosting. Comprehensive experiments across eight scientific regression tasks demonstrate that OA-GBM outperforms baseline methods, including specialised deep learning architectures and manually-constrained gradient boosting.
Asela Hevapathige (Tue,) studied this question.