The urgent need for accurate credit risk models often clashes with the practical reality of operating with small, imbalanced datasets, where standard deep learning configurations can be inefficient. This research addresses the critical gap in optimizing models for such common yet understudied scenarios. We conduct a rigorous empirical study using a Multi-Layer Perceptron on a 381-instance mortgage dataset, systematically comparing the performance of a quasi-Newton optimizer (L-BFGS) against first-order methods (Adam, SGD) across four distinct activation functions. The methodology is grounded in the theory that L-BFGS is better suited for deterministic, full-batch optimization. The results show that while the Adam optimizer with a Logistic function achieved the highest F1-Score (0.8966), the L-BFGS optimizer produced a highly competitive F1-Score (0.8849). Critically, L-BFGS converged in just 55 iterations compared to Adam's 1556, validating its theoretical efficiency for this problem scale and revealing a significant performance-versus-cost trade-off. This study concludes that L-BFGS provides a superior practical solution for small-scale credit risk assessment, offering a near-optimal balance between high predictive accuracy and exceptional computational efficiency, especially in resource-constrained environments where rapid convergence is essential and model interpretability, robustness, and reproducibility are equally prioritized for deployment.
Ngoc Thien Nguyen (Sat,) studied this question.