Predictive performance of mid-infrared FTIR spectroscopy and machine learning models for soil property estimation in Japan

Soil analysis plays a crucial role in precision agriculture and environmental management by providing insights into soil health and fertility. Traditional laboratory-based soil property assessment is often time-consuming and costly, necessitating alternative approaches for rapid and accurate soil characterization. Fourier-transform infrared (FTIR) spectroscopy has gained attention for its nondestructive nature and potential to replace conventional soil analysis methods. However, the complexity of spectral data requires advanced analytical techniques. We evaluated the feasibility of FTIR spectroscopy combined one of three machine learning models – Partial Least Squares Regression (PLSR), Random Forest (RF), and one-dimensional Convolutional Neural Networks (1D-CNN) – for predicting soil properties using samples of soils obtained from across Japan. In addition, we compared two cross-validation approaches: random cross-validation (Random-CV) to simulate interpolation within known conditions and leave-one-prefecture-out cross-validation (LOPO-CV) to assess extrapolation performance across different regions. 1D-CNN outperformed both PLSR and RF in mineralizable nitrogen (Min N) prediction across both validation settings, achieving the lowest root mean squared error (RMSE) (77.44 mg N kg−1 with LOPO-CV and 44.06 mg N kg−1 with Random-CV). For pH, PLS achieved superior performance with Random-CV (R2 = 0.66, RMSE = 0.33), whereas 1D-CNN achieved the lowest RMSE (0.56) with LOPO-CV. K₂O prediction showed poor extrapolation performance across all models with LOPO-CV (R2 values from − 0.12 to 0.01). In summary, our findings demonstrate that (1) spectroscopic analysis combined with CNN is an effective and feasible approach for enhancing the measurement of Min N, particularly in interpolation tasks; and (2) although 1D-CNN performed strongly in interpolation, its negative R2 in extrapolation underscores the inherent limitations of machine learning models when applied beyond the training domain, emphasizing the need for rigorous validation and robust sampling strategies.