Alternative evaluation method for multi-source soil moisture datasets: In-situ monitoring network or triple collocation?

Accurate soil moisture (SM) information is vital for enhancing weather prediction, agricultural drought monitoring, water resource management and climate studies. Multi-source SM products provide valuable large-scale observations. However, their accuracy is affected by variations in retrieval algorithms and radiative transfer models. While in situ observations are commonly used as ground truth for product evaluation, their limited spatial coverage and scaling uncertainties pose challenges. The triple collocation (TC) method, which does not require reference data, offers an alternative for relative performance assessment across large areas. However, previous studies have rarely examined the performance differences between in situ-based evaluations and TC methods at large regional scales. In particular, the relationship between the accuracy of TC-based results and the density of in situ monitoring networks remains poorly understood. This study focused on the Huai River Basin as a representative region and integrated a dense in situ observation network of 234 gravimetric stations with the extended triple collocation (ETC) method to evaluate three representative SM products—SMOS, ESA CCI and ERA5-Land—from absolute and relative perspectives over the period from July 2012 to December 2017. Results showed that the station-based evaluation, benefiting from high-quality gravimetric observations, provided stable and physically consistent assessments of product accuracy and error characteristics. Amongst the three datasets, ERA5-Land exhibited the highest accuracy in site-based correlation and RMSE metrics, whereas ETC rankings favored ESA CCI in RMSE, revealing metric-dependent uncertainties, with SMOS generally performing the worst. ETC enabled large-scale reference-free evaluation, showing ESA CCI outperformed in correlation and RMSE, whereas ERA5-Land exhibited the highest sensitivity to SM dynamics. This inconsistency highlights the inherent uncertainty of ETC-based evaluations and suggests the need to develop a composite performance index that integrates multiple metrics. A “hit event” analysis further demonstrated strong consistency between ETC and station-based evaluations when using correlation coefficients, but weaker agreement under RMSE metrics. This study provides a systematic comparison between ground-based and ETC evaluations, clarifies their respective strengths and limitations and offers guidance for improving multi-source SM product validation and fusion strategies in future research. • Dense gravimetric network validates soil moisture reliably in flat agricultural regions. • Extended Triple Collocation (ETC) reveals metric-dependent uncertainties without ground truth data. • Scale-Aware Validation uses method agreement to map reliable zones and guide monitoring design.