A Monte Carlo based adaptive Kalman filtering framework for soil moisture data assimilation

The main sources of global soil moisture information are remote sensing observations and land surface model estimates. Data assimilation (DA) aims at optimally combining these data sources through statistical merging. To properly parameterize such merging, one needs to obtain accurate knowledge of model and observation uncertainties, which is the crux of a successful DA system. In this paper, we propose a new Monte Carlo based adaptive Kalman filtering framework (MadKF) that estimates model and observation uncertainties (Q and R) and updates soil moisture forecasts simultaneously. Spatially distributed uncertainties are estimated by applying triple collocation analysis (TCA) to Monte Carlo simulations of the model open-loop, the model analysis, and the observation time series at each grid cell. Error cross-covariances, which are inevitable between these time series, are diagnosed from their ensembles and used to iteratively correct biases they cause in Q and R estimates, and hence, in the Kalman filter gain. The proposed MadKF is tested in a synthetic environment and by assimilating real satellite soil moisture retrievals from the Advanced SCATterometer (ASCAT) into the Antecedent Precipitation Index (API) model forced with daily aggregated satellite precipitation. Synthetic experiments indicate a good convergence of Q and R estimates. Internal DA diagnostics, i.e., the innovation auto-correlation (IAC) and the variance of the normalized innovations, asymptotically converge to their desired values, which indicates that the filter is operating near its optimum and reliably estimates analysis uncertainty. Real-data experiments assimilating ASCAT observations into the API model further indicate that the MadKF is robust against observation error auto-correlations, which typically cause problems in conventional IAC-tuning based adaptive filtering approaches. A performance evaluation over 264 in situ sites within the contiguous United States shows that the MadKF leads to a significant skill gain in surface soil moisture estimation. The MadKF is also potentially applicable when assimilating Level 1 backscatter or brightness temperature observations, or when updating/assimilating other geophysical variables.