Close form maximum likelihood covariance matrix estimation under a knowledge‐aided constraint

Knowledge‐aided (KA) space–time adaptive processing (STAP) is an appealing scheme for improving the detection performance of slow‐moving target in sample starved heterogeneous environments. The authors address the maximum likelihood (ML) estimation problem of the interference covariance matrix under a KA constraint. To reduce the complexity of interior point method, a close form ML estimator for the interference covariance matrix is derived. Moreover, for the hyper‐parameter selection in the KA constraint, which remains an unsolved open problem, an efficient and fully automatic method based on likelihood function and cross validation is proposed. The authors find that the proposed estimator consists of a prewhitening step and an eigenvalue‐truncation step of the whitened sample covariance matrix (SCM), which is somewhat similar to the existing fast ML with assumed clutter covariance (FMLACC) method. However, different ways for truncating the eigenvalues of the whitened SCM are exploited. The numerical simulations also demonstrate that by appropriately choosing the hyper‐parameter, the proposed estimator can remarkably outperform the FMLACC method in some situations.