Efficient DOA Estimation in Hybrid Analog–Digital Structures: Mitigating the Impact of Mutual Coupling

Hybrid analog-digital structures (HADS) have emerged as an efficient solution for mitigating transmission loss and reducing power consumption in multiple-input multiple-output (MIMO) systems. However, the limited number of radio frequency (RF) chains and the presence of array mutual coupling (MC) pose significant challenges to achieving high-performance direction-of-arrival (DOA) estimation, thereby hindering effective downlink beamforming. To address these challenges, an efficient DOA estimation method specifically designed for HADS is proposed, effectively mitigating the impact of MC by employing a two-stage framework. The first stage reconstructs the spatial covariance matrix (SCM) by adjusting switch states and leveraging a middle subarray, combined with the real-valued subspace technique for initial DOA estimation. Using these initial estimates, MC is modeled and compensated by adjusting amplifiers and phase shifters. In the second stage, enhanced DOA estimation is achieved by fully exploiting the data from the entire array. Simulation results validate the effectiveness of the proposed method, demonstrating its capability to mitigate the MC effect and deliver accurate DOA estimation under practical conditions.