Performance analysis and optimization of holographic MIMO communication systems

The rapid evolution of sixth-generation wireless communication systems has stimulated interest in holographic multiple-input multiple-output (HMIMO) technology, which promises unprecedented spatial resolution and spectral efficiency through densely packed antenna arrays within physically constrained apertures. This dissertation investigates fundamental limits of HMIMO systems, including energy efficiency, capacity, and advanced beamforming strategies. First, we consider a multi-user HMIMO system under an electromagnetic (EM) wave-compliant stochastic channel model and answer a key question: how many antennas should be deployed within a spatially constrained surface to maximize energy efficiency. Closed-form achievable rate expressions are derived under maximum ratio transmission (MRT), zero-forcing (ZF) precoding, and their uplink counterparts. An alternating optimization framework that jointly optimizes power allocation and antenna count is developed. Our key finding is that the optimal antenna count depends on the operating regime. Specifically, under MRT and MRC, exceeding the channel degrees-of-freedom improves energy efficiency only in noise-limited scenarios, while ZF precoding benefits from excess antennas across all power budgets. Second, we examine assumptions about HMIMO capacity gains using a bilateral multiport network model that captures mutual coupling, evanescent waves, and near-field interactions, phenomena neglected in most prior stochastic models. Capacity is evaluated and compared against conventional half-wavelength-spaced arrays in both line-of-sight (LoS) and non-line-of-sight (NLoS) environments. We show that sub-wavelength packing can yield capacity improvements only above a transmit power threshold that increases with separation distance and becomes more stringent under realistic antenna losses. Power thresholds are reduced in NLoS environments, yet remain practically high. Third, we investigate stacked intelligent surface (SIS)-assisted multi-user MISO systems, where beamforming is realized in the EM wave domain with a minimal number of RF chains. A hybrid digital-wave domain channel estimator is proposed, and the SIS is optimized to approximate regularized ZF precoding. Closed-form rate bounds derived via random matrix theory reveal dependence on training duration and coherence time. Optimized training yields substantial net sum-rate gains over fixed schemes, with the optimal training length decreasing for highly correlated channels, underscoring the importance of jointly designing training protocols and surface configurations. Together, these works advance the theoretical understanding and practical design of next-generation HMIMO systems.