Intrinsic interference-constrained time–frequency collaborative prototype filter design for filter bank multi-carrier systems

Prototype-filter design in Filter Bank Multi-Carrier with Offset Quadrature Amplitude Modulation (FBMC/OQAM) requires a coordinated trade-off among local real-field orthogonality, spectral localization and temporal localization. To address the coupling among these characteristics, an intrinsic-interference-constrained time-frequency collaborative prototype-filter design framework was proposed. Firstly, the intrinsic-interference potential is characterized by using the imaginary cross-correlation between locally shifted time-frequency basis functions. This term is jointly considered with normalized stop-band leakage energy and time-domain tail energy, and a fixed-reference normalization strategy is introduced to eliminate differences in numerical scale. A weighted logarithmic aggregation is then employed to construct a unified optimization objective. Secondly, an Elite-guided Adaptive Differential Evolution (EADE) algorithm is developed as the numerical solver. EADE combines stage-dependent parameter adaptation, elite-guided differential mutation, decaying Gaussian local refinement and explicit elite preservation to coordinate global exploration and local exploitation in the non-convex prototype-filter parameter space. The proposed solver is evaluated under an equal fitness-evaluation budget against standard and adaptive DE variants, while the resulting prototype filter is further compared with established FBMC prototype-filter designs. Multi-run statistical analysis, ablation experiments, objective-weight sensitivity, computational-cost evaluation and BER tests under both AWGN and frequency-selective fading channels are conducted. The results provide a systematic assessment of the proposed model and solver in terms of optimization behavior, time-frequency localization, spectral leakage, transmission reliability and robustness.