Text‐to‐image generation method based on cognitive‐inspired bidirectional attention mechanisms

Backgroud The human visual system can effectively perceive and understand complex visual scenes through the synergistic effect of bottom‐up primary visual cortex feature extraction and top‐down prefrontal cortex semantic guidance. This biological mechanism achieves an optimal balance between capturing fine‐grained detail and comprehending global semantics, providing valuable inspiration for multimodal generation research. Methods Inspired by this neural mechanism, this study proposes a method for text‐to‐image multimodal tasks and presents a new diffusion model framework that integrates bottom‐up and top‐down attention mechanisms. The bottom‐up mechanism employs global‐shape convolution, local key aggregation, and hyperbolic space mapping to hierarchically extract multi‐scale visual features, preserving both global structural integrity and local fine‐grained details by adaptively highlighting salient image regions. The top‐down mechanism dynamically modulates cross‐attention weights using mutual information, which quantifies text–image semantic correlation to enhance task‐relevant features, and information entropy, which measures feature uncertainty to suppress irrelevant noise. Results Extensive experiments conducted on the Common Objects in Context (COCO), Fill50K, and Large Scale Scene Understanding (LSUN) datasets show that our model compared favorably with the Stable Diffusion model in generation quality and diversity, achieving an inception score of 21.64 and a Fréchet inception distance (FID) of 23.65 on the COCO dataset. The model also achieved FID scores of 3.06 on LSUN Bedrooms and 4.98 on LSUN Churches. Ablation studies further validated the synergistic effect of the bidirectional framework and the independent contribution of each component, with notable improvements observed in text‐image alignment and fine‐grained detail handling. Conclusions This study demonstrates that cognitive‐inspired bidirectional attention mechanisms can effectively enhance text‐to‐image generation by balancing structural fidelity and semantic consistency. The integration of mutual information and information entropy provides a novel theoretical approach for multimodal fusion tasks and offers new directions for advancing the quality and applicability of generative models.