Mitigating position bias in hybrid summarisation via a Gated Dual-Encoder network

Generative summarisation models have demonstrated remarkable fluency; however, they frequently encounter hallucinations and factual inconsistencies, especially in specialised domains that involve lengthy documents. Hybrid approaches aim to alleviate these issues by integrating extractive key sentences as guidance. Nevertheless, current methods generally utilise an ‘early fusion’ strategy, which involves merely concatenating key sentences with the source text. This approach results in two significant bottlenecks: representation mismatch and position bias. This paper proposes a novel Gated Dual-Encoder Summarisation Framework: unlike unified encoders, it uses two parameter-independent encoders to separately process source text and key information, avoiding semantic confusion. Additionally, we present a Dynamic Gated Fusion Module that adaptively weighs the contributions of the source context and guidance signals through a learnable gating scalar. Experimental results on CNN/DailyMail, XSum, and PubMed demonstrate that our approach significantly outperforms state-of-the-art baselines. Specifically, on the PubMed dataset, our model achieves a ROUGE-L score of 40.05 and a FactCC score of 86.4%, outperforming the BART-Large baseline by 7.2% in factual consistency. Furthermore, our model demonstrates high robustness with a performance degradation rate (PDR) of only 2.8% under sentence shuffling tests.