Structure-enhanced pre-trained language model for requirement-cyber threat intelligence knowledge graph completion

Critical Information Infrastructures (CIIs) underpin a nation’s socio-economic systems and are increasingly targeted by sophisticated cyberattacks. Current models often fail to capture the dynamic interplay between CII management strategies and evolving cyber threats, resulting in defensive measures that lag behind attack developments. To bridge this gap, we propose Requirement–Cyber Threat Intelligence Knowledge Graph Plus (RCTI+), an enhanced knowledge graph that integrates cyber threat intelligence (CTI) with cybersecurity management requirements into a heterogeneous attributed graph, which explicitly characterizes the relationships between management strategies and CTI. To conduct reasoning over RCTI+, we introduce Structure-Enhanced BERT (SeBert), a structure-enhanced pre-trained language model (PLM) with a dual-stage framework: (1) Graph Neural Networks (GNNs) extract structural representations, and (2) a PLM encodes textual attributes while capturing the mutual information between semantic and structural features. This hybrid approach produces rich contextual embeddings that capture both semantic content and topological structure. Extensive experiments on RCTI+, FB15k-237, and WN18RR demonstrate that SeBert achieves state-of-the-art performance and exhibits strong generalization across datasets. The RCTI+ is publicly available at: https://github.com/weiyongyy/RCTI-plus .