SNOWTEC: Synthetic Natural Language Oversampling with transformer-based information extraction for automated compliance checking

The optimisation of Automated Compliance Checking (ACC) in Architecture, Engineering, and Construction (AEC) necessitates the interpretation of building codes into machine-processable formats. As these codes primarily exist in textual form, Information Extraction (IE) became integral to decoding this data, encouraging various IE techniques spanning manual, rule-based, and machine-learning methodologies. Recent research has shown promise in adopting deep learning; however, as far as we know, within AEC, the transformers/language models’ potential remains untapped/unexplored, yet they hold state-of-the-art performance across various text-based tasks. To address this gap, we propose an approach based on Synthetic Natural language Oversampling With Transformer-based information ExtraCtion (SNOWTEC), designed to extract entities and relations from regulatory text to convert them into machine-processable knowledge graphs. We involve transformer-based architectures and introduce an innovative data oversampling/augmentation approach addressing data scarcity, which impedes model performance. Our experiments across multiple sub-domains highlight the transformers’ strength in identifying relations but also reveal challenges in recognising entities within the AEC domain, providing insights for future research. Data oversampling played a crucial role in improving relation extraction, resulting in a notable 26% average F1 increase. • Transformer-based information extraction for Automated Compliance Checking (ACC). • Text data oversampling to improve models’ performance addressing data scarcity. • Reformulation of Information Extraction from text as knowledge graph generation. • Entity-relation knowledge graphs for human-in-the-loop reviewing processes. • Open-source codebase/implementation, data and fine-tuned models.