Research on BERT Model Compression and Acceleration Based on Knowledge Distillation and Dynamic Routing

The BERT model has achieved strong performance in natural language processing, but its large parameter scale and high computational cost limit deployment in resource-constrained and real-time scenarios. This study proposes a lightweight BERT compression and acceleration method that integrates knowledge distillation and dynamic routing. In the knowledge distillation stage, a multi-layer feature alignment and attention transfer mechanism is developed to transfer deep semantic knowledge from the teacher model to a compact student model. The strategy uses output-layer distillation, intermediate-layer feature alignment, and attention-map transfer to preserve semantic representation ability. In the dynamic routing stage, an adaptive inference structure based on early exit is constructed, enabling the model to adjust computational depth according to the complexity of different input samples and reduce redundant computation. A joint optimization strategy is used to balance final-output accuracy, early-exit classifier performance, and distillation effectiveness. Experimental results on multiple natural language processing tasks show that the proposed method substantially reduces parameter count, storage space, and inference latency while maintaining competitive performance relative to the original BERT model.