Multi-Layer feature fusion with Layer-GAT and adaptive readout for transformer text classification

In recent years, Transformer-based pre-trained language models have significantly advanced text classification. However, the complementary semantic information embedded in their multilayer hidden representations remains underutilized, often leading to inconsistent performance across different categories and text lengths. To address challenges such as class imbalance, variable text lengths, and ambiguous semantic boundaries, we propose MLF-LAR, a novel Transformer-based text classification model that integrates hierarchical information through a three-stage pipeline: extracting sentence-level representations from multiple layers, modeling inter-layer dependencies via a Graph Attention Network (GAT), and adaptively fusing layer-wise features with a learnable readout mechanism. The adaptive readout dynamically assigns layer weights per sample, regularized by entropy to prevent weight collapse and ensure stable fusion. Evaluated on four public datasets (Ohsumed, R8, MR, and SST-2), MLF-LAR achieves accuracies of 76.16%, 98.13%, 91.39%, and 95.99%, respectively. Ablation studies validate the contribution of each component, while confusion matrix analysis shows that our inter-layer interaction and adaptive fusion mechanisms enhance discriminability among confusable categories, improving overall classification performance and robustness.