A Federated Fine-Tuning Framework for Large Language Models via Graph Representation Learning and Structural Segmentation

This paper focuses on the efficient fine-tuning of large language models within the federated learning framework. To address the performance bottlenecks caused by multi-source heterogeneity and structural inconsistency, a structure-aware federated fine-tuning method is proposed. The method incorporates a graph representation module (GRM) to model internal structural relationships within text and employs a segmentation mechanism (SM) to reconstruct and align semantic structures across inputs, thereby enhancing structural robustness and generalization under non-IID (non-Independent and Identically Distributed) settings. During training, the method ensures data locality and integrates structural pruning with gradient encryption (SPGE) strategies to balance privacy preservation and communication efficiency. Compared with representative federated fine-tuning baselines such as FedNLP and FedPrompt, the proposed method achieves consistent accuracy and F1-score improvements across multiple tasks. To evaluate the effectiveness of the proposed method, extensive comparative experiments are conducted across tasks of text classification, named entity recognition, and question answering, using multiple datasets with diverse structures and heterogeneity levels. Experimental results show that the proposed approach significantly outperforms existing federated fine-tuning strategies on most tasks, achieving higher performance while preserving privacy, and demonstrating strong practical applicability and generalization potential.