Full-Parameter Fine-Tuning Method of LLMs for Sports Injury Prevention and Treatment

Fine-tuning large language models (LLMs) for sports injury prevention and treatment in resource-constrained environments poses significant challenges due to memory demands and growing size of data. This paper proposes an efficient full-parameter fine-tuning approach based on Gradient Low-Rank Projection (GaLore) to reduce memory usage. Further, a data augmentation strategy for sports injury prevention and treatment is utilized to finetune a question-and-answer (Q&A) model with 0.5B parameter on consumer GPUs with 24GB memory. Experiment results show that the proposed method enhanced by GaLore is superior to SOTA methods such as low-rank adaptation (LoRA) in terms of convergence accuracy, training time, memory consumption, and indicators of BLEU-4 and ROUGE-2. Meanwhile, the empirical effect of injury prevention Q&A cases indicate that Qwen2-0.5B-Instruct trained by the proposed method have obvious advantages in professional knowledge understanding and overcoming hallucinations.