ILL: A Lightweight Large Language Model for Legal and Courtroom Assistance

Manual case review in legal and courtroom workflows faces efficiency bottlenecks. While LLMs offer potential for vertical domains, they often struggle with domain-specific accuracy and hallucinations. This paper introduces ILL, a lightweight model for legal and courtroom assistance trained via QLoRA. By employing 4-bit quantization on an RTX 4060 GPU, ILL achieves precise knowledge transfer with low computational costs. The model attained a BertScore F1 of 0.8037, and a perplexity of 1.89, while largely preserving TruthfulQA performance after fine-tuning and demonstrating competitive results on MMLU tasks. Experiments demonstrate that this method performs excellently on small-scale datasets and shows approximate convergence at scales of about 1200 sentences. This work validates the feasibility and efficiency of constructing high-quality vertical auxiliary models using limited computational resources.