Manual case review in legal and courtroom workflows faces efficiency bottlenecks. While LLMs offer potential for vertical domains, they often struggle with domain-specific accuracy and hallucinations. This paper introduces ILL, a lightweight model for legal and courtroom assistance trained via QLoRA. By employing 4-bit quantization on an RTX 4060 GPU, ILL achieves precise knowledge transfer with low computational costs. The model attained a BertScore F1 of 0.8037, and a perplexity of 1.89, while largely preserving TruthfulQA performance after fine-tuning and demonstrating competitive results on MMLU tasks. Experiments demonstrate that this method performs excellently on small-scale datasets and shows approximate convergence at scales of about 1200 sentences. This work validates the feasibility and efficiency of constructing high-quality vertical auxiliary models using limited computational resources.
Lin et al. (Mon,) studied this question.