Combining LoRA to GPT-Neo to Reduce Large Language Model Hallucination

Abstract The deployment of Large Language Models (LLMs) often suffers from generating hallucinations, leading to outputs that appear plausible but are factually inaccurate or nonsensical. Incorporating Low-Rank Adaptation (LoRA) into GPT-Neo presents a novel approach to mitigating these hallucinations by leveraging the efficiency of low-rank approximations. This research details the integration of LoRA into GPT-Neo, demonstrating significant improvements in predictive performance, factual accuracy, and reduction in hallucination rates. The augmented model shows enhanced robustness and efficiency, making it more suitable for applications requiring high accuracy and reliability. Through comprehensive evaluations involving perplexity, BLEU, ROUGE-L scores, and qualitative analysis, the study highlights the augmented model's ability to generate more coherent and contextually appropriate text. The findings demonstrate the potential of LoRA to transform LLM deployment by reducing computational complexity and memory footprint, thus facilitating the use of large-scale models in resource-constrained environments. This advancement opens new possibilities for LLM applications across various domains, ensuring the accuracy and coherence of generated content.