Text Anomaly Detection with LLM Embeddings: A Review

Abstract— Anomaly detection in text data is essential for maintaining information integrity across domains such as cybersecurity, fraud detection, and content moderation. This review examines how Large Language Models (LLMs), including BERT, GPT-3, and LLaMA, are used in existing research to support text anomaly detection through contextualised embeddings that enhance shallow methods such as K-Nearest Neighbors (KNN) and Isolation Forest, as well as hybrid approaches that incorporate scoring heads or metric learning modules. By synthesising findings from recent literature, the review highlights that LLM embeddings capture rich semantic cues and enable strong performance in tasks such as fake news identification. However, the reviewed works reveal several limitations, including limited exploration of lightweight hybrid models, insufficient cross-domain evaluations, lack of calibration guidelines, and strong dependence on embedding quality. This review outlines future research opportunities, including cross-domain benchmarking, development of interpretability tools, calibration frameworks, exploration of lightweight hybrid architectures, and embedding-level audits to improve reliability. Keywords— Anomaly Detection, Large Language Models, LLM Embeddings, Hybrid Models, Text Mining, Natural Language Processing.