HERALD: A Hybrid Edge-RAG Architecture for Privacy-Preserving WhatsApp Humanoid Response Generation Using Local LLMs

HERALD introduces a novel hybrid architecture that enables fully local, privacy-preserving humanoid response generation on WhatsApp using Retrieval-Augmented Generation (RAG) and QLoRA-based fine-tuned large language models. Unlike existing cloud-based conversational AI systems that compromise user privacy, HERALD operates entirely on-device, ensuring zero data egress while maintaining high conversational fidelity. The system’s core innovation lies in the Dual-Engine Personal Adaptation (DEPA) mechanism, which separates stylistic learning (via QLoRA fine-tuning on personal chat data) from episodic memory retrieval (via FAISS-based RAG indexing). This resolves the long-standing trade-off between style fidelity and factual recall in personal AI systems. Additionally, HERALD introduces the Conversational Context Graph (CCG), a relationship-aware modeling layer that captures per-contact communication patterns such as emoji usage, response latency, and conversational tone. The architecture integrates local LLM inference (Phi-3-mini, 3.8B), hybrid retrieval (FAISS + BM25), and WhatsApp Cloud API for real-time deployment, along with humanoid behavior simulation including typing delays and message fragmentation. A 21-day user study (n=119) demonstrates: Turing Pass Rate: 78.3% Style Fidelity (BERTScore F1): 0.871 Episodic Recall Accuracy: 91.4% Average Latency: 2.3 seconds These results establish HERALD as the first empirically validated system combining local LLMs, RAG, and relationship-aware context modeling for real-time messaging. The work contributes two reusable primitives — DEPA and CCG — for future research in personal AI, digital twins, and edge intelligence.