Fully homomorphic encryption on LlaMA 3 model for partial privacy preserving LLM inference

Generative Artificial Intelligence applications have delivered significant improvements in service efficiency and language understanding. However, the surge in Large Language Model deployment raises critical security and privacy concerns regarding data exposure and unauthorized access to sensitive information. Quantum computing attacks pose an imminent threat to existing cryptographic protections, necessitating post-quantum cryptographic approaches. We present a partially privacy-preserving inference pipeline for LLaMA-3 by integrating fully homomorphic encryption into the transformer’s Multi-Query Attention mechanism. Our approach encrypts attention weights and computations across all 32 decoder layers of LLaMA-3.2-3B using lattice-based cryptography via the concrete-ml library, while feed-forward sublayers, layer normalization, and output projections remain in plaintext, a deliberate trade-off achieving practical feasibility on commodity CPUs. Evaluation on the instruction-tuned LLaMA-3.2-3B model yields: (i) strong semantic preservation (BERTScore-F1 ∼ 84.4%), (ii) controlled perplexity degradation (1.18 FP16 vs. 2.09 FHE), (iii) competitive BLEU-4 and ROUGE-L scores ( ∼ 35.6% and ∼ 51.8%), (iv) downstream GLUE benchmark accuracy, and (v) 825 ms latency at 17 tokens/second on AMD EPYC CPUs. We further present quantization-aware FHE and quantization bit-width ablation studies to disentangle quantization effects from FHE noise, demonstrating that partial FHE-secured LLM inference is practically feasible for privacy-sensitive applications while mitigating attacks on attention layers under RLWE hardness assumptions.