ABSTRACT—The rapid expansion of natural language processing (NLP) and the widespread adoption of large language models—such as BERT, GPT, LLaMa, and DeepSeek—have intensified global concerns about energy consumption and computational sustainability. Data centers now use approximately 1.5% of global electricity, and AI workloads are projected to contribute an additional 85–134 TWh annually by 2027. Training a single large transformer model can emit hundreds of tons of CO₂-equivalent, raising critical questions about the long-term scalability of current AI paradigms. Spiking Neural Networks (SNNs) offer a biologically inspired pathway toward energy-efficient NLP, utilizing event-driven processing and temporal coding to approach the extraordinary efficiency of the human brain's 20-watt operation. This paper comprehensively surveys opportunities and challenges in integrating SNNs with modern NLP systems. We examine spiking architectures, learning rules, and neuromorphic hardware, analyzing how key transformer components—attention mechanisms, tokenization strategies, and pre-training objectives—can be adapted to the spiking domain while maintaining linguistic capability. We highlight major opportunities, including ultra-low-power neuromorphic inference, spike-based token representation, efficient temporal sequence modeling, and energy-aware training protocols. Concurrently, we address critical challenges such as training complexity, limited memory scalability on neuromorphic hardware, temporal encoding constraints for discrete language tokens, and the absence of standardized evaluation frameworks. By synthesizing current advances and open problems, this review provides a comprehensive roadmap for developing sustainable, scalable, and energy-efficient NLP systems through brain-inspired spiking computation.
Alam Mohammad Zahangir (2025) studied this question.