Spiking Neural Networks (SNNs) promise extreme efficiency for large language models (LLMs) through event-driven sparse computation, yet current SNN-LLM hybrids inherit three fundamental flaws: surrogate gradients erode event-driven sparsity, dense self-attention introduces quadratic complexity, and floating-point word embeddings contradict spike-based representations. We propose CTP-Hybrid, a causal topological pulsing hybrid architecture that addresses these flaws with three native spiking components: (1) IST (Inter-Spike Topology Encoding) replaces embeddings with inter-spike interval ratios; (2) TCP (Temporal Causal Precedence) replaces the standard causal mask with a partial order derived from first-spike timings, enforcing physical causality in information flow; (3) TD-SCA (Temporal Dynamic Sparse Columnar Array) replaces feed-forward layers with biologically plausible cortical microcolumns. To our knowledge, we present the first successful 7B-parameter spiking LLM fine-tuning under 8~GB consumer GPU constraints (NVIDIA RTX~5060 Laptop) and its reproduction on a cloud RTX~4090. With only 2,000 training samples and QLoRA, the TD-SCA microcolumn variant achieves a WikiText-2 perplexity of 16.58 locally; on the cloud, training loss reaches 2.18 but test perplexity rises to 624.73, indicating overfitting on the extremely limited data. All models generate coherent Chinese text, proving trainability under extreme hardware constraints. The complete IST+TCP+TD-SCA forward pass is verified with correct logits and 7.7~GB memory, confirming architectural self-consistency. Full tri-module joint training is currently blocked by quantization kernel gaps and gradient flow limitations in current toolchains, not by any architectural defect. All code and experiment logs are archived and ready for immediate resumption. This work establishes the first reproducible blueprint for consumer-grade spiking LLMs and introduces a novel causal-topological computing paradigm.
Shutong Hou (Fri,) studied this question.