Answers are grounded in this paper and cite their sources.
Deploying Small Language Models (SLMs) with 0.5 billion to 1.5 billion parameters directly on consumer edge hardware preserves data privacy and removes cloud network latency, yet practical execution remains constrained by static Dynamic Random-Access Memory (DRAM) capacity and memory-bandwidth bottlenecks. The objective of this study is to compress pretrained SLMs below 4-bit average precision without suffering the severe multi-step reasoning degradation observed in uniform Post-Training Quantization (PTQ) or isolated block-diagonal Hessian approximations. To achieve this objective, we propose Q-Slim, a hybrid quantum-classical post-training compression framework combining tensor-network structural profiling with variational quantum combinatorial optimization. First, Q-Slim folds transformer Self-Attention and Multilayer Perceptron (MLP) weight matrices into higher-order tensors and decomposes them into canonical Matrix Product State (MPS) tensor trains to compute bond-wise von Neumann entanglement entropy as a calibration-free indicator of layer information capacity. Second, we derive an unconstrained two-qubit binary-radix variable encoding per layer group that maps discrete bit-width ({2, 3, 4, 8}-bit) and bond-truncation choices onto an 8-to-10 qubit Ising spin Hamiltonian using an exact single-layer distortion polynomial and a two-body quadratic coupling and memory surrogate, followed by deterministic greedy feasibility repair. Solving overlapping block windows with the Quantum Approximate Optimization Algorithm (QAOA) and unitary-folding Richardson Zero-Noise Extrapolation (ZNE) on the Qiskit Aer noisy density-matrix simulator yields a 97.6% min-max normalized approximation ratio against exact exhaustive enumeration. Evaluated on Qwen2.5-1.5B and Llama-3.2-1B under strictly matched 72.0% and 78.4% memory reductions, Q-Slim achieves 9.21 and 10.92 WikiText-2 perplexity on Qwen2.5-1.5B, outperforming uniform 4-bit and 3-bit GPTQ and AWQ while delivering 13.4 to 14.5 tokens per second on a Raspberry Pi 5. In conclusion, entanglement-guided quantum combinatorial allocation provides a practical, reproducible pathway for extreme edge SLM deployment via an interactive FastAPI and React studio.
No takes yet. Share an insight, caveat, or question.
I. et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: