Run inference

DeepSeek V3 and R1 represent the frontier of Mixture-of-Experts language models at 671 billion total parameters (37B active per token). Current hardware limitations restrict deployment to multi-GPU clusters: no single H100 (80GB), A100 (80GB), or consumer GPU can load the model at any practical quantization level. We demonstrate that the Apple M2 Ultra with 192GB unified memory is the only consumer-grade hardware capable of running this architecture, at IQ1_M quantization (139GB) fitting entirely in RAM with 53GB remaining for a 500K+ token KV cache. We analyze the technical requirements, quantization tradeoffs (IQ1_M vs IQ1_S vs Q2_K), inference performance projections, and competitive implications. At a total hardware cost of $7,000 and power draw of 200W peak, this configuration achieves inference capabilities that require $300K+ GPU clusters to match. This positions Apple Silicon as uniquely capable in the high-end local inference landscape. We provide a roadmap for practical deployment including storage requirements, llama.cpp configuration, and performance benchmarks.