What question did this study set out to answer?

This research aims to optimize the inference capabilities of 2B-parameter LoRA Mixture of Experts models on Apple Silicon.

June 3, 2026Open Access

Inference on Consumer Silicon: Phase-Gated Compute, Shared-Base LoRA Sharding, and MPS Isolation for 2B-Parameter MoE Models

Key Points

This research aims to optimize the inference capabilities of 2B-parameter LoRA Mixture of Experts models on Apple Silicon.
Developed a production system to achieve 38-61 TPS and sub-200ms TTFT p95 on M2 Pro with 32 GB RAM.
Implemented phase-gated layer depth, semantic KV cache flushing, zero-parameter symbolic routing, shared-base LoRA serving, and MPS isolation.
Utilized a 22-suite evaluation harness alongside Cohen's d and paired t-tests for validation.
Achieved 38-61 TPS, indicating strong inference performance.
Utilized techniques led to sub-200ms TTFT p95 and peak RSS under 6 GB on M2 Pro.
Statistical validation demonstrated significant performance improvements with Cohen's d values.

Abstract

We present a production system for interactive inference of a 2B-parameter, 9-expert LoRA MoE on Apple Silicon achieving 38-61 TPS, sub-200ms TTFT p95, and peak RSS under 6 GB on M2 Pro (32 GB). Five techniques combine: phase-gated layer depth (PTPS, WP-03); semantic KV cache flushing at EXHALE/EMPTY phases based on generation semantics rather than attention scores; zero-parameter symbolic routing (VortexGate, WP-01); shared-base LoRA shard serving (base model in MPS + N x 1-5 MB adapters CPU-resident); and MPS process isolation preventing Electron V8 heap from competing for Apple Silicon unified memory. Validated by 22-suite eval harness with Cohen's d and paired t-tests.

Read Full Paperexternally

AI에게 질문

Bookmark

View Full Paper