Low‐Latency English Writing Assistance via End‐Edge‐Cloud Collaborative Inference

The deployment of Large Language Models (LLMs) in real‐time English Writing Assistance (EWA) is often hindered by the conflicting requirements of high linguistic precision and low interaction latency. Traditional cloud‐centric or end‐side deployment paradigms suffer from either significant network delays or degraded generation quality due to hardware constraints. In this paper, we propose a low‐latency collaborative inference framework that integrates end, edge, and cloud resources. We introduce a semantic‐complexity aware gating mechanism that dynamically routes queries based on linguistic difficulty, alongside an asynchronous speculative parallelism architecture to mask transmission overhead. Experimental results on WikiText‐103, CNN/DailyMail, SQuAD 2.0, and the CoNLL‐2014 Grammatical Error Correction benchmarks demonstrate that our framework reduces end‐to‐end latency by up to 75% and energy consumption by 39% compared to cloud‐only baselines, while maintaining over 98% of the original model's accuracy. By effectively balancing computational loads and network jitter, the proposed system provides a scalable solution for responsive and high‐quality LLM‐driven intelligent writing services.