Tiny Transformers for TinyML: A Systematic Review of Deployment Readiness, Trustworthiness, and Benchmarking

Tiny Machine Learning (TinyML) is expanding beyond compact convolutional and feed-forward models toward Transformer-family architectures adapted to microcontrollers and extreme-edge systems. This systematic review evaluates the deployment readiness of Tiny Transformers and closely related compact sequence models under severe memory, latency, energy, and runtime constraints, while examining security, privacy, reliability, uncertainty, reproducibility, sustainability, benchmarking, and application maturity. The review followed PRISMA 2020, and its methodological record was retrospectively registered on the Open Science Framework (OSF identifier 7fazq; review period: 2 March–1 June 2026). Searches across major bibliographic databases and supplementary sources identified 355 records; 213 unique records were screened, 87 full texts were assessed, and 81 reports were included in the qualitative synthesis. Direct Tiny Transformer deployment evidence remains limited and heterogeneous but converges on four complementary feasibility pathways: memory-aware scheduling, task-specific architectural downsizing, reduced-precision and runtime co-design, and sparse architecture–deployment co-design. Across the evidence base, support was substantially stronger for real-device execution and resource-aware optimization than for implemented security and privacy mechanisms, uncertainty calibration and out-of-distribution handling, reproducible deployment pipelines, standardized cross-platform benchmarking, system-level sustainability assessment, and longitudinal field validation. The evidence therefore supports a restrained conclusion: Tiny Transformers are becoming technically viable for selected constrained intelligent systems, but trustworthy, reproducible, and operationally validated deployment remains immature. Future research should prioritize cross-platform evaluation, transparent memory and energy reporting, quantified trustworthiness overhead, reproducible deployment artifacts, and field-oriented validation under realistic operational constraints.