SwarmInfer: Intra-Layer Tensor-Parallel Distributed Inference on ESP32-S3 Microcontroller Clusters via ESP-NOW

Deploying neural networks on microcontrollers is constrained by severe memory limits, typically 256 to 512 kilobytes of static random-access memory, and when a single convolutional layer’s weights exceed device capacity, compression alone is insufficient. This paper presents SwarmInfer, a system for intra-layer tensor-parallel distributed inference on clusters of ESP32-S3 microcontrollers communicating via ESP-NOW, a connectionless wireless protocol. SwarmInfer partitions convolutional layers by output channels across worker nodes, broadcasts shared input activations, and gathers partial results—a mathematically exact partitioning that eliminates inter-worker synchronization during computation. We implement the complete system in 313 lines of C using 8-bit integer quantization with fixed-point requantization. On CIFAR-10 with five ESP32-S3 boards, a 408-thousand-parameter model distributed across four workers reaches 79.1% accuracy (versus 74.4% for a single-node 103-thousand-parameter baseline that fits one device) at 2,115 milliseconds per inference over 1,000 images; the 95% Wilson confidence intervals overlap by only 0.5 percentage points, indicating the gain is not sampling variation. Scaling from two to four workers gives a 1.73 times speedup, reaching 1.99 times at five workers, with near-linear scaling on compute-dominant deeper layers. Direct power measurement shows the four-worker cluster uses roughly ten times the per-inference energy of the single-node baseline, framing distribution as a capability—running models that exceed per-device memory—rather than an efficiency mechanism; the same exact partitioning runs a scaled MobileNet at 96-by-96 resolution, where the workload becomes communication-bound. The system completes 3,000 consecutive benchtop inferences with zero timeouts.