Neural Network Optimization for Resource-Constrained IoT Devices

The rapid expansion of the Internet of Things (IoT) has created an urgent need for neural networks that deliver reliable intelligence under stringent constraints of memory, compute, and energy. This paper presents a unified, deployment-oriented framework for neural network optimization on resource-constrained IoT devices, integrating structured pruning, post-training and quantization-aware quantization, knowledge distillation, and lightweight architectural redesign. We formalize a multi-objective cost function that balances accuracy, latency, model size, and energy per inference, and we operationalize it via a staged pipeline: (i) sparsity-inducing pruning with topology preservation for microcontroller kernels, (ii) mixed-precision quantization to 8- and 4-bit pathways with calibration on device-representative data, (iii) teacher-student distillation with temperature-scaled soft targets to recover accuracy, and (iv) hardware–software co-tuning for common IoT platforms (Raspberry Pi, ESP32, and Cortex-M microcontrollers).