What does this research mean for the field?

The Malak Python toolkit effectively streamlines edge AI model optimization, achieving up to a 3.48x reduction in model size and a 2.16x inference speedup without significant accuracy degradation across standard convolutional architectures. Novelty: ClaimNovelty.METHODOLOGICAL. Consensus alignment: ConsensusAlignment.NEUTRAL.

What question did this study set out to answer?

This work aims to simplify the process of model optimization for edge AI by providing an intuitive toolkit.

June 7, 2026Open Access

Malak: A Python Toolkit for Edge AI Model Optimization

Key Points

This work aims to simplify the process of model optimization for edge AI by providing an intuitive toolkit.
Developed Malak, an open-source Python toolkit for model optimization
empirically validated compression techniques on benchmarks including CIFAR-10 and Fashion-MNIST
experimented with various model architectures like MobileNetV2, ResNet18, and a custom SimpleCNN.
Quantization-aware training achieved 3.48× reduction in MobileNetV2 model size with comparable accuracy
Dynamic post-training quantization maintained accuracy within 0.13% across architectures
On-device testing showed a 2.16× speedup for statically quantized SimpleCNN compared to the 32-bit floating-point model.

Abstract

Deploying deep learning models on resource-constrained edge devices demands systematic model compression and optimization. PyTorch supplies low-level quantization and pruning primitives, yet a typical quantization-aware training workflow still requires approximately 40 to 60 lines of observer wiring, calibration, and conversion boilerplate, which slows iteration for researchers and students. We present Malak, an open-source Python toolkit that wraps these primitives behind a small task-oriented application programming interface exposed through both a Python module and an edgeai command-line interface. The toolkit covers the full prototyping loop: model training; post-training quantization (dynamic and static) and quantization-aware training; magnitude and structured pruning; knowledge distillation; export to the Open Neural Network Exchange format; per-layer latency profiling; and a Kullback–Leibler-divergence drift check for deployed models. We empirically validate the com-pression subset (post-training quantization, quantization-aware training, pruning, and knowledge distillation) on the CIFAR-10 and Fashion-MNIST image-classification benchmarks with five architectures: MobileNetV2, ResNet18, ResNet50, EfficientNet-B0, and a custom convolutional network we refer to as Sim-pleCNN. Across three random seeds, quantization-aware training yields a 3.48× reduction in model size on MobileNetV2 with accuracy statistically indistinguishable from the 32-bit floating-point baseline (77.91±0.83 per cent versus 77.68±0.92 per cent). Dynamic post-training quantization preserves accuracy within 0.13 per cent across all tested architectures, and magnitude pruning at 50 per cent sparsity holds within roughly one percentage point of the baseline after three fine-tuning epochs. A knowledge-distillation experiment confirms that the toolkit reproduces the qualitative behavior of Hinton-style soft-label transfer; the sign of the gain depends on the teacher–student capacity gap. On-device latency measured on a deployment-class server processor (single-input inference, 200 measurements) shows a 2.16× wall-clock speedup for the statically quantized 8-bit-integer SimpleCNN over its 32-bit floating-point counterpart, and the same 8-bit-integer binary builds, fits within 445 kilobytes of on-chip memory, and runs end-to-end on a simulated STM32H7 (Arm Cortex-M7) microcontroller target under the Renode hardware simulator. The toolkit is released under the MIT license.

Mark Helpful

Bookmark

Relay

View Full Paper