Efficient Neural Architecture Search with Model-Size Constraints: A Gradient-Based Approach

Abstract Neural architecture search (NAS) has recently gained significant attention in the field of AutoML, driven by advances in deep learning techniques. Although many existing NAS methods prioritize classification performance, they require a small architecture to operate effectively in memory-constrained environments. Because model size is a crucial factor in deep learning, NAS methods that balance model size and classification performance are essential. However, owing to their large architectural search spaces, conventional NAS methods tend to converge on large architectures and consume substantial computational resources. To address this issue, this study proposes a novel NAS method that introduces constraints on model size to reduce search time and computational cost. Specifically, our approach involves adjustments to the learning process to improve efficiency, rearranging algorithmic sequences, and integrating a technique called “change alpha’s priority”. Experimental results confirm that the proposed method can identify search for smaller network architectures than the original unconstrained NAS method while maintaining high classification accuracy. The proposed method can search for neural network architectures suitable for various devices and applications.