Early-quit evolutionary search of hybrid channel attention networks for image classification

This paper introduces a groundbreaking Neural Architecture Search (NAS) technique, addressing the critical balance between computational efficiency and model performance within the context of Convolutional Neural Networks (CNNs). Recognizing the significant impact of architecture on CNN performance, and the computational intensity of evolutionary NAS methods despite their superior global search capabilities, we propose a novel solution. Our method integrates two key innovations: an Early-Quit (EQ) strategy to reduce the model's Floating Point Operations Per Second (FLOPS) and the incorporation of channel attention mechanisms into the NAS search space. Unlike traditional approaches that embed these mechanisms directly into the network, our strategy minimizes computational waste by terminating training early for less promising architectures and enhances feature extraction through attention mechanisms. Applied to standard benchmarks like CIFAR-10 and CIFAR-100 our approach demonstrates a substantial decrease in FLOPS while maintaining or even improving learning efficacy. This NAS methodology represents a significant advancement by efficiently generating lean yet competitive neural network architectures, marrying computational efficiency with strategic innovation.