IZeroDNAS: innovative differentiable NAS using zero-cost metrics and SuperNet initialization

This paper improves the stability and hardware-efficiency of zero-cost differentiable neural architecture search (NAS) for object detection, addressing two weaknesses of prior work: zero-cost rankings sensitive to SuperNet initialization and neglected edge-device deployment constraints. The authors propose IZeroDNAS, which integrates the NASWOT zero-cost metric into differentiable NAS. The authors introduce a SuperNet warm-up initialization, plus a depthwise-accelerated variant, to stabilize the rankings and a hardware-aware loss combining latency-aware and depth-aware terms built from a device look-up table. The authors evaluate on the ScaledYOLOv4 and YOLOv7 search spaces using Pascal VOC and COCO and deploy on Qualcomm RB5, Jetson TX2 and Jetson AGX Orin. Warm-up initialization reduces the mean average precision variance across random seeds from 5.687 to 0.0025 while raising accuracy. IZeroDNAS discovers shallower, wider models that are smaller and faster than manually designed detectors and the baseline ZeroDNAS, completing a search in under 5 h on a single RTX 2080, three times faster than comparable methods. The warm-up’s stabilizing effect is shown empirically rather than proven theoretically and a strictly fair comparison with proprietary methods such as YOLO-NAS is infeasible due to their undisclosed search spaces. Future work targets broader zero-cost proxies and more accurate device-performance predictors. IZeroDNAS is the first method to jointly tackle ranking instability and hardware-awareness in zero-cost differentiable NAS for object detection. Its warm-up and depthwise-accelerated variant offer a simple, modular, resource-efficient way to stabilize zero-cost metrics in large-scale SuperNets, benefiting practitioners deploying detectors on edge devices.