Poster: Design of Elastic Deep Neural Network Candidate Spaces for Inference on Diverse Devices

Deep Neural Network (DNN) inference on edge devices is now a common practice. However, tailoring a model for multiple devices involves a lot of time and effort. While elastic models, also known as weight-sharing models, have been proposed as an efficient solution to create high-accuracy, low-latency modules, the design of an elastic model's candidate space (search space) has been underexplored. We identified a new characteristic in candidate spaces, which we named sensitivity, made a design rationale for candidate spaces based on it, and then built a preliminary algorithm to generate candidate spaces. Results show that we can get a range of models (with a 2.75× FLOPs range) on the Pareto frontier of the space by training only once.