Timing-Constrained Composable Inference for Intermittent Systems Using Reinforcement Learning

The increasing maturity of energy harvesting technologies has brought intermittent systems to the forefront as viable solutions for a range of applications. One critical area is environmental monitoring, where timely and accurate reporting of environmental conditions is essential. Existing approaches on systems powered by unstable ambient energy focus on maintaining the freshness of collected data but fall short when applied to neural network workloads, as they often neglect model accuracy. Prior studies have explored deploying neural networks on intermittent systems using branchy architectures, which prioritize energy-accuracy tradeoffs by terminating inference early. However, these approaches fail to address time constraints, often resulting in system failures due to processing expired results. This paper introduces iTRAIN, a novel timing-aware framework for deploying neural network models on intermittent systems. iTRAIN holistically accounts for energy availability, timing constraints, and model accuracy. Unlike previous studies that depend on branchy architectures, iTRAIN leverages a composable neural network framework to broaden the solution space, enabling diverse energy-time-accuracy tradeoffs. This is achieved through runtime selection among various layer implementations, such as pruning and quantization, guided by a reinforcement learning algorithm. Experimental results demonstrate that iTRAIN outperforms state-of-the-art approaches, achieving a 65% improvement in delivered model accuracy with minimal memory and runtime overhead. iTRAIN sets a foundation for enabling complex applications on intermittent systems.