Distributional Reinforcement Learning for task offloading, resource allocation and early exit selection at the edge

Early Exiting (EE) is an emerging paradigm in deep learning that equips Deep Neural Networks (DNNs) with intermediate classifiers, enabling a trade-off between inference accuracy and latency. In this work, we investigate the integration of EE mechanisms into edge computing architectures, focusing on a representative use case involving task execution in resource-constrained computing and communications environments for connected and automated vehicles (CAVs). We develop a detailed system model that captures the complex interplay among time-varying system components, including wireless channel coherence and the dynamic availability of computational and communication resources. Building on this model, we formulate a joint optimization problem encompassing task offloading, resource allocation, and early exit selection. We demonstrate how EE enhances system adaptability under stringent constraints, such as limited bandwidth, computing capacity, or delay requirements. To tackle the complexity of the proposed optimization, we adopt a novel solution approach based on the distributional Soft Actor-Critic (SAC) Deep Reinforcement Learning (DRL) algorithm, which quantifies the uncertainty of the learned policy. Simulation results confirm that integrating EE with edge computing significantly improves the trade-off between inference accuracy and latency, achieving up to 212% improvement in the average task completion ratio compared to edge computing systems without EE, under the considered simulation settings.