Multi-Agent Computing-Energy-Efficiency Optimization in Vehicular Edge Computing: Non-Cooperative Versus Cooperative Solutions

Vehicular edge computing (VEC) has driven the proliferation of computation-intensive and delay-sensitive vehicular services by deploying computing and energy resources at the edge. However, the exploitation of edge resources faces challenges due to unpredictable environmental dynamics and partial observability. To this end, this paper investigates the computing energy efficiency (CEE) problem in twin-timescale VEC scenarios by dynamically adjusting the offloading policy. Based upon modeling the problem as a decentralized partially observable Markov decision process (Dec-POMDP), a pair of non-cooperative and cooperative offloading solutions are proposed relying on multi-agent reinforcement learning (MARL), respectively. Specifically, the non-cooperative solution employs multi-agent independent proximal policy optimization (IPPO) to enable vehicular user equipments (VUEs) to learn their policies in a fully distributed manner without any information sharing. By contrast, the cooperative solution combines the multi-agent shared PPO with graph attention networks (MAPPO-GAT), where the relationship among agents is learned cooperatively and the historical learning experience is shared. Additionally, we compare the computational complexity and analyze the convergence. Simulation results show that in terms of the trade-off between offloading delay and offloading energy consumption, the proposed cooperative solution is superior to the non-cooperative counterpart with the cost of moderate training overhead for cooperative learning.