Attention-based QoE-aware Digital Twin Empowered Edge Computing for Immersive Virtual Reality

Metaverse applications such as virtual reality (VR) content streaming, require optimal resource allocation strategies for mobile edge computing (MEC) to ensure a high-quality user experience. In contrast to online reinforcement learning (RL) algorithms, which can incur substantial communication overheads and longer delays, the majority of existing works employ offline-trained RL algorithms for resource allocation decisions in MEC systems. However, they neglect the impact of desynchronization between the physical and digital worlds on the effectiveness of the allocation strategy. In this paper, we tackle this desynchronization using a continual RL (CRL) framework that facilitates the resource allocation dynamically for MEC-enabled VR content streaming. We first design a digital twin-empowered edge computing (DTEC) system and formulate a quality of experience (QoE) maximization problem based on attention-based resolution perception. This problem optimizes the allocation of computing and bandwidth resources while adapting the attention-based resolution of the VR content. The CRL framework in DTEC enables adaptive online execution in a time-varying environment. We propose three variants of CRL, namely Continual Deep Deterministic Policy Gradient (CDDPG), Prioritized Experience Replay - CDDPG (PER-CDDPG), and Freshness Prioritized Experience Replay - CDDPG (FPER-CDDPG). We evaluate these algorithms, including two other benchmarks, using extensive experiments. FPER-CDDPG shows superior performance in terms of average latency, QoE, and successful delivery rate as well as meeting the hfQoE requirements over long-term execution while ensuring system scalability with the increasing number of users.