Intelligent Link Adaptation for Integrated Data and Energy Transfer: An Enhanced DRL Approach for Long-Term Constraints

Modulation scheme and power control simultaneously impact the performance of integrated data and energy transfer (IDET). Therefore, some efforts have been invested in deep reinforcement learning (DRL) algorithms to realize adaptive modulation (AM) and adaptive power control (APC), in order to achieve long-term performance improvement. However, the optimal DRL algorithm design for the long-term performance optimization having long-term constraints is still a challenge, while the optimal patterns of IDET-oriented joint AM and APC are not fully understood. This paper aims to maximize the long-term performance of energy harvesting (EH), while satisfying the long-term constraints of spectrum efficiency, bit-error-rate and transmit power, by jointly optimizing the modulation selection and transmit power allocation. Then, a novel DRL algorithm, named constrained parameterized action deep deterministic policy gradient (C-PADDPG), is proposed to find the feasible policy of joint AM and APC for the transformed constraint satisfaction problem. Meanwhile, the optimal policy is searched for via bisection method. Simulation results demonstrate that our solution can achieve significant gain on the long-term EH performance, compared to the traditional genetic algorithm-based solution and other DRL benchmark. Moreover, the communication-efficient and EH-efficient patterns of joint AM and APC generated by the C-PADDPG algorithm are explicitly illustrated and analyzed.