User-Centric Resource Allocation in FD-RAN: A Stepwise Reinforcement Learning Approach

To improve resource utilization flexibility and enhance resource cooperation, a novel fully-decoupled radio access network (FD-RAN) architecture was conceived, allowing separate resource allocation of uplink and downlink. One of the envisions of FD-RAN and future 6G is to provide personalized services to users, namely satisfying users' demands differently. To achieve this goal, we utilize the idea from user-centric resource allocation (UCRA), which specifically takes into account users' subjective values of services during resource allocation. We first define a novel user utility function based on the prospect theory. Then, we study a subchannel allocation problem with an underlying heterogeneous network. Confronted with the complex solution space, we develop a stepwise reinforcement learning (RL) method which takes an action for only one user at each step. Furthermore, an action filter is utilized to select only feasible actions that meet the problem's constraints, such that the generated training data samples for RL are all valid, making training more efficient and stable. The method is also extended to multi-agent case, where users can choose their actions with their own agents. Owing to the stepwise action process, the nonstationary environment problem in standard multi-agent RL is naturally avoided. As a result, our method can be scaled to more agents. We have performed extensive simulations and the results validate the effectiveness of our proposed methods.