Joint User Pairing and Beamforming Design of Multi-STAR-RISs-Aided NOMA in the Indoor Environment via Multi-Agent Reinforcement Learning

The development of sixth-generation (6G)/Beyond Fifth-Generation (B5G) wireless networks, which have requirements that go beyond current 5G networks, is gaining interest from academia and industry. However, to increase 6G/B5G network quality, conventional cellular networks that rely on terrestrial base stations are constrained geographically and economically. Meanwhile, Non-Orthogonal Multiple Access (NOMA) allows multiple users to share the same resources, which improves the spectral efficiency of the system and has the advantage of supporting a larger number of users. Additionally, by intelligently manipulating the phase and amplitude of both the reflected and transmitted signals, Simultaneously Transmitting and Reflecting RISs (STAR-RISs) can achieve improved coverage, increased spectral efficiency, and enhanced communication reliability. However, STAR-RISs must simultaneously optimize the amplitude and phase shift corresponding to reflection and transmission, which makes the existing terrestrial networks more complicated and is considered a major challenging issue. Motivated by the above, we study the joint user pairing for NOMA and beamforming design of Multi-STAR-RISs in an indoor environment. Then, we formulate the optimization problem with the objective of maximizing the total throughput of mobile users (MUs) by jointly optimizing the decoding order, user pairing, active beamforming, and passive beamforming. However, the formulated problem is a mixed-integer non-linear programming (MINLP). To address this challenge, we first introduce the decoding order for NOMA networks. Next, we decompose the original problem into two subproblems, namely: 1) MU pairing and 2) Beamforming optimization under the optimal decoding order. For the first subproblem, we employ correlation-based K-means clustering to solve the user pairing problem. Then, to jointly deal with beamforming vector optimizations, we propose Multi-Agent Proximal Policy Optimization (MAPPO), which can make quick decisions in the given environment owing to its low complexity. Finally, simulation results prove that our proposed MAPPO algorithm is superior to Proximal Policy Optimization (PPO) and Advanced Actor-Critic (A2C) by a maximum of 1% and 6%, respectively.