Learning-Based Spectrum Sharing and Spatial Reuse in mm-Wave Ultradense Networks

In this paper, the throughput maximization of millimeter-wave (mm-Wave) ultradense networks (UDN) using dynamic spectrum sharing (DSS) is considered. Most of the existing works only allow temporal-domain access and admit at most one user at each time slot, resulting in significant underutilization of spectrum resource, which will be less attractive to mm-wave UDN applications. A generalized temporal-spatial sharing scheme is proposed in this paper for UDN by exploiting the location information of incumbent devices, where multiple users are allowed to access each channel simultaneously via spatial separations. For distributed applications, the global information exchange among secondary users (SU) tends to be impractical, given the unaffordable signaling overhead and latency. Thus, a noncooperative game with fine-grained two-dimensional reuse is formulated, which leads to a more efficient access strategy. It is then proved to be an ordinary potential game (OPG), which guarantees the existence of the strategy Nash equilibrium (NE). Finally, an improved decentralized reinforcement learning algorithm is designed, with which SUs can learn from wireless environments and adapt toward an NE point, relying on the individual observation and the historical action reward (rather than the global information exchanging). The convergence efficiency of the new scheme is also rigorously proved. Numerical simulations are provided to validate the performances of the proposed schemes.