Optimizing UAV location for deployment in Cell-Free Massive MIMO Networks Using a Soft Actor-Critic Reinforcement Learning

The unmanned aerial vehicle communication network has recently garnered significant attention and motivation in academic and industrial research due to its flexible mobility concept of adjusting its location to provide coverage in temporary congested hotspot areas and repetitive user interaction. Although unmanned aerial vehicles offer relatively stable network connectivity in these challenging environments due to their unique mobility and adaptability, their integration with cell-free Massive MIMO (CF-mMIMO) presents technical challenges related to their optimal deployment and coverage limitations. To address these issues, we employ soft actor-critic optimization based on deep reinforcement learning to automatically identify an ideal unmanned aerial vehicle (UAV) placement within a determined coverage area, considering channel capacity and throughput. Lastly, the unmanned aerial vehicle (UAV) is strategically deployed to alleviate congestion in specific regions while addressing communication dependencies with the cell-free access point (CF-AP). Furthermore, we also used a gradient descent algorithm in conjunction with modified K-means to assign users to each UAV cluster. Numerical simulations confirm that soft actor critic (SAC)-based deployment achieves significant throughput and total channel capacity improvement compared to standard static K-means and heuristic deployment techniques. In particular, the SAC algorithm provides a 152.93 Mbps throughput improvement at 19.636 Gbps total channel capacity improvement over conventional deployment techniques. The results confirm the ability of the soft actor critic (SAC) to dynamically adjust UAV locations effectively, providing seamless connections and optimal performance for CF-mMIMO systems under urban and high-density traffic conditions.