Monopolistic Models for Resource Allocation: A Probabilistic Reinforcement Learning Approach

Under the environment of cognitive radio networks, users are equipped with intelligent capabilities so that they can sense the conditions of networks and act optimally to maximize their revenues. Thus, dynamic spectrum access (DSA), which focuses on the management and distribution of the resources, is a critical problem, especially when only limited resources are available. Currently, this problem is handled by the game theory and auction theory, since this is a problem involving multiple agents. In this paper, we propose agent-based modeling method to model this multi-agent environment and probabilistic reinforcement learning to learn the optimal strategies. We focus on a simple scenario with only one primary user (PU) and multiple secondary users (SUs), and try to maximize the revenues of both sides, which may be further extended to other scenarios with different number of agents. First, we model this environment as a monopolistic market from the perspective of economics, where a PU acts as the monopoly and the SUs are passive buyers. Then, we propose probabilistic reinforcement learning methods to handle DSA so that both the PU and SUs can behave optimally by learning from the feedback of the others. Experimental results prove the flexibility and superior performance of our proposed methods.