Reverse Auctions with Multiple Reinforcement Learning Agents *

ABSTRACT Reverse auctions in business‐to‐business (B2B) exchanges provide numerous benefits to participants. Arguably the most notable benefit is that of lowered prices driven by increased competition in such auctions. The competition between sellers in reverse auctions has been analyzed using a game‐theoretic framework and equilibria have been established for several scenarios. One finding of note is that, in a setting in which sellers can meet total demand with the highest‐bidding seller being able to sell only a fraction of the total capacity, the sellers resort to a mixed‐strategy equilibrium. Although price randomization in industrial bidding is an accepted norm, one might argue that in reality managers do not utilize advanced game theory calculations in placing bids. More likely, managers adopt simple learning strategies. In this situation, it remains an open question as to whether the bid prices converge to the theoretical equilibrium over time. To address this question, we model reverse‐auction bidding behavior by artificial agents as both two‐player and n ‐player games in a simulation environment. The agents begin the game with a minimal understanding of the environment but over time analyze wins and losses for use in determining future bids. To test for convergence, the agents explore the price space and exploit prices where profits are higher, given varying cost and capacity scenarios. In the two‐player case, the agents do indeed converge toward the theoretical equilibrium. The n ‐player case provides results that reinforce our understanding of the theoretical equilibria. These results are promising enough to further consider the use of artificial learning mechanisms in reverse auctions and other electronic market transactions, especially as more sophisticated mechanisms are developed to tackle real‐life complexities. We also develop the analytical results when one agent does not behave strategically while the other agent does and show that our simulations for this environment also result in convergence toward the theoretical equilibrium. Because the nature of the best response in the new setting is very different (pure strategy as opposed to mixed), it indicates the robustness of the devised algorithm. The use of artificial agents can also overcome the limitations in rationality demonstrated by human managers. The results thus have interesting implications for designing artificial agents in automating bid responses for large numbers of bids where human intervention might not always be possible.