Open AccessModeling the formation of social conventions from embodied real-time interactions
Author(s) -
Ismael T. Freire,
Clément Moulin-Frier,
Martí Sánchez-Fibla,
Xerxes D. Arsiwalla,
Paul F. M. J. Verschure
Publication year - 2020
Publication title -
plos one
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.99
H-Index - 332
ISSN - 1932-6203
DOI - 10.1371/journal.pone.0234434
Subject(s) - reinforcement learning , embodied cognition , computer science , control (management) , social learning , task (project management) , game theory , perception , embodied agent , artificial intelligence , human–computer interaction , psychology , engineering , knowledge management , systems engineering , neuroscience , economics , microeconomics
What is the role of real-time control and learning in the formation of social conventions? To answer this question, we propose a computational model that matches human behavioral data in a social decision-making game that was analyzed both in discrete-time and continuous-time setups. Furthermore, unlike previous approaches, our model takes into account the role of sensorimotor control loops in embodied decision-making scenarios. For this purpose, we introduce the Control-based Reinforcement Learning (CRL) model. CRL is grounded in the Distributed Adaptive Control (DAC) theory of mind and brain, where low-level sensorimotor control is modulated through perceptual and behavioral learning in a layered structure. CRL follows these principles by implementing a feedback control loop handling the agent’s reactive behaviors (pre-wired reflexes), along with an Adaptive Layer that uses reinforcement learning to maximize long-term reward. We test our model in a multi-agent game-theoretic task in which coordination must be achieved to find an optimal solution. We show that CRL is able to reach human-level performance on standard game-theoretic metrics such as efficiency in acquiring rewards and fairness in reward distribution.