English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Exploration-exploitation is a powerful and practical tool in multi-agentlearning (MAL), however, its effects are far from understood. To make progressin this direction, we study a smooth analogue of Q-learning. We start byshowing that our learning model has strong theoretical justification as anoptimal model for studying exploration-exploitation. Specifically, we provethat smooth Q-learning has bounded regret in arbitrary games for a cost modelthat explicitly captures the balance between game and exploration costs andthat it always converges to the set of quantal-response equilibria (QRE), thestandard solution concept for games under bounded rationality, in weightedpotential games with heterogeneous learning agents. In our main task, we thenturn to measure the effect of exploration in collective system performance. Wecharacterize the geometry of the QRE surface in low-dimensional MAL systems andlink our findings with catastrophe (bifurcation) theory. In particular, as theexploration hyperparameter evolves over-time, the system undergoes phasetransitions where the number and stability of equilibria can change radicallygiven an infinitesimal change to the exploration parameter. Based on this, weprovide a formal theoretical treatment of how tuning the exploration parametercan provably lead to equilibrium selection with both positive as well asnegative (and potentially unbounded) effects to system performance.

Exploration-Exploitation in Multi-Agent Learning: Catastrophe Theory Meets Game Theory