English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study reinforcement learning (RL) for learning a Quantal StackelbergEquilibrium (QSE) in an episodic Markov game with a leader-follower structure.In specific, at the outset of the game, the leader announces her policy to thefollower and commits to it. The follower observes the leader's policy and, inturn, adopts a quantal response policy by solving an entropy-regularized policyoptimization problem induced by leader's policy. The goal of the leader is tofind her optimal policy, which yields the optimal expected total return, byinteracting with the follower and learning from data. A key challenge of thisproblem is that the leader cannot observe the follower's reward, and needs toinfer the follower's quantal response model from his actions against leader'spolicies. We propose sample-efficient algorithms for both the online andoffline settings, in the context of function approximation. Our algorithms arebased on (i) learning the quantal response model via maximum likelihoodestimation and (ii) model-free or model-based RL for solving the leader'sdecision making problem, and we show that they achieve sublinear regret upperbounds. Moreover, we quantify the uncertainty of these estimators and leveragethe uncertainty to implement optimistic and pessimistic algorithms for onlineand offline settings. Besides, when specialized to the linear and myopicsetting, our algorithms are also computationally efficient. Our theoreticalanalysis features a novel performance-difference lemma which incorporates theerror of quantal response model, which might be of independent interest.

Actions Speak What You Want: Provably Sample-Efficient Reinforcement Learning of the Quantal Stackelberg Equilibrium from Strategic Feedbacks