English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The aim of this paper is to improve the understanding of the optimizationlandscape for policy optimization problems in reinforcement learning.Specifically, we show that the superlevel set of the objective function withrespect to the policy parameter is always a connected set both in the tabularsetting and under policies represented by a class of neural networks. Inaddition, we show that the optimization objective as a function of the policyparameter and reward satisfies a stronger "equiconnectedness" property. To ourbest knowledge, these are novel and previously unknown discoveries.  We present an application of the connectedness of these superlevel sets tothe derivation of minimax theorems for robust reinforcement learning. We showthat any minimax optimization program which is convex on one side and isequiconnected on the other side observes the minimax equality (i.e. has a Nashequilibrium). We find that this exact structure is exhibited by an interestingrobust reinforcement learning problem under an adversarial reward attack, andthe validity of its minimax equality immediately follows. This is the firsttime such a result is established in the literature.

Connected Superlevel Set in (Deep) Reinforcement Learning and its Application to Minimax Theorems