English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

In many reinforcement learning (RL) applications, we want policies that reachdesired states and then keep the controlled system within an acceptable regionaround the desired states over an indefinite period of time. This latterobjective is called stability and is especially important when the state spaceis unbounded, such that the states can be arbitrarily far from each other andthe agent can drift far away from the desired states. For example, instochastic queuing networks, where queues of waiting jobs can grow withoutbound, the desired state is all-zero queue lengths. Here, a stable policyensures queue lengths are finite while an optimal policy minimizes queuelengths. Since an optimal policy is also stable, one would expect that RLalgorithms would implicitly give us stable policies. However, in this work, wefind that deep RL algorithms that directly minimize the distance to the desiredstate during online training often result in unstable policies, i.e., policiesthat drift far away from the desired state. We attribute this instability topoor credit-assignment for destabilizing actions. We then introduce an approachbased on two ideas: 1) a Lyapunov-based cost-shaping technique and 2) statetransformations to the unbounded state space. We conduct an empirical study onvarious queueing networks and traffic signal control problems and find that ourapproach performs competitively against strong baselines with knowledge of thetransition dynamics. Our code is available here:https://github.com/Badger-RL/STOP.

Learning to Stabilize Online Reinforcement Learning in Unbounded State Spaces