English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep reinforcement learning (DRL) algorithms have been demonstrated to beeffective in a wide range of challenging decision making and control tasks.However, these methods typically suffer from severe action oscillations inparticular in discrete action setting, which means that agents select differentactions within consecutive steps even though states only slightly differ. Thisissue is often neglected since the policy is usually evaluated by itscumulative rewards only. Action oscillation strongly affects the userexperience and can even cause serious potential security menace especially inreal-world domains with the main concern of safety, such as autonomous driving.To this end, we introduce Policy Inertia Controller (PIC) which serves as ageneric plug-in framework to off-the-shelf DRL algorithms, to enables adaptivetrade-off between the optimality and smoothness of the learned policy in aformal way. We propose Nested Policy Iteration as a general training algorithmfor PIC-augmented policy which ensures monotonically non-decreasing updatesunder some mild conditions. Further, we derive a practical DRL algorithm,namely Nested Soft Actor-Critic. Experiments on a collection of autonomousdriving tasks and several Atari games suggest that our approach demonstratessubstantial oscillation reduction in comparison to a range of commonly adoptedbaselines with almost no performance degradation.

Addressing Action Oscillations through Learning Policy Inertia