English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Offline reinforcement learning (RL) methods strike a balance betweenexploration and exploitation by conservative value estimation -- penalizingvalues of unseen states and actions. Model-free methods penalize values at allunseen actions, while model-based methods are able to further exploit unseenstates via model rollouts. However, such methods are handicapped in theirability to find unseen states far away from the available offline data due totwo factors -- (a) very short rollout horizons in models due to cascading modelerrors, and (b) model rollouts originating solely from states observed inoffline data. We relax the second assumption and present a novel unseen stateaugmentation strategy to allow exploitation of unseen states where the learnedmodel and value estimates generalize. Our strategy finds unseen states byvalue-informed perturbations of seen states followed by filtering out stateswith epistemic uncertainty estimates too high (high error) or too low (toosimilar to seen data). We observe improved performance in several offline RLtasks and find that our augmentation strategy consistently leads to overalllower average dataset Q-value estimates i.e. more conservative Q-valueestimates than a baseline.

Exploiting Generalization in Offline Reinforcement Learning via Unseen State Augmentations