English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

One of the central challenges faced by a reinforcement learning (RL) agent isto effectively learn a (near-)optimal policy in environments with large statespaces having sparse and noisy feedback signals. In real-world applications, anexpert with additional domain knowledge can help in speeding up the learningprocess via \emph{shaping the environment}, i.e., making the environment morelearner-friendly. A popular paradigm in literature is \emph{potential-basedreward shaping}, where the environment's reward function is augmented withadditional local rewards using a potential function. However, the applicabilityof potential-based reward shaping is limited in settings where (i) the statespace is very large, and it is challenging to compute an appropriate potentialfunction, (ii) the feedback signals are noisy, and even with shaped rewards theagent could be trapped in local optima, and (iii) changing the rewards alone isnot sufficient, and effective shaping requires changing the dynamics. Weaddress these limitations of potential-based shaping methods and propose anovel framework of \emph{environment shaping using state abstraction}. Our keyidea is to compress the environment's large state space with noisy signals toan abstracted space, and to use this abstraction in creating smoother and moreeffective feedback signals for the agent. We study the theoreticalunderpinnings of our abstraction-based environment shaping, and show that theagent's policy learnt in the shaped environment preserves near-optimal behaviorin the original environment.

Environment Shaping in Reinforcement Learning using State Abstraction