English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Temporal-Difference (TD) learning is a standard and very successfulreinforcement learning approach, at the core of both algorithms that learn thevalue of a given policy, as well as algorithms which learn how to improvepolicies. TD-learning with eligibility traces provides a way to boost sampleefficiency by temporal credit assignment, i.e. deciding which portion of areward should be assigned to predecessor states that occurred at differentprevious times, controlled by a parameter $\lambda$. However, tuning thisparameter can be time-consuming, and not tuning it can lead to inefficientlearning. For better sample efficiency of TD-learning, we propose ameta-learning method for adjusting the eligibility trace parameter, in astate-dependent manner. The adaptation is achieved with the help of auxiliarylearners that learn distributional information about the update targets online,incurring roughly the same computational complexity per step as the usual valuelearner. Our approach can be used both in on-policy and off-policy learning. Weprove that, under some assumptions, the proposed method improves the overallquality of the update targets, by minimizing the overall target error. Thismethod can be viewed as a plugin to assist prediction with functionapproximation by meta-learning feature (observation)-based $\lambda$ online, oreven in the control case to assist policy improvement. Our empirical evaluationdemonstrates significant performance improvements, as well as improvedrobustness of the proposed algorithm to learning rate variation.

META-Learning State-based Eligibility Traces for More Sample-Efficient Policy Evaluation