English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Auxiliary tasks have been argued to be useful for representation learning inreinforcement learning. Although many auxiliary tasks have been empiricallyshown to be effective for accelerating learning on the main task, it is not yetclear what makes useful auxiliary tasks. Some of the most promising results areon the pixel control, reward prediction, and the next state predictionauxiliary tasks; however, the empirical results are mixed, showing substantialimprovements in some cases and marginal improvements in others. Carefulinvestigations of how auxiliary tasks help the learning of the main task isnecessary. In this paper, we take a step studying the effect of the targetpolicies on the usefulness of the auxiliary tasks formulated as general valuefunctions. General value functions consist of three core elements: 1) policy 2)cumulant 3) continuation function. Our focus on the role of the target policyof the auxiliary tasks is motivated by the fact that the target policydetermines the behavior about which the agent wants to make a prediction andthe state-action distribution that the agent is trained on, which furtheraffects the main task learning. Our study provides insights about questionssuch as: Does a greedy policy result in bigger improvement gains compared toother policies? Is it best to set the auxiliary task policy to be the same asthe main task policy? Does the choice of the target policy have a substantialeffect on the achieved performance gain or simple strategies for setting thepolicy, such as using a uniformly random policy, work as well? Our empiricalresults suggest that: 1) Auxiliary tasks with the greedy policy tend to beuseful. 2) Most policies, including a uniformly random policy, tend to improveover the baseline. 3) Surprisingly, the main task policy tends to be lessuseful compared to other policies.

What makes useful auxiliary tasks in reinforcement learning: investigating the effect of the target policy