English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We investigate the "Visual Pushing for Grasping" (VPG) system by Zeng et al.and the "Hourglass" system by Ewerton et al., an evolution of the former. Thefocus of our work is the investigation of the capabilities of both systems tolearn long-term rewards and policies. Zeng et al. original task only needs alimited amount of foresight. Ewerton et al. attain their best performance usingan agent which only takes the most immediate action under consideration. We areinterested in the ability of their models and training algorithms to accuratelypredict long-term Q-Values. To evaluate this ability, we design a new binsorting task and reward function. Our task requires agents to accuratelyestimate future rewards and therefore use high discount factors in theirQ-Value calculation. We investigate the behaviour of an adaptation of the VPGtraining algorithm on our task. We show that this adaptation can not accuratelypredict the required long-term action sequences. In addition to the limitationsidentified by Ewerton et al., it suffers from the known Deep Q-Learning problemof overestimated Q-Values. In an effort to solve our task, we turn to theHourglass models and combine them with the Double Q-Learning approach. We showthat this approach enables the models to accurately predict long-term actionsequences when trained with large discount factors. Our results show that theDouble Q-Learning technique is essential for training with very high discountfactors, as the models Q-Value predictions diverge otherwise. We alsoexperiment with different approaches for discount factor scheduling, losscalculation and exploration procedures. Our results show that the latterfactors do not visibly influence the model's performance for our task.

Optimizing the Long-Term Behaviour of Deep Reinforcement Learning for Pushing and Grasping