English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Can humans get arbitrarily capable reinforcement learning (RL) agents to dotheir bidding? Or will sufficiently capable RL agents always find ways tobypass their intended objectives by shortcutting their reward signal? Thisquestion impacts how far RL can be scaled, and whether alternative paradigmsmust be developed in order to build safe artificial general intelligence. Inthis paper, we study when an RL agent has an instrumental goal to tamper withits reward process, and describe design principles that prevent instrumentalgoals for two different types of reward tampering (reward function tamperingand RF-input tampering). Combined, the design principles can prevent both typesof reward tampering from being instrumental goals. The analysis benefits fromcausal influence diagrams to provide intuitive yet precise formalizations.

Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspective