English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Reward hacking -- where RL agents exploit gaps in misspecified rewardfunctions -- has been widely observed, but not yet systematically studied. Tounderstand how reward hacking arises, we construct four RL environments withmisspecified rewards. We investigate reward hacking as a function of agentcapabilities: model capacity, action space resolution, observation space noise,and training time. More capable agents often exploit reward misspecifications,achieving higher proxy reward and lower true reward than less capable agents.Moreover, we find instances of phase transitions: capability thresholds atwhich the agent's behavior qualitatively shifts, leading to a sharp decrease inthe true reward. Such phase transitions pose challenges to monitoring thesafety of ML systems. To address this, we propose an anomaly detection task foraberrant policies and offer several baseline detectors.

The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models