English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

As AI systems become more intelligent and their behavior becomes morechallenging to assess, they may learn to game the flaws of human feedbackinstead of genuinely striving to follow instructions; however, this risk can bemitigated by controlling how LLMs generalize human feedback to situations whereit is unreliable. To better understand how reward models generalize, we craft69 distribution shifts spanning 8 categories. We find that reward models do notlearn to evaluate `instruction-following' by default and instead favor personasthat resemble internet text. Techniques for interpreting reward models'internal representations achieve better generalization than standardfine-tuning, but still frequently fail to distinguish instruction-followingfrom conflated behaviors. We consolidate the 15 most challenging distributionshifts into the GENeralization analogIES (GENIES) benchmark, which we hope willenable progress toward controlling reward model generalization.

Generalization Analogies: A Testbed for Generalizing AI Oversight to Hard-To-Measure Domains