English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Artificial agents have traditionally been trained to maximize reward, whichmay incentivize power-seeking and deception, analogous to how next-tokenprediction in language models (LMs) may incentivize toxicity. So do agentsnaturally learn to be Machiavellian? And how do we measure these behaviors ingeneral-purpose models such as GPT-4? Towards answering these questions, weintroduce MACHIAVELLI, a benchmark of 134 Choose-Your-Own-Adventure gamescontaining over half a million rich, diverse scenarios that center on socialdecision-making. Scenario labeling is automated with LMs, which are moreperformant than human annotators. We mathematize dozens of harmful behaviorsand use our annotations to evaluate agents' tendencies to be power-seeking,cause disutility, and commit ethical violations. We observe some tensionbetween maximizing reward and behaving ethically. To improve this trade-off, weinvestigate LM-based methods to steer agents' towards less harmful behaviors.Our results show that agents can both act competently and morally, so concreteprogress can currently be made in machine ethics--designing agents that arePareto improvements in both safety and capabilities.

Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark