English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Human preference alignment is essential to improve the interaction quality oflarge language models (LLMs). Existing alignment methods depend on manuallyannotated preference data to guide the LLM optimization directions. However,continuously updating LLMs for alignment raises a distribution gap betweenmodel-generated samples and human-annotated responses, hindering trainingeffectiveness. To mitigate this issue, previous methods require additionalpreference annotation on newly generated samples to adapt to the shifteddistribution, which consumes a large amount of annotation resources. Targetingmore efficient human preference optimization, we propose an AdversarialPreference Optimization (APO) framework, in which the LLM and the reward modelupdate alternatively via a min-max game. Through adversarial training, thereward model can adapt to the shifted generation distribution of the LLMwithout any additional annotation. With comprehensive experiments, we find theproposed adversarial training framework further enhances existing alignmentbaselines in terms of LLM helpfulness and harmlessness. The code is athttps://github.com/Linear95/APO.

Adversarial Preference Optimization: Enhancing Your Alignment via RM-LLM Game