English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large language models (LLMs) often demonstrate inconsistencies with humanpreferences. Previous research typically gathered human preference data andthen aligned the pre-trained models using reinforcement learning or instructiontuning, a.k.a. the finetuning step. In contrast, aligning frozen LLMs withoutrequiring alignment data is more appealing. This work explores the potential ofthe latter setting. We discover that by integrating self-evaluation and rewindmechanisms, unaligned LLMs can directly produce responses consistent with humanpreferences via self-boosting. We introduce a novel inference method,Rewindable Auto-regressive INference (RAIN), that allows pre-trained LLMs toevaluate their own generation and use the evaluation results to guide rewindand generation for AI safety. Notably, RAIN operates without the need of extradata for model alignment and abstains from any training, gradient computation,or parameter updates. Experimental results evaluated by GPT-4 and humansdemonstrate the effectiveness of RAIN: on the HH dataset, RAIN improves theharmlessness rate of LLaMA 30B from 82% of vanilla inference to 97%, whilemaintaining the helpfulness rate. On the TruthfulQA dataset, RAIN improves thetruthfulness of the already-well-aligned LLaMA-2-chat 13B model by 5%.

RAIN: Your Language Models Can Align Themselves without Finetuning