English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Aligning large language models (LLMs) with human values and intentscritically involves the use of human or AI feedback. While dense feedbackannotations are expensive to acquire and integrate, sparse feedback presents astructural design choice between ratings (e.g., score Response A on a scale of1-7) and rankings (e.g., is Response A better than Response B?). In this work,we analyze the effect of this design choice for the alignment and evaluation ofLLMs. We uncover an inconsistency problem wherein the preferences inferred fromratings and rankings significantly disagree 60% for both human and AIannotators. Our subsequent analysis identifies various facets of annotatorbiases that explain this phenomena, such as human annotators would rate denserresponses higher while preferring accuracy during pairwise judgments. To oursurprise, we also observe that the choice of feedback protocol also has asignificant effect on the evaluation of aligned LLMs. In particular, we findthat LLMs that leverage rankings data for alignment (say model X) are preferredover those that leverage ratings data (say model Y), with a rank-basedevaluation protocol (is X/Y's response better than reference response?) but notwith a rating-based evaluation protocol (score Rank X/Y's response on a scaleof 1-7). Our findings thus shed light on critical gaps in methods forevaluating the real-world utility of language models and their strongdependence on the feedback protocol used for alignment. Our code and data areavailable at https://github.com/Hritikbansal/sparse_feedback.

Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models