English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Human preference judgments are pivotal in guiding large language models(LLMs) to produce outputs that align with human values. Human evaluations arealso used in summarization tasks to compare outputs from various systems,complementing existing automatic metrics. Despite their significance, however,there has been limited research probing these pairwise or $k$-wise comparisons.The collective impact and relative importance of factors such as output length,informativeness, fluency, and factual consistency are still not wellunderstood. It is also unclear if there are other hidden factors influencinghuman judgments. In this paper, we conduct an in-depth examination of acollection of pairwise human judgments released by OpenAI. Utilizing theBradley-Terry-Luce (BTL) model, we reveal the inherent preferences embedded inthese human judgments. We find that the most favored factors vary across tasksand genres, whereas the least favored factors tend to be consistent, e.g.,outputs are too brief, contain excessive off-focus content or hallucinatedfacts. Our findings have implications on the construction of balanced datasetsin human preference evaluations, which is a crucial step in shaping thebehaviors of future LLMs.

DecipherPref: Analyzing Influential Factors in Human Preference Judgments via GPT-4