English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

With the recent appearance of LLMs in practical settings, having methods thatcan effectively detect factual inconsistencies is crucial to reduce thepropagation of misinformation and improve trust in model outputs. When testingon existing factual consistency benchmarks, we find that a few large languagemodels (LLMs) perform competitively on classification benchmarks for factualinconsistency detection compared to traditional non-LLM methods. However, acloser analysis reveals that most LLMs fail on more complex formulations of thetask and exposes issues with existing evaluation benchmarks, affectingevaluation precision. To address this, we propose a new protocol forinconsistency detection benchmark creation and implement it in a 10-domainbenchmark called SummEdits. This new benchmark is 20 times more cost-effectiveper sample than previous benchmarks and highly reproducible, as we estimateinter-annotator agreement at about 0.9. Most LLMs struggle on SummEdits, withperformance close to random chance. The best-performing model, GPT-4, is still8\% below estimated human performance, highlighting the gaps in LLMs' abilityto reason about facts and detect inconsistencies when they occur.

LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond