English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Retrieval-augmented generation (RAG) grounds large language model (LLM)output by leveraging external knowledge sources to reduce factualhallucinations. However, prior works lack a comprehensive evaluation ofdifferent language families, making it challenging to evaluate LLM robustnessagainst errors in external retrieved knowledge. To overcome this, we establishNoMIRACL, a human-annotated dataset for evaluating LLM robustness in RAG across18 typologically diverse languages. NoMIRACL includes both a non-relevant and arelevant subset. Queries in the non-relevant subset contain passages manuallyjudged as non-relevant or noisy, whereas queries in the relevant subset includeat least a single judged relevant passage. We measure LLM robustness using twometrics: (i) hallucination rate, measuring model tendency to hallucinate ananswer, when the answer is not present in passages in the non-relevant subset,and (ii) error rate, measuring model inaccuracy to recognize relevant passagesin the relevant subset. We build a GPT-4 baseline which achieves a 33.2%hallucination rate on the non-relevant and a 14.9% error rate on the relevantsubset on average. Our evaluation reveals that GPT-4 hallucinates frequently inhigh-resource languages, such as French or English. This work highlights animportant avenue for future research to improve LLM robustness to learn how tobetter reject non-relevant information in RAG.

NoMIRACL: Knowing When You Don't Know for Robust Multilingual Retrieval-Augmented Generation