English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We introduce RoMQA, the first benchmark for robust, multi-evidence,multi-answer question answering (QA). RoMQA contains clusters of questions thatare derived from related constraints mined from the Wikidata knowledge graph.RoMQA evaluates robustness of QA models to varying constraints by measuringworst-case performance within each question cluster. Compared to prior QAdatasets, RoMQA has more human-written questions that require reasoning overmore evidence text and have, on average, many more correct answers. Inaddition, human annotators rate RoMQA questions as more natural or likely to beasked by people. We evaluate state-of-the-art large language models inzero-shot, few-shot, and fine-tuning settings, and find that RoMQA ischallenging: zero-shot and few-shot models perform similarly to naivebaselines, while supervised retrieval methods perform well below gold evidenceupper bounds. Moreover, existing models are not robust to variations inquestion constraints, but can be made more robust by tuning on clusters ofrelated questions. Our results show that RoMQA is a challenging benchmark forlarge language models, and provides a quantifiable test to build more robust QAmethods.

RoMQA: A Benchmark for Robust, Multi-evidence, Multi-answer Question Answering