English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Building compositional explanations requires models to combine two or morefacts that, together, describe why the answer to a question is correct.Typically, these "multi-hop" explanations are evaluated relative to one (or asmall number of) gold explanations. In this work, we show these evaluationssubstantially underestimate model performance, both in terms of the relevanceof included facts, as well as the completeness of model-generated explanations,because models regularly discover and produce valid explanations that aredifferent than gold explanations. To address this, we construct a large corpusof 126k domain-expert (science teacher) relevance ratings that augment a corpusof explanations to standardized science exam questions, discovering 80kadditional relevant facts not rated as gold. We build three strong models basedon different methodologies (generation, ranking, and schemas), and empiricallyshow that while expert-augmented ratings provide better estimates ofexplanation quality, both original (gold) and expert-augmented automaticevaluations still substantially underestimate performance by up to 36% whencompared with full manual expert judgements, with different models beingdisproportionately affected. This poses a significant methodological challengeto accurately evaluating explanations produced by compositional reasoningmodels.

On the Challenges of Evaluating Compositional Explanations in Multi-Hop Inference: Relevance, Completeness, and Expert Ratings