English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Reference-based metrics such as ROUGE or BERTScore evaluate the contentquality of a summary by comparing the summary to a reference. Ideally, thiscomparison should measure the summary's information quality by calculating howmuch information the summaries have in common. In this work, we analyze thetoken alignments used by ROUGE and BERTScore to compare summaries and arguethat their scores largely cannot be interpreted as measuring informationoverlap, but rather the extent to which they discuss the same topics. Further,we provide evidence that this result holds true for many other summarizationevaluation metrics. The consequence of this result is that it means thesummarization community has not yet found a reliable automatic metric thataligns with its research goal, to generate summaries with high-qualityinformation. Then, we propose a simple and interpretable method of evaluatingsummaries which does directly measure information overlap and demonstrate howit can be used to gain insights into model behavior that could not be providedby other methods alone.

Understanding the Extent to which Summarization Evaluation Metrics Measure the Information Quality of Summaries