English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Systems that generate natural language text from abstract meaningrepresentations such as AMR are typically evaluated using automatic surfacematching metrics that compare the generated texts to reference texts from whichthe input meaning representations were constructed. We show that besideswell-known issues from which such metrics suffer, an additional problem ariseswhen applying these metrics for AMR-to-text evaluation, since an abstractmeaning representation allows for numerous surface realizations. In this workwe aim to alleviate these issues by proposing $\mathcal{M}\mathcal{F}_\beta$, adecomposable metric that builds on two pillars. The first is the principle ofmeaning preservation $\mathcal{M}$: it measures to what extent a given AMR canbe reconstructed from the generated sentence using SOTA AMR parsers andapplying (fine-grained) AMR evaluation metrics to measure the distance betweenthe original and the reconstructed AMR. The second pillar builds on a principleof (grammatical) form $\mathcal{F}$ that measures the linguistic quality of thegenerated text, which we implement using SOTA language models. In two extensivepilot studies we show that fulfillment of both principles offers benefits forAMR-to-text evaluation, including explainability of scores. Since$\mathcal{M}\mathcal{F}_\beta$ does not necessarily rely on gold AMRs, it mayextend to other text generation tasks.

Towards a Decomposable Metric for Explainable Evaluation of Text Generation from AMR