English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Data collection for natural language (NL) understanding tasks hasincreasingly included human explanations alongside data points, allowing pastworks to introduce models that both perform a task and generate NL explanationsfor their outputs. Yet to date, model-generated explanations have beenevaluated on the basis of surface-level similarities to human explanations,both through automatic metrics like BLEU and human evaluations. We argue thatthese evaluations are insufficient, since they fail to indicate whetherexplanations support actual model behavior (faithfulness), rather than simplymatch what a human would say (plausibility). In this work, we address theproblem of evaluating explanations from the model simulatability perspective.Our contributions are as follows: (1) We introduce a leakage-adjustedsimulatability (LAS) metric for evaluating NL explanations, which measures howwell explanations help an observer predict a model's output, while controllingfor how explanations can directly leak the output. We use a model as a proxyfor a human observer, and validate this choice with two human subjectexperiments. (2) Using the CoS-E and e-SNLI datasets, we evaluate two existinggenerative graphical models and two new approaches; one rationalizing method weintroduce achieves roughly human-level LAS scores. (3) Lastly, we frameexplanation generation as a multi-agent game and optimize explanations forsimulatability while penalizing label leakage, which can improve LAS scores. Weprovide code for the experiments in this paper athttps://github.com/peterbhase/LAS-NL-Explanations

Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?