English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Given the increasingly prominent role NLP models (will) play in our lives, itis important for human expectations of model behavior to align with actualmodel behavior. Using Natural Language Inference (NLI) as a case study, weinvestigate the extent to which human-generated explanations of models'inference decisions align with how models actually make these decisions. Morespecifically, we define three alignment metrics that quantify how well naturallanguage explanations align with model sensitivity to input words, as measuredby integrated gradients. Then, we evaluate eight different models (the base andlarge versions of BERT, RoBERTa and ELECTRA, as well as anRNN and bag-of-wordsmodel), and find that the BERT-base model has the highest alignment withhuman-generated explanations, for all alignment metrics. Focusing in ontransformers, we find that the base versions tend to have higher alignment withhuman-generated explanations than their larger counterparts, suggesting thatincreasing the number of model parameters leads, in some cases, to worsealignment with human explanations. Finally, we find that a model's alignmentwith human explanations is not predicted by the model's accuracy, suggestingthat accuracy and alignment are complementary ways to evaluate models.

To what extent do human explanations of model behavior align with actual model behavior?