English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent studies indicate that NLU models are prone to rely on shortcutfeatures for prediction, without achieving true language understanding. As aresult, these models fail to generalize to real-world out-of-distribution data.In this work, we show that the words in the NLU training set can be modeled asa long-tailed distribution. There are two findings: 1) NLU models have strongpreference for features located at the head of the long-tailed distribution,and 2) Shortcut features are picked up during very early few iterations of themodel training. These two observations are further employed to formulate ameasurement which can quantify the shortcut degree of each training sample.Based on this shortcut measurement, we propose a shortcut mitigation frameworkLTGR, to suppress the model from making overconfident predictions for sampleswith large shortcut degree. Experimental results on three NLU benchmarksdemonstrate that our long-tailed distribution explanation accurately reflectsthe shortcut learning behavior of NLU models. Experimental analysis furtherindicates that LTGR can improve the generalization accuracy on OOD data, whilepreserving the accuracy on in-distribution data.

Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models