English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Common explanations for shortcut learning assume that the shortcut improvesprediction under the training distribution but not in the test distribution.Thus, models trained via the typical gradient-based optimization ofcross-entropy, which we call default-ERM, utilize the shortcut. However, evenwhen the stable feature determines the label in the training distribution andthe shortcut does not provide any additional information, like in perceptiontasks, default-ERM still exhibits shortcut learning. Why are such solutionspreferred when the loss for default-ERM can be driven to zero using the stablefeature alone? By studying a linear perception task, we show that default-ERM'spreference for maximizing the margin leads to models that depend more on theshortcut than the stable feature, even without overparameterization. Thisinsight suggests that default-ERM's implicit inductive bias towards max-marginis unsuitable for perception tasks. Instead, we develop an inductive biastoward uniform margins and show that this bias guarantees dependence only onthe perfect stable feature in the linear perception task. We develop lossfunctions that encourage uniform-margin solutions, called margin control(MARG-CTRL). MARG-CTRL mitigates shortcut learning on a variety of vision andlanguage tasks, showing that better inductive biases can remove the need forexpensive two-stage shortcut-mitigating methods in perception tasks.

Don't blame Dataset Shift! Shortcut Learning due to Gradients and Cross Entropy