English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Neural networks trained with (stochastic) gradient descent have an inductivebias towards learning simpler solutions. This makes them highly prone tolearning simple spurious features that are highly correlated with a labelinstead of the predictive but more complex core features. In this work, we showthat, interestingly, the simplicity bias of gradient descent can be leveragedto identify spurious correlations, early in training. First, we prove on atwo-layer neural network, that groups of examples with high spuriouscorrelation are separable based on the model's output, in the initial trainingiterations. We further show that if spurious features have a small enoughnoise-to-signal ratio, the network's output on the majority of examples in aclass will be almost exclusively determined by the spurious features and willbe nearly invariant to the core feature. Finally, we propose SPARE, whichseparates large groups with spurious correlations early in training, andutilizes importance sampling to alleviate the spurious correlation, bybalancing the group sizes. We show that SPARE achieves up to 5.6% higherworst-group accuracy than state-of-the-art methods, while being up to 12xfaster. We also show the applicability of SPARE to discover and mitigatespurious correlations in Restricted ImageNet.

Identifying Spurious Biases Early in Training through the Lens of Simplicity Bias