English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Neural networks trained with SGD were recently shown to rely preferentiallyon linearly-predictive features and can ignore complex, equally-predictiveones. This simplicity bias can explain their lack of robustness out ofdistribution (OOD). The more complex the task to learn, the more likely it isthat statistical artifacts (i.e. selection biases, spurious correlations) aresimpler than the mechanisms to learn. We demonstrate that the simplicity biascan be mitigated and OOD generalization improved. We train a set of similarmodels to fit the data in different ways using a penalty on the alignment oftheir input gradients. We show theoretically and empirically that this inducesthe learning of more complex predictive patterns. OOD generalizationfundamentally requires information beyond i.i.d. examples, such as multipletraining environments, counterfactual examples, or other side information. Ourapproach shows that we can defer this requirement to an independent modelselection stage. We obtain SOTA results in visual recognition on biased dataand generalization across visual domains. The method - the first to evade thesimplicity bias - highlights the need for a better understanding and control ofinductive biases in deep learning.

Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization