English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The existence of spurious correlations such as image backgrounds in thetraining environment can make empirical risk minimization (ERM) perform badlyin the test environment. To address this problem, Kirichenko et al. (2022)empirically found that the core features that are related to the outcome canstill be learned well even with the presence of spurious correlations. Thisopens a promising strategy to first train a feature learner rather than aclassifier, and then perform linear probing (last layer retraining) in the testenvironment. However, a theoretical understanding of when and why this approachworks is lacking. In this paper, we find that core features are only learnedwell when their associated non-realizable noise is smaller than that ofspurious features, which is not necessarily true in practice. We provide boththeories and experiments to support this finding and to illustrate theimportance of non-realizable noise. Moreover, we propose an algorithm calledFreeze then Train (FTT), that first freezes certain salient features and thentrains the rest of the features using ERM. We theoretically show that FTTpreserves features that are more beneficial to test time probing. Across twocommonly used spurious correlation datasets, FTT outperforms ERM, IRM, JTT andCVaR-DRO, with substantial improvement in accuracy (by 4.5%) when the featurenoise is large. FTT also performs better on general distribution shiftbenchmarks.

Freeze then Train: Towards Provable Representation Learning under Spurious Correlations and Feature Noise