English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Self-training and contrastive learning have emerged as leading techniques forincorporating unlabeled data, both under distribution shift (unsuperviseddomain adaptation) and when it is absent (semi-supervised learning). However,despite the popularity and compatibility of these techniques, their efficacy incombination remains unexplored. In this paper, we undertake a systematicempirical investigation of this combination, finding that (i) in domainadaptation settings, self-training and contrastive learning offer significantcomplementary gains; and (ii) in semi-supervised learning settings,surprisingly, the benefits are not synergistic. Across eight distribution shiftdatasets (e.g., BREEDs, WILDS), we demonstrate that the combined method obtains3--8% higher accuracy than either approach independently. We then theoreticallyanalyze these techniques in a simplified model of distribution shift,demonstrating scenarios under which the features produced by contrastivelearning can yield a good initialization for self-training to further amplifygains and achieve optimal performance, even when either method alone wouldfail.

Complementary Benefits of Contrastive Learning and Self-Training Under Distribution Shift