English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

To avoid collapse in self-supervised learning (SSL), a contrastive loss iswidely used but often requires a large number of negative samples. Withoutnegative samples yet achieving competitive performance, a recent work hasattracted significant attention for providing a minimalist simple Siamese(SimSiam) method to avoid collapse. However, the reason for how it avoidscollapse without negative samples remains not fully clear and our investigationstarts by revisiting the explanatory claims in the original SimSiam. Afterrefuting their claims, we introduce vector decomposition for analyzing thecollapse based on the gradient analysis of the $l_2$-normalized representationvector. This yields a unified perspective on how negative samples and SimSiamalleviate collapse. Such a unified perspective comes timely for understandingthe recent progress in SSL.

How Does SimSiam Avoid Collapse Without Negative Samples? A Unified Understanding with Self-supervised Contrastive Learning