English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Joint Embedding Self-Supervised Learning (JE-SSL) has seen rapid developmentsin recent years, due to its promise to effectively leverage large unlabeleddata. The development of JE-SSL methods was driven primarily by the search forever increasing downstream classification accuracies, using huge computationalresources, and typically built upon insights and intuitions inherited from aclose parent JE-SSL method. This has led unwittingly to numerous pre-conceivedideas that carried over across methods e.g. that SimCLR requires very largemini batches to yield competitive accuracies; that strong and computationallyslow data augmentations are required. In this work, we debunk several suchill-formed a priori ideas in the hope to unleash the full potential of JE-SSLfree of unnecessary limitations. In fact, when carefully evaluatingperformances across different downstream tasks and properly optimizinghyper-parameters of the methods, we most often -- if not always -- see thatthese widespread misconceptions do not hold. For example we show that it ispossible to train SimCLR to learn useful representations, while using a singleimage patch as negative example, and simple Gaussian noise as the only dataaugmentation for the positive pair. Along these lines, in the hope todemocratize JE-SSL and to allow researchers to easily make more extensiveevaluations of their methods, we introduce an optimized PyTorch library forSSL.

Towards Democratizing Joint-Embedding Self-Supervised Learning