English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Self-supervised learning of deep neural networks has become a prevalentparadigm for learning representations that transfer to a variety of downstreamtasks. Similar to proposed models of the ventral stream of biological vision,it is observed that these networks lead to a separation of category manifoldsin the representations of the penultimate layer. Although this observationmatches the manifold hypothesis of representation learning, currentself-supervised approaches are limited in their ability to explicitly modelthis manifold. Indeed, current approaches often only apply augmentations from apre-specified set of "positive pairs" during learning. In this work, we proposea contrastive learning approach that directly models the latent manifold usingLie group operators parameterized by coefficients with a sparsity-promotingprior. A variational distribution over these coefficients provides a generativemodel of the manifold, with samples which provide feature augmentationsapplicable both during contrastive training and downstream tasks. Additionally,learned coefficient distributions provide a quantification of whichtransformations are most likely at each point on the manifold while preservingidentity. We demonstrate benefits in self-supervised benchmarks for imagedatasets, as well as a downstream semi-supervised task. In the former case, wedemonstrate that the proposed methods can effectively apply manifold featureaugmentations and improve learning both with and without a projection head. Inthe latter case, we demonstrate that feature augmentations sampled from learnedLie group operators can improve classification performance when using fewlabels.

Manifold Contrastive Learning with Variational Lie Group Operators