English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Contrastive learning is a powerful framework for learning self-supervisedrepresentations that generalize well to downstream supervised tasks. We showthat multiple existing contrastive learning methods can be reinterpreted aslearning kernel functions that approximate a fixed positive-pair kernel. Wethen prove that a simple representation obtained by combining this kernel withPCA provably minimizes the worst-case approximation error of linear predictors,under a straightforward assumption that positive pairs have similar labels. Ouranalysis is based on a decomposition of the target function in terms of theeigenfunctions of a positive-pair Markov chain, and a surprising equivalencebetween these eigenfunctions and the output of Kernel PCA. We givegeneralization bounds for downstream linear prediction using our Kernel PCArepresentation, and show empirically on a set of synthetic tasks that applyingKernel PCA to contrastive learning models can indeed approximately recover theMarkov chain eigenfunctions, although the accuracy depends on the kernelparameterization as well as on the augmentation strength.

Contrastive Learning Can Find An Optimal Basis For Approximately View-Invariant Functions