English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Self-supervised pretraining is the method of choice for natural languageprocessing models and is rapidly gaining popularity in many vision tasks.Recently, self-supervised pretraining has shown to outperform supervisedpretraining for many downstream vision applications, marking a milestone in thearea. This superiority is attributed to the negative impact of incompletelabelling of the training images, which convey multiple concepts, but areannotated using a single dominant class label. Although Self-SupervisedLearning (SSL), in principle, is free of this limitation, the choice of pretexttask facilitating SSL is perpetuating this shortcoming by driving the learningprocess towards a single concept output. This study aims to investigate thepossibility of modelling all the concepts present in an image without usinglabels. In this aspect the proposed SSL frame-work MC-SSL0.0 is a step towardsMulti-Concept Self-Supervised Learning (MC-SSL) that goes beyond modellingsingle dominant label in an image to effectively utilise the information fromall the concepts present in it. MC-SSL0.0 consists of two core design concepts,group masked model learning and learning of pseudo-concept for data token usinga momentum encoder (teacher-student) framework. The experimental results onmulti-label and multi-class image classification downstream tasks demonstratethat MC-SSL0.0 not only surpasses existing SSL methods but also outperformssupervised transfer learning. The source code will be made publicly availablefor community to train on bigger corpus.

MC-SSL0.0: Towards Multi-Concept Self-Supervised Learning