English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Decentralized learning has been advocated and widely deployed to makeefficient use of distributed datasets, with an extensive focus on supervisedlearning (SL) problems. Unfortunately, the majority of real-world data areunlabeled and can be highly heterogeneous across sources. In this work, wecarefully study decentralized learning with unlabeled data through the lens ofself-supervised learning (SSL), specifically contrastive visual representationlearning. We study the effectiveness of a range of contrastive learningalgorithms under decentralized learning settings, on relatively large-scaledatasets including ImageNet-100, MS-COCO, and a new real-world roboticwarehouse dataset. Our experiments show that the decentralized SSL (Dec-SSL)approach is robust to the heterogeneity of decentralized datasets, and learnsuseful representation for object classification, detection, and segmentationtasks. This robustness makes it possible to significantly reduce communicationand reduce the participation ratio of data sources with only minimal drops inperformance. Interestingly, using the same amount of data, the representationlearned by Dec-SSL can not only perform on par with that learned by centralizedSSL which requires communication and excessive data storage costs, but alsosometimes outperform representations extracted from decentralized SL whichrequires extra knowledge about the data labels. Finally, we provide theoreticalinsights into understanding why data heterogeneity is less of a concern forDec-SSL objectives, and introduce feature alignment and clustering techniquesto develop a new Dec-SSL algorithm that further improves the performance, inthe face of highly non-IID data. Our study presents positive evidence toembrace unlabeled data in decentralized learning, and we hope to provide newinsights into whether and why decentralized SSL is effective.

Does Learning from Decentralized Non-IID Unlabeled Data Benefit from Self Supervision?