English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Federated learning (FL) is a privacy-preserving machine learning paradigm inwhich the server periodically aggregates local model parameters from clientswithout assembling their private data.  Constrained communication and personalization requirements pose severechallenges to FL. Federated distillation (FD) is proposed to simultaneouslyaddress the above two problems, which exchanges knowledge between the serverand clients, supporting heterogeneous local models while significantly reducingcommunication overhead. However, most existing FD methods require a proxydataset, which is often unavailable in reality.  A few recent proxy-data-free FD approaches can eliminate the need foradditional public data, but suffer from remarkable discrepancy among localknowledge due to client-side model heterogeneity, leading to ambiguousrepresentation on the server and inevitable accuracy degradation.  To tackle this issue, we propose a proxy-data-free FD algorithm based ondistributed knowledge congruence (FedDKC). FedDKC leverages well-designedrefinement strategies to narrow local knowledge differences into an acceptableupper bound, so as to mitigate the negative effects of knowledge incongruence.  Specifically, from perspectives of peak probability and Shannon entropy oflocal knowledge, we design kernel-based knowledge refinement (KKR) andsearching-based knowledge refinement (SKR) respectively, and theoreticallyguarantee that the refined-local knowledge can satisfy an approximately-similardistribution and be regarded as congruent.  Extensive experiments conducted on three common datasets demonstrate that ourproposed FedDKC significantly outperforms the state-of-the-art on variousheterogeneous settings while evidently improving the convergence speed.

Exploring the Distributed Knowledge Congruence in Proxy-data-free Federated Distillation