English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Federated learning (FL) is a promising paradigm to enable privacy-preservingdeep learning from distributed data. Most previous works are based on federatedaverage (FedAvg), which, however, faces several critical issues, including ahigh communication overhead and the difficulty in dealing with heterogeneousmodel architectures. Federated Distillation (FD) is a recently proposedalternative to enable communication-efficient and robust FL, which achievesorders of magnitude reduction of the communication overhead compared withFedAvg and is flexible to handle heterogeneous models at the clients. However,so far there is no unified algorithmic framework or theoretical analysis forFD-based methods. In this paper, we first present a generic meta-algorithm forFD and investigate the influence of key parameters through empiricalexperiments. Then, we verify the empirical observations theoretically. Based onthe empirical results and theory, we propose a communication-efficient FDalgorithm with active data sampling to improve the model performance and reducethe communication overhead. Empirical simulations on benchmark datasets willdemonstrate that our proposed algorithm effectively and significantly reducesthe communication overhead while achieving a satisfactory performance.

Communication-Efficient Federated Distillation with Active Data Sampling