English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Federated learning enables multiple distributed devices to collaborativelylearn a shared prediction model without centralizing their on-device data. Mostof the current algorithms require comparable individual efforts for localtraining with the same structure and size of on-device models, which, however,impedes participation from resource-constrained devices. Given the widespreadyet heterogeneous devices nowadays, in this paper, we propose an innovativefederated learning framework with heterogeneous on-device models throughZero-shot Knowledge Transfer, named by FedZKT. Specifically, FedZKT allowsdevices to independently determine the on-device models upon their localresources. To achieve knowledge transfer across these heterogeneous on-devicemodels, a zero-shot distillation approach is designed without any prerequisitesfor private on-device data, which is contrary to certain prior research basedon a public dataset or a pre-trained data generator. Moreover, thiscompute-intensive distillation task is assigned to the server to allow theparticipation of resource-constrained devices, where a generator isadversarially learned with the ensemble of collected on-device models. Thedistilled central knowledge is then sent back in the form of the correspondingon-device model parameters, which can be easily absorbed on the device side.Extensive experimental studies demonstrate the effectiveness and robustness ofFedZKT towards on-device knowledge agnostic, on-device model heterogeneity, andother challenging federated learning scenarios, such as heterogeneous on-devicedata and straggler effects.

FedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Models