English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Knowledge distillation has recently become popular as a method of modelaggregation on the server for federated learning. It is generally assumed thatthere are abundant public unlabeled data on the server. However, in reality,there exists a domain discrepancy between the datasets of the server domain anda client domain, which limits the performance of knowledge distillation. How toimprove the aggregation under such a domain discrepancy setting is still anopen problem. In this paper, we first analyze the generalization bound of theaggregation model produced from knowledge distillation for the client domains,and then describe two challenges, server-to-client discrepancy andclient-to-client discrepancy, brought to the aggregation model by the domaindiscrepancies. Following our analysis, we propose an adaptive knowledgeaggregation algorithm FedD3A based on domain discrepancy aware distillation tolower the bound. FedD3A performs adaptive weighting at the sample level in eachround of FL. For each sample in the server domain, only the client models ofits similar domains will be selected for playing the teacher role. To achievethis, we show that the discrepancy between the server-side sample and theclient domain can be approximately measured using a subspace projection matrixcalculated on each client without accessing its raw data. The server can thusleverage the projection matrices from multiple clients to assign weights to thecorresponding teacher models for each server-side sample. We validate FedD3A ontwo popular cross-domain datasets and show that it outperforms the comparedcompetitors in both cross-silo and cross-device FL settings.

Domain Discrepancy Aware Distillation for Model Aggregation in Federated Learning