English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large-scale neural networks possess considerable expressive power. They arewell-suited for complex learning tasks in industrial applications. However,large-scale models pose significant challenges for training under the currentFederated Learning (FL) paradigm. Existing approaches for efficient FL trainingoften leverage model parameter dropout. However, manipulating individual modelparameters is not only inefficient in meaningfully reducing the communicationoverhead when training large-scale FL models, but may also be detrimental tothe scaling efforts and model performance as shown by recent research. Toaddress these issues, we propose the Federated Opportunistic Block Dropout(FedOBD) approach. The key novelty is that it decomposes large-scale modelsinto semantic blocks so that FL participants can opportunistically uploadquantized blocks, which are deemed to be significant towards training themodel, to the FL server for aggregation. Extensive experiments evaluatingFedOBD against four state-of-the-art approaches based on multiple real-worlddatasets show that it reduces the overall communication overhead by more than88% compared to the best performing baseline approach, while achieving thehighest test accuracy. To the best of our knowledge, FedOBD is the firstapproach to perform dropout on FL models at the block level rather than at theindividual parameter level.

FedOBD: Opportunistic Block Dropout for Efficiently Training Large-scale Neural Networks through Federated Learning