English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This work presents a new method for enhancing communication efficiency instochastic Federated Learning that trains over-parameterized random networks.In this setting, a binary mask is optimized instead of the model weights, whichare kept fixed. The mask characterizes a sparse sub-network that is able togeneralize as good as a smaller target network. Importantly, sparse binarymasks are exchanged rather than the floating point weights in traditionalfederated learning, reducing communication cost to at most 1 bit per parameter.We show that previous state of the art stochastic methods fail to find thesparse networks that can reduce the communication and storage overhead usingconsistent loss objectives. To address this, we propose adding a regularizationterm to local objectives that encourages sparser solutions by eliminatingredundant features across sub-networks. Extensive experiments demonstratesignificant improvements in communication and memory efficiency of up to fivemagnitudes compared to the literature, with minimal performance degradation invalidation accuracy in some instances.

Sparser Random Networks Exist: Enforcing Communication-Efficient Federated Learning via Regularization