English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Federated Learning (FL) has been gaining significant traction acrossdifferent ML tasks, ranging from vision to keyboard predictions. In large-scaledeployments, client heterogeneity is a fact and constitutes a primary problemfor fairness, training performance and accuracy. Although significant effortshave been made into tackling statistical data heterogeneity, the diversity inthe processing capabilities and network bandwidth of clients, termed as systemheterogeneity, has remained largely unexplored. Current solutions eitherdisregard a large portion of available devices or set a uniform limit on themodel's capacity, restricted by the least capable participants. In this work,we introduce Ordered Dropout, a mechanism that achieves an ordered, nestedrepresentation of knowledge in deep neural networks (DNNs) and enables theextraction of lower footprint submodels without the need of retraining. Wefurther show that for linear maps our Ordered Dropout is equivalent to SVD. Weemploy this technique, along with a self-distillation methodology, in the realmof FL in a framework called FjORD. FjORD alleviates the problem of clientsystem heterogeneity by tailoring the model width to the client's capabilities.Extensive evaluation on both CNNs and RNNs across diverse modalities shows thatFjORD consistently leads to significant performance gains over state-of-the-artbaselines, while maintaining its nested structure.

FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout