English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Neural network pruning is an essential technique for reducing the size andcomplexity of deep neural networks, enabling large-scale models on devices withlimited resources. However, existing pruning approaches heavily rely ontraining data for guiding the pruning strategies, making them ineffective forfederated learning over distributed and confidential datasets. Additionally,the memory- and computation-intensive pruning process becomes infeasible forrecourse-constrained devices in federated learning. To address thesechallenges, we propose FedTiny, a distributed pruning framework for federatedlearning that generates specialized tiny models for memory- andcomputing-constrained devices. We introduce two key modules in FedTiny toadaptively search coarse- and finer-pruned specialized models to fit deploymentscenarios with sparse and cheap local computation. First, an adaptive batchnormalization selection module is designed to mitigate biases in pruning causedby the heterogeneity of local data. Second, a lightweight progressive pruningmodule aims to finer prune the models under strict memory and computationalbudgets, allowing the pruning policy for each layer to be gradually determinedrather than evaluating the overall model structure. The experimental resultsdemonstrate the effectiveness of FedTiny, which outperforms state-of-the-artapproaches, particularly when compressing deep models to extremely sparse tinymodels. FedTiny achieves an accuracy improvement of 2.61% while significantlyreducing the computational cost by 95.91% and the memory footprint by 94.01%compared to state-of-the-art methods.

Distributed Pruning Towards Tiny Neural Networks in Federated Learning