English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The theoretical landscape of federated learning (FL) undergoes rapidevolution, but its practical application encounters a series of intricatechallenges, and hyperparameter optimization is one of these criticalchallenges. Amongst the diverse adjustments in hyperparameters, the adaptationof the learning rate emerges as a crucial component, holding the promise ofsignificantly enhancing the efficacy of FL systems. In response to thiscritical need, this paper presents FedHyper, a novel hypergradient-basedlearning rate adaptation algorithm specifically designed for FL. FedHyperserves as a universal learning rate scheduler that can adapt both global andlocal rates as the training progresses. In addition, FedHyper not onlyshowcases unparalleled robustness to a spectrum of initial learning rateconfigurations but also significantly alleviates the necessity for laboriousempirical learning rate adjustments. We provide a comprehensive theoreticalanalysis of FedHyper's convergence rate and conduct extensive experiments onvision and language benchmark datasets. The results demonstrate that FEDHYPERconsistently converges 1.1-3x faster than FedAvg and the competing baselineswhile achieving superior final accuracy. Moreover, FedHyper catalyzes aremarkable surge in accuracy, augmenting it by up to 15% compared to FedAvgunder suboptimal initial learning rate settings.

FedHyper: A Universal and Robust Learning Rate Scheduler for Federated Learning with Hypergradient Descent