English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

State-of-the-art federated learning methods can perform far worse than theircentralized counterparts when clients have dissimilar data distributions. Forneural networks, even when centralized SGD easily finds a solution that issimultaneously performant for all clients, current federated optimizationmethods fail to converge to a comparable solution. We show that thisperformance disparity can largely be attributed to optimization challengespresented by nonconvexity. Specifically, we find that the early layers of thenetwork do learn useful features, but the final layers fail to make use ofthem. That is, federated optimization applied to this non-convex problemdistorts the learning of the final layers. Leveraging this observation, wepropose a Train-Convexify-Train (TCT) procedure to sidestep this issue: first,learn features using off-the-shelf methods (e.g., FedAvg); then, optimize aconvexified problem obtained from the network's empirical neural tangent kernelapproximation. Our technique yields accuracy improvements of up to +36% onFMNIST and +37% on CIFAR10 when clients have dissimilar data.

TCT: Convexifying Federated Learning using Bootstrapped Neural Tangent Kernels