English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In recent years, decentralized bilevel optimization problems have receivedincreasing attention in the networking and machine learning communities thanksto their versatility in modeling decentralized learning problems overpeer-to-peer networks (e.g., multi-agent meta-learning, multi-agentreinforcement learning, personalized training, and Byzantine-resilientlearning). However, for decentralized bilevel optimization over peer-to-peernetworks with limited computation and communication capabilities, how toachieve low sample and communication complexities are two fundamentalchallenges that remain under-explored so far. In this paper, we make the firstattempt to investigate the class of decentralized bilevel optimization problemswith nonconvex and strongly-convex structure corresponding to the outer andinner subproblems, respectively. Our main contributions in this paper aretwo-fold: i) We first propose a deterministic algorithm called INTERACT(inner-gradient-descent-outer-tracked-gradient) that requires the samplecomplexity of $\mathcal{O}(n \epsilon^{-1})$ and communication complexity of$\mathcal{O}(\epsilon^{-1})$ to solve the bilevel optimization problem, where$n$ and $\epsilon > 0$ are the number of samples at each agent and the desiredstationarity gap, respectively. ii) To relax the need for full gradientevaluations in each iteration, we propose a stochastic variance-reduced versionof INTERACT (SVR-INTERACT), which improves the sample complexity to$\mathcal{O}(\sqrt{n} \epsilon^{-1})$ while achieving the same communicationcomplexity as the deterministic algorithm. To our knowledge, this work is thefirst that achieves both low sample and communication complexities for solvingdecentralized bilevel optimization problems over networks. Our numericalexperiments also corroborate our theoretical findings.

INTERACT: Achieving Low Sample and Communication Complexities in Decentralized Bilevel Learning over Networks