English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Although model-agnostic meta-learning (MAML) is a very successful algorithmin meta-learning practice, it can have high computational cost because itupdates all model parameters over both the inner loop of task-specificadaptation and the outer-loop of meta initialization training. A more efficientalgorithm ANIL (which refers to almost no inner loop) was proposed recently byRaghu et al. 2019, which adapts only a small subset of parameters in the innerloop and thus has substantially less computational cost than MAML asdemonstrated by extensive experiments. However, the theoretical convergence ofANIL has not been studied yet. In this paper, we characterize the convergencerate and the computational complexity for ANIL under two representativeinner-loop loss geometries, i.e., strongly-convexity and nonconvexity. Ourresults show that such a geometric property can significantly affect theoverall convergence performance of ANIL. For example, ANIL achieves a fasterconvergence rate for a strongly-convex inner-loop loss as the number $N$ ofinner-loop gradient descent steps increases, but a slower convergence rate fora nonconvex inner-loop loss as $N$ increases. Moreover, our complexity analysisprovides a theoretical quantification on the improved efficiency of ANIL overMAML. The experiments on standard few-shot meta-learning benchmarks validateour theoretical findings.

Convergence of Meta-Learning with Task-Specific Adaptation over Partial Parameters