English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this work we study generalization of neural networks in gradient-basedmeta-learning by analyzing various properties of the objective landscapes. Weexperimentally demonstrate that as meta-training progresses, the meta-testsolutions, obtained after adapting the meta-train solution of the model, to newtasks via few steps of gradient-based fine-tuning, become flatter, lower inloss, and further away from the meta-train solution. We also show that thosemeta-test solutions become flatter even as generalization starts to degrade,thus providing an experimental evidence against the correlation betweengeneralization and flat minima in the paradigm of gradient-based meta-leaning.Furthermore, we provide empirical evidence that generalization to new tasks iscorrelated with the coherence between their adaptation trajectories inparameter space, measured by the average cosine similarity betweentask-specific trajectory directions, starting from a same meta-train solution.We also show that coherence of meta-test gradients, measured by the averageinner product between the task-specific gradient vectors evaluated atmeta-train solution, is also correlated with generalization. Based on theseobservations, we propose a novel regularizer for MAML and provide experimentalevidence for its effectiveness.

Towards Understanding Generalization in Gradient-Based Meta-Learning