English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper studies few-shot learning via representation learning, where oneuses $T$ source tasks with $n_1$ data per task to learn a representation inorder to reduce the sample complexity of a target task for which there is only$n_2 (\ll n_1)$ data. Specifically, we focus on the setting where there existsa good \emph{common representation} between source and target, and our goal isto understand how much of a sample size reduction is possible. First, we studythe setting where this common representation is low-dimensional and provide afast rate of $O\left(\frac{\mathcal{C}\left(\Phi\right)}{n_1T} +\frac{k}{n_2}\right)$; here, $\Phi$ is the representation function class,$\mathcal{C}\left(\Phi\right)$ is its complexity measure, and $k$ is thedimension of the representation. When specialized to linear representationfunctions, this rate becomes $O\left(\frac{dk}{n_1T} + \frac{k}{n_2}\right)$where $d (\gg k)$ is the ambient input dimension, which is a substantialimprovement over the rate without using representation learning, i.e. over therate of $O\left(\frac{d}{n_2}\right)$. This result bypasses the$\Omega(\frac{1}{T})$ barrier under the i.i.d. task assumption, and can capturethe desired property that all $n_1T$ samples from source tasks can be\emph{pooled} together for representation learning. Next, we consider thesetting where the common representation may be high-dimensional but iscapacity-constrained (say in norm); here, we again demonstrate the advantage ofrepresentation learning in both high-dimensional linear regression and neuralnetwork learning. Our results demonstrate representation learning can fullyutilize all $n_1T$ samples from source tasks.

Few-Shot Learning via Learning the Representation, Provably