English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Mixup is a regularization technique that artificially produces new samplesusing convex combinations of original training points. This simple techniquehas shown strong empirical performance, and has been heavily used as part ofsemi-supervised learning techniques such asmixmatch~\citep{berthelot2019mixmatch} and interpolation consistent training(ICT)~\citep{verma2019interpolation}. In this paper, we look at Mixup through a\emph{representation learning} lens in a semi-supervised learning setup. Inparticular, we study the role of Mixup in promoting linearity in the learnednetwork representations. Towards this, we study two questions: (1) how does theMixup loss that enforces linearity in the \emph{last} network layer propagatethe linearity to the \emph{earlier} layers?; and (2) how does the enforcementof stronger Mixup loss on more than two data points affect the convergence oftraining? We empirically investigate these properties of Mixup on visiondatasets such as CIFAR-10, CIFAR-100 and SVHN. Our results show that supervisedMixup training does not make \emph{all} the network layers linear; in fact the\emph{intermediate layers} become more non-linear during Mixup trainingcompared to a network that is trained \emph{without} Mixup. However, when Mixupis used as an unsupervised loss, we observe that all the network layers becomemore linear resulting in faster training convergence.

When does mixup promote local linearity in learned representations?