English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Previous work has proposed many new loss functions and regularizers thatimprove test accuracy on image classification tasks. However, it is not clearwhether these loss functions learn better representations for downstream tasks.This paper studies how the choice of training objective affects thetransferability of the hidden representations of convolutional neural networkstrained on ImageNet. We show that many objectives lead to statisticallysignificant improvements in ImageNet accuracy over vanilla softmaxcross-entropy, but the resulting fixed feature extractors transfersubstantially worse to downstream tasks, and the choice of loss has littleeffect when networks are fully fine-tuned on the new tasks. Using centeredkernel alignment to measure similarity between hidden representations ofnetworks, we find that differences among loss functions are apparent only inthe last few layers of the network. We delve deeper into representations of thepenultimate layer, finding that different objectives and hyperparametercombinations lead to dramatically different levels of class separation.Representations with higher class separation obtain higher accuracy on theoriginal task, but their features are less useful for downstream tasks. Ourresults suggest there exists a trade-off between learning invariant featuresfor the original task and features relevant for transfer tasks.

Why Do Better Loss Functions Lead to Less Transferable Features?