English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study deep neural networks for the multi-label classification (MLab) taskthrough the lens of neural collapse (NC). Previous works have been restrictedto the multi-class classification setting and discovered a prevalent NCphenomenon comprising of the following properties for the last-layer features:(i) the variability of features within every class collapses to zero, (ii) theset of feature means form an equi-angular tight frame (ETF), and (iii) the lastlayer classifiers collapse to the feature mean upon some scaling. We generalizethe study to multi-label learning, and prove for the first time that ageneralized NC phenomenon holds with the "pick-all-label" formulation. Underthe natural analog of the unconstrained feature model (UFM), we establish thatthe only global classifier of the pick-all-label cross entropy loss display thesame ETF geometry which further collapse to multiplicity-1 feature class means.Besides, we discover a combinatorial property in generalized NC which is uniquefor multi-label learning that we call "tag-wise average" property, where thefeature class-means of samples with multiple labels are scaled average of thefeature class-means of single label tags. Theoretically, we establish globaloptimality result for the pick-all-label cross-entropy risk for the UFM.Additionally, We also provide empirical evidence to support our investigationinto training deep neural networks on multi-label datasets, resulting inimproved training efficiency.

Neural Collapse in Multi-label Learning with Pick-all-label Loss