English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Dataset Distillation is a newly emerging area that aims to distill largedatasets into much smaller and highly informative synthetic ones to acceleratetraining and reduce storage. Among various dataset distillation methods,trajectory-matching-based methods (MTT) have achieved SOTA performance in manytasks, e.g., on CIFAR-10/100. However, due to exorbitant memory consumptionwhen unrolling optimization through SGD steps, MTT fails to scale tolarge-scale datasets such as ImageNet-1K. Can we scale this SOTA method toImageNet-1K and does its effectiveness on CIFAR transfer to ImageNet-1K? Toanswer these questions, we first propose a procedure to exactly compute theunrolled gradient with constant memory complexity, which allows us to scale MTTto ImageNet-1K seamlessly with ~6x reduction in memory footprint. We furtherdiscover that it is challenging for MTT to handle datasets with a large numberof classes, and propose a novel soft label assignment that drastically improvesits convergence. The resulting algorithm sets new SOTA on ImageNet-1K: we canscale up to 50 IPCs (Image Per Class) on ImageNet-1K on a single GPU (allprevious methods can only scale to 2 IPCs on ImageNet-1K), leading to the bestaccuracy (only 5.9% accuracy drop against full dataset training) whileutilizing only 4.2% of the number of data points - an 18.2% absolute gain overprior SOTA. Our code is available at https://github.com/justincui03/tesla

Scaling Up Dataset Distillation to ImageNet-1K with Constant Memory