English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent vision architectures and self-supervised training methods enablevision models that are extremely accurate and general, but come with massiveparameter and computational costs. In practical settings, such as camera traps,users have limited resources, and may fine-tune a pretrained model on (oftenlimited) data from a small set of specific categories of interest. These usersmay wish to make use of modern, highly-accurate models, but are oftencomputationally constrained. To address this, we ask: can we quickly compresslarge generalist models into accurate and efficient specialists? For this, wepropose a simple and versatile technique called Few-Shot Task-Aware Compression(TACO). Given a large vision model that is pretrained to be accurate on a broadtask, such as classification over ImageNet-22K, TACO produces a smaller modelthat is accurate on specialized tasks, such as classification across vehicletypes or animal species. Crucially, TACO works in few-shot fashion, i.e. only afew task-specific samples are used, and the procedure has low computationaloverheads. We validate TACO on highly-accurate ResNet, ViT/DeiT, and ConvNeXtmodels, originally trained on ImageNet, LAION, or iNaturalist, which wespecialize and compress to a diverse set of "downstream" subtasks. TACO canreduce the number of non-zero parameters in existing models by up to 20xrelative to the original models, leading to inference speedups of up to3$\times$, while remaining accuracy-competitive with the uncompressed models onthe specialized tasks.

Vision Models Can Be Efficiently Specialized via Few-Shot Task-Aware Compression