English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Vision Transformers (ViTs) have underpinned the recent breakthroughs incomputer vision. However, designing the architectures of ViTs is laborious andheavily relies on expert knowledge. To automate the design process andincorporate deployment flexibility, one-shot neural architecture searchdecouples the supernet training and architecture specialization for diversedeployment scenarios. To cope with an enormous number of sub-networks in thesupernet, existing methods treat all architectures equally important andrandomly sample some of them in each update step during training. Duringarchitecture search, these methods focus on finding architectures on the Paretofrontier of performance and resource consumption, which forms a gap betweentraining and deployment. In this paper, we devise a simple yet effectivemethod, called FocusFormer, to bridge such a gap. To this end, we propose tolearn an architecture sampler to assign higher sampling probabilities to thosearchitectures on the Pareto frontier under different resource constraintsduring supernet training, making them sufficiently optimized and henceimproving their performance. During specialization, we can directly use thewell-trained architecture sampler to obtain accurate architectures satisfyingthe given resource constraint, which significantly improves the searchefficiency. Extensive experiments on CIFAR-100 and ImageNet show that ourFocusFormer is able to improve the performance of the searched architectureswhile significantly reducing the search cost. For example, on ImageNet, ourFocusFormer-Ti with 1.4G FLOPs outperforms AutoFormer-Ti by 0.5% in terms ofthe Top-1 accuracy.

FocusFormer: Focusing on What We Need via Architecture Sampler