English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Due to the complex attention mechanisms and model design, most existingvision Transformers (ViTs) can not perform as efficiently as convolutionalneural networks (CNNs) in realistic industrial deployment scenarios, e.g.TensorRT and CoreML. This poses a distinct challenge: Can a visual neuralnetwork be designed to infer as fast as CNNs and perform as powerful as ViTs?Recent works have tried to design CNN-Transformer hybrid architectures toaddress this issue, yet the overall performance of these works is far away fromsatisfactory. To end these, we propose a next generation vision Transformer forefficient deployment in realistic industrial scenarios, namely Next-ViT, whichdominates both CNNs and ViTs from the perspective of latency/accuracytrade-off. In this work, the Next Convolution Block (NCB) and Next TransformerBlock (NTB) are respectively developed to capture local and global informationwith deployment-friendly mechanisms. Then, Next Hybrid Strategy (NHS) isdesigned to stack NCB and NTB in an efficient hybrid paradigm, which boostsperformance in various downstream tasks. Extensive experiments show thatNext-ViT significantly outperforms existing CNNs, ViTs and CNN-Transformerhybrid architectures with respect to the latency/accuracy trade-off acrossvarious vision tasks. On TensorRT, Next-ViT surpasses ResNet by 5.5 mAP (from40.4 to 45.9) on COCO detection and 7.7% mIoU (from 38.8% to 46.5%) on ADE20Ksegmentation under similar latency. Meanwhile, it achieves comparableperformance with CSWin, while the inference speed is accelerated by 3.6x. OnCoreML, Next-ViT surpasses EfficientFormer by 4.6 mAP (from 42.6 to 47.2) onCOCO detection and 3.5% mIoU (from 45.1% to 48.6%) on ADE20K segmentation undersimilar latency. Our code and models are made public at:https://github.com/bytedance/Next-ViT

Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarios