English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Vision transformers (ViTs) have achieved remarkable performance in variouscomputer vision tasks. However, intensive memory and computation requirementsimpede ViTs from running on resource-constrained edge devices. Due to thenon-normally distributed values after Softmax and GeLU, post-trainingquantization on ViTs results in severe accuracy degradation. Moreover,conventional methods fail to address the high channel-wise variance inLayerNorm. To reduce the quantization loss and improve classification accuracy,we propose a two-scaled post-training quantization scheme for visiontransformer (TSPTQ-ViT). We design the value-aware two-scaled scaling factors(V-2SF) specialized for post-Softmax and post-GeLU values, which leverage thebit sparsity in non-normal distribution to save bit-widths. In addition, theoutlier-aware two-scaled scaling factors (O-2SF) are introduced to LayerNorm,alleviating the dominant impacts from outlier values. Our experimental resultsshow that the proposed methods reach near-lossless accuracy drops (<0.5%) onthe ImageNet classification task under 8-bit fully quantized ViTs.

TSPTQ-ViT: Two-scaled post-training quantization for vision transformer