English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Transformers have astounding representational power but typically consumeconsiderable computation which is quadratic with image resolution. Theprevailing Swin transformer reduces computational costs through a local windowstrategy. However, this strategy inevitably causes two drawbacks: (1) the localwindow-based self-attention hinders global dependency modeling capability; (2)recent studies point out that local windows impair robustness. To overcomethese challenges, we pursue a preferable trade-off between computational costand performance. Accordingly, we propose a novel factorization self-attentionmechanism (FaSA) that enjoys both the advantages of local window cost andlong-range dependency modeling capability. By factorizing the conventionalattention matrix into sparse sub-attention matrices, FaSA captures long-rangedependencies while aggregating mixed-grained information at a computationalcost equivalent to the local window-based self-attention. Leveraging FaSA, wepresent the factorization vision transformer (FaViT) with a hierarchicalstructure. FaViT achieves high performance and robustness, with linearcomputational complexity concerning input image spatial resolution. Extensiveexperiments have shown FaViT's advanced performance in classification anddownstream tasks. Furthermore, it also exhibits strong model robustness tocorrupted and biased data and hence demonstrates benefits in favor of practicalapplications. In comparison to the baseline model Swin-T, our FaViT-B2significantly improves classification accuracy by 1% and robustness by 7%,while reducing model parameters by 14%. Our code will soon be publiclyavailable at https://github.com/q2479036243/FaViT.

Factorization Vision Transformer: Modeling Long Range Dependency with Local Window Cost