English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent studies have integrated convolution into transformers to introduceinductive bias and improve generalization performance. However, the staticnature of conventional convolution prevents it from dynamically adapting toinput variations, resulting in a representation discrepancy between convolutionand self-attention as self-attention calculates attention matrices dynamically.Furthermore, when stacking token mixers that consist of convolution andself-attention to form a deep network, the static nature of convolution hindersthe fusion of features previously generated by self-attention into convolutionkernels. These two limitations result in a sub-optimal representation capacityof the constructed networks. To find a solution, we propose a lightweight DualDynamic Token Mixer (D-Mixer) that aggregates global information and localdetails in an input-dependent way. D-Mixer works by applying an efficientglobal attention module and an input-dependent depthwise convolution separatelyon evenly split feature segments, endowing the network with strong inductivebias and an enlarged effective receptive field. We use D-Mixer as the basicbuilding block to design TransXNet, a novel hybrid CNN-Transformer visionbackbone network that delivers compelling performance. In the ImageNet-1K imageclassification task, TransXNet-T surpasses Swin-T by 0.3% in top-1 accuracywhile requiring less than half of the computational cost. Furthermore,TransXNet-S and TransXNet-B exhibit excellent model scalability, achievingtop-1 accuracy of 83.8% and 84.6% respectively, with reasonable computationalcosts. Additionally, our proposed network architecture demonstrates stronggeneralization capabilities in various dense prediction tasks, outperformingother state-of-the-art networks while having lower computational costs. Code isavailable at https://github.com/LMMMEng/TransXNet.

TransXNet: Learning Both Global and Local Dynamics with a Dual Dynamic Token Mixer for Visual Recognition