English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

The recent success of multiple neural architectures like CNNs, Transformers,and MLP-Mixers motivated us to look for similarities and differences betweenthem. We found that these architectures can be interpreted through the lens ofa general concept of dimension mixing. Research on coupling flows and thebutterfly transform shows that partial and hierarchical signal mixing schemesare sufficient for efficient and expressive function approximation. In thiswork, we study group-wise sparse, non-linear, multi-layered and learnablemixing schemes of inputs and find that they are complementary to many standardneural architectures. Following our observations and drawing inspiration fromthe Fast Fourier Transform, we generalize Butterfly Structure to use non-linearmixer function allowing for MLP as mixing function called Butterfly MLP. Wewere also able to sparsely mix along sequence dimension for Transformer-basedarchitectures called Butterfly Attention. Experiments on CIFAR and LRA datasetsdemonstrate that the proposed Non-Linear Butterfly Mixers are efficient andscale well when the host architectures are used as mixing function.Additionally, we propose Patch-Only MLP-Mixer for processing spatial 2D signalsdemonstrating a different dimension mixing strategy.

Dimension Mixer: Group Mixing of Input Dimensions for Efficient Function Approximation