English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We combine the capacity of sparsely gated Mixture-of-Experts (MoE) with thespeed and stability of linear, mixing transformations to design the SparseMixer encoder model. Sparse Mixer slightly outperforms (<1%) BERT on GLUE andSuperGLUE, but more importantly trains 65% faster and runs inference 61%faster. We also present a faster variant, prosaically named Fast Sparse Mixer,that marginally underperforms BERT on SuperGLUE, but trains and runs nearlytwice as fast. We justify the design of these two models by carefully ablatingthrough various mixing mechanisms, MoE configurations and hyperparameters.Sparse Mixer overcomes many of the latency and stability concerns of MoE modelsand offers the prospect of serving sparse student models, without resorting todistilling them to dense variants.

Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT