English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Ensembling BERT models often significantly improves accuracy, but at the costof significantly more computation and memory footprint. In this work, wepropose Multi-CLS BERT, a novel ensembling method for CLS-based predictiontasks that is almost as efficient as a single BERT model. Multi-CLS BERT usesmultiple CLS tokens with a parameterization and objective that encourages theirdiversity. Thus instead of fine-tuning each BERT model in an ensemble (andrunning them all at test time), we need only fine-tune our single Multi-CLSBERT model (and run the one model at test time, ensembling just the multiplefinal CLS embeddings). To test its effectiveness, we build Multi-CLS BERT ontop of a state-of-the-art pretraining method for BERT (Aroca-Ouellette andRudzicz, 2020). In experiments on GLUE and SuperGLUE we show that our Multi-CLSBERT reliably improves both overall accuracy and confidence estimation. Whenonly 100 training samples are available in GLUE, the Multi-CLS BERT_Base modelcan even outperform the corresponding BERT_Large model. We analyze the behaviorof our Multi-CLS BERT, showing that it has many of the same characteristics andbehavior as a typical BERT 5-way ensemble, but with nearly 4-times lesscomputation and memory.

Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling