English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Transformers-based models, such as BERT, have dramatically improved theperformance for various natural language processing tasks. The clinicalknowledge enriched model, namely ClinicalBERT, also achieved state-of-the-artresults when performed on clinical named entity recognition and naturallanguage inference tasks. One of the core limitations of these transformers isthe substantial memory consumption due to their full self-attention mechanism.To overcome this, long sequence transformer models, e.g. Longformer andBigBird, were proposed with the idea of sparse attention mechanism to reducethe memory usage from quadratic to the sequence length to a linear scale. Thesemodels extended the maximum input sequence length from 512 to 4096, whichenhanced the ability of modeling long-term dependency and consequently achievedoptimal results in a variety of tasks. Inspired by the success of these longsequence transformer models, we introduce two domain enriched language models,namely Clinical-Longformer and Clinical-BigBird, which are pre-trained fromlarge-scale clinical corpora. We evaluate both pre-trained models using 10baseline tasks including named entity recognition, question answering, anddocument classification tasks. The results demonstrate that Clinical-Longformerand Clinical-BigBird consistently and significantly outperform ClinicalBERT aswell as other short-sequence transformers in all downstream tasks. We have madeour source code available at[https://github.com/luoyuanlab/Clinical-Longformer] the pre-trained modelsavailable for public download at:[https://huggingface.co/yikuan8/Clinical-Longformer].

Clinical-Longformer and Clinical-BigBird: Transformers for long clinical sequences