English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Pretrained language models such as Bidirectional Encoder Representations fromTransformers (BERT) have achieved state-of-the-art performance in naturallanguage processing (NLP) tasks. Recently, BERT has been adapted to thebiomedical domain. Despite the effectiveness, these models have hundreds ofmillions of parameters and are computationally expensive when applied tolarge-scale NLP applications. We hypothesized that the number of parameters ofthe original BERT can be dramatically reduced with minor impact on performance.In this study, we present Bioformer, a compact BERT model for biomedical textmining. We pretrained two Bioformer models (named Bioformer8L and Bioformer16L)which reduced the model size by 60% compared to BERTBase. Bioformer uses abiomedical vocabulary and was pre-trained from scratch on PubMed abstracts andPubMed Central full-text articles. We thoroughly evaluated the performance ofBioformer as well as existing biomedical BERT models including BioBERT andPubMedBERT on 15 benchmark datasets of four different biomedical NLP tasks:named entity recognition, relation extraction, question answering and documentclassification. The results show that with 60% fewer parameters, Bioformer16Lis only 0.1% less accurate than PubMedBERT while Bioformer8L is 0.9% lessaccurate than PubMedBERT. Both Bioformer16L and Bioformer8L outperformedBioBERTBase-v1.1. In addition, Bioformer16L and Bioformer8L are 2-3 fold asfast as PubMedBERT/BioBERTBase-v1.1. Bioformer has been successfully deployedto PubTator Central providing gene annotations over 35 million PubMed abstractsand 5 million PubMed Central full-text articles. We make Bioformer publiclyavailable via https://github.com/WGLab/bioformer, including pre-trained models,datasets, and instructions for downstream use.

Bioformer: an efficient transformer language model for biomedical text mining