English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This work presents biomedical and clinical language models for Spanish byexperimenting with different pretraining choices, such as masking at word andsubword level, varying the vocabulary size and testing with domain data,looking for better language representations. Interestingly, in the absence ofenough clinical data to train a model from scratch, we applied mixed-domainpretraining and cross-domain transfer approaches to generate a performantbio-clinical model suitable for real-world clinical data. We evaluated ourmodels on Named Entity Recognition (NER) tasks for biomedical documents andchallenging hospital discharge reports. When compared against the competitivemBERT and BETO models, we outperform them in all NER tasks by a significantmargin. Finally, we studied the impact of the model's vocabulary on the NERperformances by offering an interesting vocabulary-centric analysis. Theresults confirm that domain-specific pretraining is fundamental to achievinghigher performances in downstream NER tasks, even within a mid-resourcescenario. To the best of our knowledge, we provide the first biomedical andclinical transformer-based pretrained language models for Spanish, intending toboost native Spanish NLP applications in biomedicine. Our best models arefreely available in the HuggingFace hub: https://huggingface.co/BSC-TeMU.

Biomedical and Clinical Language Models for Spanish: On the Benefits of Domain-Specific Pretraining in a Mid-Resource Scenario