English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

In recent years, major advancements in natural language processing (NLP) havebeen driven by the emergence of large language models (LLMs), which havesignificantly revolutionized research and development within the field.Building upon this progress, our study delves into the effects of variouspre-training methodologies on Turkish clinical language models' performance ina multi-label classification task involving radiology reports, with a focus onaddressing the challenges posed by limited language resources. Additionally, weevaluated the simultaneous pretraining approach by utilizing limited clinicaltask data for the first time. We developed four models, includingTurkRadBERT-task v1, TurkRadBERT-task v2, TurkRadBERT-sim v1, andTurkRadBERT-sim v2. Our findings indicate that the general Turkish BERT model(BERTurk) and TurkRadBERT-task v1, both of which utilize knowledge from asubstantial general-domain corpus, demonstrate the best overall performance.Although the task-adaptive pre-training approach has the potential to capturedomain-specific patterns, it is constrained by the limited task-specific corpusand may be susceptible to overfitting. Furthermore, our results underscore thesignificance of domain-specific vocabulary during pre-training for enhancingmodel performance. Ultimately, we observe that the combination ofgeneral-domain knowledge and task-specific fine-tuning is essential forachieving optimal performance across a range of categories. This study offersvaluable insights for developing effective Turkish clinical language models andcan guide future research on pre-training techniques for other low-resourcelanguages within the clinical domain.

Harnessing the Power of BERT in the Turkish Clinical Domain: Pretraining Approaches for Limited Data Scenarios