English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Pre-training and fine-tuning have emerged as a promising paradigm acrossvarious natural language processing (NLP) tasks. The effectiveness ofpretrained large language models (LLM) has witnessed further enhancement,holding potential for applications in the field of medicine, particularly inthe context of Traditional Chinese Medicine (TCM). However, the application ofthese general models to specific domains often yields suboptimal results,primarily due to challenges like lack of domain knowledge, unique objectives,and computational efficiency. Furthermore, their effectiveness in specializeddomains, such as Traditional Chinese Medicine, requires comprehensiveevaluation. To address the above issues, we propose a novel domain specificTCMDA (TCM Domain Adaptation) approach, efficient pre-training withdomain-specific corpus. Specifically, we first construct a large TCM-specificcorpus, TCM-Corpus-1B, by identifying domain keywords and retreving fromgeneral corpus. Then, our TCMDA leverages the LoRA which freezes the pretrainedmodel's weights and uses rank decomposition matrices to efficiently trainspecific dense layers for pre-training and fine-tuning, efficiently aligningthe model with TCM-related tasks, namely TCM-GPT-7B. We further conductedextensive experiments on two TCM tasks, including TCM examination and TCMdiagnosis. TCM-GPT-7B archived the best performance across both datasets,outperforming other models by relative increments of 17% and 12% in accuracy,respectively. To the best of our knowledge, our study represents the pioneeringvalidation of domain adaptation of a large language model with 7 billionparameters in TCM domain. We will release both TCMCorpus-1B and TCM-GPT-7Bmodel once accepted to facilitate interdisciplinary development in TCM and NLP,serving as the foundation for further study.

TCM-GPT: Efficient Pre-training of Large Language Models for Domain Adaptation in Traditional Chinese Medicine