English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Rare diseases (RDs) are collectively common and affect 300 million peopleworldwide. Accurate phenotyping is critical for informing diagnosis andtreatment, but RD phenotypes are often embedded in unstructured text andtime-consuming to extract manually. While natural language processing (NLP)models can perform named entity recognition (NER) to automate extraction, amajor bottleneck is the development of a large, annotated corpus for modeltraining. Recently, prompt learning emerged as an NLP paradigm that can lead tomore generalizable results without any (zero-shot) or few labeled samples(few-shot). Despite growing interest in ChatGPT, a revolutionary large languagemodel capable of following complex human prompts and generating high-qualityresponses, none have studied its NER performance for RDs in the zero- andfew-shot settings. To this end, we engineered novel prompts aimed at extractingRD phenotypes and, to the best of our knowledge, are the first the establish abenchmark for evaluating ChatGPT's performance in these settings. We comparedits performance to the traditional fine-tuning approach and conducted anin-depth error analysis. Overall, fine-tuning BioClinicalBERT resulted inhigher performance (F1 of 0.689) than ChatGPT (F1 of 0.472 and 0.591 in thezero- and few-shot settings, respectively). Despite this, ChatGPT achievedsimilar or higher accuracy for certain entities (i.e., rare diseases and signs)in the one-shot setting (F1 of 0.776 and 0.725). This suggests that withappropriate prompt engineering, ChatGPT has the potential to match oroutperform fine-tuned language models for certain entity types with just onelabeled sample. While the proliferation of large language models may provideopportunities for supporting RD diagnosis and treatment, researchers andclinicians should critically evaluate model outputs and be well-informed oftheir limitations.

Identifying and Extracting Rare Disease Phenotypes with Large Language Models