English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Knowledge probing is crucial for understanding the knowledge transfermechanism behind the pre-trained language models (PLMs). Despite the growingprogress of probing knowledge for PLMs in the general domain, specialised areassuch as biomedical domain are vastly under-explored. To catalyse the researchin this direction, we release a well-curated biomedical knowledge probingbenchmark, MedLAMA, which is constructed based on the Unified Medical LanguageSystem (UMLS) Metathesaurus. We test a wide spectrum of state-of-the-art PLMsand probing approaches on our benchmark, reaching at most 3% of acc@10. Whilehighlighting various sources of domain-specific challenges that amount to thisunderwhelming performance, we illustrate that the underlying PLMs have a higherpotential for probing tasks. To achieve this, we propose Contrastive-Probe, anovel self-supervised contrastive probing approach, that adjusts the underlyingPLMs without using any probing data. While Contrastive-Probe pushes the acc@10to 28%, the performance gap still remains notable. Our human expert evaluationsuggests that the probing performance of our Contrastive-Probe is stillunder-estimated as UMLS still does not include the full spectrum of factualknowledge. We hope MedLAMA and Contrastive-Probe facilitate furtherdevelopments of more suited probing techniques for this domain.

Rewire-then-Probe: A Contrastive Recipe for Probing Biomedical Knowledge of Pre-trained Language Models