English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Knowledge-Enhanced Pre-trained Language Models (KEPLMs) improve theperformance of various downstream NLP tasks by injecting knowledge facts fromlarge-scale Knowledge Graphs (KGs). However, existing methods for pre-trainingKEPLMs with relational triples are difficult to be adapted to close domains dueto the lack of sufficient domain graph semantics. In this paper, we propose aKnowledge-enhanced lANGuAge Representation learning framework for variousclOsed dOmains (KANGAROO) via capturing the implicit graph structure among theentities. Specifically, since the entity coverage rates of closed-domain KGscan be relatively low and may exhibit the global sparsity phenomenon forknowledge injection, we consider not only the shallow relationalrepresentations of triples but also the hyperbolic embeddings of deephierarchical entity-class structures for effective knowledge fusion.Moreover,as two closed-domain entities under the same entity-class often have locallydense neighbor subgraphs counted by max point biconnected component, we furtherpropose a data augmentation strategy based on contrastive learning oversubgraphs to construct hard negative samples of higher quality. It makes theunderlying KELPMs better distinguish the semantics of these neighboringentities to further complement the global semantic sparsity. In theexperiments, we evaluate KANGAROO over various knowledge-aware and general NLPtasks in both full and few-shot learning settings, outperforming various KEPLMtraining paradigms performance in closed-domains significantly.

Learning Knowledge-Enhanced Contextual Language Representations for Domain Natural Language Understanding