English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Symbolic knowledge graphs (KGs) play a pivotal role in knowledge-centricapplications such as search, question answering and recommendation. Ascontemporary language models (LMs) trained on extensive textual data havegained prominence, researchers have extensively explored whether the parametricknowledge within these models can match up to that present in knowledge graphs.Various methodologies have indicated that enhancing the size of the model orthe volume of training data enhances its capacity to retrieve symbolicknowledge, often with minimal or no human supervision. Despite theseadvancements, there is a void in comprehensively evaluating whether LMs canencompass the intricate topological and semantic attributes of KGs, attributescrucial for reasoning processes. In this work, we provide an exhaustiveevaluation of language models of varying sizes and capabilities. We constructnine qualitative benchmarks that encompass a spectrum of attributes includingsymmetry, asymmetry, hierarchy, bidirectionality, compositionality, paths,entity-centricity, bias and ambiguity. Additionally, we propose novelevaluation metrics tailored for each of these attributes. Our extensiveevaluation of various LMs shows that while these models exhibit considerablepotential in recalling factual information, their ability to capture intricatetopological and semantic traits of KGs remains significantly constrained. Wenote that our proposed evaluation metrics are more reliable in evaluating theseabilities than the existing metrics. Lastly, some of our benchmarks challengethe common notion that larger LMs (e.g., GPT-4) universally outshine theirsmaller counterparts (e.g., BERT).

Rethinking Language Models as Symbolic Knowledge Graphs