English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Current approaches to evaluating large language models (LLMs) withpre-existing Knowledge Graphs (KG) mostly ignore the structure of the KG andmake arbitrary choices of which part of the graph to evaluate. In this paper,we introduce KGLens, a method to evaluate LLMs by generating natural languagequestions from a KG in a structure aware manner so that we can characterize itsperformance on a more aggregated level. KGLens uses a parameterized KG, whereeach edge is augmented with a beta distribution that guides how to sample edgesfrom the KG for QA testing. As the evaluation proceeds, different edges of theparameterized KG are sampled and assessed appropriately, converging to a moreglobal picture of the performance of the LLMs on the KG as a whole. In ourexperiments, we construct three domain-specific KGs for knowledge assessment,comprising over 19,000 edges, 700 relations, and 21,000 entities. The resultsdemonstrate that KGLens can not only assess overall performance but alsoprovide topic, temporal, and relation analyses of LLMs. This showcases theadaptability and customizability of KGLens, emphasizing its ability to focusthe evaluation based on specific criteria.

KGLens: A Parameterized Knowledge Graph Solution to Assess What an LLM Does and Doesn't Know