English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Enterprise applications of Large Language Models (LLMs) hold promise forquestion answering on enterprise SQL databases. However, the extent to whichLLMs can accurately respond to enterprise questions in such databases remainsunclear, given the absence of suitable Text-to-SQL benchmarks tailored toenterprise settings. Additionally, the potential of Knowledge Graphs (KGs) toenhance LLM-based question answering by providing business context is not wellunderstood. This study aims to evaluate the accuracy of LLM-powered questionanswering systems in the context of enterprise questions and SQL databases,while also exploring the role of knowledge graphs in improving accuracy. Toachieve this, we introduce a benchmark comprising an enterprise SQL schema inthe insurance domain, a range of enterprise queries encompassing reporting tometrics, and a contextual layer incorporating an ontology and mappings thatdefine a knowledge graph. Our primary finding reveals that question answeringusing GPT-4, with zero-shot prompts directly on SQL databases, achieves anaccuracy of 16%. Notably, this accuracy increases to 54% when questions areposed over a Knowledge Graph representation of the enterprise SQL database.Therefore, investing in Knowledge Graph provides higher accuracy for LLMpowered question answering systems.

A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databases