English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In executable task-oriented semantic parsing, the system aims to translateusers' utterances in natural language to machine-interpretable programs (APIcalls) that can be executed according to pre-defined API specifications. Withthe popularity of Large Language Models (LLMs), in-context learning offers astrong baseline for such scenarios, especially in data-limited regimes.However, LLMs are known to hallucinate and therefore pose a formidablechallenge in constraining generated content. Thus, it remains uncertain if LLMscan effectively perform task-oriented utterance-to-API generation whererespecting API's structural and task-specific constraints is crucial.  In this work, we seek to measure, analyze and mitigate such constraintsviolations. First, we identify the categories of various constraints inobtaining API-semantics from task-oriented utterances, and define fine-grainedmetrics that complement traditional ones. Second, we leverage these metrics toconduct a detailed error analysis of constraints violations seen instate-of-the-art LLMs, which motivates us to investigate two mitigationstrategies: Semantic-Retrieval of Demonstrations (SRD) and API-awareConstrained Decoding (API-CD). Our experiments show that these strategies areeffective at reducing constraints violations and improving the quality of thegenerated API calls, but require careful consideration given theirimplementation complexity and latency.

Measuring and Mitigating Constraint Violations of In-Context Learning for Utterance-to-API Semantic Parsing