English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Many Natural Language Generation (NLG) tasks aim to generate a single outputtext given an input prompt. Other settings require the generation of multipletexts, e.g., for Synthetic Traffic Generation (STG). This generation task iscrucial for training and evaluating QA systems as well as conversationalagents, where the goal is to generate multiple questions or utterancesresembling the linguistic variability of real users. In this paper, we showthat common NLG metrics, like BLEU, are not suitable for evaluating STG. Wepropose and evaluate several metrics designed to compare the generated trafficto the distribution of real user texts. We validate our metrics with anautomatic procedure to verify whether they capture different types of qualityissues of generated data; we also run human annotations to verify thecorrelation with human judgements. Experiments on three tasks, i.e., ShoppingUtterance Generation, Product Question Generation and Query Auto Completion,demonstrate that our metrics are effective for evaluating STG tasks, andimprove the agreement with human judgement up to 20% with respect to common NLGmetrics. We believe these findings can pave the way towards better solutionsfor estimating the representativeness of synthetic text data.

Evaluation Metrics of Language Generation Models for Synthetic Traffic Generation Tasks