English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Precisely assessing the progress in natural language generation (NLG) tasksis challenging, and human evaluation to establish a preference in a model'soutput over another is often necessary. However, human evaluation is usuallycostly, difficult to reproduce, and non-reusable. In this paper, we propose anew and simple automatic evaluation method for NLG called Near-NegativeDistinction (NND) that repurposes prior human annotations into NND tests. In anNND test, an NLG model must place a higher likelihood on a high-quality outputcandidate than on a near-negative candidate with a known error. Modelperformance is established by the number of NND tests a model passes, as wellas the distribution over task-specific errors the model fails on. Throughexperiments on three NLG tasks (question generation, question answering, andsummarization), we show that NND achieves a higher correlation with humanjudgments than standard NLG evaluation metrics. We then illustrate NNDevaluation in four practical scenarios, for example performing fine-grain modelanalysis, or studying model training dynamics. Our findings suggest that NNDcan give a second life to human annotations and provide low-cost NLGevaluation.

Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets