English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

There has always been criticism for using $n$-gram based similarity metrics,such as BLEU, NIST, etc, for evaluating the performance of NLG systems.However, these metrics continue to remain popular and are recently being usedfor evaluating the performance of systems which automatically generatequestions from documents, knowledge graphs, images, etc. Given the risinginterest in such automatic question generation (AQG) systems, it is importantto objectively examine whether these metrics are suitable for this task. Inparticular, it is important to verify whether such metrics used for evaluatingAQG systems focus on answerability of the generated question by preferringquestions which contain all relevant information such as question type(Wh-types), entities, relations, etc. In this work, we show that currentautomatic evaluation metrics based on $n$-gram similarity do not alwayscorrelate well with human judgments about answerability of a question. Toalleviate this problem and as a first step towards better evaluation metricsfor AQG, we introduce a scoring function to capture answerability and show thatwhen this scoring function is integrated with existing metrics, they correlatesignificantly better with human judgments. The scripts and data developed as apart of this work are made publicly available athttps://github.com/PrekshaNema25/Answerability-Metric

Towards a Better Metric for Evaluating Question Generation Systems