English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Automatic metrics are fundamental for the development and evaluation ofmachine translation systems. Judging whether, and to what extent, automaticmetrics concur with the gold standard of human evaluation is not astraightforward problem. We show that current methods for judging metrics arehighly sensitive to the translations used for assessment, particularly thepresence of outliers, which often leads to falsely confident conclusions abouta metric's efficacy. Finally, we turn to pairwise system ranking, developing amethod for thresholding performance improvement under an automatic metricagainst human judgements, which allows quantification of type I versus type IIerrors incurred, i.e., insignificant human differences in system quality thatare accepted, and significant human differences that are rejected. Together,these findings suggest improvements to the protocols for metric evaluation andsystem performance evaluation in machine translation.

Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics