English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Quality Estimation (QE) is the task of predicting the quality of MachineTranslation (MT) system output, without using any gold-standard translationreferences. State-of-the-art QE models are supervised: they requirehuman-labeled quality of some MT system output on some datasets for training,making them domain-dependent and MT-system-dependent. There has been researchon unsupervised QE, which requires glass-box access to the MT systems, orparallel MT data to generate synthetic errors for training QE models. In thispaper, we present Perturbation-based QE - a word-level Quality Estimationapproach that works simply by analyzing MT system output on perturbed inputsource sentences. Our approach is unsupervised, explainable, and can evaluateany type of blackbox MT systems, including the currently prominent largelanguage models (LLMs) with opaque internal processes. For language directionswith no labeled QE data, our approach has similar or better performance thanthe zero-shot supervised approach on the WMT21 shared task. Our approach isbetter at detecting gender bias and word-sense-disambiguation errors intranslation than supervised QE, indicating its robustness to out-of-domainusage. The performance gap is larger when detecting errors on a nontraditionaltranslation-prompting LLM, indicating that our approach is more generalizableto different MT systems. We give examples demonstrating our approach'sexplainability power, where it shows which input source words have influence ona certain MT output word.

Perturbation-based QE: An Explainable, Unsupervised Word-level Quality Estimation Method for Blackbox Machine Translation