English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large Language Models (LLMs) have achieved remarkable results in the machinetranslation evaluation task, yet there remains a gap in knowledge regarding howthey utilize the provided data to conduct evaluations. This study aims toexplore how LLMs leverage source and reference information in evaluatingtranslations, with the ultimate goal of better understanding the workingmechanism of LLMs. To this end, we design the controlled experiments acrossvarious input modes and model types, and employ both coarse-grained andfine-grained prompts to discern the utility of source versus referenceinformation. Surprisingly, we find that reference information significantlyenhances the evaluation accuracy, while source information sometimes iscounterproductive, indicating a lack of cross-lingual capability when usingLLMs to evaluate translations. We further conduct a meta-evaluation fortranslation error detection of LLMs, observing a similar phenomenon. Thesefindings also suggest a potential research direction for LLMs that fullyexploits the cross-lingual capability of LLMs to achieve better performance inmachine translation evaluation tasks.

Lost in the Source Language: How Large Language Models Evaluate the Quality of Machine Translation