Open AccessBeyond MT metrics in specialised translation: Automated and manual evaluation of machine translation output for freelance translators and small LSPs in the context of EU documents
Author(s) -
Krzysztof Łoboda
Publication year - 2021
Publication title -
beyond philology
Language(s) - English
Resource type - Journals
eISSN - 2451-1498
pISSN - 1732-1220
DOI - 10.26881/bp.2020.4.02
Subject(s) - machine translation , computer science , terminology , context (archaeology) , evaluation of machine translation , translation (biology) , natural language processing , process (computing) , artificial intelligence , computer assisted translation , transfer based machine translation , machine translation software usability , example based machine translation , information retrieval , linguistics , programming language , paleontology , philosophy , biochemistry , chemistry , messenger rna , gene , biology
This paper discusses simplified methods of translation evaluation in two seemingly disparate areas: machine translation (MT) technology and translation for EU institutions. It provides a brief overview of methods for evaluating MT output and proposes simplified solutions for small LSPs and freelancers dealing with specialised translation of this kind. After discussing the context of the study and the process of machine translation, an analysis of fragments of the selected specialist text (an EU regulation) is carried out. The official English and Polish versions of this document provide the basis for a comparative evaluation of raw machine translation output obtained with selected commercially available (paid) neural machine translation engines (NMT). Quantitative analysis, including the Damerau-Levenshstein edit distance parameters and the number of erroneous segments in the text, combined with a manual qualitative analysis of errors and terminology can be a serviceable method for small LSPs and freelance translators to evaluate the usefulness of neural machine translation engines.