Translation Quality and Error Recognition in Professional Neural Machine Translation Post-Editing

This study aims to analyse how translation experts from the German department of the European Commission&#8217;s Directorate-General for Translation (DGT) identify and correct different error categories in neural machine translated texts (NMT) and their post-edited versions (NMTPE). The term <i>translation expert</i> encompasses <i>translator</i>, <i>post-editor</i> as well as <i>revisor</i>. Even though we focus on neural machine-translated segments, <i>translator</i> and <i>post-editor</i> are used synonymously because of the combined workflow using CAT-Tools as well as machine translation. Only the distinction between <i>post-editor</i>, which refers to a DGT translation expert correcting the neural machine translation output, and <i>revisor</i>, which refers to a DGT translation expert correcting the post-edited version of the neural machine translation output, is important and made clear whenever relevant. Using an automatic error annotation tool and the more fine-grained manual error annotation framework to identify characteristic error categories in the DGT texts, a corpus analysis revealed that quality assurance measures by post-editors and revisors of the DGT are most often necessary for lexical errors. More specifically, the corpus analysis showed that, if post-editors correct mistranslations, terminology or stylistic errors in an NMT sentence, revisors are likely to correct the same error type in the same post-edited sentence, suggesting that the DGT experts were being primed by the NMT output. Subsequently, we designed a controlled eye-tracking and key-logging experiment to compare participants&#8217; eye movements for test sentences containing the three identified error categories (mistranslations, terminology or stylistic errors) and for control sentences without errors. We examined the three error types&#8217; effect on early (first fixation durations, first pass durations) and late eye movement measures (e.g., total reading time and regression path durations). Linear mixed-effects regression models predict what kind of behaviour of the DGT experts is associated with the correction of different error types during the post-editing process.

Abstract licence: CC BY

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.