With 90% of the members of the three groups who attended the call considering that post-editors are usually negative in terms of the MTPE compensation models, one can conclude that the subject deserves attention.
Indeed, a quick search in Proz about MTPE compensation offers a snapshot of certain lacks and assumptions: lack of experience with MTPE or familiarization with MT systems, error types and metrics, differences between revision and translation as a process, differences in compensation according to the type of text and other actors, and compensation seen as an opportunity for LSPs to trick translators and save money, to mention just a few.
In the translation and localization industry, several MTPE compensation methods coexist. Companies and LSPs usually adopt one or another based on aspects such as language pair, content domain, text type, quality of the MT output or actual effort put into the process.
Post-editing distance or edit distance is an automated metric which consists of comparing the raw MT output with the final version of the target language to calculate the minimum number of changes required to get from the original MT output to that final version of the target text. It considers additions, deletions, substitutions and sometimes shifts (changes in position), and can be calculated at the character level or the word level. Its adoption in the localization industry has been increasing as it gets integrated into more CAT tools.
Edit distance, however, is not a good indicator alone because it doesn’t necessarily correlate with the time spent on editing, as the latter depends on other factors such as the error type and the post-editor qualification. When the compensation method is based exclusively in edit distance, it creates a conflict of interest for the post-editor to do more edits, while the goal is precisely to avoid changing what is good enough.
Actual time per segment tracked seem the ideal method to capture productivity in words per hour. However, tracking edit time is complex and raises many questions, some of them related to implementation (what does actual editing time refer to, anyway? Should it exclude pauses and/or research time?) or ethics (up to what extent it is intrusive, or productizing the job of post-editors?). These may be some of the reasons why time tracking has had so far a discreet integration into CAT tools; hence less adopted by LSPs and companies alike.