MTPE of Poor Quality Source Texts: Some Practical Suggestions

To achieve the best MT results, you should first correct the source text, when it is a scanned hard copy or an automatic transcription of recorded speech. Here are a few practical suggestions:

• Choose the correct settings before running OCR. In particular, select the correct source language (you’ll see better suggestions during verification), select the correct graphic resolution for each page and the correct text direction for each piece of text, and de-skew and clean each page that requires it. Verification should be run by someone familiar with both the source language and the subject.
• Correct misspelled or wrongly transcribed words.
• Add “[sic]” after any word that you cannot identify and that you suspect is an artifact of the OCR process. This helps the post-editor focus on problem areas.
• Capitalize proper nouns and acronyms.
• Lower case incorrectly capitalized words.
• Reassemble sentences broken up by periods (hard returns) or new lines (soft returns).
• Feed the source text to the MT engine only after completing such corrections; doing otherwise will yield substandard results and will take longer to post-edit.

When the source text is good, you can skip pre-editing, but, when it is questionable or poor, pre-editing enhances the quality of the resulting machine translation and helps the post-editor achieve the desired quality.

Guest post: Translators’ Attitudes towards Machine Translation

By Irene Chamali

In my dissertation, I tackled the topic of Machine Translation vs. Translators, not only because I want to later become a translator myself, but also because I was always fascinated with technology and how it is used in different professions. My key question was: What are professional translators’ attitudes towards the technological tools created for their profession?

1. Research Questions and Hypotheses

My first question was “Do professional translators believe that Machine Translation (MT) increases their productivity?” What I found (from the answers received and existing research) was that such software is easy to use, offers fast results and, according to professional translators, it improves their productivity. 

My second question, “Do translators view MT as a threat?” looked at how translators feel about automated programs which can translate entire texts automatically. I found that there is no fear that MT will replace translators, since, according to research participants, it is not quite advanced yet and there are aspects of language which MT software cannot yet cope with. So, translators do not view MT as a threat (yet). 

Moving on to the third question, “What are the requirements that MT software has to fulfill in order for translators to use it?” I originally believed that it would be difficult to pinpoint specific requirements. Previous research claimed that speed, usefulness and ease of use are the main factors driving MT software adoption, and my research confirmed this: I found that ease of use, fast results, a target text which requires only minor corrections, the availability of training and support for MT software are the requirements for MT software adoption.

My last question was “Is experience one of the factors which lead translators to the acceptance of MT?”, and the answers showed that more experienced translators are more likely to use MT software.

2. Participants and Data Collection

The participants were 42 professional translators (freelancers, in-house, working in companies or in the EU) from all over the world, of different ages and experience. I collected data through online questionnaires and then examined it with the help of SPSS (a statistical tool).

3. Results

Not all the results were what I was expecting, but this didn’t discourage me, because unexpected findings can encourage further research. 

The results regarding perceived increase in productivity thanks to MT software showed that most participants recognize the advantage of using such software, since it can increase productivity. Most respondents, however, appear not to trust the quality of machine translation. Not all groups of translators (freelancers, in-house translators, etc.) have the same opinion regarding perceived productivity. For example, none of the in-house translators agreed that MT software can increase productivity, although most of the other groups thought otherwise. The reason may be that they are urged by their companies to use software which does not suit their needs.

Almost no participant feared that MT will replace human translators, since MT still needs to improve considerably. The younger the participants were, the less they believed that MT software can replace them. I think this is because younger translators are more used to using technology and seeing such tools complementing one’s work instead of taking their place, so they are less intimidated by MT. Gender, on the other hand, did not seem to play any role in perceived threat. What played a role, according to the results, was nationality, as the answers to questions regarding perceived threat differed from one nationality group to the other. For example, Turkish, Spanish, Australian, Swedish, Bulgarian and Danish participants did not seem to agree that MT software can replace human translators. French participants, on the other hand, agreed, and Portuguese, Moldovan and Austrian ones were generally neutral. Regarding the requirements for MT software, the participants’ ranking showed that the most important are usefulness, fast results and ease of use. It was interesting to see that the answers that MT software users gave did not differ from those of non-users, which could mean that non-users have a realistic view of what MT software can offer.

Finally, the outcome of my last research question about work experience as a determining factor for MT software use was that groups with different working experience gave similar answers. The small number of participants could explain the fact that my results differed from those of previous studies.

I think that conducting research surveys like the one I did for my university is not only important for academic purposes but is also useful to help software developers tailor MT software to the needs of their clients. I will be very glad if my paper makes a contribution, however small it may be, to the investigation and enhancement of the relationship between human and machine.

About the author:

Irene Chamali is a recent graduate from CITY College, International Faculty of the University of Sheffield, in Thessaloniki, Greece. She was accepted in 2017, studied in the English Studies Department for three years, and was awarded the BA (Honors) degree in English Language and Linguistics. After her BA studies, Irene was accepted for an MA in Translation and Interpreting from CITY College, which she is currently undertaking. Her article summarizes the research she completed for her dissertation.

Translators’ Attitudes towards Machine Translation

I’ve received the following message, about a questionnaire regarding translators’ attitudes towards Machine Translation, together with the request to share it with other translators:

Greetings,

I’m a BA student from the English Studies Department of the University of Sheffield and I would appreciate it if you took the time to fill in the questionnaire for my dissertation regarding translators’ attitudes towards Machine Translation. It would also be very helpful if you shared it with other potential participants. Thank you in advance! Here’s the link: https://docs.google.com/forms/d/e/1FAIpQLSdgRSy8Ys-zlxXSKtw--oOELkvr5BjOcfzncua20l0HwGTK-g/viewform?usp=sf_link

Kind regards, Irene Chamali

The questionnaire includes, at the beginning, before any questions are asked, a full “Participant Information Sheet”. I’ve checked (and answered) the questionnaire, and I believe it deserves that translators answer it, as it comes from a legitimate study.

Interesting article on post-editing machine translation

Recently, Isabella Massardo published on interesting article on post-editing machine translation:

5 Effective Strategies for Post-Editing MT

Because of the misleading fluency of NMT systems, we now have to get the meaning of the source text first and then compare the text with the MT raw output to make sure that the translation is correct and adequate.

Good to know that there are still people who realize that advocating for "Monolingual Post-Editing", as some zealots increasingly do is ever more dangerous, especially now that machine translation is becoming even more "fluent".

Edit:
The link was not correct but it should work now.

An interesting article on "Google Translate" and its legal implications

The Volokh Conspiracy, a law blog, recently published an interesting article on some legal implications of the use of Google Translate: "Google Translate" and the Law of Consent Searches.

GT4T - A tool for translators, instead of a tool to replace translators

Guest post by Dallas Cao, developer of GT4T

Many translators believe that machine translation (MT) is a horror story, and that using machine translation (MT) in our work only results in bad quality. Indeed, after I started advertising GT4T (Google Translate for Translators) on Facebook, the reactions I got from many translators were negative.

They are right to think that the overall quality of machine translation is bad, and that any translator who mindlessly uses machine translation puts his or her career at risk; but the quality of machine translation is improving: Google’s neural translation engine, for example, has surprised many, to the point that some agencies have started using it to replace human translators, relying afterwards on translators as post-editors--a situation that creates even greater hostility against MT among translators, who are rightfully afraid that post-editing means for them toiling at mind-numbing grunt work.

Most of us use on-line reference tools in our work; when an online reference tool gets better, it helps us more. In my opinion, MT is the most advanced technology in translation, and, therefore, it should benefit professional translators first. If we consider MT as a reference tool rather than a threat, shouldn’t we be glad when our tool gets better?

I never liked the idea of letting MT translate and translators confined to an unrewarding task of post-editing; however, we can use MT to “translate” a word, a term, a phrase, or a part of a sentence that we judge it will translate well. Sometimes MT returns nonsense, true, but most of time, when used carefully it provides a surprisingly useful translation.

I developed GT4T because I wanted a tool that could help translators (and not translation companies) make the most of Google Translate, without becoming ourselves post-editors. Copying and pasting between Google Translate and your work is not a good solution, as it takes too much time. Some TM tools already include MT, but they all submit the whole sentence to MT: you cannot choose to have MT translate only part of a sentence.

GT4T is a tool that lets you submit any portion of a sentence of your choice to MT with ease. It’s very simple: you select some text anywhere (including from inside a CAT tool), press a keyboard shortcut, and the selection is replaced by translation from MT. Simple as it is, I believe it is the correct way of using MT. As we use keyboards most of time, GT4T painlessly incorporates MT into our workflow.

A usual problem with MT is inconsistency--the MT engine translates the same term differently in different sentences. GT4T has a simple glossary feature to solve this issue. You press a keyboard shortcut to add a term to GT4T’s glossary, and that term will be pre-translated before submission to MT; thus the results suggested by MT will be consistent.

GT4T - Glossary Setup

GT4T also offers the option to use both Google Translate and Microsoft Translator at the same time. The results from both engines appear in a popup, and you can then press 1 or 2 to paste the corresponding translation.

GT4T - Alternative Translations

I expect there are still many years ahead before MT can effectively replace us. Before that happens, MT can be a great aid--a tool that can increase both the speed and the quality of our translations, if used properly. A tool for translators, instead of a tool to replace translators.

---

You can find GT4T at: https://gt4t.net/en/

(Bull)Shift Happens

Keith Laska, CEO of SDL, has recently published a self-serving, jargon-filled post on SDL's community blog.

His claim is not all that novel: that there is so much content to translate now, that MT must be part of the equation. He then goes out on a limb with some unsubstantiated claims about how MT has become so much better, in recent years (can you spot the logical fallacy in a statement like "As for MT quality concerns: the machine translation quality debate is dead. Over 75% of our language markets report the use - or pending use - of some form of machine translation solution."? - a hint: "use" or "pending use" are not the same as "successful use", and stating, without a shred of evidence, that the MT debate is over doesn't mean that it is).

But my question is another: is there a special secret pact between CEOs that requires them to spew such corporate drivel as "value is now at the critical intersection between machines and humans"? Is that sentence supposed to mean something, or is it there just to give the impression that it carries some momentous meaning? Am I alone in thinking that "thought leadership", in "to drive high-quality, secure MT improvements, innovation and thought leadership" sounds creepy?

Keith, if you do have some good MT product or strategy, write your post again, in a way that does not make the reader think that your product or service is so poor you have to hide it in a fog of jargon lest your prospects realize how hollow the vaunted MT progress actually is.

Speaking as an SDL customer, in fact, I have a suggestion: why don't you redirect some of the efforts you are spending on pushing MT onto the unwary, and instead concentrate on actually improving those products of yours that we human translators use every day? A hint: starting with long overdue improvements to fuzzy-matching algorithms would be a good idea.

Don’t worry: MT is not exactly perfect yet

While I believe that for certain applications MT will keep on improving (and will become a useful tool even for many translators), the sky is not falling on our profession. At least not yet:

The verb “Switch”, in “Switch your TV to the corresponding Component Video input to view your XYZ video playback” is translated in opposite ways by Google Translate and by Bing Translator – and both of them are wrong. Google translates “switch” as if it were “switch on”; Bing as if it were “switch off” – when of course the meaning is neither the former nor the latter.

This is just anecdotal evidence, of course, and by itself means little, but it underlines the fact that a machine translation program does not understand the text, and that relying on MT can lead to some disastrous errors.

Which free machine translation works best? The results are in

Some time ago I wrote about the study that Chinese translator Ethan Shen was conducting to compare three different free MT engines (for my earlier articles about this study, see Google, Bing and Babelfish and Google, Bing and Babelfish: some preliminary results).

Ethan has now completed phase 1 of his study, and the results are both interesting and - for me, at least - unexpected. Here below you can read a short report on Ethan's study.

From Ethan’s website you can download the full report, if you prefer to have all the details.

---

Real World Comparison of Online Machine Translators

by Ethan Shen
Gabble On Research Project
research@gabble-on.com

Abstract

This paper evaluates the relative quality of three popular online translation tools: Google Translate, Bing (Microsoft) Translator, and Yahoo Babelfish. The results published below are based on a 6 week survey open to the general internet population which allowed survey takers to choose any language, enter any free-form text, and vote on the best of all translation results side-by-side (www.gabble-on.com/research). The final data reveals that while Google Translate is widely preferred when translating long passages, Microsoft Bing Translator and Yahoo Babelfish often produce better translations for phrases below 140 characters. Also, in general Babelfish performs well in East Asian Languages such as Chinese and Korean and Bing Translator performs well in Spanish, German, and Italian.

Results

Most Preferred Engine and Margin of Preference by Language Pair and Text Length

The above table describes the relationship between user preferences and translated text character length for 15 single direction languages pairings. The most preferred engine is given at each intersection (Google, Babelfish, or Bing) along with the magnitude of its lead over its closest competitor in that category (colored percentage). The language pairings excluded from this table represent sets for which preferences were overwhelming (over 100%) or insufficient data was available.

From this data, the following conclusions can be drawn:

1. For long passages of text up to 2000 characters, survey takers generally prefer Google Translate's results across the board.
   
   a. The extent of Google’s lead varies dramatically from language to language. In some languages such as French, the strength of Google Translate’s engine is overwhelming. However, in several others like German, Italian, and Portuguese, Google holds only a very slim lead when compared to its biggest competitors.
   
   b. These observations validate our Hypothesis 1 that no single engine can perform equally well across a spectrum of languages or conditions.
   
   
2. The greatest relative strength of statistical translation focused engine (Google Translate) has not clustered around the European Union working languages as expected. German, Italian, and Portuguese, all EU working languages are the most hotly contested from a performance perspective.
   
   a. One possible explanation is that large additional bodies of parallel English-French text are available from the government of Canada for which are official documents are translated into both. To a lesser extent this could explain the strength of Google Translate in Spanish as many Latin American country offer English Translations of official documents.
   
   b. This data partially refutes Hypothesis 2.
   
   
3. Traditional Rules Based Translation Engines (Babelfish) performed generally well in East Asian languages such and Chinese and Korean.
   
   a. One possible reason for this outperformance is likely that the language specific grammar and word usages rules are more effective that association based transliteration in these situations.
   
   b. These finding are in line with Hypothesis 3, but the size of the data set is not large enough to confirm in a statistical significant manner.
   
   
4. Across almost every language Bing Translator and Yahoo Babelfish gain ground or surpass Google Translate as the text length gets shorter.
   
   a. In Chinese, the gradual erosion of Google relative performance as total text length shrinks from 2000 characters to 50 characters is stark and representative of the comparative strength Rules Based or Hybrid Translation Engines as phrases get shorter and more straight forward.
   
   b. It appears that at 150 characters or less, the fiercest competition between performance of different translation models become the most heated. Some similar effects were seen at 200 characters, but to a less significant extent.
   
   c. Though data is not shown, a similar effect is seen for passages that are only one sentence compared to passages with multiple sentences
   
   d. This data strongly validates Hypothesis 4.
   
   
5. The most interesting observation is that translation quality is not a two way street. The engine that is best for translating in one direction is not necessarily the best tool to translate back the other way.
   
   a. The two most obvious cases of this are French and German. Though Google Translation dominates when translating both these languages to English. It faces heavy competition when translating back from English to the foreign language.

These results are taken from a longer full research write-up.
To read the hypothesis, experiment design, extended results, practical applications and references, the full report is provided here: http://www.gabble-on.com/files/phase1_full_research_report.pdf.

PC World's shallow comparison of Chrome, Bing and Babel Fish

I’ve recently written about Ethan Shen’s survey to determine which free MT platform is best. Earlier this month, PC World published a review of the machine translation capabilities of the new beta version of Google Chrome, comparing it to Bing Translator and to Yahoo’s Babel Fish.

Ethan’s approach is more interesting and will prove more useful. PC World’s review is really too facile: saying that “It's fair to say, however, that Chrome's translator is up to the task.” on the basis of a single, short piece of translation, is really not doing a good service to PC World’s readers.

Update

I had not seen the comparison of the three machine/translation platforms that had appeared on the New York Times a few days ago. It is much more interesting and well done than PC World's, but I fear it might have a built bias that favors statistical MT platforms such as Google: isn't it likely that such famous lines as the opening of One Hundred Years of Solitude aren't already in the giant databases that feed Google Translate and similar systems?

Google, Bing and Babelfish: some preliminary results

In a recent post, I mentioned a study that Chinese translator Ethan Shen is conducting to find out which of three major free machine translation platforms is best.

Yesterday I received the following message from Ethan, about some preliminary results from the study. He also reiterates his invitation to take part in the survey (you can participate in the survey at: Which Engine Translates Best?).

With his permission, I’m reposting here the message Ethan sent me:

Thanks for helping me promote my research project. We’ve reached the half-way point of our research period and I’ve made some quick observations of the data trends we’re seeing so far. I’ve made some recent changes to the survey engine to eliminate brand bias and first-result bias, if you think your readers would be interested in the below early results, I’d love to make one more publicity push to help reach our 10,000 vote goal. I’ll keep you up to date!

1. At the highest level, it appears that survey takers prefer Google Translate's results across the board.
   
   
   • In a few languages (Arabic, Polish, Dutch) the preference is overwhelming with votes for Google doubling its nearest competitor
     
   
   
2. However, once you remove voters that have self defined their fluency in the source or target language as “limited”, the contest becomes closer for some of the heavily trafficked languages
   
   
   • Bing Translator leads in German
   • Babelfish leads in Chinese
   • Google maintains its lead in Spanish, Japanese, and French
     
   
   
3. Observing just the self defined “Limited fluency” voter reveals a strong brand bias. If your fluency in the target translation language is limited, it would stand to reason your ability to assess the quality of the translation is very limited. And yet…
   
   
   • Limited fluency voters choose Google over Bing by 2 to 1
   • They also chose Google over Yahoo Babelfish by 5 to 1
     
   
   
4. As I had guessed in my hypothesis, Systran’s and Microsoft’s hybrid RBMT model performs better on shorter passages
   
   
   • For phrases below 50 characters, Google’s lead in Spanish, Japanese, and French disappear. And Microsoft’s lead in German widens
   • Beyond 50 characters, Google’s relative performance seems to improve across the board.
   • For passages that are only one sentence, the same effect is seen, though to a lesser extent than under 50 characters.
     
   
   
5. After March 4th, we’ve implemented changes to our survey-taking platform to hide the brands and randomize the positions of the results before voting. There has not been enough data collected since then to draw conclusions, but Yahoo Babelfish seems to be receiving the biggest boost, perhaps showing the effects of the recent neglect of that tool.

Ethan Shen

Google, Bing and Babelfish

I’ve just participated in the Which Engine Translates Best? survey, designed to test which among three leading free translation engines is best (for more details, see this post of mine, from a few days ago).

At least for gisting, both Bing Translator and Google Translate can prove surprisingly useful. I had expected Google to be the best of three engines, and, in my opinion, so it proved in this test, but Bing came a close second. A nice thing about Bing is that, unlike Google, it warns of its limitations: “Automatic translation can help you understand the gist of the translated text but is no substitute for a professional human translator” is prominently displayed in the Bing page, while Google Translate says nothing of the sort.

Babelfish made a complete mess of all translation samples (at least for English into Italian: I’ll probably test again the engines using different language pairs), and also turned everything to all uppercase.

Pity that Babelfish is so clearly outmatched by the other engines: with a name that directly refers to Douglas Adams’ stroke of genius, the science-fiction fan in me would love to see it shine brighter.

Which (free) MT is best?

Ethan Shen, a Chinese translator, used for years, while studying in high school and college,  a variety of free MT translation engines. A question remained unsolved for him, however: which MT translation system is best?

To finally settle the question, Shen has devised a comparative study, and is looking for volunteers. Shen has set up a web site in which you can paste or input text to translate. The survey site feeds the source text to three diferent free MT systems (Google Translate, Yahoo/Babelfish and Microsoft Bing's). You are then asked to review the resulting translations, rate them, and add your comments.

Shen is looking for 10,000 testers between now and the end of March, for any of the language pairs supported by these machine translations systems.He will then analyze the results, and I believe he plans to publish the results of his research, or write about them.

If you would like to participate, you can do so by following the link to Which Engine Translates Best? March Madness Edition.

As an enticement to participate in the survey, Shen's company will award a new Apple iPad to a participant in the March Madness contest (you can find the details on Shen's survey site).

Misleading software descriptions: Site Translator

The ZDNet's overview of Site Translator, an automatic web localization tool, states

Site Translator uses automated machine translation technology [that] is capable of translating entire Web sites in a matter of minutes and you do not need to know the translated language. If you need to improve accuracy, Site Translator has a feature called translation memory, which helps you fine-tune exact language phrases [Italics mine].

For all I know, Site Translator might be a useful program, in the right hands. Used by someone who "[does] not need to know the translated language", and who might be mislead into thinking that translation memory, by itself, will somehow help him to "fine-tune exact language phrases", it is a sure recipe for localization disaster.

European Commission Translation Memories Available for Download

The European Directorate General for Translation (DGT) has made publicly accessible its multilingual Translation Memory for the Acquis Communautaire.

The Acquis communautaire is a collection of texts and their translation in 22 languages. It comprises the entire European legislation, including all the treaties, regulations and directives adopted by the European Union (EU) and the rulings of the European Court of Justice.

The memories can be downloaded from The DGT Multilingual Translation Memory
of the Acquis Communautaire: DGT-TM, which also contains an explanation of what the materials available are and how they can be used.

I found the announcement on the Global Watchtower, the bulletin of Common Sense Advisory. The original announcement also includes valuable insight about how translation companies (and I think also translation professionals), will be able to take advantage of this multilingual corpus.

New Azeri-English Translation Software Released

(From Trend)

According to this press release, a new Azeri-English translation software, called Dilmanc, has been released, and it already claims 6500 users.

From what I know, most commercial MT programs have normally been aimed first at much more widespread languages, to take advantage of the larger translation market for those language combinations.

I wonder, though, whether MT translation isn't actually more useful for languages combinations (such as Azeri-English, perhaps?) where the number of professional translators available is limited: in such cases it mught be argued that the choice would not be between machine translation and (better) human translation, but between machine translation and no translation at all.

Consequences of "frictionless communication across languages"

iSpeak.net has an article about the future impact of machine translation: hopes, dreams, possibilities, etc. One thing that jumped to my eye was this sentence:

"Some even believe that frictionless communication across languages would help different cultures and religions to see eye to eye, helping to bring about peace on earth"

Whoever said that, must not have paid much attention to Douglas Adams's Hitchhiker's Guide to the Galaxy:

"Meanwhile, the poor Babel Fish, by effectively removing all barriers to communication between different races and cultures, has caused more and bloodier wars than anything else in the history of creation."

Forthcoming Article about Machine Translation in Scientific American

In the March 2006 issue, Scientific American is going to have an article on "The Elusive Goal of Machine Translation".

The brief summary that appears on Scientific American.com doesn't reveal much: it starts with an example of bad machine translation (this seems to be almost de rigueur in such articles), but at least the subtitle holds promise, as it mentions statistical methods as a way of moving MT "out of the doldrums".

Global Content Management

EContent published a few days ago an interesting article on Global Content Management. Among the highlights of the articles are sound suggestions such as

First and foremost, you must learn to write for translation, which means to write simply, clearly and, above all, to write for reuse.

There are also links to useful sites; for instance after suggesting the above, it links to the home page for Simplified Technical English, where one should be able to find help and guidelines for clear writing.

The article continues with more useful information on translation software (both TM and MT), XML, and other related products.

Although the article is clearly not aimed at translators, but at the users of translation, localization, and allied services, it should also be of interest to most translators.

Interesting article on Google Translator

Google Blogoscoped (Philipp Lenssen) has an interesting article on the current state of the Google machine translation system: Google Translator: The Universal Language.

The article was followed by some lively discussion, and was followed by another interesting article on the Qwikly.com blog.