Saturday, March 14, 2009

Translation tests v. translation samples

Often translators complain about translation tests, and ask why, instead, they couldn't just send out samples of their translations, perhaps one for each language combination or one for each field in which they specialize.

There may various reasons why a translation company may prefer a translation test to a translation sample:

  • Often the purpose of the test is not only to assess the quality of the translation, but also to see how well the translator follows the instructions given with the test. For example, when I worked in the translation department of a major software company, we used tests to assess candidates for staff translator positions. The tests were short portions of longer documents (about 250 words to translate in a 750-word document), with clear instructions about what to translate and what not, what to do in case of doubt and so on. We rejected many candidates because they would not read through the instructions: if I am looking for a technical translator when I know that each project will come with detailed instructions, I want to screen out the translators who skip the instructions and plunge directly in the translation. This screening prevents many serious problems later.

  • A test translation lets the translation company see how each candidate solved specific translation problems, and compare the quality of a translation with the quality of a different translation of the same source text. This is not possible with translation samples.

  • A translation sample lets translators present teir best work. Fine for them, but less useful for the company: a test shows how you tackle the type of work the translation company would send you.

  • The quality-control process adopted by the translation company may require a test. For example, most ISO-certified companies follow elaborate QC procedures throughout the translation process, including the selection of freelancers. If this is so, they are not going to change their process just because a translator has some sample translation.

  • When you have to evaluate many candidates, it is faster when each test translates the same original, than if you have to shift gears every time, and look at a different translation of a different original. A well-designed test represents a considerable investment of time for the translation company or translation department:
    • the time spent selecting the texts to be translated,
    • the time spent designing the test (choosing which parts of the text to translate, perhaps adding translation problems to see how they would be solved by the candidates, writing and reviewing the instructions for the test, sending out the tests),
    • the time devoted to a first screening of all the tests received to see which could be dismissed out of hand, and then
    • the time spent assessing the tests.
    Bear in mind that well-designed test is not assessed by a person only: at the software company I mentioned before, two translators assessed each test, but when the two evaluations differed, a third evaluator also took part.
These are the main reason a translation test may be more useful than sample translations. This, of course, applies only to tests that are well designed and well administered. Tests that are not well planned are a waste of time for all.


10 comments:

  1. This is true when you're sending out a generic call for translators. The best way to test a new translator (IMHO) is to have them do a small, paid job. You'll only know how good a translator really is when you pay them. This way, you also avoid the problem that most of the people submitting the unpaid trials will be scrubs, while the established translators will be too busy to bother.

    Of course, this approach is impossible when you're dealing with hundreds of applications. It does work, however, when you identify and approach a few potential new translators yourself. It's probably also more effective.

    This is also why, as a translator, I much prefer to have potential clients come to me via word-of-mouth introductions, rather than responding to the monthly resume trawls by McAgencies.

    ReplyDelete
  2. Hi Ryan:

    I don't think that the difference is generic v. specific (specific to a particular project or to certain particular requirements, I presume), but rather on such things as how many translators you are looking for, how much time you have available to find the right ones, and the resources you have for the evaluation process.

    For a small company which is looking for one translator only (or at most just a few of them), and that has time enough that if the first one doesn't pan out, they can try a second (or third) one, small assignments as you described are indeed a good strategy (it is in fact what we normally do, given our very small size and the fact that we look for outside translators only very infrequently).

    ReplyDelete
  3. Hi Riccardo: By "generic" I mean an open call for resumes, rather than one targeted to specific people.

    Of course, if you need a lot of people right away, that's pretty much your only option, but you'll have to know that a good portion of chaff is going to get through (you'll also miss out on some wheat, but that isn't really the problem in this scenario).

    ReplyDelete
  4. Great post, Riccardo. I agree with everything you've said, I just wish that more agencies would pay for test translations and realize that they are a cost of doing business. Yes, you're taking a gamble that the person will fail, but the translator is absorbing the cost of the rest of the application process; filling out the application form, providing references, etc.

    ReplyDelete
  5. Riccardo's points very much apply also to tests for localizer, language specialist and terminologist positions. Over the years I designed quite a few and also in my experience a surprisingly large number of candidates failed to follow instructions and/or did not realize the importance of an additional section where they were asked to comment on the test itself (e.g. how they had dealt with any problem they might have found). It was disappointing but it made the first screening much quicker.

    ReplyDelete
  6. many customers prefer to get translation test samples before they choose a translation company. so the translation companies request the translators to try a sample before they get assigned with the work. this applies to large volumes of work. for small ones i think we go straight with the preferred translators.

    ReplyDelete
  7. Maybe my question may sound quite silly (but, it always comes to my mind that people from the law field are allowed to do tests using some "tools", such as rules, and regulation books - Context: Brazil). So, may translators be also permitted to use dictionaries in tests?

    ReplyDelete
  8. Never heard of a translation test for a translation company that required it to be done without a dictionary.
    These are not exams in a classroom... they are tests done by the translators at their own computers.

    ReplyDelete
  9. One of my regular clients was an agency in Delhi. It was competing for a big assignment from a new client, who wanted a test translation. This test was passed on to me with the request that I should not charge it as the agency itself will not be paid.

    I pointed out very politely, while the agency is a new service provider for the end client, I am already well-known to the agency and I need not prove myself to it. Further, this was the agency's bid to get a new client and it has to invest something in that process, which in this case is represented by its payment to me.

    The agency was convinced and paid me. It got the work and I got the maximum of the same forwarded to me afterwards.

    Regards,
    Dondu N. Raghavan

    ReplyDelete
  10. Nice post Ricardo. I agree with most of your points.

    ReplyDelete

Thank you for your comment!

Unfortunately, comment spam has grown to the point that all comments need to be moderated. All legitimate comments will be published as soon as possible.