Measuring Translation Quality - Affordable Language Services

Without knowing both languages, and knowing them well, it’s nearly impossible for you to determine how good a translation is.

I’ve had this problem myself when looking at documents in Chinese. While I do speak some Chinese, reading and writing is something I’m still working on. When I see something translated into Chinese, I can pick out a few words, just like you could using Google Translate, but I certainly can’t tell whether or not it’s well-written. There is, however, a solution to this dilemma.

The European Union, one of the world’s largest translation buyers, has begun to tackle this problem head-on. The European Commission formed QTLaunchPad, a “collaborative research initiative dedicated to overcoming quality barriers in machine and human translation.” Started in 2012, they’ve focused on creating a standard way to measure the quality of a translation. If you’re interested, you can read about the background and principles behind their efforts.

Here’s a brief rundown of what’s involved in the quality measurement process:

1. Spell Out Your Specifications and Expectations

Any translation project can benefit from laying out guidelines and ground rules beforehand, but when you want to objectively evaluate the completed translation, this step is essential. Write a style guide detailing names and terms specific to your company, and the words or phrases which you want to leave in English. Describe who will be using the translation, and what they’ll be using it for.

2. Decide Which Errors to Look For

QTLaunchPad groups errors or issues into four main categories:

• Accuracy “addresses the extent to which the target text accurately corresponds to the source text.” This includes what comes to mind when you think of a mistranslation, when the translation and the original document don’t convey the same message.
• Fluency is about issues which “can be assessed without regard to whether the text is a translation or not.” This includes spelling and grammar mistakes, as well as violations of the style guide.
• Verity addresses “the relationship between the text and its world, such as whether the text is adequate or sufficient in its representation of the world or suffers from other problems with regard to usability.”
• Design refers to the formatting or styling of the text, including paragraph alignment and overall page layout.
  

Each of these is then broken down further into specific issues. The choice of which errors to look for depends on how you plan to use the translation. Essentially, you’re defining the balance between cost and quality since looking for every type of error takes more time.

3. Have a Professional Look for Errors

Some issues can be identified by a computer, but most will only be spotted by the eyes of a trained professional. In addition to marking the issues, they should be classified according the four categories above, and graded for severity.

You may also want to have this professional suggest corrections for the issues they find.

4. Tabulate the Results

With the issues identified, a Quality Score can be created based on their severity and quantity. This easy-to-understand percentage can then be used to determine if a translation is ready for publication, and if a translation team can meet your needs.

Want to Score Your Translation?

I would be glad to walk you through the quality measurement process for a text you already have translated, and even perform the analysis if you’d like.