Sign up free

Translation test questions: how to write and mark them

What a translation question measures, what research on translation in language teaching found, how to write one, and how to mark several right versions.

The EdProcto editorial teamWritten and checked against the research it cites. Last checked . How the guides are made

Guides · Question shapes and when to use them · · 13 min read

How do I write and mark translation test questions for a language class?

Translation test questions should be short, carry one teaching point a sentence, and give enough context that one reading is intended. Mark each against a list of every version you accept, or a rubric with points for each element, fixed before the test. Translating out of the mother tongue tests production; translating into it tests comprehension.

Translation test questions ask a student to put a sentence or a short passage into the language being learned, or out of it. They are quick to write and hard to mark, because a sentence has more than one right version.

This guide covers the research, what the shape measures, and how to write and mark an item, with examples in French and English. EdProcto carries the shape as its Translate template: the student types a version, and you mark it against your own model answer and rubric.

One caution first: a free translation tool on a phone now handles a classroom sentence, so where the test is sat matters too.

Is translation a good way to test a language?

Translation is a sound way to test a language when the item is short, the direction is chosen on purpose and the marking is settled beforehand. The evidence that translating helps learning is real but narrow.

Reverter Oliver (2021) records the traditional view that the Direct Method discouraged translation and that many teachers have since held it to be harmful, although, citing Cook (2010), nothing has shown scientifically that it is. We read only the abstract. The communicative approach is usually named in the same story; no source we could read says so in terms.

Hall and Cook (2012), reviewing the literature on own-language use, write that until recently the teaching literature assumed a new language is best learned monolingually. Students' own languages stayed in wide use in classrooms all the same, and the assumption is now increasingly questioned. Cook (2010) is the book-length argument: its publisher describes a survey of the arguments on both sides that presents translation as an aid to acquisition, teaching and testing. We read that description, not the book.

What studies of translation exercises found

Two experiments point in opposite directions.

  • A gain. Laufer and Girsai (2008) taught ten unfamiliar words and ten collocations to three groups of high-school learners of English: one did meaning-focused tasks, one did vocabulary exercises, and one translated in both directions while the teacher contrasted the two languages. With time on task equal, the translation group significantly outscored the others on every test, straight afterwards and a week later.
  • No advantage. Hummel (2010) had 191 French-speaking students meet English vocabulary by translating into English, translating into French, or copying. All three led to short-term recall, the two directions did not differ, and copying beat both.

The evidence is thin: both measured vocabulary over days. Neither shows what a term of translation exercises does for language learners' speaking or listening.

What does a translation question measure, and what does it miss?

A translation question measures control of two languages at once, and the direction decides which is under test.

Direction It tests What can spoil the reading In EdProcto
Out of the mother tongue Production: vocabulary, grammar and word order together Little: the source is easy to follow Translate
Into the mother tongue Comprehension of the foreign text Weak writing in the mother tongue; a class with several home languages Translate

Buck (1992) tested the second row. Seven raters with no training or instructions marked translations into their first language by 121 Japanese junior college students. Every estimate of reliability was acceptable, and the scores agreed satisfactorily with other measures of passage comprehension. A second study found acceptable construct validity. Buck concluded that rejecting translation on measurement grounds was probably unwarranted, then warned that such a test would often have an unwelcome effect on classroom practice and should be used with great care.

The shape misses speaking, listening and any writing that starts from the student's own idea.

How do you write translation test questions?

Write a short sentence around one teaching point, add the context that fixes one reading, and list the versions you will accept before anyone sits it.

  1. Pick the direction for the skill. Out of the mother tongue for production, into it for comprehension.
  2. Keep it short. A sentence of six to twelve words, or a passage of 30 to 60. That is our working rule, not a research finding: every extra clause multiplies the versions you must list.
  3. Set one teaching point a sentence, a tense, a false friend or a word order, so a wrong answer tells you which rule failed.
  4. Add context until one reading is intended. "Where do you live?" has at least four correct French versions: Où habites-tu ?, Où est-ce que tu habites ?, Où habitez-vous ? and Où est-ce que vous habitez ?. Adding "(to a friend)" halves the list.
  5. Avoid the idiom that rewards a memorised phrase. "It is raining cats and dogs" shows whether one fixed expression was learned, not whether the student can build a sentence.
  6. List the versions, then try the item on a colleague. In EdProcto, choose Translate, type the sentence as the prompt, and put the accepted versions in the model answer box, one a line.

Which versions should you accept? Three sentences into French

Accept every version that is correct French and keeps the meaning, and refuse one only for a reason you could write down.

Sentence and point Accept Refuse, and why
"I have lived in Lyon for three years" (and still do). Depuis with the present. J'habite à Lyon depuis trois ans. Je vis à Lyon depuis trois ans. J'habite Lyon depuis trois ans. Ça fait trois ans que j'habite à Lyon. Il y a trois ans que j'habite à Lyon. J'ai habité à Lyon pendant trois ans. A finished stay.
"She is going to the library this afternoon." A false friend. Elle va à la bibliothèque cet après-midi. The same with cette après-midi, or with the time first. Elle va à la librairie cet après-midi. That is a bookshop.
"I miss my friends." Manquer à. Mes amis me manquent. Mes amies me manquent. Je manque à mes amis. Correct French for "My friends miss me". Je manque mes amis. With a direct object the verb means failing to catch or reach.

We checked the French twice. The grammar of the University of Texas at Austin (n.d.) gives depuis with the present for a state that still holds, pendant with the passé composé for a finished one, and ça fait … que and il y a … que for duration. Larousse (n.d.) builds habiter with or without a preposition, gives après-midi as masculine or feminine, defines librairie as a bookshop, with "library" as an old sense, and illustrates manquer with Ses enfants lui manquent.

The third row shows why the list comes first: the best wrong answer is good French that says the opposite.

How do you mark a translation?

Mark a single sentence against the list of accepted versions, and a passage against a rubric with points for each element. Professional schemes add two ideas worth borrowing: named error categories and severity.

The American Translators Association (2022) sorts errors into three groups: target-language mechanics (grammar, spelling, punctuation, and diacritical marks as a category of their own), meaning transfer (omission, faux ami, literalness, misunderstanding) and writing quality (usage, register). Points follow how far an error disrupts the text; a passage with 18 or more fails, and two graders mark each exam, with a third if they disagree (American Translators Association, n.d.).

The sample scorecard of the MQM framework (MQM Council, n.d.) has four severity levels, neutral, minor, major and critical, weighted 0, 1, 5 and 25. Its evaluators often work with samples of 500 to 20,000 words, so borrow the categories and the idea of severity for a class, not the pass lines.

Holistic or analytic marking, and do markers agree?

Both can work, and less is known about agreement than markers assume. Waddington (2001) applied four methods, error analysis and holistic marking among them, to a translation exam sat by 64 university students. All four correlated significantly with one factor of translation competence, and none proved more valid than another. Yet Eyckmans and Anckaert (2017) observe that, for all the calls for objectivity, the reliability of translation assessment had not been addressed.

For a class, give element points to sentences and short passages, and a band description to longer work. Our guide to short answer questions and a rubric to grade them covers building one.

A rubric for a short passage, with the points

The passage (37 words): Hier, il faisait très froid. Marie a raté son bus, donc elle est arrivée en retard au collège. Elle venait de s'asseoir quand le professeur lui a demandé ses devoirs. Elle les avait oubliés à la maison.

Each of seven elements earns 2 points when meaning and English are both right, 1 when the meaning is there but the English is faulty or the shade wrong, and 0 when the meaning is wrong or missing.

Element Full marks for One student wrote Points Category
il faisait très froid it was very cold it made very cold 1 Literalness
a raté son bus missed her bus missed her bus 2
est arrivée en retard arrived late arrived late 2
au collège at school at college 0 Faux ami
venait de s'asseoir had just sat down came to sit down 0 Misunderstanding
lui a demandé ses devoirs asked her for her homework demanded her homework 1 Faux ami
les avait oubliés à la maison had left it at home had forgotten them at home 1 Grammar

Added by a script, not by hand, that is 7 of 14 points, or 50%. By severity the same answer has three minor errors and two major ones: 13 penalty points on the sample weights above. With the two major errors put right the score would be 11 of 14, or 79%: the case for weighting meaning above polish.

Has machine translation made the translation question pointless?

No, but an unsupervised translation task now shows what the student and a tool can do together. Stapleton and Leung Ka Kin (2019) had primary pupils in Hong Kong write on one topic in English and in Chinese, put the Chinese scripts through Google Translate, and gave both sets to 12 teachers who were not told. The machine-translated scripts were graded equal or higher.

O'Neill (2019) cites research finding that 87.7% of intermediate students of Spanish and French use online translators for graded work at least sometimes. In O'Neill's own study of more than 1,000 compositions, students trained to use Google Translate scored highest of five groups and students with no tool lowest, though two later posttests gave mixed results for the translators.

What follows is practice, not research:

  • Sit it in the room when the score counts.
  • Change the channel. A phrase heard in a clip, or a version given as a spoken answer, is harder to pass to a tool. See our guide to listening questions.
  • Do not lean on the browser. Of EdProcto's integrity measures, "Leaving the tab" and "Copy and paste off" each act with the consequence you chose, and a flag is a record to read, not proof. Nothing in a browser stops a student using a translation tool on another device, and the "Dictionary" and "AI assistants not allowed" settings only state a house rule on the start screen.

How does EdProcto run a translation question?

EdProcto is the best tool for translation questions when several versions are right, because it does not pretend a rule can read a sentence: every answer comes to you with your accepted versions and your element points beside it. Buck's untrained raters reached acceptable reliability; a written scheme can only help.

EdProcto has Translate in the builder now, among its 111 templates. You type the prompt, a passage under it if there is one, a model answer, a rubric and the points. At hand-in the answer waits for you and counts for nothing until it is marked. On the grading page you enter the points and a note, or press "Ask Procto AI" on that answer; it reads one answer at a time against your model answer and rubric, and its mark is saved under its own name for you to keep or change. No rule compares a Translate answer with a list, so accents count as your rubric says.

The Free plan has the core templates. Every template opens on Tutor, which also carries grading by Procto AI when you ask on an answer; pricing has the detail.

If you want a sentence marked by rule

Set it as fill in the blank, which takes "Accepted answers, one a line" and marks all or nothing at hand-in. We ran EdProcto's comparison on the three sentences above.

  • Ignored: capitals, extra spaces, commas, question marks, guillemets and a final full stop. Curly and straight apostrophes are treated alike.
  • Kept: accents, hyphens and apostrophes. "a" is not à, "apres-midi" is not après-midi, "soeur" is not sœur, and "3 ans" is not trois ans.
  • The slip. "One slip in a word of five letters or more still counts" starts switched on. It forgives one slip in the sentence, inside one word of five letters or more: one character wrong, missing or extra, or two neighbours swapped. Every other word is held to the key. With it on, Tes amis me manquent and Mes ami me manquent were both marked wrong against Mes amis me manquent, because the changed words are short, and Mes amis me manquant was marked right. So a pronoun and a short plural are held exactly, and a wrong ending on a longer verb is forgiven. Switch it off for translation.

Finish the list before the quiz opens. A form added afterwards reaches the attempts already handed in when you press "Re-mark this quiz" on the results page. See one-word answers and spelling.

Quick questions

Should a translation question go into or out of the language being learned?

Out of the mother tongue to test production, into it to test comprehension.

How many versions of one sentence should I accept?

Every version that is correct and keeps the meaning. Context in the prompt keeps the list short.

Do accents count in a typed translation?

In EdProcto's Translate shape you read each answer, so they count as your rubric says. In the shapes marked by rule accents are kept, so a for à is wrong unless the slip is on.

Can students use a translation tool during an online quiz?

Nothing in a browser stops a tool on another device. For a score that counts, sit the quiz in the room.

References

Translate

A sentence in one language, typed in another.

In the builder now

See Translate

Read next