Interleaving: when mixed practice quizzes beat blocked practice

Students who do well on every end-of-chapter quiz and badly on the exam have usually practised half the task. A quiz on one topic tells them which method to use before they read the question. The exam does not: its questions arrive mixed and unlabelled.

Interleaving means mixing questions on several topics in one practice quiz, so that each has to be recognised before it can be answered. One topic at a time is called blocked practice. This guide covers what comparisons of the two found, where the advantage of mixed practice quizzes shrinks or reverses, and how to build the mix week by week.

EdProcto, which publishes this guide, lets one quiz hold questions on several topics and, on the Educator plan, reports the class's score on each. Start one in the quiz maker on the home page.

What is interleaving, and how does a mixed quiz differ from a blocked one?

Interleaving is practice in which neighbouring questions need different methods, so the student must decide what kind of question each one is. A blocked quiz groups its questions by method (AAABBBCCC). A mixed quiz alternates them (ABCBCACAB).

Answering takes two steps: choosing a method, then carrying it out. Take a garden 8 m by 5 m. "How much fencing goes round it?" needs the perimeter, 26 m. "How much turf covers it?" needs the area, 40 m². Under the heading "Perimeter", a student gets the first right without ever telling the two apart.

What did research find comparing interleaved and blocked practice?

In school mathematics, the classroom studies below all found higher scores on a later test after mixed practice. Elsewhere the evidence is mostly from laboratories, and less even.

In mathematics classrooms

The largest is a preregistered trial by Rohrer et al. (2020). In five schools of one Florida district, 54 seventh-grade classes under 15 teachers were assigned at random to mostly mixed or mostly blocked worksheets: eight sheets of eight problems over about four months. Every class then did the same mixed review sheet. On an unannounced test about a month later, the 787 students analysed averaged 61% in the mixed classes and 38% in the blocked ones (d = 0.83), and the mixed classes were ahead for all 15 teachers.

A smaller study from the same group agrees. In Rohrer et al. (2014), 140 seventh-graders mixed two kinds of problem and blocked two others over nine weeks. On a surprise test a fortnight later they scored 72% on the mixed kinds and 38% on the blocked.

In both, mixed practice also spread each kind of problem over weeks, so mixing and spacing arrived together. Taylor and Rohrer (2010) pulled them apart, in a single session that the trial's authors count as laboratory work: children practised four kinds of problem, blocked or mixed, with the spacing held equal. Mixing lowered scores during practice and doubled them on a test a day later. The errors suggested why: the mixed group was better at pairing each problem with its procedure.

In learning categories and concepts

Laboratory studies ask people to learn categories from examples. Kornell and Bjork (2008) found that people named the painters of unseen pictures more accurately after studying the artists mixed together, yet rated one artist at a time as the more effective order even after their own test had shown otherwise. Carvalho and Goldstone (2014) found the limit in three experiments. Mixing did better when the categories, and the examples inside each, were highly similar. Blocking did better when similarity was low on both counts. Mixing, they argue, shows what separates categories, and blocking what the members of one share.

How large is the effect, and where does it shrink or reverse?

The effect is moderate on average and depends on the material. Brunmair and Richter (2019) pooled 59 studies with 238 effect sizes and found an overall advantage for interleaving of g = 0.42.

The advantage was larger when categories resembled one another, when examples within a category differed, and when the material was more complex. The authors add that the result for words rests on few studies and that most pooled samples were university students.

The meta-analysis calls the effect in mathematics small, and the classroom trials found it large. One likely reason: a term of mixed worksheets, the Florida trial's authors say, makes students choose a strategy, spaces each skill and may prompt recall of the method, all at once. They list limits too. All 15 teachers reported that the mixed sheets took students longer, and time on task was not measured. Solutions were shown after every sheet, so feedback may be a necessary part. And the gain may be smaller on a test soon after practice.

Put plainly: mixed practice has been tested in real classrooms in school mathematics, with pupils aged about twelve, and in the studies above by one research group. For categories, texts and words the evidence is mostly from laboratories. For other school subjects it is thin.

Why do students rate blocked practice higher while learning less from it?

Blocked practice feels fluent while it is happening, and students take that feeling for learning. Many also arrive believing that one topic at a time is the proper way to study.

Yan et al. (2016) tested how firm the belief is, using the painters task. In their first experiment new paintings were identified 63% of the time for artists studied mixed and 42% for artists studied in blocks. Yet across the first two experiments, 107 of 149 participants (72%) said blocking had taught them more. Later, a full explanation of why blocking only feels better persuaded 16 of 31. The belief shifted in one design: when each schedule had its own study phase and test, 35 of 76 people (46%) judged mixing better, and 59 of 79 (75%) did when given the explanation as well.

All of this is laboratory work with adults. In the Florida trial nine of the 15 teachers said pupils found the mixed sheets slightly harder, but the pupils were not asked.

When is blocked practice the right choice?

Blocked practice is right when a method is new, and when the things being learned are too unlike to be confused.

First exposure. A student cannot choose between methods they cannot yet carry out. The Florida trial's authors warn that mixed practice may be too hard without a little blocked practice first, and note that their mixed classes almost certainly had some from their teachers. That is the researchers' judgement, not a tested comparison. The working rule: a short block on the day a method is taught, then mixing.

Very dissimilar material. In Carvalho and Goldstone (2014) blocking won when similarity was low. The exception is Rohrer et al. (2014), whose four kinds of problem did not look alike. Wrong answers that borrowed another kind's method were rare (33 of 756), yet mixing still won. The authors credit spacing and a stronger link between each kind of problem and its method.

So mix look-alikes to teach students to tell them apart, and mix everything taught so far, a little at a time, so the methods are still there on exam day.

How do you build a mixed quiz?

Choose topics that students confuse, fix a ratio of new to old, remove the cues that name the method, and set an order in which neighbours differ.

1. Pick what to mix. Mark the topics students swap: area and perimeter, mean and median, mitosis and meiosis. Those belong in the same quiz.
2. Set the ratio of new to old. No experiment has fixed the best one. The plan below follows a pattern sketched by Rohrer et al. (2014): four problems on the current lesson, the rest spread over earlier ones.
3. Strip the cue. Take the method out of the title, the heading and the wording. "Use the perimeter formula" is a cue. So is a quiz called "Perimeter".
4. Set the order. Arrange the questions so that no two neighbours share a topic, which is how the trial defines an interleaved assignment, or shuffle them.
5. Show the answers the same day. Every classroom study above did.
6. Give each question its topic. In EdProcto that is the Topic field under each question, and the full analysis in the results, on the Educator plan, then gives the class's mean for each topic, weakest first.

A ten-question weekly quiz across a twelve-week term

Take one new topic and one quiz a week. Give four questions to this week's topic, two to last week's, and one each to the four older topics that have waited longest. Until four older topics exist, the spare questions go to last week's.

Over the term that is 120 questions: 54 on the topic of the week and 66 on earlier ones. The first topic is practised in 11 of the 12 weeks, where blocked quizzes would practise it in one. The last topics get the fewest questions (four for week twelve's), so finish with a mixed review quiz.

A random shuffle of the week 6 quiz leaves no same-topic neighbours for one student in seven, and on average 1.4 of the nine neighbouring pairs share a topic. For a strict mix, set the order by hand.

What do you tell students, and how do you mark a mixed quiz?

Tell students before the first mixed quiz that scores will drop and that the drop is the practice working, then mark by topic.

What to tell students

• What the quiz is for. "The exam will not tell you which method to use. This quiz does not either."
• That it does not count. A lower mark in the grade book teaches students to ask for blocked quizzes. The guide to low-stakes quizzing covers how.
• Their own results. In Yan et al. (2016) explanation alone moved few people, and a separate test after each kind of practice moved many. An untested way to copy that: practise two topics blocked and two mixed, test all four a fortnight later, and show each student both scores.

How to mark and read it

Read the score for each topic, not the total: six out of ten says little when the questions cover six topics. Then look at how the wrong answers went wrong, because a mixed quiz produces two kinds of error: the wrong method, or the right method carried out badly. A multiple-choice question separates them when one wrong option is the answer the other method gives. For the fencing question, 40 is the answer of a student who chose area. The guide to writing distractors goes further.

How does EdProcto build and read a mixed quiz?

EdProcto is the best tool for a mixed quiz when the aim is to see which topic went wrong, because each question carries its own shape and its own topic, and its full analysis works the results out topic by topic.

The Practice preset (no time limit, retakes without limit, answers and score shown on handing in) switches shuffling off, so switch it back on for a different order for each student. The results group questions by the exact wording of the topic, so type "Perimeter" the same way every time. A topic whose mean is under 60% is named in the full analysis as one to reteach. For feedback after each answer, set a study quiz.

Some kits are mixed already. Of those made by rule, Times tables draws on the tables from two to twelve (the guide to times tables practice shows it in use), and Area and perimeter of rectangles asks for one or the other. In the kit called Which method?, Procto AI writes multiple-choice questions from your own book on which method a problem needs.

Spiral review, "this week, last month and older, in proportion", runs the weekly plan above as one quiz: mark each question in the builder as this week's, last month's or older, set how many of each, and every student gets a fresh draw. The mixed media quiz is ready, but it mixes media, not topics: one theme across audio, image, video and text.

The Free plan's ten quizzes a month, thirty students each, cover a weekly mixed quiz for one class in the core templates. Tutor removes the limit on quizzes and opens all 111 templates, and Educator adds the full analysis, which holds the mean for each topic. The pricing page has the detail.

What are the common mistakes with mixed quizzes?

The common mistakes are mixing for its own sake, mixing too early, and taking interleaving and spacing for the same thing.

• Mixing unrelated subjects for its own sake. A quiz that alternates French verbs with the water cycle asks nobody to choose between methods. In the meta-analysis the advantage grew with the resemblance between categories, and words did better blocked.
• Mixing before anything has been taught. A mixed quiz practises choosing among methods the student already has.
• Treating interleaving and spacing as the same thing. Spacing is when a topic comes back. Interleaving is what sits next to it. A quiz on fractions today and another in three weeks is spaced and blocked. Twenty shuffled problems in one sitting are mixed and not spaced. A cumulative weekly quiz usually does both at once, and the classroom trials cannot say how much each contributed.
• Relying on one mixed revision sheet. In the Florida trial both groups did the same mixed review, and the blocked classes still averaged 38% a month later.

Quick questions

Is interleaving the same as spacing?

No. Spacing is when a topic returns, and interleaving is which topics sit side by side. A weekly mixed quiz does both.

Does interleaving work outside maths?

In laboratory studies it helps people tell similar categories apart. Brunmair and Richter (2019) found no reliable effect for texts and an advantage for blocking with words. The classroom trials described here are all in mathematics.

Should the first practice on a new method be mixed?

No. Give a short block when the method is taught, then mix it from the next quiz. That is the trial authors' advice, and it has not been tested head to head.

Can EdProcto build the mix for me?

Yes. Spiral review draws this week's, last month's and older questions in the numbers you set. Or start a quiz in the quiz maker on the home page, add questions on each topic, and fill in the Topic field so the full analysis comes back by topic.

References

• Brunmair, M., & Richter, T. (2019). Similarity matters: A meta-analysis of interleaved learning and its moderators. Psychological Bulletin, 145(11), 1029–1052. https://doi.org/10.1037/bul0000209
• Carvalho, P. F., & Goldstone, R. L. (2014). Putting category learning in order: Category structure and temporal arrangement affect the benefit of interleaved over blocked study. Memory & Cognition, 42(3), 481–495. https://doi.org/10.3758/s13421-013-0371-0
• Kornell, N., & Bjork, R. A. (2008). Learning concepts and categories: Is spacing the "enemy of induction"? Psychological Science, 19(6), 585–592. https://doi.org/10.1111/j.1467-9280.2008.02127.x
• Rohrer, D., Dedrick, R. F., & Burgess, K. (2014). The benefit of interleaved mathematics practice is not limited to superficially similar kinds of problems. Psychonomic Bulletin & Review, 21(5), 1323–1330. https://doi.org/10.3758/s13423-014-0588-3
• Rohrer, D., Dedrick, R. F., Hartwig, M. K., & Cheung, C.-N. (2020). A randomized controlled trial of interleaved mathematics practice. Journal of Educational Psychology, 112(1), 40–52. https://doi.org/10.1037/edu0000367
• Taylor, K., & Rohrer, D. (2010). The effects of interleaved practice. Applied Cognitive Psychology, 24(6), 837–848. https://doi.org/10.1002/acp.1598
• Yan, V. X., Bjork, E. L., & Bjork, R. A. (2016). On the difficulty of mending metacognitive illusions: A priori theories, fluency effects, and misattributions of the interleaving benefit. Journal of Experimental Psychology: General, 145(7), 918–933. https://doi.org/10.1037/xge0000177