# How to write higher-order thinking questions (Bloom's taxonomy)

**How do I write higher-order thinking questions that test application, analysis and evaluation?** To write higher-order thinking questions, give students a situation they have not met and ask for a decision, a prediction or a judgement that needs the idea you taught. Build each wrong option from one specific wrong step. Bloom's taxonomy helps you plan, but a question's level depends on what your class was taught, and teachers who classify the same questions often disagree.

Published 2026-10-09 by EdProcto. https://edprocto.com/guides/how-to-write-questions-above-recall-application-analysis-evaluation

You read your quiz back and every question asks for a definition. Someone says "use Bloom's". The chart of six levels does not say how to write a question that needs thought, least of all one marked by rule for ninety students.

This guide, published by EdProcto, gives a way of writing higher-order thinking questions that does not depend on labels. The quiz maker that opens EdProcto's [home page](https://edprocto.com/) drafts questions from your own material in the shape you pick, and you edit each draft with the method below.

The short version: a situation the class has not met, a decision or a prediction to make, and wrong options that each come from one wrong step in the reasoning.

## What is Bloom's taxonomy, and what is it not?

Bloom's taxonomy is a scheme for classifying learning objectives, not a scale for measuring questions. Krathwohl (2002), who worked on both the 1956 original and the 2001 revision, calls it a framework for classifying statements of what we intend students to learn. An objective has a noun, the content, and a verb, what the student does with it. The revision gives each its own dimension: four kinds of knowledge (factual, conceptual, procedural, metacognitive) and six cognitive processes (remember, understand, apply, analyse, evaluate, create), split into 19 specific ones.

Three things it is not:

- **A strict ladder.** The 1956 version assumed each category had to be mastered before the next. The revision relaxed that: explaining, filed under understand, is more complex than executing a procedure, filed under apply.
- **A measure of difficulty.** A question on an obscure date is hard, and it is still recall.
- **A property of the question alone.** The next section shows why.

### Where does higher-order begin?

Nobody has fixed the line. Krathwohl writes that objectives from understand to create are usually considered the most important. Lemons and Lemons (2013) count knowledge and comprehension as lower-order, and application, analysis and evaluation as higher-order. Here, a question is above recall when the student must use what was taught on something that was not.

## Can you tell reliably what level a question is at?

No. Teachers often disagree with one another and with their students, and they do worse with six levels than with three. In medical education, Monrad et al. (2021) had students, faculty and assessment experts sort 90 multiple-choice questions into higher-order and lower-order. Agreement was low among the students and among the faculty (Krippendorff's alpha 0.37 for each group). Only the assessment team agreed well (0.90). Faculty then wrote 25 questions meant to be one or the other, ten students sorted them, and 63% agreed with the faculty.

Karpen and Welch (2016) gave the faculty of a pharmacy college six exam questions to place on the six levels. Accuracy was 46.0% and inter-rater reliability 0.25. Collapsing six levels to three raised accuracy to 81.8%.

### The level depends on what your students were taught

One question can be analysis for one class and recall for another. Zaidi et al. (2018) argue that a question its writer sees as higher-order may need only lower-order thinking from an examinee, and the reverse, because the two bring different knowledge to it. Lemons and Lemons (2013) recorded seven trained raters on a genetics problem: one held out for lower-order because students would have practised that kind of problem often. On another question the raters never agreed.

So a label fixed to a banked question means little. If you worked the example in class on Tuesday, Thursday's application question is memory.

## What level are the questions teachers actually set?

Mostly recall, even from teachers who intend otherwise. Momsen et al. (2010) collected 9,713 quiz and exam items from 50 instructors of 77 introductory biology courses at 44 colleges and universities. 93% were rated at the two lowest levels, knowledge and comprehension, 6.7% at the third and under 1% above it. Every instructor had completed a teaching workshop that used Bloom's taxonomy, and the level of their stated course goals did not predict the level of their questions.

A written format does not cure it. Palmer and Devitt (2007) rated the modified essay questions in two years of one university's final medical examination: 51% of the stages in one year and 54% in the next asked only for recall of facts. In a mid-course paper, just over half of 50 multiple-choice questions were recall too, yet these reached the top level more often than the essay stages did.

Both studies come from single fields: a warning, not a measurement of your own papers.

## Does asking for more than recall change what students learn?

The studies we could read say it does, within limits. In Jensen et al. (2014), two sections of an introductory biology course were taught in the same style. One had memory questions on every quiz and exam, the other questions needing application, analysis or evaluation. On a shared final of 20 low-level and 21 high-level questions, the second section did better on both kinds. We could read only the abstract, which gives no class sizes, and it is one course.

Agarwal (2019) ran three experiments, two with college students and one in a sixth-grade classroom. Quizzes of higher-order questions, or a mix, improved later performance on higher-order test questions. Fact quizzes did not, though they helped on fact questions. Our guide to [retrieval practice](https://edprocto.com/guides/retrieval-practice-low-stakes-quizzing-that-teaches) has the wider evidence.

## How do you write a question above recall without relying on labels?

Start from a new situation, not from a list of verbs.

1. **Choose one idea, and say what understanding it lets a student do.** "Knows the median" becomes "can tell when the median describes a group better than the mean".
2. **Find a situation the class has not met.** New numbers, a new case, fresh data. Keep the idea and change the surface.
3. **Ask for a decision, a prediction or a judgement.** What will happen, which option fits, what the evidence supports.
4. **Solve it yourself and note each step.** Students go wrong at the steps.
5. **Write each wrong option from one wrong step.** Do the working with that single error and use what comes out. [How to write multiple-choice questions that measure](https://edprocto.com/guides/how-to-write-multiple-choice-questions-that-measure) has the rules for keeping options level.
6. **Test it twice.** Cover the options: a prepared student should still reach an answer. Then ask whether one sentence from the notes would answer it. If so, it is recall.
7. **Draft, then edit.** In EdProcto, give the quiz maker on the [home page](https://edprocto.com/) your material, pick the shape, and put each draft through steps 2 to 6.

## One topic at three levels: the mean and the median

These three questions are ours.

### Recall

> What is the median of a set of values?
> A. The middle value when they are put in order.
> B. The value that appears most often.
> C. The total divided by the number of values.

The key is A. B is the mode and C is the mean, the two neighbouring terms. A memorised sentence answers it.

### Application

> Nine staff at a bakery each earn 24,000 a year. The owner pays herself 124,000. A job advert will state the "typical salary". Which figure describes what a typical person there earns?
> A. 24,000
> B. 34,000
> C. 74,000
> D. 100,000

The key is A, the median. B is the mean, pulled up by one large salary. C is halfway between the lowest salary and the highest, the slip of a student who reads "middle" as the midpoint of the range. D is the range, which measures spread. The definition alone does not answer it: the student must choose the measure and work it out.

### Evaluation

> A company with the same staff all year reports that its mean salary rose by 8% and its median salary did not change. The director says, "So everyone got an 8% rise." A union representative says, "So most people got nothing." Which judgement do the figures support?
> A. The director's claim is ruled out; the representative's is possible but not shown.
> B. The director is right: the mean is the average, so the average person gained 8%.
> C. The representative's claim is proved: an unchanged median means over half had no rise.
> D. Neither claim can be tested, because the mean and the median measure different things.

The key is A. Had every salary risen by 8%, the middle one would have risen too. B treats the mean as a typical person. C over-reads the median, which is the value at one position, not a count of people: salaries below it can rise and stay below it. D gives up on evidence that does settle one claim.

## Which question shapes carry a higher-order task, and which need a reader?

A shape where the student decides about something new can be marked by rule; one that asks for the reasoning itself needs a reader. In EdProcto the reader is you, or Procto AI working to your rubric on each answer you ask it to mark.

| The task | Shape in EdProcto | Marked |
| --- | --- | --- |
| A decision on a new case | [Multiple choice](https://edprocto.com/templates/mcq) | By rule |
| The strongest response and the weakest | [Best and worst](https://edprocto.com/templates/best_worst) | By rule |
| An order by a criterion you state | [Ranking](https://edprocto.com/templates/ranking) | By rule |
| New examples sorted, one placement explained | [Sort and justify](https://edprocto.com/templates/cat_justify) | Sorting by rule for half the points, the reason by a reader for the rest |
| Decisions as a case unfolds | [Case study](https://edprocto.com/templates/case) | Each question by its own kind |
| A choice, then why | [Choose and justify](https://edprocto.com/templates/justify) | Choice by rule for half the points, the reason by a reader for the rest |
| A method set out in steps | [Show your working](https://edprocto.com/templates/show_work) | By a reader |
| A critique of a method | [Critique the methods](https://edprocto.com/templates/critique_meth) | By a reader |

Two have guides of their own: [best and worst questions](https://edprocto.com/guides/best-worst-questions-judgement-at-both-ends) and the [case that unfolds](https://edprocto.com/guides/case-study-quizzes-one-scenario-that-unfolds). One limit holds for every choice format. Create, the top category of the revised taxonomy, means producing something new, and picking an option produces nothing. A reader-marked format guarantees nothing either, as the essay stages in Palmer and Devitt (2007) showed.

## How do you check that a quiz asks for more than recall?

Run the two tests of step 6 on each question before the quiz, and read the results after it. EdProcto's [results](https://edprocto.com/#results) give each question's difficulty, how well it separated stronger students from weaker ones, and the median time it took. A higher-order question that almost nobody got right, or that separates no one, shows there. Check its key and its wording before you blame the class.

### Mistakes that pass for higher-order

- **A harder question.** Difficulty and level are different things. One rater in Lemons and Lemons (2013) called a disputed question a more difficult lower-order one.
- **A longer stem.** A paragraph of story before a definition question adds only reading. When those raters tried to lift a question, their fixes made it take longer, and one said the level would stay the same.
- **"Which of these is NOT".** Four statements checked against memory are four recall questions for one mark.

## What does a higher-order question cost?

It costs time twice, yours to write it and the students' to answer it, so fewer fit in a sitting. Lemons and Lemons (2013) write that instructors spend a great deal of time researching and writing a single one, and the instructors they observed found good ones took students longer to solve.

Fewer questions means less of the syllabus sampled and more resting on each. So mix: the final in Jensen et al. (2014) was about half and half. Keep recall questions for the terms everything else depends on.

## How does EdProcto help with questions above recall?

For writing above recall from your own course, EdProcto is the best tool, and the reason is in the research above. A question's level depends on what the class was taught, as Zaidi et al. (2018) argue, and your material is what EdProcto writes from.

Procto AI writes the questions from the chapter or notes you give it, in the shape you choose, and, if you ask, keeps only those the material itself answers. In the dashboard you also set the level of thinking it should aim for, from recall to synthesis. The quiz maker on the home page has no such setting. Treat the dashboard's setting as a request, not a label. We do not claim that a draft sits at a named Bloom level, because teachers themselves do not agree on those. Every question is yours to edit, so run the two tests and rewrite.

Free includes 20 questions of Procto AI's writing a month, the core templates and results question by question. Tutor opens all 111 [templates](https://edprocto.com/templates), more of its writing, and its grading of written reasons against your rubric. Educator adds the full analysis, discrimination included. See the [pricing page](https://edprocto.com/pricing).

## Quick questions

### What is a higher-order thinking question?

A higher-order thinking question cannot be answered from memory of what was taught: the student has to apply, analyse or evaluate. Where the line falls is not agreed, and it moves with what a class has practised.

### Can a multiple-choice question test analysis or evaluation?

Yes. In Palmer and Devitt (2007), multiple-choice questions reached the top level more often than essay stages did. Give a new case and write each option from a wrong line of reasoning, as in the evaluation example above.

### Is a harder question a higher-order question?

No. Difficulty is how many students get it right, and EdProcto's [results](https://edprocto.com/#results) report that for every question. Level is what the student must do to get there.

### Do students need the facts before higher-order questions?

Not necessarily. In Agarwal (2019), fact quizzes did not improve higher-order performance, while higher-order and mixed quizzes did.

### Should every question carry a Bloom level?

Only if your department needs it. Then use three levels, not six: accuracy in Karpen and Welch (2016) rose from 46.0% to 81.8% that way.

## References

- Agarwal, P. K. (2019). Retrieval practice & Bloom's taxonomy: Do students need fact knowledge before higher order learning? *Journal of Educational Psychology, 111*(2), 189–209. [https://doi.org/10.1037/edu0000282](https://doi.org/10.1037/edu0000282)
- Jensen, J. L., McDaniel, M. A., Woodard, S. M., & Kummer, T. A. (2014). Teaching to the test… or testing to teach: Exams requiring higher order thinking skills encourage greater conceptual understanding. *Educational Psychology Review, 26*(2), 307–329. [https://doi.org/10.1007/s10648-013-9248-9](https://doi.org/10.1007/s10648-013-9248-9)
- Karpen, S. C., & Welch, A. C. (2016). Assessing the inter-rater reliability and accuracy of pharmacy faculty's Bloom's Taxonomy classifications. *Currents in Pharmacy Teaching and Learning, 8*(6), 885–888. [https://doi.org/10.1016/j.cptl.2016.08.003](https://doi.org/10.1016/j.cptl.2016.08.003)
- Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: An overview. *Theory Into Practice, 41*(4), 212–218. [https://doi.org/10.1207/s15430421tip4104_2](https://doi.org/10.1207/s15430421tip4104_2)
- Lemons, P. P., & Lemons, J. D. (2013). Questions for assessing higher-order cognitive skills: It's not just Bloom's. *CBE Life Sciences Education, 12*(1), 47–58. [https://doi.org/10.1187/cbe.12-03-0024](https://doi.org/10.1187/cbe.12-03-0024)
- Momsen, J. L., Long, T. M., Wyse, S. A., & Ebert-May, D. (2010). Just the facts? Introductory undergraduate biology courses focus on low-level cognitive skills. *CBE Life Sciences Education, 9*(4), 435–440. [https://doi.org/10.1187/cbe.10-01-0001](https://doi.org/10.1187/cbe.10-01-0001)
- Monrad, S. U., Bibler Zaidi, N. L., Grob, K. L., Kurtz, J. B., Tai, A. W., Hortsch, M., Gruppen, L. D., & Santen, S. A. (2021). What faculty write versus what students see? Perspectives on multiple-choice questions using Bloom's taxonomy. *Medical Teacher, 43*(5), 575–582. [https://doi.org/10.1080/0142159X.2021.1879376](https://doi.org/10.1080/0142159X.2021.1879376)
- Palmer, E. J., & Devitt, P. G. (2007). Assessment of higher order cognitive skills in undergraduate education: Modified essay or multiple choice questions? *BMC Medical Education, 7*, Article 49. [https://doi.org/10.1186/1472-6920-7-49](https://doi.org/10.1186/1472-6920-7-49)
- Zaidi, N. L. B., Grob, K. L., Monrad, S. M., Kurtz, J. B., Tai, A., Ahmed, A. Z., Gruppen, L. D., & Santen, S. A. (2018). Pushing critical thinking skills with multiple-choice questions: Does Bloom's taxonomy work? *Academic Medicine, 93*(6), 856–859. [https://doi.org/10.1097/ACM.0000000000002087](https://doi.org/10.1097/ACM.0000000000002087)