# Item analysis for teachers: what the numbers mean

**What do the numbers in an item analysis mean?** Item analysis turns a class's answers into four checks. Difficulty is the share who got a question right. Discrimination shows whether students who scored well overall got it right more often than those who scored badly, and a negative value usually means a wrong key. Option counts show which wrong answers drew whom. Cronbach's alpha says how consistently the whole quiz ranked the class.

Published 2026-10-09 by EdProcto. https://edprocto.com/guides/item-analysis-for-teachers-what-the-numbers-mean

A results page gives every question a row of figures, and each answers one plain question. Item analysis is the name for that arithmetic: how hard the question was, whether it separated the students who know the material from those who do not, which wrong answers were chosen and by whom, and whether the total can be trusted.

EdProcto runs these calculations on a quiz's answers and shows them in its [results](https://edprocto.com/#results), each with a sentence saying what it found.

The short version: a question nearly everyone got right, or wrong, tells you little. A discrimination figure below zero means check the key. A wrong option favoured by your strongest students points at the question. The reliability of a short quiz is a hint, not a verdict.

## How hard was the question? Reading item difficulty

Item difficulty is the share of students who got the question right, so a higher figure means an easier question: at 0.90, nine in ten were right. With part marks, EdProcto counts the share of the available marks earned.

Very easy and very hard questions tell you little: almost nobody is left to tell apart. The University of Washington (n.d.), in its guide to test-scoring reports, says a question separates students best a little above halfway between a guessing score and a perfect one: about 74% correct for four options and 85% for true or false.

Easy is not a fault: the same guide notes that very easy and very hard questions are often needed to cover the course. EdProcto prints a sentence only at the ends: at 0.90 or above the question "says little about who knows more", and below 0.30 the advice is "Check the key first, then whether this was taught".

## Did the question separate students who know the material from those who do not?

A question discriminates when students who did well on the whole quiz got it right more often than those who did badly.

### The discrimination index: upper and lower groups

Rank the class by total score, take the top 27% and the bottom 27%, and subtract the share of the lower group who were right from the share of the upper group. The index runs from −1 to 1: at 0 the groups did equally well, and below 0 the lower group did better.

The 27% comes from Kelley (1939), who argued that when scores follow a normal curve, 27% at each end gives the two groups that are most certainly different. It is a convention: upper and lower quarters are also common (Rush et al., 2016). EdProcto uses 27%, rounded to whole students, and gives no index under six submissions. Students tied on the score at the edge of a group share its last places, each counting for a part, so the index does not depend on the order the attempts came in.

### The point-biserial correlation

The point-biserial uses every student, not only the two ends. It is the correlation, from −1 to 1, between the mark on this question and the score on the rest of the quiz, which is how EdProcto computes it: the question is never compared with itself.

### What a low or negative value usually means

A negative value usually means a wrong key, and a value near zero an ambiguous question or content that was not taught. On the University of Washington (n.d.) reading, low values often go with ambiguous wording, and a negative one may mean that students who knew the material chose a correct answer the key does not credit. Where nobody met the material, both groups guess.

Rules of thumb are conventions, and they vary between textbooks. Rush et al. (2016), from 1,925 veterinary examination questions, give about 0.20 as the target for the index, except on questions meant to be easy or hard, and want anything under 0.10 or negative reviewed. The University of Washington page calls the correlation good above 0.30 and poor below 0.10. EdProcto flags every negative index, and an index under 0.20 only when the difficulty lies between 0.30 and 0.90.

## Which wrong answers were chosen, and by whom?

Option counts show how many students chose each answer, and the useful part is who they were. Examiners call wrong options distractors, and reading them distractor analysis. For each option of a choice question, such as [multiple choice](https://edprocto.com/templates/mcq), EdProcto lists the number who took it and their average score on the whole quiz.

A wrong option favoured by the strongest students points at the question: if its takers outscore the key's, suspect the key. One whose takers scored low is a shared misconception, to be fixed in the teaching. One nobody chose is doing no work. EdProcto says so when a wrong option with three or more takers draws more than the key, and when, among ten or more students, one draws none.

Idle options are common: among 1,542 wrong options in 514 questions written by nursing teachers, Tarrant et al. (2009) found 35.1% chosen by fewer than 5% of students and 30.6% with a positive discrimination statistic, drawing the upper group more than the lower. Our guide to [writing multiple-choice questions](https://edprocto.com/guides/how-to-write-multiple-choice-questions-that-measure) shows how to build wrong options from real errors.

## Can the total be trusted? Reliability and Cronbach's alpha

Reliability asks whether the quiz ranked the class consistently, and Cronbach's alpha is the usual estimate. When students who get one question right tend to get the others right, alpha climbs towards 1. Cronbach (1951) showed that it is the mean of every split-half coefficient a test could yield.

### Why short quizzes have low alpha

Alpha depends on length: Tavakol and Dennick (2011) state that too short a test reduces it and that adding related questions raises it. A five-question check with a low alpha is not broken, only short, and like [exit tickets](https://edprocto.com/guides/exit-tickets-three-questions-and-a-reflection) it was written to decide what to teach next, not to rank anyone.

### What alpha does and does not mean

A low alpha, on Tavakol and Dennick's (2011) account, comes from few questions, questions that do not hang together, or a quiz covering several different things. It also belongs to one class on one day: they call it a property of one group's scores.

A healthy alpha does not prove the questions sound. On four medical school papers, Downing (2005) found flawed questions were harder and changed who passed, yet had little effect on test reliability.

Conventions vary here too: Tavakol and Dennick (2011) report acceptable values from 0.70 to 0.95. The University of Washington (n.d.) reads classroom tests more gently: 0.70 to 0.80 is good, and 0.50 to 0.60 suggests revision unless the test has ten questions or fewer. Once eight students have submitted, EdProcto prints one of three sentences: the quiz measured inconsistently under 0.50, consistently at 0.80 or above, and fairly consistently in between. An estimate below zero, which a few questions that disagree with one another can give, is shown as 0.

## A worked example: ten students and four questions

In this invented class each cell is the option the student chose, the key is in the heading, and Q4 had a fourth option, D, that nobody chose.

| Student | Q1 (key B) | Q2 (key C) | Q3 (key A) | Q4 (key B) | Marks |
| --- | --- | --- | --- | --- | --- |
| 1 | B | C | B | B | 3 |
| 2 | B | C | B | B | 3 |
| 3 | B | C | B | B | 3 |
| 4 | B | C | B | C | 2 |
| 5 | B | C | B | C | 2 |
| 6 | B | C | B | A | 2 |
| 7 | B | C | C | C | 2 |
| 8 | B | A | B | C | 1 |
| 9 | B | B | C | A | 1 |
| 10 | C | A | A | C | 1 |

**Difficulty.** Count the right answers in each column and divide by ten.

**Discrimination index.** 27% of ten is 2.7, so each group is three students: 1 to 3 and 8 to 10. On Q2 all of the upper group were right and none of the lower: 1.00 − 0.00 = 1.00. On Q3 it was none against one: 0.00 − 0.33 = −0.33.

| Question | Difficulty | Discrimination index | Point-biserial |
| --- | --- | --- | --- |
| Q1 | 0.90 | 0.33 | 0.05 |
| Q2 | 0.70 | 1.00 | 0.43 |
| Q3 | 0.10 | −0.33 | −0.67 |
| Q4 | 0.30 | 1.00 | 0.43 |

**Point-biserial.** On Q3 the one student marked right scored 0 out of 3 on the other questions, and the nine marked wrong averaged 2.1. Hence −0.67.

**Options.** On Q3 the seven who chose B averaged 57.1% on the quiz, against 25% for the one who chose the keyed A. On Q4 the three who chose the key averaged 75% and the five who chose C 40%.

**Alpha.** Alpha is the number of questions over one fewer, times one minus the sum of the questions' variances over the variance of the totals. A right-or-wrong question's variance is its difficulty times one minus its difficulty, scaled by 10/9 for ten students: 0.100, 0.233, 0.100 and 0.233, or 0.667 in all. The totals' variance is also 0.667, so alpha is 4/3 × (1 − 0.667/0.667) = 0.00.

### Reading the four questions

Q1 is easy, and its 0.33 is one student: in a group of three each student is worth 0.33. Q2 works. Q3 has every sign of a wrong key: one in ten right, negative on both measures, and a "wrong" option held by the best scorers. Q4 is hard and sound: its popular wrong answer belongs to the weaker students, so it is a misconception to teach.

Suppose you reread Q3 and find B is right. Re-marked, its difficulty becomes 0.70, the class mean rises from 50% to 65%, six students instead of three reach a pass mark of 60%, and alpha moves from 0.00 to 0.75. Students 7 and 8 are now tied on the edge of the lower group, so they share its last place, and the index on Q2 and on Q3 comes out at 0.83.

EdProcto's own analysis code, run on this class, returned the same figures and the sentence "Check the key on Q3: the stronger students got it wrong more often than the weaker ones."

## What should you do with each finding?

Do one of four things: fix the key and re-mark, rewrite the question, reteach the topic, or leave it alone. Start with the keys, because a wrong key bends every other figure. EdProcto releases no score that you have not released, so on a quiz that counts, read the analysis first.

| What you see | Likely cause | What to do |
| --- | --- | --- |
| Negative discrimination, and a wrong option taken by the top scorers | Wrong key | Fix the key and re-mark |
| Discrimination near zero at middling difficulty | Ambiguous wording | Rewrite the question |
| One wrong option popular among low scorers | A misconception | Reteach it and keep the question |
| A topic mean under 60% in EdProcto's [results](https://edprocto.com/#results) | Not taught, or not learned | Reteach the topic |
| An option nobody chose | An implausible wrong answer | Replace it before reuse |
| Difficulty of 0.90 or more, on purpose | A question everyone should get | Leave it alone |

## How rough are these figures in a small class?

With thirty students the figures are rough, and with ten they are hints. In a class of thirty each 27% group is eight students, so one changed answer moves the index by 0.125. With ten each group is three, and one answer moves it by 0.33. Alpha is as jumpy: change any single mark in the re-marked example and it lands between 0.51 and 0.82.

The University of Washington (n.d.) passes on a textbook caution that item analysis data are tentative, swayed by the number of students and by chance. Tarrant et al. (2009), with 73 to 146 students a paper, still warn that their idle options might behave differently elsewhere.

So use a small class's figures to choose which questions to reread, not to settle an argument. Where several classes sit one quiz the pooled figures are steadier: see [common assessments](https://edprocto.com/guides/common-assessments-one-quiz-across-every-class-in-a-year-group). Under five submissions, EdProcto says that the numbers describe those students, not yet the class.

## What can item analysis not tell you?

Item analysis cannot tell you whether the quiz covered the right content. Every figure judges a question against the total on the same quiz, so a paper that consistently tests the wrong things will produce handsome numbers. The University of Washington (n.d.) quotes the textbook caution: these figures reflect internal consistency, and judging validity needs a criterion from outside the test. It adds that a question on a different skill from the rest may correlate poorly with the total and still be needed.

Nor does it say where a pass mark belongs: [setting a pass mark you can defend](https://edprocto.com/guides/setting-a-pass-mark-you-can-defend) has a short table of what a panel checks in these figures after an exam.

## How does EdProcto show item analysis?

EdProcto is the best tool for this job when there is a class tomorrow and no hour for a spreadsheet, because it does the arithmetic above for you and writes each finding as a sentence, such as the instruction to check the key on Q3.

Build the quiz on the [home page](https://edprocto.com/), in a shape from the [templates](https://edprocto.com/templates). Objective questions are marked by rule at once. The analysis gives each question's difficulty, its discrimination as upper minus lower 27%, its point-biserial, the takers of each option with their mean score, and Cronbach's alpha for the quiz.

Each topic you tag gets a mean. Pace flags a question that took over three times as long as a typical one, and a student who finished in a quarter of the usual time with a low score.

The Free plan carries marking by rule and the results, question by question. The Educator plan adds the full analysis, which its plan card lists as discrimination, distractors, reliability and what to reteach. [Pricing](https://edprocto.com/pricing) has every plan.

## Quick questions

### What is a good discrimination index?

There is no single line: about 0.20 is one published target (Rush et al., 2016), and in a class of ten any value is only a hint.

### Is a Cronbach's alpha below 0.7 bad for a classroom quiz?

Not for a short one. Alpha falls as a quiz gets shorter, and the University of Washington (n.d.) asks for revision at 0.50 to 0.60 only when a test has more than ten questions.

### Should I delete a question with negative discrimination?

Read it first. If the key was wrong, correct it and re-mark. If two answers are defensible, rewrite the question and decide whether it should count this time.

### Where does EdProcto show these numbers?

In the [results](https://edprocto.com/#results) of a quiz, with the findings in plain sentences. Free carries the results question by question and Educator the full analysis: see [pricing](https://edprocto.com/pricing).

## References

- Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. *Psychometrika, 16*(3), 297–334. [https://doi.org/10.1007/BF02310555](https://doi.org/10.1007/BF02310555)
- Downing, S. M. (2005). The effects of violating standard item writing principles on tests and students: The consequences of using flawed test items on achievement examinations in medical education. *Advances in Health Sciences Education, 10*(2), 133–143. [https://doi.org/10.1007/s10459-004-4019-5](https://doi.org/10.1007/s10459-004-4019-5)
- Kelley, T. L. (1939). The selection of upper and lower groups for the validation of test items. *Journal of Educational Psychology, 30*(1), 17–24. [https://doi.org/10.1037/h0057123](https://doi.org/10.1037/h0057123)
- Rush, B. R., Rankin, D. C., & White, B. J. (2016). The impact of item-writing flaws and item complexity on examination item difficulty and discrimination value. *BMC Medical Education, 16*, Article 250. [https://doi.org/10.1186/s12909-016-0773-3](https://doi.org/10.1186/s12909-016-0773-3)
- Tarrant, M., Ware, J., & Mohammed, A. M. (2009). An assessment of functioning and non-functioning distractors in multiple-choice questions: A descriptive analysis. *BMC Medical Education, 9*, Article 40. [https://doi.org/10.1186/1472-6920-9-40](https://doi.org/10.1186/1472-6920-9-40)
- Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach's alpha. *International Journal of Medical Education, 2*, 53–55. [https://doi.org/10.5116/ijme.4dfb.8dfd](https://doi.org/10.5116/ijme.4dfb.8dfd)
- University of Washington. (n.d.). *Understanding item analyses*. Institutional Assessment & Evaluation. [https://www.washington.edu/assessment/scanning-scoring/scoring/reports/item-analysis/](https://www.washington.edu/assessment/scanning-scoring/scoring/reports/item-analysis/)