Beside each question on a results page sits a column headed "Separates", holding a figure such as 0.60 or −0.20. That figure is the discrimination index: whether the students with the best totals on the quiz were right on this question more often than the students with the worst.
EdProcto, which publishes this guide, works it out for every question in its results. The item analysis guide covers every figure on that page in brief; this one stays with one column.
The short version: near zero, reread the question. Below zero, check the key first. And in a class of twenty, one student's answer can move it by 0.20.
What does the "Separates" figure measure?
It measures the gap between two groups on one question: the share of your strongest students who got it right, less the share of your weakest who did. On a question marked right or wrong, 1.00 means all of the top group and none of the bottom group were right, 0.00 means the groups did equally well, and a minus sign means the bottom group did better.
Strongest and weakest are decided by the total score on that same quiz, so the index asks whether a question agrees with the rest of the paper.
Where the 27% groups come from
The groups are the top and bottom 27% of the class, a convention from Kelley (1939). We read only the abstract of his paper, which is its opening passage. There he restates his conclusion that taking 27% from each extreme yields the upper and lower groups most surely different, for scores on a normal curve and questions marked right or wrong, and calls the problem as he set it "too constricted to be real".
Rudolph et al. (2019), read in full, report the reasoning: the groups must be extreme yet hold enough students, and 27% is about where the two needs balance. Quarters are common too (Rush et al., 2016): the 27% is a habit, not a law.
The point-biserial correlation, and how the two relate
The point-biserial puts the same test to the whole class instead of its two edges: it correlates each student's mark on one question with their score on all the others. Rudolph et al. (2019) say the different ways of working out discrimination mostly give comparable results.
They part at the extremes. The index can reach 1 and −1; the University of Washington (n.d.) says its correlation will seldom pass 0.50. And the index ignores the middle of the class: in the worked example below, one question has an index of 0.00 and a point-biserial of −0.32, because across the class the students marked right on it did worse on the other nine questions than those marked wrong.
What is a good discrimination index?
No single line is agreed, and the best-known bands circulate in more than one version. They are credited to a textbook by Ebel and Frisbie, which we have not read, so here they are as two peer-reviewed papers report them.
- Four bands. Amer and Nemenqani (2020): 0.40 and above very good, 0.30 to 0.39 reasonably good, 0.20 to 0.29 marginal, 0.19 or less poor.
- Five bands. Young et al. (2017): under 0.10 poor, 0.10 to 0.19 low, 0.20 to 0.29 acceptable, 0.30 to 0.39 good, above 0.40 excellent.
Both call 0.40 strong and rate anything under 0.20 low or poor.
Later writers warn against using these as rules: Rudolph et al. (2019) tell instructors to set no hard cut-off for removing a question, and the small-class evidence below shows why.
Why can a very easy or very hard question not discriminate much?
Because the index is a gap between two shares, and when nearly everyone is right, or nearly everyone wrong, there is little room for one.
The table gives the highest index a right-or-wrong question could reach, with every wrong answer in the bottom group or every right one in the top. It is our own arithmetic: the smaller share divided by the size of a group, so 0.10 ÷ 0.27 = 0.37.
| Share who got it right | Large class, exact 27% groups | Class of 20, groups of five |
|---|---|---|
| 0.05 or 0.95 | 0.19 | 0.20 |
| 0.10 or 0.90 | 0.37 | 0.40 |
| 0.20 or 0.80 | 0.74 | 0.80 |
| 0.27 to 0.73 | 1.00 | 1.00 |
Real questions fall short of the ceiling, so judge an easy question against what it could have scored: at 0.90, an index of 0.20 is over half the possible 0.37.
Why does the figure jump in a small class?
Because each group is a handful of students. In a class of 20, 27% is 5.4 students, which EdProcto rounds to five, so one answer in either group moves the index by 0.20.
In the worked example's invented class, four of the top five and three of the bottom five got Q7 right: 0.80 − 0.60 = 0.20. We changed one answer at a time and ran EdProcto's analysis code again.
- Student 17, in the bottom five, wrong instead of right: 0.40.
- Student 16, in the bottom five, right instead of wrong: 0.00.
- Student 4, in the top five, wrong instead of right: 0.12, and the index moved on five other questions.
Student 4's total fell level with the four students next below, so five shared the last place in the top group: one answer changed who was in it.
Young et al. (2017) found the same in real examinations: across 22 cohorts of 10 to 15 medical students, 29 of 32 repeated questions fell in both the poorest and the best band at least once. Rudolph et al. (2019) warn that 27% groups may give unreliable estimates in classes of about 50 or fewer, and suggest widening them to a third.
So use one class's figure to choose which questions to reread, and pool classes that sit the same paper: see common assessments.
What does a negative discrimination index mean?
A negative figure means the bottom group did better than the top group, and the usual cause is a fault in the question. Tavakol and O'Brien (2022) list four: wording that confuses the strongest students, guessing by everyone, a question that measures something other than the rest of the paper and, most often, a wrong answer key. In a small class, add chance.
The count for each option, read beside the question, tells them apart.
| Cause | What the top scorers chose | What rereading shows | What to do |
|---|---|---|---|
| A wrong key | One wrong option, whose takers outscore the key's | That option is right | Correct the key, re-mark |
| An ambiguous stem | The key and one other option | Both can be defended | Credit both, rewrite |
| Something untaught | Every option, like the rest | It was not in the lessons | Leave it out, teach it |
| A trap | One wrong option the weakest ignored | The stem allows a second reading | Keep the key, reword |
Finding misconceptions covers the opposite pattern, a wrong option the weaker students share.
When is a low figure fine?
A low figure is fine when the question was never meant to sort the class.
A mastery check everyone should pass
If the teaching worked, nearly everyone is right and the ceiling in the table above applies. Rudolph et al. (2019) say a test meant to confirm basic understanding should contain questions most students answer correctly.
A question on a separate topic
The index measures agreement with the total. One question on reading graphs in a quiz on forces can be answered well by students weak on forces, and sit near zero through no fault of its own. The University of Washington (n.d.) notes that discrimination runs lower on tests covering a wide range of content. If the topic matters, add questions on it.
What should you do with a question that does not separate?
Work in this order, before any score is released.
- Read the question. Answer it yourself: the flaws listed in how to write multiple-choice questions often show at once.
- Check the key. Compare the option marked right in the builder with your source.
- Read which wrong option the top group chose. In EdProcto's results, press the question's row: each option shows how many chose it and their mean score.
- Fix or drop. Correct a wrong key, or credit both answers where two can be defended. Dropping takes a mark from every student who earned it: see when to throw a question out. Then rebuild it with distractors that work.
- Re-mark. Correct the key in the builder, under "Options · mark the right one", then press "Re-mark this quiz" on the results page and "Yes, re-mark": every attempt handed in is marked again from the keys as they stand, a mark you gave by hand is kept, and the page's figures are read from the new marks. To credit one answer, open the student and press "Add a note, or set the points by hand".
For what to teach next, see reading quiz results.
A worked example: ten questions, twenty students
The class is invented, and every figure is what EdProcto's analysis code returned for these answers. Twenty students sat ten multiple-choice questions of four options and one point each, with nobody tied at the edge of a group.
| Question | Got it | Top five right | Bottom five right | Separates | Point-biserial |
|---|---|---|---|---|---|
| Q1 | 95% | 5 | 4 | 0.20 | 0.09 |
| Q2 | 65% | 5 | 1 | 0.80 | 0.42 |
| Q3 | 65% | 5 | 2 | 0.60 | 0.17 |
| Q4 | 25% | 0 | 3 | −0.60 | −0.60 |
| Q5 | 40% | 4 | 0 | 0.80 | 0.36 |
| Q6 | 25% | 4 | 0 | 0.80 | 0.57 |
| Q7 | 65% | 4 | 3 | 0.20 | −0.05 |
| Q8 | 40% | 2 | 2 | 0.00 | −0.32 |
| Q9 | 80% | 5 | 3 | 0.40 | 0.17 |
| Q10 | 55% | 5 | 0 | 1.00 | 0.37 |
- Q1 is the mastery check: one student missed it, and 0.20 is the ceiling at 95% in a class of 20.
- Q4 has a wrong key. Keyed B, it drew five students who averaged 44% on the quiz. Twelve chose C and averaged 65%, the whole top five among them.
- Q6 is as hard as Q4, and sound: four of the top five were right, and the ten who chose C averaged 47% against 76% for the key's five: a misconception to teach.
- Q7 is the one question on a separate topic. It agrees little with the rest, which is no fault.
- Q8 has two defensible answers: three of the top five chose D, whose nine takers averaged 60% against 55% for the key's eight.
Re-marked by hand with C as Q4's key and run through the same code, it went from −0.60 to 1.00 and the class mean from 55.5% to 59%. The index on the other nine questions stayed put, as the same students filled both groups, but the point-biserial rose from 0.17 to 0.35 on Q3 and from 0.37 to 0.59 on Q10. A wrong key bends the totals every other figure rests on, so fix it first.
How does EdProcto show the discrimination index?
EdProcto is the best tool for this job when the scores go out tomorrow: it ranks the class, shares out the ties and writes each finding as a sentence, which in the example above put Q4's wrong key among the first lines of the page.
The arithmetic. EdProcto reads the first full sitting each student handed in: your own preview, a retry of the misses and a second sitting are left out. It ranks them by their share of the points on the answers already marked and takes 27% at each end, rounded to whole students; students level on the score where a group ends divide its remaining places between them. It works in shares of a question's points, so part marks count. Under six scored sittings it gives no figure. An answer waiting for a grade is left out of its own question and of the ranking. The point-biserial is worked out against the total less that question's points.
The page. Under "Each question", the table sets "Got it", "Separates", "Median t" and "What the numbers say" beside each question. "Separates" is printed to two decimal places in plain figures; the colour and the bar belong to "Got it". Press the row of a question with one right choice and a table opens: "Option", "Chose it" and "Their mean", the key marked "key".
The sentences. Below zero, the row reads:
On Q4 the weaker students did better than the stronger ones. That usually means a wrong key or an ambiguous wording.
From zero to under 0.20, with 30% to 90% of the points earned:
Q8 does not separate the students who know the material from those who do not.
Under 30% of the points, from three marked answers:
Most of the class missed Q4. Check the key first, then whether this was taught.
A wrong option with three or more takers, and more than the key has:
More students chose "C" than the right answer on Q4. Either the key is wrong or the class shares a misconception.
At the top, under "What it says about the teaching", every question below zero:
Check the key on Q4: the stronger students got it wrong more often than the weaker ones.
And at zero or above, where such an option's takers outscored the key's:
Check the key on Q8: the students who chose the most popular wrong answer scored higher on the rest of the quiz than those who chose the keyed one.
The plans. Free takes thirty students a quiz and Tutor 100, with "Got it" and "Median t" for each question. Educator takes 500 and adds the full analysis: "Separates", the option table and every sentence above. Below it a card reads "The full analysis is part of the Educator plan". See pricing, then choose a shape from the templates and start on the home page.
Quick questions
Is a discrimination index of 0.20 good enough?
It is the first rung that is not low or poor: marginal in one version of the bands above, acceptable in the other. In a class of twenty it is one answer from 0.00.
Why is my discrimination index negative?
Your weaker students did better on the question than your stronger ones. Check the key first, then look for a second defensible answer.
Can a question everyone got right have a high discrimination index?
No. With everyone right it is 0.00, and at 95% correct the ceiling is about 0.19. Neither is a reason to remove a question every student should get.
Where does EdProcto show the discrimination index?
In the "Separates" column of a quiz's results, from six students up, on the Educator plan and above: see pricing.
References
- Amer, M. G., & Nemenqani, D. M. (2020). Successful use of virtual microscopy in the assessment of practical histology during pandemic COVID-19: A descriptive study. Journal of Microscopy and Ultrastructure, 8(4), 156–161. https://doi.org/10.4103/JMAU.JMAU_67_20
- Kelley, T. L. (1939). The selection of upper and lower groups for the validation of test items. Journal of Educational Psychology, 30(1), 17–24. https://doi.org/10.1037/h0057123
- Rudolph, M. J., Daugherty, K. K., Ray, M. E., Shuford, V. P., Lebovitz, L., & DiVall, M. V. (2019). Best practices related to examination item construction and post-hoc review. American Journal of Pharmaceutical Education, 83(7), Article 7204. https://doi.org/10.5688/ajpe7204
- Rush, B. R., Rankin, D. C., & White, B. J. (2016). The impact of item-writing flaws and item complexity on examination item difficulty and discrimination value. BMC Medical Education, 16, Article 250. https://doi.org/10.1186/s12909-016-0773-3
- Tavakol, M., & O'Brien, D. (2022). Psychometrics for physicians: Everything a clinician needs to know about assessments in medical education. International Journal of Medical Education, 13, 100–106. https://doi.org/10.5116/ijme.625f.bfb1
- University of Washington. (n.d.). Understanding item analyses. Institutional Assessment & Evaluation. https://www.washington.edu/assessment/scanning-scoring/scoring/reports/item-analysis/
- Young, M., Cummings, B.-A., & St-Onge, C. (2017). Ensuring the quality of multiple-choice exams administered to small cohorts: A cautionary tale. Perspectives on Medical Education, 6(1), 21–28. https://doi.org/10.1007/s40037-016-0322-0