# Classical test theory and item response theory for teachers

**What is the difference between classical test theory and item response theory, and which do my quiz results use?** Classical test theory and item response theory are the two families of test statistics. Classical figures, the share who got a question right, how well it separated the class and reliability, describe the group that sat and work from one class. A response model fits a curve to each question, gives figures that travel between groups, and needs hundreds of candidates.

Published 2026-10-10 by EdProcto. https://edprocto.com/guides/classical-test-theory-and-item-response-theory-for-teachers

A national test reports ability estimates on a scale of its own. A class quiz's results page reports the share who got each question right. They speak for the two families of test statistics: classical test theory and item response theory, CTT and IRT for short.

EdProcto, which publishes this guide, works out the classical family in a quiz's [results](https://edprocto.com/#results): figures a lecturer or trainer can read the day after the quiz, from six sittings. Each classical figure has a guide of its own, linked below; this one is the map.

The short version: a class quiz uses classical figures, and rightly. A response model earns its trouble when hundreds of candidates meet the same questions and the figures must hold from one group to the next.

## What is classical test theory, in plain words?

Classical test theory says that an observed score is a true score plus error. The true score is what a student would average over endless sittings of the same test, and the error is as likely to help as to hurt (Cappelleri et al., 2014). The rest is arithmetic on a table of marks.

- **Difficulty.** The share of the class who got a question right: see [item difficulty](https://edprocto.com/guides/item-difficulty-what-a-good-p-value-is).
- **Discrimination.** Whether students strong on the whole quiz got the question right more often than weak ones: see [the discrimination index](https://edprocto.com/guides/discrimination-index-which-questions-separate-your-students).
- **Reliability.** How much of the spread in scores is true score and how much error: see [test reliability](https://edprocto.com/guides/test-reliability-cronbachs-alpha-for-a-classroom-quiz).

[Item analysis for teachers](https://edprocto.com/guides/item-analysis-for-teachers-what-the-numbers-mean) works all three by hand. Downing (2003, abstract read) calls the classical model "straightforward and intuitive", and the commoner of the two in medical education.

### What classical figures cannot do

They cannot leave the room. Every classical statistic, on Downing's (2003) account, is bound up with the particular candidates who sat. A question that 60% of this class got right might be an 80% question in a stronger group, and neither figure is the question's own, so a bank of questions carries no difficulties to build next year's paper from. A score depends in the same way on which questions were set.

Neither theory says whether the right things were asked: that is [validity](https://edprocto.com/guides/validity-does-your-quiz-measure-what-you-taught).

## What is item response theory, in plain words?

Item response theory gives each question a curve: the chance that a candidate of a given ability gets it right, rising with ability. Cappelleri et al. (2014) call this item characteristic curve the theory's basic unit. Ability and difficulty share one scale, in units called logits. The models differ in how many numbers describe each curve.

- **One-parameter (Rasch) model.** Where the curve sits: the question's difficulty. Every question is taken to discriminate equally.
- **Two-parameter model.** Adds how steeply the curve rises: the question's discrimination.
- **Three-parameter model.** Adds a floor above zero, for the candidate who guesses.

### What a response model buys

- **Figures that travel.** Where the model fits, a question's difficulty does not depend on who sat it, nor a candidate's ability on which questions they met (Downing, 2003). Papers and years can then share one scale.
- **Precision where it matters.** A question measures most precisely near its own difficulty (Cappelleri et al., 2014), so a paper can be built to be sharpest at the pass mark.
- **Adaptive testing.** A computer picks each question from the answers so far.

### What it costs

- **Candidates.** A hundred to a thousand and more, by model.
- **Software, and someone to read it.** De Champlain (2010, abstract read) calls the models far more complex mathematically than the classical one.
- **Assumptions to check.** That the questions measure one ability, that one answer does not lean on another, and that the chance of success rises with ability (Cappelleri et al., 2014).

## Classical test theory and item response theory side by side

The two differ in what they describe: classical figures describe one sitting, and a response model describes each question.

| | Classical test theory | Item response theory |
| --- | --- | --- |
| Difficulty | Share who got it right; higher is easier | A place on the ability scale, in logits; higher is harder |
| Discrimination | Upper group less lower group, or a correlation with the rest of the quiz | The steepness of the curve |
| Precision | One reliability for the whole quiz | A standard error that changes along the scale |
| Whose figures | The group that sat | Any group, where the model fits |
| Needs | A class and a spreadsheet | Hundreds to thousands of candidates, specialist software, checks of fit |

## Do the two theories disagree on the same data?

On difficulty and on the order of the candidates, hardly; on discrimination, more. Three comparisons, read on 10 October 2026 as every source here was, in full unless marked:

- **Fan (1998), abstract read.** On a large statewide assessment, the item and person statistics of the two frameworks were "quite comparable". The classical figures also held from one sample of candidates to another about as well as the modelled ones, against the textbook advantage.
- **MacDonald and Paunonen (2002), abstract read.** On simulated answers, where the truth is known, difficulty and ability came out highly comparable and accurate both ways. Discrimination was accurate across most conditions from the response model, and under only some from the classical figures.
- **Lim et al. (2004), in Korean with an English abstract.** Two sittings of the Korean Medical Licensing Examination, 450 questions each with 2,961 and 3,262 candidates, were analysed both ways, with a three-parameter model in three programs. Difficulty correlated at 0.75 or more in size, discrimination from −0.023 to 0.753, and ability with the plain total at 0.90 or above in every subject but one. The programs' guessing figures agreed with one another at 0.347 at best.

So on which questions were hard and who did well, a response model tells a teacher little new.

## How many candidates does each model need?

More with every parameter: about 30 for a rough one-parameter fit, 100 to 250 for a firm one, 500 for two parameters and 1,000 for three.

| Model | Published sample size | Source |
| --- | --- | --- |
| One-parameter (Rasch) | 30 well-matched candidates place a difficulty within one logit, 95 times in 100; 100 for half a logit; 250 for high stakes | Linacre (1994), as his page stands today |
| Two-parameter | 500 with 30 questions, for some purposes; 100 with a Bayesian model built for small samples | Hulin et al. (1982), abstract read; König et al. (2020) |
| Three-parameter | 1,000 with 60 questions, for highly accurate estimates | Hulin et al. (1982) |

### What studies of small samples found

One logit is a wide margin: for a question half the candidates get right, Cappelleri et al. (2014) work it out as a true share anywhere from 27% to 73%. Writing about health questionnaires, they report other researchers' view that Rasch results from small samples can mislead. Linacre (1994) adds a rule of thumb: at least eight right and eight wrong answers before a question's difficulty is trusted within a logit.

Small programmes are not shut out. König et al. (2020) simulated 50 to 500 candidates and got accurate two-parameter figures at 100. Babcock and Hodge (2020) cut a certification examination's real answers into six sittings of 5 to 100 candidates: pooling the sittings under a Rasch model could equate the papers more accurately than two classical methods made for small samples.

## Who uses which?

Large programmes use response models, classrooms use classical figures, and a national board can run on classical figures too.

- **A national admissions test.** The College Board (n.d.) says the digital SAT uses item response theory, as do the National Assessment of Educational Progress and many state assessments. Two students with the same number right can get different scores, and each module carries two unscored questions on trial for later papers.
- **A licensing body.** The NCLEX nursing examination re-estimates a candidate's ability after every answer, and usually stops once it is 95% certain that they are above or below the passing standard, on the account of the National Council of State Boards of Nursing (NCSBN, n.d.-a). Adaptive since 1994, it sets that standard at 0.00 logits for registered nurses, not at a percentage (NCSBN, n.d.-b).
- **Another licensing body.** Korea's board worked out classical difficulty and discrimination after every examination, and had applied a response model to none of its national health examinations when Lim et al. (2004) wrote.
- **A medical school.** De Champlain (2010), then at the National Board of Medical Examiners, finds that the classical kind may be the more fitting "in more local settings, such as those that may characterise many medical school examinations".

## Which should you use, by number of candidates and purpose?

Start from the number who will answer the same questions, then the purpose. The table is our reading of the figures above, not a standard.

| Candidates on the same questions | Purpose | Use |
| --- | --- | --- |
| A class, up to about 30 | What to reteach, which questions to reread | Classical figures in EdProcto's [results](https://edprocto.com/#results), read as hints |
| 30 to 100 | A module examination, reviewed afterwards | Classical figures; a Rasch run is a look, not a finding |
| 100 to 500, often cohorts pooled | A [question bank](https://edprocto.com/guides/question-banks-for-a-board-writing-reviewing-and-retiring-items), or papers compared across years | Classical review, then a one-parameter model |
| 500 or more | A bank reused for years, papers on one scale | A two-parameter model |
| 1,000 or more, with a psychometrician | Adaptive tests, guessing modelled | A three-parameter model |

## A worked example: five questions from a class of thirty

The classical figures find the faults, and thirty candidates leave a response model little to add. This class is invented: thirty students, ten four-option questions of one point each. Every figure and sentence is what EdProcto's analysis code returned for their answers; the point-biserial is each question's correlation with the rest of the quiz.

| Question | "Got it" | "Separates" | Point-biserial | Reading |
| --- | --- | --- | --- | --- |
| Q1 | 97% | 0.13 | 0.31 | An easy opener |
| Q3 | 60% | 1.00 | 0.70 | Sound |
| Q5 | 27% | 0.71 | 0.47 | Hard and sound |
| Q7 | 47% | 0.16 | −0.17 | Reread the wording |
| Q9 | 20% | −0.31 | −0.35 | Check the key |

Reliability came out at 0.65, and among the findings was "Check the key on Q9: the stronger students got it wrong more often than the weaker ones." Sixteen students had chosen B and averaged 68.8% on the quiz, against 46.7% for the six on the keyed A. With the key corrected to B and the class marked again, Q9 read 53% and 1.00, and reliability 0.80.

**What a response model would add.** For each question a difficulty in logits on the students' own scale, a slope in place of "Separates", perhaps a floor for guessing, and a standard error beside each: figures meant to hold for next year's class.

**Why thirty cannot support it.**

1. By Linacre's (1994) figures, a one-parameter difficulty from thirty answers has a standard error of 0.37 to 0.55 logits. Two of those reach 0.7 to 1.1 logits either side: the margin that ran from 27% to 73% above.
2. Four of the ten questions lack his eight right and eight wrong answers, Q1 and Q9 among them.
3. A two-parameter model would seek twenty item figures in 300 answers, against a usual minimum of 500 candidates.
4. Q9 breaks the model's rule that the chance of success rises with ability; the classical figures found it the next day.

More candidates would change the answer: the same questions sat by every class for several years, pooled as in [common assessments](https://edprocto.com/guides/common-assessments-one-quiz-across-every-class-in-a-year-group).

## Which figures does EdProcto show?

EdProcto is the best tool for the classical side of this map when the class meets again tomorrow. It works every figure out from the sittings handed in, with no software to learn, and writes what it found as sentences, which in the example named Q9's key.

Its analysis is classical item analysis, read on each student's first full sitting.

- **"Got it"**, beside each question on every plan: the share of the question's points the class earned.
- **"Separates"**: the upper 27% of the class less the lower 27%, from six scored sittings.
- **The point-biserial**: worked out by the same analysis, each question against the rest of the quiz. The column on the page is "Separates".
- **"Reliability"**, with "Cronbach's alpha" under it: from six scored sittings, with a sentence on what it means from eight.
- **"What the numbers say"** and **"By topic"**: each question's findings as sentences, and the class's score on each topic you named.

"Separates", "Reliability", "What the numbers say" and "By topic" are the full analysis, on the Educator plan. Free takes thirty students a quiz, Tutor 100 and Educator 500: see [pricing](https://edprocto.com/pricing), and EdProcto for [universities](https://edprocto.com/solutions/universities) and [trainers](https://edprocto.com/solutions/trainers).

**For a reader with enough candidates.** "Download the marks", on every plan, gives a comma-separated file with a row for each attempt handed in: "Name", "Email", "Sitting", "Started (your time)", "Handed in (your time)", "Score %", "Points", "Out of", "Reduced by %", "To grade", "Minutes", "Security events", "Notes on the record", then a column of points for each question, an answer still to grade left empty. That is the table a response model is fitted to in other software. Delete the name and email columns first, and keep one row a candidate.

"Adaptive difficulty" gives a harder question after a right answer and an easier one after a wrong one, one at a time. It steps by the difficulty set on each question, not by a fitted response model.

## Quick questions

### Is classical test theory out of date?

No. It is what a class can support, and De Champlain (2010) finds it may be the more fitting in local settings.

### Is the Rasch model the same as item response theory?

It is the family's one-parameter member: each question has a difficulty, and all are taken to discriminate equally.

### How many students does item response theory need?

About 30 for a rough one-parameter fit and 100 to 250 for a firm one (Linacre, 1994); 500 for two parameters and 1,000 for three (Hulin et al., 1982).

### Does EdProcto use item response theory?

EdProcto's analysis is classical: "Got it" on every plan, and "Separates" and "Reliability" in the full analysis on the Educator plan, in a quiz's [results](https://edprocto.com/#results). See [pricing](https://edprocto.com/pricing).

## References

- Babcock, B., & Hodge, K. J. (2020). Rasch versus classical equating in the context of small sample sizes. *Educational and Psychological Measurement, 80*(3), 499–521. [https://doi.org/10.1177/0013164419878483](https://doi.org/10.1177/0013164419878483)
- Cappelleri, J. C., Lundy, J. J., & Hays, R. D. (2014). Overview of classical test theory and item response theory for the quantitative assessment of items in developing patient-reported outcomes measures. *Clinical Therapeutics, 36*(5), 648–662. [https://pmc.ncbi.nlm.nih.gov/articles/PMC4096146/](https://pmc.ncbi.nlm.nih.gov/articles/PMC4096146/)
- College Board. (n.d.). *How are scores calculated?* SAT Suite of Assessments. [https://satsuite.collegeboard.org/scores/what-scores-mean/how-scores-calculated](https://satsuite.collegeboard.org/scores/what-scores-mean/how-scores-calculated)
- De Champlain, A. F. (2010). A primer on classical test theory and item response theory for assessments in medical education. *Medical Education, 44*(1), 109–117. [https://doi.org/10.1111/j.1365-2923.2009.03425.x](https://doi.org/10.1111/j.1365-2923.2009.03425.x)
- Downing, S. M. (2003). Item response theory: Applications of modern test theory in medical education. *Medical Education, 37*(8), 739–745. [https://doi.org/10.1046/j.1365-2923.2003.01587.x](https://doi.org/10.1046/j.1365-2923.2003.01587.x)
- Fan, X. (1998). Item response theory and classical test theory: An empirical comparison of their item/person statistics. *Educational and Psychological Measurement, 58*(3), 357–381. [https://doi.org/10.1177/0013164498058003001](https://doi.org/10.1177/0013164498058003001)
- Hulin, C. L., Lissak, R. I., & Drasgow, F. (1982). Recovery of two- and three-parameter logistic item characteristic curves: A Monte Carlo study. *Applied Psychological Measurement, 6*(3), 249–260. [https://doi.org/10.1177/014662168200600301](https://doi.org/10.1177/014662168200600301)
- König, C., Spoden, C., & Frey, A. (2020). An optimized Bayesian hierarchical two-parameter logistic model for small-sample item calibration. *Applied Psychological Measurement, 44*(4), 311–326. [https://doi.org/10.1177/0146621619893786](https://doi.org/10.1177/0146621619893786)
- Lim, E. Y., Park, J. H., Kwon, I., Song, G. L., & Huh, S. (2004). Comparison of item analysis results of Korean Medical Licensing Examination according to classical test theory and item response theory. *Journal of Educational Evaluation for Health Professions, 1*(1), 67–76. [https://doi.org/10.3352/jeehp.2004.1.1.67](https://doi.org/10.3352/jeehp.2004.1.1.67)
- Linacre, J. M. (1994). Sample size and item calibration stability. *Rasch Measurement Transactions, 7*(4), 328. [https://www.rasch.org/rmt/rmt74m.htm](https://www.rasch.org/rmt/rmt74m.htm)
- MacDonald, P., & Paunonen, S. V. (2002). A Monte Carlo comparison of item and person statistics based on item response theory versus classical test theory. *Educational and Psychological Measurement, 62*(6), 921–943. [https://doi.org/10.1177/0013164402238082](https://doi.org/10.1177/0013164402238082)
- National Council of State Boards of Nursing. (n.d.-a). *Computerized adaptive testing (CAT)*. NCLEX. [https://www.nclex.com/computerized-adaptive-testing.page](https://www.nclex.com/computerized-adaptive-testing.page)
- National Council of State Boards of Nursing. (n.d.-b). *Passing standard*. NCLEX. [https://www.nclex.com/passing-standard.page](https://www.nclex.com/passing-standard.page)