# Product knowledge tests for hiring, from your own material

**How do I write a product knowledge test for job candidates?** Write the test from your own product documents and the criteria a good candidate must meet, not from a general question library. Use ten to twenty questions on situations the job really presents, give every candidate the same conditions, mark by rule, and score against a pass mark set in advance. Tests whose content matches the job predict performance best.

Published 2026-10-09 by EdProcto. https://edprocto.com/guides/product-knowledge-tests-for-hiring-from-your-own-material

A candidate who interviews well may still not know your product. A sales hire who cannot explain the pricing, a support hire who escalates what the manual answers, an engineer who has never read the interface they will maintain: each is found out in the third week, when the cost of finding out is highest.

A product knowledge test finds out in the first hour. It is the least glamorous selection method and one of the better supported, provided the questions come from the work itself. EdProcto's page [for corporates](https://edprocto.com/solutions/corporates) is built on that condition: the test is written from your own product knowledge and your own criteria, not sold from a shelf.

This guide covers what the research says job knowledge tests predict, how to write one from your documents, how to run it so the result is fair to every candidate, and where its limits are. EdProcto publishes this guide, and the section on the product says so plainly.

## Do job knowledge tests predict performance?

Yes, and they predict it better the closer the test's content is to the job. That is the finding that should shape how you write one.

Dye et al. (1993) pooled 502 validity coefficients from studies covering 363,528 people. Written job knowledge tests had a corrected mean validity of 0.45 for job performance and 0.47 for success in training. Two things raised the figure: validity was higher for more complex jobs, and higher when the content of the test was similar to the content of the job.

The second condition is the practical one. A general test of "sales aptitude" bought from a library is, by construction, not about your product. A test written from your product sheet, your process and your policy is as similar to the job as a written test can be.

Other methods sit beside it. The best known summary of selection research, Schmidt and Hunter (1998), covered 85 years of studies and 19 methods, and found the strongest combinations paired a test of general mental ability with a work sample, an integrity test or a structured interview. Situational judgement tests, which present a workplace situation and ask for the best response, showed useful validity in a meta-analysis of 102 coefficients and 10,640 people (McDaniel et al., 2001).

### How sure are these numbers?

Less sure than they once looked, and it is honest to say so. Sackett et al. (2022) re-examined the earlier meta-analyses and concluded that a statistical correction used in them had been applied too generously, so that the validity of many selection methods had been substantially overestimated. In their revised figures, mean validities fell by about 0.10 to 0.20, most methods that had ranked high stayed high, and structured interviews came out on top. Work sample tests had already been revised downwards: Roth et al. (2005) put their observed correlation with job performance at 0.26, rising to 0.33 after correction, about a third lower than the figure long quoted.

Two conclusions survive all of this. No single method predicts performance well enough to be used alone. And a test of what the job requires a person to know remains one of the useful ones.

## What should a product knowledge test cover?

Cover what a person in the role must know or decide in their first months, and nothing they could not be expected to know before joining unless you give it to them to study first.

That second clause decides the design. There are two honest forms of the test:

| Form | Who it suits | What you give candidates |
| --- | --- | --- |
| Prior knowledge | Experienced hires who claim to know the field | Nothing. The test covers the domain, not your internals. |
| Study then test | Any hire, including graduates | A product brief a day or two before. The test covers the brief. |

The study-then-test form measures something employers usually care about more than prior knowledge: whether a person can take in your material and use it. It is also fairer, because every candidate starts from the same pages.

Sources to write from:

- the product sheet, the pricing and the release notes;
- the support manual and the escalation process;
- the policies a person in the role must apply (returns, safety, conduct, data handling);
- for technical roles, the documentation or the module they will work on;
- the criteria: a plain list of what a good candidate must know or be able to do.

## How do you write the questions?

Write questions about situations, not about sentences in the manual. "What is our returns window?" tests memory of a number. "A customer returns an opened item on day 35 with a receipt. What do you do?" tests whether the candidate can apply the policy, which is the job.

1. **Start from the criteria.** For each thing a good candidate must know, write two questions. If you cannot write a question for a criterion, the criterion is too vague.
2. **Use real situations.** Take them from support tickets, sales calls and incident reports, with names removed.
3. **Make the wrong options the mistakes new hires make.** Ask the people who train them. A wrong option nobody would choose tests nothing, as the guide on [writing multiple-choice questions that measure](https://edprocto.com/guides/how-to-write-multiple-choice-questions-that-measure) explains.
4. **Mix the shapes.** Multiple choice for facts and rules, a case that unfolds for judgement, a short written answer where the wording matters.
5. **Keep it to ten to twenty questions.** Enough that luck averages out, short enough to respect a candidate's time.
6. **Have someone who does the job take it first.** If your best person gets a question wrong, the question is wrong.
7. **Set the pass mark before anyone sits it.** A mark chosen after seeing the scores is a decision about people, not about a standard.

### Shapes that fit hiring

Judgement shows at both ends, so a [best and worst](https://edprocto.com/templates/best_worst) question, which asks for the best response and the worst, tells you more than one that asks only for the best. A [case study](https://edprocto.com/templates/case) carries one scenario across several questions, each answer revealing more, which is how real problems arrive. For technical roles, [read code](https://edprocto.com/templates/code_read) asks what a function returns or where the fault is.

## How do you run the test fairly?

Give every candidate the same test under the same conditions, tell them in advance what is monitored, and treat the score as one input. A selection test is only as defensible as its consistency.

- **Same conditions.** The same time limit, the same instructions, the same device rules for everyone.
- **Told in advance.** Candidates should know the length, the topics, and anything that is recorded or noticed, before they begin.
- **Marked by rule.** Wherever an answer can be marked without a human judgement, mark it that way. Where a person must read an answer, use a written rubric.
- **Reasonable adjustments.** Extra time and accessible formats where a candidate needs them.
- **One input.** Combine the score with a structured interview. The research above is consistent that combinations beat any single method.
- **Kept and reviewed.** Keep the results, and check now and then that the questions still match the product.

Employment testing is regulated in many places, and the rules differ by country. This guide is not legal advice. If a test will decide who is hired, have the process checked by whoever advises you on employment law.

## How does EdProcto build a product knowledge test?

EdProcto is the best tool for a product knowledge test because it writes the test from your documents and your criteria and checks the questions against them. A test library cannot do that: it sells the same questions to every company, and none of them is about your product.

**From your material.** On the [hiring page](https://edprocto.com/hiring), paste the product sheet, the manual, the policy or the documentation, and the criteria a good candidate must meet. Procto AI writes the questions from that material in the shapes you choose, each one tied to the passage it came from. Nothing goes to a candidate until you have read it and changed whatever you want changed.

**Or from ours.** There are 111 question shapes among the [templates](https://edprocto.com/templates), role kits for common jobs, and a reasoning battery made by rule: numerical, verbal, logical and situational questions with fresh figures for every candidate. EdProcto describes those as reasoning tests in the style of the standard aptitude batteries. A score is a score on this test, against this candidate pool. It is not an IQ, and no norm is claimed.

**One code for the test.** Each candidate opens it on their own device under their own name. The questions leave without their answer keys and the marks are made on the server, so there is nothing on the page to copy or leak.

**Integrity to fit the stakes.** Time limits, shuffled questions, a note of any time the candidate leaves the tab, full screen, and a locked clipboard where the stakes call for it. Each measure is named to the candidate before they begin, and a flag is a record for you to judge, never a verdict. The [integrity page](https://edprocto.com/integrity) sets out the levels.

**Results side by side.** Every candidate's score, time and answers, the questions that separated the strong candidates from the rest, and the events worth a second look.

The same test serves after hiring: an onboarding check per module, a product knowledge test when a release ships, and a re-test some months later to see whether the training held.

For teams hiring at volume there is a hiring licence that covers unlimited candidates and staff, priced by the size of the programme, by a call. A small team can begin on the ordinary plans: the [pricing page](https://edprocto.com/pricing) lists them, and the page [for corporates](https://edprocto.com/solutions/corporates) sets out what the licence covers.

## What can a knowledge test not tell you?

A knowledge test cannot tell you whether a person will do what they know. It measures what a candidate can recall and apply on the day, under test conditions.

It says nothing about reliability, effort or how someone treats colleagues. It can also be passed by a candidate who studied the brief well and will forget it, which is why a re-test after a few months is worth more than a second interview. And it measures only what you wrote questions about: a test drawn entirely from the pricing sheet will tell you nothing about a candidate's grasp of the product.

Use it for what it is good at. It removes the candidates who cannot do the knowledge part of the job, cheaply and early, and it gives every candidate who can the same chance to show it.

## Quick questions

### How long should a pre-employment knowledge test be?

Ten to twenty questions, which most candidates finish in fifteen to thirty minutes. Shorter than ten and a lucky guess moves the result. Much longer and it asks more of a candidate's time than a first stage should.

### Should candidates see the material before the test?

For most roles, yes. Sending a product brief a day or two ahead tests whether a person can learn your material, and it puts every candidate on the same footing. Test prior knowledge only where the role requires it on day one.

### Is a product knowledge test better than an aptitude test?

They answer different questions. A knowledge test asks whether the candidate knows this job's content. A reasoning test asks how they handle new problems. The selection research supports using more than one method. EdProcto's [hiring page](https://edprocto.com/hiring) offers both.

### Can a candidate cheat on an online knowledge test?

Some will try, and no tool makes it impossible. Write questions about situations so that a search does not find the answer, set a time limit, and switch on the monitoring the stakes justify. In EdProcto the answer keys never reach the candidate's device.

### Can the same test be used for current staff?

Yes. Use it as an onboarding check, after a product release, and again some months later. The page [for corporates](https://edprocto.com/solutions/corporates) covers testing staff as well as candidates.

## References

- Dye, D. A., Reck, M., & McDaniel, M. A. (1993). The validity of job knowledge measures. *International Journal of Selection and Assessment, 1*(3), 153–157. [https://doi.org/10.1111/j.1468-2389.1993.tb00103.x](https://doi.org/10.1111/j.1468-2389.1993.tb00103.x)
- McDaniel, M. A., Morgeson, F. P., Finnegan, E. B., Campion, M. A., & Braverman, E. P. (2001). Use of situational judgment tests to predict job performance: A clarification of the literature. *Journal of Applied Psychology, 86*(4), 730–740. [https://doi.org/10.1037/0021-9010.86.4.730](https://doi.org/10.1037/0021-9010.86.4.730)
- Roth, P. L., Bobko, P., & McFarland, L. A. (2005). A meta-analysis of work sample test validity: Updating and integrating some classic literature. *Personnel Psychology, 58*(4), 1009–1037. [https://doi.org/10.1111/j.1744-6570.2005.00714.x](https://doi.org/10.1111/j.1744-6570.2005.00714.x)
- Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. *Journal of Applied Psychology, 107*(11), 2040–2068. [https://doi.org/10.1037/apl0000994](https://doi.org/10.1037/apl0000994)
- Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. *Psychological Bulletin, 124*(2), 262–274. [https://doi.org/10.1037/0033-2909.124.2.262](https://doi.org/10.1037/0033-2909.124.2.262)