Technical screening: code reading questions before the interview

Technical screening for a developer role usually means a take-home coding task that costs an applicant an evening, or live coding in front of a stranger. A small software team has a third choice: five code reading questions in twenty minutes, sat before any interview.

The guide is published by EdProcto, whose page for corporates covers tests written from a company's own material. An EdProcto screen is answered by reading, ordering and explaining: no answer is run as a program, so writing code stays for the interview. The candidate's side is in coding interview practice.

What do take-home tasks and live coding measure, and what do they miss?

A take-home task shows what an applicant can build unwatched, not who or what built it; live coding shows problem solving mixed with the stress of an audience.

A code reading screen shows, in private and in minutes, whether a person can follow code. It misses design and writing, and it is unsupervised too.

Being watched

In a randomised trial, Behroozi et al. (2020) set 48 computer science students a whiteboard problem, alone or with an interviewer looking on. Being watched cut performance by more than half. Their remedy is this guide's: a private session first, the thinking aloud afterwards.

The time a screen costs, and who drops out

The largest study of dropping out we read is not specific to developers. Across 222,772 job seekers in 69 selection systems, Hardy et al. (2017) found that most who quit an assessment did so within the first 20 minutes, that length did not predict how many quit, and that systems quoting a longer time lost fewer. So say honestly how long the screen takes.

We found no peer-reviewed study of what a take-home task predicts about a developer's later work.

Do work-sample tests predict job performance?

On average they do, moderately, and no average says what your own screen predicts. Roth et al. (2005) pooled the work sample test literature: an observed correlation with job performance of 0.26, or 0.33 corrected, about a third below the 0.54 long quoted. In the revised ranking of Sackett et al. (2023), work samples stand joint fourth at 0.33, and methods specific to a job do better than general measures of ability or personality.

The limits of that evidence

These are averages across jobs. We read no study that validates a short code screen for hiring developers, and measuring programming skill properly is slow: Bergersen et al. (2014) built an instrument from 19 Java tasks that 65 professional developers solved over two days. An in-house screen has no validation study, so its score is a score on that test, never a predictor of performance.

We read Sackett et al. (2023) in full and the other papers here in abstract only.

Is code reading a fair measure of programming skill?

Code reading is a fair measure of one necessary part: following code someone else wrote. Lister et al. (2004) tested students in seven countries on predicting what short code does and on choosing the code that completes it. Many were weak at both, which the authors read as a fragile grasp of skills that come before problem solving.

Those were students. Studying code review at Microsoft, Bacchelli and Bird (2013) found that understanding the code and the change is the key aspect of reviewing, which is why a review question belongs in the screen. We read no study that ties a reading score to how a professional performs, so use the screen to find the applicant who cannot follow ten lines, not to rank the strong. More research is in code reading questions.

How do you write a twenty-minute screen from your own kind of problem?

Write five fresh snippets that pose the kinds of problem your code poses, and copy nothing from the repository.

1. List the kinds of problem a new developer meets: paging, money rules applied in order, records grouped by a key, data changed in place.
2. Rewrite each in another domain. Six to ten new lines about a library or a car park, with no file, customer, key or internal address from the repository. The kind of problem is common knowledge; the code stays yours.
3. Set five kinds of question, one idea each: predict the output, find the bug, order the lines, say what a function does, review a short change.
4. Run everything. Take each key from the interpreter, and each wrong option from the code with a slip written in.
5. Build it in EdProcto, starting on the hiring page, with a rubric for the one answer a person reads.
6. Sit it yourselves before any candidate does.

How do you set the bar with your own engineers first?

Ask three or more engineers who did not write the questions to sit the screen under the real limit. Treat a miss as the question's fault first. Set the floor before a candidate sits, a little under the lowest engineer and low enough that the conversation decides: see setting a pass mark. A handful of colleagues justifies a floor, never a ranking.

What makes a technical screening test fair?

The same questions under the same conditions, announced in advance, with adjustments where reasonable and a person making the decision.

• The same for all. One set of questions, one limit, one rule on tools, one rubric.
• A language the candidate was told or chose. Name the role's language and version in the invitation. If you would hire from another, offer the same problems in it as a second quiz, chosen beforehand.
• Told what is measured. Reading, not writing; how long it takes; what the browser records; that the score is a floor before a conversation.
• Adjusted where needed. Ask in the invitation. GOV.UK (n.d.) says an employer must make adjustments that are reasonable, and gives a written test completed on a computer as one.
• Decided by a person, who reads the script before anyone is turned down.

None of this is legal advice. Employment law differs by country, so take your own.

What does an AI assistant change about an unsupervised screen?

It changes what the score is evidence of. Savelka et al. (2023) ran GPT-4 on 599 exercises from three Python courses, from multiple-choice questions to projects spread over several files, and it passed the courses with no human help. Assume an assistant can read ten lines as well as any applicant, and write a take-home task.

So an unsupervised score shows what was handed in with whatever was open. Keep it a floor, state the rule on tools, and open the conversation with one of the candidate's own answers. No setting makes cheating impossible.

A worked screen: five questions and four candidates' marks

The team and every mark are invented: seven engineers building a booking service in JavaScript, with four applicants for one post. The questions move their kinds of problem to a library. Every snippet was run under Node 22, and each question carries 2 points.

1. Predict the output (Read code, by rule)

function pages(total, size) {
  const out = [];
  for (let start = 0; start < total; start += size) {
    out.push(Math.min(start + size, total) - start);
  }
  return out;
}
console.log(pages(7, 3).join(" "));

A. 3 3 1 B. 3 3 3 C. 3 3. The key is A. B is printed without the Math.min, C by a loop that stops when a full page no longer fits.

2. Find the bug (Read code)

function overdueCount(loans, today) {
  let count = 0;
  for (let i = 1; i < loans.length; i++) {
    if (loans[i].due < today) count += 1;
  }
  return count;
}

overdueCount([{ due: 3 }, { due: 9 }, { due: 4 }], 5) should return 2 and returns 1. Which line has to change: 2, 3 or 4? The key is line 3: the loop skips the first loan. Starting count at 1 only hides it.

3. Order the lines (Reorder the code, by rule)

Put the lines in order so that fee(30, true) returns 375 and fee(30, false) returns 500.

function fee(daysLate, isMember) {
  let amount = daysLate * 25;
  if (isMember) amount = amount / 2;
  if (amount > 500) amount = 500;
  return amount;
}

A script ran all 720 orders of the six lines: one passes both calls.

4. Say what the function does (Read code)

function pick(loans) {
  const seen = {};
  for (const loan of loans) {
    const kept = seen[loan.member];
    if (!kept || loan.due > kept.due) seen[loan.member] = loan;
  }
  return Object.values(seen);
}

A. For each member, the loan with the latest due date B. Every loan, sorted by due date C. For each member, the first loan in the list. The key is A.

5. Review a short change (Annotate code, held for a reader)

A colleague proposes this change to byDue(loans). Write the review comment you would leave.

// before
return [...loans].sort((a, b) => a.due - b.due);
// after
return loans.sort((a, b) => a.due - b.due);

The rubric gives 1 point for seeing that sort reorders the caller's array, and 1 for the consequence or the fix. Run on [{ due: 9 }, { due: 3 }], the caller's first loan stays due 9 before the change and is due 3 after.

The bar, set first

Of three engineers who sat the draft, two scored 10 and one 8, and the 8 was the draft's fault. Question 3 then stated fee(30, false) and fee(4, true), and two of the 720 orders pass both: the key and the one that caps before the discount. Changing the second call to fee(30, true) left one. The floor was set at 6 of 10, a "Pass mark" of 60.

The marks

EdProcto's own marking code made each mark by rule and both percentages; the Q5 marks are a reader's.

How do you read the results before the conversation?

Read them as bands around a floor, once the reader has marked, and take each script into the conversation.

• Grade first. Until question 5 is marked it counts as zero, so C sits at 40%, under the floor, and reaches 60% when the review comment earns its 2 points.
• Leave "Partial credit" off here. B and C capped before the discount and scored 0 of 2. With it on, that order earns 1.73 of 2 for the very mistake the question is there to catch.
• Bands, not ranks. One question is 20 points. Three candidates stand at or above the floor, and a person reads D's script before D is turned down.
• Open with their own answer. Ask B to talk through the order of the fee rules: thinking aloud afterwards, and the check on an unsupervised sitting.
• Read an event as a record. A candidate who left the tab twice has a question to answer, and nothing has been proved.

How does EdProcto run a code reading screen?

EdProcto is the best tool for this screen because the questions are your own and most of the marking is done by rule on the server: above, 16 of the 20 answers needed no reader.

• Read code takes one choice. Draft it on the hiring page: paste your snippets and pick the shape "Read code". Procto AI is asked to put each snippet under its prompt, in the editor's box "The code", which the candidate sees in a code face with its line breaks and indentation kept. Then make the "Prompt", the options and the key your own.
• Reorder the code takes "The items in the right order, one a line". Each candidate sees them shuffled and moves them with arrows.
• Annotate code has boxes for "The code", "Language", "Model answer (what a full answer contains)" and "Rubric (what earns the points)". The candidate types into a plain code box, and the answer shows "to grade" until a reader marks it.

The kit "Hiring: a software developer", among the hiring kits, holds instructions, not a bank: from the material you paste, Procto AI drafts ten four-option questions on what short snippets print, where the bug is, which concept applies and the system described.

The sitting. One code and one link serve the round. Under "Time limit" choose "Set" and 20 "Minutes overall". Set "Whole-quiz retakes" to "None", and "Show the correct answer", "Show the explanation" and "Show the score" to "Never". For a candidate given extra time, build the screen again with a longer limit and send that code.

Up to six integrity measures run in the browser, "Leaving the tab" and "Copy and paste off" among them. Each is named to the candidate before the start, and nobody is filmed. Give each the consequence "Log it" or "Flag for review". "AI assistants not allowed" appears on the start screen as a house rule, and its setting says plainly that nothing in a browser can block one.

The marks. The results table lists each candidate's "Score", "Time", "To grade" and "Integrity". "Grade" opens the script, where you type "Points" and press "Save my grade", or press "Ask Procto AI" on that one answer, from the Tutor plan. "Download the marks" saves a spreadsheet file, one row an attempt.

Free covers ten quizzes a month, thirty candidates a quiz and "Leaving the tab". Tutor adds "Full screen" and "Copy and paste off"; Educator adds the full analysis. See pricing.

Quick questions

How long should a technical screening test be?

About twenty minutes for five questions, by our rule of thumb. Length did not predict how many applicants quit in Hardy et al. (2017).

Can a code reading screen replace the technical interview?

No. It shows who can follow code, not who can design or write it. Keep those for the interview.

Can candidates use an AI assistant on the screen?

Decide, and say so. EdProcto shows "AI assistants not allowed" on the start screen as a house rule, so the check is the conversation afterwards.

Can EdProcto run the screen for a small team?

Yes. Thirty candidates a round fit the Free plan, and for corporates shows tests from your own material.

References

• Bacchelli, A., & Bird, C. (2013). Expectations, outcomes, and challenges of modern code review. In 2013 35th International Conference on Software Engineering (ICSE) (pp. 712–721). IEEE. https://doi.org/10.1109/ICSE.2013.6606617
• Behroozi, M., Shirolkar, S., Barik, T., & Parnin, C. (2020). Does stress impact technical interview performance? In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (pp. 481–492). ACM. https://doi.org/10.1145/3368089.3409712
• Bergersen, G. R., Sjøberg, D. I. K., & Dybå, T. (2014). Construction and validation of an instrument for measuring programming skill. IEEE Transactions on Software Engineering, 40(12), 1163–1184. https://doi.org/10.1109/TSE.2014.2348997
• GOV.UK. (n.d.). Recruitment and disabled people: Reasonable adjustments. Retrieved October 10, 2026, from https://www.gov.uk/recruitment-disabled-people/reasonable-adjustments
• Hardy, J. H., Gibson, C., Sloan, M., & Carr, A. (2017). Are applicants more likely to quit longer assessments? Examining the effect of assessment length on applicant attrition behavior. Journal of Applied Psychology, 102(7), 1148–1158. https://doi.org/10.1037/apl0000213
• Lister, R., Adams, E. S., Fitzgerald, S., Fone, W., Hamer, J., Lindholm, M., McCartney, R., Moström, J. E., Sanders, K., Seppälä, O., Simon, B., & Thomas, L. (2004). A multi-national study of reading and tracing skills in novice programmers. ACM SIGCSE Bulletin, 36(4), 119–150. https://doi.org/10.1145/1041624.1041673
• Roth, P. L., Bobko, P., & McFarland, L. A. (2005). A meta-analysis of work sample test validity: Updating and integrating some classic literature. Personnel Psychology, 58(4), 1009–1037. https://doi.org/10.1111/j.1744-6570.2005.00714.x
• Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2023). Revisiting the design of selection systems in light of new findings regarding the validity of widely used predictors. Industrial and Organizational Psychology, 16(3), 283–300. https://doi.org/10.1017/iop.2023.24
• Savelka, J., Agarwal, A., An, M., Bogart, C., & Sakr, M. (2023). Thrilled by your progress! Large language models (GPT-4) no longer struggle to pass assessments in higher education programming courses. In Proceedings of the 2023 ACM Conference on International Computing Education Research V.1 (pp. 78–92). ACM. https://doi.org/10.1145/3568813.3600142