Sign up free

Assessment design after chatbots: what a department can change

What quality bodies and studies say about generative AI and assessment in higher education, what each secured format costs, and a one-year plan for a department.

The EdProcto editorial teamWritten and checked against the research it cites. Last checked . How the guides are made

Guides · Integrity and cheating · · 13 min read

How should a university department redesign its assessment now that generative AI can write the coursework?

Assessment design after chatbots is decided for the whole programme, not module by module. A department assures the award at a few secured points in every year, such as invigilated tests, vivas, in-class writing and observed practicals, opens everything else and teaches generative AI use there, and stops treating AI detectors or a marker's instinct as evidence, because studies found both unreliable.

Assessment design in higher education has a new fact to absorb: a chatbot can write a passable answer to much of the written work a degree sets, and when one university tested its own markers blind, they almost never noticed. One lecturer can change a module. Only a department or a programme team can decide where the degree as a whole is still assured.

EdProcto publishes this guide and makes the paper for a supervised sitting; its page for universities says what it offers a course team. One teacher's view of the evidence is in academic integrity in the age of AI.

In short: map every assessed item, secure a few points in each year, open the rest and teach with generative AI there, retire the detector, and count the staff hours first.

What have quality bodies and regulators said about assessment and generative AI?

Two say to reform assessment across the programme, and a third says how to investigate fairly. Each was read at source on 10 October 2026; your own regulator comes first.

  • The Quality Assurance Agency (2023), a quality body for UK higher education, advised that the output of these tools cannot be detected reliably. It set three aims for a review: fewer assessed items, more synoptic assessment that tests programme-level outcomes across modules, and more authentic tasks. It rates the integrity of handwritten invigilated exams high, and calls relying on them a step backwards for accessibility.
  • Australia's higher education regulator, the Tertiary Education Quality and Standards Agency (2025), describes its 2023 guidance as principles for institutions and its later resource as examples of them in practice. Both aim to protect the assurance of learning while students learn to use the tools responsibly. We read only its summary page, in an archived copy.
  • The Office of the Independent Adjudicator (2025), the student complaints ombuds for England and Wales, set out how to investigate suspected misuse: the provider must prove the case, decision-makers must understand the limits of detection software, and they should ask whether their assumptions are biased against a disabled student or one whose first language is not English.

Can markers or AI detectors tell machine-written work from a student's own?

Mostly not, in the studies so far. Scarfe et al. (2024) slipped answers written entirely by GPT-4 into the take-home exams of five undergraduate psychology modules at the University of Reading without telling the markers. Of those scripts, 94% went unflagged, and their grades averaged about half a degree-class boundary above the real students'. That is one degree at one university in 2023, and the single final-year module went the other way on three scripts.

Are AI detectors reliable enough to act on?

No. Weber-Wulff et al. (2023) tested 12 public tools and two commercial systems and concluded that they were neither accurate nor reliable, erring mainly towards calling machine text human (abstract read). Liang et al. (2023) ran seven detectors over 91 essays written for the TOEFL English test by non-native speakers and 88 by American eighth-grade pupils. The American essays were classified accurately; the TOEFL essays were called AI-generated 61.3% of the time on average. Both tests date from 2023 and may not describe today's tools. A detector score can start a conversation; it cannot carry a penalty.

What is the two-lane approach to assessment?

The two-lane approach sorts every assessed task into one of two kinds, secured or open. It comes from the University of Sydney, where Liu and Bridgeman (2023) set it out. Lane 1 is assessment of learning, done live under supervision to assure that outcomes are met: in-class tasks, vivas, simulations, supervised exams. Lane 2 is assessment as learning, where students work with generative AI and are taught to do it well. The authors see no middle ground: any task outside lane 1 should be assumed to involve AI.

Their later answers to staff (Bridgeman & Liu, 2024) add that anything not supervised face to face counts as open, online quizzes and presentations by video call included. Corbin et al. (2025) explain why rules alone fail: a discursive change alters only the instructions, which students remain free to ignore, while a structural change alters how the task works (abstract read).

Why decide at programme level and not module by module?

Because securing every module costs staff time few departments have, and the award is what must be assured. Bridgeman and Liu (2024) place secured assessment mainly at programme level, at chosen points through the degree. About 80% of Sydney's units, with just under 60% of enrolments, had no exam in 2024, they report, and adding secured tasks to each would take considerable resources.

Which kinds of secured assessment are there, and what does each cost?

There are four, and each buys assurance at a different price.

Secured format What the evidence says Main cost
Invigilated test or exam The QAA rates integrity high for handwritten papers and medium for digital ones, which may be sat off site Rooms and invigilators. On devices, EdProcto can be the paper
Viva or oral examination Huxham et al. (2012), 99 students split at random on the same biology questions: higher marks than in writing, more nerves, no group seen to be disadvantaged (abstract read) Staff time; the QAA calls it suitable for individuals or small groups
In-class writing We found no study of it against generative AI Teaching time
Observed practical Brannick et al. (2011), 39 studies of clinical examinations: reliability across stations 0.66 on average, better with more stations and examiners, and overall often not very reliable (abstract read) Stations, examiners, scheduling

For orals the QAA describes structured interviews with two or more examiners, a clear rubric and safeguards for vulnerable students.

Is a remotely proctored exam a secured assessment?

Not on this evidence. The QAA notes that even with digital proctoring a remote candidate can reach the paper on another device, and Bridgeman and Liu (2024) expect remote exams to become harder to secure. A secured sitting means a supervised room. EdProcto is the paper in that room, not a substitute for it.

Do drafts, logs and a short oral check prove who did the work?

Drafts and logs do not; a short oral check in front of an examiner is the stronger evidence. Drafts, notes and version histories are made out of sight, as the essay was, and we found no study of whether a marker can tell from them who wrote the work. Typing logs have been tested: a detector built by Kundu et al. (2024) told assisted from unassisted writing with 52% to 86% accuracy, depending on the scenario (abstract read), which is far from proof.

The OIA accepts a viva on the content of submitted work as a test of understanding, and suggests asking for notes and drafts in a fair investigation. The QAA describes mini-vivas, in which small groups are interviewed about their submissions, as costly in staff time and a strong deterrent to contract cheating. Keep process evidence for teaching, and put ten minutes of questions on the work at a secured point.

How do you keep a redesigned programme fair and accessible?

By designing the adjustments in, because each secured format shuts someone out when run carelessly.

  • Handwriting. The QAA says fewer students arrive prepared to handwrite at length; a test on a device spares them that.
  • Speaking. The QAA lists orals as stressful for some students and hard for those with speech or hearing disabilities, and asks that accents be treated equitably. Offer a practice sitting.
  • Suspicion. Detectors flagged second-language writers in Liang et al. (2023), and the OIA asks decision-makers to look for the same bias in themselves.

For a sitting in EdProcto, use the browser's zoom and the device's own settings, and give a candidate with extra time a second quiz with the longer limit.

What does programme-level assessment design look like before and after?

Fewer items, with secured points in every year. The map is an illustration we invented, not a real programme: a three-year degree of 18 modules, 90 students in each year. Each cell counts the assessed items in years 1, 2 and 3.

Assessed item Before After
Open: essays, reports, take-home exams, the dissertation 8, 9, 8 4, 4, 4
Open: online tests sat at home 4, 2, 0 0, 0, 0
Open: group projects and prepared presentations 1, 2, 1 1, 1, 1
Secured: invigilated exams and tests 1, 1, 1 1, 1, 1
Secured: in-class writing 0, 0, 0 2, 1, 0
Secured: vivas on submitted work 0, 0, 0 0, 1, 1
Secured: observed practicals 0, 0, 0 1, 1, 1
All items 14, 14, 10 9, 9, 8
Secured share 7.1%, 7.1%, 10.0% 44.4%, 44.4%, 37.5%

Across the degree 38 items become 26, and the secured share rises from 3 of 38 (7.9%) to 11 of 26 (42.3%). Work written at home falls from 25 items (65.8%) to 12 (46.2%). The six home tests become uncounted practice quizzes, where retrieval practice belongs.

What does it cost in staff hours?

Less than the items removed, on our assumptions: put your own minutes in. Thirteen fewer home-written items, at 20 minutes a script for 90 students, free 390 marking hours a year. The new work costs 202.5: three in-class pieces at 15 minutes a script (67.5), two viva points at ten minutes a student plus five to change over, with two examiners (90), and three practicals at ten minutes a student (45). That leaves 187.5 hours, before rooms and scheduling. A first-year module of several hundred is a different sum: Bridgeman and Liu (2024) say formal orals may not belong there.

How do you redesign assessment for AI in one academic year?

A department can map, decide, pilot and approve in one year, in this order.

  1. Before teaching starts: draw the map. List every assessed item: its type, its weight, the programme outcomes it tests, and whether anyone watches it being done. The QAA asks teams to look for duplication, gaps and items open to generative AI.
  2. First month: try your own briefs. Liu and Bridgeman (2023) advise running each task through a capable model.
  3. First term: retire the detector as evidence. Write the procedure the OIA describes: the department proves the case, and the student sees the evidence.
  4. First term: choose the secured points. At least one in each year, together covering every programme outcome, in a format the discipline already trusts.
  5. Second term: cut and open. Drop duplicated items, turn home tests into practice, declare the remaining home work open, and teach the AI use you expect.
  6. Second term: pilot one secured point in each year. A rubric, two examiners for an oral, a practice sitting, adjustments agreed. For a test in a room, build the paper in EdProcto from the module's own material.
  7. Third term: approve, tell students, review. Take the changes through programme approval, publish the map to students, and read each secured test's item analysis before reusing it.

Where does EdProcto fit in a redesigned programme?

EdProcto is the best tool for the paper at a secured point, because it is built round the two things the evidence above asks of a test: nothing on the screen worth carrying to a chatbot, and marks made where the candidate cannot reach them. It writes each quiz from the module's own material.

In the room. The Exam preset switches on all six browser measures (leaving the tab, full screen, copy and paste, right-click, keyboard shortcuts, printing), each with the consequence you choose, allows no going back, and holds every score until you release it. That is its Medium level. The measures act in the browser, so the invigilator checks who is sitting. Questions and options are shuffled for each student, and the answer key never leaves the server.

Shapes that leave a chatbot least to do. All five are in the builder. A clip that plays once, a click on an image and a question shown for a few seconds are marked by rule. A case that unfolds, with no going back, is a structure for the whole quiz. A recorded spoken answer waits for you to mark it.

Defending work. Defend a position asks for a claim, evidence, reasoning and a counter-argument; peer review for major comments, minor comments and a recommendation. Both are in the builder and wait in the grading queue for a reader: you, or Procto AI when you ask it on one answer, with its mark yours to change. An oral defence puts the student before several examiners' questions, answered aloud and recorded for you to grade. See graduate assessment.

Afterwards. The results give each question's difficulty and discrimination, who took each wrong option, the paper's reliability, topic means and pace.

Free takes thirty students a quiz, with Light integrity. Tutor takes 100, opens all 111 templates and saves the Exam preset. Educator takes 500 and adds the full analysis. A licence for a whole programme is arranged by a call: see pricing and the plans for institutions. A score is a record for the examiners, never a grade by itself.

Quick questions

Are AI detectors reliable?

Not enough to act on. Weber-Wulff et al. (2023) judged 14 tools neither accurate nor reliable, and Liang et al. (2023) found seven detectors calling most essays by non-native English writers machine-written.

Should a department go back to handwritten exams for everything?

No. The QAA rates their integrity high and lists accessibility among their weaknesses. Use them at a few points, beside vivas, in-class writing and practicals.

How many secured assessments does a programme need?

Enough to assure every programme outcome once, with a secured point in each year. That is our working rule; no source we read gives a number.

Can EdProcto secure an exam that students sit at home?

Not by itself. Its browser measures record what happens on the screen; at the Strict and Maximum levels the webcam, the shared screen, a phone watching the room and an ID checked at the start add a record of who sat and what was on the desk, for you to read, and none of it is proof. For a result that must be certain, seat students in a supervised room with EdProcto as the paper: see how much proctoring a quiz needs and the universities page.

References

EdProcto for universities

Assessment that holds in the era of the chatbot.

Instructors, course teams and examinations offices in higher education.

See the page for universities

Read next