Eight approaches compared

A working comparison of eight redesign moves for AI-era assessment. Ratings are our evidence-based judgments for first- and second-year undergraduate courses (the kit’s scope), not a systematic review. Where a rating is arguable, the “evidence base” column says so — and the literature notes show what was checked.

Evidence-base key: strong = multiple independent studies or institutional adoption; emerging = single-site studies or documented pilots; anecdotal = practitioner reports and reasoned argument only.

The eight approaches

1. In-class writing

Writing done live in class (or a scheduled exam slot), unaided by AI.

• Validity benefit: strong — the strongest possible link between the student and the words on the page.
• Workload: low change to grading, but class time is consumed and drafts cannot be meaningfully iterative in the room.
• Scalability: moderate — works in lecture halls with seating spread; grading is the same as any essay.
• Equity/accessibility risks: high for students with slow typing, writing disabilities, anxiety, or extra-time accommodations — timing pressure is the assessment barrier. Extended time and typing accommodations are mandatory, which strains “everyone writes at once”.
• Student experience: often negative for second-language writers and deep, slow thinkers; writing quality under time pressure ≠ thinking quality.
• Evidence base: emerging — long-standing exam practice, but the AI-era literature on in-class writing as a redesign response is mostly practitioner reports.

2. Oral checks (viva-lite)

Short structured spoken Q&A: student defends choices, explains their work, answers “why did you do X?”.

• Validity benefit: strong — the student must use the work in real time; the best single check that the artefact is theirs.
• Workload: high per student at scale; mitigable by sampling (see below).
• Scalability: low without adaptation — 200 students × 5 minutes ≈ 17 hours. Scaling moves: sample (e.g., every 5th student plus anyone whose process evidence raises questions), run in tutorial groups, use TAs with a shared question bank, or offer group-scheduled slots.
• Equity/accessibility risks: significant for communication disabilities, anxiety, and non-native speakers — always provide a written or recorded alternative of equal weight. Oral checks measure speaking as well as knowing, which is usually not the outcome being assessed.
• Student experience: intimidating for some, valuable for others; predictable questions and a published rubric reduce fear.
• Evidence base: emerging — institutional adoption at scale is documented (Sydney’s secure lane; interactive oral programs), but outcome studies are young.

3. Staged submissions

One artefact built in graded stages: proposal → annotated sources → draft → final → short defence.

• Validity benefit: strong — the chain of stages produces a traceable process; each stage is cheap to grade.
• Workload: moderate upfront (design + deadlines), but per-stage grading is quick with the right rubric; total grading similar to marking one big essay badly.
• Scalability: moderate — feedback per stage is the constraint; in large classes, stages can be graded complete/incomplete against checklists with the final stage carrying the grade.
• Equity/accessibility risks: low if deadlines are spaced and extensions are real; students who start late get less value but no penalty beyond the late policy.
• Student experience: positive for most — checkpoints prevent the last-minute collapse — but students who like working in one burst resist it.
• Evidence base: strong — staged/scaffolded assessment is a long-established practice with substantial literature.

4. Process portfolios (drafts, logs, version history)

Students submit the artefact and the trail: drafts, an AI-use log, a lab notebook, document version history.

• Validity benefit: moderate-to-strong — reveals the journey, but a determined fabricator can manufacture a trail; combine with a short defence for assurance.
• Workload: moderate — grading focuses on the log/reflection, not re-reading every draft.
• Scalability: moderate — reflection and logs grade fast with analytic rubrics; version history is spot-checked, not audited in full.
• Equity/accessibility risks: low — asynchronous by nature; watch for students whose lives make “steady documented progress” unrealistic (partial solution: grade the log as evidence, not as a work diary).
• Student experience: positive for reflective students; others find it bureaucratic — keep the format short and purposeful.
• Evidence base: emerging — portfolios are well established in some fields; their AI-era use (logs of AI interaction) is newer and mostly practitioner-documented.

5. Personalized or local data

The task runs on data the student (or class) collected: a mini-survey, a local case, their own experiment, this semester’s numbers.

• Validity benefit: strong — generic AI output can’t match specific local context, and the data-collection step itself is assessable.
• Workload: moderate — designing a collectable-in-a-week data task is the hard part; grading is comparable to the original.
• Scalability: moderate — cohort-level collection (one shared class dataset, student-owned subsamples) scales well; 200 individual projects do not.
• Equity/accessibility risks: low-to-moderate — data collection must not require travel, expensive equipment, or access to particular communities; always provide a provided-dataset alternative.
• Student experience: generally the most positive of the eight — students see their own data mean something.
• Evidence base: emerging — well-grounded in active-learning and lab pedagogy; AI-era framing is new.

6. Critiquing AI output

Students are given AI-generated work (or generate some) and must find, fix and justify the errors and weak reasoning.

• Validity benefit: strong for critical-evaluation outcomes — the skill of checking AI is itself a learning goal.
• Workload: low-to-moderate — the AI draft is the prompt; grading targets the critique.
• Scalability: high — one AI artefact serves a whole cohort; critique items are quick to grade.
• Equity/accessibility risks: low — asynchronous and text-based; ensure the AI artefact is accessible and the critique format allows structure (bullets, tables) rather than pure prose.
• Student experience: engaging — feels like real work. Caveat: students need instruction in how to critique, or they find nothing to say.
• Evidence base: anecdotal-to-emerging — a fast-moving area; no settled outcome literature yet. (This is stated plainly rather than oversold.)

7. Open AI use with reflection

The task invites AI use at Full AI level; students document what they used, evaluate the output, and are graded on the quality of their AI-assisted process and reflection.

• Validity benefit: strong if the outcome is AI-collaborative capability (which is often exactly what a course should teach); weak as an assurance of unaided skill — that’s what the secure lane is for.
• Workload: low-to-moderate — grading the reflection is additional but light.
• Scalability: high — no proctoring, no policing.
• Equity/accessibility risks: moderate — paid-tool access varies; levels must be satisfiable with free tiers, and students without any access need an equivalent path. Benefits multilingual students who use AI legitimately as a language aid — with disclosure.
• Student experience: the most honest match to real-world practice; the risk is students outsourcing learning itself, mitigated by grading the reflection, not the product.
• Evidence base: emerging — AIAS deployments (K-10 pilot) and practice reports exist; long-term evidence is not in.

8. Proctored/secure checks

Formal supervised assessment: in-person quizzes, tests, exams (the classic secure lane, used sparingly).

• Validity benefit: strong for assurance of individual achievement; but it samples performance under constraint, not durable learning.
• Workload: high logistics (rooms, invigilation, scheduling), moderate grading (MCQ/short-answer grades fast).
• Scalability: moderate-to-high for MCQ-style checks; low for anything requiring spread-out seating at scale.
• Equity/accessibility risks: high if used exclusively — exam anxiety, accommodations processing, and the research on exam performance vs. later application all argue for a limited role. Also the least interesting thing to grade.
• Student experience: the most hated of the eight, and the least formative.
• Evidence base: strong as practice (centuries old), though as an AI response it’s a retreat, not a design.

Summary table

#	Approach	Validity benefit	Workload	Scalability	Equity/accessibility risk	Student experience	Evidence base
1	In-class writing	Strong	Low	Moderate	High	Poor	Emerging
2	Oral checks	Strong	High	Low (sampled: moderate)	Significant (mitigable)	Mixed	Emerging
3	Staged submissions	Strong	Moderate	Moderate	Low	Positive	Strong
4	Process portfolio	Moderate–strong	Moderate	Moderate	Low	Positive	Emerging
5	Personal/local data	Strong	Moderate	Moderate	Low–moderate	Very positive	Emerging
6	Critique AI output	Strong (for that goal)	Low–moderate	High	Low	Engaging	Anecdotal–emerging
7	Open AI + reflection	Strong (for AI-collab goals)	Low–moderate	High	Moderate (tool access)	Positive	Emerging
8	Proctored checks	Strong (narrow)	Logistics-heavy	Moderate–high	High (if exclusive)	Poor	Strong (as practice)

How the cases use these

• Case A (essay) uses 3 + 4 + 5 + 2: staged build, source log, local connection, and a five-minute defense of one argument choice.
• Case B (problem set) uses 7 + 8 + 5 + 6: open AI-as-tutor practice, weighted secure in-class checks, class-collected data, and critique-the-AI-solution items.
• Case C (research report) uses 5 + 4 + 2 + 7: student-collected data, dated lab notebook, structured Q&A on limitations, and AI allowed for language editing with disclosure.

Honesty notes

• These ratings are the kit author’s analysis, informed by the sources in the literature notes — not a meta-analysis. The evidence-base column is deliberately coarse.
• Two good approaches can still be a bad combination: stacking all eight into one course produces a workload nobody will sustain (see principle 5).
• The University of Sydney two-lane work (Liu & Bridgeman 2023) is the closest published framework to the “secure a few, open the rest” stance this kit takes.