Slide 1 — What you will leave with
- A redesign of one of your own assessments, drafted on paper today
- A shared vocabulary: AI-use levels and the two-lane idea
- One committed next step
We will not hand you a policy. We will hand you a pencil.
Speaker notes
Set the contract: working session, not lecture. People redesign THEIR assignment in pairs from minute 42. Ask for phones on the poll now (next slide).
Slide 2 — First: what worries you?
Anonymous poll (or sticky wall). One question:
When you think about AI in your assessments, what worries you most?
Results stay visible while we talk. We will come back to them at the end.
Speaker notes
Read 4–6 answers aloud (accessibility: blind participants hear them). Expect cheating/detection concerns to dominate — acknowledge, park until slide 8. If someone pushes now: "there's an evidence slide coming; I'd rather you argue with the evidence than with me."
Slide 3 — Live: stress-testing a real-style brief
The sample assignment: a first-year sociology essay ("Does social media harm adolescent mental health?", 1,500 words, five scholarly sources).
Watch the AI write. Then you grade it — with the rubric on your handout.
- Round 1: the brief, pasted as a student would
- Round 2: brief + rubric + "revise toward the top band"
Gut-score each round in 60 seconds — first impressions only. The totals are a vibe, not a mark: real scoring of 1,500 words is a six-minute read per round, and if a glance satisfies the rubric, that is the finding.
Speaker notes
Run the naive prompt, then the determined one. Participants score both against the rubric (handout has the 20-point rubric). Keep the tool on the projector, logged in as the facilitator. If the network fails: appendix backup (pre-generated the day before — never improvised). Timing: 5 min setup, 7 min demo+scoring, 3 min debrief.
Slide 4 — What just happened?
Compare your scores. The pattern (from the kit's stress-test analysis of three assignment types):
- Fluency and structure: AI passes for free — it no longer discriminates between students
- Plausible citations: formatted perfectly, sometimes invented — unless someone checks every source
- Generic analysis: competent, abstract, interchangeable
The honest conclusion: the grade stopped telling us who learned what.
A live demo shows capability, not a measured study. The kit's stress-test pages add one agent-simulated run per case (one AI agent, self-scored, not blind), still not a measured study. Reasoning first, labelled numbers where they exist.
Speaker notes
Collect scores aloud or by poll. Name the rubric criteria AI passed easily; they are usually the heaviest-weighted ones. Do NOT claim measured scores here; point to the kit's protocol and its labelled agent-simulated results. This honesty is a feature: it models the behaviour we want from faculty.
Slide 5 — Principle 1: decide what you're assessing, then decide about AI
The AI question is a validity question: does this task still show the learning it claims to show?
Example (Case A): the essay certified "argument + evidence + analysis + writing". AI can now produce three of the four. The redesign certifies verification of sources and defending choices instead.
Speaker notes
Constructive alignment in one line: outcomes, activities, assessment must point at the same capability. Ask the room: "what does YOUR assignment claim to certify — and can AI produce that artefact without it?" 60 seconds to a neighbour.
Slide 6 — Principle 2: name the AI-use level for every part of every task
The AI Assessment Scale (AIAS 2.1 — Perkins, Furze, Roe & MacVaugh), five levels, exact names:
- No AI
- AI Planning
- AI Collaboration
- Full AI
- AI Exploration
Set the level per stage, not per course. Pair every icon with its text label. Vague rules produce accidental misconduct.
Speaker notes
Show the five levels; emphasise the stage-level thinking with Case A's pattern (planning=open, sources=closed, draft feedback=open, defence=closed). Level names verified against aiassessmentscale.com; the kit's literature notes carry the citations and access dates.
Slide 7 — Principles 3 & 4: assess the process; secure a few checks, open the rest
Process evidence where process is the learning: verified source lists, lab notebooks, use logs, short defences.
Two lanes (after University of Sydney):
- Lane 1 — secure, weighted, few: in-class checks, defences — these certify
- Lane 2 — open, most of the work: practice, drafts, AI-as-tutor — these teach
Case B's weights say it plainly: open practice 10%, secure checks 20%.
Speaker notes
The two-lane framing is Sydney's (Liu & Bridgeman 2023); TEQSA's national advice says the same. The workable insight for skeptics: you cannot secure everything — you can secure a few things and say so.
Slide 8 — Principles 5 & 6: equity and workload; and why not detection
Design for equity, access and workload — or the redesign won't last: oral components get written alternatives; AI levels work on free tiers; faculty notes carry the grading-time arithmetic (Case B: roughly 30% of the original marking load — recomputed line by line).
Why not detection:
- 14 tools tested, none above 80% accuracy; degraded on edited and machine-translated text (Weber-Wulff et al. 2023)
- Detectors misclassify non-native English writers (Liang et al. 2023, Patterns)
- OpenAI retired its own classifier in July 2023 for "low rate of accuracy"
A detector is a smoke alarm with no fire code.
Speaker notes
This is the slide for the "just ban it / just detect it" pushback — the facilitator guide has scripted answers. The equity point lands hardest: detector false positives concentrate on multilingual students, precisely the group already over-disciplined.
Slide 9 — The decision tool
Six questions, keyboard-friendly, nothing stored. It recommends 2–3 approaches with the reasoning shown — because the reasoning is the deliverable, not the ranking.
Open the decision tool (works on phones).
Speaker notes
Demo it live if the room has Wi-Fi (30 seconds, answer 3 questions, show a result card). In the sprint, pairs use it as a cross-check on their canvas — not as an oracle. The "Why this recommendation?" box is the part to show.
Slide 10 — Redesign sprint (minutes 42–70)
Pairs. One real assignment per pair. The redesign canvas walks it — five steps, 26 minutes, ~2 minutes of buffer:
- What learning must it show? (4)
- Where is it vulnerable? (4)
- AI-use level for each stage (6)
- Design moves (8)
- Sketch the rubric change (4)
Finish step 4 early? Run the decision tool as a cross-check. Time-box hard: drafts, not polish.
Speaker notes
Circulate. Broadcast live examples every ~10 min. Most common stall: redesigning the whole course — park it in the canvas's "what this does NOT solve" box. At minute 68 call the switch regardless of state. Designed for 12–30 participants (6–15 pairs); bigger rooms split or run two sprints.
Slide 11 — Gallery walk: give feedback that changes drafts
Three prompts, written ON the canvas:
- Which rubric criterion is still easiest for AI to satisfy?
- What would you cut to protect the grading time?
- Which student would struggle with this redesign — and what would help them?
Visit exactly two other canvases — 2 × 4 minutes — then a 4-minute harvest. Online: pre-assigned doc pairs, comments during each window.
Speaker notes
Ban "looks great". Prompt 3 is the equity canary — read a few answers aloud in the harvest before closing. Two canvases is a hard cap: at 12 minutes, more rotations mean nobody finishes a read.
Slide 12 — Commit to one step
On your canvas footer: one next step, within two weeks, with a date.
While you write: the post-survey (same five items as the pre-survey, plus two open questions) — scan the QR code now, it runs in parallel. Then six volunteered pairs say their step aloud, 15 seconds each.
Kit URL goes round with the slides: the three cases, the decision tool, the templates.
Speaker notes
Collect canvases ONLY with permission, said explicitly. Cap the go-around at six pairs — a full go-around at 15 pairs eats the closing; collect the rest in writing or chat. Post-survey data (anonymous) feeds the workshop's own evaluation loop. Close by naming one worry from slide 2 that the session addressed — and one it didn't (honesty buys credibility for the next session).
Appendix — backup for the live demo
For the facilitator: run the two demo prompts (naive, determined — exact text in the stress-test protocol) the day before, and paste the real outputs here. Label them with tool, model, date — e.g. "Generated with [tool], [model], [date]".
Do not write your own "AI-looking" text as a backup: a fabricated sample defeats the point of the session. If you cannot generate a backup, make the demo a scoring exercise on a colleague's anonymised past submission instead — with their permission.
Speaker notes
This appendix stays empty on purpose in the published kit: outputs belong to the facilitator's own verified run. The labelling rule (tool, model, date) matches the kit's honesty rules for any AI output.