Case B — faculty notes
Cost, scaling and equity notes for the redesign. Setting is the fictional STAT 201 at Lakeside State University — a fictional US public university preparing for ADA Title II digital accessibility compliance — 250 students, one coordinator, four TAs.
Grading time
| Component | Original | Redesign | Notes |
|---|---|---|---|
| Weekly sets ×5 | 10–12 min × 250 × 5 weeks (≈ 210–250 h per term across TAs) | ~1.5 min × 250 × 5 (≈ 31 h) — completion + log check | The answer key stops mattering; the log is 5 lines |
| Secure checks ×2 | (nothing equivalent before the midterm) | ~4 min × 250 × 2 (≈ 33 h) | Four questions each; Q4 is 1 mark; marks fast |
| Net | ≈ 210–250 h | ≈ 64 h of marking + calibration (≈ 4 h) ≈ 68 h | Roughly 30% of the original marking load (≈ 68 h vs ≈ 210–250 h), with far more assurance |
Where the grade weight comes from: the original five sets were worth 10% each (≈ 50% of the course grade — see the original brief). The redesign cuts the practice lane to 10% total and moves 20% of the freed weight into the two secure checks; the sets-and-checks block drops from ≈ 50% to 30% of the course grade, and the remaining 20% is re-allocated by the course team at adoption (at Lakeside State, to the final examination). The midterm (20%) is unchanged.
The design trades marking answers for marking logs and short checks. If logs feel too light, the alternative is sampling a deep read of 10% of sets per week (25 scripts × 10–12 min ≈ 4–5 h per week, ≈ 20–25 h across the term).
Scaling to 200+ students
- The secure checks run inside existing tutorial slots — no invigilation hiring, no room booking. Sections of 30 take 20 minutes with the TA present; nothing else fits their slot that week.
- Log grading is binary (complete/partial) — a checklist, not an essay mark. New TAs calibrate on 15 sample logs in one 45-minute session.
- The class dataset is collected once (week 2, LMS form) and cleaned by the coordinator; every section uses the same file. No per-section variation to manage.
- “Find and fix” items are pre-written (from last year’s anonymised errors); adding one to a set costs nothing.
- If TAs are scarce, the checks can be marked on a 20-point scheme by Q4 auto-marking (LMS quiz format for Q1 and Q4, hand-marked for Q2–Q3) — about 2 min/script.
Accessibility and accommodations
- Secure checks: any student with extended-time or rest-break accommodations takes the check with the TA’s standard arrangement; because the check is only 20 minutes, doubling time still fits a tutorial slot.
- Missed check: a make-up slot in the same week; a second miss converts to a 15-minute oral equivalent (or written Q&A) with the coordinator — accommodations don’t require disclosure of reasons.
- Open lane: logs can be voice notes, bullet points, or table rows — format doesn’t matter, honesty does. Screen-reader users and students with RSI can use AI more, not less: the levels never require handwriting, and the tutorial checks have a typed equal-weight alternative per the university’s accommodation process.
- The class survey is optional to answer (it informs the dataset, not your grade); students who opt out use the anonymised dataset like everyone else. No sensitive questions; the collection form is reviewed by the teaching team against the university’s minimal-data guidance.
Multilingual students
- Interpretation-in-context sentences are graded on meaning, not idiom — a plain, slightly awkward sentence that gets the numbers and limits right scores full marks.
- The open lane explicitly legitimises AI language help (“ask AI to explain the question in simpler words” is a logged, allowed use) — which is exactly the support multilingual students have been using silently under the old ban.
- Detector use stays off the table: the evidence on false positives for non-native writers (Liang et al. 2023; Weber-Wulff et al. 2023) is the reason the redesign needs no detector.
- The formula sheet may include the student’s own brief wording of each formula in any language — transcription into English is not being tested.
Common pushback, with answers
- “Full AI on problem sets is surrender.” The sets stop certifying anything; the checks certify. The alternative — banning AI from unenforceable take-home sets — certifies nothing and manufactures suspicion.
- “Two 20-minute checks can’t measure everything.” Correct. They measure the four highest-priority skills, weighted honestly (20% > the practice lane’s 10%), and the midterm remains.
- “Students will split logs — write the log after copying answers.” Some will; that’s why the checks exist and why the log grades judgment (catching an AI error) rather than compliance. The redesign doesn’t claim to make everyone honest; it makes honesty the cheap, safe strategy and makes certainty possible on the secure lane.
What we would measure after the first term
- Check-score correlation with midterm score (validity signal for the checks).
- Log-quality distribution and its relationship to check performance (does engaged AI use predict secure-lane scores?).
- TA hours actually spent vs. the estimates above.