Case B — faculty notes

Cost, scaling and equity notes for the redesign. Setting is the fictional STAT 201 at Lakeside State University — a fictional US public university preparing for ADA Title II digital accessibility compliance — 250 students, one coordinator, four TAs.

Grading time

Component Original Redesign Notes
Weekly sets ×5 10–12 min × 250 × 5 weeks (≈ 210–250 h per term across TAs) ~1.5 min × 250 × 5 (≈ 31 h) — completion + log check The answer key stops mattering; the log is 5 lines
Secure checks ×2 (nothing equivalent before the midterm) ~4 min × 250 × 2 (≈ 33 h) Four questions each; Q4 is 1 mark; marks fast
Net ≈ 210–250 h ≈ 64 h of marking + calibration (≈ 4 h) ≈ 68 h Roughly 30% of the original marking load (≈ 68 h vs ≈ 210–250 h), with far more assurance

Where the grade weight comes from: the original five sets were worth 10% each (≈ 50% of the course grade — see the original brief). The redesign cuts the practice lane to 10% total and moves 20% of the freed weight into the two secure checks; the sets-and-checks block drops from ≈ 50% to 30% of the course grade, and the remaining 20% is re-allocated by the course team at adoption (at Lakeside State, to the final examination). The midterm (20%) is unchanged.

The design trades marking answers for marking logs and short checks. If logs feel too light, the alternative is sampling a deep read of 10% of sets per week (25 scripts × 10–12 min ≈ 4–5 h per week, ≈ 20–25 h across the term).

Scaling to 200+ students

  1. The secure checks run inside existing tutorial slots — no invigilation hiring, no room booking. Sections of 30 take 20 minutes with the TA present; nothing else fits their slot that week.
  2. Log grading is binary (complete/partial) — a checklist, not an essay mark. New TAs calibrate on 15 sample logs in one 45-minute session.
  3. The class dataset is collected once (week 2, LMS form) and cleaned by the coordinator; every section uses the same file. No per-section variation to manage.
  4. “Find and fix” items are pre-written (from last year’s anonymised errors); adding one to a set costs nothing.
  5. If TAs are scarce, the checks can be marked on a 20-point scheme by Q4 auto-marking (LMS quiz format for Q1 and Q4, hand-marked for Q2–Q3) — about 2 min/script.

Accessibility and accommodations

  • Secure checks: any student with extended-time or rest-break accommodations takes the check with the TA’s standard arrangement; because the check is only 20 minutes, doubling time still fits a tutorial slot.
  • Missed check: a make-up slot in the same week; a second miss converts to a 15-minute oral equivalent (or written Q&A) with the coordinator — accommodations don’t require disclosure of reasons.
  • Open lane: logs can be voice notes, bullet points, or table rows — format doesn’t matter, honesty does. Screen-reader users and students with RSI can use AI more, not less: the levels never require handwriting, and the tutorial checks have a typed equal-weight alternative per the university’s accommodation process.
  • The class survey is optional to answer (it informs the dataset, not your grade); students who opt out use the anonymised dataset like everyone else. No sensitive questions; the collection form is reviewed by the teaching team against the university’s minimal-data guidance.

Multilingual students

  • Interpretation-in-context sentences are graded on meaning, not idiom — a plain, slightly awkward sentence that gets the numbers and limits right scores full marks.
  • The open lane explicitly legitimises AI language help (“ask AI to explain the question in simpler words” is a logged, allowed use) — which is exactly the support multilingual students have been using silently under the old ban.
  • Detector use stays off the table: the evidence on false positives for non-native writers (Liang et al. 2023; Weber-Wulff et al. 2023) is the reason the redesign needs no detector.
  • The formula sheet may include the student’s own brief wording of each formula in any language — transcription into English is not being tested.

Common pushback, with answers

  • “Full AI on problem sets is surrender.” The sets stop certifying anything; the checks certify. The alternative — banning AI from unenforceable take-home sets — certifies nothing and manufactures suspicion.
  • “Two 20-minute checks can’t measure everything.” Correct. They measure the four highest-priority skills, weighted honestly (20% > the practice lane’s 10%), and the midterm remains.
  • “Students will split logs — write the log after copying answers.” Some will; that’s why the checks exist and why the log grades judgment (catching an AI error) rather than compliance. The redesign doesn’t claim to make everyone honest; it makes honesty the cheap, safe strategy and makes certainty possible on the secure lane.

What we would measure after the first term

  • Check-score correlation with midterm score (validity signal for the checks).
  • Log-quality distribution and its relationship to check performance (does engaged AI use predict secure-lane scores?).
  • TA hours actually spent vs. the estimates above.

Back to top

Content licensed CC BY 4.0 · Code licensed MIT · Case study context: Lakeside State University is fictional.

This site uses Just the Docs, a documentation theme for Jekyll.