Case B — new rubric
4 criteria × 4 levels. The two secure checks (10% each) are marked on criteria 1–3; the open practice sets are marked on criterion 4 (the AI-use log). Alignment to the course outcome is in the table at the end.
The rubric
| Criterion | 10 — Accomplished | 7.5 — Proficient | 5 — Developing | 2.5 — Beginning |
|---|---|---|---|---|
| 1. Method selection (10) | Names the right procedure and why (assumptions, design); chooses correctly in unfamiliar contexts | Correct procedure chosen; reasoning mostly stated | Right procedure but reasoning unstated, or near-miss procedures | Procedure mismatched to question |
| 2. Execution & accuracy (10) | Computation correct; units and notation clean; arithmetic checks shown | Computation correct; minor notation slips | One significant slip (wrong df, dropped unit) | Multiple computational failures |
| 3. Interpretation in context (10) | Result interpreted with this data’s meaning, limits and a next-step implication; numbers cited correctly | Correct in-context interpretation; limits implied | Template interpretation; context partly generic | Bare number, or interpretation contradicts the result |
| 4. AI-use judgment (10, open lane only) | Log specific per question; shows catching and correcting at least one AI error; choices about when not to use AI explained | Log honest and complete; engagement mostly explained | Log present but thin (“used AI for Q3”) | No log; log contradicts work; or wholesale unlogged AI use |
How the marks map to the course grade (also in faculty notes):
| Component | Weight | Criteria |
|---|---|---|
| Weekly sets ×5 (2 pts each) | 10% | 4 (log quality, completion-based) |
| Secure check 1 (week 3) | 10% | 1–3 (scaled to 30 pts → 10%) |
| Secure check 2 (week 6) | 10% | 1–3 |
| Midterm (unchanged, secure) | 20% | 1–3 |
| Assessment-change summary | Practice lane 10% total (the five sets were 10% each under the original design — ≈50% of the course grade, graded for answers); secure lane 20%, new in weeks 1–6 |
Alignment to the course outcome
| Outcome element (“choose, execute, interpret, check”) | Rubric criterion | Evidence source |
|---|---|---|
| Choose the appropriate procedure | 1 | Secure checks (No AI) |
| Execute correctly | 2 | Secure checks (No AI) |
| Interpret in context | 3 | Class-data items + secure checks |
| Check and critique solutions (incl. AI output) | 4 + “find and fix” items | Open-lane logs |
Why this rubric replaced the original
- The original 20-point scheme graded method+answer per question — exactly the two things models do reliably. It couldn’t see judgment, critique, or in-context interpretation tied to real data.
- Criterion 3 now requires our dataset’s meaning — anchoring interpretation to something generic output doesn’t have.
- Criterion 4 grades the AI-use log as a skill (judgment, catching errors), so honest heavy AI use scores higher than silent copying — the incentive points at learning.
- The rubric is deliberately short (4 criteria) so TAs can mark a 20-minute secure check in ~4 minutes.