Case C — redesign

What learning must this assessment show?

Course outcome (PSY 250, weeks 7–12): “Students can conduct a small empirical study responsibly — design a measure, collect and manage data, choose and run an appropriate analysis, and defend the inferential choices and limitations — and communicate the study to professional standards.”

What counts as valid evidence:

1. A dataset the student (or their team) actually collected, with a dated record of decisions and problems.
2. Analysis choices the student can justify under questioning.
3. Correct professional write-up — allowed to be AI-assisted with disclosure, because write-up fluency is not the capability being assured.
4. A short structured Q&A on limitations and next steps.

What no longer counts: a solo-authored-seeming report about data the course handed over.

Vulnerability summary (from the stress test)

Original element	Risk	Reason (analysis)
Provided dataset + full report	Structural	Nothing in the task belongs to the student; models write the whole thing
Method + Results marks (12/30)	Highest	Rewards the prose of statistics; unverifiable at scale
Introduction & references (6/30)	High	Fluent lit-review is a model strength; fabricated refs plausible
Discussion (7/30)	Moderate	Generic without lived data-collection experience
“No AI for any part” rule	Structural	Unenforceable; discourages honest language support

Options considered

Option	How it helps validity	Workload	Equity/accessibility risks	Verdict
Proctored in-class report writing	Assurance of unaided writing	Heavy (2-hour sessions × 120)	High for slow processors, EAL writers	Rejected as main form
Individual data collection for all	Strong personal anchor	Unmanageable at 120 individual studies	Approval burden for 120 protocols	Rejected
Team-collected class study + notebook + Q&A	Strong: decisions, mess and defence are assessable	Moderate: one shared protocol, team data	Low with accommodations	Chosen
AI-written report + critique task	Fast; builds evaluation	Low	Low	Partially adopted (analysis-choices memo)
Oral defence for all 120	Strong assurance	~10 h+	Anxiety; alternatives needed	Sampled Q&A chosen

Chosen design and rationale

A team-run class study with a graded lab notebook, a disclosed-AI report, and a short Q&A. The write-up stays AI-assisted on purpose; the decisions move into evidence AI cannot supply: dated notebook entries, justified analysis choices, and a defence.

The structure (weeks 7–12). Component weights below are raw marks summing to 38 points; the course handbook caps the task at 30% of the final grade, so they are scaled by 30/38 in the gradebook (shown honestly in the rubric).

1. Week 8 — team setup (2%). Teams of 4–5. Each team registers a twist on the shared protocol: they choose the measure variant, the sampling slice, or the context (e.g., collect during exam week vs. ordinary weeks). The twist must be registered and justified in 200 words. AI level: AI Planning — AI may help brainstorm the twist; the choice is the team’s.
2. Weeks 8–9 — data collection + dataset (8%). Each team collects 30–50 responses on the shared topic (stress and sleep before assessments — a safe, approved topic with the department’s standing class-study approval). A provided-dataset fallback exists for anyone who cannot collect (see faculty notes).
3. Weeks 8–11 — lab notebook (8%). Dated entries (minimum six): what was decided, what went wrong, what changed. One entry must analyse a problem (e.g., a misread response, a data cleaning decision) and how it was handled. AI level: No AI for the notebook — it is the record of their work. Spot-checked, not audited.
4. Week 11 — analysis-choices memo (4%). 500 words: which test, why, what assumptions were checked, what result would have changed the decision. AI level: AI Collaboration — AI may help check logic; the decision and its defence are the student’s, and the Q&A probes them.
5. Week 12 — report (10%). Full APA report on the team’s data. AI level: AI Collaboration for language and clarity only, with disclosure — not for analysis decisions or results sentences. The rubric grades structure, accuracy and interpretation, not solo fluency.
6. Week 12–13 — structured Q&A (6%). Ten minutes per team, in tutorial: each member answers on their own role — three anchors, one method decision, one result or limitation, and one next step. Written alternative available with equal weight. AI level: No AI.

AI-use level for each part of the task

Part	AIAS 2.1 level	Why
Team setup / twist brainstorm	AI Planning (2)	Ideas are cheap; the justified choice is the learning
Data collection	No AI (1) for the human part; code help allowed (see below)	The collection is the team’s own work
Data cleaning	AI Collaboration (3) — code/debugging help allowed, logged	Real practice; the log shows judgment
Lab notebook	No AI (1)	It certifies the journey; AI would defeat its purpose
Analysis-choices memo	AI Collaboration (3)	Decision + justification are the student’s; AI may check logic
Report	AI Collaboration (3) — language/clarity with disclosure	Write-up fluency is not the assured capability
Structured Q&A	No AI (1)	The secure lane

What students submit

1. Team registration + twist justification (200 words).
2. Dataset (de-identified) + collection notes.
3. Lab notebook (≥6 dated entries).
4. Analysis-choices memo (500 words).
5. APA report (~1,800 words) with a one-paragraph AI-use statement (template: assignment AI statement).
6. Q&A participation (live, recorded, or written alternative).

How it’s graded

The new rubric has 5 criteria × 4 levels: research design & ownership, data practice & integrity, analytic reasoning, communication, and reflective practice. The notebook and memo feed criteria 1–3 and 5; the report feeds criteria 3–4; the Q&A feeds criteria 1–3.

Implementation notes

• One shared protocol + team twists keeps ethics simple: the department’s standing class-study approval covers the topic; twists stay within it (no new measures, no vulnerable groups).
• Teams hand in one notebook but each member tags the entries they led; the Q&A is individual.
• TA marking time drops to roughly 30% of the original load (see faculty notes); the notebook is checked against a checklist and the Q&A replaces re-running everyone’s statistics.
• Limitations are stated honestly in what this does not solve.

What this does NOT solve

• The report can still be substantially AI-assisted within the rules. Language and clarity help are allowed with disclosure; the design no longer treats the prose as the evidence. A student who uses AI for analysis decisions but hides it will be exposed in the memo and Q&A, not by a detector.
• Team free-riding is reduced (tagged notebook entries, individual Q&A), not eliminated.
• Data quality. Class-collected data is messy by nature; that’s a feature for the notebook criterion and a cost everywhere else.
• Students who cannot collect data use the provided-dataset fallback — a real but lesser version of the same assessment; the fallback’s evidence is thinner, which the Q&A compensates for.
• A determined team could still fabricate a notebook. The Q&A + twist registration + spot-checks raise the cost substantially; no design eliminates determined fraud (see the stress-test limits).