TraceBook · course content in, evidence ledger out

Live demo: the fixture run's claims ledger

This is the actual output of npm run verify and npm run report on the repository's test fixtures — fictional content with five seeded errors and one correct control claim. The fixture ledger's claims and stored quotes were hand-authored to isolate the deterministic steps; only the verdicts and verification notes below come from the pipeline. Fixture source URLs point at a local server (127.0.0.1:8641) that only runs during npm test; that is why the links here are local addresses. The result: 1 verified · 1 snippet-verified · 4 unverified. The pipeline refused to green-light every seeded error.

Reminder: a green ledger surfaces claims for human judgment. It does not certify content as "fact-checked."

Summary

6 claim(s) · 1 verified · 1 snippet-verified · 4 unverified · 0 corrected · 0 not-applicable

The ledger

Every row is one factual claim with its source, verdict, and machine-generated verification note.
ID Claim Seeded error / purpose Verdict Verification note (from verify.mjs)
C001 "Large-scale randomized studies show that high-impact tutoring lifts achievement by 0.41 standard deviations on average." Wrong statistic. The source says 0.10–0.20 SD. UNVERIFIED NUMERIC-MISMATCH — the stored quote was found in the source, but the claim states number(s) [0.41] that do not appear in the quote.
C002 "Spaced practice typically improves long-term retention by 10 to 30 percent compared with massed practice in classroom studies." Control: a correct claim. VERIFIED QUOTE-FOUND — stored quote found verbatim in the live source; numbers consistent.
C003 "One 2024 randomized trial found that structured tutoring reduced course dropout by 18 percent." Paywalled source (HTTP 403). UNVERIFIED ACCESS-RESTRICTED — source returned HTTP 403 (paywalled or blocks automated access). Labeled honestly; the source was not circumvented.
C004 "As of this writing, the CourseKit platform runs on version 3.8.5." Stale version string. The live changelog says 4.2.1. UNVERIFIED QUOTE-NOT-FOUND — similarity 0.67. Either the source changed (content drift), the quote was mis-transcribed, or the source never contained it.
C005 "Retrieval practice strengthens memory more than rereading in most classroom experiments." Drifted wording (source now says "restudying"). VERIFIED-SNIPPET PARTIAL-MATCH — similarity 0.86. Possible content drift or paraphrase. Human judgment required; never treated as fully verified.
C006 "The 2019 accreditation guideline requires programs to publish their evidence base annually." Dead link (HTTP 404). UNVERIFIED SOURCE-UNREACHABLE — source unreachable (HTTP 404).

What the negative test adds

npm test runs this exact pipeline and asserts every verdict above, plus: the agent-run extractor imports only claims found verbatim in the content and makes no network calls; --dry-run writes stubs labeled "DRY RUN – NOT REAL RESULTS"; report --correct rewrites the wrong statistic in place with a dated correction note; and the weekly rot-check flags the dead link (LINK-ROT), the paywall (ACCESS-RESTRICTED), and both changed sources (CONTENT-DRIFT) — then opens no duplicate findings on the next run. 47 assertions, all passing.

Machine-readable ledger: ledger/claims-verified.json · Markdown version: CLAIMS.md