Initial phase-1 baseline of the karpathy/autoresearch-style loop. The formatter module is the inner-loop artifact; parser and linter are infra. The linter carries a LINTER_VERSION hash (v0.2.0) that will force a re-baseline on any rule change. Components: - harness/diff.py: case-sensitive field-level substring diff - harness/score.py: three-axis scoring (field, linter, canary exact) - src/cmos/linter.py: 9 CMOS 18 structural rules, each Purdue/CMOS cited - src/cmos/parser.py: locate ## Bibliography section, split entries - src/cmos/formatter.py: prompt + OpenAI call with caller injection - src/cmos/cli.py: cmos format path/to/draft.md - scripts/run_loop.py: loop runner with --fake mode for no-API runs - exemplars/: 3 canary seed exemplars (book, journal w/DOI, web), sourced from chicagomanualofstyle.org quick guide Tests: 48 passing. Fake-mode baseline scalar = 0.000 on the 3 seed exemplars (identity caller fails the linter on every rule). This is the floor the real GPT-5 formatter needs to improve from.
Exemplars
Each .toml file is one exemplar: a messy input, a canonical structured
record, and an expected CMOS 18 bibliography string.
Format
# IMPORTANT: all top-level keys (including expected_bibliography) must come
# BEFORE any [section] header, or TOML will absorb them into that section.
source = "Purdue OWL CMOS 18 NB PDF, p. X" # provenance, required
type = "book" # book | journal | chapter | web | ai | ...
tags = ["book", "single-author"]
canary = false # if true, exact-string match required
messy_input = "..."
expected_bibliography = "Last, First. *The Book Title*. Publisher Name, 2023."
[canonical]
# Structured fields for the field-level diff. Fields vary by type.
author = "Last, First"
title = "The Book Title"
publisher = "Publisher Name"
year = 2023
Primary sources
- Purdue OWL CMOS 18 NB PDF: https://owl.purdue.edu/owl/research_and_citation/chicago_manual_18th_edition/documents/cmos-nb-20250731.pdf
- CMOS official quick guide: https://www.chicagomanualofstyle.org/tools_citationguide/citation-guide-1.html
- Widener "New in 18th" LibGuide: https://widener.libguides.com/Chicago18/New