Scaffold CMOS 18 reformatter: harness, linter, parser, formatter, CLI

Initial phase-1 baseline of the karpathy/autoresearch-style loop.
The formatter module is the inner-loop artifact; parser and linter
are infra. The linter carries a LINTER_VERSION hash (v0.2.0) that
will force a re-baseline on any rule change.

Components:
- harness/diff.py: case-sensitive field-level substring diff
- harness/score.py: three-axis scoring (field, linter, canary exact)
- src/cmos/linter.py: 9 CMOS 18 structural rules, each Purdue/CMOS cited
- src/cmos/parser.py: locate ## Bibliography section, split entries
- src/cmos/formatter.py: prompt + OpenAI call with caller injection
- src/cmos/cli.py: cmos format path/to/draft.md
- scripts/run_loop.py: loop runner with --fake mode for no-API runs
- exemplars/: 3 canary seed exemplars (book, journal w/DOI, web),
  sourced from chicagomanualofstyle.org quick guide

Tests: 48 passing. Fake-mode baseline scalar = 0.000 on the 3 seed
exemplars (identity caller fails the linter on every rule). This is
the floor the real GPT-5 formatter needs to improve from.
This commit is contained in:
cmos dev
2026-04-10 20:48:33 -04:00
commit 4cad38ef30
29 changed files with 2267 additions and 0 deletions
+67
View File
@@ -0,0 +1,67 @@
# Program goal: CMOS 18 bibliography reformatter
## Task
Given a markdown draft that contains a bibliography section (delimited by a
`## Bibliography` heading), rewrite each bibliography entry in **Chicago Manual
of Style, 18th edition, notes-and-bibliography format** — specifically the
bibliography form (not the note form).
All other text in the draft MUST be left unchanged. Entry order MUST be
preserved (we do not re-alphabetize — that is the user's responsibility).
## Hard constraints
1. **Do not fabricate or look up missing metadata.** If an entry is missing a
field (e.g., no publisher, no year), leave the field missing and flag the
entry with an inline `<!-- cmos: missing X -->` comment. Never invent data.
2. **CMOS 18th edition rules only.** Several 17th-ed conventions are obsolete:
- No place of publication for books.
- No chapter page ranges in the bibliography entry for book chapters.
- Maximum 6 authors listed in a bibliography entry; above that, use first 3
+ "et al."
- No "Ibid." in subsequent notes (this tool does not emit notes in v1).
- Repeat author names instead of using a 3-em dash.
- AI-generated sources have explicit formatting guidance.
3. **Italic style markers:** render italicized titles as Markdown `*Title*`.
4. **Output scope:** bibliography section only. In-prose citation rewriting is
deferred to v2 and MUST NOT be attempted.
5. **Determinism:** `temperature=0`. Log OpenAI model id and
`system_fingerprint` on every run.
## Inputs and outputs
**Input:** a single markdown file containing (among other things) a
`## Bibliography` section.
**Output:** the same file with the bibliography section rewritten in CMOS 18
form. Everything outside the bibliography section is passed through byte-for-
byte.
## Success metric
For each exemplar (messy_input, expected_bibliography), the emitted entry must:
1. Match the expected output's **structured fields** (author, title, publisher,
year, pages, doi, url) case-sensitively.
2. Pass **all applicable CMOS 18 linter rules** in `src/cmos/linter.py`.
3. For canary exemplars, match the expected string **exactly**.
Overall loop scalar: `(field_match_rate) * (linter_pass_rate)`.
Canary exact-match rate is a separate gate — any regression blocks acceptance.
## Out of scope (v1)
- In-prose citation rewriting ("(Smith 2020, p.15)" → footnote marker).
- Author-date CMOS form.
- Non-bibliography CMOS elements (short notes, tables, figures).
- Fetching metadata from Crossref, OpenLibrary, etc.
- Non-English drafts.
## Why this design
This project follows a karpathy/autoresearch-style dev-time loop: the
formatter module (`src/cmos/formatter.py`) is iteratively edited against a
frozen harness (`harness/`) scoring against a growing exemplar corpus
(`exemplars/`). The agent running the loop is Claude Code + pytest. No runtime
self-critique; the shipped formatter is a single-shot transform.