cmos dev
4cad38ef30
Initial phase-1 baseline of the karpathy/autoresearch-style loop. The formatter module is the inner-loop artifact; parser and linter are infra. The linter carries a LINTER_VERSION hash (v0.2.0) that will force a re-baseline on any rule change. Components: - harness/diff.py: case-sensitive field-level substring diff - harness/score.py: three-axis scoring (field, linter, canary exact) - src/cmos/linter.py: 9 CMOS 18 structural rules, each Purdue/CMOS cited - src/cmos/parser.py: locate ## Bibliography section, split entries - src/cmos/formatter.py: prompt + OpenAI call with caller injection - src/cmos/cli.py: cmos format path/to/draft.md - scripts/run_loop.py: loop runner with --fake mode for no-API runs - exemplars/: 3 canary seed exemplars (book, journal w/DOI, web), sourced from chicagomanualofstyle.org quick guide Tests: 48 passing. Fake-mode baseline scalar = 0.000 on the 3 seed exemplars (identity caller fails the linter on every rule). This is the floor the real GPT-5 formatter needs to improve from.
2.9 KiB
Program goal: CMOS 18 bibliography reformatter
Task
Given a markdown draft that contains a bibliography section (delimited by a
## Bibliography heading), rewrite each bibliography entry in Chicago Manual
of Style, 18th edition, notes-and-bibliography format — specifically the
bibliography form (not the note form).
All other text in the draft MUST be left unchanged. Entry order MUST be preserved (we do not re-alphabetize — that is the user's responsibility).
Hard constraints
- Do not fabricate or look up missing metadata. If an entry is missing a
field (e.g., no publisher, no year), leave the field missing and flag the
entry with an inline
<!-- cmos: missing X -->comment. Never invent data. - CMOS 18th edition rules only. Several 17th-ed conventions are obsolete:
- No place of publication for books.
- No chapter page ranges in the bibliography entry for book chapters.
- Maximum 6 authors listed in a bibliography entry; above that, use first 3
- "et al."
- No "Ibid." in subsequent notes (this tool does not emit notes in v1).
- Repeat author names instead of using a 3-em dash.
- AI-generated sources have explicit formatting guidance.
- Italic style markers: render italicized titles as Markdown
*Title*. - Output scope: bibliography section only. In-prose citation rewriting is deferred to v2 and MUST NOT be attempted.
- Determinism:
temperature=0. Log OpenAI model id andsystem_fingerprinton every run.
Inputs and outputs
Input: a single markdown file containing (among other things) a
## Bibliography section.
Output: the same file with the bibliography section rewritten in CMOS 18 form. Everything outside the bibliography section is passed through byte-for- byte.
Success metric
For each exemplar (messy_input, expected_bibliography), the emitted entry must:
- Match the expected output's structured fields (author, title, publisher, year, pages, doi, url) case-sensitively.
- Pass all applicable CMOS 18 linter rules in
src/cmos/linter.py. - For canary exemplars, match the expected string exactly.
Overall loop scalar: (field_match_rate) * (linter_pass_rate).
Canary exact-match rate is a separate gate — any regression blocks acceptance.
Out of scope (v1)
- In-prose citation rewriting ("(Smith 2020, p.15)" → footnote marker).
- Author-date CMOS form.
- Non-bibliography CMOS elements (short notes, tables, figures).
- Fetching metadata from Crossref, OpenLibrary, etc.
- Non-English drafts.
Why this design
This project follows a karpathy/autoresearch-style dev-time loop: the
formatter module (src/cmos/formatter.py) is iteratively edited against a
frozen harness (harness/) scoring against a growing exemplar corpus
(exemplars/). The agent running the loop is Claude Code + pytest. No runtime
self-critique; the shipped formatter is a single-shot transform.