Scaffold CMOS 18 reformatter: harness, linter, parser, formatter, CLI
Initial phase-1 baseline of the karpathy/autoresearch-style loop. The formatter module is the inner-loop artifact; parser and linter are infra. The linter carries a LINTER_VERSION hash (v0.2.0) that will force a re-baseline on any rule change. Components: - harness/diff.py: case-sensitive field-level substring diff - harness/score.py: three-axis scoring (field, linter, canary exact) - src/cmos/linter.py: 9 CMOS 18 structural rules, each Purdue/CMOS cited - src/cmos/parser.py: locate ## Bibliography section, split entries - src/cmos/formatter.py: prompt + OpenAI call with caller injection - src/cmos/cli.py: cmos format path/to/draft.md - scripts/run_loop.py: loop runner with --fake mode for no-API runs - exemplars/: 3 canary seed exemplars (book, journal w/DOI, web), sourced from chicagomanualofstyle.org quick guide Tests: 48 passing. Fake-mode baseline scalar = 0.000 on the 3 seed exemplars (identity caller fails the linter on every rule). This is the floor the real GPT-5 formatter needs to improve from.
This commit is contained in:
+67
@@ -0,0 +1,67 @@
|
||||
# Program goal: CMOS 18 bibliography reformatter
|
||||
|
||||
## Task
|
||||
|
||||
Given a markdown draft that contains a bibliography section (delimited by a
|
||||
`## Bibliography` heading), rewrite each bibliography entry in **Chicago Manual
|
||||
of Style, 18th edition, notes-and-bibliography format** — specifically the
|
||||
bibliography form (not the note form).
|
||||
|
||||
All other text in the draft MUST be left unchanged. Entry order MUST be
|
||||
preserved (we do not re-alphabetize — that is the user's responsibility).
|
||||
|
||||
## Hard constraints
|
||||
|
||||
1. **Do not fabricate or look up missing metadata.** If an entry is missing a
|
||||
field (e.g., no publisher, no year), leave the field missing and flag the
|
||||
entry with an inline `<!-- cmos: missing X -->` comment. Never invent data.
|
||||
2. **CMOS 18th edition rules only.** Several 17th-ed conventions are obsolete:
|
||||
- No place of publication for books.
|
||||
- No chapter page ranges in the bibliography entry for book chapters.
|
||||
- Maximum 6 authors listed in a bibliography entry; above that, use first 3
|
||||
+ "et al."
|
||||
- No "Ibid." in subsequent notes (this tool does not emit notes in v1).
|
||||
- Repeat author names instead of using a 3-em dash.
|
||||
- AI-generated sources have explicit formatting guidance.
|
||||
3. **Italic style markers:** render italicized titles as Markdown `*Title*`.
|
||||
4. **Output scope:** bibliography section only. In-prose citation rewriting is
|
||||
deferred to v2 and MUST NOT be attempted.
|
||||
5. **Determinism:** `temperature=0`. Log OpenAI model id and
|
||||
`system_fingerprint` on every run.
|
||||
|
||||
## Inputs and outputs
|
||||
|
||||
**Input:** a single markdown file containing (among other things) a
|
||||
`## Bibliography` section.
|
||||
|
||||
**Output:** the same file with the bibliography section rewritten in CMOS 18
|
||||
form. Everything outside the bibliography section is passed through byte-for-
|
||||
byte.
|
||||
|
||||
## Success metric
|
||||
|
||||
For each exemplar (messy_input, expected_bibliography), the emitted entry must:
|
||||
|
||||
1. Match the expected output's **structured fields** (author, title, publisher,
|
||||
year, pages, doi, url) case-sensitively.
|
||||
2. Pass **all applicable CMOS 18 linter rules** in `src/cmos/linter.py`.
|
||||
3. For canary exemplars, match the expected string **exactly**.
|
||||
|
||||
Overall loop scalar: `(field_match_rate) * (linter_pass_rate)`.
|
||||
Canary exact-match rate is a separate gate — any regression blocks acceptance.
|
||||
|
||||
## Out of scope (v1)
|
||||
|
||||
- In-prose citation rewriting ("(Smith 2020, p.15)" → footnote marker).
|
||||
- Author-date CMOS form.
|
||||
- Non-bibliography CMOS elements (short notes, tables, figures).
|
||||
- Fetching metadata from Crossref, OpenLibrary, etc.
|
||||
- Non-English drafts.
|
||||
|
||||
## Why this design
|
||||
|
||||
This project follows a karpathy/autoresearch-style dev-time loop: the
|
||||
formatter module (`src/cmos/formatter.py`) is iteratively edited against a
|
||||
frozen harness (`harness/`) scoring against a growing exemplar corpus
|
||||
(`exemplars/`). The agent running the loop is Claude Code + pytest. No runtime
|
||||
self-critique; the shipped formatter is a single-shot transform.
|
||||
Reference in New Issue
Block a user