Extend the v2 parser and CLI to handle [N] text footnote definitions
in addition to pandoc-style [^marker]: text. The numbered format is
what docx-to-text conversion of footnoted Word documents produces.
The user's "Anti-Communist Formations of LIS" draft uses this format
for all 107 of its notes; without this support, v2 was structurally
incapable of running on real-world docx-derived inputs.
src/cmos/parser.py:
- Add NoteDefinition.original_prefix field so reassembly can
round-trip the source's marker syntax (pandoc input → pandoc
output, numbered input → numbered output) without consumers
needing to know which format was matched.
- Update find_notes() to populate original_prefix as "[^N]: ".
- Add find_numbered_notes() targeting "[N] text" definitions.
Marker must be all digits (rejects [Smith 2020], [foo], etc.);
caret prefix is rejected (rejects pandoc-style cleanly).
src/cmos/cli.py:
- reformat_notes now auto-detects format: tries find_notes first,
falls back to find_numbered_notes if no pandoc definitions found.
Uses definition.original_prefix for reassembly so both formats
round-trip correctly.
tests/test_parser.py:
- 11 new tests for find_numbered_notes covering: single/multiple
definitions, multi-digit markers, line number recording, trailing
whitespace stripping, ignoring pandoc/non-numeric markers, original
prefix recording, and the actual Anti-Communist draft format.
- 1 new test for find_notes original_prefix population.
tests/test_cli.py:
- 2 new tests for reformat_notes auto-detect: numbered input round-
trips as numbered output, pandoc input still round-trips as pandoc.
Total suite: 136/136 (was 123, +13 net new). v1 untouched, 96/96
v1 tests still passing.
Real-draft validation: ran cmos format-notes against the 107-note
Anti-Communist Formations of LIS draft (108 calls in parallel via
the existing concurrency=8 thread pool, completed cleanly). Output
saved to /tmp (not committed). All 107 notes preserved through the
pipeline; ~30-40 first-occurrence full notes produced clean CMOS 18
note form; ~30 shortened-form refs correctly left unchanged;
2 real bugs surfaced for the next iteration (empty input → conver-
sational reply, Ibid → empty string), plus several lower-priority
issues (substantive note truncation, retry waste on shortened
forms, lossy month dropping). Not addressed in this chunk per the
"collect signal, don't fix" plan.
Wire the v2 note formatter into the CLI so it can be invoked on
real markdown drafts. The format-notes subcommand mirrors v1's
format subcommand: parser → formatter → reassemble, with concurrent
API calls.
src/cmos/cli.py:
- Import find_notes and format_note_entry alongside the existing
v1 imports.
- Add reformat_notes(text, formatter, concurrency) that finds
pandoc-style markdown footnote definitions via find_notes,
formats each definition's text via the v2 note formatter (or
an injected fake), and substitutes the formatted text back into
the original line position. Non-definition lines preserved
byte-for-byte. Returns text unchanged when no definitions found.
- Register the format-notes argparse subparser with the same
--concurrency flag as v1's format.
- Dispatch args.command == "format-notes" to reformat_notes.
- Module docstring updated to document both subcommands.
tests/test_cli.py:
- 7 new tests for reformat_notes covering: in-place substitution,
order preservation under concurrency, prose preservation,
reference markers staying verbatim, no-op on empty input,
trailing newline preservation.
- Extended test_python_dash_m_invocation_actually_runs_main to
also assert "format-notes" appears in --help, catching accidental
subcommand removal.
Path B integrity: formatter.py, note_formatter.py, parser.py,
linter.py, harness/score.py all unchanged. No LINTER_VERSION
bump. 96/96 v1 tests still passing. Total suite: 123/123.
Real-API end-to-end smoke test on a temp draft with 2 footnote
definitions: both reformatted byte-for-byte, prose and headings
preserved, ## Conclusion section after the notes preserved.
Two prompt edits to SYSTEM_PROMPT_NOTES driven by chunk 1 smoke test
gaps on chapter_first_doyle.toml.
Rule 6 (chapter form) expanded with an explicit invariance statement:
"ed." is the canonical abbreviation regardless of editor count — do
NOT pluralize to "eds." for multiple editors. CMOS NB treats it as
an invariant abbreviation, not a number-agreeing word.
New rule 18 (publisher expansion) mirrors v1 formatter.py rule 14:
publisher names must be in full canonical form, with note-form-
specific examples ("U of Chicago Press" → "University of Chicago
Press"). Includes the same MIT Press / ALA Editions / MLA carve-out
for publishers whose canonical self-presentation legitimately uses
initials.
Two new prompt-content unit tests added to tests/test_note_formatter.py
following the v1 test_formatter.py discipline. TDD cycle: red-green-
verified end-to-end.
Real-API smoke test, 3 v2 exemplars × 2 runs each: 6/6 byte-perfect
matches (was 4/6 in chunk 1; chapter_first_doyle went 0/2 → 2/2,
book and journal still 2/2). v1 untouched, 96/96 v1 tests still
passing. Total suite: 116/116.
Begin v2 (in-text citation note form) as a parallel artifact to the
v1 bibliography formatter, per the Path B architectural decision:
no shared mutable state, no shared prompt content, no v1 changes.
New artifacts:
- src/cmos/parser.py: add find_notes() / NoteDefinition /
NotesParseResult as siblings to split_bibliography(). Targets
pandoc-style markdown footnote definitions [^marker]: text.
Single-line only; multi-line continuation deferred.
- src/cmos/note_formatter.py: new module mirroring formatter.py.
17-rule SYSTEM_PROMPT_NOTES for CMOS 18 first-occurrence note
form. Reuses cmos.runtime_validator.validate() unchanged — its
structural checks all apply to note form too. Caller injection,
retry loop, and model selection mirror v1.
- tests/test_parser.py: 7 new tests for find_notes().
- tests/test_note_formatter.py: 11 new tests mirroring v1 test
discipline (no API calls, fake-caller injection, retry semantics,
prompt smoke checks).
- exemplars/notes/: new subdirectory with 3 hand-synthesized
first-occurrence note exemplars (book, journal article, chapter
in edited book). Uses expected_note as the field name (not v1's
expected_bibliography). harness/score.py:load_exemplars uses a
non-recursive glob, so the v1 canary loader does not see these —
Path B isolation is automatic.
v1 untouched. v1 canary still loads exactly 17 exemplars. Full
test suite: 96 v1 + 18 new v2 = 114 passing.
Smoke-tested all 3 v2 exemplars against real GPT-5 (not fakes),
2 runs each. book_first_yu and journal_first_kwon: 4/4 byte-perfect
on the first try. chapter_first_doyle: 0/2, surfacing two known
prompt gaps for the next iteration:
1. Publisher abbreviation not expanded ("U of Chicago Press"
preserved instead of "University of Chicago Press"). v1's
formatter.py rule 14 is missing from the v2 prompt.
2. "ed." pluralized to "eds." for multiple editors. CMOS NB
uses "ed." invariantly regardless of editor count.
Both gaps are addressable prompt edits, not architectural problems —
exactly the kind of finding the dev loop is designed to surface.
Out of scope (deferred to chunk 2 and later): CLI extension,
shortened-form generation, document reassembly, harness scoring
loop integration, v2 linter rules, real-draft testing.
The user's drafts in rough_drafts/ are real in-progress academic
work. Project discipline (per memory) is to keep them untracked,
but a single accidental `git add -A` could leak unpublished
scholarship into git history. Adding the pattern to .gitignore
makes the protection structural rather than discipline-based.
Scoped to .txt only (the docx->txt converted form the project
actually consumes); does not affect sample.md, .gitkeep, or any
other extensions that might land in rough_drafts/ later.
Without `if __name__ == "__main__": sys.exit(main())` at the bottom
of cli.py, `python -m cmos.cli format <path>` imports the module
but never invokes main(), so the process silently exits 0 with
empty stdout — indistinguishable from a successful run that
produced no output. Discovered during real-draft testing on
2026-04-11.
Adds a regression test that subprocesses the CLI with --help and
asserts on stdout content. argparse --help exits 0 in both broken
and fixed states; stdout content is the only discriminator.
Both invocation paths now work:
- uv run cmos format <path> (pyproject script entry)
- uv run python -m cmos.cli format <path> (module invocation)
Two new real-draft-derived journal exemplars locking down prompt
rule 29 from the preceding commit. Follows the Hasanah precedent
for promoting real-draft failures into the canary corpus, per the
"defense in depth" feedback in project memory.
- journal_preposition_huttunen.toml exercises "among" mid-title,
plus four other CMOS 8.159 lowercased words ("on", "in", "a",
and implicitly "among"). Real article: Huttunen and Kortelainen,
JASIST 72, no. 7 (2021). The messy_input uses the ASCII hyphen
in "Meaning-Making" for clarity — the source draft actually had
a U+2010 non-breaking hyphen from docx->txt conversion, but
mixing Unicode hyphen normalization into this exemplar would
have conflated two independent failure modes. The U+2010 issue
is noted in the exemplar comment for a separate iteration.
- journal_preposition_mehra.toml exercises "beyond" mid-title,
plus mid-title exclamation-point preservation (rule 25) and
leading-"The" drop from "The Library Quarterly" (rule 17).
Real article: Mehra, Library Quarterly 91, no. 2 (2021).
Both are canary-enabled and will block any future prompt iteration
that regresses on these patterns via the exact-match canary axis.
The canonical dicts additionally lock them down via the field-
level diff, per the upstream-refinement feedback in project
memory.
Canary corpus size: 15 -> 17.
Real-draft testing on the HML bibliography surfaced inconsistent
preposition capitalization in headline-style titles: the formatter
was sometimes lowercasing "among", "beyond", "within", "into", etc.
and sometimes capitalizing them, violating CMOS 8.159. The behavior
was nondeterministic across similar entries — smoking-gun evidence
of GPT-5 judgment drift on a rule the existing prompt only
implicitly covered via rule 6's "headline-style capitalization".
Rule 29 makes the carve-out explicit and instructs the model to
apply CMOS 8.159 independently of the source's casing, with:
- a substantial but illustrative preposition list including "as"
(which CMOS 8.159 calls out as always lowercased)
- an explicit scope restriction to PREPOSITIONS, with subordinating
conjunctions (If, That, Because, Although, Unless, etc.)
capitalized
- a CMOS 8.161 carve-out for hyphenated compounds: the first
element is always capitalized (so "In-School" stays, not
"in-School")
- a first/last-word exception covering "last word of the main
title immediately before a subtitle colon", which the model
was treating as mid-title
Verified on the 134-entry HML draft: ~17 entries now produce more
CMOS-correct forms, and two consecutive re-runs show no new rule-
29-related regressions. Observed GPT-5 drift on orthogonal axes
(multi-author inversion, inner-quote style, periodical
italicization) washed out across re-runs, consistent with the
~5% single-run variance documented in project memory.
Rule count: 28 -> 29. No linter version bump required.
Mark Eaton