553240c15388861f2e2450b5fbd56cf1a185450f
19
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Mark Eaton
|
553240c153 |
v2 chunk 3: bug fixes from real-draft triage + polish
Six independent fixes addressing the Anti-Communist Formations of LIS real-draft test findings (chunk 2c). Four are bug fixes for issues that destroyed user data or wasted API calls; two are polish quality improvements revisited from the deferred list. src/cmos/note_formatter.py — empty input guard (fix #1) Empty / whitespace-only input now short-circuits the API entirely and returns the input verbatim. Surfaced by note [66] in the Anti-Communist draft, where GPT-5 broke character on empty input and returned a conversational meta-reply ("Please paste the citation entry...") that then got substituted into the document. Whitespace preserved so cli.reformat_notes is byte-exact on empty notes. v2 only. src/cmos/note_formatter.py — deprecated Latin guard (fix #2) New Python guard catches the full CMOS-18-deprecated Latin citation set (ibid, idem, id., op. cit., loc. cit.) and returns input verbatim before the API call. Rule 10 in the prompt also rewritten: explicitly says "return verbatim" instead of the old "return cleanest possible full-form note", which the model interpreted as "return empty when there's no information", silently destroying the marker. Surfaced by note [61] = "Ibid.". v2 only. src/cmos/runtime_validator.py — shortened-form carve-out (fix #3) The "must contain italics" check is now skipped when the candidate looks like a CMOS shortened-form note (author last name, optional page, no italic content). Surfaced by ~30 of 107 notes in the Anti-Communist draft that were correctly returned as shortened forms ("Rosen, 7.", "Mitchell, 197.") but rejected by the validator's strict italic check, firing the full retry budget on valid output (~60 wasted API calls per run). The carve-out is conservative: name-token regex + terminal period; doesn't match unstructured prose. Shared infrastructure — affects v1 and v2, no-op in v1's normal workflow because v1 outputs always have italics. src/cmos/note_formatter.py — substantive prose pass-through (fix #4) Long discursive prose with no citation skeleton markers now short-circuits the API and returns the input verbatim. Surfaced by notes [8] (689 chars), [9] (893 chars), [75] (163 chars) in the Anti-Communist draft — substantive notes that the formatter was extracting a single citation from and silently discarding the surrounding commentary. CMOS 14.39 explicitly allows substantive notes; preserving them is the user's explicit policy ("preserve all free text as long as that does not break other formatting"). Detection: length > 150 chars AND no citation skeleton markers (parenthesized year, URL, vol./no./pp., DOI, terminal page or terminal year). Conservative — does not false-positive on legitimate first-occurrence notes. v2 only. src/cmos/formatter.py + note_formatter.py — month/season preservation (polish #5) Both v1 rule 10 (journal article format) and v2 rule 5 (journal article note form) now explicitly instruct the model to preserve (Month YEAR) and (Season YEAR) parentheticals when the source provides them. Surfaced by entries in both Reading_Disrepair and Anti-Communist where (Spring, 1993) and August 1952 were silently collapsed to (1993) and 1952. Both forms are valid CMOS but month/season is more informative when the source has it. Affects v1 and v2 in mirror. src/cmos/note_formatter.py — government documents rule 19 (polish #7) v2 now has an explicit rule mirroring v1's rule 27: government bodies, institutional reports, and similar standalone documents get italicized titles and book-form treatment, NOT quoted-article treatment. Includes the Anti-Communist Senate of California Tenth Report and a hypothetical GAO example. Implicit handling worked in chunk 2c, but explicit rule provides regression protection. v2 only. Tests: +14 net new (across test_note_formatter.py, test_formatter.py, test_runtime_validator.py). Total suite 150/150 (was 136 before this chunk). All v1 tests still passing. Path B status: formatter.py and runtime_validator.py edits cross the v1/v2 boundary, but only by user-explicit approval per the relevant fix discussions. The shared validator was always shared; the v1 formatter rule 10 mirror is a small additive edit that doesn't affect the v1 prompt's existing behavior on its existing inputs. |
||
|
Mark Eaton
|
45f51c6341 |
v2 chunk 2c: numbered-note format support + real-draft test
Extend the v2 parser and CLI to handle [N] text footnote definitions in addition to pandoc-style [^marker]: text. The numbered format is what docx-to-text conversion of footnoted Word documents produces. The user's "Anti-Communist Formations of LIS" draft uses this format for all 107 of its notes; without this support, v2 was structurally incapable of running on real-world docx-derived inputs. src/cmos/parser.py: - Add NoteDefinition.original_prefix field so reassembly can round-trip the source's marker syntax (pandoc input → pandoc output, numbered input → numbered output) without consumers needing to know which format was matched. - Update find_notes() to populate original_prefix as "[^N]: ". - Add find_numbered_notes() targeting "[N] text" definitions. Marker must be all digits (rejects [Smith 2020], [foo], etc.); caret prefix is rejected (rejects pandoc-style cleanly). src/cmos/cli.py: - reformat_notes now auto-detects format: tries find_notes first, falls back to find_numbered_notes if no pandoc definitions found. Uses definition.original_prefix for reassembly so both formats round-trip correctly. tests/test_parser.py: - 11 new tests for find_numbered_notes covering: single/multiple definitions, multi-digit markers, line number recording, trailing whitespace stripping, ignoring pandoc/non-numeric markers, original prefix recording, and the actual Anti-Communist draft format. - 1 new test for find_notes original_prefix population. tests/test_cli.py: - 2 new tests for reformat_notes auto-detect: numbered input round- trips as numbered output, pandoc input still round-trips as pandoc. Total suite: 136/136 (was 123, +13 net new). v1 untouched, 96/96 v1 tests still passing. Real-draft validation: ran cmos format-notes against the 107-note Anti-Communist Formations of LIS draft (108 calls in parallel via the existing concurrency=8 thread pool, completed cleanly). Output saved to /tmp (not committed). All 107 notes preserved through the pipeline; ~30-40 first-occurrence full notes produced clean CMOS 18 note form; ~30 shortened-form refs correctly left unchanged; 2 real bugs surfaced for the next iteration (empty input → conver- sational reply, Ibid → empty string), plus several lower-priority issues (substantive note truncation, retry waste on shortened forms, lossy month dropping). Not addressed in this chunk per the "collect signal, don't fix" plan. |
||
|
Mark Eaton
|
87cb70b4fb |
v2 chunk 2b: cli format-notes subcommand and reformat_notes()
Wire the v2 note formatter into the CLI so it can be invoked on real markdown drafts. The format-notes subcommand mirrors v1's format subcommand: parser → formatter → reassemble, with concurrent API calls. src/cmos/cli.py: - Import find_notes and format_note_entry alongside the existing v1 imports. - Add reformat_notes(text, formatter, concurrency) that finds pandoc-style markdown footnote definitions via find_notes, formats each definition's text via the v2 note formatter (or an injected fake), and substitutes the formatted text back into the original line position. Non-definition lines preserved byte-for-byte. Returns text unchanged when no definitions found. - Register the format-notes argparse subparser with the same --concurrency flag as v1's format. - Dispatch args.command == "format-notes" to reformat_notes. - Module docstring updated to document both subcommands. tests/test_cli.py: - 7 new tests for reformat_notes covering: in-place substitution, order preservation under concurrency, prose preservation, reference markers staying verbatim, no-op on empty input, trailing newline preservation. - Extended test_python_dash_m_invocation_actually_runs_main to also assert "format-notes" appears in --help, catching accidental subcommand removal. Path B integrity: formatter.py, note_formatter.py, parser.py, linter.py, harness/score.py all unchanged. No LINTER_VERSION bump. 96/96 v1 tests still passing. Total suite: 123/123. Real-API end-to-end smoke test on a temp draft with 2 footnote definitions: both reformatted byte-for-byte, prose and headings preserved, ## Conclusion section after the notes preserved. |
||
|
Mark Eaton
|
d061f78b7f |
v2 chunk 2a iter 1: rules 6+18 — ed. invariant, expand publishers
Two prompt edits to SYSTEM_PROMPT_NOTES driven by chunk 1 smoke test
gaps on chapter_first_doyle.toml.
Rule 6 (chapter form) expanded with an explicit invariance statement:
"ed." is the canonical abbreviation regardless of editor count — do
NOT pluralize to "eds." for multiple editors. CMOS NB treats it as
an invariant abbreviation, not a number-agreeing word.
New rule 18 (publisher expansion) mirrors v1 formatter.py rule 14:
publisher names must be in full canonical form, with note-form-
specific examples ("U of Chicago Press" → "University of Chicago
Press"). Includes the same MIT Press / ALA Editions / MLA carve-out
for publishers whose canonical self-presentation legitimately uses
initials.
Two new prompt-content unit tests added to tests/test_note_formatter.py
following the v1 test_formatter.py discipline. TDD cycle: red-green-
verified end-to-end.
Real-API smoke test, 3 v2 exemplars × 2 runs each: 6/6 byte-perfect
matches (was 4/6 in chunk 1; chapter_first_doyle went 0/2 → 2/2,
book and journal still 2/2). v1 untouched, 96/96 v1 tests still
passing. Total suite: 116/116.
|
||
|
Mark Eaton
|
28ca3ac928 |
v2 chunk 1: scaffold note formatter (Path B parallel artifact)
Begin v2 (in-text citation note form) as a parallel artifact to the
v1 bibliography formatter, per the Path B architectural decision:
no shared mutable state, no shared prompt content, no v1 changes.
New artifacts:
- src/cmos/parser.py: add find_notes() / NoteDefinition /
NotesParseResult as siblings to split_bibliography(). Targets
pandoc-style markdown footnote definitions [^marker]: text.
Single-line only; multi-line continuation deferred.
- src/cmos/note_formatter.py: new module mirroring formatter.py.
17-rule SYSTEM_PROMPT_NOTES for CMOS 18 first-occurrence note
form. Reuses cmos.runtime_validator.validate() unchanged — its
structural checks all apply to note form too. Caller injection,
retry loop, and model selection mirror v1.
- tests/test_parser.py: 7 new tests for find_notes().
- tests/test_note_formatter.py: 11 new tests mirroring v1 test
discipline (no API calls, fake-caller injection, retry semantics,
prompt smoke checks).
- exemplars/notes/: new subdirectory with 3 hand-synthesized
first-occurrence note exemplars (book, journal article, chapter
in edited book). Uses expected_note as the field name (not v1's
expected_bibliography). harness/score.py:load_exemplars uses a
non-recursive glob, so the v1 canary loader does not see these —
Path B isolation is automatic.
v1 untouched. v1 canary still loads exactly 17 exemplars. Full
test suite: 96 v1 + 18 new v2 = 114 passing.
Smoke-tested all 3 v2 exemplars against real GPT-5 (not fakes),
2 runs each. book_first_yu and journal_first_kwon: 4/4 byte-perfect
on the first try. chapter_first_doyle: 0/2, surfacing two known
prompt gaps for the next iteration:
1. Publisher abbreviation not expanded ("U of Chicago Press"
preserved instead of "University of Chicago Press"). v1's
formatter.py rule 14 is missing from the v2 prompt.
2. "ed." pluralized to "eds." for multiple editors. CMOS NB
uses "ed." invariantly regardless of editor count.
Both gaps are addressable prompt edits, not architectural problems —
exactly the kind of finding the dev loop is designed to surface.
Out of scope (deferred to chunk 2 and later): CLI extension,
shortened-form generation, document reassembly, harness scoring
loop integration, v2 linter rules, real-draft testing.
|
||
|
Mark Eaton
|
2a8c72ae22 |
gitignore: protect rough_drafts/*.txt from accidental commits
The user's drafts in rough_drafts/ are real in-progress academic work. Project discipline (per memory) is to keep them untracked, but a single accidental `git add -A` could leak unpublished scholarship into git history. Adding the pattern to .gitignore makes the protection structural rather than discipline-based. Scoped to .txt only (the docx->txt converted form the project actually consumes); does not affect sample.md, .gitkeep, or any other extensions that might land in rough_drafts/ later. |
||
|
Mark Eaton
|
22404fbd6b |
cli: add __main__ guard so python -m cmos.cli actually runs
Without `if __name__ == "__main__": sys.exit(main())` at the bottom of cli.py, `python -m cmos.cli format <path>` imports the module but never invokes main(), so the process silently exits 0 with empty stdout — indistinguishable from a successful run that produced no output. Discovered during real-draft testing on 2026-04-11. Adds a regression test that subprocesses the CLI with --help and asserts on stdout content. argparse --help exits 0 in both broken and fixed states; stdout content is the only discriminator. Both invocation paths now work: - uv run cmos format <path> (pyproject script entry) - uv run python -m cmos.cli format <path> (module invocation) |
||
|
Mark Eaton
|
1d970e84f9 |
exemplars: canary for CMOS 8.159 preposition rule (Huttunen, Mehra)
Two new real-draft-derived journal exemplars locking down prompt
rule 29 from the preceding commit. Follows the Hasanah precedent
for promoting real-draft failures into the canary corpus, per the
"defense in depth" feedback in project memory.
- journal_preposition_huttunen.toml exercises "among" mid-title,
plus four other CMOS 8.159 lowercased words ("on", "in", "a",
and implicitly "among"). Real article: Huttunen and Kortelainen,
JASIST 72, no. 7 (2021). The messy_input uses the ASCII hyphen
in "Meaning-Making" for clarity — the source draft actually had
a U+2010 non-breaking hyphen from docx->txt conversion, but
mixing Unicode hyphen normalization into this exemplar would
have conflated two independent failure modes. The U+2010 issue
is noted in the exemplar comment for a separate iteration.
- journal_preposition_mehra.toml exercises "beyond" mid-title,
plus mid-title exclamation-point preservation (rule 25) and
leading-"The" drop from "The Library Quarterly" (rule 17).
Real article: Mehra, Library Quarterly 91, no. 2 (2021).
Both are canary-enabled and will block any future prompt iteration
that regresses on these patterns via the exact-match canary axis.
The canonical dicts additionally lock them down via the field-
level diff, per the upstream-refinement feedback in project
memory.
Canary corpus size: 15 -> 17.
|
||
|
Mark Eaton
|
0238f5e5d7 |
iter 8: prompt rule 29 for CMOS 8.159 headline-case prepositions
Real-draft testing on the HML bibliography surfaced inconsistent preposition capitalization in headline-style titles: the formatter was sometimes lowercasing "among", "beyond", "within", "into", etc. and sometimes capitalizing them, violating CMOS 8.159. The behavior was nondeterministic across similar entries — smoking-gun evidence of GPT-5 judgment drift on a rule the existing prompt only implicitly covered via rule 6's "headline-style capitalization". Rule 29 makes the carve-out explicit and instructs the model to apply CMOS 8.159 independently of the source's casing, with: - a substantial but illustrative preposition list including "as" (which CMOS 8.159 calls out as always lowercased) - an explicit scope restriction to PREPOSITIONS, with subordinating conjunctions (If, That, Because, Although, Unless, etc.) capitalized - a CMOS 8.161 carve-out for hyphenated compounds: the first element is always capitalized (so "In-School" stays, not "in-School") - a first/last-word exception covering "last word of the main title immediately before a subtitle colon", which the model was treating as mid-title Verified on the 134-entry HML draft: ~17 entries now produce more CMOS-correct forms, and two consecutive re-runs show no new rule- 29-related regressions. Observed GPT-5 drift on orthogonal axes (multi-author inversion, inner-quote style, periodical italicization) washed out across re-runs, consistent with the ~5% single-run variance documented in project memory. Rule count: 28 -> 29. No linter version bump required. |
||
|
cmos dev
|
8b14dbc9ca |
formatter: linter-guided retry to suppress GPT-5 reasoning drift
GPT-5 is a reasoning model and is not bit-deterministic even at the API level (no temperature override allowed). Variance testing in iter 7 showed ~1 in 6 loop runs hit a nondeterministic regression: in one run the magazine_mead exemplar lost its italic markers around "New Yorker" even though the exact same input had produced a clean output 5 times prior. The scoring linter caught it (rule_magazine_name_italicized) but the formatter still emitted the bad output to the user. This commit adds a runtime guardrail: - New module src/cmos/runtime_validator.py — type-independent structural sanity checks. Operates on a single candidate string with no Exemplar context, because at runtime we don't know the source type. Checks: ends with period, no Ibid., has at least one italic span (almost every CMOS bibliography entry italicizes something), balanced * and " markers. Deliberately weaker than the scoring linter; it's a fast guardrail, not a full validator. - format_bibliography_entry now retries up to DEFAULT_MAX_RETRIES (2) times when the validator rejects a candidate. Independent re-calls are usually enough because the failures are stochastic. If every attempt fails, the LAST attempt is returned (no exception) — the caller still gets something usable, and the failure surfaces through the scoring linter or human review. The retry path costs zero on the common case (1 call per entry); ~1-2% extra calls on noisy drafts. Empirical: 3 consecutive loop runs after this change are scalar 1.000 canary 1.000 (vs 5/6 clean in the variance test before). Sample is too small to claim full suppression but the signal is positive. Also adds a new exemplar journal_multiauthor_secondary_first_last.toml captured from the user's HML draft (Hasanah et al., IJIDI 2024). It exercises the case where a multi-author entry has the first author inverted and the rest in First Last form — which the iter 7 HML run got wrong on one entry. Variance testing showed the exemplar passes 6/6 in isolation, so the original HML failure was nondeterminism, not a missing rule. Keeping the exemplar regardless: it adds canary coverage of a real-world multi-author pattern, no-DOI / JSTOR-URL variant, and the year-suffix author-date holdover stripping. Tests: 85 → 95 (8 new for runtime_validator + 2 new for formatter retry). All passing. |
||
|
cmos dev
|
a21d2614d3 |
iter 7: prompt rules 27-29 driven by real-draft testing
First exercise of the tool against unfamiliar real-world drafts (user's
in-progress academic papers, not derived from CMOS quick guide
examples). Found three systemic patterns affecting ~14% of entries
across 134-entry and 13-entry drafts; this commit addresses all three.
Prompt additions:
27. GOVERNMENT DOCUMENTS / INSTITUTIONAL REPORTS — when the author is
a government body (GAO, US Census, WHO, Pew, etc.) and the source
is a standalone report, italicize the title like a book; do NOT
wrap it in quotes. CMOS 14.272.
28. PRESERVE DELIBERATE LOWERCASING of proper nouns. Examples cited in
the prompt: the journal "portal: Libraries and the Academy"
(deliberately lowercase "p"), authors bell hooks / danah boyd /
e e cummings, brand names iPhone / eBay. Headline-case
normalization must NOT touch these.
Rule 17 also extended (was newspaper/magazine only):
17. PERIODICAL NAMES — drop leading "The" from ANY periodical:
newspapers, magazines, AND scholarly journals. Write "Journal of
Academic Librarianship", not "The Journal of Academic
Librarianship"; "Library Quarterly", not "The Library Quarterly";
"American Archivist", not "The American Archivist". CMOS 14.191.
Rule explicitly excepts BOOK titles and report titles, which keep
their leading "The" (e.g., *The Library's Guide to Sexual and
Reproductive Health Information*).
Empirical impact, Reading_Disrepair (13 entries):
before: 3 errors (2 leading-The, 1 GAO format, 1 portal capitalized)
after: 0 errors
Empirical impact, HML Submission (134 entries):
before: ~21 errors (18 leading-The, 0 GAO in this draft, 3 portal)
after: ~1 error (a multi-author 2nd-author inversion bug newly
surfaced — separate fix)
Also adds scripts/analyze_draft_output.py — a triage helper that
diff-walks input and output, surfacing common failure patterns
(leading-The, doi: prefix, bare-hyphen ranges, gov-report quoted, and
intentional lowercasing stripped). False positives are tolerated since
it's a human-review tool, not a validator.
Exemplar corpus is unchanged. Re-baseline against 14 canary exemplars
under the new rules: scalar 1.000, canary 1.000 (no regressions).
User's real drafts under rough_drafts/ remain untracked — they are
in-progress academic work and don't belong in git history.
|
||
|
cmos dev
|
2bd1bdf0d4 |
cli: parallelize formatter calls with thread pool
reformat_draft now uses concurrent.futures.ThreadPoolExecutor with a default of 8 workers. OpenAI SDK calls are synchronous but network- bound, so threads release the GIL during I/O and give real speedup. ThreadPoolExecutor.map preserves input order regardless of completion order, so output is deterministic. Empirical: HML draft (134 entries) went from ~25 min serial to 1m55s with concurrency=12. ~12x speedup; further increases hit OpenAI rate limits. CLI gains a --concurrency flag (default 8) for tuning per draft size / rate limit headroom. concurrency=1 forces serial execution for debugging. New test asserts that order is preserved when concurrent calls finish out of input order (uses a sleep-by-index fake formatter). |
||
|
cmos dev
|
1b11a03a0b |
judge: LLM-as-judge triage for canary failures (advisory only)
Add harness/judge.py and scripts/triage.py. The judge reads the latest loop run from logs/runs.jsonl, finds canary failures (exact_match=False but fields/linter passed), and asks GPT-5 to classify each as "regression", "variant", or "unclear" with CMOS 18 section citations. CRITICAL: verdicts are ADVISORY ONLY. They are written to logs/triage.jsonl and never feed back into the loop scalar. Using the judge as ground truth would let the formatter-LLM optimize against a judge-LLM from the same model family, inviting shared-bias drift. Design: - harness/judge.py: Verdict dataclass, build_judge_prompt, parse_verdict (handles code fences and normalizes unknown labels to "unclear"), judge() with caller-injection seam matching cmos.formatter. Uses the same _is_reasoning_model branching to skip temperature for gpt-5/o*. Judge model is separately overridable via CMOS_JUDGE_MODEL env var (defaults to OPENAI_MODEL, which defaults to gpt-5). - scripts/triage.py: CLI that walks runs.jsonl, locates a target run (default: latest), filters canary failures, calls judge on each, appends a verdict record to logs/triage.jsonl. --dry-run available for offline testing. Exits 0 with a note when there are no failures. Tests: 6 new unit tests covering prompt building, JSON parsing (including code-fence stripping and unknown-label normalization), and caller injection. No real API calls in the test suite. Validated on iter 3's run (canary 0.286, 10 failures): - 8 correctly flagged as regressions, each with a cited CMOS section (14.72, 14.76, 14.128, 14.190, 14.206, 14.212, 14.267, ...). - 2 flagged as variants: "Kindle" vs "Kindle edition" (CMOS 14.159– 14.161 allows flexibility) and "The New Yorker" vs "New Yorker" (CMOS 14.191 — leading "The" is optional). These surface that the formatter's current rules 17 and 18 are stricter than CMOS strictly requires; documenting here but not acting on yet. |
||
|
cmos dev
|
fe5e071066 |
linter: expand to 21 rules covering all 8 source types (v0.3.0)
Close the defense-in-depth gap flagged after iter 6: chapter, magazine,
newspaper, social_media, podcast, and video exemplars had ZERO applicable
linter rules, so their 100% linter pass rate was trivially true. Every
exemplar now has 3-6 structural rules firing.
Bump LINTER_VERSION to v0.3.0. Per harness discipline, this invalidates
prior run logs' scores for cross-version comparison (they stay in the
log as history). The re-baseline run still scores scalar 1.000, canary
1.000 on all 14 exemplars — formatter was already producing output that
matches the new structural conventions, so tightening the linter didn't
surface any regressions.
New rules (12):
- chapter_book_title_italicized
- chapter_in_book_marker
- magazine_name_italicized
- newspaper_name_italicized
- periodical_comma_before_date (magazine + newspaper; not journal)
- social_media_post_quoted
- social_media_platform_comma_date
- podcast_series_italicized
- podcast_episode_quoted
- podcast_format_label
- video_title_quoted (in quotes, NOT italicized)
- video_format_label
Extended:
- article_title_quoted now covers journal, magazine, newspaper,
and accepts ?/! as terminal punctuation
Each rule has a positive + negative unit test with the CMOS/Purdue source
cited in the docstring. Tests: 48 → 78 (30 new).
Per-exemplar applicable-rule counts after this change:
book 4 journal 5-6 web 3
chapter 4 magazine 5 social 4
podcast 5 video 4 newspaper 5
|
||
|
cmos dev
|
ced9c73474 |
iter 3-6: prompt rules 13-26 to handle expanded corpus
Four-iteration sweep over the 14-exemplar corpus, driven by loop
failures at each step. Scalar progression:
iter 3 (v0 prompt, 14 exemplars) → 0.842
iter 4 (+rules 13-23) → 0.951
iter 5 (+rules 24-25) → 0.988
iter 6 (+rule 26, rule 12 refined) → 1.000 canary 1.000
New SYSTEM_PROMPT rules:
13. Drop chapter page range from bibliography (CMOS 18 change from 17).
14. Use full canonical publisher names — no "U of Chicago Press" etc.
15. Do not comma-invert non-Western family-first names (Liu Xinwu,
Murakami Haruki).
16. Author threshold: up to 6 listed; 7+ → first 3 + "et al." with full
given names when source provides them.
17. Expand abbreviated newspaper names (NYT → New York Times); drop
leading "The" from newspaper/magazine names; use comma between
italicized name and date.
18. E-book format: "Kindle." not "Kindle edition."
19. Database indicator: just the name, not "Accessed via X".
20. Social media: preserve original casing of post content; comma
between Platform and Date.
21. Podcast: series italicized, episode in quotes, preserve
Season/episode/duration.
22. Video / TED Talk: preserve venue, date, Video label, duration, URL.
23. URLs preserved verbatim including "www." subdomain when present.
24. CMOS 9.61 inclusive-number elision for page ranges: 1818–59 not
1818–1859; 101–8; 1100–1113; 1496–1504.
25. Preserve terminal punctuation (? !) inside titles.
26. Edition indicators abbreviated: "2nd ed." not "Second edition".
Also scoped rule 12 (date qualifier preservation) to web-page sources
only — podcasts and videos use bare dates, so "Released September 13,
2022" should emit as just "September 13, 2022".
Also refined three messy inputs (podcast, social media, video) to
include "www." in the URL, and the snyder messy input to explicitly
list 7 authors so the et-al threshold is unambiguous to the formatter.
|
||
|
cmos dev
|
abb75888c5 |
Expand exemplar corpus to 14 across 8 CMOS 18 source types
Add 11 new exemplars pulled verbatim from the CMOS official quick guide (chicagomanualofstyle.org/tools_citationguide/citation-guide-1.html): - book_two_authors_binder: Binder & Kidder (two-author book) - book_chapter_doyle: Doyle in Marks & Parkin (chapter in edited volume, no page range per CMOS 18) - book_translated_liu: Liu Xinwu, trans. Tiang (translated book, non- Western name not comma-inverted) - book_edition_borel: Borel, 2nd ed. via EBSCOhost (edition + database) - book_ebook_roy: Roy, Kindle format - journal_many_authors_snyder: 7 authors in PLOS ONE, exercises the CMOS 18 "first 3 + et al." threshold and article-ID page format - magazine_mead: New Yorker (tests leading-"The" drop) - newspaper_blum: NYT with URL (tests abbreviation expansion) - social_media_cmos_facebook: Facebook post with case preservation - podcast_ober: Pushkin podcast with season/episode/duration - video_cowan_ted: TED Talk with venue and duration Every exemplar is marked canary=true — we want byte-exact regression detection on all of them. Source citations and CMOS rule justifications are in the TOML doc comments. All 14 canonical outputs lint clean under LINTER_VERSION v0.2.0. The harness will score them in the following commit. |
||
|
cmos dev
|
1621d0d502 |
iter 1: preserve date qualifiers in web entries
First real loop iteration found that GPT-5 silently drops date qualifiers
("Effective", "Published", "Accessed", etc.) when reformatting web
sources. The field diff was satisfied because the canonical date field
did not include the qualifier, but the canary exact-match axis caught
the regression.
Two fixes, per the canary-upstream policy:
1. Tighten the web-page exemplar canonical: date field now includes the
"Effective" qualifier so the field diff will catch future regressions
without relying on the canary.
2. Add SYSTEM_PROMPT rule 12 instructing the formatter to preserve
semantic date qualifiers from the input.
After these fixes: scalar 1.000, canary exact-match 1.000 on all three
seed exemplars.
|
||
|
cmos dev
|
ee0bcd107e |
formatter: skip temperature override for reasoning models
GPT-5 and the o-series reject temperature=0 with a 400 BadRequestError — only the default (1) is supported for reasoning models. Add an _is_reasoning_model helper and pass temperature only when the model name does not start with gpt-5/o1/o3/o4. Determinism on reasoning models is a property of the architecture, not a parameter. Discovered on the first real LLM run against the seed exemplars. |
||
|
cmos dev
|
4cad38ef30 |
Scaffold CMOS 18 reformatter: harness, linter, parser, formatter, CLI
Initial phase-1 baseline of the karpathy/autoresearch-style loop. The formatter module is the inner-loop artifact; parser and linter are infra. The linter carries a LINTER_VERSION hash (v0.2.0) that will force a re-baseline on any rule change. Components: - harness/diff.py: case-sensitive field-level substring diff - harness/score.py: three-axis scoring (field, linter, canary exact) - src/cmos/linter.py: 9 CMOS 18 structural rules, each Purdue/CMOS cited - src/cmos/parser.py: locate ## Bibliography section, split entries - src/cmos/formatter.py: prompt + OpenAI call with caller injection - src/cmos/cli.py: cmos format path/to/draft.md - scripts/run_loop.py: loop runner with --fake mode for no-API runs - exemplars/: 3 canary seed exemplars (book, journal w/DOI, web), sourced from chicagomanualofstyle.org quick guide Tests: 48 passing. Fake-mode baseline scalar = 0.000 on the 3 seed exemplars (identity caller fails the linter on every rule). This is the floor the real GPT-5 formatter needs to improve from. |