v2 chunk 3: bug fixes from real-draft triage + polish

Six independent fixes addressing the Anti-Communist Formations of LIS
real-draft test findings (chunk 2c). Four are bug fixes for issues
that destroyed user data or wasted API calls; two are polish quality
improvements revisited from the deferred list.

src/cmos/note_formatter.py — empty input guard (fix #1)
  Empty / whitespace-only input now short-circuits the API entirely
  and returns the input verbatim. Surfaced by note [66] in the
  Anti-Communist draft, where GPT-5 broke character on empty input
  and returned a conversational meta-reply ("Please paste the
  citation entry...") that then got substituted into the document.
  Whitespace preserved so cli.reformat_notes is byte-exact on empty
  notes. v2 only.

src/cmos/note_formatter.py — deprecated Latin guard (fix #2)
  New Python guard catches the full CMOS-18-deprecated Latin
  citation set (ibid, idem, id., op. cit., loc. cit.) and returns
  input verbatim before the API call. Rule 10 in the prompt also
  rewritten: explicitly says "return verbatim" instead of the old
  "return cleanest possible full-form note", which the model
  interpreted as "return empty when there's no information",
  silently destroying the marker. Surfaced by note [61] = "Ibid.".
  v2 only.

src/cmos/runtime_validator.py — shortened-form carve-out (fix #3)
  The "must contain italics" check is now skipped when the
  candidate looks like a CMOS shortened-form note (author last
  name, optional page, no italic content). Surfaced by ~30 of 107
  notes in the Anti-Communist draft that were correctly returned
  as shortened forms ("Rosen, 7.", "Mitchell, 197.") but rejected
  by the validator's strict italic check, firing the full retry
  budget on valid output (~60 wasted API calls per run). The
  carve-out is conservative: name-token regex + terminal period;
  doesn't match unstructured prose. Shared infrastructure — affects
  v1 and v2, no-op in v1's normal workflow because v1 outputs
  always have italics.

src/cmos/note_formatter.py — substantive prose pass-through (fix #4)
  Long discursive prose with no citation skeleton markers now
  short-circuits the API and returns the input verbatim. Surfaced
  by notes [8] (689 chars), [9] (893 chars), [75] (163 chars) in
  the Anti-Communist draft — substantive notes that the formatter
  was extracting a single citation from and silently discarding
  the surrounding commentary. CMOS 14.39 explicitly allows
  substantive notes; preserving them is the user's explicit policy
  ("preserve all free text as long as that does not break other
  formatting"). Detection: length > 150 chars AND no citation
  skeleton markers (parenthesized year, URL, vol./no./pp., DOI,
  terminal page or terminal year). Conservative — does not
  false-positive on legitimate first-occurrence notes. v2 only.

src/cmos/formatter.py + note_formatter.py — month/season preservation
  (polish #5)
  Both v1 rule 10 (journal article format) and v2 rule 5 (journal
  article note form) now explicitly instruct the model to preserve
  (Month YEAR) and (Season YEAR) parentheticals when the source
  provides them. Surfaced by entries in both Reading_Disrepair and
  Anti-Communist where (Spring, 1993) and August 1952 were silently
  collapsed to (1993) and 1952. Both forms are valid CMOS but
  month/season is more informative when the source has it. Affects
  v1 and v2 in mirror.

src/cmos/note_formatter.py — government documents rule 19 (polish #7)
  v2 now has an explicit rule mirroring v1's rule 27: government
  bodies, institutional reports, and similar standalone documents
  get italicized titles and book-form treatment, NOT quoted-article
  treatment. Includes the Anti-Communist Senate of California
  Tenth Report and a hypothetical GAO example. Implicit handling
  worked in chunk 2c, but explicit rule provides regression
  protection. v2 only.

Tests: +14 net new (across test_note_formatter.py, test_formatter.py,
test_runtime_validator.py). Total suite 150/150 (was 136 before this
chunk). All v1 tests still passing.

Path B status: formatter.py and runtime_validator.py edits cross
the v1/v2 boundary, but only by user-explicit approval per the
relevant fix discussions. The shared validator was always shared;
the v1 formatter rule 10 mirror is a small additive edit that
doesn't affect the v1 prompt's existing behavior on its existing
inputs.
This commit is contained in:
Mark Eaton
2026-04-11 20:06:59 -04:00
parent 45f51c6341
commit 553240c153
6 changed files with 673 additions and 21 deletions
+72
View File
@@ -50,6 +50,78 @@ def test_missing_italics_anywhere_fails():
assert any("italic" in f.lower() for f in result.failures)
# --- Shortened-form note carve-out (chunk 3 fix #3, Option D) -------------
def test_shortened_form_author_only_passes_without_italics():
"""CMOS shortened note form for an author-only reference (no page,
no title) is just the author's last name with a period: 'Ettarh.',
'Murch.', 'O'Mara.'. These contain no italics by definition.
Without a carve-out, the validator's "must have italics" rule
rejects them and the formatter's retry loop fires its full budget
on valid output. Surfaced by the Anti-Communist Formations of LIS
real-draft run; ~30 of 107 notes were correct shortened-forms
that wasted ~60 API calls retrying.
The carve-out: if a candidate has no italics AND looks like a
shortened-form note (short, name-token shape, optional page,
terminal period), the missing-italics check is skipped.
"""
for candidate in ["Ettarh.", "Murch.", "O'Mara.", "Burden-Stelly."]:
result = validate(candidate)
assert result.passed is True, (
f"shortened-form author-only note {candidate!r} should pass "
f"validation; got failures {result.failures}"
)
def test_shortened_form_author_plus_page_passes_without_italics():
"""The other common CMOS shortened form is author last name + page:
'Rosen, 7.', 'Mitchell, 197.', 'CBS, 47.'. Same carve-out applies."""
for candidate in [
"Rosen, 7.",
"Mitchell, 197.",
"CBS, 47.",
"Seybold, 282.",
"Lawrence Powell, 45.", # multi-word name
"Mitchell, 137-39.", # page range with hyphen
]:
result = validate(candidate)
assert result.passed is True, (
f"shortened-form note {candidate!r} should pass validation; "
f"got failures {result.failures}"
)
def test_long_no_italic_output_still_fails():
"""The carve-out must NOT be a free pass for any short string.
A long output with no italics is still likely a malformed
full-form citation (model dropped the italic title), and the
retry loop should still fire on it.
"""
candidate = (
"Steve Batterson, The Prosecution of Professor Chandler Davis: "
"McCarthyism, Communism, and the Myth of Academic Freedom (NYU Press, 2023)."
)
result = validate(candidate)
assert result.passed is False
assert any("italic" in f.lower() for f in result.failures)
def test_short_but_unstructured_no_italic_output_still_fails():
"""A short but non-name-shaped string (e.g., a stray sentence
fragment with no name structure) should still fail. The carve-out
is NOT just "short and ends with period" — it requires the
candidate to look like a shortened-form note specifically.
"""
candidate = "this is some prose without italics."
result = validate(candidate)
assert result.passed is False, (
"lowercase prose fragment must not be accepted as a shortened-form "
"note; the carve-out should require name-token shape"
)
def test_unbalanced_italic_markers_fails():
candidate = "Yu, Charles. *Interior Chinatown. Pantheon Books, 2020."
result = validate(candidate)