v2 chunk 3: bug fixes from real-draft triage + polish

Six independent fixes addressing the Anti-Communist Formations of LIS
real-draft test findings (chunk 2c). Four are bug fixes for issues
that destroyed user data or wasted API calls; two are polish quality
improvements revisited from the deferred list.

src/cmos/note_formatter.py — empty input guard (fix #1)
  Empty / whitespace-only input now short-circuits the API entirely
  and returns the input verbatim. Surfaced by note [66] in the
  Anti-Communist draft, where GPT-5 broke character on empty input
  and returned a conversational meta-reply ("Please paste the
  citation entry...") that then got substituted into the document.
  Whitespace preserved so cli.reformat_notes is byte-exact on empty
  notes. v2 only.

src/cmos/note_formatter.py — deprecated Latin guard (fix #2)
  New Python guard catches the full CMOS-18-deprecated Latin
  citation set (ibid, idem, id., op. cit., loc. cit.) and returns
  input verbatim before the API call. Rule 10 in the prompt also
  rewritten: explicitly says "return verbatim" instead of the old
  "return cleanest possible full-form note", which the model
  interpreted as "return empty when there's no information",
  silently destroying the marker. Surfaced by note [61] = "Ibid.".
  v2 only.

src/cmos/runtime_validator.py — shortened-form carve-out (fix #3)
  The "must contain italics" check is now skipped when the
  candidate looks like a CMOS shortened-form note (author last
  name, optional page, no italic content). Surfaced by ~30 of 107
  notes in the Anti-Communist draft that were correctly returned
  as shortened forms ("Rosen, 7.", "Mitchell, 197.") but rejected
  by the validator's strict italic check, firing the full retry
  budget on valid output (~60 wasted API calls per run). The
  carve-out is conservative: name-token regex + terminal period;
  doesn't match unstructured prose. Shared infrastructure — affects
  v1 and v2, no-op in v1's normal workflow because v1 outputs
  always have italics.

src/cmos/note_formatter.py — substantive prose pass-through (fix #4)
  Long discursive prose with no citation skeleton markers now
  short-circuits the API and returns the input verbatim. Surfaced
  by notes [8] (689 chars), [9] (893 chars), [75] (163 chars) in
  the Anti-Communist draft — substantive notes that the formatter
  was extracting a single citation from and silently discarding
  the surrounding commentary. CMOS 14.39 explicitly allows
  substantive notes; preserving them is the user's explicit policy
  ("preserve all free text as long as that does not break other
  formatting"). Detection: length > 150 chars AND no citation
  skeleton markers (parenthesized year, URL, vol./no./pp., DOI,
  terminal page or terminal year). Conservative — does not
  false-positive on legitimate first-occurrence notes. v2 only.

src/cmos/formatter.py + note_formatter.py — month/season preservation
  (polish #5)
  Both v1 rule 10 (journal article format) and v2 rule 5 (journal
  article note form) now explicitly instruct the model to preserve
  (Month YEAR) and (Season YEAR) parentheticals when the source
  provides them. Surfaced by entries in both Reading_Disrepair and
  Anti-Communist where (Spring, 1993) and August 1952 were silently
  collapsed to (1993) and 1952. Both forms are valid CMOS but
  month/season is more informative when the source has it. Affects
  v1 and v2 in mirror.

src/cmos/note_formatter.py — government documents rule 19 (polish #7)
  v2 now has an explicit rule mirroring v1's rule 27: government
  bodies, institutional reports, and similar standalone documents
  get italicized titles and book-form treatment, NOT quoted-article
  treatment. Includes the Anti-Communist Senate of California
  Tenth Report and a hypothetical GAO example. Implicit handling
  worked in chunk 2c, but explicit rule provides regression
  protection. v2 only.

Tests: +14 net new (across test_note_formatter.py, test_formatter.py,
test_runtime_validator.py). Total suite 150/150 (was 136 before this
chunk). All v1 tests still passing.

Path B status: formatter.py and runtime_validator.py edits cross
the v1/v2 boundary, but only by user-explicit approval per the
relevant fix discussions. The shared validator was always shared;
the v1 formatter rule 10 mirror is a small additive edit that
doesn't affect the v1 prompt's existing behavior on its existing
inputs.

src/cmos/formatter.py

@@ -79,6 +79,18 @@ Rules you must follow (CMOS 18th edition specifically):
 10. Journal article format: Lastname, First. "Article Title." *Journal Title*
     VOLUME, no. ISSUE (YEAR): PAGES. DOI-URL.
 
+    PRESERVE MONTH OR SEASON information when the source provides it.
+    Some scholarly journals organize their issues by month or season
+    rather than (or in addition to) volume/issue numbers, and CMOS
+    accepts both `(YEAR)` and `(Month YEAR)` / `(Season YEAR)` forms
+    in the parenthetical. If the source has a month name (January
+    through December) or a season name (Spring, Summer, Fall, Autumn,
+    Winter), preserve it as `(Month YEAR)` or `(Season YEAR)` — note
+    NO comma between the month/season and the year. Do NOT collapse
+    `(Spring 1993)` to `(1993)` or `August 1952` to `1952` when the
+    source had the month/season; that loses information unnecessarily.
+    If the source has only a year, just use `(YEAR)` as before.
+
 11. Web page format: Author-or-Organization. "Page Title." Site Name.
     Effective/Published Date. URL.
 

src/cmos/note_formatter.py

@@ -29,12 +29,96 @@ yet implemented.
 from __future__ import annotations
 
 import os
+import re
 from typing import Callable
 
 from dotenv import load_dotenv
 
 from cmos.runtime_validator import validate
 
+# Pure-token CMOS-18-deprecated Latin citation abbreviations. When the
+# input matches this exactly (after .strip()), the formatter
+# short-circuits and returns the input verbatim instead of calling the
+# API. The model has no information to expand these into a full
+# citation (shortened-form generation is out of scope), so the only
+# correct behavior is to preserve the marker for human review.
+#
+# Detection is intentionally STRICT — input must match the deprecated
+# token exactly. Inputs like "Ibid., 47" (token + page number) are
+# left for the LLM to handle per prompt rule 10, which says return
+# verbatim in those cases too. This split avoids over-matching real
+# citations that happen to mention "ibid" inside a longer string.
+_DEPRECATED_LATIN_RE = re.compile(
+    r"^(?:"
+    r"ibid\.?"            # ibid, ibid.
+    r"|idem\.?"           # idem, idem.
+    r"|id\."              # id. (period required to avoid matching "id" in unrelated contexts)
+    r"|op\.?\s*cit\.?"    # op cit, op. cit, op.cit, op cit., etc.
+    r"|loc\.?\s*cit\.?"   # loc cit, loc. cit, loc.cit, etc.
+    r")$",
+    re.IGNORECASE,
+)
+
+# Substantive note pass-through (chunk 3 fix #4, Option A). When the
+# input is long discursive prose with no obvious citation skeleton, the
+# formatter returns it VERBATIM without calling the API. CMOS 14.39
+# explicitly allows substantive notes (commentary, qualifications,
+# cross-references), and the previous behavior extracted whatever
+# citation it could find and silently discarded the surrounding prose.
+# Surfaced by notes [8], [9], and [75] in the Anti-Communist Formations
+# of LIS draft.
+#
+# Detection is conservative: input must be longer than the threshold
+# AND must NOT contain ANY of the citation skeleton markers below.
+# Real first-occurrence citations virtually always have at least one
+# skeleton marker (terminal year, terminal page reference, URL, vol.,
+# no., pp., DOI, parenthesized year). Substantive prose typically has
+# none of these. The combination of length-floor + marker-absence
+# catches the egregious cases without false-positiving on legitimate
+# citations that happen to be long.
+#
+# The user's explicit policy (chunk 3 fix #4): preserve ALL free text
+# as long as that does not break other formatting. Erring on the side
+# of preservation is intentional.
+_SUBSTANTIVE_PROSE_LENGTH_THRESHOLD = 150
+
+_CITATION_SKELETON_MARKERS_RE = re.compile(
+    r"(?:"
+    r"\([^)]*\d{4}[^)]*\)"               # parenthesized year — (2024), (NYU Press, 2023), (March 2024)
+    r"|https?://"                        # URL
+    r"|\bvol\.\s*\d+"                    # volume number — vol. 7
+    r"|\bno\.\s*\d+"                     # issue number — no. 4
+    r"|\bpp?\.\s*\d+"                    # page reference — p. 45, pp. 45-67
+    r"|\bdoi[:.\s]"                      # DOI marker — doi:, doi., doi space
+    r"|,\s*\d+(?:[-\u2013]\d+)?\.\s*$"   # terminal page reference — ", 145." or ", 145-67."
+    r"|\b\d{4}\.?\s*$"                   # year at end of string — "2023" or "2023."
+    r")",
+    re.IGNORECASE,
+)
+
+
+def _looks_like_substantive_prose(text: str) -> bool:
+    """True if ``text`` looks like substantive note prose, not a citation.
+
+    Conservative detection: only matches input that is BOTH long AND
+    lacks any obvious citation skeleton markers. Designed to catch
+    long discursive notes (689+ chars in the Anti-Communist draft)
+    while preserving legitimate first-occurrence notes — which all
+    have at least one skeleton marker even when long. The shorter
+    [75]-style cases (~163 chars, no markers) are also caught because
+    the threshold is set well below the typical citation length floor
+    when markers are present.
+
+    See ``_SUBSTANTIVE_PROSE_LENGTH_THRESHOLD`` and
+    ``_CITATION_SKELETON_MARKERS_RE`` above for the detection details.
+    """
+    stripped = text.strip()
+    if len(stripped) < _SUBSTANTIVE_PROSE_LENGTH_THRESHOLD:
+        return False
+    if _CITATION_SKELETON_MARKERS_RE.search(stripped):
+        return False
+    return True
+
 # Default model. Override with OPENAI_MODEL=... in your .env.
 MODEL = os.environ.get("OPENAI_MODEL", "gpt-5")
 
@@ -96,6 +180,19 @@ Rules you must follow (CMOS 18th edition specifically, NOTE form):
    the quotes, and the colon between (YEAR) and the page is NOT
    preceded by a space.
 
+   PRESERVE MONTH OR SEASON information when the source provides
+   it. Some scholarly journals organize their issues by month or
+   season rather than (or in addition to) volume/issue numbers, and
+   CMOS accepts both `(YEAR)` and `(Month YEAR)` / `(Season YEAR)`
+   forms in the parenthetical. If the source has a month name
+   (January through December) or a season name (Spring, Summer,
+   Fall, Autumn, Winter), preserve it as `(Month YEAR)` or
+   `(Season YEAR)` — note NO comma between the month/season and
+   the year. Do NOT collapse `(Spring 1993)` to `(1993)` or
+   `August 1952` to `1952` when the source had the month/season;
+   that loses information unnecessarily. If the source has only a
+   year, just use `(YEAR)` as before.
+
 6. CHAPTER IN EDITED BOOK NOTE FORM:
    Author, "Chapter Title," in *Book Title*, ed. Editor Names
    (Publisher, Year), specific-page.
@@ -119,12 +216,34 @@ Rules you must follow (CMOS 18th edition specifically, NOTE form):
    as a full URL: https://doi.org/10.xxxx/yyyy (no "doi:" prefix,
    no bare DOI).
 
-10. NO IBID. CMOS 18 deprecates "Ibid." entirely. Do not produce it
+10. NO DEPRECATED LATIN ABBREVIATIONS. CMOS 18 deprecates the entire
-    under any circumstances, even if the input contains it. If the
+    family of older Latin citation abbreviations in favor of
-    input is an "Ibid." reference, this iteration of the formatter
+    shortened-form notes:
-    cannot resolve it (shortened-form generation is out of scope).
+    - "ibid." / "ibid" (ibidem — same as previous)
-    Return the cleanest possible full-form note from whatever
+    - "idem" (the same author as previous)
-    information the messy input contains.
+    - "id." (also "the same")
+    - "op. cit." (opere citato — in the work cited)
+    - "loc. cit." (loco citato — in the place cited)
+    Do NOT produce ANY of these abbreviations in your output, even
+    if the input contains them.
+
+    If the input contains a deprecated abbreviation that this
+    iteration cannot resolve (because shortened-form generation is
+    out of scope and the formatter has no access to the previously-
+    cited source), return the input VERBATIM. Do NOT return an empty
+    string. Do NOT invent missing context (no fabricated author
+    names, titles, or page numbers). Do NOT silently delete the
+    marker. Returning the input verbatim signals to the user that
+    this entry needs manual resolution against the surrounding notes
+    — losing the marker silently is worse than any other failure
+    mode here.
+
+    The pure-token cases ("Ibid.", "idem", "op. cit.", etc. on their
+    own with no other content) are caught deterministically by a
+    Python-side guard before the API call ever fires; this rule
+    covers the cases where a deprecated abbreviation appears inline
+    with other content (e.g., "Ibid., 47" or "Smith, op. cit., 12").
+    In all such cases: return the input verbatim.
 
 11. PRESERVE TERMINAL PUNCTUATION inside titles. If a title ends in
     "?" or "!", keep that punctuation inside the closing quote and
@@ -186,6 +305,37 @@ Rules you must follow (CMOS 18th edition specifically, NOTE form):
     "MLA") stay in that form. When in doubt, prefer the longer
     form over the abbreviation.
 
+19. GOVERNMENT DOCUMENTS AND INSTITUTIONAL REPORTS — CMOS 14.272.
+    When the author is a government body, agency, institution, or
+    NGO (e.g., "Government Accountability Office", "U.S. Department
+    of Education", "Senate of the State of California", "World Health
+    Organization", "Pew Research Center", "American Library
+    Association") and the source is a STANDALONE report or document
+    (not an article in a periodical), treat the report like a BOOK:
+    italicize the title with Markdown *...*. Do NOT wrap the title in
+    quotes — that is the article/chapter form, which is wrong for a
+    standalone institutional report.
+
+    Note form:
+    Author-Body, *Title of Report*, Report Number (Publisher, Date),
+    specific-page, URL.
+
+    The Report Number (e.g., "GAO-26-107262") is an identifier and
+    sits after the italicized title with comma separation. The
+    publisher is often the same body as the author (institutional
+    self-publication is common); include both even when they
+    duplicate. If there is no separate publisher and only the author-
+    body is named, that is the publisher.
+
+    Examples:
+    - Government Accountability Office, *Public Libraries: Many
+      Buildings Are Reported to Be in Poor Condition*, GAO-26-107262
+      (Government Accountability Office, December 18, 2025), 4,
+      https://files.gao.gov/reports/GAO-26-107262/index.html.
+    - Senate of the State of California, *Tenth Report: Senate
+      Investigating Committee on Education, the 1951 Hearings*,
+      66, https://hdl.handle.net/2027/mdp.39015070361673.
+
 Output ONLY the single reformatted note. No preamble, no explanation,
 no leading number, no code fences.
 """
@@ -243,6 +393,41 @@ def format_note_entry(
     Pass a ``caller`` shim to avoid the real API (used by tests and the
     harness when running with a fake formatter for loop smoke tests).
 
+    Empty input short-circuit: if ``messy_entry`` is empty or contains
+    only whitespace, the function returns it VERBATIM without calling
+    the API. Surfaced by the Anti-Communist Formations of LIS draft
+    (note [66] is an empty placeholder line) — without the guard, GPT-5
+    occasionally breaks character on empty input and returns a
+    conversational meta-reply asking for content. The whitespace is
+    preserved verbatim so reassembly in ``cmos.cli.reformat_notes`` is
+    byte-exact on empty notes; the source's spacing is part of the
+    input contract, not noise to normalize. v2 only — v1's
+    ``format_bibliography_entry`` does not currently have this guard.
+
+    Deprecated-Latin short-circuit: if ``messy_entry.strip()`` exactly
+    matches one of the CMOS-18-deprecated Latin citation abbreviations
+    (ibid, idem, id., op. cit., loc. cit.), the function returns the
+    input VERBATIM without calling the API. This iteration cannot
+    resolve shortened-form references (no access to the previously-
+    cited source), and returning empty would silently delete the
+    marker the user needs for manual resolution. Surfaced by note [61]
+    in the Anti-Communist Formations of LIS draft. Detection is
+    strict (token-only); inputs with extra content like "Ibid., 47"
+    fall through to the LLM, which is governed by prompt rule 10.
+    v2 only.
+
+    Substantive prose pass-through: if ``messy_entry`` looks like a
+    substantive note (long discursive prose with no obvious citation
+    skeleton markers), the function returns the input VERBATIM without
+    calling the API. CMOS 14.39 explicitly allows substantive notes,
+    and the previous behavior extracted whatever citation it could
+    find and silently discarded the surrounding prose. Surfaced by
+    notes [8], [9], and [75] in the Anti-Communist Formations of LIS
+    draft. Detection is conservative (length-floor + skeleton-marker
+    absence) — see ``_looks_like_substantive_prose``. The user's
+    explicit policy: preserve ALL free text as long as that does not
+    break other formatting. v2 only.
+
     Reasoning models (gpt-5, o-series) are nondeterministic. The candidate
     is checked against ``cmos.runtime_validator`` after each call; if it
     fails any structural sanity check (missing terminal period, dropped
@@ -258,6 +443,13 @@ def format_note_entry(
     don't raise — the caller still gets something usable, and the
     failure will surface via the v2 scoring linter or human review).
     """
+    stripped = messy_entry.strip()
+    if not stripped:
+        return messy_entry
+    if _DEPRECATED_LATIN_RE.match(stripped):
+        return messy_entry
+    if _looks_like_substantive_prose(messy_entry):
+        return messy_entry
     call = caller or _openai_caller
     user_message = build_user_message(messy_entry)
     last: str = ""

src/cmos/runtime_validator.py

@@ -1,34 +1,57 @@
 """Runtime structural validator — type-independent sanity checks on a single
-formatted bibliography entry.
+formatted bibliography or note entry.
 
-This is the gate used by ``cmos.formatter.format_bibliography_entry`` to
+This is the gate used by ``cmos.formatter.format_bibliography_entry`` and
-decide whether a candidate output is well-formed enough to return, or
+``cmos.note_formatter.format_note_entry`` to decide whether a candidate
-whether to retry the API call. It runs WITHOUT an Exemplar, because at
+output is well-formed enough to return, or whether to retry the API call.
-runtime we have only the messy input and the candidate output — no
+It runs WITHOUT an Exemplar, because at runtime we have only the messy
-canonical fields, no source-type metadata.
+input and the candidate output — no canonical fields, no source-type
+metadata.
 
 Because it knows neither the source type nor the canonical fields, it
 deliberately checks only structural properties that hold for ALL CMOS 18
-bibliography entries:
+bibliography entries (and CMOS 18 first-occurrence note entries):
 
 - Ends with a period.
 - Contains no "Ibid." (deprecated in CMOS 18).
 - Contains at least one italic span (``*...*``). This catches one of
   the most common nondeterministic regressions: GPT-5 occasionally
   drops italic markers around a journal/book/magazine title. Almost
-  every CMOS bibliography entry italicizes SOMETHING (book title,
+  every CMOS full citation italicizes SOMETHING (book title, journal
-  journal name, magazine name, podcast series, report title), so an
+  name, magazine name, podcast series, report title), so a full
-  entry with no italics at all is highly likely to be malformed.
+  citation with no italics at all is highly likely to be malformed.
 - Italic markers (``*``) are balanced (even count).
 - Straight double quotes (``"``) are balanced (even count).
 
-This is intentionally weaker than ``cmos.linter`` (the scoring linter
+**Shortened-form note carve-out (chunk 3 fix #3, Option D):** the
-that needs an Exemplar). The point is to be a fast, deterministic
+"must contain italics" check is SKIPPED when the candidate looks like
-guardrail at runtime, not to validate every CMOS rule.
+a CMOS shortened-form note — author last name (or "Lastname, page")
+with no title and no italic content. Without this carve-out, the
+validator rejects valid shortened forms ("Rosen, 7.", "Ettarh.",
+"Mitchell, 197.") and the formatter's retry loop fires its full budget
+on correct output. Surfaced by the Anti-Communist Formations of LIS
+real-draft run where ~30 of 107 notes were valid shortened forms that
+wasted ~60 API calls between them.
+
+The carve-out is conservative: it only matches one or two name-token
+forms (capitalized, allowing apostrophes and hyphens for names like
+"O'Mara" and "Burden-Stelly") optionally followed by a comma and a
+page number / page range, ending in a period. Anything more complex
+(quoted short titles, italicized short titles, longer prose) still
+falls under the strict "must have italics" check. The goal is to
+catch the most common shortened-form cases without false positives
+on actual full-form citations that happen to be short.
+
+This validator is intentionally weaker than ``cmos.linter`` (the
+scoring linter that needs an Exemplar). The point is to be a fast,
+deterministic guardrail at runtime, not to validate every CMOS rule.
+Shared between v1 (bibliography formatter) and v2 (note formatter)
+because both use the same retry loop pattern. Edits affect both.
 """
 
 from __future__ import annotations
 
+import re
 from dataclasses import dataclass, field
 
 
@@ -41,6 +64,44 @@ class ValidationResult:
         return not self.failures
 
 
+# Shortened-form note pattern. Matches the two most common CMOS shortened
+# forms that legitimately have no italics:
+#   - Author last name only:        "Ettarh.", "Murch.", "O'Mara.", "Burden-Stelly."
+#   - Author last name + page:      "Rosen, 7.", "Mitchell, 197.", "CBS, 47."
+#                                   "Lawrence Powell, 45." (multi-word name)
+#                                   "Mitchell, 137-39." (page range)
+#
+# Each name token must start with an uppercase letter (rejects "this is some
+# prose without italics."). Apostrophes and hyphens are allowed inside name
+# tokens for names like "O'Mara" and "Burden-Stelly". A multi-word name is
+# space-separated capitalized tokens. The optional page is a comma followed
+# by digits, optionally with an ASCII or en-dash range. The whole thing
+# ends with a required period.
+#
+# Does NOT match shortened forms with embedded short titles (quoted or
+# italicized) — those are harder to disambiguate from real first-occurrence
+# notes and are left for the strict italics check / future iteration.
+_SHORTENED_FORM_RE = re.compile(
+    r"^"
+    r"[A-Z][\w'\-]*"               # First name token (Lastname, possibly hyphenated/apostrophe)
+    r"(?:\s+[A-Z][\w'\-]*)*"       # Optional additional name tokens
+    r"(?:,\s*\d+(?:[-\u2013]\d+)?)?"  # Optional ", page" or ", page-range" / page–range
+    r"\.$"                         # Required terminal period
+)
+
+
+def _looks_like_shortened_form(text: str) -> bool:
+    """True if ``text`` matches the conservative shortened-form pattern.
+
+    Used by ``validate`` to skip the "must have italics" check on
+    legitimate shortened-form notes that have no italic content. Only
+    catches author-only and author-plus-page shapes; more complex
+    shortened forms (with quoted or italicized short titles) are not
+    matched here and remain subject to the strict italic check.
+    """
+    return _SHORTENED_FORM_RE.match(text.strip()) is not None
+
+
 def validate(candidate: str) -> ValidationResult:
     result = ValidationResult()
     text = candidate.rstrip()
@@ -51,9 +112,11 @@ def validate(candidate: str) -> ValidationResult:
     if "ibid" in text.lower():
         result.failures.append("contains 'Ibid.' (deprecated in CMOS 18)")
 
-    if "*" not in text:
+    # Strict italic check, with a carve-out for legitimate shortened-form
+    # notes that have no italics by definition. See module docstring.
+    if "*" not in text and not _looks_like_shortened_form(text):
         result.failures.append(
-            "no italic span found (CMOS bibliography entries usually italicize "
+            "no italic span found (CMOS full citations usually italicize "
             "a book/journal/magazine/series/report title)"
         )
 

tests/test_formatter.py

@@ -47,6 +47,22 @@ def test_system_prompt_mentions_italic_markers():
     assert "*" in SYSTEM_PROMPT and "italic" in SYSTEM_PROMPT.lower()
 
 
+def test_system_prompt_mentions_month_or_season_preservation():
+    """Polish fix #5: when the source provides month or season information
+    in a scholarly journal date, the formatter should preserve it in CMOS
+    form `(Month YEAR)` or `(Season YEAR)` rather than collapsing to
+    year-only. Surfaced by Reading_Disrepair (v1) and Anti-Communist (v2)
+    real-draft runs where dates like `(Spring, 1993)` and `August 1952`
+    were silently collapsed to `(1993)` and `1952`. Both forms are valid
+    CMOS but month/season is more informative when the source has it.
+    """
+    lower = SYSTEM_PROMPT.lower()
+    assert "month" in lower
+    assert "season" in lower
+    # And some indication of preservation, not collapse.
+    assert "preserve" in lower
+
+
 def test_build_user_message_contains_the_messy_input():
     msg = build_user_message("yu, charles. interior chinatown. 2020")
     assert "yu, charles. interior chinatown. 2020" in msg

tests/test_note_formatter.py

@@ -53,6 +53,33 @@ def test_system_prompt_mentions_no_ibid():
     assert "Ibid" in SYSTEM_PROMPT_NOTES or "ibid" in SYSTEM_PROMPT_NOTES.lower()
 
 
+def test_system_prompt_mentions_broader_deprecated_latin_set():
+    """Rule 10 was rewritten in fix #2 to cover the broad CMOS 18
+    deprecated set, not just Ibid. The prompt must mention idem,
+    op. cit., loc. cit. so the LLM knows to refuse all of them
+    consistently — the Python guard catches pure-token cases, but
+    the prompt must cover the cases where a deprecated abbreviation
+    appears alongside other content (e.g. "Ibid., 47")."""
+    lower = SYSTEM_PROMPT_NOTES.lower()
+    assert "idem" in lower
+    assert "op. cit" in lower or "op cit" in lower
+    assert "loc. cit" in lower or "loc cit" in lower
+
+
+def test_system_prompt_rule_10_says_return_verbatim_not_empty():
+    """Rule 10 must explicitly say "return verbatim" (or equivalent)
+    when the input contains a deprecated abbreviation the formatter
+    cannot resolve. The previous wording said "return the cleanest
+    possible full-form note from whatever information the messy input
+    contains" which the model interpreted as "return empty when there's
+    no information" — destroying the marker the user needs for review.
+    """
+    lower = SYSTEM_PROMPT_NOTES.lower()
+    assert "verbatim" in lower
+    # And the prompt should explicitly forbid the empty-string output.
+    assert "do not return an empty" in lower or "not return empty" in lower
+
+
 def test_system_prompt_mentions_publisher_full_form():
     """Surfaced by chunk 1 smoke test on chapter_first_doyle: the formatter
     preserved 'U of Chicago Press' from the messy input instead of
@@ -72,6 +99,35 @@ def test_system_prompt_mentions_publisher_full_form():
     )
 
 
+def test_system_prompt_mentions_month_or_season_preservation():
+    """Polish fix #5 (v2 mirror): preserve month/season info in scholarly
+    journal dates rather than collapsing to year-only. Mirror of the v1
+    test in tests/test_formatter.py — both prompts get the same rule.
+    Surfaced by entries [19] and [54] in the Anti-Communist Formations
+    of LIS draft where (Spring, 1993) and August 1952 were collapsed.
+    """
+    lower = SYSTEM_PROMPT_NOTES.lower()
+    assert "month" in lower
+    assert "season" in lower
+    assert "preserve" in lower
+
+
+def test_system_prompt_mentions_government_reports_rule():
+    """Polish fix #7: v2 should explicitly handle government documents
+    and institutional reports the same way v1's rule 27 does — italicize
+    the title like a book, do NOT wrap in quotes. Implicit handling
+    worked in the chunk 2c real-draft test (entries [11] and [83]) but
+    explicit rule provides regression protection if a future prompt
+    iteration loses this implicit knowledge.
+    """
+    lower = SYSTEM_PROMPT_NOTES.lower()
+    assert "government" in lower or "institutional" in lower
+    assert "report" in lower
+    # And the key behavior: treat like a book (italicize title), not a
+    # quoted article.
+    assert "italic" in lower  # already in the prompt for general italics
+
+
 def test_system_prompt_mentions_ed_invariant_for_multiple_editors():
     """Surfaced by chunk 1 smoke test on chapter_first_doyle: the formatter
     pluralized 'ed.' to 'eds.' when there were two editors. CMOS NB form
@@ -147,3 +203,244 @@ def test_note_formatter_returns_last_attempt_after_max_retries():
     output = format_note_entry("messy", caller=always_bad, max_retries=2)
     assert len(attempts) == 3  # 1 initial + 2 retries
     assert "*" not in output
+
+
+def test_format_note_entry_short_circuits_deprecated_latin_abbreviations():
+    """When the input is ONLY a CMOS-18-deprecated Latin citation
+    abbreviation (ibid, idem, id., op. cit., loc. cit.), the formatter
+    must short-circuit the API and return the input verbatim. Surfaced
+    by note [61] in the Anti-Communist Formations of LIS draft, where
+    the formatter was given "Ibid." and (after retrying through the
+    runtime validator) returned an empty string — silently destroying
+    the marker that the user needs to find later for manual resolution.
+
+    The deprecated set per CMOS 18:
+        ibid, ibid., idem, id., op. cit., loc. cit.
+    plus common spelling/spacing variants (Op.Cit., loc cit, etc.).
+
+    The guard MUST be deterministic (Python-side, not LLM-side) and
+    MUST NOT call the API. Cost guarantee enforced via counting caller.
+    The whitespace of the original input is preserved verbatim so
+    reassembly in cmos.cli.reformat_notes is byte-exact.
+    """
+    call_count = 0
+
+    def counting_caller(system: str, user: str) -> str:
+        nonlocal call_count
+        call_count += 1
+        return "should not be called"
+
+    deprecated_inputs = [
+        "Ibid.",
+        "ibid",
+        "IBID.",
+        "Idem",
+        "idem.",
+        "id.",
+        "Id.",
+        "op. cit.",
+        "Op. Cit.",
+        "op.cit.",
+        "Op Cit",
+        "loc. cit.",
+        "Loc. Cit.",
+        "loc.cit.",
+        # With surrounding whitespace preserved verbatim
+        "  Ibid.  ",
+        "\tIbid.\n",
+    ]
+    for inp in deprecated_inputs:
+        result = format_note_entry(inp, caller=counting_caller)
+        assert result == inp, (
+            f"Input {inp!r} must round-trip verbatim; got {result!r}"
+        )
+
+    assert call_count == 0, (
+        f"API caller was invoked {call_count} times for deprecated "
+        "Latin abbreviations — the short-circuit guard is missing or "
+        "not effective. Deprecated tokens must NEVER reach the API."
+    )
+
+
+def test_format_note_entry_does_not_short_circuit_normal_citations():
+    """The deprecated-token guard must NOT match real citations that
+    happen to start with one of the tokens. e.g. 'Ibid., 47' is a
+    common usage where a page number is appended; for now we let the
+    LLM see those (the prompt rule covers them). Pure-token-only
+    inputs short-circuit; anything with extra content goes to the API.
+    """
+    calls: list[str] = []
+
+    def recording_caller(system: str, user: str) -> str:
+        calls.append(user)
+        return "Charles Yu, *Interior Chinatown* (Pantheon Books, 2020), 45."
+
+    # Real-looking inputs that contain "ibid" but are not pure tokens
+    not_short_circuited = [
+        "Ibid., 47.",  # ibid + page — common usage
+        "Smith, Ibid., 47.",
+        "Title with the word ibid in it.",
+        "Charles Yu, Interior Chinatown.",  # totally unrelated
+    ]
+    for inp in not_short_circuited:
+        format_note_entry(inp, caller=recording_caller)
+
+    assert len(calls) == len(not_short_circuited), (
+        "Some non-pure-token inputs were short-circuited — guard is "
+        "too aggressive."
+    )
+
+
+def test_format_note_entry_passes_through_substantive_prose_verbatim():
+    """When the input is substantive prose — long discursive commentary
+    with no obvious citation skeleton — the formatter must return it
+    VERBATIM without calling the API. CMOS 14.39 explicitly allows
+    substantive notes (commentary, qualifications, cross-references).
+    The previous behavior extracted whatever citation it could find and
+    silently discarded the surrounding prose, which is data loss.
+
+    Surfaced by notes [8], [9], and [75] in the Anti-Communist
+    Formations of LIS draft. The user's explicit policy (chunk 3
+    fix #4): preserve ALL free text, as long as that does not break
+    other formatting.
+
+    Detection rule (conservative): length > 150 chars AND no obvious
+    citation skeleton markers (parenthesized year, URL, vol./no./pp.,
+    DOI, terminal page reference, terminal year). This catches the
+    egregious cases without false-positiving on legitimate citations,
+    which all have at least one skeleton marker.
+    """
+    call_count = 0
+
+    def counting_caller(system: str, user: str) -> str:
+        nonlocal call_count
+        call_count += 1
+        return "should not be called"
+
+    # [8]-style: long discursive note about DuBois/Morehouse, no markers
+    note_8_style = (
+        "W.E.B. DuBois' popularization of philanthropist Henry Lyman "
+        "Morehouse's theory of the talented tenth is celebrated, missing "
+        "the note that he himself retracted such a theory later in his "
+        "life, as written about by Joy James. The violence of Jim Crow "
+        "becomes diminished into many monstrosities, since traditionally "
+        "black colleges and universities became monuments to human hope "
+        "through philanthropic funding."
+    )
+    assert format_note_entry(note_8_style, caller=counting_caller) == note_8_style
+
+    # [9]-style: discursive about Wilder/Murch, no markers
+    note_9_style = (
+        "Craig Steven Wilder's Ebony and Ivy discusses the political "
+        "economy of anti-Blackness in higher education, and the influence "
+        "of not only capital and capitalists in the creation of "
+        "Historically Black Colleges and Universities and white elite "
+        "universities, but also enslaved labor to support the latter."
+    )
+    assert format_note_entry(note_9_style, caller=counting_caller) == note_9_style
+
+    # [75]-style: shorter cross-archive prose, no markers
+    note_75_style = (
+        "This letter appeared in multiple archives, including California "
+        "State Archives, Berkeley and UCLA special collections, and is "
+        "uploaded on UC Santa Barbara Servers."
+    )
+    assert format_note_entry(note_75_style, caller=counting_caller) == note_75_style
+
+    assert call_count == 0, (
+        f"API caller was invoked {call_count} times for substantive prose — "
+        "the substantive-note pass-through guard is missing or not "
+        "effective. Free text must NEVER be sent to the API for "
+        "reformatting under this fix."
+    )
+
+
+def test_format_note_entry_does_not_pass_through_real_citations():
+    """The substantive-prose guard must NOT match real citations that
+    happen to be long. The detection requires absence of citation
+    skeleton markers; real citations have at least one (year, URL,
+    vol/no/pp, DOI, terminal page). This test pins the negative cases
+    so the guard doesn't drift toward over-aggression.
+    """
+    calls: list[str] = []
+
+    def recording_caller(system: str, user: str) -> str:
+        calls.append(user)
+        # Return validator-passing output so the retry loop doesn't fire
+        # and inflate the call count.
+        return "Fake Author, *Fake Title* (Fake Publisher, 2024), 1."
+
+    # Real first-occurrence book note (158 chars, ends with year+period)
+    note_6_style = (
+        "Steve Batterson, The Prosecution of Professor Chandler Davis: "
+        "McCarthyism, Communism, and the Myth of Academic Freedom. "
+        "New York: NYU Press, 2023."
+    )
+    format_note_entry(note_6_style, caller=recording_caller)
+
+    # Real journal article note (URL marker)
+    note_44_style = (
+        "Lawrence W.S. Auld, \"The King Report: New Directions in Library "
+        "and Information Science Education.\" ACRL College & Research "
+        "Libraries News 48, no. 4, 1987. https://crln.acrl.org/example."
+    )
+    format_note_entry(note_44_style, caller=recording_caller)
+
+    # Real govt report (parenthesized year + URL)
+    note_54_style = (
+        "Robert D. Leigh, The California Librarian Education Survey: a "
+        "Report to President Robert G. Sproul of the University of "
+        "California. (New York: Columbia University, August 1952). "
+        "https://hdl.handle.net/example"
+    )
+    format_note_entry(note_54_style, caller=recording_caller)
+
+    # Real master's thesis (year at end)
+    note_107_style = (
+        "Christine Michele Curley, \"The School of the Library,\" "
+        "Unpublished Master Thesis, University of California Los "
+        "Angeles, 2017."
+    )
+    format_note_entry(note_107_style, caller=recording_caller)
+
+    assert len(calls) == 4, (
+        "Some real first-occurrence notes were short-circuited as "
+        "substantive prose — the detection is too aggressive. Each of "
+        "these has at least one citation skeleton marker and must be "
+        "sent to the API for reformatting."
+    )
+
+
+def test_format_note_entry_returns_empty_input_verbatim_without_calling_api():
+    """Empty or whitespace-only input must short-circuit the API call and
+    return the input verbatim. Without this guard, GPT-5 occasionally breaks
+    character on empty input and returns a conversational meta-reply asking
+    for content (e.g., "Please paste the citation entry you'd like
+    formatted..."). Surfaced by note [66] in the Anti-Communist Formations
+    of LIS draft.
+
+    The whitespace must be preserved verbatim so reassembly in
+    cmos.cli.reformat_notes is byte-exact on empty notes — the source's
+    spacing is part of the input contract, not noise to be normalized.
+
+    Critically, the API must NOT be called for empty input — the counting
+    caller asserts call_count == 0 to lock in the cost guarantee.
+    """
+    call_count = 0
+
+    def counting_caller(system: str, user: str) -> str:
+        nonlocal call_count
+        call_count += 1
+        return "should not be called"
+
+    # Each whitespace flavor must round-trip verbatim and not call the API.
+    assert format_note_entry("", caller=counting_caller) == ""
+    assert format_note_entry("   ", caller=counting_caller) == "   "
+    assert format_note_entry("\n", caller=counting_caller) == "\n"
+    assert format_note_entry("\t \n", caller=counting_caller) == "\t \n"
+
+    assert call_count == 0, (
+        "API caller was invoked on empty input — the short-circuit "
+        "guard is missing or not effective. Empty input must NEVER "
+        "reach the API."
+    )

tests/test_runtime_validator.py

@@ -50,6 +50,78 @@ def test_missing_italics_anywhere_fails():
     assert any("italic" in f.lower() for f in result.failures)
 
 
+# --- Shortened-form note carve-out (chunk 3 fix #3, Option D) -------------
+
+
+def test_shortened_form_author_only_passes_without_italics():
+    """CMOS shortened note form for an author-only reference (no page,
+    no title) is just the author's last name with a period: 'Ettarh.',
+    'Murch.', 'O'Mara.'. These contain no italics by definition.
+    Without a carve-out, the validator's "must have italics" rule
+    rejects them and the formatter's retry loop fires its full budget
+    on valid output. Surfaced by the Anti-Communist Formations of LIS
+    real-draft run; ~30 of 107 notes were correct shortened-forms
+    that wasted ~60 API calls retrying.
+
+    The carve-out: if a candidate has no italics AND looks like a
+    shortened-form note (short, name-token shape, optional page,
+    terminal period), the missing-italics check is skipped.
+    """
+    for candidate in ["Ettarh.", "Murch.", "O'Mara.", "Burden-Stelly."]:
+        result = validate(candidate)
+        assert result.passed is True, (
+            f"shortened-form author-only note {candidate!r} should pass "
+            f"validation; got failures {result.failures}"
+        )
+
+
+def test_shortened_form_author_plus_page_passes_without_italics():
+    """The other common CMOS shortened form is author last name + page:
+    'Rosen, 7.', 'Mitchell, 197.', 'CBS, 47.'. Same carve-out applies."""
+    for candidate in [
+        "Rosen, 7.",
+        "Mitchell, 197.",
+        "CBS, 47.",
+        "Seybold, 282.",
+        "Lawrence Powell, 45.",  # multi-word name
+        "Mitchell, 137-39.",  # page range with hyphen
+    ]:
+        result = validate(candidate)
+        assert result.passed is True, (
+            f"shortened-form note {candidate!r} should pass validation; "
+            f"got failures {result.failures}"
+        )
+
+
+def test_long_no_italic_output_still_fails():
+    """The carve-out must NOT be a free pass for any short string.
+    A long output with no italics is still likely a malformed
+    full-form citation (model dropped the italic title), and the
+    retry loop should still fire on it.
+    """
+    candidate = (
+        "Steve Batterson, The Prosecution of Professor Chandler Davis: "
+        "McCarthyism, Communism, and the Myth of Academic Freedom (NYU Press, 2023)."
+    )
+    result = validate(candidate)
+    assert result.passed is False
+    assert any("italic" in f.lower() for f in result.failures)
+
+
+def test_short_but_unstructured_no_italic_output_still_fails():
+    """A short but non-name-shaped string (e.g., a stray sentence
+    fragment with no name structure) should still fail. The carve-out
+    is NOT just "short and ends with period" — it requires the
+    candidate to look like a shortened-form note specifically.
+    """
+    candidate = "this is some prose without italics."
+    result = validate(candidate)
+    assert result.passed is False, (
+        "lowercase prose fragment must not be accepted as a shortened-form "
+        "note; the carve-out should require name-token shape"
+    )
+
+
 def test_unbalanced_italic_markers_fails():
     candidate = "Yu, Charles. *Interior Chinatown. Pantheon Books, 2020."
     result = validate(candidate)