v2 chunk 5: 5 fixes from cross-draft triage (HML + Reading_Disrepair)

After chunk 4 verification on the Anti-Communist draft, ran v2
against the previously-untested ## Notes sections of HML (72 notes)
and Reading_Disrepair (25 notes) — 97 new notes for 204 total
across 3 drafts. The wider corpus surfaced 5 distinct issues that
weren't visible in the Anti-Communist data alone.

src/cmos/note_formatter.py:

1. Tighten substantive prose detector — exclude semicolon-bearing
   inputs (chunk 5 task 1). The HML draft was dominated by compound
   shortened references like:
     See Author1, "Title;" Author2, "Title;" Author3, "Title."
   17 of 72 HML notes matched this pattern, exceeded the 150-char
   threshold, and had no citation skeleton markers, so they were
   passed through verbatim by the chunk 3 detector. Compound
   references use semicolons as item separators; substantive prose
   doesn't. Adding `if ";" in stripped: return False` cleanly
   separates the two without affecting the genuine substantive
   notes (which contain none).

2. Rule 3 — distinguish comma-inside vs period-inside for
   article/chapter title closing quote based on whether MORE
   content follows the title. Surfaced by 8+ entries across drafts
   producing the wrong `"Title,".` pattern (comma inside followed
   by stray period outside) when the title is the last element of
   the entry. Rule 3 now explicitly says: comma inside when more
   content follows; period inside when title is the last element.

3. New rule 20 — preserve signal phrases verbatim. Surfaced by HML
   [69] and [70] where "For example," was stripped while "See" was
   preserved elsewhere. CMOS notes commonly open with signal
   phrases (See, See also, For example, Cf., Compare, But see,
   Contra, Accord, Quoted in) that indicate how a citation relates
   to the surrounding argument. Stripping them is data loss.

4. Rule 1 — explicit "no comma between author name and 'et al.'"
   in non-inverted note form. Surfaced by HML [69] where "Bignoli
   et al." became "Bignoli, et al." (extra comma). The comma
   before "et al." is a feature of inverted bibliography form,
   not non-inverted note form. Rule 1 now includes WRONG examples.

5. Citation skeleton URL marker — also accept bare-domain URLs
   without the https:// scheme. Surfaced by Reading_Disrepair [1]
   GAO entry which had `files.gao.gov/...` without `https://`,
   so the existing https?:// marker didn't match and the 190-char
   citation was passed through as substantive prose. New pattern:
   `\b\w{2,}(?:\.\w{2,})+/\S*` catches bare domains like
   files.gao.gov/path without false-positiving on common things
   like "e.g./" (single-char tokens excluded by {2,}).

Tests: 157/157 (was 153, +4 net new for prompt-content and regex
verification). The +4 is: compound-reference detection, genuine
substantive prose regression check, rule 3 / rule 20 / rule 1
prompt-content tests, bare-domain URL detection.

Path B: rule 18 (publishers, chunk 4) and rule 20 (signal phrases,
this commit) are v2-only. Rule 1 et al. clarification could in
principle apply to v1 too but v1 doesn't see "et al." in the same
non-inverted form. v1 untouched. 96/96 v1 tests still passing.

Real-draft validation pending — verification re-run against all
3 drafts will follow in the next step.

src/cmos/note_formatter.py

@@ -59,33 +59,48 @@ _DEPRECATED_LATIN_RE = re.compile(
     re.IGNORECASE,
 )
 
-# Substantive note pass-through (chunk 3 fix #4, Option A). When the
+# Substantive note pass-through (chunk 3 fix #4, Option A; tightened in
-# input is long discursive prose with no obvious citation skeleton, the
+# chunk 5 task 1). When the input is long discursive prose with no
-# formatter returns it VERBATIM without calling the API. CMOS 14.39
+# obvious citation skeleton, the formatter returns it VERBATIM without
-# explicitly allows substantive notes (commentary, qualifications,
+# calling the API. CMOS 14.39 explicitly allows substantive notes
-# cross-references), and the previous behavior extracted whatever
+# (commentary, qualifications, cross-references), and the previous
-# citation it could find and silently discarded the surrounding prose.
+# behavior extracted whatever citation it could find and silently
-# Surfaced by notes [8], [9], and [75] in the Anti-Communist Formations
+# discarded the surrounding prose. Surfaced by notes [8], [9], and [75]
-# of LIS draft.
+# in the Anti-Communist Formations of LIS draft.
 #
 # Detection is conservative: input must be longer than the threshold
-# AND must NOT contain ANY of the citation skeleton markers below.
+# AND must NOT contain ANY of the citation skeleton markers below
+# AND must NOT contain a semicolon (the chunk 5 tightening).
 # Real first-occurrence citations virtually always have at least one
 # skeleton marker (terminal year, terminal page reference, URL, vol.,
 # no., pp., DOI, parenthesized year). Substantive prose typically has
-# none of these. The combination of length-floor + marker-absence
+# none of these.
-# catches the egregious cases without false-positiving on legitimate
-# citations that happen to be long.
 #
-# The user's explicit policy (chunk 3 fix #4): preserve ALL free text
+# The semicolon check (chunk 5 task 1) was added after the HML draft
-# as long as that does not break other formatting. Erring on the side
+# verification revealed ~17 of 72 notes were COMPOUND SHORTENED
-# of preservation is intentional.
+# REFERENCES — multiple shortened citations separated by semicolons —
+# that had no skeleton markers and were >150 chars, so they matched the
+# original (chunk 3) detector and got passed through verbatim. Compound
+# references look like:
+#
+#   Author1, "Short Title 1;" Author2, "Short Title 2;" Author3, "..."
+#
+# Substantive prose almost never uses semicolons as item separators
+# (they use commas inside English sentences). Adding "no semicolons" as
+# a third condition cleanly separates the two classes without affecting
+# the genuine substantive notes (which contain none).
+#
+# The user's explicit policy: preserve ALL free text as long as that
+# does not break other formatting. The chunk 5 tightening reflects the
+# observation that the original detector WAS breaking other formatting
+# (preventing legit compound references from being processed).
 _SUBSTANTIVE_PROSE_LENGTH_THRESHOLD = 150
 
 _CITATION_SKELETON_MARKERS_RE = re.compile(
     r"(?:"
     r"\([^)]*\d{4}[^)]*\)"               # parenthesized year — (2024), (NYU Press, 2023), (March 2024)
-    r"|https?://"                        # URL
+    r"|https?://"                        # URL with scheme
+    r"|\b\w{2,}(?:\.\w{2,})+/\S*"        # bare-domain URL — files.gao.gov/path, example.com/abc
     r"|\bvol\.\s*\d+"                    # volume number — vol. 7
     r"|\bno\.\s*\d+"                     # issue number — no. 4
     r"|\bpp?\.\s*\d+"                    # page reference — p. 45, pp. 45-67
@@ -100,23 +115,30 @@ _CITATION_SKELETON_MARKERS_RE = re.compile(
 def _looks_like_substantive_prose(text: str) -> bool:
     """True if ``text`` looks like substantive note prose, not a citation.
 
-    Conservative detection: only matches input that is BOTH long AND
+    Conservative detection: only matches input that meets ALL of:
-    lacks any obvious citation skeleton markers. Designed to catch
+    - longer than ``_SUBSTANTIVE_PROSE_LENGTH_THRESHOLD`` chars
-    long discursive notes (689+ chars in the Anti-Communist draft)
+    - contains no citation skeleton markers (parenthesized year, URL,
-    while preserving legitimate first-occurrence notes — which all
+      vol./no./pp., DOI, terminal page reference, terminal year)
-    have at least one skeleton marker even when long. The shorter
+    - contains no semicolons
-    [75]-style cases (~163 chars, no markers) are also caught because
-    the threshold is set well below the typical citation length floor
-    when markers are present.
 
-    See ``_SUBSTANTIVE_PROSE_LENGTH_THRESHOLD`` and
+    Designed to catch long discursive notes (689+ chars in the
-    ``_CITATION_SKELETON_MARKERS_RE`` above for the detection details.
+    Anti-Communist draft) while preserving legitimate first-occurrence
+    citations (which always have at least one skeleton marker) AND
+    legitimate compound shortened references (which use semicolons as
+    item separators — see the chunk 5 tightening note in the constants
+    block above).
+
+    See ``_SUBSTANTIVE_PROSE_LENGTH_THRESHOLD``,
+    ``_CITATION_SKELETON_MARKERS_RE``, and the module constants for
+    the detection details.
     """
     stripped = text.strip()
     if len(stripped) < _SUBSTANTIVE_PROSE_LENGTH_THRESHOLD:
         return False
     if _CITATION_SKELETON_MARKERS_RE.search(stripped):
         return False
+    if ";" in stripped:
+        return False
     return True
 
 # Default model. Override with OPENAI_MODEL=... in your .env.
@@ -147,9 +169,23 @@ Rules you must follow (CMOS 18th edition specifically, NOTE form):
    - One author: "Charles Yu"
    - Two authors: "Charles Yu and Hyeyoung Kwon"
    - Three authors: "Charles Yu, Hyeyoung Kwon, and Kathleen Doyle"
-   - Four or more authors: first author followed by ", et al." (CMOS
+   - Four or more authors: the first author's name followed DIRECTLY
-     14.76: notes use first author + et al. for 4+ authors, even
+     by "et al." with NO comma between the name and "et al." (CMOS
+     14.76; notes use first author + et al. for 4+ authors, even
      though bibliography lists more before truncating).
+     - Correct: "Charles Yu et al."
+     - Correct: "Bignoli et al."
+     - WRONG:   "Charles Yu, et al."  (extra comma)
+     - WRONG:   "Bignoli, et al."     (extra comma)
+     The comma before "et al." is a feature of INVERTED bibliography
+     form ("Bignoli, A., et al."), not non-inverted note form. In note
+     form there is no comma between the author's last name (or full
+     name) and "et al."
+
+     This applies equally to shortened-form notes — when the input is
+     just "Author et al., Short Title" the output should also be
+     "Author et al., 'Short Title'" with no comma before "et al."
+
    Corporate authors (e.g., "Google", "Modern Language Association")
    appear as-is, not reordered.
    Non-Western names already in family-first order (e.g., "Liu Xinwu",
@@ -164,9 +200,31 @@ Rules you must follow (CMOS 18th edition specifically, NOTE form):
 3. ITALIC AND QUOTE CONVENTIONS are the same as bibliography form:
    book titles and journal titles are italicized using Markdown
    asterisks (*Title*); article titles and chapter titles are
-   wrapped in straight double quotes. In NOTE form the closing
+   wrapped in straight double quotes.
-   comma goes INSIDE the closing quote of the article/chapter
+
-   title: "Article Title,"  (not "Article Title",).
+   The CLOSING PUNCTUATION INSIDE the article/chapter title's
+   closing quote depends on what comes next in the note:
+
+   (a) If MORE CONTENT FOLLOWS the title (a journal name, a page
+       number, a publisher, etc.), use a COMMA inside the closing
+       quote, NOT outside:
+       - Correct: "Article Title," *Journal* 12...
+       - WRONG:   "Article Title", *Journal* 12...
+
+   (b) If the title is the LAST ELEMENT of the note (nothing
+       follows except the entry-terminal period), use a PERIOD
+       inside the closing quote, NOT a comma:
+       - Correct: Mattern, "Maintenance and Care."
+       - WRONG:   Mattern, "Maintenance and Care,".
+       - WRONG:   Mattern, "Maintenance and Care".
+
+   The (b) form is especially common in shortened-form notes that
+   give just an author and a short title (no page number, no
+   journal). The model commonly errs on this case by producing
+   `"Title,".` (comma inside quote followed by stray period
+   outside) — that is incorrect punctuation. The correct form is
+   `"Title."` with the period inside the closing quote and nothing
+   after.
 
 4. BOOK NOTE FORM:
    Author, *Book Title* (Publisher, Year), specific-page.
@@ -356,6 +414,36 @@ Rules you must follow (CMOS 18th edition specifically, NOTE form):
       Investigating Committee on Education, the 1951 Hearings*,
       66, https://hdl.handle.net/2027/mdp.39015070361673.
 
+20. SIGNAL PHRASES — preserve them verbatim. CMOS notes commonly
+    open with a signal phrase that tells the reader how the cited
+    source relates to the surrounding argument. Common signal phrases
+    that appear at the start of notes:
+
+    - "See" / "See also" / "See, e.g.,"
+    - "For example,"
+    - "Cf." (compare)
+    - "Compare"
+    - "But see" / "But cf."
+    - "Contra"
+    - "Accord"
+    - "Quoted in"
+
+    These phrases are MEANINGFUL — they tell the reader whether the
+    citation supports, contradicts, exemplifies, or qualifies the
+    author's claim. PRESERVE them VERBATIM at the start of the note.
+    Do NOT strip them, do NOT normalize one to another (don't convert
+    "For example," to "See" or vice versa), and do NOT move them to
+    a different position in the note.
+
+    Examples:
+    - Source: `For example, Bignoli et al., "Status in Academic Libraries."`
+    - Correct: `For example, Bignoli et al., "Status in Academic Libraries."`
+    - WRONG:   `Bignoli et al., "Status in Academic Libraries."` (signal phrase stripped)
+
+    - Source: `See Smith, "Article Title;" Jones, "Other Title."`
+    - Correct: `See Smith, "Article Title"; Jones, "Other Title."`
+    - WRONG:   `Smith, "Article Title"; Jones, "Other Title."` (signal phrase stripped)
+
 Output ONLY the single reformatted note. No preamble, no explanation,
 no leading number, no code fences.
 """

tests/test_note_formatter.py

@@ -112,6 +112,108 @@ def test_system_prompt_mentions_month_or_season_preservation():
     assert "preserve" in lower
 
 
+def test_system_prompt_rule_3_distinguishes_comma_and_period_at_title_end():
+    """Chunk 5 task 2: rule 3's punctuation guidance for note form must
+    distinguish between (a) title followed by more content (comma inside
+    quote) and (b) title as the last element of the entry (period inside
+    quote). Surfaced by 8+ entries across drafts producing the wrong
+    `"Title,".` pattern instead of `"Title."`.
+    """
+    lower = SYSTEM_PROMPT_NOTES.lower()
+    # Either explicit "last element" wording or both forms shown
+    assert "last element" in lower or "nothing follows" in lower
+    # Both punctuation patterns should be mentioned
+    assert "period inside" in lower or 'title."' in SYSTEM_PROMPT_NOTES.lower()
+
+
+def test_system_prompt_mentions_signal_phrases_to_preserve():
+    """Chunk 5 task 3: explicit rule that signal phrases at the start
+    of notes must be preserved verbatim. Surfaced by HML [69] and [70]
+    where "For example," was stripped while "See" was preserved
+    elsewhere — inconsistent behavior that loses meaningful authorial
+    intent.
+    """
+    lower = SYSTEM_PROMPT_NOTES.lower()
+    assert "signal phrase" in lower
+    # The list should mention at least the common ones that were
+    # actually observed in the drafts
+    assert "see also" in lower
+    assert "for example" in lower
+    assert "cf." in lower or "compare" in lower
+
+
+def test_system_prompt_rule_1_says_no_comma_before_et_al():
+    """Chunk 5 task 4: rule 1 must explicitly say "et al." follows the
+    author name with NO comma in note form (non-inverted). Surfaced by
+    HML [69] where "Bignoli et al." became "Bignoli, et al." with a
+    spurious comma. CMOS 14.76: in non-inverted note form, the comma
+    that would precede "et al." in inverted bibliography form is NOT
+    used.
+
+    Test logic: the "no comma" prohibition must appear NEAR the
+    "et al." mention in rule 1 (within 300 chars), not just somewhere
+    else in the prompt (the polish #5 month/season rule also says "no
+    comma" but that's a different context).
+    """
+    text = SYSTEM_PROMPT_NOTES.lower()
+    # Find all occurrences of "et al" and check if "no comma" is within
+    # 300 chars before or after each one
+    found_proximity = False
+    idx = text.find("et al")
+    while idx != -1:
+        window = text[max(0, idx - 300) : min(len(text), idx + 300)]
+        if "no comma" in window:
+            found_proximity = True
+            break
+        idx = text.find("et al", idx + 1)
+
+    # Also accept an explicit wrong-form example
+    has_wrong_example = (
+        '"Bignoli, et al."' in SYSTEM_PROMPT_NOTES
+        or '"Smith, et al."' in SYSTEM_PROMPT_NOTES
+        or 'wrong: "' in text and "et al" in text
+    )
+
+    assert found_proximity or has_wrong_example, (
+        "rule 1 must explicitly forbid the comma between author name "
+        "and 'et al.' in non-inverted note form. The 'no comma' "
+        "prohibition must appear near the 'et al.' rule, or an "
+        "explicit wrong-form example must be present."
+    )
+
+
+def test_substantive_prose_detector_recognizes_domain_only_urls():
+    """Chunk 5 task 5: the citation skeleton marker for URLs must also
+    recognize bare-domain URLs (without the https:// scheme), so that
+    real first-occurrence citations with bare-domain URLs aren't
+    false-positively passed through as substantive prose. Surfaced by
+    Reading_Disrepair [1] GAO entry which had files.gao.gov/... instead
+    of https://files.gao.gov/...
+    """
+    # 190+ char input with no parens-year, no vol/no/pp, no DOI marker,
+    # no terminal page reference — but WITH a bare-domain URL.
+    input_with_bare_domain = (
+        "Government Accountability Office. Public Libraries: Many Buildings "
+        "Are Reported to Be in Poor Condition. GAO-26-107262. 18 December "
+        "2025. files.gao.gov/reports/GAO-26-107262/index.html"
+    )
+    assert len(input_with_bare_domain) > 150  # confirms it would otherwise hit the threshold
+
+    call_count = 0
+
+    def counting_caller(system: str, user: str) -> str:
+        nonlocal call_count
+        call_count += 1
+        return "Fake Author, *Fake Title* (Fake Publisher, 2024), 1."
+
+    format_note_entry(input_with_bare_domain, caller=counting_caller)
+    assert call_count == 1, (
+        "Citation with bare-domain URL was incorrectly passed through "
+        "as substantive prose; the URL marker should also recognize "
+        "bare-domain URLs without the https:// scheme."
+    )
+
+
 def test_system_prompt_publisher_exception_list_includes_nyu_press():
     """Chunk 4 task 2: rule 18's publisher exception list should include
     NYU Press alongside the existing MIT Press / ALA Editions / MLA.
@@ -366,6 +468,92 @@ def test_format_note_entry_passes_through_substantive_prose_verbatim():
     )
 
 
+def test_format_note_entry_does_not_pass_through_compound_shortened_references():
+    """Chunk 5 task 1: long compound shortened references (multiple
+    citations separated by semicolons) must NOT be treated as substantive
+    prose, even when they exceed the length threshold and have no
+    citation skeleton markers. Surfaced by the HML draft, where ~17 of
+    72 notes were compound references like:
+
+        See Meadowbrooke et al., "Information Behavior and HIV Testing;"
+        Kim and Syn, "Health Information on Facebook;" ...
+
+    These have no parenthesized year, no URL, no vol/no/pp markers, and
+    they're 200-400 chars long — so they matched the chunk 3 fix #4
+    substantive prose detector and were passed through verbatim, leaving
+    curly quotes and incorrect semicolon-quote ordering unfixed.
+
+    The discriminator: substantive prose doesn't typically use
+    semicolons as item separators. Compound references do. So adding
+    "no semicolons" as a third condition (alongside "long" and "no
+    skeleton markers") cleanly separates the two classes.
+    """
+    calls: list[str] = []
+
+    def recording_caller(system: str, user: str) -> str:
+        calls.append(user)
+        return "Fake Author, *Fake Title* (Fake Publisher, 2024), 1."
+
+    # All of these are real-shaped HML compound references (synthesized to
+    # match observed patterns). Each is over the 150-char threshold and has
+    # no citation skeleton markers (no parenthesized year, no URL, no
+    # vol/no/pp). They MUST be sent to the LLM for formatting.
+    compound_refs = [
+        # 2-reference compound (smallest)
+        'Knapp, "Creating Safe and Inclusive Spacing;" Padrón, "Defending Queer Youth;" Pinsky and Brenner, "Threats to Accessing Sexuality Information."',
+        # 4-reference compound with "See" prefix
+        'See Dilevko and Gottlieb, "Pornography and Bibliographic Access;" Drabinski, "Queering the Catalog;" Billey et al., "Critique of RDA 9.7;" Baucom, "Archival Descriptions of LGBTQ Materials."',
+        # 3-reference compound, no prefix
+        'Ajayi and Omotayo, "Challenges of HIV/AIDS;" Kachota and Kassim, "Sexual and Reproductive Health Information-Seeking Behaviour;" Bankole and Busayo, "Reproductive Health Information Seeking Behaviour."',
+    ]
+    for ref in compound_refs:
+        format_note_entry(ref, caller=recording_caller)
+
+    assert len(calls) == len(compound_refs), (
+        f"some compound references were short-circuited as substantive prose; "
+        f"expected {len(compound_refs)} calls, got {len(calls)}. The "
+        f"detector is over-matching."
+    )
+
+
+def test_format_note_entry_still_passes_through_genuine_substantive_prose():
+    """Regression test for chunk 5 task 1: tightening the detector with a
+    semicolon check must NOT break the genuine substantive-prose pass-
+    through that chunk 3 fix #4 was designed for. The Anti-Communist
+    notes [8], [9], and [75] (long discursive commentary, NO semicolons)
+    must still be preserved verbatim.
+    """
+    call_count = 0
+
+    def counting_caller(system: str, user: str) -> str:
+        nonlocal call_count
+        call_count += 1
+        return "should not be called"
+
+    # Synthesized to match Anti-Communist note shapes — long discursive
+    # prose, no semicolons, no skeleton markers.
+    note_8_style = (
+        "W.E.B. DuBois' popularization of philanthropist Henry Lyman "
+        "Morehouse's theory of the talented tenth is celebrated, missing "
+        "the note that he himself retracted such a theory later in his "
+        "life, as written about by Joy James. The violence of Jim Crow "
+        "becomes diminished into many monstrosities, since traditionally "
+        "black colleges and universities became monuments to human hope "
+        "through philanthropic funding."
+    )
+    note_75_style = (
+        "This letter appeared in multiple archives, including California "
+        "State Archives, Berkeley and UCLA special collections, and is "
+        "uploaded on UC Santa Barbara Servers."
+    )
+    assert format_note_entry(note_8_style, caller=counting_caller) == note_8_style
+    assert format_note_entry(note_75_style, caller=counting_caller) == note_75_style
+    assert call_count == 0, (
+        "genuine substantive prose was incorrectly sent to the API; the "
+        "semicolon-tightening should not have affected these (no semicolons)."
+    )
+
+
 def test_format_note_entry_does_not_pass_through_real_citations():
     """The substantive-prose guard must NOT match real citations that
     happen to be long. The detection requires absence of citation