cmos dev
8b14dbc9ca
GPT-5 is a reasoning model and is not bit-deterministic even at the API level (no temperature override allowed). Variance testing in iter 7 showed ~1 in 6 loop runs hit a nondeterministic regression: in one run the magazine_mead exemplar lost its italic markers around "New Yorker" even though the exact same input had produced a clean output 5 times prior. The scoring linter caught it (rule_magazine_name_italicized) but the formatter still emitted the bad output to the user. This commit adds a runtime guardrail: - New module src/cmos/runtime_validator.py — type-independent structural sanity checks. Operates on a single candidate string with no Exemplar context, because at runtime we don't know the source type. Checks: ends with period, no Ibid., has at least one italic span (almost every CMOS bibliography entry italicizes something), balanced * and " markers. Deliberately weaker than the scoring linter; it's a fast guardrail, not a full validator. - format_bibliography_entry now retries up to DEFAULT_MAX_RETRIES (2) times when the validator rejects a candidate. Independent re-calls are usually enough because the failures are stochastic. If every attempt fails, the LAST attempt is returned (no exception) — the caller still gets something usable, and the failure surfaces through the scoring linter or human review. The retry path costs zero on the common case (1 call per entry); ~1-2% extra calls on noisy drafts. Empirical: 3 consecutive loop runs after this change are scalar 1.000 canary 1.000 (vs 5/6 clean in the variance test before). Sample is too small to claim full suppression but the signal is positive. Also adds a new exemplar journal_multiauthor_secondary_first_last.toml captured from the user's HML draft (Hasanah et al., IJIDI 2024). It exercises the case where a multi-author entry has the first author inverted and the rest in First Last form — which the iter 7 HML run got wrong on one entry. Variance testing showed the exemplar passes 6/6 in isolation, so the original HML failure was nondeterminism, not a missing rule. Keeping the exemplar regardless: it adds canary coverage of a real-world multi-author pattern, no-DOI / JSTOR-URL variant, and the year-suffix author-date holdover stripping. Tests: 85 → 95 (8 new for runtime_validator + 2 new for formatter retry). All passing.
34 lines
2.3 KiB
TOML
34 lines
2.3 KiB
TOML
# Source: derived from a real-world entry in the user's HML draft (Hasanah
|
||
# et al., IJIDI 2024). This exemplar specifically covers the CMOS rule that
|
||
# in a multi-author bibliography entry, ONLY the first author is comma-
|
||
# inverted; subsequent authors are written in "First Last" form.
|
||
#
|
||
# This is a regression test for a bug found in iter 7: the formatter saw a
|
||
# messy input where the first author was already inverted and the secondary
|
||
# authors were in First-Last form, and "fixed" the perceived inconsistency by
|
||
# inverting everyone. Per CMOS 14.72 / 14.76, only the first author is
|
||
# inverted in a bibliography entry; secondary authors stay in given-name-
|
||
# first order.
|
||
#
|
||
# Also exercises:
|
||
# - Year-suffix author-date holdover that should be stripped: "(2024). "
|
||
# - Leading "The" in journal name to be dropped (rule 17)
|
||
# - Hyphen→en-dash for page range, no elision (under 100)
|
||
# - Source provides a JSTOR URL instead of a DOI (rare variant)
|
||
source = "user draft (HML Submission); article: Hasanah et al., IJIDI 8, no. 2 (2024)"
|
||
type = "journal"
|
||
tags = ["journal", "multi-author", "no-doi", "jstor-url"]
|
||
canary = true
|
||
messy_input = "Hasanah, Anis Karunia Uswatun, Fitri Mutia, and Norhuda Salleh. (2024). “Crossing the Barriers of Library Anxiety: A Quantitative Evaluation of Indonesian Undergraduate Students with Visual Disabilities Navigating Their Academic Library.” The International Journal of Information, Diversity, & Inclusion 8, no. 2 (2024): 29-51. https://www.jstor.org/stable/48786446."
|
||
expected_bibliography = "Hasanah, Anis Karunia Uswatun, Fitri Mutia, and Norhuda Salleh. \"Crossing the Barriers of Library Anxiety: A Quantitative Evaluation of Indonesian Undergraduate Students with Visual Disabilities Navigating Their Academic Library.\" *International Journal of Information, Diversity, & Inclusion* 8, no. 2 (2024): 29–51. https://www.jstor.org/stable/48786446."
|
||
|
||
[canonical]
|
||
authors = "Hasanah, Anis Karunia Uswatun, Fitri Mutia, and Norhuda Salleh"
|
||
article_title = "Crossing the Barriers of Library Anxiety: A Quantitative Evaluation of Indonesian Undergraduate Students with Visual Disabilities Navigating Their Academic Library"
|
||
journal = "International Journal of Information, Diversity, & Inclusion"
|
||
volume = 8
|
||
issue = 2
|
||
year = 2024
|
||
pages = "29–51"
|
||
url = "https://www.jstor.org/stable/48786446"
|