Scaffold CMOS 18 reformatter: harness, linter, parser, formatter, CLI

Initial phase-1 baseline of the karpathy/autoresearch-style loop.
The formatter module is the inner-loop artifact; parser and linter
are infra. The linter carries a LINTER_VERSION hash (v0.2.0) that
will force a re-baseline on any rule change.

Components:
- harness/diff.py: case-sensitive field-level substring diff
- harness/score.py: three-axis scoring (field, linter, canary exact)
- src/cmos/linter.py: 9 CMOS 18 structural rules, each Purdue/CMOS cited
- src/cmos/parser.py: locate ## Bibliography section, split entries
- src/cmos/formatter.py: prompt + OpenAI call with caller injection
- src/cmos/cli.py: cmos format path/to/draft.md
- scripts/run_loop.py: loop runner with --fake mode for no-API runs
- exemplars/: 3 canary seed exemplars (book, journal w/DOI, web),
  sourced from chicagomanualofstyle.org quick guide

Tests: 48 passing. Fake-mode baseline scalar = 0.000 on the 3 seed
exemplars (identity caller fails the linter on every rule). This is
the floor the real GPT-5 formatter needs to improve from.
This commit is contained in:
cmos dev
2026-04-10 20:48:33 -04:00
commit 4cad38ef30
29 changed files with 2267 additions and 0 deletions
+45
View File
@@ -0,0 +1,45 @@
"""Field-level structured diff for CMOS 18 candidate outputs.
Approach: for each (field, value) in the canonical record, check whether the
candidate formatted string contains `str(value)` as a case-sensitive
substring. This is deliberately simpler than parsing the candidate — the
linter is responsible for structural checks (punctuation, ordering, italics),
so the diff only answers: "did the right facts survive the reformat?"
Frozen harness module. Edits here invalidate prior scores.
"""
from __future__ import annotations
from dataclasses import dataclass, field
@dataclass
class DiffResult:
matched: list[str] = field(default_factory=list)
missing: list[str] = field(default_factory=list)
@property
def total(self) -> int:
return len(self.matched) + len(self.missing)
@property
def rate(self) -> float:
if self.total == 0:
return 1.0
return len(self.matched) / self.total
def field_diff(candidate: str, canonical: dict) -> DiffResult:
"""Compare a formatted candidate string against a canonical field dict.
Each canonical field counts as matched iff its stringified value appears
as a case-sensitive substring of `candidate`.
"""
result = DiffResult()
for key, value in canonical.items():
if str(value) in candidate:
result.matched.append(key)
else:
result.missing.append(key)
return result