Scaffold CMOS 18 reformatter: harness, linter, parser, formatter, CLI
Initial phase-1 baseline of the karpathy/autoresearch-style loop. The formatter module is the inner-loop artifact; parser and linter are infra. The linter carries a LINTER_VERSION hash (v0.2.0) that will force a re-baseline on any rule change. Components: - harness/diff.py: case-sensitive field-level substring diff - harness/score.py: three-axis scoring (field, linter, canary exact) - src/cmos/linter.py: 9 CMOS 18 structural rules, each Purdue/CMOS cited - src/cmos/parser.py: locate ## Bibliography section, split entries - src/cmos/formatter.py: prompt + OpenAI call with caller injection - src/cmos/cli.py: cmos format path/to/draft.md - scripts/run_loop.py: loop runner with --fake mode for no-API runs - exemplars/: 3 canary seed exemplars (book, journal w/DOI, web), sourced from chicagomanualofstyle.org quick guide Tests: 48 passing. Fake-mode baseline scalar = 0.000 on the 3 seed exemplars (identity caller fails the linter on every rule). This is the floor the real GPT-5 formatter needs to improve from.
This commit is contained in:
@@ -0,0 +1,68 @@
|
||||
"""Tests for src/cmos/cli.py — end-to-end draft reformatting.
|
||||
|
||||
The CLI orchestrates parser → formatter → reassemble. Tests inject a fake
|
||||
formatter so no API calls happen; the goal is to pin the glue logic, not the
|
||||
LLM behavior.
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from cmos.cli import reformat_draft
|
||||
from cmos.parser import NoBibliographyError
|
||||
|
||||
|
||||
def _fake_formatter(messy: str) -> str:
|
||||
# Deterministic fake: return a cleaned-up marker string so the output is
|
||||
# recognizable.
|
||||
return f"FORMATTED({messy.strip()})"
|
||||
|
||||
|
||||
def test_reformat_draft_rewrites_only_bibliography_section():
|
||||
draft = """\
|
||||
# My Paper
|
||||
|
||||
Some prose with citations.
|
||||
|
||||
## Bibliography
|
||||
|
||||
yu, charles. interior chinatown. 2020.
|
||||
kwon, hyeyoung. inclusion work. 2022.
|
||||
"""
|
||||
output = reformat_draft(draft, formatter=_fake_formatter)
|
||||
assert "# My Paper" in output
|
||||
assert "Some prose with citations." in output
|
||||
assert "FORMATTED(yu, charles. interior chinatown. 2020.)" in output
|
||||
assert "FORMATTED(kwon, hyeyoung. inclusion work. 2022.)" in output
|
||||
# Original messy lines must not also survive.
|
||||
assert "yu, charles. interior chinatown. 2020." not in output.replace(
|
||||
"FORMATTED(yu, charles. interior chinatown. 2020.)", ""
|
||||
)
|
||||
|
||||
|
||||
def test_reformat_draft_preserves_sections_after_bibliography():
|
||||
draft = """\
|
||||
## Bibliography
|
||||
|
||||
one entry.
|
||||
|
||||
## Appendix
|
||||
|
||||
appendix text.
|
||||
"""
|
||||
output = reformat_draft(draft, formatter=_fake_formatter)
|
||||
assert "FORMATTED(one entry.)" in output
|
||||
assert "## Appendix" in output
|
||||
assert "appendix text." in output
|
||||
|
||||
|
||||
def test_reformat_draft_raises_when_no_bibliography():
|
||||
with pytest.raises(NoBibliographyError):
|
||||
reformat_draft("## Intro\n\nno bib here.\n", formatter=_fake_formatter)
|
||||
|
||||
|
||||
def test_reformat_draft_bibliography_heading_preserved():
|
||||
draft = "## Bibliography\n\nentry.\n"
|
||||
output = reformat_draft(draft, formatter=_fake_formatter)
|
||||
assert "## Bibliography" in output
|
||||
Reference in New Issue
Block a user