First real loop iteration found that GPT-5 silently drops date qualifiers
("Effective", "Published", "Accessed", etc.) when reformatting web
sources. The field diff was satisfied because the canonical date field
did not include the qualifier, but the canary exact-match axis caught
the regression.
Two fixes, per the canary-upstream policy:
1. Tighten the web-page exemplar canonical: date field now includes the
"Effective" qualifier so the field diff will catch future regressions
without relying on the canary.
2. Add SYSTEM_PROMPT rule 12 instructing the formatter to preserve
semantic date qualifiers from the input.
After these fixes: scalar 1.000, canary exact-match 1.000 on all three
seed exemplars.
21 lines
1.1 KiB
TOML
21 lines
1.1 KiB
TOML
# Source: CMOS official citation guide (chicagomanualofstyle.org), 18th ed.
|
|
# Demonstrates: web page with "Effective" date (rather than access date).
|
|
# Site owner / corporate author sits in the author slot when no personal author exists.
|
|
source = "CMOS official citation guide (chicagomanualofstyle.org/tools_citationguide/citation-guide-1.html)"
|
|
type = "web"
|
|
tags = ["web", "corporate-author"]
|
|
canary = true
|
|
messy_input = "google, \"privacy policy,\" privacy & terms, effective nov 15 2023, policies.google.com/privacy"
|
|
expected_bibliography = "Google. \"Privacy Policy.\" Privacy & Terms. Effective November 15, 2023. https://policies.google.com/privacy."
|
|
|
|
[canonical]
|
|
author = "Google"
|
|
title = "Privacy Policy"
|
|
site = "Privacy & Terms"
|
|
# Include the date qualifier ("Effective", "Published", "Accessed", etc.) in
|
|
# the canonical date field. CMOS 18 preserves these qualifiers in web-source
|
|
# entries; dropping them is a real regression even though the resulting
|
|
# string is structurally well-formed.
|
|
date = "Effective November 15, 2023"
|
|
url = "https://policies.google.com/privacy"
|