Repository navigation
Re-baseline the upstream surface watch at Claude Code 2.1.300; hold cached-artifact checks to the baseline - #1155
Open
seathatflowsinourveins wants to merge 2 commits into
Conversation
…d cached-artifact checks to the baseline The command center's re-baseline follow-up, approved 2026-10-15T01:48Z. Four CachedArtifactTests pinned counts read from an older cache (SDK 0.3.289, Codex rust-v0.160.0) and failed against the host's watch cache at 2.1.300. Re-baseline: `python3 -E -s -B scripts/upstream_surface_watch.py --network --write-baseline --force`, run 2026-10-15T06:38:40Z under a scratch XDG_STATE_HOME. Every source came from the network (coverage.from_cache empty), and the host's watch state directory was left unchanged (mtimes compared). Watched versions, baseline before -> after: - Claude Code, npm latest: 2.1.296 -> 2.1.300 (dist-tags: stable 2.1.292, next 2.1.300) - @anthropic-ai/claude-agent-sdk: 0.3.296 -> 0.3.300 (claudeCodeVersion 2.1.300, matched) - Codex stable release rust-v0.162.1 and binary codex-cli 0.162.1: unchanged Counts that changed: - claude:env: 417 -> 422 (+5 names) - claude:setting from settings-reference.md: 171 -> 172 (a key sdk.d.ts already typed; the union stays 184 and sdk.d.ts stays 174) Unchanged: claude:hook 33, claude:mod 6, codex:config 1057, codex:feature 167. No name was removed and no feature stage changed. The baseline no longer records the claude-doc-docs-index and claude-doc-platform-release-notes fetches, because no enabled doc row reads those pages any more. Not dispositioned here (the dispositions catalog is a separate review). Four of the five new env names have no row, and the new baseline now holds them, so daily diffs will not list them again: - claude:env:CLAUDE_CODE_ARTIFACT_PUBLISH_EDITS - claude:env:CLAUDE_CODE_DISABLE_FILE_MENTIONS - claude:env:CLAUDE_CODE_IDLE_COMPACT_MIN_TOKENS - claude:env:CLAUDE_CODE_YOU_SHOULD_KNOW_DEBUG The fifth, claude:env:CLAUDE_CODE_OVERLOADED_RETRY_MAX_DELAY_MS, already has a row. The claude:doc:memory and claude:doc:skills bodies differ from their reviewed digests and stay unreviewed. The dispositions catalog's baseline_versions still names 2.1.296. Tests: CachedArtifactTests now hold the parse of each unmodified cached artifact to the committed baseline instead of a release-specific literal (173, (33, 173), 171, 1034, and the docs-only 10 of two more tests in the class). When the cached body is the artifact the baseline recorded (same sha256), the names must equal the baseline's. For a per-source parse, the count must match and the names must lie within the kind. When the cache holds another release, the count must keep the watch's own 80% floor (usw.floor_failure), and at least one docs-only key must exist. The comparisons with reformatted, escaped and corrupted copies are unchanged. One test loses "ten" from its name, now test_closing_markers_on_the_docs_only_headings_preserve_the_real_settings. Fail-first, at f165498 against the host cache with no UPSTREAM_SURFACE_TEST_CACHE: four failures (172 != 171; (33, 174) != (33, 173); 174 != 173; 1057 != 1034). After: all six CachedArtifactTests pass against the host cache and against the scratch cache the baseline was built from. Mutations, in a scratch copy of the watch script, against the host cache: - u-escape decoding removed from ts_string_value: 2 failures (both escape tests) - ATX closing hashes no longer stripped: 5 errors (both closing-marker tests) - one sdk.d.ts key lost on every parse: 4 failures (173 != 174; docs-only 11 != 10), which the escaped-copy comparisons alone would miss - unresolved $ref ignored: 4 failures; one schema path lost: 1 failure With the base commit's 2.1.296 baseline (floor branch for sdk.d.ts and the settings page), the unmutated tests pass, and the lost-key mutant also passes: the floor tolerates a 20% loss. docs/upstream-surface-watch.md no longer ties these checks to SDK 0.3.289 and Codex rust-v0.160.0; its line count is unchanged. Checks: python3 scripts/validate.py exits 0 after registering the three changed files in manifests/evidence.json. python3 -m unittest tests.test_upstream_surface_watch tests.test_currency_due tests.test_currency_due_notice ran 233 tests, OK. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This was referenced Oct 15, 2026
…nvariant The command center's FULL read of #1155 at bf9153a (PASS, one fold): the four env names the 2.1.300 re-baseline added (CLAUDE_CODE_ARTIFACT_PUBLISH_EDITS, CLAUDE_CODE_DISABLE_FILE_MENTIONS, CLAUDE_CODE_IDLE_COMPACT_MIN_TOKENS, CLAUDE_CODE_YOU_SHOULD_KNOW_DEBUG) had no disposition row, so the baseline grandfathered them out of unreviewed and currency_due (upstream_surface_watch.py:1597-1600). Each gets a baseline-unreviewed row (the catalog's value for this case, upstream_surface_watch.py:210-220, docs/upstream-surface-watch.md:192-193); #1131's rebase turns them into rulings. test_every_name_a_re_baseline_added_has_a_disposition_row compares the baseline with the 2.1.296 baseline at f165498 (from git history) and requires a row for every added name. Names that predate that baseline (1,566) are grandfathered under the earlier rule, so the invariant covers re-baseline additions, not every baseline name. Fail-before at bf9153a: the four names. After: tests.test_upstream_surface_watch, test_currency_due and test_currency_due_notice OK; --check-dispositions valid (367 rows); validate rc 0. Co-Authored-By: Claude Opus 5.5 <[email protected]>
5 tasks done
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
f165498aalane:foundationcatalogs/foundation/upstream-surface-baseline.json;tests/test_upstream_surface_watch.py;docs/upstream-surface-watch.md, lines 91-92 reworded with the same line count;manifests/evidence.json.Re-baseline:
scripts/upstream_surface_watch.py --network --write-baseline --forceran at 2026-10-15T06:38:40Z under a scratchXDG_STATE_HOME. Every source came from the network, and the host's watch state was unchanged (31 entries, mtimes and sizes compared).Versions:
latest): 2.1.296 → 2.1.300.claudeCodeVersion2.1.300.Count changes:
claude:env: 417 → 422.claude:settingfrom settings-reference.md: 171 → 172. The union stays 184, and sdk.d.ts stays 174.codex:config(1057) andcodex:feature(167) are unchanged.Test change:
New names with no disposition row are left undecided for the dispositions review:
CLAUDE_CODE_ARTIFACT_PUBLISH_EDITSCLAUDE_CODE_DISABLE_FILE_MENTIONSCLAUDE_CODE_IDLE_COMPACT_MIN_TOKENSCLAUDE_CODE_YOU_SHOULD_KNOW_DEBUGSOTA sources
These are the upstream artifacts the new baseline records:
9ebce26c…).8ba90434…, fetched 2026-10-15).rust-v0.162.1config-schema.json(sha2567933a705…, checked against GitHub's published release digest).@anthropic-ai/claude-code, read 2026-10-15T06:38Z: latest 2.1.300, stable 2.1.292, next 2.1.300.Evidence-class table
native_provensyntheticLocal commands run
Fold (4313309)
This is the fold for the command center's FULL read at bf9153a (PASS):
unreviewedand currency_due (upstream_surface_watch.py:1597-1600)baseline-unreviewedrow for each name. This is the catalog's value for this case (upstream_surface_watch.py:210-220;docs/upstream-surface-watch.md:192-193). #1131's rebase turns these rows into rulings.test_every_name_a_re_baseline_added_has_a_disposition_rowcompares the baseline with the 2.1.296 baseline at f165498 and requires a row for every added name. It failed at bf9153a on the four names.Scope of the invariant: it covers names a re-baseline adds. The 1,566 names that predate the 2.1.296 baseline were grandfathered under the earlier rule, so it is not a check on every baseline name.
Dropped
sourcesrows: the new baseline no longer records theclaude-doc-docs-indexandclaude-doc-platform-release-notesfetches. No enabled doc row reads those pages, so this is the script's correct output (docs/upstream-surface-watch.md:96-98).Notes for the reads
CC FULL P3 (
tests/test_upstream_surface_watch.py:61-76): the count-plus-subset check passes when one key is swapped for another, one for one.Floor gap: when the cache is newer than the baseline, a parser that silently loses one key passes until the next re-baseline. The exact-match path catches it when the cache and the baseline agree.
Unreviewed names: the four names above are now in the baseline, so they leave the daily diff until the dispositions review adds their rows.
claude:doc:memoryandclaude:doc:skillsstill differ from their reviewed digests.Follow-up: the dispositions catalog's
baseline_versionsstill says 2.1.296.Test scope: only the three named suites and validate ran, not the full sharded suite.
Checklist
🤖 Generated with Claude Code