Skip to content

Commit c148e04

Browse files
Use the vendor TypeSafe SDK for crosswalk transport and retry (#725)
### Scope The crosswalk's `call()` duplicates HTTP transport and retries already shipped by TypeSafe. Replace that loop with the published `typesafe-sdk==0.7.2` client and its native retry policy, imported only for `judge`. Preserve request construction, validation, thresholds, ledger code and recorded judgments. - Base commit after the requested rebase: `4af7417b7d4db8936c0136235a86717f31d16f27` (descends from `77d7516d8`). - Lane: `lane:foundation`. - Owned paths: `tools/sota-convergence/gap_crosswalk.py`, its new SDK test, dated decision, compact synthetic evidence and the evidence registry. - Claude cross-family read completed: 0 P1, 0 P2; both P3 documentation corrections are addressed. The command center authorized leaving draft after these fixes; the command center alone lands the PR. This is separate from the currency handoff in #721. - Rebased conflict-free onto current main on 2026-10-05. The five owned source/evidence files remain byte-identical to their original qualification; dated evidence retains its original input revision. Main's registry was restored and native registration rerun, preserving every main entry and all non-file registry fields. ## SOTA sources - [typesafe-ai/[email protected], f078f1e2: native client](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/client/sync/client.py#L22), [structured Choice types](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/question_types.py#L39) and [raw-question normalization](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/questions.py#L10). All three prerequisite compatibility gates pass at this pin. - [Usage](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/response_types.py#L65), [choice probabilities](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_schemas/models.py#L25) and [public HTTP metadata](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/schemas/base.py#L81) preserve the existing validator input and optional request ID. - The [strict SDK schema](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/schemas/base.py#L21) uses `extra="ignore"` and `strict=True`, requires [confidence](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_schemas/models.py#L19), and [drops unknown answer types](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/response_types.py#L86) before `validate()`/ledger. No modeled field is lost on any of the 695 retained successful rows (2,254 choice answers); future schema-mismatched responses can change acceptance or lose extra fields. The decision now makes this boundary explicit. - [RetryPolicy](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/retry.py#L52) and [native retry-count header](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/transport.py#L68). Explicit settings retain five attempts, 429/529 only, 1/2/4/8-second delays, no jitter or Retry-After, no connection/timeout retries and a 60-second attempt timeout. - [Vendor-owned headers](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/transport.py#L116) explain the authorized User-Agent change to `typesafe-sdk/0.7.2`. [pydantic/[email protected], d426e2c4: public elapsed contract](https://github.com/pydantic/httpx2/blob/d426e2c44d1247fcf3326ed1cb496e56353efa4a/src/httpx2/httpx2/_models.py#L579) supplies successful-attempt HTTP timing, excluding SDK decoding; the prior timer also included final JSON decoding. Earlier attempts/backoff remain excluded. - [Upstream MockTransport seam](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/tests/conftest.py#L33) supplies the offline test boundary. [PyPI 0.7.2](https://pypi.org/pypi/typesafe-sdk/0.7.2/json) publishes the wheel; its downloaded SHA256 matches `0a961148187d52e18276ed7f2d02617cfac48e3b97673cb631a8749397d43d1e`. Installation used an isolated uv environment with source builds disabled. - [Repository hot-file protocol at 4af7417](https://github.com/seathatflowsinourveins/native-agent-stack/blob/4af7417b7d4db8936c0136235a86717f31d16f27/docs/lanes.md#L94): native `host_receipts.register_file` updated the five owned files from main's registry and re-registered the revised decision; the registry is committed last at `ad6979a866317456cf80e32855c0a591f8371d88`. ### Evidence-class table | Claim | Evidence class | Command / receipt | | --- | --- | --- | | SDK accepts structured questions and exposes all required response/retry fields | `source_review` | Pinned sources above and dated decision | | Native SDK transport/parser/retry preserves modeled choice payload, metadata and selected retry policy with injected responses | `synthetic` | 6 MockTransport tests plus 3 offline import and version checks; all 22 touched tests pass, no skips | | Existing crosswalk output remains current without the SDK or site packages | `local_integration` | Ordinary and `python3 -S` offline `build --check`, exit 0 | | Registered evidence is internally consistent | `local_integration` | `validate.py` after final P3 registry commit: exit 0, 10,098 hashes; integrity and scope only | The stub tests are locally authored integration checks through the unchanged SDK implementation, not an unchanged upstream suite or live provider acceptance. No live judge call occurred. The original environment-only credential consumer remains unchanged. No credential inventory, install script, host receipt or host-stack pin changed. Dependency pinning in the runtime install hint remains an explicit limitation: the hint pins only `typesafe-sdk==0.7.2`. The observed dependency set includes `httpx2==2.13.1`, `pydantic==2.13.5` and `tenacity==9.1.4`; another resolution can select versions outside that tested set. The optional hint change is deferred, as allowed by the cross-family verdict. ### Local commands run Checks used private scratch and an isolated uv cache; the private environment path prefixes are recorded in the lane status. Runtime checks are reused from the unchanged code/test inputs at `9a87ae07`; the P3 follow-up changes only the decision and its registration. Validation was rerun before the doc/registry commits and after the final registry commit. ```text python3 tools/sota-convergence/gap_crosswalk.py build --check exit 0: crosswalk up to date python3 -S tools/sota-convergence/gap_crosswalk.py build --check exit 0: crosswalk up to date uv run --no-project --no-config --no-env-file --no-build --no-managed-python --no-python-downloads --offline --with typesafe-sdk==0.7.2 python3 -m unittest discover -s tests -p 'test_gap_crosswalk*.py' -v exit 0: 22 tests, no skips python3 scripts/component_matrix.py --write exit 0; outputs unchanged python3 scripts/new_host_grand_list.py --write exit 0; outputs unchanged python3 scripts/validate.py exit 0 after the final P3 registry commit; final commit is registry-only git diff --check exit 0 ``` The first eight-test stub run exited 1 because three preloaded mock responses lacked elapsed metadata. The final streamed fixture exercises native read/close timing; that failed attempt and correction remain in `evidence/artifacts/typesafe-sdk-transport-20261005/checks.json`, with final returned output alongside it. Independent review also corrected the missing-SDK hint to preserve the original command's required `--set`/`--out` arguments; no material findings remain. ### Decision record `docs/decisions/2026-10-05-typesafe-sdk-transport.md` records the three gates, retry comparison, header/timing changes, alternatives, overturn conditions and completeness critic. ### Host evidence Not applicable: no host receipts changed. ### Checklist - [x] No GitHub Actions changes; existing offline CI build remains stdlib-only. - [x] No secrets printed or committed; no new required secret. - [x] No new paid hosting, subscription or billing surface. - [x] Peer-owned files and #721's worktree preserved. - [x] Registry committed last; required post-commit validation passed.
1 parent 4fd7106 commit c148e04

6 files changed

Lines changed: 465 additions & 19 deletions

File tree

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
# TypeSafe crosswalk transport through the vendor SDK
2+
3+
Date: 2026-10-05. Scope: `tools/sota-convergence/gap_crosswalk.py::call()`.
4+
This serves the north-star research stack by maintaining the existing landscape
5+
gap judgments without duplicating the vendor's HTTP and retry implementation.
6+
It authorizes no new model run or reinterpretation of recorded judgments.
7+
8+
## Decision and compatibility gates
9+
10+
Use the published `typesafe-sdk==0.7.2` wheel, from
11+
`typesafe-ai/[email protected]`, commit
12+
`f078f1e208a0d885154dc758344ae4fce77ac168`, through its public
13+
`TypeSafeClient.system_one()` and `RetryPolicy`. The release was published on
14+
2026-09-26. Import it only inside `call()`, reached by `judge`; an absent or
15+
different SDK fails with the pinned `uv run --with typesafe-sdk==0.7.2` command.
16+
`build`, `build --check` and `score` retain their standard-library dependency
17+
boundary. The existing environment-only API-key consumer is unchanged.
18+
19+
All three prerequisite gates pass by source review:
20+
21+
1. Dict-shaped instructions and criteria are JSON content in
22+
[ChoiceModel:39-49](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/question_types.py#L39).
23+
[normalize_questions:10-23](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/questions.py#L10)
24+
accepts the existing raw choice dictionaries; no prompt conversion is added.
25+
2. The response preserves
26+
[usage:65-73](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/response_types.py#L65),
27+
[choice probabilities:25-30](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_schemas/models.py#L25),
28+
and [public raw HTTP metadata:81-94](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/schemas/base.py#L81).
29+
Native `model_dump(mode="json")` supplies the existing `validate()` input.
30+
Reading the raw header retains the prior optional request ID; the SDK's
31+
stricter `request_id` property would reject a missing header.
32+
A strict SDK schema now sits between the wire response and `validate()`
33+
and the ledger. The [base schema:21](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/schemas/base.py#L21)
34+
uses `extra="ignore"` and `strict=True`; the
35+
[ChoiceAnswer schema:19-24](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_schemas/models.py#L19)
36+
requires `confidence`. The
37+
[response preparation:86-93](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/response_types.py#L86)
38+
drops answers of unknown type from the parsed response. This loses nothing
39+
on every retained successful recorded row:
40+
[eval judgments](../../evidence/artifacts/gap-crosswalk-92bb279/typesafe-eval.json)
41+
contain 315 rows / 1,039 answers and
42+
[current judgments](../../evidence/artifacts/gap-crosswalk-92bb279/typesafe-current.json)
43+
contain 380 rows / 1,215 answers. All 695 rows contain modeled usage fields
44+
and all 2,254 answers contain exactly the four modeled Choice
45+
fields, including confidence. This static artifact census is source review,
46+
not a new SDK or model run. For a future response that does not match the
47+
schema, the boundary can change acceptance and drops extra fields before our
48+
unchanged validator and ledger. Raw HTTP data remains available through the
49+
SDK, but the ledger receives its parsed model dump. This schema boundary was
50+
omitted from the initial decision and is now explicit following cross-family
51+
source review.
52+
3. [RetryPolicy:52-123](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/retry.py#L52)
53+
supports both 429 and 529 and explicit backoff configuration.
54+
55+
The published wheel's PyPI SHA256 is
56+
`0a961148187d52e18276ed7f2d02617cfac48e3b97673cb631a8749397d43d1e`;
57+
the distribution metadata is available at
58+
[PyPI 0.7.2](https://pypi.org/pypi/typesafe-sdk/0.7.2/json).
59+
Acceptance installs the published wheel in a private uv cache with source builds
60+
disabled. It does not install a new host-stack component.
61+
62+
Dependency-resolution limitation: the install hint pins the SDK only, while its
63+
[dependency declarations:35-40](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/pyproject.toml#L35)
64+
allow dependency ranges. The tested environment used `httpx2==2.13.1`,
65+
`pydantic==2.13.5` and `tenacity==9.1.4`, as retained in
66+
`evidence/artifacts/typesafe-sdk-transport-20261005/checks.json`. Pinning that
67+
tested dependency set in the hint is deferred; another resolution may select
68+
versions outside this observed set. This task does not qualify every supported
69+
dependency resolution.
70+
71+
## Retry and response metadata contract
72+
73+
| Setting | Legacy urllib transport | SDK defaults | Selected SDK policy |
74+
| --- | --- | --- | --- |
75+
| Total attempts | 5 | 3 | 5 (`max_retries=4`) |
76+
| Retried statuses | 429, 529 | 408, 429, all 5xx | 429, 529 |
77+
| Delay before retries | 1, 2, 4, 8 seconds | Initial 0.5, cap 5, jitter 0.25 | 1, 2, 4, 8; no jitter |
78+
| Retry-After | Ignored | Honored | Ignored |
79+
| Connection/timeout retries | Disabled | Enabled | Disabled |
80+
| Overall retry budget | None | 30 seconds | None |
81+
| Per-attempt HTTP timeout | 60 seconds | SDK default | 60 seconds |
82+
83+
The existing retry field remains the zero-based successful attempt number,
84+
obtained from the SDK's own
85+
[X-TypeSafe-Retry-Count:68-74](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/transport.py#L68)
86+
on the public originating request. Its absence on the first attempt means zero.
87+
88+
Two transport details change explicitly. The SDK sets `User-Agent` to
89+
`typesafe-sdk/0.7.2`, replacing `agent-lab-gap-crosswalk/1 (python-urllib)`;
90+
its [header construction:116-127](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/src/typesafe_sdk/_core/transport.py#L116)
91+
overwrites a caller-supplied User-Agent. Accept the vendor's header contract.
92+
The latency field now uses the successful response's public `elapsed` HTTP
93+
exchange duration, excluding SDK decoding. The legacy timer also covered final
94+
JSON decoding. Both exclude earlier attempts and backoff; existing ledger bytes
95+
remain unchanged. The public dependency contract is
96+
[pydantic/[email protected], d426e2c4:_models.py:579-604](https://github.com/pydantic/httpx2/blob/d426e2c44d1247fcf3326ed1cb496e56353efa4a/src/httpx2/httpx2/_models.py#L579).
97+
98+
## Alternatives and overturn condition
99+
100+
Keeping the urllib loop duplicates the supported vendor implementation. Using
101+
SDK defaults changes the five-attempt retry contract. A wrapper or fork to
102+
override vendor headers or count retries adds unnecessary transport glue.
103+
Use the public client and its metadata directly.
104+
105+
Keep the prior glue if a prerequisite fails at the selected pin. Revisit this
106+
decision if a future supported SDK cannot preserve structured questions,
107+
probabilities, usage, request metadata or the retry policy. A comparison through
108+
the upstream MockTransport seam must establish any proposed replacement before
109+
another transport change.
110+
111+
## Acceptance boundary and completeness critic
112+
113+
`tests/test_gap_crosswalk_sdk.py` uses the upstream
114+
[MockTransport seam:33](https://github.com/typesafe-ai/typesafe-sdk-python/blob/f078f1e208a0d885154dc758344ae4fce77ac168/tests/conftest.py#L33)
115+
and the vendor's unchanged parser and retry implementation. These are synthetic
116+
integration checks, not an unchanged upstream suite or a live judgment. They
117+
comprise six MockTransport tests plus three offline import and version checks.
118+
They cover structured questions, usage/probabilities/request metadata, both retry
119+
statuses, exhaustion, a non-retried 500, absent request ID and missing usage.
120+
Wrong probability keys still fail the unchanged validator. Streamed mock
121+
responses exercise the native client's HTTP elapsed metadata. The first fixture
122+
used preloaded responses without elapsed metadata and failed three tests; that
123+
failed attempt and the fixture correction are retained in
124+
`evidence/artifacts/typesafe-sdk-transport-20261005/checks.json`.
125+
Offline import checks cover absent and mismatched SDK installations. Independent
126+
review caught that an initial install hint omitted `judge`'s required `--set`
127+
and `--out` flags. The final hint asks users to prefix their original command,
128+
preserving those flags and their output path; the guard test checks that hint.
129+
130+
The source review also checked the omitted contracts: optional metadata,
131+
retry-count provenance, per-attempt timing, fixed endpoint, SDK-owned headers
132+
and the offline CI build. Native service behavior remains unmeasured because
133+
this task authorizes no live judge run. `request_body()`, `validate()`, the
134+
threshold, ledger construction and recorded judgments are preserved.
Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
{
2+
"id": "typesafe-sdk-transport-20261005",
3+
"date": "2026-10-05",
4+
"base_commit": "f946c6d4ca988a17b6fa4392ecb488909f147883",
5+
"purpose": "Replace only the landscape crosswalk's transport and retry with the public vendor SDK after all compatibility gates pass.",
6+
"sources": {
7+
"repository": "https://github.com/typesafe-ai/typesafe-sdk-python",
8+
"tag": "v0.7.2",
9+
"commit": "f078f1e208a0d885154dc758344ae4fce77ac168",
10+
"registry": "https://pypi.org/pypi/typesafe-sdk/0.7.2/json",
11+
"wheel": "typesafe_sdk-0.7.2-py3-none-any.whl",
12+
"published_sha256": "0a961148187d52e18276ed7f2d02617cfac48e3b97673cb631a8749397d43d1e",
13+
"downloaded_sha256": "0a961148187d52e18276ed7f2d02617cfac48e3b97673cb631a8749397d43d1e",
14+
"download_exit_code": 0,
15+
"sha256sum_exit_code": 0,
16+
"compatibility_gates": {
17+
"structured_questions": "pass; src/typesafe_sdk/_core/question_types.py:39-49 and questions.py:10-23",
18+
"response_metadata": "pass; src/typesafe_sdk/_core/response_types.py:65-120, schemas/base.py:81-94 and src/typesafe_sdk/_schemas/models.py:25-30",
19+
"retry_policy": "pass; src/typesafe_sdk/_core/retry.py:52-123"
20+
},
21+
"evidence_class": "source_review"
22+
},
23+
"environment": {
24+
"installation": "isolated uv --with environment; --no-build, --no-config and --no-env-file; scratch and cache outside /tmp",
25+
"installed_packages": {
26+
"typesafe-sdk": "0.7.2",
27+
"httpx2": "2.13.1",
28+
"httpcore2": "2.13.1",
29+
"pydantic": "2.13.5",
30+
"pydantic_core": "2.46.5",
31+
"tenacity": "9.1.4",
32+
"typing_extensions": "4.16.0",
33+
"typing-inspection": "0.4.4",
34+
"anyio": "4.15.1",
35+
"h11": "0.16.0",
36+
"annotated-types": "0.8.0",
37+
"idna": "3.20",
38+
"truststore": "0.10.4"
39+
}
40+
},
41+
"checks": [
42+
{
43+
"command": "python3 tools/sota-convergence/gap_crosswalk.py build --check",
44+
"phase": "baseline with SDK absent",
45+
"exit_code": 0,
46+
"returned_stdout": "crosswalk up to date\n",
47+
"returned_stderr": "",
48+
"evidence_class": "local_integration"
49+
},
50+
{
51+
"command": "uv run --no-project --no-config --no-env-file --no-build --no-managed-python --no-python-downloads --default-index https://pypi.org/simple --with typesafe-sdk==0.7.2 python3 -m unittest discover -s tests -p test_gap_crosswalk_sdk.py -v",
52+
"phase": "first stub fixture",
53+
"exit_code": 1,
54+
"returned_summary": "Ran 8 tests in 0.324s; FAILED (errors=3)",
55+
"failure": "test_missing_request_id_stays_optional, test_missing_usage_remains_invalid, and test_structured_questions_metadata_and_final_attempt_retry_count failed at raw.elapsed: RuntimeError: '.elapsed' may only be accessed after the response has been read or closed.",
56+
"correction": "The preloaded MockTransport fixture had no elapsed metadata. A subsequent explicit synthetic elapsed fixture passed all eight tests (exit 0). The final fixture instead streams its response through native ByteStream so the native client read/close sets elapsed; production transport code is unchanged by this correction.",
57+
"evidence_class": "synthetic"
58+
},
59+
{
60+
"command": "python3 -S tools/sota-convergence/gap_crosswalk.py build --check",
61+
"phase": "changed transport with site packages disabled",
62+
"exit_code": 0,
63+
"returned_stdout": "crosswalk up to date\n",
64+
"returned_stderr": "",
65+
"evidence_class": "local_integration"
66+
},
67+
{
68+
"command": "uv run --no-project --no-config --no-env-file --no-build --no-managed-python --no-python-downloads --offline --with typesafe-sdk==0.7.2 python3 -m unittest discover -s tests -p 'test_gap_crosswalk*.py' -v",
69+
"phase": "final streamed SDK stub plus existing crosswalk and routing tests",
70+
"exit_code": 0,
71+
"returned_summary": "Ran 22 tests in 0.439s; OK",
72+
"returned_output_path": "evidence/artifacts/typesafe-sdk-transport-20261005/unittest-output.txt",
73+
"skipped": 0,
74+
"new_sdk_tests": 9,
75+
"evidence_class": "synthetic"
76+
},
77+
{
78+
"command": "python3 tools/sota-convergence/gap_crosswalk.py build --check",
79+
"phase": "changed transport with ordinary standard-library interpreter",
80+
"exit_code": 0,
81+
"returned_stdout": "crosswalk up to date\n",
82+
"returned_stderr": "",
83+
"evidence_class": "local_integration"
84+
}
85+
],
86+
"preservation": {
87+
"byte_identical_functions": ["request_body", "validate", "judge", "build"],
88+
"threshold": 0.3,
89+
"recorded_judgments_changed": false,
90+
"catalog_ledger_changed": false,
91+
"user_agent": "SDK-owned typesafe-sdk/0.7.2; explicit authorized change from agent-lab-gap-crosswalk/1 (python-urllib)",
92+
"latency_s": "Final successful HTTP exchange; excludes prior attempts, backoff and SDK decoding. Prior timer also included final JSON decoding."
93+
},
94+
"independent_review_correction": "The first missing-SDK hint omitted required judge --set/--out arguments. Final hint asks to prefix the original judge command, preserving its flags/output path; strengthened guard test passes.",
95+
"limitations": [
96+
"No live judge call, provider acceptance, credential value read or new judgment.",
97+
"Stub tests are locally authored synthetic integration checks through the unchanged SDK implementation; they are not an unchanged upstream suite.",
98+
"Stub timing does not establish live service latency.",
99+
"Source gates establish compatibility at the selected pin, not live service correctness.",
100+
"Repository validation checks integrity and declared consistency; its final exit code is reported in the draft PR and private lane status after the registry commit."
101+
]
102+
}
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
test_build_check_is_up_to_date (test_gap_crosswalk.GapCrosswalkTests.test_build_check_is_up_to_date) ... ok
2+
test_build_rejects_candidates_outside_the_rule (test_gap_crosswalk.GapCrosswalkTests.test_build_rejects_candidates_outside_the_rule) ... ok
3+
test_build_rejects_receipt_content_changed_since_judging (test_gap_crosswalk.GapCrosswalkTests.test_build_rejects_receipt_content_changed_since_judging) ... ok
4+
test_every_gap_of_the_source_revision_is_covered (test_gap_crosswalk.GapCrosswalkTests.test_every_gap_of_the_source_revision_is_covered) ... ok
5+
test_every_layer_has_one_known_owner (test_gap_crosswalk.GapCrosswalkTests.test_every_layer_has_one_known_owner) ... ok
6+
test_judgment_and_review_files_match_their_recorded_hashes (test_gap_crosswalk.GapCrosswalkTests.test_judgment_and_review_files_match_their_recorded_hashes) ... ok
7+
test_no_host_paths_or_session_identifiers (test_gap_crosswalk.GapCrosswalkTests.test_no_host_paths_or_session_identifiers) ... ok
8+
test_status_follows_the_final_decisions (test_gap_crosswalk.GapCrosswalkTests.test_status_follows_the_final_decisions) ... ok
9+
test_a3_a_gap_with_a_digit_sends_every_pair_to_review (test_gap_crosswalk_routing.CrosswalkRoutingTests.test_a3_a_gap_with_a_digit_sends_every_pair_to_review) ... ok
10+
test_a7_a_gap_with_no_reviewed_pair_is_not_open (test_gap_crosswalk_routing.CrosswalkRoutingTests.test_a7_a_gap_with_no_reviewed_pair_is_not_open) ... ok
11+
test_a7_a_pair_no_reviewer_saw_has_no_final_decision (test_gap_crosswalk_routing.CrosswalkRoutingTests.test_a7_a_pair_no_reviewer_saw_has_no_final_decision) ... ok
12+
test_negative_control_the_2026_09_23_rule_fails_both_checks (test_gap_crosswalk_routing.CrosswalkRoutingTests.test_negative_control_the_2026_09_23_rule_fails_both_checks) ... ok
13+
test_each_rule_on_the_retained_inputs (test_gap_crosswalk_routing.RetainedLedgerRoutingTests.test_each_rule_on_the_retained_inputs) ... ok
14+
test_missing_sdk_fails_with_pinned_install_command (test_gap_crosswalk_sdk.OptionalSDKTests.test_missing_sdk_fails_with_pinned_install_command) ... ok
15+
test_module_import_does_not_load_sdk (test_gap_crosswalk_sdk.OptionalSDKTests.test_module_import_does_not_load_sdk) ... ok
16+
test_other_sdk_version_fails_before_transport (test_gap_crosswalk_sdk.OptionalSDKTests.test_other_sdk_version_fails_before_transport) ... ok
17+
test_missing_request_id_stays_optional (test_gap_crosswalk_sdk.SDKTransportTests.test_missing_request_id_stays_optional) ... ok
18+
test_missing_usage_remains_invalid (test_gap_crosswalk_sdk.SDKTransportTests.test_missing_usage_remains_invalid) ... ok
19+
test_non_retryable_status_stops_immediately (test_gap_crosswalk_sdk.SDKTransportTests.test_non_retryable_status_stops_immediately) ... ok
20+
test_retries_stop_after_five_attempts_with_legacy_backoff (test_gap_crosswalk_sdk.SDKTransportTests.test_retries_stop_after_five_attempts_with_legacy_backoff) ... ok
21+
test_structured_questions_metadata_and_final_attempt_retry_count (test_gap_crosswalk_sdk.SDKTransportTests.test_structured_questions_metadata_and_final_attempt_retry_count) ... ok
22+
test_wrong_probability_keys_remain_invalid (test_gap_crosswalk_sdk.SDKTransportTests.test_wrong_probability_keys_remain_invalid) ... ok
23+
24+
----------------------------------------------------------------------
25+
Ran 22 tests in 0.439s
26+
27+
OK

‎manifests/evidence.json‎

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16682,6 +16682,11 @@
1668216682
"sha256": "d7698470de9eb7ed30f0764f8b1b17acd8a926298b08630829e1b955ba177dfe",
1668316683
"bytes": 23344
1668416684
},
16685+
{
16686+
"path": "docs/decisions/2026-10-05-typesafe-sdk-transport.md",
16687+
"sha256": "d197ab297186fe55e6c83624c78a06ead264e00d66a7fe2d5e62cfc1b653dbd6",
16688+
"bytes": 9226
16689+
},
1668516690
{
1668616691
"path": "docs/direct-results.md",
1668716692
"sha256": "12a511fee11f22c90b9b622701d5485be9c8247167102b5768d694244f939a5b",
@@ -49552,6 +49557,16 @@
4955249557
"sha256": "189a1b80b559e74fc12d0b7e73f6415e4829041bfe8f03d89ca5a18fb11be4f5",
4955349558
"bytes": 30252
4955449559
},
49560+
{
49561+
"path": "evidence/artifacts/typesafe-sdk-transport-20261005/checks.json",
49562+
"sha256": "0c2d0c3a40a5f4fdfbf707d30bc49a9d58cddbd01fdad8965fcb495248f312ac",
49563+
"bytes": 5642
49564+
},
49565+
{
49566+
"path": "evidence/artifacts/typesafe-sdk-transport-20261005/unittest-output.txt",
49567+
"sha256": "a2c5d1bde040183fcd96890e3ce1b5dce70197478942641dce45ca75976ddb15",
49568+
"bytes": 3275
49569+
},
4955549570
{
4955649571
"path": "evidence/artifacts/u11-part1-source-repair-20261001/integration-main-20261002.json",
4955749572
"sha256": "6382ce9bc43e18e5797b0e0c7df0f2b76f2485ca6a908e3b7aba13ec1c1eb638",
@@ -53677,6 +53692,11 @@
5367753692
"sha256": "3fee83f89033d101c71cbfcdb4eb2efb12166a459ae80f7c9a96cb58bb5937aa",
5367853693
"bytes": 13100
5367953694
},
53695+
{
53696+
"path": "tests/test_gap_crosswalk_sdk.py",
53697+
"sha256": "5bdf42e3dbd407909c1d21a1e8314d2f9d5b19623f26d8ee1d86689bf3184dda",
53698+
"bytes": 7316
53699+
},
5368053700
{
5368153701
"path": "tests/test_gap_resolution_ledger.py",
5368253702
"bytes": 5845,
@@ -54594,8 +54614,8 @@
5459454614
},
5459554615
{
5459654616
"path": "tools/sota-convergence/gap_crosswalk.py",
54597-
"sha256": "7545a9aa0a709aaca636e69b6ca7a1d6d9e317a53027098df7133e952d4f705f",
54598-
"bytes": 30150
54617+
"sha256": "77f9c58561e656dfa6b42a346ed6063d7a3716089c2900c84808e0527594c22c",
54618+
"bytes": 30850
5459954619
},
5460054620
{
5460154621
"path": "tools/sota-convergence/gap_wave_ledger.py",

0 commit comments

Comments
 (0)