Repository navigation
fix: honor checkpoint object in thread state updates - #7115
Conversation
Prefer the checkpoint object over the deprecated checkpoint_id, validate its thread_id, and preserve checkpoint namespace when updating state. Add regression coverage for checkpoint-object forks and thread mismatch.
|
@gmll-star, thanks for your contribution. Please click the CLA button to sign the CLA first, otherwise we cannot merge your code. |
willem-bd
left a comment
There was a problem hiding this comment.
Reviewed at head 155b439 (verified against merge-base 92e9afe).
The checkpoint-object fix itself checks out — I verified each claim independently against this head:
- The sync builders (
build_checkpoint_state_accessor,build_checkpoint_state_mutation_accessor) acceptcheckpoint_ns: str = ""as keyword-only params and carry it into the checkpointer config. checkpoint_nsis forwarded only when non-empty at every hop (router → mutation boundary → read/mutation builders →abuild_checkpoint_state_accessor), so existing test doubles that only acceptcheckpoint_idkeep working — 447 tests acrosstest_gateway_services.py+test_threads_router.pypass apart from the issue below.- The ns-aware
aget_tupleexistence check runs before any write, so an unknown (or trimmed, per #7041) selector still 404s explicitly instead of writing to head; thread-id mismatch → 400; the deprecated string form and object-wins precedence behave as #7077 specifies. - Both new tests pass at head and fail at base (I transplanted the head test file onto 92e9afe: both fail there — the fork silently parents to head).
So relative to #7100, this is the complete, working fix for #7077 — with one blocker that needs to be dropped before merge.
| else: | ||
| existing_runtime_context["client"] = existing_client_context | ||
|
|
||
| if not context: |
There was a problem hiding this comment.
This PR silently reverts the Gateway-side client-context handling from #7051, which has nothing to do with the checkpoint fix and is not mentioned in the description.
At this head, services.py deletes _sanitize_request_client_context, the configurable["client"] strip, the config['context']['client'] sanitization, and the body.context["client"] forwarding (the sanitize_client_context import is gone too — its only remaining callers are inside deerflow.runtime.client_context itself). Consequences at head:
body.config.configurable.clientis no longer stripped, so arbitrary caller-suppliedclientdata persists verbatim intoconfigurable— which is written to the checkpoint store and echoed by the run API. This is exactly what the removed code guarded against.body.config.context.clientreachesruntime.contextunsanitized (unbounded/nested caller data; the prompt block is still safe becauserender_client_context_for_promptre-sanitizes at render time, but the raw dict persists).- Top-level
body.context.clientis now silently dropped instead of forwarded sanitized, so the feat(context): support client context capabilities #7051 feature stops working for SDK clients (dynamic_context_middlewarereadsruntime.context["client"]and now finds nothing).
Hard evidence this is a regression, not a refactor: 9 tests in backend/tests/test_gateway_services.py — a file this PR does not touch — pass at merge-base 92e9afe and fail at this head (..._propagates_sanitized_client_context_to_runtime_only, ..._sanitizes_config_context_client_without_body_context, ..._drops_configurable_client, ..._logs_dropped_client_fields_without_values, ..._does_not_log_a_clean_client, and test_start_run_forwards_sanitized_client_context_to_runtime_context_only x4 params). I ran both sides to confirm.
The description's validation note ("Related gateway services tests passed (422 cases)") does not hold at 155b439. Please drop every services.py hunk unrelated to checkpoint_ns — including the unrelated run_after_metadata comment reword further down — so the PR contains only the checkpoint fix, which as noted above is correct and complete.
440a528 to
7f3563c
Compare
| "configurable": { | ||
| "thread_id": thread_id, | ||
| "checkpoint_ns": "", | ||
| "checkpoint_ns": checkpoint_ns, |
There was a problem hiding this comment.
[P2] Bind the mutation graph before using a nonempty checkpoint namespace
A valid checkpoint with a nonempty namespace passes the new existence check, but the subsequent accessor.aupdate() fails. LangGraph interprets checkpoint_ns as a subgraph path, while build_state_mutation_graph() contains only the mutation node and CheckpointStateAccessor.bind() does not provide the configurable checkpointer override needed to operate directly within that namespace.
I reproduced this with the locked LangGraph 1.2.9: persist a checkpoint by invoking a parent graph containing a child subgraph, then POST /api/threads/{thread_id}/state with that checkpoint's actual {thread_id, checkpoint_ns, checkpoint_id} selector and a title update. The lookup succeeds, but the request returns HTTP 500 (Failed to update thread state), with ValueError: Subgraph child not found from abulk_update_state. The same endpoint's root-namespace control succeeds.
Please make the mutation accessor support writes and readback in the selected namespace, and add a regression using a real persisted nonroot checkpoint that verifies the updated values and parent checkpoint ID. Both new tests currently use checkpoint_ns: "", so they miss this path.
willem-bd
left a comment
There was a problem hiding this comment.
Follow-up review at head 72e51b8 (one new commit, "fix: support state updates in checkpoint namespaces"), covering only what changed since 7f3563c. Both prior asks are resolved, verified at this head:
- The undeclared #7051 revert is gone from the diff —
services.pykeeps_sanitize_request_client_contextand all client-context handling, and all 9 previously failing tests intest_gateway_services.pypass again (I ran them at this head). - The namespace-write 500 from the earlier review is fixed:
direct_namespace=Truebinds the checkpointer viaCONFIG_KEY_CHECKPOINTERfor nonempty namespaces, and the newtest_update_thread_state_in_persisted_subgraph_namespaceregression (real persisted subgraph checkpoint, bothfullanddeltamodes, parent-checkpoint-id + root-untouched assertions) passes at this head and matches what was asked.
Two small new items below: one behavior gap in namespace-only selectors, one nit on the private LangGraph import.
| checkpoint_id = str(raw_checkpoint_id) if raw_checkpoint_id else raw_checkpoint_id | ||
| raw_checkpoint_ns = body.checkpoint.get("checkpoint_ns") | ||
| if raw_checkpoint_ns is not None: | ||
| checkpoint_ns = str(raw_checkpoint_ns) |
There was a problem hiding this comment.
[Suggestion] A checkpoint object with a nonempty checkpoint_ns but no checkpoint_id skips the if checkpoint_id is not None: existence check below, and with direct_namespace=True the write now succeeds into an arbitrary namespace. I probed this head: POST /api/threads/{id}/state with checkpoint: {"thread_id": ..., "checkpoint_ns": "child:ghost"} (no checkpoints exist in that namespace) returns 200 and materializes a brand-new checkpoint in child:ghost, confirmed via checkpointer.aget_tuple — root state stays intact, but the endpoint's 404 discipline for unknown selectors (the branch just below) does not apply to namespace-only selectors, and before this PR the state API could never create a nonroot namespace at all. Consider running the same ns-aware aget_tuple existence check whenever checkpoint_ns is nonempty, even without checkpoint_id, returning 404 when the namespace has no checkpoints.
| # A state-only mutation graph has no subgraphs to delegate to. Bind | ||
| # the saver explicitly so LangGraph reads/writes this namespace on | ||
| # the graph itself, including when reading back an updated config. | ||
| from langgraph._internal._constants import CONFIG_KEY_CHECKPOINTER |
There was a problem hiding this comment.
[Nit] This reaches into LangGraph's private internals (langgraph._internal._constants); the repo so far imports private LangGraph modules only from tests. It works at the locked langgraph >=1.2.9,<1.3 — I verified CONFIG_KEY_CHECKPOINTER resolves at 1.2.9 — and the deferred import confines any breakage to the nonempty-namespace mutation path, but a minor upgrade that moves the constant would turn subgraph-namespace state updates into runtime 500s. Worth a brief comment naming that version coupling so the next langgraph bump knows to grep for it.
willem-bd
left a comment
There was a problem hiding this comment.
Reviewed 3b9d2cc. The earlier checkpoint selection and namespace failures are addressed. I found one remaining P2 side effect in nonroot title updates, detailed inline.
Validation: 669 related checkpoint, Gateway service/router, checkpoint-mode, compaction, rollback and factory-offload tests passed. All seven new regressions fail against the original router as expected. Additional HTTP probes through the real accessor builders reproduce the title-isolation failure in both full and delta modes; an in-memory root-only title-sync guard makes those probes pass. Ruff lint/format, guidance checks and current CI passed. No production source files were changed during this review.
| checkpoint_id = str(raw_checkpoint_id) if raw_checkpoint_id else raw_checkpoint_id | ||
| raw_checkpoint_ns = body.checkpoint.get("checkpoint_ns") | ||
| if raw_checkpoint_ns is not None: | ||
| checkpoint_ns = str(raw_checkpoint_ns) |
There was a problem hiding this comment.
[P2] Keep child titles out of root conversation metadata
Now that a nonempty checkpoint_ns targets a child checkpoint, the later title synchronization at lines 1718–1725 also needs to respect that namespace. It currently calls thread_store.update_display_name(thread_id, new_title, ...) for every title update, so changing a child's title renames the root conversation in thread search results even though its root checkpoint was correctly left unchanged.
I reproduced this through the real thread/assistant/accessor builders with a persisted child subgraph in both full and delta modes: first name the root Root conversation, then POST a title update of forked with the child's {thread_id, checkpoint_ns, checkpoint_id} selector. The response is 200 and the child checkpoint has the correct selected parent; GET /api/threads/{id}/state still returns the root title Root conversation, but POST /api/threads/search now lists that conversation as forked. The root checkpoint itself is byte-for-byte unchanged.
Please restrict the update_display_name synchronization to the root namespace (not checkpoint_ns) and extend the namespace regression to assert that thread metadata/search titles remain unchanged after a child title update. A temporary in-memory version of that guard makes the failing full/delta probes pass without affecting the root controls.
Only synchronize state titles into conversation metadata for the root checkpoint namespace. Cover persisted child updates with and without checkpoint IDs in full and delta modes, and retain root title synchronization and branch sequence cleanup.
…rs/AGENTS.md skipped - file absent on fork base)
Summary
Validation
tests/test_gateway_services.py tests/test_threads_router.py: 456 passed.tests/blocking_io: 255 passed, 8 skipped.app/gateway/services.py.Fixes #7077
Non-root namespace review follow-up
tests/test_checkpoint_state.py tests/test_gateway_services.py tests/test_threads_router.py: 470 passed.tests/blocking_io/test_agent_factory_construction.py tests/blocking_io/test_tool_assembly_offloop.py: 9 passed.Namespace-only selector follow-up
tests/test_checkpoint_state.py tests/test_gateway_services.py tests/test_threads_router.py: 473 passed; Ruff check and format check passed for the three modified files.