Repository navigation
Python: [Bug]: Checkpoint encoding silently loses values when dictionary keys collide after string conversion #8256
Description
Description
Checkpoint encoding converts every dictionary key with str(k) before writing JSON. Distinct Python keys can therefore collapse to the same JSON key and silently overwrite checkpoint state. For example, 1 and "1" are distinct dictionary keys, but both encode as "1".
The existing test documents conversion of ordinary non-string keys to strings, so this report is specifically about collision and silent data loss rather than requesting an unconditional behavior change for all non-string keys.
Expected behavior: checkpoint encoding should either preserve a dictionary whose keys collide after string conversion (for example, by using the existing pickle envelope for that dictionary) or reject it explicitly. It should not silently discard a value.
I would like maintainer guidance on the compatibility approach before submitting a PR. A narrow option is to pickle only dictionaries for which len({str(k) for k in value}) != len(value), preserving the existing behavior for non-colliding keys.
Code Sample
import asyncio
import tempfile
from agent_framework import FileCheckpointStorage, WorkflowCheckpoint
async def main() -> None:
with tempfile.TemporaryDirectory() as directory:
storage = FileCheckpointStorage(directory)
checkpoint = WorkflowCheckpoint(
workflow_name="wf",
graph_signature_hash="hash",
state={"mapping": {1: "integer-key", "1": "string-key"}},
)
await storage.save(checkpoint)
restored = await storage.load(checkpoint.checkpoint_id)
print(checkpoint.state["mapping"])
print(restored.state["mapping"])
asyncio.run(main())Output:
{1: 'integer-key', '1': 'string-key'}
{'1': 'string-key'}
Error Messages / Stack Traces
No exception or warning is emitted; one value is silently lost.
Package Versions
agent-framework-core: source checkout of main at d7823b2
Python Version
Python 3.11.9
Additional Context
The module describes the hybrid format as providing full Python object fidelity for data values, and it already pickles dictionaries containing reserved marker keys. Reusing the pickle envelope only for string-key collisions would avoid introducing a new serialization format.
AI assistance was used to audit the encoding path and prepare the reproduction; the round-trip result was verified against the referenced source checkout.
Metadata
Metadata
Labels
Type
Projects
- Status
Activity
github-actions commentedon Sep 10, 2026
🤖 Automated triage reproduction notes (agent-authored — trust but verify)
Agent analysis
The bug occurs in
python/packages/core/agent_framework/_workflows/_checkpoint_encoding.py::_encodeat line 318 when distinct dictionary keys produce the same string. The dictionary comprehension silently retains only the later value. Minimal repro: round-trip{1: "integer-key", "1": "string-key"}throughencode_checkpoint_valueanddecode_checkpoint_valueand assert equality.python/packages/core/tests/workflow/test_checkpoint_encode.pymain@d7823b21.18.0fix(python): pickle checkpoint dicts with colliding str keys
he-yufeng commentedon Sep 14, 2026
Verified on current main (0ff9e40):
_encodeinpython/packages/core/agent_framework/_workflows/_checkpoint_encoding.pystill doesencoded_dict = {str(k): _encode(v) for k, v in typed_dict.items()}, so1and"1"collide into one JSON key and the earlier value is silently overwritten.On the compatibility question you asked about: the narrow shape you sketched is the right one, and it has an in-tree precedent. The encoder already pickles a dict wholesale when any key stringifies into
_RESERVED_DICT_KEYS(__pickled__/__type__). Extending that trigger tolen({str(k) for k in value}) != len(value)keeps every existing checkpoint byte-identical (non-colliding dicts take the plain path exactly as today) and only collisions take the envelope. On decode, the envelope preserves the original key types, while checkpoints written before the fix decode as they always did — the already-collapsed value stays collapsed, which is the honest outcome for data written with the loss.One nuance worth pinning in the PR's tests: the whole dict rides in one envelope, so a custom object nested under a colliding dict is unpickled under the restricted loader's allowed types just as it would be standalone — same security posture, but worth a test with
allowed_typesso nobody "fixes" that later by widening it.Happy to leave this to you since you said you wanted the PR. If you'd rather not carry it, say the word and I'll pick it up.
Python: fix(checkpoint): preserve dicts whose keys collide after str() (
ktz03 commentedon Sep 15, 2026
Fixed by #8272 (merged). The merged PR body mistakenly said
Fixes #8200; this issue (#8256) was the intended target.he-yufeng commentedon Sep 15, 2026
Confirming from the merged diff: #8272 resolves this exactly along the lines analyzed above. Stringified-key collisions now fall back to pickle encoding (so
{1: "a", "1": "b"}survives the round trip), and each key is stringified once and reused across the reserved-key check, the collision check, and the encode pass, which removes the stateful-str disagreement between the three reads. The new tests in test_checkpoint_encode.py pin both the int/str collision and the pickle fallback. Nothing left on this one from my side; thanks for the quick pickup, and noting the body reference should read #8256 rather than #8200 for anyone tracing it later.eavanvalkenburg commentedon Sep 16, 2026
Resolved by #8272, which detects stringified dictionary-key collisions and preserves those dictionaries through the checkpoint pickle envelope.