Skip to content

Commit 833ed99

Browse files
committed
fix(sessions): preserve turns persisted after a response's recorded boundary
previous_response_id compaction covers the server side history through one response only, but the replacement classified everything in the snapshot as the compacted baseline. When another run appended its turn between that response's persisted batch and the late snapshot, the prefix and generation checks saw no rewrite and the replacement dropped the newer turn. The runner now persists each response's batch through a session hook that records the exact local item count in the same locked region as the append, pairing the response id with its ownership boundary. run_compaction reads the recorded boundary under the lock and preserves every item past it, so turns appended after the batch survive no matter when the snapshot happens. Without a recorded boundary, direct calls and input mode keep the snapshot length as the boundary, since their compaction input is captured under the same lock as the snapshot. clear_session and pop_item drop recorded boundaries the same way they already invalidate the caches, so a destroyed history can never feed a stale count into a replacement. A successful replacement translates the surviving boundaries instead of clearing them, shifting each count past the rewritten prefix onto the new history, so overlapping previous_response_id compactions still preserve the turns past their own boundaries. A boundary that ended inside the rewritten prefix has no counterpart afterwards; it is kept as a tombstone and a later compaction keyed on it skips its replacement, because falling back to its snapshot would classify newer turns and the earlier summary into its baseline and drop them. Resumed pending session writes route through the same hook when the resume itself performs the append and a response id is known, so those batches record boundaries too. The new regressions drive the exact orderings from review: a turn persisted between a response's batch and its late snapshot survives that response's replacement, a second compaction lands on its translated boundary after an overlapping replacement and keeps the newer turn, and a compaction whose recorded prefix was rewritten away skips instead of dropping the newer turn and the earlier summary. Without the source change each ordering loses items.
1 parent 49f34b9 commit 833ed99

3 files changed

Lines changed: 400 additions & 22 deletions

File tree

‎src/agents/memory/openai_responses_compaction_session.py‎

Lines changed: 109 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,10 @@
2727

2828
DEFAULT_COMPACTION_THRESHOLD = 10
2929
_ALL_SESSION_ITEMS_LIMIT = 2_147_483_647
30+
# Recorded response boundaries are pruned oldest first past this size so ids whose
31+
# compaction never runs (for example turns whose compaction was deferred) cannot
32+
# grow the map without bound.
33+
_MAX_RECORDED_RESPONSE_BOUNDARIES = 50
3034

3135
OpenAIResponsesCompactionMode = Literal["previous_response_id", "input", "auto"]
3236

@@ -137,6 +141,16 @@ def __init__(
137141
self._compaction_candidate_items: list[TResponseInputItem] | None = None
138142
self._session_items: list[TResponseInputItem] | None = None
139143
self._response_id: str | None = None
144+
# Each response id is paired with the exact local item count at the moment
145+
# its batch was persisted, recorded under the mutation lock. A
146+
# previous_response_id compaction covers the server side history through
147+
# that response only, so the replacement must preserve every local item
148+
# past the recorded boundary, not past a snapshot taken later. A None
149+
# value is a tombstone: the response was recorded, but a later
150+
# replacement rewrote the prefix its boundary counted, so no count on
151+
# the current history describes that response's coverage and a
152+
# compaction keyed on it must skip instead of guessing.
153+
self._response_boundaries: dict[str, int | None] = {}
140154
self._deferred_response_id: str | None = None
141155
self._last_unstored_response_id: str | None = None
142156
# Serialize wrapper mutations against compaction snapshot/replace/restore so a
@@ -232,6 +246,33 @@ async def run_compaction(
232246
)
233247
return
234248

249+
# The boundary recorded when this response's batch was persisted is
250+
# the authority on what previous_response_id compaction covers. The
251+
# snapshot below can already contain turns another run appended after
252+
# that persist, so it only anchors the divergence check and never the
253+
# ownership boundary. clear_session and pop_item drop recorded
254+
# boundaries outright and a successful replacement translates the
255+
# survivors, so a stale count can never be read here. A tombstone
256+
# means an earlier replacement rewrote the prefix this boundary
257+
# counted; the snapshot fallback would classify turns persisted
258+
# after this response into its baseline and drop them, so skip. An
259+
# id with no entry was never persisted through the response hook;
260+
# such direct callers persist before compacting and own that
261+
# ordering, which keeps the snapshot fallback sound for them.
262+
recorded_boundary: int | None = None
263+
if resolved_mode == "previous_response_id" and response_id is not None:
264+
if response_id in self._response_boundaries:
265+
recorded_boundary = self._response_boundaries[response_id]
266+
if recorded_boundary is None:
267+
logger.warning(
268+
"Skipped compaction for %s (mode=%s): an earlier replacement "
269+
"rewrote the history prefix this response's boundary counted, "
270+
"so no boundary on the current history describes its coverage.",
271+
response_id,
272+
resolved_mode,
273+
)
274+
return
275+
235276
# Capture the full stored history while the lock still excludes writers.
236277
# It anchors the post-flight check that detects concurrent mutations.
237278
snapshot_items = await self._get_all_underlying_session_items()
@@ -283,9 +324,15 @@ async def run_compaction(
283324
resolved_mode,
284325
)
285326
return
286-
# Items appended concurrently while the request was in flight sit past
287-
# the snapshot; carry them over so the replacement cannot drop them.
288-
concurrent_tail = previous_items[baseline_count:]
327+
# Preserve every item past the recorded boundary for this response.
328+
# Turns another run appended between this response's persisted batch
329+
# and the snapshot are not covered by previous_response_id compaction,
330+
# so they must survive alongside items appended while the request was
331+
# in flight. Without a recorded boundary (direct run_compaction calls,
332+
# or input mode where the compaction input was captured under the same
333+
# lock as the snapshot) the snapshot length is the boundary.
334+
preserve_from = recorded_boundary if recorded_boundary is not None else baseline_count
335+
concurrent_tail = previous_items[preserve_from:]
289336
await self._replace_underlying_session_items(
290337
output_items=output_items + concurrent_tail,
291338
previous_items=previous_items,
@@ -298,6 +345,22 @@ async def run_compaction(
298345
# a concurrent tail. With a non empty snapshot the prefix check would
299346
# catch that; when both snapshots were empty only the counter can.
300347
self._destructive_generation += 1
348+
# Translate every recorded boundary onto the rewritten history. The
349+
# replacement rewrote previous_items[:preserve_from] into output_items
350+
# and kept the tail, so a boundary at or past preserve_from still
351+
# counts the same persisted batch at its shifted position, and an
352+
# overlapping compaction keyed on that response stays sound. A
353+
# boundary inside the rewritten prefix has no counterpart in the new
354+
# history, and any count would claim newer items for that response;
355+
# keep a tombstone so its compaction skips instead.
356+
self._response_boundaries = {
357+
rid: (
358+
boundary - preserve_from + len(output_items)
359+
if boundary is not None and boundary >= preserve_from
360+
else None
361+
)
362+
for rid, boundary in self._response_boundaries.items()
363+
}
301364

302365
logger.debug(
303366
"compact: done for %s (mode=%s, output=%s, candidates=%s, concurrent_tail=%s)",
@@ -457,21 +520,45 @@ def _clear_deferred_compaction(self) -> None:
457520

458521
async def add_items(self, items: list[TResponseInputItem]) -> None:
459522
async with self._mutation_lock:
460-
try:
461-
await self.underlying_session.add_items(items)
462-
except (Exception, asyncio.CancelledError):
463-
# The backend may have committed before acknowledgement failed. Re-read its
464-
# authoritative history before compaction instead of retaining a stale cache.
465-
self._compaction_candidate_items = None
466-
self._session_items = None
467-
raise
468-
if self._compaction_candidate_items is not None:
469-
new_items = _normalize_compaction_session_items(items)
470-
new_candidates = select_compaction_candidate_items(new_items)
471-
if new_candidates:
472-
self._compaction_candidate_items.extend(new_candidates)
473-
if self._session_items is not None:
474-
self._session_items.extend(_normalize_compaction_session_items(items))
523+
await self._add_items_locked(items)
524+
525+
async def _add_items_for_response(
526+
self, items: list[TResponseInputItem], *, response_id: str
527+
) -> None:
528+
"""Append a response's persisted batch and record its compaction boundary.
529+
530+
The runner calls this instead of add_items when the batch belongs to a
531+
specific response. Recording the boundary in the same locked region as
532+
the append keeps the pairing exact: no other writer can slip items in
533+
between the batch and the count recorded for it, so run_compaction can
534+
later preserve everything past the boundary regardless of what other
535+
runs appended before its own snapshot.
536+
"""
537+
async with self._mutation_lock:
538+
await self._add_items_locked(items)
539+
boundary = len(await self._get_all_underlying_session_items())
540+
self._response_boundaries.pop(response_id, None)
541+
self._response_boundaries[response_id] = boundary
542+
while len(self._response_boundaries) > _MAX_RECORDED_RESPONSE_BOUNDARIES:
543+
del self._response_boundaries[next(iter(self._response_boundaries))]
544+
545+
async def _add_items_locked(self, items: list[TResponseInputItem]) -> None:
546+
try:
547+
await self.underlying_session.add_items(items)
548+
except (Exception, asyncio.CancelledError):
549+
# The backend may have committed before acknowledgement failed. Read its
550+
# authoritative history again before compaction instead of retaining a
551+
# stale cache.
552+
self._compaction_candidate_items = None
553+
self._session_items = None
554+
raise
555+
if self._compaction_candidate_items is not None:
556+
new_items = _normalize_compaction_session_items(items)
557+
new_candidates = select_compaction_candidate_items(new_items)
558+
if new_candidates:
559+
self._compaction_candidate_items.extend(new_candidates)
560+
if self._session_items is not None:
561+
self._session_items.extend(_normalize_compaction_session_items(items))
475562

476563
async def pop_item(self) -> TResponseInputItem | None:
477564
async with self._mutation_lock:
@@ -480,6 +567,9 @@ async def pop_item(self) -> TResponseInputItem | None:
480567
self._compaction_candidate_items = None
481568
self._session_items = None
482569
self._destructive_generation += 1
570+
# Recorded boundaries are item counts, so removing an item makes
571+
# them stale even when the pop only trimmed the tail.
572+
self._response_boundaries.clear()
483573
return popped
484574

485575
async def clear_session(self) -> None:
@@ -489,6 +579,7 @@ async def clear_session(self) -> None:
489579
self._session_items = []
490580
self._deferred_response_id = None
491581
self._destructive_generation += 1
582+
self._response_boundaries.clear()
492583

493584
async def _ensure_compaction_candidates(
494585
self,

‎src/agents/run_internal/session_persistence.py‎

Lines changed: 48 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -662,9 +662,29 @@ async def save_result_to_session(
662662
resumed_write_state._current_turn_persisted_item_count + saved_run_items_count
663663
),
664664
}
665-
await resume_pending_session_write(resumed_write_state, session, wrapper=wrapper)
665+
await resume_pending_session_write(
666+
resumed_write_state, session, response_id=response_id, wrapper=wrapper
667+
)
666668
else:
667-
await _session_add_items(session, items_to_save, wrapper=wrapper)
669+
add_items_for_response = (
670+
getattr(session, "_add_items_for_response", None)
671+
if response_id and is_openai_responses_compaction_aware_session(session)
672+
else None
673+
)
674+
if callable(add_items_for_response):
675+
# Persisting this response's batch is the only moment its response id
676+
# and its exact local item boundary coincide, so record the pairing
677+
# here. A later previous_response_id compaction preserves everything
678+
# past the recorded boundary, including turns other runs append
679+
# before that compaction takes its own snapshot.
680+
await _call_session_method(
681+
add_items_for_response,
682+
items_to_save,
683+
response_id=response_id,
684+
wrapper=wrapper,
685+
)
686+
else:
687+
await _session_add_items(session, items_to_save, wrapper=wrapper)
668688

669689
if run_state is not None:
670690
run_state._current_turn_persisted_item_count = already_persisted + saved_run_items_count
@@ -751,13 +771,19 @@ async def resume_pending_session_write(
751771
run_state: RunState,
752772
session: Session | None,
753773
*,
774+
response_id: str | None = None,
754775
wrapper: RunContextWrapper[Any] | None = None,
755776
) -> None:
756777
"""Settle a resumed output batch before allowing further model work.
757778
758779
The application must supply the original backend and serialize access to its history,
759780
including independently restored RunState copies. Session has no distributed compare-and-swap
760781
or backend identity contract. A changed tail is not repaired or searched for similar items.
782+
783+
When a response_id is supplied and this call performs the append itself, the batch is
784+
persisted through the compaction boundary hook so the response id is paired with its
785+
item count. The serialized pending write carries no response id, so resumes from a
786+
restored RunState leave it unset and record no boundary.
761787
"""
762788
pending = run_state._pending_session_write
763789
if pending is None:
@@ -801,7 +827,26 @@ def digests(items: Sequence[TResponseInputItem]) -> list[str]:
801827
append = unchanged
802828
if append:
803829
# Backends may retain or transform their input; the durable checkpoint stays detached.
804-
await _session_add_items(session, copy.deepcopy(pending["items"]), wrapper=wrapper)
830+
items_to_append = copy.deepcopy(pending["items"])
831+
add_items_for_response = (
832+
getattr(session, "_add_items_for_response", None)
833+
if response_id and is_openai_responses_compaction_aware_session(session)
834+
else None
835+
)
836+
if callable(add_items_for_response):
837+
# Only the branch that appends can record a boundary: the hook pairs
838+
# the response id with the item count in one locked region. When an
839+
# earlier attempt already committed the batch, other writers may
840+
# have appended since, so any count read now could claim their
841+
# items for this response; no boundary is recorded in that case.
842+
await _call_session_method(
843+
add_items_for_response,
844+
items_to_append,
845+
response_id=response_id,
846+
wrapper=wrapper,
847+
)
848+
else:
849+
await _session_add_items(session, items_to_append, wrapper=wrapper)
805850
run_state._current_turn_persisted_item_count = pending["persisted_count"]
806851
run_state._pending_session_write = None
807852
finally:

0 commit comments

Comments
 (0)