Repository navigation
Conversation
Contributor
|
Thanks for the pull request. One thing worth sorting out before it is reviewed:
If you think it belongs here anyway, say so and a maintainer will take a look. |
2 tasks done
Every RedisStorage upsert wrote the record and its index entries as separate awaited round trips. A connection failure or process interruption between the writes left the durable record outside every index: an interrupted upsert_schedule never fires (the scheduler only discovers work through the global index) and is invisible to the user, an interrupted upsert_channel is not restored at startup, and the mcp/skill name indexes desynchronize from their records. agentscope-ai#3840 showed the failure with fault injection on fakeredis; agentscope-ai#3816 fixed the same invariant for knowledge documents. Wrap each upsert's record and index writes in a MULTI/EXEC pipeline (client.pipeline(transaction=True)), covering credential, agent, session creation, schedule, channel, mcp and skill upserts including the mcp/skill rename paths. A failure inside the transaction discards the queued writes, so no record survives outside its indexes.
Self-review caught four more upserts with the same separate-write shape that the first pass missed: upsert_sop, upsert_sop_run (record + two index entries), upsert_team and upsert_voice_profile. Convert them to the same atomic pipeline and extend the fault-injection tests with an interrupted/successful team pair.
…diom Rebase onto the merge of agentscope-ai#3806, which collapsed _set_with_ttl into a single SET ... EX command: replace the SET + conditional EXPIRE pairs inside the upsert pipelines with the same single-command form (and drop the redundant voice_profile gap — voice profiles now carry the configured TTL like other records), so every upsert keeps writing its record and index entries in one MULTI/EXEC transaction while speaking the same idiom as the rest of the file.
konlue
force-pushed
the
fix/redis-upsert-index-atomicity
branch
from
November 9, 2026 15:47
7e42809 to
ba623b8
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AgentScope Version
2.0.12dev (main @
d8ebf89, rebased over #3806)Description
RedisStoragepersisted every record together with its index entries in separate awaited round trips. A connection failure or process interruption between the writes left the durable record outside every index: an interruptedupsert_schedulenever fires (the scheduler discovers work only throughlist_all_schedules(), which reads the global index, and there is no reconciliation) and is invisible to the user; an interruptedupsert_channelis silently not restored at startup (list_all_channels()); interruptedupsert_mcp/upsert_skillcalls desynchronize the name indexes used byget_*_by_nameand the duplicate-name checks (#3840). #3816 fixed the same invariant forupsert_knowledge_document; this extends it to all remaining upserts.Changes — every upsert's record and index writes now commit in one
MULTI/EXECpipeline (client.pipeline(transaction=True), the idiom already used byupsert_message), covering:upsert_schedule(record + per-user index + global scheduler index),upsert_channel(record + per-user index + all-channels index + bot-id binding),upsert_mcp/upsert_skill(record + id set + name hash, including the renameHDEL),upsert_sop/upsert_sop_run(record + per-user index + per-procedure index),upsert_team,upsert_voice_profile,upsert_agent,upsert_credential,upsert_session(creation path; record + indexes),upsert_knowledge_base.The record value is written with the single-command
SET ... EXform introduced by #3806 (pipe.set(key, value, ex=self.key_ttl)), so the pipelines speak the same idiom as the rest of the file — voice profiles now carry the configured TTL like other records, unchanged from #3806's semantics.Pre-transaction reads (the name-conflict check, the previous-record read for renames and
created_atpreservation) stay outside the pipeline with unchanged semantics; a failure inside theasync withblock discards the queued writes, so no record survives outside its indexes. (upsert_knowledge_documentitself is left to #3816.)How to test
New
TestUpsertIndexAtomicityin the existingtests/storage_redis_test.py, using a pipeline wrapper whoseexecutealways raises (fault injection, mirroring #3805's reproduction):test_interrupted_schedule_upsert_is_invisible/test_interrupted_channel_upsert_is_invisible/test_interrupted_skill_upsert_is_invisible/test_interrupted_team_upsert_is_invisible— fail onmain(the record key survives whilelist_schedules/list_all_schedules/list_all_channels/get_*_by_name/list_teamsall come back empty), pass with this fix (nothing applied);test_successful_upserts_reach_every_index— schedule, channel (including the bot-id binding), skill and team all land in every index;test_skill_rename_updates_the_name_index_atomically— renaming drops the previous name hash entry in the same commit.python -m pytest tests/storage_redis_test.py # 105 passed, 25 warnings (includes #3806's TestRecordKeyTTL).Fixes #3840
Checklist