Adjusting disconnectedBehavior Option to Prevent Timeout When Redis Shutdown#1
Open
MagicalLas wants to merge 1 commit intomainfrom
Open
Adjusting disconnectedBehavior Option to Prevent Timeout When Redis Shutdown#1MagicalLas wants to merge 1 commit intomainfrom
MagicalLas wants to merge 1 commit intomainfrom
Conversation
… Shutdown We often need to shut down Redis for maintenance, such as version upgrades. During these times, requests do not fail immediately but instead experience timeouts, increasing application latency. This issue can be resolved by adjusting some options. Currently, our Redis client options are configured as follows: ```java ClientOptions options = ClientOptions.builder() .autoReconnect(true) .disconnectedBehavior(DisconnectedBehavior.REJECT_COMMANDS) ``` The DisconnectedBehavior.REJECT_COMMANDS option appears to cancel commands when the connection is lost. However, if autoReconnect is not set to false, commands in the CommandHandler.stack are not canceled but are placed into the disconnectedBuffer. Therefore, ongoing commands are not rejected if autoReconnect is true, even with the client option modified. For services heavily relying on Redis, latency is crucial. Additionally, we want to avoid writing custom code for reconnections by using the auto-reconnect feature. Adjusting the autoReconnect option can solve this issue immediately, but it would require significant changes to implement automatic reconnection. Proposal We propose that the condition for canceling commands in the CommandHandler.stack should be based solely on rejectCommandsWhileDisconnected and not combined with autoReconnect. I've submitted a PR with a simple code change. Please let me know your thoughts. === Additionally our applications, to maintain low latency and avoid queuing many requests in the buffer when the connection is down, we have configured the client to execute commands only when the connection is active. As a result, only a few requests in the CommandHandler.stack encounter timeouts, but this still negatively impacts the application.
MagicalLas
changed the title
MagicalLas
pushed a commit
that referenced
this pull request
Mar 9, 2026
* Introducing the FT.CREATE command * Formatter issues * Polishing #1
MagicalLas
pushed a commit
that referenced
this pull request
Mar 9, 2026
…redis#3461) * feat(CAE-1130): Add comprehensive connection testing for Redis Enterprise maintenance events - Add ConnectionTesting class with 9 test scenarios for maintenance handoff behavior - Test old connection graceful shutdown during MOVING operations - Validate traffic resumption with autoconnect after handoff - Verify maintenance notifications only work with RESP3 protocol - Test new connection establishment during migration and bind phases - Add memory leak validation for multiple concurrent connections - Include TLS support testing for maintenance events - Replace .supportMaintenanceEvents(true) with MaintenanceEventsOptions.enabled() - Add comprehensive monitoring and validation of connection lifecycle Tests cover CAE-1130 requirements for Redis Enterprise maintenance event handling including connection draining, autoconnect behavior, and notification delivery. * Add comprehensive maintenance events tests for CLIENT MAINT_NOTIFICATIONS - connectionHandshakeIncludesEnablingNotificationsTest: Verifies all 5 notification types (MOVING, MIGRATING, MIGRATED, FAILING_OVER, FAILED_OVER) are received when maintenance events are enabled - disabledDontReceiveNotificationsTest: Verifies no notifications received when maintenance events are disabled - clientHandshakeWithEndpointTypeTest: Tests CLIENT MAINT_NOTIFICATIONS with 'none' endpoint type (nil IP scenario) - clientMaintenanceNotificationInfoTest: Verifies CLIENT MAINT_NOTIFICATIONS configuration with moving-endpoint-type Based on CLIENT MAINT_NOTIFICATIONS implementation from commit bd408cf * Update Redis Enterprise maintenance event notification protocol - Update push notification patterns to include sequence numbers (4-element format) - Fix MOVING notification parsing to handle new address format with sequence and time - Update MIGRATING, MIGRATED, FAILING_OVER, and FAILED_OVER patterns with sequence numbers - Improve FaultInjectionClient status handling: change from 'pending' to 'running' checks - Enhance JSON response parsing with better output field handling and debugging - Remove deprecated maintenance sequence functionality and associated unit test - Add test phase isolation to prevent cleanup notification interference - Extend monitoring timeout from 2 to 5 minutes for longer maintenance operations - Add @AfterEach cleanup to restore cluster state between tests - Remove hardcoded optimal node selection logic in RedisEnterpriseConfig This aligns with the updated Redis Enterprise maintenance events specification and improves test reliability by handling the new notification protocol format. * Fix moving tests for timeout de-relaxation after moving * fix notification capture logic and several tests. * fix up resp2 test, and add proper test for None, will rebase to master * Fix None test * Fix several tests related to handling. 5 tests left to fix up. * fix up new connection test and connection leak tests * fix up traffic test and remove un-needed code. * fix more tests, remove more un-needed code * revert log changes * revert the re-throw change, to be discussed * remove resp3 test after offline discussion * change endpoint name * temporarely reduce number of tests * add more tests * reduce test execution time by 50% * remove hardcoded target config and enable working with 6 nodes and multiple dbs * fix up relaxedtimeoutconfig to use newest functions and add connection handoff test * add 1 more handoff test, add more logging, fix some issues that were raised during review * fix some bugs and remove the un-needed clean-up of testing, to speed up tests by 50% * Merge pull request #1 from kiryazovi-redis/CI-fix-functional-handoff-and-connection-testing-of-maint-events-2 Ci fix functional handoff and connection testing of maint events 2 * optimise relaxed timeoutest, fix compilation issues after renaming done by Ivo * remove MaintenanceNotificationTest, as all functionality is covered now, also implement more fixes and improvements * renamed memoryleak infra, refactored several tests, implemented more fixes from review, started removin comments * remove any useless sleeps, refactor several tests again to be more functionally correct, enable logging for testing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
sample implementation