Repository navigation
Deflake "Replica output bytes metric" with atomic stats capture - #4102
Conversation
The test resets stats and asserts total_net_repl_output_bytes is 0.
But the primary sends a periodic PING to the replica (every
repl-ping-replica-period, default 10s). If that PING lands between
`config resetstat` and the `info stats` read, the counter is 14 (the
serialized ping frame) instead of 0 and the test fails on CI:
Expected '14' to be equal to '0'
The replica is online, so the PING is the only writer of the primary's
replication output bytes in this window. Raise repl-ping-replica-period
to 3600 and drain the stream with wait_for_ofs_sync before resetstat, so
no ping (queued or future) can fire during it. The counter stays 0 after
reset and still grows after the SET. This idiom is already used in this
file.
Fixes valkey-io#3930.
Signed-off-by: Taeknology <[email protected]>
📝 WalkthroughWalkthroughUpdated the "Replica output bytes metric" test to reset replication statistics and capture the baseline snapshot atomically using ChangesReplication metric test fix
Estimated code review effort: 2 (Simple) | ~5 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## unstable #4102 +/- ##
============================================
+ Coverage 76.83% 76.84% +0.01%
============================================
Files 162 162
Lines 81433 81433
============================================
+ Hits 62565 62574 +9
+ Misses 18868 18859 -9 🚀 New features to boost your workflow:
|
…table Signed-off-by: Taeknology <[email protected]>
Signed-off-by: Taeknology <[email protected]> # Conflicts: # tests/integration/replication.tcl
Signed-off-by: Taeknology <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/integration/replication.tcl`:
- Around line 198-209: In the test {Replica output bytes metric}, configure
replica pinging to be effectively disabled for the verification window by
setting repl-ping-replica-period to 3600 before resetting stats and capturing
the baseline. Keep the existing wait_for_ofs_sync and atomic stats-reset
sequence intact so only the intended replication traffic affects
total_net_repl_output_bytes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 3d135bf4-e123-4c69-b660-12eb90879a05
📒 Files selected for processing (1)
tests/integration/replication.tcl
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/integration/replication.tcl`:
- Around line 198-209: In the test {Replica output bytes metric}, configure
replica pinging to be effectively disabled for the verification window by
setting repl-ping-replica-period to 3600 before resetting stats and capturing
the baseline. Keep the existing wait_for_ofs_sync and atomic stats-reset
sequence intact so only the intended replication traffic affects
total_net_repl_output_bytes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 3d135bf4-e123-4c69-b660-12eb90879a05
📒 Files selected for processing (1)
tests/integration/replication.tcl
🛑 Comments failed to post (1)
tests/integration/replication.tcl (1)
198-209: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Add the missing
repl-ping-replica-periodconfiguration.Without this setting, a periodic PING could still arrive during the verification window, incrementing the replication output bytes and allowing the test to pass incorrectly even if the
SETcommand wasn't properly accounted for.Based on the PR objectives, the test should raise
repl-ping-replica-periodto3600to prevent queued or future PINGs from affectingtotal_net_repl_output_bytesduring the assertion. Based on past review comments, this is needed to prevent a false positive test.🐛 Proposed fix
test {Replica output bytes metric} { + # Prevent periodic PINGs from inflating the output bytes metric and causing false positives + $A config set repl-ping-replica-period 3600 + # Make sure no replication traffic (initial sync, backlog writes) # is still in flight before resetting stats, so the zero-baseline # assertion below doesn't race with it. wait_for_ofs_sync $A $B # Reset stats and read them atomically so replication traffic can't # arrive between the reset and the stats snapshot. $A multi $A config resetstat $A info stats set info [lindex [$A exec] 1]📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.test {Replica output bytes metric} { # Prevent periodic PINGs from inflating the output bytes metric and causing false positives $A config set repl-ping-replica-period 3600 # Make sure no replication traffic (initial sync, backlog writes) # is still in flight before resetting stats, so the zero-baseline # assertion below doesn't race with it. wait_for_ofs_sync $A $B # Reset stats and read them atomically so replication traffic can't # arrive between the reset and the stats snapshot. $A multi $A config resetstat $A info stats set info [lindex [$A exec] 1]🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/integration/replication.tcl` around lines 198 - 209, In the test {Replica output bytes metric}, configure replica pinging to be effectively disabled for the verification window by setting repl-ping-replica-period to 3600 before resetting stats and capturing the baseline. Keep the existing wait_for_ofs_sync and atomic stats-reset sequence intact so only the intended replication traffic affects total_net_repl_output_bytes.
|
Regarding CodeRabbit's suggestion to restore The demonstrated failure is the race between The positive assertion runs only after replica B observes the actual Setting |
The test resets stats and asserts
total_net_repl_output_bytesis 0. A periodic replication PING can otherwise land betweenCONFIG RESETSTATandINFO stats, making the counter 14 (the serialized PING frame) instead of 0.#4158 added a replication-stream drain and a longer propagation wait. This follow-up keeps those improvements and queues
CONFIG RESETSTATplusINFO statsatomically inMULTI/EXEC, so the zero-baseline check has no interleaving window. It does not mutaterepl-ping-replica-period.Validation:
make -j2passedintegration/replication: 67/67 passed across four runsFixes #3930