Skip to content

[2.8] MOD-12212: Fix index load from RDB temporary memory overhead#7366

Merged
Itzikvaknin merged 2 commits into2.8from
ivaknin-rdb_load_refactor_2_8
Nov 17, 2025
Merged

[2.8] MOD-12212: Fix index load from RDB temporary memory overhead#7366
Itzikvaknin merged 2 commits into2.8from
ivaknin-rdb_load_refactor_2_8

Conversation

@Itzikvaknin
Copy link
Collaborator

@Itzikvaknin Itzikvaknin commented Nov 13, 2025

Test: https://github.com/redislabsdev/Redis-Enterprise/actions/runs/19403079189/job/55513530613


Note

During RDB index load, initialize GC only when adding a new spec, not for duplicates that are discarded.

  • RDB load path (src/spec.c):
    • Start IndexSpec_StartGC only after verifying the spec isn't already in specDict_g (inside the else branch).
    • Remove the earlier unconditional GC start prior to Cursors_initSpec.

Written by Cursor Bugbot for commit ce3d3f3. This will update automatically on new commits. Configure here.

@Itzikvaknin Itzikvaknin changed the base branch from master to 2.8 November 13, 2025 15:35
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Race Condition: Iterator Pending Count Corruption

Race condition in MRIteratorCallback_Done: the function decrements ctx->it->ctx.pending using a non-atomic operation (--ctx->it->ctx.pending) while this field is accessed atomically elsewhere via __atomic_load_n. This creates a data race where concurrent decrements from multiple threads can be lost, leading to incorrect pending counts and potential memory leaks or premature iterator cleanup.

coord/src/rmr/rmr.c#L729-L738

"depleted(should be false): %d, Pending: (%d), inProcess: %d, itRefCount: %d, channel size: "
"%zu, shard_slot: %d",
ctx->cmd.depleted, ctx->it->ctx.pending, ctx->it->ctx.inProcess, ctx->it->ctx.itRefCount,
MRChannel_Size(ctx->it->ctx.chan), ctx->cmd.targetSlot);
ctx->cmd.depleted = true;
short pending = --ctx->it->ctx.pending; // Decrease `pending` before decreasing `inProcess`
RS_ASSERT(pending >= 0);
MRIteratorCallback_ProcessDone(ctx);
}

Fix in Cursor Fix in Web


Bug: Race Condition: Inconsistent In-Process State

Race condition in MR_ManuallyTriggerNextIfNeeded: the function directly assigns it->ctx.inProcess = it->ctx.pending without using atomic operations, while inProcess is accessed atomically elsewhere via __atomic_sub_fetch and __atomic_load_n. This creates a data race that can lead to incorrect tracking of in-process commands and potential iterator lifecycle issues.

coord/src/rmr/rmr.c#L787-L790

// We have <= channelThreshold replies to process, so if there are pending commands we want to trigger them.
if (it->ctx.pending) {
// We have more commands to send
it->ctx.inProcess = it->ctx.pending;

Fix in Cursor Fix in Web


Copy link

@jit-ci jit-ci bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Jit has detected 24 important findings in this PR that you should review.


Jit encountered an internal error and cannot comment on each finding.

You can ask a Jit admin to comment #jit_ignore_all on this PR to ignore the findings.

Here are the findings in this PR:

  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/task-test.yml
    • Lines: 164-174
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/flow-build-artifacts.yml
    • Lines: 82-93
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/task-build-artifacts.yml
    • Lines: 75-105
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/task-build-artifacts.yml
    • Lines: 59-66
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/task-test.yml
    • Lines: 86-116
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/task-build-artifacts.yml
    • Lines: 52-52
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/flow-build-artifacts.yml
    • Lines: 58-66
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/task-test.yml
    • Lines: 252-252
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/benchmark-flow.yml
    • Lines: 73-76
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/task-test.yml
    • Lines: 177-179
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/task-test.yml
    • Lines: 185-190
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/task-build-artifacts.yml
    • Lines: 127-137
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/task-test.yml
    • Lines: 71-78
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/task-test.yml
    • Lines: 67-68
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/task-get-linux-configurations.yml
    • Lines: 49-155
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/event-release.yml
    • Lines: 46-66
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/task-test.yml
    • Lines: 64-64
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/event-release.yml
    • Lines: 97-105
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/task-test.yml
    • Lines: 227-227
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/benchmark-flow.yml
    • Lines: 109-126
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/task-build-artifacts.yml
    • Lines: 55-56
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/benchmark-flow.yml
    • Lines: 81-84
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/task-get-latest-tag.yml
    • Lines: 52-63
  • Security Control: Static Code Analysis Semgrep Pro
    • Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
    • Description: Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
    • Severity: HIGH
    • Learn More: Link
    • Filename: .github/workflows/task-get-latest-tag.yml
    • Lines: 37-46

@codecov
Copy link

codecov bot commented Nov 16, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.44%. Comparing base (d06815f) to head (ce3d3f3).
⚠️ Report is 2 commits behind head on 2.8.

Additional details and impacted files
@@            Coverage Diff             @@
##              2.8    #7366      +/-   ##
==========================================
- Coverage   87.45%   87.44%   -0.01%     
==========================================
  Files         203      203              
  Lines       34782    34782              
==========================================
- Hits        30418    30415       -3     
- Misses       4364     4367       +3     
Flag Coverage Δ
flow 81.81% <100.00%> (-0.22%) ⬇️
unit 42.48% <0.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Itzikvaknin Itzikvaknin requested a review from oshadmi November 17, 2025 08:05
@Itzikvaknin Itzikvaknin added this pull request to the merge queue Nov 17, 2025
Merged via the queue into 2.8 with commit b503484 Nov 17, 2025
13 checks passed
@Itzikvaknin Itzikvaknin deleted the ivaknin-rdb_load_refactor_2_8 branch November 17, 2025 09:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants