[2.8] MOD-12212: Fix index load from RDB temporary memory overhead#7366
[2.8] MOD-12212: Fix index load from RDB temporary memory overhead#7366Itzikvaknin merged 2 commits into2.8from
Conversation
There was a problem hiding this comment.
Bug: Race Condition: Iterator Pending Count Corruption
Race condition in MRIteratorCallback_Done: the function decrements ctx->it->ctx.pending using a non-atomic operation (--ctx->it->ctx.pending) while this field is accessed atomically elsewhere via __atomic_load_n. This creates a data race where concurrent decrements from multiple threads can be lost, leading to incorrect pending counts and potential memory leaks or premature iterator cleanup.
coord/src/rmr/rmr.c#L729-L738
RediSearch/coord/src/rmr/rmr.c
Lines 729 to 738 in 2837ff7
Bug: Race Condition: Inconsistent In-Process State
Race condition in MR_ManuallyTriggerNextIfNeeded: the function directly assigns it->ctx.inProcess = it->ctx.pending without using atomic operations, while inProcess is accessed atomically elsewhere via __atomic_sub_fetch and __atomic_load_n. This creates a data race that can lead to incorrect tracking of in-process commands and potential iterator lifecycle issues.
coord/src/rmr/rmr.c#L787-L790
RediSearch/coord/src/rmr/rmr.c
Lines 787 to 790 in 2837ff7
There was a problem hiding this comment.
❌ Jit has detected 24 important findings in this PR that you should review.
Jit encountered an internal error and cannot comment on each finding.
You can ask a Jit admin to comment #jit_ignore_all on this PR to ignore the findings.
Here are the findings in this PR:
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/task-test.yml
- Lines: 164-174
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/flow-build-artifacts.yml
- Lines: 82-93
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/task-build-artifacts.yml
- Lines: 75-105
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/task-build-artifacts.yml
- Lines: 59-66
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/task-test.yml
- Lines: 86-116
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/task-build-artifacts.yml
- Lines: 52-52
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/flow-build-artifacts.yml
- Lines: 58-66
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/task-test.yml
- Lines: 252-252
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/benchmark-flow.yml
- Lines: 73-76
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/task-test.yml
- Lines: 177-179
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/task-test.yml
- Lines: 185-190
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/task-build-artifacts.yml
- Lines: 127-137
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/task-test.yml
- Lines: 71-78
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/task-test.yml
- Lines: 67-68
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/task-get-linux-configurations.yml
- Lines: 49-155
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/event-release.yml
- Lines: 46-66
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/task-test.yml
- Lines: 64-64
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/event-release.yml
- Lines: 97-105
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/task-test.yml
- Lines: 227-227
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/benchmark-flow.yml
- Lines: 109-126
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/task-build-artifacts.yml
- Lines: 55-56
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/benchmark-flow.yml
- Lines: 81-84
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/task-get-latest-tag.yml
- Lines: 52-63
- Security Control: Static Code Analysis Semgrep Pro
- Type: Yaml.Github-Actions.Security.Run-Shell-Injection.Run-Shell-Injection
- Description: Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". - Severity: HIGH
- Learn More: Link
- Filename: .github/workflows/task-get-latest-tag.yml
- Lines: 37-46
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## 2.8 #7366 +/- ##
==========================================
- Coverage 87.45% 87.44% -0.01%
==========================================
Files 203 203
Lines 34782 34782
==========================================
- Hits 30418 30415 -3
- Misses 4364 4367 +3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Test: https://github.com/redislabsdev/Redis-Enterprise/actions/runs/19403079189/job/55513530613
Note
During RDB index load, initialize GC only when adding a new spec, not for duplicates that are discarded.
src/spec.c):IndexSpec_StartGConly after verifying the spec isn't already inspecDict_g(inside theelsebranch).Cursors_initSpec.Written by Cursor Bugbot for commit ce3d3f3. This will update automatically on new commits. Configure here.