MOD-13735 MOD-13181 Fix FT.PROFILE shard total profile time by ofiryanai · Pull Request #8129 · RediSearch/RediSearch

ofiryanai · 2026-01-22T11:19:33Z

Problem

In cluster mode, FT.PROFILE reports incorrect "Total profile time" values for each shard. The total time appears smaller than individual Result Processor times within the same shard, which is mathematically impossible since the total should encompass all processor times. This even worse because some non-negligible time is not included in any of the result processors times (e.g shard results serialization in cursor read).

Root Cause

The issue occurs when internal cursors are used (cluster mode with FT.AGGREGATE). The timing calculation has a mismatch between how total time and individual processor times are tracked:

initClock is reset on each cursor read at src/aggregate/aggregate_exec.c:1382:

rs_wall_clock_init(&req->initClock); // Reset the clock for the current cursor read

profileTotalTime is calculated only from the last cursor read in src/profile.c:145:

req->profileTotalTime += rs_wall_clock_elapsed_ns(&req->initClock);

This only captures elapsed time since the last initClock reset.

Result Processor times accumulate across ALL cursor reads in src/result_processor.c:1144:

self->profileTime += rs_wall_clock_elapsed_ns(&start);

These times are never reset between cursor reads.

Result: When multiple cursor reads occur, RP times accumulate across all reads while total time only measures the final read.

Fix

Accumulate profileTotalTime at the end of each cursor read in finishSendChunk(), matching how RP times are accumulated. This ensures total profile time reflects the sum of all cursor read durations, maintaining mathematical consistency with individual processor times.

Note

Fixes shard profiling time under cursored executions by accumulating per-read durations into req->profileTotalTime.

Update finishSendChunk() in aggregate_exec.c to add elapsed time for intermediate cursor reads to profileTotalTime (final read remains accounted in Profile_Print); use the measured duration for global stats
Add tests in tests/pytests/test_profile.py to verify timing consistency: Total profile time ≥ sum of Result Processor Time across RESP2/RESP3, cluster/standalone, and various SEARCH/AGGREGATE pipelines; include helper sum_rp_times() and ProfileTotalTimeConsistency()

^{Written by Cursor Bugbot for commit 382aa9d. This will update automatically on new commits. Configure here.}

jit-ci · 2026-01-22T11:19:40Z

Hi, I’m Jit, a friendly security platform designed to help developers build secure applications from day zero with an MVS (Minimal viable security) mindset.

In case there are security findings, they will be communicated to you as a comment inside the PR.

Hope you’ll enjoy using Jit.

Questions? Comments? Want to learn more? Get in touch with us.

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

src/aggregate/aggregate_exec.c

codecov · 2026-01-22T11:47:05Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.26%. Comparing base (a6bb950) to head (382aa9d).
⚠️ Report is 4 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8129      +/-   ##
==========================================
+ Coverage   83.21%   83.26%   +0.05%     
==========================================
  Files         369      369              
  Lines       55455    55457       +2     
  Branches    14432    14432              
==========================================
+ Hits        46149    46179      +30     
+ Misses       9148     9120      -28     
  Partials      158      158

Flag	Coverage Δ
flow	`84.53% <100.00%> (+0.08%)`	⬆️
unit	`50.29% <0.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

src/aggregate/aggregate_exec.c

lerman25

Nice! Should we backport?

redisearch-backport-pull-request · 2026-01-22T19:52:28Z

Backport failed for 2.8, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 2.8
git worktree add -d .worktree/backport-8129-to-2.8 origin/2.8
cd .worktree/backport-8129-to-2.8
git switch --create backport-8129-to-2.8
git cherry-pick -x fd63e0c3cf864804885bb2b2688e4444d861b6aa

redisearch-backport-pull-request · 2026-01-22T19:52:29Z

Backport failed for 2.10, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 2.10
git worktree add -d .worktree/backport-8129-to-2.10 origin/2.10
cd .worktree/backport-8129-to-2.10
git switch --create backport-8129-to-2.10
git cherry-pick -x fd63e0c3cf864804885bb2b2688e4444d861b6aa

redisearch-backport-pull-request · 2026-01-22T19:52:31Z

Backport failed for 8.2, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 8.2
git worktree add -d .worktree/backport-8129-to-8.2 origin/8.2
cd .worktree/backport-8129-to-8.2
git switch --create backport-8129-to-8.2
git cherry-pick -x fd63e0c3cf864804885bb2b2688e4444d861b6aa

redisearch-backport-pull-request · 2026-01-22T19:52:32Z

Backport failed for 8.4, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 8.4
git worktree add -d .worktree/backport-8129-to-8.4 origin/8.4
cd .worktree/backport-8129-to-8.4
git switch --create backport-8129-to-8.4
git cherry-pick -x fd63e0c3cf864804885bb2b2688e4444d861b6aa

* Fix FT.PROFILE shard total profile time * Avoid double count on non-cursor Timeout Return failures * Move out of the if (Ok | Timeout) block * Add tests * Enhance tests to cover more cases (cherry picked from commit fd63e0c)

redisearch-backport-pull-request · 2026-01-22T19:52:39Z

Successfully created backport PR for 8.6:

[8.6] MOD-13735 MOD-13181 Fix FT.PROFILE shard total profile time #8140

* Fix FT.PROFILE shard total profile time * Avoid double count on non-cursor Timeout Return failures * Move out of the if (Ok | Timeout) block * Add tests * Enhance tests to cover more cases (cherry picked from commit fd63e0c)

… (#8150) MOD-13181 Fix FT.PROFILE shard total profile time (#8129) * Fix FT.PROFILE shard total profile time * Avoid double count on non-cursor Timeout Return failures * Move out of the if (Ok | Timeout) block * Add tests * Enhance tests to cover more cases (cherry picked from commit fd63e0c)

… (#8151) MOD-13181 Fix FT.PROFILE shard total profile time (#8129) * Fix FT.PROFILE shard total profile time * Avoid double count on non-cursor Timeout Return failures * Move out of the if (Ok | Timeout) block * Add tests * Enhance tests to cover more cases (cherry picked from commit fd63e0c)

* Fix FT.PROFILE shard total profile time * Avoid double count on non-cursor Timeout Return failures * Move out of the if (Ok | Timeout) block * Add tests * Enhance tests to cover more cases (cherry picked from commit fd63e0c)

…8154) * MOD-13181 Fix FT.PROFILE shard total profile time (#8129) * Fix FT.PROFILE shard total profile time * Avoid double count on non-cursor Timeout Return failures * Move out of the if (Ok | Timeout) block * Add tests * Enhance tests to cover more cases (cherry picked from commit fd63e0c) * passing tests * simplify tests * pass also coord tests * simplify extract_profile_coordinator_and_shards * Fix profile parsing to handle FT.SEARCH and FT.AGGREGATE Original get_shards_profile only handled FT.AGGREGATE (assumed res['Shards']). FT.SEARCH has different structure: lowercase 'shards' with nested Coordinator (RESP3), different flat list format (RESP2). New extract_profile_coordinator_and_shards handles all 4 cases (FT.SEARCH/ FT.AGGREGATE × RESP2/RESP3) and extracts Coordinator data needed by testProfileTotalTimeConsistencyCluster* tests. * Fix profile parsing for standalone mode Handle standalone mode in extract_profile_coordinator_and_shards by returning empty coordinator and shards when not in cluster mode. * Fix bug

) * pass also coord tests * MOD-13181 Fix FT.PROFILE shard total profile time (#8129) * Fix FT.PROFILE shard total profile time * Avoid double count on non-cursor Timeout Return failures * Move out of the if (Ok | Timeout) block * Add tests * Enhance tests to cover more cases (cherry picked from commit fd63e0c) * fix * Align test_profile.py helper functions with 2.10 backport * Fix RESP2 parsing to capture all values after key Fixed parse_resp2_shards_list and standalone format parsing to use item[1:] instead of item[1] to capture ALL values after the key. This fixes multi-valued fields like 'Result processors profile' which have format ['key', val1, val2, ...]. Added single-element unwrapping to maintain backward compatibility. * Fix RESP3 parsing to handle case-insensitive keys and filter Coordinator - Check for both 'shards'/'Shards' and 'coordinator'/'Coordinator' keys - Filter out Coordinator from shards list (has 'Total Coordinator time' not 'Total profile time') - Handle standalone mode with 'profile' key * Fix RESP3 Coordinator filter to use negative check Filter out entries with 'Total Coordinator time' key instead of filtering for entries with 'Total profile time', since shards may not always have the 'Total profile time' field.

Fix FT.PROFILE shard total profile time

146b903

github-actions bot added the size:XS label Jan 22, 2026

cursor bot reviewed Jan 22, 2026

View reviewed changes

src/aggregate/aggregate_exec.c Outdated Show resolved Hide resolved

GuyAv46 reviewed Jan 22, 2026

View reviewed changes

src/aggregate/aggregate_exec.c Outdated Show resolved Hide resolved

ofiryanai added 2 commits January 22, 2026 14:43

Avoid double count on non-cursor Timeout Return failures

d104eb5

Move out of the if (Ok | Timeout) block

7243e27

github-actions bot added the size:S label Jan 22, 2026

Add tests

72365ad

ofiryanai force-pushed the fix-profile-shard-total-time branch from 6f0ca61 to 72365ad Compare January 22, 2026 14:56

Enhance tests to cover more cases

382aa9d

github-actions bot added the size:M label Jan 22, 2026

GuyAv46 approved these changes Jan 22, 2026

View reviewed changes

lerman25 approved these changes Jan 22, 2026

View reviewed changes

ofiryanai added backport 8.2 backport 8.4 backport 8.6 labels Jan 22, 2026

ofiryanai enabled auto-merge January 22, 2026 16:41

ofiryanai marked this pull request as draft January 22, 2026 16:51

auto-merge was automatically disabled January 22, 2026 16:51
Pull request was converted to draft

ofiryanai marked this pull request as ready for review January 22, 2026 16:51

ofiryanai added backport 2.8 backport 2.10 labels Jan 22, 2026

ofiryanai enabled auto-merge January 22, 2026 17:20

ofiryanai added this pull request to the merge queue Jan 22, 2026

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 22, 2026

ofiryanai added this pull request to the merge queue Jan 22, 2026

Merged via the queue into master with commit fd63e0c Jan 22, 2026
96 checks passed

ofiryanai deleted the fix-profile-shard-total-time branch January 22, 2026 19:52

redisearch-backport-pull-request bot mentioned this pull request Jan 22, 2026

[8.6] MOD-13735 MOD-13181 Fix FT.PROFILE shard total profile time #8140

Merged

ofiryanai mentioned this pull request Jan 25, 2026

[8.4] MOD-13735 MOD-13181 Fix FT.PROFILE shard total profile time #8140 #8150

Merged

ofiryanai mentioned this pull request Jan 25, 2026

[8.2] MOD-13735 MOD-13181 Fix FT.PROFILE shard total profile time #8140 #8151

Merged

ofiryanai changed the title ~~MOD-13181 Fix FT.PROFILE shard total profile time~~ MOD-13735 MOD-13181 Fix FT.PROFILE shard total profile time Jan 25, 2026

ofiryanai mentioned this pull request Jan 25, 2026

[2.10] MOD-13735 MOD-13181 Fix FT.PROFILE shard total profile time #8154

Merged

ofiryanai mentioned this pull request Jan 25, 2026

[2.8] MOD-13735 MOD-13181 Fix FT.PROFILE shard total profile time #8163

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MOD-13735 MOD-13181 Fix FT.PROFILE shard total profile time#8129

MOD-13735 MOD-13181 Fix FT.PROFILE shard total profile time#8129
ofiryanai merged 5 commits intomasterfrom
fix-profile-shard-total-time

ofiryanai commented Jan 22, 2026 •

edited by cursor bot

Loading

Uh oh!

jit-ci bot commented Jan 22, 2026

Uh oh!

cursor bot left a comment

Uh oh!

Uh oh!

codecov bot commented Jan 22, 2026 •

edited

Loading

Uh oh!

Uh oh!

lerman25 left a comment

Uh oh!

Uh oh!

Uh oh!

redisearch-backport-pull-request bot commented Jan 22, 2026

Uh oh!

redisearch-backport-pull-request bot commented Jan 22, 2026

Uh oh!

redisearch-backport-pull-request bot commented Jan 22, 2026

Uh oh!

redisearch-backport-pull-request bot commented Jan 22, 2026

Uh oh!

redisearch-backport-pull-request bot commented Jan 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ofiryanai commented Jan 22, 2026 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Root Cause

Fix

Uh oh!

jit-ci bot commented Jan 22, 2026

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

lerman25 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

redisearch-backport-pull-request bot commented Jan 22, 2026

Uh oh!

redisearch-backport-pull-request bot commented Jan 22, 2026

Uh oh!

redisearch-backport-pull-request bot commented Jan 22, 2026

Uh oh!

redisearch-backport-pull-request bot commented Jan 22, 2026

Uh oh!

redisearch-backport-pull-request bot commented Jan 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ofiryanai commented Jan 22, 2026 •

edited by cursor bot

Loading

codecov bot commented Jan 22, 2026 •

edited

Loading