MOD-11751: Support WITHCOUNT keyword in FT.AGGREGATE by nafraf · Pull Request #7202 · RediSearch/RediSearch

nafraf · 2025-10-31T20:33:12Z

Describe the changes in the pull request

Support WITHCOUNT keyword in FT.AGGREGATE for accurate total results.
The root of the issue is that the RESP2 response format has the total_results appear before the results themselves.
For efficiency, we do not return an accurate count.
If user is interested with the accurate count, adding the WITHCOUNT to FT.AGGREGATE will do cost with more cpu and memory, and return the accurate count.

Adding a depleting result processor step (when missing) that will deplete all matching docs and will return an accurate count.
With RESP3, with sorter/grouper, there are already depleters being used - so no need to add another.
Notice: With ON_TIMEOUT FAIL policy, we buffer results until making sure no timeout occurs, but not necessarily having a depleter. In addition, depleting normalizers are only used in FT.HYBDRID which we do not handle in this PR.
Also adding a ShardResponseBarrier to make sure the coordinator got a first reply from each shard and accumulated the total count.

No included in this PR:

Fix for FT.AGGREGATE + WITHCOUNT + WITHCURSORS
Fix for FT.AGGREGATE + KNN + WITHCOUNT

Note:
The default value in FT.SEARCH is WITHCOUNT (dialects 1-3), WITHOUTCOUNT (dialect 4).
The default value in FT.AGGREGATE is WITHOUTCOUNT

A clear and concise description of what the PR is solving, including:

Current: The current state briefly
Change: What is the change
Outcome: Adding the outcome

Which additional issues this PR fixes

MOD-...
#...

Main objects this PR modified

...

Mark if applicable

This PR introduces API changes
This PR introduces serialization changes

Note

Adds WITHCOUNT to FT.AGGREGATE with accurate total_results via a depleter pipeline and a shard response barrier, plus coordinator/runtime plumbing and extensive tests.

FT.AGGREGATE — WITHCOUNT support:
- New request flags (HAS_WITHCOUNT, HAS_SORTBY, HAS_GROUPBY, HAS_DEPLETER) and helpers; parse and validate (including rejecting WITHCOUNT+WITHCURSOR).
- Pipeline: inject RPDepleter (sync) when needed; pager placement updated; optimizer/path checks adjusted.
- Arrange/plan: create arrange steps lazily; treat empty SORTBY 0 MAX n; use array_len checks.
Coordinator and runtime:
- Introduce ShardResponseBarrier to wait for first reply from each shard and accumulate total_results; wired through RPNet, MRIterator private data lifecycle, and net callback.
- Add channel/iterator timed pop APIs to support coordinated waiting with timeouts; handle EOF/timed-out flows and profiling drains.
- Preserve WITHCOUNT in fanout command; fix draining loop and RESP handling.
Result processors:
- Rename/background-safe depleter (RPSafeDepleter) for hybrid; add new synchronous RPDepleter for aggregate.
- Profile prints new RP types.
Hybrid:
- Switch to RPSafeDepleter; cursor/startup paths updated.
Utilities/bug fixes:
- array_del uses memmove; add timer helpers.
Tests:
- Extensive pytests for WITHCOUNT semantics, shard barrier behavior, limits, profiles, pagers, KNN cases, and expiration; C++ tests for safe depleter and hybrid pipelines.

^{Written by Cursor Bugbot for commit 96865aa. This will update automatically on new commits. Configure here.}

src/aggregate/aggregate_request.c

This reverts commit 7d94c44.

codecov · 2025-11-03T09:00:55Z

Codecov Report

❌ Patch coverage is 93.60000% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.73%. Comparing base (9c09598) to head (96865aa).
⚠️ Report is 4 commits behind head on master.

Files with missing lines	Patch %	Lines
src/coord/rmr/chan.c	86.48%	5 Missing ⚠️
src/coord/dist_aggregate.c	76.47%	4 Missing ⚠️
src/coord/rpnet.c	96.55%	4 Missing ⚠️
src/coord/dist_utils.c	87.50%	3 Missing ⚠️
src/pipeline/pipeline_construction.c	90.62%	3 Missing ⚠️
src/result_processor.c	96.42%	3 Missing ⚠️
src/util/timeout.h	60.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7202      +/-   ##
==========================================
+ Coverage   84.70%   84.73%   +0.03%     
==========================================
  Files         349      349              
  Lines       54139    54428     +289     
  Branches    14500    14500              
==========================================
+ Hits        45858    46122     +264     
- Misses       8088     8113      +25     
  Partials      193      193

Flag	Coverage Δ
flow	`85.13% <93.33%> (-0.06%)`	⬇️
unit	`51.74% <19.46%> (-0.23%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

tests/pytests/test.py

src/aggregate/aggregate_exec.c

src/coord/dist_aggregate.c

src/coord/rpnet.c

src/pipeline/pipeline_construction.c

…ctly

src/aggregate/aggregate_request.c

src/coord/rpnet.c

src/pipeline/pipeline_construction.c

src/util/arr/arr.h

src/util/timeout.h

src/result_processor.h

tests/pytests/test_multibyte_char_terms.py

kei-nan

left some comments,
I think asking another person to review would be a good idea.

cursor · 2025-12-04T10:13:23Z

src/coord/dist_aggregate.c

+      return RS_RESULT_ERROR;
+    }
+    nc->shardResponseBarrier = barrier;
+  }


Bug: WITHCOUNT + GROUPBY returns wrong total in cluster mode

The ShardResponseBarrier is created for all WITHCOUNT aggregate queries including those with GROUPBY. For GROUPBY queries, each shard returns its group count in total_results, and the barrier accumulates these (e.g., 25 + 25 + 25 = 75 for 3 shards). However, the coordinator's final GROUPBY may produce fewer unique groups (e.g., 25) due to overlapping group keys across shards. Since rpnetNext skips per-reply accumulation when the barrier is set, totalResults remains at the incorrect accumulated value. The barrier creation condition at line 56 checks HasWithCount and IsAggregate but doesn't exclude GROUPBY queries where the shard totals don't represent the final count.

Additional Locations (1)

src/coord/rpnet.c#L155-L165

cursor · 2025-12-04T12:05:40Z

src/coord/rpnet.c

+
+      // Store reply for later processing
+      if (!nc->pendingReplies) {
+        nc->pendingReplies = array_new(MRReply *, numShards);


Bug: Array allocated with zero capacity when numShards not yet initialized

When the barrier is first waiting for shard responses, the local variable numShards is captured from atomic_load in the while loop condition (line 237) before blocking on MRIterator_NextWithTimeout. If a reply arrives before iterStartCb has initialized the barrier, numShards will be 0, causing array_new(MRReply *, numShards) to allocate an array with zero initial capacity. While array_append handles dynamic growth, this is inefficient and doesn't match the comment's claim that "once a reply arrives, iterStartCb has finished and numShards will be set" since the local variable isn't re-read after unblocking.

src/pipeline/pipeline_construction.c

…omments for clarity

cursor · 2025-12-04T14:07:54Z

src/coord/rpnet.c

+      if (!nc->pendingReplies) {
+        nc->pendingReplies = array_new(MRReply *, numShards);
+      }
+      array_append(nc->pendingReplies, reply);


Bug: Coordinator waits indefinitely when numShards is 0 at timeout check

When the barrier waiting loop times out before iterStartCb initializes numShards (i.e., numShards remains 0), array_new(MRReply *, numShards) creates an array with capacity 0 at line 252. While this isn't a crash (the array grows dynamically), it's an inefficiency. More importantly, if numShards is still 0 when shardResponseBarrier_HandleTimeout is called, it will return a timeout error even if replies exist in pendingReplies. The condition at line 191 treats numShards == 0 as a timeout case regardless of actual reply availability.

cursor · 2025-12-04T14:07:54Z

src/coord/rpnet.c

+          array_append(nc->shardsProfile, error);
+        }
+        return RS_RESULT_OK;
+      }


Bug: Missing totalResults update when shard error is later ignored

When shardResponseBarrier_HandleError detects a shard error and returns early at line 265, shardResponseBarrier_UpdateTotalResults at line 276 is never called. If this error is later ignored in rpnetNext due to policy settings like ON_TIMEOUT RETURN, the query continues but totalResults was never set from the barrier's accumulated value. The IO thread callbacks already accumulated totals from successful shards into accumulatedTotal, but this partial count is lost. For WITHCOUNT queries with error-tolerant policies, users would receive results but with an incorrect total_results (likely 0).

approved by oshadmi, kei-nans comments were addressed

redisearch-backport-pull-request · 2025-12-04T15:55:09Z

Backport failed for 2.10, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 2.10
git worktree add -d .worktree/backport-7202-to-2.10 origin/2.10
cd .worktree/backport-7202-to-2.10
git switch --create backport-7202-to-2.10
git cherry-pick -x b9e1fe9270c72dd8d6b64815fd7dcb5f7b0e3d6a 6faa9b5d2a53970fcdabe64566645099a940d264 590c01d86c3001827a9d11647e647967273e19b0 1353a4a519be78f5c6d440c6e32b36d66cba476e 5f1b4f345ccd302854b9080025a49800373e272d fc72f1aff73d232398ec492f2eeac03ff3807f61 7d94c4453234893335bacdd541540f2f9cb8a096 b254b4f00cf8081f360ca3ce2bbc3ad48c379816 dabd1c8c6d6f1f0121421302639bb73935bdb619 8bcda5f1b168e16170db0c15de6a99b8ab696383 b9520c3962d40e09f80af916b1745ece857a526c ee73faec9177c461945583163cf6e4aeb49ba167 d2c5257a9d501a62b8eb701b4c1faf3b7cb997ed e83797f95c8443a5534bdafa2746d63fd9e4b826 98debd8875522bfa5ff05e4bdf4bb75e311cfd15 3a3cc5b89edfcaf0f1aab105aa9c1782583c6e63 21ac10a81fc74217aa203067408e5c3a1ae471f0 0acca45f8822a2c4edf6cf191edc62ad4a6fb882 568910963e009443a0cc866bc994abf505173a8a 81cc473de4b4c9dd312d6c706f397592ffe1829f b14371e41464b7bc7cee1ce9333b3eabf8c1cad0 f1ef205a1961cc67243ffbeb7ed4fc2665752f41 e4690f31785438253d245a539cf3c70688f45e2a 794f56e53a50ada5c9593ae971fdf81aa02ba2ac fcf5bc1a423696fbc70d95e03af1f2420bc71344 800f4da4688a2a7fbaa642b87a562b3ecd84fa87 b6bfeacdc0e6033a943918de5e07c4d345132269 bb3a421a5414bdb98826f241ed95768d220cd692 8d31eb9ade8f3d333246743b28ff34ddd7b2502e 6f3a33e34a833787e39ddc7ee3b2e52c30b8fb4b 8a1daf86de3b29a505ee35159b3b247acbe4127a 06340eef2980d4d3e49baae7240d597b4b6c9068 8ed83d8842377283da83089aa5b547ba0e67f93c 14b13f7b1fe60924932bc64c7a846df73180fd56 4c2698ad284b58397811f063c1bc3d24a34cb96d ed6f5d1ea4f02b8663bd03c2e174ae1563b17d3c bf082b0c6ec1c0e1d1847323c3003347386b5e0a 194f569def87afa684992e6cd6310cd850da5773 5b86ba846e696a4f73fc8ad55d1800a5664d4baf 36686c6d4fd6f08b42bbee3aaaf8d21db781313a 78f35c36e2df5f7f4d369afbe408e67679286bc6 b5a679eb7909f583de04cb9cbbff8fb31083c717 5134787ad91db4a80a6f3abb0693bed81ff4dbee e200b6983e188eee7b19c42e57be562337a34bfa 3548f730bddae94ef5a89a4327e62405d42cc9b4 205ed788bbd80a298a77b1ab5b6c87275ac4c57d d88b048e915300a259918dcda704878b77eba68a e0e9723a3863d3b3f078c46b04bab3b8e8fdc746 883a499c4f60ba8c2460d69ffc24245b9d28b276 546d4aaaca5b73a92f7e55e21f16abec3412f11b 589f83e5ee059e493610aea01bf0a739e5c7a3ee 0fdb36c384eb5f0341a96602240020437ad6d32c 746b16264816493d534e44bcda2bd701344b6258 6b8197481c7795c86534feaaaaa78c005e2e1de3 b529cce68cc03105f70d93aed65e63814036636d b13563d0e73e8537eebd2391bf1b265f4d531767 4cee2c93d3d48870a15af06fd1e3213d095cf140 4bfb160f8c964f274b97d092b01e386dd9a90243 8a7a74b46e860d00c7e25b44362bc00f1f9f2fc0 a29158db15745e564c55e689b60b408d9cd3ad1b c1029b3b9165d9815fb4d218b693fd3a89b7c4fb cbd20925f936e0d102de56d44edda470baf49510 9618db9d4caf120fac579691b0d8b71e6cc4cb69 b1935726948eb81e80ca8a94aa820f5f5d433b50 238ba5bb6288d67d8f84cefe271034d77c715388 000de3c98cb418da78ce7119b6fa9942aadee15d e721a15f69c88a782806448e3638f99dd33a3eba 7a92af385fa5878607281b8b71e4a645fb9e83d9 f9e99db8aecaf327c7d9dca88abd626a4c409b39 437f143fd17a34531d4ff7cfe6628db4a92ab859 a48e7c966911563082dd974354405a861fcab6e2 47c86c54f90ae30e458080cea9cf860952c3941c 5764857b819f036438396ce1d45afd1e1acaafd7 ef793557708fd7bf20960e5d23dd3231f323e012 523bf261c891d4738847896643b1e140973a1a94 d6291d2db7724350765085fa74b365b1b9c9e270 d3beb32b75082e65a8051d6e40cd55521b616927 2847b61f52aba9c2e259456f55aeb469813f1d72 397ed1f0311e474b909cb3274baba3d4b65b79f2 2c3271497cd711695d6158eb90f3d29d95fe257d d29a9b27c4058a89e0e691823c869730fd9c758e 740b2772f9cee755f09e173ec5d6b5c193f841e5 6f1b8806df5a179e3c1bb0c1a78d98cce591b448 22caa6fc166c4cf79f80d982cd7089dd3f46675e e7c1b7041fe61782691817aa164f632b9a9fa49f ece995e7d6a78d86b6ad18171c4a29a506ab9bec 4fc8c21c1acc97aabd92ce954d112471460a1ad2 7a34bd0d8c663bbfbac6e9213da8ed1a9cb88745 a1e62ba53ea216cc6922f2b0f092a13a9f029499 ebd310c32697347f447eea5f3c3f4e4348c964a2 cc027bde8f1aaa1a10d0c5e8e3ca0bd93aaf91d9 fbba14bf6638efd9ae54ec98f2566cddd7132925 4b9456787539d0738e499a6e319ac0c5dfd7d5e8 d493bc54e9624c6da328c86f6e821b6bb9b1cec8 f253e7cee3fef3fc31dfbd7516dcd4a460440fea 1d010a3e45b1277a99f1123844cbf749ffdfb964 d9e536a03e5426ede2f4d587a8c2ac3230ef4845 d26df0a1f9e087d11f933950f8b72fac904d2371 c7777000782e4183de233bdcda01baef4589bf5c 1d062c5120d769228ff9e09e264048e87abacc44 6cd7875196eb82430fc8355e696e316294bb9e5e c2ea61dd8dd847705688b23b5705123a15b60432 ec9a9bc7c42bfd4b4dc63942352b48bc68d24966 cea90a0ed923d6f32931f081f73eefb1d910ef84 8b6afd0e2bf4ad009dadf8758b24f02c39cfd118 16bf9c95cd9a185332d17a7b9d529f606fe2a72e 5107f53e0cb480fd0acc7feec19d6cb5cdd3ee25 f0c03aeb24c8c8f18faa9f596eb70a7bd6225d49 9fbd9303feffa927eebfc5f12be4e0317b096dc0 8fa93313e5f42f5551a97dff043526ac093f8f02 c305335e4e5a47edbcfd84183be4300a85091bac 2d6593c8c248db31659a07780197ace22e843dec 6e3f08a16744a84e47813ed11acb26f0fef8dc80 18d8bbb3aa683372d8645d56f67c628571c14de7 0ea8bfd7c99e78ae7b36023ce72bb6be550c610a 668676e9b04207c0945154d0d3485b47a3d33948 32d81a1d34fde3d0cee96805d83dd68e5ddaec91 a55e9a2c2329f32987bfbc1460ce45b02d70ee67 e78cb58b9b0d0f336ea824b120dedf27e1bc643e 5f1ffc2b6a29fee23e614556edde836902e55a11 893e82754f6222a836ad735b8da9c6b120364afd 590c299607b8bc95abfac216161e62e8d8885477 499749c5f74cad29f64f7535c2917aa8236c88e0 1bce4c06eb33bb00fd4e1b422ee134a9f8147c18 42daef8deda59e5dd2ffb575e1a8b0755264812e efd0a5e1da1d19b38bf1b91ca6be3d8b1ca55b95 ee728950872020e57ad41c3fab65fe6f22ff4837 c803784020b7713a6b5063b838aa032fcfe88f33 533594dff507e93b4a2e4b71631b6297308294f6 a8593ae657f10d1f3c4d814c4506269d83960bcc a97a82203cd17ca1638aeca5df5fee52b7d6938d 96865aaca468b16c299f0d49988882d542933274

redisearch-backport-pull-request · 2025-12-04T15:55:11Z

Backport failed for 8.2, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 8.2
git worktree add -d .worktree/backport-7202-to-8.2 origin/8.2
cd .worktree/backport-7202-to-8.2
git switch --create backport-7202-to-8.2
git cherry-pick -x b9e1fe9270c72dd8d6b64815fd7dcb5f7b0e3d6a 6faa9b5d2a53970fcdabe64566645099a940d264 590c01d86c3001827a9d11647e647967273e19b0 1353a4a519be78f5c6d440c6e32b36d66cba476e 5f1b4f345ccd302854b9080025a49800373e272d fc72f1aff73d232398ec492f2eeac03ff3807f61 7d94c4453234893335bacdd541540f2f9cb8a096 b254b4f00cf8081f360ca3ce2bbc3ad48c379816 dabd1c8c6d6f1f0121421302639bb73935bdb619 8bcda5f1b168e16170db0c15de6a99b8ab696383 b9520c3962d40e09f80af916b1745ece857a526c ee73faec9177c461945583163cf6e4aeb49ba167 d2c5257a9d501a62b8eb701b4c1faf3b7cb997ed e83797f95c8443a5534bdafa2746d63fd9e4b826 98debd8875522bfa5ff05e4bdf4bb75e311cfd15 3a3cc5b89edfcaf0f1aab105aa9c1782583c6e63 21ac10a81fc74217aa203067408e5c3a1ae471f0 0acca45f8822a2c4edf6cf191edc62ad4a6fb882 568910963e009443a0cc866bc994abf505173a8a 81cc473de4b4c9dd312d6c706f397592ffe1829f b14371e41464b7bc7cee1ce9333b3eabf8c1cad0 f1ef205a1961cc67243ffbeb7ed4fc2665752f41 e4690f31785438253d245a539cf3c70688f45e2a 794f56e53a50ada5c9593ae971fdf81aa02ba2ac fcf5bc1a423696fbc70d95e03af1f2420bc71344 800f4da4688a2a7fbaa642b87a562b3ecd84fa87 b6bfeacdc0e6033a943918de5e07c4d345132269 bb3a421a5414bdb98826f241ed95768d220cd692 8d31eb9ade8f3d333246743b28ff34ddd7b2502e 6f3a33e34a833787e39ddc7ee3b2e52c30b8fb4b 8a1daf86de3b29a505ee35159b3b247acbe4127a 06340eef2980d4d3e49baae7240d597b4b6c9068 8ed83d8842377283da83089aa5b547ba0e67f93c 14b13f7b1fe60924932bc64c7a846df73180fd56 4c2698ad284b58397811f063c1bc3d24a34cb96d ed6f5d1ea4f02b8663bd03c2e174ae1563b17d3c bf082b0c6ec1c0e1d1847323c3003347386b5e0a 194f569def87afa684992e6cd6310cd850da5773 5b86ba846e696a4f73fc8ad55d1800a5664d4baf 36686c6d4fd6f08b42bbee3aaaf8d21db781313a 78f35c36e2df5f7f4d369afbe408e67679286bc6 b5a679eb7909f583de04cb9cbbff8fb31083c717 5134787ad91db4a80a6f3abb0693bed81ff4dbee e200b6983e188eee7b19c42e57be562337a34bfa 3548f730bddae94ef5a89a4327e62405d42cc9b4 205ed788bbd80a298a77b1ab5b6c87275ac4c57d d88b048e915300a259918dcda704878b77eba68a e0e9723a3863d3b3f078c46b04bab3b8e8fdc746 883a499c4f60ba8c2460d69ffc24245b9d28b276 546d4aaaca5b73a92f7e55e21f16abec3412f11b 589f83e5ee059e493610aea01bf0a739e5c7a3ee 0fdb36c384eb5f0341a96602240020437ad6d32c 746b16264816493d534e44bcda2bd701344b6258 6b8197481c7795c86534feaaaaa78c005e2e1de3 b529cce68cc03105f70d93aed65e63814036636d b13563d0e73e8537eebd2391bf1b265f4d531767 4cee2c93d3d48870a15af06fd1e3213d095cf140 4bfb160f8c964f274b97d092b01e386dd9a90243 8a7a74b46e860d00c7e25b44362bc00f1f9f2fc0 a29158db15745e564c55e689b60b408d9cd3ad1b c1029b3b9165d9815fb4d218b693fd3a89b7c4fb cbd20925f936e0d102de56d44edda470baf49510 9618db9d4caf120fac579691b0d8b71e6cc4cb69 b1935726948eb81e80ca8a94aa820f5f5d433b50 238ba5bb6288d67d8f84cefe271034d77c715388 000de3c98cb418da78ce7119b6fa9942aadee15d e721a15f69c88a782806448e3638f99dd33a3eba 7a92af385fa5878607281b8b71e4a645fb9e83d9 f9e99db8aecaf327c7d9dca88abd626a4c409b39 437f143fd17a34531d4ff7cfe6628db4a92ab859 a48e7c966911563082dd974354405a861fcab6e2 47c86c54f90ae30e458080cea9cf860952c3941c 5764857b819f036438396ce1d45afd1e1acaafd7 ef793557708fd7bf20960e5d23dd3231f323e012 523bf261c891d4738847896643b1e140973a1a94 d6291d2db7724350765085fa74b365b1b9c9e270 d3beb32b75082e65a8051d6e40cd55521b616927 2847b61f52aba9c2e259456f55aeb469813f1d72 397ed1f0311e474b909cb3274baba3d4b65b79f2 2c3271497cd711695d6158eb90f3d29d95fe257d d29a9b27c4058a89e0e691823c869730fd9c758e 740b2772f9cee755f09e173ec5d6b5c193f841e5 6f1b8806df5a179e3c1bb0c1a78d98cce591b448 22caa6fc166c4cf79f80d982cd7089dd3f46675e e7c1b7041fe61782691817aa164f632b9a9fa49f ece995e7d6a78d86b6ad18171c4a29a506ab9bec 4fc8c21c1acc97aabd92ce954d112471460a1ad2 7a34bd0d8c663bbfbac6e9213da8ed1a9cb88745 a1e62ba53ea216cc6922f2b0f092a13a9f029499 ebd310c32697347f447eea5f3c3f4e4348c964a2 cc027bde8f1aaa1a10d0c5e8e3ca0bd93aaf91d9 fbba14bf6638efd9ae54ec98f2566cddd7132925 4b9456787539d0738e499a6e319ac0c5dfd7d5e8 d493bc54e9624c6da328c86f6e821b6bb9b1cec8 f253e7cee3fef3fc31dfbd7516dcd4a460440fea 1d010a3e45b1277a99f1123844cbf749ffdfb964 d9e536a03e5426ede2f4d587a8c2ac3230ef4845 d26df0a1f9e087d11f933950f8b72fac904d2371 c7777000782e4183de233bdcda01baef4589bf5c 1d062c5120d769228ff9e09e264048e87abacc44 6cd7875196eb82430fc8355e696e316294bb9e5e c2ea61dd8dd847705688b23b5705123a15b60432 ec9a9bc7c42bfd4b4dc63942352b48bc68d24966 cea90a0ed923d6f32931f081f73eefb1d910ef84 8b6afd0e2bf4ad009dadf8758b24f02c39cfd118 16bf9c95cd9a185332d17a7b9d529f606fe2a72e 5107f53e0cb480fd0acc7feec19d6cb5cdd3ee25 f0c03aeb24c8c8f18faa9f596eb70a7bd6225d49 9fbd9303feffa927eebfc5f12be4e0317b096dc0 8fa93313e5f42f5551a97dff043526ac093f8f02 c305335e4e5a47edbcfd84183be4300a85091bac 2d6593c8c248db31659a07780197ace22e843dec 6e3f08a16744a84e47813ed11acb26f0fef8dc80 18d8bbb3aa683372d8645d56f67c628571c14de7 0ea8bfd7c99e78ae7b36023ce72bb6be550c610a 668676e9b04207c0945154d0d3485b47a3d33948 32d81a1d34fde3d0cee96805d83dd68e5ddaec91 a55e9a2c2329f32987bfbc1460ce45b02d70ee67 e78cb58b9b0d0f336ea824b120dedf27e1bc643e 5f1ffc2b6a29fee23e614556edde836902e55a11 893e82754f6222a836ad735b8da9c6b120364afd 590c299607b8bc95abfac216161e62e8d8885477 499749c5f74cad29f64f7535c2917aa8236c88e0 1bce4c06eb33bb00fd4e1b422ee134a9f8147c18 42daef8deda59e5dd2ffb575e1a8b0755264812e efd0a5e1da1d19b38bf1b91ca6be3d8b1ca55b95 ee728950872020e57ad41c3fab65fe6f22ff4837 c803784020b7713a6b5063b838aa032fcfe88f33 533594dff507e93b4a2e4b71631b6297308294f6 a8593ae657f10d1f3c4d814c4506269d83960bcc a97a82203cd17ca1638aeca5df5fee52b7d6938d 96865aaca468b16c299f0d49988882d542933274

redisearch-backport-pull-request · 2025-12-04T15:55:12Z

Backport failed for 8.4, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 8.4
git worktree add -d .worktree/backport-7202-to-8.4 origin/8.4
cd .worktree/backport-7202-to-8.4
git switch --create backport-7202-to-8.4
git cherry-pick -x b9e1fe9270c72dd8d6b64815fd7dcb5f7b0e3d6a 6faa9b5d2a53970fcdabe64566645099a940d264 590c01d86c3001827a9d11647e647967273e19b0 1353a4a519be78f5c6d440c6e32b36d66cba476e 5f1b4f345ccd302854b9080025a49800373e272d fc72f1aff73d232398ec492f2eeac03ff3807f61 7d94c4453234893335bacdd541540f2f9cb8a096 b254b4f00cf8081f360ca3ce2bbc3ad48c379816 dabd1c8c6d6f1f0121421302639bb73935bdb619 8bcda5f1b168e16170db0c15de6a99b8ab696383 b9520c3962d40e09f80af916b1745ece857a526c ee73faec9177c461945583163cf6e4aeb49ba167 d2c5257a9d501a62b8eb701b4c1faf3b7cb997ed e83797f95c8443a5534bdafa2746d63fd9e4b826 98debd8875522bfa5ff05e4bdf4bb75e311cfd15 3a3cc5b89edfcaf0f1aab105aa9c1782583c6e63 21ac10a81fc74217aa203067408e5c3a1ae471f0 0acca45f8822a2c4edf6cf191edc62ad4a6fb882 568910963e009443a0cc866bc994abf505173a8a 81cc473de4b4c9dd312d6c706f397592ffe1829f b14371e41464b7bc7cee1ce9333b3eabf8c1cad0 f1ef205a1961cc67243ffbeb7ed4fc2665752f41 e4690f31785438253d245a539cf3c70688f45e2a 794f56e53a50ada5c9593ae971fdf81aa02ba2ac fcf5bc1a423696fbc70d95e03af1f2420bc71344 800f4da4688a2a7fbaa642b87a562b3ecd84fa87 b6bfeacdc0e6033a943918de5e07c4d345132269 bb3a421a5414bdb98826f241ed95768d220cd692 8d31eb9ade8f3d333246743b28ff34ddd7b2502e 6f3a33e34a833787e39ddc7ee3b2e52c30b8fb4b 8a1daf86de3b29a505ee35159b3b247acbe4127a 06340eef2980d4d3e49baae7240d597b4b6c9068 8ed83d8842377283da83089aa5b547ba0e67f93c 14b13f7b1fe60924932bc64c7a846df73180fd56 4c2698ad284b58397811f063c1bc3d24a34cb96d ed6f5d1ea4f02b8663bd03c2e174ae1563b17d3c bf082b0c6ec1c0e1d1847323c3003347386b5e0a 194f569def87afa684992e6cd6310cd850da5773 5b86ba846e696a4f73fc8ad55d1800a5664d4baf 36686c6d4fd6f08b42bbee3aaaf8d21db781313a 78f35c36e2df5f7f4d369afbe408e67679286bc6 b5a679eb7909f583de04cb9cbbff8fb31083c717 5134787ad91db4a80a6f3abb0693bed81ff4dbee e200b6983e188eee7b19c42e57be562337a34bfa 3548f730bddae94ef5a89a4327e62405d42cc9b4 205ed788bbd80a298a77b1ab5b6c87275ac4c57d d88b048e915300a259918dcda704878b77eba68a e0e9723a3863d3b3f078c46b04bab3b8e8fdc746 883a499c4f60ba8c2460d69ffc24245b9d28b276 546d4aaaca5b73a92f7e55e21f16abec3412f11b 589f83e5ee059e493610aea01bf0a739e5c7a3ee 0fdb36c384eb5f0341a96602240020437ad6d32c 746b16264816493d534e44bcda2bd701344b6258 6b8197481c7795c86534feaaaaa78c005e2e1de3 b529cce68cc03105f70d93aed65e63814036636d b13563d0e73e8537eebd2391bf1b265f4d531767 4cee2c93d3d48870a15af06fd1e3213d095cf140 4bfb160f8c964f274b97d092b01e386dd9a90243 8a7a74b46e860d00c7e25b44362bc00f1f9f2fc0 a29158db15745e564c55e689b60b408d9cd3ad1b c1029b3b9165d9815fb4d218b693fd3a89b7c4fb cbd20925f936e0d102de56d44edda470baf49510 9618db9d4caf120fac579691b0d8b71e6cc4cb69 b1935726948eb81e80ca8a94aa820f5f5d433b50 238ba5bb6288d67d8f84cefe271034d77c715388 000de3c98cb418da78ce7119b6fa9942aadee15d e721a15f69c88a782806448e3638f99dd33a3eba 7a92af385fa5878607281b8b71e4a645fb9e83d9 f9e99db8aecaf327c7d9dca88abd626a4c409b39 437f143fd17a34531d4ff7cfe6628db4a92ab859 a48e7c966911563082dd974354405a861fcab6e2 47c86c54f90ae30e458080cea9cf860952c3941c 5764857b819f036438396ce1d45afd1e1acaafd7 ef793557708fd7bf20960e5d23dd3231f323e012 523bf261c891d4738847896643b1e140973a1a94 d6291d2db7724350765085fa74b365b1b9c9e270 d3beb32b75082e65a8051d6e40cd55521b616927 2847b61f52aba9c2e259456f55aeb469813f1d72 397ed1f0311e474b909cb3274baba3d4b65b79f2 2c3271497cd711695d6158eb90f3d29d95fe257d d29a9b27c4058a89e0e691823c869730fd9c758e 740b2772f9cee755f09e173ec5d6b5c193f841e5 6f1b8806df5a179e3c1bb0c1a78d98cce591b448 22caa6fc166c4cf79f80d982cd7089dd3f46675e e7c1b7041fe61782691817aa164f632b9a9fa49f ece995e7d6a78d86b6ad18171c4a29a506ab9bec 4fc8c21c1acc97aabd92ce954d112471460a1ad2 7a34bd0d8c663bbfbac6e9213da8ed1a9cb88745 a1e62ba53ea216cc6922f2b0f092a13a9f029499 ebd310c32697347f447eea5f3c3f4e4348c964a2 cc027bde8f1aaa1a10d0c5e8e3ca0bd93aaf91d9 fbba14bf6638efd9ae54ec98f2566cddd7132925 4b9456787539d0738e499a6e319ac0c5dfd7d5e8 d493bc54e9624c6da328c86f6e821b6bb9b1cec8 f253e7cee3fef3fc31dfbd7516dcd4a460440fea 1d010a3e45b1277a99f1123844cbf749ffdfb964 d9e536a03e5426ede2f4d587a8c2ac3230ef4845 d26df0a1f9e087d11f933950f8b72fac904d2371 c7777000782e4183de233bdcda01baef4589bf5c 1d062c5120d769228ff9e09e264048e87abacc44 6cd7875196eb82430fc8355e696e316294bb9e5e c2ea61dd8dd847705688b23b5705123a15b60432 ec9a9bc7c42bfd4b4dc63942352b48bc68d24966 cea90a0ed923d6f32931f081f73eefb1d910ef84 8b6afd0e2bf4ad009dadf8758b24f02c39cfd118 16bf9c95cd9a185332d17a7b9d529f606fe2a72e 5107f53e0cb480fd0acc7feec19d6cb5cdd3ee25 f0c03aeb24c8c8f18faa9f596eb70a7bd6225d49 9fbd9303feffa927eebfc5f12be4e0317b096dc0 8fa93313e5f42f5551a97dff043526ac093f8f02 c305335e4e5a47edbcfd84183be4300a85091bac 2d6593c8c248db31659a07780197ace22e843dec 6e3f08a16744a84e47813ed11acb26f0fef8dc80 18d8bbb3aa683372d8645d56f67c628571c14de7 0ea8bfd7c99e78ae7b36023ce72bb6be550c610a 668676e9b04207c0945154d0d3485b47a3d33948 32d81a1d34fde3d0cee96805d83dd68e5ddaec91 a55e9a2c2329f32987bfbc1460ce45b02d70ee67 e78cb58b9b0d0f336ea824b120dedf27e1bc643e 5f1ffc2b6a29fee23e614556edde836902e55a11 893e82754f6222a836ad735b8da9c6b120364afd 590c299607b8bc95abfac216161e62e8d8885477 499749c5f74cad29f64f7535c2917aa8236c88e0 1bce4c06eb33bb00fd4e1b422ee134a9f8147c18 42daef8deda59e5dd2ffb575e1a8b0755264812e efd0a5e1da1d19b38bf1b91ca6be3d8b1ca55b95 ee728950872020e57ad41c3fab65fe6f22ff4837 c803784020b7713a6b5063b838aa032fcfe88f33 533594dff507e93b4a2e4b71631b6297308294f6 a8593ae657f10d1f3c4d814c4506269d83960bcc a97a82203cd17ca1638aeca5df5fee52b7d6938d 96865aaca468b16c299f0d49988882d542933274

nafraf added 6 commits October 31, 2025 10:07

WIP: Support WITHCOUNT in FT.AGGREGATE

b9e1fe9

buffered results in sendChunk_Resp2()

6faa9b5

Don't add QEXEC_OPTIMIZE flag to FT.AGGREGATE

590c01d

Add RPDepleter_NewSync()

1353a4a

Disable optimization if FT.AGGREGATE + SORTBY (bwc)

5f1b4f3

Fixing backward compatibility

fc72f1a

github-actions bot added the size:M label Oct 31, 2025

nafraf commented Oct 31, 2025

View reviewed changes

src/aggregate/aggregate_request.c Outdated Show resolved Hide resolved

nafraf and others added 9 commits October 31, 2025 18:12

WIP: disable assertion.

7d94c44

Revert "WIP: disable assertion."

b254b4f

This reverts commit 7d94c44.

Don't use QOptimizer_UpdateTotalResults() for FT.AGGREGATE

dabd1c8

Fix src/aggregate/aggregate_request.c comment

8bcda5f

Fix resp2 + LIMIT

b9520c3

Fix RESP3 + SORTBY in cluster mode. Get total_results from meta data

ee73fae

Refactor test_aggregate_count.py

d2c5257

Fix test:test_timeoutCoordSearch_NonStrict()

e83797f

Simplify RPDepleter_NewSync() args

98debd8

nafraf marked this pull request as ready for review November 3, 2025 08:22

nafraf requested a review from oshadmi November 3, 2025 08:22

This comment was marked as outdated.

Sign in to view

oshadmi reviewed Nov 4, 2025

View reviewed changes

tests/pytests/test.py Outdated Show resolved Hide resolved

oshadmi reviewed Nov 4, 2025

View reviewed changes

src/aggregate/aggregate_exec.c Outdated Show resolved Hide resolved

oshadmi reviewed Nov 4, 2025

View reviewed changes

src/coord/dist_aggregate.c Show resolved Hide resolved

oshadmi reviewed Nov 4, 2025

View reviewed changes

src/coord/rpnet.c Outdated Show resolved Hide resolved

oshadmi reviewed Nov 4, 2025

View reviewed changes

src/pipeline/pipeline_construction.c Outdated Show resolved Hide resolved

oshadmi reviewed Nov 4, 2025

View reviewed changes

src/pipeline/pipeline_construction.c Outdated Show resolved Hide resolved

Fix SORTBY handling in handleCommonArgs to ensure flags are set corre…

3a3cc5b

…ctly

cursor bot reviewed Nov 4, 2025

View reviewed changes

src/aggregate/aggregate_request.c Outdated Show resolved Hide resolved

Add assertion for unreachable code in RPDepleter_Next_Sync

21ac10a