[MOD-12627] Add Debug Support for `FT.PROFILE` Command by meiravgri · Pull Request #7510 · RediSearch/RediSearch

meiravgri · 2025-11-25T15:21:25Z

This PR extends query debug capabilities to support the FT.PROFILE command.

Example Usage

# Standalone: Profile with timeout after 5 results
_FT.DEBUG FT.PROFILE idx SEARCH QUERY "@field:value*" DEBUG_PARAMS_COUNT 2 TIMEOUT_AFTER_N 5

Commands added:

_FT.DEBUG FT.PROFILE <index> <SEARCH|AGGREGATE> [LIMITED] QUERY <query> [options] DEBUG_PARAMS_COUNT <count> [debug_params...]
_FT.DEBUG _FT.PROFILE (internal shard command)

Execution Flows

Standalone Mode

User: _FT.DEBUG FT.PROFILE idx SEARCH|AGGREGATE QUERY ... DEBUG_PARAMS_COUNT 2 TIMEOUT_AFTER_N 5
  ↓
ProfileCommandCommand_DebugWrapper (debug_commands.c)
  ↓ 
ProfileCommandHandlerImp(ctx, argv, argc, isDebug=true) (module.c)
  ↓ (NumShards == 1)
RSProfileCommandImp(ctx, argv, argc, isDebug=true) (aggregate_exec.c)
  ↓ 
DEBUG_execCommandCommon (with EXEC_DEBUG | EXEC_WITH_PROFILE flags)
  ↓
Builds query pipeline with debug RPs (RP_TIMEOUT, RP_PAUSE, etc.)
  ↓
Returns profile with timeout warning

Cluster Mode

User: _FT.DEBUG FT.PROFILE idx SEARCH|AGGREGATE QUERY ... DEBUG_PARAMS_COUNT 2 TIMEOUT_AFTER_N 3
  ↓
ProfileCommandCommand_DebugWrapper (coordinator)
  ↓
ProfileCommandHandlerImp(ctx, argv, argc, isDebug=true)
  ↓ (detects SEARCH or AGGREGATE)
DistSearchCommandImp / DistAggregateCommandImp (ctx, argv, argc, isDebug=true)
  ↓ (selects DEBUG_DistSearchCommandHandler / DEBUG_RSExecDistAggregate callback)
Coordinator sends commands to all shards
  ↓
[Shard 1, 2, 3]: _FT.DEBUG _FT.PROFILE idx SEARCH|AGGREGATE QUERY ... DEBUG_PARAMS_COUNT 2 TIMEOUT_AFTER_N 3
  ↓
RSProfileCommandShard (debug_commands.c) → RSProfileCommandImp(isDebug=true)
  ↓
Each shard executes with debug RPs and returns profile
  ↓
Coordinator aggregates results and profiles from all shards
  ↓
Returns combined profile with Shards[] and Coordinator sections

Implementation Details

Profile Printing: Handling Debug Result Processors

File: src/profile.c (lines 79-82)

Debug result processors (RP_TIMEOUT, RP_CRASH, RP_PAUSE) have types `>= RP_MAX`. Added an early return in `_recursiveProfilePrint` to skip debug RPs before opening the Map, preventing them from appearing in profile output

Refactoring: `*Imp` Pattern for Debug Support

Files: src/module.c, src/module.h, src/aggregate/aggregate_exec.c

Created *Imp versions of command functions that accept a bool isDebug parameter:

DistSearchCommandImp(ctx, argv, argc, isDebug)
DistAggregateCommandImp(ctx, argv, argc, isDebug)
ProfileCommandHandlerImp(ctx, argv, argc, isDebug)
RSProfileCommandImp(ctx, argv, argc, isDebug)

Pattern:

Original functions call *Imp versions with isDebug=false
Debug wrappers call *Imp versions with isDebug=true
Removed manual _FT.DEBUG detection logic (previously used RMUtil_ArgIndex)

This allow to reuses distributed command code while maintaining existing assumptions about argument order, avoiding code duplication.

Note

Add debug-mode support for FT.PROFILE across SA/cluster with new *Imp handlers and debug registration, refactor distributed paths to pass isDebug, hide debug RPs in profiling, and add comprehensive tests.

Debug/Profiling:
- Add RSProfileCommandImp and ProfileCommandHandlerImp with isDebug flag; route debug profile to DEBUG_execCommandCommon and set EXEC_DEBUG.
- Register debug commands for profiling: FT.PROFILE and _FT.PROFILE under _FT.DEBUG, plus shard wrappers.
- Refactor distributed entrypoints to DistSearchCommandImp/DistAggregateCommandImp (and profile handler) with isDebug propagation; update debug wrappers to use these.
- Profile output: introduce RP_MAX_DEBUG and skip debug RPs in profile printer.
Result Processors:
- Guard profile print on RP type bounds; minor formatting fixes.
Tests:
- Add SA/cluster RESP2/RESP3 tests validating debug FT.PROFILE behavior (TIMEOUT_AFTER_N, structure parity, warnings) and update debug help list.
- Add countFlatElements utility in test common.

^{Written by Cursor Bugbot for commit d426fe6. This will update automatically on new commits. Configure here.}

introduce in mocule.h: RSProfileCommandImp RSProfileCommand calls RSProfileCommandImp(isDebug = false) for regular execution ProfileCommandCommand_DebugWrapper mcalls it with isDebug=true and skips _FT.DEBUG introduce entrypoint for _FT.DEBUG FT.PROFILE in debug_commands: ProfileCommandCommand_DebugWrapper RSProfileCommandImp calls DEBUG_execCommandCommon is its debug _recursiveProfilePrint skips printing debug RP

src/aggregate/aggregate_debug.c

module.h: replace declaration: DistAggregateCommand DistSearchCommand with Imp version that receives isDebug expose ProfileCommandHandlerImp align debug_commands introducr _FT.DEBUG _FT.PROFILE

src/module.c

tests/pytests/test_profile.py

tests/pytests/test_debug_commands.py

src/profile.c

codecov · 2025-11-26T06:38:13Z

Codecov Report

❌ Patch coverage is 91.17647% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.98%. Comparing base (874ce68) to head (d426fe6).
⚠️ Report is 22 commits behind head on master.

Files with missing lines	Patch %	Lines
src/debug_commands.c	83.33%	2 Missing ⚠️
src/profile.c	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7510      +/-   ##
==========================================
- Coverage   85.00%   84.98%   -0.03%     
==========================================
  Files         349      349              
  Lines       53877    53896      +19     
  Branches    14384    14384              
==========================================
+ Hits        45797    45801       +4     
- Misses       7884     7899      +15     
  Partials      196      196

Flag	Coverage Δ
flow	`85.03% <91.17%> (-0.16%)`	⬇️
unit	`52.36% <2.94%> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

lerman25 · 2025-11-27T13:50:10Z

src/debug_commands.c

+    return RedisModule_WrongArity(ctx);
+  }
+
+  return ProfileCommandHandlerImp(ctx, ++argv, --argc, true);


Major nitpick -
I think using ++ and -- implies you intend to use those variables afterwards,
Isn't +1 and -1 more straightforward?

Better to be aligned with all other similar commands, no ?

lerman25

Great work!

redisearch-backport-pull-request · 2025-11-30T00:07:34Z

Backport failed for 2.8, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 2.8
git worktree add -d .worktree/backport-7510-to-2.8 origin/2.8
cd .worktree/backport-7510-to-2.8
git switch --create backport-7510-to-2.8
git cherry-pick -x 00ca3cfa686f171ddc0af800e4daca275ed39219

redisearch-backport-pull-request · 2025-11-30T00:07:35Z

Backport failed for 2.10, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 2.10
git worktree add -d .worktree/backport-7510-to-2.10 origin/2.10
cd .worktree/backport-7510-to-2.10
git switch --create backport-7510-to-2.10
git cherry-pick -x 00ca3cfa686f171ddc0af800e4daca275ed39219

redisearch-backport-pull-request · 2025-11-30T00:07:36Z

Backport failed for 8.2, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 8.2
git worktree add -d .worktree/backport-7510-to-8.2 origin/8.2
cd .worktree/backport-7510-to-8.2
git switch --create backport-7510-to-8.2
git cherry-pick -x 00ca3cfa686f171ddc0af800e4daca275ed39219

* imp debug profile for SA: introduce in mocule.h: RSProfileCommandImp RSProfileCommand calls RSProfileCommandImp(isDebug = false) for regular execution ProfileCommandCommand_DebugWrapper mcalls it with isDebug=true and skips _FT.DEBUG introduce entrypoint for _FT.DEBUG FT.PROFILE in debug_commands: ProfileCommandCommand_DebugWrapper RSProfileCommandImp calls DEBUG_execCommandCommon is its debug _recursiveProfilePrint skips printing debug RP * pass is debug instead of extracting: module.h: replace declaration: DistAggregateCommand DistSearchCommand with Imp version that receives isDebug expose ProfileCommandHandlerImp align debug_commands introducr _FT.DEBUG _FT.PROFILE * add test for cluster * return res * augi fixes * fix spell check * fix for real * fix test * skip tests according to env * revrt test_profile changes * reove changes from internal_only (cherry picked from commit 00ca3cf)

redisearch-backport-pull-request · 2025-11-30T00:07:41Z

Successfully created backport PR for 8.4:

[8.4] [MOD-12627] Add Debug Support for FT.PROFILE Command #7573

[MOD-12627] Add Debug Support for `FT.PROFILE` Command (#7510) * imp debug profile for SA: introduce in mocule.h: RSProfileCommandImp RSProfileCommand calls RSProfileCommandImp(isDebug = false) for regular execution ProfileCommandCommand_DebugWrapper mcalls it with isDebug=true and skips _FT.DEBUG introduce entrypoint for _FT.DEBUG FT.PROFILE in debug_commands: ProfileCommandCommand_DebugWrapper RSProfileCommandImp calls DEBUG_execCommandCommon is its debug _recursiveProfilePrint skips printing debug RP * pass is debug instead of extracting: module.h: replace declaration: DistAggregateCommand DistSearchCommand with Imp version that receives isDebug expose ProfileCommandHandlerImp align debug_commands introducr _FT.DEBUG _FT.PROFILE * add test for cluster * return res * augi fixes * fix spell check * fix for real * fix test * skip tests according to env * revrt test_profile changes * reove changes from internal_only (cherry picked from commit 00ca3cf) Co-authored-by: meiravgri <[email protected]>

* imp debug profile for SA: introduce in mocule.h: RSProfileCommandImp RSProfileCommand calls RSProfileCommandImp(isDebug = false) for regular execution ProfileCommandCommand_DebugWrapper mcalls it with isDebug=true and skips _FT.DEBUG introduce entrypoint for _FT.DEBUG FT.PROFILE in debug_commands: ProfileCommandCommand_DebugWrapper RSProfileCommandImp calls DEBUG_execCommandCommon is its debug _recursiveProfilePrint skips printing debug RP * pass is debug instead of extracting: module.h: replace declaration: DistAggregateCommand DistSearchCommand with Imp version that receives isDebug expose ProfileCommandHandlerImp align debug_commands introducr _FT.DEBUG _FT.PROFILE * add test for cluster * return res * augi fixes * fix spell check * fix for real * fix test * skip tests according to env * revrt test_profile changes * reove changes from internal_only (cherry picked from commit 00ca3cf)

…7574) * [MOD-12627] Add Debug Support for `FT.PROFILE` Command (#7510) * imp debug profile for SA: introduce in mocule.h: RSProfileCommandImp RSProfileCommand calls RSProfileCommandImp(isDebug = false) for regular execution ProfileCommandCommand_DebugWrapper mcalls it with isDebug=true and skips _FT.DEBUG introduce entrypoint for _FT.DEBUG FT.PROFILE in debug_commands: ProfileCommandCommand_DebugWrapper RSProfileCommandImp calls DEBUG_execCommandCommon is its debug _recursiveProfilePrint skips printing debug RP * pass is debug instead of extracting: module.h: replace declaration: DistAggregateCommand DistSearchCommand with Imp version that receives isDebug expose ProfileCommandHandlerImp align debug_commands introducr _FT.DEBUG _FT.PROFILE * add test for cluster * return res * augi fixes * fix spell check * fix for real * fix test * skip tests according to env * revrt test_profile changes * reove changes from internal_only (cherry picked from commit 00ca3cf) * include module.h

@alonre24

* fix: Avoid Rust cache contamination across platforms. (#7569) * [MOD-12170] Implement ASM State Machine on notifications (#7331) * first commit * fix: handle lock in two phases * inform ownership in regular command process handlers * fix: handle proper init * fix: use conditional variable * fix: add some comments * test: add test proving deadlock * fix: changes as per comments * fix: remove this locking from conn * remove code where certainity is not so high * fix potential issue * test: add first unit tests version * test: add proper testing with the value proposition * test: improve testing * alternative using auxiliary lock * simplify thpool a bit * make sure RedisModule_Yield is protected * fix: fix issue raised by cursor * fix: handle potential deadlock in drain also * fix: fix potential TOCTOU concurrency bug * fix: fix order of release * fix: fix potential overflow issue * fix tests * fix: fix counts in all variables * small refactor * some refactoring of Shared Exclusive Lock * simplify * protect GILOwned simple bool * clarify some comments * set GILAlternativeLockHeld to true properly * fix spelling * add assertion * fix: fix import RS_LOG_ASSERT * test: add more conditions to testing * add some more logic * improve testing to proper signal main thread can finish while other threads may be waiting for the Shared Lock in the loop * test: handl test properly * add another pattern of tests * fix comments from cursor * fix concurrency bug * fix: fix potential race condition at release lock time * fix: add condition * test: add more testing * force testing further to capture more potential errors * parametrize tests * test: make tests a little faster * test: add micro benchmark * fix compile microbenchmarks * fix: avoid potential reentrant deadlocks * fix: avoid potential reentrant deadlocks * test: avoid leak in test * test: avoid leak in test * fix: fix assertion * fix: fix assertion * Simplify Shared Lock internals (#7267) * simplify shared lock * small improvement to set_timeout * fix comment * fix nd improve comments * condition fix * remove lock type from release API * Add lock type back to the release API * remove Unlocked from enum and handle clock init for macOS * adapt to use new API * chnange according to comments * handle PR comments * handle PR comments * fix: take shared lock in other cases * change as PR comments * test: simplify test, do not allow query errors * fix tests as per comments * fix: handle number of high priority jobs running * fix: fix test comment * add ASM to help slot tracking in notifications * test: add some testing idea * change as per PR comments * compile and link test fix * change draining method to drain high priority * add ctests for ASM State Machine * test: add ASM tests * test: complete tests * change as per PR comments * add micro benchmarks with jobs in threads * remove changes not wanted * checkout redis feature branch in task test * remove draining * move atomic to new header * remove _internal naming * fix formatting --------- Co-authored-by: GuyAv46 <[email protected]> * [MOD-12627] Add Debug Support for `FT.PROFILE` Command (#7510) * imp debug profile for SA: introduce in mocule.h: RSProfileCommandImp RSProfileCommand calls RSProfileCommandImp(isDebug = false) for regular execution ProfileCommandCommand_DebugWrapper mcalls it with isDebug=true and skips _FT.DEBUG introduce entrypoint for _FT.DEBUG FT.PROFILE in debug_commands: ProfileCommandCommand_DebugWrapper RSProfileCommandImp calls DEBUG_execCommandCommon is its debug _recursiveProfilePrint skips printing debug RP * pass is debug instead of extracting: module.h: replace declaration: DistAggregateCommand DistSearchCommand with Imp version that receives isDebug expose ProfileCommandHandlerImp align debug_commands introducr _FT.DEBUG _FT.PROFILE * add test for cluster * return res * augi fixes * fix spell check * fix for real * fix test * skip tests according to env * revrt test_profile changes * reove changes from internal_only * [MOD-12694] [MOD-12069] Add active_coord_threads metric (#7546) * Add multi-threading statistics tracking for active I/O threads * fix comment * add cpp test * fix spelling * address comment * remove unnecessary nre line * add "active_worker_threads" metric * fix comment imp tests * test cleanups * add test comne about the num queries and cleanups * fix declartion * remove coord threads * add active_coord_threads expose ConcurrentSearchPool_WorkingThreadCount * make the tests run... * add "active_worker_threads" metric * fix declartion * remove coord threads * make the tests run... * introduce workersThreadPool_isInitialized assert is initizlied in GlobalStats_GetMultiThreadingStats * cleanup * rename workersThreadPool_isCreated * introduce ConcurrentSearchPool_IsCreated * fix test * we dont need workers * remove ConcurrentSearchPool_IsCreated and workersThreadPool_isInitialized * fix merge * [MOD-12789] test: fix flaky thpool test (#7581) test: fix flaky thpool test * Support Multiple Slot Ranges in search.CLUSTERSET - [MOD-11657] (#7508) * support multiple slot ranges * implement and move around helpers * add tests * ignore shards with no slots * improve parsing * better error message * fix flow tests * fix tests * sort by node id * improve error testing * cover missing cases * more error messages improvements * address AI review * stabilize Unexpected argument error path * stabilize more error paths * fix tests accordingly * last fix * add logs to cluster set command * add more logs per @alonre24 request * rename MetricIterator to Metric (#7586) The other iterators are not suffixed with 'Iterator'. * [MOD-12701] Split the execution of Rust and C/C++ unit tests across two different CI steps. (#7587) Split the execution of Rust and C/C++ unit tests across two different CI steps. * [MOD-12519] implement skip multi in II iterators (#7426) * remove it->skipMulti No longer needed as the full iterators have been removed. The query ones always set it to true. * implement skip multi II query iterators are supposed to skip results having the same ids. Test ported from test_cpp_iterator_index.cpp * inline read() and skip_to() * test more ii iterator edge cases Port of GetCorrectValue and EOFAfterFiltering from test_cpp_iterator_index.cpp * Compress layers prior to exporting them to the Docker layer cache (#7529) * Compress Docker layers prior to exporting them. * Ignore boost subfolders recursively * [MOD-12417] Track maxprefixexpansions errors and warnings in info (#7570) track maxprefixexpanions * [MOD-12409]: Port DocumentType enum to Rust (#7590) Port DocumentType enum to Rust Port DocumentType enum to Rust as `document::DocumentType`, removing it from `redisearch.h`. Use `document::DocumentType` in `rlookup` instead of `rlookup::bindings::DocumentType`. * [MOD-12069] Add `*_pending_jobs` metrics (#7556) * align info/* to active_coord * add APIs to get queues length * add to info * fix * test * fix test * catch general error * rename * fix moduleArgs * rename * rename test_active_worker_threads * rename to wworketrs * [MOD-12392] Remove numDocs parameter from non-optimized Wildcard iterator (#7602) Remove numDocs parameter from non-optimized Wildcard iterator * [MOD-12701] Enforce a per-test timeout in the C++ rstest suite using ctest (#7588) * Enforce a per-test timeout in the C++ rstest suite using ctest * Disable problematic tests * Raise timeout to 60s * Raise timeout * Add a timeout for coordinator tests too * Skip ActivateIoThreadsMetric test * Register Cursor Sub-Commands as such - [MOD-12807, MOD-12808] (#7571) * split cursor command * fix and improve tests * cover error cases * fix cursor leaks * Add "TODO: run hybrid cursor" back * remove new empty line * small test improvement * fix FT.CURSOR GC * de-flake test * make CURSOR PROFILE internal only * test the free * Keep just the prints * test on macos and noble as well * Move the free * Remove * Add to non container * Back to regular test * Remove spaces * Moved the remove and add the repo size * Moved it again * Move it to after repo build * Move print after repo build * test all * run the temp flow * add mount to container * Fix curly --------- Co-authored-by: Luca Palmieri <[email protected]> Co-authored-by: Joan Fontanals <[email protected]> Co-authored-by: GuyAv46 <[email protected]> Co-authored-by: meiravgri <[email protected]> Co-authored-by: Guillaume Desmottes <[email protected]> Co-authored-by: lerman25 <[email protected]> Co-authored-by: Henk Oordt <[email protected]> Co-authored-by: alonre24 <[email protected]>

@alonre24

* Reduce merge queue * add ability to run manually * CR comments * allow workflow call for testing * Fix naming in build image + remove quick from flow intel as well * use ubuntu nobel rather than latest * fix noble typo * CR fixes * CR fixes 2 * don't use container for cov and san * restore leftover * remove mac + intel and workflow call * remove macos intel from matrix * change back ubuntu:latest to ubuntu:noble in merge-to-queue as per Jonathan comment * measure disk space * fix step name * Free disk on container (RediSearch#7613) * fix: Avoid Rust cache contamination across platforms. (RediSearch#7569) * [MOD-12170] Implement ASM State Machine on notifications (RediSearch#7331) * first commit * fix: handle lock in two phases * inform ownership in regular command process handlers * fix: handle proper init * fix: use conditional variable * fix: add some comments * test: add test proving deadlock * fix: changes as per comments * fix: remove this locking from conn * remove code where certainity is not so high * fix potential issue * test: add first unit tests version * test: add proper testing with the value proposition * test: improve testing * alternative using auxiliary lock * simplify thpool a bit * make sure RedisModule_Yield is protected * fix: fix issue raised by cursor * fix: handle potential deadlock in drain also * fix: fix potential TOCTOU concurrency bug * fix: fix order of release * fix: fix potential overflow issue * fix tests * fix: fix counts in all variables * small refactor * some refactoring of Shared Exclusive Lock * simplify * protect GILOwned simple bool * clarify some comments * set GILAlternativeLockHeld to true properly * fix spelling * add assertion * fix: fix import RS_LOG_ASSERT * test: add more conditions to testing * add some more logic * improve testing to proper signal main thread can finish while other threads may be waiting for the Shared Lock in the loop * test: handl test properly * add another pattern of tests * fix comments from cursor * fix concurrency bug * fix: fix potential race condition at release lock time * fix: add condition * test: add more testing * force testing further to capture more potential errors * parametrize tests * test: make tests a little faster * test: add micro benchmark * fix compile microbenchmarks * fix: avoid potential reentrant deadlocks * fix: avoid potential reentrant deadlocks * test: avoid leak in test * test: avoid leak in test * fix: fix assertion * fix: fix assertion * Simplify Shared Lock internals (RediSearch#7267) * simplify shared lock * small improvement to set_timeout * fix comment * fix nd improve comments * condition fix * remove lock type from release API * Add lock type back to the release API * remove Unlocked from enum and handle clock init for macOS * adapt to use new API * chnange according to comments * handle PR comments * handle PR comments * fix: take shared lock in other cases * change as PR comments * test: simplify test, do not allow query errors * fix tests as per comments * fix: handle number of high priority jobs running * fix: fix test comment * add ASM to help slot tracking in notifications * test: add some testing idea * change as per PR comments * compile and link test fix * change draining method to drain high priority * add ctests for ASM State Machine * test: add ASM tests * test: complete tests * change as per PR comments * add micro benchmarks with jobs in threads * remove changes not wanted * checkout redis feature branch in task test * remove draining * move atomic to new header * remove _internal naming * fix formatting --------- Co-authored-by: GuyAv46 <[email protected]> * [MOD-12627] Add Debug Support for `FT.PROFILE` Command (RediSearch#7510) * imp debug profile for SA: introduce in mocule.h: RSProfileCommandImp RSProfileCommand calls RSProfileCommandImp(isDebug = false) for regular execution ProfileCommandCommand_DebugWrapper mcalls it with isDebug=true and skips _FT.DEBUG introduce entrypoint for _FT.DEBUG FT.PROFILE in debug_commands: ProfileCommandCommand_DebugWrapper RSProfileCommandImp calls DEBUG_execCommandCommon is its debug _recursiveProfilePrint skips printing debug RP * pass is debug instead of extracting: module.h: replace declaration: DistAggregateCommand DistSearchCommand with Imp version that receives isDebug expose ProfileCommandHandlerImp align debug_commands introducr _FT.DEBUG _FT.PROFILE * add test for cluster * return res * augi fixes * fix spell check * fix for real * fix test * skip tests according to env * revrt test_profile changes * reove changes from internal_only * [MOD-12694] [MOD-12069] Add active_coord_threads metric (RediSearch#7546) * Add multi-threading statistics tracking for active I/O threads * fix comment * add cpp test * fix spelling * address comment * remove unnecessary nre line * add "active_worker_threads" metric * fix comment imp tests * test cleanups * add test comne about the num queries and cleanups * fix declartion * remove coord threads * add active_coord_threads expose ConcurrentSearchPool_WorkingThreadCount * make the tests run... * add "active_worker_threads" metric * fix declartion * remove coord threads * make the tests run... * introduce workersThreadPool_isInitialized assert is initizlied in GlobalStats_GetMultiThreadingStats * cleanup * rename workersThreadPool_isCreated * introduce ConcurrentSearchPool_IsCreated * fix test * we dont need workers * remove ConcurrentSearchPool_IsCreated and workersThreadPool_isInitialized * fix merge * [MOD-12789] test: fix flaky thpool test (RediSearch#7581) test: fix flaky thpool test * Support Multiple Slot Ranges in search.CLUSTERSET - [MOD-11657] (RediSearch#7508) * support multiple slot ranges * implement and move around helpers * add tests * ignore shards with no slots * improve parsing * better error message * fix flow tests * fix tests * sort by node id * improve error testing * cover missing cases * more error messages improvements * address AI review * stabilize Unexpected argument error path * stabilize more error paths * fix tests accordingly * last fix * add logs to cluster set command * add more logs per @alonre24 request * rename MetricIterator to Metric (RediSearch#7586) The other iterators are not suffixed with 'Iterator'. * [MOD-12701] Split the execution of Rust and C/C++ unit tests across two different CI steps. (RediSearch#7587) Split the execution of Rust and C/C++ unit tests across two different CI steps. * [MOD-12519] implement skip multi in II iterators (RediSearch#7426) * remove it->skipMulti No longer needed as the full iterators have been removed. The query ones always set it to true. * implement skip multi II query iterators are supposed to skip results having the same ids. Test ported from test_cpp_iterator_index.cpp * inline read() and skip_to() * test more ii iterator edge cases Port of GetCorrectValue and EOFAfterFiltering from test_cpp_iterator_index.cpp * Compress layers prior to exporting them to the Docker layer cache (RediSearch#7529) * Compress Docker layers prior to exporting them. * Ignore boost subfolders recursively * [MOD-12417] Track maxprefixexpansions errors and warnings in info (RediSearch#7570) track maxprefixexpanions * [MOD-12409]: Port DocumentType enum to Rust (RediSearch#7590) Port DocumentType enum to Rust Port DocumentType enum to Rust as `document::DocumentType`, removing it from `redisearch.h`. Use `document::DocumentType` in `rlookup` instead of `rlookup::bindings::DocumentType`. * [MOD-12069] Add `*_pending_jobs` metrics (RediSearch#7556) * align info/* to active_coord * add APIs to get queues length * add to info * fix * test * fix test * catch general error * rename * fix moduleArgs * rename * rename test_active_worker_threads * rename to wworketrs * [MOD-12392] Remove numDocs parameter from non-optimized Wildcard iterator (RediSearch#7602) Remove numDocs parameter from non-optimized Wildcard iterator * [MOD-12701] Enforce a per-test timeout in the C++ rstest suite using ctest (RediSearch#7588) * Enforce a per-test timeout in the C++ rstest suite using ctest * Disable problematic tests * Raise timeout to 60s * Raise timeout * Add a timeout for coordinator tests too * Skip ActivateIoThreadsMetric test * Register Cursor Sub-Commands as such - [MOD-12807, MOD-12808] (RediSearch#7571) * split cursor command * fix and improve tests * cover error cases * fix cursor leaks * Add "TODO: run hybrid cursor" back * remove new empty line * small test improvement * fix FT.CURSOR GC * de-flake test * make CURSOR PROFILE internal only * test the free * Keep just the prints * test on macos and noble as well * Move the free * Remove * Add to non container * Back to regular test * Remove spaces * Moved the remove and add the repo size * Moved it again * Move it to after repo build * Move print after repo build * test all * run the temp flow * add mount to container * Fix curly --------- Co-authored-by: Luca Palmieri <[email protected]> Co-authored-by: Joan Fontanals <[email protected]> Co-authored-by: GuyAv46 <[email protected]> Co-authored-by: meiravgri <[email protected]> Co-authored-by: Guillaume Desmottes <[email protected]> Co-authored-by: lerman25 <[email protected]> Co-authored-by: Henk Oordt <[email protected]> Co-authored-by: alonre24 <[email protected]> * Revert "Free disk on container (RediSearch#7613)" This reverts commit 9c68740. * add free disk step * rename + remove temp test * remove leftover * better readability in container input Co-authored-by: GuyAv46 <[email protected]> * fix double defaults --------- Co-authored-by: dor-forer <[email protected]> Co-authored-by: Luca Palmieri <[email protected]> Co-authored-by: Joan Fontanals <[email protected]> Co-authored-by: GuyAv46 <[email protected]> Co-authored-by: meiravgri <[email protected]> Co-authored-by: Guillaume Desmottes <[email protected]> Co-authored-by: lerman25 <[email protected]> Co-authored-by: Henk Oordt <[email protected]>

backport #7510 to 2.8

github-actions bot added the size:M label Nov 25, 2025

cursor bot reviewed Nov 25, 2025

View reviewed changes

src/aggregate/aggregate_debug.c Outdated Show resolved Hide resolved

pass is debug instead of extracting:

5196b17

module.h: replace declaration: DistAggregateCommand DistSearchCommand with Imp version that receives isDebug expose ProfileCommandHandlerImp align debug_commands introducr _FT.DEBUG _FT.PROFILE

cursor bot reviewed Nov 25, 2025

View reviewed changes

src/module.c Show resolved Hide resolved

meiravgri added 2 commits November 25, 2025 16:58

add test for cluster

ab6dd0f

return res

1d6678e

cursor bot reviewed Nov 25, 2025

View reviewed changes

src/module.c Outdated Show resolved Hide resolved

meiravgri added 3 commits November 25, 2025 17:09

augi fixes

e551fca

fix spell check

c3c9819

fix for real

532eb5a

cursor bot reviewed Nov 25, 2025

View reviewed changes

tests/pytests/test_profile.py Outdated Show resolved Hide resolved

tests/pytests/test_debug_commands.py Show resolved Hide resolved

meiravgri added 5 commits November 25, 2025 17:38

fix test

5a93fd4

Merge branch 'master' into meiravg_debug_profile_query

09a4ca4

skip tests according to env

664c458

revrt test_profile changes

133bc09

reove changes from internal_only

d426fe6

cursor bot reviewed Nov 26, 2025

View reviewed changes

src/profile.c Show resolved Hide resolved

meiravgri added backport 2.8 backport 2.10 backport 8.2 backport 8.4 labels Nov 26, 2025

meiravgri changed the title ~~imp debug profile for SA:~~ [MOD-12627] Add Debug Support for FT.PROFILE Command Nov 26, 2025

meiravgri requested a review from lerman25 November 26, 2025 08:52

lerman25 reviewed Nov 27, 2025

View reviewed changes

lerman25 approved these changes Nov 27, 2025

View reviewed changes

meiravgri added this pull request to the merge queue Nov 27, 2025

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 27, 2025

meiravgri added this pull request to the merge queue Nov 27, 2025

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 27, 2025

meiravgri added this pull request to the merge queue Nov 28, 2025

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 28, 2025

meiravgri added this pull request to the merge queue Nov 28, 2025

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 28, 2025

meiravgri added this pull request to the merge queue Nov 29, 2025

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 29, 2025

meiravgri added this pull request to the merge queue Nov 29, 2025

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 29, 2025

meiravgri added this pull request to the merge queue Nov 29, 2025

Merged via the queue into master with commit 00ca3cf Nov 30, 2025
51 of 53 checks passed

meiravgri deleted the meiravg_debug_profile_query branch November 30, 2025 00:07

redisearch-backport-pull-request bot mentioned this pull request Nov 30, 2025

[8.4] [MOD-12627] Add Debug Support for FT.PROFILE Command #7573

Merged

meiravgri mentioned this pull request Nov 30, 2025

[8.2] [MOD-12627] Add Debug Support for FT.PROFILE Command (#7510) #7574

Merged

meiravgri mentioned this pull request Nov 30, 2025

[2.10] [MOD-12627] Add Debug Support for FT.PROFILE Command #7575

Merged

meiravgri added a commit that referenced this pull request Dec 7, 2025

backport #7510 to 2.8

9212bee

meiravgri mentioned this pull request Dec 7, 2025

[2.8] [MOD-12627] Add Debug Support for FT.PROFILE Command #7662

Merged

github-merge-queue bot pushed a commit that referenced this pull request Dec 7, 2025

[2.8] [MOD-12627] Add Debug Support for FT.PROFILE Command (#7662)

306ce8e

backport #7510 to 2.8

Conversation

meiravgri commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Example Usage

Execution Flows

Standalone Mode

Cluster Mode

Implementation Details

Profile Printing: Handling Debug Result Processors

Debug result processors (RP_TIMEOUT, RP_CRASH, RP_PAUSE) have types >= RP_MAX. Added an early return in _recursiveProfilePrint to skip debug RPs before opening the Map, preventing them from appearing in profile output

Refactoring: *Imp Pattern for Debug Support

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

lerman25 Nov 27, 2025

Choose a reason for hiding this comment

Uh oh!

meiravgri Nov 27, 2025

Choose a reason for hiding this comment

Uh oh!

lerman25 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

redisearch-backport-pull-request bot commented Nov 30, 2025

Uh oh!

redisearch-backport-pull-request bot commented Nov 30, 2025

Uh oh!

redisearch-backport-pull-request bot commented Nov 30, 2025

Uh oh!

redisearch-backport-pull-request bot commented Nov 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

meiravgri commented Nov 25, 2025 •

edited

Loading

Debug result processors (RP_TIMEOUT, RP_CRASH, RP_PAUSE) have types `>= RP_MAX`. Added an early return in `_recursiveProfilePrint` to skip debug RPs before opening the Map, preventing them from appearing in profile output

Refactoring: `*Imp` Pattern for Debug Support

codecov bot commented Nov 26, 2025 •

edited

Loading