Skip to content

[8.4] [MOD-12320] Ensure full profile output on timeout with RETURN policy#7431

Merged
redisearch-backport-pull-request[bot] merged 2 commits into8.4from
backport-7345-to-8.4
Nov 20, 2025
Merged

[8.4] [MOD-12320] Ensure full profile output on timeout with RETURN policy#7431
redisearch-backport-pull-request[bot] merged 2 commits into8.4from
backport-7345-to-8.4

Conversation

@redisearch-backport-pull-request
Copy link
Contributor

@redisearch-backport-pull-request redisearch-backport-pull-request bot commented Nov 19, 2025

Description

Backport of #7345 to 8.4.


Note

Ensures full profile output is returned after timeouts (RETURN policy) by tracking timeout state, draining pending shard replies, and introducing FT.CURSOR PROFILE to fetch profile data from timed-out cursors.

  • Profiling & Timeout Handling:
    • Add AREQ::has_timedout and use a shared createProfilePrinterCtx() to consistently include timeout/warning state in profile replies (incl. empty-result paths).
    • Coordinator printAggProfile now drains remaining iterator replies and validates shard profile count; logs partial shard profiles.
    • Propagate timeout warnings from shards (RESP3) so profiles reflect timeouts across shards and coordinator.
  • New Cursor Capability:
    • Introduce FT.CURSOR PROFILE {index} {CID} to return (and then free) profile data for internal cursor requests; validate non-profile cursors.
    • Extend routing with C_PROFILE; on timeout during profiling, resend as _FT.CURSOR PROFILE to retrieve profile data.
  • RMR/RPNet Infrastructure:
    • Expose getNextReply and add iterator helpers MRIterator_GetChannelSize and MRIterator_GetNumShards.
    • Wire coordinator resend logic to switch between READ/DEL/PROFILE on timeouts.
  • Tests:
    • Add test_cursor_profile and strengthen profile timeout tests to assert coordinator warning and shard count (RESP2/RESP3).

Written by Cursor Bugbot for commit 74b9ac7. This will update automatically on new commits. Configure here.

…7345)

* initial imp

req->has_timedout

FT.CURSOR PROFILE

while (getNextReply(rpnet)) to printAggProfile

expose getNextReply

set timeout in netCursor Callabck
send ft.cursor profile it timeout

* don't send C_DEL if profile
getNextReply read profile when cursor is depleted
use sendChunk_ReplyOnly_EmptyResults in cursor PROFILE

* sendChunk_ReplyOnly_EmptyResults: move createProfilePrinterCtx to a callback

netCursorCallback: fix set timeout from reply

expose MRIterator_GetChannelSize

done check if its not md->rootCommand = C_PROFILE

* cleanups

* address review:

Cursors_Purge->Cursors_Free

free reply

netCursorCallback: move timeout check after the profile validation

add test_cursor_profile

* fix

* fix test

* address review:

use it->len for num shards

* take spec ref

* remove return statement

* Update src/aggregate/aggregate_exec.c

Co-authored-by: GuyAv46 <[email protected]>

---------

Co-authored-by: GuyAv46 <[email protected]>
(cherry picked from commit 38408d7)
meiravgri
meiravgri previously approved these changes Nov 20, 2025
} else {
sendChunk_ReplyOnly_EmptyResults(reply, req);
IndexSpecRef_Release(execution_ref);
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Resource leak: StrongRef not released on error path

When the spec is invalid (!StrongRef_Get(execution_ref) is true) at line 1404, the strong reference execution_ref promoted at line 1403 is never released. The IndexSpecRef_Release(execution_ref) at line 1410 is only called in the else branch, so the reference leaks when the spec check fails.

Fix in Cursor Fix in Web

@codecov
Copy link

codecov bot commented Nov 20, 2025

Codecov Report

❌ Patch coverage is 97.29730% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.71%. Comparing base (313236b) to head (74b9ac7).
⚠️ Report is 8 commits behind head on 8.4.

Files with missing lines Patch % Lines
src/aggregate/aggregate_exec.c 97.22% 1 Missing ⚠️
src/coord/dist_aggregate.c 85.71% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##              8.4    #7431      +/-   ##
==========================================
+ Coverage   85.70%   85.71%   +0.01%     
==========================================
  Files         330      330              
  Lines       52323    52408      +85     
  Branches    12001    12001              
==========================================
+ Hits        44842    44922      +80     
- Misses       7314     7319       +5     
  Partials      167      167              
Flag Coverage Δ
flow 84.21% <97.29%> (-0.08%) ⬇️
unit 52.44% <1.35%> (-0.07%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@meiravgri meiravgri self-requested a review November 20, 2025 12:11
@meiravgri meiravgri requested a review from GuyAv46 November 20, 2025 12:30
@redisearch-backport-pull-request redisearch-backport-pull-request bot added this pull request to the merge queue Nov 20, 2025
Merged via the queue into 8.4 with commit 0f3b5fa Nov 20, 2025
22 checks passed
@redisearch-backport-pull-request redisearch-backport-pull-request bot deleted the backport-7345-to-8.4 branch November 20, 2025 17:29
alonre24 added a commit to alonre24/redis that referenced this pull request Jan 26, 2026
**Bug Fixes:**

* [redis#7385](RediSearch/RediSearch#7385) Fix high temporary memory consumption when loading multiple search indexes from RDB
* [redis#7430](RediSearch/RediSearch#7430) Fix a potential deadlock in `FT.HYBRID` in cluster mode during updates.
* [redis#7454](RediSearch/RediSearch#7454) Fix a garbage collection performence regression
* [redis#7460](RediSearch/RediSearch#7460) Fix potential double-free in Fork GC error paths
* [redis#7455](RediSearch/RediSearch#7455) Fix internal cursors not being deleted promptly in cluster mode
* [redis#7667](RediSearch/RediSearch#7667) Fix a cursor logical leak upon dropping the index
* [redis#7796](RediSearch/RediSearch#7796) Fix a potential use-after-free when removing connections
* [redis#7792](RediSearch/RediSearch#7792) Fix string comparison for binary data with embedded NULLs in TOLIST reducer in FT.AGGREGATE
* [redis#7823](RediSearch/RediSearch#7823) Update `FT.HYBRID` to accept vector blobs only via parameters
* [redis#7903](RediSearch/RediSearch#7903) Fix a memory leak in Hybrid ASM
* [redis#8052](RediSearch/RediSearch#8052) Fix `FT.HYBRID` behavior when used with `LOAD *`
* [redis#8082](RediSearch/RediSearch#8082) Fix incorrect FULLTEXT field metric counts
* [redis#8089](RediSearch/RediSearch#8089) Fix an edge case in `CLUSTERSET` handling
* [redis#8152](RediSearch/RediSearch#8152) Fix configuration registration issues

**Improvements:**

* [redis#7427](RediSearch/RediSearch#7427) Enhance `FT.PROFILE` with vector search execution details
* [redis#7431](RediSearch/RediSearch#7431) Ensure full `FT.PROFILE` output is returned on timeout with RETURN policy
* [redis#7507](RediSearch/RediSearch#7507) Track timeout warnings and errors in INFO
* [redis#7576](RediSearch/RediSearch#7576) Track OOM warnings and errors in INFO
* [redis#7612](RediSearch/RediSearch#7612) Track `maxprefixexpansions` warnings and errors in INFO
* [redis#7960](RediSearch/RediSearch#7960) Persist query warnings across cursor reads
* [redis#7551](RediSearch/RediSearch#7551), [redis#7616](RediSearch/RediSearch#7616), [redis#7622](RediSearch/RediSearch#7622), [redis#7625](RediSearch/RediSearch#7625) Add runtime thread and pending-jobs metrics
* [redis#7589](RediSearch/RediSearch#7589) Support multiple slot ranges in `search.CLUSTERSET`
* [redis#7707](RediSearch/RediSearch#7707) Add `WITHCOUNT` support to `FT.AGGREGATE`
* [redis#7862](RediSearch/RediSearch#7862) Add support for subquery `COUNT` in `FT.HYBRID`
* [redis#8087](RediSearch/RediSearch#8087) Add warnings when cursor results may be affected by ASM and expose ASM warnings in `FT.PROFILE`
* [redis#8049](RediSearch/RediSearch#8049) Add logging for index-related commands
* [redis#8150](RediSearch/RediSearch#8150) Fix shard total profile time reporting in `FT.PROFILE`
YaacovHazan pushed a commit to redis/redis that referenced this pull request Jan 26, 2026
**Bug Fixes:**

* [#7385](RediSearch/RediSearch#7385) Fix high
temporary memory consumption when loading multiple search indexes from
RDB
* [#7430](RediSearch/RediSearch#7430) Fix a
potential deadlock in `FT.HYBRID` in cluster mode during updates.
* [#7454](RediSearch/RediSearch#7454) Fix a
garbage collection performence regression
* [#7460](RediSearch/RediSearch#7460) Fix
potential double-free in Fork GC error paths
* [#7455](RediSearch/RediSearch#7455) Fix
internal cursors not being deleted promptly in cluster mode
* [#7667](RediSearch/RediSearch#7667) Fix a
cursor logical leak upon dropping the index
* [#7796](RediSearch/RediSearch#7796) Fix a
potential use-after-free when removing connections
* [#7792](RediSearch/RediSearch#7792) Fix string
comparison for binary data with embedded NULLs in TOLIST reducer in
FT.AGGREGATE
* [#7704](RediSearch/RediSearch#7704) Use
asynchronous jobs for GC in SVS to accelerate execution
* [#7823](RediSearch/RediSearch#7823) Update
`FT.HYBRID` to accept vector blobs only via parameters
* [#7903](RediSearch/RediSearch#7903) Fix a
memory leak in Hybrid ASM
* [#8052](RediSearch/RediSearch#8052) Fix
`FT.HYBRID` behavior when used with `LOAD *`
* [#8082](RediSearch/RediSearch#8082) Fix
incorrect FULLTEXT field metric counts
* [#8089](RediSearch/RediSearch#8089) Fix an
edge case in `CLUSTERSET` handling
* [#8152](RediSearch/RediSearch#8152) Fix
configuration registration issues

**Improvements:**

* [#7427](RediSearch/RediSearch#7427) Enhance
`FT.PROFILE` with vector search execution details
* [#7431](RediSearch/RediSearch#7431) Ensure
full `FT.PROFILE` output is returned on timeout with RETURN policy
* [#7507](RediSearch/RediSearch#7507) Track
timeout warnings and errors in INFO
* [#7576](RediSearch/RediSearch#7576) Track OOM
warnings and errors in INFO
* [#7612](RediSearch/RediSearch#7612) Track
`maxprefixexpansions` warnings and errors in INFO
* [#7960](RediSearch/RediSearch#7960) Persist
query warnings across cursor reads
* [#7551](RediSearch/RediSearch#7551),
[#7616](RediSearch/RediSearch#7616),
[#7622](RediSearch/RediSearch#7622),
[#7625](RediSearch/RediSearch#7625) Add runtime
thread and pending-jobs metrics
* [#7589](RediSearch/RediSearch#7589) Support
multiple slot ranges in `search.CLUSTERSET`
* [#7707](RediSearch/RediSearch#7707) Add
`WITHCOUNT` support to `FT.AGGREGATE`
* [#7862](RediSearch/RediSearch#7862) Add
support for subquery `COUNT` in `FT.HYBRID`
* [#8087](RediSearch/RediSearch#8087) Add
warnings when cursor results may be affected by ASM and expose ASM
warnings in `FT.PROFILE`
* [#8049](RediSearch/RediSearch#8049) Add
logging for index-related commands
* [#8150](RediSearch/RediSearch#8150) Fix shard
total profile time reporting in `FT.PROFILE`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants