Skip to content

[8.4] [MOD-12069] Add *_pending_jobs metrics#7625

Merged
meiravgri merged 5 commits into8.4from
backport-7556-to-8.4
Dec 3, 2025
Merged

[8.4] [MOD-12069] Add *_pending_jobs metrics#7625
meiravgri merged 5 commits into8.4from
backport-7556-to-8.4

Conversation

@meiravgri
Copy link
Collaborator

@meiravgri meiravgri commented Dec 3, 2025

MOD-12069 Multi threaded section
MOD-12696 number of queries waiting in the shards queue
MOD-12697 number of pending jobs in “low priority queue”
MOD-12831 test_info_modules:test_pending_jobs_metrics_search Extend output on timeout

backport #7556 and #7609 to 8.4


Note

Expose workers’ high/low priority pending job metrics via INFO multi_threading, backed by threadpool APIs; add tests and test utilities.

  • Info/Stats:
    • Add workers_low_priority_pending_jobs and workers_high_priority_pending_jobs to GlobalStats_GetMultiThreadingStats() and export them in INFO MODULES under search_multi_threading.
  • Threadpool/Workers:
    • Add atomic getters redisearch_thpool_high_priority_pending_jobs and redisearch_thpool_low_priority_pending_jobs.
    • Expose counts via workersThreadPool_{High,Low}PriorityPendingJobsCount().
  • Tests/Utils:
    • Extend tests/pytests/test_info_modules.py with multi-threading metrics tests (active threads, pending jobs for SEARCH/AGGREGATE).
    • Add helpers in tests/pytests/common.py (wait_for_condition, launch_cmds_in_bg_with_exception_check).
    • Minor cleanup in C++ test comment and verify active IO threads behavior.

Written by Cursor Bugbot for commit 8df7093. This will update automatically on new commits. Configure here.

@github-actions github-actions bot added the size:S label Dec 3, 2025
* align info/* to active_coord

* add APIs to get queues length

* add to info

* fix

* test

* fix test

* catch general error

* rename

* fix moduleArgs

* rename

* rename test_active_worker_threads

* rename to wworketrs

(cherry picked from commit ea0476a)
@github-actions github-actions bot added the size:M label Dec 3, 2025
meiravgri and others added 3 commits December 3, 2025 08:22
…nd output on timeout (#7609)

* move queries to run_cmds_in_bg
add workers stats to state if will fail

* refcator run_cmds_in_bg

(cherry picked from commit 7db7c2e)
@meiravgri meiravgri requested a review from JoanFM December 3, 2025 12:16
@meiravgri meiravgri enabled auto-merge December 3, 2025 12:23
@codecov
Copy link

codecov bot commented Dec 3, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.93%. Comparing base (c476509) to head (8df7093).
⚠️ Report is 1 commits behind head on 8.4.

Additional details and impacted files
@@            Coverage Diff             @@
##              8.4    #7625      +/-   ##
==========================================
- Coverage   85.99%   85.93%   -0.06%     
==========================================
  Files         331      331              
  Lines       52827    52841      +14     
  Branches    12004    12004              
==========================================
- Hits        45427    45411      -16     
- Misses       7233     7263      +30     
  Partials      167      167              
Flag Coverage Δ
flow 84.50% <100.00%> (-0.18%) ⬇️
unit 52.60% <85.71%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@meiravgri meiravgri added this pull request to the merge queue Dec 3, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Dec 3, 2025
@meiravgri meiravgri added this pull request to the merge queue Dec 3, 2025
Merged via the queue into 8.4 with commit a4ada3a Dec 3, 2025
26 checks passed
@meiravgri meiravgri deleted the backport-7556-to-8.4 branch December 3, 2025 15:52
alonre24 added a commit to alonre24/redis that referenced this pull request Jan 26, 2026
**Bug Fixes:**

* [redis#7385](RediSearch/RediSearch#7385) Fix high temporary memory consumption when loading multiple search indexes from RDB
* [redis#7430](RediSearch/RediSearch#7430) Fix a potential deadlock in `FT.HYBRID` in cluster mode during updates.
* [redis#7454](RediSearch/RediSearch#7454) Fix a garbage collection performence regression
* [redis#7460](RediSearch/RediSearch#7460) Fix potential double-free in Fork GC error paths
* [redis#7455](RediSearch/RediSearch#7455) Fix internal cursors not being deleted promptly in cluster mode
* [redis#7667](RediSearch/RediSearch#7667) Fix a cursor logical leak upon dropping the index
* [redis#7796](RediSearch/RediSearch#7796) Fix a potential use-after-free when removing connections
* [redis#7792](RediSearch/RediSearch#7792) Fix string comparison for binary data with embedded NULLs in TOLIST reducer in FT.AGGREGATE
* [redis#7823](RediSearch/RediSearch#7823) Update `FT.HYBRID` to accept vector blobs only via parameters
* [redis#7903](RediSearch/RediSearch#7903) Fix a memory leak in Hybrid ASM
* [redis#8052](RediSearch/RediSearch#8052) Fix `FT.HYBRID` behavior when used with `LOAD *`
* [redis#8082](RediSearch/RediSearch#8082) Fix incorrect FULLTEXT field metric counts
* [redis#8089](RediSearch/RediSearch#8089) Fix an edge case in `CLUSTERSET` handling
* [redis#8152](RediSearch/RediSearch#8152) Fix configuration registration issues

**Improvements:**

* [redis#7427](RediSearch/RediSearch#7427) Enhance `FT.PROFILE` with vector search execution details
* [redis#7431](RediSearch/RediSearch#7431) Ensure full `FT.PROFILE` output is returned on timeout with RETURN policy
* [redis#7507](RediSearch/RediSearch#7507) Track timeout warnings and errors in INFO
* [redis#7576](RediSearch/RediSearch#7576) Track OOM warnings and errors in INFO
* [redis#7612](RediSearch/RediSearch#7612) Track `maxprefixexpansions` warnings and errors in INFO
* [redis#7960](RediSearch/RediSearch#7960) Persist query warnings across cursor reads
* [redis#7551](RediSearch/RediSearch#7551), [redis#7616](RediSearch/RediSearch#7616), [redis#7622](RediSearch/RediSearch#7622), [redis#7625](RediSearch/RediSearch#7625) Add runtime thread and pending-jobs metrics
* [redis#7589](RediSearch/RediSearch#7589) Support multiple slot ranges in `search.CLUSTERSET`
* [redis#7707](RediSearch/RediSearch#7707) Add `WITHCOUNT` support to `FT.AGGREGATE`
* [redis#7862](RediSearch/RediSearch#7862) Add support for subquery `COUNT` in `FT.HYBRID`
* [redis#8087](RediSearch/RediSearch#8087) Add warnings when cursor results may be affected by ASM and expose ASM warnings in `FT.PROFILE`
* [redis#8049](RediSearch/RediSearch#8049) Add logging for index-related commands
* [redis#8150](RediSearch/RediSearch#8150) Fix shard total profile time reporting in `FT.PROFILE`
YaacovHazan pushed a commit to redis/redis that referenced this pull request Jan 26, 2026
**Bug Fixes:**

* [#7385](RediSearch/RediSearch#7385) Fix high
temporary memory consumption when loading multiple search indexes from
RDB
* [#7430](RediSearch/RediSearch#7430) Fix a
potential deadlock in `FT.HYBRID` in cluster mode during updates.
* [#7454](RediSearch/RediSearch#7454) Fix a
garbage collection performence regression
* [#7460](RediSearch/RediSearch#7460) Fix
potential double-free in Fork GC error paths
* [#7455](RediSearch/RediSearch#7455) Fix
internal cursors not being deleted promptly in cluster mode
* [#7667](RediSearch/RediSearch#7667) Fix a
cursor logical leak upon dropping the index
* [#7796](RediSearch/RediSearch#7796) Fix a
potential use-after-free when removing connections
* [#7792](RediSearch/RediSearch#7792) Fix string
comparison for binary data with embedded NULLs in TOLIST reducer in
FT.AGGREGATE
* [#7704](RediSearch/RediSearch#7704) Use
asynchronous jobs for GC in SVS to accelerate execution
* [#7823](RediSearch/RediSearch#7823) Update
`FT.HYBRID` to accept vector blobs only via parameters
* [#7903](RediSearch/RediSearch#7903) Fix a
memory leak in Hybrid ASM
* [#8052](RediSearch/RediSearch#8052) Fix
`FT.HYBRID` behavior when used with `LOAD *`
* [#8082](RediSearch/RediSearch#8082) Fix
incorrect FULLTEXT field metric counts
* [#8089](RediSearch/RediSearch#8089) Fix an
edge case in `CLUSTERSET` handling
* [#8152](RediSearch/RediSearch#8152) Fix
configuration registration issues

**Improvements:**

* [#7427](RediSearch/RediSearch#7427) Enhance
`FT.PROFILE` with vector search execution details
* [#7431](RediSearch/RediSearch#7431) Ensure
full `FT.PROFILE` output is returned on timeout with RETURN policy
* [#7507](RediSearch/RediSearch#7507) Track
timeout warnings and errors in INFO
* [#7576](RediSearch/RediSearch#7576) Track OOM
warnings and errors in INFO
* [#7612](RediSearch/RediSearch#7612) Track
`maxprefixexpansions` warnings and errors in INFO
* [#7960](RediSearch/RediSearch#7960) Persist
query warnings across cursor reads
* [#7551](RediSearch/RediSearch#7551),
[#7616](RediSearch/RediSearch#7616),
[#7622](RediSearch/RediSearch#7622),
[#7625](RediSearch/RediSearch#7625) Add runtime
thread and pending-jobs metrics
* [#7589](RediSearch/RediSearch#7589) Support
multiple slot ranges in `search.CLUSTERSET`
* [#7707](RediSearch/RediSearch#7707) Add
`WITHCOUNT` support to `FT.AGGREGATE`
* [#7862](RediSearch/RediSearch#7862) Add
support for subquery `COUNT` in `FT.HYBRID`
* [#8087](RediSearch/RediSearch#8087) Add
warnings when cursor results may be affected by ASM and expose ASM
warnings in `FT.PROFILE`
* [#8049](RediSearch/RediSearch#8049) Add
logging for index-related commands
* [#8150](RediSearch/RediSearch#8150) Fix shard
total profile time reporting in `FT.PROFILE`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants