Skip to content

[8.4] [MOD-12694] [MOD-12069] Add active_worker_threads metric #7616

Merged
meiravgri merged 4 commits into8.4from
backport-7538-to-8.4
Dec 2, 2025
Merged

[8.4] [MOD-12694] [MOD-12069] Add active_worker_threads metric #7616
meiravgri merged 4 commits into8.4from
backport-7538-to-8.4

Conversation

@meiravgri
Copy link
Collaborator

@meiravgri meiravgri commented Dec 2, 2025

backport #7538 to 8.4


Note

Adds active_worker_threads metric to global multithreading stats and exposes it via INFO MODULES, with worker pool creation checks and tests for SA and cluster.

  • Metrics / Global Stats:
    • Extend MultiThreadingStats with active_worker_threads in src/info/global_stats.h.
    • Populate active_worker_threads via workersThreadPool_WorkingThreadCount() in GlobalStats_GetMultiThreadingStats() (src/info/global_stats.c), asserting workersThreadPool_isCreated().
  • INFO output:
    • Add search_active_worker_threads to multi_threading section in src/info/info_redis/info_redis.c.
  • Workers pool:
    • Add workersThreadPool_isCreated() in src/util/workers.c and declare in src/util/workers.h.
  • Tests:
    • Add helper and tests to validate active_worker_threads behavior in standalone and cluster (tests/pytests/test_info_modules.py).
    • Keep existing active_io_threads checks; verify new field presence and transitions during paused queries.

Written by Cursor Bugbot for commit ba16602. This will update automatically on new commits. Configure here.

* add "active_worker_threads" metric

* fix declartion

* remove coord threads

* make the tests run...

* introduce workersThreadPool_isInitialized
assert is initizlied in GlobalStats_GetMultiThreadingStats

* cleanup

* rename workersThreadPool_isCreated

* Unskip test_active_io_threads_stats

Removed skip decorator from test_active_io_threads_stats.

(cherry picked from commit d9574d8)
@github-actions github-actions bot added the size:M label Dec 2, 2025
@meiravgri meiravgri changed the title [MOD-12694] [MOD-12069] Add active_worker_threads metric (#7538) [8.4] [MOD-12694] [MOD-12069] Add active_worker_threads metric (#7538) Dec 2, 2025
@meiravgri meiravgri changed the title [8.4] [MOD-12694] [MOD-12069] Add active_worker_threads metric (#7538) [8.4] [MOD-12694] [MOD-12069] Add active_worker_threads metric Dec 2, 2025
@meiravgri meiravgri requested a review from JoanFM December 2, 2025 15:38
@meiravgri meiravgri enabled auto-merge December 2, 2025 15:38
JoanFM
JoanFM previously approved these changes Dec 2, 2025
@codecov
Copy link

codecov bot commented Dec 2, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.91%. Comparing base (6af7bf2) to head (ba16602).
⚠️ Report is 3 commits behind head on 8.4.

Additional details and impacted files
@@            Coverage Diff             @@
##              8.4    #7616      +/-   ##
==========================================
- Coverage   85.95%   85.91%   -0.05%     
==========================================
  Files         331      331              
  Lines       52803    52808       +5     
  Branches    12004    12004              
==========================================
- Hits        45388    45368      -20     
- Misses       7248     7273      +25     
  Partials      167      167              
Flag Coverage Δ
flow 84.75% <100.00%> (+0.12%) ⬆️
unit 52.58% <80.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

if (QueryError_HasQueryOOMWarning(qctx->err)) {
QueryWarningsGlobalStats_UpdateWarning(QUERY_WARNING_CODE_OUT_OF_MEMORY_COORD, 1, !IsInternal(req));
}
if (QueryError_HasReachedMaxPrefixExpansionsWarning(qctx->err)) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems unrelated to the PR being backported?

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Unrelated change included in backport PR

The removal of the commented # @skip(cluster=False) decorator from test_active_io_threads_stats is unrelated to the purpose of this backport PR (adding active_worker_threads metric). This was flagged by a reviewer in the PR discussion as "unrelated to the PR being backported" and likely shouldn't be included in this change.

tests/pytests/test_info_modules.py#L1614-L1615

def test_active_io_threads_stats(env):

Fix in Cursor Fix in Web


@meiravgri meiravgri added this pull request to the merge queue Dec 2, 2025
Merged via the queue into 8.4 with commit 6bbb4e8 Dec 2, 2025
26 checks passed
@meiravgri meiravgri deleted the backport-7538-to-8.4 branch December 2, 2025 19:00
alonre24 added a commit to alonre24/redis that referenced this pull request Jan 26, 2026
**Bug Fixes:**

* [redis#7385](RediSearch/RediSearch#7385) Fix high temporary memory consumption when loading multiple search indexes from RDB
* [redis#7430](RediSearch/RediSearch#7430) Fix a potential deadlock in `FT.HYBRID` in cluster mode during updates.
* [redis#7454](RediSearch/RediSearch#7454) Fix a garbage collection performence regression
* [redis#7460](RediSearch/RediSearch#7460) Fix potential double-free in Fork GC error paths
* [redis#7455](RediSearch/RediSearch#7455) Fix internal cursors not being deleted promptly in cluster mode
* [redis#7667](RediSearch/RediSearch#7667) Fix a cursor logical leak upon dropping the index
* [redis#7796](RediSearch/RediSearch#7796) Fix a potential use-after-free when removing connections
* [redis#7792](RediSearch/RediSearch#7792) Fix string comparison for binary data with embedded NULLs in TOLIST reducer in FT.AGGREGATE
* [redis#7823](RediSearch/RediSearch#7823) Update `FT.HYBRID` to accept vector blobs only via parameters
* [redis#7903](RediSearch/RediSearch#7903) Fix a memory leak in Hybrid ASM
* [redis#8052](RediSearch/RediSearch#8052) Fix `FT.HYBRID` behavior when used with `LOAD *`
* [redis#8082](RediSearch/RediSearch#8082) Fix incorrect FULLTEXT field metric counts
* [redis#8089](RediSearch/RediSearch#8089) Fix an edge case in `CLUSTERSET` handling
* [redis#8152](RediSearch/RediSearch#8152) Fix configuration registration issues

**Improvements:**

* [redis#7427](RediSearch/RediSearch#7427) Enhance `FT.PROFILE` with vector search execution details
* [redis#7431](RediSearch/RediSearch#7431) Ensure full `FT.PROFILE` output is returned on timeout with RETURN policy
* [redis#7507](RediSearch/RediSearch#7507) Track timeout warnings and errors in INFO
* [redis#7576](RediSearch/RediSearch#7576) Track OOM warnings and errors in INFO
* [redis#7612](RediSearch/RediSearch#7612) Track `maxprefixexpansions` warnings and errors in INFO
* [redis#7960](RediSearch/RediSearch#7960) Persist query warnings across cursor reads
* [redis#7551](RediSearch/RediSearch#7551), [redis#7616](RediSearch/RediSearch#7616), [redis#7622](RediSearch/RediSearch#7622), [redis#7625](RediSearch/RediSearch#7625) Add runtime thread and pending-jobs metrics
* [redis#7589](RediSearch/RediSearch#7589) Support multiple slot ranges in `search.CLUSTERSET`
* [redis#7707](RediSearch/RediSearch#7707) Add `WITHCOUNT` support to `FT.AGGREGATE`
* [redis#7862](RediSearch/RediSearch#7862) Add support for subquery `COUNT` in `FT.HYBRID`
* [redis#8087](RediSearch/RediSearch#8087) Add warnings when cursor results may be affected by ASM and expose ASM warnings in `FT.PROFILE`
* [redis#8049](RediSearch/RediSearch#8049) Add logging for index-related commands
* [redis#8150](RediSearch/RediSearch#8150) Fix shard total profile time reporting in `FT.PROFILE`
YaacovHazan pushed a commit to redis/redis that referenced this pull request Jan 26, 2026
**Bug Fixes:**

* [#7385](RediSearch/RediSearch#7385) Fix high
temporary memory consumption when loading multiple search indexes from
RDB
* [#7430](RediSearch/RediSearch#7430) Fix a
potential deadlock in `FT.HYBRID` in cluster mode during updates.
* [#7454](RediSearch/RediSearch#7454) Fix a
garbage collection performence regression
* [#7460](RediSearch/RediSearch#7460) Fix
potential double-free in Fork GC error paths
* [#7455](RediSearch/RediSearch#7455) Fix
internal cursors not being deleted promptly in cluster mode
* [#7667](RediSearch/RediSearch#7667) Fix a
cursor logical leak upon dropping the index
* [#7796](RediSearch/RediSearch#7796) Fix a
potential use-after-free when removing connections
* [#7792](RediSearch/RediSearch#7792) Fix string
comparison for binary data with embedded NULLs in TOLIST reducer in
FT.AGGREGATE
* [#7704](RediSearch/RediSearch#7704) Use
asynchronous jobs for GC in SVS to accelerate execution
* [#7823](RediSearch/RediSearch#7823) Update
`FT.HYBRID` to accept vector blobs only via parameters
* [#7903](RediSearch/RediSearch#7903) Fix a
memory leak in Hybrid ASM
* [#8052](RediSearch/RediSearch#8052) Fix
`FT.HYBRID` behavior when used with `LOAD *`
* [#8082](RediSearch/RediSearch#8082) Fix
incorrect FULLTEXT field metric counts
* [#8089](RediSearch/RediSearch#8089) Fix an
edge case in `CLUSTERSET` handling
* [#8152](RediSearch/RediSearch#8152) Fix
configuration registration issues

**Improvements:**

* [#7427](RediSearch/RediSearch#7427) Enhance
`FT.PROFILE` with vector search execution details
* [#7431](RediSearch/RediSearch#7431) Ensure
full `FT.PROFILE` output is returned on timeout with RETURN policy
* [#7507](RediSearch/RediSearch#7507) Track
timeout warnings and errors in INFO
* [#7576](RediSearch/RediSearch#7576) Track OOM
warnings and errors in INFO
* [#7612](RediSearch/RediSearch#7612) Track
`maxprefixexpansions` warnings and errors in INFO
* [#7960](RediSearch/RediSearch#7960) Persist
query warnings across cursor reads
* [#7551](RediSearch/RediSearch#7551),
[#7616](RediSearch/RediSearch#7616),
[#7622](RediSearch/RediSearch#7622),
[#7625](RediSearch/RediSearch#7625) Add runtime
thread and pending-jobs metrics
* [#7589](RediSearch/RediSearch#7589) Support
multiple slot ranges in `search.CLUSTERSET`
* [#7707](RediSearch/RediSearch#7707) Add
`WITHCOUNT` support to `FT.AGGREGATE`
* [#7862](RediSearch/RediSearch#7862) Add
support for subquery `COUNT` in `FT.HYBRID`
* [#8087](RediSearch/RediSearch#8087) Add
warnings when cursor results may be affected by ASM and expose ASM
warnings in `FT.PROFILE`
* [#8049](RediSearch/RediSearch#8049) Add
logging for index-related commands
* [#8150](RediSearch/RediSearch#8150) Fix shard
total profile time reporting in `FT.PROFILE`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants