Skip to content

Backport metrics info 2.8 [MOD-8388]#5407

Merged
alonre24 merged 17 commits into2.8from
backport_metrics_info_2.8
Jan 1, 2025
Merged

Backport metrics info 2.8 [MOD-8388]#5407
alonre24 merged 17 commits into2.8from
backport_metrics_info_2.8

Conversation

@alonre24
Copy link
Collaborator

Describe the changes in the pull request

Adds all the relevant info fields from 2.10 version to this version as well, including the mechanisms that are responsible for populating the info fields.

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

alonre24 and others added 13 commits December 25, 2024 13:26
…D-7584] (#5146)

Add metrics of min and max memory consumption of indexes - [MOD-7584] (#5143)

* implement min and max memory usage metrics

* added to info module test

(cherry picked from commit b0d56ad)

Co-authored-by: GuyAv46 <[email protected]>
(cherry picked from commit 094318f)
* Add a counter of total query execution time - [MOD-7961] (#5145)

* complete CP of "Count Queries" (#5072) content

* always count query execution time (on non-internal commands)

* add test

* give more time

* review fixes

(cherry picked from commit 97f34e2)

* add missing if

(cherry picked from commit d45365d)
Add VecSim related metrics - [MOD-7967] (#5151)

* added vector info

* added a test

* drain work queue before checking memory

* move vector metrics info field statistics section

* fix test

* fix test

* tidy up

* added a flat index to the test

(cherry picked from commit 388f81e)

Co-authored-by: GuyAv46 <[email protected]>
(cherry picked from commit 242f314)
…llow the test of marked deleted info to pass
Improve counting active writes - [MOD-8151] (#5354)

* improve counting active writes

* add a test

(cherry picked from commit 2b5182d)

Co-authored-by: GuyAv46 <[email protected]>
(cherry picked from commit a3d447d)
@codecov
Copy link

codecov bot commented Dec 27, 2024

Codecov Report

Attention: Patch coverage is 98.00000% with 8 lines in your changes missing coverage. Please review.

Project coverage is 85.79%. Comparing base (edc69b3) to head (ccd8849).
Report is 2 commits behind head on 2.8.

Files with missing lines Patch % Lines
src/gc.c 90.90% 3 Missing ⚠️
src/info/info_redis.c 98.25% 3 Missing ⚠️
src/info/indexes_info.c 97.56% 1 Missing ⚠️
src/spec.c 96.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##              2.8    #5407      +/-   ##
==========================================
+ Coverage   85.68%   85.79%   +0.10%     
==========================================
  Files         195      198       +3     
  Lines       33981    34126     +145     
==========================================
+ Hits        29118    29278     +160     
+ Misses       4863     4848      -15     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

GuyAv46
GuyAv46 previously approved these changes Dec 31, 2024
meiravgri
meiravgri previously approved these changes Dec 31, 2024
@alonre24 alonre24 added this pull request to the merge queue Dec 31, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Dec 31, 2024
@alonre24 alonre24 dismissed stale reviews from meiravgri and GuyAv46 via dd99124 December 31, 2024 17:03
@alonre24 alonre24 enabled auto-merge January 1, 2025 07:45
@alonre24 alonre24 requested review from GuyAv46 and meiravgri January 1, 2025 07:45
@alonre24 alonre24 added this pull request to the merge queue Jan 1, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 1, 2025
env.expect(config_cmd(), 'SET', 'FORK_GC_CLEAN_THRESHOLD', '0').ok()
set_doc = lambda: env.expect('HSET', '1', 'vec', '????????')
field_infos = lambda: [to_dict(env.cmd(debug_cmd(), 'VECSIM_INFO', f'idx{i}', 'vec')) for i in range(1, 4)]
get_field_infos = lambda: [to_dict(env.cmd(debug_cmd(), 'VECSIM_INFO', f'idx{i}', 'vec')) for i in range(1, 4)]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not to use FT.DEBUG WORKER_THREADS DRAIN as we do in master?

Copy link
Collaborator Author

@alonre24 alonre24 Jan 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. I thought we don't have it in 2.8 but we do ....

@alonre24 alonre24 added this pull request to the merge queue Jan 1, 2025
Merged via the queue into 2.8 with commit f428033 Jan 1, 2025
11 checks passed
@alonre24 alonre24 deleted the backport_metrics_info_2.8 branch January 1, 2025 12:51
kei-nan added a commit that referenced this pull request Jan 30, 2025
kei-nan added a commit that referenced this pull request Jan 30, 2025
This reverts commit ef25dca.

Revert "[2.8] Fix Max Frequency Misscalculation - [MOD-8158] (#5560)"

This reverts commit a6864e3.

Revert "[2.8] MOD-8561: Fix Inverted Index SeekTo Edge Case (#5542)"

This reverts commit bb873af.

Revert "[2.8] Fix legacy geofilter leak - [MOD-8568] (#5518)"

This reverts commit 979f3d2.

Revert "Backport 4243 to 2.8 (#5508)"

This reverts commit 9af7959.

Revert "MOD-7570 Backport 5422 to 2.8 (#5470)"

This reverts commit 540b5f3.

Revert "[2.8] MOD-7872: Add Types To Hide User Input (#5489)"

This reverts commit b2bf242.

Revert "[2.8] MOD-7864: Obfuscation API (#5488)"

This reverts commit 4866bf1.

Revert "[2.8] MOD-7634: Fix NOSTEM  (#5475)"

This reverts commit a015a55.

Revert "[2.8] [MOD-8462] rename total_active_writes -> total_active_write_threads (#5444)"

This reverts commit baadb4d.

Revert "[2.8] MOD-8058: Avoid error with _NUM_SSTRING (#5187) (#5427)"

This reverts commit 5b49276.

Revert "Backport metrics info 2.8 [MOD-8388] (#5407)"

This reverts commit f428033.

Revert " [2.8] Fixes for inverted indexes encoding - [MOD-8248] (#5389)"

This reverts commit 0c7227e.

Revert "[2.8] MOD-8221: backport info fields (for metrics) to 2.8 (#5358)"

This reverts commit b7eaa56.

Revert "[2.8] Improve "Raw doc id" encoding - [MOD-8255] (#5371)"

This reverts commit 152043b.

Revert "[2.8] Improve SkipToBlock logic - [MOD-8255] (#5369)"

This reverts commit c1a2c8f.

Revert "[2.8][MOD-8115] Free spec resources in the main thread (#5324) (#5330)"

This reverts commit 5f3ce64.

Revert "[2.8] Fix Sorting Vector - [MOD-6783] (#5316)"

This reverts commit 04f7778.

Revert "[2.8] Fix a flakiness in testCursorOnCoordinator - [MOD-8016] (#5305)"

This reverts commit 4fc1b27.

Revert "[MOD-7949] Avoid lazy expire upon scan keys in background [2.8] (#5303)"

This reverts commit 16e4ff4.

Revert "[2.8] Fix long PREFIX/SUFFIX/INFIX/WILDCARD queries - [MOD-7882] (#5299)"

This reverts commit 09f5b62.

Revert "[2.8] Remove assertion in optimizer (#5296)"

This reverts commit 45db16a.

Revert "[2.8] MOD-8188: Fix OnReopen Comments (#5288)"

This reverts commit 001746f.

Revert "[2.8] Fix missing expansions in text queries - [MOD-8142] (#5287)"

This reverts commit ed13507.

Revert "[2.8] Fix Tag OnReopen Callback - [MOD-8011] (#5280)"

This reverts commit 40b8012.

Revert "[2.8] Fix MRIterator ownership mechanism - [MOD-8108] (#5269)"

This reverts commit 90a5685.

Revert "[2.8] [MOD-8125] Create inverted index in write operations only (#5260)"

This reverts commit ebb9484.

Revert "[2.8] MOD-8129: Fix indexed_percent info stat (#5249)"

This reverts commit fb9781a.

Revert "[2.8] MOD-8097, MOD-8114 Fix memory counting (#5242)"

This reverts commit a21a909.

Revert "[2.8] Cleanup for the trimming tree logic (#5225)"

This reverts commit 1b9126b.

Revert "[2.8] [MOD-8035] Add field index error for JSON field indexing failures (#5205)"

This reverts commit 3a13ddb.

Revert "[2.8] Fix Numeric Tree Balance - [MOD-8081, MOD-8082] (#5200)"

This reverts commit a2948c4.

Revert "[2.8] Performance improvement in indexBulkFields - [MOD-8093] (#5192)"

This reverts commit 2b0e0a0.

Revert "[2.8] MOD-8009: Allow Users To Configure Cursor Index Limitation Through Global Config (#5167)"

This reverts commit f860505.

Revert "[2.8] Skip test_multithread:test_async_updates_sanity with sanitizer  (#5124)"

This reverts commit 938369a.

Revert "[2.8] Improve Error Message on VecSim Syntax Error - [MOD-7887] (#5108)"

This reverts commit ba614ca.

Revert "[2.8] Fix flaky test (#5084)"

This reverts commit f579156.

Revert "[2.8] Fix flaky test_async_updates_sanity - avoid timeout in force invoke GC [MOD-7867] (#5080)"

This reverts commit 1a0eabc.

Revert "[2.8] Handle VecSim in-place deletion after async [MOD-7643, MOD-7732] (#5048)"

This reverts commit fcc19e9.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants