[MOD-9560] Revert marking index as faulty after reaching OOM#6140
Merged
[MOD-9560] Revert marking index as faulty after reaching OOM#6140
Conversation
alonre24
reviewed
May 20, 2025
src/query_error.h
Outdated
| X(QUERY_EALIASCONFLICT, "Alias conflicts with an existing index name") \ | ||
| X(QUERY_INDEXBGOOMFAIL, "Index background scan failed due to OOM. Queries cannot be executed on\ | ||
| an incomplete index.") \ | ||
| X(QUERY_INDEXBGOOMFAIL, "Index background scan failed due to OOM") \ |
Collaborator
There was a problem hiding this comment.
Suggested change
| X(QUERY_INDEXBGOOMFAIL, "Index background scan failed due to OOM") \ | |
| X(QUERY_INDEXBGOOMFAIL, "Index background scan did not complete due to OOM") \ |
| @@ -259,56 +265,6 @@ def test_change_config_during_bg_indexing(env): | |||
| memory_ratio = get_memory_consumption_ratio(env) | |||
| env.assertAlmostEqual(memory_ratio, 0.85, delta=0.1) | |||
Collaborator
There was a problem hiding this comment.
Can we validate that the resp3 warning is also in ft.profile output under "warnings" section (should be available for resp2 and resp AFAIR)?
Collaborator
Author
There was a problem hiding this comment.
Added warning to resp2, validated in verbosity test for both resp3 & 2
alonre24
reviewed
May 21, 2025
src/aggregate/aggregate_exec.c
Outdated
| RedisModule_Reply_LongLong(reply, 0); | ||
| if (IsProfile(req)) { | ||
| req->profile(reply, req, has_timedout, req->qiter.err->reachedMaxPrefixExpansions); | ||
| req->profile(reply, req, has_timedout, req->qiter.err->reachedMaxPrefixExpansions, req->sctx->spec && req->sctx->spec->scan_failed_OOM); |
Collaborator
There was a problem hiding this comment.
Not actually related to this PR, but can we just create ProfilePrinterCtx and move it here by value? The list of arguments to this function is getting long...
This reverts commit 6b73b3c.
alonre24
approved these changes
May 22, 2025
JoanFM
pushed a commit
that referenced
this pull request
May 27, 2025
* Add config * insert oom_scan field to scanner, remove check for oom in debug pause on oom * fix comment * create basis for function * handle last scanned key * change sleep function name * Add pause before and after reset * Add pause before and after statuses code * Add pause before and after reset to bg scan * wait if config >0 * adjust default sleep time * fix last scanned key * temp tests * bsaic test strcuture * styling * Add scanner status strings * Add first test * Add update option for dbug scanner and more tests * remove scanner cancleation * Alter and Drop tests * spellcheck * skip cluster in tests * Add config test for new config * shortern tests * styling * remove pause on OOM from drop test * Add tests, style * style * style * Remove unused pause after OOM reset * debug commands tests and cluster tests * Naming, styling, formatting * rename, change structure for simplicity * improve test robustness * remove unused and move to better location * remove OOM from ACL * remove oom from aggregate_exec * Change config help message * remove from ft.search * remove query error test * Add warning to query_error * Add warning to resp3 * Add resp3 test * format * revert pytest * resp3 tests * Ben comments round1 * Add 0 thresh test * comment * ADd skip cluster * change config and other Alon's comments * small tess * remove pause after Remove duplicates in tests * more test compression * fix assert * FT.PROFILE and OOM string * Add resp2 warning * Revert "Add resp2 warning" This reverts commit 6b73b3c. * Add resp2 warning * format
JoanFM
pushed a commit
that referenced
this pull request
May 27, 2025
* Add config * insert oom_scan field to scanner, remove check for oom in debug pause on oom * fix comment * create basis for function * handle last scanned key * change sleep function name * Add pause before and after reset * Add pause before and after statuses code * Add pause before and after reset to bg scan * wait if config >0 * adjust default sleep time * fix last scanned key * temp tests * bsaic test strcuture * styling * Add scanner status strings * Add first test * Add update option for dbug scanner and more tests * remove scanner cancleation * Alter and Drop tests * spellcheck * skip cluster in tests * Add config test for new config * shortern tests * styling * remove pause on OOM from drop test * Add tests, style * style * style * Remove unused pause after OOM reset * debug commands tests and cluster tests * Naming, styling, formatting * rename, change structure for simplicity * improve test robustness * remove unused and move to better location * remove OOM from ACL * remove oom from aggregate_exec * Change config help message * remove from ft.search * remove query error test * Add warning to query_error * Add warning to resp3 * Add resp3 test * format * revert pytest * resp3 tests * Ben comments round1 * Add 0 thresh test * comment * ADd skip cluster * change config and other Alon's comments * small tess * remove pause after Remove duplicates in tests * more test compression * fix assert * FT.PROFILE and OOM string * Add resp2 warning * Revert "Add resp2 warning" This reverts commit 6b73b3c. * Add resp2 warning * format
lerman25
added a commit
that referenced
this pull request
May 27, 2025
* Add config * insert oom_scan field to scanner, remove check for oom in debug pause on oom * fix comment * create basis for function * handle last scanned key * change sleep function name * Add pause before and after reset * Add pause before and after statuses code * Add pause before and after reset to bg scan * wait if config >0 * adjust default sleep time * fix last scanned key * temp tests * bsaic test strcuture * styling * Add scanner status strings * Add first test * Add update option for dbug scanner and more tests * remove scanner cancleation * Alter and Drop tests * spellcheck * skip cluster in tests * Add config test for new config * shortern tests * styling * remove pause on OOM from drop test * Add tests, style * style * style * Remove unused pause after OOM reset * debug commands tests and cluster tests * Naming, styling, formatting * rename, change structure for simplicity * improve test robustness * remove unused and move to better location * remove OOM from ACL * remove oom from aggregate_exec * Change config help message * remove from ft.search * remove query error test * Add warning to query_error * Add warning to resp3 * Add resp3 test * format * revert pytest * resp3 tests * Ben comments round1 * Add 0 thresh test * comment * ADd skip cluster * change config and other Alon's comments * small tess * remove pause after Remove duplicates in tests * more test compression * fix assert * FT.PROFILE and OOM string * Add resp2 warning * Revert "Add resp2 warning" This reverts commit 6b73b3c. * Add resp2 warning * format
github-merge-queue bot
pushed a commit
that referenced
this pull request
Jun 8, 2025
* [MOD-9372 , MOD-9733] Stop indexing OOM - Add wait before OOM (#6114) * Add config * insert oom_scan field to scanner, remove check for oom in debug pause on oom * fix comment * create basis for function * handle last scanned key * change sleep function name * Add pause before and after reset * Add pause before and after statuses code * Add pause before and after reset to bg scan * wait if config >0 * adjust default sleep time * fix last scanned key * temp tests * bsaic test strcuture * styling * Add scanner status strings * Add first test * Add update option for dbug scanner and more tests * remove scanner cancleation * Alter and Drop tests * spellcheck * skip cluster in tests * Add config test for new config * shortern tests * styling * remove pause on OOM from drop test * Add tests, style * style * style * Remove unused pause after OOM reset * debug commands tests and cluster tests * Naming, styling, formatting * rename, change structure for simplicity * improve test robustness * remove unused and move to better location * Ben comments round1 * Add 0 thresh test * comment * ADd skip cluster * change config and other Alon's comments * small tess * remove pause after Remove duplicates in tests * more test compression * fix assert * [MOD-9560] Revert marking index as faulty after reaching OOM (#6140) * Add config * insert oom_scan field to scanner, remove check for oom in debug pause on oom * fix comment * create basis for function * handle last scanned key * change sleep function name * Add pause before and after reset * Add pause before and after statuses code * Add pause before and after reset to bg scan * wait if config >0 * adjust default sleep time * fix last scanned key * temp tests * bsaic test strcuture * styling * Add scanner status strings * Add first test * Add update option for dbug scanner and more tests * remove scanner cancleation * Alter and Drop tests * spellcheck * skip cluster in tests * Add config test for new config * shortern tests * styling * remove pause on OOM from drop test * Add tests, style * style * style * Remove unused pause after OOM reset * debug commands tests and cluster tests * Naming, styling, formatting * rename, change structure for simplicity * improve test robustness * remove unused and move to better location * remove OOM from ACL * remove oom from aggregate_exec * Change config help message * remove from ft.search * remove query error test * Add warning to query_error * Add warning to resp3 * Add resp3 test * format * revert pytest * resp3 tests * Ben comments round1 * Add 0 thresh test * comment * ADd skip cluster * change config and other Alon's comments * small tess * remove pause after Remove duplicates in tests * more test compression * fix assert * FT.PROFILE and OOM string * Add resp2 warning * Revert "Add resp2 warning" This reverts commit 6b73b3c. * Add resp2 warning * format * [MOD-9372] - Add GIL release in OOM wait (#6203) release gil * fix merge
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR changes the following:
Continuing the work of #6114 , #6053 , #5778 .