MOD-9609: Pass Private Data Context Back To UnblockClient API#6237
Merged
MOD-9609: Pass Private Data Context Back To UnblockClient API#6237
Conversation
…t in order for the free function to get called
GuyAv46
previously approved these changes
May 29, 2025
GuyAv46
approved these changes
May 29, 2025
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Contributor
|
Backport failed for Please cherry-pick the changes locally and resolve any conflicts. git fetch origin 2.8
git worktree add -d .worktree/backport-6237-to-2.8 origin/2.8
cd .worktree/backport-6237-to-2.8
git switch --create backport-6237-to-2.8
git cherry-pick -x 608e4cbb674c0dc13c63c12ffc846a494a3ad060 |
Contributor
|
Backport failed for Please cherry-pick the changes locally and resolve any conflicts. git fetch origin 2.10
git worktree add -d .worktree/backport-6237-to-2.10 origin/2.10
cd .worktree/backport-6237-to-2.10
git switch --create backport-6237-to-2.10
git cherry-pick -x 608e4cbb674c0dc13c63c12ffc846a494a3ad060 |
Contributor
|
Backport failed for Please cherry-pick the changes locally and resolve any conflicts. git fetch origin 8.0
git worktree add -d .worktree/backport-6237-to-8.0 origin/8.0
cd .worktree/backport-6237-to-8.0
git switch --create backport-6237-to-8.0
git cherry-pick -x 608e4cbb674c0dc13c63c12ffc846a494a3ad060 |
nafraf
pushed a commit
that referenced
this pull request
Jun 2, 2025
* Apparently private data had to be sent back when unblocking the client in order for the free function to get called * code review comment * solve memory leak
kei-nan
added a commit
that referenced
this pull request
Jun 3, 2025
* Apparently private data had to be sent back when unblocking the client in order for the free function to get called * code review comment * solve memory leak (cherry picked from commit 608e4cb)
github-merge-queue bot
pushed a commit
that referenced
this pull request
Jun 16, 2025
* Normalize multi-byte char terms * Fix rm_strdupcase_utf8 * Remove debug log * Add Russian alphabet and diacritic tests * Don't use DefaultNormalize_utf8() for chinese * print available locales * Check locale before using utf8 normalization * Fix DefaultNormalize() * Skip multibyte tests if 'en_US.UTF-8' locale is not available * Test multibyte stopwords * Revert changes in install_script.sh * run sanitizer using ubuntu:latest * Test_cn: fix language_field * Remove unused strtolower function from misc.c and misc.h * Fixes from code review * Enhance multibyte character tests * Validate queries using multi-byte stopwords * Add MULTIBYTE_CHARS config param and install locales * Use setlocale instead of querylocale for better compatibility * revert changes in event-pull_request.yml and install French locale for debian/ubuntu * Convert to lowercase using nunicode library * nunicode_tolower() returns zero terminated string * Refactor nunicode_tolower() to improve readability * Refactor nunicode_tolower() to use destination buffer and improve memory management * Remove null termination in nunicode_tolower() * Improve documentation * Support multi-byte chars for tags * Support multi-byte chars for synonyms * unicode_tolower(): avoid duplicating encoded input * Increase SSO_MAX_LENGTH * Update test to use FT.DEBUG DUMP_TAGIDX instead of FT.TAGVALS * Keep previous test_cn:testSynonym and test_cn:testMixedHighlight * Update CMake configuration for consistent multi-byte char sorting * Test JSON index * Fix JSON test and remove unneeded tolower() * Fix query_EvalSingleTagNode() * Refactor string normalization to use case folding instead of lowercase conversion * Refactor tests to use run_command_on_all_shards for setting DEFAULT_DIALECT * BWC: Add function to convert string to single codepoint folded runes * Rename rm_strdupcase() to rm_normalize() * Add stemming test * Refactor string normalization to use lowercase transformation instead of folding. Modify FT.SUGGET flow to be BWC * Refactor filtering functions to use a transformation callback for rune processing. Remove dead code. * Update testDFAFilter to use strToSingleCodepointFoldedRunes for rune processing * Refactor string normalization to use case folding (single codepoint) instead of lowercase transformation * Revert "Refactor string normalization to use case folding (single codepoint) instead of lowercase transformation" This reverts commit ebcf994. * Add case sensitivity option to tag string processing functions * Rename tag_strtofold to tag_strtolower for clarity and update related references to reflect lowercase transformation * Simplify length checks in tag_strtolower and rm_normalize functions * Add documentation after review * WIP: Enhance unicode_tolower function to handle memory allocation for transformed strings * WIP: Fixing unicode_tolower usage * Fix tag_strtolower() to support memory realloc * skip testToLowerSize() on cluster * Fix rm_normalize and add more tests * Fix memory leak * Fix memory leak 2 * MOD-9609: Pass Private Data Context Back To UnblockClient API (#6237) * Apparently private data had to be sent back when unblocking the client in order for the free function to get called * code review comment * solve memory leak * MOD-9774: Stop Compiling Redis-With Coverage (#6231) * stop compiling redis-server with coverage * bring back coverage as requirement * Add unit test for unicode_tolower * Fix memory management in UnicodeToLowerTest and update expected results in multibyte character tests * Add tests for unsupported Unicode characters in tolower conversion. * MOD-9768: Add Rust abstractions for Buffer* (#6175) * Add Rust abstractions for Buffer* * Add tests * Make Buffer fields pub For consistency with BufferReader and BufferWriter. Also there is a very good chance of us needing to use them directly in the near future. * Expand the docs a bit more * Add and improve upon safety comments * fmt fix * Add forgotten license headers * Revert "Make Buffer fields pub" This reverts commit 1243590. * Address review comments * Address review comments * Better document safety of `Buffer` fields * Apply suggestions from code review Co-authored-by: Luca Palmieri <[email protected]> * Remove an unneeded import * Use reallocation in mock `Buffer_Grow` * Satisfy MIRI about some pointer access * Drop unneeded dep * Remove unnecessary #includes * Split FFI into a separate crate * Add buffer.h API to ffi crate * Buffer API now based on generated bindings for the C Buffer API * CI: Set `CARGO_PROFILE_OPTIMISED_TEST_BUILD_OVERRIDE_DEBUG` env Just acting on the advice from a panic: ``` To improve backtraces for build dependencies, set the CARGO_PROFILE_OPTIMISED_TEST_BUILD_OVERRIDE_DEBUG=true environment variable to enable debug information generation. ``` * Drop linking of internal libs in ffi crate * Apply suggestions from code review Co-authored-by: Luca Palmieri <[email protected]> * Revert "CI: Set `CARGO_PROFILE_OPTIMISED_TEST_BUILD_OVERRIDE_DEBUG` env" This reverts commit 9bd1d87. * Drop a redundant import * Add a few debug_asserts * Bring back an accidentally deleted assertion in a test * Fix a .gitignore file * Make a safety assumption explicit * Dont build `buffer` crate as static lib * Don't build bindings for oxidized triemap code * Make use of BufferWritter::buffer_mut * Fix build on Alpine --------- Co-authored-by: Luca Palmieri <[email protected]> * refactor EmptyIterator (#6240) * refactor EmptyIterator * fix useless imports and doc * update empty_iterator.c to add newline * add gcc11:bullseye to platforms [MOD-9529] (#6198) * add gcc11:bullseye to platforms * add naming * remove deps * add nightly platforms option * set merge queue to false * fix flow * fix flow * chore: Improve granularity of build invalidation logic for FFI bindings (#6244) * Add `tests/deps/RedisJSON` to `.gitignore` (#6249) * Skip tests of unsupported unicode 9.0.0 chars * Fix mem leak 3 * Test stopwords which require memory reallocation * unicode_tolower update out_len always * Refactor unicode_tolower function to receive a single *inout_len argument * Update documentation for tag_strtolower() * Fixes after review * tokenizeTagString: Rename origLen as len * Test prefix/infix/suffix search with supported unicode chars * Update test description * Fix testToLowerConversionExactMatch * testTextToLowerConversionSimilarMatch: remove unused doc_id in queries * fix crash in test * Fix test failure on focal and bullseye: avoid nested f-strings with curly brackets * Fix test failure on mac: invalid escape sequence * Use SSO in runesToStr and strToLowerRunes * Add utf8_len arg to strToLowerRunes() * Align and move strToLowerRunes comment to header --------- Co-authored-by: kei-nan <[email protected]> Co-authored-by: Zeeshan Ali Khan <[email protected]> Co-authored-by: Luca Palmieri <[email protected]> Co-authored-by: Joan Fontanals <[email protected]> Co-authored-by: BenGoldberger <[email protected]> Co-authored-by: oshadmi <[email protected]>
redisearch-backport-pull-request bot
pushed a commit
that referenced
this pull request
Jun 16, 2025
* Normalize multi-byte char terms * Fix rm_strdupcase_utf8 * Remove debug log * Add Russian alphabet and diacritic tests * Don't use DefaultNormalize_utf8() for chinese * print available locales * Check locale before using utf8 normalization * Fix DefaultNormalize() * Skip multibyte tests if 'en_US.UTF-8' locale is not available * Test multibyte stopwords * Revert changes in install_script.sh * run sanitizer using ubuntu:latest * Test_cn: fix language_field * Remove unused strtolower function from misc.c and misc.h * Fixes from code review * Enhance multibyte character tests * Validate queries using multi-byte stopwords * Add MULTIBYTE_CHARS config param and install locales * Use setlocale instead of querylocale for better compatibility * revert changes in event-pull_request.yml and install French locale for debian/ubuntu * Convert to lowercase using nunicode library * nunicode_tolower() returns zero terminated string * Refactor nunicode_tolower() to improve readability * Refactor nunicode_tolower() to use destination buffer and improve memory management * Remove null termination in nunicode_tolower() * Improve documentation * Support multi-byte chars for tags * Support multi-byte chars for synonyms * unicode_tolower(): avoid duplicating encoded input * Increase SSO_MAX_LENGTH * Update test to use FT.DEBUG DUMP_TAGIDX instead of FT.TAGVALS * Keep previous test_cn:testSynonym and test_cn:testMixedHighlight * Update CMake configuration for consistent multi-byte char sorting * Test JSON index * Fix JSON test and remove unneeded tolower() * Fix query_EvalSingleTagNode() * Refactor string normalization to use case folding instead of lowercase conversion * Refactor tests to use run_command_on_all_shards for setting DEFAULT_DIALECT * BWC: Add function to convert string to single codepoint folded runes * Rename rm_strdupcase() to rm_normalize() * Add stemming test * Refactor string normalization to use lowercase transformation instead of folding. Modify FT.SUGGET flow to be BWC * Refactor filtering functions to use a transformation callback for rune processing. Remove dead code. * Update testDFAFilter to use strToSingleCodepointFoldedRunes for rune processing * Refactor string normalization to use case folding (single codepoint) instead of lowercase transformation * Revert "Refactor string normalization to use case folding (single codepoint) instead of lowercase transformation" This reverts commit ebcf994. * Add case sensitivity option to tag string processing functions * Rename tag_strtofold to tag_strtolower for clarity and update related references to reflect lowercase transformation * Simplify length checks in tag_strtolower and rm_normalize functions * Add documentation after review * WIP: Enhance unicode_tolower function to handle memory allocation for transformed strings * WIP: Fixing unicode_tolower usage * Fix tag_strtolower() to support memory realloc * skip testToLowerSize() on cluster * Fix rm_normalize and add more tests * Fix memory leak * Fix memory leak 2 * MOD-9609: Pass Private Data Context Back To UnblockClient API (#6237) * Apparently private data had to be sent back when unblocking the client in order for the free function to get called * code review comment * solve memory leak * MOD-9774: Stop Compiling Redis-With Coverage (#6231) * stop compiling redis-server with coverage * bring back coverage as requirement * Add unit test for unicode_tolower * Fix memory management in UnicodeToLowerTest and update expected results in multibyte character tests * Add tests for unsupported Unicode characters in tolower conversion. * MOD-9768: Add Rust abstractions for Buffer* (#6175) * Add Rust abstractions for Buffer* * Add tests * Make Buffer fields pub For consistency with BufferReader and BufferWriter. Also there is a very good chance of us needing to use them directly in the near future. * Expand the docs a bit more * Add and improve upon safety comments * fmt fix * Add forgotten license headers * Revert "Make Buffer fields pub" This reverts commit 1243590. * Address review comments * Address review comments * Better document safety of `Buffer` fields * Apply suggestions from code review Co-authored-by: Luca Palmieri <[email protected]> * Remove an unneeded import * Use reallocation in mock `Buffer_Grow` * Satisfy MIRI about some pointer access * Drop unneeded dep * Remove unnecessary #includes * Split FFI into a separate crate * Add buffer.h API to ffi crate * Buffer API now based on generated bindings for the C Buffer API * CI: Set `CARGO_PROFILE_OPTIMISED_TEST_BUILD_OVERRIDE_DEBUG` env Just acting on the advice from a panic: ``` To improve backtraces for build dependencies, set the CARGO_PROFILE_OPTIMISED_TEST_BUILD_OVERRIDE_DEBUG=true environment variable to enable debug information generation. ``` * Drop linking of internal libs in ffi crate * Apply suggestions from code review Co-authored-by: Luca Palmieri <[email protected]> * Revert "CI: Set `CARGO_PROFILE_OPTIMISED_TEST_BUILD_OVERRIDE_DEBUG` env" This reverts commit 9bd1d87. * Drop a redundant import * Add a few debug_asserts * Bring back an accidentally deleted assertion in a test * Fix a .gitignore file * Make a safety assumption explicit * Dont build `buffer` crate as static lib * Don't build bindings for oxidized triemap code * Make use of BufferWritter::buffer_mut * Fix build on Alpine --------- Co-authored-by: Luca Palmieri <[email protected]> * refactor EmptyIterator (#6240) * refactor EmptyIterator * fix useless imports and doc * update empty_iterator.c to add newline * add gcc11:bullseye to platforms [MOD-9529] (#6198) * add gcc11:bullseye to platforms * add naming * remove deps * add nightly platforms option * set merge queue to false * fix flow * fix flow * chore: Improve granularity of build invalidation logic for FFI bindings (#6244) * Add `tests/deps/RedisJSON` to `.gitignore` (#6249) * Skip tests of unsupported unicode 9.0.0 chars * Fix mem leak 3 * Test stopwords which require memory reallocation * unicode_tolower update out_len always * Refactor unicode_tolower function to receive a single *inout_len argument * Update documentation for tag_strtolower() * Fixes after review * tokenizeTagString: Rename origLen as len * Test prefix/infix/suffix search with supported unicode chars * Update test description * Fix testToLowerConversionExactMatch * testTextToLowerConversionSimilarMatch: remove unused doc_id in queries * fix crash in test * Fix test failure on focal and bullseye: avoid nested f-strings with curly brackets * Fix test failure on mac: invalid escape sequence * Use SSO in runesToStr and strToLowerRunes * Add utf8_len arg to strToLowerRunes() * Align and move strToLowerRunes comment to header --------- Co-authored-by: kei-nan <[email protected]> Co-authored-by: Zeeshan Ali Khan <[email protected]> Co-authored-by: Luca Palmieri <[email protected]> Co-authored-by: Joan Fontanals <[email protected]> Co-authored-by: BenGoldberger <[email protected]> Co-authored-by: oshadmi <[email protected]> (cherry picked from commit dfe511e)
github-merge-queue bot
pushed a commit
that referenced
this pull request
Jun 17, 2025
MOD-8799: Support special utf8 (#5637) * Normalize multi-byte char terms * Fix rm_strdupcase_utf8 * Remove debug log * Add Russian alphabet and diacritic tests * Don't use DefaultNormalize_utf8() for chinese * print available locales * Check locale before using utf8 normalization * Fix DefaultNormalize() * Skip multibyte tests if 'en_US.UTF-8' locale is not available * Test multibyte stopwords * Revert changes in install_script.sh * run sanitizer using ubuntu:latest * Test_cn: fix language_field * Remove unused strtolower function from misc.c and misc.h * Fixes from code review * Enhance multibyte character tests * Validate queries using multi-byte stopwords * Add MULTIBYTE_CHARS config param and install locales * Use setlocale instead of querylocale for better compatibility * revert changes in event-pull_request.yml and install French locale for debian/ubuntu * Convert to lowercase using nunicode library * nunicode_tolower() returns zero terminated string * Refactor nunicode_tolower() to improve readability * Refactor nunicode_tolower() to use destination buffer and improve memory management * Remove null termination in nunicode_tolower() * Improve documentation * Support multi-byte chars for tags * Support multi-byte chars for synonyms * unicode_tolower(): avoid duplicating encoded input * Increase SSO_MAX_LENGTH * Update test to use FT.DEBUG DUMP_TAGIDX instead of FT.TAGVALS * Keep previous test_cn:testSynonym and test_cn:testMixedHighlight * Update CMake configuration for consistent multi-byte char sorting * Test JSON index * Fix JSON test and remove unneeded tolower() * Fix query_EvalSingleTagNode() * Refactor string normalization to use case folding instead of lowercase conversion * Refactor tests to use run_command_on_all_shards for setting DEFAULT_DIALECT * BWC: Add function to convert string to single codepoint folded runes * Rename rm_strdupcase() to rm_normalize() * Add stemming test * Refactor string normalization to use lowercase transformation instead of folding. Modify FT.SUGGET flow to be BWC * Refactor filtering functions to use a transformation callback for rune processing. Remove dead code. * Update testDFAFilter to use strToSingleCodepointFoldedRunes for rune processing * Refactor string normalization to use case folding (single codepoint) instead of lowercase transformation * Revert "Refactor string normalization to use case folding (single codepoint) instead of lowercase transformation" This reverts commit ebcf994. * Add case sensitivity option to tag string processing functions * Rename tag_strtofold to tag_strtolower for clarity and update related references to reflect lowercase transformation * Simplify length checks in tag_strtolower and rm_normalize functions * Add documentation after review * WIP: Enhance unicode_tolower function to handle memory allocation for transformed strings * WIP: Fixing unicode_tolower usage * Fix tag_strtolower() to support memory realloc * skip testToLowerSize() on cluster * Fix rm_normalize and add more tests * Fix memory leak * Fix memory leak 2 * MOD-9609: Pass Private Data Context Back To UnblockClient API (#6237) * Apparently private data had to be sent back when unblocking the client in order for the free function to get called * code review comment * solve memory leak * MOD-9774: Stop Compiling Redis-With Coverage (#6231) * stop compiling redis-server with coverage * bring back coverage as requirement * Add unit test for unicode_tolower * Fix memory management in UnicodeToLowerTest and update expected results in multibyte character tests * Add tests for unsupported Unicode characters in tolower conversion. * MOD-9768: Add Rust abstractions for Buffer* (#6175) * Add Rust abstractions for Buffer* * Add tests * Make Buffer fields pub For consistency with BufferReader and BufferWriter. Also there is a very good chance of us needing to use them directly in the near future. * Expand the docs a bit more * Add and improve upon safety comments * fmt fix * Add forgotten license headers * Revert "Make Buffer fields pub" This reverts commit 1243590. * Address review comments * Address review comments * Better document safety of `Buffer` fields * Apply suggestions from code review * Remove an unneeded import * Use reallocation in mock `Buffer_Grow` * Satisfy MIRI about some pointer access * Drop unneeded dep * Remove unnecessary #includes * Split FFI into a separate crate * Add buffer.h API to ffi crate * Buffer API now based on generated bindings for the C Buffer API * CI: Set `CARGO_PROFILE_OPTIMISED_TEST_BUILD_OVERRIDE_DEBUG` env Just acting on the advice from a panic: ``` To improve backtraces for build dependencies, set the CARGO_PROFILE_OPTIMISED_TEST_BUILD_OVERRIDE_DEBUG=true environment variable to enable debug information generation. ``` * Drop linking of internal libs in ffi crate * Apply suggestions from code review * Revert "CI: Set `CARGO_PROFILE_OPTIMISED_TEST_BUILD_OVERRIDE_DEBUG` env" This reverts commit 9bd1d87. * Drop a redundant import * Add a few debug_asserts * Bring back an accidentally deleted assertion in a test * Fix a .gitignore file * Make a safety assumption explicit * Dont build `buffer` crate as static lib * Don't build bindings for oxidized triemap code * Make use of BufferWritter::buffer_mut * Fix build on Alpine --------- * refactor EmptyIterator (#6240) * refactor EmptyIterator * fix useless imports and doc * update empty_iterator.c to add newline * add gcc11:bullseye to platforms [MOD-9529] (#6198) * add gcc11:bullseye to platforms * add naming * remove deps * add nightly platforms option * set merge queue to false * fix flow * fix flow * chore: Improve granularity of build invalidation logic for FFI bindings (#6244) * Add `tests/deps/RedisJSON` to `.gitignore` (#6249) * Skip tests of unsupported unicode 9.0.0 chars * Fix mem leak 3 * Test stopwords which require memory reallocation * unicode_tolower update out_len always * Refactor unicode_tolower function to receive a single *inout_len argument * Update documentation for tag_strtolower() * Fixes after review * tokenizeTagString: Rename origLen as len * Test prefix/infix/suffix search with supported unicode chars * Update test description * Fix testToLowerConversionExactMatch * testTextToLowerConversionSimilarMatch: remove unused doc_id in queries * fix crash in test * Fix test failure on focal and bullseye: avoid nested f-strings with curly brackets * Fix test failure on mac: invalid escape sequence * Use SSO in runesToStr and strToLowerRunes * Add utf8_len arg to strToLowerRunes() * Align and move strToLowerRunes comment to header --------- (cherry picked from commit dfe511e) Co-authored-by: nafraf <[email protected]> Co-authored-by: kei-nan <[email protected]> Co-authored-by: Zeeshan Ali Khan <[email protected]> Co-authored-by: Luca Palmieri <[email protected]> Co-authored-by: Joan Fontanals <[email protected]> Co-authored-by: BenGoldberger <[email protected]> Co-authored-by: oshadmi <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Weird not so well documented behaviour.
Redis has
RedisModule_BlockClientSetPrivateDatawhich we used after blocking the client.We assumed based on the documentation of the
BlockClientAPI that the free callback will be called with the context we set.Apparently it uses the context that gets passed to
UnblockClient, since we passed NULL it didn't call the free function for the query nodes.This caused the server to crash during the exit flow, we think RLTest sometimes missed this and sometimes didn't
Now getting the context and passing it back to unblock client.
Saw that the free function now gets called.
A clear and concise description of what the PR is solving, including:
Main objects this PR modified
Mark if applicable