Improve valgrind support for cluster tests#7725
Merged
oranagra merged 1 commit intoredis:unstablefrom Sep 6, 2020
oranagra:cluster-valgrind-tests
Merged
Improve valgrind support for cluster tests#7725oranagra merged 1 commit intoredis:unstablefrom oranagra:cluster-valgrind-tests
oranagra merged 1 commit intoredis:unstablefrom
oranagra:cluster-valgrind-tests
Conversation
Member
Author
|
Some tests currently fail (probably on timing issues), so we cannot yet use it in daily CI. |
Member
Author
|
did this to try to catch the problem in #7719 but no luck. |
- redirect valgrind reports to a dedicated file rather than console - try to avoid killing instances with SIGKILL so that we get the memory leak report (killing with SIGTERM before resorting to SIGKILL) - search for valgrind reports when done, print them and fail the tests - add --dont-clean option to keep the logs on exit - fix exit error code when crash is found (would have exited with 0) changes that affect the normal redis test suite: - refactor check_valgrind_errors into two functions one to search and one to report - move the search half into util.tcl to serve the cluster tests too - ignore "address range perms" valgrind warnings which seem non relevant.
guybe7
approved these changes
Sep 3, 2020
yossigo
reviewed
Sep 3, 2020
| set errfile [file join $dirname err.txt] | ||
| if {$::valgrind} { | ||
| set pid [exec valgrind --track-origins=yes --suppressions=../../../src/valgrind.sup --show-reachable=no --show-possibly-lost=no --leak-check=full ../../../src/${prgname} $cfgfile &] | ||
| set pid [exec valgrind --track-origins=yes --suppressions=../../../src/valgrind.sup --show-reachable=no --show-possibly-lost=no --leak-check=full ../../../src/${prgname} $cfgfile 2>> $errfile &] |
Collaborator
There was a problem hiding this comment.
@oranagra Is there a particular reason you're redirecting stderr rather than use the valgrind --log-file option?
It may not matter if we don't expect $prgname to use stderr but that may not always be the case and error messages that were printed to console in the past would no longer be visible at real time.
Member
Author
There was a problem hiding this comment.
@yossigo I followed the same technique that is used by the main redis tests.
The main redis tests redirect stdout to one file and stderr to another.
The cluster tests are using the logfile config instead of redirecting the stdout.
Anyway, redis doesn't normally outputs anything to stderr, and if we want to change the valgrind log approach, it's more important to change the main redis test suite too.
Although come to think of it, both run multiple servers in parallel, so putting things, to stderr rather than a file may be confusing.
Let me know if you still think this should be changed and if you wanna change both tests infra or just the one
yossigo
approved these changes
Sep 6, 2020
oranagra
added a commit
that referenced
this pull request
Sep 10, 2020
- redirect valgrind reports to a dedicated file rather than console - try to avoid killing instances with SIGKILL so that we get the memory leak report (killing with SIGTERM before resorting to SIGKILL) - search for valgrind reports when done, print them and fail the tests - add --dont-clean option to keep the logs on exit - fix exit error code when crash is found (would have exited with 0) changes that affect the normal redis test suite: - refactor check_valgrind_errors into two functions one to search and one to report - move the search half into util.tcl to serve the cluster tests too - ignore "address range perms" valgrind warnings which seem non relevant. (cherry picked from commit 2b998de)
JackieXie168
pushed a commit
to JackieXie168/redis
that referenced
this pull request
Nov 4, 2020
- redirect valgrind reports to a dedicated file rather than console - try to avoid killing instances with SIGKILL so that we get the memory leak report (killing with SIGTERM before resorting to SIGKILL) - search for valgrind reports when done, print them and fail the tests - add --dont-clean option to keep the logs on exit - fix exit error code when crash is found (would have exited with 0) changes that affect the normal redis test suite: - refactor check_valgrind_errors into two functions one to search and one to report - move the search half into util.tcl to serve the cluster tests too - ignore "address range perms" valgrind warnings which seem non relevant.
jschmieg
pushed a commit
to memKeyDB/memKeyDB
that referenced
this pull request
Nov 6, 2020
- redirect valgrind reports to a dedicated file rather than console - try to avoid killing instances with SIGKILL so that we get the memory leak report (killing with SIGTERM before resorting to SIGKILL) - search for valgrind reports when done, print them and fail the tests - add --dont-clean option to keep the logs on exit - fix exit error code when crash is found (would have exited with 0) changes that affect the normal redis test suite: - refactor check_valgrind_errors into two functions one to search and one to report - move the search half into util.tcl to serve the cluster tests too - ignore "address range perms" valgrind warnings which seem non relevant. (cherry picked from commit 2b998de)
oranagra
added a commit
that referenced
this pull request
Oct 28, 2021
When stopping an instance in the cluster tests, disable appendonly first, so that SIGTERM won't be ignored. Recently in #9679 i change the test infra to use SIGSEGV to kill servers that refuse the SIGTERM rather than do SIGKILL directly. This surfaced an issue that i've added in #7725 which changed SIGKILL to SIGTERM (to resolve valgrind issues). So the current situation in the past months was that sometimes servers refused the SIGTERM and waited 10 seconds for the SIGKILL, and this commit resolves that (faster termination).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
leak report (killing with SIGTERM before resorting to SIGKILL)
changes that affect the normal redis test suite:
one to report