fix broken PEXPIREAT test#7791
fix broken PEXPIREAT test#7791oranagra merged 1 commit intoredis:unstablefrom oranagra:fix_pexpireat_test
Conversation
This test was nearly always failing on MacOS github actions. This is because of bugs in the test that caused it to nearly always run all 3 attempts and just look at the last one as the pass/fail creteria. i.e. the test was nearly always running all 3 attempts and still sometimes succeed. this is because the break condition was different than the test completion condition. The reason the test succeeded is because the break condition tested the results of all 3 tests (PSETEX/PEXPIRE/PEXPIREAT), but the success check at the end was only testing the result of PSETEX. The reason the PEXPIREAT test nearly always failed is because it was getting the current time wrong: getting the current second and loosing the sub-section time, so the only chance for it to succeed is if it run right when a certain second started. Because i now get the time from redis, adding another round trip, i added another 100ms to the PEXPIRE test to make it less fragile, and also added many more attempts. Adding many more attempts before failure to account for slow platforms, github actions and valgrind
This test was nearly always failing on MacOS github actions. This is because of bugs in the test that caused it to nearly always run all 3 attempts and just look at the last one as the pass/fail creteria. i.e. the test was nearly always running all 3 attempts and still sometimes succeed. this is because the break condition was different than the test completion condition. The reason the test succeeded is because the break condition tested the results of all 3 tests (PSETEX/PEXPIRE/PEXPIREAT), but the success check at the end was only testing the result of PSETEX. The reason the PEXPIREAT test nearly always failed is because it was getting the current time wrong: getting the current second and loosing the sub-section time, so the only chance for it to succeed is if it run right when a certain second started. Because i now get the time from redis, adding another round trip, i added another 100ms to the PEXPIRE test to make it less fragile, and also added many more attempts. Adding many more attempts before failure to account for slow platforms, github actions and valgrind (cherry picked from commit ed9bfe2)
This test was nearly always failing on MacOS github actions. This is because of bugs in the test that caused it to nearly always run all 3 attempts and just look at the last one as the pass/fail creteria. i.e. the test was nearly always running all 3 attempts and still sometimes succeed. this is because the break condition was different than the test completion condition. The reason the test succeeded is because the break condition tested the results of all 3 tests (PSETEX/PEXPIRE/PEXPIREAT), but the success check at the end was only testing the result of PSETEX. The reason the PEXPIREAT test nearly always failed is because it was getting the current time wrong: getting the current second and loosing the sub-section time, so the only chance for it to succeed is if it run right when a certain second started. Because i now get the time from redis, adding another round trip, i added another 100ms to the PEXPIRE test to make it less fragile, and also added many more attempts. Adding many more attempts before failure to account for slow platforms, github actions and valgrind
This test was nearly always failing on MacOS github actions. This is because of bugs in the test that caused it to nearly always run all 3 attempts and just look at the last one as the pass/fail creteria. i.e. the test was nearly always running all 3 attempts and still sometimes succeed. this is because the break condition was different than the test completion condition. The reason the test succeeded is because the break condition tested the results of all 3 tests (PSETEX/PEXPIRE/PEXPIREAT), but the success check at the end was only testing the result of PSETEX. The reason the PEXPIREAT test nearly always failed is because it was getting the current time wrong: getting the current second and loosing the sub-section time, so the only chance for it to succeed is if it run right when a certain second started. Because i now get the time from redis, adding another round trip, i added another 100ms to the PEXPIRE test to make it less fragile, and also added many more attempts. Adding many more attempts before failure to account for slow platforms, github actions and valgrind (cherry picked from commit ed9bfe2)
|
The test failed in daily CI: https://github.com/redis/redis/runs/3986363437?check_suite_focus=true#step:5:3758 maybe we should do something like: 85b69af |
|
@enjoy-binbin what do you mean? these thresholds in the commit you referred to where already applied. do you mean to take them further apart? please note that before this PR the test was actually broken and passed anyway sometimes. |
|
@oranagra I ran a loop and cycled the test more than 200 times in my daily CI (test-macos-latest), here is the result.
arrays: [130, 14, 99, 42, 15, 57, 45, 21, 40, 1, 20, 149, 159, 21, 10, 18, 43, 65, 64, 0, 54, 25, 134, 2, 58, 23, 15, 97, 4, 155, 13, 22, 0, 16, 109, 51, 64, 77, 181, 58, 138, 14, 51, 112, 64, 0, 47, 2, 7, 91, 26, 16, 48, 4, 15, 76, 43, 138, 142, 91, 14, 33, 83, 123, 74, 41, 44, 119, 47, 1, 0, 44, 7, 6, 123, 5, 84, 19, 30, 19, 271, 89, 23, 73, 170, 41, 88, 113, 57, 417, 19, 50, 68, 17, 175, 44, 86, 15, 13, 17, 31, 1, 13, 31, 6, 8, 19, 30, 5, 137, 86, 99, 150, 89, 30, 29, 2, 26, 5, 42, 32, 4, 41, 10, 68, 106, 25, 86, 70, 90, 125, 29, 11, 117, 4, 91, 13, 74, 275, 95, 62, 73, 27, 15, 81, 57, 35, 119, 62, 11, 14, 94, 10, 69, 20, 18, 42, 27, 14, 18, 65, 93, 136, 116, 19, 111, 160, 2, 102, 35, 95, 7, 134, 3, 2, 108, 36, 33, 68, 42, 11, 54, 33, 20, 76, 48, 31, 103, 81, 20, 12, 1, 48, 2, 137, 57, 81, 58, 72, 31, 117, 157, 76, 231, 25, 53, 51, 49, 55, 26, 40, 46, 20, 36, 26, 24, 0, 3, 37]
|
|
that's odd, i downloaded the last daily results and got this: i.e. most on the first attempt and some on the second (not even lose to the threshold of 30). anyway, what do you suggest? |
|
and again, let's change the threshold from 30 to 50 |
3 participants
This test was nearly always failing on MacOS github actions.
This is because of bugs in the test that caused it to nearly always run
all 3 attempts and just look at the last one as the pass/fail creteria.
i.e. the test was nearly always running all 3 attempts and still sometimes
succeed. this is because the break condition was different than the test
completion condition.
The reason the test succeeded is because the break condition tested the
results of all 3 tests (PSETEX/PEXPIRE/PEXPIREAT), but the success check
at the end was only testing the result of PSETEX.
The reason the PEXPIREAT test nearly always failed is because it was
getting the current time wrong: getting the current second and loosing
the sub-section time, so the only chance for it to succeed is if it run
right when a certain second started.
Because i now get the time from redis, adding another round trip, i
added another 100ms to the PEXPIRE test to make it less fragile, and
also added many more attempts.
Adding many more attempts before failure to account for slow platforms,
github actions and valgrind