update BATS & helper + minor updates to BATS variables#2988
update BATS & helper + minor updates to BATS variables#2988georglauterbach merged 5 commits intomasterfrom
Conversation
BATS core: previous: 1.7.0 now: 1.8.2 BATS assert: previous: ffe84ea5dd43b568851549b3e241db150c12929c now: 2.1.0
georglauterbach
added
area/tests
kind/update
Move from `TEST_NAME_PREFIX` to `BATS_TEST_NAME_PREFIX` which BATS will automatically prepend to tests in a given file. Didn't know why I did not see this earlier. The prefix names were made uniform too.
georglauterbach
changed the title
The test would very sporadically fail in case Amavis could not print its log in-time. We will now wait at most 5 secs for Amavis to print its log.
search-and-replace did not catch everything the last time
| @ shopt -s globstar ; ./test/bats/bin/bats $(BATS_FLAGS) \ | ||
| --no-parallelize-within-files --jobs $(BATS_PARALLEL_JOBS) test/$@/**/*.bats |
There was a problem hiding this comment.
Prefer vertically listing the explicit options, easier to grok and track changes 😅
| @ shopt -s globstar ; ./test/bats/bin/bats $(BATS_FLAGS) \ | |
| --no-parallelize-within-files --jobs $(BATS_PARALLEL_JOBS) test/$@/**/*.bats | |
| @ shopt -s globstar ; ./test/bats/bin/bats $(BATS_FLAGS) \ | |
| --no-parallelize-within-files \ | |
| --jobs $(BATS_PARALLEL_JOBS) \ | |
| test/$@/**/*.bats |
Side-notes (nothing below requiring change for this PR)
A reminder that --no-parallelize-within-files prevents --jobs 1 for running a parallel set sequentially.
Although I wouldn't worry about this much as I'm doubtful the default of --jobs 2 will cause anyone running these tests a problem, since we can still run a few individual tests with make test/file1,file2,file3 easily enough.
If of interest to you:
- There is also a tagging feature to be selective on what test cases to run in a file(s).
--no-parallelize-within-filescould be considered for removal if we can confidently evaluate which test files need to explicitly opt-out viaexport BATS_NO_PARALLELIZE_WITHIN_FILE=trueinsetup_file().
There was a problem hiding this comment.
I have merged this PR to make CI pass again, but we can tackle the suggestions in another PR - I don't really mind the changes to the Makefile since it's only a stylistic change :)
| @test "SpamAssassin integration should be active" { | ||
| # give Amavis just a bit of time to print out its full debug log | ||
| run repeat_in_container_until_success_or_timeout 5 "${CONTAINER_NAME}" grep 'ANTI-SPAM-SA' /var/log/mail/mail.log | ||
| assert_success | ||
| assert_output --partial 'loaded' | ||
| refute_output --partial 'NOT loaded' |
There was a problem hiding this comment.
This is works too 👍 (I noticed this locally and was going to use an existing helper method to wait for the amavis service to be ready)
There was a problem hiding this comment.
Alternatively, instead of "grep-ing until success", grep until a string appears:
tail -F /var/log/mail/mail.log | grep --max-count 1 'ANTI-SPAM-SA'
timeout 5 tail -F /var/log/mail/mail.log | grep --max-count 1 'ANTI-SPAM-SA'There was a problem hiding this comment.
grep until a string appears
I have been refactoring / debugging a test of the past couple days where a similar requirement with counting was necessary and being a hassle for me to shell-fu my way through, I may want to switch to your tip here now 😂
There was a problem hiding this comment.
fyi: if you care for the output, use --max-count 1, otherwise just -q (grep will then quit after first match)
3 participants
Description
NOTE: Please review commit by commit :)
BATS core:
--> previous: 1.7.0
--> now: 1.8.2
BATS assert:
--> previous: ffe84ea5dd43b568851549b3e241db150c12929c
--> now: 2.1.0
Type of change
Checklist: