Prevent testProviderRefresh test failures due to log ordering differences #897
+50
−19
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…nces Accept either the typical ordering of log messages or an alternate ordering of log messages in testProviderRefresh. The same messages are used in each case, but the sequence of messages changes. Accepting either ordering assures that we won't have plugin BOM failures due to timing differences or other differences in test infrastructure. jenkinsci/bom#5718 (comment) detected a failure in this test when run within the plugin bill of materials. jenkinsci/bom#5720 (comment) tested more deeply and found that it failed more than 5% of my test runs locally. Additional local testing confirmed that the order of the log messages was sometimes slightly different than the original assertion expected. This change accepts either ordering. Testing done: * Before this change, the test would fail 5% of the time on my AMD Ryzen 5 5600X 6-Core Processor on Red Hat Enterprise Linux 8 when run with `mvn -Dtest=GithubAppCredentialsTest#testProviderRefresh test` * After this change, the test passes 100% of the time in the 36 test runs that I used to check it
jglick
approved these changes
Sep 30, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Prevent testProviderRefresh test failures due to log ordering differences
Accept either the typical ordering of log messages or an alternate ordering of log messages in testProviderRefresh. The same messages are used in each case, but the sequence of messages changes.
Accepting either ordering assures that we won't have plugin BOM failures due to timing differences or other differences in test infrastructure.
Change prompted by test failures in:
Discussion started when jenkinsci/bom#5718 (comment) detected a failure in this test when run within the plugin bill of materials.
Discussion continued when jenkinsci/bom#5720 (comment) tested more deeply and found that it failed more than 5% of my test runs locally.
Additional local testing confirmed that the order of the log messages was sometimes slightly different than the original assertion expected. This change accepts either ordering.
Testing done
Before this change, the test would fail 5% of the time on in the 60+ tests I ran on my AMD Ryzen 5 5600X 6-Core Processor on Red Hat Enterprise Linux 8 when run with
mvn -Dtest=GithubAppCredentialsTest#testProviderRefresh testAfter this change, the test passes 100% of the time in the 36 test runs that I ran
No documentation changes are needed. This is only a change in a test.
May be easier to review with whitespace ignored.
Submitter checklist
Link to JIRA ticket in description, if appropriate.Reviewer's manual test instructions provided in PR description. See Reviewer's first task below.Reviewer checklist
Documentation changes
Users/aliases to notify