-
Notifications
You must be signed in to change notification settings - Fork 1.5k
WIP: Unit Test Fixes and Upgrades #700
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
No need for all these when NETStandard.Library is in play.
For now, also uses local host as the remote, we need a more multi-user friendly structure for this (and to move Booksleeve tests under the main project).
Not perfect, more tweaks to ordering and parallelization to make. When running on an underpowered maching (e.g. this laptop), Redis servers lose connection. Enhancements: - dotnet xunit works - Tests are run in parallel - Current CLI Counts: Total: 730, Errors: 0, Failed: 105 - VS includes and passes more, see: parallization conflicts and CPU starvation above - Output is now properly in the test cases (rather than a ton of Console.WriteLine) for much easier debugging, e.g. every test has a nicely formatted and output connection log, and only when the test fails (unless diagnostic logging is enabled via argument or runner config) - Lots of formatting fixes - Initial Skippable tests implemented TODOs: - Need skips for Cluster, SSL, and latest features - Need skips for Theory (assuming you can do this) - Move redis binaries to get rid of \packages to avoid confusion
Also removes /packages to eliminate any confusion. Restores redis-cli.exe as well, not sure what happened there.
Let's get some initial testing back online, I hope.
This will let us skip based on feature sets the same way everywhere, e.g. when testing against an older server. This also makes the base [Fact] and [Theory] skippable, meaning no custom attributes everywhere.
Breaking Master/Slave setup obviusly screws a ton of other tests, so we need to isolate these at the end of the run.
This appears to be broken for the Sync and Async type, but getting more data logged for fixing it.
Previously, tests would fail if the computer didn't have a D:\ drive at all (e.g. this laptop...) Now, tests have a default config in TestConfig. To override anything, use a _TestConfig.json file (interntionally excluded from git) to specify things like RedisLabs and SSL servers. Or generally any non-localhost style configuration. This should be the 1 place to add endpoints, for example if we add some IPv6 tests. It also makes it easier to swap tests to be against another endpoint like Azure, off-site test clusters, etc.
Some test don't actually need 1 master to check anything valid, so make them cope with the tests that split better.
Moving to an embedded resource approach given how xUnit command line runs.
These tests take several minutes each. To re-enable, use _TestConfig.json like this:
{
"RunLongRunning": true
}
Sentinel moved to skip (unless configured) since we don't have a local setup/run script yet. Will get to that at the end.
Aggregate nonsense otherwise when we're looking at specific throw types.
Some of the global exceptions can be tracked per-instance (those created on the multiplexer), so do that. This reduces cross-erroring of tests for ambient exceptions known from the multiplexer.
These needed updating for multiple servers in the default test multiplexer
Collaborator
Author
|
Merging into master to continue work on other PRs and getting decent CI validation up. Remaining tests are: "I don't think the test is wrong" failures I need to get with Marc on. The landscape has changed a bit. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Before converting most of the project to C# 7,
netstandard2.0, etc. I want to get unit tests in shape. There's a fair bit of complication to doing so in that we need to test against:This work does a few things:
Console.WriteLinemixed before)\packagesfolder completely (it wasn't used for anything but theredis-*.exebinariesnet462in testing (was onlynetcoreapp1.0before)Still TODO
[Theory]skipsLong-term TODOs
.jsonconfig override that's in.gitignorePosting this for review as I go, hopefully if people have time to eyeball they can spot existing errors with unit tests as well. Certainly far more cleanup can be done, and comments are appreciated.