Fix Allure wiring: remove duplicate attributes and fix CI check for transitive dependencies#1784
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses Allure/NUnit wiring issues after splitting tests into multiple projects, and updates CI to correctly validate Allure inheritance across transitive dependencies. It also introduces several new split test projects and adds/relocates a number of test suites into those projects.
Changes:
- Remove duplicate Allure attributes from derived fixtures and ensure derived fixtures are still discovered by NUnit.
- Update CI “Verify Allure wiring” to resolve
AllureTestBasefrom transitive dependencies via loaded AppDomain assemblies. - Split tests into additional projects (collections/ACL/scripting/complexstring/vectorset/rangeindex/extensions + cluster sub-suites) and update InternalsVisibleTo, solution, and workflow matrices accordingly.
Reviewed changes
Copilot reviewed 29 out of 91 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| test/Garnet.test/Garnet.test.csproj | Adds InternalsVisibleTo entries to support newly split test projects; removes linked global usings include. |
| test/Garnet.test.vectorset/VectorCleanupVsResetRaceTests.cs | Adds regression test for VectorManager cleanup vs store reset race using UnsafeAccessor. |
| test/Garnet.test.vectorset/Garnet.test.vectorset.csproj | New vector-set focused test project. |
| test/Garnet.test.scripting/Garnet.test.scripting.csproj | New scripting-focused test project. |
| test/Garnet.test.rangeindex/Garnet.test.rangeindex.csproj | New range-index focused test project. |
| test/Garnet.test.extensions/ScratchBufferAllocatorTests.cs | Adds tests for ScratchBufferAllocator behavior. |
| test/Garnet.test.extensions/RespRevivificationTests.cs | Adds revivification behavior tests (RMW/Upsert/AOF interactions). |
| test/Garnet.test.extensions/ReadOptimizedLockTests.cs | Adds unit tests for ReadOptimizedLock correctness under contention. |
| test/Garnet.test.extensions/ReadCacheTests.cs | Adds tests validating read-cache population/eviction for string and object records. |
| test/Garnet.test.extensions/IndexGrowthTests.cs | Adds tests validating index resizing behavior (including checkpoint/recovery). |
| test/Garnet.test.extensions/GlobalUsings.cs | Provides local aliases used by extensions tests. |
| test/Garnet.test.extensions/GarnetJSON/JSONPath/QueryExpressionTests.cs | Adds JSONPath query expression tests under extensions suite. |
| test/Garnet.test.extensions/GarnetJSON/JSONPath/JsonPathRegressionTests.cs | Adds JSONPath regression suite runner using embedded JSON cases. |
| test/Garnet.test.extensions/GarnetJSON/JSONPath/JsonAssert.cs | Adds JSON element/node equality helpers for JSONPath tests. |
| test/Garnet.test.extensions/Garnet.test.extensions.csproj | New extensions test project (includes GarnetJSON + DiskANN tests/resources). |
| test/Garnet.test.extensions/DiskANN/DiskANNSyntheticRecallTests.cs | Adds DiskANN synthetic recall tests (grid/circle datasets). |
| test/Garnet.test.complexstring/Garnet.test.complexstring.csproj | New complex-string focused test project. |
| test/Garnet.test.collections/RespListGarnetClientTests.cs | Adds/relocates GarnetClient list command tests into collections suite. |
| test/Garnet.test.collections/GeoHashTests.cs | Adds GeoHash encode/decode tests into collections suite. |
| test/Garnet.test.collections/GarnetObjectTests.cs | Adds object serialization/checkpoint/recovery tests using Tsavorite directly. |
| test/Garnet.test.collections/Garnet.test.collections.csproj | New collections-focused test project (links shared global usings). |
| test/Garnet.test.cluster/Garnet.test.cluster.csproj | Adds InternalsVisibleTo for new cluster sub-projects; links AllureTestBase. |
| test/Garnet.test.cluster/CustomProcs/ClusterDelRmw.cs | Makes ClusterDelRmw public for cross-assembly use. |
| test/Garnet.test.cluster.vectorsets/Garnet.test.cluster.vectorsets.csproj | New cluster vector-sets test project. |
| test/Garnet.test.cluster.replication/ReplicationTests/ClusterResetDuringReplicationTests.cs | Adds DEBUG-only tests for CLUSTER RESET HARD during replication attach. |
| test/Garnet.test.cluster.replication/Garnet.test.cluster.replication.csproj | New cluster replication test project; exposes internals to dependent suites. |
| test/Garnet.test.cluster.replication.tls/ReplicationTests/ClusterReplicationTLS.cs | Removes duplicate Allure attribute by relying on base; ensures fixture discovery. |
| test/Garnet.test.cluster.replication.tls/Garnet.test.cluster.replication.tls.csproj | New TLS-specific replication test project. |
| test/Garnet.test.cluster.replication.disklesssync/Garnet.test.cluster.replication.disklesssync.csproj | New diskless-sync focused replication test project. |
| test/Garnet.test.cluster.replication.asyncreplay/ReplicationTests/ClusterReplicationAsyncReplay.cs | Removes duplicate Allure attribute by relying on base; ensures fixture discovery. |
| test/Garnet.test.cluster.replication.asyncreplay/Garnet.test.cluster.replication.asyncreplay.csproj | New async-replay replication test project. |
| test/Garnet.test.cluster.multilog/MultiLogTests/ClusterReplicationShardedLog.cs | Adds multilog/sharded-log replication test fixture wrapper. |
| test/Garnet.test.cluster.multilog/MultiLogTests/ClusterReplicationDisklessSyncShardedLog.cs | Adds multilog wrapper for diskless-sync replication tests. |
| test/Garnet.test.cluster.multilog/Garnet.test.cluster.multilog.csproj | New multilog-focused cluster test project. |
| test/Garnet.test.cluster.migrate/RedirectTests/TestClusterProc.cs | Adds custom txn procedures and redirect test commands for migrate suite. |
| test/Garnet.test.cluster.migrate/Garnet.test.cluster.migrate.csproj | New cluster migrate test project. |
| test/Garnet.test.cluster.migrate/ClusterMigrateTLSTests.cs | Adds TLS variant wrapper fixture for cluster migrate tests. |
| test/Garnet.test.acl/Resp/GarnetAuthenticatorTests.cs | Adds tests for authenticator invalidation behavior. |
| test/Garnet.test.acl/Resp/ACL/UserAclResult.cs | Adds helper for parsing ACL GETUSER responses. |
| test/Garnet.test.acl/Resp/ACL/ParallelTests.cs | Adds parallel stress tests for AUTH/password hashing/ACL SETUSER. |
| test/Garnet.test.acl/Resp/ACL/GetUserTests.cs | Adds ACL GETUSER tests including multi-user scenarios. |
| test/Garnet.test.acl/Resp/ACL/DeleteUserTests.cs | Adds ACL DELUSER behavior tests. |
| test/Garnet.test.acl/Resp/ACL/BasicTests.cs | Adds tests for basic ACL commands and NoAuth flag validation. |
| test/Garnet.test.acl/Resp/ACL/AclTest.cs | Introduces ACL test base class with shared setup/teardown and constants. |
| test/Garnet.test.acl/Resp/ACL/AclParserTests.cs | Adds ACL parser reduction/timeout regression tests. |
| test/Garnet.test.acl/Resp/ACL/AclConfigurationFileTests.cs | Adds tests around ACL file load/save semantics and error handling. |
| test/Garnet.test.acl/Garnet.test.acl.csproj | New ACL-focused test project. |
| playground/TstRunner/TstRunner.csproj | Adds references to new cluster migrate/replication test projects. |
| modules/GarnetJSON/GarnetJSON.csproj | Adds InternalsVisibleTo for Garnet.test.extensions. |
| main/GarnetServer/GarnetServer.csproj | Adds InternalsVisibleTo for newly split test projects. |
| libs/storage/Tsavorite/cs/src/core/Tsavorite.core.csproj | Adds InternalsVisibleTo for Garnet.test.extensions. |
| libs/server/Properties/AssemblyInfo.cs | Adds InternalsVisibleTo for newly split test projects. |
| libs/host/Garnet.host.csproj | Adds InternalsVisibleTo for newly split test projects. |
| Garnet.slnx | Adds new test projects to the solution. |
| .github/workflows/nightly.yml | Expands nightly test matrix to include new test projects. |
| .github/workflows/ci.yml | Expands CI test matrix and fixes Allure wiring verification for transitive deps. |
Split Garnet.test.cluster into separate projects to enable parallel CI: - Garnet.test.cluster: shared infra + basic cluster tests (144 tests) - Garnet.test.cluster.migrate: migrate + slot verification (67 tests) - Garnet.test.cluster.replication: all replication tests (372 tests) - Garnet.test.cluster.vectorsets: vector set cluster tests (30 tests) - Garnet.test.cluster.multilog: sharded log replication tests (163 tests) Child projects reference base via ProjectReference + InternalsVisibleTo. Updated CI and nightly workflow matrices to run all 5 in parallel. Fixed Allure wiring check to find AllureTestBase in referenced assemblies. Co-authored-by: Copilot <[email protected]>
Separate Garnet.test.cluster.replication into 3 additional projects for parallel CI execution: - Garnet.test.cluster.replication.tls: TLS replication tests (inherits ClusterReplicationBaseTests with useTLS=true) - Garnet.test.cluster.replication.asyncreplay: Async replay tests (inherits ClusterReplicationBaseTests with asyncReplay=true) - Garnet.test.cluster.replication.disklesssync: Diskless sync tests (standalone ClusterReplicationDisklessSyncTests) Base replication project retains ClusterReplicationBaseTests and ClusterResetDuringReplicationTests. Updated InternalsVisibleTo, multilog project references, CI and nightly workflow matrices, and solution file. Co-authored-by: Copilot <[email protected]>
Split the monolithic Garnet.test project into 8 focused test projects: - Garnet.test (base): RESP core, config, admin, infra (~774 tests) - Garnet.test.collections: Hash, List, Set, SortedSet, Geo (~746 tests) - Garnet.test.acl: ACL and auth tests (~426 tests) - Garnet.test.scripting: Lua, custom commands, transactions, AOF, modules (~585 tests) - Garnet.test.complexstring: Bitmap, HyperLogLog (~386 tests) - Garnet.test.vectorset: VectorSet tests (~36 tests) - Garnet.test.rangeindex: RangeIndex tests (~58 tests) - Garnet.test.extensions: JSON, DiskANN, revivification, storage internals (~527 tests) Each child project references Garnet.test for shared infrastructure (TestUtils, AllureTestBase, extensions). InternalsVisibleTo entries added to Garnet.server, Garnet.host, GarnetServer, GarnetJSON, and Tsavorite.core for child projects. Updated CI and nightly workflows with expanded test matrices. Co-authored-by: Copilot <[email protected]>
…ures Added [AllureNUnit] and [TestFixture] to ClusterReplicationTLS, ClusterReplicationAsyncReplay, ClusterReplicationShardedLog, and ClusterReplicationDisklessSyncShardedLog to pass CI Allure wiring check. Co-authored-by: Copilot <[email protected]>
…r transitive dependencies - Remove duplicate [AllureNUnit] from derived test classes (TLS, AsyncReplay, MultiLog) that inherit it from base classes, fixing runtime error 'Unable to change the container context because the test context is active' - Fix CI Allure wiring check to search AppDomain.GetAssemblies() instead of using Assembly.Load() on direct references, which failed for AllureTestBase in transitive dependencies (e.g. Garnet.test.cluster via Garnet.test.cluster.replication) Co-authored-by: Copilot <[email protected]>
Add actions/cache@v4 for ~/.nuget/packages to format-garnet, format-tsavorite, build-test-garnet, and build-test-tsavorite jobs. Cache key is based on runner.os and hash of *.csproj and Directory.Packages.props files, with a fallback restore-key for partial matches. This reduces dotnet restore time, especially on Windows runners where install dependencies averaged 1.1 min (up to 4.1 min) per job. Co-authored-by: Copilot <[email protected]>
Split build-test-garnet into two jobs: - build-garnet: 8 jobs (os × framework × config) that restore, build, and upload bin/obj artifacts with 1-day retention - test-garnet: 128 jobs that download build artifacts and run dotnet test --no-build --no-restore This eliminates 120 redundant builds (~316 CPU-minutes per run). Test jobs no longer need Rust toolchain, NuGet cache, or restore steps. The pipeline-success job now depends on test-garnet instead of build-test-garnet. Co-authored-by: Copilot <[email protected]>
Split the monolithic Tsavorite.test project into 5 test subprojects to enable parallel CI execution and reduce the critical-path bottleneck: - Tsavorite.test (core tests, shared infrastructure) - Tsavorite.test.recordops (revivification, delete/dispose, record lifecycle) - Tsavorite.test.session (session, unsafe context, read cache chain tests) - Tsavorite.test.hlog (log, scan, device, spanbyte, compaction tests) - Tsavorite.test.recovery (recovery, checkpoint, object recovery tests) CI workflow changes: - ci.yml: Split build-test-tsavorite into separate build-tsavorite and test-tsavorite jobs with 5x4 matrix (os x framework x config x subproject) - nightly.yml: Added new subprojects to test matrix with proper allure-results staging and cleanup paths - Scoped NuGet restore to Tsavorite.slnx only in build job Co-authored-by: Copilot <[email protected]>
- Add npm cache (actions/cache) for Azurite in both ci.yml and nightly.yml - In ci.yml, conditionally skip Node.js setup, Azurite install, and RunAzureTests env var for test.recordops, test.session, and test.recovery (only test and test.hlog use Azure storage device tests) - Saves ~200s on Windows / ~20s on Linux per skipped subproject job Co-authored-by: Copilot <[email protected]>
upload-artifact strips the common ancestor (libs/storage/Tsavorite/cs/) from the glob paths. Specify path: on download-artifact to restore the prefix so the Allure check and test run steps find DLLs at the expected location. Co-authored-by: Copilot <[email protected]>
MSBuild graph build constructs the project dependency graph upfront and schedules builds optimally, avoiding redundant project evaluations. Co-authored-by: Copilot <[email protected]>
Run format-garnet and format-tsavorite in parallel with builds instead of blocking them. Format failures still gate pipeline-success, but builds and tests can start immediately, saving ~1-2 min off the critical path. Co-authored-by: Copilot <[email protected]>
dotnet test with --no-build against project directories silently exits with 0 tests when MSBuild evaluation fails to locate the pre-built assemblies. Running against the DLL directly bypasses MSBuild entirely and reliably discovers and executes tests from downloaded artifacts. Co-authored-by: Copilot <[email protected]>
upload-artifact strips the Unix execute bit. Add chmod +x for the GarnetServer binary on Linux after downloading build artifacts, so tests that launch it as a subprocess (e.g., GarnetBitmapTests) work. Co-authored-by: Copilot <[email protected]>
Split the Tsavorite.test.session subproject (23.1 min worst case) into two for better CI parallelism: - test.session: ReadCacheChainTests, ReadAddressTests, NativeReadCacheTests, RandomReadCacheTests (read-cache focused) - test.session.context: TransactionalUnsafeContextTests, UnsafeContextTests, SessionTests, FunctionPerSessionTests (session/context focused) Extracted shared helpers into test/SessionContextTestUtils.cs to resolve cross-project dependencies. Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Make manySegments imply lowMemory instead of requiring both parameters. This reduces test combinations from 16 to 8 (removes redundant cases where lowMemory=false,manySegments=true which was already a no-op due to the existing manySegments = lowMemory && manySegments guard). Co-authored-by: Copilot <[email protected]>
badrishc
reviewed
May 13, 2026
Move Garnet.test* projects under test/standalone/ and Garnet.test.cluster* projects under test/cluster/ for better organization. - Move 8 standalone test projects to test/standalone/ - Move 8 cluster test projects to test/cluster/ - Update all .csproj relative paths (ProjectReference, Compile Include, EmbeddedResource, AssemblyOriginatorKeyFile, testcerts) - Update Garnet.slnx with nested folder structure - Update TstRunner.csproj project references - Update CI workflows (ci.yml, nightly.yml) with subdir resolution - Update runGarnetTests.cmd path Co-authored-by: Copilot <[email protected]>
Move satellite test projects (test.hlog, test.recordops, test.recovery, test.session, test.session.context) from cs/ into cs/test/ as subdirectories. - Move 5 satellite test project directories under cs/test/ - Update satellite csproj relative paths (src/, Garnet.snk, Tsavorite.test ref) - Add Compile Remove glob to Tsavorite.test.csproj to exclude subdirectories - Update Tsavorite.slnx project paths - Fix Tsavorite.test.csproj AllureTestBase.cs path for prior Garnet test move - Update CI workflows (ci.yml, nightly.yml) Tsavorite directory maps Co-authored-by: Copilot <[email protected]>
Pre-built native BfTree binaries exist for all platforms (linux-x64, win-x64, osx-x64, osx-arm64). The BfTreeInterop.csproj already gracefully skips cargo when it's not installed, falling back to pre-built binaries. Rust is only needed to rebuild from source for bftree/range-index development. Co-authored-by: Copilot <[email protected]>
- Remove AllureTestBase class, rename to TestBase (keep RunningTests tracking) - Remove [AllureNUnit] attribute from all 158 test fixtures - Remove Allure.Net.Commons and Allure.NUnit package references from all csproj files - Remove Allure packages from Directory.Packages.props - Remove Allure wiring verification steps from ci.yml (Garnet + Tsavorite) - Remove Allure CLI install, results staging, report generation from nightly.yml - Remove Allure artifact/history/report steps from deploy-website.yml - Delete test/Allure/ directory (GenerateAllureReport.ps1, categories.json) - Update docs: README badge, copilot-instructions, skills, onboarding Co-authored-by: Copilot <[email protected]>
The -graph flag on dotnet build silently skips projects inside nested <Folder> elements in .slnx files. After restructuring test directories into test/standalone/ and test/cluster/ with nested solution folders, only 4 of 16 test projects were built, causing 'DLL not found' errors in 33 of 36 test jobs. Changes: - Remove -graph flag from build-garnet and build-tsavorite jobs - Remove artifact upload/download pipeline between build and test jobs - Test jobs now build inline via dotnet test <project-dir> (matching the pattern used on main branch) - Add NuGet cache and restore to test jobs - Remove redundant framework dimension from build-garnet matrix - Keep build jobs as fast-fail compilation gates Co-authored-by: Copilot <[email protected]>
After moving test projects into test/standalone/ and test/cluster/,
Split("Garnet.test")[0] returns test/standalone/ instead of test/.
Navigate up one level to reach the correct test/ root directory.
Fixes DocsTests, RespCommandTests, RespCustomCommandTests, and
RespModuleTests failures caused by incorrect relative path resolution.
Co-authored-by: Copilot <[email protected]>
badrishc
approved these changes
May 13, 2026
badrishc
approved these changes
May 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes two CI issues with Allure test wiring in the newly split cluster replication test projects.
Issue 1 - Runtime Allure error
Derived test classes (TLS, AsyncReplay, MultiLog) had
[AllureNUnit]applied directly and inherited it from their base classes. NUnit discoversITestActionattributes from both, causing Allure'sBeforeTestto run twice and crash with:Fix: Removed the duplicate
[AllureNUnit]from the 4 derived classes - the inherited attribute from the base class is sufficient.Issue 2 - CI Allure wiring check failure
The CI
Verify Allure wiringstep usedAssembly.Load()to findAllureTestBasein referenced assemblies. This failed because:Assembly.Load()does not probe the build output directory (only works for GAC/default paths)AllureTestBaselives inGarnet.test.cluster, which is a transitive dependency for some projects (e.g., TLS references it throughGarnet.test.cluster.replication, not directly)Fix: Updated the CI script to search
AppDomain.GetAssemblies()- all transitive dependencies are already loaded by the runtime duringGetTypes().Changes
ClusterReplicationTLS.cs- remove duplicate[AllureNUnit]ClusterReplicationAsyncReplay.cs- remove duplicate[AllureNUnit]ClusterReplicationShardedLog.cs- remove duplicate[AllureNUnit]ClusterReplicationDisklessSyncShardedLog.cs- remove duplicate[AllureNUnit].github/workflows/ci.yml- fixAllureTestBaseresolution for transitive dependencies