Skip to content

Fix Allure wiring: remove duplicate attributes and fix CI check for transitive dependencies#1784

Merged
vazois merged 33 commits into
mainfrom
vazois/ci-runtime
May 13, 2026
Merged

Fix Allure wiring: remove duplicate attributes and fix CI check for transitive dependencies#1784
vazois merged 33 commits into
mainfrom
vazois/ci-runtime

Conversation

@vazois

@vazois vazois commented May 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes two CI issues with Allure test wiring in the newly split cluster replication test projects.

Issue 1 - Runtime Allure error

Derived test classes (TLS, AsyncReplay, MultiLog) had [AllureNUnit] applied directly and inherited it from their base classes. NUnit discovers ITestAction attributes from both, causing Allure's BeforeTest to run twice and crash with:

System.InvalidOperationException: Unable to change the container context because the test context is active.

Fix: Removed the duplicate [AllureNUnit] from the 4 derived classes - the inherited attribute from the base class is sufficient.

Issue 2 - CI Allure wiring check failure

The CI Verify Allure wiring step used Assembly.Load() to find AllureTestBase in referenced assemblies. This failed because:

  1. Assembly.Load() does not probe the build output directory (only works for GAC/default paths)
  2. AllureTestBase lives in Garnet.test.cluster, which is a transitive dependency for some projects (e.g., TLS references it through Garnet.test.cluster.replication, not directly)

Fix: Updated the CI script to search AppDomain.GetAssemblies() - all transitive dependencies are already loaded by the runtime during GetTypes().

Changes

  • ClusterReplicationTLS.cs - remove duplicate [AllureNUnit]
  • ClusterReplicationAsyncReplay.cs - remove duplicate [AllureNUnit]
  • ClusterReplicationShardedLog.cs - remove duplicate [AllureNUnit]
  • ClusterReplicationDisklessSyncShardedLog.cs - remove duplicate [AllureNUnit]
  • .github/workflows/ci.yml - fix AllureTestBase resolution for transitive dependencies

Copilot AI review requested due to automatic review settings May 9, 2026 04:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses Allure/NUnit wiring issues after splitting tests into multiple projects, and updates CI to correctly validate Allure inheritance across transitive dependencies. It also introduces several new split test projects and adds/relocates a number of test suites into those projects.

Changes:

  • Remove duplicate Allure attributes from derived fixtures and ensure derived fixtures are still discovered by NUnit.
  • Update CI “Verify Allure wiring” to resolve AllureTestBase from transitive dependencies via loaded AppDomain assemblies.
  • Split tests into additional projects (collections/ACL/scripting/complexstring/vectorset/rangeindex/extensions + cluster sub-suites) and update InternalsVisibleTo, solution, and workflow matrices accordingly.

Reviewed changes

Copilot reviewed 29 out of 91 changed files in this pull request and generated no comments.

Show a summary per file
File Description
test/Garnet.test/Garnet.test.csproj Adds InternalsVisibleTo entries to support newly split test projects; removes linked global usings include.
test/Garnet.test.vectorset/VectorCleanupVsResetRaceTests.cs Adds regression test for VectorManager cleanup vs store reset race using UnsafeAccessor.
test/Garnet.test.vectorset/Garnet.test.vectorset.csproj New vector-set focused test project.
test/Garnet.test.scripting/Garnet.test.scripting.csproj New scripting-focused test project.
test/Garnet.test.rangeindex/Garnet.test.rangeindex.csproj New range-index focused test project.
test/Garnet.test.extensions/ScratchBufferAllocatorTests.cs Adds tests for ScratchBufferAllocator behavior.
test/Garnet.test.extensions/RespRevivificationTests.cs Adds revivification behavior tests (RMW/Upsert/AOF interactions).
test/Garnet.test.extensions/ReadOptimizedLockTests.cs Adds unit tests for ReadOptimizedLock correctness under contention.
test/Garnet.test.extensions/ReadCacheTests.cs Adds tests validating read-cache population/eviction for string and object records.
test/Garnet.test.extensions/IndexGrowthTests.cs Adds tests validating index resizing behavior (including checkpoint/recovery).
test/Garnet.test.extensions/GlobalUsings.cs Provides local aliases used by extensions tests.
test/Garnet.test.extensions/GarnetJSON/JSONPath/QueryExpressionTests.cs Adds JSONPath query expression tests under extensions suite.
test/Garnet.test.extensions/GarnetJSON/JSONPath/JsonPathRegressionTests.cs Adds JSONPath regression suite runner using embedded JSON cases.
test/Garnet.test.extensions/GarnetJSON/JSONPath/JsonAssert.cs Adds JSON element/node equality helpers for JSONPath tests.
test/Garnet.test.extensions/Garnet.test.extensions.csproj New extensions test project (includes GarnetJSON + DiskANN tests/resources).
test/Garnet.test.extensions/DiskANN/DiskANNSyntheticRecallTests.cs Adds DiskANN synthetic recall tests (grid/circle datasets).
test/Garnet.test.complexstring/Garnet.test.complexstring.csproj New complex-string focused test project.
test/Garnet.test.collections/RespListGarnetClientTests.cs Adds/relocates GarnetClient list command tests into collections suite.
test/Garnet.test.collections/GeoHashTests.cs Adds GeoHash encode/decode tests into collections suite.
test/Garnet.test.collections/GarnetObjectTests.cs Adds object serialization/checkpoint/recovery tests using Tsavorite directly.
test/Garnet.test.collections/Garnet.test.collections.csproj New collections-focused test project (links shared global usings).
test/Garnet.test.cluster/Garnet.test.cluster.csproj Adds InternalsVisibleTo for new cluster sub-projects; links AllureTestBase.
test/Garnet.test.cluster/CustomProcs/ClusterDelRmw.cs Makes ClusterDelRmw public for cross-assembly use.
test/Garnet.test.cluster.vectorsets/Garnet.test.cluster.vectorsets.csproj New cluster vector-sets test project.
test/Garnet.test.cluster.replication/ReplicationTests/ClusterResetDuringReplicationTests.cs Adds DEBUG-only tests for CLUSTER RESET HARD during replication attach.
test/Garnet.test.cluster.replication/Garnet.test.cluster.replication.csproj New cluster replication test project; exposes internals to dependent suites.
test/Garnet.test.cluster.replication.tls/ReplicationTests/ClusterReplicationTLS.cs Removes duplicate Allure attribute by relying on base; ensures fixture discovery.
test/Garnet.test.cluster.replication.tls/Garnet.test.cluster.replication.tls.csproj New TLS-specific replication test project.
test/Garnet.test.cluster.replication.disklesssync/Garnet.test.cluster.replication.disklesssync.csproj New diskless-sync focused replication test project.
test/Garnet.test.cluster.replication.asyncreplay/ReplicationTests/ClusterReplicationAsyncReplay.cs Removes duplicate Allure attribute by relying on base; ensures fixture discovery.
test/Garnet.test.cluster.replication.asyncreplay/Garnet.test.cluster.replication.asyncreplay.csproj New async-replay replication test project.
test/Garnet.test.cluster.multilog/MultiLogTests/ClusterReplicationShardedLog.cs Adds multilog/sharded-log replication test fixture wrapper.
test/Garnet.test.cluster.multilog/MultiLogTests/ClusterReplicationDisklessSyncShardedLog.cs Adds multilog wrapper for diskless-sync replication tests.
test/Garnet.test.cluster.multilog/Garnet.test.cluster.multilog.csproj New multilog-focused cluster test project.
test/Garnet.test.cluster.migrate/RedirectTests/TestClusterProc.cs Adds custom txn procedures and redirect test commands for migrate suite.
test/Garnet.test.cluster.migrate/Garnet.test.cluster.migrate.csproj New cluster migrate test project.
test/Garnet.test.cluster.migrate/ClusterMigrateTLSTests.cs Adds TLS variant wrapper fixture for cluster migrate tests.
test/Garnet.test.acl/Resp/GarnetAuthenticatorTests.cs Adds tests for authenticator invalidation behavior.
test/Garnet.test.acl/Resp/ACL/UserAclResult.cs Adds helper for parsing ACL GETUSER responses.
test/Garnet.test.acl/Resp/ACL/ParallelTests.cs Adds parallel stress tests for AUTH/password hashing/ACL SETUSER.
test/Garnet.test.acl/Resp/ACL/GetUserTests.cs Adds ACL GETUSER tests including multi-user scenarios.
test/Garnet.test.acl/Resp/ACL/DeleteUserTests.cs Adds ACL DELUSER behavior tests.
test/Garnet.test.acl/Resp/ACL/BasicTests.cs Adds tests for basic ACL commands and NoAuth flag validation.
test/Garnet.test.acl/Resp/ACL/AclTest.cs Introduces ACL test base class with shared setup/teardown and constants.
test/Garnet.test.acl/Resp/ACL/AclParserTests.cs Adds ACL parser reduction/timeout regression tests.
test/Garnet.test.acl/Resp/ACL/AclConfigurationFileTests.cs Adds tests around ACL file load/save semantics and error handling.
test/Garnet.test.acl/Garnet.test.acl.csproj New ACL-focused test project.
playground/TstRunner/TstRunner.csproj Adds references to new cluster migrate/replication test projects.
modules/GarnetJSON/GarnetJSON.csproj Adds InternalsVisibleTo for Garnet.test.extensions.
main/GarnetServer/GarnetServer.csproj Adds InternalsVisibleTo for newly split test projects.
libs/storage/Tsavorite/cs/src/core/Tsavorite.core.csproj Adds InternalsVisibleTo for Garnet.test.extensions.
libs/server/Properties/AssemblyInfo.cs Adds InternalsVisibleTo for newly split test projects.
libs/host/Garnet.host.csproj Adds InternalsVisibleTo for newly split test projects.
Garnet.slnx Adds new test projects to the solution.
.github/workflows/nightly.yml Expands nightly test matrix to include new test projects.
.github/workflows/ci.yml Expands CI test matrix and fixes Allure wiring verification for transitive deps.

@vazois vazois force-pushed the vazois/ci-runtime branch from 0d136c5 to ea78bcc Compare May 11, 2026 22:30
@vazois vazois changed the base branch from dev to main May 11, 2026 22:31
vazois and others added 7 commits May 11, 2026 15:31
Split Garnet.test.cluster into separate projects to enable parallel CI:
- Garnet.test.cluster: shared infra + basic cluster tests (144 tests)
- Garnet.test.cluster.migrate: migrate + slot verification (67 tests)
- Garnet.test.cluster.replication: all replication tests (372 tests)
- Garnet.test.cluster.vectorsets: vector set cluster tests (30 tests)
- Garnet.test.cluster.multilog: sharded log replication tests (163 tests)

Child projects reference base via ProjectReference + InternalsVisibleTo.
Updated CI and nightly workflow matrices to run all 5 in parallel.
Fixed Allure wiring check to find AllureTestBase in referenced assemblies.

Co-authored-by: Copilot <[email protected]>
Separate Garnet.test.cluster.replication into 3 additional projects
for parallel CI execution:

- Garnet.test.cluster.replication.tls: TLS replication tests
  (inherits ClusterReplicationBaseTests with useTLS=true)
- Garnet.test.cluster.replication.asyncreplay: Async replay tests
  (inherits ClusterReplicationBaseTests with asyncReplay=true)
- Garnet.test.cluster.replication.disklesssync: Diskless sync tests
  (standalone ClusterReplicationDisklessSyncTests)

Base replication project retains ClusterReplicationBaseTests and
ClusterResetDuringReplicationTests. Updated InternalsVisibleTo,
multilog project references, CI and nightly workflow matrices,
and solution file.

Co-authored-by: Copilot <[email protected]>
Split the monolithic Garnet.test project into 8 focused test projects:
- Garnet.test (base): RESP core, config, admin, infra (~774 tests)
- Garnet.test.collections: Hash, List, Set, SortedSet, Geo (~746 tests)
- Garnet.test.acl: ACL and auth tests (~426 tests)
- Garnet.test.scripting: Lua, custom commands, transactions, AOF, modules (~585 tests)
- Garnet.test.complexstring: Bitmap, HyperLogLog (~386 tests)
- Garnet.test.vectorset: VectorSet tests (~36 tests)
- Garnet.test.rangeindex: RangeIndex tests (~58 tests)
- Garnet.test.extensions: JSON, DiskANN, revivification, storage internals (~527 tests)

Each child project references Garnet.test for shared infrastructure (TestUtils,
AllureTestBase, extensions). InternalsVisibleTo entries added to Garnet.server,
Garnet.host, GarnetServer, GarnetJSON, and Tsavorite.core for child projects.

Updated CI and nightly workflows with expanded test matrices.

Co-authored-by: Copilot <[email protected]>
…ures

Added [AllureNUnit] and [TestFixture] to ClusterReplicationTLS,
ClusterReplicationAsyncReplay, ClusterReplicationShardedLog, and
ClusterReplicationDisklessSyncShardedLog to pass CI Allure wiring check.

Co-authored-by: Copilot <[email protected]>
…r transitive dependencies

- Remove duplicate [AllureNUnit] from derived test classes (TLS, AsyncReplay,
  MultiLog) that inherit it from base classes, fixing runtime error
  'Unable to change the container context because the test context is active'
- Fix CI Allure wiring check to search AppDomain.GetAssemblies() instead of
  using Assembly.Load() on direct references, which failed for AllureTestBase
  in transitive dependencies (e.g. Garnet.test.cluster via Garnet.test.cluster.replication)

Co-authored-by: Copilot <[email protected]>
Add actions/cache@v4 for ~/.nuget/packages to format-garnet,
format-tsavorite, build-test-garnet, and build-test-tsavorite jobs.
Cache key is based on runner.os and hash of *.csproj and
Directory.Packages.props files, with a fallback restore-key for
partial matches.

This reduces dotnet restore time, especially on Windows runners
where install dependencies averaged 1.1 min (up to 4.1 min) per job.

Co-authored-by: Copilot <[email protected]>
Split build-test-garnet into two jobs:
- build-garnet: 8 jobs (os × framework × config) that restore, build,
  and upload bin/obj artifacts with 1-day retention
- test-garnet: 128 jobs that download build artifacts and run
  dotnet test --no-build --no-restore

This eliminates 120 redundant builds (~316 CPU-minutes per run).
Test jobs no longer need Rust toolchain, NuGet cache, or restore steps.
The pipeline-success job now depends on test-garnet instead of
build-test-garnet.

Co-authored-by: Copilot <[email protected]>
@vazois vazois force-pushed the vazois/ci-runtime branch from ea78bcc to 92601a4 Compare May 11, 2026 22:32
vazois and others added 12 commits May 12, 2026 10:27
Split the monolithic Tsavorite.test project into 5 test subprojects to enable
parallel CI execution and reduce the critical-path bottleneck:

- Tsavorite.test (core tests, shared infrastructure)
- Tsavorite.test.recordops (revivification, delete/dispose, record lifecycle)
- Tsavorite.test.session (session, unsafe context, read cache chain tests)
- Tsavorite.test.hlog (log, scan, device, spanbyte, compaction tests)
- Tsavorite.test.recovery (recovery, checkpoint, object recovery tests)

CI workflow changes:
- ci.yml: Split build-test-tsavorite into separate build-tsavorite and
  test-tsavorite jobs with 5x4 matrix (os x framework x config x subproject)
- nightly.yml: Added new subprojects to test matrix with proper allure-results
  staging and cleanup paths
- Scoped NuGet restore to Tsavorite.slnx only in build job

Co-authored-by: Copilot <[email protected]>
- Add npm cache (actions/cache) for Azurite in both ci.yml and nightly.yml
- In ci.yml, conditionally skip Node.js setup, Azurite install, and
  RunAzureTests env var for test.recordops, test.session, and test.recovery
  (only test and test.hlog use Azure storage device tests)
- Saves ~200s on Windows / ~20s on Linux per skipped subproject job

Co-authored-by: Copilot <[email protected]>
upload-artifact strips the common ancestor (libs/storage/Tsavorite/cs/)
from the glob paths. Specify path: on download-artifact to restore the
prefix so the Allure check and test run steps find DLLs at the expected
location.

Co-authored-by: Copilot <[email protected]>
MSBuild graph build constructs the project dependency graph upfront
and schedules builds optimally, avoiding redundant project evaluations.

Co-authored-by: Copilot <[email protected]>
Run format-garnet and format-tsavorite in parallel with builds instead
of blocking them. Format failures still gate pipeline-success, but
builds and tests can start immediately, saving ~1-2 min off the
critical path.

Co-authored-by: Copilot <[email protected]>
dotnet test with --no-build against project directories silently exits
with 0 tests when MSBuild evaluation fails to locate the pre-built
assemblies. Running against the DLL directly bypasses MSBuild entirely
and reliably discovers and executes tests from downloaded artifacts.

Co-authored-by: Copilot <[email protected]>
upload-artifact strips the Unix execute bit. Add chmod +x for the
GarnetServer binary on Linux after downloading build artifacts, so
tests that launch it as a subprocess (e.g., GarnetBitmapTests) work.

Co-authored-by: Copilot <[email protected]>
Split the Tsavorite.test.session subproject (23.1 min worst case) into
two for better CI parallelism:

- test.session: ReadCacheChainTests, ReadAddressTests,
  NativeReadCacheTests, RandomReadCacheTests (read-cache focused)
- test.session.context: TransactionalUnsafeContextTests,
  UnsafeContextTests, SessionTests, FunctionPerSessionTests
  (session/context focused)

Extracted shared helpers into test/SessionContextTestUtils.cs to
resolve cross-project dependencies.

Co-authored-by: Copilot <[email protected]>
Make manySegments imply lowMemory instead of requiring both parameters.
This reduces test combinations from 16 to 8 (removes redundant cases
where lowMemory=false,manySegments=true which was already a no-op due
to the existing manySegments = lowMemory && manySegments guard).

Co-authored-by: Copilot <[email protected]>
Comment thread libs/storage/Tsavorite/cs/test/test.hlog/BasicStorageTests.cs
Move Garnet.test* projects under test/standalone/ and Garnet.test.cluster*
projects under test/cluster/ for better organization.

- Move 8 standalone test projects to test/standalone/
- Move 8 cluster test projects to test/cluster/
- Update all .csproj relative paths (ProjectReference, Compile Include,
  EmbeddedResource, AssemblyOriginatorKeyFile, testcerts)
- Update Garnet.slnx with nested folder structure
- Update TstRunner.csproj project references
- Update CI workflows (ci.yml, nightly.yml) with subdir resolution
- Update runGarnetTests.cmd path

Co-authored-by: Copilot <[email protected]>
vazois and others added 11 commits May 13, 2026 11:01
Move satellite test projects (test.hlog, test.recordops, test.recovery,
test.session, test.session.context) from cs/ into cs/test/ as subdirectories.

- Move 5 satellite test project directories under cs/test/
- Update satellite csproj relative paths (src/, Garnet.snk, Tsavorite.test ref)
- Add Compile Remove glob to Tsavorite.test.csproj to exclude subdirectories
- Update Tsavorite.slnx project paths
- Fix Tsavorite.test.csproj AllureTestBase.cs path for prior Garnet test move
- Update CI workflows (ci.yml, nightly.yml) Tsavorite directory maps

Co-authored-by: Copilot <[email protected]>
Pre-built native BfTree binaries exist for all platforms (linux-x64,
win-x64, osx-x64, osx-arm64). The BfTreeInterop.csproj already gracefully
skips cargo when it's not installed, falling back to pre-built binaries.
Rust is only needed to rebuild from source for bftree/range-index development.

Co-authored-by: Copilot <[email protected]>
- Remove AllureTestBase class, rename to TestBase (keep RunningTests tracking)
- Remove [AllureNUnit] attribute from all 158 test fixtures
- Remove Allure.Net.Commons and Allure.NUnit package references from all csproj files
- Remove Allure packages from Directory.Packages.props
- Remove Allure wiring verification steps from ci.yml (Garnet + Tsavorite)
- Remove Allure CLI install, results staging, report generation from nightly.yml
- Remove Allure artifact/history/report steps from deploy-website.yml
- Delete test/Allure/ directory (GenerateAllureReport.ps1, categories.json)
- Update docs: README badge, copilot-instructions, skills, onboarding

Co-authored-by: Copilot <[email protected]>
The -graph flag on dotnet build silently skips projects inside nested
<Folder> elements in .slnx files. After restructuring test directories
into test/standalone/ and test/cluster/ with nested solution folders,
only 4 of 16 test projects were built, causing 'DLL not found' errors
in 33 of 36 test jobs.

Changes:
- Remove -graph flag from build-garnet and build-tsavorite jobs
- Remove artifact upload/download pipeline between build and test jobs
- Test jobs now build inline via dotnet test <project-dir> (matching
  the pattern used on main branch)
- Add NuGet cache and restore to test jobs
- Remove redundant framework dimension from build-garnet matrix
- Keep build jobs as fast-fail compilation gates

Co-authored-by: Copilot <[email protected]>
After moving test projects into test/standalone/ and test/cluster/,
Split("Garnet.test")[0] returns test/standalone/ instead of test/.
Navigate up one level to reach the correct test/ root directory.

Fixes DocsTests, RespCommandTests, RespCustomCommandTests, and
RespModuleTests failures caused by incorrect relative path resolution.

Co-authored-by: Copilot <[email protected]>
@vazois vazois merged commit d6d4888 into main May 13, 2026
194 checks passed
@vazois vazois deleted the vazois/ci-runtime branch May 13, 2026 23:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants