Harden RESTORE validation and block malformed HLL payload misuse in PF operations by vazois · Pull Request #1599 · microsoft/garnet

vazois · 2026-03-04T00:06:59Z

Summary

This PR fixes RESTORE misuse paths that allowed tampered dump payloads to be accepted and later used by HyperLogLog operations, including when checksum validation is explicitly skipped.

Changes

Fixed RESTORE checksum gate in libs/server/Resp/KeyAdminCommands.cs to validate CRC by default (!SkipRDBRestoreChecksumValidation).
Hardened HLL value validation in libs/server/Resp/HyperLogLog/HyperLogLog.cs:
- Corrected dense/sparse length validation logic.
- Enforced sparse payload bound (SparseRLESize <= allocated sparse payload).
- Added full sparse stream semantic validation (IsValidSparseStream) to verify:
  - opcode validity,
  - non-zero register value bounds,
  - exact register-space coverage (no over/under-run).
Added defense-in-depth PF-path checks in libs/server/Storage/Session/MainStore/HyperLogLogOps.cs to reject malformed HLL blobs before PFCOUNT/PFMERGE processing.
Extended test server factory in test/Garnet.test/TestUtils.cs with skipRDBRestoreChecksumValidation to exercise checksum-skip scenarios.
Added/updated security regressions in test/Garnet.test/HyperLogLogTests.cs:
- corrupted dump payload rejection on RESTORE,
- zero-CRC payload rejection on RESTORE,
- corrupted sparse-RLE payload rejection on RESTORE,
- sparse stream validator rejection cases,
- checksum-skip mode test asserting malformed restored HLL returns WRONGTYPE for PFCOUNT/PFADD/PFMERGE,
- representation/length parsing coverage for sparse/dense and 6/14/32-bit parser branches.

Copilot

Pull request overview

This PR strengthens Garnet’s handling of RESTORE payloads and HyperLogLog (PF*) operations to prevent malformed/tampered serialized values from being accepted and later misused during PF processing.

Changes:

Enables RESTORE checksum validation by default (unless explicitly skipped via server option).
Hardens HyperLogLog blob validation, including strict sparse-stream semantic checks.
Adds/extends test utilities and regression tests to cover corrupted dumps and checksum-skip scenarios.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
libs/server/Resp/KeyAdminCommands.cs	Flips RESTORE checksum-validation gate to validate by default.
libs/server/Resp/HyperLogLog/HyperLogLog.cs	Tightens HLL length checks and validates sparse opcode stream semantics/coverage.
libs/server/Storage/Session/MainStore/HyperLogLogOps.cs	Adds PFCOUNT/PFMERGE defense-in-depth validation before processing blobs.
test/Garnet.test/TestUtils.cs	Extends test server factory to optionally skip RESTORE checksum validation.
test/Garnet.test/HyperLogLogTests.cs	Adds regression tests for corrupted dumps, sparse-stream validator cases, and checksum-skip behavior.

Comments suppressed due to low confidence (1)

libs/server/Resp/KeyAdminCommands.cs:81

RESTORE CRC computation appears to hash the value-type byte as well (valueSpan.Slice(0, valueSpan.Length - 8)), but DUMP computes CRC starting after the value-type byte (payloadToHash slice starts after writing the type byte). With checksum validation now enabled by default, this mismatch would cause RESTORE to reject dumps produced by DUMP. Adjust the RESTORE hash input to skip the leading type byte so both commands use the same CRC coverage.

            if (!storeWrapper.serverOptions.SkipRDBRestoreChecksumValidation)
            {
                // crc is calculated over the encoded payload length, payload and the rdb version bytes
                // skip's the value type byte and crc64 bytes
                var calculatedCrc = new ReadOnlySpan<byte>(Crc64.Hash(valueSpan.Slice(0, valueSpan.Length - 8)));

                // skip's rdb version bytes
                var payloadCrc = footer[2..];

                if (calculatedCrc.SequenceCompareTo(payloadCrc) != 0)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

* Bump the nuget-deps group with 14 updates Bumps diskann-garnet from 1.0.23 to 1.0.25 Bumps Microsoft.Extensions.Configuration.Binder from 10.0.2 to 10.0.3 Bumps Microsoft.Extensions.Configuration.Json from 10.0.2 to 10.0.3 Bumps Microsoft.Extensions.Logging from 10.0.2 to 10.0.3 Bumps Microsoft.Extensions.Logging.Configuration from 9.0.8 to 10.0.3 Bumps Microsoft.Extensions.Logging.Console from 9.0.8 to 10.0.3 Bumps Microsoft.IdentityModel.Protocols.OpenIdConnect from 8.6.1 to 8.16.0 Bumps Microsoft.IdentityModel.Validators from 8.6.1 to 8.16.0 Bumps Microsoft.NET.Test.Sdk from 18.0.1 to 18.3.0 Bumps NUnit from 4.1.0 to 4.5.0 Bumps NUnit3TestAdapter from 4.6.0 to 6.1.0 Bumps StackExchange.Redis from 2.9.25 to 2.11.8 Bumps System.IdentityModel.Tokens.Jwt from 8.6.1 to 8.16.0 Bumps System.Numerics.Tensors from 9.0.9 to 10.0.3 --- updated-dependencies: - dependency-name: diskann-garnet dependency-version: 1.0.25 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Configuration.Binder dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Configuration.Json dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Logging dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Logging.Configuration dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-major dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Logging.Console dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-major dependency-group: nuget-deps - dependency-name: Microsoft.IdentityModel.Protocols.OpenIdConnect dependency-version: 8.16.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: System.IdentityModel.Tokens.Jwt dependency-version: 8.16.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: Microsoft.IdentityModel.Validators dependency-version: 8.16.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: Microsoft.NET.Test.Sdk dependency-version: 18.3.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: NUnit dependency-version: 4.5.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: NUnit3TestAdapter dependency-version: 6.1.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: nuget-deps - dependency-name: StackExchange.Redis dependency-version: 2.11.8 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: System.Numerics.Tensors dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-major dependency-group: nuget-deps ... Signed-off-by: dependabot[bot] <[email protected]> * fix changes in API * revert unit test upgrade * Add ConfigureAwait(false) to all test await expressions to fix NUnit 4.5 deadlock (#1593) * Initial plan * Add .ConfigureAwait(false) to all await expressions in test files Add .ConfigureAwait(false) to every await expression across 38 test files under test/ to follow best practices for library/test code and avoid potential deadlocks in synchronization contexts. - Handles single-line and multi-line await expressions - Handles ternary await expressions (both branches) - Skips await foreach and await using (already correct) - Excludes Task.Yield() which returns YieldAwaitable (no ConfigureAwait support) - No changes outside test/ directory Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Vasileios Zois <[email protected]> * upgrade nunit * Fix MigrateVector cluster tests for StackExchange.Redis 2.11.8 (#1595) * Initial plan * Fix MigrateVector tests to handle transient errors from StackExchange.Redis 2.11.8 Co-authored-by: vazois <[email protected]> * Fix MigrateVector tests: use Exception catch + specific MOVED message patterns for SE.Redis 2.11.8 Co-authored-by: vazois <[email protected]> --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: vazois <[email protected]> * revert diskann version --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Vasileios Zois <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Vasileios Zois <[email protected]> Co-authored-by: Tal Zaccai <[email protected]>

…L payload

…F operations (#1599) * SkipChecksum validation testing and fix * validate HLL format with skipRDBRestoreChecksumValidation * add tests that cover different length encoding * add full sparse stream validation * skipRDBRestoreChecksumValidation for slot validation tests * remove double validation and fix buffer swap * reduce test overhead * fix redirection tests for RESTORE * eliminate SkipRDBRestoreChecksumValidation option * Bump the nuget-deps group with 14 updates (#1592) * Bump the nuget-deps group with 14 updates Bumps diskann-garnet from 1.0.23 to 1.0.25 Bumps Microsoft.Extensions.Configuration.Binder from 10.0.2 to 10.0.3 Bumps Microsoft.Extensions.Configuration.Json from 10.0.2 to 10.0.3 Bumps Microsoft.Extensions.Logging from 10.0.2 to 10.0.3 Bumps Microsoft.Extensions.Logging.Configuration from 9.0.8 to 10.0.3 Bumps Microsoft.Extensions.Logging.Console from 9.0.8 to 10.0.3 Bumps Microsoft.IdentityModel.Protocols.OpenIdConnect from 8.6.1 to 8.16.0 Bumps Microsoft.IdentityModel.Validators from 8.6.1 to 8.16.0 Bumps Microsoft.NET.Test.Sdk from 18.0.1 to 18.3.0 Bumps NUnit from 4.1.0 to 4.5.0 Bumps NUnit3TestAdapter from 4.6.0 to 6.1.0 Bumps StackExchange.Redis from 2.9.25 to 2.11.8 Bumps System.IdentityModel.Tokens.Jwt from 8.6.1 to 8.16.0 Bumps System.Numerics.Tensors from 9.0.9 to 10.0.3 --- updated-dependencies: - dependency-name: diskann-garnet dependency-version: 1.0.25 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Configuration.Binder dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Configuration.Json dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Logging dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Logging.Configuration dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-major dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Logging.Console dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-major dependency-group: nuget-deps - dependency-name: Microsoft.IdentityModel.Protocols.OpenIdConnect dependency-version: 8.16.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: System.IdentityModel.Tokens.Jwt dependency-version: 8.16.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: Microsoft.IdentityModel.Validators dependency-version: 8.16.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: Microsoft.NET.Test.Sdk dependency-version: 18.3.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: NUnit dependency-version: 4.5.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: NUnit3TestAdapter dependency-version: 6.1.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: nuget-deps - dependency-name: StackExchange.Redis dependency-version: 2.11.8 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: System.Numerics.Tensors dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-major dependency-group: nuget-deps ... Signed-off-by: dependabot[bot] <[email protected]> * fix changes in API * revert unit test upgrade * Add ConfigureAwait(false) to all test await expressions to fix NUnit 4.5 deadlock (#1593) * Initial plan * Add .ConfigureAwait(false) to all await expressions in test files Add .ConfigureAwait(false) to every await expression across 38 test files under test/ to follow best practices for library/test code and avoid potential deadlocks in synchronization contexts. - Handles single-line and multi-line await expressions - Handles ternary await expressions (both branches) - Skips await foreach and await using (already correct) - Excludes Task.Yield() which returns YieldAwaitable (no ConfigureAwait support) - No changes outside test/ directory Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Vasileios Zois <[email protected]> * upgrade nunit * Fix MigrateVector cluster tests for StackExchange.Redis 2.11.8 (#1595) * Initial plan * Fix MigrateVector tests to handle transient errors from StackExchange.Redis 2.11.8 Co-authored-by: vazois <[email protected]> * Fix MigrateVector tests: use Exception catch + specific MOVED message patterns for SE.Redis 2.11.8 Co-authored-by: vazois <[email protected]> --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: vazois <[email protected]> * revert diskann version --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Vasileios Zois <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Vasileios Zois <[email protected]> Co-authored-by: Tal Zaccai <[email protected]> * cleanup unused properties * add more comments at HLL validation * add test that bypasses crc check by faking it while corrupting the HLL payload --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Copilot <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Tal Zaccai <[email protected]>

* Bump @easyops-cn/docusaurus-search-local in /website (#1584) Bumps [@easyops-cn/docusaurus-search-local](https://github.com/easyops-cn/docusaurus-search-local/tree/HEAD/packages/docusaurus-search-local) from 0.52.3 to 0.55.1. - [Release notes](https://github.com/easyops-cn/docusaurus-search-local/releases) - [Commits](https://github.com/easyops-cn/docusaurus-search-local/commits/v0.55.1/packages/docusaurus-search-local) --- updated-dependencies: - dependency-name: "@easyops-cn/docusaurus-search-local" dependency-version: 0.55.1 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * add grouping wildcard (#1591) * Bump dotnet-sdk from 10.0.101 to 10.0.103 (#1585) Bumps [dotnet-sdk](https://github.com/dotnet/sdk) from 10.0.101 to 10.0.103. - [Release notes](https://github.com/dotnet/sdk/releases) - [Commits](dotnet/sdk@v10.0.101...v10.0.103) --- updated-dependencies: - dependency-name: dotnet-sdk dependency-version: 10.0.103 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Vasileios Zois <[email protected]> * Fix Stored Proc Replay (#1583) * fix failing ClusterReplicationStoredProc * revert logging * Update docker files to latest (#1600) * update dockerfiles * Fix Docker build issues: libaio path, tdnf, and workflow image names - Dockerfile: Add libaio.so.1t64 -> libaio.so.1 symlink for Ubuntu 24.04 t64 compat - Dockerfile.ubuntu: Copy libaio.so.1t64 and create libaio.so.1 symlink (consistent with CI workaround and Dockerfile) - Dockerfile.cbl-mariner: Revert dnf back to tdnf (Azure Linux 3.0 uses tdnf) - docker-linux.yml: Update GHCR image names to match new base OS (jammy -> noble, cbl-mariner2.0 -> azurelinux3.0) Co-authored-by: Copilot <[email protected]> * Fix Alpine Lua SIGSEGV: remove glibc-compiled KeraLua liblua54.so The KeraLua NuGet package bundles a glibc-compiled liblua54.so that gets published to /app/liblua54.so. On Alpine (musl libc), .NET loads this glibc binary first (app dir takes priority), causing SIGSEGV on any Lua EVAL command. Remove it in the build stage so .NET falls through to the musl-compiled system Lua library via the runtime symlink. Co-authored-by: Copilot <[email protected]> * Add libaio to chiseled image for native device support The chiseled (distroless) image was missing libaio.so.1, causing --device-type Native to fail with 'libaio.so.1: cannot open shared object file'. Add libaio1t64 to the libs-builder stage and copy both the library and compat symlink into the final chiseled image. Co-authored-by: Copilot <[email protected]> * Fix Dockerfile.ubuntu libaio arm64 arch mismatch Remove the hardcoded --platform=linux/amd64 builder stage that always copied x86_64 libaio into the runtime image regardless of target arch. Install libaio1t64 directly in the runtime stage via apt-get (alongside liblua5.4-0) so the correct architecture library is installed, matching the approach used in the default Dockerfile. Co-authored-by: Copilot <[email protected]> * Add Docker image validation test script Comprehensive Python script that verifies all 5 Linux Docker images: - Build all Dockerfiles - Basic server tests (PING, SET, GET) - Lua EVAL scripting - Default device persistence (all platforms incl. Alpine) - Native device persistence (glibc platforms) - Library resolution checks (libaio, liblua54, libnative_device) - Optional multi-platform buildx (amd64+arm64) Usage: python3 test/docker-tests/validate_docker_images.py [--skip-build] [--multiplatform] [--images ...] Co-authored-by: Copilot <[email protected]> * Remove dead libaio copy in chiseled prep-runtime stage The libaio files copied into prep-runtime's /usr/lib were never transferred to the final chiseled image (only /usr/share/dotnet/shared is copied from prep-runtime). The final image already gets libaio directly from libs-builder. Remove the unused copy/symlink steps. Co-authored-by: Copilot <[email protected]> * Rename Dockerfile.cbl-mariner to Dockerfile.azurelinux The runtime base image is now azurelinux3.0 and the CI publishes as -azurelinux3.0. Rename the Dockerfile to match and update the workflow matrix and test script references. Co-authored-by: Copilot <[email protected]> * Use native lua-libs package on Azure Linux instead of Ubuntu copy Azure Linux 3.0 ships lua-libs (providing /usr/lib/liblua-5.4.so) in the base runtime image. Use a symlink to this native library instead of copying a cross-distro binary from an Ubuntu builder stage. This avoids potential glibc ABI compatibility issues and removes an unnecessary build stage. Co-authored-by: Copilot <[email protected]> * Use synchronous SAVE instead of BGSAVE+sleep in Docker tests SAVE blocks until the checkpoint completes and returns, eliminating the race condition from the sleep-based BGSAVE approach. Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: Copilot <[email protected]> * Fix 7 Dependabot security vulnerabilities in website npm dependencies (#1601) * Initial plan * initial plan Co-authored-by: TalZaccai <[email protected]> * Fix 7 Dependabot security vulnerabilities in website npm dependencies - Add yarn resolutions for minimatch (3.1.5) and serialize-javascript (7.0.4) - Add npm overrides for minimatch (3.1.5) and serialize-javascript (7.0.4) - ajv updated to 6.14.0 (^6.x) and 8.18.0 (^8.x) via yarn re-resolution - Fixes: GHSA-2g4f-4pwh-qvx6 (ajv ReDoS), GHSA-3ppc-4f35-3m26, GHSA-7r86-cg39-jmmj, GHSA-23c5-xmqv-rm74 (minimatch ReDoS), GHSA-5c6j-r48x-rmvq (serialize-javascript RCE) Co-authored-by: TalZaccai <[email protected]> * Remove yarn v1 resolutions from website/package.json Per review feedback, the resolutions field is unnecessary since yarn v1 is not used. The npm overrides field handles dependency pinning. Co-authored-by: TalZaccai <[email protected]> --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: TalZaccai <[email protected]> Co-authored-by: Tal Zaccai <[email protected]> * Bump the nuget-deps group with 14 updates (#1592) * Bump the nuget-deps group with 14 updates Bumps diskann-garnet from 1.0.23 to 1.0.25 Bumps Microsoft.Extensions.Configuration.Binder from 10.0.2 to 10.0.3 Bumps Microsoft.Extensions.Configuration.Json from 10.0.2 to 10.0.3 Bumps Microsoft.Extensions.Logging from 10.0.2 to 10.0.3 Bumps Microsoft.Extensions.Logging.Configuration from 9.0.8 to 10.0.3 Bumps Microsoft.Extensions.Logging.Console from 9.0.8 to 10.0.3 Bumps Microsoft.IdentityModel.Protocols.OpenIdConnect from 8.6.1 to 8.16.0 Bumps Microsoft.IdentityModel.Validators from 8.6.1 to 8.16.0 Bumps Microsoft.NET.Test.Sdk from 18.0.1 to 18.3.0 Bumps NUnit from 4.1.0 to 4.5.0 Bumps NUnit3TestAdapter from 4.6.0 to 6.1.0 Bumps StackExchange.Redis from 2.9.25 to 2.11.8 Bumps System.IdentityModel.Tokens.Jwt from 8.6.1 to 8.16.0 Bumps System.Numerics.Tensors from 9.0.9 to 10.0.3 --- updated-dependencies: - dependency-name: diskann-garnet dependency-version: 1.0.25 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Configuration.Binder dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Configuration.Json dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Logging dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Logging.Configuration dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-major dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Logging.Console dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-major dependency-group: nuget-deps - dependency-name: Microsoft.IdentityModel.Protocols.OpenIdConnect dependency-version: 8.16.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: System.IdentityModel.Tokens.Jwt dependency-version: 8.16.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: Microsoft.IdentityModel.Validators dependency-version: 8.16.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: Microsoft.NET.Test.Sdk dependency-version: 18.3.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: NUnit dependency-version: 4.5.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: NUnit3TestAdapter dependency-version: 6.1.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: nuget-deps - dependency-name: StackExchange.Redis dependency-version: 2.11.8 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: System.Numerics.Tensors dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-major dependency-group: nuget-deps ... Signed-off-by: dependabot[bot] <[email protected]> * fix changes in API * revert unit test upgrade * Add ConfigureAwait(false) to all test await expressions to fix NUnit 4.5 deadlock (#1593) * Initial plan * Add .ConfigureAwait(false) to all await expressions in test files Add .ConfigureAwait(false) to every await expression across 38 test files under test/ to follow best practices for library/test code and avoid potential deadlocks in synchronization contexts. - Handles single-line and multi-line await expressions - Handles ternary await expressions (both branches) - Skips await foreach and await using (already correct) - Excludes Task.Yield() which returns YieldAwaitable (no ConfigureAwait support) - No changes outside test/ directory Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Vasileios Zois <[email protected]> * upgrade nunit * Fix MigrateVector cluster tests for StackExchange.Redis 2.11.8 (#1595) * Initial plan * Fix MigrateVector tests to handle transient errors from StackExchange.Redis 2.11.8 Co-authored-by: vazois <[email protected]> * Fix MigrateVector tests: use Exception catch + specific MOVED message patterns for SE.Redis 2.11.8 Co-authored-by: vazois <[email protected]> --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: vazois <[email protected]> * revert diskann version --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Vasileios Zois <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Vasileios Zois <[email protected]> Co-authored-by: Tal Zaccai <[email protected]> * fixing merge error * Harden RESTORE validation and block malformed HLL payload misuse in PF operations (#1599) * SkipChecksum validation testing and fix * validate HLL format with skipRDBRestoreChecksumValidation * add tests that cover different length encoding * add full sparse stream validation * skipRDBRestoreChecksumValidation for slot validation tests * remove double validation and fix buffer swap * reduce test overhead * fix redirection tests for RESTORE * eliminate SkipRDBRestoreChecksumValidation option * Bump the nuget-deps group with 14 updates (#1592) * Bump the nuget-deps group with 14 updates Bumps diskann-garnet from 1.0.23 to 1.0.25 Bumps Microsoft.Extensions.Configuration.Binder from 10.0.2 to 10.0.3 Bumps Microsoft.Extensions.Configuration.Json from 10.0.2 to 10.0.3 Bumps Microsoft.Extensions.Logging from 10.0.2 to 10.0.3 Bumps Microsoft.Extensions.Logging.Configuration from 9.0.8 to 10.0.3 Bumps Microsoft.Extensions.Logging.Console from 9.0.8 to 10.0.3 Bumps Microsoft.IdentityModel.Protocols.OpenIdConnect from 8.6.1 to 8.16.0 Bumps Microsoft.IdentityModel.Validators from 8.6.1 to 8.16.0 Bumps Microsoft.NET.Test.Sdk from 18.0.1 to 18.3.0 Bumps NUnit from 4.1.0 to 4.5.0 Bumps NUnit3TestAdapter from 4.6.0 to 6.1.0 Bumps StackExchange.Redis from 2.9.25 to 2.11.8 Bumps System.IdentityModel.Tokens.Jwt from 8.6.1 to 8.16.0 Bumps System.Numerics.Tensors from 9.0.9 to 10.0.3 --- updated-dependencies: - dependency-name: diskann-garnet dependency-version: 1.0.25 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Configuration.Binder dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Configuration.Json dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Logging dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Logging.Configuration dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-major dependency-group: nuget-deps - dependency-name: Microsoft.Extensions.Logging.Console dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-major dependency-group: nuget-deps - dependency-name: Microsoft.IdentityModel.Protocols.OpenIdConnect dependency-version: 8.16.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: System.IdentityModel.Tokens.Jwt dependency-version: 8.16.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: Microsoft.IdentityModel.Validators dependency-version: 8.16.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: Microsoft.NET.Test.Sdk dependency-version: 18.3.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: NUnit dependency-version: 4.5.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: NUnit3TestAdapter dependency-version: 6.1.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: nuget-deps - dependency-name: StackExchange.Redis dependency-version: 2.11.8 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: System.Numerics.Tensors dependency-version: 10.0.3 dependency-type: direct:production update-type: version-update:semver-major dependency-group: nuget-deps ... Signed-off-by: dependabot[bot] <[email protected]> * fix changes in API * revert unit test upgrade * Add ConfigureAwait(false) to all test await expressions to fix NUnit 4.5 deadlock (#1593) * Initial plan * Add .ConfigureAwait(false) to all await expressions in test files Add .ConfigureAwait(false) to every await expression across 38 test files under test/ to follow best practices for library/test code and avoid potential deadlocks in synchronization contexts. - Handles single-line and multi-line await expressions - Handles ternary await expressions (both branches) - Skips await foreach and await using (already correct) - Excludes Task.Yield() which returns YieldAwaitable (no ConfigureAwait support) - No changes outside test/ directory Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Vasileios Zois <[email protected]> * upgrade nunit * Fix MigrateVector cluster tests for StackExchange.Redis 2.11.8 (#1595) * Initial plan * Fix MigrateVector tests to handle transient errors from StackExchange.Redis 2.11.8 Co-authored-by: vazois <[email protected]> * Fix MigrateVector tests: use Exception catch + specific MOVED message patterns for SE.Redis 2.11.8 Co-authored-by: vazois <[email protected]> --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: vazois <[email protected]> * revert diskann version --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Vasileios Zois <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Vasileios Zois <[email protected]> Co-authored-by: Tal Zaccai <[email protected]> * cleanup unused properties * add more comments at HLL validation * add test that bypasses crc check by faking it while corrupting the HLL payload --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Copilot <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Tal Zaccai <[email protected]> * Pin dompurify to 3.3.1 to resolve Dependabot security alerts (#1604) * Initial plan * Initial plan for fixing dependabot security issues Co-authored-by: TalZaccai <[email protected]> * Fix dompurify vulnerability: upgrade to 3.3.1 via overrides and resolutions Co-authored-by: TalZaccai <[email protected]> * Remove resolutions from package.json per review feedback Co-authored-by: TalZaccai <[email protected]> --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: TalZaccai <[email protected]> Co-authored-by: Tal Zaccai <[email protected]> * Bump svgo from 3.3.2 to 3.3.3 in /website (#1608) Bumps [svgo](https://github.com/svg/svgo) from 3.3.2 to 3.3.3. - [Release notes](https://github.com/svg/svgo/releases) - [Commits](svg/svgo@v3.3.2...v3.3.3) --- updated-dependencies: - dependency-name: svgo dependency-version: 3.3.3 dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Avoid NUnit async test-context capture in Tsavorite tests (#1614) * add configureAwait(false) to async await calls * manual search configureAwait * fix formatting * fix formatting for Tsavorite * Harden Offset Validation for Bitmap Operations (#1610) * bound validation for max 512MB strings * add BITPOS validation * add BITCOUNT offset validation * addressing comments * fix slot verification test * addressing comments * making check in backend bitfield call Debug.Assert * fix formatting * fix Release build * merge error fix * Fix TTL rounding to match Redis behavior (#1602) * Fix TTL rounding to match Redis behavior (#1389) Redis rounds TTL seconds to the nearest integer (equivalent to (ms+500)/1000), while Garnet was truncating (flooring) via a direct (long) cast. This caused SET+EXPIRE 10+TTL to return 9 instead of 10 when called immediately. Fix: Use Math.Round(MidpointRounding.AwayFromZero) in both SecondsFromDiffUtcNowTicks and MillisecondsFromDiffUtcNowTicks in ConvertUtils.cs. Added TtlRoundingTest to verify string and object keys return the expected TTL value immediately after EXPIRE. Co-authored-by: Copilot <[email protected]> * updates * updates * increase test timing tolerance * update --------- Co-authored-by: Copilot <[email protected]> * ci: use orphan commits for deploy branches to reduce clone size (#1603) * ci: use orphan commits for deploy branches to reduce clone size Use orphan commits (single commit, no history) for gh-pages, allure_data_history, and benchmark branches to prevent large generated files from accumulating across hundreds of commits, which bloats the repository pack file and slows git clone. - deploy-website: add force_orphan: true to peaceiris/actions-gh-pages - nightly: rewrite allure_data_history push as orphan commit - new workflow: daily squash of continuousbenchmark branches Co-authored-by: Copilot <[email protected]> * updates --------- Co-authored-by: badrishc <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Tal Zaccai <[email protected]> * Bump dompurify from 3.3.1 to 3.3.3 in /website (#1617) Bumps [dompurify](https://github.com/cure53/DOMPurify) from 3.3.1 to 3.3.3. - [Release notes](https://github.com/cure53/DOMPurify/releases) - [Commits](cure53/DOMPurify@3.3.1...3.3.3) --- updated-dependencies: - dependency-name: dompurify dependency-version: 3.3.3 dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * update lua block list (#1621) * add more functions to block list * fix test --------- Co-authored-by: badrishc <[email protected]> * format * merge error fix * remove merge error --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Vasileios Zois <[email protected]> Co-authored-by: Badrish Chandramouli <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Vasileios Zois <[email protected]> Co-authored-by: badrishc <[email protected]>

Copilot AI review requested due to automatic review settings March 4, 2026 00:07

Copilot started reviewing on behalf of vazois March 4, 2026 00:08 View session

vazois added 4 commits March 3, 2026 16:08

SkipChecksum validation testing and fix

b8b8c66

validate HLL format with skipRDBRestoreChecksumValidation

0545a27

add tests that cover different length encoding

220008f

add full sparse stream validation

0c6a10d

vazois force-pushed the vazois/hll-store-corrupt-fix branch from 09cb009 to 0c6a10d Compare March 4, 2026 00:11

Copilot AI reviewed Mar 4, 2026

View reviewed changes

Comment thread libs/server/Storage/Session/MainStore/HyperLogLogOps.cs Outdated

Comment thread libs/server/Storage/Session/MainStore/HyperLogLogOps.cs Outdated

Comment thread test/Garnet.test/HyperLogLogTests.cs Outdated

vazois and others added 6 commits March 3, 2026 16:52

skipRDBRestoreChecksumValidation for slot validation tests

0d9760d

Merge branch 'main' into vazois/hll-store-corrupt-fix

59dde74

remove double validation and fix buffer swap

e96e2a1

reduce test overhead

ebbfbe1

Merge branch 'main' into vazois/hll-store-corrupt-fix

1caad4c

fix redirection tests for RESTORE

e1dd3e6

vazois force-pushed the vazois/hll-store-corrupt-fix branch from d9b3f60 to 21cc936 Compare March 4, 2026 18:49

vazois and others added 2 commits March 4, 2026 11:03

eliminate SkipRDBRestoreChecksumValidation option

44322c2

vazois force-pushed the vazois/hll-store-corrupt-fix branch from 21cc936 to e4bf81c Compare March 4, 2026 19:03

vazois and others added 2 commits March 4, 2026 11:04

Merge branch 'main' into vazois/hll-store-corrupt-fix

3342f21

cleanup unused properties

80fce46

badrishc approved these changes Mar 4, 2026

View reviewed changes

add more comments at HLL validation

5d9901d

badrishc approved these changes Mar 4, 2026

View reviewed changes

add test that bypasses crc check by faking it while corrupting the HL…

732c33e

…L payload

badrishc approved these changes Mar 4, 2026

View reviewed changes

vazois merged commit b5b1030 into main Mar 4, 2026
68 of 71 checks passed

vazois deleted the vazois/hll-store-corrupt-fix branch March 4, 2026 22:09

github-actions Bot locked and limited conversation to collaborators May 4, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Harden RESTORE validation and block malformed HLL payload misuse in PF operations#1599

Harden RESTORE validation and block malformed HLL payload misuse in PF operations#1599
vazois merged 16 commits into
mainfrom
vazois/hll-store-corrupt-fix

vazois commented Mar 4, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

vazois commented Mar 4, 2026

Summary

Changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants