This PR extends Garnet’s Vector Set + DiskANN integration to support background quantization/backfill workflows, updates the FFI to reflect DiskANN’s new insert/search signatures, and cleans up/extends quantizer + vector-format handling (including backward-compatible aliases). It also adds a new server option to control quantization task parallelism, updates docs, bumps the diskann-garnet package, and expands test coverage across standalone + cluster suites.

Changes:

• Add VectorManager background quantization task pipeline (build_quant_table then parallel backfill_quant_vectors) and queue quantization based on DiskANN insert/recreate signals.
• Update vector quantizer/format enums + parsing to support XNOQUANT_{U8,I8}, XBIN_{U8,I8}, and XU8/XI8 (with aliases for XPREQ8 and XB8).
• Update DiskANN interop signatures (insert/search) and add extensive tests/docs/config wiring.

Copilot's findings

• Files reviewed: 26/26 changed files
• Comments generated: 8

Comments suppressed due to low confidence (1)

libs/server/Resp/Vector/RespServerSessionVectors.cs:444

• When REDUCE is used with extension quantizers, this branch currently reports the generic "asked quantization mismatch" error because customErrMsg is left empty. This makes the actual validation failure hard to understand.

                // We need to reject these HERE because validation during create_index is very awkward
                GarnetStatus res;
                VectorManagerResult result;
                ReadOnlySpan<byte> customErrMsg;
                if (quantType is VectorQuantType.XBin_U8 or VectorQuantType.XBin_I8 or VectorQuantType.XNoQuant_U8 or VectorQuantType.XNoQuant_I8 && reduceDim != 0)
                {
                    result = VectorManagerResult.BadParams;
                    res = GarnetStatus.OK;
                    customErrMsg = default;
                }

• Files reviewed: 26/26 changed files
• Comments generated: 11

• Files reviewed: 26/26 changed files
• Comments generated: 7

Comments suppressed due to low confidence (1)

libs/server/Resp/Vector/RespServerSessionVectors.cs:444

• The extension-quantizer REDUCE rejection has a precedence bug: as written, reduceDim != 0 only applies to the last pattern, so XBin_* and XNoQuant_U8 are rejected even when REDUCE is not specified. This also sets customErrMsg to default, which later surfaces as a misleading quantization-mismatch error. Gate on reduceDim != 0 first and return a specific error message.

                if (quantType is VectorQuantType.XBin_U8 or VectorQuantType.XBin_I8 or VectorQuantType.XNoQuant_U8 or VectorQuantType.XNoQuant_I8 && reduceDim != 0)
                {
                    result = VectorManagerResult.BadParams;
                    res = GarnetStatus.OK;
                    customErrMsg = default;
                }

• Files reviewed: 26/26 changed files
• Comments generated: 7

• Files reviewed: 26/26 changed files
• Comments generated: 5