[RDMA] Keep EQDS reported and spendable credits consistent - #1108

Description

Fixes #1092.

EQDS reported speculative credit after speculation had stopped, even though the sender could no longer spend it. It also accepted a chunk larger than the available pull credit without charging the remainder. The pull-target computation subtracted the same unusable credit and tried to compensate by draining it over subsequent calls.

• Report only spendable credit and reject insufficient-credit spends without changing counters.
• Charge the entire accepted chunk, using pull credit first and then any permitted speculative credit.
• Compute pull demand from that same spendable-credit value; remove the stale-credit draining heuristic.
• Keep speculation on ordinary ACKs, which do not supply pull credit. Stop it when a credit packet arrives.

Add credit-accounting and native transport regression tests, with running instructions. The default CCA configuration is unchanged; EQDS remains opt-in.

Type of Change

• Bug fix
• New feature
• Documentation update

How Has This Been Tested?

Compared upstream 54e216ee against the fix on Linux x86-64, GCC 13.3, CUDA 12.8 headers, and an active mlx5 InfiniBand port on an NVIDIA B200 node.

Credit and pull-target checks

eqds_credit_test executes 30,784 boundary/seeded credit-spend cases, 512 grant/drain cycles including pull-number wraparound and duplicate grants, and production compute_pull_target() transitions before/after speculation stops. It checks exact byte conservation, admission decisions, unchanged counters on rejection, and stable demand under repeated computation.

• Baseline: 72,537 failed assertions.
• Fixed: all checks pass.
• ASan + UBSan, including leak checking, passes for the test and production eqds.cc/param.cc.

Native data-integrity checks

eqds_transfer_test forks before transport/CUDA initialization, connects two local peers through the actual UCCL transport, and checks every received byte at 1 B, 1 KiB, 64 KiB, 1 MiB, and 32 MiB. Each peer has a timeout.

Enabled kReceiverCCA = RECEIVER_CCA_EQDS in both test trees, retaining default Timely sender CCA. Used UCCL_IB_HCA=mlx5_0:1, one engine, default 32-QP entropy, and UC mode. Both baseline and fix pass this sequence; this is a compatibility check rather than a reproduction of the issue's reported AllReduce stall. Repeated the final fixed binary three times: all 15 transfers passed byte-for-byte.

An exploratory EQDS-only configuration (sender CCA disabled, 4 QPs) failed at 32 MiB in both baseline and fix with ibv_post_send failure. That configuration remains unresolved; this PR does not claim to fix every EQDS progress failure.

Builds and formatting

Built all transport objects, both new tests, and libnccl-net-uccl.so with default CCA and with EQDS enabled. Used the repository's pinned NCCL headers (2ea4ee94) and the existing native Makefiles. This host lacks unused gflags/gtest libraries, so used LIBS="-lz -lelf -libverbs -L/usr/local/cuda-12.8/lib64 -lcudart -lpthread" and LIBS_SHARED="-lz -lelf -libverbs".

Changed C++ files pass clang-format 14.0.6; git diff --check passes. ROCm, RoCE, GPU-buffer data transfers, cross-node collectives, and throughput comparisons were not tested. No bandwidth or latency improvement is claimed.

• Unit tests
• Integration tests
• Manual testing

Checklist

• I have run format.sh to follow the style guidelines. (Ran clang-format 14 directly on changed files.)
• I have run build.sh to verify compilation. (Used native Makefile builds above.)
• I have removed redundant variables and comments.
• I have updated the documentation.
• I have added tests.