perf(appsec): make rate limiter synchronous to remove idle goroutine cost#4956
Conversation
…icker goroutine) Replaces the per-instance background goroutine and 500µs time.Ticker with a synchronous golang.org/x/time/rate.Limiter that lazily refills via an injected clock. Eliminates idle-pod CPU cost and the apisec proxy-sampler goroutine leak while preserving the public API (TokenTicker, constructors, Start/Stop no-ops, Allow) and behavioral test assertions. Also fixes the broken BenchmarkLimiter.
…ly and test via synctest Removes the TokenTicker wrapper struct, its Start()/Stop() no-op methods, and the injected now func() time.Time clock. The NewTokenTicker/NewTokenTickerWithInterval constructors now build a golang.org/x/time/rate.Limiter and return it as the limiter.Limiter interface (which *rate.Limiter already satisfies via Allow()). Drops all Start()/Stop() calls in the WAF listener and apisec proxy sampler; the WAF feature field is now limiter.Limiter. Migrates limiter unit tests from an injected fake clock to the standard testing/synctest package (deterministic virtual time); concurrency and benchmark tests remain on real time. No behavior change to rate limiting; public limiter behavior and all assertions preserved. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <[email protected]>
Resolves the golangci-lint 'modernize' finding by replacing the manual if-clamp with the min builtin. No behavior change.
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 24dbd17 | Docs | Datadog PR Page | Give us feedback! |
BenchmarksBenchmark execution time: 2026-06-25 12:59:35 Comparing candidate commit 24dbd17 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 297 metrics, 2 unstable metrics, 1 flaky benchmarks without significant changes.
|
…cost (#4956) ### What does this PR do? Replaces the AppSec rate limiter's background-goroutine token bucket with a synchronous implementation backed by `golang.org/x/time/rate.Limiter`. - Removes the per-instance background goroutine and hard-coded 500µs `time.Ticker` that woke ~2000×/sec on **every** pod, even idle ones. - `Allow()` now refills the bucket lazily on call via `rate.Limiter`, so idle pods incur **zero** limiter CPU cost. - Deletes the `TokenTicker` struct and its `Start()`/`Stop()` lifecycle. The `NewTokenTicker` / `NewTokenTickerWithInterval` constructors now return the `limiter.Limiter` interface backed directly by `*rate.Limiter` (which already satisfies `Allow() bool`). - Drops the now-unused `Start()`/`Stop()` calls in the WAF listener and the apisec proxy sampler. As a side effect, the proxy sampler's previously-leaked goroutine (the `Sampler` interface had no `Stop`) is eliminated. - Migrates the limiter unit tests from an injected fake clock to the standard `testing/synctest` package (deterministic virtual time); concurrency and benchmark tests remain on real time. Also fixes `BenchmarkLimiter`, which no longer built on modern Go (`B.Loop called with timer stopped`). No change to rate-limit values, defaults, env vars, or the `Limiter` interface contract; behavior is preserved. ### Motivation The limiter's background goroutine consumed roughly 0.3% of a CPU core per instance **continuously** — even on idle pods with no traffic — which adds up meaningfully across the fleet. `golang.org/x/time/rate` is already a direct dependency and is fully synchronous (no goroutine, no ticker), so it gives zero idle cost while preserving the existing token-bucket behavior. Per-call cost (~50ns uncontended) is irrelevant given `Allow()` is invoked at most ~100/s. ### Performance notes Rough local microbenchmarks (single machine, quick comparison — directional only, not a rigorous A/B) informed the choice of `rate.Limiter`: - **Idle cost (what we actually care about):** the old background ticker burned on the order of ~0.3% of a CPU core *per limiter instance, continuously*, even with zero traffic (it woke ~2000×/s regardless of load). The synchronous `rate.Limiter` does no background work, so idle cost drops to ~zero. - **Per-call `Allow()`:** the old lock-free atomic path was a few ns uncontended; `rate.Limiter` is in the tens of ns uncontended, and the two are roughly comparable under contention. Both are allocation-free. At our call frequency (`Allow()` fires at most ~100/s) the difference is noise. - **Why `rate.Limiter` vs a bespoke lazy bucket:** it's already a direct dependency, fully synchronous, and well-tested upstream — not worth hand-rolling and maintaining an equivalent for a sub-microsecond, low-frequency path. Net trade: a negligible per-call overhead in exchange for eliminating a continuous, fleet-wide idle CPU cost. ### Reviewer's Checklist - [x] Changed code has unit tests for its functionality at or near 100% coverage. - [ ] [System-Tests](https://github.com/DataDog/system-tests/) covering this feature have been added and enabled with the va.b.c-dev version tag. - [x] There is a benchmark for any new code, or changes to existing code. - [ ] If this interacts with the agent in a new way, a system test has been added. - [x] New code is free of linting errors. You can check this by running `make lint` locally. - [x] New code doesn't break existing tests. You can check this by running `make test` locally. - [ ] Add an appropriate team label so this PR gets put in the right place for the release notes. - [x] All generated files are up to date. You can check this by running `make generate` locally. - [x] Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild. Make sure all nested modules are up to date by running `make fix-modules` locally. Co-authored-by: eliott.bouhana <[email protected]>
What does this PR do?
Replaces the AppSec rate limiter's background-goroutine token bucket with a synchronous implementation backed by
golang.org/x/time/rate.Limiter.time.Tickerthat woke ~2000×/sec on every pod, even idle ones.Allow()now refills the bucket lazily on call viarate.Limiter, so idle pods incur zero limiter CPU cost.TokenTickerstruct and itsStart()/Stop()lifecycle. TheNewTokenTicker/NewTokenTickerWithIntervalconstructors now return thelimiter.Limiterinterface backed directly by*rate.Limiter(which already satisfiesAllow() bool).Start()/Stop()calls in the WAF listener and the apisec proxy sampler. As a side effect, the proxy sampler's previously-leaked goroutine (theSamplerinterface had noStop) is eliminated.testing/synctestpackage (deterministic virtual time); concurrency and benchmark tests remain on real time. Also fixesBenchmarkLimiter, which no longer built on modern Go (B.Loop called with timer stopped).No change to rate-limit values, defaults, env vars, or the
Limiterinterface contract; behavior is preserved.Motivation
The limiter's background goroutine consumed roughly 0.3% of a CPU core per instance continuously — even on idle pods with no traffic — which adds up meaningfully across the fleet.
golang.org/x/time/rateis already a direct dependency and is fully synchronous (no goroutine, no ticker), so it gives zero idle cost while preserving the existing token-bucket behavior. Per-call cost (~50ns uncontended) is irrelevant givenAllow()is invoked at most ~100/s.Performance notes
Rough local microbenchmarks (single machine, quick comparison — directional only, not a rigorous A/B) informed the choice of
rate.Limiter:rate.Limiterdoes no background work, so idle cost drops to ~zero.Allow(): the old lock-free atomic path was a few ns uncontended;rate.Limiteris in the tens of ns uncontended, and the two are roughly comparable under contention. Both are allocation-free. At our call frequency (Allow()fires at most ~100/s) the difference is noise.rate.Limitervs a bespoke lazy bucket: it's already a direct dependency, fully synchronous, and well-tested upstream — not worth hand-rolling and maintaining an equivalent for a sub-microsecond, low-frequency path.Net trade: a negligible per-call overhead in exchange for eliminating a continuous, fleet-wide idle CPU cost.
Reviewer's Checklist
make lintlocally.make testlocally.make generatelocally.make fix-moduleslocally.