What problem are you trying to solve?

The /scan-secrets endpoint rebuilds the SDS Scanner from scratch on every request. This involves deserializing rules from JSON, compiling regex patterns, building automata, and configuring validators - all expensive operations. In contrast, the /analyze endpoint already caches compiled rules via RuleCache with DashMap.

In the IDE use-case, the same set of rules is sent with every file scan request (rules come from the Datadog API and rarely change), meaning the scanner is unnecessarily rebuilt hundreds of times with identical rules.

What is your solution?

Introduced a SecretScannerCache - a single-entry, RwLock-based cache that stores the most recently built Scanner along with its parsed SecretRules.

• Cache hit (fast path): acquires a read lock only, returns Arc-cloned scanner + rules. Multiple concurrent scans proceed without blocking each other.
• Cache miss (slow path): deserializes rules and builds the scanner outside any lock, then briefly acquires a write lock to store the result.
• Cache key: u64 hash of the serialized JSON rules. Compared on every request to detect rule changes.

On cache hit, three expensive operations are skipped entirely:

1. Rule deserialization (serde_json::Value -> SecretRule)
2. SDS rule config conversion (convert_to_sds_ruleconfig with regex compilation)
3. Scanner building (Scanner::builder().build() with automaton construction)

Alternatives considered

• DashMap multi-entry cache (like RuleCache for /analyze): considered but unnecessary since the IDE client sends the same rule set every time. A single-entry cache is simpler and has the same hit rate.
• Opt-in via CLI flag (like --use-rules-cache for /analyze): considered but there's no downside to always caching - memory is minimal (one entry) and the cache self-invalidates on rule changes. So it's always enabled.
• Caching individual rules instead of the whole scanner: not feasible because the SDS Scanner is built atomically from all rules at once - you can't incrementally add/remove rules from a built scanner.

What the reviewer should know

• dd_sds::Scanner is confirmed Send + Sync (the dd-sds benchmarks use Arc<Scanner> across 32 threads). Both scan() and validate_matches() take &self.
• use_debug is deliberately excluded from the cache key - it only controls eprintln logging in convert_to_sds_ruleconfig / try_to_secondary_validator, not the scanner's behavior or output.
• Scanner is re-exported from the secrets crate (pub use dd_sds::Scanner) so the bins crate can reference the type without adding a direct dd_sds dependency.
• process_secret_scan_request accepts Option<&SecretScannerCache> so the non-cache path is preserved for testing.