[BUG] Rate-limited providers silently lose analyzer coverage: max_concurrency=10 hardcoded, 429 batches dropped without retry, partial LLM pass not flagged #303

Summary

On a rate-limited provider (typical free tier), arun_batches bursts up to a hardcoded max_concurrency=10 and drops any 429'd batch ("logged and omitted from the result") with no retry/backoff. There is no way to lower the concurrency or make the client respect the provider's RPM/TPM. The result is a scan that silently loses analyzer coverage while still emitting a full risk_assessment — including losing the security-critical semantic_security_discovery batch, so a genuinely malicious skill can read as clean.

This is distinct from #290 (provider unavailable): here the provider is available and answering, but throttling, and the partial result is not surfaced as such.

Evidence

Provider openai pointed at Gemini's OpenAI-compatible endpoint, free tier (5 RPM):

WARNING [skillspector.nodes.analyzers.semantic_security_discovery] semantic_security_discovery failed:
Error code: 429 - ... 'quotaId': 'GenerateRequestsPerMinutePerProjectPerModel-FreeTier', 'quotaValue': '5' ...

metadata.llm_calls_attempted: 4, llm_calls_succeeded: 3, yet risk_assessment.severity is emitted normally and issues: 0. On a known-malicious test skill (natural-language exfiltration of ~/.config/gh/hosts.yml), the dropped batch was exactly the analyzer that catches it — the scan reported LOW/clean.

Same pattern on Groq free tier: the ~33k-token semantic_security_discovery prompt exceeds the free TPM cap → 413 → dropped → clean-looking report.

Why the current design makes this invisible

1. max_concurrency=10 is a fixed default in arun_batches; nothing threads a lower value or reads an env/flag. Ten parallel requests against a 5 RPM budget guarantee 429s.
2. 429s are caught per-batch and omitted; there is no retry honoring Retry-After.
3. llm_calls_succeeded < llm_calls_attempted is recorded in metadata but the risk assessment does not treat a partial LLM pass as degraded, so downstream consumers see a normal verdict.

Requests (any subset helps)

1. Configurable concurrency — an env var / CLI flag (e.g. SKILLSPECTOR_MAX_LLM_CONCURRENCY) so users on rate-limited tiers can serialize.
2. Retry with backoff on 429/RESOURCE_EXHAUSTED, honoring Retry-After, before giving up on a batch.
3. Surface incompleteness in the verdict: when any batch is dropped, mark the report degraded / refuse to assert a clean risk_assessment — same failure mode Deep scan can report SAFE when provider is unavailable #290 guards against, one step earlier. A metadata flag like llm_batches_dropped: N would let callers gate on it.

Happy to send a PR for (1) and (3) if the direction is agreeable.

Environment

SkillSpector (current main), SKILLSPECTOR_PROVIDER=openai against generativelanguage.googleapis.com/v1beta/openai/ and api.groq.com/openai/v1, free tiers.