Skip to content

feat(sim): hold service, worker and lambda slots until their downstream calls answer - #109

Merged
xevrion merged 1 commit into
mainfrom
feat/blocking-calls
Nov 7, 2026
Merged

xevrion merged 1 commit into
mainfrom
feat/blocking-calls

Conversation

@xevrion

@xevrion xevrion commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator

A service used to free its slot as soon as its own work finished, before the calls it made came back. So a slow database never backed up the API in front of it, which isn't how thread-per-request servers behave.

  • Service, worker and lambda now keep their slot until their downstream calls answer, with timeouts and retries per attempt. A queue, cache, load balancer or store still hands the request on, so a queue still shields its callers from a slow consumer
  • Slowing a database 6x behind an 8-slot API now fills the API (about 7 of 8 slots waiting), sheds and times out, where before it showed 0% errors
  • Cycles fail fast instead of deadlocking, crashed or removed nodes release the threads parked on them, and a direct call into a worker or transcoder is served from its own line
  • The inspector shows slots held while waiting separately from busy work, and the MCP summary ranks blocked callers properly
  • Presets are retuned, mostly by giving callers more slots, so every example still teaches what it says. Measured across seeds; hold-the-line now runs at 200 rps and the autoscaling ceiling reads ~590

@vercel

vercel Bot commented Nov 7, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
breakscale Ready Ready Preview Nov 7, 2026 4:36am UTC

Request Review

@xevrion
xevrion merged commit 0db6703 into main Nov 7, 2026
9 checks passed
@xevrion
xevrion deleted the feat/blocking-calls branch November 7, 2026 04:38

This branch was successfully deployed

1 active deployment
Preview – breakscale — d7bd4ef9 Deployed Nov 7, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant