feat(sandbox): e2b code execution primitive#30898
Conversation
Add a provider-agnostic code execution primitive that runs model-generated code in an isolated sandbox and returns the output, with e2b as the first backend over raw httpx (no SDK dependency). Public API: litellm.acode_interpreter_tool (ephemeral create -> run -> delete) plus the low-level lifecycle litellm.acreate_sandbox / arun_code / adelete_sandbox. Each is @client-decorated so operations are logged like litellm.asearch. Backends implement BaseSandboxConfig; resolved via ProviderConfigManager.get_provider_sandbox_config.
|
|
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Greptile SummaryThis PR adds a provider-agnostic code execution primitive backed by e2b, following the same structural pattern as
Confidence Score: 5/5Safe to merge; implementation is well-structured with a single minor edge-case connection leak in the output-cap error path. All new code is isolated to new modules and follows established litellm patterns (AsyncHTTPHandler reuse, provider config manager, @client logging). The ephemeral teardown guarantee is correctly implemented via try/finally. The one notable finding is that _read_capped_lines does not close the streaming httpx response when the output-size ValueError is raised mid-iteration, leaving the connection open until timeout — but this only fires in the >10 MB output edge case and does not affect correctness of results. litellm/llms/e2b/sandbox/transformation.py — the streaming response should be explicitly closed in _read_capped_lines's finally block.
|
| Filename | Overview |
|---|---|
| litellm/llms/e2b/sandbox/transformation.py | New e2b sandbox provider implementation; streaming response not closed on early exit from _read_capped_lines |
| litellm/sandbox/main.py | New public entrypoints for sandbox operations; correct use of try/finally for cleanup, proper internal-kwarg stripping |
| litellm/llms/base_llm/sandbox/transformation.py | New base classes ContainerHandle, CodeExecutionResult, and BaseSandboxConfig; clean, no issues |
| litellm/utils.py | Adds get_provider_sandbox_config to ProviderConfigManager, mirroring the existing search provider pattern |
| tests/test_litellm/sandbox/test_e2b_sandbox.py | Mock-only unit tests covering request shapes, NDJSON parsing, teardown guarantee, and error edge cases |
| tests/integration/sandbox/test_e2b_sandbox.py | Real-network integration tests gated on E2B_API_KEY; correctly skipped when key is absent |
Reviews (3): Last reviewed commit: "chore(sandbox): re-trigger automated rev..." | Re-trigger Greptile
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 781d4cdd42
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 1 · PR risk: 0/10 |
- document e2b provider in provider_endpoints_support.json and add a sandbox endpoint definition - regenerate dashboard CallTypes after the sandbox call-type additions - guard explicit timeout=0 instead of coercing it to the default - require a ContainerHandle access token before running code; reject bare-id runs - return False on a 404 delete now that the shared http handler raises for status - skip non-JSON NDJSON lines and cap streamed output to bound memory - move the real-network integration tests out of tests/test_litellm into tests/integration/sandbox
- modernize annotations in the new sandbox modules to PEP 585/604 (list/dict, X | None) and drop the now-unnecessary quoted forward refs so the strict-rule budget delta for UP006/UP037/UP045 returns to zero - add __all__ to litellm/sandbox/main.py so 'import *' only re-exports the four public entrypoints instead of leaking module-level imports
utils.py uses 'from __future__ import annotations', so the quoted forward ref tripped UP037; the unquoted union is lazily evaluated and keeps the strict-rule delta at zero
|
@greptileai review |
53593f6
into
litellm_internal_staging
* docs(sandbox): document e2b code execution primitive Documents the SDK surface added in BerriAI/litellm#30898: acode_interpreter_tool for ephemeral runs and the acreate_sandbox / arun_code / adelete_sandbox lifecycle. Covers the e2b backend setup, parameters, the CodeExecutionResult passthrough shape, and what is intentionally out of scope for this phase (proxy endpoints, OpenAI container interceptor, gpt-5 routing, container reuse, file IO, OpenSandbox, Daytona). Adds a sidebar entry under LiteLLM Python SDK > SDK Functions. * docs(sandbox): add Responses API code interpreter interceptor section Documents BerriAI/litellm#30905: how a client can call OpenAI's code interpreter through /v1/responses and have the code run in the configured sandbox (e2b) instead of OpenAI's container, with no client-side change. Covers the sandbox_tools registry and code_interpreter_interception callback, the curl shape, response-shape parity with native code_interpreter_call (id/type/status/code/container_id/outputs as an OpenAI-shaped logs array), streaming via the synthetic SSE wrapper, forced tool_choice rewrite, sandbox lifecycle and per-request isolation on a server-minted token, the stripped control fields that prevent client forgery, hot-reload behaviour, and the v0 limitation that file upload and download are not yet supported. Also notes the new api_base passthrough on the SDK entrypoints. * docs(sandbox): cross-link the interceptor from the Code Interpreter guide A proxy user looking for code interpreter lands on guides/code_interpreter, not the SDK sandbox page, so the interceptor was effectively undiscoverable from the proxy side. Adds a tip in the Code Interpreter guide pointing at the new interceptor section, and a second sidebar entry under Tool Calling labelled 'Code Interpreter Sandbox Interception' next to the existing Web Search Interception entry, mirroring how websearch_interception is exposed. * docs(sandbox): turn the proxy section into a step-by-step tutorial The interceptor section had the config and curl but no narrative walking a proxy user from a clean checkout to a working call. Restructures into five numbered steps (get the e2b key, write config.yaml with a model_list plus sandbox_tools registry, start the proxy, call /v1/responses via curl or the OpenAI SDK, verify the code ran in e2b via the cntr_-wrapped container_id and the e2b dashboard sandbox count). Calls out enabled_providers as the safety gate so flipping the callback on does not silently change behaviour for other providers. * docs(sandbox): tighten the tutorial section * docs(sandbox): collapse the proxy section notes into prose --------- Co-authored-by: Krrish Dholakia <[email protected]>
* feat(sandbox): add e2b code execution primitive Add a provider-agnostic code execution primitive that runs model-generated code in an isolated sandbox and returns the output, with e2b as the first backend over raw httpx (no SDK dependency). Public API: litellm.acode_interpreter_tool (ephemeral create -> run -> delete) plus the low-level lifecycle litellm.acreate_sandbox / arun_code / adelete_sandbox. Each is @client-decorated so operations are logged like litellm.asearch. Backends implement BaseSandboxConfig; resolved via ProviderConfigManager.get_provider_sandbox_config. * fix(sandbox): address review feedback and CI gates - document e2b provider in provider_endpoints_support.json and add a sandbox endpoint definition - regenerate dashboard CallTypes after the sandbox call-type additions - guard explicit timeout=0 instead of coercing it to the default - require a ContainerHandle access token before running code; reject bare-id runs - return False on a 404 delete now that the shared http handler raises for status - skip non-JSON NDJSON lines and cap streamed output to bound memory - move the real-network integration tests out of tests/test_litellm into tests/integration/sandbox * fix(sandbox): satisfy strict ruff gate and scope star-exports - modernize annotations in the new sandbox modules to PEP 585/604 (list/dict, X | None) and drop the now-unnecessary quoted forward refs so the strict-rule budget delta for UP006/UP037/UP045 returns to zero - add __all__ to litellm/sandbox/main.py so 'import *' only re-exports the four public entrypoints instead of leaking module-level imports * fix(sandbox): drop quotes on sandbox config return annotation utils.py uses 'from __future__ import annotations', so the quoted forward ref tripped UP037; the unquoted union is lazily evaluated and keeps the strict-rule delta at zero * chore(sandbox): re-trigger automated review after addressing feedback
Relevant issues
Part of #30891 (phase 1: the code execution primitive)
Linear ticket
n/a
Pre-Submission checklist
make test-unitType
🆕 New Feature
Changes
This is the first slice of the code interpreter work in #30891: a provider-agnostic primitive that runs model-generated code in an isolated sandbox and returns whatever the sandbox produced. e2b is the first backend, talked to directly over httpx with no e2b SDK dependency added to the core.
The public API is a high-level ephemeral call plus the low-level lifecycle:
codeis just an executable string; there is no language knob and no invented result schema, theCodeExecutionResultis a passthrough of the sandbox's own output (stdout, stderr, results such as base64 charts, error name/value/traceback, execution count).templateis an optional passthrough that defaults to e2b'scode-interpreter-v1.Backends implement
BaseSandboxConfig(acreate_sandbox/arun_code/adelete_sandbox) and are resolved throughProviderConfigManager.get_provider_sandbox_config, mirroring how search providers resolve throughget_provider_search_config. Each public entrypoint is@client-decorated, so every operation logs the same waylitellm.asearchdoes. The low-level names are sandbox-scoped on purpose so they do not collide with the existing OpenAI Containers API (litellm.create_container), which stays untouched.The e2b wire calls were derived from e2b's open-source SDK: create is
POST api.e2b.app/sandboxeswithtemplateID, run is a streamed NDJSONPOSTto the per-sandbox code-interpreter host on port 49999 with the envd access token, and delete isDELETE api.e2b.app/sandboxes/{id}.Out of scope here and tracked on the ticket for later phases: the proxy
/v1/containersendpoints, the interceptor that swaps OpenAI's container for the configured sandbox, gpt-5code_interpreterrouting, container reuse, file read/write, and the OpenSandbox and Daytona backends.Tests
tests/test_litellm/sandbox/test_e2b_sandbox.py. Unit tests inject a fake async HTTP client (dependency injection, not monkeypatching) and assert the request shapes (templateIDnottemplate, the 49999 execute host, theX-Access-Tokenheader,DELETE /sandboxes/{id}), the NDJSON-to-result mapping, and that the ephemeral path still deletes the sandbox when the run raises. An integration test hits the real e2b API and is skipped unlessE2B_API_KEYis set.Proof of fix
Run against the real e2b API (this is the public surface the PR ships; there is no proxy endpoint in this phase):
Output:
The full suite including the gated integration tests, with a real key set: