Skip to content

feat(sandbox): e2b code execution primitive#30898

Merged
krrish-berri-2 merged 5 commits into
litellm_internal_stagingfrom
litellm_e2b_code_execution
Jun 20, 2026
Merged

feat(sandbox): e2b code execution primitive#30898
krrish-berri-2 merged 5 commits into
litellm_internal_stagingfrom
litellm_e2b_code_execution

Conversation

@krrish-berri-2

Copy link
Copy Markdown
Contributor

Relevant issues

Part of #30891 (phase 1: the code execution primitive)

Linear ticket

n/a

Pre-Submission checklist

  • I have added meaningful tests
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible; it only solves 1 specific problem

Type

🆕 New Feature

Changes

This is the first slice of the code interpreter work in #30891: a provider-agnostic primitive that runs model-generated code in an isolated sandbox and returns whatever the sandbox produced. e2b is the first backend, talked to directly over httpx with no e2b SDK dependency added to the core.

The public API is a high-level ephemeral call plus the low-level lifecycle:

# ephemeral: create -> run -> delete (delete runs even if the code raises)
out = await litellm.acode_interpreter_tool(provider="e2b", code="print(sum(range(10)))")

# low-level: you own the container
c   = await litellm.acreate_sandbox(provider="e2b")
out = await litellm.arun_code(provider="e2b", container=c, code="print(6*7)")
await litellm.adelete_sandbox(provider="e2b", container=c)

code is just an executable string; there is no language knob and no invented result schema, the CodeExecutionResult is a passthrough of the sandbox's own output (stdout, stderr, results such as base64 charts, error name/value/traceback, execution count). template is an optional passthrough that defaults to e2b's code-interpreter-v1.

Backends implement BaseSandboxConfig (acreate_sandbox / arun_code / adelete_sandbox) and are resolved through ProviderConfigManager.get_provider_sandbox_config, mirroring how search providers resolve through get_provider_search_config. Each public entrypoint is @client-decorated, so every operation logs the same way litellm.asearch does. The low-level names are sandbox-scoped on purpose so they do not collide with the existing OpenAI Containers API (litellm.create_container), which stays untouched.

The e2b wire calls were derived from e2b's open-source SDK: create is POST api.e2b.app/sandboxes with templateID, run is a streamed NDJSON POST to the per-sandbox code-interpreter host on port 49999 with the envd access token, and delete is DELETE api.e2b.app/sandboxes/{id}.

Out of scope here and tracked on the ticket for later phases: the proxy /v1/containers endpoints, the interceptor that swaps OpenAI's container for the configured sandbox, gpt-5 code_interpreter routing, container reuse, file read/write, and the OpenSandbox and Daytona backends.

Tests

tests/test_litellm/sandbox/test_e2b_sandbox.py. Unit tests inject a fake async HTTP client (dependency injection, not monkeypatching) and assert the request shapes (templateID not template, the 49999 execute host, the X-Access-Token header, DELETE /sandboxes/{id}), the NDJSON-to-result mapping, and that the ephemeral path still deletes the sandbox when the run raises. An integration test hits the real e2b API and is skipped unless E2B_API_KEY is set.

Proof of fix

Run against the real e2b API (this is the public surface the PR ships; there is no proxy endpoint in this phase):

export E2B_API_KEY=...   # real key
python - <<'PY'
import asyncio, litellm
async def main():
    r = await litellm.acode_interpreter_tool(provider="e2b", code="print(sum(range(10)))")
    print("ephemeral stdout:", repr(r.stdout.strip()), "| error:", r.error)
    e = await litellm.acode_interpreter_tool(provider="e2b", code="1/0")
    print("error name:", e.error and e.error["name"])
    c = await litellm.acreate_sandbox(provider="e2b")
    print("created sandbox id:", c.id, "| domain:", c.domain)
    run = await litellm.arun_code(provider="e2b", container=c, code="print(6*7)")
    print("run stdout:", repr(run.stdout.strip()))
    print("deleted:", await litellm.adelete_sandbox(provider="e2b", container=c))
asyncio.run(main())
PY

Output:

ephemeral stdout: '45' | error: None
error name: ZeroDivisionError
created sandbox id: i20qlm1j8ykre99l9zq0s | domain: e2b.app
run stdout: '42'
deleted: True

The full suite including the gated integration tests, with a real key set:

11 passed in 1.15s

Add a provider-agnostic code execution primitive that runs model-generated
code in an isolated sandbox and returns the output, with e2b as the first
backend over raw httpx (no SDK dependency).

Public API: litellm.acode_interpreter_tool (ephemeral create -> run -> delete)
plus the low-level lifecycle litellm.acreate_sandbox / arun_code /
adelete_sandbox. Each is @client-decorated so operations are logged like
litellm.asearch. Backends implement BaseSandboxConfig; resolved via
ProviderConfigManager.get_provider_sandbox_config.
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@codecov

codecov Bot commented Jun 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 31.60920% with 119 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
litellm/llms/e2b/sandbox/transformation.py 0.00% 86 Missing ⚠️
litellm/sandbox/main.py 41.86% 25 Missing ⚠️
litellm/llms/base_llm/sandbox/transformation.py 85.18% 4 Missing ⚠️
litellm/utils.py 33.33% 4 Missing ⚠️

📢 Thoughts on this report? Let us know!

@greptile-apps

greptile-apps Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a provider-agnostic code execution primitive backed by e2b, following the same structural pattern as litellm.asearch: a BaseSandboxConfig abstract class, an E2BSandboxConfig concrete implementation that talks directly to e2b's REST API over AsyncHTTPHandler, a public sandbox/main.py module with @client-decorated entrypoints, and a ProviderConfigManager.get_provider_sandbox_config resolver.

  • Adds acreate_sandbox, arun_code, adelete_sandbox, and the ephemeral convenience wrapper acode_interpreter_tool; sandbox names are chosen to avoid collision with the existing OpenAI Containers API.
  • e2b wire calls (create POST, streamed NDJSON execute on port 49999, DELETE) are implemented directly over httpx without adding an e2b SDK dependency; access tokens from create are stored in ContainerHandle._hidden_params and threaded through to the execute call.
  • Unit tests use dependency-injected fake HTTP clients and cover request shapes, NDJSON parsing, the MAX_OUTPUT_BYTES cap, the teardown-on-error guarantee, and the zero-timeout edge case; real-network tests are gated on E2B_API_KEY.

Confidence Score: 5/5

Safe to merge; implementation is well-structured with a single minor edge-case connection leak in the output-cap error path.

All new code is isolated to new modules and follows established litellm patterns (AsyncHTTPHandler reuse, provider config manager, @client logging). The ephemeral teardown guarantee is correctly implemented via try/finally. The one notable finding is that _read_capped_lines does not close the streaming httpx response when the output-size ValueError is raised mid-iteration, leaving the connection open until timeout — but this only fires in the >10 MB output edge case and does not affect correctness of results.

litellm/llms/e2b/sandbox/transformation.py — the streaming response should be explicitly closed in _read_capped_lines's finally block.

Important Files Changed

Filename Overview
litellm/llms/e2b/sandbox/transformation.py New e2b sandbox provider implementation; streaming response not closed on early exit from _read_capped_lines
litellm/sandbox/main.py New public entrypoints for sandbox operations; correct use of try/finally for cleanup, proper internal-kwarg stripping
litellm/llms/base_llm/sandbox/transformation.py New base classes ContainerHandle, CodeExecutionResult, and BaseSandboxConfig; clean, no issues
litellm/utils.py Adds get_provider_sandbox_config to ProviderConfigManager, mirroring the existing search provider pattern
tests/test_litellm/sandbox/test_e2b_sandbox.py Mock-only unit tests covering request shapes, NDJSON parsing, teardown guarantee, and error edge cases
tests/integration/sandbox/test_e2b_sandbox.py Real-network integration tests gated on E2B_API_KEY; correctly skipped when key is absent

Reviews (3): Last reviewed commit: "chore(sandbox): re-trigger automated rev..." | Re-trigger Greptile

Comment thread tests/test_litellm/sandbox/test_e2b_sandbox.py Outdated
Comment thread litellm/llms/e2b/sandbox/transformation.py Outdated
Comment thread litellm/llms/e2b/sandbox/transformation.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 781d4cdd42

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread litellm/llms/e2b/sandbox/transformation.py Outdated
Comment thread litellm/llms/e2b/sandbox/transformation.py
Comment thread litellm/llms/e2b/sandbox/transformation.py Outdated
@veria-ai

veria-ai Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

- document e2b provider in provider_endpoints_support.json and add a sandbox endpoint definition
- regenerate dashboard CallTypes after the sandbox call-type additions
- guard explicit timeout=0 instead of coercing it to the default
- require a ContainerHandle access token before running code; reject bare-id runs
- return False on a 404 delete now that the shared http handler raises for status
- skip non-JSON NDJSON lines and cap streamed output to bound memory
- move the real-network integration tests out of tests/test_litellm into tests/integration/sandbox
- modernize annotations in the new sandbox modules to PEP 585/604 (list/dict,
  X | None) and drop the now-unnecessary quoted forward refs so the strict-rule
  budget delta for UP006/UP037/UP045 returns to zero
- add __all__ to litellm/sandbox/main.py so 'import *' only re-exports the four
  public entrypoints instead of leaking module-level imports
utils.py uses 'from __future__ import annotations', so the quoted forward ref
tripped UP037; the unquoted union is lazily evaluated and keeps the strict-rule
delta at zero
@krrish-berri-2

Copy link
Copy Markdown
Contributor Author

@greptileai review

@krrish-berri-2

Copy link
Copy Markdown
Contributor Author

@greptileai

@krrish-berri-2 krrish-berri-2 enabled auto-merge (squash) June 20, 2026 23:21
@krrish-berri-2 krrish-berri-2 merged commit 53593f6 into litellm_internal_staging Jun 20, 2026
116 of 121 checks passed
@krrish-berri-2 krrish-berri-2 deleted the litellm_e2b_code_execution branch June 20, 2026 23:30
krrish-berri-2 added a commit to BerriAI/litellm-docs that referenced this pull request Jun 21, 2026
* docs(sandbox): document e2b code execution primitive

Documents the SDK surface added in BerriAI/litellm#30898: acode_interpreter_tool for ephemeral runs and the acreate_sandbox / arun_code / adelete_sandbox lifecycle. Covers the e2b backend setup, parameters, the CodeExecutionResult passthrough shape, and what is intentionally out of scope for this phase (proxy endpoints, OpenAI container interceptor, gpt-5 routing, container reuse, file IO, OpenSandbox, Daytona). Adds a sidebar entry under LiteLLM Python SDK > SDK Functions.

* docs(sandbox): add Responses API code interpreter interceptor section

Documents BerriAI/litellm#30905: how a client can call OpenAI's code interpreter through /v1/responses and have the code run in the configured sandbox (e2b) instead of OpenAI's container, with no client-side change. Covers the sandbox_tools registry and code_interpreter_interception callback, the curl shape, response-shape parity with native code_interpreter_call (id/type/status/code/container_id/outputs as an OpenAI-shaped logs array), streaming via the synthetic SSE wrapper, forced tool_choice rewrite, sandbox lifecycle and per-request isolation on a server-minted token, the stripped control fields that prevent client forgery, hot-reload behaviour, and the v0 limitation that file upload and download are not yet supported. Also notes the new api_base passthrough on the SDK entrypoints.

* docs(sandbox): cross-link the interceptor from the Code Interpreter guide

A proxy user looking for code interpreter lands on guides/code_interpreter, not the SDK sandbox page, so the interceptor was effectively undiscoverable from the proxy side. Adds a tip in the Code Interpreter guide pointing at the new interceptor section, and a second sidebar entry under Tool Calling labelled 'Code Interpreter Sandbox Interception' next to the existing Web Search Interception entry, mirroring how websearch_interception is exposed.

* docs(sandbox): turn the proxy section into a step-by-step tutorial

The interceptor section had the config and curl but no narrative walking a proxy user from a clean checkout to a working call. Restructures into five numbered steps (get the e2b key, write config.yaml with a model_list plus sandbox_tools registry, start the proxy, call /v1/responses via curl or the OpenAI SDK, verify the code ran in e2b via the cntr_-wrapped container_id and the e2b dashboard sandbox count). Calls out enabled_providers as the safety gate so flipping the callback on does not silently change behaviour for other providers.

* docs(sandbox): tighten the tutorial section

* docs(sandbox): collapse the proxy section notes into prose

---------

Co-authored-by: Krrish Dholakia <[email protected]>
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
* feat(sandbox): add e2b code execution primitive

Add a provider-agnostic code execution primitive that runs model-generated
code in an isolated sandbox and returns the output, with e2b as the first
backend over raw httpx (no SDK dependency).

Public API: litellm.acode_interpreter_tool (ephemeral create -> run -> delete)
plus the low-level lifecycle litellm.acreate_sandbox / arun_code /
adelete_sandbox. Each is @client-decorated so operations are logged like
litellm.asearch. Backends implement BaseSandboxConfig; resolved via
ProviderConfigManager.get_provider_sandbox_config.

* fix(sandbox): address review feedback and CI gates

- document e2b provider in provider_endpoints_support.json and add a sandbox endpoint definition
- regenerate dashboard CallTypes after the sandbox call-type additions
- guard explicit timeout=0 instead of coercing it to the default
- require a ContainerHandle access token before running code; reject bare-id runs
- return False on a 404 delete now that the shared http handler raises for status
- skip non-JSON NDJSON lines and cap streamed output to bound memory
- move the real-network integration tests out of tests/test_litellm into tests/integration/sandbox

* fix(sandbox): satisfy strict ruff gate and scope star-exports

- modernize annotations in the new sandbox modules to PEP 585/604 (list/dict,
  X | None) and drop the now-unnecessary quoted forward refs so the strict-rule
  budget delta for UP006/UP037/UP045 returns to zero
- add __all__ to litellm/sandbox/main.py so 'import *' only re-exports the four
  public entrypoints instead of leaking module-level imports

* fix(sandbox): drop quotes on sandbox config return annotation

utils.py uses 'from __future__ import annotations', so the quoted forward ref
tripped UP037; the unquoted union is lazily evaluated and keeps the strict-rule
delta at zero

* chore(sandbox): re-trigger automated review after addressing feedback
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants