skip to content
e2e raised $8m to build the software verification layer for the agentic coding era
13.8kstar on github

docs / Models and agents

[ view as markdown ]

Subscriptions

Use your ChatGPT, GitHub Copilot, OpenCode Console, or SuperGrok subscription, your Claude Max or Team plan's API credits, or your Claude Code or Codex login for agent steps.

Run agent steps on a plan you already pay for. Sign in one time, then use the plan’s model in e2e.config.ts.

PlanHow to connect
ChatGPT Plus or Pronpx e2e login openai
GitHub Copilotnpx e2e login github-copilot
OpenCode Console (OpenCode Zen and OpenCode Go)npx e2e login opencode-console
SuperGrok or X Premium+npx e2e login spacexai
Claude Max or Team (Claude Code)API key, paid from the plan’s monthly API credits, not its usage limits
Claude Code or Codex loginRun steps on your coding agent

Set up

For a new project, run e2e init and select your plan (quickstart). For an existing project, install the packages, sign in, and set agents: { default: agent } in e2e.config.ts.

npm install -D ai @ai-sdk/openai
npx e2e login openai
Model setup
import { chatgpt } from 'e2e/oauth/chatgpt';

const agent = { model: chatgpt('gpt-6-luna') };

Pick a model

e2e models lists the model ids your login can use:

npx e2e models            # every stored login
npx e2e models openai     # one provider

The id is not always the name the vendor’s app shows.

With OPENCODE_API_KEY set, e2e models opencode-console lists the ids that key serves, the same key opencodeConsole() uses in place of the login.

  • Copilot marks not enabled (turn the model on in your Copilot settings on GitHub, or ask the admin who manages your seat) and no chat or responses (use a different model). It can’t see what your plan serves: on Copilot Free, GitHub serves models through auto model selection only, so a model called by id is usually refused. The run then stops with MODEL_UNAVAILABLE naming it; pick another id or an API provider.
  • OpenCode Console lists go/ models only if you have Go. It leaves out models your workspace turned off, and Zen’s free models. Key providerOptions by opencode for chat completions models, else by openai, anthropic, or google.

Sign-in help

  • ChatGPT opens a browser. To use a device code, add --device.
  • Copilot uses your GitHub CLI login. Without gh, pass --client-id with your own GitHub OAuth app that has device flow on. For GitHub Enterprise, add --enterprise-url.
  • OpenCode Console shows a device code. Approve it and select the workspace. The page shows e2e as an unknown client. To change the workspace, run npx e2e logout opencode-console and sign in again.
  • SuperGrok shows a device code and the URL where you enter it.

npx e2e login with no provider opens a picker that shows which accounts are signed in. npx e2e logout removes a login. When a saved login stops working, the step fails with MODEL_PROVIDER_FAILED, and the message gives the npx e2e login <provider> command to run. See the CLI reference for every flag.

Model-call cancellation also applies while waiting for a token refresh.

CI

Use API keys in CI. Claude Max or Team already uses one. For OpenCode Console, set OPENCODE_API_KEY to an OpenCode Go subscription API key, or to a service account API key created in the Console (Service Accounts, then API Keys). For ChatGPT, Copilot, and SuperGrok, use an API provider.

E2E_OAUTH_CREDENTIALS can give a stored login to CI, but e2e doesn’t save refreshed tokens back to it. A login with a rotating refresh token works there only for a limited time.

Use your coding agent

@e2e-dev/acp runs agent.act and agent.assert on Claude Code or Codex over the Agent Client Protocol. The agent signs in with its own login or API key, so e2e needs no model provider.

The agent gets complete_step and the built-in agent’s action tools for the verbs the target’s engine supports. The runner records every action, so steps replay from the cache without the agent. Your tests don’t change.

Install the package and Zed’s Claude Code adapter. Sign in with claude, or set ANTHROPIC_API_KEY.

npm install -D @e2e-dev/acp @agentclientprotocol/claude-agent-acp
e2e.config.ts
import type { E2EConfig } from 'e2e';
import { web } from '@e2e-dev/web';
import { claudeCode } from '@e2e-dev/acp';

export default {
  targets: [{ engine: web(), app: { url: 'http://localhost:3000' } }],
  agents: { default: { executor: claudeCode({ model: 'sonnet' }) } },
} satisfies E2EConfig;

claudeCode() and codex() take these options:

OptionWhat it does
modelA model the agent offers, such as sonnet. When the agent doesn’t offer it, the session fails to start and the error lists the models it does offer. Omitted, the agent’s default runs.
envVariables added to the agent’s environment.
systemText appended to the rules the agent gets with the first step.
nameThe executor name in reports.

Unexpected forwarding failures stop the ACP session and appear in the runner’s secret-redacted failure output. The HTTP client receives a generic error or a closed stream, without the exception text.

What the agent can use

An action the agent takes with its own tools isn’t recorded, can’t replay, and can reach files or the web. So e2e keeps the agent to the step tools:

  • The agent process starts in your project. Its session opens in an empty temporary directory, and e2e offers it no file system or terminal access.
  • e2e allows permission requests for step tools and rejects all others.
  • When a tool of the agent’s own runs without asking, e2e stops the turn and fails the step with POLICY_DENIED. Later step tool calls take no action. The session closes, so the next step starts a fresh agent that never saw what the tool read.

Each preset also turns off the agent’s own tools that run without asking:

  • Claude Code: no built-in tools, no user, project, or local settings, no MCP servers but the step tools, no saved session, and no bypass of permission prompts.
  • Codex: read-only mode, no shell, web search, apps, plugins, browser, or computer use. Every MCP server from your Codex config is off for these sessions only. Codex can still start a subagent, which fails the step like any tool of its own.

Secrets

A secret param reaches the agent by name only. When the target’s engine can fill secrets, the agent calls type_secret and the runner types the value, as for the built-in agent.

The agent’s process gets your environment without the E2E_SECRET_* and E2E_USER_* variables.

A secret in any other variable reaches the agent. Examples are API_KEY behind secrets: { apiKey: process.env.API_KEY }, or the variable an E2E_SECRET_* value was copied from. Keep secrets in E2E_SECRET_* and E2E_USER_* only.

How a test runs

One agent session serves one test attempt. The first agent step that the cache can’t replay starts it. Each later step is one more message in the same conversation, and the session closes when the attempt ends, or earlier on a POLICY_DENIED step.

Each message carries the step, its non-secret params, the steps done so far, and the current screen. Action results say what changed on screen, and a screenshot reaches the agent as an image.

An assertion can’t change the screen. observe, screenshot, scrolling, and hovering still work. The agent ends every step with complete_step: passed, failed, or blocked with an error code, as the built-in agent does.

Limits
  • The Agent Client Protocol doesn’t report model requests. Each prompt turn counts as one model call against maxModelCalls, with its tokens and cost when the agent reports them. maxSteps and the step timeout bound the turn.
  • waitFor and extract use the built-in judgments, so they need the agents entry’s model.
  • An agents entry with an executor takes no tools, so the agent gets no project tools from defineTool.

Next

Models

Use an API key, a gateway, or a local model instead.

[ → ]

Agents and personas

Give tests different models and instructions.

[ → ]

was this page helpful?