Academic

Terminal-Bench 4.0

Updated 11/16/2026

Frontier-difficulty terminal tasks across software, science, ML, operations, hardware, security, and media

As of November 16, 2026, Claude Fable 5.2 (Mini-SWE-agent) and Claude Opus 5.5 (Mini-SWE-agent) tie for first on Terminal-Bench 4.0 with 65.15%, followed by Claude Sonnet 5.5 (Mini-SWE-agent) at 64.14%.

Terminal-Bench 4.0Frontier-difficulty terminal tasks
Terminal-Bench 4.0•Nov 16, 2026
ACCURACY
Lab:
Pinch or Ctrl/Cmd + scroll to zoom. Once zoomed, scroll or swipe with two fingers to pan, or drag to pan. Shift + drag draws a box to zoom into; turn on box zoom to draw one without Shift. Plus and minus keys zoom, arrow keys pan, 0 resets.

Terminal-Bench 4.0 leaderboard

Rank Model Harness Score Cost / Test Tokens / Test Duration / Test
1 Claude Fable 5.2 Mini-SWE-agent 65.15% $14.56 20.62m 55m07s
2 Claude Opus 5.5 Mini-SWE-agent 65.15% $12.04 28.27m 1h04m
3 Claude Sonnet 5.5 Mini-SWE-agent 64.14% $10.75 58.53m 1h22m
4 Kimi K3.1 Pro Mini-SWE-agent 63.64% $4.57 16.57m 49m21s
5 GPT-6.2 Astra Mini-SWE-agent 63.13% $4.69 3.10m 34m17s
6 Gemini 4 Carbon Mini-SWE-agent 63.13% $7.45 12.46m 45m07s
7 GPT-6.2 Sol Mini-SWE-agent 60.61% $1.13 3.24m 23m21s
8 GPT-6 Astra Mini-SWE-agent 59.60% $9.58 3.01m 35m42s
9 Claude Fable 5.1 Mini-SWE-agent 58.08% $17.18 18.82m 1h00m
10 Wonder Full Mini-SWE-agent 58.08% $10.53 31.82m 1h12m
11 Gemini 4 Argon Mini-SWE-agent 57.58% $17.64 16.66m 53m35s
12 GPT-6.1 Sol Mini-SWE-agent 55.05% $1.72 2.86m 45m12s
13 Gemini 4 Hydrogen Mini-SWE-agent 54.04% $7.56 22.03m 30m40s
14 Claude Opus 5 Mini-SWE-agent 53.53% $18.60 23.71m 1h07m
15 GPT-6.1 Terra Mini-SWE-agent 49.49% $2.16 16.82m 33m53s
16 Wonder Mini-SWE-agent 49.49% $14.66 34.66m 54m41s
17 Qwen 4 Max Mini-SWE-agent 48.48% $3.90 34.66m 1h15m
18 GPT-6 Sol Mini-SWE-agent 44.44% $5.79 13.79m 34m56s
19 Claude Fable 5 Mini-SWE-agent 41.41% $30.34 19.75m 1h08m
20 Muse Ember Mini-SWE-agent 39.39% $0.22 30.93m 26m21s
21 Hy4 Mini-SWE-agent 39.39% $1.28 16.16m 43m20s
22 GLM 5.3 Mini-SWE-agent 38.89% $9.37 31.54m 1h23m
23 Qwen 3.8 Max 0902 Mini-SWE-agent 38.89% $12.15 39.16m 2h23m
24 GPT-5.6 Sol Mini-SWE-agent 37.88% $7.98 10.49m 33m46s
25 GLM 5.3 Flash Mini-SWE-agent 37.37% $1.22 35.66m 1h01m
26 Claude Haiku 5.5 Mini-SWE-agent 36.36% $3.12 45.50m 51m56s
27 Qwen 3.8 Max Mini-SWE-agent 34.34% $10.63 33.91m 2h28m
28 DeepSeek V4.1 Flash 1017 Mini-SWE-agent 31.82% $0.54 39.65m 41m10s
29 Step 5 Preview Mini-SWE-agent 31.82% $3.11 40.82m 1h58m
30 MiMo V2.6 Pro Mini-SWE-agent 31.31% $0.46 26.60m 2h42m
31 Grok 4.7 Mini-SWE-agent 28.79% $18.01 18.23m 50m08s
32 Mistral Large 4 Preview Mini-SWE-agent 27.78% $5.18 41.07m 1h40m
33 Mistral Large 4 Mini-SWE-agent 26.26% $2.73 25.04m 1h05m
34 MiniMax-M3.1-Flash Mini-SWE-agent 25.25% $0.73 38.09m 50m31s
35 Muse Spark 1.3 Max Mini-SWE-agent 24.75% $6.65 32.43m 52m56s
36 MiMo V2.6 Flash Mini-SWE-agent 24.24% $0.21 33.20m 2h25m
37 Claude Opus 4.8 Mini-SWE-agent 23.23% $17.14 17.17m 1h11m
38 GPT-5.6 Terra Mini-SWE-agent 22.73% $5.60 13.66m 31m29s
39 Ember-1 Mini-SWE-agent 20.71% $19.48 46.75m 1h55m
40 DeepSeek V4.1 Flash Mini-SWE-agent 19.70% $0.50 31.53m 48m22s
41 Gemini 3.8 Flash Mini-SWE-agent 19.70% $8.65 36.56m 1h47m
42 DeepSeek V4 Flash 0731 Mini-SWE-agent 18.69% $0.77 30.13m 1h46m
43 Grok 4.6 Mini-SWE-agent 17.17% $5.06 8.40m 29m25s
44 Kimi K3 Mini-SWE-agent 17.17% $12.48 27.56m 3h09m
45 Hy4 Flash Mini-SWE-agent 15.15% $0.10 18.32m 32m27s
46 DeepSeek V4 Pro 0813 Mini-SWE-agent 14.14% $3.31 54.70m 1h32m
47 GPT-6 Luna Mini-SWE-agent 13.64% $0.35 15.38m 32m46s
48 Inkling 2 Small Mini-SWE-agent 12.63% $8.22 254.59m 1h41m
49 Gemini 3.7 Flash Mini-SWE-agent 12.12% $8.71 23.32m 1h24m
50 GPT-5.6 Luna Mini-SWE-agent 11.62% $0.73 19.00m 35m55s
51 DeepSeek V4 Mini-SWE-agent 11.11% $3.45 56.99m 1h26m
52 Claude Sonnet 5 Mini-SWE-agent 10.61% $26.05 67.55m 1h45m
53 Muse Spark 1.3 Mini-SWE-agent 10.61% $7.04 33.50m 1h59m
54 Grok 4.5 Mini-SWE-agent 8.59% $5.52 9.64m 33m43s
55 Hy4 Preview Mini-SWE-agent 8.08% $3.07 41.06m 2h10m
56 Muse Spark Open Mini-SWE-agent 6.57% $7.26 13.99m 2h04m
57 Gemini 3.5 Flash Mini-SWE-agent 6.06% $5.98 18.68m 1h11m
58 Muse Spark 1.2 Mini-SWE-agent 6.06% $4.14 21.55m 1h18m
59 Beam 1.0 Mini-SWE-agent 4.04% - 12.27m 19m15s
60 Mercury 3 Mini-SWE-agent 3.03% $0.42 18.93m 33m07s
61 Gemini 3.1 Pro Preview (02/26) Mini-SWE-agent 2.52% $4.11 7.96m 18m44s
62 GPT 5.4 Mini Mini-SWE-agent 2.52% $1.44 7.38m 29m30s
63 MiniMax-M3 Mini-SWE-agent 1.01% $7.23 50.32m 1h16m
64 Inkling Mini-SWE-agent 0.51% $2.50 11.93m 22m59s
65 MiMo V2.5 Pro Mini-SWE-agent 0.51% $0.23 9.12m 2h07m
66 Inkling Small Mini-SWE-agent 0.00% $0.32 3.97m 11m45s
67 Nemotron 3.5 Super VL Mini-SWE-agent 0.00% $0.86 14.87m 14m56s
68 Mercury 2.5 Mini-SWE-agent 0.00% $0.37 3.85m 15m14s
Reasoning EffortAccuracy at three reasoning efforts per model, labelled with each provider's own setting
Terminal-Bench 4.0•Nov 16, 2026
Accuracy
Model:
Error bars show ±1 standard error.

Background

Terminal-Bench is an open-source benchmark that measures how well an agent can complete long, realistic pieces of work in a sandboxed terminal. Version 4.0 is a fresh set of 66 community-contributed, maintainer-reviewed tasks, none of which appear in Terminal-Bench 2.1. Each task is built around a real deliverable — a working service, a proof, a CAD model, a trained kernel, a forensic report — and carries an expert time estimate: the median task is estimated at 4 hours of expert work, and the range runs from under an hour to 60 hours.

Where Terminal-Bench 2.x was mostly software engineering and system administration, 4.0 is deliberately wide. The tasks span seven categories:

  • Software (18 tasks): databases, distributed systems, frontend performance, data engineering, algorithms
  • Science (14 tasks): biology, chemistry, physics, formal mathematics, robotics, linguistics, earth science
  • ML (11 tasks): training runs, inference serving, GPU kernels, evaluation harnesses
  • Operations (9 tasks): logistics, finance, insurance claims, compliance, supply chain
  • Hardware (5 tasks): CAD modeling and RTL design
  • Security (5 tasks): cryptography, application security, forensics
  • Media (4 tasks): music theory, audio transcription, layout design

We include Terminal-Bench because it is widely reported by model providers, because it reflects the kind of end-to-end terminal work that agentic coding products (Claude Code, Codex, Cursor and others) are expected to do, and because 4.0 raises the ceiling considerably: the tasks are long, multi-step, and graded strictly on the final artifact.

Terminal-Bench is developed by the Terminal-Bench community and the Harbor team as an open-source effort. We thank the community for building the benchmark and for helping us integrate it into our evaluation suite. To learn more or contribute tasks, visit tbench.ai.

Below is an example task from the Operations category (you can find the full task on the Harbor Hub).

Speed up the worker startup for the financial transaction processing pipeline so overdraft notifications are correct, not missed or duplicated, and arrive within 5s during worker respawns and rolling deploys in fresh containers.

Add any extra Python packages to /app/src/worker/requirements.txt.


Methodology

All models were benchmarked using mini-swe-agent, a minimal, model-agnostic agent harness. The model is given the task instruction and a single tool, bash: each turn it issues one or more shell commands, which run in a fresh subshell inside the task sandbox, and it reads back their output. There is no step or cost limit; the agent works until it signals that it has finished or reaches the task’s time limit.

All results reported are avg@3: each model is run through the full benchmark three times, each run scored as pass@1 (one attempt per task), and the displayed score is the mean of the three run-level scores, with error bars showing the standard error across those three runs. Each task is graded by its own verifier, which runs after the agent finishes and inspects the final state of the environment or the artifact the agent produced. A model must pass the verifier’s full test suite to get any credit for a task — there is no partial credit.

Tasks are run in isolated sandboxes on Daytona, following the upstream Harbor setup, including the three tasks that require a GPU. Each task has an 8-hour agent time limit, and each verifier has its own task-specific timeout; an agent that exceeds the time limit, or a verifier that cannot produce a result from the agent’s submission, scores zero for that task, matching Harbor’s semantics.


Comparison to Terminal-Bench 2.1

Our Terminal-Bench 2.1 results used the Terminus 2 harness; Terminal-Bench 4.0 uses mini-swe-agent. The Daytona sandboxes and pass@1 scoring are the same. The task set also changed:

  • New tasks. All 66 tasks are new; there is no overlap with the 89 tasks in 2.1.
  • Categories instead of difficulty tiers. 2.1 tagged tasks as easy/medium/hard. 4.0 tags them by domain (the seven categories above) and by an expert time estimate. We report scores per category.
  • Long time limits. Every 4.0 task allows the agent 8 hours.
  • Broader scope. 2.1 was predominantly software engineering. Roughly three quarters of 4.0 is outside traditional software engineering — science, ML systems, operations, hardware, security, and media.

Because the task set is entirely different, scores are not comparable across versions.