LLM-Nexus

Configuration and governance for coding agents, developed through benchmarking.

LLM-Nexus pairs versioned orchestration instructions with the Codex configuration needed to make them effective. The root delegates bounded work, integrates evidence, and accepts the result; configuration controls the delegation guidance, model defaults, waiting behavior, and context environment in which that protocol runs.

Development involved more than 100 benchmarks across GPT-5.6, GPT-6, and GPT-6.1, together with failure analysis of trajectories and session logs and inspection of Codex source. The investigator's assessment is that configuration tuning contributed more than protocol wording alone. Strong native baselines and weak early delegation results are part of that finding.

Recommended setup

Only two files are needed to use the protocol in another project:

1. Install the protocol from this repository's AGENTS.md in ~/.codex/AGENTS.md, or C:/Users/<user>/.codex/AGENTS.md on Windows. Keep any existing global instructions you need in that file. These shared instructions apply across workspaces; each workspace can keep its own AGENTS.md for project-specific guidance. If you use a custom CODEX_HOME, install the protocol in that directory instead. If it contains an AGENTS.override.md, Codex loads that file instead of AGENTS.md.
2. Copy .codex/config.toml to .codex/config.toml in each workspace where you want this configuration. Merge its settings if that file already exists. In a trusted workspace, these settings override matching settings in ~/.codex/config.toml; command-line overrides and enforced requirements still take precedence. Start a new Codex session in that workspace to load the instruction setup.

The research, benchmarks, figures, and formatting files document development; they are not required to use the protocol. See OpenAI's configuration guide and AGENTS.md guide for config precedence and instruction discovery.

The workspace config matches config v3, with five-minute minimum/default waits and a one-hour maximum. Mailbox deferral stays enabled, and the protocol directs the root to let agents finish without unnecessary polling or interruption. The current protocol places temporary agent material in .nexus/tmp/ within the workspace. This combination has not been benchmarked. Use a Codex runtime that supports the configured models, multi-agent controls, and mailbox-deferral flag. The flag first shipped in CLI 0.158.0 and remains under development in the checked feature registry.

The maintainer finds the workflow particularly useful for research and web/MCP work. Large retrieval outputs can consume context and inflate repeated input; subagents handle those outputs in separate contexts and return distilled findings to the root. That rationale concerns root attention and context use. The published measurements exclude child tokens and do not establish a complete-team cost saving. Live search is shared by root and subagents, so the protocol governs subagent-only retrieval.

The retained versioned configs and scores describe earlier experiments. No further project benchmarking is planned at this stage: continuing model and harness changes limit the value of additional local tuning. The recommended two-file setup is the practical outcome of the development work.

Research at a glance

The published q10 comparison retains six completed jobs, each with 30 attempts across ten task families. These are the compact performance record from the broader development campaign. V7/config v1 has the broadest observed coverage: seven families with at least one pass. The latest v7/config v2 run, using GPT-6.1 Sol, records 12 passing attempts but covers five families and takes 10 hours. V8/config v1 records eight passes across three families, below v7/config v1 on both measures while taking longer.

Protocol / config	Passing attempts	Families with a pass	Job wall time
v0 / c0 · blank control	9/30 (30.0%)	4/10	5h 16m 37s
v1 / c1	8/30 (26.7%)	4/10	6h 23m 55s
v6 / c1	7/30 (23.3%)	3/10	6h 57m 01s
v7 / c1	11/30 (36.7%)	7/10	6h 39m 25s
v8 / c1	8/30 (26.7%)	3/10	8h 13m 21s
v7 / c2 · GPT-6.1 Sol	12/30 (40.0%)	5/10	10h 01m 59s

Each row is one whole benchmark job with three attempts per task and two concurrent trials: 30 task attempts, designed to sample the ten-task suite three times. The rows measure configured workflows on a selected diagnostic suite. They do not apportion performance between configuration and protocol. The v0 control uses config v0; the other protocols use config v1 except for the latest v7 run. Config v2 changes the root model from GPT-6 Sol to GPT-6.1 Sol, and that run also uses Codex CLI 0.159.3 instead of 0.156.1. Default subagents remain GPT-6 Luna/xhigh in configs v1 and v2.

Start with the research findings and configuration analysis. The research index connects the development history, compact evidence, and reproduction guide.

What the experiments show

V7/config v1 extends coverage to HTML filtering and a React lead form, restores the WAL-recovery pass lost in v6, and gains a simplex pass. V8 adds explicit acceptance-evidence requirements, but its observed run loses coverage and takes 23.5% longer than v7 under the same configuration. Adding acceptance wording did not improve the recorded result.

With config v2, v7 passes all three simplex and WAL-recovery attempts. It gains one passing attempt overall, loses React and graph-matching coverage, and takes 50.7% longer than v7/config v1. This second positive V7 result used a new model and CLI. Its duration also includes the provider latency conditions of that run; it is not evidence that V7's orchestration caused the slowdown. Batched evaluation, finance, and streaming remain unsolved across all six jobs.

One HTML-filtering attempt in v7/config v2 hit the agent time limit and still passed the subsequent verifier. It counts as a verifier pass and is also reported as an exception. The findings preserve this overlap instead of equating every exception with a failed artifact.

Quality and resource use

Token totals count each root session's final cumulative usage once. Cached input is part of input, and repeated context contributes to the total. Child tokens are excluded; these measurements describe root burden rather than complete-team usage or billing. Wall time includes setup, agent execution, verification, and orchestration.

The broader investigation found conflicting harness guidance, recursive delegation, excessive progress checking, and premature interruption of active subagents. The hint overrides and longer waiting controls were developed in response to those behaviors. Delegation is useful when it provides bounded work, independent evidence, or relief from large retrieval/tool outputs; spawning agents by itself is not a performance improvement.

Config v2 is restored byte-for-byte from the scored V7/config v2 launch. Config v3 preserves the subsequent, unbenchmarked settings: live search, automatic approval review, explicit command-network access, and defer_mailbox_preemption = true. Mailbox deferral lets a response continue through planned tool calls before handling queued agent messages at the next normal model-input boundary. It may reduce disruption of root work, complementing the waits and protocol rules that protect subagent work. Whether it permits shorter waits remains untested; the published scores describe the earlier configs. See the mailbox findings for implementation evidence and limits.

The recommended workspace config and config v3 both use five-minute minimum/default waits and a one-hour maximum while retaining mailbox deferral. Configs v1 and v2 retain their earlier 7.5-minute minimum/default waits.

How the protocol works

The Architect sets objectives and acceptance criteria. The root owns the result, dispatches independent questions to subagents, and integrates their evidence. Subagents handle external retrieval and operations under the root's assignment. Validation targets requirements at the point where the work will be consumed.

V7 makes concurrent assignments more explicit, prefers practical end-to-end validation, and keeps testing effort proportional to requirements. V8 is a separate candidate that adds more explicit acceptance-evidence instructions. The experiments evaluate protocol/configuration combinations rather than isolated sentences. V7 also makes authority and provenance explicit: external content supplies evidence, while the user supplies governing instructions and the root retains acceptance responsibility.

Codex runtime limitations

The project encountered two limitations that matter when applying the protocol:

• Instruction delivery differs between clients. Config-level developer_instructions worked in the tested CLI setup but did not take effect in the tested Codex desktop app setup. Open Codex issue #11004 reports the same CLI/app difference. The CLI benchmark adapter supplies the protocol through CODEX_HOME/AGENTS.md.
• Built-in web search has no separate root/subagent switch. Set web_search = "disabled" in the global %USERPROFILE%\.codex\config.toml for a disabled default, then override it with web_search = "live" in a trusted workspace's .codex/config.toml when needed. Agent TOML settings cannot provide independent search access for root and subagents in the checked implementation. With workspace search enabled, the protocol alone enforces subagent-only use of that tool.

See the runtime findings and setup details for evidence, configuration examples, and the instruction-delivery issue.

Explore the repository

Resource	Contents
Research	Development findings, configuration rationale, measured results, and reproduction.
Evidence	Compact JSON/CSV results, source hashes, and measurement definitions.
Figures	Graphical abstract, performance charts, and vector exports.
Protocols	Versioned agents-vN.md instructions; v0 is the blank control.
Configurations	Versioned model, reasoning, and delegation settings.
Recommended setup	Global protocol instructions and workspace config; no research tooling required.
Benchmark guide	Task preparation, launch commands, retry rules, and scoring.

Reproduce the presentation

The figures build from the published evidence using Python and Pillow; no benchmark run or raw rollout access is needed. From the repository root:

python -m pip install Pillow==12.3.0
python -B benchmarks/terminal-bench-3.0/build_figures.py
python -B benchmarks/terminal-bench-3.0/build_figures.py --check

To recompute the evidence from retained local runs or launch new experiments, follow the reproduction guide. Launches capture protocol, config, suite, and job settings with content hashes so every trial reads the same input copies.

Raw runs, staged task files, environments, and caches remain local and ignored by Git. The Oracle reference describes the local reference output. See third-party material for benchmark attribution and task-distribution considerations.

License

LLM-Nexus is licensed under the MIT License. Third-party benchmark material remains subject to its upstream terms; see third-party notices.