LLM‑driven agent that writes, evaluates & improves machine‑learning code.
AIDE ML is the open‑source “reference build” of the AIDE algorithm, a tree‑search agent that autonomously drafts, debugs and benchmarks code until a user‑defined metric is maximised (or minimised). It ships as a research‑friendly Python package with batteries‑included utilities (CLI, visualisation, config presets) so that academics and engineer‑researchers can replicate the paper, test new ideas, or prototyping ML pipelines.
| Layer | Description | Where to find it |
|---|---|---|
| AIDE algorithm | LLM‑guided agentic tree search in the space of code. | Described in our paper. |
| AIDE ML repo (this repo) | Lean implementation for experimentation & extension. | pip install aideml |
| Weco product | The platform generalizes AIDE's capabilities to broader code optimization scenarios, providing experiment tracking and enhanced user control. | weco.ai |
- Agent‑architecture researchers – swap in new search heuristics, evaluators or LLM back‑ends.
- ML Practitioners – quickly build a high performance ML pipelines given a dataset.
- Natural‑language task specification Point the agent at a dataset and describe goal + metric in plain English. No YAML grids or bespoke wrappers.
aide data_dir=… goal="Predict churn" eval="AUROC" - Iterative agentic tree search Each python script becomes a node in a solution tree; LLM‑generated patches spawn children; metric feedback prunes and guides the search. OpenAI’s MLE‑Bench (75 Kaggle comps) found the tree‑search of AIDE wins 4 × more medals than the best linear agent (OpenHands).
Utility features provided by this repo
- HTML visualiser – inspect the full solution tree and code attached to each node.
- Streamlit UI – prototype ML solution .
- Model‑neutral plumbing – OpenAI, Anthropic, Gemini, or any local LLM that speaks the OpenAI API.
| Institution | Paper / Project Name | Links |
|---|---|---|
| OpenAI | MLE-bench: Evaluating Machine-Learning Agents on Machine-Learning Engineering | Paper, GitHub |
| METR | RE-Bench: Evaluating frontier AI R&D capabilities of language-model agents against human experts | Paper, GitHub |
| Sakana AI | The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search | Paper, GitHub |
| Meta | The Automated LLM Speedrunning Benchmark: Reproducing NanoGPT Improvements | Paper, GitHub |
| Meta | AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench | Paper, GitHub |
| SJTU | ML-Master: Towards AI-for-AI via Integration of Exploration and Reasoning | Paper, GitHub |
Know another public project that cites or forks AIDE?
Open a PR and add it to the table!
# 1 Install
pip install -U aideml
# 2 Set an LLM key
export OPENAI_API_KEY=<your‑key> # https://platform.openai.com/api-keys
# 3 Run an optimisation
aide data_dir="example_tasks/house_prices" \
goal="Predict the sales price for each house" \
eval="RMSE between log‑prices"After the run finishes you’ll find:
logs/<id>/best_solution.py– best code foundlogs/<id>/tree_plot.html– click to inspect the solution tree
pip install -U aideml # adds streamlit
cd aide/webui
streamlit run app.pyUse the sidebar to paste your API key, upload data, set Goal & Metric, then press Run AIDE.
The UI shows live logs, the solution tree, and the best code.
# Choose a different coding model and run 50 steps
aide agent.code.model="claude-4-sonnet" \
agent.steps=50 \
data_dir=… goal=… eval=…Common flags
| Flag | Purpose | Default |
|---|---|---|
agent.code.model |
LLM used to write code | gpt-4-turbo |
agent.steps |
Improvement iterations | 20 |
agent.search.num_drafts |
Drafts per step | 5 |
agent.search.code_ahead |
Generated code drafts to keep queued ahead of execution | 0 |
agent.code.codex_branch_sessions |
Reuse Codex history for siblings and fork it when expanding a node | false |
codex.limits.usedPercent |
Gracefully stop after a step when Codex primary usage exceeds this percentage | null |
agent.mode=ag_generator creates a deterministic, no-LLM experiment panel from
one successfully scored AutoGluon artifact:
uv run aide \
--seed-from-sha feff2f3363 \
--seed-source-run 2-romantic-guan-of-eternity \
agent.mode=ag_generatorThe seed remains the common parent and control. Six children add exactly one
model-specific generator each, in this order: GroupBy, RSFC, arithmetic
interactions, categorical interactions, out-of-fold target encoding, and
categorical frequency encoding. Original features and all other seed settings
are preserved. Use agent.steps=N to run only the first N experiments.
This mode requires the experimental AutoGluon build that exposes these feature generators. It disables code/feedback model calls, research, synthesis, refactoring, and the generated LLM report; existing modes are unchanged.
import aide
import logging
def main():
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
aide_logger = logging.getLogger("aide")
aide_logger.setLevel(logging.INFO)
print("Starting experiment...")
exp = aide.Experiment(
data_dir="example_tasks/bitcoin_price", # replace this with your own directory
goal="Build a time series forecasting model for bitcoin close price.", # replace with your own goal description
eval="RMSLE" # replace with your own evaluation metric
)
best_solution = exp.run(steps=2)
print(f"Best solution has validation metric: {best_solution.valid_metric}")
print(f"Best solution code: {best_solution.code}")
print("Experiment finished.")
if __name__ == '__main__':
main()Use an explicit prefix when the model should run through an agent server:
# Codex app-server (started automatically for each call)
export AIDE_AGENT_CODE_MODEL="codex:gpt-5.6-sol:high"
export AIDE_AGENT_FEEDBACK_MODEL="codex:gpt-5.6-luna:low"OpenCode uses one persistent HTTP server. Start it before AIDE:
opencode serve --hostname 127.0.0.1 --port 4096
export AIDE_OPENCODE_SERVER_URL="http://127.0.0.1:4096"
export AIDE_AGENT_CODE_MODEL="opencode:x-preview-f-free:high"
export AIDE_AGENT_FEEDBACK_MODEL="opencode:x-preview-f-free:low"The format is <backend>:<model>:<effort>. OpenCode models from another
provider use <backend>:<provider>/<model>:<effort>, for example
opencode:anthropic/claude-sonnet-4-5:high.
If OpenCode reports finish=length, the backend fails immediately instead of
retrying the same truncated response.
Each OpenCode call runs in a temporary shallow clone of the current Git project. This matches the interactive TUI project context while isolating any tool writes from the working tree. Run AIDE from within a Git project.
To protect the OpenCode server with HTTP Basic Auth, set
OPENCODE_SERVER_PASSWORD (and optionally OPENCODE_SERVER_USERNAME) in both
the server and AIDE environments. The username defaults to opencode.
Unprefixed gpt-* and o* model names use the OpenAI API. The codex: prefix
is required for Codex app-server; no implicit GPT-to-Codex routing is applied.
export OPENAI_BASE_URL="http://localhost:11434/v1"
aide agent.code.model="qwen2.5" data_dir=… goal=… eval=…Note: evaluator defaults to gpt‑4o.
export OPENAI_BASE_URL="http://localhost:11434/v1"
aide agent.code.model="qwen2.5" agent.feedback.model="qwen2.5" data_dir=… goal=… eval=…
Tip: Expect some performance drop with fully local models.
docker build -t aide .
docker run -it --rm \
-v "${LOGS_DIR:-$(pwd)/logs}:/app/logs" \
-v "${WORKSPACE_BASE:-$(pwd)/workspaces}:/app/workspaces" \
-v "$(pwd)/aide/example_tasks:/app/data" \
-e OPENAI_API_KEY="your-actual-api-key" \
aide data_dir=/app/data/house_prices goal="Predict price" eval="RMSE"git clone https://github.com/WecoAI/aideml.git
cd aideml && pip install -e .If you use AIDE in your work, please cite the following paper:
@article{aide2025,
title={AIDE: AI-Driven Exploration in the Space of Code},
author={Zhengyao Jiang and Dominik Schmidt and Dhruv Srikanth and Dixing Xu and Ian Kaplan and Deniss Jacenko and Yuxiang Wu},
year={2025},
eprint={2502.13138},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2502.13138},
}