Research and benchmarks

Can many computer-use training attempts share one running computer without touching each other?

This is the public record of what Drums has tried, what broke, what was measured, and what has not been run yet. Nothing here is a claim of speed over anything until the matched numbers exist.

verified 4preliminary 7failed 9planned 7All records

Latest (2026-10-28): our first head-to-head benchmark, in plain language: where Drums wins, where it loses, and what is next.

The question

Training a computer-use agent by reinforcement learning needs thousands of isolated desktop environments, usually one virtual machine each. Every VM duplicates the operating system and the applications, which makes resets, checkpoints and scaling expensive. Drums is investigating a shared-runtime architecture: many attempts share one booted operating system and one set of initialised applications while keeping independent files, clipboard, display and input. The goal is more correctly verified training rollouts per dollar, faster resets, lower overhead, and isolation that holds for training correctness. This is an experimental thesis, not a proven improvement, and the evidence so far includes a pre-registered negative result and an unresolved restore failure.

The idea under test: one shared base runtime, private state per attempt. Not a system that exists.

Isolation and sharing pull against each other. Every experiment on this site is a measurement of where that line can sit without a reward being wrong.

The three experiments

Runtime correctness and efficiency against tuned incumbents. Faithful checkpoint and restore, clean resets, isolation, memory per environment, and desktop throughput, compared with QEMU/KVM warm VM snapshots as the primary baseline, an OCI desktop with CRIU, the custom Drums runtime, and an OSGym or CUA-Sandbox style optimised implementation where it is reproducible and licensed. Equivalent apps, OS versions, tasks, checker, hardware allocation and reset semantics; every difference recorded.
Real-agent rollout economics. Real multimodal agents on a frozen set of about 20 to 30 desktop tasks using genuine benchmark task definitions and external checkers where supported. Measured: completed and failed-but-valid trajectories, invalid or corrupted ones, task success, inference time and cost, environment time and cost, verification cost, recovery overhead.
Actual RL with only the environment changed. Identical open-weight models, algorithm and rewards; only the runtime changes; a pre-registered target success rate; cost and time to reach it; multiple seeds; "never reached" reported honestly.

The third experiment cannot be interpreted until the first two pass. A learning-curve difference on a runtime whose restore is unfaithful or whose isolation is unmeasured is a difference in noise.

Targets exist (two times valid rollouts per hour, half the environment cost per rollout) and they are targets, not results. They were not chosen to be attractive, and the economic threshold that matters should be set by prospective customers, not chosen for a headline. Reset correctness of 100% and zero cross-rollout contamination are requirements, not targets.

Research status

As of 2026-10-25.

Verified (software evidence, not benchmark results)

2026-10-22. The macOS sessions-per-Studio harness builds and both test suites pass on the target Mac Studio. EXP-001

Verified (harness validation, both arms Drums, not a comparison)

2026-10-24. Branching a real task setup state: a LibreOffice Writer launch restored 3.6x faster than cold plus setup; the snapshot-API bridge preserved and branched task state on real sessions (measured, one dev sandbox). EXP-021
2026-10-24. Crash fault windows: a SIGKILL at each of 22 registered points, 22 of 22 clean in three runs after one failed run; orphan cleanup after a supervisor crash (measured). EXP-022
2026-10-24. OSWorld task survey: 13 of 369 public tasks pass a static filter for the Drums rootfs; a manual audit leaves 1 usable and 1 conditional (static analysis). EXP-025

Preliminary

2026-10-23 to 24. Linux L0 spikes, the first-check driver smoke and per-session cgroup limits: real sessions, no incumbent, no cost number. EXP-024, EXP-023
2026-10-22. Day 0 host setup on the Studio and a host baseline with no environment logged in; two environment accounts created. EXP-002
2026-10-25. Shared-OS spike in an authoring sandbox: a private-namespace virtual-display tree launched, probed and cleaned; CRIU absent, so no checkpoint. EXP-006
2026-10-25. CRIU 4.3 loses a POSIX lock on a mapped file across restore; a patch candidate exists; unresolved. EXP-009
2026-10-25. OCI reference arm (runc 1.5.2 plus CRIU 4.3) restores with a broken network namespace; cause read from runc source; a holder fix shown on a minimal container. EXP-010
2026-10-25. Bedrock computer-use adapter: early real-endpoint calls failed; a reported five-call success has no artifact. EXP-012

Failed

2026-10-23. Positive control on the Studio: six attempts, none ran past its prerequisites. No contamination detector has been shown to fire. EXP-003
2026-10-23. L1 fork prototype on Linux: sharing the server process by fork saved 6.2% memory per added session against a pre-registered 15% bar (measured); forking a live GUI process broke in five concrete ways. EXP-004
2026-10-24. Mechanism B on Linux: one shared Chromium with a profile per session cut memory 54% (measured) and failed isolation: a sibling read another session's secret and ran script in its page. EXP-019
2026-10-24. Mechanism K on Linux: kernel page merging saved 7% with ASLR on and 41% only with ASLR off, at 0.7 to 0.9 vCPU against a 0.25 bar, and opened a cross-session timing channel (measured). EXP-020
2026-10-24. Mechanism C on Linux: CRIU restore of a warm session with the required hygiene reseed was 1.76x faster than cold against a 2x bar (measured). EXP-005
2026-10-24 to 25. Drums full-root sessions on the OSWorld image with GNOME: a CRIU 4.3 dump failed on configfs and the pre-dump capture was black (author-reported). EXP-026
2026-10-25. Custom rootfs runtime: CRIU dump and restore attempts of a GNOME tree; the lock gate failed on the first full cycle. EXP-008
2026-10-25. QEMU/KVM warm-snapshot harness, first real OSWorld smoke run: pre-check 0, post-check 0; the keystroke script renamed nothing. EXP-007
2026-10-25. First paid AWS host launch terminated by its own heartbeat backstop before the first heartbeat. EXP-011

Planned

Real-agent rollouts on matched arms through Bedrock. EXP-013
Open-weight agent serving ladder. EXP-014
Matched baseline vs candidate comparison with the benchmark harness. EXP-015
Matched incumbent runs on AWS: stock VirtualBox on bare metal, Docker/KVM small stage. EXP-027
macOS probes, gates, sweep, soak and cost per valid rollout (the experiment 1 headline). EXP-016
Linux VM with N virtual displays on the same Mac. EXP-017
RL training A/B with only the environment changed. EXP-018

Current blockers

CRIU POSIX lock fidelity. A restored tree does not hold a lock on a file it also maps. Until a restored tree holds every lock it was dumped with, no custom-Drums Linux performance comparison is run or reported. EXP-009
The positive control must fire. No macOS isolation number is published until the detector has been shown to fire. EXP-003
On-screen permission grants. Each macOS environment session needs a human login and Accessibility and Screen Recording clicks after every boot; the Studio is FileVault-enabled and institution-managed, so a reboot needs a person at the machine.
The first AWS launch must complete. The Linux baseline needs a KVM-capable host; the first launch was killed by its own backstop. EXP-011

What is verified vs preliminary

Claim	Status	Where
The macOS harness and its tests work on the target Mac	verified software evidence	EXP-001
Forking a shared server process saves 6.2% memory per session, not 15%	measured, pre-registered negative	EXP-004
CRIU 4.3 drops a POSIX lock on a mapped file after restore	author-claimed probe in a sandbox; unresolved	EXP-009
runc always asks CRIU for an empty network namespace	read from source; live result author-claimed	EXP-010
Any density, throughput, reset, isolation or cost number for any arm	none exists	benchmarks
Drums is faster or cheaper than a VM	not claimed

How a number reaches this site: raw logs in a results folder, a result file validated by schema at build, an experiment record that names hardware, commit and evidence paths, and only then a table. Author-claimed numbers without an artifact stop at the record and are quoted as claims.

How to read this site

Every record carries one of four statuses. verified: backed by measurement files we can point to, on named hardware, with the run recorded. preliminary: a real observation that needs more runs, a positive control or an independent check. failed: ran and missed a correctness, isolation, reset or pre-registered bar; kept on the record. planned: written as code or protocol, not executed, no numbers. When in doubt we use the lower status.

Every number carries a label the first time it appears: measured, author-reported (a paper), author-claimed (text with no artifact), vendor claim, design value, or estimate.

Experiment records: every run, in ID order, including the failures.
Benchmarks: measured numbers only, generated from validated result files.
Research notes: the model, the loop, checkpoint and restore, macOS isolation, prior art, verifiers, open questions, glossary.
Methodology: workloads, baseline fairness, gates, cost accounting, statistics, reproduction.
Source: github.com/sahiljagtap08/drums.