pi-cua-integration

Cua (trycua/cua) computer-use integration for the pi coding agent.

pi-cua-integration exposes Cua sandboxes (local Docker / QEMU / Lume / Tart and optional cloud) plus unsandboxed Localhost host control as Pi tools and skills. Local mode is the default. No Cua account is required to use it.

Modes

Mode	Where actions run	Sandbox	Needs API key	Runtimes
local ★	Local container or VM	yes	no	Docker (XFCE/Kasm), QEMU, Lume, Tart
localhost	Your host machine	no	no	cua-auto (pynput + Pillow/AppKit)
cloud	cua.ai-hosted VM	yes	CUA_API_KEY	Cua cloud regions

★ default. If mode: "cloud" is selected but CUA_API_KEY is missing, the extension falls back to local with a warning.

Quick start

# 1. Install
pi install npm:pi-cua-integration
pip install cua

# 2. (Optional) project policy
mkdir -p .pi && cat > .pi/cua.jsonc <<'EOF'
{
  "mode": "local",
  "local": {
    "runtime": "docker",
    "image": { "os": "linux", "kind": "container" }
  }
}
EOF

# 3. Run pi
pi

When the session starts you should see [pi-cua] ready (mode=local, ...).

Tools

Tool	Purpose
cua_sandbox_start	Start (or reconnect) a sandbox
cua_sandbox_stop	Destroy a sandbox
cua_sandbox_list	List active sandboxes
cua_screenshot	Capture a PNG screenshot
cua_click	Click at (x, y)
cua_type	Type text
cua_key	Press a key chord (ctrl+s, Return, etc.)
cua_scroll	Scroll at coordinates

See docs/TOOLS.md for full schemas and examples.

Skills

The extension contributes five markdown skills via the resources_discover event. Pi's skill loader can auto-discover them, and the agent learns when to use the tools without prompt engineering.

Skill	When the agent loads it
cua-overview	Anytime any cua_* tool comes up
cua-local-sandbox	Local Docker/QEMU/Lume sandbox details
cua-localhost	Unsandboxed host control safety notes
cua-cloud-sandbox	Cloud (cua.ai) sandbox configuration
cua-control	Mouse / keyboard / scroll primitives

See docs/SKILLS.md.

Configuration

Policy lives in JSONC files (project beats global, key-by-key merge):

• .pi/cua.jsonc - project policy
• ~/.pi/cua.json - global user policy

Schema: schema/cua.schema.json. Regenerate with npm run generate:schema.

See docs/CONFIG.md for the full schema and annotated examples.

Environment variables

Variable	Default	Purpose
CUA_API_KEY	unset	Required for mode: "cloud". Env name configurable.
CUA_TELEMETRY_ENABLED	false (forced off)	Cua opt-out; this extension forces it off unless overridden.
ANTHROPIC_API_KEY etc.	unset	The pi agent that drives cua reads these for its own LLM calls.

Architecture

+--------------------------------------------------+
|  pi-mono session                                  |
|  +---------------------------------------------+  |
|  |  pi-cua-integration extension (TypeScript)   |  |
|  |   - skills via resources_discover            |  |
|  |   - tools via pi.registerTool                |  |
|  |   - /cua command via pi.registerCommand      |  |
|  |                ^                             |  |
|  |                | JSON-RPC over stdio          |  |
|  |                v                             |  |
|  |   python/daemon.py (Python subprocess)       |  |
|  |   - manages Sandbox/Localhost instances      |  |
|  |   - delegates to cua + cua-auto              |  |
|  +---------------------------------------------+  |
+--------------------------------------------------+

Documentation

• docs/MODES.md - local / localhost / cloud mode details
• docs/TOOLS.md - tool reference
• docs/SKILLS.md - skill bundle
• docs/CONFIG.md - JSONC schema and examples
• docs/SECURITY.md - threat model and safety notes
• CHANGELOG.md - release history
• AGENTS.md - conventions for AI agents working on this repo

Development

npm install
npm run check
npm test

Manual QA scripts live in scripts/qa-*.sh once written. The Python daemon contract is verified by test/integration/python-daemon.test.ts (gated by PI_CUA_INTEGRATION_TEST=1).

License

MIT - see LICENSE and NOTICE.

Related

• trycua/cua - the upstream Cua SDK
• senpi - the fork/runtime this extension targets
• pi-anthropic-computer-use - register Anthropic's native computer tool (compatible side-by-side)