LeanToken

Token-bounded repository context for coding agents

Give agents the source they need without repeatedly sending whole files.

Install · Why LeanToken · Tools · CLI · How it works · Docs

---

Quick start

Add LeanToken to Claude Code, Cursor, OpenCode, Codex, Gemini CLI, or Antigravity:

npx leantoken setup

Current releases stop setup before writing when npx resolves a stale project-local or ancestor install, and point to npx leantoken@latest setup. Older releases that predate this check can be bootstrapped directly with that versioned command.

The setup wizard labels supported clients it detects, but leaves every client unselected so you choose exactly which coding agents receive LeanToken. Before writing anything, it shows the exact configuration paths and MCP launcher and asks for confirmation. An npx-based setup pins the exact LeanToken version that ran setup, so restarting a client cannot silently move to a newer release.

Global setup never stores the repository where setup happened. OpenCode gets a workspace-relative working directory; other supported clients launch LeanToken from the workspace cwd selected by the host. If a host instead starts it from the home directory or a filesystem root, LeanToken refuses to index that broad root by default.

Restart or reload the configured clients, then verify the complete MCP handshake and first retrieval from a repository:

npx leantoken doctor

Start with a broad task such as: Use LeanToken to map the relevant repository context before editing. The MCP initialization guidance routes the agent to leantoken.context first and keeps native tools available for edits, builds, and tests.

Local by default Source is indexed on your machine in SQLite. LeanToken is a read-only discovery and retrieval layer.

Explicit token budgets Every source response is bounded, so repository context does not quietly crowd out the task.

Built for agent workflows Browse paths, search identifiers, inspect outlines, read exact ranges, trace symbol history, query structured JSON, and inspect cumulative token accounting through eight focused MCP tools.

To skip the wizard, select clients explicitly or configure all supported clients:

npx leantoken setup --claude --codex --yes
npx leantoken setup --all --yes

Use --private-runtime to copy the exact package-native executable into LeanToken's versioned application-data directory and configure clients to launch it directly, without persistent npm/Node wrappers. Preview its path and digest with --dry-run.

Automation never treats detection as consent: --yes requires explicit client flags, --all, or --refresh for entries already managed by LeanToken. Preview the same resolved plan without changing files:

npx leantoken setup --codex --cursor --dry-run

Setup adds the leantoken MCP entry plus a small owned discovery skill in the host-standard user skill directories. The skill advertises routing metadata; it does not duplicate tool schemas, add rules, or install shell hooks. Remove the owned integration with:

npx leantoken remove

Refresh only existing LeanToken MCP entries after explicitly choosing a new version, or use an older version to roll back:

npx --yes leantoken@latest setup --refresh --yes
npx --yes [email protected] setup --refresh --yes --allow-outdated

Why LeanToken

Repository exploration often starts with broad searches, whole-file reads, and the same source being loaded again after a handoff. LeanToken replaces that loop with progressive disclosure:

Typical repository exploration	With LeanToken
Scan broad directory listings	Browse a compact, ignore-aware tree
Read entire files to find structure	Request signatures, definitions, imports, and ranges
Repeat searches after handoffs	Suppress unchanged evidence with content hashes
Let source reads grow with file size	Apply an explicit token limit to every retrieval
Guess which files matter	Rank task-specific evidence when scope is uncertain

The host agent still owns editing, commands, tests, conversation state, and model orchestration. LeanToken handles repository discovery and bounded source retrieval.

Available tools

Tool	Purpose
leantoken.context	Default first call for broad tasks; preview or materialize ranked evidence under a token budget.
leantoken.search	Prefer over grep/rg for ranked text, regex, identifier, symbol, or reference search.
leantoken.files	Prefer over find/ls/glob for compact, ignore-aware path discovery.
leantoken.outline	Inspect definitions, signatures, imports, and ranges without whole-file reads.
leantoken.read	Prefer over cat/head/sed for one exact symbol or inclusive line range.
leantoken.history	Read, diff, or trace one parsed symbol across immutable Git revisions.
leantoken.json	Query, summarize, or compare bounded live JSON with paged keys and typed diagnostics.
leantoken.savings	Report source compression and cumulative full-response token accounting.

Every index-backed retrieval tool accepts consistency: "reconcile_working_tree" when completed edits must be reconciled before the query. The default, "indexed_generation", returns the latest completed index generation without scanning or waiting for filesystem changes; it is not a Git revision boundary. leantoken.history reads immutable Git objects and leantoken.json reads exact live files, so neither accepts an index consistency mode. To constrain context to immutable history, pass BASE..HEAD as leantoken.context.base_revision with strict_changed_paths: true.

For an uncertain broad task, set plan_only: true to receive bounded ranked candidate metadata without source fragments or receipt mutation. Confirm the paths and coverage, then repeat the same request with plan_only: false to materialize the selected source. Context omission diagnostics are compact by default; set verbose_diagnostics: true only when full path, file-type, reason, score-band, focus, and changed-path facets are needed.

The catalog stays intentionally small because every tool description and schema also consumes model context.

CLI usage

Run LeanToken directly through npx:

npx leantoken status
npx leantoken savings
npx leantoken doctor
npx leantoken --root /path/to/repo search handle_request

Or use a globally installed binary:

npm install --global leantoken@latest

leantoken --root /path/to/repo index
leantoken --root /path/to/repo search handle_request --mode identifier --max-tokens 800
leantoken --root /path/to/repo context \
  --task "fix request cancellation during shutdown" \
  --budget 2000

npm install leantoken installs the command in the current project's node_modules/.bin; it does not add leantoken to the shell PATH. Invoke a project-local install through npx leantoken, a package script, or ./node_modules/.bin/leantoken.

Run the MCP server manually over stdio:

leantoken --root /path/to/repo mcp

Manual MCP client configuration

{
  "mcpServers": {
    "leantoken": {
      "command": "leantoken",
      "args": ["--root", "/path/to/repo", "mcp"]
    }
  }
}

Installation options

The npm package includes native binaries for:

• macOS on ARM64 and x64
• glibc Linux on ARM64 and x64
• Windows on x64

Installation does not run lifecycle scripts or download an executable from a postinstall hook. Other targets, including musl Linux, must build from source. Install Rust 1.95 or later and a native C/C++ toolchain, then run:

cargo install --git https://github.com/morluto/leantoken

Updating

MCP entries created through npx are pinned to the exact version that ran setup. They change only when setup is run again for selected clients or with setup --refresh. The configured launcher may contact npm to obtain that exact package, but it never falls forward to @latest. Removing the npm cache while offline can therefore make startup fail rather than execute an unapproved version.

For a globally installed CLI or a CLI installed with Cargo:

leantoken upgrade --check
leantoken upgrade --yes

update is an alias for upgrade. For a project-local npm installation, update the dependency with npm:

npm install leantoken@latest

A persistent CLI upgrade leaves existing MCP entries unchanged so exact pins and intentional rollbacks remain stable. After a successful upgrade, LeanToken prints the exact version-pinned setup --refresh --yes command for users who want to advance only their existing registrations.

Cache management

LeanToken keeps one SQLite cache per canonical repository in the platform cache directory. Inspect usage and preview an age- or size-based cleanup before applying it:

leantoken cache list
leantoken cache prune --older-than 30 --dry-run
leantoken cache prune --max-total-bytes 1073741824 --yes

Active MCP leaders and followers hold a lifetime lease and are skipped. A missing repository is retained unless it also meets another criterion or --remove-missing-roots is passed explicitly, because removable and offline volumes can be temporarily unavailable. Cache commands never inspect or delete an explicit --database outside the managed cache directory.

How it works

repository
    │
    ▼
ignore-aware discovery ──► syntax extraction ──► SQLite + FTS5 index
                                                    │
                                                    ▼
agent request ──► ranked / exact retrieval ──► token-bounded evidence

LeanToken indexes source once, then serves compact paths, ranked matches, structural outlines, exact source ranges, and task-specific context. Content hashes reduce repeated evidence across turns and model handoffs.

The primary metric is useful repository evidence delivered per model token.

Documentation

Guide	Contents
Usage and tool reference	Commands, MCP tools, request options, and examples
Architecture and reliability	Components, data flow, storage, and failure behavior
Roadmap	Current direction and planned work
Development and testing	Local setup, validation, and release workflow
Benchmark methodology	Token-economy measurements and interpretation
Measurement harnesses	Experiment, wire-cost, and profiling tools

License

Licensed under either of the following, at your option:

• Apache License, Version 2.0
• MIT License