# pyscriptc - Full Context > Generated by docmd Developer and agent documentation for pyscriptc: compile annotated Python to native binaries. Island is the floor and native is the ceiling. --- ## [Examples](https://pyscriptc.pages.dev/docs/examples/) --- title: Examples description: Small programs that compile and run today. --- Programs that compile and run today live in the repository: - [Examples folder](https://github.com/sigilco/pyscriptc/tree/master/examples), with an [index](https://github.com/sigilco/pyscriptc/blob/master/examples/README.md) of what each program shows. - [Benchmarks folder](https://github.com/sigilco/pyscriptc/tree/master/benchmarks), small kernels and stdlib calls that compile and run on every CI push. - [Scripts folder](https://github.com/sigilco/pyscriptc/tree/master/scripts), the repo's own tooling, which is compiled and exercised in CI where possible. The per-program walkthrough, with the command and the value each one prints: [Example programs](programs.md). --- ## [Example programs](https://pyscriptc.pages.dev/docs/examples/programs/) --- title: Example programs description: Programs in the repository that compile today, with the value each one prints. --- # Example programs The programs below, from [`examples/`](https://github.com/sigilco/pyscriptc/tree/master/examples), are an invoice, an access-log gate, a log token, a release note, a record digest, a URL allow-list, a route label, and a hostname lookup. Each one compiles, and the printed value matches CPython. `--args` passes `int`, `float`, `bool`, and `str`. Lists and bytes are built inside the function. ## Invoice `examples/invoice.py` bills a meter and totals cents with a floor-divided tax. A negative line returns `-1`. ```bash uv run pyscriptc compile examples/invoice.py --run --call charge --args '[10, 2, 50]' -o /tmp/charge uv run pyscriptc compile examples/invoice.py --run --call invoice --args '[10]' -o /tmp/invoice uv run pyscriptc compile examples/invoice.py --run --call bad_invoice -o /tmp/bad-invoice ``` `charge` prints `600` (`(10 + 2) * 50`). `invoice` prints `4290` (lines of 1000, 2500, and 400 cents, plus 10 percent). `bad_invoice` prints `-1`. ## Access log `examples/access_log.py` admits one JSON object of strings. GET, a path under `/api`, and status `200` return `0`. Anything else returns `1`. ```bash uv run pyscriptc compile examples/access_log.py --run --call admit \ --args '["{\"method\":\"GET\",\"path\":\"/api/health\",\"status\":\"200\"}"]' \ -o /tmp/admit ``` A POST to the same path prints `1`. A missing field raises `KeyError`. ## Request id `examples/request_id.py` returns the first digit run in a log line. ```bash uv run pyscriptc compile examples/request_id.py --run --call request_id \ --args '["id=88421 status=500"]' -o /tmp/request-id ``` That prints `88421`. A line with no digits prints an empty line. ## Release note `examples/release_note.py` adds whole days to an ISO date, writes the result, and reads it back. ```bash uv run pyscriptc compile examples/release_note.py --run --call stamp \ --args '["2020-01-01", 40, "/tmp/release.txt"]' -o /tmp/stamp ``` That prints `2020-02-10`, and `/tmp/release.txt` contains the same text. ## Record digest `examples/blob_digest.py` packs two integers, checks a base64 round trip, and returns the SHA-256 hex digest. ```bash uv run pyscriptc compile examples/blob_digest.py --run --call fingerprint \ --args '[1, 2]' -o /tmp/fingerprint ``` That prints `34fb5c825de7ca4aea6e712f19d439c1da0c92c37b423936c5f618545ca4fa1f`. ## URL gate `examples/url_gate.py` matches a URL path with `fnmatch` and percent-encodes a path. ```bash uv run pyscriptc compile examples/url_gate.py --run --call allow \ --args '["https://example.com/api/health", "/api/*"]' -o /tmp/allow uv run pyscriptc compile examples/url_gate.py --run --call quote_path \ --args '["https://example.com/a b"]' -o /tmp/quote ``` `allow` prints `1`. The same host with pattern `/api/*` and path `/health` prints `0`. `quote_path` prints `/a%20b`. ## Route label `examples/route_label.py` takes the last segment of a path and a status. The status defaults to `200`. ```bash uv run pyscriptc compile examples/route_label.py --run --call label \ --args '["/api/health", 201]' -o /tmp/label ``` That prints `health 201`. Omitting the status prints `health 200`. ## Hostname `examples/hostname.py` runs through embedded CPython (fallback mode). The binary links libpython. ```bash uv run pyscriptc compile examples/hostname.py --run --call hostname -o /tmp/hostname ``` It prints the same name `socket.gethostname()` prints under CPython. ## Word count (a program directory) `examples/wordcount/` is a program: `__main__.py` is the entry point and `tally.py` is imported by its plain name. The whole tree compiles into one binary. ```bash uv run pyscriptc compile examples/wordcount --run -o /tmp/wordcount ``` It prints `three 3`, the word that appears most often in the sentence and how often it appears. --- ## [Getting started](https://pyscriptc.pages.dev/docs/getting-started/) --- title: Getting started description: Install pyscriptc, check the toolchain, and compile your first Python file to a native binary. --- # Getting started pyscriptc compiles typed Python to a native binary. This page gets you from nothing to a running binary in a few minutes. ## Install Run the install script for your platform. It checks the prerequisites, installs `uv` and, with the flag, `zig` where it can, then installs pyscriptc and runs `pyscriptc doctor`. It never uses `sudo` or an admin shell: for anything it will not install itself it prints the exact command to run as your normal user. ::: tabs ::: tab "macOS / Linux" ```sh curl -LsSf https://raw.githubusercontent.com/sigilco/pyscriptc/master/docs/install.sh | sh ``` ::: /tab ::: tab "Windows" ```powershell powershell -ExecutionPolicy ByPass -c "irm https://raw.githubusercontent.com/sigilco/pyscriptc/master/docs/install.ps1 | iex" ``` ::: /tab ::: /tabs The script is a convenience: it does the two commands below and nothing more. To install by hand, use `uv tool install pyscriptc`. From a checkout of the repository, run the script from the file instead of the URL: ```sh sh docs/install.sh # macOS and Linux .\docs\install.ps1 # Windows ``` Pass `--yes` (or `-Yes` on Windows) to let the script install `uv` and `zig` too. ## Prerequisites You need two things: - CPython 3.11 or later, to run the compiler. On an older CPython, `uv tool install --python 3.14 pyscriptc` fetches a suitable interpreter. - The compiled program always targets CPython 3.14 semantics, whatever host runs the compiler. A source the host cannot parse (syntax newer than the host) is a syntax error before the compiler sees it; see [Divergences](reference/divergences.md). - A C toolchain on PATH, to link compiled programs. `zig` also enables cross-compilation. The type checker ships inside the package, so there is nothing extra to install for it. ### C toolchain by platform | Platform | Install a compiler | | --- | --- | | macOS | `xcode-select --install` (or `brew install zig`) | | Debian / Ubuntu | `sudo apt install clang` (or `sudo apt install zig` on newer releases) | | Fedora | `sudo dnf install clang` | | Arch | `sudo pacman -S clang` | | Any Linux or macOS | `mise use -g zig` | | Windows | `winget install zig.zig` (or `scoop install zig`) | You only need one of `clang` or `zig`. Run `pyscriptc doctor` after installing to see what was found. ### No system zig? Install it as a wheel If the host has no zig and no clang, install pyscriptc with the `zig` extra: ```sh uv tool install "pyscriptc[zig]" ``` The extra pulls the `ziglang` wheel, which ships the zig toolchain. The wheel registers only a `python-zig` console script, so pyscriptc installs a `zig` PATH shim over `python -m ziglang` and reports it as `zig: ... [PATH shim for pip install ziglang]` in `doctor`. With that in place `--driver zig` and cross-compilation (`--target`) work with no other setup. The same works with `pipx install "pyscriptc[zig]"` and `pip install "pyscriptc[zig]"`. The extra is the wheel route only: a system zig (`brew`, `winget`, `mise`, the distro package) is found directly and needs no shim. ## Install pyscriptc These pages describe master; the newest released wheel can lag it. `uv tool install pyscriptc` installs the newest release; installing from the repository (`uv pip install git+https://github.com/sigilco/pyscriptc`) installs master. `pyscriptc doctor` prints the installed version, so you can see which one you have. Install pyscriptc as a tool: ```sh uv tool install pyscriptc ``` `pipx install pyscriptc` and `pip install pyscriptc` do the same thing. Prebuilt binaries for Linux, macOS and an experimental Windows build are on the [releases page](https://github.com/sigilco/pyscriptc/releases); the binary still needs `clang` or `zig` on PATH to link programs. Then check the toolchain: ```sh pyscriptc doctor ``` `doctor` prints the pyscriptc version, the checker it found, the C compiler or zig it will link with, and the cache locations. If something is missing, fix it with the table above and run `doctor` again. For an automated check, `pyscriptc doctor --json` prints the same facts as JSON. The `zig.found` and `clang.found` fields answer whether this machine can compile a binary at all. ## First run Create a small typed script: ```python # sum_squares.py def sum_squares(n: int) -> int: total: int = 0 for i in range(1, n + 1): total = total + i * i return total if __name__ == "__main__": print(sum_squares(10)) ``` Compile and run it: ```sh pyscriptc run sum_squares.py ``` `pyscriptc run` compiles once, caches the binary under `~/.cache/pyscriptc/bin`, and runs it. A second run of an unchanged file skips the compile. Arguments after `--` reach the program as `sys.argv`. To compile a function and run it directly: ```sh pyscriptc compile sum_squares.py --run --call sum_squares --args '[10]' -o /tmp/sum_squares ``` `--call` names the function to run and prints its return value, `--args` passes a Python list of arguments, and `-o` names the output binary. Every subcommand and flag is listed in the [CLI reference](reference/cli.md). ## When a compile stops If a file cannot be compiled, pyscriptc prints a table naming each construct and a code such as `PS2003`, and exits with a status that says what happened: | Exit | Cause | | --- | --- | | 0 | Compiled (plus the binary's own status with `--run`) | | 1 | An internal compiler error | | 2 | The type checker rejected the file (or a command-line syntax error) | | 3 | The file uses something pyscriptc cannot compile yet | | 4 | A version or target mismatch | Run `pyscriptc explain PS2003` to see what a code means and how to change your code. The full list of diagnostic codes is in [Diagnostics](reference/diagnostics.md), and the general reference for unsupported behavior is [Divergences from CPython](reference/divergences.md). ## If your app is framework-heavy Reflection-heavy and framework-heavy code (pydantic models, Flask routes, agent frameworks) does not compile to native code. It does not fail either: once an untyped import sits at a module top level, that module's functions are pulled onto the embedded CPython island with it, and the program runs there. What it costs is the island: those parts run at interpreter speed, and every call crossing between a native function and an island one pays a marshalling cost. The fences that remain in your own code (a missing annotation, an `Any` annotation, a construct outside the subset) still stop the build, and each names the line to change. Before building, run `pyscriptc assess` on the tree and read the plane map it prints: what runs native, what runs on the island and why, and the policy lines that route each unsupported import. The rules are in [What runs native and what runs on the island](guide/python-subset.md#what-runs-native-and-what-runs-on-the-island), the island mechanics and what a crossing costs in [Libraries and fallback mode](guide/libraries.md#the-two-planes), and how this compares with other tools in [Comparison with other tools](guide/comparison.md). --- ## [Comparison with other tools](https://pyscriptc.pages.dev/docs/guide/comparison/) --- title: "Comparison with other tools" description: "How pyscriptc relates to Nuitka, Cython, mypyc, Codon, Numba and scriptc, by what each project's own documentation says." --- Several tools compile or accelerate Python. They differ in what they take as input, what they produce, and how close they stay to CPython semantics. This page states the differences without claiming one is faster than another: the projects measure themselves differently, and pyscriptc publishes its own numbers on the [landing page](https://pyscriptc.pages.dev/) instead of comparing here. The one measured cross-tool row on this page is the micro-benchmark at the bottom, run on one VPS with the code attached; treat it as an anecdote, not a ranking. ## At a glance | | Input | Output | Types | CPython semantics | | --- | --- | --- | --- | --- | | **pyscriptc** | typed Python subset | native binary; untyped imports run on an embedded CPython island | annotations required, checked by pyrefly before lowering | subset; deviations are fenced with a `PSxxxx` diagnostic, numbered divergences documented | | **CPython** | standard Python (baseline) | interpreted bytecode, needs an interpreter | not needed | the reference | | **PyPy** | standard Python | JIT inside its own interpreter process | not needed | compatible with caveats (C-extension support through `cpyext`, slow first call) | | **Nuitka** | standard Python | extension module, standalone folder or onefile executable | not needed | aims for full compatibility, including Python 2.6/2.7 modes | | **Cython** | Python or the Cython language (`.pyx`) | CPython extension module (`.so`) | optional; static types add speed and C interop | its own compiled language; documented limitations page | | **mypyc** | typed Python | CPython extension module per compiled module | standard hints, enforced at runtime (`TypeError` at boundaries) | mostly compatible; restricts some dynamic features | | **Codon** | Python-like source | native executable or extension module | not required; its own Hindley-Milner inference, monomorphization | explicitly not a drop-in replacement (64-bit `int`, no monkey-patching, C-style numerics by default) | | **Numba** | decorated Python functions | JIT-compiled machine code in-process | not required; argument types inferred at call time | nopython-mode subset with a documented supported-features list; object mode covers the rest | | **Shed Skin** | implicitly statically typed Python | C++ extension module or standalone | none written; types inferred and must not vary per variable | restricted subset, standard-library slice documented | | **scriptc** | TypeScript/JavaScript | native executable and WASI module | standard TypeScript via `tsc` | not Python; listed here for the architectural parallel | ## The tools ### CPython The baseline: bytecode interpreted on a stack machine, every dynamic feature, every C extension, no build step. Distribution means shipping an interpreter or depending on one being installed. Every tool on this page exists to trade some of that flexibility for speed or a self-contained artifact. ### PyPy [PyPy](https://www.pypy.org/) runs standard Python on its own JIT-compiling virtual machine. No annotations, no build step, a drop-in interpreter replacement; the program speeds up after warmup while the first iterations pay JIT and garbage-collection tuning costs. Its known weak spot is C extensions: the `cpyext` compatibility layer works but costs more than CPython's, so NumPy-class wheels are the ecosystem pain point (the Fastest CPython/PyPyson pages and FAQ document this). Distribution is an interpreter runtime, not a binary of your program. pyscriptc differs in the artifact (a compiled binary vs an interpreter runtime) and in refusing, at compile time, the dynamic patterns PyPy handles at run time. ### Nuitka Nuitka translates a whole Python program into a C-level program that links `libpython`, so the compiled result executes the way CPython does. It compiles standard Python as-is: no type annotations are needed, and its docs claim full compatibility with Python 3.4 through 3.14 (plus Python 2.6/2.7), with optimizations that never remove compatibility. Its three modes are `--module` (an importable extension), `--mode=standalone` (a distributable folder with the runtime bundled) and `--mode=onefile`. The typical use case in its docs is distributing applications. pyscriptc differs in the input contract and the output: it compiles a typed subset, refuses the rest with a named diagnostic instead of lowering it slowly, and a native binary carries no interpreter except where you opt into the island. ### Cython Cython compiles `.pyx` (or plain `.py`) files to C, and a C compiler turns that into an extension module that you import back into a CPython session. Static typing is optional; adding it speeds up hot paths and enables C/C++ interop. The language is a superset of Python with its own limitations page, and the output is always an extension module, not a standalone program. pyscriptc differs in output shape (a binary instead of a module for CPython to import) and in making annotations mandatory rather than an accelerator. ### mypyc mypyc compiles Python modules with standard type hints into C extensions, using mypy for checking and inference. It is a strict, gradually typed Python variant: dynamic features are restricted to gain performance, and annotations are enforced at runtime with `TypeError` at static/dynamic boundaries. Output is one extension module per compiled module (or a whole package via `mypycify`), which mypy itself uses to speed up its own codebase. This is the closest input contract to pyscriptc's: typed Python in, checked before lowering. The difference is the artifact and the boundary. mypyc produces code that runs inside a CPython process; pyscriptc produces a binary, and unknown or untyped imports route to an embedded CPython island rather than to `Any`-erased compiled code. ### Codon Codon, from Exaloop, is an ahead-of-time compiler for Python-like source with its own type checker (a modified Hindley-Milner system) and LLVM backend. Annotations are not required because the compiler infers types and monomorphizes per call site. It builds native executables and extension modules, and its docs state plainly that it is not a drop-in replacement for CPython: `int` is 64-bit rather than arbitrary precision, static checking forbids monkey-patching and heterogeneous containers, numerics follow C semantics by default, and unimplemented stdlib modules run under a real CPython via `from python import`. pyscriptc shares the "native binary, no interpreter inside" goal but keeps CPython semantics as the target within the subset: an `int` is a Python `int`, and anything the subset cannot compile is refused with a diagnostic or routed to an embedded CPython, never reinterpreted. ### Numba Numba is a JIT: you decorate functions with `@numba.jit` (or `@njit`), and LLVM-compiled machine code is produced at first call, with argument types inferred then. It is aimed at array and numerical code, with deep NumPy integration and CPU/GPU targets, and its `nopython` mode supports a documented subset of Python while object mode catches the rest. Compilation happens inside a running CPython process; there is no standalone binary. pyscriptc is ahead-of-time and whole-program: the compile happens once, before distribution, and the output runs with no CPython in the loop outside the island. ### Shed Skin [Shed Skin](https://shedskin.readthedocs.io/) is an experimental compiler from implicitly statically typed Python to C++. No annotations are written, but the inference requires each variable to hold one type, and the supported standard-library slice is documented per module. Output is a CPython extension module or a standalone binary; the project describes itself as experimental. pyscriptc differs in demanding the types be written (they steer the lowering instead of being inferred once) and in routing what it cannot compile to embedded CPython rather than leaving the program without a path. ### scriptc [scriptc](https://github.com/vercel-labs/scriptc) (Apache-2.0) is Vercel Labs' experimental compiler for a related problem in a different language: it compiles TypeScript and JavaScript to native executables and WASI modules, with the real `tsc` checking types and unsupported operations fenced with an `SC` diagnostic plus a rewrite hint. It also ships a `scriptc coverage` command that reports, per construct, what compiles. pyscriptc is an independent project, but the shape of the problem is the same one scriptc describes for the Node ecosystem: a typed subset lowered through LLVM IR to native code, with everything outside the subset routed to a dynamic engine instead of guessed at. The architectural difference comes from the languages: scriptc's dynamic fallback runs on an embedded QuickJS, while pyscriptc embeds CPython itself, because Python's runtime introspection is idiomatic in its ecosystem (frameworks read `__annotations__` and build schemas at import time) and only real CPython provides it. pyscriptc keeps that island as a permanent tier rather than a transition state, and credits scriptc as the closest prior art for the coverage-diagnostic developer experience. ## Choosing - You want to ship your existing, untyped script as an executable and need full-language behavior: Nuitka's territory. - You want to run untyped code faster without a build step and accept interpreter-runtime distribution: PyPy. - You want to speed up a module inside a CPython process: Cython or mypyc. - You want a native binary from numeric, statically-inferable code and accept its numeric and typing divergences: Codon. - You want decorated numeric/array functions to run faster inside a running interpreter: Numba. - You have typed Python and want a native binary that refuses, names, and explains what it cannot compile, with untyped dependencies delegated to an embedded CPython: that is pyscriptc. ## A micro-benchmark, with the code One task, one machine, one afternoon: count the words of a 44-character sentence repeated 20,000 times, 20 passes, timing the whole loop (`perf_counter`). This is a dict-update and string-splitting loop, not a numeric kernel, so it favors none of the tools in particular. The machine is a 6-vCPU Haswell-class VPS; each tool ran the same logic, and the times below are the best of 2-3 runs after warmup. The code is inline so the comparison can be checked; treat this as an anecdote about tool classes, not a ranking. Task code (the untyped spelling; the typed spelling adds `str`/`dict[str, int]`/`int` annotations only): ```python import time def count_words(text): counts = {} for word in text.split(): counts[word] = counts.get(word, 0) + 1 return counts def main(): text = "the quick brown fox jumps over the lazy dog " * 20000 t0 = time.perf_counter() total = 0 for _ in range(20): total += sum(count_words(text).values()) dt = time.perf_counter() - t0 print(total, dt) ``` | Tool | What ran | Version | Time (s) | Notes | | --- | --- | --- | --- | --- | | CPython 3.14 | same code | 3.14.9 | 0.96 | baseline | | PyPy | same code, untyped | PyPy 8.0.1 (Python 3.12.15) | 0.67 | warmup included | | mypyc | typed spelling, compiled to an extension | mypy/mypyc 2.4.1 | 1.04 | little gain on a dict-heavy loop | | Cython | same code, `cythonize` to an extension | Cython 3.3.1 | 0.72 | | | pyscriptc | typed spelling, native binary | master `9d6c831` | 0.84 | plane map: all functions native, no island | A second, numeric-shaped loop (sum `i % 7` over 840,000 elements, 20 passes, typed integer only) shows the class difference between dict-heavy application code and tight numeric loops: numba 0.68.1's `@njit` ran it in 0.038 s against CPython's 1.66 s, about 40x, because the loop lowers to machine code with no container operations; the same dict-heavy task above stays within one order of magnitude for every tool. numba only reaches numbers like that on code it can type and lower in `nopython` mode; it does not compile the dict/string task at all. Versions and runner code: the drivers were `python typed.py` (CPython, PyPy, the mypyc and Cython extension modules through a `drive.py` that imports and calls `main()`), and the compiled pyscriptc binary invoked with `--call main`. The mypyc build required the variable annotation `counts: dict[str, int] = {}` to pass its checker; the untyped spelling failed mypy with `var-annotated`. No other tool was benchmarked; everything else on this page is from the projects' own documentation. --- ## [Debugging and logs](https://pyscriptc.pages.dev/docs/guide/debugging/) --- title: Debugging and logs description: Stage timings, refusal tables, IR dumps, and audit logs. --- ## Compiler debug mode `pyscriptc compile FILE --debug` prints a stage table to stderr: parse, check, lower, emit, link, each with wall time. Narrow it with `--debug=stages,fences,cc,paths` or set `PYSC_DEBUG=1`. `--timings` prints the same table on its own. ```bash pyscriptc compile app.py --debug=fences # every unsupported construct with its span pyscriptc compile app.py --debug-dir out/ # dumps ir.json, out.ll, cc.sh, summary.json ``` `--debug-dir` never changes the emitted IR; it only writes the intermediates next to it, so it is safe to diff two runs. The full reference for the debug switches, the stderr topics and the `--debug-dir` artifacts lives in [`.agents/docs/debug.md`](https://github.com/sigilco/pyscriptc/blob/master/.agents/docs/debug.md) in the repo; `pyscriptc compile --help` lists the flags themselves. ## The `PYSC_MIR` backend lane `PYSC_MIR=1 pyscriptc compile FILE` routes the functions the new MIR backend covers through it instead of the default LLVM emitter. It is an escape hatch for comparing the two backends, not a feature switch: the lane is experimental, off by default, and any value other than empty or `0` turns it on for the whole compile. The lane is held to byte-identical output, so on a native target a program behaves exactly as without it. Only scalar leaf functions (`int`, `float` and `bool` parameters, locals and returns, with no calls, containers, strings or closures) take the lane; every other function and the module's entry point still come from the default emitter. What differs or fails with `PYSC_MIR` set: - A `wasm32` target (`--target wasm32-wasi-musl`, `wasm32-wasi-p2`) emits the program entry point as `@main` instead of `@__main_argc_argv`, so the wasm link fails. Unset `PYSC_MIR` for wasm builds. - `--emit binding` ignores the variable and always uses the default emitter. - Any other difference from the default output is a bug: report it with both `.ll` files (`--emit llvm` with and without `PYSC_MIR=1`). ## Reducer `pyscriptc reduce FILE --fence PS1104` shrinks a failing source to the smallest variant that still reproduces, and writes it to `FILE.min.py`. Use it when a `PSxxxx` code or a crash shows up in a file far bigger than the bug. ```bash pyscriptc reduce big.py --fence PS1104 --json # the removal log pyscriptc reduce big.py --crash # still raises an error that is not a PSxxxx code pyscriptc reduce big.py --predicate 'pytest -x "$PSC_SOURCE"' # your own check ``` The loop is AST-level and monotone: each pass drops one statement the bug does not need, re-checks, and keeps the removal only when the check still holds. No C toolchain is needed. ## Audit logs Every invocation appends JSONL records to `~/.cache/pyscriptc/log/`. INFO records are always on; `-v` / `--verbose` or `PYSC_DEBUG=1` adds DEBUG records for each pipeline stage. ```bash pyscriptc logs # print the log path and its last 50 lines ``` Attach the log tail (or a `--debug-dir` dump) to a bug report and we can usually reproduce without your source. The logging directives are in [`.agents/docs/pyscriptc-debug.md`](https://github.com/sigilco/pyscriptc/blob/master/.agents/docs/pyscriptc-debug.md). --- ## [HTTP](https://pyscriptc.pages.dev/docs/guide/http/) --- title: HTTP description: The native urllib.request client, what it covers, and where it still differs from CPython. --- HTTP goes through a native client: `urllib.request` and `urllib.error` compile to `urlopen` calls served by `ps_http.c`, with no subprocess and no host `curl`. This page shows what compiles, the caveats that remain, and the `curl` escape hatch for the cases the client does not cover. ## What compiles `urlopen(url)` returns an `HttpResponse` with `status`, `headers` and `body` (`bytes`), plus `getcode()`, `info()` and `read()`. `data` sends a POST, `timeout` bounds the whole request and raises `TimeoutError` on expiry. ```python import json import urllib.request def fetch(url: str) -> str: resp: urllib.request.HTTPResponse = urllib.request.urlopen(url) body: bytes = resp.read() return body.decode("utf-8") def status(body: str) -> str: doc: dict[str, str] = json.loads(body) return doc["status"] ``` `json.loads` needs the result annotation (`dict[str, str]` above), and the decoded string must be UTF-8, since `bytes.decode` compiles the `utf-8` encoding only. `Request(url, data=..., headers=..., method=...)` builds the request record that `urlopen` consumes, for methods beyond GET and POST and for custom headers. Redirects (301, 302, 303, 307, 308) follow CPython's method rules, relative `Location` resolution included, with a 10-hop cap. Chunked transfer-encoding is decoded. ## Divergences from CPython - TLS is not there yet: an `https://` URL raises `OSError`. TLS via mbedTLS is [#235](https://github.com/sigilco/pyscriptc/issues/235). - `urlopen` takes `url`, `data` and `timeout`; other keywords are unsupported. Headers go on the `Request`, as a dict display with constant string keys and values; a header dict built at run time stops the compile. - 4xx and 5xx raise `HTTPError` carrying `status`, `headers` and a readable body; malformed URLs and connection failures raise `URLError`. Both catch through the `OSError` chain like CPython, but the error types are synthetic records, not the full `urllib.error` class surface. - Only `urlopen` and `Request` compile; the rest of `urllib.request` (`urlretrieve`, openers, proxies) is unsupported, and `http.client` is not supported (#237). - No streaming: the whole body arrives as one `bytes` value. - These are the numbered divergences 27 and 28 in the [divergence list](../reference/divergences.md). ## The curl escape hatch For the cases the native client does not cover, such as an unusual TLS configuration or a proxy setup, the `curl` bridge still works by running the call through embedded CPython (fallback mode): ```python import subprocess def fetch(url: str) -> str: out: bytes = subprocess.check_output(["curl", "-s", url]) return out.decode("utf-8") ``` It needs `curl` on `PATH` at run time and the libpython-linked binary shape; see divergence 15 in the [divergence list](../reference/divergences.md). --- ## [Libraries and fallback mode](https://pyscriptc.pages.dev/docs/guide/libraries/) --- title: "Libraries and fallback mode" description: "How to use third-party and unsupported modules: embed them with --fallback=cpython or call compiled code from Python via bindings." --- pyscriptc compiles the typed subset of Python 3.14. Every dependency gets one of two execution modes: it compiles to native code when it fits the subset, or it runs through embedded CPython (fallback mode) when it does not. An untyped dependency is never impossible to use: it runs in fallback mode. ## The two planes Every compiled program is split into two execution planes, decided per module and shown by `pyscriptc assess` (the plane map): | Plane | What runs there | What it costs | | --- | --- | --- | | **native** | compiled to machine code; no interpreter in the loop | the subset's speed and binary size; anything outside the subset fences the build | | **island** | an embedded CPython 3.14 interpreter in the same process | interpreter speed for those parts, plus a marshalling cost on every native-island call crossing | The split is not a per-package verdict of "works or not": a program can run entirely on the island tier and still produce correct output, at interpreter speed, in a single binary that needs no system Python. What the split decides is how much of the program gets native speed. `pyscriptc assess --json` prints the planned route per definition (`planeMap`, with `demotedBy` naming the import or call that pulled a function onto the island) and `pyscriptc run --report=report.json` prints the route that a run actually took. A fence (a `PSxxxx` diagnostic) is the compile-time signal that a construct cannot be lowered; the island is the run-time answer for the code that a fence would otherwise stop. Support status per package, including which packages are measured to run on which plane, is on the [compat site]({{PYSC_COMPAT_URL}}); this page states the mechanics and the costs. ## Route a module to fallback mode Fallback mode links your binary against `libpython` and runs the unsupported parts in an in-process CPython 3.14 interpreter, while the typed functions still compile to native code. Enable it on the command line: ```bash pyscriptc compile script.py --fallback=cpython -o script_bin pyscriptc run --fallback=cpython script.py ``` `--island` is an alias for `--fallback=cpython`. You can also set it in `pyproject.toml` so no flag is needed: ```toml [tool.pyscriptc.fallback] definitions = true ``` Consider a script that mixes a typed function with an untyped one: ```python import json def helper(x: int) -> int: return x * 2 def parse_data(raw): return json.loads(raw) def main() -> None: data = parse_data('{"count": 21}') result: int = helper(data["count"]) print(f"result: {result}") if __name__ == "__main__": main() ``` Without the flag the compile stops on the unannotated `raw` parameter (`PS1001`). With `--fallback=cpython` the unsupported definitions run in the embedded interpreter. Note that `helper` runs there too: when an unsupported function calls a typed one, the compiler moves the callee into the same interpreter so both run together. ## See what compiles and what does not `pyscriptc assess` reports which definitions compile and which do not, with the fix for each refusal: ```bash pyscriptc assess script.py ``` Its summary counts definitions inside the subset; that is not the share that runs native. On a fallback build the deciding report is the plane map: `assess --json` prints the planned route per function plus `demotedBy` (the call that pulled a function onto the island), and `run --report=report.json` prints the executed route after a run. A module whose top level imports an untyped dependency has its whole top level routed first (divergence 42), so framework-heavy trees can show a high per-definition share with everything island at run time; see [What runs native and what runs on the island](python-subset.md#what-runs-native-and-what-runs-on-the-island). `pyscriptc imports SOURCE` classifies every imported module as `native` or as requiring libpython. `pyscriptc run --fallback=cpython --report=report.json script.py` writes the same classification as JSON after a run. ## The imports table The `[tool.pyscriptc.imports]` table in `pyproject.toml` routes each top-level module name to a lane: `compile` (default), `refuse` (always a compile error naming the policy site), or opt-in fallback entries. Set it when you want the decision recorded in the project instead of on the command line. The full key reference is [Config](../reference/config.md). ## How values cross - Scalars (`int`, `float`, `bool`), `str`, `bytes` and lists of scalars cross by value. A list is copied, so a change made on the embedded side is never visible to native code. - Dictionaries and class instances cross as opaque handles supporting attribute access, subscript lookups, method calls and `str()` formatting. - Exceptions raised in the embedded interpreter are catchable from native code: a native `try ... except ValueError:` catches a `ValueError` raised there. Unknown exception types surface as `RuntimeError` carrying the original class name and message. - `raise SystemExit(code)` out of the embedded side exits the process with that status. ```python def risky(x): y = [x, "tag"] if x < 0: raise ValueError("negative value") return len(y) + x def main() -> None: try: val: int = risky(-5) print(f"val: {val}") except ValueError as exc: print(f"handled ValueError: {exc}") if __name__ == "__main__": main() ``` ```bash pyscriptc run --fallback=cpython script.py # handled ValueError: negative value ``` ## Call compiled code from Python with bindings The other direction works across module boundaries with `--emit binding`, which produces a shared library plus a `ctypes` loader module: ```bash pyscriptc compile typed_math.py --emit binding -o typed_math ``` ```python from typed_math_binding import add print(add(20, 22)) ``` Use this when a plain CPython script, or code running in fallback mode, needs to call compiled functions from another module. ## Environment and limits Fallback mode picks up the host Python environment: - If `VIRTUAL_ENV` is set, its `site-packages` is on the path. - `PYTHONPATH` entries are added. - The directory containing the source file is on `sys.path`, and dependency roots from `pyproject.toml` or `--dependency-root` join both the type-checker search path and the embedded interpreter's path. Limits to know: - Fallback mode provides compatibility, not native speed: the embedded parts run under the interpreter, calls crossing the boundary pay a marshalling cost, and native threads serialize at the GIL. - Closures and generators cannot be passed across the boundary. - Cross-compiling a libpython-linked binary to a foreign target is refused (divergence 31); fallback mode needs the host libpython and its installed packages. - Uncaught exceptions from the embedded side print the type and message but not a traceback (divergences 12 and 42). - Unsupported module-level statements plan the whole module top level for the embedded interpreter (divergence 42). - The island plan propagates through calls: a native-eligible function that names an island-planned import or an unannotated value reaching one is routed there too (divergence 59). On one measured Flask-plus-LLM backend this took the run-time native share from 41% (definitions passing in isolation) to 0% (functions planned island). The complete crossing rules and numbered divergences: [Divergences from CPython](../reference/divergences.md). Per-module and per-library support status: the [compat site]({{PYSC_COMPAT_URL}}). --- ## [Mobile apps](https://pyscriptc.pages.dev/docs/guide/mobile/) --- title: Mobile apps description: Compile Python into a static library and link it into an iOS or Android app. --- pyscriptc can compile a Python module into a static library that a native iOS or Android app links and calls. The library mode exists because a phone app has no use for a standalone binary: the app itself is the executable, and the compiled Python rides inside it as `libpyscriptc.a`. ## The three targets ```bash pyscriptc compile hello.py --emit lib --target aarch64-apple-ios-simulator -o lib pyscriptc compile hello.py --emit lib --target aarch64-apple-ios -o lib pyscriptc compile hello.py --emit lib --target aarch64-linux-android -o lib ``` `-o` names a directory that receives three files: `libpyscriptc.a`, `pyscriptc.h` and `module.modulemap`. The simulator target builds and links from a macOS host with no extra setup. The device target needs Xcode. The Android target needs the NDK and refuses to start without one: point `PYSC_ANDROID_NDK` at the NDK root (the directory that contains `toolchains/llvm/prebuilt`), and optionally set `PYSC_ANDROID_API` (24 by default). ## Calling it from the app The exported surface is exactly two functions, `pyscriptc_init` and `pyscriptc_main`. Everything else is hidden at compile time (`-fvisibility=hidden`), so the archive cannot leak the runtime namespace into the app's symbol table. ```c #include "pyscriptc.h" if (pyscriptc_main() != 0) { /* the module raised or exited nonzero */ } ``` `pyscriptc_init()` sets up the per-process runtime state (the argv list is empty in a library); it is idempotent. `pyscriptc_main()` is the module entry: runtime init, the exception table, the module globals, then the module's own entry. It returns 0 on success. Call `pyscriptc_init()` yourself when the host drives the runtime without the module entry; `pyscriptc_main()` already calls it. A Swift app imports the same pair through the `module.modulemap` beside the archive. Link the archive into the app target, add the header search path, and `import pyscriptc` gives both functions. ## What a mobile target refuses - `--emit bin` and `--run` fail on every mobile triple with `PS4101` (see `pyscriptc explain PS4101`), and `--emit lib` fails on a desktop triple the same way. - `--driver zig` is refused for a library build: zig bundles neither bionic nor the Apple SDKs, so the build compiles through clang with an explicit sysroot. - `subprocess`, `os.open` and the other raw file-descriptor calls stop the compile with the mobile reason (`PS2001`), because the runtime files that implement them are left out of the mobile archive. A call that compiled anyway would only fail later, as an undefined symbol at link. ## Android page-size rule An Android app links the archive into a shared object, and that link carries the 16 KB alignment flags Google Play requires for Android 15+: ```bash -Wl,-z,max-page-size=16384 -Wl,-z,common-page-size=16384 ``` The archive itself has no program headers, so the flags belong to the consumer's link line. `pyscriptc` prints this reminder after every Android archive build. ## Not in the library yet Distribution packaging (XCFramework, AAR, SPM binary target), generated Swift and Kotlin bindings, and host callbacks are separate follow-up work; this page documents the archive and its calling convention. --- ## [Proven patterns](https://pyscriptc.pages.dev/docs/guide/proven-patterns/) --- title: Proven patterns description: Measured cases where pyscriptc paid off, with the command that produced the number, the receipt, and the limit. --- This page records patterns that were measured on real programs, not a feature list. Each pattern states when it fits, the exact command that produced the number, the receipt, and the honest limit. The measurements were taken on macOS arm64 with CPython 3.14.9 as the reference; a different machine moves the absolute numbers, not the shape of them. Where a pattern did not pay off, it says so. The support state of every construct lives on the [compat site]({{PYSC_COMPAT_URL}}); the gaps live in the numbered [divergences](../reference/divergences.md). ## Pattern 1: island-compile an existing CLI into one binary **When it fits.** A pure-Python CLI you want to ship as a single file, where the behavior has to match CPython exactly and the program is too large or too dynamic to lower natively. **How to do it.** Install the program, compile the package in program mode with the island tier on, and compare the output against CPython byte for byte. ```bash uv pip install --target ./ytpkg yt-dlp rm -rf ./ytpkg/yt_dlp/__pyinstaller uv run pyscriptc compile ./ytpkg/yt_dlp --program --island --embed-python host \ --emit bin --skip-check -o ./ytdlp ``` The `--program` flag treats the directory as a program (`python -m pkg` parity) and embeds every module in the binary, so the result runs with no `PYTHONPATH` and no source tree next to it. The `--island` flag (alias of `--fallback=cpython`) routes the definitions the typed subset refuses to the embedded CPython, which is what makes an untyped package compile at all. Non-Python files in the program directory (data files, JavaScript assets) are staged beside the binary under `pyscriptc-data//`. The `rm -rf` line removes `yt_dlp/__pyinstaller/hook-yt_dlp.py`, a packaging hook that is not an importable module; the compiler refuses a program directory that holds one, because the file name is not a chain of identifiers. Verify the result against CPython before you ship it: ```bash ./ytdlp --list-extractors > bin.txt python -m yt_dlp --list-extractors > cpy.txt cmp bin.txt cpy.txt ``` **Receipt.** yt-dlp 2026.10.26, 1,043 files and 10,024 definitions, compiled in 2 min 20 s to a 11.4 MB binary. `--version` and `--list-extractors` (1,755 lines) are byte-identical to `python -m yt_dlp` on stdout, stderr and exit code, verified with `cmp`. The binary links one shared library, the host `libpython3.14.dylib`; the whole package source travels inside the binary. **The limit.** This is a packaging win, not a speed win, and the numbers say so plainly: | Measurement | CPython | Island binary | | --- | --- | --- | | `--version` (best of 7) | 150 ms | 523 ms | | `--list-extractors` (best of 3) | 0.18 s | 0.63 s | | HLS manifest over a local server | 0.62 s | 1.06 s | | DASH manifest over a local server | 0.66 s | 1.76 s | | HTML extraction over a local server | 1.19 s | 2.35 s | | Binary size | 24 MB of source tree | 10.9 MB binary plus a 18.1 MB `libpython` | The binary is 1.5 to 3.4x slower than CPython on the same work, and the reason is measurable: the island recompiles the embedded sources at every start (`exec(compile(source, path, "exec"), module.__dict__)`), while CPython reads cached `.pyc` files. Importing `yt_dlp.extractor` costs 994 ms with a cold bytecode cache and 151 ms with a warm one, so roughly 0.84 s of every cold start is source compilation that CPython does not pay. The binary also does not remove the Python requirement: it links the host `libpython` by absolute path, so it runs only where that library exists. That build now says so itself: the compile ends with `PS4304` naming the dylib path, and `--allow-host-libpython` accepts the host link for a build that never leaves this machine (divergence 64). The honest summary is one binary, one command, byte-identical output, and no interpreter install for the user, at the price of startup time. ## Pattern 2: profile first, then compile only the hot module **When it fits.** A program where a few modules carry the runtime and the rest is glue. **How to do it.** Profile the CPython run, find the module that holds the time, and check whether it lowers natively before you plan anything. ```bash python -m cProfile -s tottime your_program.py > profile.txt uv run pyscriptc coverage path/to/hot_module.py # 0 fences means it lowers ``` **The receipt, and the rule that came out of it.** Two seam costs decide whether splitting a program pays. Both were measured with the compiler on the same machine, 200,000 crossings per run: | Direction | Measured cost per crossing | Wins when | | --- | --- | --- | | Native calls island (in-binary trampoline) | 0.44 to 0.59 us | the island body costs more than the crossing | | Island calls native (`--emit binding`, ctypes) | 0.675 us (0.36 us raw ctypes) | the CPython body costs more than the crossing | The ctypes direction has a sharp cliff, and it is worth writing down because it is the direction most people try first. A helper whose CPython body is `return x + 1` costs 0.064 us in CPython and 0.675 us through the generated binding loader: **10x slower than the code it replaced**. A helper whose body is a ten-iteration loop costs 0.475 us in CPython and 0.681 us through the binding, which is where the direction starts to pay. The rule that falls out of it: a binding only pays when the CPython body is worth more than the crossing. At the measured 12 ns per bytecode, that is a body of roughly 60 to 100 bytecodes, not a three-line helper. Measure the body before you build the seam, not after. ## Pattern 3: a typed hot loop in the subset **When it fits.** A request path, a parse loop, or a dispatch table you control, written to the subset's rules. **How to do it.** Write the hot path to the subset's rules, compile it, and time it against CPython on the same source. ```bash uv run pyscriptc coverage micro.py # 0 fences (0 root) in 0 of 10 definitions (10 native) uv run pyscriptc compile micro.py --emit bin -o micro ``` **Receipt.** A router, request and response records, a query-string parser and a JSON response helper, all annotated and all native, dispatching 20,000 requests: | Side | Time for 20,000 dispatches | Requests/s | | --- | --- | --- | | Compiled native | 25 ms | 790,607 | | CPython 3.14.9 | 51 ms | 391,273 | | Ratio | **2.0x** | | Across three runs on the same machine the ratio was 2.0x to 3.2x; the absolute numbers move with the host, the direction does not. The binary is 585 KB and links no `libpython`, because nothing in it needs the island. **The limit.** The subset is the price of the speed. Fields are declared in the class body and assigned in `__init__`, not annotated in `__init__`. A list is mutated through a named local, not through an attribute expression. `json.dumps` over a dict with mixed value types does not lower, so the JSON is built by concatenation. The full per-construct table is in [The Python subset](python-subset.md); the rewrite cost is real and belongs in the estimate, not after it. ## What did not pay off Two directions were measured and did not earn their place, and both are recorded so nobody repeats them. **Compiling a hot helper and calling it from island code.** A binding call costs 0.675 us before any work happens. The yt-dlp hot leaf it would replace, `FormatSorter._calculate_field_preference_from_value`, is about thirty lines of straight-line Python and costs well under the crossing, so the port is a measured regression. The profitable direction is native calls island, and only for bodies heavy enough to pay the crossing of 0.44 to 0.59 us. **Inferring types into untyped code.** The compiler infers what it can prove and refuses the rest: a missing annotation is a `PS1001` fence, never a guessed `Any`. An annotation overlay applied to yt-dlp's untyped parameters moved 66 of 7,646 fenced definitions to native on the `url`-shaped parameters alone, and 223 when every remaining parameter was annotated `str`, because the wrong annotations create type-mismatch and name-resolution fences of their own. A class-body constant table (3,949 fences) is the larger lever, but it is a compiler feature, not an inference one. The [libraries and fallback guide](libraries.md) is the reference for what the fallback tier does and does not do. --- ## [The Python subset](https://pyscriptc.pages.dev/docs/guide/python-subset/) --- title: "The Python subset" description: "What pyscriptc compiles, how to write code inside the subset, and where to check the live support status." --- pyscriptc compiles the typed subset of Python 3.14. Code inside the subset becomes a native binary with no interpreter inside. Code outside it either stops the build with a `PSxxxx` code that names the exact line, or runs through embedded CPython (fallback mode) when you enable it. The compiler never guesses. Check whether your script is inside the subset before compiling: ```bash pyscriptc coverage script.py ``` `coverage` needs no C toolchain and reports, per line, what would stop the build. `pyscriptc explain PSxxxx` looks up any code it prints and shows the smallest rewrite that clears it. ## What runs native and what runs on the island A compiled program has two execution planes and every definition lands on one of them. Definitions inside the subset lower to native machine code. Everything else either stops the build with a `PSxxxx` code, or, when fallback mode is on, runs inside the embedded CPython island (see [Libraries and fallback mode](libraries.md)). The split is decided per module and then propagates: a module whose top level imports something the subset cannot compile has its whole top level planned for the island, and functions in that module that name island-planned imports or unannotated values reaching them are planned there too, transitively through calls (divergence 59). On a framework-heavy app this compounds fast: on one measured Flask-plus-LLM backend, 41% of the definitions passed the per-definition check in isolation, yet the run-time plane routed 100% of them to the island because untyped framework imports sat at the module top level of every file. A definition that "compiles in isolation" is not a definition that runs native. How to tell before you build: ```bash pyscriptc imports app/ # per import: native, fenced, or island pyscriptc assess app/ # per definition counts, refusals, and the island policy to add pyscriptc assess app/ --json # adds the plane map: island route per function and what demoted it pyscriptc run --fallback=cpython --report=report.json script.py # the executed plane map after a run ``` `assess` prints a "Path to the island" section: the exact `[tool.pyscriptc.imports]` lines that route each unsupported import to embedded CPython, each with the import site that forced it. The `--json` plane map reports, per function, `native` or `island`, plus the `demotedBy` function that pulled each island demotion. Read the plane map, not the per-definition share: the per-definition share answers "how much of this tree is inside the subset", the plane map answers "what actually runs native", and on a framework-heavy app those differ by the whole framework boundary. ## When pyscriptc fits and when it does not pyscriptc fits typed application code: CLIs, data pipelines, file and protocol processing, parsers, simulation and math cores, background jobs. The pattern that works is a typed core with all annotations, closed data shapes (`dataclass`, `dict[str, int]`, `list[T]`), imports from the closed stdlib table, and third-party interaction pushed to explicit boundaries. pyscriptc does not fit, as native code, applications whose structure is runtime reflection: - **pydantic / attrs / dataclasses-with-validators**: these read annotations at run time to build schemas and validators. A native binary has no `typing` object and no annotation surface to read. Route the models to the island and keep the typed logic around them. - **Flask, FastAPI and other decorator-routing frameworks**: context proxies (`request`, `g`), decorator-registered route tables and `**kwargs` dispatch are dynamic-object-model patterns the subset refuses, and the route table only exists at run time. The framework part runs on the island or stays on CPython; compile the handlers' typed cores instead. - **Agent and plugin frameworks (dynamic registries, getattr dispatch, loosely typed message dicts)**: the dispatch is the program. There is no static call graph to lower. - **C-extension-heavy code** (PyMuPDF, NumPy, database drivers): there is no Python source to lower; these cross the island boundary per call and pay marshalling costs. The honest test is `pyscriptc assess` on your tree and then the plane map. If the plane map routes your hot code to the island, pyscriptc gives you a single-file binary that embeds CPython, not a speedup. If it routes your hot code native, the binary has no interpreter inside and the subset's guarantees hold. ## How to write code that compiles Annotate every variable and function parameter. The annotations are the input the compiler works from: a type-checker preflight ([pyrefly](https://pyrefly.org/)) checks them first, then the compiler uses them to build the binary. ```python def sum_squares(xs: list[int]) -> int: total: int = 0 for x in xs: total = total + x * x return total def find(xs: list[str], t: str) -> str | None: for x in xs: if x == t: return x return None def main() -> None: counts: dict[str, int] = {"a": 1, "b": 2} best: int | None = None for v in counts.values(): if best is None or v > best: best = v print(sum_squares([1, 2, 3]), best, find(["x"], "x")) ``` The rules that matter most: - Assign to annotated names. `counts: dict[str, int] = {}` compiles; `counts = {}` stops with `PS1001`, and every later read of the name stops with `PS2002` because of it. - Narrow optionals with `is None`. `best: int | None = None` followed by `if best is None or v > best:` compiles, including inside loops. - Return `T | None` instead of a sentinel value, then narrow at the call site. - Type function values with `Callable`: `c: Callable[[int], int] = lambda v: v + 1` compiles and calls through `c` are direct native calls. - Write generic helpers in the PEP 484 shape: `T = TypeVar("T")` at module level, then `def first(xs: list[T]) -> T`. The helper is specialized per call-site type, including over your own classes. - Use `if TYPE_CHECKING:` for imports that only feed annotations (for example `collections.abc` shapes or third-party types). The branch body is skipped by the compiler and the checker still checks it. - Prefer `from __future__ import annotations` for self-referencing classes, so annotations resolve without forward-reference refusals. - Read schema-like data into annotated locals: `d: dict[str, int] = json.loads(s)`, then pull typed fields out of `d`. `typing` and `typing_extensions` are annotation-only modules. Importing them is free, but a typing name used as a value (`cast(...)`, `get_type_hints(...)`, `NewType(...)`) stops the build with `PS2002`. There is no `typing` module object in the binary. ## Annotations and idioms | Write | What happens | | --- | --- | | `list[int]`, `dict[str, int]`, `set[int]`, `tuple[int, str]`, `tuple[int, ...]` | compiles; the annotation fixes the element layout | | `T \| None`, `Optional[T]` | compiles as a tagged optional; `is None` narrowing drives the branches | | `A \| B` (neither arm `None`), `Union[A, B, None]` | unsupported (`PS1101` / `PS1102`) | | `Final`, `Final[int]`, `ClassVar[T]` | the inner annotation compiles; the checker enforces no reassignment | | `Annotated[int, "..."]` | the first argument compiles, the rest is dropped | | `Callable[[int], int]` | compiles over functions and lambdas; bare `Callable` is unsupported (`PS1104`) | | `Iterator[int]`, `Iterable[int]`, `Generator[...]`, `Sequence[int]` | compiles (`Sequence[T]` reads as `list[T]`) from `typing` or `collections.abc`, imported directly, as `collections.abc.X`, or under `if TYPE_CHECKING:` | | `Mapping[K, V]`, `MutableMapping[K, V]`, `MutableSequence[T]`, `Collection[T]`, `AbstractSet[T]`, `MutableSet[T]` | compiles as the concrete container (`dict[K, V]`, `list[T]`, `set[T]`); the argument must be that container, so a `set` passed to a `Collection[T]` parameter is refused (`PS1003`) | | `collections.abc` or `typing` names used as values (`isinstance(v, Sequence)`, `print(Mapping)`) | unsupported (`PS2002`, or `PS2001` inside `isinstance`); they are annotation-only | | `TypeVar("T")` generic functions and `Generic[T]` classes | compiles, specialized per call site, including `T \| None`, `Callable[[T], U]` and a TypeVar key in `dict[K, V]` or `Mapping[K, V]`; a bound `TypeVar` erases to its bound; constraints and variance flags are accepted and not enforced (as in CPython) | | PEP 695 `def f[T]`, `ParamSpec`, `TypeVar(..., default=...)` | unsupported (`PS1106`, `PS2003`) | | `Protocol`, `ABC`, `@abstractmethod`, `@overload` | the annotation is checked, then dropped; no runtime enforcement (divergence 14) | | `NamedTuple`, `TypedDict`, `NewType`, `Literal[...]`, `Self` | unsupported; use a `@dataclass`, a plain `dict[K, V]`, an alias, a plain `str`, or the class name | | `Any` as a container element (`dict[K, Any]`, `list[Any]`) | compiles in the builtin and the `typing` spelling; reads need a proven type at the read site (annotation or checker fact), an unproven read fences `PS1107`; bare `Any` or `object` as an annotation is unsupported (`PS1104`) | | `async def` and `await` | compiles | | `dataclasses.dataclass` with no options | compiles as a native record; `frozen=True` and other decorators are unsupported (`PS1201`) | | `cast(t, v)`, `get_type_hints`, `__annotations__` reads | unsupported (`PS2002`); write the type on an annotated local instead | | `isinstance(v, list[int])` | unsupported (`PS2001`); runtime checks take scalar and class names only | | quoted forward references (`"Node"`) | unsupported (`PS1105`); define the class first and write the name unquoted | Constructs with runtime meaning beyond their annotation (pydantic models, `attrs` validators, anything that reads annotations at run time) are not typing failures: run them through embedded CPython (fallback mode), see [Libraries and fallback mode](libraries.md). For background on the typing system itself, the upstream references are the best source and pyscriptc does not re-explain them: - [The typing module](https://docs.python.org/3/library/typing.html) and [Type hints cheat sheet](https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html) from the mypy docs - [PEP 484](https://peps.python.org/pep-0484/) (type hints), [PEP 526](https://peps.python.org/pep-0526/) (variable annotations), [PEP 544](https://peps.python.org/pep-0544/) (protocols), [PEP 570](https://peps.python.org/pep-0570/) (positional-only parameters), [PEP 585](https://peps.python.org/pep-0585/) (builtin generics), [PEP 604](https://peps.python.org/pep-0604/) (`X | Y` unions), [PEP 647](https://peps.python.org/pep-0647/) (type guards), [PEP 695](https://peps.python.org/pep-0695/) (type parameter syntax) ## Networked and concurrent subsets The stdlib subsets that touch the outside world keep to a closed surface; each names its divergences in [Divergences from CPython](../reference/divergences.md). - `selectors`: `DefaultSelector` over the compiled event loop, with `SelectorKey`, `EVENT_READ`/`EVENT_WRITE` and `register`/`unregister`/`modify`/`select`/`get_key`/`close`. `register` takes a bare int fd, not a `fileobj` (the typed socket handle is issue #229), `select()` returns `list[SelectorKey]` rather than CPython's `(key, events)` pairs, and `get_key` of an unregistered fd returns `None` where CPython raises `KeyError`. ## What stays unsupported - Runtime reflection: `inspect.signature`, `getattr` dispatch, `**kwargs` forwarding, annotation reads. - Typing names used as values: `cast`, `NewType`, `get_type_hints`, `Annotated` outside an annotation. - Dynamic object model features: `issubclass`, `type(v)`, `vars`, `__import__`. - Anything not covered above stops the build with a named `PSxxxx` code, never a guess. The full divergence list is in [Divergences from CPython](../reference/divergences.md). ## Where the live status lives Per-construct, per-builtin, per-module and per-library support status is tracked on the [compat site]({{PYSC_COMPAT_URL}}), fed by the [nightly CI workflow](https://github.com/sigilco/pyscriptc/actions/workflows/nightly.yml), so it cannot go stale the way a page would. This page states the rules and the known gaps only. ## Scripts as a target pyscriptc aims at real CLI and automation scripts, and is measured against third-party scripts vendored from public repositories (`benchmarks/real_scripts/`). A script that stops with a `PSxxxx` code is treated as a signal about the compiler, so corpus scripts are never edited to make them compile. Common failure modes on real scripts: - An assignment without an annotation (`files = os.listdir(d)`) stops with `PS1001`, and every later read of the name stops with `PS2002`; the first one is the cause and the rest follow from it. - A keyword argument to a stdlib call (`shutil.copy(src, dst, follow_symlinks=False)`) stops with `PS2003`. - A module outside the closed import table (`platform`, `calendar`, `zipfile`, `tarfile`, `sqlite3`) stops with `PS2002` on the import. Run the module through embedded CPython (fallback mode) instead, see [Libraries and fallback mode](libraries.md). Related commands: ```bash pyscriptc assess script.py # what compiles, what does not, and why pyscriptc imports script.py # how each imported module is classified pyscriptc verify script.py --call main # run the binary and CPython side by side and diff the output ``` --- ## [Serving Flask and FastAPI](https://pyscriptc.pages.dev/docs/guide/web-frameworks/) --- title: "Serving Flask and FastAPI" description: "Compiling a Flask or FastAPI app into one binary: what works, what it costs, measured numbers, and the known gaps." --- # Serving Flask and FastAPI A Flask or FastAPI app compiles into one binary with `--island` (the alias is `--fallback=cpython`). The framework is dynamic Python, so it runs on the island floor: the binary embeds CPython and the app inside it. What the compile buys is the deployable shape: one artifact for app plus framework plus interpreter, no venv and no `pip install` on the target. It does not buy native speed; the request path is the framework's own. Every number on this page was measured on the reference setup below, with the commands shown, so the page can be re-measured when the compiler moves. The receipts are statements about one machine and one framework version, not guarantees. ## The apps `examples/web/flask_app.py` serves `/` and `/health` through Flask 3.1. `examples/web/fastapi_app.py` serves the same two paths plus the generated `/docs` through FastAPI 0.162 with uvicorn. ```bash uv run pyscriptc compile examples/web/flask_app.py --island --embed-python host --emit bin -o flask_bin uv run pyscriptc compile examples/web/fastapi_app.py --island --embed-python host --emit bin -o fastapi_bin ``` `--emit bin` is explicit because `--emit` defaults to `llvm` (the textual IR), which is what the compiler's own pipelines want. The compile needs the framework importable so the type-checker and the island seed can read it (run the compile from a venv that has `flask` or `fastapi` installed, or pass `PYTHONPATH`). The binary needs it importable at run time too: the `--embed-python host` build adds `VIRTUAL_ENV`'s `site-packages` and `PYTHONPATH` entries to the embedded interpreter's path, the same rule the [libraries page](../guide/libraries.md) documents for fallback mode. With the framework on the path, the binary serves: ```bash PYTHONPATH=.venv/lib/python3.14/site-packages ./flask_bin & curl -s localhost:8080/ # hello from flask ``` ## Divergences and gaps - The binary is not self-contained under `--embed-python host`: it links the host `libpython` and reads the framework from disk. A target without the framework cannot run it. - `--embed-python bundle` writes a relocatable directory (`bin/` + `lib/`) that carries `libpython`, the trimmed stdlib and the app's pure-Python dependencies, and it runs with `env -i` (no inherited environment). Neither example can use it today, both refused with `PS4302` by design: Markupsafe for Flask and pydantic-core for FastAPI ship compiled extensions, and a bundle carries pure Python only. A program whose dependency closure is pure (`anyio` alone) bundles and runs with `env -i`; the same refusals bound the shape for every framework with a compiled wheel. - Dev-server warnings are the frameworks' own: Flask's "use a production WSGI server" banner and uvicorn's startup lines print from inside the binary exactly as they print under CPython. - No inbound socket compiles natively: `socket`, `ssl` and `http.server` are fenced (`PS2001`/`PS2002`), so a server in the native tier does not exist yet (divergence 29). The servers above run entirely on the island floor; `ps_loop.c` has no file-descriptor demultiplexing and the native HTTP client is outbound-only. - The framework's own divergences apply unchanged: the embedded CPython is the host interpreter, not the 3.14 pin (divergence 15). ## Measured receipts Reference setup: Apple M-series laptop, macOS 26 arm64, CPython 3.14.9 host and island, Flask 3.1.3 / Werkzeug 3.1.9, FastAPI 0.162 / Starlette 1.10 / uvicorn 0.54 / pydantic 2.14, uv venv. `ab` was avoided: the Werkzeug dev server answers `Connection: close`, and a macOS load client that opens a fresh socket per request exhausts its ephemeral port range in seconds (`OSError 49`), which reads as a server fault but is a client artifact. The load generator below pins a source port per worker. | Metric | Flask compiled | Flask venv + `python app.py` | FastAPI compiled | FastAPI venv + `python app.py` | |---|---|---|---|---| | Binary (host embed) | 629 KB | n/a | 629 KB | n/a | | Deployed size | 629 KB + framework on disk | 20.3 MB venv | 629 KB + framework on disk | 20.3 MB venv | | Bundle directory (`--embed-python bundle`) | refused (PS4302, Markupsafe) | n/a | refused (PS4302, pydantic-core) | n/a | | Cold start to first 200 | 128 ms median | 211 ms median | 198 ms median | 203 ms median | | RSS after warm-up | 28-35 MiB | 33-35 MiB | 30-56 MiB | 56-203 MiB | | Throughput, 8 workers, loopback | ~690 req/s | ~686 req/s | ~3,400 req/s | ~3,370 req/s | | Latency p50 / p99 | 11.5 / 12.9 ms | 11.6 / 13.3 ms | 2.3 / 3.0 ms | 2.3 / 3.0 ms | Reading the table honestly: - Throughput and latency are the framework's dev servers on both sides of every row; the compiled column and the venv column are within noise of each other. There is no speed claim. - The compiled Flask binary's RSS overlaps the venv run's because both are the same interpreter running the same code. The FastAPI RSS range is uvicorn's single-worker spread; the two 200 MiB readings were multi-worker starts. - The size win is real but conditional: 629 KB plus a framework that must already be on disk is smaller than a 20 MB venv only when several apps share one framework install. The `env -i`-runnable bundle, measured on a pure-dependency program (`anyio`), is 39 MB against a 20 MB venv, because it also carries the stdlib and `libpython`. - Cold start saves roughly 40 percent on Flask (128 ms vs 211 ms) and is inside noise on FastAPI. Both include the framework's own import cost, which dominates. The load generator used for the throughput and latency rows, kept so the numbers can be reproduced: ```python import json, socket, statistics, sys, threading, time, urllib.parse results, lock = [], threading.Lock() def worker(host, port, path, stop): local = [] sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.settimeout(5) local_port = sock.getsockname()[1] while time.monotonic() < stop: t0 = time.perf_counter() try: sock.connect((host, port)) sock.sendall(b"GET %s HTTP/1.1\r\nHost: %s\r\nConnection: close\r\n\r\n" % (path.encode(), host.encode())) data = b"" while True: chunk = sock.recv(65536) if not chunk: break data += chunk if data.startswith(b"HTTP/1.1 200"): local.append((time.perf_counter() - t0) * 1000.0) except OSError: pass finally: sock.close() sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.settimeout(5) try: sock.bind(("127.0.0.1", local_port)) except OSError: sock.bind(("127.0.0.1", 0)) local_port = sock.getsockname()[1] with lock: results.extend(local) u = urllib.parse.urlsplit(sys.argv[1]) stop = time.monotonic() + float(sys.argv[2]) if len(sys.argv) > 2 else time.monotonic() + 8.0 ts = [threading.Thread(target=worker, args=(u.hostname, u.port or 80, u.path or "/", stop)) for _ in range(8)] t0 = time.monotonic() for t in ts: t.start() for t in ts: t.join() elapsed = time.monotonic() - t0 results.sort() n = len(results) print( json.dumps( { "requests": n, "rps": round(n / elapsed, 1), "p50_ms": round(results[int(n * 0.5)], 2), "p95_ms": round(results[int(n * 0.95)], 2), "p99_ms": round(results[int(n * 0.99)], 2), } ) ) ``` ## When this shape is the right one - One artifact per app for a deployment target that has no Python tooling: the binary plus the framework tree is the whole contract, and `env -i` works for the bundle shape. - Reproducible-ish deploys without Docker: no `pip install` step can drift, because there is none. - Where it is not the right shape: when the target already has a Python installation and a venv workflow, the venv column of the table is the same app with fewer moving parts; when the framework needs a compiled dependency that `PS4302` refuses, the bundle shape is closed and the host-embed shape still needs the venv. --- ## [pyscriptc](https://pyscriptc.pages.dev/docs/) --- title: pyscriptc description: Compile annotated Python to a native binary. ---

pyscriptc

# pyscriptc Compile annotated Python to a native binary. [Get started](/getting-started/) · [GitHub](https://github.com/sigilco/pyscriptc) · [PyPI](https://pypi.org/project/pyscriptc/)

Sponsor

For agents: [llms.txt]({{PYSC_DOCS_URL}}/llms.txt), [llms-full.txt]({{PYSC_DOCS_URL}}/llms-full.txt) ## Where it differs from CPython pyscriptc compiles a typed subset of Python to machine code. Anything the subset cannot prove is refused with a numbered diagnostic (`PSxxxx`) that names the line, or routed to an embedded CPython island inside the same binary. The rules, the numbered divergences from CPython and the known gaps are in [Divergences from CPython](/reference/divergences/) and [The Python subset](/guide/python-subset/); the live per-module and per-library status is on the [compat site]({{PYSC_COMPAT_URL}}). Two things to know before you plan around it: - Annotations are required. A parameter or local without one stops the build (`PS1001`). - Code outside the subset does not fail silently, but it does not become native either: it runs on the island, at interpreter speed, and every call crossing the boundary pays a marshalling cost. [Libraries and fallback mode](/guide/libraries/) states what the two planes cost. ## Try it now ```bash uv tool install pyscriptc pyscriptc compile examples/sum_squares.py --run --call sum_squares --args '[10]' -o /tmp/sum_squares ``` That prints `285`, the same value CPython returns, from a native binary. ## Read next - [Getting started](/getting-started/) - [Examples](/examples/) - [The Python subset](/guide/python-subset/) - [Using libraries](/guide/libraries/) - [Comparison with other tools](/guide/comparison/) - [CLI reference](/reference/cli/) - [Diagnostic codes](/reference/diagnostics/) - [Mobile apps](/guide/mobile/) - [Stability and contracts](/reference/stability/) --- ## [Contributing](https://pyscriptc.pages.dev/docs/project/contributing/) --- title: Contributing description: How to set up the repo and land a change. --- ## Setup ```bash git clone https://github.com/sigilco/pyscriptc.git cd pyscriptc mise trust && mise install # python, uv, node, pnpm per mise.toml mise run setup # uv sync + pnpm install (the node deps live in docs/.docmd) ``` Tools already on your PATH that satisfy the `mise.toml` pins are reused, not reinstalled. You also need `clang` for linking compiled binaries. ## Tasks `mise` is the default task runner. Every command in this document is `mise run ` first, with the raw command it wraps shown once below the table. The task names and what they do: | Task | What it does | | --- | --- | | `mise run setup` | First-time install: `uv sync` and the pnpm installs (`docs/.docmd/`, `apps/web/`) | | `mise run docs:dev` | Public docs dev server over `docs/` | | `mise run docs:build` | Build the public site to `dist/docs` | | `mise run docs:validate` | Check every internal link in `docs/` | | `mise run docs:internal` | Dev server for the internal notes in `.agents/docs/` (no build step) | | `mise run test` | The python test suite, `linux_container` excluded | | `mise run lint` | `ruff check` | | `mise run format-check` | `ruff format --check`, with the same excludes as CI | | `mise run check` | `lint` + `format-check` + the `tests/compat2` unit selection | | `mise run bench:baseline` | The perf regression gate against `benchmarks/baseline/reference*.json` | | `mise run compat:json` | Rebuild `data/compat/compat.json` (append `--check` to compare) | | `mise run compat:ladder` | Rebuild `data/compat/ladder.json` | | `mise run compat:r0` | Add the R0 real-scripts point to `data/compat/ladder.json` | | `mise run compat:census` | Compat2 census over the visible corpus (append `--top N` to limit) | | `mise run support:table` | Regenerate the supported-API tables under `.agents/docs/support` (append `--check` to compare) | | `mise run boxed:cost` | The boxed ledger window, per `AGENTS.md` | | `mise run tir:hashes` | Canonical IR hash compare against the committed baseline (linux oracle; see the note in `mise.toml` for the 3.14 micro stamp and macOS behavior) | | `mise run compile` | `pyscriptc compile`, extra args pass through: `mise run compile examples/hello.py --emit bin` | | `mise run run` | `pyscriptc run` (content-cached compile and execute), args pass through | | `mise run wheel:smoke` | Build the wheel, install it in a fresh venv, compile and run hello | | `mise run release:rehearse` | Dispatch `release.yml` on the master tip (the rehearsal run, about 20 minutes) | Extra args after the task name are appended to the wrapped command, so `mise run compat:json --check` is `uv run python scripts/build_compat_json.py --check`. The same raw commands are available without mise (`uv run pytest`, `pnpm --dir docs/.docmd exec docmd build --config docmd.config.js`, `pnpm --dir apps/web run docs:validate`), but mise pins the toolchain versions and is what a local checkout is set up for. CI runs the raw commands, not `mise run`. `mise.toml` pins `uv = "latest"`, and the workflows hold a `>= 0.13.4` uv floor from the emSign CA incident, so routing CI through mise would change which uv the checks run under. The job names and the required check names are the same either way, since the commands are identical. ## The documentation site The public site has two parts: the docs, built from `docs/` with [docmd](https://docmd.io) (configuration in `docs/.docmd/`, with the placeholders plugin; the internal notes in `.agents/docs/` are served by the second config `docs/.docmd/docmd.internal.config.js`), and the web app in `apps/web/` (pnpm, one Astro app: the landing page at `/` and the compat page at `/compat`). The canonical site data lives in `data/` at the repo root and is synced into the app at build time; web tooling is pnpm-only. ```bash mise run web:dev # site app dev server (landing + compat) mise run site:build # build web + docs, assemble apps/web/deploy/ mise run site:check # validate the assembled tree ``` ```bash mise run docs:dev # preview the public site mise run docs:build # build to dist/docs mise run docs:validate # every internal link must resolve mise run docs:internal # preview the internal notes ``` Two details worth knowing before you edit a page: - The output directory is `dist/docs`, outside the source tree. Cloudflare Pages must publish `dist/docs`, not `docs/dist`. - Pages may use the `{{PYSC_COMPAT_URL}}` and `{{PYSC_DOCS_URL}}` placeholders. They are expanded at build time from the environment variable of the same name, falling back to the default in `docs/.docmd/plugins/placeholders.js`. ## Checks before a PR ```bash mise run check mise run test mise run bench:baseline ``` `mise run check` wraps `uv run ruff check`, `uv run ruff format --check` and `uv run pytest -q tests/compat2 -m "not linux_container"`, and `mise run bench:baseline` wraps `uv run python -m benchmarks.baseline.run --check`. The raw commands work too. ## Conventions - Conventional commits (`feat(lower): ...`, `fix(llvm): ...`), no co-author trailers. - The compat2 scorer files listed in `AGENTS.md` are frozen; fence messages are never reworded. - Boxed (CPython island) code stays behind `boxed:begin`/`boxed:end` markers. - Agent-facing notes and the debug/logging directives live in `AGENTS.md` and `.agents/docs/`. ## Reporting bugs Attach `pyscriptc logs` output or a `--debug-dir` dump, the `PSxxxx` fence codes you hit, and the smallest source file that reproduces. --- ## [Telemetry and privacy](https://pyscriptc.pages.dev/docs/project/telemetry/) --- title: Telemetry and privacy description: What pyscriptc collects, and how to turn it off. --- Release builds of pyscriptc send pseudonymous usage events to PostHog (EU cloud, `eu.i.posthog.com`). Telemetry exists to answer one question: which unsupported constructs block real users the most, so the roadmap follows demand. See the [privacy notice](https://github.com/sigilco/pyscriptc/blob/master/PRIVACY.md) for what is collected, the retention and how to opt out. ## What is sent The CLI sends the fields below. The website sends page paths, referrers and campaign parameters, click events on declared links and buttons, and scroll depth; see the [website analytics](#website-analytics) section above. Every field of every event, in full: | Event | Field | Example | Why | | ------------------- | ------------------------------------------------------------------------ | --------------------------------- | ------------------------------------------------------------------------------------------------- | | all | `distinct_id` | `3f2b8c1e-...` (random uuid4) | counts distinct installs, not people; see below | | all | `pyscriptc_version` | `0.4.0` | which release has which problem | | all | `python_version` | `3.14` (major.minor only) | which Python versions to support | | all | `platform` | `darwin`, `linux` | which targets matter | | all | `internal` | `false` | true when the event comes from a source checkout, an editable install, `PYSC_INTERNAL=1` or CI (CI never sends events; the flag marks our own smoke and dev traffic in the dashboard) | | all | `$ip` (null), `$geoip_disable` (true), `$process_person_profile` (false) | | ask PostHog not to store the IP, not to geolocate and not to build a person profile | | `cli.invoked` | `subcommand` | `compile` | which commands are used | | `compile.finished` | `duration_ms` | `412` | performance regressions | | `compile.finished` | `outcome` | `fences` | `ok`, `fences`, `check-failed`, `lower-error` or `build-error` | | `compile.finished` | `fence_count` | `3` | how often programs are blocked | | `compile.finished` | `fence_families` | `{"union-types": 2, "stdlib": 1}` | which kinds of constructs block users (fixed vocabulary, `stdlib:` collapsed to `stdlib`) | | `compile.finished` | `fence_codes` | `{"PS2002": 2, "PS1101": 1}` | how often each diagnostic code blocks users (the fixed PSxxxx vocabulary, counts only) | | `compile.finished` | `binary_bytes` | `282616` | binary size; present only when a binary was linked | | `compile.finished` | `checker`, `emit`, `driver` | `pyrefly`, `bin`, `clang` | which toolchain options are used (flag choices from a fixed list) | | `compile.finished` | `skip_check` | `false` | explains an absent `checker` | | `run.finished` | `exit_code` | `0` | how often compiled programs succeed | | `run.finished` | `duration_ms`, `cache_hit`, `binary_bytes` | `731`, `true`, `282616` | run latency and the content-hash binary cache hit rate | | `run.finished` | `driver`, `target`, `skip_check` | `clang`, `` (absent), `false` | which toolchain options are used (absent rather than empty) | | `coverage.reported` | `definitions`, `nativeDefinitions`, `fencedDefinitions`, `fences`, `rootFences` | `40`, `37`, `3`, `5`, `2` | aggregate coverage only | | `coverage.reported` | `byFamily` | `{"union-types": 5}` | family to fence count | Not sent, ever: file names or paths, user names, host names, machine ids, MAC addresses, source text or snippets, imported module names, fence messages, per-fence locations, definition names, argument values, program arguments or output, environment variables. The PSxxxx codes in `fence_codes` are the fixed diagnostic vocabulary of the compiler (`pyscriptc explain` lists them); a code says which construct is unsupported, never which statement, file or name produced it. The server connection necessarily exposes your IP address to the host; the project is configured to discard it (see PRIVACY.md in the repo). ## The install id The id is a random uuid4 created the first time an event is sent and stored in `telemetry-id` next to the config file (`~/.config/pyscriptc/telemetry-id`, or under `$XDG_CONFIG_HOME`), with mode 0600. Nothing about your machine is read to make it: no machine id, MAC address or hostname. It is still a persistent identifier for the install, so it counts as pseudonymous data, not anonymous data. Run `pyscriptc telemetry reset-id` (or delete the file) to throw it away; a new unrelated id is created at the next send. Older releases derived the id from a hash of machine identifiers; this release stops using it and does not migrate it. ## Opting out ```bash pyscriptc telemetry disable # persistent off pyscriptc telemetry status # current state pyscriptc telemetry reset-id # new random id ``` Or set `PYSC_NO_TELEMETRY=1` or `DO_NOT_TRACK=1`. Telemetry is always silent on CI, in development builds built without `POSTHOG_PROJECT_TOKEN`, and for our own smoke jobs (they run the release binary with `PYSC_INTERNAL=1`, since `env -i` strips `CI`). ## First-run notice The first time you run pyscriptc in an interactive terminal, it prints a notice on stderr listing what is collected, the retention period and how to opt out, then records `notice_seen = true` in `~/.config/pyscriptc/config.toml` (or under `$XDG_CONFIG_HOME`). No event is sent before that notice was shown. The notice never prints, and so nothing is sent, when stderr is not a TTY, when `CI` is set, or when telemetry is disabled. The `pyscriptc telemetry` command itself sends nothing. ## Maintainer dashboard The aggregates are read in the PostHog EU project (no external dashboard service). Three saved insights cover the roadmap question: - Fence histogram: insight type "SQL", `SELECT properties.fence_codes AS code, count() FROM events WHERE event = 'compile.finished' AND outcome = 'fences'`, ordered by count. After a few weeks switch to trends per code (`breakdown` on `fence_codes`) for the per-code PSxxxx histogram over time. - Run outcomes: trend of `run.finished` broken down by `exit_code`, and a second trend broken down by `cache_hit` for the binary cache hit rate. - Compile outcomes: trend of `compile.finished` broken down by `outcome`. Keep the raw event retention at 90 days (matches the privacy notice); the saved insights read the same events and need no extra retention. ## Website analytics The public website (landing page, compat page, docs) sends events to the same PostHog EU project, loaded lazily so it never blocks rendering. The loader is baked in at build time from `PUBLIC_POSTHOG_TOKEN`; without that variable no analytics code ships at all. It carries a `site` super property from `PUBLIC_SITE_URL`, so the deployment host is never hardcoded and a domain move is one variable change. The website posture matches the CLI: cookieless (`persistence: "memory"`, no cookies and no local storage, so every page view starts from a fresh anonymous id), `person_profiles: "never"`, session recording, surveys, heatmaps, dead-click, rage-click and exception capture all off, `respect_dnt: true` plus an explicit Global Privacy Control check, and `$ip` deleted from every event (backed by the project setting "Discard client IP data"). Autocapture is limited to elements explicitly tagged `data-ph-event`, so no element text, form input or untagged link is ever captured. Website events, in full: | Event | Trigger | Properties | | ------------------ | ------------------------------------------ | -------------------------------------------- | | `$pageview` | every page load (all three site parts) | path, referrer, `utm_*` (PostHog defaults) | | `$pageleave` | leaving a page | PostHog defaults | | `cta_click` | a call-to-action link (hero, final CTA, modal) | `label` (which button) | | `install_copy` | copying the install command | `source` (`playground_modal`) or docs variant | | `outbound_click` | outbound link (GitHub, PyPI, X) | `label` | | `scroll_depth` | landing page scroll quarters | `percent` (25, 50, 75, 100) | | docs `*_click` variants | docs links, tab switches, copy buttons, search open | `label`, `tab`, `source` | `llms.txt`, `llms-full.txt` and the og-image are plain files fetched without the loader running, so they are not visible to PostHog; Cloudflare's own Web Analytics (server side) is the way to count those if needed. Keep the 90-day retention for website events too; they are the same project and the same bill. --- ## [Supported CPython versions](https://pyscriptc.pages.dev/docs/project/versions/) --- title: Supported CPython versions description: Which Python versions pyscriptc supports and why. --- pyscriptc supports [Python versions still supported upstream](https://devguide.python.org/versions/); **CPython 3.14** is the primary reference. The parser is CPython 3.14's `ast`, and `pyproject.toml` requires `>=3.14` to run the compiler. Two different version axes matter: - **Compiler host**: the CPython that runs pyscriptc itself (`>=3.14`, enforced in `pyproject.toml`). This is also the reference interpreter the behavior is checked against. - **Emitted binaries**: native executables with no Python version dependency at all. The `cpython` IR stamp is a consistency check, not a runtime requirement. ## Why 3.14 is the minimum - The front end parses with the running interpreter's `ast` module, and the reference interpreter compares behavior against that same interpreter, so supporting older hosts means maintaining a second set of parser and semantics expectations for no user benefit. - Binaries compiled by pyscriptc do not embed Python, so users running compiled programs are unaffected by the host floor. - The version axis only affects which interpreter runs the compiler and which CPython the compiler matches; it never affects the produced binaries. The detailed assessment behind the window (lifecycle table, IR stamping, oracle installs, and the host-versus-target split) is in [`docs/python-version-window.md`](https://github.com/sigilco/pyscriptc/blob/master/.agents/docs/release/python-version-window.md). --- ## [CLI](https://pyscriptc.pages.dev/docs/reference/cli/) --- title: CLI description: Every pyscriptc subcommand and flag. --- # CLI Install once (`uv tool install pyscriptc`) and every subcommand is `pyscriptc COMMAND [options]`. `-v` / `--verbose` turns on DEBUG audit records for any subcommand; the log is what `pyscriptc logs` prints. Exit codes and the stability contract are in [Stability and contracts](stability.md), the diagnostic codes in [Diagnostics](diagnostics.md). ### `compile` ```sh pyscriptc compile SOURCE [options] ``` | Flag | Value | Effect | | ---- | ----- | ------ | | `--program` | | treat the directory as a program (`python -m pkg` parity): `__main__.py` is the entry point even when the directory has an `__init__.py`, and its modules are namespaced under the directory name | | `--emit EMIT` | `ir\|mir\|llvm\|asm\|obj\|bin\|lib\|binding` | output artifact: ir (JSON IR), mir (textual MIR of the leaf slice), llvm (textual LLVM), asm, obj, bin, lib or binding; default llvm (default `llvm`) | | `--out-dir DIR` | `DIR` | write artifacts to DIR/ instead of beside the source (useful for directory compiles) | | `-o PATH` | `PATH` | output path; defaults to the source stem | | `--ffi MANIFEST` | `MANIFEST` | bind a typed extern manifest (TOML) and link its C sources; repeatable. Also read from the `ffi` key of pyscriptc.toml / PEP 723 metadata | | `--fallback TIER` | `cpython` | route a function pyscriptc cannot compile to embedded CPython (fallback mode); the only tier is cpython. Needs libpython and the packages on the host | | `--island` | | alias of --fallback=cpython: route every import outside the closed table and every function pyscriptc cannot compile to embedded CPython (fallback mode) | | `--embed-python MODE` | `host\|bundle\|auto` | libpython embedded CPython links: host (the compiler interpreter), bundle (the pinned runtime pack, relocatable output) or auto (host when this interpreter has a shared libpython, bundle otherwise; the default). Also read from the `embed` key of [tool.pyscriptc.island] | | `--embed-stdlib TRIM` | `safe\|minimal` | stdlib trim of a bundle: safe (the default; everything except test, idlelib, tkinter, turtledemo, ensurepip, pydoc_data, __phello__) or minimal (an opt-in frozen closure plus allowlist). Also read from the `stdlib` key of [tool.pyscriptc.island] | | `--bundle-exclude ROOT` | `ROOT` | import root a bundle does not carry: the host must provide it (an optional or native-backed dependency); repeatable | | `--island-engine ENGINE` | `ENGINE` | island engine descriptor to use (a TOML path, a name searched in PYSC_ISLAND_ENGINES, or cpython for the built-in embedded-CPython engine); resolved, validated and recorded in the debug summary. Also read from the `engine` key of [tool.pyscriptc.island] | | `--embed-headers` | | stage the runtime pack's CPython headers into a bundle: a bundle that is itself a compiler (the self-compiled pyscriptc) needs Python.h to compile island programs at run time | | `--onefile` | | write the bundle as one executable: the payload travels appended to a small stub, which extracts it into the cache (keyed by the payload's sha256) on the first run and execs it on every run. Needs --embed-python=bundle | | `--strip-runtime` | | strip the staged libpython of a bundle: the dynamic symbols the interpreter exports stay, the debug and static symbols go (about 12 MB on Linux) | | `--allow-host-libpython` | | accept a binary that links the build host's libpython: the post-link audit warns instead of refusing (issue #1247). The binary then runs only where that exact interpreter library exists | | `--call FUNC` | `FUNC` | function the generated main calls | | `--args ARGS` | `ARGS` | Python literal list of arguments for --call, parsed with ast.literal_eval (default `[]`) | | `--run` | | compile a binary and run it; arguments after `--` go to the binary | | `--driver DRIVER` | `clang\|zig` | C toolchain driver; zig means `zig cc` (preferred, required for --target); env PYSC_CC, defaults to zig if found, else clang | | `--target TRIPLE` | `TRIPLE` | cross-compilation triple for `zig cc`, e.g. aarch64-linux-gnu.2.39 (env PYSC_TARGET) | | `--linker LINKER` | `clang\|zig` | driver for the final link step only; env PYSC_LINKER, defaults to --driver | | `--sanitize LIST` | `LIST` | comma list of sanitizers for the C runtime and the emitted code, e.g. address,undefined (env PYSC_SANITIZE); needs a binary, so --emit bin only | | `--debug-info` | | emit DWARF debug metadata (source file and line per function and statement) so lldb/gdb can step in the Python source (env PYSC_DEBUG_INFO) | | `--print PRINT_WHAT` | `diagnostics` | print JSON instead of compiling: diagnostics is the coverage report; exit 3 on a refusal | | `--skip-check` | | skip the type-checker preflight | | `--checker CHECKER` | `pyrefly\|mypy\|pyright` | type-checker preflight; mypy and pyright are deprecated (default `pyrefly`) | | `--bridge BRIDGE` | `mypy\|pyrefly` | feed the checker's inferred-type map to the compiler so unannotated code compiles; needs mypy installed, or pyrefly for --bridge pyrefly | | `--infer` | | apply the package-wide pins `pyscriptc infer` proves: every call site of a parameter that agrees on its type pins the parameter, so an unannotated parameter lowers without the source being annotated | | `--debug[=WHAT]` | `WHAT` | print a compact stage table to stderr; WHAT is a comma list of stages,fences,cc,paths,all (also via PYSC_DEBUG) | | `--debug-dir DIR` | `DIR` | write debug artifacts (ir.json, out.ll, cc.sh, summary.json) to DIR | | `--timings` | | print per-stage wall times (parse, check, lower, emit, cc/link) to stderr | ### `run` ```sh pyscriptc run SOURCE [options] ``` | Flag | Value | Effect | | ---- | ----- | ------ | | `--program` | | treat the directory as a program (see compile) | | `--fallback TIER` | `cpython` | route a function pyscriptc cannot compile to embedded CPython (fallback mode); the only tier is cpython | | `--island` | | alias of --fallback=cpython | | `--embed-python MODE` | `host\|bundle\|auto` | libpython embedded CPython links (see compile); the run cache keys on it | | `--report[=FILE.json]` | `FILE.json` | print or write the executed plane map after the run (default: silent) | | `--call FUNC` | `FUNC` | function the generated main calls | | `--args ARGS` | `ARGS` | Python literal list of arguments for --call (default `[]`) | | `--driver DRIVER` | `clang\|zig` | C toolchain driver (see compile) | | `--target TRIPLE` | `TRIPLE` | cross-compilation triple for `zig cc` (see compile) | | `--linker DRIVER` | `clang\|zig` | link driver (see compile) | | `--sanitize LIST` | `LIST` | comma list of sanitizers for the C runtime and the emitted code (env PYSC_SANITIZE) | | `--skip-check` | | skip the type-checker preflight | | `--checker CHECKER` | `pyrefly\|mypy\|pyright` | type-checker preflight (default `pyrefly`) | | `--no-cache` | | always recompile (env PYSC_NO_RUN_CACHE=1) | | `--ffi MANIFEST` | `MANIFEST` | bind a typed extern manifest (TOML) and link its C sources; repeatable | ### `verify` ```sh pyscriptc verify SOURCE [options] ``` | Flag | Value | Effect | | ---- | ----- | ------ | | `--call FUNC` | `FUNC` | function to call on both sides, not the program entry | | `--args ARGS` | `ARGS` | Python literal args list for --call (default `[]`) | | `--input FILE` | `FILE` | feed FILE's bytes on stdin to both runs (default: an empty stdin, so input() sees EOF) | | `--json` | | print the receipt as JSON instead of text | | `--timeout S` | `S` | seconds each run may take before it is a divergence (default 60) | | `--program` | | treat the directory as a program (see compile) | | `--driver DRIVER` | `clang\|zig` | C toolchain driver (see compile) | | `--target TRIPLE` | `TRIPLE` | cross-compilation triple for `zig cc` (see compile) | | `--linker DRIVER` | `clang\|zig` | link driver (see compile) | | `--skip-check` | | skip the type-checker preflight | | `--checker CHECKER` | `pyrefly\|mypy\|pyright` | type-checker preflight (default `pyrefly`) | | `--no-cache` | | always recompile (env PYSC_NO_RUN_CACHE=1) | | `--ffi MANIFEST` | `MANIFEST` | bind a typed extern manifest (TOML) and link its C sources; repeatable | ### `assess` ```sh pyscriptc assess SOURCE [options] ``` | Flag | Value | Effect | | ---- | ----- | ------ | | `--json` | | print the assessment report as JSON | | `--all` | | print the full report without capping refusals (default top-3 per root cause) | | `--island-engine ENGINE` | `ENGINE` | island engine to report the capability surface against (same syntax as compile --island-engine); when the engine carries a manifest, the report gains an engine capability section; also read from [tool.pyscriptc.island] engine | | `--bench` | | time CPython against native binary on compilable entry points and print ratio | | `--verify` | | run differential verify on compilable entry points and show receipt inline | | `--call FUNC` | `FUNC` | function to call on entry points, not the program entry | | `--args ARGS` | `ARGS` | Python literal args list for --call (default `[]`) | | `--input FILE` | `FILE` | feed FILE's bytes on stdin to verify and bench | | `--timeout S` | `S` | seconds each run may take (default 60) | | `--program` | | treat the directory as a program | | `--driver DRIVER` | `clang\|zig` | C toolchain driver (see compile) | | `--target TRIPLE` | `TRIPLE` | cross-compilation triple for `zig cc` (see compile) | | `--linker DRIVER` | `clang\|zig` | link driver (see compile) | | `--skip-check` | | skip the type-checker preflight | | `--checker CHECKER` | `pyrefly\|mypy\|pyright` | type-checker preflight (default `pyrefly`) | | `--no-cache` | | always recompile (env PYSC_NO_RUN_CACHE=1) | | `--ffi MANIFEST` | `MANIFEST` | bind a typed extern manifest (TOML) and link its C sources; repeatable | ### `coverage` ```sh pyscriptc coverage SOURCE [options] ``` | Flag | Value | Effect | | ---- | ----- | ------ | | `--json` | | print the report as JSON | | `--fail-on FAIL_ON` | `blockers\|any` | exit 1 when root refusals (blockers) or any refusal (any) exist; default always exits 0 | ### `infer` ```sh pyscriptc infer SOURCE [options] ``` | Flag | Value | Effect | | ---- | ----- | ------ | | `--json` | | print the report as JSON | | `--overlay FILE` | `FILE` | write the inferred annotations to FILE as a sidecar the user can apply | | `--bridge BRIDGE` | `pyrefly\|own` | inference arm to run (default: pyrefly) (default `pyrefly`) | ### `config` ```sh pyscriptc config SOURCE [options] ``` | Flag | Value | Effect | | ---- | ----- | ------ | | `--json` | | print the resolved config as JSON | ### `explain` ```sh pyscriptc explain CODE [options] ``` ### `imports` ```sh pyscriptc imports SOURCE [options] ``` | Flag | Value | Effect | | ---- | ----- | ------ | | `--json` | | print the report as JSON | | `--reach FILE` | `FILE` | append the embedded-CPython reach share to a JSON file (the compat ledger reads it) | | `--island-engine ENGINE` | `ENGINE` | island engine to report the capability surface against (same syntax as compile --island-engine); when the engine carries a manifest, imports gain an engine availability note; also read from [tool.pyscriptc.island] engine | ### `reduce` ```sh pyscriptc reduce SOURCE [options] ``` | Flag | Value | Effect | | ---- | ----- | ------ | | `--predicate CMD` | `CMD` | shell command run on each candidate file; it exits 0 when the candidate still reproduces (the candidate path arrives in $PSC_SOURCE) | | `--fence CODE` | `CODE` | shorthand: compiling still reports a refusal with this code | | `--crash` | | shorthand: compiling still raises a non-refusal error | | `--timeout TIMEOUT` | | seconds per --predicate command (default 120) | | `--steps STEPS` | | cap on removal attempts (default 10000) | | `--print` | | print the reduced source instead of writing FILE.min.py | | `--json` | | print the removal log as JSON | ### `build` ```sh pyscriptc build DIRECTORY [options] ``` | Flag | Value | Effect | | ---- | ----- | ------ | | `--script SCRIPT` | | build only this console script; default every script | | `--out-dir DIR` | `DIR` | write the binaries to DIR instead of /build | | `--driver DRIVER` | `clang\|zig` | C toolchain driver (see compile) | | `--target TRIPLE` | `TRIPLE` | cross-compilation triple for `zig cc` (see compile) | | `--linker DRIVER` | `clang\|zig` | link driver (see compile) | | `--skip-check` | | skip the type-checker preflight | | `--checker CHECKER` | `pyrefly\|mypy\|pyright` | type-checker preflight (default `pyrefly`) | ### `logs` ```sh pyscriptc logs [options] ``` ### `telemetry` ```sh pyscriptc telemetry ACTION [options] ``` ### `doctor` ```sh pyscriptc doctor [options] ``` | Flag | Value | Effect | | ---- | ----- | ------ | | `--json` | | print the report as JSON | ### `deps` ```sh pyscriptc deps SOURCE [options] ``` | Flag | Value | Effect | | ---- | ----- | ------ | | `--json` | | print the report as JSON | | `--frozen` | | use the existing lock, never resolve | | `--upgrade` | | re-resolve to the newest allowed versions | | `--offline` | | no network: uv offline, cached wheels only | | `--no-download` | | classify from wheel tags only | | `--cache-dir DIR` | `DIR` | cache root (env PYSC_CACHE_DIR, default ~/.cache/pyscriptc) | ### `embed` ```sh pyscriptc embed fetch [options] ``` #### `embed fetch` ```sh pyscriptc embed fetch [options] ``` | Flag | Value | Effect | | ---- | ----- | ------ | | `--platform PLATFORM` | `PLATFORM` | pack platform (default: this machine) | | `--force` | | re-download even when the pack is cached | | `--pypy` | | fetch the pinned PyPy runtime pack (the island engine's) | ## Notes on `compile` `--emit bin` and `--run` refuse mobile targets with `PS4101` (see `pyscriptc explain PS4101`). When compilation stops on unsupported constructs, pyscriptc prints the root cause and counts the cascade (`and N more`) rather than printing every consequence. Use `pyscriptc coverage FILE` to see every individual code. `PYSC_CC`, `PYSC_LINKER` and `PYSC_TARGET` fill `--driver`, `--linker` and `--target` when the flag is absent; an explicit flag always wins. ## Notes on `verify` Exit codes: 0 when every check matches, 1 on a divergence, 3 when the compiler refuses the program, 4 when no CPython 3.14 oracle is available. The oracle is resolved from `PYSC_ORACLE` (an explicit path, refused if it is not 3.14), `PYSC_ORACLE_DIR`, `python3.14` on PATH, `uv python find 3.14`, then the interpreter running pyscriptc. It is probed before use: verify never compares a binary against the wrong interpreter. Verify is a differential smoke test, not a proof. It covers the paths the argv and stdin exercised in that run. Code either compiles to native code or runs through embedded CPython (fallback mode), so a divergence is evidence to inspect, not automatically a compiler bug. `--call` compares the call, not the module body: the compiled main runs the global initializers and then the function, so the oracle side hides the output of the module's top-level statements. ## Notes on `assess` On a single script, assess reports whether it compiles fully, partly, or is refused, and `--bench` prints the measured speedup against CPython when an entry point exists. On a library directory it inspects every module without needing an entry point. On an application directory (one with `__main__.py` or `main.py`) it evaluates the whole package tree and prints the `[tool.pyscriptc.imports]` lines that route the unsupported imports to embedded CPython. See [Config](config.md) for what those lines go into. ## Notes on `coverage` Needs no C toolchain. The text view carries a capability line per definition: `native` (compiles), `fenced` (a rewrite clears it) or `island` (routes to embedded CPython). The JSON has the same split under `capabilities.byFile` and `capabilities.totals`. ## Notes on `reduce` The result is written to `FILE.min.py` beside the input; the input is never modified. Exit code is 0 when the final variant still satisfies the predicate. ## Notes on `doctor` `doctor` reports the type checker it found (argv, version, how it was resolved), clang and zig paths and versions, the cache and log directories, the default target, and the install shape (`wheel`, `tool`, `binary`, `source`). Paste this when filing an issue. ## Notes on `embed` `embed fetch` downloads the runtime pack recorded in the runtime lock, verifies its sha256, and unpacks it under `~/.cache/pyscriptc/runtime/`. A `--embed-python=bundle` compile without a cached pack names this command before downloading. `--force` re-downloads a cached pack; `--platform` fetches for another machine. The pack itself, its pinned versions and the licenses that travel with it are on [Runtime pack](runtime-pack.md). --- ## [Config](https://pyscriptc.pages.dev/docs/reference/config/) --- title: Config description: "The pyscriptc config surface: pyscriptc.toml, pyproject.toml and PEP 723." --- # Config One config file feeds every kind of project: a single-file script, a multi-module tree, or a pyproject package. Sources, nearest first: 1. `pyscriptc.toml` at or above the source, holding a `[pyscriptc]` table (or the keys at the file root). 2. `pyproject.toml` at or above the source, holding a `[tool.pyscriptc]` table. A pyproject without that table does not shadow anything; discovery keeps walking up. 3. The script's own PEP 723 inline metadata (`# /// script`), read for `requires-python` and `dependencies`. Nearest directory wins between the two TOML sources. Project config wins over PEP 723 for the keys it sets; PEP 723 is the only source of script dependency metadata. ## Keys ```toml [tool.pyscriptc] entry = "main" # entry point used by project mode src = "src" # module root(s), project mode checker = "pyrefly" # preflight type checker driver = "clang" # C toolchain driver target = "" # zig cc cross-compilation triple fail-on = "any" # exit rule for pyscriptc coverage stub-path = "stubs" # extra stub roots, relative to this file ffi = "ffi.toml" # extern manifests, relative to this file [tool.pyscriptc.imports] orjson = "replace" # curated native replacement sqlite3 = "extern" # C ABI via an ffi.toml manifest numpy = "island" # embedded CPython, marks the binary as libpython-linked rich = "refuse" # always a compile error naming the lane "*" = "compile" # default [tool.pyscriptc.island] embed = "auto" # host | bundle | auto: where the embedded CPython's libpython comes from stdlib = "safe" # safe | minimal: stdlib trim of a bundle ``` The accepted keys, their types and their allowed values: | Key | Type | Values | What it does | | --- | ---- | ------ | ----------- | | `entry` | `string` | | entry point function or module stem, used by project mode | | `src` | `string` or `list` | | module root, used by project mode | | `checker` | `string` | pyrefly or mypy or pyright | type checker run before compiling | | `driver` | `string` | clang or zig | C toolchain driver | | `target` | `string` | | cross-compilation triple for `zig cc` | | `fail-on` | `string` | blockers or any | exit code rule for `pyscriptc coverage` | | `stub-path` | `string` or `list` | | extra stub roots, relative to this file | | `ffi` | `string` or `list` | | extern manifests, relative to this file | | `imports` | `table` | | import lane per top-level module name | | `fallback` | `table` or `string` or `boolean` | | fallback mode settings: `definitions`, or the tier `cpython` | | `island` | `table` | | embedded CPython settings: `embed`, `stdlib`, `engine`, `allow_host_libpython` | | Lane | What it does | | ---- | ------------ | | `compile` | compile from source, and stop with a lane hint when it fails | | `replace` | a curated native replacement covers this import | | `extern` | bound through the C ABI, with an `ffi.toml` manifest | | `island` | run through embedded CPython; the binary needs libpython | | `refuse` | refuse the compile with a message naming the lane | Without a matching entry the lane is `compile`. Unknown keys, wrong value types and unknown lanes are `PS3102` errors naming the offending key (`pyscriptc explain PS3102`). ## Import lanes `imports` maps a top-level module name to a lane. An exact name wins over `*`; without either, the lane is `compile`, so the zero-config single-file product is unchanged. The `refuse` lane is enforced while compiling: an import routed to `refuse` stops the compile with `PS2002`, naming the lane and the policy site. The other lanes are recorded in config and reported by `pyscriptc config`. The `island` lane is what `--fallback=cpython` (alias `--island`) acts on; see [Runtime pack](runtime-pack.md) for what a libpython-linked binary carries. ## PEP 723 A script may open with: ```python # /// script # requires-python = ">=3.14" # dependencies = ["orjson>=13"] # /// ``` The block is parsed as TOML. More than one `# /// script` block is an error. `requires-python` and `dependencies` are validated and reported by `pyscriptc config`; resolving the dependencies is the separate `pyscriptc deps` command. Each dependency is also mapped to a lane, shown by `pyscriptc config` as `dependency-lanes` in its JSON. The `[imports]` table wins when it names the distribution or its import name; otherwise the offline import classifier decides. ## Project mode A pyproject with a `[project.scripts]` table builds one binary per console script: ```sh pyscriptc build [DIR] # every script pyscriptc build DIR --script NAME # one script ``` The binary is written to `/build/` (or `--out-dir`). The module path resolves against the project's module roots: `src`, default `src`, falling back to the project root when the directory does not exist (a flat layout). `pyscriptc config DIR --json` reports the scripts. ## CLI interaction `pyscriptc compile` fills unset `--checker`, `--driver`, `--target` and `--fail-on` from config; an explicit flag always wins, and a note on stderr names the config origin. `pyscriptc coverage` reads `fail-on` the same way. Run `pyscriptc config SOURCE` to see the resolved merge for any source. --- ## [Diagnostics](https://pyscriptc.pages.dev/docs/reference/diagnostics/) --- title: Diagnostics description: Every PSxxxx diagnostic code, what it means, and how to change your code. --- # Diagnostics When a compile stops, pyscriptc names a code and prints the line that caused it. Run `pyscriptc explain PS2001` (or any code) to see one entry in the terminal. Codes are grouped by kind: `PS0xxx` are compiler internals (never your fault), `PS1xxx` are source and type problems, `PS2xxx` are constructs and calls the subset does not take, `PS3xxx` are environment and configuration problems, and `PS4xxx` are extern, target and object-model refusals. A code is never reused or renumbered; the stability contract is in [Stability and contracts](stability.md). ### Compiler internals | Code | What it means | How to change your code | | ---- | ------------- | ---------------------- | | `PS0001` | The source or the generated IR failed a structural check before the backend could run. | Fix the input path or syntax error named in the message. | | `PS0101` | The IR mixes types the emitter cannot pair: operands of one operator disagree, a compare does not yield bool, a condition is not bool, or a returned value has another type than the function returns. Debug-mode verification reports this before the backend runs. | This is a compiler bug: reduce the source and report it with the debug dump. | | `PS0102` | A name the IR reads is not a parameter, local or env slot of its function, or a call names a function or class the module does not define. Debug-mode verification reports this before the backend runs. | This is a compiler bug: reduce the source and report it with the debug dump. | | `PS0103` | A statement follows a return or raise that ends its body, or a function with a value return type has a path that reaches the end without one. Debug-mode verification reports this before the backend runs. | This is a compiler bug: reduce the source and report it with the debug dump. | | `PS0104` | The LLVM module verifier rejected the emitted text (a missing terminator, an undefined value, a malformed instruction). Debug mode runs the verifier over the .ll it dumps. | This is a compiler bug: attach the --debug-dir dump when reporting it. | ### Source and types | Code | What it means | How to change your code | | ---- | ------------- | ---------------------- | | `PS1001` | A parameter, return, or variable needs an annotation the subset can read, or an annotation form has no more specific code. | Annotate the name with int, float, bool, str, bytes, list[T], dict[K, V], tuple[...], or a class the compiler takes. | | `PS1003` | An operation received a type the compiler does not accept for it. | Convert the operand explicitly, for example int(flag) before math calls. | | `PS1004` | A function with a non-None return type has a path that reaches the end without a return. | End every path with a return or a raise. | | `PS1101` | The IR has no tagged union yet, so `A | B` and `Union[A, B]` have no representation. | Narrow the annotation to one arm, or split the function per arm. | | `PS1102` | An optional of one type (`T | None`, `Optional[T]`) compiles. `A | B | None` and `Union[A, B, None]` need a multi-arm union, which the IR does not have yet. | Narrow to `T | None` with one arm, or split the function per arm. | | `PS1103` | A subscripted annotation such as Mapping[K, V], Sequence[T], Iterator[T], Callable[...], or type[T] is not a container the compiler takes. | Use list[T], dict[K, V], or tuple[...] in the signature. | | `PS1104` | object, Any, Protocol, TypeVar, Generic, Callable and ABC bases name dynamic shapes the IR has no slot for. Typing names are annotation-only: they steer compilation, they are never runtime values. | Annotate with the concrete type the function really takes. | | `PS1105` | A quoted annotation such as 'Node' is not resolved at compile time. | Define the class before use and write the name unquoted. | | `PS1106` | The annotation is a plain name that is not a builtin scalar, a compiled class, or a known alias. | Import the concrete class, or replace the alias with the concrete type. | | `PS1107` | dict[K, Any] and list[Any] read their element back as a value of no proven type. A read nothing proves fences: the IR cannot pick a slot or a load width for it. Assign, pass or annotate the element into a typed slot first. | Annotate the read target with a concrete type, e.g. `x: int = d[k]`. | | `PS1201` | Only a closed set of decorators compiles. Other decorators change behavior the compiler cannot see. | Remove the decorator, or inline what it does. | | `PS1202` | Positional-only, keyword-only, *args and **kwargs parameters are not supported. An unannotated **kwargs parameter needs a runtime keyword mapping. | Use plain positional parameters, with defaults where needed. An **kwargs parameter runs through embedded CPython (fallback mode); declare its keys with PEP 692 Unpack[TypedDict]. | | `PS1203` | Class keywords (metaclass=...) and multiple inheritance are not supported. A metaclass builds the class at run time and native code fixes the layout at compile time. | Use a single base class the compiler takes and no class keywords. Write the class explicitly, or use PEP 695 generics instead of metaclass synthesis. | | `PS1204` | A class body holds only annotated fields without values, method definitions, assignments whose value has a runtime form (a call, a container, a comprehension, a name bound at module scope), and assignments whose value folds to a constant. | Move an assignment the compiler cannot compute into __init__, or declare the field with an annotation only. | | `PS1205` | `except E as name` and bare `except:` are not supported. | Use a compiled exception class and drop the `as` binding. | | `PS1206` | `with ... as name` compiles only for a closed set of context managers. | Use a supported context manager without the binding, or open and close explicitly. | ### Constructs and calls | Code | What it means | How to change your code | | ---- | ------------- | ---------------------- | | `PS2001` | The module imports, but this attribute is outside the surface the compiler takes. An attribute that reads runtime metadata (inspect.signature, __annotations__) is runtime reflection and routes the definition to embedded CPython. | Use an attribute of the module the compiler takes. For reflection, use the typed idiom the message names, or compile with --fallback=cpython. | | `PS2002` | A statement, expression, import, or definition has no compiled form. Dynamic forms such as **kwargs keyword forwarding or an annotation read at run time are runtime reflection and cannot be compiled natively. | Rewrite it with the typed idiom the message names (PEP 692 Unpack[TypedDict] for **kwargs), or compile with --fallback=cpython. | | `PS2003` | A call has keyword arguments, or the wrong number of arguments, for the form the compiler takes. | Pass positional arguments in the arity the compiler takes. | | `PS2004` | The operator is not supported for these operand types. | Convert the operands to a supported pair of types. | | `PS2005` | Strings are byte sequences in the runtime, so only Latin-1 text compiles. | Use Latin-1 text, or move the literal to bytes. | | `PS2006` | A user decorator compiles as a direct call of a function value, so the decorator expression must be typed as a function that takes the decorated function and returns one. An expression such as `@click.command()` returns a class instance, an untyped value or an embedded-CPython value, which carries no parameter list to check the wrapped function against. | Use a decorator typed as a function returning a function, or apply the wrapper explicitly in the function body. Under `--fallback=cpython` the decorated definition runs through embedded CPython. | | `PS2101` | The module is planned but not compiled yet. | Avoid the module, or wait for the queue entry to land. | ### Environment and configuration | Code | What it means | How to change your code | | ---- | ------------- | ---------------------- | | `PS3001` | The running interpreter is not the CPython version the subset targets. | Run under the pinned CPython version. | | `PS3002` | The emitted program exceeds a fixed limit of the backend. | Reduce the offending size or count. | | `PS3099` | A class decorator other than `@dataclass`, or a method/function decorator outside the closed set that does not synthesize behavior, compiles as erased: the definition lowers undecorated and the wrap never runs, where CPython applies the decorator to the object (or replaces the name). The compile prints a note and succeeds; the binary keeps the definition's own behavior. A decorator stack that carries a behavior-synthesizing member (`functools.lru_cache`, `cache`, `wraps`, `singledispatchmethod`) keeps its fence instead: the erase would drop the synthesis with it. | Remove the decorator, or apply what it does explicitly after the definition. Under `--fallback=cpython` the decorated definition runs through embedded CPython with the decorator applied. | | `PS3101` | `pyscriptc deps` could not read, resolve, download or verify a PyPI dependency (bad PEP 723 or pyproject metadata, sdist-only package, stale lock under --frozen, hash mismatch). | Fix the metadata or lock as the message says; pyscriptc never builds sdists, so pick a version with a wheel. | | `PS3102` | A `pyscriptc.toml`, a `[tool.pyscriptc]` table, or PEP 723 script metadata failed schema validation. | Fix the key named in the message; `pyscriptc explain` lists no other config diagnostics. | ### Externs, targets and the object model | Code | What it means | How to change your code | | ---- | ------------- | ---------------------- | | `PS4001` | The manifest passed to --ffi could not be read or is not valid TOML. | Fix the file path or the TOML syntax named in the message. | | `PS4002` | The [manifest] section is missing, is not a table, or its module name is not a dotted identifier. | Add [manifest] with a module, ps_abi and format key. | | `PS4003` | The manifest's format pin is missing or names a schema version this compiler does not read. | Rewrite the manifest against a supported format version. | | `PS4004` | The manifest's ps_abi pin is missing or does not match the PS_ABI epoch this compiler stamps. | Regenerate or update the manifest for the compiler's ps_abi epoch. | | `PS4005` | A manifest key is missing, has the wrong type, or is not part of the schema. | Keep the manifest to the keys documented in .agents/docs/ffi/ffi-manifest.md. | | `PS4006` | A py signature or type in the manifest is outside the vocabulary the FFI lane marshals. | Use the scalar and container types, or a declared handle, from .agents/docs/ffi/ffi-manifest.md. | | `PS4007` | A name is declared twice in one manifest, or two manifests claim the same module. | Give each handle, const, function and module a unique name. | | `PS4008` | An ownership annotation or handle reference names something the manifest does not declare. | Declare the handle, or fix the reference to a declared handle or parameter. | | `PS4009` | A [[source]] entry names a C file, archive or object that does not exist on disk. | Point the manifest at the file that ships with the adapter, or fetch the vendored C first. | | `PS4010` | A call whose manifest entry lists consumes took ownership of the handle, so the variable is dead: reading it would use a freed pointer. | Do not use the handle after its consuming call. Open the resource again, or use what the consuming call returns. | | `PS4011` | The call consumes a handle, but the argument is not a handle variable this function owns (a literal, a call result, or a temporary), so the compiler cannot prove the value is dead afterwards. | Bind the handle to a local variable first, then pass that variable to the consuming call. | | `PS4012` | Two handle types are both opaque pointers at the ABI, so passing one for the other would be accepted by the link and corrupt memory. | Pass the handle type the manifest signature names; open the resource with the right connect call if needed. | | `PS4013` | The manifest declares a handle but no function consumes it, so a scope-exit free has no thunk to call and the handle would leak. | Add a [[fn]] whose consumes names this handle (a close or finalize wrapper), or pass every handle to the consuming call explicitly. | | `PS4014` | A manifest handle names a type, like a class name. The value is an opaque pointer that only crosses FFI calls. | Annotate a variable with the handle type, or pass it to a manifest function; never use the handle name itself as a value. | | `PS4015` | A handle is an opaque pointer at the ABI: it has no fields, and a read would be a wild access into foreign memory. | Pass the handle to a manifest function instead of reading its fields; the adapter's thunk is where the C struct is touched. | | `PS4016` | Arithmetic on a handle is pointer arithmetic on a foreign object. The ABI passes handles as opaque tokens and never computes with them. | Use the handle only as a token: pass it to manifest calls. Convert a C-side value to an int through a manifest function if you need a number. | | `PS4101` | The target (an iOS device, the iOS simulator or Android) has no standalone executable: a mobile app links a static library, so a `--emit bin` build has nothing to run on the device. | Use `--emit lib --target ` to produce libpyscriptc.a plus pyscriptc.h and module.modulemap, and call pyscriptc_init() from the app. | | `PS4201` | `__setattr__`, `__getattribute__`, `__get__` and `__set__` intercept attribute access at run time. Attribute access is resolved statically to declared fields, so a hook that can intercept any name lookup or write has nothing to run in, and the hook would be silently dropped or silently bypassed. | Use plain annotated fields and methods. `@property` getters compile; `__getattr__` compiles for unknown-name reads (issue #434); unknown-name delegation otherwise becomes an explicit method call. | | `PS4202` | `__delattr__` intercepts `del obj.name`, which the compiled subset does not model: fields are fixed at compile time and never removed, so the hook would never run. | Use plain annotated fields. Model 'removed' with a sentinel value or a flag field. | | `PS4203` | `__del__` is a finalizer: CPython calls it when the object is garbage-collected, and that timing is explicitly unspecified. The runtime has no GC to trigger it, so the method would be silently dropped and the cleanup would never run. | Use `with` and an explicit `close()` for resource lifetime. | | `PS4204` | An unknown-name read on a class with `__getattr__` compiles as a direct call of the hook with the attribute name as a compile-time constant. The hook's declared return type is what the read gets, so it must cover the read site's expected type exactly; a hook that returns `None`, `bool` or an optional cannot feed an arbitrary read. | Narrow the hook's return annotation to one value type that every read site expects, or split the class: one `__getattr__` per return shape, or explicit methods instead of the hook. | | `PS4303` | The island engine selected for this build implements an older language level than the compiler targets (PyPy 3.11 has no PEP 649 deferred annotations, no PEP 695 type parameter lists, no PEP 750 t-strings and no PEP 758 `except A, B:`), so the construct cannot run on that engine. | Rewrite the construct in syntax the engine's level has, or build the program against the embedded CPython engine. | | `PS4304` | The produced binary links the libpython of the interpreter that built it, by absolute path. It runs on the build machine and breaks on a target that has no such Python, and the build output says so now: the post-link audit reads the artifact's dynamic dependencies (otool -L, readelf -d) after the link. | Rebuild with `--embed-python=bundle` to ship the pinned runtime next to the binary (relocatable), or pass `--allow-host-libpython` to accept the host link on purpose. | ## What a refusal means for your program A `PSxxxx` code is not an error inside a working program: it means this one construct has no compiled form yet. You have two ways forward: - Change your code so the construct fits the subset (the table above names the change for each code). The page to check what the subset takes is [The Python subset](../guide/python-subset.md). - Compile with `--fallback=cpython` (alias `--island`) so the functions that carry the code run through embedded CPython (fallback mode) instead. This needs libpython and the packages on the host; see [Runtime pack](runtime-pack.md). Runtime differences that survive a successful compile are not diagnostic codes; they are the numbered list in [Divergences from CPython](divergences.md). --- ## [Divergences from CPython](https://pyscriptc.pages.dev/docs/reference/divergences/) --- title: Divergences from CPython description: The parity contract and every numbered difference from CPython 3.14. --- pyscriptc targets CPython 3.14.x. Parsing, probes and the compatibility oracle follow 3.14. Compiled programs match CPython 3.14 on stdout for everything that compiles, except the numbered divergences below. The list is load-bearing: anything not listed here is claimed identical, so a difference you find that is not listed is a bug. Constructs pyscriptc cannot compile are not divergences. They stop the compile with a `PSxxxx` code, and every code is in [Diagnostics](diagnostics.md). ## Parity contract - stdout: byte-exact for every compiled call. The corpus scorer compares whitespace-stripped stdout; the nightly strict corpus A/B lane (`scripts/strict_corpus_ab.py`) compares untrimmed bytes over the visible corpus and reports every difference it finds. - stderr: for a `raises` probe, only the exception name must appear. Full traceback text is not matched. - exit code: an uncaught exception prints its name to stderr and exits 1, as CPython does. An uncaught `SystemExit` follows CPython's own rule instead: an `int` value is the exit status, a `str` value is written to stderr and the process exits 1, and no value exits 0, with no class name or traceback text printed. `raise SystemExit() from ` does not compile (`PS2003`). ## The tool host and the target The compiler runs on CPython 3.11 through 3.15 (`requires-python = ">=3.11"`); the program it emits claims CPython 3.14 (`src/pyscriptc/target.json`). The two are separate, and the split is the contract: - The **source grammar** is the host's. The compiler parses user source with the host interpreter's `ast`, so a source the host cannot parse is a `SyntaxError` before pyscriptc sees it. Syntax newer than the host (PEP 646 `def f[*Ts]`, for example) needs a host that knows it; a 3.11 host refuses such a source where a 3.14 host lowers it. - The **emitted semantics** are the target's. The error wording (`division by zero`), the baked `sys.version` constant, the `sys.version_info` branch folds, the typeshed version gate, the dependency resolution's Python minor and the runtime pack's stdlib tree all read `target.json`, not the host. A 3.11 host and a 3.14 host emit the same program for the same source; the canonical IR of the whole probe corpus is byte-identical across 3.11, 3.13, 3.14 and 3.15 hosts once the `cpython` stamp line is stripped (the stamp names the lowering interpreter by design, and the typed hash serializer (`pyscriptc/tirhash.py`) never emits it, so the IR hash gate is unaffected). - The **island** mirrors the host it links. A program that embeds CPython runs the interpreter the binary carries, and `sys.executable` is the binary's own path (divergence 41 and 42 describe the embedded interpreter's version). - The **verify oracle** still needs a 3.14 interpreter. The PS3001 refusal names `uv python install 3.14`. Two constants are deliberately *not* target-pinned, because they describe the machine rather than the program: `sys.executable` (an environment path, which the IR hash serializer classifies as incidental) and the host's own `sys.platform`. ## Numbered divergences Every entry has a row and a probe in `divergence-ledger.json`, regenerated by `scripts/gen_divergence_ledger.py`; a test replays each row's recorded bytes, so a divergence that is fixed or that moves is a reviewed change. 1. `int` is `i64`. Overflow aborts the process instead of growing a big int. 2. `//` and `%` by zero raise `ZeroDivisionError` matching CPython 3.14's `division by zero` (or 3.13's variant on a 3.13 host), rather than aborting the process. 3. The `str` model is mid-migration to the CPython code-point model (epic [#472](https://github.com/sigilco/pyscriptc/issues/472); this item is rewritten again when it lands). A `str` is a UTF-8 byte buffer with a code-point count and an ASCII flag, so `len`, indexing, slicing and the find/count/justify family count code points, the case family applies the full Unicode 16.0.0 mappings, and text output (stdout, text-mode files, stdin reads) is UTF-8 bytes as CPython writes it. A literal may hold any scalar code point; a lone surrogate stops the compile with `PS2005` (no UTF-8 bytes to store). `chr()` with a literal surrogate stops with `PS2005`; at run time, `chr()` with a variable surrogate code point raises `ValueError: chr() arg not in range(0x110000)` where CPython returns the surrogate character. `format` widths and precisions count code points, `repr` follows CPython's escape rules (`repr("café")` is `'caf\xe9'`), `%c` and the `c` spec take a code point, and the six is* predicates plus `isascii` decide on code points. The is* predicates that need Unicode category tables beyond the case data (`isalpha`/`isalnum` on a non-cased code point, `isspace` beyond ASCII) raise `NotImplementedError` naming the predicate and the code point. `translate`, `maketrans`, `istitle`, `isdecimal`, `isnumeric`, `isprintable`, `iscapitalize` do not compile (`PS2001`). Still Latin-1-bound: `unicodedata` covers only Latin-1 code points and a normalization that would leave Latin-1 raises `ValueError`; `re` and `json` are separate items (4 and 5). 4. `re` is the runtime's byte-oriented backtracking subset, not CPython's `re`. `L`, `T`, `DEBUG` and unknown flag bits raise `ValueError`, `A` and `U` are no-ops, and a pattern that can loop without consuming stops with `RuntimeError` once its fuel is spent. Five or more groups are refused. 5. `json` has no non-ASCII text. 6. Set and `frozenset` iteration is insertion order, not CPython's hash order. Length and membership match. 7. After `for i in range(0)`, a new loop target reads as `0` instead of raising `UnboundLocalError`. 8. A subclass `__init__` that never assigns an inherited field leaves it at the zero value of its type instead of raising `AttributeError`. 9. `raise AppError("no")` for a user exception class stores a code and an optional `str` and does not call `__init__`. 10. `@classmethod` and `@staticmethod` of the form `Class.method(...)` do not pass the class object. 11. Some library failures surface as a different exception type: `importlib.resources.files` on a zip or missing package raises `ValueError` rather than `ModuleNotFoundError`, `shutil.copyfile` of the same path raises `RuntimeError` rather than `SameFileError`, other `shutil` and `os` failures (except `os.mkdir`, `os.makedirs` and `Path.mkdir` which raise `FileExistsError`) are `RuntimeError`, and type errors on `unicodedata` calls surface as `ValueError`. 12. `assert` is always evaluated; a compiled binary has no `-O`. 13. `x is None` compiles as `==` against `None`. Identical for `None` because it is a singleton; every other `is` on a scalar is refused. 14. `ABC`, `ABCMeta`, `Protocol`, `@abstractmethod` and `@overload` compile as plain classes and methods with no CPython runtime semantics. 15. Calls that run through embedded CPython (`subprocess.check_output`, `socket.gethostname`, `threading`, `logging`) follow the interpreter the binary links, not the pin. 16. `dotenv` compiles a closed subset of python-dotenv's parsing rules, and it differs from the library in three places. The subset: a UTF-8 BOM is stripped, blank lines and whole-line comments are skipped, an `export ` prefix is allowed, keys are bare (`KEY` alone binds `None`), an unquoted value runs to the end of the line with the ` #` comment cut and the rest right-stripped, a double-quoted value decodes `\" \\' \\ \a \b \f \n \r \t \v` and a single-quoted one only `\\` and `\'`, and a later binding overwrites an earlier one. No interpolation (`$NAME` stays literal), no multi-line values, no quoted keys, and the `stream`, `verbose`, `interpolate`, `override` and `encoding` arguments are refused. The three differences: a malformed line raises `ValueError` where python-dotenv skips it with a warning, a missing file parses as the empty document where python-dotenv searches with `find_dotenv()`, and `load_dotenv` does not overwrite a variable that is already in the environment (its `override=False` default) while still returning `True` for a document with bindings. 17. `yaml.safe_load` covers a closed YAML subset, not all of PyYAML: timestamps and sexagesimal numbers (`1:30`) raise `ValueError` where PyYAML resolves them to `date` and `int` objects, and errors surface as `ValueError` rather than `yaml.YAMLError`. 18. Plaintext `http://` is native: the compiled program makes the request itself, with no subprocess and no host `curl`. `https://` rides the system libcurl, linked at build time as a shared library (issue [#235](https://github.com/sigilco/pyscriptc/issues/235)): a program whose emitted IR reaches an `https://` URL gains libcurl's flags (resolved through `curl-config --libs` or `pkg-config libcurl`) on its link line, so the libcurl development package is a build dependency of such a program and the compile fails with a diagnostic naming the package to install when neither tool finds it (`apt install libcurl-dev` on Debian/Ubuntu, `brew install curl` on macOS, `dnf install libcurl-devel` on Fedora). `PSC_CURL_DYNAMIC=1` opts out of the build-time link; the `https://` request then raises `OSError` at run time where CPython makes the request. Peer and hostname verification stay on in every shape; there is no insecure fallback. An `http://`-only program carries no libcurl dependency. See [HTTP](https://pyscriptc.pages.dev/docs/guide/http/). 19. `generator.send(v)` on a generator that has not started ignores `v` and runs the body from the top, where CPython raises `TypeError: can't send non-None value to a just-started generator`. `None` (the priming `send`) behaves identically in both. 20. `generator.throw(E)` marks the frame done and raises `E` out of the call site. CPython's "the generator catches it, execution continues" outcome is not reachable through this surface: `throw` is a separate entry point that never runs the generator's own `except` handlers. `GeneratorExit` semantics do not apply either: `close()` marks the frame done without running a pending `finally`, where CPython raises `GeneratorExit` inside the generator so its `finally` runs. 21. `yield from ` drains the subgenerator and discards its return value (`StopIteration.value`), where CPython's `yield from` expression evaluates to it. Statement-form `yield from` (the only form that compiles) never reads the value, so no observable program differs. A `return ` in a generator compiles (the value rides in the frame) but no surface reads it back. 22. The curated `pyscriptc-jwt` distribution package compiles the HMAC family only: `HS256`, `HS384` and `HS512` through `jwt.encode` and `jwt.decode` over a `dict[str, str]` payload. RS/ES algorithms and a non-literal algorithm argument stop the compile where PyJWT signs with a private key, claims beyond `str` values (`exp`, `aud`) are refused, errors surface as `ValueError` rather than PyJWT's `InvalidTokenError` hierarchy, and a multi-claim payload is emitted with the claims sorted by key where PyJWT emits insertion order (decode does not depend on the order; single-claim tokens are byte-identical to PyJWT's). 23. The curated `pyscriptc-orjson` distribution package compiles `orjson.dumps` and `orjson.loads` over the same C JSON writer and reader as `json`: compact `bytes` output, `OPT_SORT_KEYS` and `OPT_INDENT_2`, with the `json` subset's value shapes. The `default=` callable, every other `OPT_*` flag, `orjson.Fragment` and orjson's extended types (`dataclass`, `UUID`, `datetime`, `numpy`) are refused, and errors surface as `ValueError` rather than `orjson.JSONDecodeError`. 24. `@contextmanager` runs the body's code after the `yield` when the `with` exits cleanly, and re-raises the in-flight exception when the body raises, but the CPython `gen.throw` path that lets the generator catch the exception is unreachable: the manager's `__exit__` marks the frame done instead of throwing into it, so a `try`/`except` inside the generator cannot see the in-flight exception. 25. A `finally` that is pending when the generator is abandoned does not run: `close()` marks the frame done and drops the pending `finally`, where CPython raises `GeneratorExit` inside the generator so its `finally` runs and its cleanup happens. A `finally` that is reached because the generator runs to completion or raises through the `try` behaves exactly like CPython's. 26. The threading surface that needs a second thread (`threading.Thread`, `threading.Condition`, `threading.Semaphore`, `threading.Event`, `queue.Queue`) is native-only: on a `wasm32` target the link is refused with a diagnostic naming the target and the runtime symbols it cannot use, because wasi-libc stubs `pthread_create` to return `EAGAIN` and its `pthread_cond_wait` traps. The `threading.Lock` primitives that need no second thread are unaffected. 27. The HTTP client is a closed subset of `urllib.request`: only `urlopen` and `Request` compile. `urlopen` accepts `url`, `data` and `timeout`; headers ride the `Request` as a dict display with constant `str` keys and values, so runtime-built header dicts are refused. `urlretrieve`, openers and proxy handling are refused. The error types are synthetic `HTTPError` and `URLError` records that match through the `OSError` chain, not the full `urllib.error` class surface. See [HTTP](https://pyscriptc.pages.dev/docs/guide/http/). 28. `https://` raises `OSError` in the compiled program: the native client speaks plaintext HTTP/1.1 only, because TLS (mbedTLS through the FFI manifest) has not landed. CPython raises nothing and makes the request. Issue [#235](https://github.com/sigilco/pyscriptc/issues/235). 29. `socket`, `ssl`, `http.client` and `http.server` do not compile: an import of any of them stops the compile with `PS2001` naming the module. CPython provides them. Inbound servers and `http.client` keep-alive are issues [#227](https://github.com/sigilco/pyscriptc/issues/227)-[#230](https://github.com/sigilco/pyscriptc/issues/230) and [#237](https://github.com/sigilco/pyscriptc/issues/237); the event loop underneath (`epoll`/`kqueue`) is [#177](https://github.com/sigilco/pyscriptc/issues/177), which is also why `asyncio` I/O without the compiled `run`/`sleep`/`gather`/`create_task`/`wait_for` surface is refused. `selectors` compiles as a closed subset over that loop (`src/pyscriptc/runtime/ps_select.c`, the `DefaultSelector` records over the #177 poller): `DefaultSelector`, `SelectorKey`, `EVENT_READ`/`EVENT_WRITE`, and `register`/`unregister`/`modify`/`select`/`get_key`/`close`. The subset takes a bare int fd where CPython takes a `fileobj` - the typed `socket` handle is [#229](https://github.com/sigilco/pyscriptc/issues/229), and until it lands the fd-carrying object has no type to hand over, so `sel.register(sock, events)` fences with `PS1003` naming #229. Two further shapes differ from CPython: `select()` returns `list[SelectorKey]`, not CPython's `list[tuple[SelectorKey, int]]` pairs, and `get_key` of an unregistered fd returns `None`, where CPython raises `KeyError`. `SelectorKey` carries `fd` and `events`; CPython's `fileobj` and `data` attributes stay outside the subset for the same #229 reason. When #229 lands, `register` reads the fd off the handle and the runtime records do not change. The scope decision is recorded in issue [#485](https://github.com/sigilco/pyscriptc/issues/485). 30. `urlopen` and `Request` are native-only: on the `wasm32` and `*-windows-*` targets the runtime stub raises `URLError` (an `OSError`) at run time with a message naming the target family, because WASI has no sockets and Windows needs winsock. Programs that never call `urlopen` link unchanged for those targets. 31. On `wasm32-wasi-musl` the program is a WASI preview1 command module, and the preview1 surface is smaller than a POSIX host: there is no subprocess (`fork`/`exec` has no WASI story, so `subprocess` calls are refused at link time with a diagnostic naming the target), no `dlopen` and therefore no native-to-embedded call (`zlib.crc32` through that path is refused the same way), no symbolic links (`os.symlink` and `os.readlink` are refused at link time with the same target-naming diagnostic), and no fallback mode (already refused for every cross target). The threading primitives that compile everywhere (`Lock`, `RLock`, `get_ident`, `active_count`, `current_thread().name`) work on the one thread a WASI program has; wasi-libc's pthread stubs never block, and `threading.Thread` is refused on every target. The filesystem is preopen-only, so the wasmtime invocation must pass `--dir` or `-S dir::/path` preopens for the program to see any file; `os.getcwd` and the tempfile helpers write into whatever is preopened. `tempfile.NamedTemporaryFile` and `tempfile.mkstemp` use an `open(O_CREAT|O_EXCL)` port (`ps_mkstemp`) because wasi-libc declares no `mkstemp`; the file lands in whatever preopen holds the temp directory. The emitted entry point is `@__main_argc_argv` on wasm targets, since wasm-ld resolves a C-style `main` only through that name. Everything else is unchanged: the smoke lane proves stdout byte-identical to the native binary on the same program. 32. On `wasm32-wasi-p2` the program is a WASI 0.2 component: the same preview1 core module wrapped by `wasm-tools component new --adapt` with the preview1 adapter pinned from the wasmtime v50.0.0 release (URL, version and sha256 recorded in `src/pyscriptc/wasm_adapter.py`; a fetched adapter whose digest does not match the pin is refused, never used). The component imports `wasi:cli/run@0.2.x` instead of the raw preview1 functions, and every constraint of divergence 31 applies unchanged. There is no thread world to spawn into on 0.2; threading arrives with the WASI 0.3 coop-threads lane, which is a separate deliverable. Generators compile through the wrap unchanged. 33. On a `*-windows-*` target the program is a Win32 PE binary built by cross-compilation, and the Win32 surface is smaller than a POSIX host in ways the runtime cannot paper over: `os.path.islink` is always false and `os.lstat` follows the link instead of reporting it, because Windows has no `lstat`; path handling is Win32-flavoured, so `\` and `/` are both separators, drive letters (`C:\`) are absolute, and `os.path.join` still emits `/`; `os.symlink` makes file links only and `os.readlink` raises `RuntimeError` at run time, because Windows has no CRT `readlink`; `urlopen` and `Request` raise `URLError` at run time because the HTTP client needs BSD sockets and Windows needs winsock. The remaining Windows semantics (CRLF stdout, `input()`, signals, threading) are unverified rather than claimed: the emitted binaries are never executed anywhere, in CI or otherwise. Windows is experimental cross-compilation only; native Windows compiles are not supported yet. 34. Only refusals in modules the compiled tree's import graph reaches stop a compile with dependency source roots (`PyPI` pure wheels, issue [#424](https://github.com/sigilco/pyscriptc/issues/424)). In a program the graph starts at the entry module; in a library package it starts at every module, because any of them can be the consumer's import target. The rule rests on CPython's import-time side effect rule: a module body runs once, at the import that reaches it, and never otherwise, so a dependency module the tree never imports never runs and a refusal inside it cannot change what the compiled program does. Such a module is pruned (named on stderr as a note, not compiled) or kept in the export table without compiling when an unreached program module imports it. A reached dependency refusal stops the compile with its code and the distribution that ships it. The reach walk is static and name-based, so a dynamic import (`importlib.import_module`, `__import__`, both refused) never extends it. The program's own modules keep the whole-tree rule: a refusal in one of them always stops the compile. CPython runs every module whose import appears anywhere in the executed call graph, so a program whose entry reaches that module through a dynamic import behaves differently: CPython refuses at run time or succeeds through the import, where the compile succeeded without the module's code. 35. The `__getattr__` hook compiles for unknown-name reads only, and only for one shape: a plain instance method `def __getattr__(self, name: str) -> T`. The read compiles as a direct call of the first MRO definer with the attribute name as a compile-time constant, and the hook's declared return type must cover the read site exactly (`PS4204` otherwise). Everything else in the dynamic attribute protocol stays refused at the definition site: `__setattr__`, `__getattribute__`, `__get__`, `__set__` (`PS4201`), `__delattr__` (`PS4202`), `__del__` (`PS4203`), and a `__getattr__` in any other shape (a decorator, a different parameter list). Where CPython dispatches a missing attribute through `__getattr__` on any shape, or an attribute write or deletion through the protocol, the compile stops instead. An unguarded read of a name the hook itself reads recurses to no limit in the binary where CPython raises `RecursionError`. Writes (`__setattr__`) are still refused. Issue [#434](https://github.com/sigilco/pyscriptc/issues/434). 36. The shortest float spelling sometimes carries an exponent form CPython does not use at that magnitude: `10.0` prints `1e+01` where CPython prints `10.0` (`100.0` prints `1e+02`). The shortest round-trip search accepts `%g`'s exponent spelling before CPython's fixed-notation preference kicks in, so the value is exact and the bytes differ for whole floats whose shortest spelling picks an exponent. Values whose shortest spelling is fixed-notation (`0.5`, `16.1`, `0.1`) are byte-identical. 37. A `round(x, n)` call whose constant `ndigits` is outside 0 to 8 stops the compile (`PS2003`) where CPython rounds for `-8 <= n <= 20`. A variable `ndigits` outside the range raises `ValueError: round ndigits is out of the supported range` at run time. In range, `round(x, n)` rounds the exact binary value of `x` the way CPython's dtoa path does (`round(2.675, 2)` is `2.67`). 38. `newline=` is refused on `open`, so text-mode reads always apply the default universal-newline translation (`\r\n` and `\r` read as `\n`); a program that needs CPython's `newline=""` (no translation) or a specific newline spelling cannot compile the call. `encoding=` compiles only as a str constant from `{utf-8, ascii, latin-1}` and their firm/short spellings, and it rides the same read path as the no-encoding call, so a file whose content is not representable under the subset's byte read differs from CPython's decode there. CPython accepts any encoding object and honors it. A third positional (`buffering`) and the other `open` keyword arguments are refused. 39. `int(text, base)` compiles for bases 2 to 36 and matches CPython's read and messages (the leading prefix sits behind the sign, base names check like CPython's, the surrounding-whitespace strip takes the Unicode space set, and the underscore digit separator follows CPython's rule: one `_` between two digits, never at an end, never doubled), with the i64 ceiling of divergence 1. Base `0` and out-of-range base constants stop the compile (`PS2003`), where CPython accepts both (base 0 prefixes off the text). One message detail stays open: an invalid literal in the `bytes` form is reported with the `str` repr (`invalid literal for int() with base 10: '1x'`) where CPython writes the `bytes` repr (`b'1x'`), because the runtime reads both forms through one buffer ABI that does not carry the str/bytes distinction. 40. `subprocess.run` takes `capture_output=`, `text=` and `check=` as bool literals: `check=True` raises the `check_call` stand-in `RuntimeError` on a non-zero exit where CPython raises `CalledProcessError` carrying the cmd and returncode. `text=True` (or an `encoding=` name) decodes the captured streams through the runtime's CPython-exact decoder (`utf-8`/`ascii`/`latin-1` with `strict`/`replace`/`ignore`/`backslashreplace`), so the record's `stdout`/`stderr` are str like CPython's text-mode record; the default bytes path is unchanged. `cwd=` compiles as a str, Path or None; `input=`, `shell=` and the remaining popen keyword arguments are refused. 41. Fallback mode (`compile --fallback=cpython`, alias `--island`) routes a function pyscriptc cannot compile to embedded CPython: that function's source segment plus the module's top-level imports and module-global initializers are exec'd once inside the binary (so a global initializer runs again in the embedded interpreter; for a deterministic initializer like `Path.home() / x` the value matches, and a change to a global made there is never visible to native code), and every call to it crosses into the embedded interpreter. Under fallback mode an annotated function that accumulates an unbounded `int` also runs there (issue [#677](https://github.com/sigilco/pyscriptc/issues/677)): a conservative static check finds the growth shape (a `*=`, `<<=`, `**=` in-place update, or the loop-carry assign `name = name anything` with a multiplying operator) and plans that function for the embedded interpreter instead of letting the native i64 overflow of divergence 1 abort it, so an annotated program keeps CPython's arbitrary-precision result where the same function compiled natively would not. A single `name = name ** k` assign with a literal exponent of at most 62 and no loop in the body stays native. Values cross one way per call: scalars, `str`, `bytes` and lists of scalars by value (a list is copied, so a change made there is not visible to native code), `dict` and everything else as an opaque handle usable through attribute reads, item reads, further calls and `str()`/f-string formatting, never by native operators. No object identity survives a crossing. A function running in the embedded interpreter that calls a compiled sibling stops the compile with a diagnostic naming both functions, where CPython would run the call. A class definition that pyscriptc cannot compile (`@attrs.define`, `@dataclasses.dataclass(frozen=True)`, any decorator the native dataclass form does not take, class keywords, a class-body statement) or whose method bodies cannot compile is routed the same way (issue [#628](https://github.com/sigilco/pyscriptc/issues/628)): the class leaves the native class table, the embedded interpreter defines it, and every compiled function that names it is routed there too. A native class method that still uses that class reaches it through the handle path: an instance is a call on the class name, a class-level attribute read goes through the handle path, and an attribute write from compiled code stops the compile because values cross one way. Functions and classes routed there run under the host CPython (3.14.x), not the compiled runtime. 42. With fallback mode on, the module top level is all-or-nothing: the synthesized entry body is probed like a function, and a refused probe (a module-level statement the subset does not take, an unannotated `yaml.safe_load` result, a `Path` global) plans the whole top level for the embedded interpreter instead of stopping the compile. The binary's `main` seeds the embedded interpreter with the module's own source text, exec'd once with `__name__ = "__main__"` (the script's main guard fires there exactly once) and `__file__` set to the compiled source path; `sys.argv` mirrors the process argv, so `sys.argv[0]` is the binary path where CPython shows the script path. `raise SystemExit(status)` out of the body delivers the process status (an int becomes the exit code, a str prints to stderr and exits 1), and any other uncaught exception surfaces as its own type and message with exit 1, matching CPython's shape but not its traceback bytes. Annotated scalar globals the top level binds (`LABEL: str = os.environ.get(...)`) unbox once at init into native shadow slots, so typed native functions read the embedded interpreter's initial value; a later rewrite of the global there is never visible to native code, and a native write to a global owned there stops the compile naming the global. Unannotated top-level globals stay in the embedded interpreter only. The native entry never runs when the top level is routed there: every function rides the seed, and it runs under the host CPython (3.14.x). 43. `argparse` `type=Path` options compile to a native `Path` in the namespace, but the namespace has no None slot, so an absent option cannot be None. A `type=Path` option without `required=True` or a str default stops the compile (`PS2003`), and a default is converted to a `Path` at parse time. `parse_args` takes `None`, a list of str, or an Optional list; the list form skips `sys.argv`. 44. A local that is first bound to a value and later assigned `None` (or an Optional call result) in a branch is not widened to Optional. The first binding fixes the type, so `lp = Path(x)` followed by `lp = find_latest()` stops the compile with `PS1003`, and `lp is None` on a plain `Path` stops with `PS2004`. The typed form `lp: Optional[Path] = None` compiles. Narrowing through `is None` and `is not None` guards combined with `and`, `or`, conditional expressions, early exits and loop bodies does compile. 45. These shapes are refused after the mechanical-delta pass, each as its own PS code: `set(d)` and `list(d)` over a dict (PS2002, the set kind takes a list of int or str), `sorted(paths)` over `Path` elements (PS2003), incremental `hashlib` `update` calls and a `hashlib` constructor with data (PS2003), `Path.open("rb")` (PS2001), `collections.abc.Sequence` annotations (PS1104), class-body values the compiler cannot compute (PS1204; a `name = ` or `name: T = ` class attribute compiles and reads through the class and its instances, and so does a value the compiler computes — a call to a lowered function, a container literal, a comprehension, a name bound at module scope — stored in a runtime slot keyed `C.` and initialized at module init, where CPython computes it in the class body; a value with no slot form, a tuple or a float or a multi-target assign, still fences), module-level list-of-dict tables (PS1106, dict values are not heterogeneous), and a module-level `Path(__file__).resolve().parents[n]` constant (PS1001). 46. Under fallback mode for a whole program, the compile stages the program directory's non-Python files beside the binary (`/pyscriptc-data//`, the exact compile-time bytes) and rebases the module origins and `__file__` of the embedded interpreter's finder onto that tree, so `Path(__file__).parent / "data.json"` reads the staged copy after the output is relocated and the source tree is gone. Lookups that leave the program directory (`Path(__file__).parent.parent / "x"`, absolute paths) are not staged and still resolve against the compile-time path: with the source tree absent they raise `FileNotFoundError` where CPython reads the real file. A single-file fallback compile stages nothing and keeps the compile-time `__file__` (the same restriction divergence 42 documents). Issue [#665](https://github.com/sigilco/pyscriptc/issues/665). 47. `os.walk(top)` is a materialized list of the `(dirpath, dirnames, filenames)` triples in CPython's topdown pre-order, so the visit order and the rows match a CPython walk over the same tree read in the same `readdir` order. The walk is computed before the loop body runs, so a body that edits `dirnames` does not prune the walk where CPython honours the edit; `topdown=`, `onerror=` and `followlinks=` are refused (`os.scandir` and the other `os` names outside the fixed rows keep their `PS2001`). `json.load(fp)` reads the whole handle and reuses the `json.loads` annotation path: a missing result annotation stops the compile (`PS1001`) where CPython returns an untyped value, and a decode failure stays the runtime `ValueError`. `json.dump(obj, fp)` serializes through the same `json.dumps` subset and writes the handle with one call, so the bytes are the `dumps` bytes (no trailing newline); the closed dumps keyword set (`indent`, `sort_keys`, `ensure_ascii=True`) rides along and any other keyword stops the compile (`PS2003`). `time.perf_counter` and `time.monotonic` read the monotonic clock as a float. 48. A `--embed-python=bundle` output carries a trimmed runtime: the standard library travels as the safe-trimmed zip (test, idlelib, tkinter, turtledemo, ensurepip, `pydoc_data` and `__phello__` are out), the pack interpreter's own `site-packages` tooling (pip) is not bundled, and the unversioned `libpython3.X.so` name of a bundled Linux runtime is a symlink to the versioned library, not a second copy. A program that imports a trimmed tree stops the compile closed (`PS4301`) where CPython imports it; a bundled program that calls `pip` at runtime gets the same `ModuleNotFoundError` a CPython interpreter without pip would. Issues [#709](https://github.com/sigilco/pyscriptc/issues/709) and [#885](https://github.com/sigilco/pyscriptc/issues/885). 49. The program island carries a precompiled code cache next to the embedded sources (issue [#782](https://github.com/sigilco/pyscriptc/issues/782)): the compiler marshals every program unit's code object at compile time, zlib-compresses the entries into one blob, and the seed's loader unmarshals a module on its first import instead of compiling it (a 523 ms to 333 ms `--version` on yt-dlp). The cache is guarded, not trusted: each entry names the compiling interpreter's `.pyc` magic number and the source's sha256, the loader re-checks both at run time against the interpreter that runs the binary and the embedded text, and any mismatch (a binary moved to a different CPython build, a corrupted blob) falls back to compiling the source, so behavior never differs - only start time. A cached program answers CPython byte for byte; traceback frames name the same `co_filename` either way. Source: `program_island_source` and `_module_code_entry` in `src/pyscriptc/boxed.py`, `ps_island_codecache` in `src/pyscriptc/runtime/ps_island.c`, and `tests/test_island_code_cache.py`. 50. A module-scope container bound without an annotation (`xs = [...]`, `d = {...}`, `s = {...}`) in a script is a compile-time constant: every read re-builds the literal, so the name has no object identity. Mutating it through a method call (`xs.sort()`, `d.update()`, `s.add()`), an item or slice store (`xs[i] = v`), or a copy-on-alias bind (`ys = xs`, `x = y = xs`) stops the compile with `PS2002` naming the annotation fix: annotating the name (`xs: list[int] = ...`) gives it a global slot and CPython identity. Two paths still read copies where CPython aliases: passing the name to a function hands the callee a fresh container, so a callee that mutates its parameter leaves the module value unchanged, and `ys = xs[i]` binds a copy of the element, so a mutation through `ys` is not shared. 51. `x += ys` and `x *= n` on a list desugar to a fresh-concat rebind, not CPython's in-place `__iadd__`/`__imul__`. The rebound name's own value is right, but a second name bound to the same list keeps the old object where CPython mutates the shared one. 52. `itertools.tee(xs)` and `tee(xs, 2)` return a pair of eager `list` copies of the materialized source (`restlower/_itertools.py`), so later mutation of one copy does not affect the other and `next` is not defined on them; CPython returns two lazy iterators that share an underlying buffer. `itertools.batched(xs, n)` yields lists of elements (a short last list when the length is not a multiple of `n`) rather than tuples, because a leftover of one element cannot be a one-field tuple in the IR. Issue [#490](https://github.com/sigilco/pyscriptc/issues/490). 53. A class decorator other than `@dataclass` is erased: the class lowers undecorated (fields, methods, constructor and every read of it unchanged) and the decorator call never runs. CPython applies the decorator to the class object after the body executes, so side effects the decorator has (a registry write, a flag, a returned replacement class) are missing in the binary. Every decorator spelling other than the single `@dataclass` form takes the erase; the compile prints a `note: PS3099 ...` line on stderr and still succeeds, and the binary runs with the class's own value semantics. Under `--fallback=cpython` the class keeps the decorator: it runs through embedded CPython with the wrap applied. Issue [#732](https://github.com/sigilco/pyscriptc/issues/732). 54. A method decorator or module-function decorator outside the closed set that does not synthesize behavior is erased: the function lowers undecorated and the decorator call never runs. CPython replaces the attribute (or the module name) with whatever the decorator returns, so side effects the decorator has (a registry write, a flag, a wrapper with different call semantics) are missing in the binary. A decorator stack that carries a behavior-synthesizing member (`functools.lru_cache`, `functools.cache`, `functools.wraps`, `functools.singledispatchmethod`) never takes the erase: the stack would lower undecorated and the synthesis would be lost with it, so it keeps its fence (`PS1201` for a method, `PS2006` for a module function). The closed set (`property`, `staticmethod`, `classmethod`, `contextmanager`, the erased typing names) keeps its existing lowering. The compile prints a `note: PS3099 ...` line per erased decorator and still succeeds. Under `--fallback=cpython` the decorated definition keeps the decorator: it runs through embedded CPython with the wrap applied. Issue [#732](https://github.com/sigilco/pyscriptc/issues/732). 55. The `uuid` subset covers `uuid4()`, `uuid1()`, `uuid5(namespace, name)` and `UUID(text)`. `uuid4` and `uuid1` draw their random bytes from the host entropy source and set the version and variant bits the way CPython does, so the shape, the version nibble and the variant bits match but the values differ from CPython's by construction. `uuid5` hashes the namespace bytes and the name bytes with SHA-1 exactly as CPython does, so `uuid5` and `UUID` are byte-identical to CPython's. `UUID` is not lowered as a class: the call returns the canonical text, so `str(UUID(text))` reads the way CPython's does, and the `.hex` attribute of a lowered uuid call is the canonical text without the dashes (what CPython's `UUID.hex` holds). The `.int`, `.urn` and `.bytes` forms are refused (`PS2001`), and `uuid.UUID` as a parameter or return annotation is refused (`PS1106`) because the lowered value is text, not a class instance. 56. The `binascii` subset covers `unhexlify(text)` and `hexlify(data)`. Both are byte-identical to CPython's for ASCII hex, including the error cases: an odd-length hex string and a non-hex character raise `ValueError`, which is what CPython's `binascii.Error` reads as at an `except (ValueError, ...)` handler (`binascii.Error` itself is a `ValueError` subclass). `hexlify` takes bytes and returns bytes, as CPython's does; a `str` argument is refused at compile time (`PS1003`) where CPython raises `TypeError`. Every other `binascii` name is refused (`PS2001`). 57. The `shlex` subset covers `split(text)`, `quote(word)` and `join(words)`. `split` runs CPython's POSIX-mode lexer, so the words match CPython's byte for byte for the quoting, escaping and comment rules; the `comments` flag must be a constant bool (a variable is refused with `PS1003`), where CPython reads it at run time. An unterminated quote and a trailing backslash raise `ValueError`, as CPython's do. `quote` leaves a word alone when it holds only the characters CPython's safe set holds and wraps it in single quotes otherwise, spelling an embedded single quote `'"'"'`; `join` spaces the quoted words. The `shlex` class itself, the `posix` flag and the `punctuation_chars` mode are refused (`PS2001`). 58. The `secrets` subset covers `token_hex(n)`, `token_bytes(n)` and `randbelow(n)`. The bytes come from the host entropy source (`/dev/urandom`), the source CPython's `SystemRandom` reads through `os.urandom`, so the shapes match CPython's but the values differ by construction. `token_hex` returns the lowercase hexadecimal text of `n` random bytes (two digits per byte), `token_bytes` returns the bytes, and `randbelow` returns an int in `[0, n)` drawn the way CPython's `SystemRandom._randbelow` draws it: `n.bit_length()` bits at a time, redrawn while the draw is out of range. `n` is a constant int (a variable is refused with `PS1003`) and defaults to CPython's 32 bytes when omitted; `randbelow(0)` raises `ValueError`, as CPython's does. Every other `secrets` name (`choice`, `randbits`, `token_urlsafe`, `compare_digest`, `SystemRandom`) is refused (`PS2001`). 59. The island plan is conservative past the module that forced it: once any import in a module is routed to embedded CPython, the module's whole top level plans there (divergence 42) and every function whose body names an island-planned import or an unannotated value reaching one is planned there too, transitively through calls. On a real Flask-plus-LLM backend this compounds: 41% of its definitions pass the per-definition check in isolation, but the plane map routes 100% of them to the island because untyped imports at the module top level demote everything below them (the `assess --json` `planeMap.demotedBy` names the function that pulled each demotion). No compiled code runs unless the forced imports are moved behind a typed boundary or routed out of the compiled tree. The program still builds and runs there: the island is the route for code the subset cannot lower, not a failure, and it costs interpreter speed on the island plane plus a marshalling cost on every native-island crossing ([Libraries and fallback mode](../guide/libraries.md#the-two-planes)). 60. `math.factorial(n)` follows divergence 1: `int` is `i64`, so the factorial past `20!` raises `OverflowError` where CPython grows a big int. Every other integer helper (`gcd`, `lcm`) stays in range the way CPython's does for the same inputs; the domain errors (`factorial(-1)` is `ValueError`) match CPython exactly. 61. Float text output is shortest-round-trip without CPython's fixed-exponent rule: a value CPython prints as `180.0` prints as `1.8e+02` here. Values CPython also spells in exponent form byte-match, and every `%.17g`-style numeric comparison is unaffected; only exact `print` text near CPython's exponent crossover can differ. 62. `struct.calcsize` and `struct.unpack_from` lower only with a string-literal format that opens with an explicit byte-order prefix (`<`, `>`, `!` or `=`); `calcsize` folds to an int constant at compile time and `unpack_from(fmt, buffer, offset=0)` takes a `bytes` buffer and an `int` offset (negative offsets count from the end, as in CPython). A computed format (`PS2001`), the native `@` default and an unprefixed format (`PS2001`, native alignment is not lowered), `pack_into`, `iter_unpack`, `Struct` and `error` stay refused. Like `struct.unpack`, `unpack_from` returns a `list`, not a tuple, and a buffer too short for the format raises `ValueError` (`struct.unpack_from buffer is too small`) where CPython raises `struct.error`. Issue [#489](https://github.com/sigilco/pyscriptc/issues/489). 63. The `random` subset runs one module-level CPython-compatible Mersenne Twister: `random.seed(int)` seeds it with `init_by_array` over the int's little-endian 32-bit limbs, and every draw consumes the stream the way CPython 3.14's `random.py` does, so a seeded program prints the same numbers as CPython (the differential probe `probes/differential/random_surface.py` byte-compares them). `seed()` with no argument is nondeterministic, as in CPython, and `seed` takes an int only (no `version=`, no str, bytes or float seeds). The subset covers `random`, `randint`, `randrange`, `uniform`, `getrandbits`, `choice`, `shuffle`, `randbytes(n)`, `sample(population, k)`, `choices(population, k=n)` and `choices(population, weights=w, k=n)`, and the draws `gauss`, `normalvariate`, `lognormvariate`, `expovariate`, `triangular`, `gammavariate` and `betavariate`. `population` is a list (a str, tuple or range is refused with `PS1003`), `weights` is a `list[int]` or `list[float]`, and `choices` takes `weights` and `k` as keywords. Integer weights are accumulated in 64 bits and compared as floats, where CPython compares exact ints, so cumulative weights beyond 2**53 can pick a different index. `lognormvariate` returns `inf` on overflow where CPython raises `OverflowError`. `sample` has no `counts=`, `choices` has no `cum_weights=`, and the `Random` and `SystemRandom` classes plus `vonmisesvariate`, `paretovariate`, `weibullvariate`, `binomialvariate`, `getstate` and `setstate` are refused (`PS2001`). 64. The `pyscriptc_arrow` FFI adapter (`--ffi packaging/adapters/pyscriptc-arrow/ffi.toml`, issue [#1209](https://github.com/sigilco/pyscriptc/issues/1209)) carries the primitive fixed-width Arrow dtypes only: int32, int64, float32, float64, bool and utf8. `array_get` and `array_sum` widen every numeric element to double, so an int64 value above 2**53 loses integer precision where CPython-level int arithmetic stays exact, and an int32 value above the f64 mantissa reads rounded. The Arrow IPC wire form is a one-column record batch named `v` whose top level is a struct (the wire format requires it), so a stream PyArrow produced with a multi-column or differently named schema refuses with `ValueError` instead of projecting a silent subset; `stream_read` returns `list[dict[str, str]]` whose values are the element's own text (ints decimal, floats `%g`, bools `True`/`False`), not typed values. Every other Arrow type (nested types, dictionaries, unions, timestamps, extension metadata, decimal) is the honest fence: the thunk refuses the dtype word with `ValueError` naming the six supported constants, never a silent cast. The adapter has no `__arrow_c_array__`/`__arrow_c_stream__` capsule protocol in-process: the exchange with a live Arrow implementation goes over the wire bytes (`export_array`/`import_stream`), and a consumer under CPython opens them with `pyarrow.ipc.open_stream`. 65. The `statistics` subset (`mean`, `fmean`, `median`, `median_low`, `median_high`, `mode`, `variance`, `pvariance`, `stdev`, `pstdev`, `quantiles` with the default exclusive method and `n` in `{2, 4, 8}`) takes a `list[int]` or `list[float]`; every other `statistics` name is refused (`PS2001`) and other container shapes are refused (`PS1003`). The functions never mutate the caller's list. Two seams come from CPython's exact-rational implementation (`statistics` converts every number to a `Fraction` and converts the result to float at the end) against the i64/double runtime: `stdev` and `pstdev` differ from CPython by one ulp on a minority of datasets (CPython's correctly-rounded square root needs `math.isqrt` of a big integer), and the mean/median/variance family over `list[int]` returns `float` where CPython returns `int` (the mean of an exactly divisible int dataset, the median of an odd-length one, a variance whose exact fraction is integral): the value is equal, the `repr` is not, so `print(statistics.mean([1, 2, 3]))` prints `2.0` where CPython prints `2`. Over `list[float]` the mean/median/variance family folds in double arithmetic, so a minority of datasets round differently from the exact-rational path. `mode` takes `list[int]` only (a `dict[int, int]` count; float keys are outside the subset, `PS1003`), breaks ties the way CPython does (the first key to reach the top count), and `quantiles` takes `n` as a constant int. See divergence 61 for the float spelling rule that can also move a `print` of a whole float. 66. `csv.DictReader` returns a materialized `list[dict[str, str | None]]` where CPython returns a lazy iterator: the rows are all parsed before the loop body runs, so a loop body that mutates the source lines does not see the edit, and the reader cannot be resumed after `list()` or a second `for` (CPython's iterator is spent the same way, so a fresh `DictReader` call is the fix either way). Two row-shape differences ride along: a record longer than the header raises `ValueError: dict reader has more fields than fieldnames` at run time where CPython stores the extra fields under the `None` restkey (a `None` key and a `list[str]` value are outside the row dict's type), and `restkey=`/`restval=` are refused (`PS2003`) where CPython re-keys or refills the row. Blank records are skipped like CPython skips them. The writer matches CPython's excel dialect byte for byte, including the `\r\n` terminator, the minimal quoting and the empty-text restval for a missing field. Issue [#488](https://github.com/sigilco/pyscriptc/issues/488). 67. An island binary that links the build host's libpython is refused at the end of the compile (`PS4304`): the audit reads the linked artifact's dynamic dependencies (`otool -L` on Mach-O, `readelf -d` on ELF) and a reference to the host interpreter's library, reached by absolute path or through an absolute runpath, fails the build with that path in the message. The binary would run on the build machine and break on a target without that Python, so the build output says so before the artifact is shipped. `--embed-python=bundle` produces a relocatable binary instead, and `--allow-host-libpython` (or `[tool.pyscriptc.island] allow_host_libpython = true`) accepts the host link on purpose and warns instead. A native binary links no libpython and is unaffected. ## Where each entry comes from Every entry names the runtime file or compiler module that produces the difference, so a report can point at the code. The live support state across modules and language features is on the [compat site]({{PYSC_COMPAT_URL}}), and the code that stops a compile is in [Diagnostics](diagnostics.md). --- ## [Island engine descriptors (schema v1)](https://pyscriptc.pages.dev/docs/reference/island-engine/) # Island engine descriptors (schema v1) Issue #666. An island engine descriptor is a TOML document describing a pluggable engine behind the boxed island tier. The built-in engine is the embedded CPython the island tier already links (`--island-engine cpython`, always resolvable); issue #668 targets PyPy as the first external engine. The selection resolves and validates the descriptor, records it in the debug summary, and, for a PyPy-kind descriptor (`kind = "pypy-interpreter"` or `[abi] libpython = "pypy-c"`), emits `-DPS_ENGINE_PYPY` into the island runtime compile (#667). A PyPy-kind descriptor may instead name the pinned PyPy **runtime pack** (`[abi] pack = "pypy"`, issue #668): the compiler fetches the pack, compiles `ps_island.c` against its headers, links its `libpypy-c` and stages a relocatable bundle with the engine's own runtime. ## Format ```toml schema_version = 1 [engine] name = "pypy" # required, non-empty string version = "7.3.16" # required, non-empty string kind = "interpreter" # required, non-empty free string capabilities = ["fallback", "reflection"] # optional, open list of strings [abi] # optional table of string -> string expectations libpython = "3.14" stable_abi = "no" ``` - `schema_version`: integer. A descriptor with a **newer major version** than the compiler supports is rejected with a clear diagnostic; same or older versions load (v1 is the first, so today only `1` loads). - `capabilities`: an **open string list**. There is no closed vocabulary yet — engines and the compiler negotiate semantics in later embed issues. Unknown capability names are kept, never rejected. - `[abi]`: string-to-string expectations of the runtime surface the engine provides (for example `libpython`, `stable_abi`). Unknown entries are kept for forward compatibility. - Unknown top-level keys and tables are kept on the loaded descriptor, so an older compiler can still load a newer descriptor. ## Loader `src/pyscriptc/island_engine.py`: - `island_engine.resolve(name_or_path)` — a path (contains a separator or ends in `.toml`) is loaded directly; a bare name is looked up as `.toml` in the `os.pathsep`-separated `PYSC_ISLAND_ENGINES` search dirs; `cpython` resolves to the built-in descriptor. - Malformed descriptors raise `island_engine.EngineError` with a one-line diagnostic (surfaced by the CLI as an `IRError`, no traceback): invalid TOML, missing/wrong-typed required fields, a newer `schema_version`. - `island_engine.builtin_descriptor()` — the built-in embedded-CPython engine (`name=cpython`, `kind=interpreter`, `capabilities=["fallback"]`). ## CLI and config - `pyscriptc compile --island-engine `; also read from the `engine` key of `[tool.pyscriptc.island]` when the flag is unset (explicit flag wins). - The resolved descriptor appears in the debug output (`--debug=stages` prints an `island-engine:` line) and as the `islandEngine` key of `--debug-dir`'s `summary.json`. - The run-cache key includes the selection, so a future engine that changes the linked output cannot collide with a cached default build. ## Capability manifest (schema v1, issue #669) A descriptor may name an engine capability manifest: a JSON document the `probe` subcommand of `scripts/engine_manifest.py` produces *inside* the engine being described (identity, stdlib coverage, language features, platform). The descriptor's `[engine] manifest` key holds the path, relative to the descriptor file: ```toml [engine] name = "pypy" version = "7.3.16" kind = "interpreter" manifest = "pypy-manifest.json" ``` The manifest shape (validated by `pyscriptc/engine_manifest.py`, class `EngineManifest`): ```json { "schema": 1, "engine": {"name": "pypy", "version": "7.3.16", "language_level": "3.11", "platform": "linux-x86_64", "free_threaded": false}, "stdlib": {"available": ["json"], "missing": ["annotationlib"]}, "features": {"walrus_pep572": true, "tstring_pep750": false} } ``` - `schema`: integer; a newer major version than the compiler supports is rejected with a one-line diagnostic. - `stdlib.available` / `stdlib.missing`: disjoint lists of stdlib module names the engine resolves or lacks. - `features`: object of booleans, one entry per probed language feature. - Unknown top-level keys are kept (`extra`) for forward compatibility. ### Consumers - `pyscriptc imports --island-engine ` (or the `[tool.pyscriptc.island] engine` key): when the engine carries a manifest, the JSON report gains an `engine` section and each import row gains `engine.stdlib` (`available` / `missing` / `unknown`; third-party imports are always `unknown` — existence in the manifest is not loadability). The text report appends one `engine:` line. - `pyscriptc assess` with the same selection: the report gains an `engine` capability section (`stdlibStatus` per import root) and the text report appends one `engine:` line. - Without an engine (or with the built-in `cpython`, which has no descriptor file and no manifest) both commands behave exactly as before; the surface is additive only. A manifest that fails to parse warns on stderr and falls back to the native-only report. ## Boundary: what pyscriptc does not own The engine contract splits responsibility at the descriptor: - **The engine owns** its descriptor file, its capability manifest (produced inside the engine; `scripts/engine_manifest.py` is the reference probe), its headers, and the library or binary the compiled island links against. pyscriptc never installs, patches or relocates an external engine. - **pyscriptc owns** descriptor schema v1 validation (`src/pyscriptc/island_engine.py`), manifest schema v1 parsing and the `imports`/`assess` consumers (`src/pyscriptc/engine_manifest.py`), the engine seam in `src/pyscriptc/runtime/ps_island.c` (`PS_ENGINE_START`, with `PS_ENGINE_PYPY` selecting the PyPy init order), and the `-DPS_ENGINE_*` define emission in the LLVM driver. - **Manifest truth is the engine's claim.** pyscriptc parses and reports the manifest; it does not verify at compile time that the engine really resolves a listed stdlib module. Consumers degrade to `unknown`/native-only on a missing or broken manifest rather than trusting it blindly. ### Unsupported engines - **GraalPy**: no C embedding path. `ps_island.c` hosts the runtime through the C API, which GraalPy does not provide. - **MicroPython**: does not implement the CPython C API. - **`rpc` transport engines**: out of scope, tracked in #651. The descriptor loader does not reject these names (the schema is open by design); what is missing is a `PS_ENGINE_*` seam, so selecting them changes nothing in the emitted binary. ### PyPy seam status Under `PS_ENGINE_PYPY`, `PS_ENGINE_START()` calls `rpython_startup_code()`, `pypy_setup_home(home, 1)` and `pypy_init_threads()` instead of `Py_InitializeEx(0)`. The order is verified against the real PyPy 7.3.23 pack (issue #668): the fractions program of the acceptance test runs through it. Exception arguments are read with `PyObject_GetAttrString(val, "args")`, which is portable across both engines' C APIs. The CPython path is unchanged and fully tested. The GIL discipline is engine-specific and lives entirely in the `ps_island_acquire` / `ps_island_release` seam (see the source comment for the measured disagreement on every step: `PyEval_SaveThread` aborts under PyPy, `PyGILState_Ensure` on the main thread segfaults on the next Python call; a foreign thread registers with `pypy_thread_attach`). `PyImport_AddModuleObject` does not exist in cpyext; the seam calls the byte-name `PyImport_AddModule` entry under PyPy. ## The conformance corpus (issue #670) Every shipped engine must behave the same on the island surface, so the corpus is a set of island programs that run on each engine and must match CPython byte for byte (stdout, stderr and the exit code). | Program | What it covers | | --- | --- | | `fractions` | `fractions.Fraction` arithmetic and `str()` of the result | | `json` | `json.dumps` with sort keys and `ensure_ascii=False`, `json.loads` | | `re` | `re.match` with groups, `re.findall`, `re.sub` | | `dataclasses` | a dataclass with a nested dataclass, `dataclasses.fields`, `asdict` | | `exceptions` | a caught exception, a chained one, a locally defined exception class | | `bytes` | empty, embedded-NUL and non-ASCII byte strings across the seam | | `lists` | int and str element kinds, the empty list, a list built island-side | The programs live in `benchmarks/engine_corpus/programs//`: an island module (declared `island` in the program's `pyscriptc.toml`, so every definition in it runs on the selected engine) and a thin typed entry that calls it. ```bash uv run python -m benchmarks.engine_corpus.run # the whole matrix uv run python -m benchmarks.engine_corpus.run --only re # one program ``` The matrix is **descriptor-driven**: every `*.toml` under `benchmarks/engine_corpus/engines/` is one engine of the matrix, so the PyPy leg turns on by itself when #668 ships a PyPy descriptor — the file lands in the directory, and neither the runner nor the CI workflow needs an edit. A descriptor opts out with a top-level `conformance = false` key (an engine without an island seam yet); the runner reports it as skipped and does not fail. The CI lane is `.github/workflows/engine-conformance.yml`: one Linux cell runs the matrix and the marshalling micro-benchmark, writes both reports into the job summary and uploads them as artifacts. ### Marshalling cost per call `benchmarks/engine_corpus/marshalling.py` measures the seam itself: the per-call cost of an island call for each type the ABI carries, against a no-argument baseline that covers the GIL re-entry, the namespace lookup and the call machinery. Each case runs a warm-up loop first and is then timed five times inside the binary; the best run is reported, because the first call pays the embedded interpreter's initialization and a single timed run on a shared machine is dominated by scheduling noise. The numbers are the release receipt for the embed epic, not a gate: they are recorded in the PR body of #670 and re-measured at release boundaries. Measured on a quiet Linux x86_64 runner (CPython 3.14.9, 20 000 calls x 5 repeats, best run): | case | ns/call | marshalling share | | --- | --- | --- | | base (no payload) | 26.4 | baseline | | int | 28.3 | +2 ns | | float | 28.2 | +2 ns | | bool | 26.9 | +0 ns | | str | 27.8 | +1 ns | | bytes | 29.5 | +3 ns | | list (4 ints) | 52.0 | +26 ns | Reading: the scalars cost one to three nanoseconds over the seam itself — the copy is not the interesting cost. A list is the first structurally bigger case (an element copy per item plus a fresh list object), and it is the one to watch when the ABI grows. The absolute figures are noise-sensitive: on a loaded host the same run reports a baseline near 245 ns/call with the same shape (scalars within a few nanoseconds of the baseline, the list an order of magnitude above it). Compare shapes across hosts, not absolutes; the CI job re-measures on a quiet runner and writes its numbers into the job summary. ## Open questions (deferred to later embed issues) - Closed capability vocabulary and what each capability obligates the compiler to do (#668+). - Whether `[abi]` values should gain types (versions, booleans) once a second engine exists. - Descriptor discovery beyond `PYSC_ISLAND_ENGINES` (a per-project descriptor directory, a registry). --- ## [Runtime pack and licenses](https://pyscriptc.pages.dev/docs/reference/runtime-pack/) --- title: Runtime pack and licenses description: The bundled CPython runtime pack for `--embed-python=bundle` compiles, its pinned versions, and the licenses that travel with it. --- # Runtime pack and licenses A `bundle` compile (`--embed-python=bundle`, or the default `auto` mode when the compiler's interpreter has no usable shared `libpython`, as inside the release binary) links and runs a pinned CPython runtime pack instead of the host interpreter. The pack is fetched once, sha256-verified, cached, and carries its licenses with it. This page is the licensing and provenance contract that the pack and every output built from it follow. ## What the pack contains The pack for each supported platform (macOS arm64 and Linux x86_64 in the first slice) is a tarball produced from the pinned [python-build-standalone](https://github.com/astral-sh/python-build-standalone) release recorded in `packaging/runtime.lock.toml`: - `libpython3.14` shared library and the version headers, - the standard library as a safe-trimmed zip (modules a program can import; test, idle, tkinter and similar trees are not bundled, and neither is the pack interpreter's own `site-packages` tooling such as pip), - the `licenses/` directory from the matching `full` archive: the CPython license text plus the license of every third-party component compiled into libpython (for example expat, libffi, mpdecimal, OpenSSL, SQLite, zlib, zstd; the authoritative list is the inventory in the pack, not this page). The lock file pins the release tag, the asset name, the sha256 and the CPython micro version. A pack that does not match the lock is refused before anything is linked. ## Fetching and caching ```sh pyscriptc embed fetch # this machine pyscriptc embed fetch --platform linux-x86_64 ``` The compressed pack is tens of megabytes (roughly 27 MB on macOS arm64, 35 MB on Linux x86_64). It is fetched on demand, not at install time: a `bundle` compile without a cached pack says so and names the fetch command before downloading. The unpacked pack is cached under `~/.cache/pyscriptc/runtime//`. Offline and air-gapped machines point `PYSC_RUNTIME_DIR` at an already-unpacked pack directory; the compiler then never touches the network. ## Licenses in the output The CPython license (PSF License Agreement) is permissive and allows redistribution in binary form provided the license text and copyright notice travel with the binary. A `bundle` compile therefore writes `OUT/share/licenses/` with: - `LICENSE.cpython.txt`, the license text of the bundled runtime itself, copied from the pack (`lib/pythonX.Y/LICENSE.txt` of the pinned asset), - one `LICENSE..txt` per third-party component compiled into libpython (expat, libffi, mpdecimal, OpenSSL, SQLite, zlib, zstd, ncurses and the rest of the inventory in `packaging/third-party.toml`); the texts are vendored in the compiler's package data (`pyscriptc/pbs_licenses/`) and copied into the output, - `NOTICE`, the third-party notice rollup of the bundled runtime, generated by `scripts/gen_third_party_notices.py` from `packaging/third-party.toml` and the runtime lock, and - one directory per bundled third-party distribution (issue #660), holding the `licenses/` tree of its `*.dist-info` metadata. The third-party texts are vendored because the pinned python-build-standalone asset the pack is cut from (`install_only_stripped`) ships no `licenses/` directory; the `full` archive of the same release does, and `scripts/sync_pbs_licenses.py` vendors its texts with their sha256 pinned in `packaging/pbs_licenses.toml`. A runtime bump refreshes the vendored texts, the inventory and `NOTICE` together (see [the operational doc](https://github.com/sigilco/pyscriptc/blob/master/.agents/docs/release/embed-runtime.md)). The compiler itself is Apache-2.0; the preflight checker (pyrefly, MIT) ships inside the install. Neither license changes when a program is compiled with a bundled runtime, but the runtime's own licenses must be present in every redistributed output directory, which the bundle step does automatically. ## Asking before shipping The third-party inventory of the first pack build is confirmed with the project owner before the first public release that ships a pack (the inventory check, not legal advice). ## The self-compiled pyscriptc pyscriptc can compile itself into a binary that needs no Python, uv or pip on the host. The build is one command from a checkout: ```sh pyscriptc compile src/pyscriptc --program --skip-check --island --emit bin \ --embed-python bundle --embed-headers --bundle-exclude mypy -o dist/selfbin ``` The output is a relocatable directory, `dist/selfbin/`, laid out as: - `bin/selfbin`, the compiler itself (about 3 MB; every module of `src/pyscriptc` is compiled into it, and the program's non-Python files live in `bin/pyscriptc-data/`: the C runtime sources `ps_*.c`, the vendored typeshed, the pyright stubs and the runtime lock), - `lib/`, the pinned runtime: `libpython3.14.so`, the stdlib zip and `lib/python3.14/` (dynload modules plus the bundled pure-Python dependencies), - `include/`, the CPython headers, staged by `--embed-headers`, so the binary can compile island programs (which include `Python.h`) at run time, - `share/licenses/`, the license texts of the bundled runtime. This is a two-file layout in spirit: the directory is the artifact, and it runs from any path. It is not a single file because the interpreter is a shared library and the stdlib is a zip; both must sit on disk next to the binary for the dynamic loader and the import system. What the binary needs at run time, and what it does not: - No Python, uv or pip. The interpreter and the stdlib travel in `lib/`. `env -i PATH=/nonexistent dist/selfbin/bin/selfbin` prints its usage, and compiling a program that does not touch the island needs nothing else. - A C toolchain on PATH (clang or zig) for `--emit bin` and `run`, exactly like the installed compiler. A compile with no toolchain fails with PS0001, which names the install options. - pyrefly for `check` and the preflight of a compile without `--skip-check`. The binary resolves it like the installed compiler: the `PYREFLY_BIN` override, then a `pyrefly` on PATH. When none is found the compile says so instead of half-running. - No network and no pack cache for `--embed-python bundle` compiles: the binary links user programs against the runtime it carries. `--bundle-exclude mypy` declares the optional mypy bridge an extern dependency: the import stays in the compiled compiler, and the host answers it exactly like CPython answers a dead import. Without the exclusion the bundle refuses the compile, because mypy ships a compiled extension and a bundle carries pure Python only (PS4302). ## The onefile output `--onefile` writes the bundle as one executable instead of a directory. The flag needs `--embed-python=bundle`. ```sh pyscriptc compile src/pyscriptc --program --skip-check --island --emit bin \ --embed-python bundle --embed-headers --embed-stdlib minimal \ --onefile --strip-runtime -o dist/selfbin-onefile ``` The output is `dist/selfbin-onefile/selfbin-onefile.onefile`, one file. It holds: - the compiled program, the pinned runtime and the staged data tree, in one zlib-compressed payload, - a small C stub in front of it (about 17 KB compiled) that finds the payload appended to its own file. The stub is the whole runtime story of the onefile. On the first run it inflates the payload into a cache directory and execs the extracted binary with the process argv; on every later run it finds the cache directory and execs straight away. The cache directory is named after the payload's sha256, so: - a rebuild whose payload differs extracts beside the old tree instead of over it, and both keep working, - two hosts that run the same file for the first time at the same time race for the extraction; the loser of the atomic rename execs the winner's tree, which holds the same bytes. The cache root is `$PYSC_ONEFILE_CACHE` when it is set, then `$XDG_CACHE_HOME/pyscriptc/onefile`, then `~/.cache/pyscriptc/onefile`. The stub links zlib and nothing else, and it needs no `tar` and no Python: a container image with only libc runs the onefile as it runs the bundle. The build host needs zlib's development files (`libz-dev` on Debian, `zlib-devel` on Fedora) and a C compiler, because the stub is built from source on the machine that builds the onefile. The cost of the shape is one decompression on the first run: about 0.4 s for the 19 MB yt-dlp payload on the VPS. The warm run adds a `stat` and an `execv` to the program's own startup, which is under a millisecond. The measured cold and warm starts of the 18.8 MB yt-dlp onefile on the VPS (2026-11-02) are 0.91 s cold and 0.48 s to 0.61 s warm, against 0.45 s for the unpacked bundle, so the extraction costs about 0.4 s once and nothing after that. ### What the host still needs The onefile carries the interpreter, the stdlib and the program's own data. It does not carry the tools around the compiler, and a self-compiled pyscriptc still needs them on the host: - A C toolchain on PATH (clang or zig) for `--emit bin` and `run`, exactly like the installed compiler. This is the known gap of the onefile: the compiler shells out to `cc` to build the emitted LLVM IR and the onefile stub, and a onefile that carries a toolchain would carry a second copy of a compiler. - `pyrefly` for `check` and for the preflight of a compile without `--skip-check`. The binary resolves it like the installed compiler: the `PYREFLY_BIN` override, then a `pyrefly` on PATH. When none is found the compile says so and names the remedy instead of half-running. - No Python, uv or pip, and no pack cache and no network. The two gaps are the tools around the compiler, not the compiler itself: the 14.5 MB self-compiled onefile runs its usage with `PATH=/nonexistent` and compiles a hello world to a working native binary with only `cc` on PATH. Closing the C-toolchain gap means shipping a second compiler inside the onefile, which costs more than the runtime it would carry; closing the pyrefly gap means shipping the checker, which is a separate program with its own release cadence. Both stay host tools for now. ### Shrinking the output Two flags cut the size, and neither is an executable compressor (zstd and upx are not used, and the note that froze the benchmark's size levers forbids them): - `--strip-runtime` strips the staged `libpython`. Plain `strip` keeps `.dynsym`, where every `PyInit_*` extension entry point and the interpreter's exported API live, and drops the debug and static symbol tables: about 12 MB off the 32 MB Linux libpython. - `--embed-stdlib minimal` trims the stdlib zip to a frozen closure over the CPython import graph instead of the safe trim's "everything except the developer-facing trees". On the yt-dlp tree it takes the zip from 6.8 MB to 4.1 MB. The size of a onefile is therefore the sum of the program, the runtime and the trimmed stdlib, compressed once with zlib. ### What the onefile measures The yt-dlp composite (`benchmarks/ytdlp_composite/`, the frozen definition in `.agents/notes/2026-10-30-ytdlp-composite.md`) measures the onefile as a candidate. The run of 2026-11-02 (CPython 3.14.9, 6 cores, load average 3.55 during the run, min of 7 interleaved samples after 2 warmups): | Variant | `--version` s | `--list-extractors` s | extract s | download s | RSS MiB | files | installed bytes | shipped bytes | needs Python | relocatable | clean container | |---|---|---|---|---|---|---|---|---|---|---|---| | V-bundle (safe trim) | 0.453 | 0.550 | 1.441 | 1.468 | 57.7 | 170 | 82 441 773 | 82 441 773 | no | yes | yes | | V-minimal (minimal trim) | 0.453 | 0.625 | 1.396 | 1.472 | 57.4 | 170 | 79 231 932 | 79 231 932 | no | yes | yes | | V-onefile (minimal trim, stripped runtime) | 0.463 | 0.545 | 1.496 | 1.447 | 57.4 | 1 | 18 766 651 | 18 766 651 | no | yes | yes | | B1 (uv venv) | 0.287 | 0.468 | 1.228 | 1.353 | 49.1 | 1181 | 111 710 283 | 111 710 283 | yes | no | no | | B2 (`yt-dlp_linux`, PyInstaller onefile) | 1.429 | 1.561 | 2.373 | 2.482 | 73.4 | 1 | 41 166 968 | 41 166 968 | no | yes | yes | Composite (geometric mean over the 11 dimensions, lower is better; a PASS needs it below 1.0 against both baselines and no dimension worse than 2x): | Candidate | vs B1 | vs B2 | |---|---|---| | V-bundle | 1.009 | 0.994 | | V-minimal | 1.010 | 0.996 | | V-onefile | 0.644 | 0.635 | The onefile is the first candidate to pass on both sides. The bundle loses `files_n` (170 files against one) and `install_bytes` (4.4x B2); the onefile turns both into wins (1.0x and 0.46x) and pays for them with nothing, since its timings are the bundle's timings plus the extraction. Its output is the bundle's output: the `-j` extract of the local sample is byte-identical to B1's once the `epoch` key is removed, and `--list-extractors` is identical (1755 lines) to B1, B2 and the bundle. The onefile also runs in `debian:trixie-slim` (glibc 2.41, no Python), which is the `clean_container` answer the note asks for. --- ## [Stability and contracts](https://pyscriptc.pages.dev/docs/reference/stability/) --- title: Stability and contracts description: Support tiers, stability guarantees, versioned axes, and internal compiler surfaces. --- # Stability and contracts pyscriptc compiles the typed subset of Python 3.14. Code either compiles to native code or runs through embedded CPython (fallback mode); the compiler provides no shims or dynamic runtime emulation for untyped Python. This page states what is unstable, not what works. The live compat status across modules and language features lives on the [compat site]({{PYSC_COMPAT_URL}}). Known runtime differences from standard CPython are cataloged in [Divergences from CPython](divergences.md), and the codes that stop a compile are in [Diagnostics](diagnostics.md). ## Support tiers The compiler codebase and its public interface are organized into three explicit stability tiers: Frozen contracts (Tier 1), Best-effort driver API (Tier 2), and Internal implementation surfaces (Tier 3). ### Tier 1: Frozen contracts Tier 1 surfaces represent contractual boundaries between the compiler, external tools, emitted artifacts, and user code. Changes to any Tier 1 contract require a formal recorded decision, following the same governance discipline as the frozen v2 evaluation gates. Every Tier 1 guarantee is grounded in an owning file: - **IR JSON schema and `irVersion`**: defined and validated in [`src/pyscriptc/ir.py`](https://github.com/sigilco/pyscriptc/blob/master/src/pyscriptc/ir.py). The textual IR JSON emitted via `pyscriptc compile --emit ir` carries an explicit `irVersion`, the typed `subset` identifier, and the CPython target stamp. Modules failing schema validation or carrying mismatched target versions are rejected by `validate_module`. - **Runtime C ABI and `PS_ABI`**: defined in [`src/pyscriptc/runtime/ps_rt.h`](https://github.com/sigilco/pyscriptc/blob/master/src/pyscriptc/runtime/ps_rt.h) and pinned by `CURRENT_PS_ABI` in [`src/pyscriptc/ffi_manifest.py`](https://github.com/sigilco/pyscriptc/blob/master/src/pyscriptc/ffi_manifest.py). The runtime header maintains append-only struct definitions and frozen `ps_*` C symbol signatures. Any modification to a `ps_*` signature or `%ps.*` LLVM struct type bumps `CURRENT_PS_ABI` or requires an explicit recorded no-bump decision. - **FFI manifest format**: defined in [`src/pyscriptc/ffi_manifest.py`](https://github.com/sigilco/pyscriptc/blob/master/src/pyscriptc/ffi_manifest.py). Manifest files (`ffi.toml`) enforce `format = 1` and require `ps_abi = CURRENT_PS_ABI`. Manifests with mismatched ABI versions fail validation up front with diagnostic `PS4004`. - **CLI exit codes**: defined in [`src/pyscriptc/cli.py`](https://github.com/sigilco/pyscriptc/blob/master/src/pyscriptc/cli.py). The CLI exit codes are contractually fixed: `0` for successful compilation or child run status, `1` for internal compiler faults (`SpikeError`), `2` for type-checker preflight failures (`CheckFailed`) or argument syntax errors, `3` for typed subset refusals and unsupported constructs, and `4` for target architecture or CPython version fit mismatches (`VersionError`, `FitError`). - **Diagnostic codes**: defined in [`src/pyscriptc/diagnostics.py`](https://github.com/sigilco/pyscriptc/blob/master/src/pyscriptc/diagnostics.py). Diagnostic codes (`PSxxxx`) are permanent, append-only identifiers divided into numeric categories: syntax and compilation (`PS1xxx`), types and static semantics (`PS2xxx`), runtime operations (`PS3xxx`), and architectural refusals (`PS4xxx`). A code is never repurposed or renumbered. - **Divergence-list numbering**: defined in [`docs/reference/divergences.md`](divergences.md). Every intentional divergence from CPython 3.14 receives a permanent numeric identifier. When an entry is resolved or removed, existing numbers are preserved or renumbered through an explicit decision record. #### Dropped pack format The pack format track (`pack.toml`, binary wheel pack distribution, and precompiled library archives) was permanently dropped per the 2026-10-25 typed-subset positioning decision (Decision 5). The pack format is not a stability tier. PyPI dependencies are consumed either by compiling pure-typed source directly or by routing through an embedded in-process CPython (fallback mode); consumption of compiled code from CPython routes through `--emit binding` (`ctypes`). Curated distribution packages (`pyscriptc-orjson`, `pyscriptc-jwt`) remain as narrow standalone exceptions under explicit alias imports. ### Tier 2: Best-effort driver API Tier 2 encompasses programmatic Python APIs and machine-readable CLI formats designed for tool integrators, build runners, and editor extensions. These interfaces avoid arbitrary breaking changes within minor release lines, and any breaking adjustment is documented in release changelogs. They are not frozen contracts. - **Programmatic driver entrypoints**: - `pyscriptc.lower.lower_source`: compiles an annotated Python AST or source string into the intermediate representation. - `pyscriptc.emitter.emit_ir_text`: serializes validated IR dictionaries into canonical JSON text. - `pyscriptc.ffi_manifest.load_manifests`: parses and validates external FFI TOML specifications. - `pyscriptc.imports.build_import_report`: categorizes module imports into standard library, native, and embedded-CPython lanes. - `pyscriptc.factsbridge.FactsProvider`: provides static checker inference facts to the compiler pipeline. - **Machine-readable JSON outputs**: - `pyscriptc compile --debug=summary` and `--print diagnostics`: emits structured stage timings, lists of `PSxxxx` codes, and capability summaries. - `pyscriptc config --json`: serializes the resolved project configuration and import lane mappings. - `pyscriptc coverage --json`: serializes per-file and aggregate counts of unsupported constructs, reflection mechanisms, and porting hints. - `pyscriptc imports --json`: serializes import classifications, reflection mechanisms, and libpython linkage requirements. - Machine-readable outputs carry a top-level `schemaVersion` integer field when versioned schemas are introduced. ### Tier 3: Internal surfaces Tier 3 covers internal compiler implementation modules, code generators, and deprecated bridge mechanisms. These surfaces carry no stability promise, can be refactored or deleted across any commit, and should not be imported or relied upon by external tools. - **Compiler passes**: all modules under `src/pyscriptc/lower/` (AST visitors, desugaring passes, type inference routines). - **LLVM code generation internals**: all modules under `src/pyscriptc/emit/llvm/` (instruction builders, ABI layout helpers, block emitters). - **Runtime shims and private C helpers**: unexported structures and static helpers within `src/pyscriptc/runtime/`. - **Mypy plugin and bridge**: the mypy plugin (`src/pyscriptc/check.py:plugin`), the mypy facts bridge (`src/pyscriptc/factsbridge.py:MypyBridge`), and the `--checker mypy` CLI lane. The default compiler preflight is `pyrefly`, bundled in release binaries. The release binary compiles with `--nofollow-import-to=mypy` and omits mypy entirely. The mypy integration is deprecated, retained solely for cross-validation (`benchmarks/factsdiff.py`), and offers no API stability. ## Versioned axes and canonicity The compiler tracks four distinct versioned axes: | Axis | Indicator | Purpose | | ---- | --------- | ------- | | IR Schema | `irVersion` in JSON IR (`src/pyscriptc/ir.py`) | Identifies the textual JSON IR structure and required node fields | | Runtime C ABI | `CURRENT_PS_ABI` (`src/pyscriptc/ffi_manifest.py`) | Identifies the struct layout and calling convention between emitted LLVM IR and `ps_rt.h` | | FFI Manifest | `format` and `ps_abi` in `ffi.toml` | Guarantees external C extension manifests pair with the compiling runtime | | Tool Output | `schemaVersion` in machine-readable JSON | Tracks schema evolutions in CLI JSON output reports | ### IR hashing and canonicity `PS_ABI` and C runtime revisions do not affect IR canonicity. In accordance with the IR canonicity specification (`.agents/docs/spec/ir-canonicity.md`, section 7) and the typed serializer (`src/pyscriptc/tirhash.py`), canonical IR hashing evaluates only the semantic IR tree, walked in schema field order. Compiler environment details, host paths, and runtime ABI versions never enter the IR hash calculation: spans, the `cpython` stamp and `_notes` are never emitted, environment absolute paths in string values are scrubbed to ``, and synthetic class, handler-code and fence orders are canonicalized by the serializer. --- ## Public Data Files Reference ### [compat.json](https://pyscriptc.pages.dev/compat/data/compat.json) Description: Live compatibility metrics across builtins, standard library, and the pinned package corpus, with the native verdict, the island run verdict and the plane split per package. Schema / Format: compat_schema.py (version 2). Direct link: https://pyscriptc.pages.dev/compat/data/compat.json --- ### [island.json](https://pyscriptc.pages.dev/compat/data/island.json) Description: Island run receipts: one consumer program per pinned package, compiled with the island fallback and diffed against CPython. Schema / Format: island receipts JSON (version 1). Direct link: https://pyscriptc.pages.dev/compat/data/island.json --- ### [ladder.json](https://pyscriptc.pages.dev/compat/data/ladder.json) Description: Adoption ladder scores across R0 (real scripts), R1 (single-file), R2 (multi-module), and R4 (dependency lanes) rungs. Schema / Format: ladder JSON (version 1). Direct link: https://pyscriptc.pages.dev/compat/data/ladder.json --- ### [support.json](https://pyscriptc.pages.dev/compat/data/support.json) Description: Evaluated support status for standard library and builtin functions from compile probes. Schema / Format: support JSON (version 1). Direct link: https://pyscriptc.pages.dev/compat/data/support.json --- ### [what_will_fail.json](https://pyscriptc.pages.dev/compat/data/what_will_fail.json) Description: Known compiler gaps and fences measured on the R0 corpus grouped by root cause. Schema / Format: what_will_fail JSON (version 1). Direct link: https://pyscriptc.pages.dev/compat/data/what_will_fail.json ---