Terrarium

Terrarium is a secure container environment that uses Envoy as an L7 egress gateway, configured via familiar Cilium network policy semantics.

See CiliumNetworkPolicy Compatibility.

Terrarium allows you to declare policies that balance security and functionality, based on your risk tolerance, environment, and use cases.

It is particularly useful for running fully autonomous AI agents.

Usage

Published images include the terrarium binary and Envoy but not language runtimes, package managers, or other tools your workload needs. Use the image as a base layer or copy the binary into your own image.

Use as a base image

The :debian variant ships with ca-certificates and Envoy pre-installed:

FROM ghcr.io/macropower/terrarium:debian
RUN apt-get update && apt-get install -y python3 && rm -rf /var/lib/apt/lists/*
COPY config.yaml /home/dev/.config/terrarium/config.yaml
ENTRYPOINT ["terrarium", "init", "--"]
CMD ["python3", "app.py"]

Copy the binary into your own image

The :latest (scratch) variant contains only the terrarium binary, which makes it useful as a copy source in a multi-stage build:

FROM ghcr.io/macropower/terrarium:latest AS terrarium

FROM ubuntu:24.04
RUN apt-get update && apt-get install -y --no-install-recommends \
      ca-certificates envoy && \
    rm -rf /var/lib/apt/lists/*
COPY --from=terrarium /usr/local/bin/terrarium /usr/local/bin/terrarium
COPY config.yaml /home/dev/.config/terrarium/config.yaml
ENTRYPOINT ["terrarium", "init", "--"]
CMD ["/bin/bash"]

Running

However you build your image, terrarium init is the main entry point. It sets up the firewall, DNS proxy, and Envoy, then drops privileges and execs your command:

docker run --cap-add=NET_ADMIN my-terrarium-image

--cap-add=NET_ADMIN is required for nftables. The default config path is ~/.config/terrarium/config.yaml (following XDG conventions); override it with --config. Use --ready-file to create a file once all infrastructure is up, useful for orchestration. See terrarium init --help for the full flag reference.

Use as a Claude Code proxy

terrarium proxy runs the same egress policy as a plain HTTP forward proxy on the host instead of a transparent gateway inside a container. It needs no root, no nftables, and no network namespaces -- it binds a loopback socket and supervises an Envoy subprocess. This makes it a drop-in for the Claude Code sandbox proxy: point sandbox.network.httpProxyPort at it and every sandboxed request is filtered by your existing network policies.

Envoy is an external dependency in this mode (it is not bundled with the host binary):

• Linux: install the Envoy project's official release (tarball or Deb/RPM, amd64/arm64) and put envoy on PATH. See the Envoy install docs.
• macOS: brew install envoy (Apple Silicon and Intel). The Envoy project does not publish macOS tarballs; Homebrew is the supported native path. Bottles can trail the Linux release by a version or two, which terrarium tolerates.

Run it, writing the generated MITM CA where the sandbox can read it:

terrarium proxy \
  --config ./policy.yaml \
  --http-port 8080 \
  --ca-out ~/.terrarium/proxy/ca.pem

Then in ~/.claude/settings.json:

{
  "sandbox": {
    "enabled": true,
    "network": {
      "httpProxyPort": 8080
    },
    "filesystem": {
      "allowRead": ["~/.terrarium/proxy/ca.pem"]
    }
  },
  "env": {
    "NODE_EXTRA_CA_CERTS": "~/.terrarium/proxy/ca.pem",
    "SSL_CERT_FILE": "~/.terrarium/proxy/ca.pem"
  }
}

Setting httpProxyPort routes every sandboxed request through terrarium and skips Claude's own built-in proxy and hostname allowlist, so terrarium's policy is the only source of truth: unknown hosts get a 403 from terrarium, never a confirmation prompt.

Domains with L7 rules (path/method/header constraints) are MITM'd: the proxy terminates TLS with a leaf signed by the generated CA, so the sandbox must trust that CA. Claude reads NODE_EXTRA_CA_CERTS for Node tooling; most other tools honor SSL_CERT_FILE / REQUESTS_CA_BUNDLE / CURL_CA_BUNDLE pointed at the same PEM. Go-based tools on macOS verify through the system keychain and ignore these variables; set sandbox.network.enableWeakerNetworkIsolation: true (Claude's documented option for an httpProxyPort MITM proxy with a custom CA), trust the CA in the keychain, or list those tools in excludedCommands. Domains without L7 rules are tunneled through CONNECT without decryption, exactly like SNI-only rules in container mode.

L7 rules also require a system CA trust bundle in the environment where terrarium itself runs: the proxy re-encrypts inspected traffic upstream and verifies the real server's certificate against that bundle. Terrarium looks for SSL_CERT_FILE, NIX_SSL_CERT_FILE, and the usual system paths (including /etc/ssl/cert.pem on macOS), and fails closed with an error when none is found. This is a different bundle from the sandbox env above: the sandboxed client points SSL_CERT_FILE at the terrarium MITM CA, while terrarium's own environment must point at a real system trust bundle. Pointing terrarium's own SSL_CERT_FILE at the MITM CA would satisfy the bundle check but break every upstream handshake.

Proxy mode filters TCP egress only. UDP egress (DNS, QUIC) is not proxied and stays blocked by the sandbox. Allow-rule kinds a forward proxy cannot enforce -- bare toCIDR/toCIDRSet ranges, icmps, open port ranges, and tcpForwards -- are skipped with a warning (this only narrows the policy). egressDeny rules are rejected, since the proxy cannot enforce them and silently dropping them would widen the policy. See terrarium proxy --help for the full flag reference.

Examples

Allow GET requests to repos in your GitHub organization, injecting an API key header:

egress:
  - toFQDNs:
      - matchName: "github.com"
      - matchPattern: "*.github.com"
    toPorts:
      - ports:
          - port: "443"
            protocol: TCP
        rules:
          http:
            - method: "GET"
              path: "/my-org/.*"
              headerMatches:
                - name: "Authorization"
                  mismatch: ADD
                  value: "Bearer ghp_xxxxxxxxxxxx"

Allow DNS resolution for internal domains, plus HTTPS access:

egress:
  - toFQDNs:
      - matchName: "internal.example.com"
      - matchPattern: "*.internal.example.com"
    toPorts:
      - ports:
          - port: "53"
            protocol: UDP
          - port: "53"
            protocol: TCP
        rules:
          dns:
            - matchName: "internal.example.com"
            - matchPattern: "*.internal.example.com"
      - ports:
          - port: "443"
            protocol: TCP

Allow TLS connections to specific SNIs within a CIDR range:

egress:
  - toCIDR:
      - "10.0.0.0/8"
    toPorts:
      - ports:
          - port: "443"
            protocol: TCP
        serverNames:
          - "db.internal.example.com"
          - "*.internal.example.com"

Allow access to the internet, deny access to your internal network:

egress:
  - toCIDRSet:
      - cidr: "0.0.0.0/0"
        except:
          - "10.0.0.0/8"
          - "172.16.0.0/12"
          - "192.168.0.0/16"

Block specific ports to private subnets:

egressDeny:
  - toCIDR:
      - "192.168.0.0/16"
    toPorts:
      - ports:
          - port: "22"
            protocol: TCP
  - toCIDRSet:
      - cidr: "172.16.0.0/12"
        except:
          - "172.16.1.0/24"
    toPorts:
      - ports:
          - port: "3306"
            protocol: TCP

Forward non-TLS TCP services (like SSH) through Envoy to a named host:

tcpForwards:
  - port: 22
    host: "github.com"

Design

Process model

All processes share the same network namespace. Communication happens through kernel data structures (nftables sets via netlink) and loopback sockets.

graph TD
    init["terrarium init\n(PID 1, root)"]
    dns["DNS proxy\n(root, 127.0.0.1:53 + [::1]:53)"]
    envoy["Envoy\n(UID 1001, CAP_NET_ADMIN)"]
    user["User command\n(UID 1000, all caps dropped)"]
    nft[("nftables\nFQDN IP sets")]
    upstream(("Upstream"))

    init --> dns
    init --> envoy
    init --> user

    dns -- "updates via netlink\non resolution" --> nft
    user -- "TCP: NAT REDIRECT\nUDP: TPROXY" --> nft
    nft -- "FQDN traffic" --> envoy
    nft -- "CIDR TCP traffic" --> envoy
    nft -- "CIDR UDP traffic" --> envoy
    envoy --> upstream

Loading

Single-container architecture

Terrarium runs all components (firewall, DNS proxy, Envoy, user process) inside a single container sharing one network namespace. This isn't a packaging convenience; it's how the security model works.

nftables rules dispatch traffic based on process UID: the terrarium user (1000) gets full rule enforcement, Envoy (1001) gets unrestricted egress (domain allowlisting is enforced in Envoy's own config), and root (0) gets DNS access. Splitting into separate containers gives each its own network namespace, which breaks UID-based filtering entirely. You could share network namespaces across containers (a la Kubernetes Pods), but that adds orchestration complexity to arrive back at the same topology.

The components also depend on each other at runtime. The DNS proxy resolves allowed FQDNs and updates nftables IP sets via netlink. nftables redirects user traffic to Envoy's loopback listeners. The init process (PID 1) manages the lifecycle of all processes, forwards signals, and reaps zombies. Privilege drop happens atomically before exec'ing the user command.

There's also no real upside to splitting. Every component is 1:1 with the user workload, so independent scaling doesn't apply. The Envoy config and nftables rules are generated together from the same policy and can't drift independently, so independent upgrades don't work either. And fault isolation is actively undesirable here: if the DNS proxy or firewall crashes, the user process should stop. A security boundary with a hole is worse than no service at all.

User process isolation

The user command (UID 1000) shares a network namespace with the firewall, DNS proxy, and Envoy. Several kernel mechanisms prevent it from tampering with those components.

Capabilities

terrarium exec launches the user command with --inh-caps=-all and --bounding-set=-all, which clears the entire Linux capability bounding set. This is the foundational constraint; everything else is defense in depth.

Without capabilities, the user process cannot:

• Modify nftables rules or IP sets (CAP_NET_ADMIN).
• Bind to port 53 to replace the DNS proxy (CAP_NET_BIND_SERVICE).
• Send signals to Envoy (UID 1001) or init/DNS proxy (UID 0) (CAP_KILL).
• Change its own UID/GID or manipulate process credentials (CAP_SETUID, CAP_SETGID).

no-new-privs

The user process runs with --no-new-privs. The kernel enforces this across execve: setuid/setgid bits are ignored and no capability can be raised, even if a suid-root binary exists in the container image.

Envoy does not run with --no-new-privs because it needs CAP_NET_ADMIN as an ambient capability. TPROXY requires IP_TRANSPARENT on the UDP listener socket, and only CAP_NET_ADMIN can grant that. This is scoped: Envoy still runs as a dedicated non-root user (UID 1001) with no other elevated capabilities.

File ownership

All configuration and certificate files are created by root before the privilege drop. Envoy config is written 0644 (root-owned, world-readable); TLS certs and private keys are also 0644 (Envoy needs read access at UID 1001). The user process can read these files but cannot modify them.

Network-level enforcement

All DNS traffic from policy-evaluated UIDs is intercepted by nftables NAT OUTPUT REDIRECT rules and sent to the local DNS proxy on port 53. This happens regardless of what /etc/resolv.conf says -- even if the user points it at an external server, the redirect fires first. Port 53 egress is only allowed for root (UID 0); the proxy forwards upstream queries as root.

Layered defense

No single mechanism is load-bearing. Capability clearing prevents privilege escalation, no-new-privs blocks setuid exploitation, UID-based nftables rules enforce network policy regardless of process behavior, and file permissions prevent configuration tampering. Bypassing the security model requires compromising multiple independent kernel subsystems simultaneously.

Security modes

The firewall, DNS proxy, and Envoy all derive their behavior from the same parsed policy. Three modes:

• Unrestricted (nil egress): all traffic allowed, routed through Envoy so access events flow into the SQLite event store.
• Blocked (egress: [{}]): all traffic denied, DNS returns REFUSED, no Envoy.
• Filtered (rules with FQDN/CIDR/L7 matchers): per-rule chain isolation with OR semantics, FQDN IP sets with per-element TTLs, Envoy MITM for L7 inspection. All TCP (FQDN and CIDR) is routed through Envoy; CIDR TCP uses a dedicated catch-all listener forwarding via original_dst. UDP is routed through Envoy via TPROXY.

Allow, deny, and error events from the DNS proxy and Envoy land in an embedded SQLite event store; query with terrarium stats (see below). Stats is opt-in; set stats.enabled: true in YAML.

Traffic routing

In non-blocked modes, all terrarium traffic passes through Envoy for access logging and policy enforcement. TCP and UDP use different interception mechanisms because of how the kernel recovers original destination addresses.

graph TD
    user["User process\n(UID 1000)"]
    tcp{"TCP"}
    udp{"UDP"}
    nat["nftables\nNAT OUTPUT\n(deny CIDR ACCEPT,\nallow CIDR REDIRECT,\nFQDN REDIRECT)"]
    filter["nftables\nOUTPUT filter\n(per-rule allow/deny)"]
    drop["DROP"]
    mangle_out["nftables\nmangle OUTPUT\n(fwmark 0x1)"]
    policy["Policy routing\n(table 100, loopback)"]
    mangle_pre["nftables\nmangle PREROUTING\n(TPROXY)"]
    envoy_fqdn["Envoy per-port listeners\n(:15443 for 443\n:15080 for 80\n:15001 catch-all)"]
    envoy_cidr["Envoy CIDR catch-all\n(:15003, original_dst)"]
    envoy_udp["Envoy UDP listener\n(:15002)"]
    dns["DNS proxy\n(:53)"]
    upstream(("Upstream"))

    user --> tcp
    user --> udp

    tcp --> nat
    udp -- "port 53" --> dns
    udp -- "other ports" --> filter
    nat -- "deny CIDR\n(ACCEPT, no redirect)" --> filter
    nat -- "CIDR TCP" --> envoy_cidr --> upstream
    nat -- "FQDN TCP" --> envoy_fqdn --> upstream
    nat -- "non-policy TCP" --> envoy_fqdn
    filter -- "denied" --> drop
    filter -- "UDP allowed" --> mangle_out

    mangle_out --> policy --> mangle_pre --> envoy_udp --> upstream

Loading

TCP (NAT REDIRECT)

nftables NAT output rules REDIRECT all terrarium-UID TCP to Envoy. The NAT chain evaluates rules in order: deny CIDR chains ACCEPT (skip redirect so the filter chain drops the traffic), allow CIDR chains REDIRECT to the CIDR catch-all listener (port 15003), then per-port FQDN REDIRECT rules. Specialized listeners handle ports 443 (TLS passthrough, port 15443) and 80 (HTTP forward, port 15080). Per-rule port restrictions get dedicated listeners at ProxyPortBase (15000) + port. Non-policy traffic hits a catch-all TCP proxy on port 15001 that rejects via a blackhole cluster. CIDR-allowed TCP is forwarded by the CIDR catch-all listener via the original_dst cluster; it also runs a TLS inspector for SNI visibility in access logs. Envoy uses the original_dst listener filter to recover the real destination from conntrack (SO_ORIGINAL_DST).

UDP (TPROXY)

UDP cannot use NAT REDIRECT because SO_ORIGINAL_DST only works for TCP. Instead, nftables mangle chains and Linux policy routing implement TPROXY:

1. A mangle output chain marks all terrarium-UID UDP packets (excluding port 53) with fwmark 0x1.
2. A policy routing rule directs marked packets to table 100, which has a local default route through loopback.
3. The packet re-enters via loopback and hits a mangle prerouting chain that applies TPROXY, delivering it to Envoy's UDP listener on port 15002.
4. Envoy binds a transparent socket (IP_TRANSPARENT) at 0.0.0.0:15002, recovering the original destination from the socket address itself.

Port 53 is excluded from TPROXY marking so DNS queries reach the DNS proxy directly on loopback. Loose reverse-path filtering (rp_filter=2) is required on loopback because TPROXY'd packets arrive with non-local source addresses. The firewall package sets net.ipv4.conf.lo.rp_filter and net.ipv4.conf.all.rp_filter to 2 during policy routing setup, since the effective value is max(conf.all, conf.<iface>) and both must be loose.

Stats event store

Every egress decision (DNS allow/deny/error, Envoy HTTP/TCP allow/deny, firewall allow/deny/leak) can be recorded into an embedded SQLite database. The terrarium stats CLI surfaces the data; the daemon never reads the DB itself.

stats:
  enabled: true # opt-in; default off
  path: $XDG_DATA_HOME/terrarium/stats.db # default
  socket: /run/terrarium/accesslog.sock # gRPC ALS UDS
  firewall:
    nflogGroup: 5000 # netlink group for kernel log emission
  retention:
    maxAge: 720h # 30 days
    maxRows: 1000000
    perSource:
      firewall: 200000 # cap so chatty firewall events do not evict DNS/Envoy
  bufferBytes: 16384 # envoy gRPC ALS buffer
  flushIntervalMs: 1000 # envoy gRPC ALS flush

Three transports feed the store: the in-process DNS proxy (one event per terminal branch in handleQuery), Envoy via a gRPC AccessLog Service over the UDS at socket, and an nflog reader bound to the kernel's log group N emissions when logging.firewall.enabled is also true. None of these paths can block the data plane. Terrarium's writer channel uses non-blocking sends with a drop counter; Envoy buffers internally; the nflog reader translates kernel log records to events and emits through the same drop-on-full channel.

Read events with the CLI:

terrarium stats                                          # last-24h summary
terrarium stats top  [--denied|--allowed]                # top-N by domain (default --denied)
terrarium stats top --by sni                             # group by TLS SNI
terrarium stats top --by path                            # group by HTTP path (query string stripped)
terrarium stats list                                     # raw events; --cursor for pagination
terrarium stats list --source firewall --since 1h        # firewall-source events only

Flags shared across subcommands: --since, --until, --limit, --source dns|envoy|firewall, --instance ID, --format table|json|csv, --db PATH. The CLI opens the DB read-only, so it has no daemon dependency.

Daemon mode

Daemon mode (terrarium daemon) runs terrarium as a VM-wide network filter, applying policy to all processes rather than a single container UID. This is designed for Lima VMs where containers run inside the VM via containerd/Docker bridge networking.

Forwarded container traffic (bridge/veth) uses a separate interception path from locally-originated traffic:

Chain	Hook	Priority	Purpose
nat_prerouting	PREROUTING	-100	DNAT forwarded IPv4 DNS + per-port TCP to Envoy on 127.0.0.1
mangle_prerouting	PREROUTING	-150	Mark forwarded UDP for TPROXY; dispatch to Envoy/DNS proxy
forward	FORWARD	0	Policy-evaluate forwarded traffic not intercepted by DNAT (ICMP)
postrouting_guard	POSTROUTING	0	Catch locally-originated leaks (forwarded traffic passes through)

All forwarded TCP uses DNAT in NAT PREROUTING. When br_netfilter is loaded (required for bridge-networked containers), bridged frames traverse netfilter inet hooks so terrarium's nftables rules see the traffic. DNAT is required because br_nf_pre_routing_finish (the br_netfilter continuation after inet PREROUTING hooks) only re-routes packets locally when DNAT has changed the destination address. Without DNAT, the packet receives a fake bridge rtable and is L2-forwarded to the original destination, bypassing Envoy entirely. TPROXY does not modify the IP header, so it cannot trigger this re-route.

Per-port DNAT rules redirect configured TCP ports to Envoy's proxy listeners (ProxyPortBase + port). The non-loopback interface check prevents these rules from matching locally-generated TPROXY traffic re-entering PREROUTING on loopback. Per-port matching avoids intercepting SSH (port 22).

DNS (UDP and TCP port 53) also uses DNAT in NAT PREROUTING. The DNS DNAT rules intentionally omit the non-local destination check so queries to bridge-local resolvers (e.g., BuildKit's embedded DNS on the CNI gateway) are also intercepted. DNS UDP DNAT requires net.ipv4.conf.default.route_localnet=1 so dynamically-created bridge interfaces allow routing to 127.0.0.0/8. Deny CIDRs skip TPROXY so the FORWARD chain's terminal DROP applies.

Guard table

Terrarium manages a single terrarium table via netlink, deleting and recreating it on every config reload. During the brief window between delete and apply (or if the daemon crashes), there are no egress rules at all. A separate terrarium-guard table at priority 10 closes this gap.

The filter output chain sets fwmark bit 0x2 on every packet that enters policy evaluation (after loopback CIDR accepts, before any allow/deny rules). Accepted packets carry this bit; dropped packets never leave the chain. The guard table checks this single bit to accept policy-evaluated traffic without duplicating terrarium-internal details like UIDs, TPROXY marks, or ICMP exceptions. When the daemon is down (no terrarium table, no mark), the guard drops all non-loopback egress. The guard mark coexists with the TPROXY mark (0x1) via bitwise OR: TPROXY'd packets carry both bits (0x3), and matchMark uses bitmask matching so each bit is identified independently.

Two tables are needed. A boot-time terrarium table provides deny-all until the daemon starts. The daemon replaces it with policy-based rules. The terrarium-guard table is never touched by the daemon and survives across reloads:

# Boot-time deny-all (terrarium replaces this on startup).
table inet terrarium {
  chain output {
    type filter hook output priority filter; policy drop;
    oifname "lo" accept
    ct state established,related accept
  }
}

# Guard table (terrarium never touches this).
table inet terrarium-guard {
  chain output {
    type filter hook output priority 10; policy accept;
    ip daddr 127.0.0.0/8 accept
    ip6 daddr ::1 accept
    meta mark & 0x2 == 0x2 accept
    ct state established,related accept
    # Add rules here for services that need egress independent of
    # terrarium (e.g., a local DNS forwarder):
    # udp dport 53 accept
    # tcp dport 53 accept
    drop
  }
}

The ct state established,related rule appears after the mark check so established connections survive a daemon restart (graceful draining) even without the mark. Load both tables at boot before the daemon starts -- on NixOS use networking.nftables.tables, on systemd systems use an nftables service with a config file, or load them via nft -f in an init script.

Host requirements

Bridge-networked containers require kernel modules and sysctls that terrarium does not configure itself (they need root and typically belong in the host's boot configuration):

# Load br_netfilter so bridged L2 frames enter netfilter inet hooks.
modprobe br_netfilter

# Force bridged traffic through iptables/nftables.
sysctl net.bridge.bridge-nf-call-iptables=1
sysctl net.bridge.bridge-nf-call-ip6tables=1

# Enable IP forwarding for container routing.
sysctl net.ipv4.ip_forward=1

# Allow DNAT to 127.0.0.1 on bridge interfaces (for DNS UDP).
# Setting "default" ensures dynamically-created interfaces inherit it.
sysctl net.ipv4.conf.default.route_localnet=1

Container runtimes that manage their own NAT rules (CNI bridge ipMasq, Docker's --iptables) can conflict with terrarium's nftables table. Disable source NAT in the CNI bridge plugin ("ipMasq": false) or Docker (--iptables=false) and let terrarium handle traffic interception exclusively.

If the host runs a strict reverse-path filter (the NixOS default), it must be relaxed to "loose" mode. With br_netfilter, bridged packets enter L3 hooks with the bridge port (veth) as iif, but routes point to the bridge master (cni0). Strict rpfilter fails because fib saddr . iif has no matching route; loose mode drops the iif constraint.

The NixOS nftables rpfilter chain (fib saddr . mark check exists) also needs an exception for TPROXY-marked packets. With fwmark 0x1, the FIB lookup uses policy routing table 100 (local default dev lo), which changes the reverse path result. Without the exception, the rpfilter chain drops TPROXY-marked forwarded UDP packets:

# NixOS: networking.firewall.checkReversePath = "loose";
# Or: sysctl net.ipv4.conf.all.rp_filter=2

Host firewall

Terrarium is an egress filter and does not create an INPUT chain. Inbound filtering is the responsibility of the host firewall (NixOS firewall, iptables, etc.) or, in container mode, the container runtime's bridge/port-mapping rules.

The host firewall must accept traffic patterns created by terrarium's NAT and TPROXY interception. Without these rules, DNATted bridge-container traffic (DNS and TCP redirected to 127.0.0.1) and TPROXY-marked forwarded UDP packets would be dropped:

# Accept DNATted bridge-container traffic (DNS + TCP).
ct status dnat accept
# Accept TPROXY-marked forwarded UDP packets.
meta mark & 0x1 == 0x1 accept

On NixOS, add these via extraInputRules (appended to the input-allow chain which is jumped to from the main INPUT chain for new connections):

networking.firewall = {
  enable = true;
  checkReversePath = "loose";
  allowedTCPPorts = [ 22 ];  # SSH or other management access
  extraInputRules = ''
    ct status dnat accept
    meta mark & 0x1 == 0x1 accept
  '';
  # TPROXY-marked packets use policy routing table 100, which
  # changes the FIB reverse path result. Without this exception
  # the rpfilter chain drops them.
  extraReversePathFilterRules = ''
    meta mark & 0x1 == 0x1 accept
  '';
};

The fwmark (meta mark) is a kernel-internal field on the socket buffer -- it never appears on the wire and arrives as 0 on all inbound packets. Only local nftables rules, TPROXY, or processes with CAP_NET_ADMIN can set it, so external traffic cannot spoof the mark to bypass the firewall.

The NixOS firewall does not create a FORWARD chain by default (networking.firewall.filterForward is false), so there is no conflict with terrarium's FORWARD chain.

Container DNS

Terrarium intercepts all forwarded DNS (UDP and TCP port 53) in NAT PREROUTING, including queries to bridge-local addresses like a CNI gateway or BuildKit's embedded resolver. This ensures container DNS works regardless of what resolv.conf the container runtime injects.

Container CA trust

When L7 rules require MITM inspection, the CA certificate is installed into the host VM's trust store and copied to /etc/terrarium/ca.pem. Containers have isolated filesystems and need separate configuration:

Containerd registry access (e.g., docker pull through L7-restricted HTTPS):

# /etc/containerd/certs.d/_default/hosts.toml
[host."https://*"]
  ca = "/etc/terrarium/ca.pem"

Application HTTPS inside containers:

• Volume mount: bind-mount /etc/terrarium/ca.pem and set SSL_CERT_FILE
• Build into image: copy and run update-ca-certificates
• Non-L7 rules: if rules only use FQDN/CIDR selectors without L7 matchers (paths/methods/headers), no MITM occurs and no CA is needed

MITM inspection also requires a system CA bundle on the host, which Envoy uses to verify upstream server certificates when it re-encrypts traffic. Generation fails closed with an error when none is found.

Workload confinement

Daemon-mode workloads sometimes need real kernel root (raw sockets, certain syscalls that user namespaces mediate away), which rules out the UID/capability model that container mode uses. terrarium jail is a launcher that arms an AppArmor profile transition and execs the workload under it:

terrarium jail -- my-workload arg1 arg2

The terrarium.workload profile denies writes under /var/lib/terrarium/**, /etc/terrarium/**, and the MITM CA directory; reads of process environ and direct memory access via /proc/*/mem; writes under /etc/nixos/**, /nix/store/**, /boot/**, and systemd/dbus sockets (no rebuild path); /dev/mem, /dev/kmem, and module loads; and writes to /proc/self/attr/**, /proc/thread-self/attr/**, and /sys/kernel/security/apparmor/** (no tampering with the policy itself). Capability mediation uses an explicit allow-list -- sys_module, sys_admin, net_admin, mac_admin, bpf, and other dangerous caps are absent, so AppArmor's implicit-deny handles them (nftables mutation, for example, requires CAP_NET_ADMIN and is blocked by that omission).

Granted caps include net_bind_service, net_raw, sys_resource, sys_ptrace, syslog, and perfmon (plus the usual DAC/kill/setuid set) so same-profile debugging, kernel-log observability, and userspace profiling keep working; cross-profile ptrace and writes to the sysctls that widen ptrace/dmesg/perf access for unconfined neighbors remain denied. See lima/terrarium-workload.profile for the authoritative list.

The boundary is the profile transition at exec, not the subcommand name. Admin paths outside terrarium jail (e.g., nixos-rebuild switch from the host) stay unconfined and keep working. terrarium jail must be called before any privilege drop, and does not compose with terrarium init in either order: init needs CAP_NET_ADMIN (not granted by the profile) to spawn Envoy, and a post-drop process cannot transition into a new profile without CAP_MAC_ADMIN.

Requirements: AppArmor enabled in the kernel, the terrarium.workload profile loaded, and lockdown=integrity on the kernel command line (lockdown backstops the /dev/mem and module-load denies). On NixOS:

security.apparmor = {
  enable = true;
  policies."terrarium.workload".profile = builtins.readFile ./terrarium-workload.profile;
};
boot.kernelParams = [ "lockdown=integrity" ];

An example profile ships at lima/terrarium-workload.profile. Because lockdown=integrity is a boot parameter, activating it requires a fresh boot with the new kernel command line -- nixos-rebuild switch alone does not reload it.

IPv6

The entire stack is dual-stack. nftables uses a single inet-family table that covers both IPv4 and IPv6 (replacing four legacy iptables tables). FQDN IP sets are created in pairs: one TypeIPAddr set for A records and one TypeIP6Addr set for AAAA records, both with per-element TTLs. The mangle prerouting chain has per-AF UDP TPROXY rules for NFPROTO_IPV4 and NFPROTO_IPV6. Policy routing rules and routes are installed for both AF_INET and AF_INET6. The DNS proxy listens on both 127.0.0.1:53 and [::1]:53. NAT OUTPUT REDIRECT rules intercept port 53 for both address families, so no /etc/resolv.conf modification is needed.

At startup, init checks whether IPv6 is actually available via net.ipv6.conf.all.disable_ipv6. If the stack appears disabled, IPv6 is explicitly turned off via sysctl on all interfaces as defense-in-depth. The ipv6Disabled flag is threaded through to the DNS proxy so it skips binding [::1].

Lifecycle

1. Capture upstream DNS from /etc/resolv.conf
2. Generate configs (firewall, Envoy, certs) if not pre-baked
3. Install CA certificates (if L7 rules need MITM)
4. Apply nftables rules atomically via netlink
5. Set up policy routing for UDP TPROXY (if egress is not blocked)
6. Start DNS proxy (nftables NAT OUTPUT redirects port 53 to loopback)
7. Start Envoy (if egress is not blocked), wait for listener readiness
8. Create ready-file if configured (signals all infrastructure is up)
9. Drop privileges, exec user command
10. Forward signals to user command, reap zombies
11. On exit: drain Envoy, stop DNS proxy, remove policy routes, flush nftables