Skip to content

Browser.setDownloadBehavior is a noop: file downloads not implemented (no disk write, no download events) #2701

Description

@navidemad

Summary

Browser.setDownloadBehavior is dispatched but its implementation is a noop — every parameter is commented out and it returns a bare success. As a result Lightpanda has no file-download path at all: a response carrying Content-Disposition: attachment is treated as a normal navigation (and for a non-displayable MIME its body bytes are discarded), the configured downloadPath is ignored, nothing is written to disk, and neither Page.downloadWillBegin nor Browser.downloadProgress is ever emitted. A CDP client therefore cannot trigger, await, or locate a download.

Today's behavior

setDownloadBehavior at src/cdp/domains/browser.zig:73-82 parses none of its params and just calls sendResult(null, …). On navigation, Frame.frameHeaderDoneCallback (src/browser/Frame.zig:1045) commits the response as a page regardless of Content-Disposition; frameDataCallback/frameDoneCallback then route a non-HTML/-text/-image body into _parse_state = .raw (Frame.zig:1171) whose bytes are appended (:1193) and then discarded when an empty document is parsed in their place (Frame.zig:1259-1264). There is no download_* notification type and no Page.downloadWillBegin / Browser.downloadProgress emission anywhere in src/.

sequenceDiagram
    participant Client as CDP Client
    participant LP as Lightpanda
    participant Frame
    Client->>LP: Browser.setDownloadBehavior allow, downloadPath
    LP-->>Client: success but params are ignored
    Client->>LP: Page.navigate to attachment URL
    LP->>Frame: response with Content-Disposition attachment
    Frame->>Frame: treat as page load, buffer then discard body
    Frame-->>LP: empty document committed
    LP--xClient: no Page.downloadWillBegin
    LP--xClient: no Browser.downloadProgress
    Note over LP,Frame: nothing written to downloadPath
Loading

Expected behavior

Per the CDP protocol, opting in via Browser.setDownloadBehavior should make a download response land on disk and surface progress events:

  • Browser.setDownloadBehavior — honor behavior (allow / allowAndName / deny / default), write downloaded files under downloadPath, and (when eventsEnabled: true) emit download events.
  • Page.downloadWillBegin{ frameId, guid, url, suggestedFilename }, fired when a download starts.
  • Browser.downloadProgress{ guid, totalBytes, receivedBytes, state } where state advances inProgresscompleted.

A response is a download when it carries Content-Disposition: attachment (and, in Chrome, when its MIME type is one the browser will not render). Clicking an <a download> anchor is the same operation initiated from script. No rendering or layout is required for any of this.

sequenceDiagram
    participant Client as CDP Client
    participant LP as Lightpanda
    participant Frame
    participant Disk
    Client->>LP: Browser.setDownloadBehavior allow, downloadPath
    LP->>LP: store behavior and downloadPath on the connection
    LP-->>Client: success
    Client->>LP: Page.navigate to attachment URL
    LP->>Frame: response with Content-Disposition attachment
    LP-->>Client: Page.downloadWillBegin guid, suggestedFilename
    Frame->>Disk: write body bytes under downloadPath
    LP-->>Client: Browser.downloadProgress state inProgress
    LP-->>Client: Browser.downloadProgress state completed
Loading

Reproducer

Four files in one directory. No Chrome, no framework. Requires node >= 18.

npm install --no-save ws     # the only dependency
bash repro.sh                # or: LIGHTPANDA_BIN=/path/to/lightpanda bash repro.sh

server.js — serves the attachment:

const http = require('http');

const PORT = process.env.HTTP_PORT || 9580;
const BODY = 'col1,col2\nhello,world\n42,1337\n';

const server = http.createServer((req, res) => {
  if (req.url === '/report.csv') {
    res.writeHead(200, {
      'Content-Type': 'text/csv',
      'Content-Disposition': 'attachment; filename="report.csv"',
      'Content-Length': Buffer.byteLength(BODY),
    });
    res.end(BODY);
    return;
  }
  res.writeHead(404, { 'Content-Type': 'text/plain' });
  res.end('not found');
});

server.listen(PORT, '127.0.0.1', () => console.error(`http server on 127.0.0.1:${PORT}`));

probe.js — set download behavior, navigate at the attachment, check four signals:

const fs = require('fs');
const path = require('path');
const os = require('os');
const { connect, sleep } = require('./cdp.js');

const HTTP_PORT = process.env.HTTP_PORT || 9580;
const EXPECTED = 'col1,col2\nhello,world\n42,1337\n';

(async () => {
  const dlDir = fs.mkdtempSync(path.join(os.tmpdir(), 'lp-dl-'));
  const c = await connect();

  const events = [];
  c.on('Page.downloadWillBegin', (p) => events.push(['downloadWillBegin', p]));
  c.on('Browser.downloadProgress', (p) => events.push(['downloadProgress', p]));

  await c.send('Browser.setDownloadBehavior', {
    behavior: 'allow',
    downloadPath: dlDir,
    eventsEnabled: true,
  });

  await c.send('Page.navigate', { url: `http://127.0.0.1:${HTTP_PORT}/report.csv` });

  let file = null;
  for (let i = 0; i < 20; i++) {
    const completed = events.some(([n, p]) => n === 'downloadProgress' && p && p.state === 'completed');
    const files = fs.readdirSync(dlDir).filter((f) => !f.startsWith('.') && !f.endsWith('.crdownload'));
    if (files.length) file = path.join(dlDir, files[0]);
    if (completed && file) break;
    await sleep(100);
  }

  c.close();

  const beginFired = events.some(([n]) => n === 'downloadWillBegin');
  const completedFired = events.some(([n, p]) => n === 'downloadProgress' && p && p.state === 'completed');
  const fileExists = !!file && fs.existsSync(file);
  const contentOk = fileExists && fs.readFileSync(file, 'utf8') === EXPECTED;

  console.log(`Page.downloadWillBegin fired:               ${beginFired}`);
  console.log(`Browser.downloadProgress {state:completed}: ${completedFired}`);
  console.log(`file written to downloadPath:               ${fileExists}${file ? ' (' + path.basename(file) + ')' : ''}`);
  console.log(`file content matches response body:         ${contentOk}`);

  try { fs.rmSync(dlDir, { recursive: true, force: true }); } catch {}

  const ok = beginFired && completedFired && fileExists && contentOk;
  console.log(ok
    ? 'PASS: setDownloadBehavior wrote the attachment to disk and emitted download events.'
    : 'FAIL: attachment response produced no download — no events fired, nothing written to disk.');
  process.exit(ok ? 0 : 1);
})().catch((e) => { console.error(e); process.exit(2); });

repro.sh — orchestration:

#!/usr/bin/env bash
set -euo pipefail

REPRO_DIR="$(cd "$(dirname "$0")" && pwd)"
LIGHTPANDA_BIN="${LIGHTPANDA_BIN:-lightpanda}"
HTTP_PORT="${HTTP_PORT:-9580}"
CDP_PORT="${CDP_PORT:-9222}"
export HTTP_PORT

lsof -ti:"$CDP_PORT"  2>/dev/null | xargs kill 2>/dev/null || true
lsof -ti:"$HTTP_PORT" 2>/dev/null | xargs kill 2>/dev/null || true

PIDS=()
cleanup() { for pid in "${PIDS[@]:-}"; do kill "$pid" 2>/dev/null || true; done; }
trap cleanup EXIT

cd "$REPRO_DIR"
if [ ! -d node_modules/ws ]; then npm install --no-save --silent ws; fi

node server.js >server.log 2>&1 &
PIDS+=("$!")

LIGHTPANDA_DISABLE_TELEMETRY=true \
  "$LIGHTPANDA_BIN" serve --host 127.0.0.1 --port "$CDP_PORT" >lightpanda.log 2>&1 &
PIDS+=("$!")

for _ in $(seq 1 50); do
  body="$(curl -fsS "http://127.0.0.1:${CDP_PORT}/json/version" 2>/dev/null || true)"
  if [[ "$body" == *'"Browser": "Lightpanda'* ]]; then break; fi
  sleep 0.1
done

exec node probe.js
cdp.js — raw-ws CDP helper (Target.createTarget + attachToTarget boilerplate)
const WebSocket = require('ws');

function makeClient(url = 'ws://127.0.0.1:9222/') {
  const ws = new WebSocket(url);
  let nextId = 1;
  const pending = new Map();
  const handlers = new Map();
  const drainPending = (reason) => {
    for (const { reject } of pending.values()) reject(new Error(reason));
    pending.clear();
  };
  ws.on('message', (raw) => {
    const m = JSON.parse(raw.toString());
    if (m.id != null && pending.has(m.id)) {
      const { resolve, reject } = pending.get(m.id);
      pending.delete(m.id);
      if (m.error) reject(new Error(`${m.error.code}: ${m.error.message}`));
      else resolve(m.result);
    } else if (m.method) {
      const fn = handlers.get(m.method);
      if (fn) fn(m.params, m.sessionId);
    }
  });
  ws.on('close', () => drainPending('WebSocket closed before response'));
  ws.on('error', (e) => drainPending(`WebSocket error: ${e.message}`));
  return {
    send(method, params = {}, sessionId) {
      const id = nextId++;
      return new Promise((resolve, reject) => {
        pending.set(id, { resolve, reject });
        ws.send(JSON.stringify({ id, method, params, sessionId }));
      });
    },
    on(method, fn) { handlers.set(method, fn); },
    ready() { return new Promise(r => ws.on('open', r)); },
    close() { ws.close(); },
  };
}

async function connect(opts = {}) {
  const url = opts.url || 'ws://127.0.0.1:9222/';
  const startUrl = opts.startUrl || 'about:blank';
  const client = makeClient(url);
  await client.ready();
  const { targetId } = await client.send('Target.createTarget', { url: startUrl });
  const { sessionId } = await client.send('Target.attachToTarget', { targetId, flatten: true });
  await client.send('Page.enable', {}, sessionId);
  await client.send('Runtime.enable', {}, sessionId);

  return {
    sessionId,
    targetId,
    raw: client,
    send: (method, params = {}) => client.send(method, params, sessionId),
    on: (method, fn) => client.on(method, fn),
    close: () => client.close(),
    async eval(expression, { awaitPromise = false } = {}) {
      const send = () => client.send('Runtime.evaluate', {
        expression, returnByValue: true, awaitPromise,
      }, sessionId);
      let r;
      for (let attempt = 0; attempt < 3; attempt++) {
        try { r = await send(); break; }
        catch (e) {
          if (!/Cannot find default execution context/.test(e.message)) throw e;
          if (attempt === 2) throw e;
          await sleep(150);
        }
      }
      if (r.exceptionDetails) throw new Error(`JS exception: ${r.exceptionDetails.text}`);
      return r.result && r.result.value;
    },
    async navigate(url) {
      let loaded = false;
      client.on('Page.loadEventFired', () => { loaded = true; });
      await client.send('Page.navigate', { url }, sessionId);
      for (let i = 0; i < 30; i++) {
        if (loaded) break;
        await sleep(100);
        try {
          const r = await client.send('Runtime.evaluate', {
            expression: 'document.readyState', returnByValue: true,
          }, sessionId);
          if (r.result && r.result.value === 'complete') break;
        } catch (e) {
          if (!/Cannot find default execution context/.test(e.message)) throw e;
        }
      }
      await this.eval('1');
    },
  };
}

function sleep(ms) { return new Promise(r => setTimeout(r, ms)); }

module.exports = { connect, makeClient, sleep };

Run output today

Page.downloadWillBegin fired:               false
Browser.downloadProgress {state:completed}: false
file written to downloadPath:               false
file content matches response body:         false
FAIL: attachment response produced no download — no events fired, nothing written to disk.

Exit code 1. After the fix, all four lines read true, it prints PASS, and exits 0. Reproduced identically against the current public nightly and a local build of main (d695ce10).

Likely fix location

  • src/cdp/domains/browser.zigsetDownloadBehavior: parse and store behavior / downloadPath / eventsEnabled.
  • src/cdp/CDP.zig (BrowserContext) — hold the per-connection download policy.
  • src/browser/Frame.zigframeHeaderDoneCallback (recognize Content-Disposition: attachment), frameDataCallback / frameDoneCallback (stream the body to disk instead of discarding the .raw buffer), and handleClick (the a.download branch at Frame.zig:4099, which currently only logs a warning).
  • src/Notification.zig — add download_will_begin / download_progress event types.
  • src/cdp/domains/page.zig + src/cdp/domains/browser.zig — subscribe and emit Page.downloadWillBegin / Browser.downloadProgress.

Proposed approach

Smallest correct slice:

  1. Store the policy. Parse behavior / downloadPath / eventsEnabled in setDownloadBehavior and stash them on the per-connection BrowserContext.
  2. Detect the download. In the navigation response path, a response whose Content-Disposition is attachment is a download rather than a document. (Optionally also a non-displayable MIME with no explicit disposition, matching Chrome.)
  3. Divert the body to disk. Stream the bytes to <downloadPath>/<suggestedFilename> — filename from the Content-Disposition filename= parameter, else the URL basename — instead of buffering-then-discarding them as the .raw path does today.
  4. Emit the events. Add the two notification types and have the CDP layer emit Page.downloadWillBegin and a terminal Browser.downloadProgress { state: "completed" }.

The <a download> click trigger (Frame.handleClick) reuses the same machinery and behavior: deny could land as a follow-up, keeping the first PR focused on the Content-Disposition navigation path.

Important

One design question before I open a PR. In Chrome, navigating to an attachment response does not change the current document — the page stays put and the download runs in the background. Lightpanda commits the pending page in frameHeaderDoneCallback as soon as headers arrive, and the Content-Disposition header is available there, so the navigation can be recognized as a download before commit. Which behavior do you prefer?

  • (A) Preserve the current page (Chrome-faithful): detect the attachment in the header callback, skip the page commit, keep the previous document live, run the download as a side effect. More correct; touches the pending/active commit path.
  • (B) Let the navigation commit to a blank document and run the download as a side effect. Simpler and contained to the response-body path; since there is no rendering the post-download page state is rarely observed, but it diverges from Chrome's "page unchanged" semantics.

I'm happy to implement either — (A) is more faithful, (B) is a smaller, lower-risk first PR. Glad to follow your preference before writing the navigation-lifecycle code.

Environment

  • Lightpanda: current public nightly + local build of main at d695ce10 (both reproduce identically)
  • OS: macOS (aarch64); the download path is platform-independent
  • CDP client: Node 18 + ws (raw WebSocket; see cdp.js above)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions