feat(cdp): implement Browser.setDownloadBehavior file downloads#2722
Conversation
|
Hi, I have really enjoyed contributing to this repo so far. Always happy to help with other open issues if useful. |
3715b14 to
2b22578
Compare
|
I have rebased this branch to the latest |
|
@Ar-maan05 thanks for the PR, It seems that the tests are failing w/ playwright.
|
|
I have fixed the merge conflicts and ensured all the tests pass. CI should be green now. Thank you for the review. Always happy to help with any other issues if useful. |
| } | ||
|
|
||
| // Strips any directory components, guarding against path traversal. | ||
| fn basename(name: []const u8) ?[]const u8 { |
There was a problem hiding this comment.
Can't we use std.fs.path.basename instead?
There was a problem hiding this comment.
Done. Replaced it with std.fs.path.basename (the helper is now sanitizeFilename). I kept a backslash strip on top of it because a Content-Disposition can carry Windows separators even on a Linux host, where the posix basename wouldn't split them, plus the ./../empty guard.
| } | ||
|
|
||
| // Derives a filename from a URL's last path segment, ignoring query/fragment. | ||
| fn urlBasename(url: []const u8) ?[]const u8 { |
There was a problem hiding this comment.
I would use a real url parsing instead. see webapi/URL.zig.
There was a problem hiding this comment.
Switched to real URL parsing: it now goes through rust-url (sys/url.zig url_parse + url_get_path) and takes the basename of the parsed path, so query/fragment are handled there rather than by hand.
| } | ||
|
|
||
| // Returns the (optionally quoted) value of `key=` within a parameter list. | ||
| fn paramValue(disposition: []const u8, key: []const u8) ?[]const u8 { |
There was a problem hiding this comment.
Is it specific to an HTTP header value isn't it?
Maybe we could move it into Header struct from http.zig.
Than you can use directly the Header you get from the response's header iterator.
There was a problem hiding this comment.
Maybe you could also implement a values iterator into Header struct, loop hover them and get the value you need directly: attachement, filename, ...
There was a problem hiding this comment.
Moved this onto the Header struct in network/http.zig. It now exposes firstValue() (the value before the first ;), param(key), and a params() ParamIterator. dispositionFilename takes the Header straight from the response header iterator and uses .param("filename") / .param("filename*"), so the Content-Disposition-specific string slicing is gone from Frame.
|
|
||
| pub fn onDownloadProgress(ctx: *anyopaque, msg: *const Notification.DownloadProgress) !void { | ||
| const self: *BrowserContext = @ptrCast(@alignCast(ctx)); | ||
| return @import("domains/browser.zig").downloadProgress(self, msg); |
There was a problem hiding this comment.
I'm not sure I get why you send Browser.downloadProgress and Page.downloadWillBegin and not also Browser.downloadWillBegin + Page.downloadProgress?
There was a problem hiding this comment.
Afterall, since we set Browser.setDownloadBehavior I think we should only dispatch Browser.XXX events and not Pages.XXX
There was a problem hiding this comment.
Agreed. Since the opt-in is Browser.setDownloadBehavior, I dropped the Page.* events and now emit Browser.downloadWillBegin + Browser.downloadProgress only. The downloadWillBegin emitter moved from page.zig to browser.zig alongside downloadProgress.
| // of being parsed as a page, and (when `download_events_enabled` is set) | ||
| // `Page.downloadWillBegin` / `Browser.downloadProgress` events are emitted. | ||
| // `download_path` is duped into the Session arena. | ||
| download_behavior: DownloadBehavior = .default, |
There was a problem hiding this comment.
I think default is useless, it should be removed and we should set directly deny instead.
And if the CDP command sends a default, then explicitly set the behavior as deny.
There was a problem hiding this comment.
Removed the .default variant; the field now defaults to .deny, and a CDP behavior: "default" is explicitly mapped to .deny.
| // })) orelse return error.InvalidParams; | ||
| const params = (try cmd.params(struct { | ||
| behavior: []const u8, | ||
| browserContextId: ?[]const u8 = null, |
There was a problem hiding this comment.
If browserContextId write a not_implemented log.
if (params.browserContextId != null) {
log.warn(.not_implemented, "Browser.setDownloadBehavior", .{ .param = "browserContextId" });
}There was a problem hiding this comment.
Added the not_implemented log as suggested.
| .download => |*download| { | ||
| download.file.writeAll(data) catch |err| { | ||
| log.err(.frame, "download write", .{ .err = err, .guid = download.guid }); | ||
| return; |
There was a problem hiding this comment.
I think we should handle the error here more correctly here, but idk exactly how.
We can't set the parse_state to err directly b/c we could hit the .err => unreachable on next chunck.
So we should either:
- change that unreachable on error
- set a failure/error on download
And maybe remove the eventually partial file written?
I suggest to write a TODO here for a follow up instead.
There was a problem hiding this comment.
Left a TODO here for a follow-up, as you suggested. Noted the two open parts: it can't set parse_state = .err without tripping the .err => unreachable on the next chunk, and it should emit a canceled progress event and remove the partial file.
|
@krichprollsch pushed a round addressing all the review comments. Full suite green ( |
0b3e326 to
67f7c89
Compare
|
Thanks for the changes, LGMT. I'm preparing a demo/test script for the feature into https://github.com/lightpanda-io/demo/ |
|
Sounds good! Let me know if I can be of any help with this or with any issues/feature implementations. |
Browser.setDownloadBehavior was a noop, so Lightpanda had no file-download path. A response with Content-Disposition: attachment is now streamed to disk under downloadPath, and Page.downloadWillBegin / Browser.downloadProgress are emitted when eventsEnabled. Fixes lightpanda-io#2701
…ontext Playwright sends Browser.setDownloadBehavior at the browser level during connection setup, before any target/context exists. Returning an error there aborted the whole connection, failing every playwright demo-runner test. Treat the no-context case as a success no-op, matching the prior behavior.
…eal url/basename, deny default
67f7c89 to
1f85ff1
Compare
5d441ad to
0f5ee79
Compare
Fixes #2701
Summary
Browser.setDownloadBehaviorwas a noop (every param commented out, bare success returned), so Lightpanda had no file-download path at all: a response carryingContent-Disposition: attachmentwas treated as a normal navigation,its body discarded,
downloadPathignored, and no download events emitted.This wires up the full path. When
Browser.setDownloadBehavioropts in (allow/allowAndName) and a navigation response carriesContent-Disposition: attachment, the body is streamed to disk underdownloadPathinstead of being parsed as a page, and, wheneventsEnabled: true,Page.downloadWillBeginandBrowser.downloadProgress(inProgress->completed) are emitted.Approach
This follows the existing notification pattern:
Framedispatches notifications and the CDP domains consume them and emit CDP events (the same shape asframe_navigated->Page.frameNavigated).Session.zig: download config (download_behavior,download_path,download_events_enabled), scoped per CDP connection.Notification.zig: newDownloadWillBegin/DownloadProgresseventFrame.zig: a.downloadparse state that holds the open file; detect the attachment inframeHeaderDoneCallback, stream the body to disk inframeDataCallback, close + emitcompletedinframeDoneCallback, then commit an empty document so the navigation still finishes cleanly (mirrorsthe existing
.rawpath). Content-Disposition parsing handles the quoted, unquoted, and RFC 5987 (filename*=) forms, and strips path components to guard against traversal.Runner.zig:.downloadhandled as a loading state.CDP.zig: idempotent registration for the download notifications.browser.zig/page.zig:setDownloadBehaviornow parses and storesits params and toggles event registration;
Browser.downloadProgressandPage.downloadWillBeginemitters.Behavior notes / scope
<a download>clicks are the same operation from JS but go through a different entry point; left as a follow-up.denyfalls through to the prior behavior (the response is not written to disk); no cancellation event is emitted.filename*values are taken verbatim (no percent-decoding yet).Testing
setDownloadBehaviorwith a tempdownloadPath, navigates to aContent-Disposition: attachmentresponse, and asserts bothPage.downloadWillBeginandBrowser.downloadProgressfire and that the file lands on disk with the correct bytes.behavior.make test), formatting clean (zig fmt --check).Checklist
make test).zig fmt --check ./*.zig ./**/*.zig).