Skip to content

feat: add support for bun.lock#379

Closed
G-Rath wants to merge 11 commits into
google:mainfrom
ackama:bun/support
Closed

feat: add support for bun.lock#379
G-Rath wants to merge 11 commits into
google:mainfrom
ackama:bun/support

Conversation

@G-Rath

@G-Rath G-Rath commented Jan 12, 2025

Copy link
Copy Markdown
Collaborator

As part of improving ergonomics, Bun has introduced a new text-based lockfile which this adds support for extracting.

While the blog post claims that it's "JSON with comments" (aka JSONC), it's actually "JSON with comments and trailing commas" (aka JWCC or HuJSON) - since the Go standard library only supports parsing standard JSON, I've had to bring in a third-party library to handle parsing, which is designed to leverage the existing standard library by instead handling "standardizing" the input into valid boring JSON.

Aside from the general question of if this library is acceptable to use here, it also seems to require Go 1.23 resulting in the addition of a toolchain line in go.mod which I've never managed to quite figure out what the right thing to do with - overall, I'd like someone from Google to confirm what library they'd prefer we use here. (I've since switched to using a library that requires Go 1.16)

I have also specified the testdata fixtures as JSON5 as that's the most appropriate format supported by both VSCode and IntelliJ/GoLand, though that technically supports more features like single quotes (which it actually seems like bun does not mind if you use in your lockfile, though it'll always use double quotes itself) - personally I think that's fine, but don't mind renaming the files to be .hujson if folks would prefer.

Resolves google/osv-scanner#1405

Comment thread extractor/filesystem/language/javascript/bunlock/bunlock.go Outdated
Comment thread extractor/filesystem/language/javascript/bunlock/bunlock.go Outdated
Comment thread extractor/filesystem/language/javascript/bunlock/bunlock.go Outdated
Comment thread extractor/filesystem/language/javascript/bunlock/bunlock.go Outdated
Comment thread extractor/filesystem/language/javascript/bunlock/bunlock.go Outdated
Comment thread extractor/filesystem/language/javascript/bunlock/bunlock_test.go Outdated
@erikvarga

Copy link
Copy Markdown
Collaborator

It's generally fine to add new libraries if we think they can't be implemented easily on our side. Can you double check that this new import doesn't cause the SCALIBR binary size to get too much bigger?

Regarding the Go version, do you mean that the lib needs Go 1.23 or beyond? We use Go 1.24 internally so requiring 1.23 or beyond should be fine for us.

@G-Rath

G-Rath commented Jan 13, 2025

Copy link
Copy Markdown
Collaborator Author

Regarding the Go version, do you mean that the lib needs Go 1.23 or beyond? We use Go 1.24 internally so requiring 1.23 or beyond should be fine for us.

Yes that's my read of it - I think bumping the Go version could be a breaking change but either way probably best done in a dedicated PR.

I'm happy to open that and we can continue any related discussion over there

@G-Rath

G-Rath commented Jan 13, 2025

Copy link
Copy Markdown
Collaborator Author

Can you double check that this new import doesn't cause the SCALIBR binary size to get too much bigger?

osv-scalibr on  main [?] via 🐹 v1.22.5 via  v20.11.0
❯ du scalibr-old scalibr-new scalibr-alt
48396   scalibr-old
49512   scalibr-new
49304   scalibr-alt

The scalibr-alt version is built with tidwall/jsonc which was another library I found that could work - I went with the current library to start as it's maintained by tailscale and looks to be actively maintained, but I don't think there's a huge risk if we wanted to use this alt library (which would also be a little nicer as it doesn't return an error, so we'd have one less condition branch)

@G-Rath

G-Rath commented Jan 13, 2025

Copy link
Copy Markdown
Collaborator Author

@erikvarga I have gone ahead with switching out the library since tidwall/jsonc only requires Go v1.16, so this means I can get CI green but happy to switch back to the tailscale library if you're prefer (or look at other alternatives, as there's a couple out there)

Comment thread extractor/filesystem/language/javascript/bunlock/bunlock.go Outdated
@erikvarga

Copy link
Copy Markdown
Collaborator

@erikvarga I have gone ahead with switching out the library since tidwall/jsonc only requires Go v1.16, so this means I can get CI green but happy to switch back to the tailscale library if you're prefer (or look at other alternatives, as there's a couple out there)

Hm, if using tailscale means we have to wait until Go 1.24 is released so we'll stay compatible with the last 2 versions then maybe we're better off using tidwall/jsonc.

Comment thread extractor/filesystem/language/javascript/bunlock/bunlock_test.go Outdated
@G-Rath
G-Rath deleted the bun/support branch January 28, 2025 19:20
@G-Rath

G-Rath commented Jan 29, 2025

Copy link
Copy Markdown
Collaborator Author

@another-rex @erikvarga it doesn't look like this was landed properly - the import has been changed from github.com/tidwall/jsonc to github.com/tidwall/jsonc/pkg/jsonc, which does not exist so everything errors.

Happy to do a PR changing that back, but not sure if that change was intentional as a result of your internals and that maybe something else should be being done?

The more I think about it, the more I think this was a mistake related to internal vs external handling, so I've opened #420

@erikvarga

Copy link
Copy Markdown
Collaborator

Oops, I think I messed up the open source package name in our internal transformation rules. I'll have to fix that internally tomorrow (unless OSV folks get around to doing it first)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

REQUEST: Support bun.lock lockfile used by Bun

4 participants