Skip to content

Add directory build/include/wasmtime#82

Merged
alexcrichton merged 3 commits intobytecodealliance:mainfrom
olivierlemasle:include-wasmtime
Jun 4, 2021
Merged

Add directory build/include/wasmtime#82
alexcrichton merged 3 commits intobytecodealliance:mainfrom
olivierlemasle:include-wasmtime

Conversation

@olivierlemasle
Copy link
Copy Markdown
Contributor

@olivierlemasle olivierlemasle commented Jun 3, 2021

Following the update to Wasmtime's new C API in #81, the header files are now not only in build/local, but also in build/local/wasmtime.

This commit adds the directory "build/include/wasmtime" as a valid empty go package, like it was done in #45 for the other build directories.

It also addapts ci/local.sh to:

  • copy the additional header files in build/include/wasmtime
  • stop this script from removing the "empty.go" files.

Following the update to Wasmtime's new C API in bytecodealliance#81, the header files are now not
only in build/local, but also in build/local/wasmtime.

This commit adds the directory "build/include/wasmtime" as a valid empty go
package, like it was done in bytecodealliance#45 for the other build directories.

It also addapts `ci/local.sh` to:
- copy the additional header files in build/include/wasmtime
- stop this script from removing the "empty.go" files.
@alexcrichton
Copy link
Copy Markdown
Member

Thanks for the PR!

I am personally still pretty confused by the empty.go files and why they're necessary. Can you be sure to update CI to fail if they're not present? I'm not sure why things are working for me locally but then they keep failing for users consuming this package?

Does this mean that everyone has to import with a _ the header file directory now too? If that's the case I'd rather fix this via some other means because I don't want to have to force all consumers to import internal details like that.

@olivierlemasle
Copy link
Copy Markdown
Contributor Author

I'm not sure why things are working for me locally but then they keep failing for users consuming this package?

It depends if you use dependency vendoring. By default, you don't, and everything is fine. Your don't have to import anything but github.com/bytecodealliance/wasmtime-go.

However, some projects use go mod vendor to make a local copy of their dependencies. In that case, Go uses creates and manages a directory vendor in the root directory of the project, and copies the vendored dependencies in it. Then Go uses this local copy of the dependencies to build the project. But as it copies only the packages that are referenced in the code, it does not copy the build directory, which is then not available for consumers :-(

You can try go mod vendor in a module-enabled version of your "hello world" example to reproduce the issue.

Does this mean that everyone has to import with a _ the header file directory now too? If that's the case I'd rather fix this via some other means because I don't want to have to force all consumers to import internal details like that.

At least one project uses this _ "hack" to reference the build subdirectories and copy them in their vendor directory: https://github.com/open-policy-agent/opa/blob/f1104893cbac176e0a9b752ba67a43af0d8ec0b0/internal/wasm/sdk/internal/wasm/vm.go#L16-L19

But it's because they decided to use this vendor feature; that's not the case for the majority of go projects.

@alexcrichton
Copy link
Copy Markdown
Member

Ah ok that makes sense, thanks for the info! Is it possible to move those import directives into this package itself without disrupting other users? Linked from that issue is another style like https://github.com/godror/godror/pull/38/files, and would something like that work?

In either case, though, I'd ideally like to add a test on CI that this works. This is something that I'll undoubtedly regress and break in the future otherwise after I forget...

@olivierlemasle
Copy link
Copy Markdown
Contributor Author

Is it possible to move those import directives into this package itself without disrupting other users? Linked from that issue is another style like https://github.com/godror/godror/pull/38/files, and would something like that work?

Yes, that's way better!

In either case, though, I'd ideally like to add a test on CI that this works. This is something that I'll undoubtedly regress and break in the future otherwise after I forget...

Sure, I'll add a CI test for that.

- Import build packages, to prevent `go mod vendor` from pruning the build
  directories;
- Update download-wasmtime.py to re-create the "empty.go" files and keep
  the Go packages in "build"
- Add a CI test to check that wasmtime-go can be used by Go projects using
  dependency vendoring
@olivierlemasle
Copy link
Copy Markdown
Contributor Author

I've updated the PR:

  • Import build packages, to prevent go mod vendor from pruning the build directories;
  • Update download-wasmtime.py to re-create the "empty.go" files and keep the Go packages in "build"
  • Add a CI test to check that wasmtime-go can be used by Go projects using dependency vendoring

If an empty.go file is missing, go build will fail as the packages are imported in ffi.go. If an import is missing in ffi.go, the script ./ci/test-vendoring.sh fails in CI.

@olivierlemasle
Copy link
Copy Markdown
Contributor Author

Bazel is failing. I guess I should add the import in BUILD.bazel... (I don't know bazel)

@alexcrichton
Copy link
Copy Markdown
Member

Looks great to me! But yeah unfortunately I don't really know much about BUILD.bazel either...

@olivierlemasle
Copy link
Copy Markdown
Contributor Author

Ok. I'm testing it locally with Bazel.

@olivierlemasle
Copy link
Copy Markdown
Contributor Author

olivierlemasle commented Jun 4, 2021

I've moved the "_" imports to a unused file (not in Bazel, and excluded from build by a build constraint); it seems to fix the Bazel issue, and it still makes go mod vendor to copy the headers and binaries.

@alexcrichton
Copy link
Copy Markdown
Member

Looks great to me, thanks again for this!

@alexcrichton alexcrichton merged commit 542c2ac into bytecodealliance:main Jun 4, 2021
@olivierlemasle olivierlemasle deleted the include-wasmtime branch June 4, 2021 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants