Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement module linker #113

Merged
merged 41 commits into from
May 2, 2018
Merged

Implement module linker #113

merged 41 commits into from
May 2, 2018

Conversation

ballercat
Copy link
Owner

@ballercat ballercat commented Apr 15, 2018

Implement module imports.

It is difficult to go beyond toy examples without breaking the application into modules.

  • Relative imports are treated as module imports. Anything which starts with a . For example, import { a: i32 } from "./a.walt"; .
  • Everything else is left up to the user to provide. All modules share environment imports (anything not a relative module path)

Basic.

TODO:

  • Data section spanning multiple modules. String declarations
  • Type inference on cross-walt imports (so that repeated type annotation can be avoided)
  • Dogfood linker in compiler tests 😎
  • Support crude type imports (syntax needs help)
  • export type syntax support

Out of scope:

  • Webpack loader integration
  • Node cli wrapper

The change set here is large enough already that these features would need to be fleshed out in another set of PRs.

@xtuc
Copy link

xtuc commented Apr 17, 2018

What happens when you import { a: i32 } from "./a.js";? You could encode it in the binary and emit the JavaScript file. As said in WebAssembly/esm-integration#5 bundlers will take care of dynamically linking it for you.

@ballercat
Copy link
Owner Author

@xtuc

Thanks for the suggestion. That would be the ideal inside something like webpack for sure, but there some complications. I don't always want to depend on bundlers either because there should be a way to do this in vanilla node as well :) (bundler related work would need to happen in the loader)

I think the problem gets much more complicated when there are implicit dependencies between the two modules. Specifically the memory layout each module expects and static data sections. Each module is pre-compiled in isolation which makes this a problem. I have some ideas on how to tackle that but it seems to be pushing me more into the approach where all the Walt Program ASTs are merged and compiled together, and imports are basically hoisted onto a single module. Nowhere near implementing that though, but it'll be interesting once I do.

The whole thing is kind of bumming me out because I have no idea how multiple different source languages are supposed to play nice together because of memory issues 🤔 . Kind of seems like that is unlikely to happen until maybe there are multiple memories allowed in WASM.

Either way, I need to implement a couple of ideas and see how it shakes out before all the tradeoffs are known.

@xtuc
Copy link

xtuc commented Apr 17, 2018

Yes indeed, that makes a lot of sense. As far as I know bundlers will be a separate target in Emscripten.

different source languages are supposed to play nice together

For JavaScript is pretty obvious (ImportObject). For other languages it's more complicated; compiling + shared library with PIC should work or multiple modules/unmanaged closure.

Arty Buldauskas added 18 commits April 21, 2018 17:15
Parse all imports without compiling and collect
all syntax trees in one place.
Iterate over all asts and combine statics into a
global statics object.
Correctly build cross-module data sections. This also
fixes cases when a non-memory/data module is imported
as part of the program
Cover that memory can is not required in a module even if
the rest of the program may use it
@ballercat ballercat changed the title WIP - implement basic module linking Implement basic module linking Apr 29, 2018
Arty Buldauskas and others added 15 commits April 29, 2018 19:21
- Something is wrong with validation. Off for now in linker
- DX + 1,000,000
- Add DI compiler to the linker for diagnostics/debugging
- Fix type inference where an imported method has arity > 1
- Add validation spec to cover un-patched type inferences
- Fix a bug in error generator
- Add basic compiling of .wasm files to cli
- Output .wasm files
- JS Wrapping to happen later
- Types must be exported before use
- Add warning to linker if they are not
- Syntax is still valid if not used in another module
- Fix a typo in filename passed in linker validation
-
@ballercat ballercat changed the title Implement basic module linking Implement module linker May 2, 2018
@ballercat ballercat merged commit 54e97d7 into master May 2, 2018
@ballercat ballercat mentioned this pull request May 2, 2018
@ballercat ballercat deleted the imports branch May 2, 2018 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants