Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve support for completely unknown architectures #10107

Merged

Conversation

alexcrichton
Copy link
Member

This commit is a step in the direction of trying to make Wasmtime more portable by default. The goal here is to enable Wasmtime to compile for architectures that it has no prior knowledge of. There's a few miscellaneous locations through Wasmtime where we need architecture-specific intrinsics and such but that's all in service of Cranelift itself. Without Cranelift support none of them are necessary.

This commit plumbs a custom #[cfg] from Wasmtime's build.rs script into the crate about whether there's a supported Cranelift backend. If this isn't available some architecture-specific intrinsics are turned off and not included. An example is that vm::arch entirely disappears which is only in service of UnwindHost, which also disappears. Furthermore the helpers.c file also entirely disappears as it's not necessary on unknown architectures.

To help keep this working I've added CI to build Wasmtime for powerpc64le-unknown-linux-gnu. Wasmtime currently has no support for this architecture, although if it grows such support in the future this'll need to be changed to some other unsupported architecture.

This commit is a step in the direction of trying to make Wasmtime more
portable by default. The goal here is to enable Wasmtime to compile for
architectures that it has no prior knowledge of. There's a few
miscellaneous locations through Wasmtime where we need
architecture-specific intrinsics and such but that's all in service of
Cranelift itself. Without Cranelift support none of them are necessary.

This commit plumbs a custom `#[cfg]` from Wasmtime's `build.rs` script
into the crate about whether there's a supported Cranelift backend. If
this isn't available some architecture-specific intrinsics are turned
off and not included. An example is that `vm::arch` entirely disappears
which is only in service of `UnwindHost`, which also disappears.
Furthermore the `helpers.c` file also entirely disappears as it's not
necessary on unknown architectures.

To help keep this working I've added CI to build Wasmtime for
`powerpc64le-unknown-linux-gnu`. Wasmtime currently has no support for
this architecture, although if it grows such support in the future
this'll need to be changed to some other unsupported architecture.
@alexcrichton alexcrichton requested review from a team as code owners January 24, 2025 21:17
@alexcrichton alexcrichton requested review from fitzgen and removed request for a team January 24, 2025 21:17
@github-actions github-actions bot added the wasmtime:api Related to the API of the `wasmtime` crate itself label Jan 24, 2025
Comment on lines 1649 to 1654
// When Cranelift has support for the host then we might be running native
// compiled code meaning we need to read the actual stack pointer. If
// Cranelift can't be used though then we're guaranteed to be running pulley
// in which case this stack poitner isn't actually used as Pulley has custom
// mechanisms for stack overflow.
#[cfg(has_cranelift_host_backend)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we rename this has_host_compiler_backend? While we don't have any target supported on Winch but not Cranelift now, it doesn't seem like it would be impossible in the future, so we should just try to be a little more precise about what we are saying.

@alexcrichton alexcrichton added this pull request to the merge queue Jan 27, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 27, 2025
Shuffle some files around to be more amenable to #[cfg]
@alexcrichton alexcrichton added this pull request to the merge queue Jan 27, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 27, 2025
That way it's got easy access to the #[cfg]'s from the build script
Even if custom signals are enabled, don't compile it in.

prtest:full
These aren't needed any more to compile Pulley
Instead of validating at `Engine` config time instead validate at
`Module` config time to enable cross-compilation.
@github-actions github-actions bot added the wasmtime:config Issues related to the configuration of Wasmtime label Jan 27, 2025
Copy link

Label Messager: wasmtime:config

It looks like you are changing Wasmtime's configuration options. Make sure to
complete this check list:

  • If you added a new Config method, you wrote extensive documentation for
    it.

    Our documentation should be of the following form:

    Short, simple summary sentence.
    
    More details. These details can be multiple paragraphs. There should be
    information about not just the method, but its parameters and results as
    well.
    
    Is this method fallible? If so, when can it return an error?
    
    Can this method panic? If so, when does it panic?
    
    # Example
    
    Optional example here.
    
  • If you added a new Config method, or modified an existing one, you
    ensured that this configuration is exercised by the fuzz targets.

    For example, if you expose a new strategy for allocating the next instance
    slot inside the pooling allocator, you should ensure that at least one of our
    fuzz targets exercises that new strategy.

    Often, all that is required of you is to ensure that there is a knob for this
    configuration option in wasmtime_fuzzing::Config (or one
    of its nested structs).

    Rarely, this may require authoring a new fuzz target to specifically test this
    configuration. See our docs on fuzzing for more details.

  • If you are enabling a configuration option by default, make sure that it
    has been fuzzed for at least two weeks before turning it on by default.


To modify this label's message, edit the .github/label-messager/wasmtime-config.md file.

To add new label messages or remove existing label messages, edit the
.github/label-messager.json configuration file.

Learn more.

Ensures that specific `--target pulleyNN` works most of the time.
No signal handlers are used at all with Pulley so when the async stack
overflows there's no message printed any more.
@alexcrichton
Copy link
Member Author

I probably should have done prtest:full sooner but as can probably be seen there's a lot of fixup commits here to get tests passing on all platforms in CI and such. Nontrivial changes include:

  • Validation of Tunables for execution at runtime is deferred until a module is loaded. This enables 32-bit platforms, which disable signals_based_traps, to compile for 64-bit platforms (e.g. the disas test suite)
  • Auto-configuration of Tunables based on crate features is skipped if --target is passed. This fixes disas cross-compilation tests from 32-to-64-bit where 64-bit hosts expect signals-based-traps.
  • Some standalone tests/*.rs files have moved from /tests/*.rs to /crates/wasmtime/tests/*.rs to be able to use the #[cfg] directives printed by crates/wasmtime/build.rs
  • Default Tunables for Pulley targets have been auto-adjusted to assume no signals-based-traps and no memory reservation. This fixes --target pulley32 on 32-bit platforms where the second bullet above wasn't otherwise applying to auto-disable from the defaults.
  • Stack overflow messages for overflowing the async stack no longer happen with Pulley. That's because we no longer (intentionally) install signal handlers.

Juggling compile options I feel still isn't great. Ideally everything "just works" but it's a narrow needle to thread trying to juggle different crate features and targetting different environments. Ideally we'd be able to query has_native_signals and has_virtual_memory based on target_lexicon::Triple, but that involves inferring cfg(unix) or cfg(windows) which is a bit of a stretch at this time. I've attempted to have at least a somewhat reasonable middle-ground in the meantime.

@alexcrichton alexcrichton added this pull request to the merge queue Jan 28, 2025
Merged via the queue into bytecodealliance:main with commit 4afa86b Jan 28, 2025
153 checks passed
@alexcrichton alexcrichton deleted the support-unknown-architecture branch January 28, 2025 01:26
This was referenced Jan 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wasmtime:api Related to the API of the `wasmtime` crate itself wasmtime:config Issues related to the configuration of Wasmtime
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants