Reimplement how unwind information is stored #3180

alexcrichton · 2021-08-11T20:21:48Z

This commit is a major refactoring of how unwind information is stored
after compilation of a function has finished. Previously we would store
the raw UnwindInfo as a result of compilation and this would get
serialized/deserialized alongside the rest of the ELF object that
compilation creates. Whenever functions were registered with
CodeMemory this would also result in registering unwinding information
dynamically at runtime, which in the case of Unix, for example, would
dynamically created FDE/CIE entries on-the-fly.

Eventually I'd like to support compiling Wasmtime without Cranelift, but
this means that UnwindInfo wouldn't be easily available to decode into
and create unwinding information from. To solve this I've changed the
ELF object created to have the unwinding information encoded into it
ahead-of-time so loading code into memory no longer needs to create
unwinding tables. This change has two different implementations for
Windows/Unix:

On Windows the implementation was much easier. The unwinding
information on Windows is already stored after the function itself in
the text section. This was actually slightly duplicated in object
building and in code memory allocation. Now the object building
continues to do the same, recording unwinding information after
functions, and code memory no longer manually tracks this.
Additionally Wasmtime will emit a special custom section in the object
file with unwinding information which is the list of
RUNTIME_FUNCTION structures that RtlAddFunctionTable expects. This
means that the object file has all the information precompiled into it
and registration at runtime is simply passing a few pointers around to
the runtime.
Unix was a little bit more difficult than Windows. Today a .eh_frame
section is created on-the-fly with offsets in FDEs specified as the
absolute address that functions are loaded at. This absolute
address hindered the ability to precompile the FDE into the object
file itself. I've switched how addresses are encoded, though, to using
DW_EH_PE_pcrel which means that FDE addresses are now specified
relative to the FDE itself. This means that we can maintain a fixed
offset between the .eh_frame loaded in memory and the beginning of
code memory. When doing so this enables precompiling the .eh_frame
section into the object file and at runtime when loading an object no
further construction of unwinding information is needed.

The overall result of this commit is that unwinding information is no
longer stored in its cranelift-data-structure form on disk. This means
that this unwinding information format is only present during
compilation, which will make it that much easier to compile out
cranelift in the future.

This commit also significantly refactors CodeMemory since the way
unwinding information is handled is not much different from before.
Previously CodeMemory was suitable for incrementally adding more and
more functions to it, but nowadays a CodeMemory either lives per
module (in which case all functions are known up front) or it's created
once-per-Func::new with two trampolines. In both cases we know all
functions up front so the functionality of incrementally adding more and
more segments is no longer needed. This commit removes the ability to
add a function-at-a-time in CodeMemory and instead it can now only
load objects in their entirety. A small helper function is added to
build a small object file for trampolines in Func::new to handle
allocation there.

Finally, this commit also refactors the wasmtime-obj crate and its
builder structure to be more amenable to this strategy of managing
unwinding tables.

It is not intentional to have any real functional change as a result of
this commit. This might accelerate loading a module from cache slightly
since less work is needed to manage the unwinding information, but
that's just a side benefit from the main goal of this commit which is to
remove the dependence on cranelift unwinding information being available
at runtime.

~~On a procedural note this draws from #3179 and #3176 so only the last commit here is what's to be reviewed.~~

github-actions · 2021-08-11T20:41:27Z

Subscribe to Label Action

cc @peterhuene

Details

This issue or pull request has been labeled: "cranelift", "cranelift:wasm", "lightbeam", "wasmtime:api"

Thus the following users have been cc'd because of the following labels:

peterhuene: wasmtime:api

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

alexcrichton · 2021-08-16T17:11:23Z

Ok I've rebased on main which involved a fair bit of code movement (as expected since this heavily conflicted with #3178). Morally though everything is the same, though, just in a different place.

This is hopefully one of the last major steps to excising cranelift!

fitzgen

Nice! Phew, that was a lot to read through, but it looks great.

crates/cranelift/src/obj.rs

fitzgen · 2021-08-17T20:13:50Z

crates/cranelift/src/obj.rs

+    /// This will generate a `.eh_frame` section, but not one that can be
+    /// naively loaded. The goal of this section is that we can create the
+    /// section once here and never again does it need to change. To describe
+    /// dynamically loaded functions though each individual FDE needs to talk
+    /// about the function's absolute address that it's referencing. Naturally
+    /// we don't actually know the function's absolute address when we're
+    /// creating an object here.
+    ///
+    /// To solve this problem the FDE address encoding mode is set to
+    /// `DW_EH_PE_pcrel`. This means that the actual effective address that the
+    /// FDE describes is a relative to the address of the FDE itself. By
+    /// leveraging this relative-ness we can assume that the relative distance
+    /// between the FDE and the function it describes is constant, which should
+    /// allow us to generate an FDE ahead-of-time here.


To clarify: the alternative to this whole DW_EH_PE_pcrel approach would be to emit relocs for the FDEs, right?

Indeed yeah. I'll admit that I know so little about relocations I didn't really try to do this at all and I didn't feel I needed to pursue this much because I got the pcrel bits working. Not having relocations is somewhat beneficial in that loading these tables is that much faster (it's just what's in-memory) and in theory amenable to the hypothetical mmap-and-go world.

crates/jit/src/code_memory.rs

fitzgen · 2021-08-17T20:40:02Z

crates/jit/src/unwind/systemv.rs

+            // libgcc stores the frame entries as a linked list in decreasing
+            // sort order based on the PC value of the registered entry.
+            //
+            // As we store the registrations in increasing order, it would be
+            // O(N^2) to deregister in that order.
+            //
+            // To ensure that we just pop off the first element in the list upon
+            // every deregistration, walk our list of registrations backwards.
+            for fde in self.registrations.iter().rev() {
+                __deregister_frame(*fde as *const _);


ugh I forgot about this bug

This commit is a major refactoring of how unwind information is stored after compilation of a function has finished. Previously we would store the raw `UnwindInfo` as a result of compilation and this would get serialized/deserialized alongside the rest of the ELF object that compilation creates. Whenever functions were registered with `CodeMemory` this would also result in registering unwinding information dynamically at runtime, which in the case of Unix, for example, would dynamically created FDE/CIE entries on-the-fly. Eventually I'd like to support compiling Wasmtime without Cranelift, but this means that `UnwindInfo` wouldn't be easily available to decode into and create unwinding information from. To solve this I've changed the ELF object created to have the unwinding information encoded into it ahead-of-time so loading code into memory no longer needs to create unwinding tables. This change has two different implementations for Windows/Unix: * On Windows the implementation was much easier. The unwinding information on Windows is already stored after the function itself in the text section. This was actually slightly duplicated in object building and in code memory allocation. Now the object building continues to do the same, recording unwinding information after functions, and code memory no longer manually tracks this. Additionally Wasmtime will emit a special custom section in the object file with unwinding information which is the list of `RUNTIME_FUNCTION` structures that `RtlAddFunctionTable` expects. This means that the object file has all the information precompiled into it and registration at runtime is simply passing a few pointers around to the runtime. * Unix was a little bit more difficult than Windows. Today a `.eh_frame` section is created on-the-fly with offsets in FDEs specified as the absolute address that functions are loaded at. This absolute address hindered the ability to precompile the FDE into the object file itself. I've switched how addresses are encoded, though, to using `DW_EH_PE_pcrel` which means that FDE addresses are now specified relative to the FDE itself. This means that we can maintain a fixed offset between the `.eh_frame` loaded in memory and the beginning of code memory. When doing so this enables precompiling the `.eh_frame` section into the object file and at runtime when loading an object no further construction of unwinding information is needed. The overall result of this commit is that unwinding information is no longer stored in its cranelift-data-structure form on disk. This means that this unwinding information format is only present during compilation, which will make it that much easier to compile out cranelift in the future. This commit also significantly refactors `CodeMemory` since the way unwinding information is handled is not much different from before. Previously `CodeMemory` was suitable for incrementally adding more and more functions to it, but nowadays a `CodeMemory` either lives per module (in which case all functions are known up front) or it's created once-per-`Func::new` with two trampolines. In both cases we know all functions up front so the functionality of incrementally adding more and more segments is no longer needed. This commit removes the ability to add a function-at-a-time in `CodeMemory` and instead it can now only load objects in their entirety. A small helper function is added to build a small object file for trampolines in `Func::new` to handle allocation there. Finally, this commit also folds the `wasmtime-obj` crate directly into the `wasmtime-cranelift` crate and its builder structure to be more amenable to this strategy of managing unwinding tables. It is not intentional to have any real functional change as a result of this commit. This might accelerate loading a module from cache slightly since less work is needed to manage the unwinding information, but that's just a side benefit from the main goal of this commit which is to remove the dependence on cranelift unwinding information being available at runtime.

Remove everything non-essential so that only the bits which will need to be refactored out of cranelift remain.

github-actions bot added cranelift Issues related to the Cranelift code generator cranelift:wasm lightbeam Issues related to the Lightbeam compiler wasmtime:api Related to the API of the `wasmtime` crate itself labels Aug 11, 2021

alexcrichton mentioned this pull request Aug 12, 2021

Wasmtime doesn't work on Linux AArch64 with PAC enabled #3183

Closed

alexcrichton force-pushed the refactor-unwind branch from 8cbf065 to 265824d Compare August 12, 2021 20:16

alexcrichton requested a review from fitzgen August 12, 2021 20:39

alexcrichton force-pushed the refactor-unwind branch 3 times, most recently from 75d2f9f to 26d74e2 Compare August 16, 2021 17:08

alexcrichton force-pushed the refactor-unwind branch 2 times, most recently from 1676ad4 to dcc7c0a Compare August 16, 2021 20:33

fitzgen approved these changes Aug 17, 2021

View reviewed changes

alexcrichton added 5 commits August 17, 2021 14:22

Remove isa reexport from wasmtime-environ

703f730

Trim down reexports of cranelift-codegen

06333f7

Remove everything non-essential so that only the bits which will need to be refactored out of cranelift remain.

Fix debug tests

e58adae

Review comments

256224c

alexcrichton force-pushed the refactor-unwind branch from f7ed3a9 to 256224c Compare August 17, 2021 21:22

alexcrichton merged commit e8aa7bb into bytecodealliance:main Aug 17, 2021

alexcrichton deleted the refactor-unwind branch August 17, 2021 22:14

kadiwa4 mentioned this pull request Jan 2, 2022

crates/jit/src/unwind/winx32.rs doesn’t compile #3634

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Reimplement how unwind information is stored #3180

Reimplement how unwind information is stored #3180

Uh oh!

alexcrichton commented Aug 11, 2021 •

edited

Loading

Uh oh!

github-actions bot commented Aug 11, 2021

Uh oh!

alexcrichton commented Aug 16, 2021

Uh oh!

fitzgen left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fitzgen Aug 17, 2021

Uh oh!

alexcrichton Aug 17, 2021

Uh oh!

Uh oh!

fitzgen Aug 17, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Reimplement how unwind information is stored #3180

Reimplement how unwind information is stored #3180

Uh oh!

Conversation

alexcrichton commented Aug 11, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 11, 2021

Subscribe to Label Action

Uh oh!

alexcrichton commented Aug 16, 2021

Uh oh!

fitzgen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fitzgen Aug 17, 2021

Choose a reason for hiding this comment

Uh oh!

alexcrichton Aug 17, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

fitzgen Aug 17, 2021

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

alexcrichton commented Aug 11, 2021 •

edited

Loading