Refactor wasi:http headers' host representation#12754
Refactor wasi:http headers' host representation#12754alexcrichton merged 2 commits intobytecodealliance:mainfrom
wasi:http headers' host representation#12754Conversation
This commit is a follow-on/extension of bytecodealliance#12748 and extends the changes made for WASIp2 headers in bytecodealliance#12652 to the WASIp3 implementation as well. This is done through a number of refactorings to make the WASIp2 and WASIp3 implementations more similar in terms of how they represent headers. Changes here are: * `FieldMap` now has its own dedicated module at the crate root instead of intermingling with other WASIp2 types. * `FieldMap` is now internally-`Arc`'d and is cheaply clonable. `FieldMap` itself now tracks whether it's mutable or immutable (WASI semantics) and doesn't need different wrappers in WASIp2 and WASIp3. * Creation of an immutable `FieldMap` can be done without needing a size limit. Flagging a `FieldMap` as mutable, however, requires a size limit. * `FieldMap::set` was added to be a bit more efficient w.r.t. clones. * `FieldMapError` is a new error type that covers all of the possible error modes of operating with a `FieldMap`. Conversions from this to WASIp{2,3} `header-error` types are now implemented as well. * WASIp2 now flags `header-error` as a trappable-error-type, allowing the use of `?` in implementing header functions (like WASIp3). * Much of WASIp2's header implementation was refactored with `?`, moving methods around, shuffling where headers are made vs `FieldMap`, some minor idioms, etc. * WASIp3 no longer uses `MaybeMutable` for headers and instead uses `FieldMap` directly. cc bytecodealliance#12674
94028fc to
a6c1996
Compare
|
Another caveat to this implementation: WASIp2 no longer uses child resources for headers, meaning that we're probably technically in violation of the spec. That being said the WASIp2 semantics now mirror WASIp3, so I'd personally consider this ok since WASIp3 is the defacto-expectations in the future. |
As I've said elsewhere previously, child resources were the wrong way to go about literally everything they were used for and I am glad we are getting rid of them in WASIp3. I don't think that lifting the restriction from the implementation in p2 is harmful - all existing guest programs that expected to follow the child rules will continue to work properly. Though there is some risk that a new p2 guest is written without expectation of the child rules and is non-portable to hosts with an older version of wasmtime-wasi-http, I don't know if we need to worry about this in practice. Also, FWIW, in the two private embeddings Chris and I work on, we didn't use child resources for headers. |
pchickey
left a comment
There was a problem hiding this comment.
Thanks, this is a much nicer structure.
…2754) * Refactor `wasi:http` headers' host representation This commit is a follow-on/extension of bytecodealliance#12748 and extends the changes made for WASIp2 headers in bytecodealliance#12652 to the WASIp3 implementation as well. This is done through a number of refactorings to make the WASIp2 and WASIp3 implementations more similar in terms of how they represent headers. Changes here are: * `FieldMap` now has its own dedicated module at the crate root instead of intermingling with other WASIp2 types. * `FieldMap` is now internally-`Arc`'d and is cheaply clonable. `FieldMap` itself now tracks whether it's mutable or immutable (WASI semantics) and doesn't need different wrappers in WASIp2 and WASIp3. * Creation of an immutable `FieldMap` can be done without needing a size limit. Flagging a `FieldMap` as mutable, however, requires a size limit. * `FieldMap::set` was added to be a bit more efficient w.r.t. clones. * `FieldMapError` is a new error type that covers all of the possible error modes of operating with a `FieldMap`. Conversions from this to WASIp{2,3} `header-error` types are now implemented as well. * WASIp2 now flags `header-error` as a trappable-error-type, allowing the use of `?` in implementing header functions (like WASIp3). * Much of WASIp2's header implementation was refactored with `?`, moving methods around, shuffling where headers are made vs `FieldMap`, some minor idioms, etc. * WASIp3 no longer uses `MaybeMutable` for headers and instead uses `FieldMap` directly. cc bytecodealliance#12674 * Clippy warnings
…12769) * Enable multiple concurrent sync host calls from different threads in the same task (#12735) * Move sync call set to thread state * Add test * Cleanup * Cleanup * Fix caller context on fused component<->component returns (#12737) In #12718 I added a test for more cases but forgot to update the `*.wast` to actually execute the test. Turns out all the cases were failing, and so this commit properly enables the tests and then fixes them. * wasip3: Limit random number generation by default (#12745) This commit extends the random-related fixes of #12652 to WASIp3's implementation of randomness-related interfaces. cc #12674 * Fix lost wakeups with stdin and wasip3 (#12711) I've been running some tests with wasip3 recently and I was running into a situation where a program would read stdin, get some data, and then stdin would be closed. The second read of stdin wouldn't get a wakeup and would get stuck forever despite stdin being closed. I'm not 100% sure what was happening but I'm highly suspect of the `Notify`-based synchronization here as I know historically that's a tricky primitive to work with. This applies a hammer and moves some lock scopes up a bit further to avoid dealing with trickiness and instead ensure everything proceeds in lockstep. * Refactor WASIp2 `wasi:http` implementation (#12748) * Sequester WASIp2 in `wasmtime-wasi-http` to a module This mirrors the `wasmtime-wasi` crate's organization where there's a `p2` module and a `p3` module at the top level. * Refactor WASIp2 `wasi:http` implementation This commit reorganizes and refactors the WASIp2 implementation of `wasi:http` to look more like other `wasmtime-wasi`-style interfaces. Specifically the old `WasiHttpImpl<T>` structure is removed in favor of as `WasiHttpCtxView<'_>` type that is used to implement bindgen-generated `Host` traits. This necessitated reorganizing the methods of the previous `WasiHttpView` trait like so: * The `WasiHttpView` trait is renamed to `WasiHttpHooks` to make space for a new `WasiHttpView` which behaves like `WasiView`, for example. * The `ctx` and `table` methods of `WasiHttpHooks` were removed since they'll be fields in `WasiHttpCtxView`. * Helper methods for WASIp2 were moved to methods on `WasiHttpCtxView` instead of default methods on `WasiHttpHooks`. With these changes in place the WASIp3 organization was also updated slightly as well. Notably WASIp3 now contains a reference to the crate's `WasiHttpCtx` structure (which has field limits for example). WASIp3's previous `WasiHttpCtx` trait is now renamed to `WasiHttpHooks` as well. This means that there are two `WasiHttpHooks` traits right now, one for WASIp2 and one for WASIp3. In the future I would like to unify these two but that will require some more work around the default `send_request`. A final note here is that the `WasiHttpHooks` trait previously, and continues to be, optional for embedders to implement. Default functions are provided as `wasmtime_wasi_http::{p2, p3}::default_hooks`. Additionally there's a `Default for &mut dyn WasiHttpHooks` implementation, too. With all that outlined: the motivation for this change is to bring the WASIp2 and WASIp3 implementations of `wasi:http` closer together. This is inspired by refactorings I was doing for #12674 to apply the same header limitations for WASIp3 as is done for WASIp2. Prior to this change there were a number of differences such as WASIp3 not having `crate::WasiHttpCtx` around, WASIp2 having a different organization of structures/borrows, etc. The goal is to bring the two implementations closer in line with each other to make refactoring across them more consistent and easier. * Make `WasiHttp` in WASIp2 public * Fix some conditional build * Fix some doctests * Fix configured build * Fixup documentation * Refactor `wasi:http` headers' host representation (#12754) * Refactor `wasi:http` headers' host representation This commit is a follow-on/extension of #12748 and extends the changes made for WASIp2 headers in #12652 to the WASIp3 implementation as well. This is done through a number of refactorings to make the WASIp2 and WASIp3 implementations more similar in terms of how they represent headers. Changes here are: * `FieldMap` now has its own dedicated module at the crate root instead of intermingling with other WASIp2 types. * `FieldMap` is now internally-`Arc`'d and is cheaply clonable. `FieldMap` itself now tracks whether it's mutable or immutable (WASI semantics) and doesn't need different wrappers in WASIp2 and WASIp3. * Creation of an immutable `FieldMap` can be done without needing a size limit. Flagging a `FieldMap` as mutable, however, requires a size limit. * `FieldMap::set` was added to be a bit more efficient w.r.t. clones. * `FieldMapError` is a new error type that covers all of the possible error modes of operating with a `FieldMap`. Conversions from this to WASIp{2,3} `header-error` types are now implemented as well. * WASIp2 now flags `header-error` as a trappable-error-type, allowing the use of `?` in implementing header functions (like WASIp3). * Much of WASIp2's header implementation was refactored with `?`, moving methods around, shuffling where headers are made vs `FieldMap`, some minor idioms, etc. * WASIp3 no longer uses `MaybeMutable` for headers and instead uses `FieldMap` directly. cc #12674 * Clippy warnings * Set current thread before lowering stream/future reads (#12736) * Set thread before lowering stream/future reads * Restore thread * Make CurrentThread pub(crate) * fmt * Remove with_thread * Remove DS_Store * Add test * Check return values * Move string data * Update some github actions versions (#12762) CI is warning us about these, so try updating. * Enable limiting wasip3 resource limits (#12761) * Enable limiting wasip3 resource limits This commit adds a new `Store::concurrent_resource_table` method which enables getting a handle to the underlying `ResourceTable` used by the concurrent implementation of component-model-async. This can in turn be used to set the max capacity on the table and limit the guest usage of the table. Closes #11552 * Adjust features * Fix imports * Consume hostcall fuel when buffering stream data in the host (#12767) For guest-to-guest communication stream reads/writes rendezvousing together will currently copy data through `Val`. This is expected to become more optimized in the future, but for now this needs to consume the concept of "hostcall fuel" introduced in #12652 to ensure that the guest can't exhaust memory in the host. This additionally tweaks some hostcall fuel calculations to more accurately reflect the size of values on the host, notably by using `size_of::<Thing>()` on the host rather than the size in the guest. Closes #12674 --------- Co-authored-by: Sy Brand <[email protected]>
This commit is a follow-on/extension of #12748 and extends the changes made for WASIp2 headers in #12652 to the WASIp3 implementation as well. This is done through a number of refactorings to make the WASIp2 and WASIp3 implementations more similar in terms of how they represent headers. Changes here are:
FieldMapnow has its own dedicated module at the crate root instead of intermingling with other WASIp2 types.FieldMapis now internally-Arc'd and is cheaply clonable.FieldMapitself now tracks whether it's mutable or immutable (WASI semantics) and doesn't need different wrappers in WASIp2 and WASIp3.FieldMapcan be done without needing a size limit. Flagging aFieldMapas mutable, however, requires a size limit.FieldMap::setwas added to be a bit more efficient w.r.t. clones.FieldMapErroris a new error type that covers all of the possible error modes of operating with aFieldMap. Conversions from this to WASIp{2,3}header-errortypes are now implemented as well.header-erroras a trappable-error-type, allowing the use of?in implementing header functions (like WASIp3).?, moving methods around, shuffling where headers are made vsFieldMap, some minor idioms, etc.MaybeMutablefor headers and instead usesFieldMapdirectly.cc #12674