feat!: Fix lower_funcs with custom extensions failing to load#2925
feat!: Fix lower_funcs with custom extensions failing to load#2925
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2925 +/- ##
==========================================
+ Coverage 83.78% 83.82% +0.03%
==========================================
Files 267 267
Lines 52975 52996 +21
Branches 46914 46935 +21
==========================================
+ Hits 44384 44422 +38
+ Misses 6320 6302 -18
- Partials 2271 2272 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| .unwrap_or(Ok(vec![]))?; | ||
| ExtensionRegistry::new(extra_extensions.into_iter().map(std::sync::Arc::new)) | ||
| let weak_registry: WeakExtensionRegistry = (&self.registry).into(); | ||
| ExtensionRegistry::new_with_extension_resolution(extra_extensions, &weak_registry) |
There was a problem hiding this comment.
drive-by: We weren't running resolution on extensions loaded from the ModelText format.
| #[serde_as(as = "Box<AsBinaryEnvelope>")] | ||
| hugr: Box<Hugr>, | ||
| #[serde(rename = "hugr")] | ||
| pkg: Box<Package>, |
There was a problem hiding this comment.
This is the fix. The encoded envelope already has all the required non-std extensions.
Decoding it as a Hugr didn't keep a strong reference to the loaded extensions, so they were being dropped.
There was a problem hiding this comment.
👍 nice for it to be so neat, and +1 for having commented that it's a single-module package.
48d24c2 to
e5de069
Compare
|
This PR contains breaking changes to the public Rust API. cargo-semver-checks summary |
e5de069 to
17260e1
Compare
acl-cqc
left a comment
There was a problem hiding this comment.
Thanks Agustin, looks good. Not sure if the suggestion of deserializing the whole variant as a FixedHugrDeserializer works.
hugr-core/src/extension/op_def.rs
Outdated
| /// We store it as a single-module package here to keep any encoded | ||
| /// extensions required to define the Hugr alive. | ||
| /// | ||
| /// The Package must contain any non-std extension required to define |
There was a problem hiding this comment.
super-nit: must -> should. If it doesn't, things might still work....(probably a lucky accident!)
| @@ -295,12 +296,20 @@ pub enum LowerFunc { | |||
| FixedHugr { | |||
| /// The extensions required by the [`Hugr`] | |||
There was a problem hiding this comment.
I think it might be good to expand this slightly now we have another thing storing the extensions. These are the runtime requirements (i.e. which the spec says may exclude types that are treated only opaquely).
We removed these (/the equivalents) from FunctionType but didn't do so here - perhaps an oversight but also they're still here to allow selecting which lower-func to use.
I had to remind myself that an ExtensionSet is a collection of extension names not definitions (that would be an ExtensionRegistry), this is easy to forget now we aren't using ExtensionSet much...
There was a problem hiding this comment.
I would avoid modifying this enum much, given that we're still using serde for extension definitions, and any change here would break the schema.
We're both thinking about dropping LowerFunc and moving extension defs to capnp, so I wouldn't try and break / change this if it's not needed.
| pub extensions: ExtensionSet, | ||
| #[serde_as(as = "Box<AsBinaryEnvelope>")] | ||
| pub hugr: Box<Hugr>, | ||
| pub hugr: Box<Package>, |
There was a problem hiding this comment.
So....anything that previously deserialized as a Hugr will now deserialize as a Package (with no extensions, if it was only a Hugr before)?
There was a problem hiding this comment.
The bytes are already an encoded envelope, we were always deserializing a package and then extracting the single module.
Now we do that step when doing the actual lowering instead, so we keep the package context in-memory.
| #[serde_as(as = "Box<AsBinaryEnvelope>")] | ||
| hugr: Box<Hugr>, | ||
| #[serde(rename = "hugr")] | ||
| pkg: Box<Package>, |
There was a problem hiding this comment.
👍 nice for it to be so neat, and +1 for having commented that it's a single-module package.
|
|
||
| for node in lower_hugr.nodes() { | ||
| let op = lower_hugr.get_optype(node); | ||
| assert!( |
There was a problem hiding this comment.
Optionally, you could check that you do find an ExtensionOp whose extension-arc is inner_ext
| /// when loading the Hugr. | ||
| /// | ||
| /// [ExtensionOp]: crate::ops::ExtensionOp | ||
| #[serde_as(as = "Box<AsBinaryEnvelope>")] |
There was a problem hiding this comment.
Do we still use this? I mean, do we still deserialize them individually, as well as deserializing lists of them via deserialize_lower_funcs?
If so, you might wanna serialize this whole variant as a FixedHugrDeserializer?
There was a problem hiding this comment.
deserialize_lower_funcs is a deserializing helper to improve errors messages, we still use derive(serde::Serialize) on the main enum for serializing.
## 🤖 New release
* `hugr-model`: 0.25.7 -> 0.26.0 (✓ API compatible changes)
* `hugr-core`: 0.25.7 -> 0.26.0 (⚠ API breaking changes)
* `hugr-llvm`: 0.25.7 -> 0.26.0 (✓ API compatible changes)
* `hugr-persistent`: 0.4.7 -> 0.5.0 (✓ API compatible changes)
* `hugr`: 0.25.7 -> 0.26.0 (✓ API compatible changes)
* `hugr-passes`: 0.25.7 -> 0.26.0 (⚠ API breaking changes)
* `hugr-cli`: 0.25.7 -> 0.26.0 (✓ API compatible changes)
### ⚠ `hugr-core` breaking changes
```text
--- failure enum_struct_variant_field_added: pub enum struct variant field added ---
Description:
An enum's exhaustive struct variant has a new field, which has to be included when constructing or matching on this variant.
ref: https://doc.rust-lang.org/reference/attributes/type_system.html#the-non_exhaustive-attribute
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/enum_struct_variant_field_added.ron
Failed in:
field pkg of variant LowerFunc::FixedHugr in /tmp/.tmpGqHhMb/hugr/hugr-core/src/extension/op_def.rs:312
--- failure enum_struct_variant_field_missing: pub enum struct variant's field removed or renamed ---
Description:
A publicly-visible enum has a struct variant whose field is no longer available under its prior name. It may have been renamed or removed entirely.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/enum_struct_variant_field_missing.ron
Failed in:
field hugr of variant LowerFunc::FixedHugr, previously in file /tmp/.tmpqeiszk/hugr-core/src/extension/op_def.rs:303
--- failure enum_variant_missing: pub enum variant removed or renamed ---
Description:
A publicly-visible enum has at least one variant that is no longer available under its prior name. It may have been renamed or removed entirely.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/enum_variant_missing.ron
Failed in:
variant BuildError::HugrInsertionError, previously in file /tmp/.tmpqeiszk/hugr-core/src/builder.rs:186
variant EnvelopeFormat::ModelText, previously in file /tmp/.tmpqeiszk/hugr-core/src/envelope/header.rs:60
variant EnvelopeFormat::ModelTextWithExtensions, previously in file /tmp/.tmpqeiszk/hugr-core/src/envelope/header.rs:70
variant Value::Function, previously in file /tmp/.tmpqeiszk/hugr-core/src/ops/constant.rs:169
variant Value::Function, previously in file /tmp/.tmpqeiszk/hugr-core/src/ops/constant.rs:169
--- failure function_missing: pub fn removed or renamed ---
Description:
A publicly-visible function cannot be imported by its prior path. A `pub use` may have been removed, or the function itself may have been renamed or removed entirely.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/function_missing.ron
Failed in:
function hugr_core::envelope::read_described_envelope, previously in file /tmp/.tmpqeiszk/hugr-core/src/envelope.rs:98
--- failure inherent_method_missing: pub method removed or renamed ---
Description:
A publicly-visible method or associated fn is no longer available under its prior name. It may have been renamed or removed entirely.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/inherent_method_missing.ron
Failed in:
Value::function, previously in file /tmp/.tmpqeiszk/hugr-core/src/ops/constant.rs:380
Value::function, previously in file /tmp/.tmpqeiszk/hugr-core/src/ops/constant.rs:380
```
### ⚠ `hugr-passes` breaking changes
```text
--- failure derive_trait_impl_removed: built-in derived trait no longer implemented ---
Description:
A public type has stopped deriving one or more traits. This can break downstream code that depends on those types implementing those traits.
ref: https://doc.rust-lang.org/reference/attributes/derive.html#derive
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/derive_trait_impl_removed.ron
Failed in:
type RedundantOrderEdgesPass no longer derives Copy, in /tmp/.tmpGqHhMb/hugr/hugr-passes/src/redundant_order_edges.rs:26
--- failure enum_missing: pub enum removed or renamed ---
Description:
A publicly-visible enum cannot be imported by its prior path. A `pub use` may have been removed, or the enum itself may have been renamed or removed entirely.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/enum_missing.ron
Failed in:
enum hugr_passes::untuple::UntupleRecursive, previously in file /tmp/.tmpqeiszk/hugr-passes/src/untuple.rs:23
--- failure function_missing: pub fn removed or renamed ---
Description:
A publicly-visible function cannot be imported by its prior path. A `pub use` may have been removed, or the function itself may have been renamed or removed entirely.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/function_missing.ron
Failed in:
function hugr_passes::remove_dead_funcs, previously in file /tmp/.tmpqeiszk/hugr-passes/src/dead_funcs.rs:179
function hugr_passes::const_fold::constant_fold_pass, previously in file /tmp/.tmpqeiszk/hugr-passes/src/const_fold.rs:224
function hugr_passes::non_local::nonlocal_edges, previously in file /tmp/.tmpqeiszk/hugr-passes/src/non_local.rs:39
function hugr_passes::nonlocal_edges, previously in file /tmp/.tmpqeiszk/hugr-passes/src/non_local.rs:39
function hugr_passes::monomorphize, previously in file /tmp/.tmpqeiszk/hugr-passes/src/monomorphize.rs:36
--- failure inherent_method_missing: pub method removed or renamed ---
Description:
A publicly-visible method or associated fn is no longer available under its prior name. It may have been renamed or removed entirely.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/inherent_method_missing.ron
Failed in:
UntuplePass::new, previously in file /tmp/.tmpqeiszk/hugr-passes/src/untuple.rs:102
UntuplePass::set_parent, previously in file /tmp/.tmpqeiszk/hugr-passes/src/untuple.rs:114
UntuplePass::recursive, previously in file /tmp/.tmpqeiszk/hugr-passes/src/untuple.rs:136
UntuplePass::find_rewrites, previously in file /tmp/.tmpqeiszk/hugr-passes/src/untuple.rs:151
UntuplePass::new, previously in file /tmp/.tmpqeiszk/hugr-passes/src/untuple.rs:102
UntuplePass::set_parent, previously in file /tmp/.tmpqeiszk/hugr-passes/src/untuple.rs:114
UntuplePass::recursive, previously in file /tmp/.tmpqeiszk/hugr-passes/src/untuple.rs:136
UntuplePass::find_rewrites, previously in file /tmp/.tmpqeiszk/hugr-passes/src/untuple.rs:151
RedundantOrderEdgesPass::new, previously in file /tmp/.tmpqeiszk/hugr-passes/src/redundant_order_edges.rs:39
RedundantOrderEdgesPass::recursive, previously in file /tmp/.tmpqeiszk/hugr-passes/src/redundant_order_edges.rs:44
RemoveDeadFuncsPass::with_module_entry_points, previously in file /tmp/.tmpqeiszk/hugr-passes/src/dead_funcs.rs:75
NormalizeCFGPass::cfgs, previously in file /tmp/.tmpqeiszk/hugr-passes/src/normalize_cfgs.rs:121
ValueHandle::new_const_hugr, previously in file /tmp/.tmpqeiszk/hugr-passes/src/const_fold/value_handle.rs:95
--- failure trait_added_supertrait: non-sealed trait added new supertraits ---
Description:
A non-sealed trait added one or more supertraits, which breaks downstream implementations of the trait
ref: https://doc.rust-lang.org/cargo/reference/semver.html#generic-bounds-tighten
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/trait_added_supertrait.ron
Failed in:
trait hugr_passes::composable::ComposablePass gained WithScope in file /tmp/.tmpGqHhMb/hugr/hugr-passes/src/composable.rs:25
trait hugr_passes::ComposablePass gained WithScope in file /tmp/.tmpGqHhMb/hugr/hugr-passes/src/composable.rs:25
--- failure trait_method_missing: pub trait method removed or renamed ---
Description:
A trait method is no longer callable, and may have been renamed or removed entirely.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#major-any-change-to-trait-item-signatures
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/trait_method_missing.ron
Failed in:
method value_from_const_hugr of trait ConstLoader, previously in file /tmp/.tmpqeiszk/hugr-passes/src/dataflow.rs:73
method with_scope_internal of trait ComposablePass, previously in file /tmp/.tmpqeiszk/hugr-passes/src/composable.rs:45
method with_scope_internal of trait ComposablePass, previously in file /tmp/.tmpqeiszk/hugr-passes/src/composable.rs:45
--- failure type_allows_fewer_generic_type_params: type now allows fewer generic type parameters ---
Description:
A type now allows fewer generic type parameters than it used to. Uses of this type that supplied all previously-supported generic types will be broken.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#trait-new-parameter-no-default
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/type_allows_fewer_generic_type_params.ron
Failed in:
Struct NormalizeCFGPass allows 1 -> 0 generic types in /tmp/.tmpGqHhMb/hugr/hugr-passes/src/normalize_cfgs.rs:101
--- failure unit_struct_changed_kind: unit struct changed kind ---
Description:
A public unit struct has been changed to a normal (curly-braces) struct, which cannot be constructed using the same struct literal syntax.
ref: rust-lang/cargo#10871
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/unit_struct_changed_kind.ron
Failed in:
struct MonomorphizePass in /tmp/.tmpGqHhMb/hugr/hugr-passes/src/monomorphize.rs:178
struct InlineDFGsPass in /tmp/.tmpGqHhMb/hugr/hugr-passes/src/inline_dfgs.rs:17
struct LocalizeEdges in /tmp/.tmpGqHhMb/hugr/hugr-passes/src/non_local.rs:23
```
<details><summary><i><b>Changelog</b></i></summary><p>
## `hugr-model`
<blockquote>
##
[0.26.0](hugr-model-v0.25.7...hugr-model-v0.26.0)
- 2026-03-16
### Testing
- Replace model text snapshots with roundtrip tests
([#2933](#2933))
- Add missing width arg in model-call example
([#2945](#2945))
</blockquote>
## `hugr-core`
<blockquote>
##
[0.26.0](hugr-core-v0.25.7...hugr-core-v0.26.0)
- 2026-03-16
### Bug Fixes
- Keep declared used_extensions in envelope description
([#2932](#2932))
### Documentation
- Move `spec/schema` and `spec/std_extensions` to `resources/`
([#2897](#2897))
### Miscellaneous Tasks
- [**breaking**] Fix new clippy warning on rust 1.94
([#2912](#2912))
### New Features
- [**breaking**] Rename ModelText envelope format to SExpression
([#2927](#2927))
- [**breaking**] Fix lower_funcs with custom extensions failing to load
([#2925](#2925))
- [**breaking**] TypeRow: add impl From array of Type, remove From<Type>
([#2784](#2784))
- Deprecate HugrView::as_petgraph
([#2944](#2944))
### Refactor
- [**breaking**] Remove deprecated Value::Function
([#2928](#2928))
- [**breaking**] Remove deprecated definitions
([#2930](#2930))
### Testing
- Fix type_row intos in doctests
([#2941](#2941))
- Add missing width arg in model-call example
([#2945](#2945))
</blockquote>
## `hugr-llvm`
<blockquote>
##
[0.26.0](hugr-llvm-v0.25.7...hugr-llvm-v0.26.0)
- 2026-03-16
### New Features
- *(llvm)* [**breaking**] Upgrade to LLVM 21
([#2901](#2901))
- Include private HUGR functions in the local symbol table
([#2831](#2831))
- [**breaking**] TypeRow: add impl From array of Type, remove From<Type>
([#2784](#2784))
### Refactor
- [**breaking**] Remove deprecated Value::Function
([#2928](#2928))
- [**breaking**] Remove deprecated stack_array codegen
([#2929](#2929))
</blockquote>
## `hugr-persistent`
<blockquote>
##
[0.5.0](hugr-persistent-v0.4.7...hugr-persistent-v0.5.0)
- 2026-03-16
### New Features
- *(llvm)* [**breaking**] Upgrade to LLVM 21
([#2901](#2901))
- [**breaking**] TypeRow: add impl From array of Type, remove From<Type>
([#2784](#2784))
</blockquote>
## `hugr`
<blockquote>
##
[0.26.0](hugr-v0.25.7...hugr-v0.26.0)
- 2026-03-16
### Bug Fixes
- Keep declared used_extensions in envelope description
([#2932](#2932))
### Documentation
- Move `spec/schema` and `spec/std_extensions` to `resources/`
([#2897](#2897))
### Miscellaneous Tasks
- [**breaking**] Fix new clippy warning on rust 1.94
([#2912](#2912))
### New Features
- [**breaking**] Update remainder of passes to use PassScope, drop
default with_scope
([#2871](#2871))
- [**breaking**] Make `WithScope` a supertrait of `ComposablePass`
([#2921](#2921))
- [**breaking**] Rename ModelText envelope format to SExpression
([#2927](#2927))
- [**breaking**] Fix lower_funcs with custom extensions failing to load
([#2925](#2925))
- Deprecate HugrView::as_petgraph
([#2944](#2944))
- [**breaking**] `hugr-passes` is no longer reexported from
`hugr::algorithms`
([#2922](#2922))
- *(llvm)* [**breaking**] Upgrade to LLVM 21
([#2901](#2901))
- [**breaking**] TypeRow: add impl From array of Type, remove From<Type>
([#2784](#2784))
### Refactor
- [**breaking**] Remove deprecated pass configuration
([#2938](#2938))
- [**breaking**] Remove deprecated Value::Function
([#2928](#2928))
- [**breaking**] Remove deprecated definitions
([#2930](#2930))
### Testing
- Fix type_row intos in doctests
([#2941](#2941))
- Add missing width arg in model-call example
([#2945](#2945))
</blockquote>
## `hugr-passes`
<blockquote>
##
[0.26.0](hugr-passes-v0.25.7...hugr-passes-v0.26.0)
- 2026-03-16
### New Features
- [**breaking**] `hugr-passes` is no longer reexported from
`hugr::algorithms`
([#2922](#2922))
- *(llvm)* [**breaking**] Upgrade to LLVM 21
([#2901](#2901))
- [**breaking**] Update remainder of passes to use PassScope, drop
default with_scope
([#2871](#2871))
- [**breaking**] Make `WithScope` a supertrait of `ComposablePass`
([#2921](#2921))
- [**breaking**] TypeRow: add impl From array of Type, remove From<Type>
([#2784](#2784))
- Deprecate HugrView::as_petgraph
([#2944](#2944))
### Refactor
- [**breaking**] Remove deprecated Value::Function
([#2928](#2928))
- [**breaking**] Remove deprecated pass configuration
([#2938](#2938))
</blockquote>
## `hugr-cli`
<blockquote>
##
[0.26.0](hugr-cli-v0.25.7...hugr-cli-v0.26.0)
- 2026-03-16
### Documentation
- Move `spec/schema` and `spec/std_extensions` to `resources/`
([#2897](#2897))
### New Features
- [**breaking**] Rename ModelText envelope format to SExpression
([#2927](#2927))
- [**breaking**] TypeRow: add impl From array of Type, remove From<Type>
([#2784](#2784))
</blockquote>
</p></details>
---
This PR was generated with
[release-plz](https://github.com/release-plz/release-plz/).
---------
Co-authored-by: Agustín Borgna <[email protected]>
Co-authored-by: Alec Edgington <[email protected]>
Closes #2520
Having a fixed hugr in an opdef lowering function that contained non-std extension reference caused an error on load.
We may deprecate lowering functions at some point, but we should try to avoid erroring out on currently valid code.
BREAKING CHANGE:
LowerFunc::FixedHugr::hugrreplaced withLowerFunc::FixedHugr::pkg