feat(hugr-py): Allow linking packages and modules from Python by maximilianruesch · Pull Request #2947 · Quantinuum/hugr

maximilianruesch · 2026-03-16T14:45:18Z

Adds capabilities to hugr-py to allow linking modules and entire packages together. This is a required feature for Guppy libraries, found in Quantinuum/guppylang#1481, and could be considered the core feature for the same.

hugr-core needed a slight modification to allow reading a module from an envelope (with all the builtin validation around expecting a single module) while also returning the extensions present in the envelope. The functions are exposed via pyo3 bindings to Python and tested there.

codecov · 2026-03-16T14:49:30Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.85%. Comparing base (93aac28) to head (705bc20).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2947      +/-   ##
==========================================
+ Coverage   83.84%   83.85%   +0.01%     
==========================================
  Files         267      267              
  Lines       52923    52953      +30     
  Branches    46854    46869      +15     
==========================================
+ Hits        44371    44402      +31     
+ Misses       6283     6282       -1     
  Partials     2269     2269

Flag	Coverage Δ
python	`88.80% <100.00%> (+0.04%)`	⬆️
rust	`83.20% <100.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

maximilianruesch · 2026-03-16T15:29:34Z

Note: I am unsure where to add tests for hugr-core, and which tests to add specifically. Currently, the function is covered by hugr-py tests implicitly.

Also, one could consider whether we should refactor load to call load_with_exts and just ignore the exsts.

croyzor · 2026-03-17T11:54:58Z

hugr-py/rust/linking.rs

+            .store_with_exts(&mut result, EnvelopeConfig::binary(), &exts_into)
+            .unwrap();
+
+        hugr_core::package::Package::load(&result[..], None)


This seems like something we don't want in production, I'd say make it a debug assert, but not sure how that works with maturin

Oop yeah, that was something that was used for debugging. I will have a look at debug asserts.

Seems to work as you would expect in regular rust. I pushed it to a debug statement.

croyzor · 2026-03-17T11:58:05Z

hugr-py/rust/linking.rs

+    }
+
+    #[pyfunction]
+    fn link_modules(module_into: &[u8], module_from: &[u8]) -> PyResult<Vec<u8>> {


I notice that we're going to have all three hugrs in memory, which seems like it could be a problem.
I don't really have strong enough rust/pyO3 foo to say the solution, but possibly:

take as arguments impl Reader so the files are read here and their contents can be dropped

Take Package as argument so we can consume them when they're linked in and release the memory

I could instead take an owned Vec<u8> if that helps... See https://pyo3.rs/v0.28.2/conversions/tables.html for allowed types / types that have an implementation of the required traits out of the box.

Yes, that'd be good

I modified the param. It seems that the reader trait to read into a hugr is not implemented for a vec though, so I still use a slice. It might also not make much of a difference since Python has no concept of ownership, so I presume the contents are copied anyway. This might actually be more expensive now 😅

I benchmarked it, and using the QAOA example using Vec<u8 I had about 0.048s per link call (from Python), whereas using &[u8] gave me about 0.031s per link call (probably due to avoiding an unnecessary copy. I reverted the change.

Yeah that's fair 👍

croyzor

LGTM 👍

…export (#2964) Does: - Correct / add missing parent module names to the existing modules (as far as I know this is mainly for the specifier displayed when printing the module in e.g. a Python REPL) - Correct module definitions for exception classes (again, I think this only has display / introspection impact) - Add a missing reexport of an exception class in the linking module (and add a test for it from Python to ensure it exists). Related to #2947

## 🤖 New release * `hugr-model`: 0.26.0 -> 0.26.1 * `hugr-core`: 0.26.0 -> 0.26.1 (✓ API compatible changes) * `hugr-llvm`: 0.26.0 -> 0.26.1 * `hugr`: 0.26.0 -> 0.26.1 (✓ API compatible changes) * `hugr-passes`: 0.26.0 -> 0.26.1 (✓ API compatible changes) * `hugr-cli`: 0.26.0 -> 0.26.1 * `hugr-persistent`: 0.5.0 -> 0.5.1 <details><summary>Changelog</summary> ## `hugr-model` <blockquote> ## [0.26.0](hugr-model-v0.25.7...hugr-model-v0.26.0) - 2026-03-16 ### Testing - Replace model text snapshots with roundtrip tests ([#2933](#2933)) - Add missing width arg in model-call example ([#2945](#2945)) </blockquote> ## `hugr-core` <blockquote> ## [0.26.1](hugr-core-v0.26.0...hugr-core-v0.26.1) - 2026-03-18 ### New Features - *(hugr-py)* Allow linking packages and modules from Python ([#2947](#2947)) ### Refactor - start cleaning up SiblingSubgraph convexity checking ([#2956](#2956)) </blockquote> ## `hugr-llvm` <blockquote> ## [0.26.0](hugr-llvm-v0.25.7...hugr-llvm-v0.26.0) - 2026-03-16 ### New Features - *(llvm)* [**breaking**] Upgrade to LLVM 21 ([#2901](#2901)) - Include private HUGR functions in the local symbol table ([#2831](#2831)) - [**breaking**] TypeRow: add impl From array of Type, remove From<Type> ([#2784](#2784)) ### Refactor - [**breaking**] Remove deprecated Value::Function ([#2928](#2928)) - [**breaking**] Remove deprecated stack_array codegen ([#2929](#2929)) </blockquote> ## `hugr` <blockquote> ## [0.26.1](hugr-v0.26.0...hugr-v0.26.1) - 2026-03-18 ### Bug Fixes - ConstantFold fails when the module contains function declarations ([#2954](#2954)) ### New Features - *(hugr-py)* Allow linking packages and modules from Python ([#2947](#2947)) ### Refactor - start cleaning up SiblingSubgraph convexity checking ([#2956](#2956)) </blockquote> ## `hugr-passes` <blockquote> ## [0.26.1](hugr-passes-v0.26.0...hugr-passes-v0.26.1) - 2026-03-18 ### Bug Fixes - ConstantFold fails when the module contains function declarations ([#2954](#2954)) </blockquote> ## `hugr-cli` <blockquote> ## [0.26.0](hugr-cli-v0.25.7...hugr-cli-v0.26.0) - 2026-03-16 ### Documentation - Move `spec/schema` and `spec/std_extensions` to `resources/` ([#2897](#2897)) ### New Features - [**breaking**] Rename ModelText envelope format to SExpression ([#2927](#2927)) - [**breaking**] TypeRow: add impl From array of Type, remove From<Type> ([#2784](#2784)) </blockquote> ## `hugr-persistent` <blockquote> ## [0.5.0](hugr-persistent-v0.4.7...hugr-persistent-v0.5.0) - 2026-03-16 ### New Features - *(llvm)* [**breaking**] Upgrade to LLVM 21 ([#2901](#2901)) - [**breaking**] TypeRow: add impl From array of Type, remove From<Type> ([#2784](#2784)) </blockquote> </details> --- This PR was generated with [release-plz](https://github.com/release-plz/release-plz/).

🤖 I have created a release *beep* *boop* --- ## [0.16.0](hugr-py-v0.15.4...hugr-py-v0.16.0) (2026-04-01) This release changes the default text serialization format from JSON to `MODEL_WITH_EXTS` (the JSON format is now deprecated). It also adds the ability to link packages and modules from Python, renders operation type arguments in HUGR visualizations, introduces a new `debug_info` module for attaching source-level debug metadata to nodes, and introduces multi-version support in `ExtensionRegistry`, allowing different versions of the same extension to coexist in the same registry. ### ⚠ BREAKING CHANGES * `ExtensionRegistry` may now contain multiple versions of the same extension. `.extensions` is now an iterator of the latest versions instead of a dictionary. `.register_updated` and `.add_extension` have been replaced with a singular `.register`. * Default `EnvelopeConfig` changed from `EnvelopeFormat.JSON` to `EnvelopeFormat.MODEL_WITH_EXTS`. * Substituted `hugr-py/src/hugr/tys.py::_type_str` with `hugr-py/src/hugr/utils.py::name_w_args`. ### Features * Add debug info metadata specification in `hugr-py` ([#2971](#2971)) ([a4da8ef](a4da8ef)) * Allow multiple versions of an extension in an ExtensionRegistry ([#3005](#3005)) ([ee90cd1](ee90cd1)) * Deprecate JSON serialization format ([#2991](#2991)) ([0f7a880](0f7a880)) * Operation parameter rendering in HUGR visualizations ([#2995](#2995)) ([d619148](d619148)) * Allow linking packages and modules from Python ([#2947](#2947)) ([329c243](329c243)) * Make `_composable_pass` and `_scope` public modules ([#2962](#2962)) ([1d6af9a](1d6af9a)), closes [#2961](#2961) * Allow missing ext versions ExtensionDesc metadata ([#2979](#2979)) ([bc1c445](bc1c445)) * Allow dfg conversion in to TrackedDfg ([#2993](#2993)) ([3bd7a88](3bd7a88)) ### Bug Fixes * Allow both _composable_pass and composable_pass imports ([#2965](#2965)) ([0a5664d](0a5664d)) * Allow both _scope and scope imports ([#3003](#3003)) ([de0458c](de0458c)) * Always do extension resolution when loading packages and hugrs ([#2953](#2953)) ([0086d2a](0086d2a)) * Correct pyo3 module definitions and add missing exception class export ([#2964](#2964)) ([3df73f4](3df73f4)) * Include extensions during link serialization ([#2992](#2992)) ([9236e89](9236e89)) * Prefix names with underscores during serialization ([#2989](#2989)) ([8aab636](8aab636)) * Use valid identifiers when constructing AST model ([#2973](#2973)) ([8750a6b](8750a6b)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). --------- Co-authored-by: Agustín Borgna <[email protected]>

maximilianruesch added 9 commits March 9, 2026 13:55

hugr-py linking

95c77f9

Pitchit hard

da1482b

Better linking

4a8a4a9

Add proper errorrage

87cd79f

Formatting

c519d20

Add entrypoint support

136ce79

Better panic

58697fc

Begin linking tests

2a1d7bc

More linking tests and fixes

76ada56

maximilianruesch mentioned this pull request Mar 16, 2026

feat!: Allow joint compilation of Guppy symbols to a library Quantinuum/guppylang#1495

Open

maximilianruesch added 2 commits March 16, 2026 14:56

Empti test

15574aa

Merge branch 'main' into mr/feat/py-linking

8ca8496

maximilianruesch marked this pull request as ready for review March 16, 2026 15:28

maximilianruesch requested a review from a team as a code owner March 16, 2026 15:28

maximilianruesch requested a review from croyzor March 16, 2026 15:28

croyzor reviewed Mar 17, 2026

View reviewed changes

maximilianruesch added 4 commits March 17, 2026 12:08

Use owned param

cb6582e

Use debug assert

bf371ab

Add comment

3a2323c

Revert for better performance

1d200f3

maximilianruesch requested a review from croyzor March 17, 2026 14:38

maximilianruesch added 2 commits March 17, 2026 15:01

Refactor load to use load_with_exts

7b7e790

Add load_with_exts test

705bc20

croyzor approved these changes Mar 17, 2026

View reviewed changes

maximilianruesch added this pull request to the merge queue Mar 17, 2026

Merged via the queue into main with commit 329c243 Mar 17, 2026
30 checks passed

maximilianruesch deleted the mr/feat/py-linking branch March 17, 2026 15:51

This was referenced Mar 17, 2026

chore(main): release hugr-py 0.16.0 #2948

Merged

chore: release #2955

Merged

maximilianruesch mentioned this pull request Mar 18, 2026

fix: Correct pyo3 module definitions and add missing exception class export #2964

Merged

hugrbot mentioned this pull request Mar 23, 2026

chore: release #2975

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(hugr-py): Allow linking packages and modules from Python#2947

feat(hugr-py): Allow linking packages and modules from Python#2947
maximilianruesch merged 17 commits intomainfrom
mr/feat/py-linking

maximilianruesch commented Mar 16, 2026

Uh oh!

codecov bot commented Mar 16, 2026 •

edited

Loading

Uh oh!

maximilianruesch commented Mar 16, 2026

Uh oh!

croyzor Mar 17, 2026

Uh oh!

maximilianruesch Mar 17, 2026

Uh oh!

maximilianruesch Mar 17, 2026

Uh oh!

croyzor Mar 17, 2026

Uh oh!

maximilianruesch Mar 17, 2026

Uh oh!

croyzor Mar 17, 2026

Uh oh!

maximilianruesch Mar 17, 2026

Uh oh!

maximilianruesch Mar 17, 2026

Uh oh!

croyzor Mar 17, 2026

Uh oh!

croyzor left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

maximilianruesch commented Mar 16, 2026

Uh oh!

codecov bot commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

maximilianruesch commented Mar 16, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

croyzor left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented Mar 16, 2026 •

edited

Loading