feat(persistent): More efficient HugrView iterators for PersistentHugr by lmondada · Pull Request #2595 · Quantinuum/hugr

lmondada · 2025-09-30T10:46:09Z

This PR improves persistent-hugr performance. It does three things

Introduces PersistentHugrCache to cache the list of children of each commit.
Improves the implementation of HugrView::children by adding a custom iterator, instead of materialising the entire hugr in memory.
small refactor: move fn node_status and enum NodeStatus from hugr-persistent/src/wire.rs to hugr-persistent/src/persistent_hugr.rs

Note:

Beyond the first two children, the order of

persistent_hugr.children(node)

and

let hugr = persistent_hugr.to_hugr();
hugr.children(node);

may no longer match when node is a Dataflow parent. This is annoying but I could not find an easy way to fix this. AFAIK this should not lead to observable differences according to current specs, but might be an issue at some point. I've created issue #2618 to track this.

codecov · 2025-09-30T10:51:12Z

Codecov Report

❌ Patch coverage is 95.37037% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.39%. Comparing base (3e12b65) to head (b9f183a).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
hugr-persistent/src/trait_impls/utils.rs	93.33%	2 Missing ⚠️
hugr-persistent/src/persistent_hugr.rs	95.65%	0 Missing and 1 partial ⚠️
hugr-persistent/src/trait_impls.rs	97.72%	0 Missing and 1 partial ⚠️
hugr-persistent/src/wire.rs	0.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2595      +/-   ##
==========================================
+ Coverage   83.36%   83.39%   +0.02%     
==========================================
  Files         257      259       +2     
  Lines       50664    50741      +77     
  Branches    46187    46264      +77     
==========================================
+ Hits        42238    42313      +75     
- Misses       6059     6061       +2     
  Partials     2367     2367

Flag	Coverage Δ
python	`91.53% <ø> (ø)`
rust	`82.60% <95.37%> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

aborgna-q

Nice!

It's a bit annoying that we need to force evaluation of the whole children when popping stuff from the back, but there's no way around it.

aborgna-q · 2025-10-13T09:33:46Z

hugr-persistent/src/persistent_hugr/cache.rs

+    children_cache: HashMap<CommitId, Vec<CommitId>>,
+}
+
+impl PersistentHugrCache {


Add some docs!

yup will do!

aborgna-q · 2025-10-13T09:52:41Z

hugr-persistent/src/trait_impls.rs

+            Either::Left(IterValidNodes::new(self, children.fuse()))
+        } else {
+            // children are precisely children of the commit hugr
+            Either::Right(children)
+        };
+        DoubleEndedIteratorAdapter::from(it)


We only need to wrap the IterValidNodes case, Either::Right(children) is already a doubleEndedIterator

true, fixing that!

lmondada · 2025-10-13T12:53:56Z

It's a bit annoying that we need to force evaluation of the whole children when popping stuff from the back, but there's no way around it.

True, but tbh it's barely (ever?) used backwards.

## 🤖 New release * `hugr-model`: 0.24.0 -> 0.24.1 (✓ API compatible changes) * `hugr-core`: 0.24.0 -> 0.24.1 (✓ API compatible changes) * `hugr-llvm`: 0.24.0 -> 0.24.1 (✓ API compatible changes) * `hugr-passes`: 0.24.0 -> 0.24.1 (✓ API compatible changes) * `hugr-persistent`: 0.3.1 -> 0.3.2 (✓ API compatible changes) * `hugr`: 0.24.0 -> 0.24.1 (✓ API compatible changes) * `hugr-cli`: 0.24.0 -> 0.24.1 (✓ API compatible changes) <details><summary>Changelog</summary> ## `hugr-model` <blockquote> ## [0.24.1](hugr-model-v0.24.0...hugr-model-v0.24.1) - 2025-11-03 ### Bug Fixes - Correct conversion of `table::Term::Tuple` to `ast::Term` ([#2653](#2653)) </blockquote> ## `hugr-core` <blockquote> ## [0.24.1](hugr-core-v0.24.0...hugr-core-v0.24.1) - 2025-11-03 ### Bug Fixes - validation outside entrypoint, normalize_cfgs w/ nonlocal edges ([#2633](#2633)) - SiblingSubgraph::try_from_nodes for non-entrypoint region ([#2655](#2655)) - Correct conversion of `table::Term::Tuple` to `ast::Term` ([#2653](#2653)) ### New Features - track package descriptions when loading ([#2639](#2639)) - *(cli)* describe sub-command ([#2650](#2650)) </blockquote> ## `hugr-llvm` <blockquote> ## [0.24.0](hugr-llvm-v0.23.0...hugr-llvm-v0.24.0) - 2025-10-13 ### New Features - LLVM lowering for borrow arrays using bitmasks ([#2574](#2574)) - *(py, core, llvm)* add `is_borrowed` op for BorrowArray ([#2610](#2610)) ### Refactor - [**breaking**] consistent inout order in borrow array ([#2621](#2621)) </blockquote> ## `hugr-passes` <blockquote> ## [0.24.1](hugr-passes-v0.24.0...hugr-passes-v0.24.1) - 2025-11-03 ### Bug Fixes - validation outside entrypoint, normalize_cfgs w/ nonlocal edges ([#2633](#2633)) </blockquote> ## `hugr-persistent` <blockquote> ## [0.3.2](hugr-persistent-v0.3.1...hugr-persistent-v0.3.2) - 2025-11-03 ### New Features - *(persistent)* More efficient HugrView iterators for PersistentHugr ([#2595](#2595)) </blockquote> ## `hugr` <blockquote> ## [0.24.1](hugr-v0.24.0...hugr-v0.24.1) - 2025-11-03 ### Bug Fixes - validation outside entrypoint, normalize_cfgs w/ nonlocal edges ([#2633](#2633)) - SiblingSubgraph::try_from_nodes for non-entrypoint region ([#2655](#2655)) - Correct conversion of `table::Term::Tuple` to `ast::Term` ([#2653](#2653)) ### New Features - track package descriptions when loading ([#2639](#2639)) - *(cli)* describe sub-command ([#2650](#2650)) </blockquote> ## `hugr-cli` <blockquote> ## [0.24.1](hugr-cli-v0.24.0...hugr-cli-v0.24.1) - 2025-11-03 ### New Features - track package descriptions when loading ([#2639](#2639)) - *(cli)* describe sub-command ([#2650](#2650)) </blockquote> </details> --- This PR was generated with [release-plz](https://github.com/release-plz/release-plz/).

## 🤖 New release * `hugr-model`: 0.24.1 -> 0.24.2 * `hugr-core`: 0.24.1 -> 0.24.2 * `hugr-llvm`: 0.24.1 -> 0.24.2 * `hugr-passes`: 0.24.1 -> 0.24.2 (✓ API compatible changes) * `hugr`: 0.24.1 -> 0.24.2 (✓ API compatible changes) * `hugr-cli`: 0.24.1 -> 0.24.2 (✓ API compatible changes) * `hugr-persistent`: 0.3.2 -> 0.3.3 <details><summary>Changelog</summary> ## `hugr-model` <blockquote> ## [0.24.1](hugr-model-v0.24.0...hugr-model-v0.24.1) - 2025-11-03 ### Bug Fixes - Correct conversion of `table::Term::Tuple` to `ast::Term` ([#2653](#2653)) </blockquote> ## `hugr-core` <blockquote> ## [0.24.1](hugr-core-v0.24.0...hugr-core-v0.24.1) - 2025-11-03 ### Bug Fixes - validation outside entrypoint, normalize_cfgs w/ nonlocal edges ([#2633](#2633)) - SiblingSubgraph::try_from_nodes for non-entrypoint region ([#2655](#2655)) - Correct conversion of `table::Term::Tuple` to `ast::Term` ([#2653](#2653)) ### New Features - track package descriptions when loading ([#2639](#2639)) - *(cli)* describe sub-command ([#2650](#2650)) </blockquote> ## `hugr-llvm` <blockquote> ## [0.24.0](hugr-llvm-v0.23.0...hugr-llvm-v0.24.0) - 2025-10-13 ### New Features - LLVM lowering for borrow arrays using bitmasks ([#2574](#2574)) - *(py, core, llvm)* add `is_borrowed` op for BorrowArray ([#2610](#2610)) ### Refactor - [**breaking**] consistent inout order in borrow array ([#2621](#2621)) </blockquote> ## `hugr-passes` <blockquote> ## [0.24.2](hugr-passes-v0.24.1...hugr-passes-v0.24.2) - 2025-11-03 ### Bug Fixes - ReplaceTypes: operate on whole Hugr, with set_regions ([#2662](#2662)) </blockquote> ## `hugr` <blockquote> ## [0.24.2](hugr-v0.24.1...hugr-v0.24.2) - 2025-11-03 ### Bug Fixes - ReplaceTypes: operate on whole Hugr, with set_regions ([#2662](#2662)) </blockquote> ## `hugr-cli` <blockquote> ## [0.24.1](hugr-cli-v0.24.0...hugr-cli-v0.24.1) - 2025-11-03 ### New Features - track package descriptions when loading ([#2639](#2639)) - *(cli)* describe sub-command ([#2650](#2650)) </blockquote> ## `hugr-persistent` <blockquote> ## [0.3.2](hugr-persistent-v0.3.1...hugr-persistent-v0.3.2) - 2025-11-03 ### New Features - *(persistent)* More efficient HugrView iterators for PersistentHugr ([#2595](#2595)) </blockquote> </details> --- This PR was generated with [release-plz](https://github.com/release-plz/release-plz/).

## 🤖 New release * `hugr-model`: 0.24.2 -> 0.24.3 (✓ API compatible changes) * `hugr-core`: 0.24.2 -> 0.24.3 * `hugr-llvm`: 0.24.2 -> 0.24.3 (✓ API compatible changes) * `hugr-passes`: 0.24.2 -> 0.24.3 (✓ API compatible changes) * `hugr`: 0.24.2 -> 0.24.3 (✓ API compatible changes) * `hugr-cli`: 0.24.2 -> 0.24.3 (✓ API compatible changes) * `hugr-persistent`: 0.3.3 -> 0.3.4 <details><summary>Changelog</summary> ## `hugr-model` <blockquote> ## [0.24.1](hugr-model-v0.24.0...hugr-model-v0.24.1) - 2025-11-03 ### Bug Fixes - Correct conversion of `table::Term::Tuple` to `ast::Term` ([#2653](#2653)) </blockquote> ## `hugr-core` <blockquote> ## [0.24.1](hugr-core-v0.24.0...hugr-core-v0.24.1) - 2025-11-03 ### Bug Fixes - validation outside entrypoint, normalize_cfgs w/ nonlocal edges ([#2633](#2633)) - SiblingSubgraph::try_from_nodes for non-entrypoint region ([#2655](#2655)) - Correct conversion of `table::Term::Tuple` to `ast::Term` ([#2653](#2653)) ### New Features - track package descriptions when loading ([#2639](#2639)) - *(cli)* describe sub-command ([#2650](#2650)) </blockquote> ## `hugr-llvm` <blockquote> ## [0.24.3](hugr-llvm-v0.24.2...hugr-llvm-v0.24.3) - 2025-11-06 ### Bug Fixes - BorrowArray discard handler allows elements to be borrowed ([#2666](#2666)) </blockquote> ## `hugr-passes` <blockquote> ## [0.24.3](hugr-passes-v0.24.2...hugr-passes-v0.24.3) - 2025-11-06 ### Bug Fixes - BorrowArray discard handler allows elements to be borrowed ([#2666](#2666)) </blockquote> ## `hugr` <blockquote> ## [0.24.3](hugr-v0.24.2...hugr-v0.24.3) - 2025-11-06 ### Bug Fixes - BorrowArray discard handler allows elements to be borrowed ([#2666](#2666)) </blockquote> ## `hugr-cli` <blockquote> ## [0.24.1](hugr-cli-v0.24.0...hugr-cli-v0.24.1) - 2025-11-03 ### New Features - track package descriptions when loading ([#2639](#2639)) - *(cli)* describe sub-command ([#2650](#2650)) </blockquote> ## `hugr-persistent` <blockquote> ## [0.3.2](hugr-persistent-v0.3.1...hugr-persistent-v0.3.2) - 2025-11-03 ### New Features - *(persistent)* More efficient HugrView iterators for PersistentHugr ([#2595](#2595)) </blockquote> </details> --- This PR was generated with [release-plz](https://github.com/release-plz/release-plz/).

lmondada force-pushed the lm/phugr-caching branch from d02aa6c to b9c152c Compare September 30, 2025 13:27

lmondada force-pushed the lm/phugr-caching branch from b9c152c to d9e4c1e Compare October 7, 2025 23:00

Base automatically changed from lm/phugr-relrc5 to main October 12, 2025 01:41

lmondada added 2 commits October 12, 2025 09:58

Make PersistentHugr::children more efficient

08491b8

feat(persistent): More efficient HugrView iterators for PersistentHugr

65d257c

lmondada force-pushed the lm/phugr-caching branch 2 times, most recently from cb8268a to 357d42a Compare October 12, 2025 02:22

Implement DoubleEndedIterator for fn children

b3f3dde

lmondada force-pushed the lm/phugr-caching branch from 357d42a to b3f3dde Compare October 12, 2025 02:26

lmondada mentioned this pull request Oct 12, 2025

PersistentHugr's HugrView::children does not guarantee ordering #2618

Open

Sort of "fix" test

bd9f343

lmondada marked this pull request as ready for review October 12, 2025 02:56

lmondada requested a review from a team as a code owner October 12, 2025 02:56

lmondada requested a review from aborgna-q October 12, 2025 02:56

aborgna-q approved these changes Oct 13, 2025

View reviewed changes

lmondada and others added 2 commits October 13, 2025 20:54

Merge branch 'main' into lm/phugr-caching

1d81a02

address comments

b9f183a

lmondada enabled auto-merge October 13, 2025 12:59

lmondada added this pull request to the merge queue Oct 13, 2025

Merged via the queue into main with commit 1d6c998 Oct 13, 2025
26 checks passed

lmondada deleted the lm/phugr-caching branch October 13, 2025 13:12

hugrbot mentioned this pull request Oct 14, 2025

chore: release #2625

Merged

hugrbot mentioned this pull request Nov 3, 2025

chore: release #2661

Merged

This was referenced Nov 5, 2025

chore: release #2667

Closed

chore: release #2669

Merged

hugrbot mentioned this pull request Nov 10, 2025

chore: release #2674

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(persistent): More efficient HugrView iterators for PersistentHugr#2595

feat(persistent): More efficient HugrView iterators for PersistentHugr#2595
lmondada merged 6 commits intomainfrom
lm/phugr-caching

lmondada commented Sep 30, 2025 •

edited

Loading

Uh oh!

codecov bot commented Sep 30, 2025 •

edited

Loading

Uh oh!

aborgna-q left a comment

Uh oh!

aborgna-q Oct 13, 2025

Uh oh!

lmondada Oct 13, 2025

Uh oh!

aborgna-q Oct 13, 2025

Uh oh!

lmondada Oct 13, 2025

Uh oh!

lmondada commented Oct 13, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

lmondada commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Note:

Uh oh!

codecov bot commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

aborgna-q left a comment

Choose a reason for hiding this comment

Uh oh!

aborgna-q Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

lmondada Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

aborgna-q Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

lmondada Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

lmondada commented Oct 13, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lmondada commented Sep 30, 2025 •

edited

Loading

codecov bot commented Sep 30, 2025 •

edited

Loading