Skip to content

test: start adding integration tests for optimization on Guppy generated hugr programs#1384

Merged
CalMacCQ merged 8 commits intomainfrom
cm/guppy_opt_integration_tests
Dec 19, 2025
Merged

test: start adding integration tests for optimization on Guppy generated hugr programs#1384
CalMacCQ merged 8 commits intomainfrom
cm/guppy_opt_integration_tests

Conversation

@CalMacCQ
Copy link
Copy Markdown
Contributor

@CalMacCQ CalMacCQ commented Dec 15, 2025

I've started adding some basic tests that validate that tket optimizations transform guppy programs as expected. This is part of #1383

Note that some key use cases are not supported yet including

  1. Some sequences of parameterized gates are not simplified as expected. Hopefully once feat: Add a pass to remove redundant order edges hugr#2739 is released and used in tket this will be fixed.
  2. Programs with arrays don't always simplify as expected (see Adjacent CX gates are not cancelled if arrays are used tket2#1307)
  3. Guppy functions which call other functions will only be optimized at the top level currently.

Once these limitations are removed I think it would be good to add test cases for the above as well.

@CalMacCQ CalMacCQ requested a review from a team as a code owner December 15, 2025 18:30
@CalMacCQ CalMacCQ requested a review from qartik December 15, 2025 18:30
@CalMacCQ
Copy link
Copy Markdown
Contributor Author

Hmm these tests all pass the precommit check locally but there seems to be an issue with importing NormalizeGuppy in the C.I. run. Will take a look.

Comment thread tests/integration/tket_opt/test_opt.py
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Dec 15, 2025

🐰 Bencher Report

Branchcm/guppy_opt_integration_tests
TestbedLinux
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
microseconds (µs)
(Result Δ%)
Upper Boundary
microseconds (µs)
(Limit %)
tests/benchmarks/test_big_array.py::test_big_array_check📈 view plot
🚷 view threshold
604,453.26 µs
(-14.46%)Baseline: 706,608.31 µs
741,938.73 µs
(81.47%)
tests/benchmarks/test_big_array.py::test_big_array_compile📈 view plot
🚷 view threshold
1,363,311.74 µs
(-6.43%)Baseline: 1,456,954.54 µs
1,529,802.27 µs
(89.12%)
tests/benchmarks/test_big_array.py::test_big_array_executable📈 view plot
🚷 view threshold
6,423,986.39 µs
(-7.89%)Baseline: 6,973,983.67 µs
7,322,682.85 µs
(87.73%)
tests/benchmarks/test_ctrl_flow.py::test_many_ctrl_flow_check📈 view plot
🚷 view threshold
44,550.18 µs
(-41.12%)Baseline: 75,660.17 µs
79,443.17 µs
(56.08%)
tests/benchmarks/test_ctrl_flow.py::test_many_ctrl_flow_compile📈 view plot
🚷 view threshold
78,831.74 µs
(-6.19%)Baseline: 84,032.32 µs
88,233.93 µs
(89.34%)
tests/benchmarks/test_ctrl_flow.py::test_many_ctrl_flow_executable📈 view plot
🚷 view threshold
517,181.50 µs
(-7.57%)Baseline: 559,540.42 µs
587,517.44 µs
(88.03%)
tests/benchmarks/test_prelude.py::test_import_guppy📈 view plot
🚷 view threshold
37.12 µs
(-15.14%)Baseline: 43.74 µs
45.93 µs
(80.82%)
🐰 View full continuous benchmarking report in Bencher

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Dec 15, 2025

🐰 Bencher Report

Branchcm/guppy_opt_integration_tests
TestbedLinux
Click to view all benchmark results
Benchmarkhugr_bytesBenchmark Result
bytes x 1e3
(Result Δ%)
Upper Boundary
bytes x 1e3
(Limit %)
hugr_nodesBenchmark Result
nodes
(Result Δ%)
Upper Boundary
nodes
(Limit %)
tests/benchmarks/test_big_array.py::test_big_array_compile📈 view plot
🚷 view threshold
143.70 x 1e3
(0.00%)Baseline: 143.70 x 1e3
145.13 x 1e3
(99.01%)
📈 view plot
🚷 view threshold
6,590.00
(0.00%)Baseline: 6,590.00
6,655.90
(99.01%)
tests/benchmarks/test_ctrl_flow.py::test_many_ctrl_flow_compile📈 view plot
🚷 view threshold
21.67 x 1e3
(0.00%)Baseline: 21.67 x 1e3
21.89 x 1e3
(99.01%)
📈 view plot
🚷 view threshold
606.00
(0.00%)Baseline: 606.00
612.06
(99.01%)
🐰 View full continuous benchmarking report in Bencher

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Dec 15, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.55%. Comparing base (5ba4fe2) to head (265aedf).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1384   +/-   ##
=======================================
  Coverage   93.55%   93.55%           
=======================================
  Files         127      127           
  Lines       11389    11389           
=======================================
  Hits        10655    10655           
  Misses        734      734           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@CalMacCQ
Copy link
Copy Markdown
Contributor Author

CalMacCQ commented Dec 15, 2025

Hmm these tests all pass the precommit check locally but there seems to be an issue with importing NormalizeGuppy in the C.I. run. Will take a look.

fixed, I needed to ensure that a recent enough version of the tket package was used in testing. See 3d9e03d.

This version bump should probably be mentioned explicitly in the changelog.

Comment thread tests/integration/tket_opt/test_opt.py
@qartik qartik requested a review from Copilot December 16, 2025 13:42
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces integration tests to validate that TKET optimization passes correctly transform Guppy-generated HUGR programs. The tests cover basic optimizations like normalization, redundant gate cancellation, and Clifford simplification.

Key changes:

  • Added new test file with three integration tests for TKET optimization on Guppy programs
  • Updated test dependencies to include tket~=0.12.13

Reviewed changes

Copilot reviewed 2 out of 4 changed files in this pull request and generated no comments.

File Description
tests/integration/tket_opt/test_opt.py Implements three integration tests validating TKET optimization passes on Guppy-generated quantum circuits
pyproject.toml Adds tket package as a test dependency with version constraint

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@CalMacCQ CalMacCQ requested a review from ferbetanzo December 16, 2025 14:06
Copy link
Copy Markdown
Contributor

@ferbetanzo ferbetanzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! :D

@CalMacCQ CalMacCQ enabled auto-merge December 19, 2025 17:22
@CalMacCQ CalMacCQ added this pull request to the merge queue Dec 19, 2025
Merged via the queue into main with commit 89bb82a Dec 19, 2025
7 checks passed
@CalMacCQ CalMacCQ deleted the cm/guppy_opt_integration_tests branch December 19, 2025 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants