TUV-x Examples and Tutorial by mattldawson · Pull Request #758 · NCAR/musica

mattldawson · 2026-01-29T16:04:12Z

This giant PR adds examples for using the TUV-x Python API. It also addresses a couple of bugs found along the way, adds some features to the TUV-x Python API, and adds tests of the v5.4 and TS1/TSMLT using the Python API vs. running with TUV-x standalone. The comparison tests are included in the "Docker" GitHub Action.

I reopened this after closing #757 to see if opening the PR on a branch of the main repo would fix the failing GH actions.

New Features

musica.tuvx.v54 and musica.tuvx.vTS1 modules for creating TUV-x calculators in Python for the 5.4 and TS1/TSMLT configurations
Two new Jupyter Notebook tutorials for TUV-x Python API (we can add more if needed)
Two example python scripts (one for v54 and one for vTS1)
Ability to directly set elements in grid, profile, and radiator arrays
Updated TUV-x JSON/YAML files that work with the Python API and Tutorials, and are tested against standalone TUV-x
Output of radiation field (actinic flux and spectral irradiance) from TUVX::run()

Bug Fixes

Fixed mismatched dimensions for output data from TUVX::run()
Fixed missing ability to set exo layer density via TUV-x Python API

Co-authored-by: Kyle Shores <[email protected]>

github-actions · 2026-01-29T16:04:23Z

📄 Documentation for this branch is available at: https://ncar.github.io/musica/branch/develop-603-tuvx-examples/

Copilot

Pull request overview

Adds end-to-end examples/tutorials for the TUV-x Python API and extends the TUV-x bindings to support richer outputs and more direct manipulation of grids/profiles/radiators; additionally introduces a Julia wrapper and related CI/docs updates.

Changes:

Add TUV-x v5.4 and TS1/TSMLT “standard configuration” + “conditions” tutorials and example scripts.
Extend TUV-x C/C++/Fortran/Python bindings to (a) expose actinic flux/spectral irradiance from TUVX::Run() and (b) enable direct element-wise mutation of backing arrays.
Add Python unit/integration tests (including standalone TUV-x comparisons via Docker) and introduce initial Julia bindings + CI/doc updates.

Reviewed changes

Copilot reviewed 88 out of 92 changed files in this pull request and generated 13 comments.

Show a summary per file

File	Description
tutorials/8.tuv-x_standard_configurations.ipynb	New notebook tutorial for standard TUV-x configs.
tutorials/9.tuv-x_conditions.ipynb	New notebook tutorial for modifying environmental conditions.
src/tuvx/tuvx_c_interface.cpp	Extend C interface `RunTuvx` to include radiation field outputs.
src/tuvx/tuvx.cpp	Track height/wavelength midpoints and pass new outputs through `Run()`.
src/tuvx/grid.cpp	Add “pointer getters” for grid edges/midpoints.
src/tuvx/profile.cpp	Add “pointer getters” for profile arrays.
src/tuvx/radiator.cpp	Add “pointer getters” for radiator optical property arrays.
src/tuvx/interface.F90	Extend `InternalRunTuvx` for radiation outputs + new dimension helpers.
src/tuvx/interface_grid.F90	Add C-bindings for grid pointer accessors.
src/tuvx/interface_profile.F90	Add C-bindings for profile pointer accessors.
src/tuvx/interface_radiator.F90	Add C-bindings for radiator pointer accessors.
src/test/unit/tuvx/tuvx_run_from_config.cpp	Update unit tests for new `RunTuvx` signature.
src/test/unit/tuvx/tuvx_c_api.cpp	Switch tests to fixed TS1/TSMLT configs.
src/test/unit/micm/parser.cpp	Adjust expectations for v0→v1 conversion output.
src/micm/convert_v0_to_v1.cpp	Add name stripping + optional unit conversion behavior.
python/bindings/tuvx/tuvx.cpp	Return actinic flux/spectral irradiance and reshape outputs.
python/bindings/tuvx/grid.cpp	Expose grid arrays as NumPy views (pointer-based).
python/bindings/tuvx/profile.cpp	Expose profile arrays as NumPy views (pointer-based).
python/bindings/tuvx/radiator.cpp	Expose radiator arrays as NumPy views (pointer-based).
python/bindings/mechanism_configuration/mechanism_configuration.cpp	Add `convert_reaction_units` flag to v0 conversion binding.
python/musica/tuvx/tuvx.py	Add radiation outputs to `run()` and config-relative path handling.
python/musica/tuvx/v54.py	New v5.4 convenience module (grids/profiles/radiators + config path).
python/musica/tuvx/vTS1.py	New TS1/TSMLT convenience module (grids/profiles/radiators + config path).
python/musica/tuvx/grid_map.py	Improve error message for key mismatch.
python/musica/tuvx/profile_map.py	Improve error message for key mismatch.
python/musica/tuvx/radiator_map.py	Improve error message and make iterators resilient to backend errors.
python/musica/tuvx/profile.py	Add `exo_layer_density` handling/validation.
python/musica/tuvx/radiator.py	Fix docstring shape descriptions for radiator arrays.
python/musica/tuvx/init.py	Export v54 module from package init.
python/musica/mechanism_configuration/utils.py	Adjust serialization filtering rules for empty values.
python/musica/main.py	Disable double unit conversion on CLI conversion path.
python/musica/examples/tuvx_5_4.py	Add runnable v5.4 TUV-x Python example.
python/musica/examples/tuvx_TS1_TSMLT.py	Add runnable TS1/TSMLT TUV-x Python example.
python/musica/examples/ts1_box_model.py	Add TS1 box-model example script.
python/musica/examples/examples.py	Register the new TS1 box-model example.
python/test/unit/tuvx/test_grid.py	Add element-wise mutation tests for grid views.
python/test/unit/tuvx/test_profile.py	Add element-wise mutation tests for profile views.
python/test/unit/tuvx/test_radiator.py	Add element-wise mutation tests for radiator views.
python/test/unit/tuvx/test_v54.py	Add v5.4 module tests (grids/profiles interpolation/repro).
python/test/integration/test_tuvx.py	Update integration tests for new shapes + config behavior.
python/test/integration/test_tuvx_v54.py	New standalone-vs-Python comparison tests for v5.4.
python/test/integration/test_tuvx_vts1.py	New standalone-vs-Python comparison tests for TS1/TSMLT.
include/musica/tuvx/tuvx.hpp	Update TUVX API for new outputs/dimension helpers.
include/musica/tuvx/tuvx_c_interface.hpp	Update C interface signature/docs for new outputs/dimensions.
include/musica/tuvx/grid.hpp	Add pointer getter declarations and C-API helpers.
include/musica/tuvx/profile.hpp	Add pointer getter declarations and C-API helpers.
include/musica/tuvx/radiator.hpp	Add pointer getter declarations and C-API helpers.
include/musica/micm/parse.hpp	Extend v0→v1 conversion API to accept a unit-conversion flag.
fortran/tuvx/tuvx.F90	Update Fortran bindings for new `RunTuvx` signature.
fortran/test/integration/test_tuvx_api.F90	Update Fortran tests to use fixed TS1/TSMLT config.
configs/tuvx/tuv_5_4.json	Remove inline aerosols block from config.
configs/tuvx/full_from_host/config_python.json	Add Python-specific host-config test fixture.
configs/tuvx/from_host/config.json	Update cross-section parameter file path.
configs/tuvx/fixed/config.json	Update cross-section parameter file path.
configs/tuvx/data/profiles/solar/extraterrestrial_flux.ts1.dat	Add TS1 extraterrestrial flux profile data.
docker/Dockerfile.python	Install `[test,tutorial]` extras in Python Docker image.
docker/Dockerfile.tuvx	Add Docker image to compare standalone TUV-x vs Python API.
pyproject.toml	Update test extras, add tutorial extra to wheel tests, adjust markers/commands.
package.json	Simplify formatting scripts; include configs/tuvx in npm package.
cmake/dependencies.cmake	Add Julia dependency setup via CxxWrap.
CMakeLists.txt	Add Julia build option and subdirectory.
julia/src/Musica.jl	New Julia package module wrapper using CxxWrap.
julia/bindings/musica_julia.cpp	New CxxWrap binding exposing `get_version`.
julia/test/runtests.jl	New minimal Julia test suite.
julia/Project.toml	New Julia package metadata/deps.
julia/CMakeLists.txt	Build/install rules for Julia wrapper library.
julia/README.md	New Julia wrapper documentation.
julia/.gitignore	Ignore Julia artifacts and built wrapper library.
docs/source/api/index.rst	Link Julia API docs from API index.
docs/source/api/julia.rst	Add Julia API documentation page.
README.md	Add Julia CI badge.
.gitignore	Ignore Julia artifacts at repo root.
.github/workflows/python-wheels.yml	Ensure PR builds test the PR head SHA.
.github/workflows/windows.yml	Disable TUV-x in Windows Fortran job.
.github/workflows/julia.yml	New Julia CI workflow.
.github/workflows/format.yml	Add Julia formatting and broaden clang-format targets.
.github/workflows/docker.yml	Add Docker-based standalone-vs-Python TUV-x comparison job.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

python/bindings/tuvx/radiator.cpp

include/musica/tuvx/tuvx.hpp

src/micm/convert_v0_to_v1.cpp

python/bindings/tuvx/tuvx.cpp

python/bindings/tuvx/grid.cpp

python/test/integration/test_tuvx_vts1.py

CMakeLists.txt

src/tuvx/interface_grid.F90

python/bindings/tuvx/tuvx.cpp

codecov-commenter · 2026-01-29T16:47:06Z

Codecov Report

❌ Patch coverage is 35.37118% with 444 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.51%. Comparing base (26e58d6) to head (0abc710).
⚠️ Report is 6 commits behind head on main.

Files with missing lines	Patch %	Lines
python/musica/examples/ts1_box_model.py	0.00%	90 Missing ⚠️
src/tuvx/radiator.cpp	0.00%	45 Missing ⚠️
src/tuvx/profile.cpp	0.00%	42 Missing ⚠️
python/musica/examples/tuvx_5_4.py	0.00%	41 Missing ⚠️
python/musica/examples/tuvx_TS1_TSMLT.py	0.00%	41 Missing ⚠️
src/tuvx/grid.cpp	0.00%	30 Missing ⚠️
python/musica/tuvx/v54.py	83.53%	27 Missing ⚠️
src/tuvx/interface_radiator.F90	0.00%	27 Missing ⚠️
python/musica/tuvx/vTS1.py	40.47%	25 Missing ⚠️
src/tuvx/interface.F90	36.11%	23 Missing ⚠️
... and 8 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #758      +/-   ##
==========================================
- Coverage   74.63%   71.51%   -3.12%     
==========================================
  Files         109      115       +6     
  Lines        8106     8739     +633     
==========================================
+ Hits         6050     6250     +200     
- Misses       2056     2489     +433

Flag	Coverage Δ
cpp_fortran	`65.87% <17.76%> (-2.63%)`	⬇️
javascript	`92.56% <ø> (ø)`
python	`73.31% <44.94%> (-5.17%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot · 2026-01-29T21:48:40Z

@K20shores I've opened a new pull request, #759, to work on those changes. Once the pull request is ready, I'll request review from you.

* Initial plan * Fix memory safety issue in TUV-x Python bindings Allocate output arrays on heap and use py::capsule to manage lifetime. This prevents dangling pointers when local vectors are destroyed. Co-authored-by: K20shores <[email protected]> * Improve exception safety using std::unique_ptr Use RAII-based memory management with std::unique_ptr instead of raw pointers to ensure automatic cleanup if exceptions occur during allocation or array construction. Co-authored-by: K20shores <[email protected]> * Fix double-delete issue in ownership transfer Store data pointers before releasing unique_ptrs to prevent double-delete if exceptions occur during numpy array construction. Co-authored-by: K20shores <[email protected]> * Ensure atomic ownership transfer from capsule to array Create each numpy array immediately after its capsule to prevent use-after-free if exceptions occur during array construction. Co-authored-by: K20shores <[email protected]> --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: K20shores <[email protected]>

Copilot · 2026-01-30T14:01:56Z

@K20shores I've opened a new pull request, #760, to work on those changes. Once the pull request is ready, I'll request review from you.

* Initial plan * Align Run() docstring array dimensions with C interface and Python bindings Co-authored-by: K20shores <[email protected]> * Use consistent dimension labels with underscores in Run() docstring Co-authored-by: K20shores <[email protected]> --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: K20shores <[email protected]>

…usica into develop-603-tuvx-examples

mattldawson and others added 23 commits January 23, 2026 14:32

add v5.4 wavelength grid

8dea150

add v5.4 height grid

fe267b7

add profile read function

a110081

add profile data for v5.4

4673f2a

add remaining profiles

b500ee1

add v5.4 aerosol radiator

cdd5224

add draft tuv-x 5.4 example

3377113

add test of tuv 5.4 against standalone tuv-x

22f99d3

added ts1 config test and example

1321795

add TUV-x yaml configs

e2db74a

tighten tolerances on tuv-x standalone tests

e7e946c

allow in-place profile updates

fbd46cb

allow in-place grid element updates

33de6eb

allow in-place radiator array updates

0e2b663

start tuv-x jupyter notebook

ffa074c

add second TUV-x tutorial

c385089

typos

15a4c2d

add tuv-x dockerfile action

e0f5164

address copilot comments and action failures

40b36d1

try to address fortran and python action failures

78fcf66

attempting to pull the fork head instead of main

3dce1e8

Update tutorials/8.tuv-x_standard_configurations.ipynb

ad65fc1

Co-authored-by: Kyle Shores <[email protected]>

merge main into develop-603-tuvx-examples

5210722

mattldawson requested review from K20shores, boulderdaze and Copilot January 29, 2026 16:04

Copilot started reviewing on behalf of mattldawson January 29, 2026 16:05 View session

Copilot AI reviewed Jan 29, 2026

View reviewed changes

packaging configs

dc30a4c

K20shores approved these changes Jan 29, 2026

View reviewed changes

ignoring duplicate named nc files

73f6b65

boulderdaze approved these changes Jan 29, 2026

View reviewed changes

Copilot AI mentioned this pull request Jan 29, 2026

Fix memory safety in TUV-x Python bindings for array returns #759

Merged

Copilot AI mentioned this pull request Jan 30, 2026

[WIP] Update TUV-x examples and tutorial based on feedback #760

Merged

5 tasks

Copilot AI and others added 4 commits January 30, 2026 08:05

responding to pr comments

751fa39

Merge branch 'develop-603-tuvx-examples' of https://github.com/NCAR/m…

a34006f

…usica into develop-603-tuvx-examples

adding ffmpeg

0abc710

K20shores merged commit 7cf369f into main Jan 30, 2026
48 of 50 checks passed

K20shores deleted the develop-603-tuvx-examples branch January 30, 2026 18:03

This was referenced Jan 30, 2026

Box model examples for Python TUV-x API #603

Closed

Allow updates to conditions in TUV-x through the Python API #596

Closed

TUV TS1 Box model #764

Closed

Package configurations with the musica python package #703

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TUV-x Examples and Tutorial#758

TUV-x Examples and Tutorial#758
K20shores merged 30 commits intomainfrom
develop-603-tuvx-examples

mattldawson commented Jan 29, 2026

Uh oh!

github-actions bot commented Jan 29, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented Jan 29, 2026 •

edited

Loading

Uh oh!

Copilot AI commented Jan 29, 2026

Uh oh!

Copilot AI commented Jan 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

mattldawson commented Jan 29, 2026

Uh oh!

github-actions bot commented Jan 29, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI commented Jan 29, 2026

Uh oh!

Copilot AI commented Jan 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

codecov-commenter commented Jan 29, 2026 •

edited

Loading