Add regression tests for C++ sample programs by speth · Pull Request #883 · Cantera/cantera

speth · 2020-06-27T20:39:41Z

Similar to the tests added in #874 for the Fortran sample program, this PR adds tests for the C++ samples so they won't break without us knowing.

Changes proposed in this pull request

Add regression tests to run all of the C++ "sample" programs and compare their output against nominal output files
Remove redundant regression tests which were essentially duplicates of the sample programs
Add ability to have data files in subdirectories of the Cantera data directory (see What should we do with all our input files? enhancements#22, Fix examples in samples folder #774)
Make some improvements to test file comparisons for the C++ regression tests
Fix issues with default composition of air.yaml and use of that file in the combustor.cpp sample
Deprecate the argument ndim to newTransportMgr which is currently ignored / unused
Fix erroneous deprecation warnings issued by including onedim.h (diagnosed thanks to running the bvp sample)

Checklist

There is a clear use-case for this code change
The commit message has a short title & references relevant issues
Build passes (scons build & scons test) and unit tests address code coverage
The pull request is ready for review

If applicable, fill in the issue number this pull request is fixing

Fixes #774

ischoegl · 2020-06-27T22:19:51Z

Fix erroneous deprecation warnings issued by including onedim.h (diagnosed thanks to running the bvp sample)

Missed that one ... thanks for fixing!

Add optional command line arguments to the sample so that it can produce stable output.

Changes to the blessed output file are a result of updates to atomic weights, and to fixes for how flow devices are updated (Cantera#886)

Remove the existing cxx_ex test which duplicates this sample. Differences in blessed output are a result of updates to atomic weights in dc96fb5.

We have plenty of tests of the equilibrium solver for ideal gas mixtures, and the reaction path diagram test doesn't make a very good C++ example.

This is the same method used for profile comparisons in the Python test suite. These comparisons are skipped (with a warning) if Numpy is not installed.

codecov · 2020-07-01T20:03:22Z

Codecov Report

Merging #883 into main will increase coverage by 0.51%.
The diff coverage is 87.62%.

@@            Coverage Diff             @@
##             main     #883      +/-   ##
==========================================
+ Coverage   70.43%   70.95%   +0.51%     
==========================================
  Files         375      376       +1     
  Lines       45688    45889     +201     
==========================================
+ Hits        32180    32560     +380     
+ Misses      13508    13329     -179

Impacted Files	Coverage Δ
include/cantera/oneD/Boundary1D.h	`53.44% <ø> (+3.44%)`	⬆️
include/cantera/transport/TransportFactory.h	`100.00% <ø> (ø)`
src/base/application.h	`75.00% <ø> (ø)`
src/transport/TransportFactory.cpp	`98.21% <50.00%> (-1.79%)`	⬇️
src/base/application.cpp	`72.95% <82.75%> (-1.38%)`	⬇️
samples/cxx/flamespeed/flamespeed.cpp	`94.82% <84.61%> (ø)`
samples/cxx/gas_transport/gas_transport.cpp	`89.74% <89.74%> (ø)`
samples/cxx/LiC6_electrode/LiC6_electrode.cpp	`92.68% <100.00%> (ø)`
samples/cxx/bvp/BoundaryValueProblem.h	`92.64% <100.00%> (ø)`
samples/cxx/bvp/blasius.cpp	`90.74% <100.00%> (ø)`
... and 28 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e922c54...fe66e6d. Read the comment docs.

speth · 2020-07-01T21:34:53Z

Rebasing this after merging #886 allows the tests to run reliably on all platforms, so I think this is ready now.

bryanwweber

Looks good, just a few small things. This also means that we no longer need the "Build Samples" step in the CI, right? Since the samples will actually be run during scons test now?

samples/cxx/gas_transport/gas_transport.cpp

src/base/application.cpp

This provides the ability to organize data files included with Cantera and differentiate data which is useful for different purposes, e.g. sample data which is only meant to demonstrate capabilities and may not be physically meaningful. See Cantera/enhancements#22. Fixes Cantera#774

The samples are now built (and run) as part of `scons test`.

speth · 2020-07-03T15:57:37Z

This also means that we no longer need the "Build Samples" step in the CI, right? Since the samples will actually be run during scons test now?

Correct. Just pushed a commit that removes these build steps.

speth force-pushed the cxx-sample-tests branch 2 times, most recently from 5b28d3c to 2c52e64 Compare June 29, 2020 02:24

speth mentioned this pull request Jun 29, 2020

Fix reactor mass flow rate #886

Merged

4 tasks

ischoegl mentioned this pull request Jun 30, 2020

Improve error message for missing phases key #880

Merged

4 tasks

speth changed the base branch from master to main June 30, 2020 23:15

speth added 13 commits July 1, 2020 15:09

[Test] Add regression test using C++ flamespeed sample

73c9aa1

Add optional command line arguments to the sample so that it can produce stable output.

[Input] Set initial composition in air.yaml to match air.cti

773f669

[Test] Add regression test using C++ combustor example

0a0c8a1

Changes to the blessed output file are a result of updates to atomic weights, and to fixes for how flow devices are updated (Cantera#886)

[Samples] Fix use of 'air' object in combustor sample

e6ea996

[Test]: Add regression test of c++ 'kinetics1' sample program

b90dbec

Remove the existing cxx_ex test which duplicates this sample. Differences in blessed output are a result of updates to atomic weights in dc96fb5.

[Transport] Deprecate unused 'ndim' parameter of newTransportMgr

eb8842d

[Test] Allow generation of CSV files without a "title" row

1f4b80a

[Test] Make cxx_ex transport tests into a (tested) c++ sample program

dec2fe1

[Test] Remove remaining cxx_ex sub-tests

ba1b3e8

We have plenty of tests of the equilibrium solver for ideal gas mixtures, and the reaction path diagram test doesn't make a very good C++ example.

[SCons/Test] Enable comparisons of profile data in regression tests

e9b5d19

This is the same method used for profile comparisons in the Python test suite. These comparisons are skipped (with a warning) if Numpy is not installed.

[Test] Add regression test using C++ Blasius BVP sample

f97362f

[Test] Use rankine C++ sample for regression test

dcfeacf

[Samples] Remove non-functional NASA_coeffs C++ sample

453af47

speth force-pushed the cxx-sample-tests branch from cf96bb4 to 35e7e47 Compare July 1, 2020 19:11

speth requested a review from bryanwweber July 1, 2020 21:34

bryanwweber requested changes Jul 2, 2020

View reviewed changes

samples/cxx/gas_transport/gas_transport.cpp Show resolved Hide resolved

src/base/application.cpp Outdated Show resolved Hide resolved

speth added 5 commits July 2, 2020 20:13

[Test] Add regression test of C++ LiC6_electrode sample

1dc2258

[Test] Add regression test based on C++ openmp_ignition sample

9f297fa

Fix erroneous deprecation warnings caused by including onedim.h

d0fc123

[Test] Improve formatting of values in regression test error messages

359f6ab

speth force-pushed the cxx-sample-tests branch from 35e7e47 to 359f6ab Compare July 3, 2020 00:13

[CI] Remove separate step to build samples

fe66e6d

The samples are now built (and run) as part of `scons test`.

bryanwweber mentioned this pull request Jul 3, 2020

[CI] Run python examples from scons #892

Closed

4 tasks

bryanwweber approved these changes Jul 3, 2020

View reviewed changes

bryanwweber merged commit 047e74b into Cantera:main Jul 3, 2020

speth deleted the cxx-sample-tests branch July 3, 2020 17:58

ischoegl mentioned this pull request Jul 11, 2020

Run Python examples during CI to catch breakages Cantera/enhancements#60

Closed

ischoegl mentioned this pull request Aug 28, 2020

Remove obsolete data files #923

Merged

4 tasks

speth mentioned this pull request Mar 22, 2022

Fix invalid unchecked cast to StFlow #1226

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add regression tests for C++ sample programs#883

Add regression tests for C++ sample programs#883
bryanwweber merged 19 commits intoCantera:mainfrom
speth:cxx-sample-tests

speth commented Jun 27, 2020 •

edited

Loading

Uh oh!

ischoegl commented Jun 27, 2020

Uh oh!

codecov bot commented Jul 1, 2020 •

edited

Loading

Uh oh!

speth commented Jul 1, 2020

Uh oh!

bryanwweber left a comment

Uh oh!

Uh oh!

Uh oh!

speth commented Jul 3, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

speth commented Jun 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ischoegl commented Jun 27, 2020

Uh oh!

codecov bot commented Jul 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

speth commented Jul 1, 2020

Uh oh!

bryanwweber left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

speth commented Jul 3, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

speth commented Jun 27, 2020 •

edited

Loading

codecov bot commented Jul 1, 2020 •

edited

Loading