Skip to content

Add regression input file for fragment + simple reactor and regression test for fragment + RMS reactor#2571

Merged
hwpang merged 7 commits intoReactionMechanismGenerator:mainfrom
hwpang:add_fragment_rms_regression
Nov 3, 2023
Merged

Add regression input file for fragment + simple reactor and regression test for fragment + RMS reactor#2571
hwpang merged 7 commits intoReactionMechanismGenerator:mainfrom
hwpang:add_fragment_rms_regression

Conversation

@hwpang
Copy link
Copy Markdown
Contributor

@hwpang hwpang commented Nov 2, 2023

Motivation or Problem

The current fragment regression test only tests that the mechanism can be generated, but not testing that the results actually stay consistent with previous runs. Additionally, currently we are not testing fragment is compatible with RMS reactor.

Description of Changes

I add a regression_input.py for the fragment + simple reactor regression test. I also add a new regression test for fragment + RMS reactor.

@JacksonBurns
Copy link
Copy Markdown
Contributor

@hwpang I added the fragment and rms fragment regression to the other line in the CI file makes it run the regression_input comparison

@hwpang
Copy link
Copy Markdown
Contributor Author

hwpang commented Nov 2, 2023

Thanks!

@codecov
Copy link
Copy Markdown

codecov bot commented Nov 2, 2023

Codecov Report

Merging #2571 (b324104) into main (3183e4b) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #2571   +/-   ##
=======================================
  Coverage   55.44%   55.44%           
=======================================
  Files         124      124           
  Lines       36818    36818           
=======================================
  Hits        20414    20414           
  Misses      16404    16404           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@JacksonBurns
Copy link
Copy Markdown
Contributor

@hwpang so the CI is failing for two reasons:

  1. The new RMS+fragment regression test has never been run on main, so there are no regression results to do the baseline comparison.
  2. The fragment regression test is raising an actual error about a missing file in the reference results, see https://github.com/ReactionMechanismGenerator/RMG-Py/actions/runs/6737504658/job/18315100139#step:17:482

Can you take a look at 2. and help me figure out what went wrong?

@hwpang
Copy link
Copy Markdown
Contributor Author

hwpang commented Nov 3, 2023

@hwpang so the CI is failing for two reasons:

  1. The new RMS+fragment regression test has never been run on main, so there are no regression results to do the baseline comparison.
  2. The fragment regression test is raising an actual error about a missing file in the reference results, see https://github.com/ReactionMechanismGenerator/RMG-Py/actions/runs/6737504658/job/18315100139#step:17:482

Can you take a look at 2. and help me figure out what went wrong?

Yeah! This is because the saveEdgeSpecies option in the input.py needs to be turned on. I pushed a commit to change that in both input files.

@JacksonBurns
Copy link
Copy Markdown
Contributor

Yeah! This is because the saveEdgeSpecies option in the input.py needs to be turned on. I pushed a commit to change that in both input files.

Thanks for the quick fix!

@JacksonBurns JacksonBurns self-requested a review November 3, 2023 14:45
Copy link
Copy Markdown
Contributor

@JacksonBurns JacksonBurns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM and will need to be merged with failing CI.

@hwpang
Copy link
Copy Markdown
Contributor Author

hwpang commented Nov 3, 2023

I got @JacksonBurns's offline approval to merge this in. The failure in CI for fragment + simple reactor is because the edge was not saved in the stable run due to not setting saveEdgeSpecies to True. The failure in CI for fragment + RMS reactor is because no stable version to compare to yet.

@hwpang hwpang merged commit e9eca53 into ReactionMechanismGenerator:main Nov 3, 2023
@hwpang hwpang deleted the add_fragment_rms_regression branch November 3, 2023 23:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants