Skip to content

Some improvements and testing for the EB covered state#704

Merged
baperry2 merged 14 commits intoAMReX-Combustion:developmentfrom
baperry2:eb-state
Oct 24, 2023
Merged

Some improvements and testing for the EB covered state#704
baperry2 merged 14 commits intoAMReX-Combustion:developmentfrom
baperry2:eb-state

Conversation

@baperry2
Copy link
Copy Markdown
Contributor

This closes #703, with the goal of ensuring that covered cells never affect fluid cell values:

  • A change in behavior for all simulations - if a NaN is detected in the state after a timestep, the simulation now stops
  • For all EB stencils, if a value in the stencil is zero, that element of the stencil is ignored rather than multiplying it by the value in the cell and adding it into the gradient being computed. This should not effect the solution, but is done so that if the value is NaN in a covered cell, it does not propagate into the domain.
  • A new function is added to abort if the stencil for gradients at the boundary tries to access covered cells (can happen for the quadratic option on coarse grids ebd.boundary_grad_stencil_type = 0)
  • the function zero_in_body has been deleted because it was not used anywhere
  • A new option to set the body state to 0 for all variables for debugging (to ensure this does not impact fluid cells)
  • The eb-c3 test is modified to set the body state to 0. This is done to prevent any future changes that would cause data from covered cells to be accessed during the fluid cell updates. Had to make a few changes to Tests/CMakeLists.txt because this test can't run with AMReX FPE trapping.

Copy link
Copy Markdown
Contributor

@hsitaram hsitaram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good! thank you, Bruce.

@baperry2
Copy link
Copy Markdown
Contributor Author

The end of timestep NaN check works at least on AMD and Nvidia GPUs (Frontier and Eagle).

On GPU, the abort for invalid stencils only works in debug mode or with USE_ASSERTION=TRUE.

Did not notice a change in performance.

file(COPY ${TEST_FILES} DESTINATION "${CURRENT_TEST_BINARY_DIR}/")
# Set some default runtime options for all tests
set(RUNTIME_OPTIONS "amr.plot_file=plt amr.checkpoint_files_output=0 amr.plot_files_output=1 amrex.the_arena_is_managed=0")
set(RUNTIME_OPTIONS_AlL "amr.plot_file=plt amr.checkpoint_files_output=0 amr.plot_files_output=1 amrex.the_arena_is_managed=0")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a typo here.

set_tests_properties(${TEST_NAME} PROPERTIES WILL_FAIL TRUE)
endfunction(add_test_rf)

# Regression tests no FPE trapping ever
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test won't have these options amr.plot_file=plt amr.checkpoint_files_output=0 amr.plot_files_output=1 amrex.the_arena_is_managed=0, which I think we would want to enforce.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I'm confused. It just has the same options as the other tests?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could leave the RUNTIME_OPTIONS as is and just add amrex.signal_handling=0 to this test category.

@baperry2 baperry2 enabled auto-merge (squash) October 23, 2023 22:47
@baperry2 baperry2 merged commit acc0f7e into AMReX-Combustion:development Oct 24, 2023
@baperry2 baperry2 deleted the eb-state branch October 24, 2023 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ensure unnecessary operations are not carried out in covered regions

4 participants