Skip to content

Feature: Auxiliary scalars#467

Merged
baperry2 merged 71 commits intoAMReX-Combustion:developmentfrom
ITV-RWTH:tlh_passive
Apr 4, 2025
Merged

Feature: Auxiliary scalars#467
baperry2 merged 71 commits intoAMReX-Combustion:developmentfrom
ITV-RWTH:tlh_passive

Conversation

@ThomasHowarth
Copy link
Copy Markdown
Contributor

@ThomasHowarth ThomasHowarth commented Jan 17, 2025

This pull request may close #465

Auxiliary variables are added through the following flags in the input:

peleLM.aux_vars = a b c

peleLM.a.advect = 1 
peleLM.a.diff_coeff = 0

peleLM.b.advect = 0
peleLM.b.diff_coeff = 1e-2

peleLM.c.advect = 1
peleLM.c.diff_coeff = 1e-2

where the advect parameter determines whether or not the variable is transported, and the diff_coeff provides a diffusion coefficient.

The test case transports/diffuses a Gaussian profile through a periodic domain. For a 1m square domain, with 1m/s upward velocity, I see the following:

  • $$t=0$$:
    image
  • Scalar A (advected, not diffused):
    • $$t=0.5$$:
      image
    • $$t=1$$:
      image
  • Scalar B (not advected, diffused):
    • $$t=0.5$$
      image
    • $$t=1$$
      image
  • Scalar C (Advected and diffused):
    • $$t=0.5$$
      image
    • $$t=1$$
      image

Discussion points:

  1. User-defined function interfaces in the setup file (i.e. pelelmex_initdata and bcnormal) have been adjusted. I've done this to all cases, but when people pull this, their current cases may not compile. Not sure how to address this.
  2. The implementation is done via the SDC iterations and is therefore not done in the case of peleLM.incompressible = 1
  3. Currently the diffusion coefficient is specified as an input, rather than linked to a particular species/temperature. It should be relatively easy to link the coefficient to it since it is set in basically the same place, but I'm not sure what the choice should be.
  4. Currently there is no source term implementation (e.g. for an age equation), however, when it is done, I suppose this could be integrated with @dmontgomeryNREL 's ODEQty in some fashion.
  5. When filling the boundary conditions, the s_aux array passed to bcnormal needs to be hardcoded. I've set this to NVAR and put a check to make sure that m_nAux < NVAR, but perhaps there is a more elegant way to do this.

@ThomasHowarth ThomasHowarth marked this pull request as draft January 17, 2025 12:55
@ThomasHowarth ThomasHowarth marked this pull request as ready for review January 20, 2025 08:51
@baperry2
Copy link
Copy Markdown
Collaborator

Regarding your discussion points:

  1. I don't like making changes that break things for users, but sometimes that can't be avoided. We should take a bit of extra time to make sure any interface changes are what we want and there aren't additional related changes we can do at the same time so that we do this as rarely as possible.
  • I don't believe it's necessary to add nAux to pelelmex_initdata: instead the user could use aux.nComp() (an Array4 member function) to get this information as needed without modifying the interface.
  • We could potentially avoid adding s_aux to bcnormal by stacking the aux data at the end of s_ext (which is just a pointer to AMReX::Real), but that's ugly and unintuitive, so a breaking change for the interface might be the lesser of two evils here.
  • We have some other potential breaking changes in the pipeline to enable inflows on EBs (Developments shopping list #156). It may make sense to delay merging this until those are done to minimize the number of times we break things for users
  1. That's ok with me for now - essentially all real use cases for LMeX are not incompressible anyway.
  2. I think it makes sense to follow what we do when we have constant non-unity Le transport. Let users specify a Schmidt number (or Lewis/Prandtl) and define the scalar diffusion coefficients based on that Sc and the viscosity.
  3. Yes, after merging this we will revisit the ODEQty stuff and fold it into this framework.
  4. This is difficult because we need a copy of the Aux values so they don't get overwritten by calls to the bcnormal function that are applying BCs for other variables. We might not be able to avoid something akin to your current approach. But an improvement would be to create a new macro like PELELMEX_MAX_NAUX that can be set in the GNUmakefile and cmake commands rather than pinning the max to NVAR - allowing users to have more AUX variables for small mechanisms and not having so much unnecessary data allocated when NVAR is large. @marchdf - any ideas for a better solution when needing dynamically sized memory within a device function?

As always, I appreciate the new capability, which I'm sure will be quite useful to many.

@ThomasHowarth
Copy link
Copy Markdown
Contributor Author

ThomasHowarth commented Jan 24, 2025

I'll leave a note here of changes made/needed:

  • Remove nAux from initdata
  • Convert diff_coeff to Schmidt number
  • Figure out what to do with s_aux, both in terms of breaking bcnormal interface, and allocating its size.

Additionally, I don't know why the Lint-codespell test has started failing - it seems to not like a naming convention for a variable that I have not touched.

@baperry2
Copy link
Copy Markdown
Collaborator

A new version of codespell was released and it catches a few more things. The fix for that is in #469.

@baperry2
Copy link
Copy Markdown
Collaborator

For the remaining issue of what to do about s_aux, I think we should be able to bypass creating the temporary array just to copy data back into the Array4 in the BCFill function. Instead of passing a pointer to amrex::Real, we could pass an amrex::CellData that gives direct access into the Array4 (see Array4::cellData()). The Aux values could also be populated with a separate bcnormal_aux function that only gets called when fillpatching the Aux variables, which would avoid the need mess with the existing bcnormal interface. @esclapez is working on a single struct that wraps all the case-specific IC/BC functions, which would make adding that additional function cleaner.

@ThomasHowarth
Copy link
Copy Markdown
Contributor Author

The CellData approach seems to work well. I think it gives a cleaner interface to the boundaries than the existing setup. It has also made the m_nAux input to the bcnormal function redundant, but we can remove that when an interface-breaking change comes. I also renamed this array to aux_ext since I think that makes more sense (I suppose the s in s_ext stands for state).

@baperry2
Copy link
Copy Markdown
Collaborator

If I compile the 2D PassiveScalar case with USE_EB=TRUE and then run with the additional flag eb2.geom_type=all_regular, which simply runs without any EB and should give the same results, then I get NaNs for the passive quantities starting in the second plt file generated.

@bssoriano noticed some issues when corse/fine boundaries intersect with inflows. Perhaps this is related.

@ThomasHowarth ThomasHowarth marked this pull request as draft March 28, 2025 09:50
@ThomasHowarth
Copy link
Copy Markdown
Contributor Author

I've traced the issue to the auxiliary boundaries not being filled properly at initialisation, but I can't figure out where the problem is. If you compile in debug mode and turn on the FPE catches, it crashes when looking for some boundary data (regardless of with/without EB). I've probably missed a fillpatch somewhere, but I can't spot it—I'll keep working on it.

bssoriano and others added 6 commits March 29, 2025 18:21
Added separate flag to turn off diffusion
Le=1 is used when schmidt < 0. Future implementation could have the user specify the Le number for each transported variable
The code returning erroneous operation when AMR was at the inflow BC for max_level = 2, and had non-physical values when max_level = 1. The problem was related to regrid.
Fix auxiliary scalars with AMR
@ThomasHowarth ThomasHowarth marked this pull request as ready for review April 1, 2025 08:54
@ThomasHowarth
Copy link
Copy Markdown
Contributor Author

Thanks to some work from @bssoriano, we spotted the missing boundary data. I've also made some things more strongly typed (more const's and constexpr's) and gone through and used the m2c/c2m unit conversions in the kernel functions where possible.

@baperry2
Copy link
Copy Markdown
Collaborator

baperry2 commented Apr 3, 2025

Some final fixes that let this run with EB in debug mode: ITV-RWTH#5

A last thing is that it looks like we don't apply redistribution for the Aux variables with EB, but that can wait. I'd like to get this merged before #477.

@baperry2
Copy link
Copy Markdown
Collaborator

baperry2 commented Apr 4, 2025

Thanks again @ThomasHowarth for the new capability and @bssoriano for helping to find the fillpatching issues. I think this is in good enough shape to merge now, although some of the EB stuff might need to be revisited eventually.

@baperry2 baperry2 merged commit 8ae6a1f into AMReX-Combustion:development Apr 4, 2025
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Transporting passive scalars

5 participants