Abstract

Providing derivatives of quantities in Cantera using automatic differentiation would enable the use of many sophisticated capabilities, such as sparse iterative solvers and machine learning algorithms. It would also eliminate the need for many tedious and error prone manual derivatives currently calculated within Cantera.

Motivation

Most of the computationally expensive routines in Cantera involve linear algebra. Integrating stiff chemical kinetics requires factorization of Jacobian matrices, which grow in size with the kinetic model. Jacobian matrix sparsity, which reduces the cost of factorization, can be obtained by evaluating the matrices through automatic differentiation, rather than the current finite difference evaluation. For the comparatively simple case of ideal gas kinetics, sparse Jacobians can be derived analytically with some effort, for example Cantera/cantera#951. However, I don't think this scales well beyond that, for example to real gases.

Many calculations in Cantera inherently involve calculations of derivatives, and manual implementations of these calculations could be avoided if they could be provided by automatic differentiation instead. For example, while c_p for an ideal gas is calculated trivially by differentiating the NASA polynomials, this calculation is much more complex in non-ideal models, e.g. RedlichKwongMFTP::cp_mole which in turn invokes the pressureDerivatives, dpdVCalc, and da_dt methods. And this is all for a comparatively simple equation of state. For a real treat, see the 4000+ line implementation of the HMWSoln class, most of which is spent calculating various derivatives.

Most machine learning algorithms require calculations of gradients. If Cantera were able to provide this information without introducing new methods for calculating every possible derivative, it would greatly enhance Cantera's ability to be used in such algorithms.

Possible Solutions

There are several libraries for doing automatic differentiation of C++ codes. I think it is probably worth experimenting with a couple of them to see which ones can be used in the least complex/invasive manner in Cantera. Some possibilities:

• Adept 2 - seems to be mature, well documented, and actively maintained
• autodiff - A relatively new header-only library that uses C++17 features
• Enzyme - Automatic differentiation based on the LLVM intermediate representation with minimal or no changes to the existing library code. Bleeding edge.

I have no experience with any of these packages, so suggestions of packages you have used and like are welcome.

One complexity I forsee is figuring out how AD would work if there's any desire to cross language barriers, such as user-defined models in other languages as envisioned in #79, or with ML algorithms. I don't know whether it is sufficient to have numerical values for derivatives at the language interface, or whether a more complex data type needs to be transmitted (if that's even possible).

References

• Adding adaptive preconditioner functionality to Cantera (draft) cantera#951 - Adding adaptive preconditioner functionality to Cantera
• Analytical Jacobian for Cantera #19 - A manual approach to calculating Jacobian elements analytically

P.S. This post is updated to reflect some of the comments below

P.P.S.: I labeled this as a "feature request" because while it is on my to-do list, I have several other projects that I'm prioritizing over this, so this could get done much sooner if someone else wants to tackle it.