Experimental `clib` from doxygen by ischoegl · Pull Request #1835 · Cantera/cantera

ischoegl · 2024-12-29T00:36:40Z

Changes proposed in this pull request

This PR seeks to implement a basis for future auto-generated Cantera APIs that are fully documented based on information provided in C++ docstrings.

New sourcegen scaffolder generates an experimental clib version ("clib-experimental")
The clib scaffolder is based on doxygen tag files and associated XML anchorfiles
A proposed minimal YAML syntax is used to reference C++ functions and methods (for details, see file interfaces/sourcegen/sourcegen/_data/README.md)
A portion of clib googletests is adopted for illustration and code coverage as test-clib-experimental
Partial CI is added to the clang-compiler action
API information is obtained without having to parse pre-existing CLib headers (i.e. the original approach for the .NET API)

This proof of concept re-implements approaches of the traditional CLib library and is newr feature-complete, although the conversion of zeroD and oneD objects is currently omitted. ~~One other currently omitted feature involves the inclusion of code that is difficult to automate, although it is expected to be rare;~~ in many instances, tweaks to the C++ interface may be the go-to approach instead. Edit: custom code is now supported.

The configuration involves YAML markup that only uses minimal information for each entry, e.g.

- name: newSolution
  implements: newSolution(const string&, const string&, const string&)
  uses: [thermo, kinetics, transport]
- name: name

which produces fully documented declarations

/**
 *  Create and initialize a new Solution manager from an input file.
 *
 *  @param infile       name of the input file
 *  @param name         name of the phase in the file. If this is blank, the first phase in the file is used.
 *  @param transport    name of the transport model.
 *  @returns            Handle to stored Solution object or -1 for exception handling.
 *
 *  @implements constructor: shared_ptr<Solution> newSolution(const string&, const string&, const string&)
 *  @relates Solution::thermo(), Solution::kinetics(), Solution::transport()
 */
int sol3_newSolution(const char* infile, const char* name, const char* transport);

/**
 *  Return the name of this Solution object.
 *
 *  @param handle       Handle to queried Solution object.
 *  @param[in] bufLen   Length of reserved array.
 *  @param[out] buf     Returned string value.
 *  @returns            Actual length of string or -1 for exception handling.
 *
 *  @implements getter: string Solution::name()
 */
int sol3_name(int handle, int bufLen, char* buf);

and implementations (click to expand)

int sol3_newSolution(const char* infile, const char* name, const char* transport)
{
    // constructor: shared_ptr<Solution> newSolution(const string&, const string&, const string&)
    try {
        auto obj = newSolution(infile, name, transport);
        int id = SolutionCabinet::add(obj);
        // add all associated objects
        if (obj->thermo()) {
            ThermoPhaseCabinet::add(obj->thermo(), id);
        }
        if (obj->kinetics()) {
            KineticsCabinet::add(obj->kinetics(), id);
        }
        if (obj->transport()) {
            TransportCabinet::add(obj->transport(), id);
        }
        return id;
    } catch (...) {
        return handleAllExceptions(-2, ERR);
    }
}

int sol3_name(int handle, int bufLen, char* buf)
{
    // getter: string Solution::name()
    try {
        string out = SolutionCabinet::at(handle)->name();
        copyString(out, buf, bufLen);
        return int(out.size()) + 1;
    } catch (...) {
        return handleAllExceptions(-1, ERR);
    }
}

Here is an example for a full YAML input used for code generation

sol3_auto.yaml (click to expand)

# Auto-generated CLib API for %Cantera's Solution class.
# Partially implements a replacement for CLib's traditional @c ct library.

# This file is part of Cantera. See License.txt in the top-level directory or
# at https://cantera.org/license.txt for license and copyright information.

prefix: sol3
base: Solution
parents: []  # List of parent classes
derived: [Interface]  # List of specializations
recipes:
- name: newSolution
  implements: newSolution(const string&, const string&, const string&)
  uses: [thermo, kinetics, transport]
- name: newInterface
  implements:
    newInterface(const string&, const string&, const vector<shared_ptr<Solution>>&)
  uses: [thermo, kinetics]
- name: del
  uses: [thermo, kinetics, transport]
- name: name
- name: setName
- name: thermo
- name: kinetics
- name: transport
- name: setTransportModel
  code: |-
    try {
        auto obj = SolutionCabinet::at(handle);
        TransportCabinet::del(
            TransportCabinet::index(*(obj->transport()), handle));
        obj->setTransportModel(model);
        return TransportCabinet::add(obj->transport(), handle);
    } catch (...) {
        return handleAllExceptions(-2, ERR);
    }
- name: nAdjacent
- name: adjacent
  implements: Solution::adjacent(size_t)
  uses: [thermo, kinetics, transport]
  what: constructor  # registers object in CLib storage
# - name: adjacentName
- name: cabinetSize

If applicable, fill in the issue number this pull request is fixing

See Cantera/enhancements#39
Partially addresses Cantera/enhancements#220

If applicable, provide an example illustrating new features this pull request is introducing

Run the following command from the Cantera root folder:

scons doxygen
python3 interfaces/sourcegen/run.py --api=clib --output=.
scons build clib_experimental=y

A partial test suite based on the traditional CLib ensures that code performs as expected:

scons test-clib-experimental

Checklist

The pull request includes a clear description of this code change
Commit messages have short titles and reference relevant issues
Build passes (scons build & scons test) and unit tests address code coverage
Style & formatting of contributed code follows contributing guidelines
The pull request is ready for review

bryanwweber

Thanks Ingmar! I know this is still in draft but I wanted to leave a small comment that might be intrusive later

interfaces/sourcegen/sourcegen/_HeaderFileParser.py

interfaces/sourcegen/sourcegen/_TagFileParser.py

codecov · 2025-01-03T02:32:52Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.39%. Comparing base (10d7754) to head (83442a2).
Report is 41 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1835   +/-   ##
=======================================
  Coverage   74.39%   74.39%           
=======================================
  Files         382      382           
  Lines       53330    53330           
  Branches     9022     9021    -1     
=======================================
  Hits        39676    39676           
  Misses      10607    10607           
  Partials     3047     3047

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

ischoegl · 2025-01-03T05:08:54Z

@speth and @bryanwweber ... this is ready for a review.

~~As an aside, there is a minor caveat for the .NET workflow on Windows, where a Python 3.10 version apparently does not respect PEP 604. I'll address this together with other comments.~~ Edit: nevermind ... windows-2022 appears to use an older system Python and my initial attempt to override failed.

bryanwweber

Thanks for all this work Ingmar! Nearly all of my comments here are suggestions to improve the type hinting. In some cases, you'll need to add an import (I think just for Iterable). Obviously these don't affect the runtime, but my thought was that if we're going to have type hints they should at least be complete 😄

interfaces/sourcegen/sourcegen/_HeaderFileParser.py

interfaces/sourcegen/sourcegen/yaml/_YamlSourceGenerator.py

test/clib_experimental/test_clib3.cpp

ischoegl · 2025-01-04T04:43:56Z

Thanks for all this work Ingmar! Nearly all of my comments here are suggestions to improve the type hinting. In some cases, you'll need to add an import (I think just for Iterable). Obviously these don't affect the runtime, but my thought was that if we're going to have type hints they should at least be complete 😄

Thanks, @bryanwweber, for the prompt review! I agree that having done most of the type hints already, it makes sense to take it all the way. I adopted almost all of the suggestions directly and added some conversions, so the googletest suite is almost fully comparable to the traditional CLib version for the parts that I have converted..

Also fix issues caused by differences of doxygen and Cantera whitespace conventions in argument lists and add recipes for functions used in googletests.

ischoegl · 2025-01-22T00:48:10Z

[...] I'm looking forward to the opportunity to continue expanding this code generation beyond clib.

Yes, I believe this can be leveraged in the near future (especially for MATLAB, as the hand-coded part covers only a portion of the actual API). Fortran could be done in analogy to CLib (without depending on it).

One question I have is on the logistics of replacing the existing manual clib. Are you expecting that we will be able to complete the transition before the next Cantera release, such that we can avoid the need for the parallel set of functions and header files with the "3" suffix?

I had already created a tracking issue for this: Cantera/enhancements#220. For now, the "3"s are needed; a future removal is imho trivial. I can likely handle the transition, but am not planning on investing in code generation for other APIs.

Adding tests to clang-compiler workflow.

ischoegl · 2025-01-22T17:07:55Z

@speth ... please let me know in case there's anything else. At this point, I have taken care of all review comments.

ischoegl force-pushed the sourcegen-doxygen-clib branch from f37d89b to 6226766 Compare December 29, 2024 01:05

ischoegl added enhancement clib labels Dec 29, 2024

ischoegl force-pushed the sourcegen-doxygen-clib branch from 6226766 to 4f4d79f Compare December 29, 2024 22:03

bryanwweber reviewed Dec 30, 2024

View reviewed changes

interfaces/sourcegen/sourcegen/_HeaderFileParser.py Outdated Show resolved Hide resolved

interfaces/sourcegen/sourcegen/_TagFileParser.py Show resolved Hide resolved

interfaces/sourcegen/sourcegen/_TagFileParser.py Outdated Show resolved Hide resolved

ischoegl force-pushed the sourcegen-doxygen-clib branch 9 times, most recently from ee04eb8 to 22be9f9 Compare January 3, 2025 02:13

ischoegl force-pushed the sourcegen-doxygen-clib branch from 22be9f9 to a06ec48 Compare January 3, 2025 04:41

ischoegl marked this pull request as ready for review January 3, 2025 05:05

ischoegl requested a review from a team January 3, 2025 05:09

ischoegl force-pushed the sourcegen-doxygen-clib branch from a06ec48 to 97bfa50 Compare January 3, 2025 05:16

bryanwweber reviewed Jan 3, 2025

View reviewed changes

ischoegl force-pushed the sourcegen-doxygen-clib branch from 97bfa50 to 2350a3d Compare January 4, 2025 03:30

ischoegl force-pushed the sourcegen-doxygen-clib branch 4 times, most recently from 739aa5d to d48f3e8 Compare January 4, 2025 14:32

ischoegl mentioned this pull request Jan 4, 2025

Use code generation to provide interfaces for C, Fortran, Matlab, Julia, etc. Cantera/enhancements#39

Open

ischoegl requested a review from a team January 4, 2025 14:51

This comment was marked as outdated.

Sign in to view

ischoegl added 19 commits January 21, 2025 17:24

[sourcegen] Improve setter/getter code generation

21dd625

[sourcegen] Simplify recipes

b47894d

[sourcegen] Add remaining CLib scaffolders

e9035b4

[sourcegen] Simplify code and templates

dc61f32

[sourcegen] Add includes

621cc78

[sourcegen] Add service methods

3623ae1

[sourcegen] Add main CLib header

f5a5964

[sourcegen] Add verbosity option to sourcegen

6c8b8a8

[sourcegen] Improve YAML source generator

7e5449f

[sourcegen] Improve CLib source generator

0231750

[sourcegen] Implement reserved CLib methods

69be698

[sourcegen] Improve nomenclature and fix glitches

0195679

[sourcegen] Document/simplify YAML specifications

03c2e39

[sourcegen] Use xml.etree to parse XML

63338a5

[sourcegen] Add vector<shared_ptr<T>> handling

1f78e4a

Also fix issues caused by differences of doxygen and Cantera whitespace conventions in argument lists and add recipes for functions used in googletests.

[sourcegen] Specify docstring as field

dc9cfb7

[sourcegen] Handle custom code

ce235f8

[unittests] Add googletests for clib_experimental

5d76578

[sourcegen] Improve type hinting

2b87eb7

ischoegl dismissed bryanwweber’s stale review via 3dc73bd January 22, 2025 00:27

ischoegl force-pushed the sourcegen-doxygen-clib branch from 19ca9a4 to 3dc73bd Compare January 22, 2025 00:27

ischoegl requested a review from speth January 22, 2025 00:50

ischoegl added 2 commits January 21, 2025 20:31

[sourcegen] Clarify output file name

a22cd9c

[CI] Add clib_experimental tests to workflow

83442a2

Adding tests to clang-compiler workflow.

ischoegl force-pushed the sourcegen-doxygen-clib branch from 3dc73bd to 83442a2 Compare January 22, 2025 02:31

speth approved these changes Jan 23, 2025

View reviewed changes

speth merged commit c426d30 into Cantera:main Jan 23, 2025

ischoegl deleted the sourcegen-doxygen-clib branch January 23, 2025 13:29

Uh oh!

Conversation

ischoegl commented Dec 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bryanwweber left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Jan 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ischoegl commented Jan 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bryanwweber left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ischoegl commented Jan 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

ischoegl commented Jan 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ischoegl commented Jan 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ischoegl commented Dec 29, 2024 •

edited

Loading

codecov bot commented Jan 3, 2025 •

edited

Loading

ischoegl commented Jan 3, 2025 •

edited

Loading

ischoegl commented Jan 4, 2025 •

edited

Loading

ischoegl commented Jan 22, 2025 •

edited

Loading