Skip to content

Generate caller-shaped thermochemistry routines - #1919

Open
sbryngelson wants to merge 3 commits into
MFlowCode:masterfrom
sbryngelson:thermochem-caller-routines
Open

sbryngelson wants to merge 3 commits into
MFlowCode:masterfrom
sbryngelson:thermochem-caller-routines

Conversation

@sbryngelson

@sbryngelson sbryngelson commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

Summary

Follow-up to #1915. MFC's hot loops chain several generated calls per gas state, and each re-evaluates the same NASA7 fits or composition. The generator now also emits routines shaped like those call sites, in the spirit of #1915's fused creation/destruction routine:

routine returns replaces per state
get_mixture_caloric_state(T, Y, cp0_r, cp, cv, e) species cp/R, mixture cp, cv, internal energy get_species_specific_heats_r + ..._cp_mass + ..._cv_mass + get_mixture_energy_mass (4 NASA7 passes -> 2)
get_mixavg_transport_state(p, T, Y, MW, X, D, lambda) molecular weight, mole fractions, mixture-averaged diffusivities, conductivity get_mixture_molecular_weight + get_mole_fractions + get_species_mass_diffusivities_mixavg + get_mixture_thermal_conductivity_mixavg (composition computed once, not three times)
get_species_enthalpies_mass(T, h) species enthalpies [J/kg] get_species_enthalpies_rt + the caller's *gas_constant*T/molecular_weights loop

Callers:

  • HLL, HLLC, LF, CBC: one get_mixture_caloric_state call per state. With gamma_method = 2 that halves the NASA7 evaluations; with the default gamma_method = 1 it is the same work in fewer calls. Unused m_thermochem imports are dropped.
  • Mixture-averaged diffusion flux (transport_model = 1): one get_mixavg_transport_state and one get_species_enthalpies_mass per side, and the hand-written enthalpy scaling loop goes away.

Design constraints:

  • Call depth. CCE has faulted when a thermochem call sits one routine deeper (see the comment in m_riemann_state.fpp), so the calls stay in the loop bodies and the shared diffusivity/conductivity bodies are Mako %defs expanded into both the existing and the fused routine, not a new callee. No device routine is nested deeper than before, and the existing routines generate identically to master.
  • Arithmetic. The fused routines reuse the separate routines' operations.

Testing

  • New test_fused_routines (h2o2, gri30): the fused routines are bitwise equal to the separate calls (gfortran -O0) and agree with Cantera.
  • amdflang 23.2.1, OpenMP offload, MI250X (OMP_TARGET_OFFLOAD=MANDATORY): --only Chemistry - 16 passed. (One test first failed because a concurrent build in the same checkout replaced its syscheck binary; it passed on re-run.)
  • gfortran 12.2 CPU: --only Chemistry - 16 passed, 0 failed.
  • ./mfc.sh precheck passes.

Optimized builds do not keep bitwise equality: on the MI250X benchmark below, 13 of 17 fields differ from master at step 500, by at most 8e-11 relative (momentum; most fields 1e-15 to 1e-13), with an identical initial condition. That is inlining/FMA contraction differing between one fused routine and separate calls. All goldens pass at their existing tolerances.

Performance

MI250X (one GCD), amdflang 23.2.1, generic --gpu mp --no-mpi, examples/2D_premixed_flame_vortex (h2o2, HLLC, gamma_method = 1, mixture-averaged diffusion), 500 steps, interleaved:

master fef3c276 this PR
s/step 0.05237, 0.05310, 0.05234 0.05224, 0.05048, 0.05215
mean 0.05260 0.05162 (-1.9%)

One run drives most of the difference, so read this as neutral to slightly faster for this case. gamma_method = 2 cases should gain more, since they save half the NASA7 passes; I did not benchmark one.

Updates after the first CI run

  • Frontier CCE (OpenACC and OpenMP) faulted in the HLLC kernel: get_mixture_caloric_state passed the species cp array on to get_species_specific_heats_r from inside the kernel. It is now a leaf routine (NASA7 expressions and averaging expanded in place, no calls), so HLLC's call is shallower than master's get_mixture_energy_mass. The bitwise test still passes. (6c93dec3)
  • NVHPC 23.11 to 24.7 (OpenACC) crashed in the front end (fort1, signal 11) compiling m_compute_levelset, which this PR does not touch. I reproduced it locally with nvfortran 24.1 and bisected it to m_thermochem.mod alone: nearly every file reads it through m_global_parameters, and adding the three routines trips a content-dependent front-end bug (removing any one avoids it; gri30's far larger module does not trip it). The three routines now live in a companion module, m_thermochem_state, generated into the same file and imported only by their callers. m_thermochem is byte-identical to master's for h2o2 and gri30. (3a47e7ec)

Re-checked after both: test_thermochem 17 passed; nvfortran 24.1 --gpu acc --mpi Chemistry build (previously crashed) completes; gfortran CPU and amdflang OpenMP offload on MI210 --only Chemistry 16 passed each.

Follow-up

#1821's surface residual (m_ibm.fpp) calls get_mixture_molecular_weight, get_species_mass_diffusivities_mixavg and the surface rates separately and recomputes mole fractions. Once both land, it can use get_mixavg_transport_state too.

Prepared with Claude Code.

Contribution Policy

We do not accept pull requests generated primarily by AI without genuine understanding or real-world usage context.

All contributions are expected to demonstrate:

  • A clear understanding of the codebase
  • Alignment with product direction
  • Thoughtful reasoning behind changes
  • Evidence of real-world usage or hands-on experience with the problem

If these expectations are not met, we would prefer to implement the changes ourselves rather than spend time reviewing low-effort submissions.


Acknowledgement

  • I confirm this PR meets the above expectations and reflects my own understanding and real-world context.

PR template credit: junegunn

MFC's hot loops chain several generated calls per gas state, each
re-evaluating the same NASA7 fits or composition. The generator now also
emits routines shaped like those call sites:

- get_mixture_caloric_state: species cp/R and mixture cp, cv and internal
  energy from one pass over the NASA7 fits.
- get_mixavg_transport_state: molecular weight, mole fractions,
  mixture-averaged diffusivities and thermal conductivity, computing the
  composition once.
- get_species_enthalpies_mass: species enthalpies in J/kg.

They reuse the separate routines' arithmetic: a new test finds them
bitwise equal to the separate calls (gfortran -O0) and in agreement with
Cantera. Optimized builds may contract or inline them differently, so
results can differ at roundoff. The shared diffusivity and conductivity
bodies are expanded in place rather than called, so no device routine is
nested deeper than before, which CCE is sensitive to.

The HLL, HLLC and LF solvers and CBC now make one caloric call per state
instead of up to four, and the mixture-averaged diffusion flux one
transport call per side instead of five. The calls stay in the loop
bodies.

Done with Claude Code.
Copilot AI balanced review requested due to automatic review settings September 23, 2026 05:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@codecov

codecov Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 70.00000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.51%. Comparing base (fef3c27) to head (3a47e7e).

Files with missing lines Patch % Lines
src/simulation/m_riemann_solver_lf.fpp 0.00% 2 Missing ⚠️
src/simulation/m_cbc.fpp 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1919      +/-   ##
==========================================
+ Coverage   61.48%   61.51%   +0.02%     
==========================================
  Files          86       86              
  Lines       22485    22458      -27     
  Branches     3298     3303       +5     
==========================================
- Hits        13826    13816      -10     
+ Misses       6206     6189      -17     
  Partials     2453     2453              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Frontier CCE (OpenACC and OpenMP) faulted in the HLLC kernel with a GPU
memory access violation: get_mixture_caloric_state passed the species cp
array on to get_species_specific_heats_r from inside the kernel, the
nested-call pattern CCE is known to fault on. The routine now expands the
NASA7 cp/R and h/RT expressions and the mass averaging in place and calls
nothing, so HLLC's call is shallower than master's
get_mixture_energy_mass. The expressions are the separate routines', so
the fused-versus-separate bitwise test still passes.

Checked: test_thermochem (17 passed), amdflang 23.2.1 OpenMP offload on
MI210 --only Chemistry (16 passed), nvfortran 24.1 --gpu mp --mpi build.

Done with Claude Code.
NVHPC 23.11 through 24.7 (OpenACC) crashed in the front end (fort1,
signal 11) compiling m_compute_levelset, a file this PR does not touch.
Reproduced with nvfortran 24.1 and bisected: master's module files plus
this PR's m_thermochem.mod alone crash it, and removing any one of the
three new routines avoids it. m_global_parameters imports m_thermochem,
so nearly every file reads that module; the crash depends on its
contents, not simply its size (gri30's much larger module compiles).

The three routines now live in a companion module, m_thermochem_state,
generated into the same file and imported only by their callers.
m_thermochem is byte-identical to master's for h2o2 and gri30.

Checked: test_thermochem (17 passed); nvfortran 24.1 --gpu acc --mpi
Chemistry build (previously crashed); gfortran CPU and amdflang OpenMP
offload on MI210 --only Chemistry (16 passed each).

Done with Claude Code.
@github-actions

Copy link
Copy Markdown

Lines of Code

File Lines Diff
src/common/m_chemistry.fpp 411 -9
src/simulation/m_riemann_solver_hll.fpp 609 -7
src/simulation/m_riemann_solver_hllc.fpp 1313 -7
src/simulation/m_riemann_solver_lf.fpp 511 -6
src/simulation/m_cbc.fpp 1115 -3
Directory Lines Diff
common 10412 -9
simulation 27825 -23
total 46648 -32

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants