Generate caller-shaped thermochemistry routines - #1919
Open
sbryngelson wants to merge 3 commits into
Open
sbryngelson wants to merge 3 commits into
sbryngelson wants to merge 3 commits into
Conversation
MFC's hot loops chain several generated calls per gas state, each re-evaluating the same NASA7 fits or composition. The generator now also emits routines shaped like those call sites: - get_mixture_caloric_state: species cp/R and mixture cp, cv and internal energy from one pass over the NASA7 fits. - get_mixavg_transport_state: molecular weight, mole fractions, mixture-averaged diffusivities and thermal conductivity, computing the composition once. - get_species_enthalpies_mass: species enthalpies in J/kg. They reuse the separate routines' arithmetic: a new test finds them bitwise equal to the separate calls (gfortran -O0) and in agreement with Cantera. Optimized builds may contract or inline them differently, so results can differ at roundoff. The shared diffusivity and conductivity bodies are expanded in place rather than called, so no device routine is nested deeper than before, which CCE is sensitive to. The HLL, HLLC and LF solvers and CBC now make one caloric call per state instead of up to four, and the mixture-averaged diffusion flux one transport call per side instead of five. The calls stay in the loop bodies. Done with Claude Code.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1919 +/- ##
==========================================
+ Coverage 61.48% 61.51% +0.02%
==========================================
Files 86 86
Lines 22485 22458 -27
Branches 3298 3303 +5
==========================================
- Hits 13826 13816 -10
+ Misses 6206 6189 -17
Partials 2453 2453 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Frontier CCE (OpenACC and OpenMP) faulted in the HLLC kernel with a GPU memory access violation: get_mixture_caloric_state passed the species cp array on to get_species_specific_heats_r from inside the kernel, the nested-call pattern CCE is known to fault on. The routine now expands the NASA7 cp/R and h/RT expressions and the mass averaging in place and calls nothing, so HLLC's call is shallower than master's get_mixture_energy_mass. The expressions are the separate routines', so the fused-versus-separate bitwise test still passes. Checked: test_thermochem (17 passed), amdflang 23.2.1 OpenMP offload on MI210 --only Chemistry (16 passed), nvfortran 24.1 --gpu mp --mpi build. Done with Claude Code.
NVHPC 23.11 through 24.7 (OpenACC) crashed in the front end (fort1, signal 11) compiling m_compute_levelset, a file this PR does not touch. Reproduced with nvfortran 24.1 and bisected: master's module files plus this PR's m_thermochem.mod alone crash it, and removing any one of the three new routines avoids it. m_global_parameters imports m_thermochem, so nearly every file reads that module; the crash depends on its contents, not simply its size (gri30's much larger module compiles). The three routines now live in a companion module, m_thermochem_state, generated into the same file and imported only by their callers. m_thermochem is byte-identical to master's for h2o2 and gri30. Checked: test_thermochem (17 passed); nvfortran 24.1 --gpu acc --mpi Chemistry build (previously crashed); gfortran CPU and amdflang OpenMP offload on MI210 --only Chemistry (16 passed each). Done with Claude Code.
Lines of Code
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #1915. MFC's hot loops chain several generated calls per gas state, and each re-evaluates the same NASA7 fits or composition. The generator now also emits routines shaped like those call sites, in the spirit of #1915's fused creation/destruction routine:
get_mixture_caloric_state(T, Y, cp0_r, cp, cv, e)get_species_specific_heats_r+..._cp_mass+..._cv_mass+get_mixture_energy_mass(4 NASA7 passes -> 2)get_mixavg_transport_state(p, T, Y, MW, X, D, lambda)get_mixture_molecular_weight+get_mole_fractions+get_species_mass_diffusivities_mixavg+get_mixture_thermal_conductivity_mixavg(composition computed once, not three times)get_species_enthalpies_mass(T, h)get_species_enthalpies_rt+ the caller's*gas_constant*T/molecular_weightsloopCallers:
get_mixture_caloric_statecall per state. Withgamma_method = 2that halves the NASA7 evaluations; with the defaultgamma_method = 1it is the same work in fewer calls. Unusedm_thermochemimports are dropped.transport_model = 1): oneget_mixavg_transport_stateand oneget_species_enthalpies_massper side, and the hand-written enthalpy scaling loop goes away.Design constraints:
m_riemann_state.fpp), so the calls stay in the loop bodies and the shared diffusivity/conductivity bodies are Mako%defs expanded into both the existing and the fused routine, not a new callee. No device routine is nested deeper than before, and the existing routines generate identically to master.Testing
test_fused_routines(h2o2, gri30): the fused routines are bitwise equal to the separate calls (gfortran-O0) and agree with Cantera.OMP_TARGET_OFFLOAD=MANDATORY):--only Chemistry- 16 passed. (One test first failed because a concurrent build in the same checkout replaced itssyscheckbinary; it passed on re-run.)--only Chemistry- 16 passed, 0 failed../mfc.sh precheckpasses.Optimized builds do not keep bitwise equality: on the MI250X benchmark below, 13 of 17 fields differ from master at step 500, by at most 8e-11 relative (momentum; most fields 1e-15 to 1e-13), with an identical initial condition. That is inlining/FMA contraction differing between one fused routine and separate calls. All goldens pass at their existing tolerances.
Performance
MI250X (one GCD), amdflang 23.2.1, generic
--gpu mp --no-mpi,examples/2D_premixed_flame_vortex(h2o2, HLLC,gamma_method = 1, mixture-averaged diffusion), 500 steps, interleaved:fef3c276One run drives most of the difference, so read this as neutral to slightly faster for this case.
gamma_method = 2cases should gain more, since they save half the NASA7 passes; I did not benchmark one.Updates after the first CI run
get_mixture_caloric_statepassed the species cp array on toget_species_specific_heats_rfrom inside the kernel. It is now a leaf routine (NASA7 expressions and averaging expanded in place, no calls), so HLLC's call is shallower than master'sget_mixture_energy_mass. The bitwise test still passes. (6c93dec3)fort1, signal 11) compilingm_compute_levelset, which this PR does not touch. I reproduced it locally with nvfortran 24.1 and bisected it tom_thermochem.modalone: nearly every file reads it throughm_global_parameters, and adding the three routines trips a content-dependent front-end bug (removing any one avoids it; gri30's far larger module does not trip it). The three routines now live in a companion module,m_thermochem_state, generated into the same file and imported only by their callers.m_thermochemis byte-identical to master's for h2o2 and gri30. (3a47e7ec)Re-checked after both:
test_thermochem17 passed; nvfortran 24.1--gpu acc --mpiChemistry build (previously crashed) completes; gfortran CPU and amdflang OpenMP offload on MI210--only Chemistry16 passed each.Follow-up
#1821's surface residual (
m_ibm.fpp) callsget_mixture_molecular_weight,get_species_mass_diffusivities_mixavgand the surface rates separately and recomputes mole fractions. Once both land, it can useget_mixavg_transport_statetoo.Prepared with Claude Code.
Contribution Policy
We do not accept pull requests generated primarily by AI without genuine understanding or real-world usage context.
All contributions are expected to demonstrate:
If these expectations are not met, we would prefer to implement the changes ourselves rather than spend time reviewing low-effort submissions.
Acknowledgement
PR template credit: junegunn