Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
622edc5
docs: design for extracting m_eos from m_variables_conversion
sbryngelson Sep 12, 2026
9f97455
docs: implementation plan for the m_eos extraction
sbryngelson Sep 12, 2026
2a50daa
refactor: move the equation-of-state machinery into m_eos
sbryngelson Sep 12, 2026
7d54ba4
refactor: import the EOS operators from m_eos directly
sbryngelson Sep 12, 2026
0196d61
refactor: give m_eos its own initialization routine
sbryngelson Sep 12, 2026
54b26b8
refactor: mark cross-module EOS device helpers and finalize m_eos sym…
sbryngelson Sep 12, 2026
e5e78a9
docs: point the equation-of-state guide at m_eos
sbryngelson Sep 12, 2026
010bf1e
docs: move s_reference_curve to the top of the EOS operator table
sbryngelson Sep 12, 2026
d66026c
docs: design and plan for collapsing the EOS family registry
sbryngelson Sep 12, 2026
d0d6199
fix: address final review nits on m_eos extraction
sbryngelson Sep 12, 2026
d747b08
refactor: collapse the EOS enum/label/param-block trio into EOS_FAMILIES
sbryngelson Sep 12, 2026
a8757cf
refactor: rewire case_validator.py to EOS_FAMILIES, fix Vinet gruneis…
sbryngelson Sep 12, 2026
59dc36d
test: pin the Vinet initial-state check to a state where gruneisen_a …
sbryngelson Sep 12, 2026
0935137
refactor: generate eos_* constants from EOS_FAMILIES instead of hand-…
sbryngelson Sep 12, 2026
6a38316
refactor: generate m_eos's family predicates and coefficient init fro…
sbryngelson Sep 12, 2026
4e7ed93
test: add EOS_FAMILIES agreement tests (R5)
sbryngelson Sep 12, 2026
cf81be3
docs: describe the EOS_FAMILIES registry in the contributing guide
sbryngelson Sep 12, 2026
e7acf77
refactor: derive the last two EOS family lists from the registry
sbryngelson Sep 12, 2026
08ba8ce
Delete docs/superpowers/plans/2026-09-11-m-eos-extraction.md
sbryngelson Sep 12, 2026
ab34a1a
Delete docs/superpowers directory
sbryngelson Sep 12, 2026
acf936f
Import s_prohibit_abort into m_eos so case-optimized builds link
sbryngelson Sep 13, 2026
3ef9c49
Merge remote-tracking branch 'origin/master' into sync-1870
sbryngelson Sep 17, 2026
865337a
Keep the EOS phase chain inlinable on NVHPC
sbryngelson Sep 18, 2026
9668f8a
Format: re-wrap the m_eos public list and drop stray blank lines
sbryngelson Sep 18, 2026
b253aac
Merge branch 'master' into module-eos
sbryngelson Sep 19, 2026
a411e10
Merge branch 'master' into module-eos
sbryngelson Sep 19, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion cmake/ParamsCodegen.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -9,28 +9,31 @@ file(GLOB_RECURSE _mfc_gen_inputs
"${CMAKE_CURRENT_SOURCE_DIR}/toolchain/mfc/params/*.py"
)

# Enumerate the 15 generated .fpp files explicitly so ninja can track them as
# Enumerate the 18 generated .fpp files explicitly so ninja can track them as
# build-time outputs and so HANDLE_SOURCES does not need a configure-time GLOB
# of ${CMAKE_BINARY_DIR}/include/<target>/ (which fails when the dir is empty).
set(_mfc_gen_inc "${CMAKE_BINARY_DIR}/include")
set(_mfc_gen_files_pre_process
"${_mfc_gen_inc}/pre_process/generated_namelist.fpp"
"${_mfc_gen_inc}/pre_process/generated_decls.fpp"
"${_mfc_gen_inc}/pre_process/generated_constants.fpp"
"${_mfc_gen_inc}/pre_process/generated_eos.fpp"
"${_mfc_gen_inc}/pre_process/generated_bcast.fpp"
"${_mfc_gen_inc}/pre_process/generated_case_opt_decls.fpp"
)
set(_mfc_gen_files_simulation
"${_mfc_gen_inc}/simulation/generated_namelist.fpp"
"${_mfc_gen_inc}/simulation/generated_decls.fpp"
"${_mfc_gen_inc}/simulation/generated_constants.fpp"
"${_mfc_gen_inc}/simulation/generated_eos.fpp"
"${_mfc_gen_inc}/simulation/generated_bcast.fpp"
"${_mfc_gen_inc}/simulation/generated_case_opt_decls.fpp"
)
set(_mfc_gen_files_post_process
"${_mfc_gen_inc}/post_process/generated_namelist.fpp"
"${_mfc_gen_inc}/post_process/generated_decls.fpp"
"${_mfc_gen_inc}/post_process/generated_constants.fpp"
"${_mfc_gen_inc}/post_process/generated_eos.fpp"
"${_mfc_gen_inc}/post_process/generated_bcast.fpp"
"${_mfc_gen_inc}/post_process/generated_case_opt_decls.fpp"
)
Expand Down
45 changes: 40 additions & 5 deletions docs/documentation/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ between them is narrow.
mfc.sh (env bootstrap, venv, module loading, lock)
└─ toolchain/mfc/build.py (config slugs, cmake invocation)
└─ CMakeLists.txt + cmake/{GPU,Fypp,ParamsCodegen,MFCTargets}.cmake
└─ toolchain/mfc/params/generators/cmake_gen.py (writes 15 generated .fpp includes)
└─ toolchain/mfc/params/generators/cmake_gen.py (writes 18 generated .fpp includes)
```

**`mfc.sh` → `build.py`.** `mfc.sh` is a thin shell wrapper that activates the Python
Expand All @@ -100,8 +100,8 @@ mode, debug, chemistry, MPI. Staging and install trees are namespaced by slug u
**CMake layer.** `cmake/Fypp.cmake` defines `HANDLE_SOURCES`, which sets up one
`add_custom_command` per `.fpp` file to run Fypp at build time. `cmake/ParamsCodegen.cmake`
registers a single ninja-tracked `add_custom_command` (DEPENDS all `params/*.py`) that
invokes `cmake_gen.py` and writes the 15 generated includes under
`build/include/<target>/`. There is no configure-time generation: all 15 files are build
invokes `cmake_gen.py` and writes the 18 generated includes under
`build/include/<target>/`. There is no configure-time generation: all 18 files are build
outputs, so changing any `params/*.py` triggers only a targeted rebuild, not a full
reconfigure.

Expand Down Expand Up @@ -466,11 +466,17 @@ If an array is allocated inside an `if` block, its deallocation must follow the

### How to Add an Equation of State

Every stiffened-gas expression lives in `src/common/m_variables_conversion.fpp`. Adding a second EOS
means supplying these, not grepping for `gammas`:
The equation-of-state operators live in `src/common/m_eos.fpp`, together with the mixture closure
rules that combine them (`s_compute_mixture_coefficients`, `s_compute_speed_of_sound` and their
`_dt`/`_avg` variants). The closure rules are not themselves equations of state, but they must stay
in the same file as the phase chain they call: on NVHPC that call only inlines within one file, and
separating them costs about a quarter of the grind time with no effect on the other backends. See
the module-boundary section of @ref gpuParallelization before moving anything out of `m_eos.fpp`.
Adding a second EOS means supplying these, not grepping for `gammas`:

| Operator | Gives |
|---|---|
| `s_reference_curve` | the reference curve \f$p_{ref}, e_{ref}\f$ and \f$\Gamma_G\f$ of a state-dependent family - one `case` per family, and nothing else |
| `s_compute_mixture_coefficients` / `_dt` | mixture \f$\Gamma, \Pi_\infty, q_v\f$ from the phase fractions, and their time derivative |
| `f_pressure` / `s_compute_energy` | \f$p(e)\f$ and \f$E(p)\f$ |
| `f_bulk_modulus` | \f$K(p)\f$ - every sound speed in MFC is \f$K/\rho\f$, differing only in how phases are mixed |
Expand All @@ -489,6 +495,35 @@ forms the user supplies (see @ref sec-stored-forms), and `isentrope_n`/`isentrop
as \f$p + B = \textrm{const}\,\rho^n\f$, derived once at start-up. Convert with the `f_isentrope_*`
operators rather than open-coding either relation.

Both are resolved once in `s_initialize_eos_module`. A state-dependent family skips this and computes
its coefficients per cell from `s_reference_curve` instead.

**Adding a family** - a new state-dependent parameter set alongside Mie-Gruneisen, JWL and Vinet -
starts at `toolchain/mfc/params/eos_families.py`: one `EosFamily` entry.

From that entry, these are generated and need no hand edit: the Fortran `eos_*` constants, the
`physical_parameters` parameter registration in `definitions.py`, the validator's per-family
required/optional parameter sets and its initial-state coefficient call, both the
`f_is_state_dependent` and `f_has_isentropic_reference` family tests, both layers of
`s_initialize_eos_module`, and the `any_state_dependent_eos` case-optimization flag.

Three things stay hand-written alongside the entry, each checked against the registry by a test
that fails on disagreement: the `case` body in `s_reference_curve` (the per-family mathematics),
the mirror function in `toolchain/mfc/eos.py`, and the family's fields on `physical_parameters` in
`src/common/m_derived_types.fpp`. Those four edits are the whole job in the normal case.

Two further edits are possible and are guarded by the compiler rather than by a test - they fail
loudly at build with a clear error, not silently at runtime:

- `m_eos.fpp`'s `use m_constants, only: ...` list must name the new `eos_<suffix>` constant. The
generated family predicates reference it, so omitting it fails to compile.
- the `eos_coefficients` type in `src/common/m_derived_types.fpp` needs a new field only if the
family stores a coefficient the existing ones do not provide. The generated assignment in
`s_initialize_eos_module` fails to compile if the field is missing.

One caveat: `f_has_isentropic_reference`'s `gruneisen_a == 0._wp` conjunct is a runtime test, not a
family property, and stays hand-written in `m_eos.fpp`.

### How to Add a Test Case

**Step 1: Create a case file**
Expand Down
2 changes: 1 addition & 1 deletion docs/documentation/equations.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ where:

The parameter `model_eqns` (1, 2, or 3) selects the governing equation set.

**Key source files:** `src/simulation/m_rhs.fpp` (RHS evaluation), `src/common/m_variables_conversion.fpp` (EOS and variable conversion).
**Key source files:** `src/simulation/m_rhs.fpp` (RHS evaluation), `src/common/m_eos.fpp` (equations of state), `src/common/m_variables_conversion.fpp` (variable conversion and mixture rules).

---

Expand Down
40 changes: 40 additions & 0 deletions docs/documentation/gpuParallelization.md
Original file line number Diff line number Diff line change
Expand Up @@ -664,6 +664,46 @@ that defines the routine. Helpers added to `m_riemann_state.fpp` are automatical
in scope for every solver module that `use`s it — no additional declare-target
annotations are needed at call sites.

## Module boundaries and NVHPC inlining

**Moving a device helper into another file can silently cost ~25% on NVHPC.** NVHPC has no
device LTO, so MFC's only cross-file inlining is the two-pass `-Mextract=lib:` / `-Minline=lib:`
scheme in `cmake/MFCTargets.cmake`. That inliner **refuses any device routine that has a
subroutine call anywhere in its call tree**, reporting:

```
subprogram not inlined -- missing prototype during crossing files: <name>
```

The refusal propagates: a caller of a refused routine is refused too. Measured on nvfortran
25.11 (A100), the following hold for a routine that must inline across files:

| in the routine's body | inlines across files? |
|---|---|
| arithmetic, branches on module logicals, module array reads, early `return` | yes |
| a call to a scalar-returning function that is itself inlinable | yes (the callee need not be inlined) |
| a call to a **subroutine** | **no** |
| the routine *returns* a derived type or an array | **no** — never inlinable |

There is no build-level escape: `-Mextract` always captures pre-inline source, so re-extracting
in stages, compiling several files in one invocation, and `levels:`/`maxsize:`/`name:`/`except:`
all fail, as does `-Mipa` (ignored in 25.x). Cray and AMD do their own whole-program IPA and are
unaffected, and CPU builds do not care — so this shows up as an NVHPC-only benchmark regression
while every other job stays green.

**Symptom.** Grind time regresses on NVHPC alone, with unchanged source semantics. Confirm by
comparing per-routine stack frames: `-Minfo=inline` and `-gpu=ptxinfo` are already on, so
`Function properties for ...` lines jumping from ~8 bytes to 200–350 bytes with matching
`spill stores`/`spill loads` is the fingerprint. Registers per thread going *down* while the
kernel gets slower is the same story seen from `ncu`.

**Rule of thumb.** Draw a module boundary where inlining already fails, not in the middle of a
chain that currently inlines. Solver kernels never inlined `s_compute_mixture_coefficients` or
`s_compute_speed_of_sound` even before `m_eos` existed, which makes that a free cut point; the
phase chain those two call (`s_phase_coefficients` → `s_eos_coefficients` → `s_reference_curve`)
must stay in the same file as them. This is why those four routines live in `src/common/m_eos.fpp`
alongside the EOS operators even though mixture closure is not, strictly, an equation of state.

------------------------------------------------------------------------------------------

# Debugging Tools and Tips for GPUs
Expand Down
1 change: 1 addition & 0 deletions docs/module_categories.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
{
"category": "Physics Models",
"modules": [
"m_eos",
"m_viscous",
"m_hb_function",
"m_surface_tension",
Expand Down
12 changes: 4 additions & 8 deletions src/common/m_constants.fpp
Original file line number Diff line number Diff line change
Expand Up @@ -113,16 +113,12 @@ module m_constants
integer, parameter :: BC_NO_SLIP_WALL = -16
integer, parameter :: BC_DIRICHLET = -17

! Synthetic turbulence array size limits
!> Equation of state per fluid. Values must match _EOS_NAMES in toolchain/mfc/params/definitions.py; compound registry keys
!! cannot be auto-generated, so these are hand-written.
integer, parameter :: eos_stiffened_gas = 1
integer, parameter :: eos_ideal_gas = 2
integer, parameter :: eos_mie_gruneisen = 3
integer, parameter :: eos_jwl = 4
integer, parameter :: eos_vinet = 5
!> Equation of state per fluid (eos_stiffened_gas, eos_ideal_gas, eos_mie_gruneisen, eos_jwl, eos_vinet) is generated from
!! EOS_FAMILIES in toolchain/mfc/params/eos_families.py; see generated_constants.fpp below.
integer, parameter :: eos_rk4_steps = 8 !< fixed-step RK4 along a phasic isentrope or a reference temperature
integer, parameter :: ode_isentrope = 1, ode_reference_temperature = 2 !< the two ODEs s_rk4 integrates

! Synthetic turbulence array size limits
integer, parameter :: num_synth_shells_max = 50 !< Max energy shells for synthetic turbulence
integer, parameter :: num_turb_sources_max = 10 !< Max Gaussian forcing zones for synthetic turbulence

Expand Down
Loading
Loading