Regression suite

The verification cases assert the physics: an exact solution, a reference, a symmetry or a conservation property. The regression suite does a different job. It freezes 39 inputs, most of them copies of a verification input cut to a few dozen steps, and detects any change of their outputs, wanted or not.

How it works

Each case directory src/tests/flume/regression/<case>/ holds a frozen input.ini (a multi-realm case: a forest manifest plus one INI per realm) and the goldens golden/cpu/ and golden/fnl/:

Golden	Content	Comparison
digest.txt	per-variable digest of every checkpoint field (src/tests/prism/regression/digest.py)	within the digest tolerances (default rtol ${10}^{-6}$)
*-residuals.dat	per-iteration residuals	header verbatim, integer columns exact, float columns within the tolerances
*-conservation_history.dat	per-iteration volume integrals of the conservative variables (and $W$, $E+W$ under self-gravity)	as the residuals

The raw HDF5 checkpoints are not committed. Every case runs on 2 MPI ranks.

Script	Role
src/tests/flume/regression/run.sh cpu	builds flume-cpu-gnu and flume-cpu-gnu-fft, runs every case, compares; runs in CI
src/tests/flume/regression/run-fnl-local.sh	the same on the GPU (flume-fnl-nvf and flume-fnl-nvf-fft, --varset local_nvf); a workstation gate, since the CI runners have no GPU
src/tests/flume/regression/run-omp-bitwise.sh	checks that the OpenMP build reproduces the serial one bit for bit (CI)

Two executables per backend: every case with self_gravity = .true. runs on the -fft executable (FFTW on CPU, cuFFT on FNL), since every self-gravity solve needs an FFT; the CI job installs libfftw3-dev. --no-build skips the build; REGRESSION_CASES="a b" restricts the run; REGRESSION_RUN_GOLDENLESS=1 runs a case that has no golden yet (the capture of a new case). Before running, the harness refuses an executable older than any of its sources.

A case also fails when its run exits non-zero, a crash or the hard timeout, even if its outputs match the goldens: a crash after the last checkpoint is still a crash (hcoll segfaulting in MPI_Finalize on a cluster passed unseen before). The outputs are compared anyway, for the report. The one exemption is the harness's own termination of a run that hung in MPI_Finalize after its end-of-run marker (the WSL GPU stack). REGRESSION_ALLOW_MPIRUN_FAILURE=1 turns the failure into a warning on an environment whose crash is known.

src/tests/flume/regression/run.sh cpu                     # build, run, compare (CPU)
src/tests/flume/regression/run-fnl-local.sh               # the same on the GPU
REGRESSION_CASES="sod-x rotor" src/tests/flume/regression/run.sh cpu --no-build

Cases

Case	Freezes	Issue
sod-x, sod-y, sod-z	Sod shock tube, along each axis	#35
sod-hllc	Sod on weno-riemann, HLLC	#47
vortex-periodic	isentropic vortex, 100 steps	#35
vortex-rk3	vortex-periodic on the low-storage runge-kutta-3 at CFL 0.8, final state digested: the only case on the low-storage stage update (every other case runs an SSP scheme, and a low-storage scheme is refused on AMR seams and multi-realm runs). At CFL 0.1 the two schemes agree to ${10}^{-7}$, inside the digest tolerances; at 0.8 the digest rejects the SSP-54 run	#95
amr-periodic-reflux	conservation across AMR seams: init-time AMR, reflux	#35
amr-periodic-reflux-stage	the same with reflux_cadence = stage	#81
shock-cylinder-ib	shock over a cylinder, immersed boundary, 120 steps	#35
sod-2realm, sod-2realm-blocks, sod-amr, sod-amr-2realm, sod-amr-refined	multi-realm runs: mirror seams lined up or not, a 2:1 face, the refined coupling	#37, #51, #52
rj2a-x, rj2a-y, rj2a-z	Ryu–Jones 2a	#41
rj2a-hlld	Ryu–Jones 2a on weno-riemann, HLLD	#47
rj2a-2realm	Ryu–Jones 2a in two realms	#41
brio-wu	Brio–Wu	#41
glm-pulse	GLM cleaning	#41
orszag-tang	Orszag–Tang	#41
orszag-tang-eglm	Orszag–Tang with EGLM	#47
rotor	MHD rotor	#41
field-loop	field loop	#41
rotated-shock-tube	rotated shock tube	#41
uniform-amr-mhd	MHD on a uniform state across 2:1 faces	#41
blast-limiter, blast3d-limiter	the Balsara–Spicer blast with the positivity limiter, 2-D and 3-D	#47
blast-amr-limiter	the limited blast across two 2:1 seams	#50
shear-viscous	viscous waves	#65
couette	Couette flow	#65
becker-shock	Becker's viscous shock	#65
alfven-resistive	the visco-resistive Alfvén wave	#65
seam-viscous-amr	the dissipative fluxes across 2:1 seams	#65
vortex-regrid	runtime regrids: the vortex, 6 regrids	#74
field-loop-regrid	runtime regrids on MHD: the field loop, 10 regrids	#74
selfgravity-jeans	Jeans modes: the unstable mode at amplitude ${10}^{-2}$, FFT solver, 20 steps	#81
selfgravity-amr-stage	self-gravity on a static AMR grid: multigrid, the seam gravity work, per-stage reflux	#81
selfgravity-star	the n = 1 polytrope, isolated boundaries, ${16}^3$	#81
selfgravity-star-amr	the n = 1 polytrope on AMR (SV-14): isolated boundaries by the multigrid's James step, the star of radius 0.3 collapsing, the box refined over ${32}^3$, per-stage reflux, 20 steps	#87
sod-ps	passive scalars (PS ps1): the sod-x shock tube with two scalars, 1 left and 0 right of the diaphragm	#92
radiative-shock	radiative cooling (CO co7): two streams at the cooling floor $T=0.1$ collide, 100 cells to $t=1$, $\mathrm{\Lambda }=0.18(T/0.1{)}^{1.5}$: the shock heating, the cooling layer, the dense layer	#93
noh-de	dual energy (DE de2): the cold planar Noh shock, Mach 845, with physical_model = euler-de, 50 cells, $t=0.6$; every cell starts cold, so the entropy floor acts every step	#91
iso-tube	isothermal equation of state (EO eo1): the isothermal shock tube, $\rho =1\mid 0.125$, $v=0.3\mid -0.2$, 100 cells in four blocks, $y$ active, WENO5 flux splitting, $t=0.2$: the isothermal library, the inert energy slot	#100
periodic-1block	the isothermal acoustic wave of EO on one block not refined at init, every face periodic, 48 cells, $t=1$: the tree and the maps rebuilt after the BC types are known (#103); equal to the 8-block run bitwise on CPU and FNL; the BC crown map with no row	#103
iso-collide-hllc	isothermal equation of state (EO eo2): two Mach-5 isothermal streams colliding, 100 cells, weno-riemann with the isothermal HLLC, $t=0.5$: the isothermal Riemann kernels, the dense layer at $M_1^2$	#100
baro-tube	barotropic equation of state (EO eb1): the tube $\rho =10\mid 0.5$ on the convex table POLY, its left fan crossing the break $\rho =8$, 100 cells in four blocks, WENO5 flux splitting, $t=0.15$: the table registry, the barotropic library	#100
baro-collide-hllc	barotropic equation of state (EO eb2): the collision $(1,3.4)\mid (1.5,-2.6)$ on POLY, both shocks into the $\mathrm{\Gamma }=5/3$ segment, 100 cells, weno-riemann HLLC, $t=0.3$: the barotropic Riemann kernels, the Roe secant speed	#100
baro-jeans-box	barotropic gases under gravity (EO eg4): the barotropic Jeans collapse (POLY on ${\rho }_0=5$, amplitude ${10}^{-2}$), 32² base with the central box refined 2:1, the multigrid Poisson solver, the seam reflux, 20 steps: gravity on the momentum only, the energy row cleared	#100
iso-sinks-shu	barotropic gases under gravity (EO eg6): the overdense singular isothermal sphere ($A=29.3$, $c_0=0.5$) with a seed sink, isothermal, 16³, isolated self-gravity, the density-only positivity limiter, 30 steps: the sinks' $c_s^2=c_0^2$; per-backend goldens	#100
iso-mhd-tube2	isothermal and barotropic MHD (EO em4): Mignone's isothermal tube 2 (rotational discontinuities), 200 cells in four blocks of one transverse cell, WENO5 flux splitting on Kim's eigenvectors, $t=0.2$: the isothermal MHD library, the inert energy slot; per-backend goldens	#100
baro-mhd-glm-hlld	isothermal and barotropic MHD (EO em4): tube 2 with the densities times 5 on POLY (the $\mathrm{\Gamma }=1.4$ segment), mixed GLM, weno-riemann with the isothermal HLLD on $P(\rho )$, 200 cells, $t=0.2$: the barotropic MHD library, the HLLD and GLM kernels; per-backend goldens	#100
sinks-shu	sink particles (SI si11): the overdense singular isothermal sphere ($A=29.3$, $\gamma =1.001$) with a seed sink at the centre, ${16}^3$, 30 steps: the deposit into the Poisson source, the CIC gather, threshold accretion, the kick-drift-kick, the sink columns of the history	#94

The self-gravity goldens agree across the backends to $9\cdot {10}^{-14}$ relative in the volume integrals and $E+W$, and to $1.8\cdot {10}^{-18}$ absolute in the momentum, which is round-off.

Refreshing a golden

Only after an intentional change, adjudicated case by case (which fields move, when, by how much, and why). The run fails against the old golden but leaves its outputs in work-cpu/ (work-fnl/ on the GPU):

REGRESSION_CASES=<case> src/tests/flume/regression/run.sh cpu --no-build
mkdir -p src/tests/flume/regression/<case>/golden/cpu
cp src/tests/flume/regression/<case>/work-cpu/{digest.txt,*-residuals.dat,*-conservation_history.dat} \
   src/tests/flume/regression/<case>/golden/cpu/