adam_flume_fnl_object

ADAM, FLUME FNL (OpenACC / OpenMP offload) backend object.

Implements the forest contract on the device. The state lives on the device for the whole run in the transposed layout (b, i, j, k, v); full-field device-to-host copies happen only on save steps. The FNL helpers (field, IB, RK, WENO) are per-realm components initialized after the common initialization from the realm's own objects; mpih_fnl is the only FNL singleton and is initialized once per process. Space operator: characteristic (or conservative) WENO flux splitting, face fluxes then flux difference (issue #35, section 3.4); the per-face physics is the shared adam_flume_euler_library, so only the kernels are FNL-specific.

Source: src/app/flume/fnl/adam_flume_fnl_object.F90

Dependencies

Contents

• equilibrium_seam_face
• flume_fnl_object
• accumulate_seam_fluxes
• flag_troubled_seam_cells
• add_dissipative_fluxes_dev
• apply_floors
• allocate_gpu
• check_nonfinite
• check_glm_ch
• compute_conservation
• compute_divb_history
• compute_dflux_norms
• compute_q_aux
• copy_cpu_gpu
• copy_gpu_cpu
• copy_phi_gpu
• refresh_potential
• copy_potential_gpu
• solve_potential_gpu
• destroy
• initialize_flume
• limit_positivity_dev
• seam_lists
• seam_sync_fluxes_dev
• seam_sync_factors_dev
• seam_sync_theta_dev
• seam_sync_blend_dev
• save_residuals
• save_simulation_data
• set_boundary_conditions
• build_bc_pass_rows
• update_ghost
• advance_one_step_forest
• after_topology_build_forest
• apply_reflux_to_stage_forest
• begin_stage_forest
• close_step_forest
• compute_local_dt_forest
• end_stage_forest
• reflux_stage
• fill_seam_from_peer_forest
• pack_seam_cells_forest
• unpack_seam_cells_forest
• finalize_forest
• finalize_mpi_forest
• initialize_forest
• is_done_forest
• open_step_forest
• post_step_forest
• regrid_forest
• copy_blocks_transposed
• refresh_equilibrium_residual
• compute_gravity_correction
• compute_residuals_weno_dev
• compute_residuals_riemann_dev
• integrate_rk_ls_dev
• integrate_rk_ssp_dev
• rk_assign_stage
• rk_compute_stage
• rk_compute_stage_ls
• rk_update_q
• seam_gather_cells
• seam_gather_stencils
• seam_gather_faces
• seam_scatter_faces
• seam_scatter_cells
• stages_per_step_forest

Derived Types

equilibrium_seam_face

The equilibrium's flux mismatch of one register face, F_coarse - F_fine_sum per skin cell (issue #73, M7-G2b).

Components

Name	Type	Attributes	Description
d	real(kind=R8P)	allocatable	Mismatch [nv, nface_cells].

flume_fnl_object

FLUME FNL backend object.

Inheritance

Extends: flume_common_object

Components

Name	Type	Attributes	Description
io	type(io_object)		IO handler.
amr	type(amr_object)		AMR marker handler.
slices	type(slices_object)		Slices handler.
blanesmoan	type(blanesmoan_object)		Blanes-Moan integrator.
cfm	type(cfm_object)		Commutator-Free Magnus integrator.
leapfrog	type(leapfrog_object)		Leapfrog integrator.
flail	type(flail_object)		Linear algebra methods handler.
weno	type(weno_object)		WENO reconstructor.
ib	type(ib_object)		Immersed boundary.
rk	type(rk_object)		Runge-Kutta integrator.
adam	type(adam_object)		ADAM, grid + tree + field + maps orchestrator.
fdv_scheme	character(len=:)	allocatable	FDV scheme, fd/fv.
fdv_order	integer(kind=I4P)		Order of finite difference/volume schemes, general order.
fdv_half_stencil	integer(kind=I4P)		Half stencil length of finite difference/volume schemes.
fdv_half_stencils	integer(kind=I4P)		Half stencil length of fdv schemes for each derivative up to 6.
ngc	integer(kind=I4P)	pointer	Number of ghost cells.
ni	integer(kind=I4P)	pointer	Number of cells in i direction.
nj	integer(kind=I4P)	pointer	Number of cells in j direction.
nk	integer(kind=I4P)	pointer	Number of cells in k direction.
nb	integer(kind=I4P)	pointer	Total blocks number for MPI.
blocks_number	integer(kind=I4P)	pointer	Actual blocks number.
nv	integer(kind=I4P)	pointer	Number of variables in q vector.
realm_index	integer(kind=I4P)		Inter-realm seam coupling — currently-published stage buffer key.
stage_active	integer(kind=I4P)		A copy of the forest's flux register (its faces and lookup, the accumulators unused), refreshed by the forest each
flux_register_copy	type(flux_register_object)		Procedure pointer TBPs for FDV operators (set at initialization by backend).
compute_block_total_variation	procedure(compute_block_total_variation_interface)	pass(self), pointer	Compute TV.
compute_curl	procedure(compute_curl_interface)	pass(self), pointer	Compute curl.
compute_curl_extended	procedure(compute_curl_interface)	pass(self), pointer	Compute curl on extended support.
compute_derivative1	procedure(compute_derivative1_interface)	pass(self), pointer	Compute deriv1.
compute_derivative2	procedure(compute_derivative2_interface)	pass(self), pointer	Compute deriv2.
compute_derivative4	procedure(compute_derivative4_interface)	pass(self), pointer	Compute deriv4.
compute_divergence	procedure(compute_divergence_interface)	pass(self), pointer	Compute dive.
compute_gradient	procedure(compute_gradient_interface)	pass(self), pointer	Compute grad.
compute_gradient_extended	procedure(compute_gradient_interface)	pass(self), pointer	Compute gradient on extended support.
compute_laplacian	procedure(compute_laplacian_interface)	pass(self), pointer	Compute laplac.
amr_locked_	logical		Runtime AMR locked after initialization.
save_auxiliary_fields	logical		Save the auxiliary variables with the fields.
glm_speed_reported	real(kind=R8P)		Largest wave speed above c_h reported so far.
divb_reported	real(kind=R8P)		Largest monitored measure above divb_tol reported so far.
divb_seam	integer(kind=I4P)	allocatable	Seam faces flags of the div(B) history [nb, 6].
q	real(kind=R8P)	allocatable	Conservative variables [nv, 1-ngc:ni+ngc, ..., nb].
dq	real(kind=R8P)	allocatable	Residuals [nv, 1-ngc:ni+ngc, ..., nb].
q_aux	real(kind=R8P)	allocatable	Auxiliary variables [nv_aux, 1-ngc:ni+ngc, ..., nb].
phi_grav	real(kind=R8P)	allocatable	Gravitational potential [1-ngc:ni+ngc, ..., nb] (issue #81);
poisson	class(poisson_fft_object)	allocatable	Self-gravity Poisson solver (issue #81; FNL: the cuFFT one).
poisson_mg	type(poisson_mg_object)	allocatable	Self-gravity on AMR grids (issue #81, P5b): the multigrid
poisson_wall	real(kind=R8P)		Wall time of the last Poisson solve (s).
q_name	type(string)	allocatable	Conservative variables names.
dq_name	type(string)	allocatable	Residuals names.
q_aux_name	type(string)	allocatable	Auxiliary variables names.
bc	type(flume_bc_object)		Boundary conditions.
diagnostics	type(flume_diagnostics_object)		Diagnostics.
ic	type(flume_ic_object)		Initial conditions.
numerics	type(flume_numerics_object)		Numerics.
physics	type(flume_physics_object)		Physics.
units	type(flume_reference_object)		Reference layer (dimensional input, issue #49).
time	type(flume_time_object)		Time handler.
field_fnl	type(field_fnl_object)		Field helper (coordinates, maps, ghost exchange).
ib_fnl	type(ib_fnl_object)		Immersed boundary helper.
rk_fnl	type(rk_fnl_object)		Runge-Kutta helper.
weno_fnl	type(weno_fnl_object)		WENO helper.
q_gpu	real(kind=R8P)	pointer	Conservative variables [nb, i, j, k, nv].
dq_gpu	real(kind=R8P)	pointer	Residuals [nb, i, j, k, nv].
q_aux_gpu	real(kind=R8P)	pointer	Auxiliary variables [nb, i, j, k, nv_aux].
flx_f_gpu	real(kind=R8P)	pointer	X-face fluxes [nb, 0:ni, 1:nj, 1:nk, nv].
fly_f_gpu	real(kind=R8P)	pointer	Y-face fluxes [nb, 1:ni, 0:nj, 1:nk, nv].
flz_f_gpu	real(kind=R8P)	pointer	Z-face fluxes [nb, 1:ni, 1:nj, 0:nk, nv].
gcor_gpu	real(kind=R8P)	pointer	Fourth-order correction of the gravity energy source
phi_grav_gpu	real(kind=R8P)	pointer	Gravitational potential (issue #81) [nb, i, j, k, 1]; a
dq_e_gpu	real(kind=R8P)	pointer	Residual of the hydrostatic equilibrium (issue #73, M7-G2b)
seam_e	type(equilibrium_seam_face)	allocatable	The equilibrium's 2:1 flux mismatch per register face.
equilibrium_stale	logical		dq_e_gpu describes another grid (init, regrid).
equilibrium_computing	logical		dq_e_gpu is being computed: no subtraction.
seam	type(flume_seam_sync_object)		Per-stage seam flux synchronisation of the limiter (issue #50).
lam_gpu	real(kind=R8P)	pointer	Positivity limiter cell factors (component 1, q-shaped for
q_inflow_gpu	real(kind=R8P)	pointer	Conservative inflow state of each face [nv, 6].
wall_sign_gpu	real(kind=R8P)	pointer	Wall mirror sign per variable and direction [nv, 3].
divb_seam_gpu	integer(kind=I4P)	pointer	Seam faces flags of the div(B) history [nb, 6].
bc_rows_gpu	integer(kind=I4P)	pointer	BC crown map rows grouped by pass [rows, ngc] (#90 F-A').
bc_pass_range	integer(kind=I4P)	allocatable	First, last entry of each pass in bc_rows_gpu [2, 3, ngc].
skin_faces	integer(kind=I4P)	allocatable	Register faces [3, n]: block, fec, column - 1 (#90 F-B).
skin_faces_gpu	integer(kind=I4P)	pointer	Device copy of skin_faces.
skins_gpu	real(kind=R8P)	pointer	Packed skins of the register faces [nv, columns].
buf_5D_R8P	real(kind=R8P)	allocatable	Transposed copy buffer, extent identical to q_gpu.
db5	integer(kind=I4P)		Device bounds of the transposed copies.
hb5	integer(kind=I4P)		Host bounds of the transposed copies.
compute_residuals_dev	procedure(compute_residuals_dev_interface)	pass(self), pointer	Space operator.
integrate_dev	procedure(integrate_dev_interface)	pass(self), pointer	Time operator.

Type-Bound Procedures

Name	Attributes	Description
load_fdv_from_file	pass(self)	Load FDV config from file.
close_block_xh5f	nopass	Close XH5F file block.
close_file_xh5f	nopass	Close XH5F file.
open_block_xh5f	pass(self)	Open block file XH5F.
open_file_xh5f	pass(self)	Open file XH5F.
save_q_xh5f	pass(self)	Save in XH5F (XDMF/HDF5) format.
compute_curl_fd_extended	pass(self)	Compute curl of vector field, finite difference, even for half gcs.
compute_gradient_fd_extended	pass(self)	Compute gradient of scalar field, finite difference, even for half
amr_update	pass(self)	Do AMR update (initialization-time only).
runtime_amr_forest	pass(self)	Whether this realm regrids during the run (issue #74).
mark_by_geometry	pass(self)	Mark blocks to be refined by a primitive geometric box.
mark_by_gradient	pass(self)	Mark blocks by the gradient of a conservative or auxiliary variable.
mark_by_solid	pass(self)	Mark blocks crossed by the surface of an immersed solid.
mark_by_lohner	pass(self)	Mark blocks by the Loehner estimator of a variable (issue #74).
mark_runtime	pass(self)	Combine the markers into the flags of a runtime regrid (issue #74).
check_regridded_state	pass(self)	Stop on an inadmissible state after a regrid (issue #74).
is_regrid_step	pass(self)	Whether the step just closed is a runtime regrid step (issue #74).
accumulate_seam_skin	pass(self)	Route one weighted seam face skin to the forest's flux register.
add_seam_gravity_work	pass(self)	Add the seam gravity work of self-gravity to a seam skin.
allocate_common	pass(self)	Allocate common data.
compute_fields_number	pass(self)	Compute the block-sized fields allocated per block.
compute_phi	pass(self)	Compute the immersed solids distance function (host).
destroy_common	pass(self)	Free common data.
glm_lambda	pass(self)	Return the GLM bound of the local dt, c_h max sum_d 1/dx_d.
initialize	pass(self)	Initialize the common data.
log_dt_limit	pass(self)	Log which limit, hyperbolic or diffusive, sets dt.
log_gravity_regime	pass(self)	Log the velocity gravity adds in a step (issue #73).
log_self_gravity	pass(self)	Log the self-gravity regime (issue #81).
poisson_iterations	pass(self)	FCG iterations of the last multigrid solve.
wall_temperature_bound	pass(self)	Return the hottest isothermal wall temperature of the dt bound.
null_freeze	pass(self)	Return the variable each null direction freezes.
output_factors	pass(self)	Return the output factors of written variables.
output_names	pass(self)	Return the names of every written variable.
report_divb	pass(self)	Reduce and save the div(B) norms, apply the divb_tol monitor.
report_glm_speed	pass(self)	Check c_h against the fastest wave (warning or stop).
nonfinite_total	pass(self)	Sum the non-finite values count over the ranks.
stop_nonfinite	pass(self)	Stop on a non-finite state, locating it on every rank.
load_restart_files	pass(self)	Load restart files.
save_restart_files	pass(self)	Save restart files.
save_slices	pass(self)	Save the slices on their cadence.
save_xh5f	pass(self)	Save fields in XH5F format.
set_divb_seam	pass(self)	Set the seam faces flags of the div(B) history.
set_glm_damping	pass(self)	Set the GLM damping once the grid exists.
set_potential	pass(self)	Set the gravitational potential of the current grid.
solve_potential	pass(self)	Solve the self-gravity potential of a host state.
solve_potential_packed	pass(self)	Solve the self-gravity potential of a packed density.
coupling_descriptor_forest	pass(self)	Return the realm coupling descriptor.
accumulate_seam_fluxes	pass(self)	Accumulate the weighted seam face fluxes of one stage.
reflux_stage	pass(self)	The per-stage reflux (issue #81, P5c).
flag_troubled_seam_cells	pass(self)	Flag the troubled seam face cells of one stage (#68 E1).
add_dissipative_fluxes_dev	pass(self)	Add the dissipative face fluxes on the device (issue #65).
apply_floors	pass(self)	Apply the MHD positivity floors of a stage.
allocate_gpu	pass(self)	Allocate device data.
check_glm_ch	pass(self)	Check the GLM c_h against the fastest wave.
compute_conservation	pass(self)	Compute and save the conservation integrals.
compute_divb_history	pass(self)	Compute and save the div(B) norms (MHD).
compute_dflux_norms	pass(self)	Compute the D_flux norms of the committed state (#68 A2).
compute_q_aux	pass(self)	Compute the auxiliary variables.
compute_gravity_correction	pass(self)	Compute the fourth-order gravity energy correction.
refresh_equilibrium_residual	pass(self)	Recompute the equilibrium residual if stale.
check_nonfinite	pass(self)	Stop on a non-finite committed state.
copy_cpu_gpu	pass(self)	Copy state and topology from host to device.
copy_gpu_cpu	pass(self)	Copy state from device to host.
copy_phi_gpu	pass(self)	Copy the immersed solids distance function to the device.
refresh_potential	pass(self)	Set the gravitational potential and copy it to the device.
copy_potential_gpu	pass(self)	Copy the gravitational potential to the device.
solve_potential_gpu	pass(self)	Solve the self-gravity potential of a device state.
destroy	pass(self)	Free device and host data.
initialize_flume	pass(self)	Initialize the FNL backend.
limit_positivity_dev	pass(self)	Apply the positivity limiter to the stage face fluxes.
seam_lists	pass(self)	Host lists of the seam skin cells and faces (issue #50).
seam_sync_blend_dev	pass(self)	Set the seam face fluxes with the seam factors (issue #50).
seam_sync_factors_dev	pass(self)	Recompute the seam cells' factors (issue #50).
seam_sync_fluxes_dev	pass(self)	Publish the donor states and the fine means (issue #50).
seam_sync_theta_dev	pass(self)	Compute the seam factors (issue #50).
save_residuals	pass(self)	Save residuals history.
save_simulation_data	pass(self)	Save fields, restart and diagnostics on their cadence.
set_boundary_conditions	pass(self)	Set boundary conditions on the device crown maps.
build_bc_pass_rows	pass(self)	Group the BC crown map rows by pass (issue #90).
update_ghost	pass(self)	Update ghost cells: local, MPI, boundary conditions.
advance_one_step_forest	pass(self)	Advance one full step (fast path).
after_topology_build_forest	pass(self)	Copy the forest-built seam and BC maps to the device.
apply_reflux_to_stage_forest	pass(self)	Apply the reflux correction.
begin_stage_forest	pass(self)	Begin an integrator stage (staged path).
close_step_forest	pass(self)	Close a step (staged path).
compute_local_dt_forest	pass(self)	Compute the local stability-limited time step.
end_stage_forest	pass(self)	End an integrator stage (staged path).
fill_seam_from_peer_forest	pass(self)	Fill inter-realm seam ghosts from a peer.
pack_seam_cells_forest	pass(self)	Pack own cells of the cross-rank seam send rows.
unpack_seam_cells_forest	pass(self)	Unpack the cross-rank seam receive rows into own ghosts.
finalize_forest	pass(self)	Finalize the realm.
finalize_mpi_forest	pass(self)	Finalize the FNL MPI handler.
initialize_forest	pass(self)	Initialize the realm.
is_done_forest	pass(self)	Return true if the realm is done.
open_step_forest	pass(self)	Open a step (staged path).
post_step_forest	pass(self)	Post-step IO and diagnostics.
regrid_forest	pass(self)	Regrid on the [amr] frequency cadence (issue #74).
stages_per_step_forest	pass(self)	Return the integrator stages per step.

Subroutines

accumulate_seam_fluxes

Accumulate the seam face fluxes of stage s, weighted by its SSP coefficient, into the forest's flux register.

The register is host-side: the skins of all the seam faces of the rank are packed on the device in one launch and copied to the host in one transfer, then each is routed by the shared accumulate_seam_skin, as on the CPU.

subroutine accumulate_seam_fluxes(self, s, flux_register, weight, with_gravity_work)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
s	integer(kind=I4P)	in		Runge-Kutta stage.
flux_register	class(flux_register_object)	inout		Forest's flux register.
weight	real(kind=R8P)	in	optional	Weight replacing beta(s) (D_flux, #68 A2).
with_gravity_work	logical	in	optional	Add the seam gravity work (default:

Call graph

flag_troubled_seam_cells

Flag the troubled seam face cells of one stage for the reflux policy troubled (issue #68 E1b), the FNL twin of the CPU flagging: the 6 cells across every owned register face cell (the coarse skin cell, two inside it, three ghosts outside) are gathered from q_aux_gpu to the host (density, velocity, pressure), where the same seam_weno5_sensor and the limiter's host seam factors (issue #50) flag them. Seam cells only: a small gather.

subroutine flag_troubled_seam_cells(self, flux_register)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
flux_register	class(flux_register_object)	inout		Forest's flux register.

Call graph

add_dissipative_fluxes_dev

Add the dissipative face fluxes of the active directions to the inviscid ones (issue #65), from the auxiliary variables of the stage, on the device: the viscous and heat fluxes when a viscosity or a conductivity is on, the Ohmic ones (MHD) when the resistivity is; the kernel of the order is selected here, never inside the loops.

subroutine add_dissipative_fluxes_dev(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

apply_floors

Apply the MHD positivity floors to the interior of a stage state, before its ghost exchange (issue #41, 3.8).

Euler has no floors (return before any work). MHD: the floored cells of the stage are logged by rank 0 when any; a non-positive density or pressure with the floors disabled (both zero) is fatal, reported with the global minimum density and pressure.

subroutine apply_floors(self, q_gpu)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
q_gpu	real(kind=R8P)	inout		Conservative variables.

Call graph

allocate_gpu

Allocate device data (every dev_alloc checked) and the host staging buffer.

subroutine allocate_gpu(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

check_nonfinite

Stop the run when the committed state holds a non-finite (NaN or infinite) value (issue #45): a run could otherwise finish with NaN fields and exit 0. One device pass over the interior cells, the model selected outside the kernels; the state is copied to the host only to locate a failure.

subroutine check_nonfinite(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

check_glm_ch

Check the GLM cleaning speed against the fastest wave of the committed state (issue #41, section 3.5); a no-op without GLM.

subroutine check_glm_ch(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

compute_conservation

Compute the volume integrals of the conservative variables on the device and save them on their cadence; with self-gravity (issue #81) the row gains W = 1/2 sum rho Phi dV and E + W, Phi solved from the committed state.

subroutine compute_conservation(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

compute_divb_history

Compute and save the div(B) norms of the committed state on the diagnostics cadence (issue #41, D-10); a no-op for Euler. The ghost cells are refreshed first and, with sibling realms, the inter-realm seam ghosts are refilled from the peers (update_ghost does not fill them, issue #31): the stencils of the seam-local cells read them. The seam faces flags are rebuilt and copied to the device at every call (6 nb integers): at step 0 the forest may not have registered the seams yet.

subroutine compute_divb_history(self, realm)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
realm	class(realm_object)	inout	target, optional	Sibling realms.

Call graph

compute_dflux_norms

Compute the D_flux norms of the committed state (issue #68 A2), the device twin of the CPU procedure: the committed state is copied into the idle first RK stage buffer and its residual evaluated into the second, the seam face fluxes accumulated with weight 1 into the realm's copy of the forest's flux register (host side) and completed across the ranks; the reflux correction of the coarse skin cells of the intra-realm 2:1 faces this rank owns is built on the host over the blocks in use and copied to the device, where compute_dflux_norms_dev reduces the norms. The caller has refreshed the ghost cells of q_gpu and copied the seam faces flags to the device.

subroutine compute_dflux_norms(self, norms)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
norms	real(kind=R8P)	out		The five norms of this rank.

Call graph

compute_q_aux

Compute the auxiliary variables on the device, ghost cells included.

subroutine compute_q_aux(self, q_gpu)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
q_gpu	real(kind=R8P)	in		Conservative variables.

Call graph

copy_cpu_gpu

Copy state and topology (coordinates, maps) from host to device.

bounded (the runtime regrid, issue #75) copies the state of the blocks in use only: the device blocks beyond blocks_number hold nothing the scheme reads. Otherwise the whole block capacity nb is copied (initialization: it also gives the unused device blocks finite values).

subroutine copy_cpu_gpu(self, verbose, bounded)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
verbose	logical	in	optional	Trigger verbose output.
bounded	logical	in	optional	Copy the blocks in use only.

Call graph

copy_gpu_cpu

Copy state and residuals from device to host, the blocks in use only (issue #75: the outputs, the restart files and the non-finite report read nothing beyond blocks_number).

subroutine copy_gpu_cpu(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

copy_phi_gpu

Copy the immersed solids distance function (computed on the host, the solids are static) to the device, transposed from (s, i, j, k, b) to (b, i, j, k, s); the staging buffer has the extent of the device array (issue #31). bounded (the runtime regrid, issue #75) copies the blocks in use only.

subroutine copy_phi_gpu(self, bounded)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
bounded	logical	in	optional	Copy the blocks in use only.

Call graph

refresh_potential

Set the gravitational potential of the current grid on the host (set_potential, issue #81) and copy it to the device, transposed from (i, j, k, b) to (b, i, j, k, 1); the device array is reallocated to the host extent (the block capacity may change at a regrid). Without a potential both are one-element placeholders, never zero bytes (issue #82). Call it after the state is on the device and after every regrid.

subroutine refresh_potential(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

copy_potential_gpu

Copy the host gravitational potential to the device, transposed from (i, j, k, b) to (b, i, j, k, 1); the device array is reallocated to the host extent.

subroutine copy_potential_gpu(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

solve_potential_gpu

Solve the self-gravity potential of the device state q_gpu (issue #81, M8-P3): a device kernel packs the interior density, the host gathers it to rank 0 through the library solver (cuFFT on rank 0's GPU), and the potential, every ghost layer filled, comes back to phi_grav and, for the active blocks only, to the device (unpack_potential_dev); on an AMR grid the multigrid solver runs on the host, its coarsest level on cuFFT (P5b). Optionally the energy sum_cells rho Phi dV (every rank). The density and the potential cross the host every stage (gather to rank 0).

Only the active blocks cross the bus: phi_grav is sized to the block capacity (23270 blocks against 16 in use on an A30 at 32^2), and copying it whole every stage cost 0.73 s per solve, 99.6% of the solve.

subroutine solve_potential_gpu(self, q_gpu, energy)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
q_gpu	real(kind=R8P)	in		Device state (b, i, j, k, v).
energy	real(kind=R8P)	out	optional	sum_cells rho Phi dV.

Call graph

destroy

Free device and host data: own device buffers, the FNL helpers (library teardown) and the common data.

subroutine destroy(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

initialize_flume

Initialize the FNL backend: MPI and device (once per process), device budget, common data, FNL helpers, device data, dispatch.

subroutine initialize_flume(self, filename, realms_number)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout	target	The equation.
filename	character(len=*)	in		Input file name.
realms_number	integer(kind=I4P)	in	optional	Realm count; divides the device budget.

Call graph

limit_positivity_dev

Apply the positivity limiter to the device face fluxes of the stage (issue #47, D-9; device twin of the CPU limit_positivity): the model's factors kernel, the ghost exchange of the factors on the device (intra-realm copies and GPU-direct MPI only: a physical-boundary ghost keeps the factor 1 it was allocated with), the blending of the active directions; the counters are reduced over the ranks and logged. With 2:1 AMR seams the seam flux is synchronised as on the CPU (issue #50, adam_flume_seam_sync_object): the seam passes run on the host over the seam cells, fed by device gathers, and scatter their factors and face fluxes back.

subroutine limit_positivity_dev(self, q_gpu, flux_register)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
q_gpu	real(kind=R8P)	in		Conservative variables of the stage.
flux_register	class(flux_register_object)	in	optional	Forest's flux register.

Call graph

seam_lists

Host lists of this rank's seam skin cells (issue #50): coarse (or fine) selects the coarse (fine) seam faces; each entry has its interior cell (i, j, k, b), its face (axis, i, j, k, b) in the flux arrays' indexing and its skin destination (cdst: skin cell of the coarse skins; fdst: compact fine store, fcc: the coarse skin cell covering it, fw: its weight in the coarse mean, 1/4, or 1/2 along the unrefined z of a quadtree, issue #46). The order is block, face, skin cell, as on the CPU.

subroutine seam_lists(self, coarse, ccell, cface, cdst, fine, fcell, fface, fdst, fcc, fw)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	in		The equation.
coarse	logical	in		Lists to build.
ccell	integer(kind=I4P)	out	allocatable	Coarse cells [4, n], faces [5, n].
cface	integer(kind=I4P)	out	allocatable	Coarse cells [4, n], faces [5, n].
cdst	integer(kind=I4P)	out	allocatable	Coarse skin destinations.
fine	logical	in		Lists to build.
fcell	integer(kind=I4P)	out	allocatable	Fine cells [4, n], faces [5, n].
fface	integer(kind=I4P)	out	allocatable	Fine cells [4, n], faces [5, n].
fdst	integer(kind=I4P)	out	allocatable	Fine store slots, covering coarse skin cells.
fcc	integer(kind=I4P)	out	allocatable	Fine store slots, covering coarse skin cells.
fw	real(kind=R8P)	out	allocatable, optional	Fine weights in the coarse means.

Call graph

seam_sync_fluxes_dev

Seam synchronisation, first phase (issue #50; host twin of the CPU seam_sync_fluxes, fed by device gathers).

subroutine seam_sync_fluxes_dev(self, q_gpu)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
q_gpu	real(kind=R8P)	in		Conservative variables of the stage.

Call graph

seam_sync_factors_dev

Seam synchronisation, second phase (issue #50; host twin of the CPU seam_sync_factors): gather the seam cells' stencils, auxiliary variables and face fluxes, recompute their factors with the seam faces overridden on the host, scatter them to the device factors.

subroutine seam_sync_factors_dev(self, q_gpu, dbad, dnonfinite)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
q_gpu	real(kind=R8P)	in		Conservative variables of the stage.
dbad	integer(kind=I4P)	inout		Inadmissible backbones count.
dnonfinite	integer(kind=I4P)	inout		Non-finite cells count.

Call graph

seam_sync_theta_dev

Seam synchronisation, third phase (issue #50; host twin of the CPU seam_sync_theta).

subroutine seam_sync_theta_dev(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

seam_sync_blend_dev

Seam synchronisation, last phase (issue #50; host twin of the CPU seam_sync_blend): the seam face fluxes with the seam factor, scattered to the device face fluxes.

subroutine seam_sync_blend_dev(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

save_residuals

Save residuals history (L2 norm of dq on the device, MPI-reduced, rank 0 writes).

subroutine save_residuals(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

save_simulation_data

Save fields, restart, slices and conservation history, each on its own cadence; state copied to host only when saved.

subroutine save_simulation_data(self, realm)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
realm	class(realm_object)	inout	target, optional	Sibling realms.

Call graph

set_boundary_conditions

Set boundary conditions on the device crown maps: three passes (rows beyond one, two, three realm faces), crown by crown within each (see the CPU set_boundary_conditions, issue #65 P0).

subroutine set_boundary_conditions(self, q_gpu)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
q_gpu	real(kind=R8P)	inout		Conservative variables.

Call graph

build_bc_pass_rows

Group the rows of the BC crown map by pass, per crown, for set_boundary_conditions (issue #90, F-A'): pass p (rows beyond p realm faces: 1 faces, 2 edges, 3 corners) of crown c is bc_rows_gpu(r1:r2,c), with [r1, r2] = bc_pass_range(:,p,c) (empty when r2 < r1), so each launch visits its own rows only. The rows are the live ones (block > 0) in map order; built from the host map, after every copy of the device map.

subroutine build_bc_pass_rows(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

update_ghost

Update ghost cells on the device: intra-realm local copies, GPU-direct MPI exchange, boundary conditions, then the positivity blend of the inadmissible face ghosts (issue #50, D2; see the CPU update_ghost).

subroutine update_ghost(self, q_gpu)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
q_gpu	real(kind=R8P)	inout		Conservative variables.

Call graph

advance_one_step_forest

Advance one full step of size dt (fast path: single realm, no AMR seam faces).

subroutine advance_one_step_forest(self, dt)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
dt	real(kind=R8P)	in		Time step from the forest's global reduction.

Call graph

after_topology_build_forest

Copy the host maps the forest built after the realm initialization to the device: the inter-realm seam map and buffers (maps%seam_local_*) and the BC crown map, whose seam rows the forest rewrote to BC_SEAM (issue #37; without it the seam fill kernel reads an unset device map and the BC kernel extrapolates over the seam ghosts).

subroutine after_topology_build_forest(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

apply_reflux_to_stage_forest

Apply the Berger-Colella reflux correction to the committed q_gpu (the forest calls it once per step, after close_step_forest, with the final stage); device twin of the CPU apply.

For every register face whose coarse side this realm and this rank own, the host mismatch slab F_coarse - F_fine_sum (step fluxes, sum_s beta_s F_s) is copied to the device and added, scaled by sgn dt / dx_coarse, to the coarse skin cells.

The scale uses the step the update used, time%dt: on the last step of a time-driven run the realm caps it to land on time_max, and the forest's dt argument is the uncapped value (with it the correction was off by the ratio of the two, 1.3e-9 of the mass on sod-amr; issue #37).

subroutine apply_reflux_to_stage_forest(self, stage, dt, flux_register)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
stage	integer(kind=I4P)	in		Integrator stage.
dt	real(kind=R8P)	in		Forest time step (unused: see the scale below).
flux_register	class(flux_register_object)	in		Forest's flux register.

Call graph

begin_stage_forest

Begin integrator stage k (staged path): publish the stage and compute its state.

subroutine begin_stage_forest(self, k, K_total, dt, realm)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
k	integer(kind=I4P)	in		Stage index (1..K_total).
K_total	integer(kind=I4P)	in		Forest-wide stage count for this step.
dt	real(kind=R8P)	in		Time step from the forest.
realm	class(realm_object)	inout	target, optional	Sibling realms (contract parity).

Call graph

close_step_forest

Close a step (staged path): assemble q, save residuals, advance time, clear the active stage.

subroutine close_step_forest(self, dt)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
dt	real(kind=R8P)	in		Time step from the forest (the local capped value is time%dt).

Call graph

compute_local_dt_forest

Compute the local stability-limited time step on the device, dt = CFL / max(sum_d (|u_d| + a) / dx_d); with GLM, also dt <= CFL / (c_h max sum_d 1 / dx_d) (glm_lambda, host data, issue #41, section 3.5); with dissipative terms the diffusive part 2 nu sum_d 1 / dx_d^2 joins the cell sum (issue #65, D-M4-3, as on the CPU).

subroutine compute_local_dt_forest(self, dt_local)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	in		The equation.
dt_local	real(kind=R8P)	out		Local stability-limited time step.

Call graph

end_stage_forest

End integrator stage k (staged path): residuals on the stage state, then stage assignment.

subroutine end_stage_forest(self, k, K_total, dt, realm, flux_register)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
k	integer(kind=I4P)	in		Stage index (1..K_total).
K_total	integer(kind=I4P)	in		Forest-wide stage count.
dt	real(kind=R8P)	in		Time step from the forest.
realm	class(realm_object)	inout	target, optional	Sibling realms (parity only).
flux_register	class(flux_register_object)	inout	optional	Forest's flux register.

Call graph

reflux_stage

The per-stage reflux ([numerics] reflux_cadence = stage, issue #81 P5c; FNL twin of the CPU reflux_stage): the seam fluxes of stage s (with the seam gravity work) into the realm's register copy with weight 1, completed across the ranks; each coarse seam face's mismatch F_coarse - F_fine goes to the device and corrects the stage residual dq_gpu by sgn / H (apply_reflux_face_dev). The end-of-step reflux is then skipped.

subroutine reflux_stage(self, s)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
s	integer(kind=I4P)	in		Runge-Kutta stage.

Call graph

fill_seam_from_peer_forest

Fill this realm's seam ghosts for peer slot p_idx from the peer's interior, on the active device buffers (q_gpu when stage_active == 0, else the active stage of q_rk_gpu) of both realms.

subroutine fill_seam_from_peer_forest(self, peer, p_idx)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
peer	class(realm_object)	in	target	Peer realm.
p_idx	integer(kind=I4P)	in		Peer slot.

Call graph

pack_seam_cells_forest

Pack the ghost values of the seam send rows of peer slot p_idx from the active device buffer (q_gpu when stage_active == 0, else the active stage of q_rk_gpu) into the host buf, nv values per row: cell copies, or the 2:1 interpolations and restrictions of a refined seam (issues #40, #52).

subroutine pack_seam_cells_forest(self, p_idx, buf)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	in		The equation.
p_idx	integer(kind=I4P)	in		Peer slot (the realm owning the ghosts).
buf	real(kind=R8P)	out		Packed values.

Call graph

unpack_seam_cells_forest

Unpack the host buf into this realm's seam ghosts of the cross-rank receive rows of peer slot p_idx, on the active device buffer, nv values per row (issue #40).

subroutine unpack_seam_cells_forest(self, p_idx, buf)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
p_idx	integer(kind=I4P)	in		Peer slot (the realm owning the cells).
buf	real(kind=R8P)	in		Packed values.

Call graph

finalize_forest

Finalize the realm: close the output files and free device and host data (MPI is finalized once by the forest).

subroutine finalize_forest(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

finalize_mpi_forest

Finalize the FNL MPI handler (called once by the forest after every realm is finalized).

subroutine finalize_mpi_forest(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation (carries no MPI state).

Call graph

initialize_forest

Initialize the realm (issue #35, section 6.1): backend init, IC (or restart) and initial AMR on the host, device topology and state sync, ghost update, initial output, output files open, AMR lock.

subroutine initialize_forest(self, filename, realms_number, memory_avail, nv, verbose)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
filename	character(len=*)	in		Input file name.
realms_number	integer(kind=I4P)	in	optional	Realm count; divides the device budget.
memory_avail	real(kind=R8P)	in	optional	Unused: the budget comes from the device.
nv	integer(kind=I4P)	in	optional	Unused: nv is decided by the physics.
verbose	logical	in	optional	Unused: initialization is always verbose.

Call graph

is_done_forest

Return true if the realm has reached its end.

subroutine is_done_forest(self, done)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	in		The equation.
done	logical	out		True if the realm is done.

Call graph

open_step_forest

Open a step (staged path): time bookkeeping and Runge-Kutta stages initialization.

subroutine open_step_forest(self, dt)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
dt	real(kind=R8P)	in		Time step from the forest.

Call graph

post_step_forest

Post-step work: the non-finite state check, the GLM c_h check, the div(B) history (MHD), fields, restart and conservation history on their cadence.

subroutine post_step_forest(self, dt, t, it, do_save_state, do_save_residuals, do_save_restart, do_amr, realm)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
dt	real(kind=R8P)	in		Time step just advanced.
t	real(kind=R8P)	in		Time after the advance.
it	integer(kind=I4P)	in		Iteration after the advance.
do_save_state	logical	in	optional	Unused: cadence is internal.
do_save_residuals	logical	in	optional	Unused: cadence is internal.
do_save_restart	logical	in	optional	Unused: cadence is internal.
do_amr	logical	in	optional	Unused: the forest regrids (regrid_forest).
realm	class(realm_object)	inout	target, optional	Sibling realms.

Call graph

regrid_forest

Regrid the realm on the cadence of [amr] frequency (issue #74, M5-P3); the forest calls it after every step, before post_step, on the committed state.

A host round trip (D-M5-7): the regrid machinery (markers, tree, transfers, redistribution, maps) is the host library's, so each sweep refreshes the ghost cells on the device (the markers and the prolongation read them), copies q to the host, marks (mark_runtime) and regrids there (adam_object%amr_update with [amr] regrid_prolongation), and, on a changed grid, recomputes the solids' distance function and copies the state, the block coordinates, the maps (a map the new grid no longer has is freed on the device, issue #74 G6) and the distance function back to the device. Then, as on the CPU, the new state is checked and its ghosts and auxiliary variables refreshed. The device arrays are sized by the block capacity nb, so nothing is reallocated; the state and distance function copies move the blocks in use only, packed and transposed on the device (issue #75: the whole capacity cost 2.3-3.1 s per regrid on the development box, whatever the grid). The wall time of the regrid (largest over the ranks) is logged with it. regridded tells the forest to rebuild the 2:1 flux register (host side); the grid is replicated, so every rank returns the same value.

subroutine regrid_forest(self, regridded)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
regridded	logical	out		True if the grid changed.

Call graph

copy_blocks_transposed

Copy blocks 1:blocks_number of a field between the host a(v, i, j, k, b) and the device a_gpu(b, i, j, k, v), ghost cells included (issue #75). The blocks in use are a strided section of a_gpu (the block is its leading index) but a contiguous slab of a: a device kernel packs (or unpacks) them, transposed, into a scratch buffer of the slab's shape, moved by one contiguous copy. The buffer has the extent of the slab, the host side is the slab itself: no staging buffer of another extent (issue #31). A rank without blocks copies nothing.

subroutine copy_blocks_transposed(blocks_number, ni, nj, nk, ngc, nv, a, a_gpu, to_device)

Arguments

Name	Type	Intent	Attributes	Description
blocks_number	integer(kind=I4P)	in		Blocks to copy.
ni	integer(kind=I4P)	in		Block dimensions, ghosts.
nj	integer(kind=I4P)	in		Block dimensions, ghosts.
nk	integer(kind=I4P)	in		Block dimensions, ghosts.
ngc	integer(kind=I4P)	in		Block dimensions, ghosts.
nv	integer(kind=I4P)	in		Variables number.
a	real(kind=R8P)	inout	target, contiguous	Host field (v, i, j, k, b).
a_gpu	real(kind=R8P)	in	pointer	Device field (b, i, j, k, v).
to_device	logical	in		Host to device, else back.

Call graph

refresh_equilibrium_residual

Recompute the residual of the hydrostatic equilibrium when the grid it describes is gone (issue #73, M7-G2b; device twin of the CPU procedure): the equilibrium is set on the host (ic%set_equilibrium) and copied into a scratch device field, its residual evaluated into dq_e_gpu; at AMR 2:1 seams the equilibrium's seam fluxes are accumulated with weight 1 into the realm's copy of the register, completed across the ranks, and their mismatch stored per face (seam_e) for the reflux to subtract.

subroutine refresh_equilibrium_residual(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

compute_gravity_correction

Compute the fourth-order correction of the gravity energy source into gcor_gpu (issue #73, M7-G2a; device twin of the CPU compute_gravity_correction): the faces of the block, then the face beyond each end of every direction with gravity, by the edge kernel of the space operator's face, selected here outside the kernels.

subroutine compute_gravity_correction(self, q_gpu)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
q_gpu	real(kind=R8P)	in		Conservative variables.

Call graph

compute_residuals_weno_dev

Compute the residuals with the WENO space operator on the device: ghost update, auxiliary variables, face fluxes of the active directions, flux difference.

The face fluxes of a null direction are never computed: they keep their zero initialization (dev_alloc). On the staged path (AMR seam faces), the seam face fluxes of every stage are accumulated into the forest's flux register.

subroutine compute_residuals_weno_dev(self, q_gpu, dq_gpu, s, flux_register)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
q_gpu	real(kind=R8P)	inout		Conservative variables.
dq_gpu	real(kind=R8P)	inout		Residuals.
s	integer(kind=I4P)	in	optional	Runge-Kutta stage.
flux_register	class(flux_register_object)	inout	optional	Forest's flux register for reflux.

Call graph

compute_residuals_riemann_dev

Compute the residuals with the weno-riemann space operator on the device (issue #47): as compute_residuals_weno_dev, with the face fluxes of the WENO interpolation, the Riemann solver and the high-order correction. The host selects the kernel of the (model, solver) pair; a pair without kernels is fatal.

subroutine compute_residuals_riemann_dev(self, q_gpu, dq_gpu, s, flux_register)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
q_gpu	real(kind=R8P)	inout		Conservative variables.
dq_gpu	real(kind=R8P)	inout		Residuals.
s	integer(kind=I4P)	in	optional	Runge-Kutta stage.
flux_register	class(flux_register_object)	inout	optional	Forest's flux register for reflux.

Call graph

integrate_rk_ls_dev

Integrate one time step with a low-storage Runge-Kutta scheme on the device.

subroutine integrate_rk_ls_dev(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

integrate_rk_ssp_dev

Integrate one time step with a strong stability preserving Runge-Kutta scheme on the device.

subroutine integrate_rk_ssp_dev(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

rk_assign_stage

Assign the residual of stage s to the Runge-Kutta stage buffer; the solid cells are masked with immersed solids.

subroutine rk_assign_stage(self, s)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
s	integer(kind=I4P)	in		Stage.

Call graph

rk_compute_stage

Compute the state of stage s; the solid cells are masked with immersed solids.

subroutine rk_compute_stage(self, s)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
s	integer(kind=I4P)	in		Stage.

Call graph

rk_compute_stage_ls

Advance low-storage stage s; the solid cells are masked with immersed solids.

subroutine rk_compute_stage_ls(self, s)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.
s	integer(kind=I4P)	in		Stage.

Call graph

rk_update_q

Assemble the committed state of a strong stability preserving step; the solid cells are masked with immersed solids.

subroutine rk_update_q(self)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	inout		The equation.

Call graph

seam_gather_cells

Gather on the host out(v, m) = the first nv components of the device field a_gpu at cells(:, m) (issue #50).

subroutine seam_gather_cells(cells, nv, ngc, a_gpu, out)

Arguments

Name	Type	Intent	Attributes	Description
cells	integer(kind=I4P)	in		Cells (i, j, k, b) [4, n].
nv	integer(kind=I4P)	in		Components.
ngc	integer(kind=I4P)	in		Ghost cells number.
a_gpu	real(kind=R8P)	in		Device field.
out	real(kind=R8P)	inout		Gathered values [nv, n].

Call graph

seam_gather_stencils

Gather on the host the axis stencils of half width s around cells(:, m) (issue #50).

subroutine seam_gather_stencils(cells, nv, ngc, s, q_gpu, out)

Arguments

Name	Type	Intent	Attributes	Description
cells	integer(kind=I4P)	in		Cells (i, j, k, b) [4, n].
nv	integer(kind=I4P)	in		Variables number.
ngc	integer(kind=I4P)	in		Ghost cells number.
s	integer(kind=I4P)	in		Stencil half width.
q_gpu	real(kind=R8P)	in		Device conservative variables.
out	real(kind=R8P)	inout		Stencils [nv, 2 s + 1, 3, n].

Call graph

seam_gather_faces

Gather on the host the device face fluxes at faces(:, m) = (axis, i, j, k, b) (issue #50).

subroutine seam_gather_faces(faces, nv, flx_f_gpu, fly_f_gpu, flz_f_gpu, out)

Arguments

Name	Type	Intent	Attributes	Description
faces	integer(kind=I4P)	in		Faces [5, n].
nv	integer(kind=I4P)	in		Variables number.
flx_f_gpu	real(kind=R8P)	in		X-face fluxes.
fly_f_gpu	real(kind=R8P)	in		Y-face fluxes.
flz_f_gpu	real(kind=R8P)	in		Z-face fluxes.
out	real(kind=R8P)	inout		Gathered fluxes [nv, n].

Call graph

seam_scatter_faces

Set the device face fluxes at faces(:, m) = (axis, i, j, k, b) to the host in(:, m) (issue #50).

subroutine seam_scatter_faces(faces, nv, in, flx_f_gpu, fly_f_gpu, flz_f_gpu)

Arguments

Name	Type	Intent	Attributes	Description
faces	integer(kind=I4P)	in		Faces [5, n].
nv	integer(kind=I4P)	in		Variables number.
in	real(kind=R8P)	in		Fluxes [nv, n].
flx_f_gpu	real(kind=R8P)	inout		X-face fluxes.
fly_f_gpu	real(kind=R8P)	inout		Y-face fluxes.
flz_f_gpu	real(kind=R8P)	inout		Z-face fluxes.

Call graph

seam_scatter_cells

Set the first nv components of the device field a_gpu at cells(:, m) to the host in(:, m) (issue #50).

subroutine seam_scatter_cells(cells, nv, ngc, in, a_gpu)

Arguments

Name	Type	Intent	Attributes	Description
cells	integer(kind=I4P)	in		Cells (i, j, k, b) [4, n].
nv	integer(kind=I4P)	in		Components.
ngc	integer(kind=I4P)	in		Ghost cells number.
in	real(kind=R8P)	in		Values [nv, n].
a_gpu	real(kind=R8P)	inout		Device field.

Call graph

Functions

stages_per_step_forest

Return the integrator stages per step: only SSP schemes are stage-splittable (staged path).

Returns: integer(kind=I4P)

function stages_per_step_forest(self) result(K)

Arguments

Name	Type	Intent	Attributes	Description
self	class(flume_fnl_object)	in		The equation.

Call graph