Appearance
adam_ripple_fnl_object
RIPPLE, the FNL (OpenACC) backend realm.
The CPU backend's twin with the state on the device: a sibling of ripple_cpu_object, extending the same common realm. The initial condition and the initial AMR run on the host, then the state goes to the device and stays there; it comes back only for the outputs. The library's FNL helpers do the device side of the library objects: field_fnl the ghost exchange (local copies, 2:1 transfers, GPU-direct MPI), rk_fnl the Runge-Kutta stages, weno_fnl the WENO coefficients. Device arrays are transposed, (b, i, j, k, v).
Source: src/app/ripple/fnl/adam_ripple_fnl_object.F90
Dependencies
Contents
- ripple_fnl_object
- allocate_gpu
- copy_state
- update_ghost
- compute_residuals
- accumulate_seam_fluxes
- initialize_forest
- advance_one_step_forest
- post_step_forest
- finalize_forest
- finalize_mpi_forest
- open_step_forest
- begin_stage_forest
- end_stage_forest
- close_step_forest
- apply_reflux_to_stage_forest
- compute_total
Derived Types
ripple_fnl_object
RIPPLE FNL realm.
Inheritance
Extends: ripple_common_object
Components
| Name | Type | Attributes | Description |
|---|---|---|---|
io | type(io_object) | IO handler. | |
amr | type(amr_object) | AMR marker handler. | |
slices | type(slices_object) | Slices handler. | |
blanesmoan | type(blanesmoan_object) | Blanes-Moan integrator. | |
cfm | type(cfm_object) | Commutator-Free Magnus integrator. | |
leapfrog | type(leapfrog_object) | Leapfrog integrator. | |
flail | type(flail_object) | FLAIL, the elliptic solver library (issue #84). | |
weno | type(weno_object) | WENO reconstructor. | |
ib | type(ib_object) | Immersed boundary. | |
rk | type(rk_object) | Runge-Kutta integrator. | |
adam | type(adam_object) | ADAM, grid + tree + field + maps orchestrator. | |
fdv_scheme | character(len=:) | allocatable | FDV scheme, fd/fv. |
fdv_order | integer(kind=I4P) | Order of finite difference/volume schemes, general order. | |
fdv_half_stencil | integer(kind=I4P) | Half stencil length of finite difference/volume schemes. | |
fdv_half_stencils | integer(kind=I4P) | Half stencil length of fdv schemes for each derivative up to 6. | |
ngc | integer(kind=I4P) | pointer | Number of ghost cells. |
ni | integer(kind=I4P) | pointer | Number of cells in i direction. |
nj | integer(kind=I4P) | pointer | Number of cells in j direction. |
nk | integer(kind=I4P) | pointer | Number of cells in k direction. |
nb | integer(kind=I4P) | pointer | Total blocks number for MPI. |
blocks_number | integer(kind=I4P) | pointer | Actual blocks number. |
nv | integer(kind=I4P) | pointer | Number of variables in q vector. |
realm_index | integer(kind=I4P) | Inter-realm seam coupling — currently-published stage buffer key. | |
stage_active | integer(kind=I4P) | A copy of the forest's flux register (its faces and lookup, the accumulators unused), refreshed by the forest each | |
flux_register_copy | type(flux_register_object) | Procedure pointer TBPs for FDV operators (set at initialization by backend). | |
compute_block_total_variation | procedure(compute_block_total_variation_interface) | pass(self), pointer | Compute TV. |
compute_curl | procedure(compute_curl_interface) | pass(self), pointer | Compute curl. |
compute_curl_extended | procedure(compute_curl_interface) | pass(self), pointer | Compute curl on extended support. |
compute_derivative1 | procedure(compute_derivative1_interface) | pass(self), pointer | Compute deriv1. |
compute_derivative2 | procedure(compute_derivative2_interface) | pass(self), pointer | Compute deriv2. |
compute_derivative4 | procedure(compute_derivative4_interface) | pass(self), pointer | Compute deriv4. |
compute_divergence | procedure(compute_divergence_interface) | pass(self), pointer | Compute dive. |
compute_gradient | procedure(compute_gradient_interface) | pass(self), pointer | Compute grad. |
compute_gradient_extended | procedure(compute_gradient_interface) | pass(self), pointer | Compute gradient on extended support. |
compute_laplacian | procedure(compute_laplacian_interface) | pass(self), pointer | Compute laplac. |
q | real(kind=R8P) | allocatable | State, point values u [1, 1-ngc:ni+ngc, ..., nb]. |
dq | real(kind=R8P) | allocatable | Residuals, same shape. |
a | real(kind=R8P) | Advection velocity. | |
cfl | real(kind=R8P) | CFL number. | |
time | real(kind=R8P) | Current time. | |
time_max | real(kind=R8P) | Final time. | |
dt | real(kind=R8P) | Current time step. | |
it | integer(kind=I4P) | Current iteration. | |
it_max | integer(kind=I4P) | Iterations number; <= 0: run to time_max. | |
history_unit | integer(kind=I4P) | Unit of the history file (rank 0). | |
amr_criteria | type(amr_criterion_pointer) | allocatable | Registered refinement criteria (issue #110). |
field_fnl | type(field_fnl_object) | Field helper: coordinates, maps, ghost exchange. | |
rk_fnl | type(rk_fnl_object) | Runge-Kutta helper: the stages on the device. | |
weno_fnl | type(weno_fnl_object) | WENO helper: the coefficients on the device. | |
q_gpu | real(kind=R8P) | pointer | State [nb, 1-ngc:ni+ngc, ..., 1]. |
dq_gpu | real(kind=R8P) | pointer | Residuals, same shape. |
flx_gpu | real(kind=R8P) | pointer | X-face fluxes [nb, 0:ni, 1:nj, 1:nk, 1]. |
fly_gpu | real(kind=R8P) | pointer | Y-face fluxes [nb, 1:ni, 0:nj, 1:nk, 1]. |
flz_gpu | real(kind=R8P) | pointer | Z-face fluxes [nb, 1:ni, 1:nj, 0:nk, 1]. |
face_gpu | real(kind=R8P) | pointer | One face's skin or reflux mismatch [1, largest face]. |
face | real(kind=R8P) | allocatable | Its host copy, the same extent. |
Type-Bound Procedures
| Name | Attributes | Description |
|---|---|---|
initialize | pass(self) | Initialize common data. |
load_fdv_from_file | pass(self) | Load FDV config from file. |
coupling_descriptor_forest | pass(self) | Report (scheme_time, rk_scheme, nv) for β admissibility. |
runtime_amr_forest | pass(self) | Whether this realm regrids during the run (#74). |
regrid_forest | pass(self) | Invoked by forest%regrid per realm per timestep (#74). |
regrid_begin_forest | pass(self) | Multi-realm regrid, phase 1: is it a regrid step (#110). |
regrid_mark_forest | pass(self) | Phase 2: mark one sweep, marks on the tree (#110). |
regrid_apply_forest | pass(self) | Phase 3: adapt to the (balanced) tree marks (#110). |
regrid_end_forest | pass(self) | Phase 4: refresh the grid-dependent state (#110). |
fill_seam_from_peer_forest | pass(self) | Receive-side roundtrip: copy peer's interior into self's ghosts. |
pack_seam_cells_forest | pass(self) | Pack own cells of the cross-rank seam send rows. |
unpack_seam_cells_forest | pass(self) | Unpack the cross-rank seam receive rows into own ghosts. |
after_topology_build_forest | pass(self) | Backend hook invoked once after the forest builds the seam maps. |
close_block_xh5f | nopass | Close XH5F file block. |
close_file_xh5f | nopass | Close XH5F file. |
open_block_xh5f | pass(self) | Open block file XH5F. |
open_file_xh5f | pass(self) | Open file XH5F. |
save_q_xh5f | pass(self) | Save in XH5F (XDMF/HDF5) format. |
compute_curl_fd_extended | pass(self) | Compute curl of vector field, finite difference, even for half gcs. |
compute_gradient_fd_extended | pass(self) | Compute gradient of scalar field, finite difference, even for half |
initialize_ripple | pass(self) | Initialize the common data. |
check_input | pass(self) | Refuse the inputs RIPPLE cannot honour. |
set_initial_conditions | pass(self) | Set the initial profile on the interior cells. |
amr_update | pass(self) | Initial AMR: box markers and registered criteria. |
register_amr_criterion | pass(self) | Register a refinement criterion by name (issue #110). |
save_output | pass(self) | Save one history row and the state (XH5F). |
advance_time | pass(self) | Count the step and cap dt on time_max. |
accumulate_seam_skin | pass(self) | Route one seam face skin to the flux register. |
apply_reflux | pass(self) | Berger-Colella correction of the coarse seam cells of q. |
compute_local_dt_forest | pass(self) | Local CFL time step. |
is_done_forest | pass(self) | True when the run is over. |
stages_per_step_forest | pass(self) | Runge-Kutta stages per step (staged path). |
allocate_gpu | pass(self) | Allocate the device data. |
copy_state | pass(self) | Copy q between host and device. |
update_ghost | pass(self) | Fill the ghost cells on the device. |
compute_residuals | pass(self) | Residuals of one device state. |
accumulate_seam_fluxes | pass(self) | Seam face fluxes of one stage to the flux register. |
compute_total | pass(self) | Total of u dV on this rank, from the device state. |
initialize_forest | pass(self) | Initialize the realm. |
advance_one_step_forest | pass(self) | Advance one whole step. |
post_step_forest | pass(self) | Save the outputs on their cadence. |
finalize_forest | pass(self) | Finalize the realm. |
finalize_mpi_forest | pass(self) | Finalize the FNL MPI handler. |
open_step_forest | pass(self) | Open a step. |
begin_stage_forest | pass(self) | Begin a stage. |
end_stage_forest | pass(self) | End a stage. |
close_step_forest | pass(self) | Close a step. |
apply_reflux_to_stage_forest | pass(self) | Reflux the committed step on the device. |
Subroutines
allocate_gpu
Allocate the device data, every dev_alloc checked.
fortran
subroutine allocate_gpu(self)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
self | class(ripple_fnl_object) | inout | The realm. |
Call graph
copy_state
Copy the blocks in use of the state between the host q(v, i, j, k, b) and the device q_gpu(b, i, j, k, v): a device kernel transposes them into a scratch buffer of the host slab's shape, moved by one contiguous copy.
fortran
subroutine copy_state(self, to_device)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
self | class(ripple_fnl_object) | inout | The realm. | |
to_device | logical | in | Host to device, else back. |
Call graph
update_ghost
Fill the ghost cells on the device: local copies and 2:1 transfers, then the GPU-direct MPI exchange.
fortran
subroutine update_ghost(self, q_gpu)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
self | class(ripple_fnl_object) | inout | The realm. | |
q_gpu | real(kind=R8P) | inout | State. |
Call graph
compute_residuals
Compute the residuals dq_gpu of the device state q_gpu (the CPU compute_residuals on the device).
fortran
subroutine compute_residuals(self, q_gpu, s, flux_register)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
self | class(ripple_fnl_object) | inout | The realm. | |
q_gpu | real(kind=R8P) | inout | State. | |
s | integer(kind=I4P) | in | optional | Runge-Kutta stage. |
flux_register | class(flux_register_object) | inout | optional | Forest's flux register. |
Call graph
accumulate_seam_fluxes
Pack the face flux skin of every registered seam face on the device, bring it to the host and route it to the (host) register, weighted by the SSP coefficient of stage s. One small copy per face: simple, not fast.
fortran
subroutine accumulate_seam_fluxes(self, s, flux_register)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
self | class(ripple_fnl_object) | inout | The realm. | |
s | integer(kind=I4P) | in | Runge-Kutta stage. | |
flux_register | class(flux_register_object) | inout | Forest's flux register. |
Call graph
initialize_forest
Initialize the realm: MPI and the device (once per process), the common data within the device budget, the FNL helpers, the device data; initial AMR and initial condition on the host, then the state to the device.
fortran
subroutine initialize_forest(self, filename, realms_number, memory_avail, nv, verbose)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
self | class(ripple_fnl_object) | inout | The realm. | |
filename | character(len=*) | in | Input file name. | |
realms_number | integer(kind=I4P) | in | optional | Unused: RIPPLE is single-realm. |
memory_avail | real(kind=R8P) | in | optional | Unused: the budget comes from the device. |
nv | integer(kind=I4P) | in | optional | Unused: nv = 1. |
verbose | logical | in | optional | Unused. |
Call graph
advance_one_step_forest
Advance one whole SSP Runge-Kutta step on the device (fast path: no 2:1 face).
fortran
subroutine advance_one_step_forest(self, dt)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
self | class(ripple_fnl_object) | inout | The realm. | |
dt | real(kind=R8P) | in | Time step from the forest. |
Call graph
post_step_forest
Append the history row every step (the total reduced on the device); bring q to the host and save the fields every [IO] it_save steps and at the end.
fortran
subroutine post_step_forest(self, dt, t, it, do_save_state, do_save_residuals, do_save_restart, do_amr, realm)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
self | class(ripple_fnl_object) | inout | The realm. | |
dt | real(kind=R8P) | in | Unused. | |
t | real(kind=R8P) | in | Unused. | |
it | integer(kind=I4P) | in | Unused. | |
do_save_state | logical | in | optional | Unused. |
do_save_residuals | logical | in | optional | Unused. |
do_save_restart | logical | in | optional | Unused. |
do_amr | logical | in | optional | Unused. |
realm | type(realm_pointer) | inout | target, optional | Unused. |
Call graph
finalize_forest
Finalize the realm: close the history, free the device data (the forest finalizes MPI once).
fortran
subroutine finalize_forest(self)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
self | class(ripple_fnl_object) | inout | The realm. |
Call graph
finalize_mpi_forest
Finalize the FNL MPI handler (the forest calls it once, after every realm is finalized).
fortran
subroutine finalize_mpi_forest(self)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
self | class(ripple_fnl_object) | inout | The realm (carries no MPI state). |
Call graph
open_step_forest
Open a step: count it, cap dt, copy q into every stage.
fortran
subroutine open_step_forest(self, dt)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
self | class(ripple_fnl_object) | inout | The realm. | |
dt | real(kind=R8P) | in | Time step from the forest. |
Call graph
begin_stage_forest
Begin stage k: publish it (stage_active) and build its state.
fortran
subroutine begin_stage_forest(self, k, K_total, dt, realm)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
self | class(ripple_fnl_object) | inout | The realm. | |
k | integer(kind=I4P) | in | Stage index. | |
K_total | integer(kind=I4P) | in | Unused. | |
dt | real(kind=R8P) | in | Unused: the capped dt is self%dt. | |
realm | type(realm_pointer) | inout | target, optional | Unused. |
Call graph
end_stage_forest
End stage k: its residuals, the seam fluxes to the register, the stage assignment.
fortran
subroutine end_stage_forest(self, k, K_total, dt, realm, flux_register)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
self | class(ripple_fnl_object) | inout | The realm. | |
k | integer(kind=I4P) | in | Stage index. | |
K_total | integer(kind=I4P) | in | Unused. | |
dt | real(kind=R8P) | in | Unused. | |
realm | type(realm_pointer) | inout | target, optional | Unused. |
flux_register | class(flux_register_object) | inout | optional | Forest's flux register. |
Call graph
close_step_forest
Close a step: commit q, advance the time, clear the active stage.
fortran
subroutine close_step_forest(self, dt)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
self | class(ripple_fnl_object) | inout | The realm. | |
dt | real(kind=R8P) | in |
Call graph
apply_reflux_to_stage_forest
Reflux the committed device state (the forest calls it after close_step_forest, with the last stage): the host mismatch F_coarse - F_fine of every coarse face this rank owns goes to the device and is added, scaled by sgn dt / dx, to its coarse skin cells.
fortran
subroutine apply_reflux_to_stage_forest(self, stage, dt, flux_register)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
self | class(ripple_fnl_object) | inout | The realm. | |
stage | integer(kind=I4P) | in | Stage: the correction acts at the last one. | |
dt | real(kind=R8P) | in | Unused: the capped dt is self%dt. | |
flux_register | class(flux_register_object) | in | Forest's flux register. |
Call graph
Functions
compute_total
Return the total of u dV over the interior cells of this rank's blocks, reduced on the device.
Returns: real(kind=R8P)
fortran
function compute_total(self) result(total)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
self | class(ripple_fnl_object) | in | The realm. |
Call graph