Appearance
adam_fnl_field_kernels
ADAM, field class FNL kernels (FNL backend of field_object).
Source: src/lib/fnl/adam_fnl_field_kernels.F90
Dependencies
Contents
- pack_blocks_transposed_dev
- unpack_blocks_transposed_dev
- pack_seam_rows_dev
- unpack_seam_rows_dev
- compute_q_gradient_dev
- compute_normL2_residuals_dev
- populate_send_buffer_ghost_gpu_dev
- receive_recv_buffer_ghost_gpu_dev
- update_ghost_local_gpu_dev
- reduce_ghost_local_gpu_dev
- pack_reduce_ghost_mpi_gpu_dev
- fold_reduce_ghost_mpi_gpu_dev
- interp_seam_ghost_gpu_dev
- restrict_seam_high_order_gpu_dev
Subroutines
pack_blocks_transposed_dev
Pack blocks 1:blocks_number of the device field a_gpu(b, i, j, k, v), ghost cells included, into buf_gpu in the host layout (v, i, j, k, b) (issue #75): buf_gpu is then one contiguous copy of the host slab a(:, :, :, :, 1:blocks_number), so a host round trip moves the blocks in use, not the block capacity nb (the leading device index is the block, so the blocks in use are a strided section of a_gpu).
fortran
subroutine pack_blocks_transposed_dev(blocks_number, ni, nj, nk, ngc, nv, a_gpu, buf_gpu)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
blocks_number | integer(kind=I4P) | in | Blocks to pack. | |
ni | integer(kind=I4P) | in | Block dimensions, ghost cells number. | |
nj | integer(kind=I4P) | in | Block dimensions, ghost cells number. | |
nk | integer(kind=I4P) | in | Block dimensions, ghost cells number. | |
ngc | integer(kind=I4P) | in | Block dimensions, ghost cells number. | |
nv | integer(kind=I4P) | in | Variables number. | |
a_gpu | real(kind=R8P) | in | Field (b, i, j, k, v). | |
buf_gpu | real(kind=R8P) | inout | Packed blocks (v, i, j, k, b). |
Call graph
unpack_blocks_transposed_dev
Unpack buf_gpu, blocks 1:blocks_number in the host layout (v, i, j, k, b), into the device field a_gpu(b, i, j, k, v), ghost cells included: the inverse of pack_blocks_transposed_dev (issue #75); the blocks beyond blocks_number are untouched.
fortran
subroutine unpack_blocks_transposed_dev(blocks_number, ni, nj, nk, ngc, nv, buf_gpu, a_gpu)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
blocks_number | integer(kind=I4P) | in | Blocks to unpack. | |
ni | integer(kind=I4P) | in | Block dimensions, ghost cells number. | |
nj | integer(kind=I4P) | in | Block dimensions, ghost cells number. | |
nk | integer(kind=I4P) | in | Block dimensions, ghost cells number. | |
ngc | integer(kind=I4P) | in | Block dimensions, ghost cells number. | |
nv | integer(kind=I4P) | in | Variables number. | |
buf_gpu | real(kind=R8P) | in | Packed blocks (v, i, j, k, b). | |
a_gpu | real(kind=R8P) | inout | Field (b, i, j, k, v). |
Call graph
pack_seam_rows_dev
Pack the ghost values of the seam send rows [rank, b, i, j, k, kind, meta] row_start..row_start+row_count-1 from q_gpu into buf_gpu, nv values per row in row order (issues #40, #52), the device twin of adam_seam_exchange%pack_seam_rows: kind 1 copies the cell, kind 4 interpolates the fine ghost around the coarse anchor cell (interp_seam_ghost_gpu_dev, the intra-realm flag-4 fill; injection copies the anchor), kind 8 is the mean of the 2x2x2 fine cells from the base cell (the intra-realm flag-8 order). The kinds are the values of adam_seam_exchange SEAM_ROW_*.
fortran
subroutine pack_seam_rows_dev(row_start, row_count, nv, ngc, regime, rows_gpu, q_gpu, buf_gpu)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
row_start | integer(kind=I4P) | in | First row. | |
row_count | integer(kind=I4P) | in | Number of rows. | |
nv | integer(kind=I4P) | in | Variables number. | |
ngc | integer(kind=I4P) | in | Ghost cells number. | |
regime | integer(kind=I4P) | in | Coarse->fine fill regime (maps%seam_ghost_fill). | |
rows_gpu | integer(kind=I4P) | in | Rows [rank, b, i, j, k, kind, meta]. | |
q_gpu | real(kind=R8P) | in | Field (b, i, j, k, v). | |
buf_gpu | real(kind=R8P) | inout | Packed values. |
Call graph
unpack_seam_rows_dev
Scatter buf_gpu (nv values per row in row order) into the cells of the cross-rank seam rows [rank, b, i, j, k] row_start..row_start+row_count-1 of q_gpu (issue #40).
fortran
subroutine unpack_seam_rows_dev(row_start, row_count, nv, ngc, rows_gpu, buf_gpu, q_gpu)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
row_start | integer(kind=I4P) | in | First row. | |
row_count | integer(kind=I4P) | in | Number of rows. | |
nv | integer(kind=I4P) | in | Variables number. | |
ngc | integer(kind=I4P) | in | Ghost cells number. | |
rows_gpu | integer(kind=I4P) | in | Rows [rank, b, i, j, k]. | |
buf_gpu | real(kind=R8P) | in | Packed values. | |
q_gpu | real(kind=R8P) | inout | Field (b, i, j, k, v). |
Call graph
compute_q_gradient_dev
Compute gradient of q(ivar).
fortran
subroutine compute_q_gradient_dev(b, ni, nj, nk, ngc, dx, dy, dz, q_gpu, ivar, gradient)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
b | integer(kind=I4P) | in | Block index. | |
ni | integer(kind=I4P) | in | Grid cells number in I direction. | |
nj | integer(kind=I4P) | in | Grid cells number in J direction. | |
nk | integer(kind=I4P) | in | Grid cells number in K direction. | |
ngc | integer(kind=I4P) | in | Ghost cells number. | |
dx | real(kind=R8P) | in | X space step. | |
dy | real(kind=R8P) | in | Y space step. | |
dz | real(kind=R8P) | in | Z space step. | |
q_gpu | real(kind=R8P) | in | Field component to which apply gradient. | |
ivar | integer(kind=I4P) | in | Ghost cells number. | |
gradient | real(kind=R8P) | out | Maximum gradient of q. |
Call graph
compute_normL2_residuals_dev
Compute L2 norm of residuals.
fortran
subroutine compute_normL2_residuals_dev(ni, nj, nk, ngc, nv, blocks_number, dq_gpu, norm)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
ni | integer(kind=I4P) | in | Grid cells number in I direction. | |
nj | integer(kind=I4P) | in | Grid cells number in J direction. | |
nk | integer(kind=I4P) | in | Grid cells number in K direction. | |
ngc | integer(kind=I4P) | in | Ghost grid number. | |
nv | integer(kind=I4P) | in | Number of states variables. | |
blocks_number | integer(kind=I4P) | in | Number of blocks. | |
dq_gpu | real(kind=R8P) | in | Residuals. | |
norm | real(kind=R8P) | inout | Residuals norm. |
Call graph
populate_send_buffer_ghost_gpu_dev
Polulate send buffer ghost GPU.
A fine->coarse row averages the ri*rj*rk fine cells of its coarse ghost: 2 per refined axis, 1 on an axis the tree never refines (quadtree z, issue #46).
fortran
subroutine populate_send_buffer_ghost_gpu_dev(ngc, seam_ghost_fill, seam_restriction, ri, rj, rk, comm_map_send_ghost_cell_gpu, send_buffer_ghost_gpu, q_gpu)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
ngc | integer(kind=I4P) | in | Ghost cells number. | |
seam_ghost_fill | integer(kind=I4P) | in | Seam ghost-fill regime (flag-4 rows). | |
seam_restriction | integer(kind=I4P) | in | Seam restriction regime (flag-8 rows). | |
ri | integer(kind=I4P) | in | Per-axis refinement factor. | |
rj | integer(kind=I4P) | in | Per-axis refinement factor. | |
rk | integer(kind=I4P) | in | Per-axis refinement factor. | |
comm_map_send_ghost_cell_gpu | integer(kind=I8P) | in | pointer | Comm map, cell information. |
send_buffer_ghost_gpu | real(kind=R8P) | inout | pointer | Send buffer of ghost cells. |
q_gpu | real(kind=R8P) | inout | Field component to be updated. |
Call graph
receive_recv_buffer_ghost_gpu_dev
Receive recv buffer ghost GPU.
fortran
subroutine receive_recv_buffer_ghost_gpu_dev(ngc, comm_map_recv_ghost_cell_gpu, recv_buffer_ghost_gpu, q_gpu)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
ngc | integer(kind=I4P) | in | Ghost cells number. | |
comm_map_recv_ghost_cell_gpu | integer(kind=I8P) | in | pointer | Comm map, cell information. |
recv_buffer_ghost_gpu | real(kind=R8P) | inout | pointer | Receive buffer of ghost cells. |
q_gpu | real(kind=R8P) | inout | Field component to be updated. |
Call graph
update_ghost_local_gpu_dev
Update (local) ghost cells.
A fine->coarse row averages the ri*rj*rk fine cells of its coarse ghost: 2 per refined axis, 1 on an axis the tree never refines (quadtree z, issue #46).
fortran
subroutine update_ghost_local_gpu_dev(ngc, seam_ghost_fill, seam_restriction, ri, rj, rk, l_map_ghost_cell_gpu, q_gpu)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
ngc | integer(kind=I4P) | in | Ghost cells number. | |
seam_ghost_fill | integer(kind=I4P) | in | Seam ghost-fill regime (flag-4 rows). | |
seam_restriction | integer(kind=I4P) | in | Seam restriction regime (flag-8 rows). | |
ri | integer(kind=I4P) | in | Per-axis refinement factor. | |
rj | integer(kind=I4P) | in | Per-axis refinement factor. | |
rk | integer(kind=I4P) | in | Per-axis refinement factor. | |
l_map_ghost_cell_gpu | integer(kind=I8P) | in | pointer | Local map of ghost cells. |
q_gpu | real(kind=R8P) | inout | Field component to be updated. |
Call graph
reduce_ghost_local_gpu_dev
Fold (local) ghost cells back onto the cells they mirror, the transpose of update_ghost_local_gpu_dev. Device twin of field_object%reduce_ghost_local: same-level rows only (bad_rows returns the number of 2:1 rows, in which case nothing is touched). A donor cell may receive from several ghosts (edges, corners): atomic updates, and the ghost refill runs in a second kernel, after every fold is complete.
fortran
subroutine reduce_ghost_local_gpu_dev(ngc, v_first, v_last, l_map_ghost_cell_gpu, q_gpu, bad_rows)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
ngc | integer(kind=I4P) | in | Ghost cells number. | |
v_first | integer(kind=I4P) | in | First variable to reduce. | |
v_last | integer(kind=I4P) | in | Last variable to reduce. | |
l_map_ghost_cell_gpu | integer(kind=I8P) | in | pointer | Local map of ghost cells. |
q_gpu | real(kind=R8P) | inout | Field component to be reduced. | |
bad_rows | integer(kind=I4P) | out | Number of 2:1 (non same-level) rows. |
Call graph
pack_reduce_ghost_mpi_gpu_dev
Pack the received ghost cells of variables v_first:v_last into their receive-buffer slots (transpose of receive_recv_buffer_ghost_gpu_dev); every other slot is zeroed, so superseded (deduplicated) slots carry nothing back.
fortran
subroutine pack_reduce_ghost_mpi_gpu_dev(ngc, v_first, v_last, comm_map_recv_ghost_cell_gpu, recv_buffer_ghost_gpu, q_gpu)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
ngc | integer(kind=I4P) | in | Ghost cells number. | |
v_first | integer(kind=I4P) | in | First variable to reduce. | |
v_last | integer(kind=I4P) | in | Last variable to reduce. | |
comm_map_recv_ghost_cell_gpu | integer(kind=I8P) | in | pointer | Comm map, cell information. |
recv_buffer_ghost_gpu | real(kind=R8P) | inout | pointer | Receive buffer of ghost cells. |
q_gpu | real(kind=R8P) | in | Field component to be reduced. |
Call graph
fold_reduce_ghost_mpi_gpu_dev
Add the slots sent back by the other ranks to their donor cells (transpose of populate_send_buffer_ghost_gpu_dev), variables v_first:v_last, same-level rows only (bad_rows returns the number of 2:1 rows, in which case nothing is touched). A donor may own several slots (edges, corners): atomic updates.
fortran
subroutine fold_reduce_ghost_mpi_gpu_dev(ngc, v_first, v_last, comm_map_send_ghost_cell_gpu, send_buffer_ghost_gpu, q_gpu, bad_rows)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
ngc | integer(kind=I4P) | in | Ghost cells number. | |
v_first | integer(kind=I4P) | in | First variable to reduce. | |
v_last | integer(kind=I4P) | in | Last variable to reduce. | |
comm_map_send_ghost_cell_gpu | integer(kind=I8P) | in | pointer | Comm map, cell information. |
send_buffer_ghost_gpu | real(kind=R8P) | in | pointer | Send buffer of ghost cells. |
q_gpu | real(kind=R8P) | inout | Field component to be reduced. | |
bad_rows | integer(kind=I4P) | out | Number of 2:1 (non same-level) rows. |
Call graph
Functions
interp_seam_ghost_gpu_dev
Device twin of field_object%interp_seam_ghost (issue #22 F3): evaluate the coarse->fine seam ghost interpolant for one (variable, fine ghost) pair on the FNL layout q_gpu(b,i,j,k,v), calling the single-source evaluators of adam_seam_interpolation_library (device-pinned by test_seam_interpolation_fnl, #22 F2).
Race discipline (CLAUDE-gpu, #22 F1-bis/F2): the footprint gather buffers are CONSTANT-BOUND locals of this acc routine seq — per-thread stack storage, the WENO device-kernels precedent — never private() clauses of a contained kernel nor array-section actuals (both are nvfortran race shapes). The anchor and the shift-clamped positions in meta guarantee the footprint reads REAL donor cells only (G3), so the gather never sees ghost cells written concurrently by other map rows.
Attributes: pure
Returns: real(kind=R8P)
fortran
function interp_seam_ghost_gpu_dev(regime, meta, ngc, q_gpu, v, b_send, i_send, j_send, k_send) result(value_)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
regime | integer(kind=I4P) | in | Active fill regime (SEAM_FILL_*). | |
meta | integer(kind=I4P) | in | Packed interpolation metadata (see seam_meta_pack). | |
ngc | integer(kind=I4P) | in | Ghost cells number (bounds of q_gpu). | |
q_gpu | real(kind=R8P) | in | Field variables, FNL layout (b,i,j,k,v). | |
v | integer(kind=I4P) | in | Variable index. | |
b_send | integer(kind=I4P) | in | Donor block index. | |
i_send | integer(kind=I4P) | in | Anchor donor cell, i. | |
j_send | integer(kind=I4P) | in | Anchor donor cell, j. | |
k_send | integer(kind=I4P) | in | Anchor donor cell, k. |
Call graph
restrict_seam_high_order_gpu_dev
Device twin of field_object%restrict_seam_high_order (issue #68 C2, C3): the 6-point Lagrange or the WENO restriction of one (variable, coarse ghost) pair on the FNL layout q_gpu(b,i,j,k,v), in the same order as the host (x innermost, then y, then z). The footprint is shifted inside the donor block by the codes of meta, so it reads REAL donor cells only and never a ghost cell another map row writes concurrently (the #22 race discipline). The Lagrange sums use scalar locals only; the WENO gathers one x line at a time (6 values) and calls seam_restriction_weno6_1d line by line, the arithmetic of the host's seam_restriction_weno6.
Attributes: pure
Returns: real(kind=R8P)
fortran
function restrict_seam_high_order_gpu_dev(meta, ngc, q_gpu, v, b_send, i_send, j_send, k_send) result(value_)Arguments
| Name | Type | Intent | Attributes | Description |
|---|---|---|---|---|
meta | integer(kind=I4P) | in | Packed restriction codes and regime (seam_restriction_meta_pack). | |
ngc | integer(kind=I4P) | in | Ghost cells number (bounds of q_gpu). | |
q_gpu | real(kind=R8P) | in | Field variables, FNL layout (b,i,j,k,v). | |
v | integer(kind=I4P) | in | Variable index. | |
b_send | integer(kind=I4P) | in | Donor block index. | |
i_send | integer(kind=I4P) | in | First fine cell of the restricted pair, i. | |
j_send | integer(kind=I4P) | in | First fine cell of the restricted pair, j. | |
k_send | integer(kind=I4P) | in | First fine cell of the restricted pair, k. |
Call graph