Unstructured memory

A host array with a copy on the device. Every procedure is generic over the kinds R8P, R4P, I8P, I4P, I2P, I1P and the ranks 1 to 7 (rank 1 shown). The array is passed as an assumed-shape dummy: any contiguous host array works, typically an allocatable, and it must stay allocated while it is mapped.

• dev_alloc_unstr
• dev_free_unstr
• dev_memcpy_to_device_unstr
• dev_memcpy_from_device_unstr
• Rules

dev_alloc_unstr

subroutine dev_alloc_unstr(fptr_dev, init_value)
VARTYPE(KKP), intent(inout)        :: fptr_dev(:)
VARTYPE(KKP), intent(in), optional :: init_value

Argument	Intent	Description
fptr_dev	inout	Host array to map: a device copy is created (OpenACC enter data create, OpenMP target enter data map(alloc:))
init_value	in, optional	Value given to every element of the device copy, by a kernel; the host copy is unchanged

Without init_value the device copy is undefined: copy the host values with dev_memcpy_to_device_unstr.

dev_free_unstr

subroutine dev_free_unstr(fptr)
VARTYPE(KKP), intent(inout) :: fptr(:)

Argument	Intent	Description
fptr	inout	Mapped host array: its device copy is deleted (OpenACC exit data delete, OpenMP target exit data map(delete:)); the host array is unchanged

dev_memcpy_to_device_unstr

subroutine dev_memcpy_to_device_unstr(dst)
VARTYPE(KKP), intent(inout) :: dst(:)

Argument	Intent	Description
dst	inout	Mapped host array: its host values are copied to its device copy (update device, target update to)

dev_memcpy_from_device_unstr

subroutine dev_memcpy_from_device_unstr(dst)
VARTYPE(KKP), intent(inout) :: dst(:)

Argument	Intent	Description
dst	inout	Mapped host array: its device values are copied to the host (update self, target update from)

Rules

• In kernels, name the host array: OpenACC with present(a), OpenMP without a clause (the array is already mapped). Do not use DEVICEVAR/DEVICEPTR, which are for structured pointers.
• The two copies are independent until a dev_memcpy_*_unstr; in a host run (fallback, CPU mode) they are the same memory, which hides a missing copy.
• To pass the device address of a mapped array to a library (a GPU-aware MPI, cuBLAS), use !$acc host_data use_device(a) or !$omp target data use_device_addr(a) around the call.
• Unstructured mappings are not in the allocation registry: the runtime present table tracks them.

allocate(t(0:n+1), tn(0:n+1))
t = [(sin(pi*i*dx), i=0, n+1)]
t(0) = 0._R8P ; t(n+1) = 0._R8P
call dev_alloc_unstr(fptr_dev=t)                      ! a device copy of t, uninitialised
call dev_memcpy_to_device_unstr(dst=t)                ! host t -> device t
call dev_alloc_unstr(fptr_dev=tn, init_value=0._R8P)  ! a device copy of tn, set on the device

call dev_memcpy_from_device_unstr(dst=t) ! device t -> host t
call dev_free_unstr(fptr=t)
call dev_free_unstr(fptr=tn)