7. Production runs

A production run must not silently run on the CPU, and must not hide a memory bug. This chapter adds the checks of a production code to the solver: heat_7 --require-device forbids the host fallback, heat_7 --double-free simulates a bug.

No silent host fallback

call dev_init(require_device=require_device, ierr=ierr)
if (ierr == FUNDAL_ERR_NO_DEVICE) then
   print '(A)', 'no device available and the host fallback is forbidden: stop'
   stop 1
endif
print '(A,L1)', 'running on the host: ', dev_is_host_fallback()

By default dev_init accepts a host fallback (no device available) with a warning on standard error, so that the same build runs on a laptop or a CI machine. With require_device=.true. a fallback is an error: dev_init returns FUNDAL_ERR_NO_DEVICE in ierr (without ierr, it stops the program). The environment can require a device without changing the code: ACC_DEVICE_TYPE=nvidia (OpenACC) or OMP_TARGET_OFFLOAD=MANDATORY (OpenMP).

The documentation runs on the host on purpose (ACC_DEVICE_TYPE=host, an explicit request: no warning), so the strict run stops:

$ heat_7 --require-device
no device available and the host fallback is forbidden: stop
[exit status 1]

A strict allocation registry

FUNDAL records every structured allocation. Its policy decides what a misuse of dev_free without ierr does: warn (the default) reports it and goes on, error stops the program, off disables the checks. A code under test, or a production code that prefers a crash to a corruption, sets error:

call dev_set_registry_policy('error') ! a misuse of dev_free without ierr stops the run (default: warn)

FUNDAL_REGISTRY=error in the environment does the same without changing the code; dev_set_registry_policy overrides it. The labels of the allocations appear in the reports, and the statistics count every live allocation:

call dev_alloc(fptr_dev=t_dev,  lbounds=[0], ubounds=[n+1], ierr=ierr, init_value=0._R8P, label='temperature')
if (ierr /= 0) error stop 'device allocation failed'
call dev_alloc(fptr_dev=tn_dev, lbounds=[0], ubounds=[n+1], ierr=ierr, init_value=0._R8P, label='temperature, next')
if (ierr /= 0) error stop 'device allocation failed'
call dev_get_alloc_stats(allocs=allocs, bytes=bytes)
print '(A,I0,A,I0,A)', 'live device allocations: ', allocs, ' (', bytes, ' bytes)'

Checked teardown

dev_free with ierr reports a misuse instead of applying the policy, and frees nothing in that case; dev_alloc_report lists what is still allocated (here, nothing):

call dev_free(t_dev, ierr=ierr)
if (ierr /= 0) error stop 'dev_free failed'
call dev_free(tn_dev, ierr=ierr)
if (ierr /= 0) error stop 'dev_free failed'
call dev_alloc_report() ! nothing should be left

$ heat_7
running on the host: T
live device allocations: 2 (272 bytes)
temperature at the centre after 50 steps: 0.61712
FUNDAL live allocations: 0 (0 bytes)

running on the host: T is dev_is_host_fallback(): true whenever a device build runs on the host, requested or not.

A double free

With --double-free a second pointer, alias, points to the buffer of t_dev, and is freed after t_dev:

if (double_free) then ! the buffer of alias has already been freed through t_dev
   call dev_free(alias, ierr=ierr)
   if (ierr == FUNDAL_ERR_NOT_REGISTERED) then
      print '(A,I0,A)', 'dev_free(alias): error ', ierr, ', not a live FUNDAL allocation, nothing freed'
      stop 1
   endif
endif

$ heat_7 --double-free
running on the host: T
live device allocations: 2 (272 bytes)
temperature at the centre after 50 steps: 0.61712
FUNDAL live allocations: 0 (0 bytes)
dev_free(alias): error 103, not a live FUNDAL allocation, nothing freed
[exit status 1]

The registry no longer knows the address, so dev_free returns FUNDAL_ERR_NOT_REGISTERED and frees nothing: without the registry, the runtime would have freed the same buffer twice.

What the registry cannot see

The registry knows addresses, not pointers. If a new allocation reuses the address of a freed buffer before the alias is freed, the alias frees the new buffer. Nullify aliases when their target is freed.

What you learned

require_device and FUNDAL_ERR_NO_DEVICE; dev_is_host_fallback; the registry policy (dev_set_registry_policy, FUNDAL_REGISTRY); labels, dev_get_alloc_stats, dev_alloc_report; dev_free with ierr. Reference: Host fallback, Registry and statistics, Errors and constants.

That is the whole tutorial. The cookbook collects short recipes for everyday tasks.