FoBiS.py

Appearance

NUM001 uncompensated-global-sum ​

info opt-in

A diagnostic quantity (residual, norm, mass, energy ...) is summed across ranks with MPI_SUM.

Why ​

A floating-point MPI_SUM reduction adds the ranks' values in an order that depends on the process count and the MPI implementation: the result changes with the decomposition, and with many terms its rounding error grows with their number. For conservation checks, convergence histories and regression baselines read at round-off level, that noise can hide or fake the effect being measured. Sum locally with a pairwise or compensated (Kahan/Neumaier) algorithm, and reduce in a fixed order (gather and sum on one rank, or a reproducible reduction) when the diagnostic must not depend on the number of ranks. Heuristic: reported for buffers whose name looks like a diagnostic.

Reported ​

fortran
call mpi_allreduce(res_local, residual, 1, MPI_DOUBLE_PRECISION, MPI_SUM, comm, ierr)

Preferred ​

fortran
res_local = pairwise_sum(r2)          ! compensated local sum
call mpi_gather(res_local, 1, MPI_DOUBLE_PRECISION, parts, 1, MPI_DOUBLE_PRECISION, 0, comm, ierr)
! rank 0 sums parts in rank order

Suppressing ​

For one occurrence, with the reason on the same line (or alone on the line before the statement):

fortran
! fobis: ignore[NUM001] <why this occurrence is fine>

For the whole project, ignore = NUM001 in the fobos [lint] section.

References ​

Sections of the shipped knowledge packs behind this rule:

  • hpc-practice:gpu-practice#diagnostic-precision-floor-pairwise-kahan-sum-in-measurement (fobis kb show hpc-practice:gpu-practice#diagnostic-precision-floor-pairwise-kahan-sum-in-measurement)
  • hpc-practice:gpu-practice#floating-point-reproducibility (fobis kb show hpc-practice:gpu-practice#floating-point-reproducibility)