3. Defects only some builds see

A new test: every rule integrates an odd function over a symmetric interval to zero. It compares relative errors with a tolerance:

!< Every rule integrates an odd function over a symmetric interval to zero.
program test_symmetry
!< Every rule integrates an odd function over a symmetric interval to zero.
use quad_rules,  only : R8P, midpoint, trapezoid, simpson
use polynomials, only : cubic
implicit none
real(R8P), parameter :: tolerance = 1.e-14_R8P !< Rounding allowance.
real(R8P), parameter :: exact = 0._R8P         !< The integral of x**3 over [-1, 1].

if (relative(midpoint(cubic, -1._R8P, 1._R8P, 4)) > tolerance) error stop 'midpoint is not symmetric'
if (relative(trapezoid(cubic, -1._R8P, 1._R8P, 4)) > tolerance) error stop 'trapezoid is not symmetric'
if (relative(simpson(cubic, -1._R8P, 1._R8P, 4)) > tolerance) error stop 'simpson is not symmetric'
print '(A)', 'zero for an odd function'

contains
   pure function relative(s) result(e)
   !< Relative error of an estimate.
   real(R8P), intent(in) :: s !< Estimate.
   real(R8P)             :: e !< Relative error.

   e = abs(s - exact) / abs(exact)
   endfunction relative
endprogram test_symmetry

The tests run in a new mode, fast: the optimised flags of the release profile, with the tests among the sources (the way a CI would run them):

[fast]
help          = Optimised, with the tests
template      = template-gnu
build_profile = release
cflags        = -c
src           = ./src/ ./test/
build_dir     = ./build/fast/
target        = src/area.F90

$ fobis test --mode fast --quiet

Building test suite (4 target(s))...

  PASS  test_exactness                 (0.00 s)
  PASS  test_order                     (0.00 s)
  PASS  test_romberg                   (0.00 s)
  PASS  test_symmetry                  (0.00 s)

  Results: 4 passed, 0 failed, 0 skipped
  Total:   0.00 s

It passes, and it cannot fail: the exact value is zero, so relative divides by zero. The midpoint rule's result is exactly 0 here, 0/0 is NaN, and NaN > tolerance is false. An optimised build computes NaN silently.

Shadow builds

A shadow rebuilds a mode with checking flags added, in its own directory (.fobis/shadow/<mode>-<shadow>/), without touching the mode's build: checks (runtime checks, floating-point traps, signalling-NaN initialisation, at -O0 with -g), sanitize (AddressSanitizer and UndefinedBehaviorSanitizer), strict (standard conformance). The flags come from a table per compiler family:

$ fobis test --mode fast --shadow checks --quiet

Building test suite (4 target(s))...

  PASS  test_exactness                 (0.00 s)
  PASS  test_order                     (0.00 s)
  PASS  test_romberg                   (0.00 s)
  FAIL  test_symmetry                  (0.12 s)
        Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation.
        
        Backtrace for this error:
        #0  0x7653b0e28d14 in ???
        #1  0x7653b0e27905 in ???
        #2  0x7653b0a4532f in ???
        	at ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
        #3  0x5d68b8e5a302 in relative
        	at test/test_symmetry.F90:21
        #4  0x5d68b8e5a3a9 in test_symmetry
        	at test/test_symmetry.F90:10
        #5  0x5d68b8e5a592 in main
        	at test/test_symmetry.F90:4

  Results: 3 passed, 1 failed, 0 skipped
  Total:   0.12 s
[exit status 1]

The floating-point trap stops the program at the division: line 21, called from line 10.

The matrix

fobis matrix builds and tests every mode × shadow combination, each from scratch:

$ fobis matrix --modes fast --shadows none,checks,sanitize
matrix: fast/none (gnu)
matrix: fast/checks (gnu)
matrix: fast/sanitize (gnu)
mode  none      checks    sanitize
fast  ok        FAIL      ok

Failures:
fast/checks (GNU Fortran (Ubuntu 13.4.0-6ubuntu1~24~ppa1) 13.4.0):
  test test_symmetry: floating-point exception at test/test_symmetry.F90:21
[exit status 1]

sanitize does not see the defect (an invalid operation is not a memory error), checks does. With several modes on several compilers (each loaded by a [verification] setup line, such as module load), the same command gives the table of what works where; --verify adds the cases, compared with their references and across cells.

With the relative error made absolute near zero (abs(s - exact) / max(abs(exact), 1)):

$ fobis matrix --modes fast --shadows none,checks,sanitize --verify
matrix: fast/none (gnu)
matrix: fast/checks (gnu)
matrix: fast/sanitize (gnu)
mode  none      checks    sanitize
fast  ok        ok        ok

What you learned

--shadow checks|sanitize|strict on build, run, test and verify, fobis matrix, --verify. Reference: the compiler matrix.

Next: 4. How accurate is the result?.