adam_fnl_poisson_mg_object

ADAM, FNL multigrid-preconditioned conjugate-gradient Poisson solver on AMR grids (OpenACC).

Extends poisson_mg_object (issue #84, E6a): the leaves, the hierarchy and every operator are assembled on the host as there; work_place then puts the operators (the composite operator, the ghost fill, the level operators, ghost terms and transfers, CSR), the level diagonals and colours, the cell volumes, the maps of the leaves into the level spaces and the work vectors on the device, where they stay until destroy. The FCG iteration and the V-cycle run there through the hooks:

• apply: the halo values packed on the device, exchanged by MPI_ALLTOALLV on the device buffers (GPU-aware MPI, as the ghost exchange; none on one rank), then one thread per row, its entries summed in the host's order;
• smooth: the red-black Gauss-Seidel sweeps, one kernel per colour after its halo exchange. A row couples only to cells of the other colour (the colour is the global cell parity, the level operators are 7-point: measured, no same-colour entry), so the parallel update of a colour is the host's sequential one;
• the vector hooks (fill, copy, combinations, dots and the max-norm by device reductions, the permutations of the leaves into the levels). The root goes through the host to the FFT solver (cuFFT by default: initialize passes poisson_fft_fnl_object when the caller gives no coarse), and so do the right-hand side, the James step and the ghost fill's placement. The results equal the host's to round-off: the dots sum in another order and the device contracts products into FMAs.

Source: src/lib/fnl/adam_fnl_poisson_mg_object.F90

Dependencies

Contents

• poisson_mg_fnl_object
• initialize_fnl
• apply_fnl
• smooth_fnl
• vec_fill_fnl
• vec_copy_fnl
• vec_lincomb_fnl
• vec_axpy_fnl
• vec_permute_fnl
• vec_update_fnl
• work_place_fnl
• halo_dev
• place_csr
• place_r
• place_i
• pack_dev
• alltoallv_dev
• spmv_dev
• sweep_dev
• fill_dev
• lincomb_dev
• axpy_dev
• permute_dev
• update_dev
• vec_dot_fnl
• vec_relmax_fnl
• dot_dev
• relmax_dev

Derived Types

poisson_mg_fnl_object

FNL multigrid-preconditioned CG Poisson solver on AMR grids (OpenACC).

Inheritance

Extends: poisson_mg_object

Components

Name	Type	Attributes	Description
is_initialized	logical		Initialization status.
nijk	integer(kind=I4P)		Cells per block per direction.
ncb	integer(kind=I4P)		Cells per block.
ngc	integer(kind=I4P)		Ghost cells per side.
nd	integer(kind=I4P)		Active directions.
act	integer(kind=I4P)		The active directions.
null	logical		Null directions.
nblk0	integer(kind=I4P)		Blocks per direction at level 0.
dx0	real(kind=R8P)		Cell spacing at level 0.
lmin	integer(kind=I4P)		Leaf levels range.
lmax	integer(kind=I4P)		Leaf levels range.
lbot	integer(kind=I4P)		The root level: lmin less the levels inside the blocks (E5c).
root_max	integer(kind=I4P)		Most cells of the root (E5c; the root solver's default).
nr	integer(kind=I4P)		Ranks, this rank.
me	integer(kind=I4P)		Ranks, this rank.
nleaf	integer(kind=I4P)		Leaves over the ranks.
leaf_c	integer(kind=I4P)	allocatable	Leaf block coordinates (i,j,k,level) [4,nleaf].
leaf_owner	integer(kind=I4P)	allocatable	Owner rank.
leaf_loc	integer(kind=I4P)	allocatable	Local block index (the field's) on its owner.
leaf_hash	type(hash_table)		Block key -> leaf id.
nnode	integer(kind=I4P)		Nodes.
node_c	integer(kind=I4P)	allocatable	Node block coordinates (i,j,k,level) [4,nnode].
node_owner	integer(kind=I4P)	allocatable	Owner rank.
node_loc	integer(kind=I4P)	allocatable	Local block index in its level space on its owner.
node_leaf	logical	allocatable	The node is a leaf.
node_hash	type(hash_table)		Block key -> node id.
nloc_lev	integer(kind=I4P)	allocatable	This rank's blocks per level [lbot:lmax].
lev_node	integer(kind=I4P)	allocatable	This rank's level blocks: node id [max blocks, lbot:lmax]
nleaf_loc	integer(kind=I4P)		This rank's leaves.
leaf_lev	integer(kind=I4P)	allocatable	This rank's leaf b: its level.
leaf_lev_loc	integer(kind=I4P)	allocatable	This rank's leaf b: its local index in its level space.
a	type(dcsr)		The composite operator on the leaves.
g	type(dcsr)		Ghost fill: the first ghost layer of the leaves.
g_dst	integer(kind=I4P)	allocatable	Ghost fill destinations (i,j,k,b) [4,rows].
al	type(dcsr)	allocatable	Level operators [lbot+1:lmax].
el	type(dcsr)	allocatable	Level ghost terms from level l-1 [lbot+1:lmax].
rl	type(dcsr)	allocatable	Restriction l -> l-1 [lbot+1:lmax].
pl	type(dcsr)	allocatable	Prolongation l-1 -> l [lbot+1:lmax].
diag	type(rvec)	allocatable	Level operator diagonals [lbot+1:lmax].
colour	type(rvec)	allocatable	Red-black colour (0, 1) of the level cells [lbot+1:lmax].
vol	real(kind=R8P)	allocatable	Leaf cell volumes.
bc	integer(kind=I4P)		Kind per face [lo/hi, axis] (adam_flail_parameters).
periodic	logical		Periodic axes (a null axis counts as periodic).
singular	logical		No Dirichlet face: the operator's kernel is the constants.
emin	real(kind=R8P)		Domain lower corner.
nbnd	integer(kind=I4P)		This rank's boundary faces.
bnd_row	integer(kind=I4P)	allocatable	Boundary face: the local leaf cell (row) it bounds.
bnd_w	real(kind=R8P)	allocatable	Boundary face: its weight in B (see the module).
bnd_x	real(kind=R8P)	allocatable	Boundary face: where its value lives, ghost or face centre.
bnd_kind	integer(kind=I4P)	allocatable	Boundary face: its kind.
bnd_dir	integer(kind=I4P)	allocatable	Boundary face: the outward axis, signed (-3..3).
bnd_gs	real(kind=R8P)	allocatable	Boundary face: its value's weight in the ghost fill.
g_bnd	integer(kind=I4P)	allocatable	Ghost-fill row: its boundary face, 0 for an interior one.
isolated	logical		Free-space boundaries: the ghost values by the James step.
nc	integer(kind=I4P)		Cells per direction at the coarsest level.
bnd_gi	integer(kind=I4P)	allocatable	Boundary face: its ghost cell in the widened coarsest box.
u0	real(kind=R8P)	allocatable	The last zero-ghost solution (warm start of the first solve).
iterations0	integer(kind=I4P)		Iterations of the last zero-ghost solve.
fft	class(poisson_fft_object)	allocatable	The root solver, on every rank (FFTW, or the caller's backend).
gfft	class(poisson_fft_object)	allocatable
tolerance	real(kind=R8P)		Relative max-norm residual to stop.
max_iterations	integer(kind=I4P)		FCG iterations at most.
sweeps	integer(kind=I4P)		Red-black sweeps per level, down and up.
iterations	integer(kind=I4P)		Iterations of the last solve.
residual	real(kind=R8P)		Relative residual of the last solve.
tolerance_used	real(kind=R8P)		Stop tolerance of the last solve (the round-off floor of the
factor_mean	real(kind=R8P)		Mean convergence factor per iteration of the last solve.
u	real(kind=R8P)	allocatable	The last solution (warm start).
has_guess	logical		u holds a solution of the current grid.
wf	real(kind=R8P)	allocatable	previous p and A p (leaf space).
wr	real(kind=R8P)	allocatable	previous p and A p (leaf space).
wz	real(kind=R8P)	allocatable	previous p and A p (leaf space).
wp	real(kind=R8P)	allocatable	previous p and A p (leaf space).
wq	real(kind=R8P)	allocatable	previous p and A p (leaf space).
wpp	real(kind=R8P)	allocatable	previous p and A p (leaf space).
wqp	real(kind=R8P)	allocatable	previous p and A p (leaf space).
w1	real(kind=R8P)	allocatable	Ones (leaf space).
wg	real(kind=R8P)	allocatable	Ghost-fill values [g%nrow].
lr	type(rvec)	allocatable	V-cycle: level residual and correction [lbot:lmax].
le	type(rvec)	allocatable	V-cycle: level residual and correction [lbot:lmax].
lt1	type(rvec)	allocatable	V-cycle: level temporaries [lbot:lmax].
lt2	type(rvec)	allocatable	V-cycle: level temporaries [lbot:lmax].
mlev	type(ivec)	allocatable	Per level: the level-space and leaf-space cells of the
mleaf	type(ivec)	allocatable	Per level: the level-space and leaf-space cells of the
root_coords	integer(kind=I4P)	allocatable	V-cycle: the root blocks [3,blocks].
root_buf	real(kind=R8P)	allocatable	V-cycle: the root's source and solution (host).
root_out	real(kind=R8P)	allocatable	V-cycle: the root's source and solution (host).
on_device	logical		The operators and the work vectors are on the device.
dsb	real(kind=R8P)	allocatable	Device: the halo send buffer (the largest operator's).
dhb	real(kind=R8P)	allocatable	Device: the halo receive buffer (the largest operator's).

Type-Bound Procedures

Name	Attributes	Description
destroy	pass(self)	Destroy the solver.
solve	pass(self)	Solve for phi from one variable of q.
solve_packed	pass(self)	Solve for phi from the packed interior source.
symmetry_defect	pass(self)	|x . A y - y . A x| / (|x| |A y|) (verification).
apply_a	pass(self)	y = A x on the leaves (verification).
apply_b	pass(self)	y = B g: the boundary values onto the boundary rows.
apply_bt	pass(self)	s = B^T x: the boundary rows onto the boundary faces.
initialize	pass(self)	As the host's, the coarsest level on cuFFT by default.
apply	pass(self)	y = M x on the device.
smooth	pass(self)	Red-black Gauss-Seidel on the device.
vec_fill	nopass	x(1:n) = a on the device.
vec_copy	nopass	y(1:n) = x(1:n) on the device.
vec_lincomb	nopass	w = x + a y on the device.
vec_axpy	nopass	y = y + a x on the device.
vec_dot	nopass	sum x y on the device and the ranks.
vec_relmax	nopass	max |r / vol| on the device and the ranks.
vec_permute	nopass	y(iy) = x(ix) on the device.
vec_update	nopass	A work vector to or from the device.
work_place	pass(self)	The operators and work vectors to or from the device.

Subroutines

initialize_fnl

As poisson_mg_object%initialize; without coarse, the coarsest level (and the James step's Green's function) on cuFFT.

subroutine initialize_fnl(self, grid, field, tolerance, max_iterations, coarse, isolated, bc, root_cells)

Arguments

Name	Type	Intent	Attributes	Description
self	class(poisson_mg_fnl_object)	inout		The solver.
grid	type(grid_object)	in		The grid.
field	type(field_object)	in		The field (block coordinates).
tolerance	real(kind=R8P)	in	optional	Relative residual to stop.
max_iterations	integer(kind=I4P)	in	optional	FCG iterations at most.
coarse	class(poisson_fft_object)	in	optional	Mold of the coarsest-level solver.
isolated	logical	in	optional	Free-space boundaries (James).
bc	integer(kind=I4P)	in	optional	Kinds per face [lo/hi, axis].
root_cells	integer(kind=I4P)	in	optional	Most cells of the root.

Call graph

apply_fnl

y = M x on the device: the halo of x packed, exchanged (GPU-aware MPI_ALLTOALLV), then the local rows.

subroutine apply_fnl(self, m, x, y)

Arguments

Name	Type	Intent	Attributes	Description
self	class(poisson_mg_fnl_object)	inout		The solver.
m	type(dcsr)	in		Operator (on the device).
x	real(kind=R8P)	in		Local column-space vector (on the device).
y	real(kind=R8P)	inout		Result (rows, on the device).

Call graph

smooth_fnl

Red-black Gauss-Seidel sweeps on level lv on the device: per colour, the halo of e, then every row of the colour (they read only the other colour's cells).

subroutine smooth_fnl(self, lv, e, rhs, order)

Arguments

Name	Type	Intent	Attributes	Description
self	class(poisson_mg_fnl_object)	inout		The solver.
lv	integer(kind=I4P)	in		Level.
e	real(kind=R8P)	inout		Correction (on the device).
rhs	real(kind=R8P)	in		Right-hand side (on the device).
order	integer(kind=I4P)	in		0: colour 0 first, 1: colour 1 first.

Call graph

vec_fill_fnl

x(1:n) = a on the device.

subroutine vec_fill_fnl(x, a, n)

Arguments

Name	Type	Intent	Attributes	Description
x	real(kind=R8P)	inout		Vector (on the device).
a	real(kind=R8P)	in		Value.
n	integer(kind=I4P)	in		Entries.

Call graph

vec_copy_fnl

y(1:n) = x(1:n) on the device.

subroutine vec_copy_fnl(x, y, n)

Arguments

Name	Type	Intent	Attributes	Description
x	real(kind=R8P)	in		Source (on the device).
y	real(kind=R8P)	inout		Destination (on the device).
n	integer(kind=I4P)	in		Entries.

Call graph

vec_lincomb_fnl

w(1:n) = x(1:n) + a y(1:n) on the device.

subroutine vec_lincomb_fnl(x, a, y, w, n)

Arguments

Name	Type	Intent	Attributes	Description
x	real(kind=R8P)	in		First vector (on the device).
a	real(kind=R8P)	in		Coefficient.
y	real(kind=R8P)	in		Second vector (on the device).
w	real(kind=R8P)	inout		Result (on the device).
n	integer(kind=I4P)	in		Entries.

Call graph

vec_axpy_fnl

y(1:n) = y(1:n) + a x(1:n) on the device.

subroutine vec_axpy_fnl(a, x, y, n)

Arguments

Name	Type	Intent	Attributes	Description
a	real(kind=R8P)	in		Coefficient.
x	real(kind=R8P)	in		Vector (on the device).
y	real(kind=R8P)	inout		Updated vector (on the device).
n	integer(kind=I4P)	in		Entries.

Call graph

vec_permute_fnl

y(iy(i)) = x(ix(i)), i = 1..n, on the device.

subroutine vec_permute_fnl(x, ix, y, iy, n)

Arguments

Name	Type	Intent	Attributes	Description
x	real(kind=R8P)	in		Source (on the device).
ix	integer(kind=I4P)	in		Source entries (on the device).
y	real(kind=R8P)	inout		Destination (on the device).
iy	integer(kind=I4P)	in		Destination entries (on the device).
n	integer(kind=I4P)	in		Entries.

Call graph

vec_update_fnl

A work vector to (to_device) or from the device.

subroutine vec_update_fnl(x, to_device)

Arguments

Name	Type	Intent	Attributes	Description
x	real(kind=R8P)	inout		Work vector (on the device).
to_device	logical	in		Direction.

Call graph

work_place_fnl

Put (attach) the operators, the level diagonals, colours and maps, the volumes and the work vectors on the device, with the halo buffers sized for the largest operator; or release them.

subroutine work_place_fnl(self, attach)

Arguments

Name	Type	Intent	Attributes	Description
self	class(poisson_mg_fnl_object)	inout		The solver.
attach	logical	in		Put, or release.

Call graph

halo_dev

The halo of x for operator m into self%dhb: x(sidx) packed into self%dsb, exchanged on the device buffers.

subroutine halo_dev(self, m, x)

Arguments

Name	Type	Intent	Attributes	Description
self	class(poisson_mg_fnl_object)	inout		The solver.
m	type(dcsr)	in		Operator.
x	real(kind=R8P)	in		Local column-space vector (on the device).

Call graph

place_csr

A CSR operator's arrays to (attach) or from the device (the exchange counts stay on the host).

subroutine place_csr(m, attach)

Arguments

Name	Type	Intent	Attributes	Description
m	type(dcsr)	inout		Operator.
attach	logical	in		Put, or release.

Call graph

place_r

A real array to (attach, copied in) or from the device (deleted); zero-size arrays stay on the host.

subroutine place_r(x, attach)

Arguments

Name	Type	Intent	Attributes	Description
x	real(kind=R8P)	inout		Array.
attach	logical	in		Put, or release.

Call graph

place_i

An integer array to (attach, copied in) or from the device (deleted); zero-size arrays stay on the host.

subroutine place_i(x, attach)

Arguments

Name	Type	Intent	Attributes	Description
x	integer(kind=I4P)	inout		Array.
attach	logical	in		Put, or release.

Call graph

pack_dev

sb(i) = x(sidx(i)), i = 1..ns, on the device.

subroutine pack_dev(ns, sidx, nx, x, nb, sb)

Arguments

Name	Type	Intent	Attributes	Description
ns	integer(kind=I4P)	in		Sizes.
sidx	integer(kind=I4P)	in		Local entries to send (present).
nx	integer(kind=I4P)	in		Sizes.
x	real(kind=R8P)	in		Vector (present).
nb	integer(kind=I4P)	in		Sizes.
sb	real(kind=R8P)	inout		Send buffer (present).

Call graph

alltoallv_dev

MPI_ALLTOALLV of the device buffers (GPU-aware MPI).

subroutine alltoallv_dev(ns, sb, sc, sd, nh, hb, rc, rd, ierr)

Arguments

Name	Type	Intent	Attributes	Description
ns	integer(kind=I4P)	in		Buffer sizes.
sb	real(kind=R8P)	in		Send buffer (present).
sc	integer(kind=I4P)	in		Counts and displacements (values).
sd	integer(kind=I4P)	in		Counts and displacements (values).
nh	integer(kind=I4P)	in		Buffer sizes.
hb	real(kind=R8P)	inout		Receive buffer (present).
rc	integer(kind=I4P)	in		Counts and displacements (values).
rd	integer(kind=I4P)	in		Counts and displacements (values).
ierr	integer(kind=I4P)	out		MPI error status.

Call graph

spmv_dev

y = M x on the device, one row per thread, the entries in the host's order.

subroutine spmv_dev(nrow, nloc, nnz, nx, ny, nh, rowptr, col, val, x, h, y)

Arguments

Name	Type	Intent	Attributes	Description
nrow	integer(kind=I4P)	in		Sizes.
nloc	integer(kind=I4P)	in		Sizes.
nnz	integer(kind=I4P)	in		Sizes.
nx	integer(kind=I4P)	in		Sizes.
ny	integer(kind=I4P)	in		Sizes.
nh	integer(kind=I4P)	in		Sizes.
rowptr	integer(kind=I4P)	in		Row pointers (present).
col	integer(kind=I4P)	in		Column slots (present).
val	real(kind=R8P)	in		Values (present).
x	real(kind=R8P)	in		Local columns (present).
h	real(kind=R8P)	in		Halo columns (present).
y	real(kind=R8P)	inout		Result (present).

Call graph

sweep_dev

One colour of a red-black Gauss-Seidel sweep on the device: e(i) = (rhs(i) - sum_{j /= i} a_ij e_j) / a_ii for the rows of colour colr, the off-diagonal entries in the host's order.

subroutine sweep_dev(nrow, nloc, nnz, ne, nr, nh, nd, colr, rowptr, col, val, diag, colour, rhs, h, e)

Arguments

Name	Type	Intent	Attributes	Description
nrow	integer(kind=I4P)	in		Sizes.
nloc	integer(kind=I4P)	in		Sizes.
nnz	integer(kind=I4P)	in		Sizes.
ne	integer(kind=I4P)	in		Sizes.
nr	integer(kind=I4P)	in		Sizes.
nh	integer(kind=I4P)	in		Sizes.
nd	integer(kind=I4P)	in		Sizes.
colr	integer(kind=I4P)	in		Colour of this pass.
rowptr	integer(kind=I4P)	in		Row pointers (present).
col	integer(kind=I4P)	in		Column slots (present).
val	real(kind=R8P)	in		Values (present).
diag	real(kind=R8P)	in		Diagonal (present).
colour	real(kind=R8P)	in		Colour of each row, 0 or 1 (present).
rhs	real(kind=R8P)	in		Right-hand side (present).
h	real(kind=R8P)	in		Halo of e (present).
e	real(kind=R8P)	inout		Correction (present).

Call graph

fill_dev

x(1:n) = a on the device.

subroutine fill_dev(nx, x, a, n)

Arguments

Name	Type	Intent	Attributes	Description
nx	integer(kind=I4P)	in		Sizes.
x	real(kind=R8P)	inout		Vector (present).
a	real(kind=R8P)	in		Value.
n	integer(kind=I4P)	in		Sizes.

Call graph

lincomb_dev

w(1:n) = x(1:n) + a y(1:n) (comb) or w(1:n) = x(1:n) on the device.

subroutine lincomb_dev(nx, x, a, ny, y, nw, w, n, comb)

Arguments

Name	Type	Intent	Attributes	Description
nx	integer(kind=I4P)	in		Sizes.
x	real(kind=R8P)	in		First vector (present).
a	real(kind=R8P)	in		Coefficient.
ny	integer(kind=I4P)	in		Sizes.
y	real(kind=R8P)	in		Second vector (present).
nw	integer(kind=I4P)	in		Sizes.
w	real(kind=R8P)	inout		Result (present).
n	integer(kind=I4P)	in		Sizes.
comb	logical	in		Combine, or copy x.

Call graph

axpy_dev

y(1:n) = y(1:n) + a x(1:n) on the device.

subroutine axpy_dev(a, nx, x, ny, y, n)

Arguments

Name	Type	Intent	Attributes	Description
a	real(kind=R8P)	in		Coefficient.
nx	integer(kind=I4P)	in		Sizes.
x	real(kind=R8P)	in		Vector (present).
ny	integer(kind=I4P)	in		Sizes.
y	real(kind=R8P)	inout		Updated vector (present).
n	integer(kind=I4P)	in		Sizes.

Call graph

permute_dev

y(iy(i)) = x(ix(i)), i = 1..n, on the device.

subroutine permute_dev(nx, x, nix, ix, ny, y, niy, iy, n)

Arguments

Name	Type	Intent	Attributes	Description
nx	integer(kind=I4P)	in		Sizes.
x	real(kind=R8P)	in		Source (present).
nix	integer(kind=I4P)	in		Sizes.
ix	integer(kind=I4P)	in		Source entries (present).
ny	integer(kind=I4P)	in		Sizes.
y	real(kind=R8P)	inout		Destination (present).
niy	integer(kind=I4P)	in		Sizes.
iy	integer(kind=I4P)	in		Destination entries (present).
n	integer(kind=I4P)	in		Sizes.

Call graph

update_dev

x to (to_device) or from the device.

subroutine update_dev(nx, x, to_device)

Arguments

Name	Type	Intent	Attributes	Description
nx	integer(kind=I4P)	in		Size.
x	real(kind=R8P)	inout		Vector (present).
to_device	logical	in		Direction.

Call graph

Functions

vec_dot_fnl

sum_i x_i y_i over the local n entries (a device reduction) and the ranks.

Returns: real(kind=R8P)

function vec_dot_fnl(x, y, n) result(d)

Arguments

Name	Type	Intent	Attributes	Description
x	real(kind=R8P)	in		First vector (on the device).
y	real(kind=R8P)	in		Second vector (on the device).
n	integer(kind=I4P)	in		Local entries.

Call graph

vec_relmax_fnl

max |r / vol| over the local n entries (a device reduction) and the ranks, over fmax (when positive).

Returns: real(kind=R8P)

function vec_relmax_fnl(r, vol, n, fmax) result(rm)

Arguments

Name	Type	Intent	Attributes	Description
r	real(kind=R8P)	in		Residual (on the device).
vol	real(kind=R8P)	in		Cell volumes (on the device).
n	integer(kind=I4P)	in		Local entries.
fmax	real(kind=R8P)	in		Normalisation.

Call graph

dot_dev

sum_{i<=n} x_i y_i on the device.

Returns: real(kind=R8P)

function dot_dev(nx, x, ny, y, n) result(d)

Arguments

Name	Type	Intent	Attributes	Description
nx	integer(kind=I4P)	in		Sizes.
x	real(kind=R8P)	in		First vector (present).
ny	integer(kind=I4P)	in		Sizes.
y	real(kind=R8P)	in		Second vector (present).
n	integer(kind=I4P)	in		Sizes.

Call graph

relmax_dev

max_{i<=n} |r_i / vol_i| on the device.

Returns: real(kind=R8P)

function relmax_dev(nr, r, nv, vol, n) result(rm)

Arguments

Name	Type	Intent	Attributes	Description
nr	integer(kind=I4P)	in		Sizes.
r	real(kind=R8P)	in		Residual (present).
nv	integer(kind=I4P)	in		Sizes.
vol	real(kind=R8P)	in		Cell volumes (present).
n	integer(kind=I4P)	in		Sizes.

Call graph