Dependencies

A dependency is another Fortran project your sources use. The fobos declares it in [dependencies], one line per dependency: a git repository pinned to a tag, a branch, a commit or a version range, or a directory on this machine. fobis fetch clones the repositories; fobis build then compiles them with your sources, or builds them as libraries with their own fobos and links them.

How it works:

1. Fetch. fobis fetch clones each repository into <deps_dir>/<name>/ (.fobis_deps/ by default), checks out its pin, builds the use=fobos ones, and writes two files there: .deps_config.ini, read by the build, and fobos.lock, the commit each dependency resolved to.
2. Build. fobis build (and fobis check, fobis doctests) reads .deps_config.ini: a use=sources dependency (the default) joins the source scan and is compiled with your flags; a use=fobos one is rebuilt by its own fobos before your build, its module directory and library wired in. Path dependencies (path=DIR) are read from the fobos at every build and need no fetch.
3. One level. fetch reads your fobos only: the dependencies of a dependency are not fetched. Declare in your fobos everything your build compiles.

Tutorial: dependencies, a project that depends on quad. Reference: fetch, tree, install.

A dependency from GitHub, pinned

[dependencies]
penf = https://github.com/szaghi/PENF :: tag=v2.0.1

fobis fetch          # clone into .fobis_deps/penf, check out v2.0.1, write the lock
fobis build          # PENF's sources are compiled with yours

$ fobis fetch
Cloning penf from https://github.com/szaghi/PENF
Checking out v2.0.1 for penf
Saved deps config to .fobis_deps/.deps_config.ini
Saved lock file to .fobis_deps/fobos.lock

$ fobis build --mode release --quiet && ./build/release/integrate

Building src/integrate.F90
Compiling .fobis_deps/penf/src/lib/penf_global_parameters_variables.F90 serially
Compiling .fobis_deps/penf/src/lib/penf_b_size.F90 serially
Compiling .fobis_deps/penf/src/lib/penf_stringify.F90 serially
Compiling src/quad_rules.F90 .fobis_deps/penf/src/lib/penf_allocatable_memory.F90 serially
Compiling src/integrands.F90 .fobis_deps/penf/src/lib/penf.F90 serially
Compiling src/integrate.F90 serially
Linking build/release/integrate
Target src/integrate.F90 has been successfully built
     n   trapezoid error     simpson error
     4        1.0388E-01        4.5598E-03
     8        2.5768E-02        2.6917E-04
    16        6.4297E-03        1.6591E-05
    32        1.6066E-03        1.0334E-06
    64        4.0161E-04        6.4530E-08
Simpson, 64 intervals: 2.0000000645300013

Recorded on a small demo project: fobis fetch, the generated .deps_config.ini, then fobis build.

The specification is name = SOURCE [:: option]...; the name is the directory under deps_dir.

Specification	Checks out
URL	the default branch
URL :: branch=develop	the branch's head
URL :: tag=v2.0.1	the tag
URL :: rev=4cb9d32	the commit
URL :: semver=^2.0	the newest tag satisfying the range (next recipe)
user/repo	https://github.com/user/repo

• Any git URL works as written: https://, ssh://, git@host:user/repo, file:///srv/git/lib (a local repository). Only a value with neither :// nor a leading git@ is read as GitHub's user/repo.
• Pin with tag= or rev=. That is what makes every checkout (a colleague's, CI's) fetch the same code: the lock is not committed. A bare URL or a branch= follows the branch.
• One pin per dependency. semver= with tag=, branch= or rev= stops fetch with exit 2. Between tag=, branch= and rev= nothing is checked: tag wins, then branch.
• A dependency already cloned is not cloned again: fetch prints already fetched and checks out the pin, so a tag= changed in the fobos is applied by a plain fobis fetch when the tag existed at clone time. A tag made later needs --update.

A version range

[dependencies]
penf   = https://github.com/szaghi/PENF :: semver=^2.0
stdlib = https://github.com/fortran-lang/stdlib :: semver=>=0.5,<1.0

fetch lists the remote's tags (git ls-remote --tags), checks out the newest one in the range and records it in the lock as resolved. Later fetches in the same checkout use the locked tag without asking the remote; --update resolves the range again.

Range	Means
^1.5	>=1.5.0, <2.0.0
~1.5.2	>=1.5.2, <1.6.0
>=1.0,<2.0	both bounds (any of >=, >, <=, <, comma-separated)
=1.5.0, 1.5.0	exactly that version
*	any version

• Tags are read as [v]MAJOR[.MINOR[.PATCH]]: v1.5 is 1.5.0. Any other tag is ignored, pre-releases included: v1.3.0-rc1 never satisfies a range.
• No tag in range stops fetch with exit 1 and lists the first 20 tags the remote has.
• A fresh checkout resolves the range again, and can get a newer tag than yours. Pin with tag= when every checkout must build the same code.

Compile it with your sources, or build it as a library

[dependencies]
; compiled with your sources and your flags (use=sources is the default)
penf   = https://github.com/szaghi/PENF :: tag=v2.0.1
; built by its own fobos in its mode, its library linked
stdlib = https://github.com/fortran-lang/stdlib :: tag=v0.5.0 :: use=fobos :: mode=gnu

	use=sources (default)	use=fobos
The dependency needs a fobos	no	yes
fobis fetch	clones	clones, then runs fobis build [--mode MODE] in it
fobis build	its directory joins your sources, unless a src path already covers it	rebuilds it first (up to date: nothing compiled), adds its mod_dir to the include path, links its library by full path

• use=sources takes your flags: one compiler, one set of options for the whole program. It suits small, source-distributed libraries, and needs nothing from the dependency's fobos.
• use=fobos takes its own: the dependency's mode must build a library (mklib, output); without mode= its default mode is used. Its directory is excluded from your source scan, so nothing is compiled twice.
• fetch --no-build only defers the build: the next fobis build builds a use=fobos dependency anyway.
• A use=fobos dependency builds with its own fobos, in its own directory: if that fobos declares dependencies from git, they are looked for in the clone's own .fobis_deps/, which nothing fetches. Declare such a library's dependencies in your fobos and use it with use=sources, or fetch inside its clone.
• mode= without use=fobos does nothing: your mode compiles a use=sources dependency.

The same wiring without [dependencies], for a project already on disk, is the mode key dependon (PATH/fobos:MODE; append ((indirect)) to link with -L/-l instead of the full path):

[default]
compiler = gnu
src      = ./src/
target   = src/area.F90
dependon = ../quad/fobos:static((direct))

$ cd ~/area && fobis build --quiet && ./area
Building dependency fobos into ../quad with mode static

Building src/quad_rules.F90
Nothing to compile, all objects are up-to-date
Linking build/static/libquad.a
Target src/quad_rules.F90 has been successfully built
Track building of src/quad_rules.F90

The following auxiliary paths have been added
Include files search paths (include):
- ../quad/build/static/mod
Libraries search paths (lib_dir):
Libraries paths:
- (libs) ../quad/build/static/libquad.a

Building src/area.F90
Nothing to compile, all objects are up-to-date
Linking ./area
Target src/area.F90 has been successfully built
area =  0.333333333333333

Where the dependencies go

[dependencies]
deps_dir = vendor
penf     = https://github.com/szaghi/PENF :: tag=v2.0.1

deps_dir (default .fobis_deps) holds the clones, .deps_config.ini and fobos.lock. Keep it out of git; the .gitignore of fobis new already does (/.fobis_deps/).

$ ls .fobis_deps && cat .fobis_deps/.deps_config.ini
fobos.lock
penf
[deps]
src = .fobis_deps/penf

• Set it in the fobos, not with --deps-dir. fobis fetch --deps-dir DIR clones into DIR, but build, check, doctests and tree read the fobos's deps_dir only: with the option alone, the build does not see the dependencies.

Update, or go back to the lock

fobis fetch --update     # git fetch in each clone, move to the newest commit its pin allows, rewrite the lock
fobis fetch --frozen     # check every dependency out at the commit of the lock; leave the lock as it is
fobis fetch --no-build   # clone or update only; use=fobos dependencies are built by the next fobis build

--update moves a branch= dependency to the remote branch's head (fast-forward), an unpinned one to its upstream head, resolves semver= ranges again, and makes newly created tags available; a tag= or rev= pin stays where it is. To move a pinned dependency, change the pin in the fobos: that is the change to commit.

--frozen refuses to run without a lock (exit 1), uses the locked tag of a semver= range without asking the remote, and never rewrites the lock. A dependency the lock does not list is fetched as usual.

$ fobis fetch --frozen
Dependency penf already fetched (use --update to refresh)
Saved deps config to .fobis_deps/.deps_config.ini

• --frozen is for a checkout that keeps its deps_dir: a workstation, a cluster account, a CI job caching .fobis_deps/. A fresh checkout has no lock: run fobis fetch, and rely on the fobos's pins.
• A plain fetch compares each clone with the lock and warns, never fails, when its commit differs (a clone moved by hand, a pin changed in the fobos): Warning: dependency 'penf' HEAD (...) does not match fobos.lock (...). Run 'fobis fetch --update' to refresh. The lock is then rewritten with the new commit.

The lock file

$ cat .fobis_deps/fobos.lock
# fobos.lock — auto-generated by FoBiS.py fetch — do not edit by hand

[penf]
url = https://github.com/szaghi/PENF
commit = 4cb9d326c08f109ed5126d83c27e8e70c98b7eaa
sha256 = 2a46c1a89c4e1e9de7a410cdf0efe4ec1cbef53526998b8552ad98c97260bd66
tag = v2.0.1

One section per dependency: url, commit (the clone's HEAD), sha256 (of git archive HEAD: the committed content), the pin as declared, resolved for a semver= range, dev = true for a test-only one. A path dependency records path and, in a git checkout, the commit holding it and dirty (tracked changes or untracked sources under the directory), else a sha256 of its source files.

• The lock lives in deps_dir and is not committed. It records what this checkout fetched, so that fetch can tell when a clone moved and --frozen can put it back. Reproducibility across checkouts comes from the pins.
• Do not edit it: every fetch but --frozen rewrites it.

Another project on this machine

[dependencies]
; its sources compiled with yours
utils = path=../utils
; built by its own fobos, its library linked
core  = path=../core :: use=fobos :: mode=static

Nothing to clone, tag or publish. Every build, check and doctests reads path dependencies from the fobos, so an edit in ../utils is recompiled at the next fobis build, and ../core is rebuilt by its own fobos first.

• The directory is relative to the fobos that declares it (absolute paths and ~ work too). A missing directory stops fetch, build and check with exit 1.
• No pin: the directory is used as it is. tag=, branch=, rev= or semver= stop with exit 2.
• A use=sources directory is scanned without its .git, .fobis, its own deps_dir and the build_dir of its fobos modes. Its own dependencies are not taken from it: declare them in your fobos (two copies of a module would clash).
• fobis fetch is optional: it checks the directories, builds the use=fobos ones (unless --no-build) and records their state in the lock. --frozen reports a path dependency that changed since as a warning, never an error: it is expected to change.

$ touch src/quad_rules.F90 && cd ~/area && fobis build --quiet
Building dependency fobos into ../quad with mode static

Building src/quad_rules.F90
Compiling src/quad_rules.F90 serially
Linking build/static/libquad.a
Target src/quad_rules.F90 has been successfully built
Track building of src/quad_rules.F90

The following auxiliary paths have been added
Include files search paths (include):
- ../quad/build/static/mod
Libraries search paths (lib_dir):
Libraries paths:
- (libs) ../quad/build/static/libquad.a

Building src/area.F90
Nothing to compile, all objects are up-to-date
Linking ./area
Target src/area.F90 has been successfully built

dependon on a sibling project: an edit there is rebuilt first. A path= dependency with use=fobos does the same.

A monorepo

repo/
├── libs/core/   fobos (mode lib), src/
├── libs/utils/  src/
└── apps/solver/ fobos, src/

; apps/solver/fobos
[dependencies]
utils = path=../../libs/utils
core  = path=../../libs/core :: use=fobos :: mode=lib

Build from apps/solver/: the dependency paths are relative to the fobos, a mode's src to the current directory. In the lock, both entries record the repository's commit and whether their own directory has uncommitted changes.

A testing framework for the tests only

[dependencies]
penf       = https://github.com/szaghi/PENF :: tag=v2.0.1
test-drive = https://github.com/fortran-lang/test-drive :: tag=v0.6.1 :: dev

fobis fetch            # fetched with the others
fobis test             # the test builds use it
fobis fetch --no-dev   # a production checkout: not fetched

• Used by fobis test, fobis doctests and fobis check only. Every other build leaves it out, even when its sources contain the dependency's directory (src = ./): it never ends up in a library, an install or a program.
• --no-dev keeps its lock entry, and a clone fetched earlier stays marked test-only, so no other build compiles it.
• Projects using yours never fetch it: fobis tree marks it [test only] and does not show a dependency's test-only dependencies.
• A build of every program (a mode without target) whose sources include test programs using it fails: give the mode a target, or keep the test directory out of its sources. More in Testing.

See the dependency tree

fobis tree             # the whole tree
fobis tree --depth 1   # direct dependencies only
fobis tree --no-dedupe # repeat a dependency met twice instead of marking it
fobis tree --json      # name, spec, version, fetched, has_fobos, duplicate, dev, path, dependencies

$ fobis tree
quad v0.1.0
└── penf v2.0.1 (https://github.com/szaghi/PENF :: tag=v2.0.1)

The root is [project] name and its version; each node is the dependency's name, its version (the clone's [project] version or latest tag, else the pin) and its specification. A fetched dependency with a fobos is read for its own [dependencies].

Mark	Means
[not fetched — run fobis fetch]	no directory under deps_dir
[directory DIR not found]	a path dependency whose directory is missing
[no fobos — cannot read transitive deps]	its dependencies are unknown
(*) [already shown]	met earlier in the tree; children not repeated
[test only]	a :: dev dependency of yours

• A dependency's dependencies are looked for in your deps_dir, by name: they show [not fetched] unless your fobos declares them too, since fetch does not follow them.
• The same view, as facts: fobis check lists the path dependencies, and fobis introspect --dependencies prints every dependency with its use, directory, pin and locked commit as JSON.

How the build finds the fetched dependencies

.deps_config.ini, written by every fetch, holds four keys of space-separated entries: src (the use=sources clones), dependon (<clone>/fobos[:MODE] of the use=fobos ones) and their test-only twins dev_src and dev_dependon. Path dependencies are not in it.

• src entries join the mode's sources, unless a source path already contains them (src = ./ does).
• dependon entries join the mode's dependon, and their directories its exclude_dirs: the dependency is built once, by its fobos, never scanned as your source.
• The dev_ keys are used by fobis test, doctests and check; any other build excludes their directories.
• Fetch again after editing [dependencies]: for repositories the build reads .deps_config.ini, not the fobos.

Install a project from GitHub

fobis install szaghi/FLAP --tag v2.0.0 --prefix ~/.local   # clone, build, copy
fobis install https://gitlab.com/user/tool --mode release  # any git URL, a mode of its fobos
fobis install szaghi/FLAP --update --prefix ~/.local       # git fetch in the cached clone first
fobis install szaghi/FLAP --no-build                       # clone only

fobis install REPO clones into ~/.fobis/<repo>/ (--deps-dir elsewhere; reused by later runs), checks out --tag, --branch or --rev, runs fobis build --track-build [--mode MODE] there, and copies what the build tracked: programs to <prefix>/bin/, libraries to <prefix>/lib/, their module files to <prefix>/include/ (--bin, --lib, --include rename them; the prefix defaults to ./). Without REPO, fobis install copies your own tracked build: see Libraries.

The local form: fobis build --tb, then fobis install --prefix ./local. The GitHub form ends with the same copy.

• The project needs a fobos whose mode (--mode, else its default) builds what you want installed; --track-build is passed by the command.
• Run it outside a project: --mode is also checked against the fobos of the current directory, and a fobos without that mode stops the command.
• An installed project is yours to manage: nothing records it, and fetch is the tool for build-time dependencies of a project.
• Known defect (3.9.11): the copy looks for the tracked files from the current directory, not from the clone, so a project whose mode has a relative build_dir installs nothing (No installable artifacts found, exit 0).

A cluster without internet

FoBiS itself: make standalone in a FoBiS checkout writes dist/fobis.pyz, one file holding FoBiS and its Python dependencies (every GitHub release carries it too). It needs Python 3.10 or newer, nothing else.

# on a machine with network
fobis fetch
rsync -a --exclude build/ myproject/ cluster:myproject/   # with .fobis_deps/ and the sibling path dependencies
scp dist/fobis.pyz cluster:

# on the cluster
cd myproject
python3 ~/fobis.pyz build --mode release
python3 ~/fobis.pyz fetch --frozen       # optional: puts every clone back at its locked commit, offline

• The build needs no network: .deps_config.ini and the lock hold paths relative to the project, and a use=fobos dependency is rebuilt by the same fobis.pyz (no fobis on the PATH needed).

• fetch without network works for clones already present: plain fetch checks out the pins, --frozen the locked commits. --update, a new clone, and a semver= range without a lock need the remote.

• Path dependencies must be copied to the same place relative to the project.

• Mirrors instead of copies: keep bare clones on the cluster and let git rewrite the URLs, so the fobos stays unchanged:

  git config --global url."file:///shared/mirrors/".insteadOf https://github.com/

More on clusters in HPC and clusters.

For CI and agents

fobis fetch --json
fobis tree --json
fobis introspect --dependencies

fetch --json prints one document (status, deps_dir, dependencies, errors) and exits 2 on a wrong specification, 1 on a missing path dependency.

• fetch --json is a reduced fetch (3.9.11): it clones at tag=/branch=/rev=, but ignores semver= (the default branch is cloned), --frozen and --no-dev, and writes no lock. Use plain fobis fetch in CI.
• A failed clone or dependency build does not fail fetch: the git or build error is printed and the exit status stays 0; the build that follows fails. Check the output, or run fobis check after fetching.
• Cache .fobis_deps/ between CI runs keyed on the fobos, and run fobis fetch --frozen on a cache hit. More in Continuous integration.