1. A local model

The last track puts a language model to work on quad: explaining failures, answering questions, writing tests, reviewing changes, writing commit messages, resolving merge conflicts. In FoBiS a model never works alone. FoBiS first computes the facts it can establish itself (where an error is, what a change touches, which module is missing), gives the model only numbered sources, and checks what comes back: an answer must cite them, and code must build and run.

The model runs on your machine (or your cluster), through llama.cpp: no account, no network after the download, no code leaving the machine. The curated models:

$ fobis llm recommended
coder-0.5b       491 MB  Apache-2.0  smoke tests of the setup only: too small to describe a diff
coder-1.5b       1.1 GB  Apache-2.0  quick experiments on CPU; copies prompt examples instead of reading the diff
coder-7b         4.7 GB  Apache-2.0  default choice: 8 GB GPU, or a recent CPU
coder-14b        9.0 GB  Apache-2.0  12-16 GB GPU
coder-30b-a3b   18.6 GB  Apache-2.0  24 GB GPU; mixture of experts (3B active), fast for its size

Pull one with: fobis llm pull <alias>   (or any hf:<org>/<repo>[@<rev>]/<file>.gguf)

fobis llm runtime install     # the llama.cpp server, pinned and checksummed
fobis llm pull coder-7b       # the model, pinned to a commit and checksummed
fobis llm serve start         # optional: keep the server running between commands

Recorded on a GPU node

The model outputs of this track were recorded with coder-7b (Qwen2.5-Coder 7B, 4-bit) on an NVIDIA A30 of a cluster, in one Slurm job, and are not regenerated by the CI: a model's wording changes with its version and the hardware. The commands and the deterministic outputs (contexts, facts) are generated and checked like the other tracks. Paths under the cluster's home directory are shown as ~/fobis-dev.

fobis llm doctor checks the whole chain:

$ fobis llm doctor
ok   provider   managed  (model coder-7b)
ok   directory  ~/fobis-dev/llm
ok   platform   linux/x86_64, variants cpu, cuda12, cuda13, vulkan, rocm, sycl; auto -> cuda12 (NVIDIA driver supports CUDA 12.9)
ok   runtime    ~/fobis-dev/llama.cpp-b11308/build/bin/llama-server  (version: 0.5.0-dev (build 1, commit feb9a3d))
ok   model      models/Qwen/Qwen2.5-Coder-7B-Instruct-GGUF/13fb94bfda8c8cf22497dc57b78f391a9acb426a/qwen2.5-coder-7b-instruct-q4_k_m.gguf
ok   start      server ready in 0.0 s
ok   sample     'ready' in 0.0 s

$ fobis llm ask "In one sentence: what does the Fortran intrinsic merge(tsource, fsource, mask) return?"
Using the running llama-server at http://127.0.0.1:41675
The Fortran intrinsic `merge(tsource, fsource, mask)` returns an array where each element is taken from `tsource` if the corresponding element in `mask` is true, otherwise it is taken from `fsource`.

fobis llm ask is a smoke test: the bare model, no sources, no checks. The next chapters never use the model this way.

What you learned

fobis llm recommended, runtime install, pull, serve, doctor, ask. Reference: local models, the managed runtime.

Next: 2. Build failures, explained.