Machine-readable Interfaces
Everything on this page is opt-in, for a program that another program calls: an agent, a job submission script, a CI step. A program that uses none of it behaves as it always did.
call cli%init(progname='solver', version='1.2.0', description='A solver', &
schema_option=.true., check_option=.true., error_format_env='SOLVER_ERROR_FORMAT')
call cli%add(switch='--steps', help='Time steps', required=.true., val_type='I4P')
call cli%add(switch='--cfl', help='CFL number', required=.false., def='0.5', envvar='SOLVER_CFL', &
val_type='R8P', min='0', max='1')
call cli%add(switch='--label', help='Label of the run', required=.false., def='run')
call cli%add(switch='--token', help='Access token', required=.false., def='', sensitive=.true.)Declared types
Values are untyped until get, unless you declare their type with val_type. Then parse validates them, whatever their source (command line, environment, configuration file):
$ solver --steps abc
solver: error: cannot convert "abc" of option "--steps" to an integer!
Try 'solver --help' for help.val_type | Accepted |
|---|---|
I1P, I2P, I4P, I8P | An integer that fits the kind |
R4P, R8P, R16P | A real |
logical | T, .true., false, ... |
character | Any text |
Typed values are also checked against their choices and their range during parse, and defaults and choices when the option is added (ERROR_TYPE_DEFINITION). The kind is part of the declaration, so a value accepted by parse also fits a get of that kind. To keep that promise, a get into another family or a narrower kind is always ERROR_TYPE_MISMATCH, whatever the value. A get into a wider kind is allowed, and a real is checked in the wider kinds too: 1.00000001 is 1.0 in R4P but above a maximum of 1 in R8P, so parse rejects it. A get into a character is always allowed and gives the text as passed. A real too big for its kind is an error; an explicit inf is a value, and nan is outside every range.
Not allowed with val_type: maps, act='config', act='alternate' and the print actions. A flag can only be logical, a count an integer, a path check a character.
Describing the program
init(schema_option=.true.) adds --schema; cli%json_schema() and cli%save_json_schema(file) give the same document without it. Hidden options and builtins are left out, and the document is the same before and after parse.
$ solver --schema | sed -n '/"options"/,$p'
"options": [
{"switch": "--steps", "help": "Time steps", "required": true, "action": "store", "value": {"type": "integer", "minimum": -2147483648, "maximum": 2147483647, "x-flap-kind": "I4P"}, "typed": true},
{"switch": "--cfl", "help": "CFL number", "required": false, "action": "store", "value": {"type": "number", "minimum": 0, "maximum": 1, "x-flap-kind": "R8P"}, "typed": true, "default": "0.5", "envvar": "SOLVER_CFL"},
{"switch": "--label", "help": "Label of the run", "required": false, "action": "store", "value": {"type": "string"}, "typed": false, "default": "run"},
{"switch": "--token", "help": "Access token", "required": false, "action": "store", "value": {"type": "string"}, "typed": false, "sensitive": true}
],
"positionals": [],
"exclusive_sets": [],
"commands": []
}It also holds schema_version, program, capabilities and errors (the meaning of every code). This is FLAP's own format, not one JSON Schema document, because commands, aliases and exclusive sets do not map onto one. The value of an option uses JSON Schema 2020-12 keywords, plus x-flap-kind, so it can be passed on as it is.
"typed": false means FLAP does not know the type: without val_type the value is described as a string, or as a number when it has a range. schema_version rises only on a breaking change.
Errors as JSON lines
init(error_format='json') fixes the format. init(error_format_env='NAME') lets each caller choose, so the people running the same program keep the text:
$ SOLVER_ERROR_FORMAT=json solver --stepz 5
{"schema_version":1,"level":"error","code":15,"class":"usage","message":"solver: error: switch \"--stepz\" is unknown! Did you mean \"--steps\"?","argument":"--stepz","suggestions":["--steps"]}Each message is one line on the error unit, with level (error or warning), code, message, and argument and suggestions when known. There are no colours, no Try ... --help line and no help after the message. cli%error_message stays the plain message. The variable takes text or json; any other value means text.
One command
A program with many commands has a long description, and an agent pays for every word. prog cmd --schema, or cli%json_schema(group='cmd'), describes one command, by its name or an alias, without the top level or the code tables.
save_json_schema(file, group='cmd') saves it. An unknown command gives '', and ERROR_MISSING_GROUP from save_json_schema.
Validating without running
init(check_option=.true.) adds --check at the top level. It may also come after a command, prog cmd --check, unless the command defines a --check of its own, as may --dry-run. It runs every stage of parse, then stops before the program uses a value:
$ solver --check --steps 100 --cfl 0.5 --label demo
solver: command line valid
2 options not type-checked: --label --token$ solver --check --steps 100 --cfl 7
solver: error: value "7" of "--cfl" is out of range [0, 1]!
Try 'solver --help' for help.
[exit status 2]The exit status is 0 if valid and otherwise the status of the error, even when an error comes before --check on the command line. With JSON errors the verdict is a line like {"schema_version":1,"level":"result","valid":true,"unchecked":["--label"]}. With standalone=.false., parse returns STATUS_CHECK or the error and the program decides.
What a valid result does not tell you:
- Untyped options. They are listed as not type-checked, because a failure would come only at
get. So are an untyped count with a range and a flag that took its value from the environment or a configuration file. - An alternate action. It skips the validation, as in a normal run. The verdict says so (
"bypassed":true) and lists every option as not checked. - Relations between values. Those are the program's business.
- Paths elsewhere. The path checks ran here: a valid result on a login node says nothing about a compute node.
Like --help, --check stops the program. A program that parses after MPI_Init stops without MPI_Finalize, so parse first, or use standalone=.false..
Run manifest
cli%provenance(format='json') and cli%save_provenance(file) record the program, the command line, the status and every value with its source. There is no date, host or directory, so the same inputs give the same bytes:
call cli%save_provenance('solver.manifest.json', error=error)$ SOLVER_CFL=0.25 solver --steps 100 --token s3cret && cat solver.manifest.json
{
"schema_version": 1,
"program": {"name": "solver", "version": "1.2.0"},
"arguments": ["--steps","100","--token","***"],
"status": 0,
"values": [
{"command": "", "name": "--steps", "value": "100", "source": "command line"},
{"command": "", "name": "--cfl", "value": "0.25", "source": "environment", "variable": "SOLVER_CFL"},
{"command": "", "name": "--label", "value": "run", "source": "default"},
{"command": "", "name": "--token", "value": "***", "source": "command line"}
]
}Values are strings exactly as resolved, so nothing is lost in a conversion. Hidden options are left out, as in the text report: a manifest may not reproduce a run that uses them.
cli%add(sensitive=.true.) masks a value as *** in both reports and in the recorded command line, and leaves its default out of the help and of the JSON description. A sensitive positional leaves the command line out altogether (null). An error message quotes it as *** too: cannot convert "***" of option "--token" to an integer!. FLAP can mask only what it attributes to the option: a misspelled --tokn=abc is an unknown argument, quoted as typed. The keys of a sensitive map stay visible.
Dry run
init(dry_run_option=.true.) adds a --dry-run flag at the top level, and cli%is_dry_run() reads it. Unlike --check, the program runs, and it is the program that must skip its side effects:
if (cli%is_dry_run()) then
print '(A)', 'would write '//trim(file)
else
call write_results(file)
endifFLAP cannot enforce this, nor detect a program that never reads the flag: such a program performs the real action while its caller believes otherwise. Enable the option only once every side effect is guarded. The JSON description declares it (capabilities.dry_run) and the run manifest records it (dry_run).
Agent Skill
cli%skill() returns a SKILL.md in the Agent Skills format, and save_skill(file) saves it. It holds what an agent needs before its first call: what the program is, the top-level options, one line per command, the machine options you enabled and the exit statuses. The options of a command stay out, read on demand.
call execute_command_line('mkdir -p heat')
call cli%save_skill('heat/SKILL.md', error=error)$ heat --write-skill && cat heat/SKILL.md
---
name: heat
description: "Solve the 2D heat equation on a square plate. Use when running heat."
---
# heat
## Options
- `--write-skill`: Write the agent skill and stop
## Commands
Read the options of a command with `heat COMMAND --schema`.
- `run`: Run a simulation
## Calling it
- `--schema` prints the full description as JSON.
- `--check` validates a command line without running it, and names what it could not validate.
- `--dry-run` reports what would be done, without doing it.
- `HEAT_ERROR_FORMAT=json` writes errors as JSON lines.
- Exit status: 0 success; 1 failure of the program; 2 fix the command line and retry; 3 fix a file or a variable; 4 the program is wrong, do not retry.The format wants the directory named as the skill: skill_name lower-cases the program name and replaces what the format forbids. A sensitive default and hidden options stay out.
Is it worth shipping? Measured once, on a small program with two commands, six runs of a small model: with the skill file, 5 calls on average to finish the task and about 62,800 tokens; without it, 5.7 calls and about 62,100 tokens, with no failed call on either side (three runs each). A program whose --help is good already gives an agent what it needs, so expect little on the calls. A skill helps an agent that does not yet know the program exists, which this test did not measure.
Using it from an agent
FLAP does not include an MCP server: that would need a JSON-RPC layer in the library, and tool calls fit long parallel runs poorly. An adapter outside the program does the same work with what is above. This one uses only the Python standard library:
#!/usr/bin/env python3
"""Expose a FLAP program as a tool for an agent, from its --schema (standard library only).
An adapter lives outside the program: FLAP describes and validates, this file translates.
flap_tool.py PROGRAM the tool definition: name, description, JSON Schema of the input
flap_tool.py PROGRAM '{"--steps": 5}' the command line of a call, validated by the program itself with --check
A long run should be submitted by the caller, not by the tool: this adapter never runs the program for real.
"""
import json
import os
import subprocess
import sys
def read_schema(program):
"""Read the description of a program."""
out = subprocess.run([program, "--schema"], capture_output=True, text=True, check=True).stdout
return json.loads(out)
def describe(schema):
"""Turn the description into a tool definition: options only, as a JSON Schema object."""
properties, required = {}, []
for option in schema["options"]:
value = dict(option["value"])
value["description"] = option["help"]
properties[option["switch"]] = value
if option["required"]:
required.append(option["switch"])
return {
"name": schema["program"]["name"],
"description": schema["program"]["description"],
"input_schema": {"type": "object", "properties": properties, "required": required},
}
def command_line(program, call):
"""Turn the input of a call into a command line; a flag takes no value, a list its items."""
argv = [program]
for switch, value in call.items():
if value is True:
argv.append(switch)
elif isinstance(value, list):
argv += [switch] + [str(item) for item in value]
elif value is not False:
argv += [switch, str(value)]
return argv
def main():
program = sys.argv[1]
schema = read_schema(program)
if len(sys.argv) == 2:
print(json.dumps(describe(schema), indent=2))
return 0
argv = command_line(program, json.loads(sys.argv[2]))
env = dict(os.environ)
if schema["capabilities"]["error_format_env"]:
env[schema["capabilities"]["error_format_env"]] = "json"
check = subprocess.run([program, "--check"] + argv[1:], capture_output=True, text=True, env=env)
print("command:", " ".join(argv))
print("verdict:", check.stdout.strip())
for line in check.stderr.splitlines():
print("error:", json.loads(line)["message"])
return check.returncode
if __name__ == "__main__":
sys.exit(main())From --schema it builds the tool definition:
$ python3 -I flap_tool.py solver
{
"name": "solver",
"description": "A solver",
"input_schema": {
"type": "object",
"properties": {
"--steps": {
"type": "integer",
"minimum": -2147483648,
"maximum": 2147483647,
"x-flap-kind": "I4P",
"description": "Time steps"
},
"--cfl": {
"type": "number",
"minimum": 0,
"maximum": 1,
"x-flap-kind": "R8P",
"description": "CFL number"
},
"--label": {
"type": "string",
"description": "Label of the run"
},
"--token": {
"type": "string",
"description": "Access token"
}
},
"required": [
"--steps"
]
}
}For a call, it builds the command line and has the program validate it, without running it:
$ python3 -I flap_tool.py solver '{"--steps": 100, "--cfl": 7}'
command: solver --steps 100 --cfl 7
verdict: {"schema_version":1,"level":"result","valid":false}
error: solver: error: value "7" of "--cfl" is out of range [0, 1]!
[exit status 2]