CLI: Build & Shipco audit (preview)

co audit — is a CLI fit for an agent harness?

An agent that drives a command-line tool has only what the tool prints. co audit asks one question of any CLI, co or anyone else's: can an agent with only this program's help find the command for a task, run it safely, and know what it will change?

Name the command as you would type it:

co audit co                  # all of co (about 30 s)
co audit co gmail            # one co group
co audit yt-dlp              # any program on PATH
co audit gh pr --review      # then a model judges each page that passed
co audit co --json

It never reads source code. It runs <command> --help (or -h), opens every subcommand the page lists, and keeps going down, many at once, each in an empty HOME and working directory with no input. Then it judges each page from what it printed. Exit 0 means fit; exit 1 lists each problem and its fix, and a table scores every rule:

gh: 228 pages
  prints         227/228
  hangs          228/228
  writes           1/228
  usage          228/228
  example        127/228
  ...
✗ not yet fit for an agent harness: 329 problems

Rules (the same for every program)

rule fails when
prints --help (or -h) does not print and exit 0
hangs it did not return within 20 seconds, e.g. waiting for input
writes reading help created a file (the finding names it)
usage no usage line
example no example
self_example no example runs this command itself
flags an example uses a flag that the page of the command it runs does not document
private an example contains a real home path or a full 0x address

Subcommands are read from the layouts real CLIs print: Typer/Rich panels (any title but Options and Arguments), Commands:-style sections (uv, click, kubectl), CORE COMMANDS with name: rows (gh), and argparse's {build,serve}. A listed word whose page is its parent's page word for word is not counted as a command.

Model review (--review)

Only pages that pass every rule are reviewed. A text-only model judges four things a rule cannot: is the first line clear to a newcomer, does the page say what the command reads or changes, is the example realistic, is it simple. It returns one concrete rewrite. Pin --model when comparing runs.

In CI

tests/unit/test_cli_help_contract.py runs the same engine on co on every PR and blocks on any problem. It adds three house conventions of ours: a fixed "what it changes" word, a Back: line, and every command in co commands reachable from co --help. The help-gate workflow runs co audit co --since base.json --review on pages a PR changed, and reports without blocking, because a model's verdict varies between runs.

co wiki keeps its own reviewed pages (#1656), which are being rewritten (#1667); co audit co reports their missing examples.

Star us on GitHub

If ConnectOnion saves you time, a ⭐ goes a long way — and earns you a coffee chat with our founder.