CLI Reference

Complete reference for every llmsoup subcommand, flag, and environment variable.

prepare

Generate a starter configuration file with ready-to-use defaults.

llmsoup prepare

Behavior

• Writes a config.yaml file to the current directory (or the path set by LLMSOUP_PREPARE_OUTPUT)
• Uses a built-in template with ready-to-use model definitions, signal configurations, and routing rules
• Fails if the target file already exists — delete or rename the existing file first

Environment variables

Variable	Description	Default
LLMSOUP_PREPARE_OUTPUT	Override the output file path	config.yaml

Example

# Generate config in the current directory
llmsoup prepare

# Generate config at a custom path
LLMSOUP_PREPARE_OUTPUT=./configs/my-config.yaml llmsoup prepare

---

serve

Start the HTTP routing server. The server exposes OpenAI-compatible endpoints at /v1/chat/completions and a Prometheus metrics endpoint at /metrics.

llmsoup serve [OPTIONS]

Flags

Flag	Short	Description	Default
--config <path>	—	Path to configuration file	config.yaml
--host <address>	—	Host address to bind to	127.0.0.1
--port <number>	—	Port number to listen on	8080
--quiet	—	Suppress the ASCII logo and startup banner	off
--no-banner	—	Alias for --quiet	off
--stats	—	Display a live TUI dashboard with metrics	off
--reset	—	Clear all persisted per-user usage before starting (non-interactive). No effect unless persistence is enabled.	off

Environment variable overrides

Each flag can also be set via an environment variable. Resolution order: CLI flag → environment variable → default.

Flag	Environment variable
--config	LLMSOUP_CONFIG
--host	LLMSOUP_HOST
--port	LLMSOUP_PORT

Live stats dashboard

Pass --stats to replace the normal log output with an interactive terminal dashboard that displays real-time routing metrics. The HTTP server runs normally in the background — requests are accepted and routed while the dashboard is displayed.

Metrics update every 500 ms. The dashboard displays the following panels:

Panel	Contents
Header	llmsoup version, uptime since start/reset, server endpoint URL, keyboard hints
Cost & Savings	Total cost, total saved, savings rate percentage
Requests	Total requests, active connections, error count
Per-Model	Gauge bars per model showing cost, input/output tokens, and latency; toggle sort between cost and tokens with m
Triggered Routes	Gauge bars per route showing match counts and proportional share (sorted by matches)
Top Errors	Error types with counts (sorted by error count)
Logs	Live log stream with level filtering and scrollable history

Keyboard shortcuts

Key	Action
q	Quit the dashboard and stop the server
r	Reset all counters and timers
m	Toggle Per-Model sort between cost and tokens
l	Toggle log target selector
f	Focus selected log target
PgUp / PgDn	Scroll log history
+ / -	Adjust log level filter
Ctrl+C	Quit (alternative)

Note

--stats implicitly suppresses the startup banner (acts as --quiet). The /metrics Prometheus endpoint remains fully functional while the dashboard is active.

Examples

# Start with defaults (127.0.0.1:8080, config.yaml)
llmsoup serve

# Bind to all interfaces on port 3000
llmsoup serve --host 0.0.0.0 --port 3000

# Use a specific config, suppress banner
llmsoup serve --config production.yaml --quiet

# Launch with live stats dashboard
llmsoup serve --stats

# Override via environment variables
LLMSOUP_CONFIG=prod.yaml LLMSOUP_HOST=0.0.0.0 LLMSOUP_PORT=9000 llmsoup serve

---

usage

Open a self-service usage dashboard for your own account. This command connects to a running llmsoup server’s authenticated /v1/usage endpoint and renders the caller’s own usage in the stats TUI. It requires the server to have per-user usage persistence enabled and an auth.tokens_file configured, so that requests can be authenticated and attributed to your account.

llmsoup usage [OPTIONS]

Flags

Flag	Short	Description	Default
--endpoint <url>	—	Server endpoint, e.g. http://host:8080 (falls back to LLMSOUP_ENDPOINT)	—

Token handling

The bearer token is read from the LLMSOUP_TOKEN environment variable only. There is deliberately no --token flag, so the token never appears in ps, /proc/<pid>/cmdline, or shell history, and is never logged. The command errors clearly if either the endpoint or the token is unset.

Keyboard shortcuts

Key	Action
w	Cycle the reporting window (lifetime → current period → last 7d → last 30d → month-to-date). Clears the reset baseline, so the new window shows real totals until you reset again
r	Reset the view baseline (client-side only)
m	Toggle the per-model gauge between cost and tokens
q	Quit the dashboard
Ctrl+C	Quit (alternative)

Environment variables

Variable	Description	Default
LLMSOUP_ENDPOINT	Server endpoint used when --endpoint is not passed	—
LLMSOUP_TOKEN	Bearer token used to authenticate against /v1/usage (never accepted on the command line)	—

Example

# Point at a running server and authenticate with a token from the environment
LLMSOUP_TOKEN=sk-my-token llmsoup usage --endpoint http://localhost:8080

# Resolve the endpoint from the environment as well
LLMSOUP_ENDPOINT=http://localhost:8080 LLMSOUP_TOKEN=sk-my-token llmsoup usage

See Self-service usage for a full walkthrough of the dashboard and its server-side requirements.

---

validate

Validate a configuration file for syntax errors, schema violations, and logical inconsistencies.

llmsoup validate [OPTIONS]

Flags

Flag	Short	Description	Default
--config <path>	—	Path to configuration file to validate	config.yaml

Environment variable overrides

Resolution order: CLI flag → environment variable → default.

Flag	Environment variable
--config	LLMSOUP_CONFIG

Example

# Validate the default config.yaml
llmsoup validate

# Validate a specific file
llmsoup validate --config staging.yaml

On success, prints:

Configuration 'staging.yaml' is valid

On failure, prints an error with file path and location information (see Error format below).

---

benchmark

Run routing benchmarks to measure accuracy, cost savings, latency, and memory footprint.

llmsoup benchmark --test-set <path> [OPTIONS]

Flags

Flag	Short	Description	Default
--test-set <path>	—	Path to labeled test set file (required)	—
--workload <path>	—	Path to benchmark workload configuration	uses test set
--config <path>	—	Path to llmsoup configuration file	config.yaml
--output <format>	—	Output format: text, json, markdown, md	text (or markdown when --export is used)
--export <path>	—	Export results to a file	—
--use-models	—	Download and use embedding/domain models for benchmarking	off (models skipped)
--download-models	—	Deprecated alias for --use-models	—

Output formats

Format	Description
text	Human-readable table output to stdout
json	Machine-readable JSON output
markdown / md	Markdown-formatted report (md is shorthand for markdown)

When --export is used without --output, the format automatically defaults to markdown.

Note

Unlike serve and validate, the benchmark command does not support the LLMSOUP_CONFIG environment variable. Use the --config flag to specify a configuration file.

Examples

# Run benchmark with text output
llmsoup benchmark --test-set tests/fixtures/benchmark_test_set.yaml

# Export results as markdown
llmsoup benchmark --test-set test-set.yaml --export results.md

# JSON output with models enabled
llmsoup benchmark --test-set test-set.yaml --output json --use-models

# Custom config and workload
llmsoup benchmark --test-set test-set.yaml --workload workload.yaml --config prod.yaml

---

Global behavior

Global flags

Flag	Description
--version	Print the llmsoup version and exit
--help	Print help information and exit

These flags are available on the root command (llmsoup --version, llmsoup --help) and on each subcommand (llmsoup serve --help).

Exit codes

Code	Meaning
0	Success
1	Runtime error (command execution failure)
2	Usage error (invalid arguments, missing required flags — handled by Clap)

Error format

Errors are printed to stderr with the following format:

Error (code: 1): path/to/file:line:column: Human-readable error message

When file location information is not available, the format simplifies to:

Error (code: 1): Human-readable error message

This file:line:column format is consistent across all subcommands and makes errors easy to parse by editors and CI tools.

Environment variables

Variable	Description	Applies to	Default
LLMSOUP_LOG	Log level filter (e.g., debug, info, warn, error)	All	info
LLMSOUP_SKIP_MODEL_DOWNLOAD	Skip embedding and domain model downloads (useful for testing and CI)	All	unset
LLMSOUP_CONFIG	Default configuration file path	serve, validate	config.yaml
LLMSOUP_PREPARE_OUTPUT	Override the output file path for prepare	prepare	config.yaml
LLMSOUP_HOST	Host address for the server	serve	127.0.0.1
LLMSOUP_PORT	Port number for the server	serve	8080

Tip

Set LLMSOUP_LOG=debug to get detailed routing and signal evaluation logs when troubleshooting.

Note

At the default info level, llmsoup logs one line per authenticated request — Authenticated request from user: <id> — giving a per-request attribution trail in the server logs. Raise LLMSOUP_LOG to warn to suppress it.