CLI-Testing — Declarative Test Runner for Shell Commands

Stop writing fragile bash scripts to test your CLI tools. Write what you expect, and let CLI-Testing do the rest.

CLI-Testing is a single-binary, zero-dependency test runner for shell commands. You describe your expected outputs in simple, readable .clitest files — no test framework, no scripting, no boilerplate. If your tool runs in a terminal, CLI-Testing can test it.

Heavily inspired by Hurl — which does the same for HTTP requests — CLI-Testing brings that same declarative, readable approach to testing CLI commands.

---

Quick Example

# Verify JSON output from our API tool
my-cli export --format json
EXIT 0
[asserts]
stdout contains "version"
stdout matches /"\d+\.\d+\.\d+"/
lineCount == 1

That's a complete test. No setup, no teardown, no imports. Just: command, expected exit code, assertions.

---

Why CLI-Testing?

As developers, we constantly build CLI tools, scripts, and pipelines. Testing them usually means one of:

• Bash scripts that grow into unmaintainable monsters with nested ifs and string comparisons
• Heavy test frameworks that require a runtime, dependencies, and ceremony just to check "did my command print the right thing?"
• Manual testing — running commands by hand and eyeballing the output (we've all been there)

CLI-Testing gives you a middle ground: tests that read like documentation and run in milliseconds.

# Health check returns OK
curl -s http://localhost:8080/health
EXIT 0
ok

You read it, you understand it, your teammates understand it. Done.

---

Features

• Exact body matching — assert full stdout output line-by-line
• Rich assertions — contains, startsWith, endsWith, matches (regex), lineCount, line-based queries
• Negation — stdout not contains "error"
• Directives — @skip, @group, @timeout, @poll, @defer, @workdir for organizing and controlling test flow
• Captures — extract values from output and reuse them across entries
• Variables — pass --var key=value from the CLI, use {{key}} in your tests
• Parallel execution — run test files concurrently with --parallel
• Recursive discovery — point it at a directory and it finds all .clitest files
• Glob support — clitest "tests/**/*.clitest"
• Zero dependencies — single Go binary, runs anywhere

---

Installation

Download a pre-built binary from the latest release, extract it, and place it in your $PATH.

Or install via go install:

go install github.com/sleipi/cli-t/cmd/clitest@latest

On Alpine Linux:

# Replace VERSION with the desired release (e.g., 1.0.0)

# Add the signing key (one-time)
wget -O /etc/apk/keys/clitest.rsa.pub \
  https://github.com/sleipi/cli-t/releases/download/vVERSION/clitest.rsa.pub

# Download and install the .apk
wget https://github.com/sleipi/cli-t/releases/download/vVERSION/clitest-VERSION-r0-x86_64.apk
apk add clitest-VERSION-r0-x86_64.apk

Or build from source:

git clone https://github.com/sleipi/cli-t.git
cd clitest
make build
# ./clitest is ready to use

---

Usage

# Run all tests in a directory (recursive)
clitest test/

# Run a specific file
clitest test/e2e/syntax/asserts.clitest

# Use a glob pattern
clitest "test/**/*.clitest"

# Pass variables
clitest --var host=localhost --var port=8080 test/

# Verbose output
clitest -v test/

# Parallel execution
clitest --parallel test/

---

.clitest File Anatomy

# Optional comment describing the test
@group smoke
my-command --flag value
EXIT 0
expected output line 1
expected output line 2
[asserts]
stdout contains "success"
stdout matches /took \d+ms/
lineCount == 2

Part	Required	Description
Comment	No	Lines starting with #, describe the test
Directives	No	@skip, @group, @timeout, @poll, @defer, @workdir
Command	Yes	Shell command executed via sh -c
EXIT	No	Expected exit code (defaults to 0)
Body	No	Exact stdout match
[asserts]	No	Rich assertions against stdout/stderr

---

Editor Support

Syntax highlighting for .clitest files is available via TextMate grammar:

• VS Code — Install the extension from editors/vscode/ (see setup instructions)
• IntelliJ IDEA — Settings > Editor > TextMate Bundles > add the editors/vscode directory
• Sublime Text — Copy the grammar file to your Packages directory

Works in all JetBrains IDEs (GoLand, WebStorm, PhpStorm, etc.).

---

Full Specification

See SPEC.md for the complete syntax reference.

---

License

MIT