Skip to content

feat: add /qa-cli skill for CLI tools, servers, and headless projects#208

Open
StressTestor wants to merge 1 commit intogarrytan:mainfrom
StressTestor:feat/qa-cli
Open

feat: add /qa-cli skill for CLI tools, servers, and headless projects#208
StressTestor wants to merge 1 commit intogarrytan:mainfrom
StressTestor:feat/qa-cli

Conversation

@StressTestor
Copy link

summary

adds /qa-cli — a QA skill for projects that don't have a web UI. CLI tools, API servers, libraries, background services.

the existing /qa and /qa-only skills are browser-based (they use the browse binary to navigate URLs, take screenshots, click elements). but a lot of projects are headless — CLIs, servers, libraries. those need a different QA approach: run the test suite, exercise the real binary with smoke tests, hit server endpoints, and report what's passing, what's broken, and what's untested.

what it does

9 phases:

  1. discovery — detect project type (node/rust/python/go), find test commands, build commands, CLI entry points, server entry points
  2. build — build the project. if it doesn't build, diagnose and fix before testing
  3. test suite — run existing tests, parse results, categorize failures (bug/flaky/stale)
  4. CLI smoke tests — exercise every subcommand with --help, run happy-path commands with safe read-only args, test error handling (bad inputs should produce clean messages, not stack traces)
  5. server smoke tests — start server, hit health endpoint, exercise documented routes, stop server
  6. coverage gap analysis — compare source modules vs test files, flag untested critical paths
  7. report — structured output with health score (0-100), tables for each phase
  8. fix loop — for each failure, diagnose root cause, make minimal fix, atomic commit, re-test
  9. before/after summary — health score delta, fixes applied, remaining issues, ship readiness

tested on

ran it manually against StressTestor/pr-prism (v2.0.0, TypeScript CLI + server, 199 tests). it found a real bug: prism scan --repo invalid produced a raw stack trace instead of a clean error message. fixed it in the same session.

changes

  • qa-cli/SKILL.md.tmpl — new skill template (9 phases, ~300 lines of prompt)
  • qa-cli/SKILL.md — generated output
  • SKILL.md.tmpl — added qa-cli to proactive suggestion list
  • SKILL.md — regenerated
  • scripts/gen-skill-docs.ts — added qa-cli to findTemplates()

test plan

  • bun test passes (static validation picks up the new skill)
  • bun run gen:skill-docs --dry-run passes (generated SKILL.md matches committed)
  • manual test: ran the full qa-cli workflow against a real project, found and fixed a bug

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant