Skip to content

Integrate micro‑pentest into demo sidecar; add API/CLI coverage doc and AlDeci podcast script#207

Closed
DevOpsMadDog wants to merge 4 commits into
mainfrom
codex/add-reachability-analysis-and-micro-pen-tests
Closed

Integrate micro‑pentest into demo sidecar; add API/CLI coverage doc and AlDeci podcast script#207
DevOpsMadDog wants to merge 4 commits into
mainfrom
codex/add-reachability-analysis-and-micro-pen-tests

Conversation

@DevOpsMadDog
Copy link
Copy Markdown
Owner

@DevOpsMadDog DevOpsMadDog commented Dec 20, 2025

Motivation

  • Surface a fast validation step in the interactive demo so users can see micro penetration tests integrated with reachability and PentAGI validation.
  • Clarify product messaging by publishing a complete API↔CLI coverage mapping and reconciling the documented endpoint counts (243 documented vs 137 core OSS inventory).
  • Provide a ready-to-use 20‑minute AlDeci podcast script that explains the decision + evidence narrative for demos and marketing.

Description

  • Add run_micro_pentest to scripts/demo_sidecar.py and integrate a Micro Pentest phase (before PentAGI) into run_scenario and full_demo flows.
  • Extend show_assessment_summary to accept micro‑pentest results and display a Micro Pentest Verdict row with colorized verdict reporting.
  • Update demo health probes to check /api/v1/micro-pentest/status/0 and add a demo fallback when micro‑pentest endpoints are missing (handle 404/501 with a demo verdict).
  • Add documentation files docs/API_CLI_COVERAGE_ANALYSIS.md (full 243‑endpoint mapping and CLI coverage notes) and docs/ALDECI_PODCAST_SCRIPT.md (20‑minute podcast script), and align wording about endpoint counts.

Testing

  • Manual smoke run: started the FastAPI demo server in demo mode with FIXOPS_MODE=demo FIXOPS_JWT_SECRET=devsecret FIXOPS_API_TOKEN=demo-token and confirmed /health returned 200.
  • Manual demo run: executed FIXOPS_BASE_URL=http://localhost:8000 FIXOPS_API_TOKEN=demo-token python scripts/demo_sidecar.py run-scenario --cve CVE-2021-44228 --skip-upload and produced terminal output and summary screens (screenshots captured), with micro‑pentest fallback exercised when the API returned 404.
  • Observed reachability worker activity (job queued) during demo; reachability attempted to clone the example repo and logged a Git clone failure in the environment, which is expected in this demo environment.
  • No automated test suite was run as part of this change; changes are demo and documentation focused and verified via the manual runs above.

Codex Task


Summary by cubic

Integrates a Micro Pentest phase into the demo sidecar to provide fast exploitability validation before PentAGI, and adds clear API↔CLI coverage documentation plus a 20‑minute AlDeci podcast script. This improves the demo flow and clarifies endpoint counts and coverage.

  • New Features
    • Added run_micro_pentest and wired it into run_scenario and full_demo before PentAGI.
    • Extended show_assessment_summary to include a colorized “Micro Pentest Verdict”.
    • Added health probe to /api/v1/micro-pentest/status/0 with a demo fallback on 404/501.
    • Added docs: API_CLI_COVERAGE_ANALYSIS.md (243-endpoint mapping, CLI coverage, reconciled counts) and ALDECI_PODCAST_SCRIPT.md (20-minute script).

Written for commit ac3c419. Summary will update on new commits.

Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 3 files

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="docs/API_CLI_COVERAGE_ANALYSIS.md">

<violation number="1" location="docs/API_CLI_COVERAGE_ANALYSIS.md:198">
P2: Arithmetic inconsistency: the CLI Command Summary states &quot;67 total&quot; but the individual counts in the table sum to 69. Update either the totals to 69 or adjust the category counts to match 67.</violation>
</file>

<file name="docs/ALDECI_PODCAST_SCRIPT.md">

<violation number="1" location="docs/ALDECI_PODCAST_SCRIPT.md:102">
P3: Product name &#39;Pentagi&#39; is inconsistent with the official branding &#39;PentAGI&#39; used elsewhere in the codebase. For brand consistency in marketing materials, consider using &#39;PentAGI&#39; to match the established naming convention.</violation>
</file>

Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR


---

## CLI Command Summary (67 total)
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai Bot Dec 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Arithmetic inconsistency: the CLI Command Summary states "67 total" but the individual counts in the table sum to 69. Update either the totals to 69 or adjust the category counts to match 67.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At docs/API_CLI_COVERAGE_ANALYSIS.md, line 198:

<comment>Arithmetic inconsistency: the CLI Command Summary states &quot;67 total&quot; but the individual counts in the table sum to 69. Update either the totals to 69 or adjust the category counts to match 67.</comment>

<file context>
@@ -0,0 +1,230 @@
+
+---
+
+## CLI Command Summary (67 total)
+
+| Category | Commands | Count |
</file context>
Fix with Cubic


---

### The Validation Layer - Pentagi (1 minute)
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai Bot Dec 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: Product name 'Pentagi' is inconsistent with the official branding 'PentAGI' used elsewhere in the codebase. For brand consistency in marketing materials, consider using 'PentAGI' to match the established naming convention.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At docs/ALDECI_PODCAST_SCRIPT.md, line 102:

<comment>Product name &#39;Pentagi&#39; is inconsistent with the official branding &#39;PentAGI&#39; used elsewhere in the codebase. For brand consistency in marketing materials, consider using &#39;PentAGI&#39; to match the established naming convention.</comment>

<file context>
@@ -0,0 +1,318 @@
+
+---
+
+### The Validation Layer - Pentagi (1 minute)
+
+*[Introduce this as the &quot;when the stakes are high&quot; option]*
</file context>
Fix with Cubic

devin-ai-integration Bot pushed a commit that referenced this pull request Jan 22, 2026
…age docs

Cherry-picked from PR #207 with fixes:
- Add run_micro_pentest function to demo_sidecar.py
- Integrate Micro Pentest phase into run_scenario and full_demo flows
- Add Micro Pentest Verdict row to assessment summary
- Add health probe for micro-pentest endpoint
- Add docs/API_CLI_COVERAGE_ANALYSIS.md (243-endpoint mapping)
- Add docs/ALDECI_PODCAST_SCRIPT.md (20-minute podcast script)
- Fix CLI command count arithmetic (67 -> 69)
@devin-ai-integration
Copy link
Copy Markdown
Contributor

Closing as part of PR consolidation. Changes have been cherry-picked into PR #240 with fixes (CLI command count arithmetic corrected from 67 to 69).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant