feat(auto-review): AI-powered auto-approve

[Cali93]

Summary

• Adds opt-in AI-powered auto-approve to the Claude auto-review action
• After the review, a separate Claude API call evaluates the diff + review findings against a repo-specific scope prompt to decide approve/reject
• Uses a dedicated GitHub App token for the approval, satisfying org-level required-review rules
• Defaults to reject when in doubt — human reviewers can always approve manually
• Fully documented in the README with setup guide and usage examples

New Inputs

Input	Required	Description
auto_approve	No (default: false)	Enable the feature
auto_approve_app_id	When enabled	GitHub App ID for approval token
auto_approve_private_key	When enabled	GitHub App private key
auto_approve_scope_prompt	No	Repo-specific criteria for Claude

Setup (one-time, org-wide)

1. Create a GitHub App

Go to your org Settings → Developer settings → GitHub Apps → New GitHub App:

• Name: e.g. Claude Reviewer (must be unique across GitHub)
• Homepage URL: your org's GitHub URL (required field, any URL works)
• Permissions: Repository permissions → Pull Requests → Read & Write
• Webhook: uncheck "Active" (no webhook needed)
• Installation: "Only on this account"

2. Generate a private key

On the App's settings page, scroll to "Private keys" → "Generate a private key". Save the downloaded .pem file.

3. Install the App

App settings → "Install App" → select your org → choose "All repositories" or select specific repos.

4. Add org secrets

Go to Org Settings → Secrets and variables → Actions → New organization secret:

• CLAUDE_REVIEWER_APP_ID — the App ID (visible on the App's "General" settings page)
• CLAUDE_REVIEWER_PRIVATE_KEY — the full contents of the .pem private key file
• Set repository access to "All repositories" or select specific repos that need auto-approve

5. Update your workflow (per-repo)

- name: Claude Review
  uses: WalletConnect/actions/claude/auto-review@master
  with:
    anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
    auto_approve: "true"
    auto_approve_app_id: ${{ secrets.CLAUDE_REVIEWER_APP_ID }}
    auto_approve_private_key: ${{ secrets.CLAUDE_REVIEWER_PRIVATE_KEY }}
    auto_approve_scope_prompt: |
      Only approve Terraform infrastructure changes.
      All changed files must be under infrastructure/ or monitoring/.
      Do NOT approve if resources are destroyed, databases deleted,
      force_destroy enabled, or prevent_destroy removed.

How It Works

1. Normal auto-review runs (unchanged)
2. Findings are extracted into findings.json (unchanged)
3. New: auto-approve-evaluation.js calls the Anthropic API with the diff + changed files + review findings + scope prompt
4. Claude returns { approved: true/false, reason: "..." }
5. If approved, generates a GitHub App token and approves the PR with the reason in the review body

Test plan

• All 165 existing tests pass
• 6 new tests for the evaluation script (approve, reject, API error, unparseable response, findings inclusion, scope prompt forwarding)
• Integration test on a real PR with the feature enabled

🤖 Generated with Claude Code

[claude]

Claude finished @Cali93's task in 2s —— View job

---

PR Review

• Gather context
• Read changed files
• Spawn breaking changes subagent → ✅ No breaking changes
• Spawn data classification subagent → ✅ No data classification issues
• Compile findings

---

🔒 External Domain URL (Non-blocking) URL: https://api.anthropic.com File: claude/auto-review/scripts/auto-approve-evaluation.js:70 — Intentional (Anthropic API), confirm acceptable per org policy.

Found 3 issue(s)

Issue 1: stdout contamination breaks auto-approve feature

ID: auto-approve-eval-stdout-contamination-b3f1
File: claude/auto-review/scripts/auto-approve-evaluation.js:162 / claude/auto-review/action.yml:463
Severity: HIGH
Category: code_quality

Context:

• Pattern: logger.log(...) calls console.log(...) (stdout). RESULT=$(node "$SCRIPT_PATH") captures all stdout. jq -r '.approved' then attempts to parse the mixed log+JSON output as JSON.
• Risk: jq fails to parse non-JSON lines like [auto-approve-evaluation.js] Evaluating auto-approve for PR #42... and exits non-zero; APPROVED is set to empty string.
• Impact: AUTO_APPROVE_DECISION is never "true" — the auto-approve feature silently does nothing even when Claude returns { "approved": true }.
• Trigger: Every time auto_approve: true is set and Claude approves.

Recommendation: Change logger.log to write to stderr in this script so stdout contains only the final JSON:

// In github-utils.js, or locally override for this script:
log: (...args) => process.stderr.write(`[${scriptName}] ` + args.join(' ') + '\n'),

Or, in action.yml, extract JSON from the last line of RESULT:

APPROVED=$(echo "$RESULT" | tail -1 | jq -r '.approved')
REASON=$(echo "$RESULT" | tail -1 | jq -r '.reason')

---

Issue 2: Command injection via ${{ env.AUTO_APPROVE_REASON }} in shell body

ID: action-approve-reason-cmd-injection-a7c2
File: claude/auto-review/action.yml:491-493
Severity: HIGH
Category: security

Context:

• Pattern: ${{ env.AUTO_APPROVE_REASON }} is a GitHub Actions expression substituted textually into the shell script before execution. The reason field is populated from Claude's API response, which is influenced by PR diff content.
• Risk: If AUTO_APPROVE_REASON contains $(command) or backticks (injected via prompt manipulation of PR content), the shell evaluates them when expanding the double-quoted string literal in the generated script.
• Impact: Arbitrary command execution in the runner context under the GitHub App token scope.
• Trigger: A malicious PR author crafts diff content that causes Claude to include shell metacharacters in the reason field.

Recommendation: Pass the reason as an env var (bash does not re-evaluate $(...) inside variable expansions):

- name: Approve PR
  if: inputs.auto_approve == 'true' && env.AUTO_APPROVE_DECISION == 'true'
  shell: bash
  env:
    GH_TOKEN: ${{ steps.approve-app-token.outputs.token }}
    APPROVE_REASON: ${{ env.AUTO_APPROVE_REASON }}
  run: |
    PR_NUMBER="${{ github.event.pull_request.number || github.event.issue.number }}"
    gh pr review "$PR_NUMBER" --approve --body "✅ **Auto-approved by Claude**

    ${APPROVE_REASON}"

---

Issue 3: No validation that App credentials are provided when auto_approve=true

ID: action-approve-missing-credential-validation-d4e9
File: claude/auto-review/action.yml:476-482
Severity: MEDIUM
Category: code_quality

Context:

• Pattern: auto_approve_app_id and auto_approve_private_key are both required: false with no validation step. actions/create-github-app-token will fail with a cryptic error if either is empty.
• Risk: Users enabling auto_approve: true without providing credentials get an opaque failure deep in the step chain after the full review has run.
• Impact: Poor DX; confusing error attribution.
• Trigger: Any workflow that sets auto_approve: true but omits one or both App credential inputs.

Recommendation: Add a validation step before the evaluation:

- name: Validate auto-approve inputs
  if: inputs.auto_approve == 'true'
  shell: bash
  run: |
    if [[ -z "${{ inputs.auto_approve_app_id }}" || -z "${{ inputs.auto_approve_private_key }}" ]]; then
      echo "::error::auto_approve_app_id and auto_approve_private_key are required when auto_approve is true"
      exit 1
    fi

[arein]

2nd thought - SOC2 Type 2 binds us to get human reviews. So we will need to get this past our SOC 2 Type 2 auditor before rolling it out