Skip to content

feat: add Tier selection logic to debug.md and plan-research.md (§7.2 audit fix) #194

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
maystudios merged 1 commit into from
Mar 25, 2026
Merged

feat: add Tier selection logic to debug.md and plan-research.md (§7.2 audit fix) #194

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 34 additions & 6 deletions templates/workflows/debug.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,32 @@ Wait for user confirmation to proceed (unless symptoms are fully specified in `$

---

## Step 3: Reproduce — Confirm the Problem
## Step 3: Tier Selection

Before spawning debugging agents, evaluate the execution tier:

1. **Check Tier 2 availability:**
- Verify `CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS` env var is set (MaxsimCLI installer enables this by default)
- Read `config.execution.parallelism.competition_strategy` from `.claude/maxsim/config.json`

Comment on lines +101 to +104
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Tier 2 availability check here only looks at the env var + config, but the subsequent condition text also references “feature not yet stable”. In execute.md’s reference tier selection, this is handled by doing a lightweight TeamCreate probe before deciding Tier 2 is available. Consider adding the same probe step here (or rewording the condition) so the workflow doesn’t attempt Agent Teams when the feature is enabled but non-functional.

Copilot uses AI. Check for mistakes.
2. **If Tier 2 is available AND `competition_strategy` is `deep`:**
- Use Agent Teams collaborative debugging pattern:
- `TeamCreate` to create a debug team
Comment on lines +105 to +107
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PROJECT.md §7.2’s tier selection table lists “Collaborative debugging → Tier 2 (Agent Teams)”, but this workflow only opts into Tier 2 when competition_strategy is deep. If competition_strategy is meant to gate Tier 2 usage outside competitive implementation, the spec/reference docs should be updated to reflect that; otherwise consider removing this gating for debugging.

Copilot uses AI. Check for mistakes.
- Spawn 2-3 hypothesis investigator teammates, each exploring a different root-cause theory
- Teammates use `SendMessage` to challenge each other's hypotheses — a hypothesis only survives if it holds up to adversarial cross-examination
- Coordinator collects surviving theories and selects the most evidence-backed root cause
- This fights LLM anchoring bias (first plausible hypothesis wins by default)

3. **If Tier 2 is NOT available (env var unset, feature not yet stable, or `competition_strategy` is `none`/`quick`/`standard`):**
- **Graceful degradation to Tier 1** — inform the user:
> "Collaborative debugging: using Tier 1 subagents (Agent Teams not available or not required). Each agent investigates independently; coordinator synthesizes findings."
- Proceed with Tier 1 subagents as described below (current default path, fully functional)

> **Graceful degradation guarantee:** Per PROJECT.md §7.2, if Agent Teams are unavailable (env var not set, unsupported plan, or feature not yet stable), MaxsimCLI falls back to Tier 1 subagents for all workflows. The user is informed but not blocked.

---

## Step 4: Reproduce — Confirm the Problem

Spawn a verifier agent to drive steps 1–4 of the `systematic-debugging` skill (Reproduce, Hypothesize, Isolate, Verify):

Expand Down Expand Up @@ -171,7 +196,7 @@ Options:
3. Manual — I'll fix it myself (close debug issue as resolved)
```

Proceed to step 4 (ExitPlanMode + Fix).
Proceed to step 5 (ExitPlanMode + Fix).

**CHECKPOINT reached:**

Expand All @@ -196,7 +221,7 @@ If the user provides more context, update `$DESCRIPTION` / `$ERROR` / `$REPRO_ST

---

## Step 4: ExitPlanMode — Confirm the Fix
## Step 5: ExitPlanMode — Confirm the Fix

Call `ExitPlanMode`.

Expand All @@ -219,7 +244,7 @@ Wait for user confirmation before spawning the executor.

---

## Step 5: Fix — Implement the Fix
## Step 6: Fix — Implement the Fix

Spawn an executor agent to implement the fix (step 5 of the `systematic-debugging` skill):

Expand Down Expand Up @@ -256,7 +281,7 @@ Extract `$COMMIT_HASH` and `$FIX_SUMMARY` from executor output.

---

## Step 6: Confirm — Verify the Fix
## Step 7: Confirm — Verify the Fix

Spawn a verifier agent to confirm the fix (step 6 of the `systematic-debugging` skill):

Expand Down Expand Up @@ -294,7 +319,7 @@ If FAIL: display issues, ask user whether to retry the fix or accept as-is.

---

## Step 7: Report — Post Results and Close Issue
## Step 8: Report — Post Results and Close Issue

Post a resolution comment to the GitHub Issue:

Expand Down Expand Up @@ -347,6 +372,9 @@ Verification: $VERIFY_STATUS
- [ ] Open debug Issues checked on GitHub before starting a new session
- [ ] GitHub Issue created with label "type:bug" and moved to "In Progress"
- [ ] EnterPlanMode called before presenting the debugging plan
- [ ] Tier selection evaluated: CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS env var and config.execution.parallelism.competition_strategy checked
- [ ] Tier 2 path (Agent Teams collaborative debugging) used when env var set and strategy is `deep`
- [ ] Graceful degradation to Tier 1 with user notification when Tier 2 is not available
- [ ] Verifier agent spawned with subagent_type="verifier" and isolation="worktree"
- [ ] Verifier drives systematic-debugging steps 1–4 (Reproduce, Hypothesize, Isolate, Verify)
- [ ] Checkpoint, INCONCLUSIVE, and ROOT_CAUSE return paths handled
Expand Down
38 changes: 33 additions & 5 deletions templates/workflows/plan-research.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,32 @@ Each domain should be independently investigatable by a separate agent.

Select 5-10 domains most relevant to this phase. Define a focused research question for each.

## Step 6: Spawn Parallel Research Agents
## Step 6: Tier Selection

Before spawning researcher agents, evaluate the execution tier:

1. **Check Tier 2 availability:**
- Verify `CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS` env var is set (MaxsimCLI installer enables this by default)
- Read `config.execution.parallelism.competition_strategy` from `.claude/maxsim/config.json`

Comment on lines +119 to +122
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Tier 2 availability check here only covers the env var + config, but the later condition text also includes “feature not yet stable”. execute.md’s tier selection covers this by doing a lightweight TeamCreate probe before opting into Tier 2. Consider adding the same probe step here (or rewording the condition) to avoid attempting Agent Teams when enabled but non-functional.

Copilot uses AI. Check for mistakes.
2. **If Tier 2 is available AND `competition_strategy` is `deep`:**
- Use Agent Teams cross-checking pattern:
- `TeamCreate` to create a research team
- Spawn 5-10 researcher teammates, each investigating a different domain independently
- Researchers use `SendMessage` to share findings and challenge each other's sources, conclusions, and recommended approaches
- Coordinator collects validated findings, flagging areas where agents agree or disagree, and synthesizes a higher-confidence research document
- This improves research quality by exposing weak sources and conflicting evidence before the planner acts on findings

3. **If Tier 2 is NOT available (env var unset, feature not yet stable, or `competition_strategy` is `none`/`quick`/`standard`):**
- **Graceful degradation to Tier 1** — inform the user:
> "Research gathering: using Tier 1 subagents (Agent Teams not available or not required). Each researcher works independently; coordinator collates findings."
- Proceed with Tier 1 batch agents as described below (current default path, fully functional)

> **Graceful degradation guarantee:** Per PROJECT.md §7.2, if Agent Teams are unavailable (env var not set, unsupported plan, or feature not yet stable), MaxsimCLI falls back to Tier 1 subagents for all workflows. The user is informed but not blocked.

Comment on lines +117 to +137
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This workflow introduces a Tier 2 (Agent Teams) path for “Research gathering”, but PROJECT.md §7.2’s tier selection table currently specifies “Research gathering → Tier 1 (Subagents)”. Either update this step to match the current spec (Tier 1 only) or adjust the text to explicitly frame Tier 2 research cross-checking as an experimental extension and align the spec/reference docs accordingly.

Suggested change
Before spawning researcher agents, evaluate the execution tier:
1. **Check Tier 2 availability:**
- Verify `CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS` env var is set (MaxsimCLI installer enables this by default)
- Read `config.execution.parallelism.competition_strategy` from `.claude/maxsim/config.json`
2. **If Tier 2 is available AND `competition_strategy` is `deep`:**
- Use Agent Teams cross-checking pattern:
- `TeamCreate` to create a research team
- Spawn 5-10 researcher teammates, each investigating a different domain independently
- Researchers use `SendMessage` to share findings and challenge each other's sources, conclusions, and recommended approaches
- Coordinator collects validated findings, flagging areas where agents agree or disagree, and synthesizes a higher-confidence research document
- This improves research quality by exposing weak sources and conflicting evidence before the planner acts on findings
3. **If Tier 2 is NOT available (env var unset, feature not yet stable, or `competition_strategy` is `none`/`quick`/`standard`):**
- **Graceful degradation to Tier 1** — inform the user:
> "Research gathering: using Tier 1 subagents (Agent Teams not available or not required). Each researcher works independently; coordinator collates findings."
- Proceed with Tier 1 batch agents as described below (current default path, fully functional)
> **Graceful degradation guarantee:** Per PROJECT.md §7.2, if Agent Teams are unavailable (env var not set, unsupported plan, or feature not yet stable), MaxsimCLI falls back to Tier 1 subagents for all workflows. The user is informed but not blocked.
Before spawning researcher agents, confirm the execution tier for this workflow:
1. **Lock research gathering to Tier 1 (Subagents):**
- Per PROJECT.md §7.2, the "Research gathering" phase MUST use Tier 1 subagents only.
- Do **not** use Agent Teams (Tier 2) for this workflow, even if `CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS` is set or `competition_strategy` is `deep`.
2. **User-facing confirmation:**
- Inform the user explicitly that research is running in Tier 1 mode:
> "Research gathering: using Tier 1 subagents as required by PROJECT.md §7.2. Each researcher works independently; the coordinator collates and synthesizes findings."
- Then proceed with Tier 1 batch agents as described below (this is the required and fully supported path).
> **Graceful degradation guarantee (project-wide):** Per PROJECT.md §7.2, if Agent Teams are unavailable (env var not set, unsupported plan, or feature not yet stable), MaxsimCLI falls back to Tier 1 subagents for all workflows. This specific research sub-workflow is **always** Tier 1 by design, so it does not depend on Agent Teams availability.

Copilot uses AI. Check for mistakes.
---

## Step 7: Spawn Parallel Research Agents

Display:
```
Expand Down Expand Up @@ -174,7 +199,7 @@ Return findings as structured markdown:
Spawn all research agents in parallel using `run_in_background=true`. Do not await any
individual agent before spawning the next.

## Step 7: Collect and Aggregate Findings
## Step 8: Collect and Aggregate Findings

After all background agents complete, collect their results.

Expand Down Expand Up @@ -221,7 +246,7 @@ For each agent result:
*Agents: {N} researchers across {N} domains*
```

## Step 8: Post Research to GitHub
## Step 9: Post Research to GitHub

Post the consolidated research as a comment on the phase GitHub Issue:

Expand All @@ -242,7 +267,7 @@ Display confirmation:
Research complete. Findings from {N} agents posted to GitHub Issue #{phase_issue_number}.
```

## Step 9: Handle Agent Failures
## Step 10: Handle Agent Failures

If any research agent fails or returns no findings:

Expand All @@ -262,7 +287,7 @@ Research partially failed. Only {N}/{total} agents returned findings.

Wait for user choice.

## Step 10: Return to Orchestrator
## Step 11: Return to Orchestrator

After research is posted to GitHub (or research is skipped), return control to the plan.md
orchestrator. Do NOT show gate confirmation or next steps -- the orchestrator handles the gate
Expand All @@ -275,6 +300,9 @@ between Research and Planning.
- [ ] Existing research detected from GitHub Issue comment (<!-- maxsim:type=research --> marker) and reused unless --force-research
- [ ] Phase context read from GitHub Issue (context comment + issue body)
- [ ] 5-10 research domains defined based on phase characteristics
- [ ] Tier selection evaluated: CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS env var and config.execution.parallelism.competition_strategy checked
- [ ] Tier 2 path (Agent Teams cross-checking) used when env var set and strategy is `deep`
- [ ] Graceful degradation to Tier 1 with user notification when Tier 2 is not available
- [ ] All research agents spawned in parallel using run_in_background=true
- [ ] Agent tool used (not Task) for all agent spawning
- [ ] Each agent investigates a distinct, focused domain
Expand Down
Loading