Skip to content

Auto-triage: label-gated follow-up, partial-progress acknowledgement, and /triage on-demand command#4324

Open
priyankatiwari08 wants to merge 4 commits into
dotnet:mainfrom
priyankatiwari08:issue-triage-followup
Open

Auto-triage: label-gated follow-up, partial-progress acknowledgement, and /triage on-demand command#4324
priyankatiwari08 wants to merge 4 commits into
dotnet:mainfrom
priyankatiwari08:issue-triage-followup

Conversation

@priyankatiwari08
Copy link
Copy Markdown
Contributor

@priyankatiwari08 priyankatiwari08 commented May 29, 2026

Summary

Today the SqlClient Issue Auto-Triage workflow runs exactly once — when an issue is opened. When that initial summary reports ⚠️ Missing: and asks the author to supply more info, the author's reply currently produces no new triage analysis, even though the previous summary is effectively stale.

This change extends the workflow with three new capabilities, all controlled cheaply at the YAML if: level so the Copilot agent only boots when worthwhile:

  1. Label-gated follow-up triage — when an initial triage flags ⚠️ Missing:, the workflow applies an internal-state label and the issue author's subsequent comments re-trigger the workflow until the env is complete.
  2. Partial-progress acknowledgement — if the author supplies some but not all required env fields, the follow-up posts an updated summary acknowledging what's now present and re-asking for what's still missing, keeping the label on.
  3. /triage on-demand command — repo maintainers can comment /triage on any issue to force a fresh triage analysis, bypassing label state and prior-summary counts.

Behavior

Situation Activation gate (if:) Agent action
Issue opened runs Initial triage; adds Auto-Triage: Waiting for Author label if env is missing
Author comment, label present, supplies no new env field ("will share soon") runs (boots agent) Silent noop — no comment posted, label stays
Author comment, label present, supplies some new env fields runs Posts ⚠️ Partial: received X, Y; still missing: A, B summary, label stays
Author comment, label present, supplies all remaining env fields runs Posts complete summary, removes label
Author comment after label was removed skipped ($0)
Comment from anyone other than the issue author skipped ($0)
Comment on a pull request (not an issue) skipped ($0)
Bot-authored comment skipped ($0)
/triage from a repo OWNER / MEMBER / COLLABORATOR runs Fresh triage regardless of label state; label untouched
/triage from a non-maintainer (drive-by commenter) skipped ($0)

How it's enforced

Layer 1 — Workflow-level if: (free, $0 if it returns false):

if: |
  github.event_name == 'issues' ||
  (github.event_name == 'issue_comment'
   && github.event.issue.pull_request == null
   && !endsWith(github.event.comment.user.login, '[bot]')
   && (
     (startsWith(github.event.comment.body, '/triage')
      && contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association))
     ||
     (github.event.comment.user.login == github.event.issue.user.login
      && contains(github.event.issue.labels.*.name, 'Auto-Triage: Waiting for Author'))
   ))

This filters out the obvious non-matches (bot commenter, non-author commenter on un-labeled issue, PR comments, non-maintainer /triage) before the agent activation job runs. Skipped activation jobs use zero compute.

Layer 2 — Prompt-level "Follow-up routing": instructs the agent to compute BEFORE and AFTER env-field snapshots and route into one of three sub-cases — No progress → silent noop, Partial progress → fresh summary + keep label, Complete → fresh summary + remove label. The agent is responsible for adding/removing the label via the new add-labels / remove-labels safe-outputs (both restricted to the single label Auto-Triage: Waiting for Author via allowed:).

safe-outputs.add-comment.max stays at 1 per run (the workflow simply runs multiple times, once per author comment that passes the gate). hide-older-comments: true collapses the previous summary so only the latest is visible.

New label

This PR introduces one new internal-state label:

  • Auto-Triage: Waiting for Author — applied by the workflow when env fields are missing, removed when they are complete. Maintainers should not apply/remove it manually; the workflow manages it.

The safe-outputs.add-labels.allowed / remove-labels.allowed lists restrict the workflow to managing only this one label.

Files changed

  • .github/workflows/issue-triage.md — new if: expression, add-labels / remove-labels safe-outputs, three-scenario overview (Initial / Follow-up / On-demand), Label-managed state section, Follow-up routing section, ⚠️ Partial: env-row format, label-management actions.
  • .github/workflows/issue-triage.lock.yml — regenerated by gh aw compile v0.72.1. Included in this PR — no maintainer compile step required.

Validation

End-to-end tested on a personal fork (priyankatiwari08/SqlClient-test-prtiwar):

Test Expected Observed
Open issue with missing env Initial summary ⚠️ Missing: + label added
Author comments "okay will share details" Silent noop, no comment, label stays
Author comments with some env fields ⚠️ Partial: summary, label stays
Author comments with remaining fields Complete summary, label removed
Author comments again after label removed Activation skipped ($0)
Maintainer comments /triage on any issue Fresh triage runs, label untouched

Cost shape

  • Initial triage: 1 agent run per opened issue (unchanged from today).
  • Follow-up triage: bounded by author comments while the label is present. Non-substantive author comments cost a single inference call (no public spam); substantive ones post a fresh summary.
  • All other comment events (non-author, non-maintainer, bots, PR comments, author comments after label removal): skipped at the YAML if: level, zero compute cost.

Copilot AI review requested due to automatic review settings May 29, 2026 12:49
@github-project-automation github-project-automation Bot moved this to To triage in SqlClient Board May 29, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends the SqlClient issue auto-triage workflow to optionally run one follow-up triage pass when (and only when) the original triage reported missing environment details and the issue author subsequently comments, with guardrails to avoid repeated/irrelevant reruns.

Changes:

  • Adds issue_comment.created as a trigger with a workflow-level if: gate to avoid booting the agent for non-author / bot / PR-comment events.
  • Introduces a prompt-level “Follow-up gate” that re-checks existing comments and calls noop unless the follow-up criteria are met.
  • Increases safe-outputs.add-comment.max to allow up to two triage summaries per issue (initial + one follow-up), relying on hide-older-comments for UX.

Comment on lines +27 to +31
permissions:
contents: read
issues: read
pull-requests: read

Extends the issue auto-triage workflow so a second (and final) triage
summary is produced when:
  1. The first triage flagged missing environment fields, AND
  2. The original issue author responds with a follow-up comment.

No further triage runs after the second summary. Comments from other
users, bots, or on PRs do not trigger re-triage.

NOTE: Maintainers must run 'gh aw compile' to regenerate
.github/workflows/issue-triage.lock.yml from this .md source.
@priyankatiwari08 priyankatiwari08 force-pushed the issue-triage-followup branch from 9f32d55 to ad13622 Compare May 29, 2026 12:56
Copilot AI review requested due to automatic review settings May 29, 2026 14:41
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread .github/workflows/issue-triage.lock.yml Outdated
Comment on lines 1 to 3
# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"e98945dcd5993e27b71e3600298e58d5205517079d21b6024c05b338fbfded9c","compiler_version":"v0.72.1","strict":true,"agent_id":"copilot"}
# gh-aw-manifest: {"version":1,"secrets":["COPILOT_GITHUB_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"bc56a0cad2f450c562810785ef38649c04db812a","version":"v0.72.1"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.25.41","digest":"sha256:cb2b565d070116d4b67e355775340528b5a2c3cb18b2c9049638bcc2df681770","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.25.41@sha256:cb2b565d070116d4b67e355775340528b5a2c3cb18b2c9049638bcc2df681770"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.25.41","digest":"sha256:fadd0de387209f69a9a7a1b8722bb5e7fdfb80ba9749a5c60f0e4cd7582a74d0","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.25.41@sha256:fadd0de387209f69a9a7a1b8722bb5e7fdfb80ba9749a5c60f0e4cd7582a74d0"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.25.41","digest":"sha256:1260445d25968dbf3ae70143964177a0e5914cf2ce07a6117f7d3caec6c3e3c4","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.25.41@sha256:1260445d25968dbf3ae70143964177a0e5914cf2ce07a6117f7d3caec6c3e3c4"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.6","digest":"sha256:2bb8eef86006a4c5963c55616a9c51c32f27bfdecb023b8aa6f91f6718d9171c","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.6@sha256:2bb8eef86006a4c5963c55616a9c51c32f27bfdecb023b8aa6f91f6718d9171c"},{"image":"ghcr.io/github/github-mcp-server:v1.0.3","digest":"sha256:2ac27ef03461ef2b877031b838a7d1fd7f12b12d4ace7796d8cad91446d55959","pinned_image":"ghcr.io/github/github-mcp-server:v1.0.3@sha256:2ac27ef03461ef2b877031b838a7d1fd7f12b12d4ace7796d8cad91446d55959"},{"image":"node:lts-alpine","digest":"sha256:d1b3b4da11eefd5941e7f0b9cf17783fc99d9c6fc34884a665f40a06dbdfc94f","pinned_image":"node:lts-alpine@sha256:d1b3b4da11eefd5941e7f0b9cf17783fc99d9c6fc34884a665f40a06dbdfc94f"}]}
# ___ _ _
@priyankatiwari08 priyankatiwari08 changed the title Auto-triage: post a one-time follow-up summary when author supplies missing env info Auto-triage: label-gated follow-up, partial-progress acknowledgement, and /triage on-demand command May 29, 2026
@priyankatiwari08 priyankatiwari08 marked this pull request as ready for review May 29, 2026 16:37
@priyankatiwari08 priyankatiwari08 requested review from a team and Copilot May 29, 2026 16:37
@priyankatiwari08 priyankatiwari08 requested a review from a team as a code owner May 29, 2026 16:37
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment on lines +33 to +39
&& (
(startsWith(github.event.comment.body, '/triage')
&& contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association))
||
(github.event.comment.user.login == github.event.issue.user.login
&& contains(github.event.issue.labels.*.name, 'Auto-Triage: Waiting for Author'))
))
@@ -1,4 +1,4 @@
# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"20b8fbbfaae4d08b3206fb6b2f4000e43b8915e37f5afc9a534056f8ce9a563b","compiler_version":"v0.72.1","strict":true,"agent_id":"copilot"}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we check in this lock file?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: To triage

Development

Successfully merging this pull request may close these issues.

3 participants