Skip to content

docs: comprehensive guide for maintaining a repo with agentic workflows #26072

@lpcox

Description

@lpcox

Summary

Create a comprehensive guide page (in the docs site under guides/) for repository maintainers who manage repos with agentic workflows at scale. The guide should focus on three core challenges:

  1. Managing the volume of incoming issues and PRs — especially from unknown or untrusted users
  2. Ensuring agents prioritize important tasks from trustworthy sources
  3. Debugging failed workflows — using the Copilot CLI and /agent agentic-workflows

Motivation

Today, the docs cover individual features in isolation (integrity filtering reference, frontmatter reference, repo-assist workflow). But there is no single guide that ties these together from a maintainer's perspective: how do I set up my repo so that agentic workflows process trusted work first, filter out noise, and scale safely?

Maintainers running agentic workflows on public repos face unique challenges:

  • Unknown users can open issues/PRs that trigger agent runs, consuming compute and tokens
  • Without integrity filtering, agents may act on low-quality or adversarial input
  • Without repo-assist, there is no triage layer to route work appropriately
  • The relationship between integrity levels, reactions, and policy configuration is not obvious
  • When workflows fail, maintainers need efficient ways to diagnose and fix issues

Proposed Content

1. Setting Up Repo-Assist as a Triage Layer

  • What repo-assist does and why it matters at scale
  • Configuring repo-assist to classify and prioritize incoming issues/PRs
  • Using repo-assist to assign agents to tasks based on trust signals
  • Example: routing trusted contributor issues to specialized agents, deferring unknown user issues for human review

2. Integrity Filtering Deep Dive

  • Integrity levels explained: none, member, approved, merged — when to use each
  • Practical guidance: choosing the right min-integrity level for different workflow types
    • Triage workflows: lower integrity (see more content, but act cautiously)
    • Code-modifying workflows: higher integrity (only act on trusted input)
  • Reactions as trust signals (proxy-only, MCPG >= v0.2.18):
    • endorsement-reactions: how thumbs-up/heart from trusted members can promote content
    • disapproval-reactions: how thumbs-down/confused can demote content
    • Default values and how to customize them
    • Limitation: reactions only work in proxy mode (not gateway mode)
  • Other policy inputs:
    • allowed-repos — scoping which repos the agent can read
    • endorser-min-integrity — minimum integrity level for reaction endorsers
    • disapproval-integrity — integrity level assigned to disapproved content

3. Scaling Strategies

  • Combining repo-assist + integrity filtering for a defense-in-depth approach
  • Token budget awareness: how filtering reduces unnecessary token consumption
  • Rate limiting workflows to avoid runaway costs
  • Monitoring: using firewall logs and audit runs to track filtering decisions

4. Debugging Failed Workflows

  • Using the Copilot CLI (copilot) for debugging:
    • Running gh aw logs <workflow-name> to download and view workflow logs
    • Running gh aw audit <run-id> to audit a specific workflow run
    • Interpreting firewall logs, DIFC filtering events, and token usage data
  • Using /agent agentic-workflows in GitHub Copilot Chat:
    • Invoking the agent with a debug intent (e.g., "debug workflow X" or "why did run [ca] Update CLI Versions: Claude Code 2.1.22, Copilot CLI 0.0.397 #12345 fail?")
    • The agent automatically routes to the debug prompt, which analyzes logs, identifies missing tools, investigates MCP configuration issues, and suggests fixes
    • Example workflows: "Analyze the logs for the last failed run of repo-assist" → agent downloads logs, identifies the root cause, and proposes a fix
  • Common failure patterns and how to diagnose them:
    • Missing tool calls (tool not configured or MCP server unavailable)
    • Authentication failures (token permissions, MCP server tokens)
    • Integrity filtering blocking expected content (check DIFC_FILTERED events in firewall logs)
    • Safe output validation failures (malformed structured output)
    • Token budget exhaustion (use token usage logs in firewall artifacts)
  • Iterative debugging workflow:
    1. Check the workflow run summary in GitHub Actions UI
    2. Use gh aw audit <run-id> for a structured analysis
    3. Use /agent agentic-workflows to interactively diagnose complex issues
    4. Fix the workflow .md file → recompile → re-run

5. Worked Examples

  • Public repo: repo-assist triages issues → integrity filtering ensures agents only act on member/approved content → reactions allow maintainers to fast-track trusted external contributions
  • Inner-source repo: lower integrity threshold with broader allowed-repos, reactions for cross-team endorsement
  • High-security repo: merged-only integrity, no reactions, strict allowed-repos

Existing Reference Material

  • docs/src/content/docs/reference/integrity.md — integrity filtering reference
  • docs/src/content/docs/reference/frontmatter.md — frontmatter configuration
  • docs/src/content/docs/reference/github-tools.md — GitHub tools reference
  • docs/adr/25948-version-gated-integrity-reactions-for-mcpg-allow-only-policy.md — reactions ADR
  • .github/workflows/repo-assist.md (if present) or repo-assist documentation
  • .github/aw/debug-agentic-workflow.md — debug prompt used by /agent agentic-workflows

Location

docs/src/content/docs/guides/maintaining-repos.md (or similar)

Acceptance Criteria

  • Single guide page covering repo-assist, integrity filtering, reactions, debugging, and scaling
  • Clear decision trees for choosing integrity levels
  • Debugging section with Copilot CLI commands and /agent agentic-workflows usage
  • Common failure patterns with diagnostic steps
  • At least 3 worked examples (public, inner-source, high-security)
  • Cross-links to existing reference pages
  • Follows Diátaxis "how-to guide" format

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions