Skip to content

Structural check: flag version/preview/beta features without stability notes #50

@dacharyc

Description

@dacharyc

Context

From a PR review patterns audit across 7 MongoDB Agent Skills, version-dependent and preview features without clear gating were identified in 8 instances across 2 PRs (9, 10). As more skills cover newer features, this pattern will grow.

Proposed check

Add two related checks for version and preview awareness:

Check 1: Preview/beta/feature-flag without stability notes

Scan for terms like "preview", "beta", "feature flag", "experimental", "early access" in skill content. Flag when these terms appear without nearby context about:

  • Production readiness / stability implications
  • API stability guarantees (or lack thereof)
  • Expected GA timeline or tracking mechanism

Severity: warning
Heuristic: Flag if the term appears but no sentence within 5 lines contains words like "stability", "production", "GA", "generally available", "not recommended for", "subject to change", "may change".

Check 2: Version requirements buried mid-document

Detect version requirements (patterns like MongoDB X.Y+, v7.0+, Atlas X.Y, requires version) that appear after the first 15 lines of a file. Version prerequisites should be stated upfront so agents can short-circuit before reading substantial content.

Severity: warning (info for reference files, warning for SKILL.md)
Message: Version requirement "{requirement}" found at line {N}. Consider moving version prerequisites to the top of the file so agents can evaluate applicability early.

Examples from PR reviews

PR Issue
9 No upfront version prerequisite (MongoDB 8.0+) — agent may read substantial content before discovering it doesn't apply
9 "Preview" label without explaining implications for production use
10 previewFeatures flag with no tracking for when to remove
9 Hybrid search path should check MongoDB version before proceeding

Related

Part of a series of checks derived from PR review pattern analysis.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions