diff --git a/.github/skills/update-otel-genai-conventions/SKILL.md b/.github/skills/update-otel-genai-conventions/SKILL.md index 7b5d4f531b8..34665c8b3dd 100644 --- a/.github/skills/update-otel-genai-conventions/SKILL.md +++ b/.github/skills/update-otel-genai-conventions/SKILL.md @@ -1,20 +1,102 @@ --- name: update-otel-genai-conventions description: >- - Analyze OpenTelemetry semantic-conventions releases or PRs with gen-ai changes - and produce compensating change plans for dotnet/extensions. Use when asked to - "update OTel conventions", "check semantic-conventions release", "plan gen-ai - convention changes", review gen-ai convention PRs, or when given a release - version, URL, or PR number/URL from open-telemetry/semantic-conventions with - area:gen-ai changes. Also use when asked to "update OpenTelemetry", "bump - semconv version", or "what changed in semantic-conventions vX.Y". + Analyze OpenTelemetry GenAI semantic-conventions changes (PRs, CHANGELOG + snapshots, date ranges, or releases when they exist) and produce + compensating change plans for dotnet/extensions. The conventions now live + in their own repo at open-telemetry/semantic-conventions-genai and cover + gen-ai, mcp, openai, anthropic, aws-bedrock, and azure-ai-inference areas; + the previous home was open-telemetry/semantic-conventions under the + area:gen-ai label. Use when asked to "update OTel conventions", "check + semantic-conventions-genai", "plan gen-ai convention changes", "bump genai + semconv version", review gen-ai/MCP/provider convention PRs, or when given + a PR number/URL, CHANGELOG snapshot, date range, or release version from + either repo. Also use for "update OpenTelemetry", "bump semconv version", + or "what changed in semantic-conventions-genai". agent: 'agent' tools: ['github/*', 'sql'] --- # Update OTel Gen-AI Conventions -Analyze OpenTelemetry [semantic-conventions](https://github.com/open-telemetry/semantic-conventions) releases or PRs with `area:gen-ai` changes and produce compensating updates in `dotnet/extensions`. +Analyze OpenTelemetry GenAI semantic-conventions changes — PRs, `CHANGELOG.md` snapshots, date ranges, or releases — primarily from [`open-telemetry/semantic-conventions-genai`](https://github.com/open-telemetry/semantic-conventions-genai), and produce compensating updates in `dotnet/extensions`. See the [Migration Note](#migration-note) below for the recent move from the core `semantic-conventions` repo and how the two repos relate. + +## Migration Note + +The OpenTelemetry GenAI semantic conventions recently moved from +[`open-telemetry/semantic-conventions`](https://github.com/open-telemetry/semantic-conventions) +(where they lived under the `area:gen-ai` label) to a dedicated repo: +[`open-telemetry/semantic-conventions-genai`](https://github.com/open-telemetry/semantic-conventions-genai). +The new repo also hosts `mcp`, `openai`, `anthropic`, `aws-bedrock`, and +`azure-ai-inference` conventions. + +Implications for this skill: + +- **Primary input source** is the new repo. The old repo is a **fallback** + for catch-up audits, historical context, and any in-flight PR that + started there before the migration. +- **No `area:gen-ai` label** exists in the new repo — every PR is in scope + by definition. The label still applies for old-repo catch-up work. +- **No releases yet** in the new repo (as of this writing). The + `CHANGELOG.md` `Unreleased` section is the most reliable "what's new" + view; pin a snapshot via commit SHA / ref for reproducible audits. +- **GenAI version is now independent** of core semconv. The schema URL + `https://opentelemetry.io/schemas/gen-ai/X.Y.Z` carries the gen-ai + version (e.g. `1.42.0`); core semconv (`v1.41.0` at the time of writing) + is a separate dependency declared in the new repo's `versions.env`. +- **Spec URL `https://opentelemetry.io/docs/specs/semconv/gen-ai/` is + unchanged** — the published page still resolves and renders the gen-ai + spec from the new repo, so the `` in dotnet/extensions source + files does not need to change. +- **PR numbering** is not interchangeable across repos. Always + disambiguate with `open-telemetry/semantic-conventions#NNN` or + `open-telemetry/semantic-conventions-genai#NNN` when there is any risk + of collision. +- **Doc-comment wording** in dotnet/extensions source still reads + "Semantic Conventions for Generative AI systems v1.XX". The next + convention-update PR should migrate this to "GenAI Semantic Conventions + vX.Y.Z" — see [references/file-inventory.md §Version References](references/file-inventory.md#version-references). + +When a referenced PR number doesn't resolve in the new repo, check the +old repo before assuming the input is invalid. + +## Cross-repo applicability + +This skill lives in `dotnet/extensions` and its file paths, build +commands, and PR-description conventions are tuned for that repo. The +new conventions repo also hosts provider-specific areas (`anthropic`, +`aws-bedrock`) whose dotnet instrumentation lives in **other** SDK +repositories that we contribute to: + +| Upstream area | Repository | Notes | +|---|---|---| +| `anthropic` | [`anthropics/anthropic-sdk-csharp`](https://github.com/anthropics/anthropic-sdk-csharp) | Anthropic's official .NET SDK. | +| `aws-bedrock` | [`aws/aws-sdk-net`](https://github.com/aws/aws-sdk-net) | AWS Bedrock instrumentation lives in the `BedrockRuntime` service library (`AWSSDK.BedrockRuntime`) inside the AWS SDK monorepo. | + +The skill can optionally be applied in those repos with the following +adaptations: + +- **Apply** the convention analysis, classification framework + ([references/change-classification.md](references/change-classification.md)), + audit-table shape, area routing, doc-comment wording target + ("GenAI Semantic Conventions vX.Y.Z"), version-reference grep + recipes, and PR-description shape + ([references/pr-description.md](references/pr-description.md)). +- **Do not assume** dotnet/extensions-specific paths + (`src/Libraries/Microsoft.Extensions.AI*/`), the + `OpenTelemetryConsts.cs` constants layout, the API-baseline workflow, + or the build/test commands in + [references/build-commands.md](references/build-commands.md). Use the + target repo's own conventions for code structure, constants + organization, and validation. +- **Scope by repo**: when running in another repo, the in-scope upstream + area is the one that repo instruments (e.g. `anthropic` in + `anthropics/anthropic-sdk-csharp`, `aws-bedrock` in the + `BedrockRuntime` library of `aws/aws-sdk-net`). Other areas are out + of scope from that repo's perspective even though they remain in + scope for `dotnet/extensions`. +- **Pre-flight** still applies — search open PRs in the target repo for + prior coverage before producing a plan. ## Mode Detection @@ -33,18 +115,58 @@ If unclear, default to **Mode 5** (Plan-then-Implement) and offer Mode 3 as an a ## Input Handling -The user provides one of: -- A **semantic-conventions release version** (e.g. `v1.40.0`) → fetch from `https://github.com/open-telemetry/semantic-conventions/releases/tag/{version}` -- A **release URL** → fetch the release notes directly -- One or more **PR references** from `open-telemetry/semantic-conventions` with `area:gen-ai` changes — as URLs, PR numbers (e.g. `#3598`), or `open-telemetry/semantic-conventions#3598` format - -When PR numbers are given without a full URL, resolve them against the `open-telemetry/semantic-conventions` repository. +The new repo (`open-telemetry/semantic-conventions-genai`) does not yet +publish releases. Until it does, the user typically provides one of: + +- **PR references** in the new repo — full URL, `#NNN`, or + `open-telemetry/semantic-conventions-genai#NNN` form. (No `area:` label + filter is needed: the new repo is gen-ai-focused by definition.) +- **A `CHANGELOG.md` snapshot** — a commit SHA, branch ref, or a + `https://github.com/open-telemetry/semantic-conventions-genai/blob/{ref}/CHANGELOG.md` + URL pinning the `Unreleased` section at a point in time. +- **A date range or "since last update"** — list of PRs merged to the + new repo's `main` between two refs / dates. +- **A release version or release URL** — once releases exist + (`https://github.com/open-telemetry/semantic-conventions-genai/releases/tag/{version}`). + +For **catch-up or historical work**, the old repo is still valid input: + +- A **semantic-conventions release version** (e.g. `v1.40.0`) → fetch from + `https://github.com/open-telemetry/semantic-conventions/releases/tag/{version}` + (filter to `area:gen-ai` PRs). +- A **release URL** from the old repo → fetch the release notes directly. +- **PR references** from the old repo — only the ones with `area:gen-ai`. + Use the `open-telemetry/semantic-conventions#NNN` form to disambiguate + from new-repo PR numbers. + +When PR numbers are given without a full URL, default to the **new** repo +and fall back to the **old** repo only if the PR doesn't exist in the new +repo or the user explicitly references the old repo. + +### In-scope areas + +The new repo hosts conventions for several areas, all of which this skill +covers (with placement guidance in +[references/implementation-patterns.md](references/implementation-patterns.md)): + +| Upstream area (`model//`) | Maps to in dotnet/extensions | +|---|---| +| `gen-ai`, `gen-ai/agent` | `Microsoft.Extensions.AI` core (e.g. `OpenTelemetryChatClient`) | +| `mcp` | Currently no instrumentation; forward-looking — flag as a watch-list item if changes appear | +| `openai` | `Microsoft.Extensions.AI.OpenAI` | +| `anthropic` | Out of scope for `dotnet/extensions` today (no provider package). Implications land in [`anthropics/anthropic-sdk-csharp`](https://github.com/anthropics/anthropic-sdk-csharp) — apply this skill there per [Cross-repo applicability](#cross-repo-applicability). | +| `aws-bedrock` | Out of scope for `dotnet/extensions` today (no provider package). Implications land in the `BedrockRuntime` service library of [`aws/aws-sdk-net`](https://github.com/aws/aws-sdk-net) — apply this skill there per [Cross-repo applicability](#cross-repo-applicability). | +| `azure-ai-inference` | Corresponding provider package, if/when one exists in this repo; otherwise out of scope | + +When classifying a change, identify its area from the path under +`model//` in the new repo (or from the doc page under +`docs//`). ### Existing dotnet/extensions PR Preflight -For **Mode 1: Audit** and **Mode 5: Plan-then-Implement**, after resolving the requested release or upstream PR identifiers but before doing deeper release analysis or creating a plan, search open pull requests in `dotnet/extensions` to determine whether another PR already appears to cover the requested GenAI/OpenTelemetry semantic-conventions update. +For **Mode 1: Audit** and **Mode 5: Plan-then-Implement**, after resolving the requested input identifiers but before doing deeper analysis or creating a plan, search open pull requests in `dotnet/extensions` to determine whether another PR already appears to cover the requested update. -Search using the requested release version, release URL, or upstream semantic-conventions PR numbers, plus relevant terms such as `gen-ai`, `GenAI`, `semantic conventions`, `OpenTelemetry`, and `OTel`. If one or more likely matching PRs are open, report the PR number, title, author, URL, and the signal that matched. Then stop and state that the audit or plan is not proceeding because an open PR already appears to cover the update. +Search using the requested release version, CHANGELOG ref, date range, or upstream PR numbers, plus relevant terms such as `gen-ai`, `GenAI`, `semantic conventions`, `semantic-conventions-genai`, `semconv-genai`, `OpenTelemetry`, `OTel`, and any in-scope area name (`MCP`, `OpenAI`, `Anthropic`, `Bedrock`, `Azure AI Inference`) that matches the changes you're working from. If one or more likely matching PRs are open, report the PR number, title, author, URL, and the signal that matched. Then stop and state that the audit or plan is not proceeding because an open PR already appears to cover the update. Do not silently ignore search failures. If GitHub search/listing is unavailable, report the problem and ask the user whether to proceed without the preflight. @@ -60,7 +182,7 @@ For Step 4, read the source files listed in [references/file-inventory.md](refer ### PR Title and Description Guidance -When creating or updating a PR after implementing semantic-conventions changes, follow [references/pr-description.md](references/pr-description.md) for the title format and the changes-table shape. +When creating or updating a PR after implementing GenAI semantic-conventions changes (from either repo), follow [references/pr-description.md](references/pr-description.md) for the title format and the changes-table shape. --- @@ -71,7 +193,7 @@ Audit the current gen-ai semantic conventions implementation against the latest 1. Complete the **Existing dotnet/extensions PR Preflight** above. If a matching open PR exists, report it and stop. 2. **Determine the current implemented version**: Read the version reference from `OpenTelemetryChatClient.cs` doc comment to identify which convention version the codebase claims to implement 3. **Check for version drift**: Verify every file with a gen-ai semantic conventions version reference uses the same version. Use the search command from [references/file-inventory.md](references/file-inventory.md#version-references). If files reference different versions, flag that as a critical gap requiring investigation. -4. **Fetch the latest convention spec**: Read the current gen-ai semantic conventions from the [published spec](https://opentelemetry.io/docs/specs/semconv/gen-ai/) and the latest release notes +4. **Fetch the latest convention spec**: Read the current conventions from the [published spec](https://opentelemetry.io/docs/specs/semconv/gen-ai/) (rendered from the new repo) and cross-check against the source of truth in [`open-telemetry/semantic-conventions-genai`](https://github.com/open-telemetry/semantic-conventions-genai) — `docs//` for human-readable docs, `model//` for the YAML registry, and `schema-snapshot/` plus `README.md` for the current schema URL (e.g. `opentelemetry.io/schemas/gen-ai/X.Y.Z`). The new repo's gen-ai version is independent of core semconv, so verify both. Until releases exist in the new repo, use the latest `Unreleased` `CHANGELOG.md` entries or recently merged PRs as the "latest release notes" equivalent. 5. **Read all current source files** listed in [references/file-inventory.md](references/file-inventory.md) to understand what is actually implemented 6. **Cross-reference**: For each attribute, metric, event, and operation name defined in the conventions: - Is the constant defined in `OpenTelemetryConsts.cs`? @@ -101,7 +223,7 @@ Modes 2, 4, and 5 share the same implementation flow. See [references/implementa ## Mode 2: Autopilot -One-shot mode that analyzes the release and implements all changes in a single pass without intermediate review. Best for end-to-end execution when the user does not need a plan checkpoint. +One-shot mode that analyzes the upstream input (release, PRs, CHANGELOG snapshot, or date range) and implements all changes in a single pass without intermediate review. Best for end-to-end execution when the user does not need a plan checkpoint. 1. Complete the **Input Handling** analysis above 2. Build an internal work plan in working memory (do not write `plan.md`): @@ -119,8 +241,8 @@ Generate a structured prompt suitable for delegating to Copilot Coding Agent on 1. Complete the **Input Handling** analysis above 2. Read [references/prompt-template.md](references/prompt-template.md) for the template structure 3. Generate the prompt following the template, filling in: - - Background with links to the upstream release/PRs - - Changes audit table + - Background with links to the upstream input (release URL, CHANGELOG snapshot ref, date range, or PR URLs) + - Changes audit table (with **Area** column) - Required changes with exact file paths and code context from the current source - Test expectations referencing [references/testing-guide.md](references/testing-guide.md) - Validation steps @@ -146,10 +268,10 @@ Generate a plan and (after user review/approval) implement it. Best when the use **Phase A: Plan** — -1. Resolve the user's input to a semantic-conventions release or upstream PR identifiers +1. Resolve the user's input to one of: a release, PR identifiers, a `CHANGELOG.md` snapshot (commit SHA / ref), or a date range in the new repo (`open-telemetry/semantic-conventions-genai`). For catch-up work, accept upstream PRs from the old `open-telemetry/semantic-conventions` repo with `area:gen-ai` 2. Complete the **Existing dotnet/extensions PR Preflight** above. If a matching open PR exists, report it and stop without creating a plan. 3. Complete the **Analyzing the Release / PRs** analysis above -4. Create `plan.md` with a problem statement linking to the upstream release, a changes audit table, and a numbered list of work items. Each work item should call out the file(s) to modify, what code/constants/attributes to add, and which tests to update. +4. Create `plan.md` with a problem statement linking to the upstream input (release URL, CHANGELOG snapshot ref, date range, or list of PR URLs — whichever applies), a changes audit table, and a numbered list of work items. Each work item should call out the file(s) to modify, what code/constants/attributes to add, and which tests to update. 5. Pause for user review/approval before proceeding to Phase B **Phase B: Implement** — @@ -187,11 +309,12 @@ Critical knowledge from past PR reviews that should inform all modes: - **Fluent chains**: Use fluent Activity API chains (`.SetStatus(...).SetTag(...)`) rather than separate statements. - **Shared code**: Cross-cutting concerns (like exception logging) shared across multiple OpenTelemetry* clients belong in `src/Libraries/Microsoft.Extensions.AI/Common/`. Before adding a new helper, method, or internal type, search `Common/`, `TelemetryHelpers.cs`, `OpenTelemetryLog.cs`, and sibling OpenTelemetry* clients for existing logic with the same purpose — reuse or extend instead of introducing a parallel implementation. When the same helper is needed in 2+ places, factor it into `Common/` from the start. The same applies to parallel internal types: if a sibling client already defines a type with the same shape (same properties, same role, e.g. `RealtimeOtelFunction` vs `OtelFunction`), unify them under a single shared definition rather than letting each client carry its own copy. - **Test augmentation**: Prefer augmenting existing test assertions over creating new test methods. Check for existing tests that validate the same scenario. -- **Version references**: When bumping the convention version, update all files that match `grep -rn "Semantic Conventions for Generative AI systems v" src/Libraries/Microsoft.Extensions.AI/`. Not all OpenTelemetry* files contain this reference — only update the ones that do. +- **Version references**: When bumping the convention version, update all files that match the transitional regex `grep -rEn "Semantic Conventions for Generative AI systems v|GenAI Semantic Conventions v" src/Libraries/Microsoft.Extensions.AI/` (handles both pre- and post-migration wording). The next convention update should also migrate the wording in lockstep — see [references/file-inventory.md §Version References](references/file-inventory.md#version-references). Not all OpenTelemetry* files contain this reference — only update the ones that do. - **No CHANGELOGs**: This repository no longer maintains per-library CHANGELOG.md files. Do NOT create or update any CHANGELOG files. - **Source-generated JSON**: Adding new OTel part types requires: (1) new inner class, (2) `[JsonSerializable]` registration on `OtelContext`, (3) switch case in `SerializeChatMessages()`. - **LoggerMessage text**: When using `[LoggerMessage]`, the message text should match the OTel event name for console logger readability. - **No orphan constants**: Never add a constant to `OpenTelemetryConsts.cs` unless the same PR also adds at least one emission site for it. If the convention defines an attribute that no current client populates, classify the change as 🟢 *Constant not yet emitted* and defer the constant — do not add it ahead of emission. Verify with `grep -rn NewConstantName src/Libraries/Microsoft.Extensions.AI/` before submitting. +- **Area-aware constants**: Pick the nested class in `OpenTelemetryConsts.cs` based on the upstream area: `GenAI.*` for `gen-ai/*`, `MCP.*` for `mcp/*`. Provider-specific attributes (`openai.*`, `anthropic.*`, `aws-bedrock.*`, `azure-ai-inference.*`) generally belong in the **provider package's** constants file, not in `Microsoft.Extensions.AI/OpenTelemetryConsts.cs`. See [references/implementation-patterns.md §Area placement guidance](references/implementation-patterns.md#area-placement-guidance). ## Validation diff --git a/.github/skills/update-otel-genai-conventions/references/change-classification.md b/.github/skills/update-otel-genai-conventions/references/change-classification.md index 40ef2264651..8a9af0a15f0 100644 --- a/.github/skills/update-otel-genai-conventions/references/change-classification.md +++ b/.github/skills/update-otel-genai-conventions/references/change-classification.md @@ -1,6 +1,21 @@ # Change Classification -Taxonomy for classifying gen-ai changes from semantic-conventions releases. Use this to assess each change's impact on dotnet/extensions. +Taxonomy for classifying GenAI semantic-conventions changes from PRs, CHANGELOG snapshots, date ranges, or releases (when they exist) in [`open-telemetry/semantic-conventions-genai`](https://github.com/open-telemetry/semantic-conventions-genai). Old-repo `area:gen-ai` PRs (pre-migration) classify the same way. Use this to assess each change's impact on dotnet/extensions. + +## Areas + +The new repo hosts conventions for several areas. Identify the **area** of each change from its path under `model//` or `docs//` in the new repo: + +| Area | dotnet/extensions location | +|---|---| +| `gen-ai`, `gen-ai/agent` | `Microsoft.Extensions.AI` core (e.g. `OpenTelemetryChatClient`) | +| `mcp` | No instrumentation today — classify all `mcp` changes as 🟢 *No client exists* unless and until an MCP client is added | +| `openai` | `Microsoft.Extensions.AI.OpenAI` (provider package) | +| `anthropic` | Out of scope for `dotnet/extensions` (no provider package). Classify as 🟢 *No client exists* **from the perspective of this repo**. The compensating change actually lands in [`anthropics/anthropic-sdk-csharp`](https://github.com/anthropics/anthropic-sdk-csharp) — see [SKILL.md §Cross-repo applicability](../SKILL.md#cross-repo-applicability). | +| `aws-bedrock` | Out of scope for `dotnet/extensions` (no provider package). Classify as 🟢 *No client exists* **from the perspective of this repo**. The compensating change actually lands in the `BedrockRuntime` service library of [`aws/aws-sdk-net`](https://github.com/aws/aws-sdk-net) — see [SKILL.md §Cross-repo applicability](../SKILL.md#cross-repo-applicability). | +| `azure-ai-inference` | Corresponding provider package if/when one exists in this repo; otherwise classify as 🟢 *No client exists* | + +When the area is provider-specific (`openai`, `anthropic`, `aws-bedrock`, `azure-ai-inference`), the compensating change usually belongs in the corresponding provider package, **not** in `Microsoft.Extensions.AI`. For `anthropic` ([`anthropics/anthropic-sdk-csharp`](https://github.com/anthropics/anthropic-sdk-csharp)) and `aws-bedrock` (the `BedrockRuntime` service library of [`aws/aws-sdk-net`](https://github.com/aws/aws-sdk-net)) this provider package lives in another SDK repo entirely; when running this skill in `dotnet/extensions`, audit-flag the change but classify it as out-of-scope and link the upstream change so a follow-up can be opened in the right SDK repo. When running this skill **in** that SDK repo, treat the area as in-scope and route the change to that repo's own constants/instrumentation files. ## Classification Categories @@ -63,28 +78,32 @@ For each gen-ai change in a release: ## Audit Table Format -When presenting the analysis, use this table format: +When presenting the analysis, use this table format. The **Area** column lets reviewers see at a glance which dotnet/extensions package each change targets: ```markdown -| Semantic Convention Change | Upstream PR | Classification | Action Required | Complexity | -|---|---|---|---|---| -| `gen_ai.new.attribute` | [#1234](link) | New required attribute | Add constant + emission + test | Low | -| `gen_ai.deferred.attribute` | [#2345](link) | Constant not yet emitted | Defer — no client populates this attribute in this PR | — | -| `retrieval` operation | [#5678](link) | N/A — No client | None | — | -| Version reference | — | Version bump | Update doc comments | Low | +| Area | Semantic Convention Change | Upstream PR | Classification | Action Required | Complexity | +|---|---|---|---|---|---| +| `gen-ai` | `gen_ai.new.attribute` | [open-telemetry/semantic-conventions-genai#1234](link) | New required attribute | Add constant + emission + test in `Microsoft.Extensions.AI` | Low | +| `gen-ai` | `gen_ai.deferred.attribute` | [open-telemetry/semantic-conventions-genai#2345](link) | Constant not yet emitted | Defer — no client populates this attribute in this PR | — | +| `mcp` | `mcp.tool.approval` | [open-telemetry/semantic-conventions-genai#3456](link) | N/A — No client | None — no MCP instrumentation today | — | +| `openai` | `openai.new.attribute` | [open-telemetry/semantic-conventions-genai#4567](link) | New required attribute | Add to `Microsoft.Extensions.AI.OpenAI` provider package | Low | +| `anthropic` | `anthropic.new.attribute` | [open-telemetry/semantic-conventions-genai#5678](link) | N/A — No client (in this repo) | Out of scope here — [`anthropics/anthropic-sdk-csharp`](https://github.com/anthropics/anthropic-sdk-csharp) is the actual target. Open a follow-up there. | — | +| `gen-ai` | Version reference | — | Version bump | Update doc comments to new wording | Low | ``` ## PR Description Table Format -When preparing a PR description, adapt the audit table into a concise reviewer-facing table grouped or sorted by semantic-conventions version. Include every analyzed gen-ai change, not just changes that required code edits. +When preparing a PR description, adapt the audit table into a concise reviewer-facing table grouped or sorted by GenAI version (the version from the schema URL or CHANGELOG release header — independent of core semconv). Include every analyzed change, not just changes that required code edits. ```markdown -| Version | Indicator | Semantic-conventions change | Classification | Compensating change / rationale | -|---|:---:|---|---|---| -| v1.XX | 🔴 | `gen_ai.new.attribute` added | New required attribute | Added constant, emission, and tests in `{files}`. | -| v1.XX | 🟡 | Version reference update | Version bump | Updated OpenTelemetry* doc comments to v1.XX. | -| v1.XX | 🟢 | Provider server span clarified | Server-side only | No client-side instrumentation change needed. | -| v1.XX | 🟢 | `gen_ai.deferred.attribute` added upstream | Constant not yet emitted | No client populates this attribute today; constant will be added in the PR that adds emission. | +| Version | Area | Indicator | Semantic-conventions change | Classification | Compensating change / rationale | +|---|---|:---:|---|---|---| +| vX.Y.Z | `gen-ai` | 🔴 | `gen_ai.new.attribute` added | New required attribute | Added constant, emission, and tests in `{files}`. | +| vX.Y.Z | `gen-ai` | 🟡 | Version reference update | Version bump | Updated OpenTelemetry* doc comments to the new wording. | +| vX.Y.Z | `gen-ai` | 🟢 | Provider server span clarified | Server-side only | No client-side instrumentation change needed. | +| vX.Y.Z | `gen-ai` | 🟢 | `gen_ai.deferred.attribute` added upstream | Constant not yet emitted | No client populates this attribute today; constant will be added in the PR that adds emission. | +| vX.Y.Z | `openai` | 🔴 | `openai.new.attribute` added | New required attribute | Added to `Microsoft.Extensions.AI.OpenAI` provider package. | +| vX.Y.Z | `mcp` | 🟢 | `mcp.tool.approval` added | N/A — No client | No MCP instrumentation in this repo today. | ``` -The final column should either describe the compensating change made or explain why no code change was made, such as "already implemented", "no local source exists", "no client exists", "server-side only", "documentation-only clarification", or "no client populates this attribute today; constant deferred until a PR adds emission". +The final column should either describe the compensating change made or explain why no code change was made, such as "already implemented", "no local source exists", "no client exists", "server-side only", "documentation-only clarification", "no client populates this attribute today; constant deferred until a PR adds emission", "provider-specific area not instrumented in this repo", or "out of scope here — implications land in `anthropics/anthropic-sdk-csharp` (anthropic) or the `BedrockRuntime` library of `aws/aws-sdk-net` (aws-bedrock); follow-up opened in that repo". diff --git a/.github/skills/update-otel-genai-conventions/references/file-inventory.md b/.github/skills/update-otel-genai-conventions/references/file-inventory.md index b1d9aa977cc..45dbf199cab 100644 --- a/.github/skills/update-otel-genai-conventions/references/file-inventory.md +++ b/.github/skills/update-otel-genai-conventions/references/file-inventory.md @@ -60,14 +60,61 @@ To discover any additional test files: `dir test\Libraries\Microsoft.Extensions. The semantic conventions version is referenced in a doc comment in specific OpenTelemetry* instrumentation client files. When bumping the version, update all files that match the grep below — not all OpenTelemetry* files contain the version reference. -The reference looks like: +### Current wording (pre-migration) + +The reference today still reads (carried over from when conventions lived in the core `open-telemetry/semantic-conventions` repo): ```csharp /// This class provides an implementation of the Semantic Conventions for Generative AI systems v1.XX, /// defined at . ``` -Find all occurrences with: +### Target wording (post-migration) + +After the GenAI conventions moved to [`open-telemetry/semantic-conventions-genai`](https://github.com/open-telemetry/semantic-conventions-genai), the doc comment should call out the standalone repo and use the GenAI-namespaced version (matching the schema URL `opentelemetry.io/schemas/gen-ai/X.Y.Z`): + +```csharp +/// This class provides an implementation of the GenAI Semantic Conventions vX.Y.Z, +/// defined at . +``` + +The `` URL is unchanged — the published spec page still resolves and renders the gen-ai spec from the new repo. + +### Wording migration + +This wording change is a **one-time edit** that should ride along with the +**next convention-update PR** (not a separate cosmetic PR). Until that +update lands, both wordings may coexist transiently in the codebase. + +### Finding occurrences during the transition + +Use a regex that matches both wordings: + +```bash +grep -rEn "Semantic Conventions for Generative AI systems v|GenAI Semantic Conventions v" src/Libraries/Microsoft.Extensions.AI/ ``` -grep -rn "Semantic Conventions for Generative AI systems v" src/Libraries/Microsoft.Extensions.AI/ + +Once every file has been migrated to the target wording, the regex can be +simplified back to a single literal: + +```bash +grep -rn "GenAI Semantic Conventions v" src/Libraries/Microsoft.Extensions.AI/ ``` + +## Provider-specific instrumentation + +The new conventions repo also covers provider-specific areas (`openai`, +`anthropic`, `aws-bedrock`, `azure-ai-inference`). Provider-specific +attributes like `openai.*` belong in the **provider package**, not in +`Microsoft.Extensions.AI`: + +| Upstream area | dotnet/extensions location | +|---|---| +| `openai` | `src/Libraries/Microsoft.Extensions.AI.OpenAI/` (e.g. `OpenAIClientExtensions.cs` for `openai.api.type` mapping) | +| `anthropic` | Out of scope for `dotnet/extensions` today — implications land in [`anthropics/anthropic-sdk-csharp`](https://github.com/anthropics/anthropic-sdk-csharp). See [SKILL.md §Cross-repo applicability](../SKILL.md#cross-repo-applicability). | +| `aws-bedrock` | Out of scope for `dotnet/extensions` today — implications land in the `BedrockRuntime` service library of [`aws/aws-sdk-net`](https://github.com/aws/aws-sdk-net). See [SKILL.md §Cross-repo applicability](../SKILL.md#cross-repo-applicability). | +| `azure-ai-inference` | Corresponding provider package, if/when one exists in this repo; otherwise out of scope | +| `mcp` | No instrumentation today — flag as a watch-list item if MCP changes appear | + +Tests for provider-specific attributes live alongside the provider package +(e.g. `test/Libraries/Microsoft.Extensions.AI.OpenAI.Tests/`). diff --git a/.github/skills/update-otel-genai-conventions/references/historical-releases.md b/.github/skills/update-otel-genai-conventions/references/historical-releases.md index e6ed6af9c3e..f77369e40b4 100644 --- a/.github/skills/update-otel-genai-conventions/references/historical-releases.md +++ b/.github/skills/update-otel-genai-conventions/references/historical-releases.md @@ -1,8 +1,15 @@ # Historical Releases -Mapping of OpenTelemetry semantic-conventions releases with gen-ai changes to dotnet/extensions PRs. +Mapping of OpenTelemetry GenAI semantic-conventions releases to dotnet/extensions PRs. -> **Note**: This file is a point-in-time reference and is not intended to be kept up to date with every new release. It provides context for how past convention updates were handled. For the latest release history, consult the [semantic-conventions releases page](https://github.com/open-telemetry/semantic-conventions/releases) and search the dotnet/extensions PR history. +> **Migration Note**: Every entry in the tables below is from the **old** [`open-telemetry/semantic-conventions`](https://github.com/open-telemetry/semantic-conventions) repo (pre-migration era), where GenAI conventions were tagged with `area:gen-ai`. GenAI conventions have since moved to a dedicated repo: [`open-telemetry/semantic-conventions-genai`](https://github.com/open-telemetry/semantic-conventions-genai). See [SKILL.md §Migration Note](../SKILL.md#migration-note) for the full picture. +> +> **Point-in-time reference**: This file is not intended to be kept up to date with every new release. It provides context for how past convention updates were handled. For current activity, consult: +> +> - The new repo's [`CHANGELOG.md`](https://github.com/open-telemetry/semantic-conventions-genai/blob/main/CHANGELOG.md) (`Unreleased` section is the live "what's new" view until releases land). +> - The new repo's [releases page](https://github.com/open-telemetry/semantic-conventions-genai/releases) (empty as of this writing). +> - The new repo's [pull request history](https://github.com/open-telemetry/semantic-conventions-genai/pulls?q=is%3Apr). +> - The dotnet/extensions PR history. ## Release History diff --git a/.github/skills/update-otel-genai-conventions/references/implementation-patterns.md b/.github/skills/update-otel-genai-conventions/references/implementation-patterns.md index 3a9f0af7f75..886b80c88e7 100644 --- a/.github/skills/update-otel-genai-conventions/references/implementation-patterns.md +++ b/.github/skills/update-otel-genai-conventions/references/implementation-patterns.md @@ -4,6 +4,19 @@ Code patterns for common convention update change types. Use these as templates > **Reuse before adding.** Before applying any of the patterns below, search the touched libraries (`Common/`, `TelemetryHelpers.cs`, `OpenTelemetryLog.cs`, and sibling OpenTelemetry* client files) for an existing helper, method, or internal type that already does the same thing. Reuse or extend it instead of adding a parallel implementation. If the same logic will be needed in two or more places, factor it into `Common/` from the start rather than duplicating it per file. The same rule applies to parallel internal types — when a sibling client already defines a type with the same shape, unify under a single shared definition. See [review-checklist.md §3](review-checklist.md#3-code-deduplication) for what reviewers look for. +## Area placement guidance + +The new conventions repo ([`open-telemetry/semantic-conventions-genai`](https://github.com/open-telemetry/semantic-conventions-genai)) hosts multiple areas. Pick the right dotnet/extensions location based on the upstream `model//` path of the change: + +| Upstream area | dotnet/extensions location | Notes | +|---|---|---| +| `gen-ai`, `gen-ai/agent` | `src/Libraries/Microsoft.Extensions.AI/` (e.g. `OpenTelemetryChatClient.cs`, `OpenTelemetryConsts.cs` under `GenAI.*`) | Generic gen-ai instrumentation — the bulk of historical work. | +| `mcp` | No instrumentation today — forward-looking | If MCP changes appear, classify as 🟢 *No client exists* and flag for follow-up. Do not add `MCP.*` constants speculatively. | +| `openai` | `src/Libraries/Microsoft.Extensions.AI.OpenAI/` (e.g. `OpenAIClientExtensions.cs` for `openai.api.type` mapping). Tests in `test/Libraries/Microsoft.Extensions.AI.OpenAI.Tests/`. | Keep `OpenTelemetryChatClient` provider-agnostic; provider-specific attributes belong in the provider package. | +| `anthropic` | Out of scope for `dotnet/extensions` today. Implications land in [`anthropics/anthropic-sdk-csharp`](https://github.com/anthropics/anthropic-sdk-csharp). | If you're using this skill in that repo, follow the target repo's own constants/file layout. See [SKILL.md §Cross-repo applicability](../SKILL.md#cross-repo-applicability). | +| `aws-bedrock` | Out of scope for `dotnet/extensions` today. Implications land in the `BedrockRuntime` service library of [`aws/aws-sdk-net`](https://github.com/aws/aws-sdk-net). | If you're using this skill in that repo, follow the target repo's own constants/file layout. See [SKILL.md §Cross-repo applicability](../SKILL.md#cross-repo-applicability). | +| `azure-ai-inference` | Corresponding provider package, if/when one exists in this repo | Currently no provider package for this in this repo — classify as 🟢 *No client exists* until one is added. | + ## Pattern 1: Adding a New Constant Location: `src/Libraries/Microsoft.Extensions.AI/OpenTelemetryConsts.cs` @@ -129,18 +142,32 @@ if (_logger is not null) ## Pattern 6: Updating Version References -When bumping the convention version (e.g. v1.39 → v1.40), update the doc comment in all matched OpenTelemetry* client files: +When bumping the convention version (e.g. an upcoming GenAI version bump), update the doc comment in all matched OpenTelemetry* client files. The wording is in the middle of a one-time migration to reflect the standalone GenAI repo — see [file-inventory.md §Version References](file-inventory.md#version-references) for full guidance. + +**Pre-migration wording** (carried over from when conventions lived in core `semantic-conventions`): ```csharp -// Before: /// Semantic Conventions for Generative AI systems v1.39, -// After: -/// Semantic Conventions for Generative AI systems v1.40, ``` -Find all occurrences: +**Target wording** (after migration to the standalone repo, with GenAI-namespaced version matching the schema URL `opentelemetry.io/schemas/gen-ai/X.Y.Z`): + +```csharp +/// GenAI Semantic Conventions v1.42.0, +``` + +The next convention-update PR should migrate every matched file from the pre-migration wording to the target wording in one shot. Do not leave files in a half-migrated state. + +Find all occurrences using a regex that matches both wordings during the transition: + +```bash +grep -rEn "Semantic Conventions for Generative AI systems v|GenAI Semantic Conventions v" src/Libraries/Microsoft.Extensions.AI/ +``` + +After the migration is complete, simplify to: + ```bash -grep -rn "Semantic Conventions for Generative AI systems v" src/Libraries/Microsoft.Extensions.AI/ +grep -rn "GenAI Semantic Conventions v" src/Libraries/Microsoft.Extensions.AI/ ``` ## Pattern 7: Modifying Message Serialization diff --git a/.github/skills/update-otel-genai-conventions/references/implementation-procedure.md b/.github/skills/update-otel-genai-conventions/references/implementation-procedure.md index 64ea28cb071..7db00e67169 100644 --- a/.github/skills/update-otel-genai-conventions/references/implementation-procedure.md +++ b/.github/skills/update-otel-genai-conventions/references/implementation-procedure.md @@ -6,9 +6,10 @@ Used by Modes 2 (Autopilot), 4 (CCA Implementation), and 5 (Plan-then-Implement) 2. Read [review-checklist.md](review-checklist.md) to anticipate review feedback 3. Apply changes in this order: - Add new constants to `OpenTelemetryConsts.cs` **only for attributes whose emission is also added in this same PR**. Do not add constants speculatively — if no OpenTelemetry* client in this repo will populate the attribute, defer the constant until the PR that wires up emission and classify the change as 🟢 *Constant not yet emitted* per [change-classification.md](change-classification.md). + - **Choose the correct constant location based on the current repo layout**: use the shared `Microsoft.Extensions.AI/OpenTelemetryConsts.cs` nested classes for shared upstream areas such as `GenAI.*` (`gen-ai/*`) and, when added, `MCP.*` (`mcp/*`). Do **not** assume there is already an `OpenAI.*` nested class there. For provider-specific attributes (`openai.*`, `anthropic.*`, `aws-bedrock.*`, `azure-ai-inference.*`), follow the provider package's existing layout. For example, OpenAI provider-specific tag names are currently defined as `internal const string` values in `OpenAIClientExtensions.cs`. If a provider accumulates enough shared constants to justify a dedicated provider constants file/class, introduce that file/class explicitly in the same PR rather than assuming one already exists. - **Before adding any new helper, method, or internal type**, search `Common/`, `TelemetryHelpers.cs`, `OpenTelemetryLog.cs`, and the sibling OpenTelemetry* client files for existing logic with the same purpose. Reuse or extend rather than introducing a parallel implementation. If the same logic is needed in two or more places, factor it into `Common/` from the start instead of duplicating it per file. The same applies to parallel internal types — unify types with identical shape under a single shared definition. - Add attribute/metric emission to the relevant OpenTelemetry* client classes - - Update version references in doc comments across all files that reference the convention version + - Update version references in doc comments across all files that reference the convention version (see [file-inventory.md §Version References](file-inventory.md#version-references) for the wording-migration guidance) - Update or augment tests 4. Self-review against [review-checklist.md](review-checklist.md) 5. Validate per the **Validation** section in `SKILL.md` diff --git a/.github/skills/update-otel-genai-conventions/references/pr-description.md b/.github/skills/update-otel-genai-conventions/references/pr-description.md index 091ace1b3ec..67679039217 100644 --- a/.github/skills/update-otel-genai-conventions/references/pr-description.md +++ b/.github/skills/update-otel-genai-conventions/references/pr-description.md @@ -1,33 +1,37 @@ # PR Title and Description Format -When asked to create or update a PR after implementing semantic-conventions changes, use this guidance. +When asked to create or update a PR after implementing GenAI semantic-conventions changes, use this guidance. ## Title ```text -Update OpenTelemetry gen-ai conventions to v{version} +Update OpenTelemetry GenAI conventions to v{version} ``` -Use the target semantic-conventions release version for `{version}`. If the PR also includes catch-up work from earlier releases, keep the title focused on the target version and explain the catch-up work in the description. +Use the target **GenAI** version for `{version}` — taken from the schema URL the new repo publishes (e.g. `opentelemetry.io/schemas/gen-ai/1.42.0` → `v1.42.0`) or from the CHANGELOG release header. The GenAI version is independent of core semconv (which has its own version line); do not conflate the two. If the PR also includes catch-up work from earlier convention versions or from the old `open-telemetry/semantic-conventions` repo, keep the title focused on the target GenAI version and explain the catch-up work in the description. ## Description -The description should include a changes table derived from the audit table and [change-classification.md](change-classification.md). Group or sort rows by semantic-conventions version and include every analyzed gen-ai change, not only the rows that produced code changes. Use the same red/yellow/green indicators as the classification guide: +The description should include a changes table derived from the audit table and [change-classification.md](change-classification.md). Group or sort rows by GenAI version and include every analyzed change, not only the rows that produced code changes. Use the same red/yellow/green indicators as the classification guide: - 🟢 for no action required - 🟡 for minor action required - 🔴 for code change required -Use this table shape: +Use this table shape (the **Area** column makes it obvious which package each row targets): ```markdown -| Version | Indicator | Semantic-conventions change | Classification | Compensating change / rationale | -|---|:---:|---|---|---| -| v1.XX | 🔴 | `gen_ai.example.attribute` added | New required attribute | Added constant, emission, and tests in `{files}`. | -| v1.XX | 🟡 | Convention version reference changed | Version bump | Updated OpenTelemetry* doc comments. | -| v1.XX | 🟢 | Server-side-only span attribute added | Server-side only | No client-side instrumentation change needed. | +| Version | Area | Indicator | Semantic-conventions change | Classification | Compensating change / rationale | +|---|---|:---:|---|---|---| +| vX.Y.Z | `gen-ai` | 🔴 | `gen_ai.example.attribute` added | New required attribute | Added constant, emission, and tests in `{files}`. | +| vX.Y.Z | `gen-ai` | 🟡 | Convention version reference changed | Version bump | Updated OpenTelemetry* doc comments. | +| vX.Y.Z | `gen-ai` | 🟢 | Server-side-only span attribute added | Server-side only | No client-side instrumentation change needed. | +| vX.Y.Z | `openai` | 🔴 | `openai.example.attribute` added | New required attribute | Added to `Microsoft.Extensions.AI.OpenAI` provider package. | +| vX.Y.Z | `mcp` | 🟢 | `mcp.example.attribute` added | N/A — No client | No MCP instrumentation in this repo today. | ``` For each row, describe the compensating change made, or explain why no change was made (already implemented, no local source, no client exists, server-side only, documentation only, etc.). +When the PR includes catch-up work whose source PRs live in the **old** `open-telemetry/semantic-conventions` repo (pre-migration `area:gen-ai` work), link to those old-repo PRs explicitly using the `open-telemetry/semantic-conventions#NNN` form so reviewers can disambiguate them from new-repo PR numbers. + Keep release-specific findings in the PR description or implementation summary; do not add them to the skill references unless they are durable cross-release guidance. diff --git a/.github/skills/update-otel-genai-conventions/references/prompt-template.md b/.github/skills/update-otel-genai-conventions/references/prompt-template.md index e0013b0260e..04d138c2894 100644 --- a/.github/skills/update-otel-genai-conventions/references/prompt-template.md +++ b/.github/skills/update-otel-genai-conventions/references/prompt-template.md @@ -4,51 +4,68 @@ Template for generating a structured prompt suitable for delegating convention u ## Template -Fill in the bracketed sections based on the analysis of the semantic-conventions release. +Fill in the bracketed sections based on the analysis of the upstream input — a release, PR set, `CHANGELOG.md` snapshot, or date range from [`open-telemetry/semantic-conventions-genai`](https://github.com/open-telemetry/semantic-conventions-genai) (or, for catch-up work, from the old `open-telemetry/semantic-conventions` repo). --- ```markdown ## Background -The OpenTelemetry semantic conventions {VERSION} release includes gen-ai changes that require compensating updates in dotnet/extensions. Release notes: {RELEASE_URL} +The OpenTelemetry GenAI semantic conventions changes summarized below +require compensating updates in dotnet/extensions. + +Source of changes: {ONE_OF: release URL | CHANGELOG.md ref URL | PR set | date range} +{IF_RELEASE} Release notes: {RELEASE_URL} +{IF_CHANGELOG_SNAPSHOT} CHANGELOG snapshot: {CHANGELOG_REF_URL} (Unreleased section at the time of audit) +{IF_DATE_RANGE} Date range: PRs merged to `open-telemetry/semantic-conventions-genai` `main` between {START} and {END} + +> The GenAI conventions live in [`open-telemetry/semantic-conventions-genai`](https://github.com/open-telemetry/semantic-conventions-genai) (recently moved from the core `semantic-conventions` repo). The new repo also covers `mcp`, `openai`, `anthropic`, `aws-bedrock`, and `azure-ai-inference` areas; see the skill's Migration Note for context. Compensating changes for `anthropic` land in [`anthropics/anthropic-sdk-csharp`](https://github.com/anthropics/anthropic-sdk-csharp); for `aws-bedrock` they land in the `BedrockRuntime` service library of [`aws/aws-sdk-net`](https://github.com/aws/aws-sdk-net) — flag both as out-of-scope here and open follow-ups there. See the skill's [Cross-repo applicability](https://github.com/dotnet/extensions/blob/main/.github/skills/update-otel-genai-conventions/SKILL.md#cross-repo-applicability) section. Key upstream PRs: {FOR_EACH_UPSTREAM_PR} -- [{PR_TITLE}]({PR_URL}) +- [{PR_TITLE}]({PR_URL}) — area: `{AREA}` {END_FOR_EACH} ## Changes Audit -| Semantic Convention Change | Upstream PR | Classification | Action Required | -|---|---|---|---| +| Area | Semantic Convention Change | Upstream PR | Classification | Action Required | +|---|---|---|---|---| {FOR_EACH_CHANGE} -| `{ATTRIBUTE_OR_CHANGE_NAME}` | [#{PR_NUMBER}]({PR_URL}) | {CLASSIFICATION} | {ACTION} | +| `{AREA}` | `{ATTRIBUTE_OR_CHANGE_NAME}` | [open-telemetry/semantic-conventions-genai#{PR_NUMBER}]({PR_URL}) | {CLASSIFICATION} | {ACTION} | {END_FOR_EACH} ## Required Changes ### 1. Version References -Update the semantic conventions version reference from `v{OLD_VERSION}` to `v{NEW_VERSION}` in doc comments across ALL OpenTelemetry* client files: +Update the GenAI semantic conventions doc-comment reference to `v{NEW_VERSION}` in ALL OpenTelemetry* client files that carry a version comment. If this is the PR that migrates the doc-comment wording from the pre-migration form, also reword each occurrence: {LIST_ALL_FILES_WITH_VERSION_REFERENCE} -The doc comment pattern to update: +Pre-migration wording: ```csharp /// Semantic Conventions for Generative AI systems v{OLD_VERSION}, ``` +→ target wording: +```csharp +/// GenAI Semantic Conventions v{NEW_VERSION}, +``` + +If the wording was already migrated in an earlier PR, just bump the version: +```csharp +/// GenAI Semantic Conventions v{OLD_VERSION}, +``` → ```csharp -/// Semantic Conventions for Generative AI systems v{NEW_VERSION}, +/// GenAI Semantic Conventions v{NEW_VERSION}, ``` ### 2. New Constants -Add these constants to `src/Libraries/Microsoft.Extensions.AI/OpenTelemetryConsts.cs`: +Add these constants to `src/Libraries/Microsoft.Extensions.AI/OpenTelemetryConsts.cs` (or to the corresponding provider package's constants file for `openai.*`, `anthropic.*`, etc.): {FOR_EACH_NEW_CONSTANT} -In the `{PARENT_CLASS}` nested class: +In the `{PARENT_CLASS}` nested class (`{PARENT_CLASS}` corresponds to the upstream area: `GenAI` for `gen-ai/*`, `MCP` for `mcp/*`, `OpenAI` for `openai/*`, etc.): ```csharp public const string {CONSTANT_NAME} = "{ATTRIBUTE_NAME}"; ``` @@ -81,7 +98,8 @@ Update tests in `{TEST_FILE_PATH}`: {END_FOR_EACH} Reference the `update-otel-genai-conventions` skill in `.github/skills/` for: -- Implementation patterns in `references/implementation-patterns.md` +- Migration context in `SKILL.md` (§Migration Note) +- Implementation patterns in `references/implementation-patterns.md` (including area placement guidance) - Testing guide in `references/testing-guide.md` - Review checklist in `references/review-checklist.md` @@ -90,14 +108,14 @@ Reference the `update-otel-genai-conventions` skill in `.github/skills/` for: After implementing changes: 1. Restore, generate the AI-filtered solution, build, and run the tests using the Linux/macOS commands in `.github/skills/update-otel-genai-conventions/references/build-commands.md` 2. If the public API surface changed, run `pwsh ./scripts/MakeApiBaselines.ps1` and keep only the baselines for the libraries actually changed -3. Verify no remaining references to the old version: `grep -rn "v{OLD_VERSION}" src/Libraries/Microsoft.Extensions.AI/` +3. Verify no remaining references to the old version using the transitional regex from `references/file-inventory.md`: `grep -rEn "Semantic Conventions for Generative AI systems v{OLD_VERSION}|GenAI Semantic Conventions v{OLD_VERSION}" src/Libraries/Microsoft.Extensions.AI/` ``` --- ## Prompt Quality Guidelines -Based on analysis of successful CCA prompts (PRs #7379, #7382, #7322): +Based on analysis of successful CCA prompts (PRs #7379, #7382, #7322 — all from the pre-migration era against the old `semantic-conventions` repo, but the prompt-shape lessons still apply): ### What makes a good prompt @@ -107,6 +125,8 @@ Based on analysis of successful CCA prompts (PRs #7379, #7382, #7322): 4. **Constant values** — specify the exact string values for new OTel attribute names 5. **Test expectations** — specify which test file and whether to augment existing tests or create new ones 6. **Validation commands** — include the build/test commands to run +7. **Disambiguated PR references** — always use the `open-telemetry/semantic-conventions-genai#NNN` form (or `open-telemetry/semantic-conventions#NNN` for catch-up) so PR numbers don't collide across repos +8. **Area on every change row** — so the implementer knows whether each item belongs in `Microsoft.Extensions.AI` or a provider package ### What to avoid @@ -114,6 +134,8 @@ Based on analysis of successful CCA prompts (PRs #7379, #7382, #7322): 2. **Missing files** — forgetting to update version references in all OpenTelemetry* files 3. **Wrong approach** — specifying `Activity.AddEvent` when `ILogger` should be used for events 4. **Incomplete scope** — only covering chat client when embedding generator also needs changes +5. **Wrong package** — putting `openai.*` (or any provider-specific) attributes in `Microsoft.Extensions.AI` instead of the provider package +6. **Bare `#NNN` PR refs** — PR numbers are not interchangeable between the two repos ### Prompt size guidance diff --git a/.github/skills/update-otel-genai-conventions/references/review-checklist.md b/.github/skills/update-otel-genai-conventions/references/review-checklist.md index ea1d9124527..e4ceabc1f3b 100644 --- a/.github/skills/update-otel-genai-conventions/references/review-checklist.md +++ b/.github/skills/update-otel-genai-conventions/references/review-checklist.md @@ -47,14 +47,16 @@ Review checklist for gen-ai convention changes. Based on patterns from past PR r **Past feedback**: PR #7379 — stephentoub asked "do we already have tests validating error.type? If so, can you just augment those". ### 6. Version Reference Completeness -- [ ] All files with a gen-ai semantic conventions version reference use the same version before starting the update +- [ ] All files with a GenAI semantic conventions version reference use the same version before starting the update - [ ] ALL OpenTelemetry* client files with a version reference have that reference updated -- [ ] Grep confirms no remaining references to the old version: `grep -rn "v1.OLD" src/Libraries/Microsoft.Extensions.AI/` +- [ ] If the next convention update is the one that migrates the doc-comment wording from `Semantic Conventions for Generative AI systems v1.XX` to `GenAI Semantic Conventions vX.Y.Z` (per [file-inventory.md §Wording migration](file-inventory.md#wording-migration)), every matched file is migrated in the same PR — no transitional drift left behind. +- [ ] Grep confirms no remaining references to the old version. Use the transitional regex while the wording migration is pending: `grep -rEn "Semantic Conventions for Generative AI systems v(OLD)|GenAI Semantic Conventions v(OLD)" src/Libraries/Microsoft.Extensions.AI/`. After the wording is migrated everywhere, simplify to `grep -rn "GenAI Semantic Conventions v(OLD)" src/Libraries/Microsoft.Extensions.AI/`. ### 7. Constants Organization - [ ] New constants added to appropriate nested class in `OpenTelemetryConsts.cs` - [ ] Constant names follow PascalCase convention - [ ] String values match the semantic convention attribute names exactly +- [ ] **Area-aware nesting**: attributes from upstream `mcp/`, `openai/`, etc. belong in their own nested class (e.g. `OpenTelemetryConsts.MCP.*`, or in a provider-package constants file for `OpenAI.*`) rather than under `GenAI.*`. Do not add MCP or provider attributes to `GenAI.*`. (See [file-inventory.md §Provider-specific instrumentation](file-inventory.md#provider-specific-instrumentation).) - [ ] **No orphan constants**: every newly added constant in `OpenTelemetryConsts.cs` is referenced by at least one emission site added in this PR. Verify with `grep -rn NewConstantName src/Libraries/Microsoft.Extensions.AI/`. If no client populates the attribute, the constant must be removed from this PR and deferred to the PR that adds emission (classify as 🟢 *Constant not yet emitted*). ### 8. Scope Completeness @@ -62,6 +64,7 @@ Review checklist for gen-ai convention changes. Based on patterns from past PR r - [ ] If a change affects embeddings, image generation, speech, etc., those clients are also updated - [ ] Function invocation changes apply to both `FunctionInvokingChatClient` and shared `Common/FunctionInvocationProcessor.cs` - [ ] Realtime function invocation via `FunctionInvokingRealtimeClientSession` is also covered if applicable +- [ ] **Provider-specific scope**: attributes from upstream `openai/`, `anthropic/`, `aws-bedrock/`, `azure-ai-inference/` areas live in the corresponding provider package (e.g. `Microsoft.Extensions.AI.OpenAI` for `openai/`), **not** in `Microsoft.Extensions.AI`. Verify the change landed in the right package and that the provider package's tests cover it. For `anthropic/` and `aws-bedrock/` the corresponding provider package lives in another dotnet SDK repo we contribute to — [`anthropics/anthropic-sdk-csharp`](https://github.com/anthropics/anthropic-sdk-csharp) and the `BedrockRuntime` library of [`aws/aws-sdk-net`](https://github.com/aws/aws-sdk-net) respectively (see [SKILL.md §Cross-repo applicability](../SKILL.md#cross-repo-applicability)); from `dotnet/extensions` they are out of scope — confirm the audit table flags them and that a follow-up has been (or will be) opened in the right SDK repo. **Past feedback**: PR #7379 — stephentoub asked to extend changes to additional client types. diff --git a/.github/skills/update-otel-genai-conventions/references/testing-guide.md b/.github/skills/update-otel-genai-conventions/references/testing-guide.md index cca6886028c..f73bd9318f9 100644 --- a/.github/skills/update-otel-genai-conventions/references/testing-guide.md +++ b/.github/skills/update-otel-genai-conventions/references/testing-guide.md @@ -142,6 +142,26 @@ Test that attributes are omitted (not set to empty/default) when the source data When an attribute appears on both spans and metrics, ensure tests verify both emission points. +## Per-area test files + +When a single instrumented client consumes attributes from multiple +upstream areas in [`open-telemetry/semantic-conventions-genai`](https://github.com/open-telemetry/semantic-conventions-genai) +(e.g. `OpenTelemetryChatClient` consumes `gen-ai/*` plus, in some +provider packages, `openai/*`), the test file still lives **with the +client**. Do not split tests by upstream area — keep all assertions for a +given client in that client's test file. + +Provider-specific attributes (`openai.*`, `anthropic.*`, +`aws-bedrock.*`, `azure-ai-inference.*`) are tested in the corresponding +provider package's test project (e.g. +`test/Libraries/Microsoft.Extensions.AI.OpenAI.Tests/` for `openai.*`). +For `anthropic.*` and `aws-bedrock.*` the corresponding provider +package — and therefore its test project — lives in another dotnet SDK +repo we contribute to: [`anthropics/anthropic-sdk-csharp`](https://github.com/anthropics/anthropic-sdk-csharp) +and the `BedrockRuntime` library of [`aws/aws-sdk-net`](https://github.com/aws/aws-sdk-net), +respectively (see [SKILL.md §Cross-repo applicability](../SKILL.md#cross-repo-applicability)); +follow that repo's own test layout when running this skill there. + ## Build and Test Commands See [build-commands.md](build-commands.md) for the canonical Windows and Linux/macOS forms, including the faster `dotnet test --filter` invocation for inner-loop iteration.