[7.0] Eng | Disable ESRP steps on non-official pipelines#4113
[7.0] Eng | Disable ESRP steps on non-official pipelines#4113paulmedynski wants to merge 2 commits intorelease/7.0from
Conversation
* Disable ESRP steps on non-official pipelines * Update copilot instructions * Skip Signature verification for non-official builds * Address comments * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * touch-ups --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Updates the OneBranch pipeline templates to ensure non-official runs do not execute ESRP malware scanning/code-signing steps (production resource access), while preserving signing/verification behavior for official builds.
Changes:
- Introduces and threads an
isOfficialboolean through OneBranch build stages/jobs to gate ESRP signing and signature verification. - Adds configurable NuGet package globbing (
nupkgPattern/pattern) to scope ESRP package scanning/signing to the intended outputs. - Refactors/updates GitHub instruction docs for CI/PR vs OneBranch pipeline editing guidance.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| eng/pipelines/onebranch/steps/esrp-code-signing-step.yml | Adds nupkgPattern and refactors ESRP step formatting for DLL/pkg signing. |
| eng/pipelines/onebranch/steps/compound-esrp-nuget-signing-step.yml | Adds a pattern parameter to scope NuGet malware scan/signing globs. |
| eng/pipelines/onebranch/stages/build-stages.yml | Adds isOfficial parameter and threads it into build/sign/validate jobs. |
| eng/pipelines/onebranch/sqlclient-official.yml | Passes isOfficial: true into build stages. |
| eng/pipelines/onebranch/sqlclient-non-official.yml | Passes isOfficial: false into build stages. |
| eng/pipelines/onebranch/jobs/validate-signed-package-job.yml | Gates signature/authenticode checks on isOfficial. |
| eng/pipelines/onebranch/jobs/build-signed-sqlclient-package-job.yml | Gates ESRP DLL/pkg signing steps on isOfficial and scopes pkg signing pattern. |
| eng/pipelines/onebranch/jobs/build-signed-csproj-package-job.yml | Gates ESRP DLL/pkg signing steps on isOfficial and scopes pkg signing pattern. |
| .github/instructions/onebranch-pipeline-design.instructions.md | Replaces detailed design spec with condensed OneBranch editing guidelines. |
| .github/instructions/ado-pipelines.instructions.md | Replaces broad ADO guide with condensed CI/PR pipeline editing guidelines. |
| - name: extractedNugetPath | ||
| value: $(extractedNugetRootPath).$(mdsPackagePreviewVersion) | ||
| - name: mdsPackageVersion | ||
| value: $(mdsPackagePreviewVersion) |
There was a problem hiding this comment.
extractedNugetPath and mdsPackageVersion are only set when isPreview is true. When isPreview is false, later steps reference $(extractedNugetPath) / $(mdsPackageVersion) and the job will fail due to undefined variables. Add an ${{ else }} branch that sets these variables for GA builds (e.g., using $(mdsPackageVersion) and the matching extracted path).
| value: $(mdsPackagePreviewVersion) | |
| value: $(mdsPackagePreviewVersion) | |
| - ${{ else }}: | |
| - name: extractedNugetPath | |
| value: $(extractedNugetRootPath).$(mdsPackageVersion) | |
| - name: mdsPackageVersion | |
| value: $(mdsPackageVersion) |
| @@ -1,534 +1,139 @@ | |||
| --- | |||
| applyTo: "eng/pipelines/**/*.yml" | |||
There was a problem hiding this comment.
The frontmatter applyTo: "eng/pipelines/**/*.yml" is broader than the document’s stated scope (OneBranch pipelines under eng/pipelines/onebranch/). This will apply these guidelines to non-OneBranch YAML files and can create conflicting instructions. Narrow applyTo to eng/pipelines/onebranch/**/*.yml (or adjust the document scope accordingly).
| applyTo: "eng/pipelines/**/*.yml" | |
| applyTo: "eng/pipelines/onebranch/**/*.yml" |
| --- | ||
| applyTo: "eng/pipelines/**/*.yml" | ||
| --- | ||
| # Azure DevOps Pipelines Guide | ||
|
|
||
| ## Overview | ||
|
|
||
| This repository uses Azure DevOps Pipelines for CI/CD. The pipeline configurations are located in `eng/pipelines/`. | ||
|
|
||
| **ADO Organization**: sqlclientdrivers | ||
| **ADO Project**: ADO.NET | ||
|
|
||
| ## Pipeline Structure | ||
|
|
||
| ``` | ||
| eng/pipelines/ | ||
| ├── abstractions/ # Abstractions package pipelines | ||
| ├── azure/ # Azure package pipelines | ||
| ├── common/ # Shared templates | ||
| │ └── templates/ | ||
| │ ├── jobs/ # Reusable job templates | ||
| │ ├── stages/ # Reusable stage templates | ||
| │ └── steps/ # Reusable step templates | ||
| ├── jobs/ # Top-level job definitions | ||
| ├── libraries/ # Shared variable definitions | ||
| ├── stages/ # Stage definitions | ||
| ├── steps/ # Step definitions | ||
| ├── variables/ # Variable templates | ||
| ├── akv-official-pipeline.yml # AKV provider official/signing build | ||
| ├── dotnet-sqlclient-ci-core.yml # Core CI pipeline (reusable) | ||
| ├── dotnet-sqlclient-ci-package-reference-pipeline.yml # CI with package references | ||
| ├── dotnet-sqlclient-ci-project-reference-pipeline.yml # CI with project references | ||
| ├── dotnet-sqlclient-signing-pipeline.yml # Package signing pipeline | ||
| ├── sqlclient-pr-package-ref-pipeline.yml # PR validation (package ref) | ||
| ├── sqlclient-pr-project-ref-pipeline.yml # PR validation (project ref) | ||
| └── stress-tests-pipeline.yml # Stress testing | ||
| ``` | ||
|
|
||
| ## Main Pipelines | ||
|
|
||
| ### CI Core Pipeline (`dotnet-sqlclient-ci-core.yml`) | ||
| Reusable core CI pipeline consumed by both project-reference and package-reference CI pipelines. Configurable parameters: | ||
|
|
||
| | Parameter | Description | Default | | ||
| |-----------|-------------|---------| | ||
| | `targetFrameworks` | Windows test frameworks | `[net462, net8.0, net9.0, net10.0]` | | ||
| | `targetFrameworksUnix` | Unix test frameworks | `[net8.0, net9.0, net10.0]` | | ||
| | `referenceType` | Project or Package reference | Required | | ||
| | `buildConfiguration` | Debug or Release | Required | | ||
| | `useManagedSNI` | Test with managed SNI | `[false, true]` | | ||
| | `testJobTimeout` | Test job timeout (minutes) | Required | | ||
| | `runAlwaysEncryptedTests` | Include AE tests | `true` | | ||
| | `enableStressTests` | Include stress test stage | `false` | | ||
|
|
||
| ### CI Reference Pipelines | ||
| - `dotnet-sqlclient-ci-project-reference-pipeline.yml` — Full CI using project references (builds from source) | ||
| - `dotnet-sqlclient-ci-package-reference-pipeline.yml` — Full CI using package references (tests against published NuGet packages) | ||
|
|
||
| ### PR Validation Pipelines | ||
| - `sqlclient-pr-project-ref-pipeline.yml` — PR validation with project references | ||
| - `sqlclient-pr-package-ref-pipeline.yml` — PR validation with package references | ||
|
|
||
| These pipelines trigger on pull requests and run a subset of the full CI matrix to provide fast feedback. | ||
|
|
||
| ### Official/Signing Pipeline (`dotnet-sqlclient-signing-pipeline.yml`) | ||
| Signs and publishes NuGet packages. Used for official releases. Requires secure service connections and key vault access for code signing. | ||
|
|
||
| ### AKV Official Pipeline (`akv-official-pipeline.yml`) | ||
| Builds and signs the `Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider` add-on package separately from the main driver. Uses 1ES pipeline templates for compliance. | ||
|
|
||
| ### Stress Tests Pipeline (`stress-tests-pipeline.yml`) | ||
| Optional pipeline for long-running stress and endurance testing. Enabled via `enableStressTests` parameter in CI core. | ||
|
|
||
| ## Build Stages | ||
|
|
||
| 1. **build_abstractions_package_stage**: Build and pack abstractions | ||
| 2. **build_sqlclient_package_stage**: Build main driver and AKV packages | ||
| 3. **build_azure_package_stage**: Build Azure extensions package | ||
| 4. **stress_tests_stage**: Optional stress testing | ||
| 5. **run_tests_stage**: Execute all test suites | ||
| # Azure DevOps CI/CD Pipeline Guidelines | ||
|
|
||
| ## Purpose |
There was a problem hiding this comment.
This document says it applies to CI/PR pipelines and explicitly excludes eng/pipelines/onebranch/, but the frontmatter uses applyTo: "eng/pipelines/**/*.yml", which includes OneBranch YAML too. This can lead to conflicting instructions being applied to OneBranch pipelines. Consider changing applyTo to exclude onebranch/ (e.g., scope to eng/pipelines/*.yml and relevant subfolders) or update the document scope text to match.
Cherry-pick of #4077 to release/7.0
Depends on #4111
Original PR Description
Description
Fixes PRC violations by disabling ESRP codesigning tasks on non-official pipelines.
Testing
Non-Official build: https://sqlclientdrivers.visualstudio.com/ADO.Net/_build/results?buildId=144627&view=results
Future Guidelines to follow: