diff --git a/.github/agents/Documentation.md b/.github/agents/Documentation.md new file mode 100644 index 0000000000..c41ba3f8d7 --- /dev/null +++ b/.github/agents/Documentation.md @@ -0,0 +1,8 @@ +# Documentation Rules + +## Important (Should Flag) + +1. **Missing RELEASENOTES update**: User-facing changes without a release note entry +1. **Missing documentation for new settings**: New or changed AL-Go settings must be documented in `Scenarios/settings.md` (including purpose, type, default/required status, and which templates/workflows honor them) and represented in the settings schema (`Actions/.Modules/settings.schema.json`) with matching descriptions and correct metadata (`type`, `enum`, `default`, `required`). +1. **Missing documentation for new functions**: New public functions (exported from modules or used as entry points) should include comment-based help (e.g., `.SYNOPSIS`, `.DESCRIPTION`, parameter help) and be described in relevant markdown documentation when they are part of the public surface. +1. **Missing documentation for new workflows or user-facing behaviors**: New or significantly changed workflows/templates in `Templates/` must have corresponding scenario documentation (or updates) in `Scenarios/`, and new user-facing commands or actions must be documented in scenarios or `README.md`. diff --git a/.github/agents/Security.md b/.github/agents/Security.md new file mode 100644 index 0000000000..445743d0cf --- /dev/null +++ b/.github/agents/Security.md @@ -0,0 +1,9 @@ +# Security Rules + +## Critical (Must Flag) + +1. **Missing error handling**: Scripts must start with `$errorActionPreference = "Stop"; $ProgressPreference = "SilentlyContinue"; Set-StrictMode -Version 2.0` +1. **Secret leakage**: Any path where a secret value could appear in logs, error messages, or output without being masked via `::add-mask::` +1. **Path traversal**: File operations that don't validate paths stay within the workspace +1. **Missing `-recurse` on ConvertTo-HashTable**: After `ConvertFrom-Json`, always chain `| ConvertTo-HashTable -recurse` for case-insensitive access +1. **Deprecated settings**: Flag usage of settings listed in `DEPRECATIONS.md` diff --git a/.github/agents/Style.md b/.github/agents/Style.md new file mode 100644 index 0000000000..2b51743f28 --- /dev/null +++ b/.github/agents/Style.md @@ -0,0 +1,13 @@ +# Style Rules + +## Important (Should Flag) + +1. **Missing tests**: New or modified functions should have corresponding Pester tests in `Tests/` +1. **Cross-platform issues**: Hardcoded path separators, PS5-only or PS7-only constructs +1. **Encoding omissions**: File read/write without explicit `-Encoding UTF8` +1. **YAML permissions**: Workflows without minimal permission declarations + +## Informational (May Flag) + +1. Opportunities to use existing helper functions from `AL-Go-Helper.ps1` or shared modules +1. Inconsistent naming (should be PascalCase functions, camelCase variables) diff --git a/.github/agents/code-review.agent.md b/.github/agents/code-review.agent.md new file mode 100644 index 0000000000..31921a9e16 --- /dev/null +++ b/.github/agents/code-review.agent.md @@ -0,0 +1,42 @@ +# AL-Go Code Review Agent + +You are a code review agent specialized in the AL-Go for GitHub repository. Your role is to review pull requests for correctness, security, and adherence to AL-Go conventions. + +## Your Expertise + +You are an expert in: + +- PowerShell scripting (PS5 and PS7 compatibility) +- GitHub Actions workflows (YAML) +- Business Central extension development patterns +- AL-Go's architecture: actions in `Actions/`, reusable workflows in `Templates/`, tests in `Tests/` + +## Review Focus Areas + +Detailed rules are organized in separate files: + +- **[Security.md](./Security.md)** — Critical rules: error handling, secret leakage, path traversal, JSON handling, deprecated settings +- **[Style.md](./Style.md)** — Style/quality rules: tests, cross-platform, encoding, YAML permissions, naming conventions +- **[Documentation.md](./Documentation.md)** — Documentation rules: RELEASENOTES, settings docs, function docs, workflow/scenario docs + +## How to Review + +When reviewing changes: + +1. Read the PR description to understand intent +1. Check each changed file against the critical and important rules in [Security.md](./Security.md) and [Style.md](./Style.md) +1. Verify that test coverage exists for logic changes +1. Check for deprecated setting usage against `DEPRECATIONS.md`, and ensure any deprecations are documented there with clear replacement guidance and reflected in settings documentation/schema descriptions. +1. Validate that workflows follow the existing patterns in `Templates/` +1. Confirm that any new or modified settings are both documented and added to the schema, with aligned descriptions and correct metadata (type/default/enum/required). See [Documentation.md](./Documentation.md). +1. Confirm that new public functions have appropriate documentation, including accurate comment-based help (parameter names and descriptions kept in sync with the implementation). +1. Confirm that new or significantly changed workflows/templates and other user-facing behaviors are documented in the appropriate scenario files and/or `README.md`, and that any breaking changes are called out in `RELEASENOTES.md`. + +## Key Repository Knowledge + +- **Settings reference**: `Scenarios/settings.md` describes all AL-Go settings +- **Settings schema**: `Actions/.Modules/settings.schema.json` defines the JSON schema for AL-Go settings +- **Action pattern**: Each action lives in `Actions//` with an `action.yaml` and PowerShell scripts +- **Template workflows**: `Templates/Per Tenant Extension/` and `Templates/AppSource App/` contain the workflow templates shipped to users +- **Shared modules**: `Actions/.Modules/` contains reusable PowerShell modules +- **Security checks**: `Actions/VerifyPRChanges/` validates that fork PRs don't modify protected files (.ps1, .psm1, .yml, .yaml, CODEOWNERS) diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 0000000000..dccaddf65e --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,111 @@ +# Copilot Instructions for AL-Go + +## Project Overview + +AL-Go for GitHub is a set of GitHub Actions and Templates for building, testing, and deploying Business Central extensions using GitHub workflows. It consists of PowerShell actions, reusable YAML workflows, and Pester-based unit tests. + +## PowerShell Conventions + +### Error Handling + +- Every action script must start with the standard header: + ```powershell + $errorActionPreference = "Stop"; $ProgressPreference = "SilentlyContinue"; Set-StrictMode -Version 2.0 + ``` +- Use `try/catch/finally` with structured error propagation. +- Check `$LASTEXITCODE` after running external commands. +- Use `Write-Host "::Error::"` for GitHub Actions error annotations. +- Use `Write-Host "::Warning::"` for non-blocking warnings. + +### JSON Processing + +- Always use `ConvertTo-HashTable -recurse` after `ConvertFrom-Json` to ensure nested objects and arrays are converted to hashtables for consistent access. +- Always specify `-Encoding UTF8` when reading or writing JSON files. + +### Function Declarations + +- Use PascalCase for function names and camelCase for variables. + +### Module Loading + +- Import modules with explicit paths: `Join-Path $PSScriptRoot` pattern. +- Use `-Force -DisableNameChecking` for re-imports. + +## Security Patterns + +### Secret Handling + +- Mask secrets with `Write-Host "::add-mask::$secret"` before any output. +- Never log raw secrets; use clean/placeholder URLs in error messages. +- Be aware that secrets in URLs use `${{ secretName }}` syntax — replacement is done before use. +- URL-encode secret values when injecting into URLs. + +### Input Sanitization + +- Sanitize filenames using `[System.IO.Path]::GetInvalidFileNameChars()`. +- Check for path traversal using `Test-PathWithinWorkspace` or equivalent. +- Sanitize container names with `-replace "[^a-z0-9\-]"`. + +### Authentication + +- Never hardcode credentials or tokens in source code. +- Use GitHub secrets or Azure KeyVault for credential storage. + +## YAML Workflow Conventions + +- Declare minimal required permissions (e.g., `contents: read`, `actions: read`). +- Use `defaults.run.shell: pwsh` for cross-platform compatibility. +- Prefix internal environment variables with `_` to distinguish from GitHub context. +- Use `${{ needs.JobName.outputs.key }}` for cross-job communication. +- Add `::Notice::` steps when conditionally skipping workflow steps. + +## Testing Requirements + +- All new functions must have Pester unit tests in the `Tests/` folder. +- Test files follow the naming convention `*.Test.ps1`. +- Use `Describe`/`It` blocks with descriptive names. +- Mock external dependencies to isolate units under test. +- Tests must pass on both Windows (PowerShell 5) and Linux (PowerShell 7). +- Use `InModuleScope` for testing private module functions. + +## Documentation Requirements + +- All new or modified AL-Go settings must be: + - Documented in `Scenarios/settings.md` with a clear description, type, default/required status, valid values (e.g., enum), and which templates/workflows honor the setting. + - Added or updated in the settings schema (`Actions/.Modules/settings.schema.json`) with aligned `description`, `type`, `enum`, `default`, and `required` metadata. + - Marked as deprecated in both `Scenarios/settings.md` and the schema description when applicable, with guidance on the replacement setting, and listed in `DEPRECATIONS.md`. +- New public functions (in `.ps1` / `.psm1` files, or used as entry points from workflows) should include comment-based help with at least `.SYNOPSIS` and, when appropriate, `.DESCRIPTION`, `.PARAMETER`, and `.EXAMPLE` blocks. Parameter names and descriptions in the help should stay in sync with the function signature. +- When adding new user-facing behaviors, workflows, or commands: + - Update the relevant scenario(s) under `Scenarios/` or the appropriate `README.md` so users can discover and understand the change. + - Call out breaking changes and notable new capabilities in `RELEASENOTES.md`. + +## Deprecated Features + +Before using or accepting settings, check `DEPRECATIONS.md` for deprecated settings: + +- `unusedALGoSystemFiles` → use `customALGoFiles.filesToExclude` +- `alwaysBuildAllProjects` → use `incrementalBuilds.onPull_Request` +- `Schedule` → use `workflowSchedule` with conditional settings +- `cleanModePreprocessorSymbols` → use `preprocessorSymbols` with conditional settings + +## Cross-Platform Considerations + +- Use `[System.IO.Path]::DirectorySeparatorChar` instead of hardcoded separators. +- Account for PowerShell 5 vs 7 differences (e.g., encoding parameters, `$IsWindows`). +- Use `Replace('\', '/')` for path normalization in URLs and artifact names. + +## Pull Request Checklist + +When reviewing PRs, verify: + +- [ ] Standard error handling header is present in new scripts +- [ ] Secrets are masked before any output +- [ ] JSON is converted with `ConvertTo-HashTable -recurse` +- [ ] File encoding is explicitly specified +- [ ] Unit tests are added or updated +- [ ] RELEASENOTES.md is updated for user-facing changes +- [ ] No deprecated settings are introduced +- [ ] YAML workflows declare minimal permissions +- [ ] Cross-platform compatibility is maintained +- [ ] New or changed settings are documented in `Scenarios/settings.md` and reflected in `Actions/.Modules/settings.schema.json` with consistent metadata +- [ ] New public functions have appropriate comment-based help and any new workflows/user-facing behaviors are documented in scenarios/READMEs diff --git a/.github/workflows/AiIssueTriager.yml b/.github/workflows/AiIssueTriager.yml index e65e6258a5..6a29fd5caa 100644 --- a/.github/workflows/AiIssueTriager.yml +++ b/.github/workflows/AiIssueTriager.yml @@ -15,7 +15,7 @@ jobs: uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Setup Node.js - uses: actions/setup-node@6044e13b5dc448c55e2357c09f80417699197238 # v6.2.0 + uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0 - name: Run AI assessment id: ai-assessment diff --git a/.github/workflows/CI.yaml b/.github/workflows/CI.yaml index f7f110ca6f..a3f8a4de86 100644 --- a/.github/workflows/CI.yaml +++ b/.github/workflows/CI.yaml @@ -30,7 +30,7 @@ jobs: steps: - name: Harden Runner if: github.repository_owner == 'microsoft' - uses: step-security/harden-runner@a90bcbc6539c36a85cdfeb73f7e2f433735f215b # v2.15.0 + uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 with: egress-policy: audit diff --git a/.github/workflows/CleanupTempRepos.yaml b/.github/workflows/CleanupTempRepos.yaml index d525ceba61..076528056c 100644 --- a/.github/workflows/CleanupTempRepos.yaml +++ b/.github/workflows/CleanupTempRepos.yaml @@ -27,7 +27,7 @@ jobs: steps: - name: Harden Runner if: github.repository_owner == 'microsoft' - uses: step-security/harden-runner@a90bcbc6539c36a85cdfeb73f7e2f433735f215b # v2.15.0 + uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 with: egress-policy: audit @@ -47,7 +47,7 @@ jobs: run: | ${{ github.workspace }}/Internal/Scripts/GetOwnerForE2ETests.ps1 -githubOwner $env:githubOwner - - uses: actions/create-github-app-token@29824e69f54612133e76f7eaac726eef6c875baf # v2.2.1 + - uses: actions/create-github-app-token@1b10c78c7865c340bc4f6099eb2f838309f1e8c3 # v3.1.1 id: app-token if: ${{ vars.E2E_APP_ID != '' }} with: diff --git a/.github/workflows/Deploy.yaml b/.github/workflows/Deploy.yaml index 145a3238ca..255120742d 100644 --- a/.github/workflows/Deploy.yaml +++ b/.github/workflows/Deploy.yaml @@ -54,7 +54,7 @@ jobs: steps: - name: Harden Runner if: github.repository_owner == 'microsoft' - uses: step-security/harden-runner@a90bcbc6539c36a85cdfeb73f7e2f433735f215b # v2.15.0 + uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 with: egress-policy: audit @@ -87,7 +87,7 @@ jobs: steps: - name: Harden Runner if: github.repository_owner == 'microsoft' - uses: step-security/harden-runner@a90bcbc6539c36a85cdfeb73f7e2f433735f215b # v2.15.0 + uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 with: egress-policy: audit @@ -129,7 +129,7 @@ jobs: steps: - name: Harden Runner if: github.repository_owner == 'microsoft' - uses: step-security/harden-runner@a90bcbc6539c36a85cdfeb73f7e2f433735f215b # v2.15.0 + uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 with: egress-policy: audit @@ -154,7 +154,7 @@ jobs: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - - uses: actions/create-github-app-token@29824e69f54612133e76f7eaac726eef6c875baf # v2.2.1 + - uses: actions/create-github-app-token@1b10c78c7865c340bc4f6099eb2f838309f1e8c3 # v3.1.1 id: app-token if: ${{ vars.APP_ID != '' }} with: @@ -214,7 +214,7 @@ jobs: - name: Create release if: github.repository_owner == 'microsoft' && needs.Inputs.outputs.createRelease == 'true' - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0 + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 id: createrelease env: branch: ${{ needs.Inputs.outputs.branch }} diff --git a/.github/workflows/E2E.yaml b/.github/workflows/E2E.yaml index eb2a5c9af7..d1804e1561 100644 --- a/.github/workflows/E2E.yaml +++ b/.github/workflows/E2E.yaml @@ -58,7 +58,7 @@ jobs: steps: - name: Harden Runner if: github.repository_owner == 'microsoft' - uses: step-security/harden-runner@a90bcbc6539c36a85cdfeb73f7e2f433735f215b # v2.15.0 + uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 with: egress-policy: audit @@ -120,7 +120,7 @@ jobs: steps: - name: Harden Runner if: github.repository_owner == 'microsoft' - uses: step-security/harden-runner@a90bcbc6539c36a85cdfeb73f7e2f433735f215b # v2.15.0 + uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 with: egress-policy: audit @@ -128,7 +128,7 @@ jobs: with: ref: ${{ github.event.inputs.ref }} - - uses: actions/create-github-app-token@29824e69f54612133e76f7eaac726eef6c875baf # v2.2.1 + - uses: actions/create-github-app-token@1b10c78c7865c340bc4f6099eb2f838309f1e8c3 # v3.1.1 id: app-token with: app-id: ${{ vars.E2E_APP_ID }} @@ -155,7 +155,7 @@ jobs: steps: - name: Harden Runner if: github.repository_owner == 'microsoft' - uses: step-security/harden-runner@a90bcbc6539c36a85cdfeb73f7e2f433735f215b # v2.15.0 + uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 with: egress-policy: audit @@ -163,7 +163,7 @@ jobs: with: ref: ${{ github.event.inputs.ref }} - - uses: actions/create-github-app-token@29824e69f54612133e76f7eaac726eef6c875baf # v2.2.1 + - uses: actions/create-github-app-token@1b10c78c7865c340bc4f6099eb2f838309f1e8c3 # v3.1.1 id: app-token with: app-id: ${{ vars.E2E_APP_ID }} @@ -281,7 +281,7 @@ jobs: steps: - name: Harden Runner if: github.repository_owner == 'microsoft' - uses: step-security/harden-runner@a90bcbc6539c36a85cdfeb73f7e2f433735f215b # v2.15.0 + uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 with: egress-policy: audit @@ -318,7 +318,7 @@ jobs: steps: - name: Harden Runner if: github.repository_owner == 'microsoft' - uses: step-security/harden-runner@a90bcbc6539c36a85cdfeb73f7e2f433735f215b # v2.15.0 + uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 with: egress-policy: audit @@ -355,7 +355,7 @@ jobs: steps: - name: Harden Runner if: github.repository_owner == 'microsoft' - uses: step-security/harden-runner@a90bcbc6539c36a85cdfeb73f7e2f433735f215b # v2.15.0 + uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 with: egress-policy: audit @@ -404,7 +404,7 @@ jobs: steps: - name: Harden Runner if: github.repository_owner == 'microsoft' - uses: step-security/harden-runner@a90bcbc6539c36a85cdfeb73f7e2f433735f215b # v2.15.0 + uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 with: egress-policy: audit @@ -453,7 +453,7 @@ jobs: steps: - name: Harden Runner if: github.repository_owner == 'microsoft' - uses: step-security/harden-runner@a90bcbc6539c36a85cdfeb73f7e2f433735f215b # v2.15.0 + uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 with: egress-policy: audit diff --git a/.github/workflows/powershell.yaml b/.github/workflows/powershell.yaml index cf0fe70f56..6b5725b1bc 100644 --- a/.github/workflows/powershell.yaml +++ b/.github/workflows/powershell.yaml @@ -22,7 +22,7 @@ jobs: steps: - name: Harden Runner if: github.repository_owner == 'microsoft' - uses: step-security/harden-runner@a90bcbc6539c36a85cdfeb73f7e2f433735f215b # v2.15.0 + uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 with: egress-policy: audit @@ -38,6 +38,6 @@ jobs: # Upload the SARIF file generated in the previous step - name: Upload SARIF results file - uses: github/codeql-action/upload-sarif@89a39a4e59826350b863aa6b6252a07ad50cf83e # v4.32.4 + uses: github/codeql-action/upload-sarif@95e58e9a2cdfd71adc6e0353d5c52f41a045d225 # v4.35.2 with: sarif_file: results.sarif diff --git a/.github/workflows/pre-commit.yml b/.github/workflows/pre-commit.yml index 3046c0d4da..a435c8af66 100644 --- a/.github/workflows/pre-commit.yml +++ b/.github/workflows/pre-commit.yml @@ -15,7 +15,7 @@ jobs: steps: - name: Harden Runner if: github.repository_owner == 'microsoft' - uses: step-security/harden-runner@a90bcbc6539c36a85cdfeb73f7e2f433735f215b # v2.15.0 + uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 with: egress-policy: audit diff --git a/.github/workflows/scorecard-analysis.yml b/.github/workflows/scorecard-analysis.yml index 39e7281de8..d910f2d11c 100644 --- a/.github/workflows/scorecard-analysis.yml +++ b/.github/workflows/scorecard-analysis.yml @@ -19,7 +19,7 @@ jobs: steps: - name: Harden Runner if: github.repository_owner == 'microsoft' - uses: step-security/harden-runner@a90bcbc6539c36a85cdfeb73f7e2f433735f215b # v2.15.0 + uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0 with: egress-policy: audit @@ -37,6 +37,6 @@ jobs: # Upload the results to GitHub's code scanning dashboard. - name: "Upload to code-scanning" - uses: github/codeql-action/upload-sarif@89a39a4e59826350b863aa6b6252a07ad50cf83e # v4.32.4 + uses: github/codeql-action/upload-sarif@95e58e9a2cdfd71adc6e0353d5c52f41a045d225 # v4.35.2 with: sarif_file: results.sarif diff --git a/Actions/.Modules/CompileFromWorkspace.psm1 b/Actions/.Modules/CompileFromWorkspace.psm1 new file mode 100644 index 0000000000..e6afb5d70c --- /dev/null +++ b/Actions/.Modules/CompileFromWorkspace.psm1 @@ -0,0 +1,843 @@ +Import-Module (Join-Path -Path $PSScriptRoot "./DebugLogHelper.psm1" -Resolve) +Import-Module (Join-Path -Path $PSScriptRoot "../TelemetryHelper.psm1" -Resolve) +Import-Module (Join-Path -Path $PSScriptRoot "../Github-Helper.psm1" -Resolve) +. (Join-Path -Path $PSScriptRoot -ChildPath "../AL-Go-Helper.ps1" -Resolve) + +$script:alTool = $null + +<# +.SYNOPSIS + Gets the list of code analyzers to use for compilation. +.DESCRIPTION + Returns an array of code analyzer names based on the settings provided. + Supports CodeCop, AppSourceCop, PTECop, and UICop. +.PARAMETER Settings + Hashtable containing the build settings with analyzer flags. +.OUTPUTS + Array of analyzer names to use for compilation. +#> +function Get-CodeAnalyzers { + param( + [Parameter(Mandatory = $true)] + [hashtable] $Settings + ) + + $analyzers = @() + + if ($Settings.enableCodeCop) { + $analyzers += "CodeCop" + } + if ($Settings.enableAppSourceCop) { + $analyzers += "AppSourceCop" + } + if ($Settings.enablePerTenantExtensionCop) { + $analyzers += "PTECop" + } + if ($Settings.enableUICop) { + $analyzers += "UICop" + } + + return $analyzers +} + +<# +.SYNOPSIS + Gets the list of custom code analyzers to use for compilation. +.DESCRIPTION + Returns an array of custom code analyzer paths based on the settings provided. + If the custom code cop is a URL, it will be downloaded to the compiler folder and the local path will be returned. +.PARAMETER Settings + Hashtable containing the build settings with custom code cop paths or URLs. +.PARAMETER CompilerFolder + The folder where the AL compiler tool is located, used for downloading custom analyzers if URLs are provided. +.OUTPUTS + Array of custom analyzer paths to use for compilation. +#> +function Get-CustomAnalyzers { + param( + [Parameter(Mandatory = $true)] + [hashtable] $Settings, + + [Parameter(Mandatory = $true)] + [string] $CompilerFolder + ) + + $analyzers = @() + if (-not $Settings.CustomCodeCops -or $Settings.CustomCodeCops.Count -eq 0) { + return $analyzers + } + + # Analyzers/ directory exists in the compiler folder by default + $binPath = Join-Path $CompilerFolder 'compiler/extension/bin' + foreach ($customCodeCop in $Settings.CustomCodeCops) { + if ($customCodeCop -like 'https://*') { + $analyzerFileName = Join-Path $binPath "Analyzers/$(Split-Path $customCodeCop -Leaf)" + try { + Invoke-WebRequest -Uri $customCodeCop -OutFile $analyzerFileName -ErrorAction Stop + } catch { + throw "Failed to download custom analyzer from '$customCodeCop': $($_.Exception.Message)" + } + $analyzers += $analyzerFileName + } + else { + $analyzers += $customCodeCop + } + } + + return $analyzers +} + +<# +.SYNOPSIS + Gets build metadata for the current build environment. +.DESCRIPTION + Returns a hashtable with build metadata including source repository URL, commit SHA, + build system identifier, and build URL. +.OUTPUTS + Hashtable with SourceRepositoryUrl, SourceCommit, BuildBy, and BuildUrl properties. +#> +function Get-BuildMetadata { + # Running in GitHub Actions + return @{ + SourceRepositoryUrl = "$env:GITHUB_SERVER_URL/$env:GITHUB_REPOSITORY" + SourceCommit = $env:GITHUB_SHA + BuildBy = "AL-Go for GitHub" + BuildUrl = "$env:GITHUB_SERVER_URL/$env:GITHUB_REPOSITORY/actions/runs/$env:GITHUB_RUN_ID" + } +} + +<# + .SYNOPSIS + Gets the path to the AL compiler tool (al.exe or al). + .DESCRIPTION + Returns the full path to the AL compiler tool located in the specified compiler folder. + .PARAMETER CompilerFolder + The folder where the AL compiler tool is located. + .OUTPUTS + The full path to the AL compiler tool. +#> +function Get-ALTool { + param( + [Parameter(Mandatory = $true)] + [string] $CompilerFolder + ) + + if ($script:alTool -and (Test-Path $script:alTool)) { + return $script:alTool + } + + # Select the platform-specific AL tool binary + if ($IsLinux) { + $platformFolder = Join-Path $CompilerFolder "compiler/extension/bin/linux" + $alExe = Join-Path $platformFolder "altool" + if (-not (Test-Path $alExe)) { + $alExe = Join-Path $platformFolder "al" + } + } + else { + $platformFolder = Join-Path $CompilerFolder "compiler/extension/bin/win32" + $alExe = Join-Path $platformFolder "altool.exe" + if (-not (Test-Path $alExe)) { + $alExe = Join-Path $platformFolder "al.exe" + } + } + + if (-not (Test-Path $alExe)) { + throw "Could not find AL tool in the compiler folder: $CompilerFolder" + } + $script:alTool = $alExe + return $script:alTool +} + +<# +.SYNOPSIS + Compiles AL apps in a workspace using the ALTool. +.DESCRIPTION + Compiles one or more AL app folders using workspace compilation from the ALTool. + Supports parallel compilation, code analyzers, preprocessor symbols, and compiler features. + Before calling this function, ensure that: + 1. A compiler folder has been created + 2. External dependencies have been fetched into the compiler folder symbols folder + 3. Baseline packages have been downloaded and AppSourceCop baseline packages set up (if applicable) +.PARAMETER Folders + Array of app folder paths to compile. +.PARAMETER CompilerFolder + Path to the compiler folder containing the ALTool and symbols. +.PARAMETER PackageCachePath + Path to the package cache folder. Defaults to the compiler folder's symbols subfolder. +.PARAMETER OutFolder + Path to the output folder for compiled .app files. Defaults to PackageCachePath. +.PARAMETER LogDirectory + Path to the directory for compilation log files. +.PARAMETER MaxCpuCount + Maximum number of parallel compilation processes. Defaults to 1. +.PARAMETER AssemblyProbingPaths + Array of assembly probing paths for the compiler. +.PARAMETER Analyzers + Array of code analyzer names to enable (e.g., CodeCop, UICop). +.PARAMETER CustomAnalyzers + Array of paths to custom code analyzer DLLs. +.PARAMETER PreprocessorSymbols + Array of preprocessor symbols to define during compilation. +.PARAMETER Features + Array of compiler features to enable (e.g., LcgTranslationFile, TranslationFile, GenerateCaptions). +.PARAMETER GenerateReportLayout + Switch to enable report layout generation during compilation. +.PARAMETER Ruleset + Path to a custom ruleset file for code analysis. +.PARAMETER SourceRepositoryUrl + URL of the source repository for build metadata. +.PARAMETER SourceCommit + Commit SHA for build metadata. +.PARAMETER ReportSuppressedDiagnostics + Switch to include suppressed diagnostics in the build output. +.PARAMETER EnableExternalRulesets + Switch to enable external rulesets for code analysis. +.PARAMETER AppType + Type of apps being compiled: 'app' or 'testApp'. +.PARAMETER PreCompileApp + Scriptblock to execute before compiling each app. +.PARAMETER PostCompileApp + Scriptblock to execute after compiling each app. +#> +function Build-AppsInWorkspace { + param( + # Mandatory parameters + [Parameter(Mandatory = $true)] + [string[]]$Folders, + [Parameter(Mandatory = $true)] + [string]$CompilerFolder, + [Parameter(Mandatory = $false)] + [string]$PackageCachePath, + [Parameter(Mandatory = $false)] + [string]$OutFolder, + [Parameter(Mandatory = $false)] + [string]$LogDirectory, + # Optional parameters + [Parameter(Mandatory = $false)] + [int]$MaxCpuCount = 1, + # Optional compiler parameters + [Parameter(Mandatory = $false)] + [string[]]$AssemblyProbingPaths, + [Parameter(Mandatory = $false)] + [string[]]$Analyzers, + [Parameter(Mandatory = $false)] + [string[]]$CustomAnalyzers, + [Parameter(Mandatory = $false)] + [string[]]$PreprocessorSymbols, + [Parameter(Mandatory = $false)] + [string[]]$Features, + [Parameter(Mandatory = $false)] + [switch]$GenerateReportLayout, + [Parameter(Mandatory = $false)] + [string]$Ruleset, + [Parameter(Mandatory = $false)] + [string]$SourceRepositoryUrl, + [Parameter(Mandatory = $false)] + [string]$SourceCommit, + [Parameter(Mandatory = $false)] + [switch]$ReportSuppressedDiagnostics, + [Parameter(Mandatory = $false)] + [switch]$EnableExternalRulesets, + [Parameter(Mandatory = $false)] + [ValidateSet('app', 'testApp')] + [string]$AppType, + [Parameter(Mandatory = $false)] + [scriptblock]$PreCompileApp, + [Parameter(Mandatory = $false)] + [scriptblock]$PostCompileApp + ) + + # Get the package cache path. Use the compiler folder symbols subfolder if not specified + if (-not $PackageCachePath) { + $PackageCachePath = Join-Path $CompilerFolder "symbols" + } + + # Determine the final output folder + if (-not $OutFolder) { + $OutputFolder = $PackageCachePath + } else { + $OutputFolder = $OutFolder + } + + # Validate MaxCpuCount + $maxAvailableProcesses = [System.Environment]::ProcessorCount + if ($MaxCpuCount -gt $maxAvailableProcesses) { + OutputWarning "Specified MaxCpuCount $MaxCpuCount is greater than available processors $maxAvailableProcesses. Using $maxAvailableProcesses instead." + $MaxProcesses = $maxAvailableProcesses + } elseif ($MaxCpuCount -lt 0) { + $MaxProcesses = $maxAvailableProcesses + } else { + $MaxProcesses = $MaxCpuCount + } + + # Get AL tool path + $alToolPath = Get-ALTool -CompilerFolder $CompilerFolder + + # Create workspace file in temp directory + $datetimeStamp = Get-Date -Format "yyyyMMddHHmmss" + $workspaceFile = Join-Path (Get-Location) "tempWorkspace$datetimeStamp.code-workspace" + New-WorkspaceFromFolders -Folders $Folders -WorkspaceFile $workspaceFile -AltoolPath $alToolPath + + $compilationParameters = @{ + ALToolPath = $alToolPath + WorkspaceFile = $workspaceFile + PackageCachePath = $PackageCachePath + OutFolder = $OutputFolder + LogDirectory = $LogDirectory + AssemblyProbingPaths = $AssemblyProbingPaths + Analyzers = $Analyzers + CustomAnalyzers = $CustomAnalyzers + PreprocessorSymbols = $PreprocessorSymbols + Features = $Features + GenerateReportLayout = $GenerateReportLayout + Ruleset = $Ruleset + SourceRepositoryUrl = $SourceRepositoryUrl + SourceCommit = $SourceCommit + ReportSuppressedDiagnostics = $ReportSuppressedDiagnostics + EnableExternalRulesets = $EnableExternalRulesets + MaxCpuCount = $MaxProcesses + } + + # Pre-Compile Apps - Invoke script override before compilation + if ($PreCompileApp) { + OutputDebug "Invoking Pre-Compile App Script..." + Invoke-Command -ScriptBlock $PreCompileApp -ArgumentList $AppType, ([ref] $compilationParameters) + } + + # Compile apps + $appFiles = CompileAppsInWorkspace @compilationParameters + + # Post-Compile Apps - Invoke script override after compilation + if ($PostCompileApp) { + OutputDebug "Invoking Post-Compile App Script..." + Invoke-Command -ScriptBlock $PostCompileApp -ArgumentList $appFiles, $AppType, $compilationParameters + } + + # Remove the workspace file again + Remove-Item $workspaceFile -Force -ErrorAction SilentlyContinue + + return $appFiles +} + +<# + .SYNOPSIS + Copies compiled app files from the package cache to the output folder, and returns the list of generated app file paths. + .DESCRIPTION + Compares the files in the package cache before and after compilation to determine which app files were generated or updated by the compilation process. Copies those files to the output folder and returns their paths. + .PARAMETER PackageCachePath + The folder where the AL compiler outputs compiled app files (package cache). + .PARAMETER OutputFolder + The folder where the generated app files should be copied to. + .PARAMETER FilesBeforeCompile + A hashtable of file paths and their last write times before compilation, used to determine which files were generated or updated by the compilation process. + .OUTPUTS + Array of file paths for the generated app files that were copied to the output folder. +#> +function Copy-CompiledAppsToOutput { + param( + [Parameter(Mandatory = $true)] + [string]$PackageCachePath, + [Parameter(Mandatory = $true)] + [string]$OutputFolder, + [Parameter(Mandatory = $true)] + [hashtable]$FilesBeforeCompile + ) + + $generatedAppFiles = @() + + if (-not (Test-Path $OutputFolder)) { + New-Item -Path $OutputFolder -ItemType Directory -Force | Out-Null + } + + $filesInPackageCache = Get-ChildItem -Path $PackageCachePath -File -Filter "*.app" + OutputArray -Message "Files in package cache after compilation:" -Array $filesInPackageCache -Debug + + # Find new or modified files by comparing timestamps + $outputFiles = $filesInPackageCache | Where-Object { + -not $FilesBeforeCompile.ContainsKey($_.FullName) -or + $_.LastWriteTimeUtc -gt $FilesBeforeCompile[$_.FullName] + } + + OutputDebug -message "Copying generated app files from package cache '$PackageCachePath' to output folder '$OutputFolder'" + foreach ($file in $outputFiles) { + $destinationPath = Join-Path $OutputFolder $file.Name + $generatedAppFiles += $destinationPath + if ($PackageCachePath -ne $OutputFolder) { + Copy-Item -Path $file.FullName -Destination $destinationPath -Force + } + } + + return $generatedAppFiles +} + +function CompileAppsInWorkspace { + param( + [Parameter(Mandatory = $true)] + [string]$ALToolPath, + + [Parameter(Mandatory = $true)] + [string]$WorkspaceFile, + + [Parameter(Mandatory = $true)] + [int]$MaxCpuCount, + + [Parameter(Mandatory = $false)] + [string]$PackageCachePath, + + [Parameter(Mandatory = $false)] + [string[]]$AssemblyProbingPaths, + + [Parameter(Mandatory = $false)] + [string[]]$Analyzers, + + [Parameter(Mandatory = $false)] + [string[]]$CustomAnalyzers, + + [Parameter(Mandatory = $false)] + [string[]]$PreprocessorSymbols, + + [Parameter(Mandatory = $false)] + [string[]]$Features, + + [Parameter(Mandatory = $false)] + [switch]$GenerateReportLayout, + + [Parameter(Mandatory = $false)] + [string]$Ruleset, + + [Parameter(Mandatory = $false)] + [string]$SourceRepositoryUrl, + + [Parameter(Mandatory = $false)] + [string]$SourceCommit, + + [Parameter(Mandatory = $false)] + [switch]$ReportSuppressedDiagnostics, + + [Parameter(Mandatory = $false)] + [switch]$EnableExternalRulesets, + + [Parameter(Mandatory = $false)] + [ValidateSet('Debug', 'Error', 'Normal', 'Verbose', 'Warning')] + [string]$LogLevel = 'Warning', + + [Parameter(Mandatory = $false)] + [string]$LogDirectory, + + [Parameter(Mandatory = $false)] + [string]$OutFolder + ) + + # Check if the workspace file exists + if (-not (Test-Path $WorkspaceFile)) { + throw "The specified workspace file '$WorkspaceFile' does not exist." + } + + # Build the command arguments dynamically + $arguments = @("workspace", "compile", $WorkspaceFile) + + # Get list of files in the package cache path with their timestamps + # Since we are outputting to the package cache path first, we can compare the files before and after compilation to determine which files were generated or updated by the compilation process. + $filesBeforeCompile = @{} + if ($PackageCachePath -and (Test-Path $PackageCachePath)) { + Get-ChildItem -Path $PackageCachePath -File -Filter "*.app" | ForEach-Object { + $filesBeforeCompile[$_.FullName] = $_.LastWriteTimeUtc + } + } + + # Add optional parameters only if they are provided + if ($MaxCpuCount -and $MaxCpuCount -ne [System.Environment]::ProcessorCount) { + $arguments += "--maxcpucount" + $arguments += $MaxCpuCount.ToString() + } + + if ($PackageCachePath) { + $arguments += "--packagecachepath" + $arguments += $PackageCachePath + + # Always output to package cache path first so compiled apps can be used as dependencies for other apps. + # Once compilation is complete the generated app files will be copied to the output folder. + $arguments += "--outfolder" + $arguments += $PackageCachePath + } + + if ($AssemblyProbingPaths -and $AssemblyProbingPaths.Count -gt 0) { + $arguments += "--assemblyprobingpaths" + $arguments += $AssemblyProbingPaths + } + + if ($Analyzers -and $Analyzers.Count -gt 0) { + $arguments += "--analyzers" + $arguments += ($Analyzers -join ",") + } + + if ($CustomAnalyzers -and $CustomAnalyzers.Count -gt 0) { + $arguments += "--customanalyzers" + $arguments += ($CustomAnalyzers -join ",") + } + + if ($PreprocessorSymbols -and $PreprocessorSymbols.Count -gt 0) { + $arguments += "--define" + $arguments += ($PreprocessorSymbols -join ";") + } + + if ($Features -and $Features.Count -gt 0) { + $arguments += "--features" + $arguments += ($Features -join ",") + } + + if ($GenerateReportLayout.IsPresent) { + $arguments += "--generatereportlayout" + } + + if ($Ruleset) { + $arguments += "--ruleset" + $arguments += $Ruleset + } + + if ($SourceRepositoryUrl) { + $arguments += "--sourcerepositoryurl" + $arguments += $SourceRepositoryUrl + } + + if ($SourceCommit) { + $arguments += "--sourcecommit" + $arguments += $SourceCommit + } + + if ($ReportSuppressedDiagnostics.IsPresent) { + OutputWarning "--reportsuppresseddiagnostics is not yet supported and will be ignored." + } + + if ($EnableExternalRulesets.IsPresent) { + OutputWarning "--enableexternalrulesets is not yet supported and will be ignored." + } + + if ($LogLevel) { + $arguments += "--loglevel" + $arguments += $LogLevel + } + + if ($LogDirectory) { + $arguments += "--logdirectory" + $arguments += $LogDirectory + } else { + $defaultLogDir = Join-Path $OutFolder "Logs" + $arguments += "--logdirectory" + $arguments += $defaultLogDir + } + + $generatedAppFiles = @() + $originalEncoding = [Console]::OutputEncoding + try { + OutputColor "Executing: $ALToolPath $($arguments -join ' ')" -Color Green + + # Temporarily set console encoding to UTF-8 to handle special characters in output + [Console]::OutputEncoding = [System.Text.Encoding]::UTF8 + RunAndCheck $ALToolPath @arguments | Out-Host + + OutputColor -message "Compilation completed successfully." -Color Green + } catch { + OutputColor -Message "Error during compilation: $_" -Color Red + throw $_ + } finally { + # Restore original encoding + [Console]::OutputEncoding = $originalEncoding + } + + $generatedAppFiles = Copy-CompiledAppsToOutput -PackageCachePath $PackageCachePath -OutputFolder $OutFolder -FilesBeforeCompile $filesBeforeCompile + + OutputArray -Message "Generated app files:" -Array $generatedAppFiles -Debug + return $generatedAppFiles +} + +<# +.SYNOPSIS + Gets the highest compatible .NET runtime version installed on the system. +.DESCRIPTION + Uses 'dotnet --list-runtimes' to detect installed .NET runtimes. Returns the highest + version that is within the supported major version range. Requires both Microsoft.NETCore.App + and Microsoft.AspNetCore.App runtimes to be installed for a version to be considered. +.PARAMETER MinimumSupportedMajorVersion + The minimum major version of .NET runtime to consider. +.PARAMETER MaximumSupportedMajorVersion + The maximum major version of .NET runtime to consider. +.OUTPUTS + System.Version of the highest compatible .NET runtime installed, or $null if none found. +#> +function Get-DotnetRuntimeVersionInstalled { + param( + [Parameter(Mandatory = $false)] + [int] $MinimumSupportedMajorVersion = 6, # TODO: Find a better way to determine minimum supported version and maximum supported version + [Parameter(Mandatory = $false)] + [int] $MaximumSupportedMajorVersion = 8 + ) + + try { + $runtimeOutput = dotnet --list-runtimes + + if (-not $runtimeOutput) { + OutputDebug -message "Could not detect .NET runtimes. 'dotnet --list-runtimes' returned no output." + return $null + } + + # Parse runtimes into a hashtable grouped by runtime name + $runtimes = @{} + $parsedRuntimes = $runtimeOutput | ConvertFrom-Csv -Delimiter ' ' -Header 'name', 'version' + foreach ($runtime in $parsedRuntimes) { + try { + $version = [System.Version]$runtime.version + if (-not $runtimes.ContainsKey($runtime.name)) { + $runtimes[$runtime.name] = @() + } + $runtimes[$runtime.name] += $version + } catch { + OutputDebug -message "Skipping runtime version '$($runtime.version)' that could not be parsed: $_" + } + } + + # Find versions where both NETCore.App and AspNetCore.App are installed + $netCoreVersions = $runtimes['Microsoft.NETCore.App'] + $aspNetVersions = $runtimes['Microsoft.AspNetCore.App'] + + if (-not $netCoreVersions -or -not $aspNetVersions) { + OutputDebug -message "Required .NET runtimes not found. Need both Microsoft.NETCore.App and Microsoft.AspNetCore.App." + return $null + } + + # Find the highest version present in both, within the supported major version range + $compatibleVersions = $netCoreVersions | Where-Object { + $_.Major -ge $MinimumSupportedMajorVersion -and + $_.Major -le $MaximumSupportedMajorVersion -and + $aspNetVersions -contains $_ + } | Sort-Object -Descending + + if ($compatibleVersions) { + return $compatibleVersions | Select-Object -First 1 + } + + return $null + } + catch { + OutputDebug -message "Failed to detect .NET runtime version: $_" + return $null + } +} + +<# + +.SYNOPSIS + Gets the assembly probing paths for the AL compiler. +.DESCRIPTION + Constructs a list of assembly probing paths based on the compiler folder and the installed .NET runtimes. + Includes paths for service assemblies, mock assemblies, OpenXML, and shared runtime folders. +.PARAMETER CompilerFolder + The folder where the AL compiler tool is located. +.OUTPUTS + Array of assembly probing paths. +#> +function Get-AssemblyProbingPaths { + param( + [Parameter(Mandatory = $true)] + [string]$CompilerFolder + ) + OutputDebug "Determining assembly probing paths..." + $probingPaths = @() + + $compilerFolderDllsPath = Join-Path $CompilerFolder "dlls" + $compilerFolderSharedPath = Join-Path $compilerFolderDllsPath "shared" + + # Use Service and Mock Assemblies folders if they exist from the compiler folder + if (Test-Path $compilerFolderDllsPath) { + $probingPaths += @((Join-Path $compilerFolderDllsPath "Service"),(Join-Path $compilerFolderDllsPath "Mock Assemblies")) + } + + # Use OpenXML and shared folder if they exist + if (Test-Path $compilerFolderSharedPath) { + $probingPaths = @((Join-Path $compilerFolderDllsPath "OpenXML"), $compilerFolderSharedPath) + $probingPaths + } elseif ($isLinux -or $isMacOS) { + $probingPaths = @((Join-Path $compilerFolderDllsPath "OpenXML")) + $probingPaths + } else { + $dotNetRuntimeVersion = (Get-DotnetRuntimeVersionInstalled) + if ($dotNetRuntimeVersion) { + $dotnetRoot = Split-Path (Get-Command dotnet).Source + $probingPaths = @((Join-Path $compilerFolderDllsPath "OpenXML"), (Join-Path $dotnetRoot "shared\Microsoft.NETCore.App\$dotNetRuntimeVersion"), (Join-Path $dotnetRoot "shared\Microsoft.AspNetCore.App\$dotNetRuntimeVersion")) + $probingPaths + } + else { + $probingPaths = @((Join-Path $compilerFolderDllsPath "OpenXML")) + $probingPaths + } + } + + OutputArray -Message "Probing Paths:" -Array $probingPaths + + return $probingPaths +} + +<# +.SYNOPSIS + Creates a workspace file from the specified folders. +.DESCRIPTION + Uses the AL compiler tool to create a workspace file that includes all the specified folders. +.PARAMETER Folders + An array of folder paths to include in the workspace. +.PARAMETER WorkspaceFile + The path where the workspace file will be created. +.PARAMETER AltoolPath + The full path to the AL compiler tool (al.exe or al). +#> +function New-WorkspaceFromFolders { + param( + [Parameter(Mandatory = $true)] + [string[]]$Folders, + + [Parameter(Mandatory = $true)] + [string]$WorkspaceFile, + + [Parameter(Mandatory = $true)] + [string]$AltoolPath + ) + $arguments = @("workspace", "create", $WorkspaceFile) + $Folders + OutputColor "Executing: $AltoolPath $($arguments -join ' ')" -Color Green + RunAndCheck $AltoolPath @arguments | Out-Null + + OutputDebug "Workspace created at $WorkspaceFile" +} + +<# +.SYNOPSIS + Updates the version property in app.json files within the specified folders. +.DESCRIPTION + Finds all app.json files in the given folders and updates their version property based on the provided major/minor version, build number, and revision number. If only the revision number is provided, it will update just the revision part of the existing version. +.PARAMETER Folders + An array of folder paths to search for app.json files. +.PARAMETER MajorMinorVersion + The major and minor version to set in the app.json files (e.g., "1.2"). If not provided, the existing major and minor version will be retained. +.PARAMETER BuildNumber + The build number to set in the app.json files. If not provided, the existing build number will be retained. +.PARAMETER RevisionNumber + The revision number to set in the app.json files. If not provided, the existing revision number will be retained. +#> +function Update-AppJsonProperties { + param( + [Parameter(Mandatory = $true)] + [string[]]$Folders, + + [Parameter(Mandatory = $false)] + [string]$MajorMinorVersion = "", + + [Parameter(Mandatory = $false)] + [int] $BuildNumber = 0, + + [Parameter(Mandatory = $false)] + [int] $RevisionNumber = 0, + + [Parameter(Mandatory = $false)] + [string]$BuildBy = "", + + [Parameter(Mandatory = $false)] + [string]$BuildUrl = "" + ) + + foreach ($folder in $Folders) { + $appJsonFiles = Get-ChildItem -Path $folder -Filter "app.json" + foreach ($appJsonFile in $appJsonFiles) { + $appJsonContent = Get-Content -Path $appJsonFile.FullName -Raw | ConvertFrom-Json + + if ($MajorMinorVersion) { + # If MajorMinorVersion is provided, use it along with BuildNumber and RevisionNumber (or 0 if not provided) to construct the new version + $version = [System.Version]"$($MajorMinorVersion).$($BuildNumber).$($RevisionNumber)" + } else { + # If MajorMinorVersion is not provided, retain the existing major and minor version from app.json and update build and revision numbers based on provided parameters (or retain existing if not provided) + $currentAppJsonVersion = [System.Version]$appJsonContent.Version + if ($BuildNumber -eq 0) { + $version = [System.Version]::new($currentAppJsonVersion.Major, $currentAppJsonVersion.Minor, $currentAppJsonVersion.Build, $RevisionNumber) + } else { + $version = [System.Version]::new($currentAppJsonVersion.Major, $currentAppJsonVersion.Minor, $BuildNumber, $RevisionNumber) + } + } + + OutputDebug "Updating app.json at $($appJsonFile.FullName) to version $version" + $appJsonContent.version = "$version" + + # Stamp build metadata into app.json + if ($BuildBy -or $BuildUrl) { + $buildObject = @{} + if ($BuildBy) { $buildObject.by = $BuildBy } + if ($BuildUrl) { $buildObject.url = $BuildUrl } + if ($appJsonContent.PSObject.Properties.Name -contains 'build') { + $appJsonContent.build = $buildObject + } else { + $appJsonContent | Add-Member -MemberType NoteProperty -Name 'build' -Value $buildObject + } + } + + # Save the updated app.json file + $appJsonContent | ConvertTo-Json -Depth 10 | Set-Content -Path $appJsonFile.FullName -Encoding UTF8 + OutputDebug "Updated app.json at $($appJsonFile.FullName)" + } + } +} + +<# +.SYNOPSIS + Creates a consolidated build output file from individual log files. +.DESCRIPTION + Collects all .log files from the specified build artifact folder, sanitizes their content, + and appends them to a single build output file. Optionally displays the output in the console + and can fail the build based on specified criteria. +.PARAMETER BuildArtifactFolder + The folder containing individual build log files. +.PARAMETER BuildOutputPath + The path where the consolidated build output file will be created. +.PARAMETER DisplayInConsole + Switch to indicate whether the build output should be displayed in the console. +.PARAMETER FailOn + Specifies the criteria for failing the build based on output severity. Options are 'none', 'error', 'warning', 'newWarning'. +.PARAMETER BasePath + The base path for relative paths in the output. Defaults to the GitHub workspace path. +#> +function New-BuildOutputFile { + param( + [Parameter(Mandatory = $true)] + [string]$BuildArtifactFolder, + [Parameter(Mandatory = $true)] + [string]$BuildOutputPath, + [Parameter(Mandatory = $false)] + [switch]$DisplayInConsole, + [Parameter(Mandatory = $false)] + [ValidateSet('none','error','warning','newWarning')] + [string]$FailOn, + [Parameter(Mandatory = $false)] + [string]$BasePath = (Get-BasePath) + ) + # Create the file path for the build output + New-Item -Path $BuildOutputPath -ItemType File -Force | Out-Null + + # Collect the log files and append their content to the build output file + $logFiles = Get-ChildItem -Path $BuildArtifactFolder -Recurse -Filter "*.log" | Select-Object -ExpandProperty FullName + OutputGroupStart -Message "Build Logs" + try { + foreach ($logFile in $logFiles) { + $sanitizedLines = Get-Content -Path $logFile | ForEach-Object { $_ -replace '^\[OUT\]\s?', '' } + Add-Content -Path $buildOutputPath -Value $sanitizedLines + + # Print build output to console (aggregated), preserving line formatting + if ($DisplayInConsole) { + Convert-AlcOutputToAzureDevOps -basePath $BasePath -AlcOutput $sanitizedLines -gitHubActions -FailOn $FailOn + } + } + } finally { + OutputGroupEnd + } + + return $buildOutputPath +} + +Export-ModuleMember -Function Build-AppsInWorkspace +Export-ModuleMember -Function New-BuildOutputFile +Export-ModuleMember -Function Get-BuildMetadata +Export-ModuleMember -Function Get-CodeAnalyzers +Export-ModuleMember -Function Get-CustomAnalyzers +Export-ModuleMember -Function Get-AssemblyProbingPaths +Export-ModuleMember -Function Update-AppJsonProperties diff --git a/Actions/.Modules/ReadSettings.psm1 b/Actions/.Modules/ReadSettings.psm1 index 90835de714..ea3bbe350a 100644 --- a/Actions/.Modules/ReadSettings.psm1 +++ b/Actions/.Modules/ReadSettings.psm1 @@ -161,6 +161,8 @@ function GetDefaultSettings "enableUICop" = $false "enableCodeAnalyzersOnTestApps" = $false "customCodeCops" = @() + "preprocessorSymbols" = @() + "features" = @() "trackALAlertsInGitHub" = $false "failOn" = "error" "treatTestFailuresAsWarnings" = $false @@ -190,6 +192,7 @@ function GetDefaultSettings "templateBranch" = "" "appDependencyProbingPaths" = @() "useProjectDependencies" = $false + "projectsToTest" = @() "runs-on" = "windows-latest" "shell" = "" "githubRunner" = "" @@ -210,6 +213,10 @@ function GetDefaultSettings "environments" = @() "buildModes" = @() "useCompilerFolder" = $false + "workspaceCompilation" = [ordered]@{ + "enabled" = $false + "parallelism" = 1 + } "pullRequestTrigger" = "pull_request" "bcptThresholds" = [ordered]@{ "DurationWarning" = 10 @@ -251,9 +258,10 @@ function GetDefaultSettings "reportSuppressedDiagnostics" = $false "workflowDefaultInputs" = @() "customALGoFiles" = [ordered]@{ - "filesToInclude" = @() + "filesToInclude" = @() "filesToExclude" = @() } + "postponeProjectInBuildOrder" = $false } } @@ -562,6 +570,11 @@ function ReadSettings { $settings.projectName = $project # Default to project path as project name } + # Interpret zero or negative parallelism as the max number of processors + if ($settings.workspaceCompilation.parallelism -le 0) { + $settings.workspaceCompilation.parallelism = [System.Environment]::ProcessorCount + } + $settings | ValidateSettings $settings diff --git a/Actions/.Modules/settings.schema.json b/Actions/.Modules/settings.schema.json index eafcb7bb64..2c90ab3b21 100644 --- a/Actions/.Modules/settings.schema.json +++ b/Actions/.Modules/settings.schema.json @@ -297,6 +297,20 @@ }, "description": "See https://aka.ms/ALGoSettings#customcodecops" }, + "preprocessorSymbols": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Preprocessor symbols to use when compiling the app. See https://aka.ms/ALGoSettings#preprocessorsymbols" + }, + "features": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Features to enable when compiling the app. See https://aka.ms/ALGoSettings#features" + }, "trackALAlertsInGitHub": { "type": "boolean", "description": "Enable tracking of AL Alerts in GitHub. See https://aka.ms/ALGoSettings#trackALAlertsInGitHub" @@ -421,6 +435,13 @@ "type": "boolean", "description": "See https://aka.ms/ALGoSettings#useprojectdependencies" }, + "projectsToTest": { + "type": "array", + "items": { + "type": "string" + }, + "description": "See https://aka.ms/ALGoSettings#projectstotest" + }, "runs-on": { "type": "string", "minLength": 1, @@ -511,6 +532,21 @@ "type": "boolean", "description": "Use the compiler folder instead of a BC container. See https://aka.ms/ALGoSettings#usecompilerfolder" }, + "workspaceCompilation": { + "type": "object", + "description": "PREVIEW: Configuration for workspace compilation. See https://aka.ms/ALGoSettings#workspacecompilation", + "properties": { + "enabled": { + "type": "boolean", + "description": "Enable workspace compilation for building apps" + }, + "parallelism": { + "type": "integer", + "description": "The number of parallel processes to use for workspace compilation. Set to 0 or -1 to use all available processors." + } + }, + "additionalProperties": false + }, "pullRequestTrigger": { "type": "string", "pattern": "^(pull_request|pull_request_target)$", @@ -744,6 +780,10 @@ } } }, + "postponeProjectInBuildOrder": { + "type": "boolean", + "description": "Indicates whether the project can be postponed in the build order to optimize build times. See https://aka.ms/ALGoSettings#postponeProjectInBuildOrder" + }, "workflowDefaultInputs": { "type": "array", "items": { diff --git a/Actions/AL-Go-Helper.ps1 b/Actions/AL-Go-Helper.ps1 index dd73c86c4e..d5b3e267c9 100644 --- a/Actions/AL-Go-Helper.ps1 +++ b/Actions/AL-Go-Helper.ps1 @@ -27,12 +27,13 @@ $RepoSettingsFile = Join-Path '.github' 'AL-Go-Settings.json' $defaultCICDPushBranches = @( 'main', 'release/*', 'feature/*' ) [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseDeclaredVarsMoreThanAssignments', 'defaultCICDPullRequestBranches', Justification = 'False positive.')] $defaultCICDPullRequestBranches = @( 'main' ) -$defaultBcContainerHelperVersion = "preview" # Must be double quotes. Will be replaced by BcContainerHelperVersion if necessary in the deploy step - ex. "https://github.com/organization/navcontainerhelper/archive/refs/heads/branch.zip" +$defaultBcContainerHelperVersion = "preview" $notSecretProperties = @("Scopes","TenantId","BlobName","ContainerName","StorageAccountName","ServerUrl","ppUserName","GitHubAppClientId","EnvironmentName") $runAlPipelineOverrides = @( "DockerPull" "NewBcContainer" + "NewBcCompilerFolder" "ImportTestToolkitToBcContainer" "CompileAppInBcContainer" "GetBcContainerAppInfo" @@ -43,7 +44,9 @@ $runAlPipelineOverrides = @( "ImportTestDataInBcContainer" "RunTestsInBcContainer" "GetBcContainerAppRuntimePackage" + "GetBcContainerEventLog" "RemoveBcContainer" + "RemoveBcCompilerFolder" "InstallMissingDependencies" "PreCompileApp" "PostCompileApp" @@ -51,6 +54,41 @@ $runAlPipelineOverrides = @( "PipelineFinalize" ) +<# + .SYNOPSIS + Gets script overrides from the AL-Go folder. + .DESCRIPTION + Checks the specified AL-Go folder for .ps1 scripts matching the provided override names. + Returns a hashtable mapping each found override name to its script block. + .PARAMETER ALGoFolderName + The folder where the AL-Go scripts are located. + .PARAMETER OverrideScriptNames + An array of script names to look for (without .ps1 extension). + .OUTPUTS + Hashtable with override script names as keys and their script blocks as values. +#> +function Get-ScriptOverrides() { + param( + [Parameter(Mandatory = $true)] + [string] $ALGoFolderName, + [Parameter(Mandatory = $true)] + [string[]] $OverrideScriptNames + ) + $overrides = @{} + foreach ($scriptName in $OverrideScriptNames) { + $scriptPath = Join-Path $ALGoFolderName "$scriptName.ps1" + if (Test-Path -Path $scriptPath -Type Leaf) { + OutputDebug "Add override for $scriptName ($scriptPath)" + $scriptBlock = (Get-Command $scriptPath | Select-Object -ExpandProperty ScriptBlock) + if (-not $scriptBlock) { + OutputError -message "Failed to get scriptblock for $scriptName.ps1, please check the override for validity." + } + $overrides[$scriptName] = $scriptBlock + } + } + return $overrides +} + # Well known AppIds $platformAppId = "8874ed3a-0643-4247-9ced-7a7002f7135d" $systemAppId = "63ca2fa4-4f03-4f2b-a480-172fef340d3f" @@ -187,6 +225,15 @@ function ConvertTo-HashTable() { } } +function Get-CurrentBranchName { + # $ENV:GITHUB_HEAD_REF is specified only for pull requests, so if it is not specified, use GITHUB_REF_NAME + $branchName = $ENV:GITHUB_HEAD_REF + if (!$branchName) { + $branchName = $ENV:GITHUB_REF_NAME + } + return $branchName.Replace('\', '_').Replace('/', '_') +} + function GetUniqueFolderName { Param( [string] $baseFolder, @@ -714,12 +761,33 @@ function AnalyzeRepo { if (!$settings.doNotBuildTests) { Write-Host "No performance test apps found in bcptTestFolders in $ALGoSettingsFile" } $settings.doNotRunBcptTests = $true } + $isTestProject = $settings.projectsToTest -and $settings.projectsToTest.Count -gt 0 if (!$settings.doNotRunTests -and -not $settings.testFolders) { - if (-not ($doNotIssueWarnings -or $settings.doNotBuildTests)) { OutputWarning -message "No test apps found in testFolders in $ALGoSettingsFile" } - $settings.doNotRunTests = $true + if ($isTestProject) { + # Test projects run tests from upstream projects via installTestApps, not from local testFolders + Write-Host "Test project: tests will be run from installed test apps" + if (-not $settings.installTestRunner) { + Write-Host "Test project: installTestRunner will be set to true as it is required to run tests." + } + $settings.installTestRunner = $true + if (-not $settings.installTestFramework) { + Write-Host "Test project: installTestFramework is not set. Add it to the project settings if upstream test apps depend on the test framework." + } + if (-not $settings.installTestLibraries) { + Write-Host "Test project: installTestLibraries is not set. Add it to the project settings if upstream test apps depend on test libraries." + } + if (-not $settings.runTestsInAllInstalledTestApps) { + Write-Host "runTestsInAllInstalledTestApps is false, but will be forced to true as no tests would be run otherwise." + } + $settings.runTestsInAllInstalledTestApps = $true + } + else { + if (-not ($doNotIssueWarnings -or $settings.doNotBuildTests)) { OutputWarning -message "No test apps found in testFolders in $ALGoSettingsFile" } + $settings.doNotRunTests = $true + } } if (-not $settings.appFolders) { - if (!$doNotIssueWarnings) { OutputWarning -message "No apps found in appFolders in $ALGoSettingsFile" } + if (!$isTestProject -and !$doNotIssueWarnings) { OutputWarning -message "No apps found in appFolders in $ALGoSettingsFile" } } $settings @@ -1546,16 +1614,7 @@ function CreateDevEnv { Push-Location $projectFolder try { - $runAlPipelineOverrides | ForEach-Object { - $scriptName = $_ - $scriptPath = Join-Path $ALGoFolderName "$ScriptName.ps1" - if (Test-Path -Path $scriptPath -Type Leaf) { - Write-Host "Add override for $scriptName" - $runAlPipelineParams += @{ - "$scriptName" = (Get-Command $scriptPath | Select-Object -ExpandProperty ScriptBlock) - } - } - } + $runAlPipelineParams += (Get-ScriptOverrides -ALGoFolderName $ALGoFolderName -OverrideScriptNames $runAlPipelineOverrides) if ($kind -eq "local") { $runAlPipelineParams += @{ @@ -1753,6 +1812,44 @@ function CheckAndCreateProjectFolder { } } +function TestIfProjectHasDependents { + Param( + [string] $project, + [string[]] $projects, + [hashtable] $appDependencies, + [array] $projectsOrder, + [hashtable] $testProjectTargets = @{} + ) + + $hasRemainingDependents = $false + foreach($otherProject in $projects) { + if ($otherProject -ne $project) { + # Grab dependencies from other project, which haven't been included in the build order yet + $otherDependencies = $appDependencies."$otherProject".dependencies | Where-Object { + $dependency = $_ + $alreadyBuilt = ($projectsOrder | ForEach-Object { $_.Projects | Where-Object { $appDependencies."$_".apps -contains $dependency } }) + return -not $alreadyBuilt + } + Write-Host "Other project $otherProject has dependencies that are not in the build order yet: $($otherDependencies -join ", ")" + foreach($dependency in $otherDependencies) { + if ($appDependencies."$project".apps -contains $dependency) { + Write-Host "Project $project is still a dependency for project $otherProject" + $hasRemainingDependents = $true + } + } + # Check whether the other project is a test project that targets the current project + if ($testProjectTargets.Keys -contains $otherProject -and $testProjectTargets."$otherProject" -contains $project) { + Write-Host "Project $project is a test target for test project $otherProject" + $hasRemainingDependents = $true + } + } + } + if (!$hasRemainingDependents) { + Write-Host "Project $project has no dependents, can be built later" + } + return $hasRemainingDependents +} + Function AnalyzeProjectDependencies { Param( [string] $baseFolder, @@ -1767,10 +1864,14 @@ Function AnalyzeProjectDependencies { # Loop through all projects # Get all apps in the project # Get all dependencies for the apps + $projectsThatCanBePostponed = @() foreach($project in $projects) { - Write-Host "- Analyzing project: $project" + Write-Host -NoNewline "Analyzing project: $project, " $projectSettings = ReadSettings -project $project -baseFolder $baseFolder + if ($projectSettings.postponeProjectInBuildOrder) { + $projectsThatCanBePostponed += $project + } ResolveProjectFolders -baseFolder $baseFolder -project $project -projectSettings ([ref] $projectSettings) # App folders are relative to the AL-Go project folder. Convert them to relative to the base folder @@ -1784,7 +1885,7 @@ Function AnalyzeProjectDependencies { Pop-Location } - OutputMessageAndArray -Message "Folders containing apps" -arrayOfStrings $folders + OutputMessageAndArray -Message "folders containing apps" -arrayOfStrings $folders $unknownDependencies = @() $apps = @() @@ -1813,8 +1914,56 @@ Function AnalyzeProjectDependencies { # "dependencies" = @("appid7", "appid8") # } # } + + # Handle projectsToTest settings: build a mapping of test projects to their target projects + # First pass: collect all test projects so we can validate that no project depends on a test project + $testProjectNames = @{} + foreach($project in $projects) { + $projectSettings = ReadSettings -project $project -baseFolder $baseFolder + if ($projectSettings.projectsToTest -and $projectSettings.projectsToTest.Count -gt 0) { + # Validate that test projects do not contain buildable code + $buildableFolders = @($appDependencies."$project".apps) + if ($buildableFolders.Count -gt 0) { + throw "Test project '$project' must not contain buildable code. Remove appFolders, testFolders, and bcptTestFolders or remove the projectsToTest setting." + } + $testProjectNames[$project] = $projectSettings.projectsToTest + } + } + # Second pass: resolve and validate target projects, build the testProjectTargets mapping + # testProjectTargets maps each test project to an array of resolved target project names + $testProjectTargets = @{} + foreach($project in $testProjectNames.Keys) { + Write-Host "Project '$project' is a test project targeting: $($testProjectNames[$project] -join ', ')" + $resolvedTargets = @() + foreach($targetProject in $testProjectNames[$project]) { + # Resolve target project name: must be the full project path + # Normalize slashes to match how project keys are stored (OS-dependent) + $normalizedTarget = $targetProject.Replace('/', [System.IO.Path]::DirectorySeparatorChar).Replace('\', [System.IO.Path]::DirectorySeparatorChar) + $resolvedTarget = $null + if ($appDependencies.Keys -contains $normalizedTarget) { + $resolvedTarget = $normalizedTarget + } + if (-not $resolvedTarget) { + OutputError "Test project '$project' references project '$targetProject' which does not exist in the repository. Use the full project path (e.g. 'projects/MyProject')." + throw + } + # Validate that the target is not itself a test project + if ($testProjectNames.Keys -contains $resolvedTarget) { + OutputError "Test project '$project' references '$resolvedTarget' which is also a test project. A test project cannot depend on another test project." + throw + } + $resolvedTargets += @($resolvedTarget) + } + $testProjectTargets[$project] = $resolvedTargets + } + $no = 1 $projectsOrder = @() + # Collect projects without dependents, which can be built later + # This is done to avoid building projects at an earlier stage than needed and increase the time until next job subsequently + # For every time we have determined a set of projects that can be build in parallel, we check whether any of these projects has no dependents + # If so, we remove these projects from the build order and add them at the end of the build order (by adding them to projectsWithoutDependents) + $projectsWithoutDependents = @() Write-Host "Analyzing dependencies" while ($projects.Count -gt 0) { $thisJob = @() @@ -1829,6 +1978,11 @@ Function AnalyzeProjectDependencies { # Loop through all dependencies and locate the projects, containing the apps for which the current project has a dependency $foundDependencies = @() foreach($dependency in $dependencies) { + # Check whether dependency is already resolved by a previous build project + $depProject = @($projectsOrder | ForEach-Object { $_.Projects | Where-Object { $_ -ne $project -and $appDependencies."$_".apps -contains $dependency } }) + if ($depProject.Count -gt 0) { + continue + } # Find the project that contains the app for which the current project has a dependency $depProjects = @($projects | Where-Object { $_ -ne $project -and $appDependencies."$_".apps -contains $dependency }) # Add this project and all projects on which that project has a dependency to the list of dependencies for the current project @@ -1839,6 +1993,18 @@ Function AnalyzeProjectDependencies { } } } + # If this project is a test project, add its target projects as direct dependencies + # Only add target projects that haven't been built yet (still in $projects), matching the normal dependency resolution pattern + if ($testProjectTargets.Keys -contains $project) { + foreach($targetProject in $testProjectTargets."$project") { + if ($projects -contains $targetProject) { + $foundDependencies += $targetProject + if ($projectDependencies.Keys -contains $targetProject) { + $foundDependencies += $projectDependencies."$targetProject" + } + } + } + } $foundDependencies = @($foundDependencies | Select-Object -Unique) # foundDependencies now contains all projects that the current project has a dependency on # Update ref variable projectDependencies for this project @@ -1881,11 +2047,28 @@ Function AnalyzeProjectDependencies { if ($thisJob.Count -eq 0) { throw "Circular project reference encountered, cannot determine build order" } + + # Check whether any of the projects in $thisJob can be built later (has postponeProjectInBuildOrder set to true and no remaining dependents) + $projectsWithoutDependents += @($thisJob | Where-Object { $projectsThatCanBePostponed -contains $_ } | Where-Object { + return -not (TestIfProjectHasDependents -project $_ -projects $projects -appDependencies $appDependencies -projectsOrder $projectsOrder -testProjectTargets $testProjectTargets) + }) + + # Remove projects in this job from the list of projects to be built (including the projects without dependents) + $projects = @($projects | Where-Object { $thisJob -notcontains $_ }) + + # Do not build jobs without dependents until the last job, remove them from this job + $thisJob = @($thisJob | Where-Object { $projectsWithoutDependents -notcontains $_ }) + + if ($projects.Count -eq 0) { + # Last job, add jobs without dependents + Write-Host "Adding projects without dependents to last build job" + $thisJob += $projectsWithoutDependents + } + Write-Host "#$no - build projects: $($thisJob -join ", ")" $projectsOrder += @{'projects' = $thisJob; 'projectsCount' = $thisJob.Count } - $projects = @($projects | Where-Object { $thisJob -notcontains $_ }) $no++ } @@ -2047,7 +2230,6 @@ function RetryCommand { try { Invoke-Command $Command -ArgumentList $argumentList if ($LASTEXITCODE -ne 0) { - $host.SetShouldExit(0); throw "Command failed with exit code $LASTEXITCODE" } break @@ -2259,7 +2441,52 @@ function RunAndCheck { & $args[0] $rest $ErrorActionPreference = 'STOP' if ($LASTEXITCODE -ne 0) { - $host.SetShouldExit(0) throw "$($args[0]) $($rest | ForEach-Object { $_ }) failed with exit code $LASTEXITCODE" } } + +<# +.SYNOPSIS +Get the version number components based on the versioning strategy +.DESCRIPTION +Get the version number components based on the versioning strategy defined in the settings. +.PARAMETER Settings +The settings object containing versioning information. +.RETURNS +A PSCustomObject with MajorMinorVersion, BuildNumber, and RevisionNumber properties. +#> +function Get-VersionNumber() { + param( + [Parameter(Mandatory=$true)] + $Settings + ) + $majorMinorVersion = "" + $appBuild = $Settings.appBuild + $appRevision = $Settings.appRevision + + if ($Settings.versioningStrategy -eq -1) { + $artifactVersion = [Version]$Settings.artifact.Split('/')[4] + $majorMinorVersion = "$($artifactVersion.Major).$($artifactVersion.Minor)" + $appBuild = $artifactVersion.Build + $appRevision = $artifactVersion.Revision + } elseif (($Settings.versioningStrategy -band 16) -eq 16) { + # For versioningStrategy +16, the version number is taken from repoVersion setting + $repoVersion = [System.Version]$Settings.repoVersion + $majorMinorVersion = "$($repoVersion.Major).$($repoVersion.Minor)" + if (($Settings.versioningStrategy -band 15) -eq 3) { + # For versioning strategy 3, we need to get the build number from repoVersion setting + $appBuild = $repoVersion.Build + if ($appBuild -eq -1) { + OutputWarning -message "RepoVersion setting only contains Major.Minor version. When using versioningStrategy 3, it should contain 3 digits" + $appBuild = 0 + } + } + } + + # Construct object to return + return [PSCustomObject]@{ + MajorMinorVersion = $majorMinorVersion + BuildNumber = $appBuild + RevisionNumber = $appRevision + } +} diff --git a/Actions/CalculateArtifactNames/CalculateArtifactNames.ps1 b/Actions/CalculateArtifactNames/CalculateArtifactNames.ps1 index b2e38fc345..805ba0819c 100644 --- a/Actions/CalculateArtifactNames/CalculateArtifactNames.ps1 +++ b/Actions/CalculateArtifactNames/CalculateArtifactNames.ps1 @@ -7,6 +7,8 @@ [string] $suffix ) +. (Join-Path -Path $PSScriptRoot -ChildPath "../AL-Go-Helper.ps1" -Resolve) + function Set-OutputVariable([string] $name, [string] $value) { Write-Host "Assigning $value to $name" Add-Content -Encoding UTF8 -Path $env:GITHUB_OUTPUT -Value "$name=$value" @@ -18,13 +20,7 @@ if ($project -eq ".") { $project = $settings.repoName } -$branchName = $ENV:GITHUB_HEAD_REF -# $ENV:GITHUB_HEAD_REF is specified only for pull requests, so if it is not specified, use GITHUB_REF_NAME -if (!$branchName) { - $branchName = $ENV:GITHUB_REF_NAME -} - -$branchName = $branchName.Replace('\', '_').Replace('/', '_') +$branchName = Get-CurrentBranchName $projectName = $project.Replace('\', '_').Replace('/', '_') # If the buildmode is default, then we don't want to add it to the artifact name diff --git a/Actions/CompileApps/Compile.ps1 b/Actions/CompileApps/Compile.ps1 new file mode 100644 index 0000000000..5676a92d71 --- /dev/null +++ b/Actions/CompileApps/Compile.ps1 @@ -0,0 +1,192 @@ +[Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSReviewUnusedParameter', 'buildMode', Justification = 'Accepted from workflow; reserved for future incremental build support')] +Param( + [Parameter(HelpMessage = "The GitHub token running the action", Mandatory = $false)] + [string] $token, + [Parameter(HelpMessage = "ArtifactUrl to use for the build", Mandatory = $false)] + [string] $artifact = "", + [Parameter(HelpMessage = "Project folder", Mandatory = $false)] + [string] $project = "", + [Parameter(HelpMessage = "Specifies a mode to use for the build steps", Mandatory = $false)] + [string] $buildMode = 'Default', + [Parameter(HelpMessage = "A path to a JSON-formatted list of dependency apps", Mandatory = $false)] + [string] $dependencyAppsJson = '', + [Parameter(HelpMessage = "A path to a JSON-formatted list of dependency test apps", Mandatory = $false)] + [string] $dependencyTestAppsJson = '', + [Parameter(HelpMessage = "RunId of the baseline workflow run", Mandatory = $false)] + [string] $baselineWorkflowRunId = '0', + [Parameter(HelpMessage = "SHA of the baseline workflow run", Mandatory = $false)] + [string] $baselineWorkflowSHA = '' +) + +. (Join-Path -Path $PSScriptRoot -ChildPath "..\AL-Go-Helper.ps1" -Resolve) +Import-Module (Join-Path -Path $PSScriptRoot "..\.Modules\CompileFromWorkspace.psm1" -Resolve) +Import-Module (Join-Path $PSScriptRoot '..\TelemetryHelper.psm1' -Resolve) +Import-Module (Join-Path -Path $PSScriptRoot -ChildPath "..\DetermineProjectsToBuild\DetermineProjectsToBuild.psm1" -Resolve) -DisableNameChecking +DownloadAndImportBcContainerHelper + +# ANALYZE - Analyze the repository and determine settings +$baseFolder = (Get-BasePath) +$settings = $env:Settings | ConvertFrom-Json | ConvertTo-HashTable +$settings = AnalyzeRepo -settings $settings -baseFolder $baseFolder -project $project -doNotCheckArtifactSetting +$settings = CheckAppDependencyProbingPaths -settings $settings -token $token -baseFolder $baseFolder -project $project + +# Check if there are any app folders or test app folders to compile +if ($settings.appFolders.Count -eq 0 -and $settings.testFolders.Count -eq 0) { + Write-Host "No app folders or test app folders specified for compilation. Skipping compilation step." + return +} + +$projectFolder = Join-Path $baseFolder $project +Push-Location $projectFolder +try { + # Set up output folders + $buildArtifactFolder = Join-Path $projectFolder ".buildartifacts" + $appOutputFolder = Join-Path $buildArtifactFolder "Apps" + $testAppOutputFolder = Join-Path $buildArtifactFolder "TestApps" + if (-not (Test-Path $buildArtifactFolder)) { + New-Item $buildArtifactFolder -ItemType Directory -Force | Out-Null + } + if (-not (Test-Path $appOutputFolder)) { + New-Item $appOutputFolder -ItemType Directory -Force | Out-Null + } + if (-not (Test-Path $testAppOutputFolder)) { + New-Item $testAppOutputFolder -ItemType Directory -Force | Out-Null + } + + # Check for precompile and postcompile overrides + $scriptOverrides = Get-ScriptOverrides -ALGoFolderName (Join-Path $projectFolder ".AL-Go") -OverrideScriptNames @("PreCompileApp", "PostCompileApp") + $scriptOverrides.Keys | ForEach-Object { Trace-Information -Message "Using override for $_" } + + # Prepare build metadata + $buildMetadata = Get-BuildMetadata + + # Get version number + $versionNumber = Get-VersionNumber -Settings $settings + + # Get ruleset file if specified + $rulesetPath = $settings.rulesetFile + if ($settings.rulesetFile) { + $rulesetPath = Join-Path $projectFolder $settings.rulesetFile -Resolve + if (-not (Test-Path $rulesetPath)) { + throw "Ruleset file specified in settings.rulesetFile not found at path '$rulesetPath'." + } + } + + # Read existing install apps and test apps from JSON files + $dependencyApps = @() + $dependencyTestApps = @() + + if ($dependencyAppsJson -and (Test-Path $dependencyAppsJson)) { + try { + $dependencyApps += Get-Content -Path $dependencyAppsJson | ConvertFrom-Json + } + catch { + throw "Failed to parse JSON file at path '$dependencyAppsJson'. Error: $($_.Exception.Message)" + } + } + + if ($dependencyTestAppsJson -and (Test-Path $dependencyTestAppsJson)) { + try { + $dependencyTestApps += Get-Content -Path $dependencyTestAppsJson | ConvertFrom-Json + } + catch { + throw "Failed to parse JSON file at path '$dependencyTestAppsJson'. Error: $($_.Exception.Message)" + } + } + + # Set up a compiler folder + $containerName = GetContainerName($project) + $cacheFolder = "" + if ($settings.gitHubRunner -like "windows-*" -or $settings.gitHubRunner -like "ubuntu-*") { + # On GitHub-hosted runners, use a folder in the runner temp directory for caching to speed up subsequent builds + $cacheFolder = Join-Path $ENV:RUNNER_TEMP ".artifactcache" + } + $compilerFolder = New-BcCompilerFolder -artifactUrl $artifact -vsixFile $settings.vsixFile -containerName "$($containerName)compiler" -cacheFolder $cacheFolder + $packageCachePath = Join-Path $compilerFolder "symbols" + + # Copy dependency apps and test apps to the package cache so the compiler can resolve them + foreach ($appFile in ($dependencyApps + $dependencyTestApps)) { + $appFile = $appFile.Trim('()') + if ($appFile -and (Test-Path $appFile)) { + Copy-Item -Path $appFile -Destination $packageCachePath -Force + OutputDebug "Copied dependency app to package cache: $(Split-Path $appFile -Leaf)" + } + elseif ($appFile) { + OutputWarning -message "Dependency app file not found: $appFile" + } + } + + # Incremental Builds - Determine unmodified apps from baseline workflow run if applicable + if ($baselineWorkflowSHA -and $baselineWorkflowRunId -ne '0' -and $settings.incrementalBuilds.mode -eq 'modifiedApps') { + #TODO: Implement support for incremental builds (AB#620492) + Write-Host "Incremental builds based on modified apps is not yet implemented." + } + + if ((-not $settings.skipUpgrade) -and $settings.enableAppSourceCop) { + # TODO: Missing implementation of around using latest release as a baseline (skipUpgrade) / Appsourcecop.json baseline implementation (AB#620310) + Write-Host "Checking for required upgrades using AppSourceCop..." + } + + # Update the app jsons with version number (and other properties) from the app manifest files + Update-AppJsonProperties -Folders ($settings.appFolders + $settings.testFolders) ` + -MajorMinorVersion $versionNumber.MajorMinorVersion -BuildNumber $versionNumber.BuildNumber -RevisionNumber $versionNumber.RevisionNumber ` + -BuildBy $buildMetadata.BuildBy -BuildUrl $buildMetadata.BuildUrl + + # Collect common parameters for Build-AppsInWorkspace + $buildParams = @{ + CompilerFolder = $compilerFolder + PackageCachePath = $packageCachePath + LogDirectory = $buildArtifactFolder + Ruleset = $rulesetPath + AssemblyProbingPaths = (Get-AssemblyProbingPaths -CompilerFolder $compilerFolder) + PreprocessorSymbols = $settings.preprocessorSymbols + Features = $settings.features + MaxCpuCount = $settings.workspaceCompilation.parallelism + SourceRepositoryUrl = $buildMetadata.SourceRepositoryUrl + SourceCommit = $buildMetadata.SourceCommit + ReportSuppressedDiagnostics = $settings.reportSuppressedDiagnostics + EnableExternalRulesets = $settings.enableExternalRulesets + PreCompileApp = $scriptOverrides['PreCompileApp'] + PostCompileApp = $scriptOverrides['PostCompileApp'] + Analyzers = (Get-CodeAnalyzers -Settings $settings) + CustomAnalyzers = (Get-CustomAnalyzers -Settings $settings -CompilerFolder $compilerFolder) + } + + # Start compilation + $appFiles = @() + $testAppFiles = @() + try { + if ($settings.appFolders.Count -gt 0) { + # Compile Apps + $appFiles = Build-AppsInWorkspace @buildParams ` + -Folders $settings.appFolders ` + -OutFolder $appOutputFolder ` + -AppType 'app' + } + + if ($settings.testFolders.Count -gt 0) { + if (-not ($settings.enableCodeAnalyzersOnTestApps)) { + $buildParams.Analyzers = @() + } + + # Compile Test Apps + $testAppFiles = Build-AppsInWorkspace @buildParams ` + -Folders $settings.testFolders ` + -OutFolder $testAppOutputFolder ` + -AppType 'testApp' + } + + } finally { + New-BuildOutputFile -BuildArtifactFolder $buildArtifactFolder -BuildOutputPath (Join-Path $projectFolder "BuildOutput.txt") -DisplayInConsole -FailOn $settings.failOn + } + + # OUTPUT - Output the updated list of dependency apps and test apps to JSON files for downstream steps + $dependencyApps += $appFiles + $dependencyTestApps += $testAppFiles + Trace-Information -message "Compilation completed. Compiled $(@($appFiles).Count) apps and $(@($testAppFiles).Count) test apps." + + ConvertTo-Json $dependencyApps -Depth 99 -Compress | Out-File -Encoding UTF8 -FilePath $dependencyAppsJson + ConvertTo-Json $dependencyTestApps -Depth 99 -Compress | Out-File -Encoding UTF8 -FilePath $dependencyTestAppsJson +} finally { + Pop-Location +} diff --git a/Actions/CompileApps/README.md b/Actions/CompileApps/README.md new file mode 100644 index 0000000000..16043496ec --- /dev/null +++ b/Actions/CompileApps/README.md @@ -0,0 +1,30 @@ +# Compile apps from workspace + +Compile AL apps by using workspace compilation from the ALTool + +## INPUT + +### ENV variables + +| Name | Description | +| :-- | :-- | +| Settings | env.Settings must be set by a prior call to the ReadSettings Action | +| Secrets | env.Secrets with secrets needed for appDependencyProbingPaths authentication must be read by a prior call to the ReadSecrets Action | + +### Parameters + +| Name | Required | Description | Default value | +| :-- | :-: | :-- | :-- | +| shell | | The shell (powershell or pwsh) in which the PowerShell script should run | powershell | +| token | | The GitHub token running the action | github.token | +| artifact | | ArtifactUrl to use for the build (optional) | '' | +| project | | Project folder | '.' | +| buildMode | | Specifies a mode to use for the build steps | Default | +| dependencyAppsJson | | Path to a JSON file containing a list of dependency apps | '' | +| dependencyTestAppsJson | | Path to a JSON file containing a list of dependency test apps | '' | +| baselineWorkflowRunId | | RunId of the baseline workflow run | '' | +| baselineWorkflowSHA | | SHA of the baseline workflow run | '' | + +## OUTPUT + +None but the action will edit the dependencyAppsJson (installAppsJson) and dependencyTestAppsJson (installTestAppsJson) to include the compiled apps. That way the apps will be installed in the RunPipeline step. diff --git a/Actions/CompileApps/action.yaml b/Actions/CompileApps/action.yaml new file mode 100644 index 0000000000..e702881a8c --- /dev/null +++ b/Actions/CompileApps/action.yaml @@ -0,0 +1,60 @@ +name: Compile Apps +author: Microsoft Corporation +inputs: + shell: + description: Shell in which you want to run the action (powershell or pwsh) + required: false + default: powershell + token: + description: The GitHub token running the action + required: false + default: ${{ github.token }} + artifact: + description: ArtifactUrl to use for the build + required: false + default: '' + project: + description: Project folder + required: false + default: '.' + buildMode: + description: Specifies a mode to use for the build steps + required: false + default: 'Default' + dependencyAppsJson: + description: A path to a JSON-formatted list of dependency apps + required: false + default: '' + dependencyTestAppsJson: + description: A path to a JSON-formatted list of dependency test apps + required: false + default: '' + baselineWorkflowRunId: + description: RunId of the baseline workflow run + required: false + default: '' + baselineWorkflowSHA: + description: SHA of the baseline workflow run + required: false + default: '' +runs: + using: composite + steps: + - name: run + shell: ${{ inputs.shell }} + env: + _token: ${{ inputs.token }} + _artifact: ${{ inputs.artifact }} + _project: ${{ inputs.project }} + _buildMode: ${{ inputs.buildMode }} + _dependencyAppsJson: ${{ inputs.dependencyAppsJson }} + _dependencyTestAppsJson: ${{ inputs.dependencyTestAppsJson }} + _baselineWorkflowRunId: ${{ inputs.baselineWorkflowRunId }} + _baselineWorkflowSHA: ${{ inputs.baselineWorkflowSHA }} + run: | + ${{ github.action_path }}/../Invoke-AlGoAction.ps1 -ActionName "Compile Apps" -Action { + ${{ github.action_path }}/Compile.ps1 -token $ENV:_token -artifact $ENV:_artifact -project $ENV:_project -buildMode $ENV:_buildMode -dependencyAppsJson $ENV:_dependencyAppsJson -dependencyTestAppsJson $ENV:_dependencyTestAppsJson -baselineWorkflowRunId $ENV:_baselineWorkflowRunId -baselineWorkflowSHA $ENV:_baselineWorkflowSHA + } +branding: + icon: terminal + color: blue diff --git a/Actions/Deliver/Deliver.ps1 b/Actions/Deliver/Deliver.ps1 index a20ed3111f..024a8da8a4 100644 --- a/Actions/Deliver/Deliver.ps1 +++ b/Actions/Deliver/Deliver.ps1 @@ -77,6 +77,7 @@ if ($deliveryTarget -eq "AppSource") { Write-Host "Artifacts $artifacts" Write-Host "Projects:" $sortedProjectList | Out-Host +$alreadyDeliveredPackages = @() $secrets = $env:Secrets | ConvertFrom-Json foreach ($thisProject in $sortedProjectList) { @@ -256,15 +257,25 @@ foreach ($thisProject in $sortedProjectList) { Get-Item -Path (Join-Path $folder[0] "*.app") | ForEach-Object { $appJson = Get-AppJsonFromAppFile -appFile $_.FullName $packageName = Get-BcNuGetPackageId -publisher $appJson.publisher -name $appJson.name -id $appJson.id -version $appJson.version - $feed, $packageId, $packageVersion = Find-BcNugetPackage -nuGetServerUrl $nuGetServerUrl -nuGetToken $nuGetToken -packageName $packageName -version $appJson.version -select Exact -allowPrerelease - if (-not $feed) { - $parameters = @{ - "gitHubRepository" = "$ENV:GITHUB_SERVER_URL/$ENV:GITHUB_REPOSITORY" - "preReleaseTag" = $preReleaseTag - "appFile" = $_.FullName + if ($alreadyDeliveredPackages -contains $packageName) { + Write-Host "Package $packageName has already been delivered in this run, skipping" + } + else { + $searchVersion = $appJson.version + if ($preReleaseTag) { + $searchVersion += "-$preReleaseTag" + } + $feed, $packageId, $packageVersion = Find-BcNugetPackage -nuGetServerUrl $nuGetServerUrl -nuGetToken $nuGetToken -packageName $packageName -version $searchVersion -select Exact -allowPrerelease + if (-not $feed) { + $parameters = @{ + "gitHubRepository" = "$ENV:GITHUB_SERVER_URL/$ENV:GITHUB_REPOSITORY" + "preReleaseTag" = $preReleaseTag + "appFile" = $_.FullName + } + $package = New-BcNuGetPackage @parameters + Push-BcNuGetPackage -nuGetServerUrl $nuGetServerUrl -nuGetToken $nuGetToken -bcNuGetPackage $package + $alreadyDeliveredPackages += $packageName } - $package = New-BcNuGetPackage @parameters - Push-BcNuGetPackage -nuGetServerUrl $nuGetServerUrl -nuGetToken $nuGetToken -bcNuGetPackage $package } } } diff --git a/Actions/DetermineArtifactsForRelease/DetermineArtifactsForRelease.ps1 b/Actions/DetermineArtifactsForRelease/DetermineArtifactsForRelease.ps1 index 2acf558b15..3dc5539da1 100644 --- a/Actions/DetermineArtifactsForRelease/DetermineArtifactsForRelease.ps1 +++ b/Actions/DetermineArtifactsForRelease/DetermineArtifactsForRelease.ps1 @@ -57,13 +57,20 @@ $projects | ForEach-Object { } else { Write-Host "Search for $project-$refname-Apps-$buildVersion or $project-$refname-PowerPlatformSolution-$buildVersion" - $artifact = $allArtifacts | Where-Object { $_.name -eq "$project-$refname-Apps-$buildVersion"-or $_.name -eq "$project-$refname-PowerPlatformSolution-$buildVersion" } | Select-Object -First 1 + $artifact = $allArtifacts | Where-Object { $_.name -eq "$project-$refname-Apps-$buildVersion" -or $_.name -eq "$project-$refname-PowerPlatformSolution-$buildVersion" } | Select-Object -First 1 } if ($artifact) { $startIndex = $artifact.name.LastIndexOf('-') + 1 $artifactsVersion = $artifact.name.SubString($startIndex) } else { + # No release artifacts (Apps or PowerPlatformSolution) were found for this project. + # This can happen when a project contains only test apps - in that case, skip the project + # rather than failing the release, as test apps are not part of the release. + if ($allArtifacts | Where-Object { $_.name -like "$project-$refname-*TestApps-*.*.*.*" }) { + Write-Host "::Warning::No release artifacts found for project $project, only test artifacts are available. Skipping project for release." + return + } throw "No artifacts found for this project" } if ($sha) { @@ -84,6 +91,9 @@ $projects | ForEach-Object { throw "No artifacts found for version $artifactsVersion" } } +if ($include.Count -eq 0 -or -not $sha) { + throw "No release artifacts found for any project. Ensure that at least one project produces release artifacts (Apps or PowerPlatformSolution) before creating a release." +} $artifacts = @{ "include" = $include } $artifactsJson = $artifacts | ConvertTo-Json -compress Add-Content -Encoding UTF8 -Path $env:GITHUB_OUTPUT -Value "artifacts=$artifactsJson" diff --git a/Actions/DetermineProjectsToBuild/DetermineProjectsToBuild.psm1 b/Actions/DetermineProjectsToBuild/DetermineProjectsToBuild.psm1 index fc1aeed1f9..6710c29089 100644 --- a/Actions/DetermineProjectsToBuild/DetermineProjectsToBuild.psm1 +++ b/Actions/DetermineProjectsToBuild/DetermineProjectsToBuild.psm1 @@ -387,6 +387,41 @@ function Get-BuildAllApps { return (IsFullBuildRequired -baseFolder $baseFolder -modifiedFiles $modifiedFiles -fullBuildPatterns @($ALGoSettingsFile) -noticeMessage "building all apps") } +<# +.Synopsis + Converts a project-relative folder path to a repo-relative path. +.Description + Converts a project-relative folder path (from settings.appFolders etc.) to a repo-relative path + that matches the format used by skipFolders (produced by GetFoldersFromAllProjects). + Settings folders may use ../ to reference paths above the project folder, + so the function resolves to an absolute path first, then strips the base folder prefix + to produce a clean repo-relative path. +.Parameter folder + The project-relative folder path to convert (e.g. '.\app' or '..\..\..\src\Apps\MyApp\App'). +.Parameter projectPath + The absolute path to the project directory. +.Parameter baseFolder + The absolute path to the repository root. +.Outputs + A repo-relative path string, or $null if the folder cannot be resolved or is outside the base folder. +#> +function ConvertTo-RepoRelativePath { + param( + [string] $folder, + [string] $projectPath, + [string] $baseFolder + ) + $fullPath = Join-Path $projectPath $folder -Resolve -ErrorAction SilentlyContinue + if (-not $fullPath) { + return $null + } + $normalizedBase = $baseFolder.TrimEnd([System.IO.Path]::DirectorySeparatorChar) + [System.IO.Path]::DirectorySeparatorChar + if ($fullPath.StartsWith($normalizedBase, [System.StringComparison]::OrdinalIgnoreCase)) { + return $fullPath.Substring($normalizedBase.Length) + } + return $null +} + <# .Synopsis Downloads unmodified artifacts from the baseline workflow run @@ -430,15 +465,9 @@ function Get-UnmodifiedAppsFromBaselineWorkflowRun { Sort-AppFoldersByDependencies -appFolders $allFolders -baseFolder $baseFolder -skippedApps ([ref] $skipFolders) -unknownDependencies ([ref]$unknownDependencies) -knownApps ([ref] $knownApps) -selectSubordinates $modifiedFolders | Out-Null OutputMessageAndArray -message "Skip folders" -arrayOfStrings $skipFolders - $projectWithSeperator = '' - if ($project) { - $projectWithSeperator = "$project$([System.IO.Path]::DirectorySeparatorChar)" - } - - # AppFolders, TestFolders and BcptTestFolders in settings are always preceded by ./ or .\, so we need to remove that (hence Substring(2)) - $downloadAppFolders = @($settings.appFolders | Where-Object { $skipFolders -contains "$projectWithSeperator$($_.SubString(2))" }) - $downloadTestFolders = @($settings.testFolders | Where-Object { $skipFolders -contains "$projectWithSeperator$($_.SubString(2))" }) - $downloadBcptTestFolders = @($settings.bcptTestFolders | Where-Object { $skipFolders -contains "$projectWithSeperator$($_.SubString(2))" }) + $downloadAppFolders = @($settings.appFolders | Where-Object { $skipFolders -contains (ConvertTo-RepoRelativePath -folder $_ -projectPath $projectPath -baseFolder $baseFolder) }) + $downloadTestFolders = @($settings.testFolders | Where-Object { $skipFolders -contains (ConvertTo-RepoRelativePath -folder $_ -projectPath $projectPath -baseFolder $baseFolder) }) + $downloadBcptTestFolders = @($settings.bcptTestFolders | Where-Object { $skipFolders -contains (ConvertTo-RepoRelativePath -folder $_ -projectPath $projectPath -baseFolder $baseFolder) }) OutputMessageAndArray -message "Download appFolders" -arrayOfStrings $downloadAppFolders OutputMessageAndArray -message "Download testFolders" -arrayOfStrings $downloadTestFolders diff --git a/Actions/DownloadProjectDependencies/ComputeDependencyArtifactPattern.ps1 b/Actions/DownloadProjectDependencies/ComputeDependencyArtifactPattern.ps1 new file mode 100644 index 0000000000..42398e34ec --- /dev/null +++ b/Actions/DownloadProjectDependencies/ComputeDependencyArtifactPattern.ps1 @@ -0,0 +1,17 @@ +$errorActionPreference = "Stop"; $ProgressPreference = "SilentlyContinue"; Set-StrictMode -Version 2.0 + +. (Join-Path $PSScriptRoot "../AL-Go-Helper.ps1" -Resolve) +Import-Module (Join-Path $PSScriptRoot "DownloadProjectDependencies.psm1" -Resolve) -Force -DisableNameChecking + +$projectDependencies = $ENV:_projectDependenciesJson | ConvertFrom-Json | ConvertTo-HashTable -recurse +$pattern = Get-DependencyArtifactPattern -Project $ENV:_project -ProjectDependencies $projectDependencies + +if ($pattern) { + Write-Host "Dependency artifact pattern: $pattern" + Add-Content -Encoding UTF8 -Path $env:GITHUB_OUTPUT -Value "hasPattern=true" + Add-Content -Encoding UTF8 -Path $env:GITHUB_OUTPUT -Value "pattern=$pattern" +} +else { + Write-Host "No dependency projects found, skipping artifact download" + Add-Content -Encoding UTF8 -Path $env:GITHUB_OUTPUT -Value "hasPattern=false" +} diff --git a/Actions/DownloadProjectDependencies/DownloadProjectDependencies.Action.ps1 b/Actions/DownloadProjectDependencies/DownloadProjectDependencies.Action.ps1 index a4b475ba53..fabf800c8e 100644 --- a/Actions/DownloadProjectDependencies/DownloadProjectDependencies.Action.ps1 +++ b/Actions/DownloadProjectDependencies/DownloadProjectDependencies.Action.ps1 @@ -23,7 +23,11 @@ function DownloadDependenciesFromProbingPaths { $settings = AnalyzeRepo -settings $settings -baseFolder $baseFolder -project $project -doNotCheckArtifactSetting -doNotIssueWarnings $settings = CheckAppDependencyProbingPaths -settings $settings -token $token -baseFolder $baseFolder -project $project if ($settings.ContainsKey('appDependencyProbingPaths') -and $settings.appDependencyProbingPaths) { - return GetDependencies -probingPathsJson $settings.appDependencyProbingPaths -saveToPath $destinationPath | Where-Object { $_ } + $dependencies = GetDependencies -probingPathsJson $settings.appDependencyProbingPaths -saveToPath $destinationPath | Where-Object { $_ } + + # GetDependencies may return .zip files (from DownloadArtifact/DownloadRelease). + # Extract .app files from any zips so downstream consumers receive clean .app paths. + return Resolve-DependencyFiles -Dependencies $dependencies -DestinationPath $destinationPath } } @@ -48,7 +52,7 @@ function DownloadDependenciesFromCurrentBuild { Write-Host "Dependency projects: $($dependencyProjects -join ', ')" # For each dependency project, calculate the corresponding probing path - $dependeciesProbingPaths = @() + $dependenciesProbingPaths = @() foreach($dependencyProject in $dependencyProjects) { Write-Host "Reading settings for project '$dependencyProject'" $dependencyProjectSettings = ReadSettings -baseFolder $baseFolder -project $dependencyProject @@ -72,7 +76,7 @@ function DownloadDependenciesFromCurrentBuild { $baseBranch = $ENV:GITHUB_REF_NAME } - $dependeciesProbingPaths += @(@{ + $dependenciesProbingPaths += @(@{ "release_status" = "thisBuild" "version" = "latest" "buildMode" = $dependencyBuildMode @@ -87,7 +91,7 @@ function DownloadDependenciesFromCurrentBuild { # For each probing path, download the dependencies $downloadedDependencies = @() - foreach($probingPath in $dependeciesProbingPaths) { + foreach($probingPath in $dependenciesProbingPaths) { $buildMode = $probingPath.buildMode $project = $probingPath.projects $branch = $probingPath.branch @@ -107,7 +111,7 @@ function DownloadDependenciesFromCurrentBuild { } } - return $downloadedDependencies + return Resolve-DependencyFiles -Dependencies $downloadedDependencies -DestinationPath $destinationPath } . (Join-Path -Path $PSScriptRoot -ChildPath "..\AL-Go-Helper.ps1" -Resolve) @@ -148,7 +152,7 @@ $downloadedDependencies | ForEach-Object { } } -# Add dependencies from settings +# Add dependencies from settings (these are already resolved to .app files) $downloadedApps += $settingsDependencies.Apps $downloadedTestApps += $settingsDependencies.TestApps diff --git a/Actions/DownloadProjectDependencies/DownloadProjectDependencies.psm1 b/Actions/DownloadProjectDependencies/DownloadProjectDependencies.psm1 index 5d00b34212..b40fb1c3ae 100644 --- a/Actions/DownloadProjectDependencies/DownloadProjectDependencies.psm1 +++ b/Actions/DownloadProjectDependencies/DownloadProjectDependencies.psm1 @@ -328,4 +328,92 @@ function Get-DependenciesFromInstallApps { return $install } -Export-ModuleMember -Function Get-AppFilesFromUrl, Get-AppFilesFromLocalPath, Get-DependenciesFromInstallApps +<# + .SYNOPSIS + Computes a minimatch-compatible glob pattern for downloading only dependency-project artifacts. + .DESCRIPTION + Given a project and its dependency map (from projectDependenciesJson), constructs a brace-expansion + pattern that matches only the Apps, TestApps, and Dependencies artifacts for the dependency projects. + This pattern is designed for use with the actions/download-artifact 'pattern' input (which uses minimatch). + + The pattern uses '*Apps' to match both 'Apps' and 'TestApps' as well as build-mode-prefixed variants + (e.g. 'CleanApps', 'CleanTestApps'). Similarly '*Dependencies' matches 'Dependencies' and variants. + .PARAMETER Project + The name of the current AL-Go project. + .PARAMETER ProjectDependencies + A hashtable mapping project names to arrays of their dependency project names (direct + transitive). + .OUTPUTS + A minimatch glob pattern string, or $null if the project has no dependencies. +#> +function Get-DependencyArtifactPattern { + param( + [Parameter(Mandatory = $true)] + [string] $Project, + [Parameter(Mandatory = $true)] + [hashtable] $ProjectDependencies + ) + + $dependencyProjects = @() + if ($ProjectDependencies.Keys -contains $Project) { + $dependencyProjects = @($ProjectDependencies."$Project") + } + + if ($dependencyProjects.Count -eq 0) { + return $null + } + + $branchName = Get-CurrentBranchName + + # Build brace-expansion entries: 2 per dependency project (*Apps covers Apps+TestApps+buildMode variants) + $entries = @() + foreach ($dep in $dependencyProjects) { + $sanitizedDep = $dep.Replace('\', '_').Replace('/', '_') + $entries += "$sanitizedDep-$branchName-*Apps-*" + $entries += "$sanitizedDep-$branchName-*Dependencies-*" + } + + return "{$($entries -join ',')}" +} + +<# + .SYNOPSIS + Resolves dependency file paths by extracting .app files from any zip archives. + .DESCRIPTION + Takes an array of dependency file paths (which may be .app files or .zip archives) + and returns an array of .app file paths. Zip archives are extracted and the contained + .app files are copied to the destination path. Test app markers (parentheses wrapping) + are preserved through extraction. + .PARAMETER Dependencies + An array of dependency file paths. Test apps are wrapped in parentheses, e.g. "(path.zip)". + .PARAMETER DestinationPath + The path where extracted .app files should be placed. + .OUTPUTS + An array of resolved .app file paths, with test app markers preserved. +#> +function Resolve-DependencyFiles { + Param( + [Parameter(Mandatory = $false)] + [string[]] $Dependencies = @(), + [Parameter(Mandatory = $true)] + [string] $DestinationPath + ) + + if (-not $Dependencies -or $Dependencies.Count -eq 0) { + return @() + } + + return @($Dependencies | ForEach-Object { + $isTestApp = $_.StartsWith('(') + $filePath = $_.Trim('()') + if ($filePath -and (Test-Path $filePath) -and (Test-IsZipFile -Path $filePath)) { + $appFiles = Expand-ZipFileToAppFiles -ZipFile $filePath -DestinationPath $DestinationPath + Remove-Item -Path $filePath -Force -ErrorAction SilentlyContinue + if ($isTestApp) { $appFiles | ForEach-Object { "($_)" } } else { $appFiles } + } + else { + $_ + } + }) +} + +Export-ModuleMember -Function Get-AppFilesFromUrl, Get-AppFilesFromLocalPath, Get-DependenciesFromInstallApps, Get-DependencyArtifactPattern, Resolve-DependencyFiles diff --git a/Actions/DownloadProjectDependencies/README.md b/Actions/DownloadProjectDependencies/README.md index 4892918f1f..687c8754c2 100644 --- a/Actions/DownloadProjectDependencies/README.md +++ b/Actions/DownloadProjectDependencies/README.md @@ -4,6 +4,8 @@ Downloads artifacts from AL-Go projects, that are dependencies of a given AL-Go The action constructs arrays of paths to .app files, that are dependencies of the apps in an AL-Go project +To optimize build performance, only artifacts from dependency projects are downloaded from the current workflow run (using a minimatch pattern filter), rather than downloading all artifacts. + ## INPUT ### ENV variables diff --git a/Actions/DownloadProjectDependencies/action.yaml b/Actions/DownloadProjectDependencies/action.yaml index d8b8bb2a05..62d77526b4 100644 --- a/Actions/DownloadProjectDependencies/action.yaml +++ b/Actions/DownloadProjectDependencies/action.yaml @@ -29,9 +29,22 @@ outputs: runs: using: composite steps: - - name: Download artifacts from current build - uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0 + - name: Compute dependency artifact pattern + id: computePattern + shell: ${{ inputs.shell }} + env: + _project: ${{ inputs.project }} + _projectDependenciesJson: ${{ inputs.projectDependenciesJson }} + run: | + ${{ github.action_path }}/../Invoke-AlGoAction.ps1 -ActionName "ComputeDependencyArtifactPattern" -Action { + ${{ github.action_path }}/ComputeDependencyArtifactPattern.ps1 + } + + - name: Download dependency artifacts from current build + if: steps.computePattern.outputs.hasPattern == 'true' + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: + pattern: ${{ steps.computePattern.outputs.pattern }} path: ${{ github.workspace }}/.dependencies - name: Download project dependencies diff --git a/Actions/GetWorkflowMultiRunBranches/GetWorkflowMultiRunBranches.ps1 b/Actions/GetWorkflowMultiRunBranches/GetWorkflowMultiRunBranches.ps1 index 822daede31..1da9a963ca 100644 --- a/Actions/GetWorkflowMultiRunBranches/GetWorkflowMultiRunBranches.ps1 +++ b/Actions/GetWorkflowMultiRunBranches/GetWorkflowMultiRunBranches.ps1 @@ -1,4 +1,6 @@ param( + [Parameter(Mandatory = $false, HelpMessage = "The GitHub event name that triggered the workflow.")] + [string] $workflowEventName = $env:GITHUB_EVENT_NAME, [Parameter(Mandatory = $false, HelpMessage = "Comma-separated value of branch name patterns to include if they exist. If not specified, only the current branch is returned. Wildcards are supported.")] [string] $includeBranches ) @@ -6,7 +8,7 @@ $gitHubHelperPath = Join-Path $PSScriptRoot '../Github-Helper.psm1' -Resolve Import-Module $gitHubHelperPath -DisableNameChecking -switch ($env:GITHUB_EVENT_NAME) { +switch ($workflowEventName) { 'schedule' { Write-Host "Event is schedule: getting branches from settings" $settings = ConvertFrom-Json $env:Settings @@ -20,8 +22,8 @@ switch ($env:GITHUB_EVENT_NAME) { $branchPatterns = @() } } - 'workflow_dispatch' { - Write-Host "Event is workflow_dispatch: getting branches from input" + { $_ -in 'workflow_dispatch', 'workflow_call' } { + Write-Host "Event is $($_): getting branches from input" $branchPatterns = @($includeBranches.Split(',') | ForEach-Object { $_.Trim() }) } } diff --git a/Actions/GetWorkflowMultiRunBranches/README.md b/Actions/GetWorkflowMultiRunBranches/README.md index a253210c15..3ed5a4f32d 100644 --- a/Actions/GetWorkflowMultiRunBranches/README.md +++ b/Actions/GetWorkflowMultiRunBranches/README.md @@ -17,6 +17,7 @@ If the workflow is run on a schedule, the branches are determined based on the ` | Name | Required | Description | Default value | | :-- | :-: | :-- | :-- | | shell | false | The shell (powershell or pwsh) in which the PowerShell script in this action should run | powershell | +| workflowEventName | false | The GitHub event name that triggered the workflow. *(override for reusable workflows)* | github.event_name | | includeBranches | false | Comma-separated value of branch name patterns to include if they exist. If not specified, only the current branch is returned. Wildcards are supported. |''| ## OUTPUT diff --git a/Actions/GetWorkflowMultiRunBranches/action.yaml b/Actions/GetWorkflowMultiRunBranches/action.yaml index 75593060f6..3cee775851 100644 --- a/Actions/GetWorkflowMultiRunBranches/action.yaml +++ b/Actions/GetWorkflowMultiRunBranches/action.yaml @@ -5,6 +5,10 @@ inputs: description: Shell in which you want to run the action (powershell or pwsh) required: false default: powershell + workflowEventName: + description: The GitHub event name that triggered the workflow. + required: false + default: ${{ github.event_name }} includeBranches: description: Comma-separated value of branch name patterns to include if they exist. If not specified, only the current branch is returned. Wildcards are supported. required: false @@ -20,10 +24,11 @@ runs: shell: ${{ inputs.shell }} id: GetWorkflowMultiRunBranches env: + _workflowEventName: ${{ inputs.workflowEventName }} _includeBranches: ${{ inputs.includeBranches }} run: | ${{ github.action_path }}/../Invoke-AlGoAction.ps1 -ActionName "GetWorkflowMultiRunBranches" -Action { - ${{ github.action_path }}/GetWorkflowMultiRunBranches.ps1 -includeBranches $env:_includeBranches + ${{ github.action_path }}/GetWorkflowMultiRunBranches.ps1 -workflowEventName $env:_workflowEventName -includeBranches $env:_includeBranches } branding: icon: terminal diff --git a/Actions/Github-Helper.psm1 b/Actions/Github-Helper.psm1 index 11fdf588b1..5a806673c1 100644 --- a/Actions/Github-Helper.psm1 +++ b/Actions/Github-Helper.psm1 @@ -141,9 +141,16 @@ function GetDependencies { $projects = $dependency.projects $buildMode = $dependency.buildMode + if ($mask -eq 'TestApps') { + $altMask = 'Apps' + } + else { + $altMask = 'TestApps' + } # change the mask to include the build mode if($buildMode -ne "Default") { $mask = "$buildMode$mask" + $altMask = "$buildMode$altMask" } Write-Host "Locating $mask artifacts for projects: $projects" @@ -169,8 +176,15 @@ function GetDependencies { } } elseif ($mask -like '*Apps') { - Write-Host "$project not built, downloading from artifacts" - $missingProjects += @($project) + # Check whether Apps/TestApps exists before determining that project isn't built + $altDownloadName = Join-Path $saveToPath "$project-$branchName-$altMask-*" + if (!(Test-Path $altDownloadName -PathType Container)) { + Write-Host "$project not built, downloading from artifacts" + $missingProjects += @($project) + } + else { + Write-Host "$project built, but $mask not found" + } } } if ($missingProjects -and $dependency.baselineWorkflowID) { @@ -796,10 +810,19 @@ function DownloadRelease { # GitHub replaces series of special characters with a single dot when uploading release assets $project = [Uri]::EscapeDataString($project.Replace('\','_').Replace('/','_').Replace(' ','.')).Replace('%2A','*').Replace('%3F','?').Replace('%','') Write-Host "project '$project'" - $assetPattern1 = "$project-*-$mask-*.zip" - $assetPattern2 = "$project-$mask-*.zip" + # Pattern 1: project-branch-mask-version.zip (branch used for release creation cannot contain -) + # Pattern 2: project-mask-version.zip (no branch) + if ($project -eq '*') { + $escapedProject = '.*' + } + else { + $escapedProject = [regex]::Escape($project) + } + $escapedMask = [regex]::Escape($mask) + $assetPattern1 = "^$escapedProject-[^-]+-$escapedMask-.+\.zip$" + $assetPattern2 = "^$escapedProject-$escapedMask-.+\.zip$" Write-Host "AssetPatterns: '$assetPattern1' | '$assetPattern2'" - $assets = @($release.assets | Where-Object { $_.name -like $assetPattern1 -or $_.name -like $assetPattern2 }) + $assets = @($release.assets | Where-Object { $_.name -match $assetPattern1 -or $_.name -match $assetPattern2 }) foreach($asset in $assets) { $uri = "$api_url/repos/$repository/releases/assets/$($asset.id)" Write-Host $uri @@ -1069,6 +1092,22 @@ function GetArtifactsFromWorkflowRun { # Get sanitized project names (the way they appear in the artifact names) $projectArr = @(@($projects.Split(',')) | ForEach-Object { $_.Replace('\','_').Replace('/','_') }) + # Get branch used in workflowRun (cached per workflow run to avoid repeated API calls) + if (-not $Script:WorkflowRunBranchCache) { + $Script:WorkflowRunBranchCache = @{} + } + + if ($Script:WorkflowRunBranchCache.ContainsKey("$repository/$workflowRun")) { + $branch = $Script:WorkflowRunBranchCache["$repository/$workflowRun"] + } + else { + $workflowRunInfo = (InvokeWebRequest -Headers $headers -Uri "$api_url/repos/$repository/actions/runs/$workflowRun").Content | ConvertFrom-Json + $branch = $workflowRunInfo.head_branch + $branch = $branch.Replace('\', '_').Replace('/', '_') + $Script:WorkflowRunBranchCache["$repository/$workflowRun"] = $branch + } + Write-Host "Branch for workflow run $workflowRun is $branch" + # Get the artifacts from the the workflow run while($true) { $artifactsURI = "$api_url/repos/$repository/actions/runs/$workflowRun/artifacts?per_page=$per_page&page=$page" @@ -1080,14 +1119,40 @@ function GetArtifactsFromWorkflowRun { } foreach($project in $projectArr) { - $artifactPattern = "$project-*-$mask-*" # e.g. "MyProject-*-Apps-*", format is: "project-branch-mask-version" + # e.g. "MyProject-main-Apps-*", format is: "project-branch-mask-version" + # Mask might include buildMode like TranslatedTestApps + $artifactPattern = "$project-$branch-$mask-*" $matchingArtifacts = @($artifacts.artifacts | Where-Object { $_.name -like $artifactPattern }) if ($matchingArtifacts.Count -eq 0) { continue } - $matchingArtifacts = @($matchingArtifacts) #enforce array + # If there are reruns of the build we found, we might see artifacts like: + # Test DBC-BHG-SAF-T-main-TestApps-1.0.48.0 + # Test DBC-BHG-SAF-T-main-TestApps-1.0.48.1 + # We want to keep only the latest version of each artifact (based on the last segment of the version) + $matchingArtifacts = @($matchingArtifacts | ForEach-Object { + # Sort on version number object + if ($_.name -match '^(.*)-(\d+\.\d+\.\d+\.\d+)$') { + [PSCustomObject]@{ + Name = $Matches[1] + Version = [version]$Matches[2] + Obj = $_ + } + } + else { + # artifacts from PR builds doesn't match the versioning pattern but are sortable + [PSCustomObject]@{ + Name = $_.name + Version = $_.name + Obj = $_ + } + } + } | Group-Object Name | ForEach-Object { + $_.Group | Sort-Object Version -Descending | Select-Object -First 1 + } | + Select-Object -ExpandProperty Obj) foreach($artifact in $matchingArtifacts) { Write-Host "Found artifact $($artifact.name) (ID: $($artifact.id)) for mask $mask and project $project" @@ -1424,3 +1489,17 @@ function Invoke-CommandWithRetry { } } } + +<# +.SYNOPSIS + Gets the base path of the GitHub workspace or the git repository root. + Assumes either GITHUB_WORKSPACE is set (GitHub Actions) or git is available and the current directory is within a git repository. +.OUTPUTS + The base path as a string. +#> +function Get-BasePath() { + if ($ENV:GITHUB_WORKSPACE) { + return $ENV:GITHUB_WORKSPACE + } + return git rev-parse --show-toplevel +} diff --git a/Actions/PipelineCleanup/PipelineCleanup.ps1 b/Actions/PipelineCleanup/PipelineCleanup.ps1 index 81209b175f..339cb8e767 100644 --- a/Actions/PipelineCleanup/PipelineCleanup.ps1 +++ b/Actions/PipelineCleanup/PipelineCleanup.ps1 @@ -3,10 +3,15 @@ [string] $project = "." ) -. (Join-Path -Path $PSScriptRoot -ChildPath "..\AL-Go-Helper.ps1" -Resolve) -DownloadAndImportBcContainerHelper +try { + . (Join-Path -Path $PSScriptRoot -ChildPath "..\AL-Go-Helper.ps1" -Resolve) + DownloadAndImportBcContainerHelper -if ($project -eq ".") { $project = "" } + if ($project -eq ".") { $project = "" } -$containerName = GetContainerName($project) -Remove-Bccontainer $containerName + $containerName = GetContainerName($project) + Remove-Bccontainer $containerName +} +catch { + Write-Host "Pipeline Cleanup failed: $($_.Exception.Message)" +} diff --git a/Actions/RunPipeline/RunPipeline.ps1 b/Actions/RunPipeline/RunPipeline.ps1 index d71cdf1a06..03e3a040ef 100644 --- a/Actions/RunPipeline/RunPipeline.ps1 +++ b/Actions/RunPipeline/RunPipeline.ps1 @@ -85,8 +85,11 @@ try { $secrets = @{} } - $appBuild = $settings.appBuild - $appRevision = $settings.appRevision + if ($settings.workspaceCompilation.enabled -and $settings.doNotPublishApps) { + OutputColor -Message "Workspace compilation complete; doNotPublishApps is set. Exiting." -Color Yellow + return + } + 'licenseFileUrl','codeSignCertificateUrl','codeSignCertificatePassword','keyVaultCertificateUrl','keyVaultCertificatePassword','keyVaultClientId','gitHubPackagesContext','applicationInsightsConnectionString' | ForEach-Object { # Secrets might not be read during Pull Request runs if ($secrets.Keys -contains $_) { @@ -119,15 +122,35 @@ try { $settings = AnalyzeRepo -settings $settings -baseFolder $baseFolder -project $project @analyzeRepoParams $settings = CheckAppDependencyProbingPaths -settings $settings -token $token -baseFolder $baseFolder -project $project + $isTestProject = $settings.projectsToTest -and $settings.projectsToTest.Count -gt 0 if ((-not $settings.appFolders) -and (-not $settings.testFolders) -and (-not $settings.bcptTestFolders)) { - Write-Host "Repository is empty, exiting" - exit + if (-not $isTestProject) { + Write-Host "Repository is empty, exiting" + exit + } + Write-Host "Test project: no local app/test folders, will install and test apps from upstream projects" } $buildArtifactFolder = Join-Path $projectPath ".buildartifacts" - New-Item $buildArtifactFolder -ItemType Directory | Out-Null + if (-not (Test-Path $buildArtifactFolder)) { + New-Item $buildArtifactFolder -ItemType Directory | Out-Null + } elseif(-not ($settings.workspaceCompilation.enabled)) { + OutputDebug -message "Build artifacts folder $buildArtifactFolder already exists. Previous build artifacts might interfere with the current build." + } + + # When using workspace compilation, apps are already compiled - pass empty folders to Run-AlPipeline + if ($settings.workspaceCompilation.enabled) { + $appFolders = @() + $testFolders = @() + $bcptTestFolders = $settings.bcptTestFolders + } + else { + $appFolders = $settings.appFolders + $testFolders = $settings.testFolders + $bcptTestFolders = $settings.bcptTestFolders + } - if ($baselineWorkflowSHA -and $baselineWorkflowRunId -ne '0' -and $settings.incrementalBuilds.mode -eq 'modifiedApps') { + if ((-not $settings.workspaceCompilation.enabled) -and $baselineWorkflowSHA -and $baselineWorkflowRunId -ne '0' -and $settings.incrementalBuilds.mode -eq 'modifiedApps') { # Incremental builds are enabled and we are only building modified apps try { $modifiedFiles = @(Get-ModifiedFiles -baselineSHA $baselineWorkflowSHA) @@ -252,28 +275,34 @@ try { $previousApps = @() if (!$settings.skipUpgrade) { - Write-Host "::group::Locating previous release" - try { - $branchForRelease = if ($ENV:GITHUB_BASE_REF) { $ENV:GITHUB_BASE_REF } else { $ENV:GITHUB_REF_NAME } - $latestRelease = GetLatestRelease -token $token -api_url $ENV:GITHUB_API_URL -repository $ENV:GITHUB_REPOSITORY -ref $branchForRelease - if ($latestRelease) { - Write-Host "Using $($latestRelease.name) (tag $($latestRelease.tag_name)) as previous release" - $artifactsFolder = Join-Path $baseFolder "artifacts" - if(-not (Test-Path $artifactsFolder)) { - New-Item $artifactsFolder -ItemType Directory | Out-Null + if ($settings.workspaceCompilation.enabled) { + OutputWarning -message "skipUpgrade is ignored when workspaceCompilation is enabled." # TODO: Missing implementation when workspace compilation is enabled (AB#620310) + } else { + OutputGroupStart -Message "Locating previous release" + try { + $branchForRelease = if ($ENV:GITHUB_BASE_REF) { $ENV:GITHUB_BASE_REF } else { $ENV:GITHUB_REF_NAME } + $latestRelease = GetLatestRelease -token $token -api_url $ENV:GITHUB_API_URL -repository $ENV:GITHUB_REPOSITORY -ref $branchForRelease + if ($latestRelease) { + Write-Host "Using $($latestRelease.name) (tag $($latestRelease.tag_name)) as previous release" + $artifactsFolder = Join-Path $baseFolder "artifacts" + if(-not (Test-Path $artifactsFolder)) { + New-Item $artifactsFolder -ItemType Directory | Out-Null + } + DownloadRelease -token $token -projects $project -api_url $ENV:GITHUB_API_URL -repository $ENV:GITHUB_REPOSITORY -release $latestRelease -path $artifactsFolder -mask "Apps" + $previousApps += @(Get-ChildItem -Path $artifactsFolder | ForEach-Object { $_.FullName }) + } + else { + OutputWarning -message "No previous release found" } - DownloadRelease -token $token -projects $project -api_url $ENV:GITHUB_API_URL -repository $ENV:GITHUB_REPOSITORY -release $latestRelease -path $artifactsFolder -mask "Apps" - $previousApps += @(Get-ChildItem -Path $artifactsFolder | ForEach-Object { $_.FullName }) } - else { - OutputWarning -message "No previous release found" + catch { + OutputError -message "Error trying to locate previous release. Error was $($_.Exception.Message)" + exit + } + finally { + OutputGroupEnd } } - catch { - OutputError -message "Error trying to locate previous release. Error was $($_.Exception.Message)" - exit - } - Write-Host "::endgroup::" } $additionalCountries = $settings.additionalCountries @@ -282,38 +311,16 @@ try { if (-not $gitHubHostedRunner) { $imageName = $settings.cacheImageName if ($imageName) { - Write-Host "::group::Flush ContainerHelper Cache" + OutputGroupStart -Message "Flush ContainerHelper Cache" Flush-ContainerHelperCache -cache 'all,exitedcontainers' -keepdays $settings.cacheKeepDays - Write-Host "::endgroup::" + OutputGroupEnd } } $authContext = $null $environmentName = "" $CreateRuntimePackages = $false - if ($settings.versioningStrategy -eq -1) { - $artifactVersion = [Version]$settings.artifact.Split('/')[4] - $runAlPipelineParams += @{ - "appVersion" = "$($artifactVersion.Major).$($artifactVersion.Minor)" - } - $appBuild = $artifactVersion.Build - $appRevision = $artifactVersion.Revision - } - elseif (($settings.versioningStrategy -band 16) -eq 16) { - # For versioningStrategy +16, the version number is taken from repoVersion setting - $repoVersion = [System.Version]$settings.repoVersion - if (($settings.versioningStrategy -band 15) -eq 3) { - # For versioning strategy 3, we need to get the build number from repoVersion setting - $appBuild = $repoVersion.Build - if ($appBuild -eq -1) { - Write-Warning "RepoVersion setting only contains Major.Minor version. When using versioningStrategy 3, it should contain 3 digits" - $appBuild = 0 - } - } - $runAlPipelineParams += @{ - "appVersion" = "$($repoVersion.Major).$($repoVersion.Minor)" - } - } + $versionNumber = Get-VersionNumber -Settings $settings $allTestResults = "testresults*.xml" $testResultsFile = Join-Path $projectPath "TestResults.xml" @@ -328,18 +335,9 @@ try { Add-Content -Encoding UTF8 -Path $env:GITHUB_ENV -Value "containerName=$containerName" Set-Location $projectPath - $runAlPipelineOverrides | ForEach-Object { - $scriptName = $_ - $scriptPath = Join-Path $ALGoFolderName "$ScriptName.ps1" - if (Test-Path -Path $scriptPath -Type Leaf) { - Write-Host "Add override for $scriptName" - Trace-Information -Message "Using override for $scriptName" - - $runAlPipelineParams += @{ - "$scriptName" = (Get-Command $scriptPath | Select-Object -ExpandProperty ScriptBlock) - } - } - } + $scriptOverrides = Get-ScriptOverrides -ALGoFolderName $ALGoFolderName -OverrideScriptNames $runAlPipelineOverrides + $scriptOverrides.Keys | ForEach-Object { Trace-Information -Message "Using override for $_" } + $runAlPipelineParams += $scriptOverrides if ($runAlPipelineParams.Keys -notcontains 'RemoveBcContainer') { $runAlPipelineParams += @{ @@ -479,21 +477,15 @@ try { } if ($buildMode -eq 'Translated') { - if ($runAlPipelineParams.Keys -notcontains 'features') { - $runAlPipelineParams["features"] = @() - } Write-Host "Adding translationfile feature" - $runAlPipelineParams["features"] += "translationfile" - } - - if ($runAlPipelineParams.Keys -notcontains 'preprocessorsymbols') { - $runAlPipelineParams["preprocessorsymbols"] = @() + $settings.features += "translationfile" } - if ($settings.ContainsKey('preprocessorSymbols')) { + if ($settings.preprocessorSymbols.Count -gt 0) { Write-Host "Adding Preprocessor symbols : $($settings.preprocessorSymbols -join ',')" - $runAlPipelineParams["preprocessorsymbols"] += $settings.preprocessorSymbols } + $runAlPipelineParams["preprocessorsymbols"] = $settings.preprocessorSymbols + $runAlPipelineParams["features"] = $settings.features Write-Host "Invoke Run-AlPipeline with buildmode $buildMode" Run-AlPipeline @runAlPipelineParams ` @@ -516,9 +508,9 @@ try { -generateDependencyArtifact ` -updateDependencies:$settings.updateDependencies ` -previousApps $previousApps ` - -appFolders $settings.appFolders ` - -testFolders $settings.testFolders ` - -bcptTestFolders $settings.bcptTestFolders ` + -appFolders $appFolders ` + -testFolders $testFolders ` + -bcptTestFolders $bcptTestFolders ` -pageScriptingTests $settings.pageScriptingTests ` -restoreDatabases $settings.restoreDatabases ` -buildOutputFile $buildOutputFile ` @@ -539,7 +531,7 @@ try { -pageScriptingTestResultsFile (Join-Path $buildArtifactFolder 'PageScriptingTestResults.xml') ` -pageScriptingTestResultsFolder (Join-Path $buildArtifactFolder 'PageScriptingTestResultDetails') ` -CreateRuntimePackages:$CreateRuntimePackages ` - -appBuild $appBuild -appRevision $appRevision ` + -appVersion ($versionNumber.MajorMinorVersion) -appBuild ($versionNumber.BuildNumber) -appRevision ($versionNumber.RevisionNumber) ` -uninstallRemovedApps if ($containerBaseFolder) { diff --git a/Internal/Deploy.ps1 b/Internal/Deploy.ps1 index 1f8c978a4c..563db98200 100644 --- a/Internal/Deploy.ps1 +++ b/Internal/Deploy.ps1 @@ -221,14 +221,15 @@ try { # replace defaultBcContainerHelperVersion $found = $false for($idx=0; $idx -lt $lines.count; $idx++) { - if ($lines[$idx] -match '^(\s*)\$defaultBcContainerHelperVersion(\s*)=(\s*)"(.*)" # (.*)$') { - $lines[$idx] = "$($Matches[1])`$defaultBcContainerHelperVersion$($Matches[2])=$($Matches[3])""$($config.defaultBcContainerHelperVersion)"" # $($Matches[5])" + if ($lines[$idx] -match '^(\s*)\$defaultBcContainerHelperVersion(\s*)=(\s*)"([^"]*)"(\s*#.*)?$') { + $comment = if ($Matches[5]) { $Matches[5] } else { "" } + $lines[$idx] = "$($Matches[1])`$defaultBcContainerHelperVersion$($Matches[2])=$($Matches[3])""$($config.defaultBcContainerHelperVersion)""$comment" $found = $true break } } if (-not $found) { - throw 'Could not find defaultBcContainerHelperVersion line in AL-Go-Helpers.ps1 matching "^(\s*)\$defaultBcContainerHelperVersion(\s*)=(\s*)"(.*)" # (.*)$"' + throw 'Could not find defaultBcContainerHelperVersion line in AL-Go-Helper.ps1 matching "^(\s*)\$defaultBcContainerHelperVersion(\s*)=(\s*)""([^""]*)""(\s*#.*)?$"' } } diff --git a/RELEASENOTES.md b/RELEASENOTES.md index 8b69114709..38745e0410 100644 --- a/RELEASENOTES.md +++ b/RELEASENOTES.md @@ -1,3 +1,71 @@ +### Optimized dependency artifact downloads for multi-project repositories + +The `DownloadProjectDependencies` action now downloads only artifacts from dependency projects instead of all workflow artifacts. For repositories with many AL-Go projects, this reduces build runner bandwidth and speeds up the dependency download step. + +### Issues + +- Incremental builds (`modifiedApps` mode) now correctly identify unmodified apps for projects whose `appFolders` reference paths outside the project directory (e.g. using `../`) +- Issue 2204 - Workspace compilation ignores vsixFile setting +- Issue 2211 - Cannot create a release if a project contains only test apps +- Issue 2214 - Workspace compilation not working with external dependencies +- Issue 2126 Deliver and Deploy actions now skip execution when no app artifacts are found, preventing errors on initial commits + +## v9.0 + +### Needs Context in Build job moved from environment variable to file + +`NeedsContext` is currently available as an environment variable in the build step of AL-Go. In some cases on repos with a large amount of projects, it's possible for this variable to exceed the max size GitHub allows for such variables. To work around this issue, we now place the contents of `NeedsContext` in a json file, where `NeedsContext` is the path to that file. + +If you have any custom processes that uses `NeedsContext`, those needs to be updated to now first read the contents of the json file. The structure of the json is identical to what was previously in the variable, so only extra step is to read the file. + +### Added support for workspace compilation + +With v28 of Business Central, the ALTool now also provides the ability to compile workspaces of apps. This has the added advantage that the ALTool can compute the dependency graph for the apps in the workspace and compile apps in parallel (if possible). For AL-Go projects with large amounts of apps that can save a lot of time. If you want to try this out you can enable it via the following setting + +```json + "workspaceCompilation": { + "enabled": true + } +``` + +By default apps are compiled sequentially but this can be changed via the parallelism property. This allows you to configure the maximum amount of parallel compilation processes. Set to 0 or -1 to use all available processors. + +```json + "workspaceCompilation": { + "enabled": true, + "parallelism": 4 + } +``` + +### Test Projects — split builds and tests for faster feedback + +AL-Go now supports **test projects**: a new project type that separates test execution from compilation. A test project does not build any apps itself — instead it depends on one or more regular projects, installs the apps they produce, and runs tests against them. + +This lets you re-run tests without waiting for a full recompilation, and makes it easy to organize large repositories where builds and test suites have different scopes or cadences. + +**Getting started** + +Add a `projectsToTest` setting to the project-level `.AL-Go/settings.json` of an empty project (no `appFolders` or `testFolders`): + +```json +{ + "projectsToTest": ["build/projects/MyProject"] +} +``` + +AL-Go will automatically: + +- Resolve the dependency so the test project always builds after its target project(s). +- Install the Test Runner, Test Framework, and Test Libraries into the container. +- Run all tests from the installed test apps. + +**Key rules** + +- A test project must **not** contain buildable code (no `appFolders`, `testFolders`, or `bcptTestFolders`). AL-Go will fail with a clear error if it detects both `projectsToTest` and buildable folders. +- A test project cannot depend on another test project. +- You can target multiple projects: `"projectsToTest": ["build/projects/ProjectA", "build/projects/ProjectB"]`. +- Use full project paths as they appear in the repository. + ### Improving error detection and build reliability when downloading project dependencies The `DownloadProjectDependencies` action now downloads app files from URLs specified in the `installApps` and `installTestApps` settings upfront, rather than validating URLs at build time. This change provides: @@ -6,10 +74,26 @@ The `DownloadProjectDependencies` action now downloads app files from URLs speci - Clearer error messages when secrets are missing or URLs are invalid - Warnings for potential issues like duplicate filenames +### Improve overall performance by postponing projects with no dependants + +The time it takes to build projects can vary significantly depending on factors such as whether you are using Linux or Windows, Containers or CompilerFolders, and whether apps are being published or tests are being run. + +By default, projects are built according to their dependency order. As soon as all dependencies for a project are satisfied, the project is added to the next layer of jobs. + +The new setting `postponeProjectInBuildOrder` allows you to delay long running jobs (f.ex. test runs) with no dependants until the final layer of the build order. This can improve overall build performance by preventing subsequent layers from waiting on projects that take longer to complete but are not required for further dependencies. + ### Issues - Attempt to start docker service in case it is not running - NextMajor (v28) fails when downloading dependencies from NuGet-feed +- Issue 2084 Multiple artifacts failure if you re-run failed jobs after flaky tests +- Issue 2085 Projects that doesn't contain both Apps and TestApps are wrongly seen as not built. +- Issue 2086 Postpone jobs, which doesn't have any dependents to the end of the build order. +- Rework input handling of workflow 'Update AL-Go System Files' for trigger 'workflow_call' + +### New Settings + +- `postponeProjectInBuildOrder` is a new project setting, which will (if set to true) cause the project to be postponed until the last build job when possible. If set on test projects, then all tests can be deferred until all builds have succeeded. ## v8.3 diff --git a/Scenarios/DeliveryTargets.md b/Scenarios/DeliveryTargets.md index 87fe57bbf0..8f4a19d10c 100644 --- a/Scenarios/DeliveryTargets.md +++ b/Scenarios/DeliveryTargets.md @@ -102,6 +102,12 @@ Create a secret named `NuGetContext` with the following format: > [!NOTE] > Replace ``, ``, ``, and `` with your actual values. +> [!TIP] +> Use the BcContainerHelper function `New-ALGoNuGetContext` to create a correctly formatted JSON structure. + +> [!WARNING] +> The secret must be in compressed JSON format (single line). Multi-line JSON will break AL-Go functionality as curly brackets will be masked in logs. + ```json {"token":"","serverUrl":"https://pkgs.dev.azure.com///_packaging//nuget/v3/index.json"} ``` @@ -294,6 +300,8 @@ Your custom delivery script receives a hash table with the following parameters: > **Note:** The folder parameters (`*Folder`) may be `$null` if no artifacts of that type were found. The plural versions (`*Folders`) contain arrays of all matching folders across different build modes. +> **Important:** The delivery step is automatically skipped at the workflow level when no app artifacts are available. This means your custom delivery script will not be executed if no app artifacts were built. This behavior prevents errors on initial commits when no apps have been built yet. + ### Branch-Specific Delivery Configure different delivery targets for different branches: diff --git a/Scenarios/settings.md b/Scenarios/settings.md index 3e15d01f2f..6a54b92c42 100644 --- a/Scenarios/settings.md +++ b/Scenarios/settings.md @@ -48,6 +48,8 @@ When running a workflow or a local script, the settings are applied by reading s | appDependencyProbingPaths | Array of dependency specifications, from which apps will be downloaded when the CI/CD workflow is starting. Every dependency specification consists of the following properties:
**repo** = repository
**version** = version (default latest)
**release_status** = latestBuild/release/prerelease/draft (default release)
**projects** = projects (default * = all)
**branch** = branch (default main)
**AuthTokenSecret** = Name of secret containing auth token (default none)
| [ ] | | preprocessorSymbols | List of preprocessor symbols to use when building the apps. This setting can be specified in [workflow specific settings files](https://aka.ms/algosettings#where-are-the-settings-located) or in [conditional settings](https://aka.ms/algosettings#conditional-settings). | [ ] | | bcptThresholds | Structure with properties for the thresholds when running performance tests using the Business Central Performance Toolkit.
**DurationWarning** = a warning is shown if the duration of a bcpt test degrades more than this percentage (default 10)
**DurationError** - an error is shown if the duration of a bcpt test degrades more than this percentage (default 25)
**NumberOfSqlStmtsWarning** - a warning is shown if the number of SQL statements from a bcpt test increases more than this percentage (default 5)
**NumberOfSqlStmtsError** - an error is shown if the number of SQL statements from a bcpt test increases more than this percentage (default 10)
*Note that errors and warnings on the build in GitHub are only issued when a threshold is exceeded on the codeunit level, when an individual operation threshold is exceeded, it is only shown in the test results viewer.* | +| postponeProjectInBuildOrder | When this setting is enabled (true), the project will be postponed to the end of the build sequence whenever possible - specifically, when no other projects depend on it. This allows, for example, test projects to run only after all build projects have completed successfully.
This setting only has effect when useProjectDependencies is also enabled, as dependency information is required to determine whether postponement is allowed. | false | +| projectsToTest | An array of project paths that this project depends on for testing. Setting this marks the project as a **test project** — it will not build any apps itself but will install the apps produced by the referenced projects and run tests against them. This lets you separate test execution from compilation so you can re-run tests without a full rebuild.
The project must not contain any buildable code (`appFolders`, `testFolders`, or `bcptTestFolders`). A test project cannot depend on another test project. Use full project paths (e.g. `"build/projects/MyApp"`). | [ ] | ## AppSource specific basic project settings @@ -110,7 +112,8 @@ The repository settings are only read from the repository settings file (.github | enableCodeCop | If enableCodeCop is set to true, the CI/CD workflow will enable the CodeCop analyzer when building. | false | | enableUICop | If enableUICop is set to true, the CI/CD workflow will enable the UICop analyzer when building. | false | | customCodeCops | CustomCodeCops is an array of paths or URLs to custom Code Cop DLLs you want to enable when building. | [ ] | -| enableCodeAnalyzersOnTestApps | If enableCodeAnalyzersOnTestApps is set to true, the code analyzers will be enabled when building test apps as well. | false | +| features | Features is an array of compiler features to enable when building. Possible values include **LcgTranslationFile**, **TranslationFile**, and **GenerateCaptions**. | [ ] | +| enableCodeAnalyzersOnTestApps| If enableCodeAnalyzersOnTestApps is set to true, the code analyzers will be enabled when building test apps as well. | false | | trackALAlertsInGitHub | If trackALAlertsInGitHub is set to true, AL code analysis results will be uploaded and tracked in the GitHub security tab. Additionally, if Advanced Security is enabled in the repo, new AL code alerts will be posted in PRs that introduce them. This setting must be enabled on the repo level, but can be optionally disabled per project.
**Note:** AL Alerts are only enabled for warnings at the moment. Support for displaying errors will come in a future release | false | | failOn | Specifies what the pipeline will fail on. Allowed values are none, warning, newWarning and error. Using 'newWarning' will lead to pull requests failing if new warnings are added, while still behaving like 'error' for normal build steps. | error | | rulesetFile | Filename of the custom ruleset file | | @@ -128,7 +131,8 @@ The repository settings are only read from the repository settings file (.github | assignPremiumPlan | Setting assignPremiumPlan to true in your project setting file, causes the build container to be created with the AssignPremiumPlan set. This causes the auto-created user to have Premium Plan enabled. This setting is needed if your tests require premium plan enabled. | false | | enableTaskScheduler | Setting enableTaskScheduler to true in your project setting file, causes the build container to be created with the Task Scheduler running. | false | | useCompilerFolder | Setting useCompilerFolder to true causes your pipelines to use containerless compiling. Unless you also set **doNotPublishApps** to true, setting useCompilerFolder to true won't give you any performance advantage, since AL-Go for GitHub will still need to create a container in order to publish and test the apps. In the future, publishing and testing will be split from building and there will be other options for getting an instance of Business Central for publishing and testing. **Note** when using UseCompilerFolder you need to sign apps using the new signing mechanism described [here](../Scenarios/Codesigning.md). | false | -| excludeEnvironments | excludeEnvironments can be an array of GitHub Environments, which should be excluded from the list of environments considered for deployment. github-pages is automatically added to this array and cannot be used as environment for deployment of AL-Go for GitHub projects. | [ ] | +| workspaceCompilation | **PREVIEW:** Configuration for workspace compilation. This uses the AL tool to compile all apps in the workspace in a single operation, which can improve build performance for repositories with multiple apps. Like **useCompilerFolder**, this is containerless compiling.
**enabled** - Set to true to enable workspace compilation. Default: false.
**parallelism** - The number of parallel compilation processes. Set to 0 or -1 to use all available processors. Default: 1.

**Current limitations:**
  • Upgrade testing is not performed (previousApps are not downloaded).
  • AppSourceCop baseline validation against the previous release is not supported.
  • BCPT test folders are not compiled by workspace compilation.
  • Linux runners are not supported yet.
| { "enabled": false, "parallelism": 1 } | +| excludeEnvironments| excludeEnvironments can be an array of GitHub Environments, which should be excluded from the list of environments considered for deployment. github-pages is automatically added to this array and cannot be used as environment for deployment of AL-Go for GitHub projects. | [ ] | | trustMicrosoftNuGetFeeds | Unless this setting is set to false, AL-Go for GitHub will trust the NuGet feeds provided by Microsoft. The feeds provided by Microsoft contains all Microsoft apps, all Microsoft symbols and symbols for all AppSource apps. | true | | trustedNuGetFeeds | trustedNuGetFeeds can be an array of NuGet feed specifications, which AL-Go for GitHub will use for dependency resolution. Every feed specification must include a URL property and can optionally include a few other properties:
**url** = The URL of the feed (examples: https://pkgs.dev.azure.com/myorg/apps/\_packaging/myrepo/nuget/v3/index.json or https://nuget.pkg.github.com/mygithuborg/index.json").
**authTokenSecret** = If the NuGet feed specified by URL is private, the authTokenSecret must be the name of a secret containing the authentication token with permissions to search and read packages from the NuGet feed.
**patterns** = AL-Go for GitHub will only trust packages, where the ID matches this pattern. Default is all packages (\*).
**fingerprints** = If specified, AL-Go for GitHub will only trust packages signed with a certificate with a fingerprint matching one of the fingerprints in this array. | [ ] | | nuGetFeedSelectMode | Determines the select mode when finding Business Central app packages from NuGet feeds, based on the dependency version specified in app.json. Options are:
- `Earliest` for earliest version of the package
- `EarliestMatching` for earliest version of the package also compatible with the Business Central version used
- `Exact` for the exact version of the package
- `Latest` for the latest version of the package
- `LatestMatching` for the latest version of the package also compatible with the Business Central version used. | LatestMatching | @@ -422,6 +426,7 @@ Note that changes to AL-Go for GitHub or Run-AlPipeline functionality in the fut | PipelineInitialize.ps1 | Initialize the pipeline | | DockerPull.ps1 | Pull the image specified by the parameter $imageName | | NewBcContainer.ps1 | Create the container using the parameters transferred in the $parameters hashtable | +| NewBcCompilerFolder.ps1 | Create the compilerFolder using the parameters transferred in the $parameters hashtable | | ImportTestToolkitToBcContainer.ps1 | Import the test toolkit apps specified by the $parameters hashtable | | CompileAppInBcContainer.ps1 | Compile the apps specified by the $parameters hashtable | | GetBcContainerAppInfo.ps1 | Get App Info for the apps specified by the $parameters hashtable | @@ -432,7 +437,9 @@ Note that changes to AL-Go for GitHub or Run-AlPipeline functionality in the fut | ImportTestDataInBcContainer.ps1 | If this function is provided, it is expected to insert the test data needed for running tests | | RunTestsInBcContainer.ps1 | Run the tests specified by the $parameters hashtable | | GetBcContainerAppRuntimePackage.ps1 | Get the runtime package specified by the $parameters hashtable | +| GetBcContainerEventLog.ps1 | Get the eventlog based on the $parameters hashtable | | RemoveBcContainer.ps1 | Cleanup based on the $parameters hashtable | +| RemoveBcCompilerFolder.ps1 | Cleanup based on the $parameters hashtable | | InstallMissingDependencies | Install missing dependencies | | BackupBcContainerDatabases | Backup Databases in container for subsequent restore(s) | | RestoreDatabasesInBcContainer | Restore Databases in container | diff --git a/Templates/AppSource App/.github/workflows/CICD.yaml b/Templates/AppSource App/.github/workflows/CICD.yaml index ca95fbd3e1..43a872b6d8 100644 --- a/Templates/AppSource App/.github/workflows/CICD.yaml +++ b/Templates/AppSource App/.github/workflows/CICD.yaml @@ -216,7 +216,7 @@ jobs: uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Download artifacts - ErrorLogs - uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0 + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 if: (success() || failure()) with: pattern: '*-*ErrorLogs-*' @@ -255,7 +255,7 @@ jobs: uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Download artifacts - uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0 + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: path: '.artifacts' @@ -272,7 +272,7 @@ jobs: - name: Setup Pages if: needs.Initialization.outputs.deployALDocArtifact == 1 - uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5.0.0 + uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6.0.0 - name: Build Reference Documentation uses: microsoft/AL-Go-Actions/BuildReferenceDocumentation@main @@ -282,14 +282,14 @@ jobs: artifactUrl: ${{ env.artifact }} - name: Upload pages artifact - uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4.0.0 + uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0 with: path: ".aldoc/_site/" - name: Deploy to GitHub Pages if: needs.Initialization.outputs.deployALDocArtifact == 1 id: deployment - uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4.0.5 + uses: actions/deploy-pages@cd2ce8fcbc39b97be8ca5fce6e763baed58fa128 # v5.0.0 Deploy: needs: [ Initialization, Build ] @@ -311,7 +311,7 @@ jobs: uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Download artifacts - uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0 + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: path: '.artifacts' @@ -338,6 +338,7 @@ jobs: - name: Deploy to Business Central id: Deploy + if: hashFiles('.artifacts/**/*.app') != '' uses: microsoft/AL-Go-Actions/Deploy@main env: Secrets: '${{ steps.ReadSecrets.outputs.Secrets }}' @@ -348,6 +349,11 @@ jobs: type: 'CD' deploymentEnvironmentsJson: ${{ needs.Initialization.outputs.deploymentEnvironmentsJson }} + - name: Deployment skipped + if: hashFiles('.artifacts/**/*.app') == '' + run: | + Write-Host "::Notice::Deployment to environment ${{ matrix.environment }} was skipped because no app artifacts were found" + - name: Deploy to Power Platform if: env.type == 'PTE' && env.powerPlatformSolutionFolder != '' uses: microsoft/AL-Go-Actions/DeployPowerPlatform@main @@ -373,7 +379,7 @@ jobs: uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Download artifacts - uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0 + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: path: '.artifacts' @@ -391,6 +397,7 @@ jobs: getSecrets: '${{ matrix.deliveryTarget }}Context' - name: Deliver + if: hashFiles('.artifacts/**/*.app') != '' uses: microsoft/AL-Go-Actions/Deliver@main env: Secrets: '${{ steps.ReadSecrets.outputs.Secrets }}' @@ -401,6 +408,11 @@ jobs: deliveryTarget: ${{ matrix.deliveryTarget }} artifacts: '.artifacts' + - name: Delivery skipped + if: hashFiles('.artifacts/**/*.app') == '' + run: | + Write-Host "::Notice::Delivery to ${{ matrix.deliveryTarget }} was skipped because no app artifacts were found" + PostProcess: needs: [ Initialization, Build, Deploy, Deliver, DeployALDoc, CodeAnalysisUpload ] if: (!cancelled()) diff --git a/Templates/AppSource App/.github/workflows/CreateRelease.yaml b/Templates/AppSource App/.github/workflows/CreateRelease.yaml index 83f18ee816..6a003d30ab 100644 --- a/Templates/AppSource App/.github/workflows/CreateRelease.yaml +++ b/Templates/AppSource App/.github/workflows/CreateRelease.yaml @@ -149,7 +149,7 @@ jobs: target_commitish: ${{ steps.determineArtifactsForRelease.outputs.commitish }} - name: Create release - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0 + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 id: createrelease env: bodyMD: ${{ steps.createreleasenotes.outputs.releaseNotes }} @@ -219,7 +219,7 @@ jobs: Invoke-WebRequest -UseBasicParsing -Headers $headers -Uri $ENV:MATRIX_URL -OutFile "$($ENV:MATRIX_NAME).zip" - name: Upload release artifacts - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0 + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: releaseId: ${{ needs.createrelease.outputs.releaseId }} MATRIX_NAME: ${{ matrix.name }} diff --git a/Templates/AppSource App/.github/workflows/DeployReferenceDocumentation.yaml b/Templates/AppSource App/.github/workflows/DeployReferenceDocumentation.yaml index 3968fbb10c..dfbbe56ffd 100644 --- a/Templates/AppSource App/.github/workflows/DeployReferenceDocumentation.yaml +++ b/Templates/AppSource App/.github/workflows/DeployReferenceDocumentation.yaml @@ -57,7 +57,7 @@ jobs: - name: Setup Pages if: steps.DetermineDeploymentEnvironments.outputs.deployALDocArtifact == 1 - uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5.0.0 + uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6.0.0 - name: Build Reference Documentation uses: microsoft/AL-Go-Actions/BuildReferenceDocumentation@main @@ -67,14 +67,14 @@ jobs: artifactUrl: ${{ env.artifact }} - name: Upload pages artifact - uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4.0.0 + uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0 with: path: ".aldoc/_site/" - name: Deploy to GitHub Pages if: steps.DetermineDeploymentEnvironments.outputs.deployALDocArtifact == 1 id: deployment - uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4.0.5 + uses: actions/deploy-pages@cd2ce8fcbc39b97be8ca5fce6e763baed58fa128 # v5.0.0 - name: Finalize the workflow if: always() diff --git a/Templates/AppSource App/.github/workflows/PullRequestHandler.yaml b/Templates/AppSource App/.github/workflows/PullRequestHandler.yaml index 4550c61206..06b2638c24 100644 --- a/Templates/AppSource App/.github/workflows/PullRequestHandler.yaml +++ b/Templates/AppSource App/.github/workflows/PullRequestHandler.yaml @@ -129,7 +129,7 @@ jobs: ref: ${{ format('refs/pull/{0}/head', github.event.pull_request.number) }} - name: Download artifacts - ErrorLogs - uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0 + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 if: (success() || failure()) with: pattern: '*-*ErrorLogs-*' diff --git a/Templates/AppSource App/.github/workflows/UpdateGitHubGoSystemFiles.yaml b/Templates/AppSource App/.github/workflows/UpdateGitHubGoSystemFiles.yaml index 204fa8810e..3f5504b7f0 100644 --- a/Templates/AppSource App/.github/workflows/UpdateGitHubGoSystemFiles.yaml +++ b/Templates/AppSource App/.github/workflows/UpdateGitHubGoSystemFiles.yaml @@ -21,6 +21,10 @@ on: default: '' workflow_call: inputs: + caller: + description: Name of the calling workflow (use github.workflow as value when calling) + type: string + required: true templateUrl: description: Template Repository URL (current is {TEMPLATEURL}) type: string @@ -50,6 +54,7 @@ defaults: shell: powershell env: + WorkflowEventName: ${{ inputs.caller && 'workflow_call' || github.event_name }} ALGoOrgSettings: ${{ vars.ALGoOrgSettings }} ALGoRepoSettings: ${{ vars.ALGoRepoSettings }} @@ -76,12 +81,13 @@ jobs: uses: microsoft/AL-Go-Actions/GetWorkflowMultiRunBranches@main with: shell: powershell - includeBranches: ${{ github.event.inputs.includeBranches }} + workflowEventName: ${{ env.WorkflowEventName }} + includeBranches: ${{ inputs.includeBranches }} - name: Determine Template URL id: DetermineTemplateUrl env: - TemplateUrlAsInput: '${{ github.event.inputs.templateUrl }}' + TemplateUrlAsInput: '${{ inputs.templateUrl }}' run: | $templateUrl = $env:templateUrl # Available from ReadSettings step if ($ENV:TemplateUrlAsInput) { @@ -133,12 +139,12 @@ jobs: - name: Calculate Commit Options env: - directCommit: '${{ github.event.inputs.directCommit }}' - downloadLatest: '${{ github.event.inputs.downloadLatest }}' + directCommit: '${{ inputs.directCommit }}' + downloadLatest: '${{ inputs.downloadLatest }}' run: | $errorActionPreference = "Stop"; $ProgressPreference = "SilentlyContinue"; Set-StrictMode -Version 2.0 - if('${{ github.event_name }}' -in 'workflow_dispatch', 'workflow_call') { - Write-Host "Using inputs from ${{ github.event_name }} event" + if($env:WorkflowEventName -in 'workflow_dispatch', 'workflow_call') { + Write-Host "Using inputs from $($env:WorkflowEventName) event" $directCommit = $env:directCommit $downloadLatest = $env:downloadLatest } diff --git a/Templates/AppSource App/.github/workflows/_BuildALGoProject.yaml b/Templates/AppSource App/.github/workflows/_BuildALGoProject.yaml index 79f563753d..b295574f7d 100644 --- a/Templates/AppSource App/.github/workflows/_BuildALGoProject.yaml +++ b/Templates/AppSource App/.github/workflows/_BuildALGoProject.yaml @@ -109,7 +109,7 @@ jobs: shell: ${{ inputs.shell }} project: ${{ inputs.project }} buildMode: ${{ inputs.buildMode }} - get: useCompilerFolder,keyVaultCodesignCertificateName,doNotSignApps,doNotRunTests,doNotRunBcptTests,doNotRunpageScriptingTests,artifact,generateDependencyArtifact,trustedSigning,useGitSubmodules,trackALAlertsInGitHub + get: useCompilerFolder,workspaceCompilation,keyVaultCodesignCertificateName,doNotSignApps,doNotRunTests,doNotRunBcptTests,doNotRunpageScriptingTests,artifact,generateDependencyArtifact,trustedSigning,useGitSubmodules,trackALAlertsInGitHub - name: Determine whether to build project id: DetermineBuildProject @@ -148,7 +148,7 @@ jobs: - name: Cache Business Central Artifacts if: steps.DetermineBuildProject.outputs.BuildIt == 'True' && env.useCompilerFolder == 'True' && inputs.useArtifactCache && env.artifactCacheKey - uses: actions/cache@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3 + uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 with: path: ${{ runner.temp }}/.artifactcache key: ${{ env.artifactCacheKey }} @@ -166,13 +166,40 @@ jobs: projectDependenciesJson: ${{ inputs.projectDependenciesJson }} baselineWorkflowRunId: ${{ inputs.baselineWorkflowRunId }} + - name: Compile Apps + id: compile + uses: microsoft/AL-Go-Actions/CompileApps@main + if: steps.DetermineBuildProject.outputs.BuildIt == 'True' && fromJson(env.workspaceCompilation).enabled == true + env: + Secrets: '${{ steps.ReadSecrets.outputs.Secrets }}' + BuildMode: ${{ inputs.buildMode }} + with: + shell: ${{ inputs.shell }} + artifact: ${{ env.artifact }} + project: ${{ inputs.project }} + buildMode: ${{ inputs.buildMode }} + dependencyAppsJson: ${{ steps.DownloadProjectDependencies.outputs.DownloadedApps }} + dependencyTestAppsJson: ${{ steps.DownloadProjectDependencies.outputs.DownloadedTestApps }} + baselineWorkflowRunId: ${{ inputs.baselineWorkflowRunId }} + baselineWorkflowSHA: ${{ inputs.baselineWorkflowSHA }} + + - name: Save needs context to file + shell: pwsh + run: | + $nc = @' + ${{ inputs.needsContext }} + '@ + $needsContextPath = Join-Path $ENV:RUNNER_TEMP 'needsContext.json' + [System.IO.File]::WriteAllText($needsContextPath, $nc.Trim()) + Add-Content -Encoding UTF8 -Path $env:GITHUB_ENV -Value "NeedsContext=$needsContextPath" + - name: Build uses: microsoft/AL-Go-Actions/RunPipeline@main if: steps.DetermineBuildProject.outputs.BuildIt == 'True' env: Secrets: '${{ steps.ReadSecrets.outputs.Secrets }}' BuildMode: ${{ inputs.buildMode }} - NeedsContext: ${{ inputs.needsContext }} + NeedsContext: ${{ env.NeedsContext }} with: shell: ${{ inputs.shell }} artifact: ${{ env.artifact }} @@ -203,7 +230,7 @@ jobs: suffix: ${{ inputs.artifactsNameSuffix }} - name: Publish artifacts - apps - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: inputs.artifactsRetentionDays >= 0 && (hashFiles(format('{0}/.buildartifacts/Apps/*',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.AppsArtifactsName }} @@ -212,7 +239,7 @@ jobs: retention-days: ${{ inputs.artifactsRetentionDays }} - name: Publish artifacts - dependencies - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: inputs.artifactsRetentionDays >= 0 && env.generateDependencyArtifact == 'True' && (hashFiles(format('{0}/.buildartifacts/Dependencies/*',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.DependenciesArtifactsName }} @@ -221,7 +248,7 @@ jobs: retention-days: ${{ inputs.artifactsRetentionDays }} - name: Publish artifacts - test apps - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: inputs.artifactsRetentionDays >= 0 && (hashFiles(format('{0}/.buildartifacts/TestApps/*',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.TestAppsArtifactsName }} @@ -230,7 +257,7 @@ jobs: retention-days: ${{ inputs.artifactsRetentionDays }} - name: Publish artifacts - build output - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: (success() || failure()) && (hashFiles(format('{0}/BuildOutput.txt',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.BuildOutputArtifactsName }} @@ -238,7 +265,7 @@ jobs: if-no-files-found: ignore - name: Publish artifacts - container event log - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: (failure()) && (hashFiles(format('{0}/ContainerEventLog.evtx',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.ContainerEventLogArtifactsName }} @@ -246,7 +273,7 @@ jobs: if-no-files-found: ignore - name: Publish artifacts - test results - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: (success() || failure()) && (hashFiles(format('{0}/.buildartifacts/TestResults.xml',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.TestResultsArtifactsName }} @@ -254,7 +281,7 @@ jobs: if-no-files-found: ignore - name: Publish artifacts - bcpt test results - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: (success() || failure()) && (hashFiles(format('{0}/.buildartifacts/bcptTestResults.json',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.BcptTestResultsArtifactsName }} @@ -262,7 +289,7 @@ jobs: if-no-files-found: ignore - name: Publish artifacts - page scripting test results - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: (success() || failure()) && (hashFiles(format('{0}/.buildartifacts/PageScriptingTestResults.xml',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.PageScriptingTestResultsArtifactsName }} @@ -270,7 +297,7 @@ jobs: if-no-files-found: ignore - name: Publish artifacts - page scripting test result details - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: (success() || failure()) && (hashFiles(format('{0}/.buildartifacts/PageScriptingTestResultDetails/*',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.PageScriptingTestResultDetailsArtifactsName }} @@ -278,7 +305,7 @@ jobs: if-no-files-found: ignore - name: Publish artifacts - ErrorLogs - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: (success() || failure()) && inputs.artifactsRetentionDays >= 0 && (hashFiles(format('{0}/.buildartifacts/ErrorLogs/*',inputs.project)) != '') && env.trackALAlertsInGitHub == 'True' with: name: ${{ steps.calculateArtifactsNames.outputs.ErrorLogsArtifactsName }} diff --git a/Templates/Per Tenant Extension/.github/workflows/CICD.yaml b/Templates/Per Tenant Extension/.github/workflows/CICD.yaml index c26c9d9b85..99820093c8 100644 --- a/Templates/Per Tenant Extension/.github/workflows/CICD.yaml +++ b/Templates/Per Tenant Extension/.github/workflows/CICD.yaml @@ -230,7 +230,7 @@ jobs: uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Download artifacts - ErrorLogs - uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0 + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 if: (success() || failure()) with: pattern: '*-*ErrorLogs-*' @@ -269,7 +269,7 @@ jobs: uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Download artifacts - uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0 + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: path: '.artifacts' @@ -286,7 +286,7 @@ jobs: - name: Setup Pages if: needs.Initialization.outputs.deployALDocArtifact == 1 - uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5.0.0 + uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6.0.0 - name: Build Reference Documentation uses: microsoft/AL-Go-Actions/BuildReferenceDocumentation@main @@ -296,14 +296,14 @@ jobs: artifactUrl: ${{ env.artifact }} - name: Upload pages artifact - uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4.0.0 + uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0 with: path: ".aldoc/_site/" - name: Deploy to GitHub Pages if: needs.Initialization.outputs.deployALDocArtifact == 1 id: deployment - uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4.0.5 + uses: actions/deploy-pages@cd2ce8fcbc39b97be8ca5fce6e763baed58fa128 # v5.0.0 Deploy: needs: [ Initialization, Build, BuildPP ] @@ -325,7 +325,7 @@ jobs: uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Download artifacts - uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0 + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: path: '.artifacts' @@ -352,6 +352,7 @@ jobs: - name: Deploy to Business Central id: Deploy + if: hashFiles('.artifacts/**/*.app') != '' uses: microsoft/AL-Go-Actions/Deploy@main env: Secrets: '${{ steps.ReadSecrets.outputs.Secrets }}' @@ -362,6 +363,11 @@ jobs: type: 'CD' deploymentEnvironmentsJson: ${{ needs.Initialization.outputs.deploymentEnvironmentsJson }} + - name: Deployment skipped + if: hashFiles('.artifacts/**/*.app') == '' + run: | + Write-Host "::Notice::Deployment to environment ${{ matrix.environment }} was skipped because no app artifacts were found" + - name: Deploy to Power Platform if: env.type == 'PTE' && env.powerPlatformSolutionFolder != '' uses: microsoft/AL-Go-Actions/DeployPowerPlatform@main @@ -387,7 +393,7 @@ jobs: uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Download artifacts - uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0 + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: path: '.artifacts' @@ -405,6 +411,7 @@ jobs: getSecrets: '${{ matrix.deliveryTarget }}Context' - name: Deliver + if: hashFiles('.artifacts/**/*.app') != '' uses: microsoft/AL-Go-Actions/Deliver@main env: Secrets: '${{ steps.ReadSecrets.outputs.Secrets }}' @@ -415,6 +422,11 @@ jobs: deliveryTarget: ${{ matrix.deliveryTarget }} artifacts: '.artifacts' + - name: Delivery skipped + if: hashFiles('.artifacts/**/*.app') == '' + run: | + Write-Host "::Notice::Delivery to ${{ matrix.deliveryTarget }} was skipped because no app artifacts were found" + PostProcess: needs: [ Initialization, Build, BuildPP, Deploy, Deliver, DeployALDoc, CodeAnalysisUpload ] if: (!cancelled()) diff --git a/Templates/Per Tenant Extension/.github/workflows/CreateRelease.yaml b/Templates/Per Tenant Extension/.github/workflows/CreateRelease.yaml index 83f18ee816..6a003d30ab 100644 --- a/Templates/Per Tenant Extension/.github/workflows/CreateRelease.yaml +++ b/Templates/Per Tenant Extension/.github/workflows/CreateRelease.yaml @@ -149,7 +149,7 @@ jobs: target_commitish: ${{ steps.determineArtifactsForRelease.outputs.commitish }} - name: Create release - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0 + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 id: createrelease env: bodyMD: ${{ steps.createreleasenotes.outputs.releaseNotes }} @@ -219,7 +219,7 @@ jobs: Invoke-WebRequest -UseBasicParsing -Headers $headers -Uri $ENV:MATRIX_URL -OutFile "$($ENV:MATRIX_NAME).zip" - name: Upload release artifacts - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0 + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: releaseId: ${{ needs.createrelease.outputs.releaseId }} MATRIX_NAME: ${{ matrix.name }} diff --git a/Templates/Per Tenant Extension/.github/workflows/DeployReferenceDocumentation.yaml b/Templates/Per Tenant Extension/.github/workflows/DeployReferenceDocumentation.yaml index 3968fbb10c..dfbbe56ffd 100644 --- a/Templates/Per Tenant Extension/.github/workflows/DeployReferenceDocumentation.yaml +++ b/Templates/Per Tenant Extension/.github/workflows/DeployReferenceDocumentation.yaml @@ -57,7 +57,7 @@ jobs: - name: Setup Pages if: steps.DetermineDeploymentEnvironments.outputs.deployALDocArtifact == 1 - uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5.0.0 + uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6.0.0 - name: Build Reference Documentation uses: microsoft/AL-Go-Actions/BuildReferenceDocumentation@main @@ -67,14 +67,14 @@ jobs: artifactUrl: ${{ env.artifact }} - name: Upload pages artifact - uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4.0.0 + uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0 with: path: ".aldoc/_site/" - name: Deploy to GitHub Pages if: steps.DetermineDeploymentEnvironments.outputs.deployALDocArtifact == 1 id: deployment - uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4.0.5 + uses: actions/deploy-pages@cd2ce8fcbc39b97be8ca5fce6e763baed58fa128 # v5.0.0 - name: Finalize the workflow if: always() diff --git a/Templates/Per Tenant Extension/.github/workflows/PullRequestHandler.yaml b/Templates/Per Tenant Extension/.github/workflows/PullRequestHandler.yaml index 4550c61206..06b2638c24 100644 --- a/Templates/Per Tenant Extension/.github/workflows/PullRequestHandler.yaml +++ b/Templates/Per Tenant Extension/.github/workflows/PullRequestHandler.yaml @@ -129,7 +129,7 @@ jobs: ref: ${{ format('refs/pull/{0}/head', github.event.pull_request.number) }} - name: Download artifacts - ErrorLogs - uses: actions/download-artifact@70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3 # v8.0.0 + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 if: (success() || failure()) with: pattern: '*-*ErrorLogs-*' diff --git a/Templates/Per Tenant Extension/.github/workflows/UpdateGitHubGoSystemFiles.yaml b/Templates/Per Tenant Extension/.github/workflows/UpdateGitHubGoSystemFiles.yaml index 204fa8810e..3f5504b7f0 100644 --- a/Templates/Per Tenant Extension/.github/workflows/UpdateGitHubGoSystemFiles.yaml +++ b/Templates/Per Tenant Extension/.github/workflows/UpdateGitHubGoSystemFiles.yaml @@ -21,6 +21,10 @@ on: default: '' workflow_call: inputs: + caller: + description: Name of the calling workflow (use github.workflow as value when calling) + type: string + required: true templateUrl: description: Template Repository URL (current is {TEMPLATEURL}) type: string @@ -50,6 +54,7 @@ defaults: shell: powershell env: + WorkflowEventName: ${{ inputs.caller && 'workflow_call' || github.event_name }} ALGoOrgSettings: ${{ vars.ALGoOrgSettings }} ALGoRepoSettings: ${{ vars.ALGoRepoSettings }} @@ -76,12 +81,13 @@ jobs: uses: microsoft/AL-Go-Actions/GetWorkflowMultiRunBranches@main with: shell: powershell - includeBranches: ${{ github.event.inputs.includeBranches }} + workflowEventName: ${{ env.WorkflowEventName }} + includeBranches: ${{ inputs.includeBranches }} - name: Determine Template URL id: DetermineTemplateUrl env: - TemplateUrlAsInput: '${{ github.event.inputs.templateUrl }}' + TemplateUrlAsInput: '${{ inputs.templateUrl }}' run: | $templateUrl = $env:templateUrl # Available from ReadSettings step if ($ENV:TemplateUrlAsInput) { @@ -133,12 +139,12 @@ jobs: - name: Calculate Commit Options env: - directCommit: '${{ github.event.inputs.directCommit }}' - downloadLatest: '${{ github.event.inputs.downloadLatest }}' + directCommit: '${{ inputs.directCommit }}' + downloadLatest: '${{ inputs.downloadLatest }}' run: | $errorActionPreference = "Stop"; $ProgressPreference = "SilentlyContinue"; Set-StrictMode -Version 2.0 - if('${{ github.event_name }}' -in 'workflow_dispatch', 'workflow_call') { - Write-Host "Using inputs from ${{ github.event_name }} event" + if($env:WorkflowEventName -in 'workflow_dispatch', 'workflow_call') { + Write-Host "Using inputs from $($env:WorkflowEventName) event" $directCommit = $env:directCommit $downloadLatest = $env:downloadLatest } diff --git a/Templates/Per Tenant Extension/.github/workflows/_BuildALGoProject.yaml b/Templates/Per Tenant Extension/.github/workflows/_BuildALGoProject.yaml index 79f563753d..b295574f7d 100644 --- a/Templates/Per Tenant Extension/.github/workflows/_BuildALGoProject.yaml +++ b/Templates/Per Tenant Extension/.github/workflows/_BuildALGoProject.yaml @@ -109,7 +109,7 @@ jobs: shell: ${{ inputs.shell }} project: ${{ inputs.project }} buildMode: ${{ inputs.buildMode }} - get: useCompilerFolder,keyVaultCodesignCertificateName,doNotSignApps,doNotRunTests,doNotRunBcptTests,doNotRunpageScriptingTests,artifact,generateDependencyArtifact,trustedSigning,useGitSubmodules,trackALAlertsInGitHub + get: useCompilerFolder,workspaceCompilation,keyVaultCodesignCertificateName,doNotSignApps,doNotRunTests,doNotRunBcptTests,doNotRunpageScriptingTests,artifact,generateDependencyArtifact,trustedSigning,useGitSubmodules,trackALAlertsInGitHub - name: Determine whether to build project id: DetermineBuildProject @@ -148,7 +148,7 @@ jobs: - name: Cache Business Central Artifacts if: steps.DetermineBuildProject.outputs.BuildIt == 'True' && env.useCompilerFolder == 'True' && inputs.useArtifactCache && env.artifactCacheKey - uses: actions/cache@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3 + uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 with: path: ${{ runner.temp }}/.artifactcache key: ${{ env.artifactCacheKey }} @@ -166,13 +166,40 @@ jobs: projectDependenciesJson: ${{ inputs.projectDependenciesJson }} baselineWorkflowRunId: ${{ inputs.baselineWorkflowRunId }} + - name: Compile Apps + id: compile + uses: microsoft/AL-Go-Actions/CompileApps@main + if: steps.DetermineBuildProject.outputs.BuildIt == 'True' && fromJson(env.workspaceCompilation).enabled == true + env: + Secrets: '${{ steps.ReadSecrets.outputs.Secrets }}' + BuildMode: ${{ inputs.buildMode }} + with: + shell: ${{ inputs.shell }} + artifact: ${{ env.artifact }} + project: ${{ inputs.project }} + buildMode: ${{ inputs.buildMode }} + dependencyAppsJson: ${{ steps.DownloadProjectDependencies.outputs.DownloadedApps }} + dependencyTestAppsJson: ${{ steps.DownloadProjectDependencies.outputs.DownloadedTestApps }} + baselineWorkflowRunId: ${{ inputs.baselineWorkflowRunId }} + baselineWorkflowSHA: ${{ inputs.baselineWorkflowSHA }} + + - name: Save needs context to file + shell: pwsh + run: | + $nc = @' + ${{ inputs.needsContext }} + '@ + $needsContextPath = Join-Path $ENV:RUNNER_TEMP 'needsContext.json' + [System.IO.File]::WriteAllText($needsContextPath, $nc.Trim()) + Add-Content -Encoding UTF8 -Path $env:GITHUB_ENV -Value "NeedsContext=$needsContextPath" + - name: Build uses: microsoft/AL-Go-Actions/RunPipeline@main if: steps.DetermineBuildProject.outputs.BuildIt == 'True' env: Secrets: '${{ steps.ReadSecrets.outputs.Secrets }}' BuildMode: ${{ inputs.buildMode }} - NeedsContext: ${{ inputs.needsContext }} + NeedsContext: ${{ env.NeedsContext }} with: shell: ${{ inputs.shell }} artifact: ${{ env.artifact }} @@ -203,7 +230,7 @@ jobs: suffix: ${{ inputs.artifactsNameSuffix }} - name: Publish artifacts - apps - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: inputs.artifactsRetentionDays >= 0 && (hashFiles(format('{0}/.buildartifacts/Apps/*',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.AppsArtifactsName }} @@ -212,7 +239,7 @@ jobs: retention-days: ${{ inputs.artifactsRetentionDays }} - name: Publish artifacts - dependencies - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: inputs.artifactsRetentionDays >= 0 && env.generateDependencyArtifact == 'True' && (hashFiles(format('{0}/.buildartifacts/Dependencies/*',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.DependenciesArtifactsName }} @@ -221,7 +248,7 @@ jobs: retention-days: ${{ inputs.artifactsRetentionDays }} - name: Publish artifacts - test apps - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: inputs.artifactsRetentionDays >= 0 && (hashFiles(format('{0}/.buildartifacts/TestApps/*',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.TestAppsArtifactsName }} @@ -230,7 +257,7 @@ jobs: retention-days: ${{ inputs.artifactsRetentionDays }} - name: Publish artifacts - build output - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: (success() || failure()) && (hashFiles(format('{0}/BuildOutput.txt',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.BuildOutputArtifactsName }} @@ -238,7 +265,7 @@ jobs: if-no-files-found: ignore - name: Publish artifacts - container event log - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: (failure()) && (hashFiles(format('{0}/ContainerEventLog.evtx',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.ContainerEventLogArtifactsName }} @@ -246,7 +273,7 @@ jobs: if-no-files-found: ignore - name: Publish artifacts - test results - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: (success() || failure()) && (hashFiles(format('{0}/.buildartifacts/TestResults.xml',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.TestResultsArtifactsName }} @@ -254,7 +281,7 @@ jobs: if-no-files-found: ignore - name: Publish artifacts - bcpt test results - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: (success() || failure()) && (hashFiles(format('{0}/.buildartifacts/bcptTestResults.json',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.BcptTestResultsArtifactsName }} @@ -262,7 +289,7 @@ jobs: if-no-files-found: ignore - name: Publish artifacts - page scripting test results - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: (success() || failure()) && (hashFiles(format('{0}/.buildartifacts/PageScriptingTestResults.xml',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.PageScriptingTestResultsArtifactsName }} @@ -270,7 +297,7 @@ jobs: if-no-files-found: ignore - name: Publish artifacts - page scripting test result details - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: (success() || failure()) && (hashFiles(format('{0}/.buildartifacts/PageScriptingTestResultDetails/*',inputs.project)) != '') with: name: ${{ steps.calculateArtifactsNames.outputs.PageScriptingTestResultDetailsArtifactsName }} @@ -278,7 +305,7 @@ jobs: if-no-files-found: ignore - name: Publish artifacts - ErrorLogs - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: (success() || failure()) && inputs.artifactsRetentionDays >= 0 && (hashFiles(format('{0}/.buildartifacts/ErrorLogs/*',inputs.project)) != '') && env.trackALAlertsInGitHub == 'True' with: name: ${{ steps.calculateArtifactsNames.outputs.ErrorLogsArtifactsName }} diff --git a/Templates/Per Tenant Extension/.github/workflows/_BuildPowerPlatformSolution.yaml b/Templates/Per Tenant Extension/.github/workflows/_BuildPowerPlatformSolution.yaml index 699884a31e..7115092b3b 100644 --- a/Templates/Per Tenant Extension/.github/workflows/_BuildPowerPlatformSolution.yaml +++ b/Templates/Per Tenant Extension/.github/workflows/_BuildPowerPlatformSolution.yaml @@ -92,7 +92,7 @@ jobs: suffix: ${{ inputs.artifactsNameSuffix }} - name: Publish artifacts - Power Platform Solution - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: inputs.publishArtifacts with: name: ${{ steps.calculateArtifactsNames.outputs.PowerPlatformSolutionArtifactsName }} diff --git a/Tests/AL-Go-Helper.Test.ps1 b/Tests/AL-Go-Helper.Test.ps1 index 18719d85de..d261545e01 100644 --- a/Tests/AL-Go-Helper.Test.ps1 +++ b/Tests/AL-Go-Helper.Test.ps1 @@ -113,4 +113,92 @@ "src$([System.IO.Path]::DirectorySeparatorChar)app4" ) } + + Describe 'Get-VersionNumber' { + # All tests share the same baseline settings to verify each strategy picks the correct source + BeforeEach { + $script:baseSettings = @{ + appBuild = 42 + appRevision = 7 + artifact = "https://bcartifacts.azureedge.net/sandbox/24.5.26928.27583/us" + repoVersion = "3.1.200" + } + } + + It 'Default versioning strategy returns settings appBuild and appRevision' { + $script:baseSettings.versioningStrategy = 0 + $result = Get-VersionNumber -Settings $script:baseSettings + $result.MajorMinorVersion | Should -Be "" + $result.BuildNumber | Should -Be 42 + $result.RevisionNumber | Should -Be 7 + } + + It 'Strategy -1 extracts version from artifact URL' { + $script:baseSettings.versioningStrategy = -1 + $result = Get-VersionNumber -Settings $script:baseSettings + $result.MajorMinorVersion | Should -Be "24.5" + $result.BuildNumber | Should -Be 26928 + $result.RevisionNumber | Should -Be 27583 + } + + It 'Strategy 16 uses repoVersion for major.minor' { + $script:baseSettings.versioningStrategy = 16 + $result = Get-VersionNumber -Settings $script:baseSettings + $result.MajorMinorVersion | Should -Be "3.1" + $result.BuildNumber | Should -Be 42 + $result.RevisionNumber | Should -Be 7 + } + + It 'Strategy 19 (16+3) gets build number from repoVersion' { + $script:baseSettings.versioningStrategy = 19 + $result = Get-VersionNumber -Settings $script:baseSettings + $result.MajorMinorVersion | Should -Be "3.1" + $result.BuildNumber | Should -Be 200 + $result.RevisionNumber | Should -Be 7 + } + + It 'Strategy 19 with two-digit repoVersion defaults build to 0 with warning' { + $script:baseSettings.versioningStrategy = 19 + $script:baseSettings.repoVersion = "2.4" + $result = Get-VersionNumber -Settings $script:baseSettings -WarningVariable warnings -WarningAction SilentlyContinue + $result.MajorMinorVersion | Should -Be "2.4" + $result.BuildNumber | Should -Be 0 + } + + It 'Strategy 17 (16+1) uses repoVersion for major.minor but does not override appBuild' { + $script:baseSettings.versioningStrategy = 17 + $result = Get-VersionNumber -Settings $script:baseSettings + $result.MajorMinorVersion | Should -Be "3.1" + $result.BuildNumber | Should -Be 42 + $result.RevisionNumber | Should -Be 7 + } + + It 'Strategy 3 without bit 16 behaves like default' { + $script:baseSettings.versioningStrategy = 3 + $result = Get-VersionNumber -Settings $script:baseSettings + $result.MajorMinorVersion | Should -Be "" + $result.BuildNumber | Should -Be 42 + $result.RevisionNumber | Should -Be 7 + } + + It 'Strategy 2 passes through date-based appBuild and appRevision' { + $script:baseSettings.versioningStrategy = 2 + $script:baseSettings.appBuild = 20260313 # Simulate date-based build number + $script:baseSettings.appRevision = 141450 # Simulate time-based revision number + $result = Get-VersionNumber -Settings $script:baseSettings + $result.MajorMinorVersion | Should -Be "" + $result.BuildNumber | Should -Be 20260313 # Build number should be passed through unchanged + $result.RevisionNumber | Should -Be 141450 # Revision number should be passed through unchanged + } + + It 'Strategy 15 passes through max build value' { + $script:baseSettings.versioningStrategy = 15 + $script:baseSettings.appBuild = [Int32]::MaxValue # Simulate max build number + $script:baseSettings.appRevision = 100 # Simulate some revision number + $result = Get-VersionNumber -Settings $script:baseSettings + $result.MajorMinorVersion | Should -Be "" + $result.BuildNumber | Should -Be ([Int32]::MaxValue) # Build number should be passed through unchanged + $result.RevisionNumber | Should -Be 100 # Revision number should be passed through unchanged + } + } } diff --git a/Tests/CompileFromWorkspace.Test.ps1 b/Tests/CompileFromWorkspace.Test.ps1 new file mode 100644 index 0000000000..aabce0a64d --- /dev/null +++ b/Tests/CompileFromWorkspace.Test.ps1 @@ -0,0 +1,872 @@ +[Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSReviewUnusedParameter', '', Justification = 'Mock/callback parameters must match function signatures')] +param() + +$errorActionPreference = "Stop"; $ProgressPreference = "SilentlyContinue"; Set-StrictMode -Version 2.0 + +. (Join-Path -Path $PSScriptRoot -ChildPath "../Actions/AL-Go-Helper.ps1" -Resolve) +Import-Module (Join-Path $PSScriptRoot '../Actions/.Modules/CompileFromWorkspace.psm1' -Resolve) -DisableNameChecking -Force + +Describe 'CompileFromWorkspace.psm1 Tests' { + BeforeAll { + <# + .SYNOPSIS + Creates a realistic AL-Go project structure for testing. + .DESCRIPTION + Sets up a complete AL-Go project with .AL-Go/settings.json, app folders with app.json files, + and test folders following the standard AL-Go conventions. + + The created folder structure mirrors a real AL-Go repository: + + BaseFolder/ + ├── .AL-Go/ + │ └── settings.json # Contains country, appFolders, testFolders, bcptTestFolders + ├── MyApp/ # App folder (one per entry in AppFolders) + │ └── app.json # App manifest with id, name, publisher, version, dependencies, idRanges + └── MyApp.Test/ # Test folder (one per entry in TestFolders) + └── app.json # Test app manifest + + .PARAMETER BaseFolder + The root folder where the project structure will be created. + .PARAMETER ProjectName + Optional project name for multi-project repos. If empty, creates single-project structure at BaseFolder. + .PARAMETER AppFolders + Array of hashtables defining app folders. Each hashtable can have: + - Name (required): Folder name for the app + - Id: App GUID (auto-generated if not specified) + - Publisher: Publisher name (defaults to "Test Publisher") + - Version: App version (defaults to "1.0.0.0") + .PARAMETER TestFolders + Array of hashtables defining test folders. Same structure as AppFolders. + .PARAMETER Settings + Additional settings to merge into .AL-Go/settings.json. + .OUTPUTS + Returns the project path (BaseFolder or BaseFolder/ProjectName). + .EXAMPLE + # Create a simple single-app project + $projectPath = New-ALGoTestProject -BaseFolder $TestDrive -AppFolders @( + @{ Name = "MyApp"; Id = "11111111-1111-1111-1111-111111111111"; Version = "1.0.0.0" } + ) + #> + function script:New-ALGoTestProject { + param( + [Parameter(Mandatory = $true)] + [string] $BaseFolder, + [Parameter(Mandatory = $false)] + [string] $ProjectName = "", + [Parameter(Mandatory = $false)] + [array] $AppFolders = @(), + [Parameter(Mandatory = $false)] + [array] $TestFolders = @(), + [Parameter(Mandatory = $false)] + [hashtable] $Settings = @{} + ) + + $projectPath = if ($ProjectName) { Join-Path $BaseFolder $ProjectName } else { $BaseFolder } + $alGoFolder = Join-Path $projectPath ".AL-Go" + + New-Item -Path $alGoFolder -ItemType Directory -Force | Out-Null + + $defaultSettings = @{ + country = "us" + appFolders = @($AppFolders | ForEach-Object { $_['Name'] }) + testFolders = @($TestFolders | ForEach-Object { $_['Name'] }) + bcptTestFolders = @() + } + + foreach ($key in $Settings.Keys) { + $defaultSettings[$key] = $Settings[$key] + } + + $defaultSettings | ConvertTo-Json -Depth 10 | Set-Content -Path (Join-Path $alGoFolder "settings.json") -Encoding UTF8 + + foreach ($app in $AppFolders) { + $appFolder = Join-Path $projectPath $app['Name'] + New-Item -Path $appFolder -ItemType Directory -Force | Out-Null + + $appJson = @{ + id = if ($app['Id']) { $app['Id'] } else { [guid]::NewGuid().ToString() } + name = $app['Name'] + publisher = if ($app['Publisher']) { $app['Publisher'] } else { "Test Publisher" } + version = if ($app['Version']) { $app['Version'] } else { "1.0.0.0" } + dependencies = @() + platform = "1.0.0.0" + application = "22.0.0.0" + idRanges = @(@{ from = 50000; to = 50100 }) + } + + $appJson | ConvertTo-Json -Depth 10 | Set-Content -Path (Join-Path $appFolder "app.json") -Encoding UTF8 + } + + foreach ($testApp in $TestFolders) { + $testFolder = Join-Path $projectPath $testApp['Name'] + New-Item -Path $testFolder -ItemType Directory -Force | Out-Null + + $testAppJson = @{ + id = if ($testApp['Id']) { $testApp['Id'] } else { [guid]::NewGuid().ToString() } + name = $testApp['Name'] + publisher = if ($testApp['Publisher']) { $testApp['Publisher'] } else { "Test Publisher" } + version = if ($testApp['Version']) { $testApp['Version'] } else { "1.0.0.0" } + dependencies = @() + platform = "1.0.0.0" + application = "22.0.0.0" + idRanges = @(@{ from = 60000; to = 60100 }) + } + + $testAppJson | ConvertTo-Json -Depth 10 | Set-Content -Path (Join-Path $testFolder "app.json") -Encoding UTF8 + } + + return $projectPath + } + } + + Describe 'Get-CodeAnalyzers' { + It 'Returns empty array when no analyzers are enabled' { + $settings = @{ + enableCodeCop = $false + enableAppSourceCop = $false + enablePerTenantExtensionCop = $false + enableUICop = $false + } + + $result = Get-CodeAnalyzers -Settings $settings + + @($result).Count | Should -Be 0 + } + + It 'Returns CodeCop when enableCodeCop is true' { + $settings = @{ + enableCodeCop = $true + enableAppSourceCop = $false + enablePerTenantExtensionCop = $false + enableUICop = $false + } + + $result = Get-CodeAnalyzers -Settings $settings + + $result | Should -Contain 'CodeCop' + @($result).Count | Should -Be 1 + } + + It 'Returns AppSourceCop when enableAppSourceCop is true' { + $settings = @{ + enableCodeCop = $false + enableAppSourceCop = $true + enablePerTenantExtensionCop = $false + enableUICop = $false + } + + $result = Get-CodeAnalyzers -Settings $settings + + $result | Should -Contain 'AppSourceCop' + @($result).Count | Should -Be 1 + } + + It 'Returns PTECop when enablePerTenantExtensionCop is true' { + $settings = @{ + enableCodeCop = $false + enableAppSourceCop = $false + enablePerTenantExtensionCop = $true + enableUICop = $false + } + + $result = Get-CodeAnalyzers -Settings $settings + + $result | Should -Contain 'PTECop' + @($result).Count | Should -Be 1 + } + + It 'Returns UICop when enableUICop is true' { + $settings = @{ + enableCodeCop = $false + enableAppSourceCop = $false + enablePerTenantExtensionCop = $false + enableUICop = $true + } + + $result = Get-CodeAnalyzers -Settings $settings + + $result | Should -Contain 'UICop' + @($result).Count | Should -Be 1 + } + + It 'Returns all analyzers when all are enabled' { + $settings = @{ + enableCodeCop = $true + enableAppSourceCop = $true + enablePerTenantExtensionCop = $true + enableUICop = $true + } + + $result = Get-CodeAnalyzers -Settings $settings + + $result | Should -Contain 'CodeCop' + $result | Should -Contain 'AppSourceCop' + $result | Should -Contain 'PTECop' + $result | Should -Contain 'UICop' + @($result).Count | Should -Be 4 + } + } + + Describe 'Get-BuildMetadata' { + It 'Returns GitHub Actions metadata when running in GitHub Actions' { + $env:GITHUB_ACTIONS = 'true' + $env:GITHUB_SERVER_URL = 'https://github.com' + $env:GITHUB_REPOSITORY = 'owner/repo' + $env:GITHUB_SHA = 'def456' + $env:GITHUB_RUN_ID = '12345' + + $result = Get-BuildMetadata + + $result.SourceRepositoryUrl | Should -Be 'https://github.com/owner/repo' + $result.SourceCommit | Should -Be 'def456' + $result.BuildBy | Should -Be 'AL-Go for GitHub' + $result.BuildUrl | Should -Be 'https://github.com/owner/repo/actions/runs/12345' + } + + AfterEach { + $env:GITHUB_ACTIONS = $null + $env:GITHUB_SERVER_URL = $null + $env:GITHUB_REPOSITORY = $null + $env:GITHUB_SHA = $null + $env:GITHUB_RUN_ID = $null + } + } + + Describe 'Get-ScriptOverrides' { + BeforeAll { + # Get-ScriptOverrides is defined in AL-Go-Helper.ps1 (dot-sourced at file scope). + # Re-dot-source here so it's available inside Pester's It blocks. + . (Join-Path -Path $PSScriptRoot -ChildPath "../Actions/AL-Go-Helper.ps1" -Resolve) + # Trace-Information is from TelemetryHelper.psm1 which isn't loaded in tests + function Trace-Information { param([string]$Message) } + } + + It 'Returns empty hashtable when no override scripts exist' { + $projectPath = New-ALGoTestProject -BaseFolder (Join-Path $TestDrive 'no-overrides') -AppFolders @( + @{ Name = "MyApp"; Id = "11111111-1111-1111-1111-111111111111" } + ) + $alGoFolder = Join-Path $projectPath ".AL-Go" + + $result = Get-ScriptOverrides -ALGoFolderName $alGoFolder -OverrideScriptNames @("PreCompileApp", "PostCompileApp") + + $result.Keys | Should -HaveCount 0 + } + + It 'Returns PreCompileApp override when script exists in .AL-Go folder' { + $projectPath = New-ALGoTestProject -BaseFolder (Join-Path $TestDrive 'precompile') -AppFolders @( + @{ Name = "MyApp"; Id = "22222222-2222-2222-2222-222222222222" } + ) + $alGoFolder = Join-Path $projectPath ".AL-Go" + Set-Content -Path (Join-Path $alGoFolder 'PreCompileApp.ps1') -Value @' +Param( + [ValidateSet('app','testApp')] + [string] $appType, + [ref] $compilationParams +) +Write-Host "Pre-compile for $appType" +'@ + + $result = Get-ScriptOverrides -ALGoFolderName $alGoFolder -OverrideScriptNames @("PreCompileApp", "PostCompileApp") + + $result.PreCompileApp | Should -Not -BeNullOrEmpty + $result.Keys | Should -Not -Contain 'PostCompileApp' + } + + It 'Returns PostCompileApp override when script exists in .AL-Go folder' { + $projectPath = New-ALGoTestProject -BaseFolder (Join-Path $TestDrive 'postcompile') -AppFolders @( + @{ Name = "MyApp"; Id = "33333333-3333-3333-3333-333333333333" } + ) + $alGoFolder = Join-Path $projectPath ".AL-Go" + Set-Content -Path (Join-Path $alGoFolder 'PostCompileApp.ps1') -Value @' +Param( + [string[]] $appFiles, + [ValidateSet('app','testApp')] + [string] $appType, + [hashtable] $compilationParams +) +Write-Host "Post-compile: $($appFiles.Count) apps" +'@ + + $result = Get-ScriptOverrides -ALGoFolderName $alGoFolder -OverrideScriptNames @("PreCompileApp", "PostCompileApp") + + $result.Keys | Should -Not -Contain 'PreCompileApp' + $result.PostCompileApp | Should -Not -BeNullOrEmpty + } + + It 'Returns both overrides when both scripts exist in .AL-Go folder' { + $projectPath = New-ALGoTestProject -BaseFolder (Join-Path $TestDrive 'both-overrides') -AppFolders @( + @{ Name = "MyApp"; Id = "44444444-4444-4444-4444-444444444444" } + ) -TestFolders @( + @{ Name = "MyApp.Test"; Id = "55555555-5555-5555-5555-555555555555" } + ) + $alGoFolder = Join-Path $projectPath ".AL-Go" + + Set-Content -Path (Join-Path $alGoFolder 'PreCompileApp.ps1') -Value @' +Param( + [ValidateSet('app','testApp')] + [string] $appType, + [ref] $compilationParams +) +Write-Host "Pre-compile for $appType" +'@ + Set-Content -Path (Join-Path $alGoFolder 'PostCompileApp.ps1') -Value @' +Param( + [string[]] $appFiles, + [ValidateSet('app','testApp')] + [string] $appType, + [hashtable] $compilationParams +) +Write-Host "Post-compile: $($appFiles.Count) apps" +'@ + + $result = Get-ScriptOverrides -ALGoFolderName $alGoFolder -OverrideScriptNames @("PreCompileApp", "PostCompileApp") + + $result.Keys | Should -Contain 'PreCompileApp' + $result.Keys | Should -Contain 'PostCompileApp' + } + } + + Describe 'Update-AppJsonProperties' { + It 'Updates version with MajorMinorVersion, BuildNumber and RevisionNumber' { + $projectPath = New-ALGoTestProject -BaseFolder $TestDrive -AppFolders @( + @{ Name = "MyApp"; Id = "11111111-1111-1111-1111-111111111111"; Version = "1.0.0.0" } + ) + $appFolder = Join-Path $projectPath "MyApp" + + Update-AppJsonProperties -Folders @($appFolder) -MajorMinorVersion "2.5" -BuildNumber 100 -RevisionNumber 50 + + $updatedAppJson = Get-Content -Path (Join-Path $appFolder 'app.json') | ConvertFrom-Json + $updatedAppJson.version | Should -Be '2.5.100.50' + } + + It 'Updates only BuildNumber and RevisionNumber when MajorMinorVersion is not provided' { + $projectPath = New-ALGoTestProject -BaseFolder $TestDrive -AppFolders @( + @{ Name = "MyApp2"; Id = "22222222-2222-2222-2222-222222222222"; Version = "1.0.0.0" } + ) + $appFolder = Join-Path $projectPath "MyApp2" + + Update-AppJsonProperties -Folders @($appFolder) -BuildNumber 200 -RevisionNumber 75 + + $updatedAppJson = Get-Content -Path (Join-Path $appFolder 'app.json') | ConvertFrom-Json + $updatedAppJson.version | Should -Be '1.0.200.75' + } + + It 'Updates only RevisionNumber when BuildNumber is 0' { + $projectPath = New-ALGoTestProject -BaseFolder $TestDrive -AppFolders @( + @{ Name = "MyApp3"; Id = "33333333-3333-3333-3333-333333333333"; Version = "1.0.0.0" } + ) + $appFolder = Join-Path $projectPath "MyApp3" + + Update-AppJsonProperties -Folders @($appFolder) -RevisionNumber 99 + + $updatedAppJson = Get-Content -Path (Join-Path $appFolder 'app.json') | ConvertFrom-Json + $updatedAppJson.version | Should -Be '1.0.0.99' + } + + It 'Updates multiple app folders in a project' { + $projectPath = New-ALGoTestProject -BaseFolder $TestDrive -AppFolders @( + @{ Name = "App1"; Id = "44444444-4444-4444-4444-444444444444"; Version = "1.0.0.0" } + @{ Name = "App2"; Id = "55555555-5555-5555-5555-555555555555"; Version = "2.0.0.0" } + ) + $appFolders = @( + (Join-Path $projectPath "App1"), + (Join-Path $projectPath "App2") + ) + + Update-AppJsonProperties -Folders $appFolders -MajorMinorVersion "3.0" -BuildNumber 50 -RevisionNumber 10 + + $app1Json = Get-Content -Path (Join-Path $projectPath "App1\app.json") | ConvertFrom-Json + $app2Json = Get-Content -Path (Join-Path $projectPath "App2\app.json") | ConvertFrom-Json + + $app1Json.version | Should -Be '3.0.50.10' + $app2Json.version | Should -Be '3.0.50.10' + } + + It 'Updates app and test folders together' { + $projectPath = New-ALGoTestProject -BaseFolder $TestDrive ` + -AppFolders @( + @{ Name = "MainApp"; Id = "66666666-6666-6666-6666-666666666666"; Version = "1.0.0.0" } + ) ` + -TestFolders @( + @{ Name = "MainApp.Test"; Id = "77777777-7777-7777-7777-777777777777"; Version = "1.0.0.0" } + ) + + $allFolders = @( + (Join-Path $projectPath "MainApp"), + (Join-Path $projectPath "MainApp.Test") + ) + + Update-AppJsonProperties -Folders $allFolders -MajorMinorVersion "4.0" -BuildNumber 123 -RevisionNumber 456 + + $appJson = Get-Content -Path (Join-Path $projectPath "MainApp\app.json") | ConvertFrom-Json + $testJson = Get-Content -Path (Join-Path $projectPath "MainApp.Test\app.json") | ConvertFrom-Json + + $appJson.version | Should -Be '4.0.123.456' + $testJson.version | Should -Be '4.0.123.456' + } + } + + Describe 'Build-AppsInWorkspace' { + BeforeAll { + # Create a mock compiler folder structure + $script:mockCompilerFolder = Join-Path $TestDrive 'compiler' + New-Item -Path $script:mockCompilerFolder -ItemType Directory -Force | Out-Null + + # Create a fake altool executable + $altoolPath = Join-Path $script:mockCompilerFolder 'altool.exe' + Set-Content -Path $altoolPath -Value "mock" + } + + It 'Uses default PackageCachePath when not specified' { + Mock Get-ALTool { return (Join-Path $TestDrive 'compiler\altool.exe') } -ModuleName CompileFromWorkspace + Mock New-WorkspaceFromFolders { } -ModuleName CompileFromWorkspace + Mock CompileAppsInWorkspace { + param($ALToolPath, $WorkspaceFile, $PackageCachePath, $OutFolder) + # Verify PackageCachePath defaults to compiler\symbols + $PackageCachePath | Should -BeLike '*compiler*symbols' + return @() + } -ModuleName CompileFromWorkspace + + $projectPath = New-ALGoTestProject -BaseFolder (Join-Path $TestDrive 'proj1') -AppFolders @( + @{ Name = "App1"; Id = "11111111-1111-1111-1111-111111111111" } + ) + + Build-AppsInWorkspace -Folders @((Join-Path $projectPath "App1")) -CompilerFolder $script:mockCompilerFolder + } + + It 'Uses specified PackageCachePath when provided' { + $customCachePath = Join-Path $TestDrive 'custom-cache' + New-Item -Path $customCachePath -ItemType Directory -Force | Out-Null + + Mock Get-ALTool { return (Join-Path $TestDrive 'compiler\altool.exe') } -ModuleName CompileFromWorkspace + Mock New-WorkspaceFromFolders { } -ModuleName CompileFromWorkspace + Mock CompileAppsInWorkspace { + param($ALToolPath, $WorkspaceFile, $PackageCachePath, $OutFolder) + $PackageCachePath | Should -Be $customCachePath + return @() + } -ModuleName CompileFromWorkspace -ParameterFilter { $PackageCachePath -eq $customCachePath } + + $projectPath = New-ALGoTestProject -BaseFolder (Join-Path $TestDrive 'proj2') -AppFolders @( + @{ Name = "App2"; Id = "22222222-2222-2222-2222-222222222222" } + ) + + Build-AppsInWorkspace -Folders @((Join-Path $projectPath "App2")) -CompilerFolder $script:mockCompilerFolder -PackageCachePath $customCachePath + } + + It 'Caps MaxCpuCount to available processor count' { + Mock Get-ALTool { return (Join-Path $TestDrive 'compiler\altool.exe') } -ModuleName CompileFromWorkspace + Mock New-WorkspaceFromFolders { } -ModuleName CompileFromWorkspace + Mock CompileAppsInWorkspace { + param($ALToolPath, $WorkspaceFile, $PackageCachePath, $OutFolder, $AssemblyProbingPaths, $Analyzers, $PreprocessorSymbols, $Features, $GenerateReportLayout, $Ruleset, $SourceRepositoryUrl, $SourceCommit, $BuildBy, $BuildUrl, $ReportSuppressedDiagnostics, $EnableExternalRulesets, $MaxCpuCount) + # MaxCpuCount should be capped to processor count + $MaxCpuCount | Should -BeLessOrEqual ([System.Environment]::ProcessorCount) + return @() + } -ModuleName CompileFromWorkspace + + $projectPath = New-ALGoTestProject -BaseFolder (Join-Path $TestDrive 'proj3') -AppFolders @( + @{ Name = "App3"; Id = "33333333-3333-3333-3333-333333333333" } + ) + + Build-AppsInWorkspace -Folders @((Join-Path $projectPath "App3")) -CompilerFolder $script:mockCompilerFolder -MaxCpuCount 9999 + } + + It 'Invokes PreCompileApp script before compilation' { + $script:preCompileInvoked = $false + + Mock Get-ALTool { return (Join-Path $TestDrive 'compiler\altool.exe') } -ModuleName CompileFromWorkspace + Mock New-WorkspaceFromFolders { } -ModuleName CompileFromWorkspace + Mock CompileAppsInWorkspace { return @() } -ModuleName CompileFromWorkspace + + $projectPath = New-ALGoTestProject -BaseFolder (Join-Path $TestDrive 'proj4') -AppFolders @( + @{ Name = "App4"; Id = "44444444-4444-4444-4444-444444444444" } + ) + + $preCompileScript = { + param($appType, $compilationParams) + $script:preCompileInvoked = $true + } + + Build-AppsInWorkspace -Folders @((Join-Path $projectPath "App4")) -CompilerFolder $script:mockCompilerFolder -PreCompileApp $preCompileScript -AppType 'app' + + $script:preCompileInvoked | Should -Be $true + } + + It 'Invokes PostCompileApp script after compilation' { + $script:postCompileInvoked = $false + + Mock Get-ALTool { return (Join-Path $TestDrive 'compiler\altool.exe') } -ModuleName CompileFromWorkspace + Mock New-WorkspaceFromFolders { } -ModuleName CompileFromWorkspace + Mock CompileAppsInWorkspace { return @('app1.app', 'app2.app') } -ModuleName CompileFromWorkspace + + $projectPath = New-ALGoTestProject -BaseFolder (Join-Path $TestDrive 'proj5') -AppFolders @( + @{ Name = "App5"; Id = "55555555-5555-5555-5555-555555555555" } + ) + + $postCompileScript = { + param($appFiles, $appType, $compilationParams) + $script:postCompileInvoked = $true + @($appFiles).Count | Should -Be 2 + } + + Build-AppsInWorkspace -Folders @((Join-Path $projectPath "App5")) -CompilerFolder $script:mockCompilerFolder -PostCompileApp $postCompileScript -AppType 'app' + + $script:postCompileInvoked | Should -Be $true + } + + It 'Returns compiled app files from CompileAppsInWorkspace' { + $expectedApps = @('MyApp_1.0.0.0.app', 'MyApp2_1.0.0.0.app') + + Mock Get-ALTool { return (Join-Path $TestDrive 'compiler\altool.exe') } -ModuleName CompileFromWorkspace + Mock New-WorkspaceFromFolders { } -ModuleName CompileFromWorkspace + Mock CompileAppsInWorkspace { return $expectedApps } -ModuleName CompileFromWorkspace + + $projectPath = New-ALGoTestProject -BaseFolder (Join-Path $TestDrive 'proj6') -AppFolders @( + @{ Name = "App6"; Id = "66666666-6666-6666-6666-666666666666" } + ) + + $result = Build-AppsInWorkspace -Folders @((Join-Path $projectPath "App6")) -CompilerFolder $script:mockCompilerFolder + + @($result).Count | Should -Be 2 + $result | Should -Contain 'MyApp_1.0.0.0.app' + $result | Should -Contain 'MyApp2_1.0.0.0.app' + } + + It 'Passes analyzers to CompileAppsInWorkspace' { + Mock Get-ALTool { return (Join-Path $TestDrive 'compiler\altool.exe') } -ModuleName CompileFromWorkspace + Mock New-WorkspaceFromFolders { } -ModuleName CompileFromWorkspace + Mock CompileAppsInWorkspace { + param($ALToolPath, $WorkspaceFile, $PackageCachePath, $OutFolder, $AssemblyProbingPaths, $Analyzers) + $Analyzers | Should -Contain 'CodeCop' + $Analyzers | Should -Contain 'UICop' + return @() + } -ModuleName CompileFromWorkspace + + $projectPath = New-ALGoTestProject -BaseFolder (Join-Path $TestDrive 'proj7') -AppFolders @( + @{ Name = "App7"; Id = "77777777-7777-7777-7777-777777777777" } + ) + + Build-AppsInWorkspace -Folders @((Join-Path $projectPath "App7")) -CompilerFolder $script:mockCompilerFolder -Analyzers @('CodeCop', 'UICop') + } + } + + Describe 'Copy-CompiledAppsToOutput' { + It 'Returns new files that appeared after compilation' { + InModuleScope CompileFromWorkspace { + $packageCache = Join-Path $TestDrive 'cache-copy1' + $outputFolder = Join-Path $TestDrive 'output-copy1' + New-Item -Path $packageCache -ItemType Directory -Force | Out-Null + + $filesBefore = @{} + + $appFile = Join-Path $packageCache 'NewApp_1.0.0.0.app' + Set-Content -Path $appFile -Value 'compiled' + + $result = @(Copy-CompiledAppsToOutput -PackageCachePath $packageCache -OutputFolder $outputFolder -FilesBeforeCompile $filesBefore) + + $result.Count | Should -Be 1 + $result[0] | Should -BeLike '*NewApp_1.0.0.0.app' + Test-Path (Join-Path $outputFolder 'NewApp_1.0.0.0.app') | Should -Be $true + } + } + + It 'Returns modified files with newer timestamps' { + InModuleScope CompileFromWorkspace { + $packageCache = Join-Path $TestDrive 'cache-copy2' + $outputFolder = Join-Path $TestDrive 'output-copy2' + New-Item -Path $packageCache -ItemType Directory -Force | Out-Null + + $appFile = Join-Path $packageCache 'Existing_1.0.0.0.app' + Set-Content -Path $appFile -Value 'old' + $oldTimestamp = (Get-Item $appFile).LastWriteTimeUtc + + $filesBefore = @{ $appFile = $oldTimestamp } + + Start-Sleep -Milliseconds 100 + Set-Content -Path $appFile -Value 'recompiled' + + $result = Copy-CompiledAppsToOutput -PackageCachePath $packageCache -OutputFolder $outputFolder -FilesBeforeCompile $filesBefore + + @($result).Count | Should -Be 1 + } + } + + It 'Skips unchanged files' { + InModuleScope CompileFromWorkspace { + $packageCache = Join-Path $TestDrive 'cache-copy3' + $outputFolder = Join-Path $TestDrive 'output-copy3' + New-Item -Path $packageCache -ItemType Directory -Force | Out-Null + + $appFile = Join-Path $packageCache 'Unchanged_1.0.0.0.app' + Set-Content -Path $appFile -Value 'content' + $timestamp = (Get-Item $appFile).LastWriteTimeUtc + + $filesBefore = @{ $appFile = $timestamp } + + $result = Copy-CompiledAppsToOutput -PackageCachePath $packageCache -OutputFolder $outputFolder -FilesBeforeCompile $filesBefore + + @($result).Count | Should -Be 0 + } + } + + It 'Skips copy when PackageCachePath equals OutputFolder' { + InModuleScope CompileFromWorkspace { + $sameFolder = Join-Path $TestDrive 'cache-copy4' + New-Item -Path $sameFolder -ItemType Directory -Force | Out-Null + + $appFile = Join-Path $sameFolder 'App_1.0.0.0.app' + Set-Content -Path $appFile -Value 'compiled' + + $result = Copy-CompiledAppsToOutput -PackageCachePath $sameFolder -OutputFolder $sameFolder -FilesBeforeCompile @{} + + @($result).Count | Should -Be 1 + Test-Path $appFile | Should -Be $true + } + } + + It 'Creates OutputFolder if it does not exist' { + InModuleScope CompileFromWorkspace { + $packageCache = Join-Path $TestDrive 'cache-copy5' + $outputFolder = Join-Path $TestDrive 'output-copy5-new' + New-Item -Path $packageCache -ItemType Directory -Force | Out-Null + + $appFile = Join-Path $packageCache 'App_1.0.0.0.app' + Set-Content -Path $appFile -Value 'compiled' + + Test-Path $outputFolder | Should -Be $false + + Copy-CompiledAppsToOutput -PackageCachePath $packageCache -OutputFolder $outputFolder -FilesBeforeCompile @{} + + Test-Path $outputFolder | Should -Be $true + } + } + } + + Describe 'New-BuildOutputFile' { + It 'Creates build output file from log files' { + $buildArtifactFolder = Join-Path $TestDrive 'artifacts-build1' + $logsFolder = Join-Path $buildArtifactFolder 'Logs' + New-Item -Path $logsFolder -ItemType Directory -Force | Out-Null + + # Create a mock log file + Set-Content -Path (Join-Path $logsFolder 'compile.log') -Value "[OUT] warning AL0001: Some warning`n[OUT] info AL0002: Some info" + + $outputPath = Join-Path $TestDrive 'BuildOutput1.txt' + + Mock Convert-AlcOutputToAzureDevOps { } -ModuleName CompileFromWorkspace + + $result = New-BuildOutputFile -BuildArtifactFolder $buildArtifactFolder -BuildOutputPath $outputPath + + $result | Should -Be $outputPath + Test-Path $outputPath | Should -Be $true + $content = Get-Content $outputPath + $content | Should -Contain 'warning AL0001: Some warning' + } + + It 'Strips [OUT] prefix from log lines' { + $buildArtifactFolder = Join-Path $TestDrive 'artifacts-build2' + $logsFolder = Join-Path $buildArtifactFolder 'Logs' + New-Item -Path $logsFolder -ItemType Directory -Force | Out-Null + + Set-Content -Path (Join-Path $logsFolder 'compile.log') -Value "[OUT] some output line" + + $outputPath = Join-Path $TestDrive 'BuildOutput2.txt' + + Mock Convert-AlcOutputToAzureDevOps { } -ModuleName CompileFromWorkspace + + New-BuildOutputFile -BuildArtifactFolder $buildArtifactFolder -BuildOutputPath $outputPath + + $content = Get-Content $outputPath + $content | Should -Contain 'some output line' + $content | Should -Not -Contain '[OUT] some output line' + } + + It 'Handles empty artifacts folder with no log files' { + $buildArtifactFolder = Join-Path $TestDrive 'artifacts-build3' + New-Item -Path $buildArtifactFolder -ItemType Directory -Force | Out-Null + + $outputPath = Join-Path $TestDrive 'BuildOutput3.txt' + + $result = New-BuildOutputFile -BuildArtifactFolder $buildArtifactFolder -BuildOutputPath $outputPath + + $result | Should -Be $outputPath + Test-Path $outputPath | Should -Be $true + } + } + + Describe 'Get-CustomAnalyzers' { + It 'Returns empty array when no custom code cops configured' { + $result = Get-CustomAnalyzers -Settings @{ CustomCodeCops = @() } -CompilerFolder $TestDrive + + @($result).Count | Should -Be 0 + } + + It 'Returns local paths as-is' { + $result = Get-CustomAnalyzers -Settings @{ CustomCodeCops = @('C:\analyzers\MyAnalyzer.dll') } -CompilerFolder $TestDrive + + @($result).Count | Should -Be 1 + $result | Should -Contain 'C:\analyzers\MyAnalyzer.dll' + } + + It 'Downloads URL-based analyzers to compiler folder' { + $compilerFolder = Join-Path $TestDrive 'compiler-analyzers' + $analyzersBin = Join-Path $compilerFolder 'compiler\extension\bin\Analyzers' + New-Item -Path $analyzersBin -ItemType Directory -Force | Out-Null + + Mock Invoke-WebRequest { + param($Uri, $OutFile) + Set-Content -Path $OutFile -Value 'mock-dll' + } -ModuleName CompileFromWorkspace + + $result = @(Get-CustomAnalyzers -Settings @{ CustomCodeCops = @('https://example.com/MyAnalyzer.dll') } -CompilerFolder $compilerFolder) + + $result.Count | Should -Be 1 + $result[0] | Should -BeLike '*MyAnalyzer.dll' + } + } + + Describe 'Get-AssemblyProbingPaths' { + It 'Includes Service and Mock Assemblies when dlls folder exists' { + $compilerFolder = Join-Path $TestDrive 'compiler-probing1' + $dllsPath = Join-Path $compilerFolder 'dlls' + New-Item -Path (Join-Path $dllsPath 'Service') -ItemType Directory -Force | Out-Null + New-Item -Path (Join-Path $dllsPath 'Mock Assemblies') -ItemType Directory -Force | Out-Null + New-Item -Path (Join-Path $dllsPath 'OpenXML') -ItemType Directory -Force | Out-Null + + Mock Get-DotnetRuntimeVersionInstalled { return $null } -ModuleName CompileFromWorkspace + + $result = Get-AssemblyProbingPaths -CompilerFolder $compilerFolder + + ($result -like '*Service') | Should -Not -BeNullOrEmpty + ($result -like '*Mock Assemblies') | Should -Not -BeNullOrEmpty + ($result -like '*OpenXML') | Should -Not -BeNullOrEmpty + } + + It 'Uses shared folder when it exists' { + $compilerFolder = Join-Path $TestDrive 'compiler-probing2' + $dllsPath = Join-Path $compilerFolder 'dlls' + $sharedPath = Join-Path $dllsPath 'shared' + New-Item -Path $sharedPath -ItemType Directory -Force | Out-Null + New-Item -Path (Join-Path $dllsPath 'OpenXML') -ItemType Directory -Force | Out-Null + New-Item -Path (Join-Path $dllsPath 'Service') -ItemType Directory -Force | Out-Null + New-Item -Path (Join-Path $dllsPath 'Mock Assemblies') -ItemType Directory -Force | Out-Null + + $result = Get-AssemblyProbingPaths -CompilerFolder $compilerFolder + + ($result -like '*shared') | Should -Not -BeNullOrEmpty + } + } + + Describe 'CompileAppsInWorkspace argument construction' { + It 'Includes --analyzers when analyzers are specified' { + InModuleScope CompileFromWorkspace { + $script:capturedArguments = @() + $wsFile = Join-Path $TestDrive 'test.code-workspace' + Set-Content -Path $wsFile -Value '{}' + $outDir = Join-Path $TestDrive 'out-args1' + New-Item -Path $outDir -ItemType Directory -Force | Out-Null + Mock RunAndCheck { + $script:capturedArguments = $args + } + Mock Copy-CompiledAppsToOutput { return @() } + + CompileAppsInWorkspace -ALToolPath 'altool.exe' -WorkspaceFile $wsFile -MaxCpuCount 1 -OutFolder $outDir -PackageCachePath $outDir -Analyzers @('CodeCop', 'UICop') + + $script:capturedArguments | Should -Contain '--analyzers' + $script:capturedArguments | Should -Contain 'CodeCop,UICop' + } + } + + It 'Includes --features when features are specified' { + InModuleScope CompileFromWorkspace { + $script:capturedArguments = @() + $wsFile = Join-Path $TestDrive 'test.code-workspace' + Set-Content -Path $wsFile -Value '{}' + $outDir = Join-Path $TestDrive 'out-args2' + New-Item -Path $outDir -ItemType Directory -Force | Out-Null + Mock RunAndCheck { + $script:capturedArguments = $args + } + Mock Copy-CompiledAppsToOutput { return @() } + + CompileAppsInWorkspace -ALToolPath 'altool.exe' -WorkspaceFile $wsFile -MaxCpuCount 1 -OutFolder $outDir -PackageCachePath $outDir -Features @('TranslationFile', 'GenerateCaptions') + + $script:capturedArguments | Should -Contain '--features' + $script:capturedArguments | Should -Contain 'TranslationFile,GenerateCaptions' + } + } + + It 'Includes --define when preprocessor symbols are specified' { + InModuleScope CompileFromWorkspace { + $script:capturedArguments = @() + $wsFile = Join-Path $TestDrive 'test.code-workspace' + Set-Content -Path $wsFile -Value '{}' + $outDir = Join-Path $TestDrive 'out-args3' + New-Item -Path $outDir -ItemType Directory -Force | Out-Null + Mock RunAndCheck { + $script:capturedArguments = $args + } + Mock Copy-CompiledAppsToOutput { return @() } + + CompileAppsInWorkspace -ALToolPath 'altool.exe' -WorkspaceFile $wsFile -MaxCpuCount 1 -OutFolder $outDir -PackageCachePath $outDir -PreprocessorSymbols @('CLEAN', 'DEBUG') + + $script:capturedArguments | Should -Contain '--define' + $script:capturedArguments | Should -Contain 'CLEAN;DEBUG' + } + } + + It 'Includes --ruleset when ruleset is specified' { + InModuleScope CompileFromWorkspace { + $script:capturedArguments = @() + $wsFile = Join-Path $TestDrive 'test.code-workspace' + Set-Content -Path $wsFile -Value '{}' + $outDir = Join-Path $TestDrive 'out-args4' + New-Item -Path $outDir -ItemType Directory -Force | Out-Null + Mock RunAndCheck { + $script:capturedArguments = $args + } + Mock Copy-CompiledAppsToOutput { return @() } + + CompileAppsInWorkspace -ALToolPath 'altool.exe' -WorkspaceFile $wsFile -MaxCpuCount 1 -OutFolder $outDir -PackageCachePath $outDir -Ruleset 'myruleset.json' + + $script:capturedArguments | Should -Contain '--ruleset' + $script:capturedArguments | Should -Contain 'myruleset.json' + } + } + + It 'Omits --maxcpucount when equal to processor count' { + InModuleScope CompileFromWorkspace { + $script:capturedArguments = @() + $wsFile = Join-Path $TestDrive 'test.code-workspace' + Set-Content -Path $wsFile -Value '{}' + $outDir = Join-Path $TestDrive 'out-args5' + New-Item -Path $outDir -ItemType Directory -Force | Out-Null + Mock RunAndCheck { + $script:capturedArguments = $args + } + Mock Copy-CompiledAppsToOutput { return @() } + + CompileAppsInWorkspace -ALToolPath 'altool.exe' -WorkspaceFile $wsFile -MaxCpuCount ([System.Environment]::ProcessorCount) -OutFolder $outDir -PackageCachePath $outDir + + $script:capturedArguments | Should -Not -Contain '--maxcpucount' + } + } + + It 'Always includes --logdirectory' { + InModuleScope CompileFromWorkspace { + $script:capturedArguments = @() + $wsFile = Join-Path $TestDrive 'test.code-workspace' + Set-Content -Path $wsFile -Value '{}' + $outDir = Join-Path $TestDrive 'out-args6' + New-Item -Path $outDir -ItemType Directory -Force | Out-Null + Mock RunAndCheck { + $script:capturedArguments = $args + } + Mock Copy-CompiledAppsToOutput { return @() } + + CompileAppsInWorkspace -ALToolPath 'altool.exe' -WorkspaceFile $wsFile -MaxCpuCount 1 -OutFolder $outDir -PackageCachePath $outDir + + $script:capturedArguments | Should -Contain '--logdirectory' + } + } + } +} diff --git a/Tests/DetermineArtifactsForRelease.Test.ps1 b/Tests/DetermineArtifactsForRelease.Test.ps1 index 1afea5bad7..e052d145af 100644 --- a/Tests/DetermineArtifactsForRelease.Test.ps1 +++ b/Tests/DetermineArtifactsForRelease.Test.ps1 @@ -1,4 +1,7 @@ -Get-Module Github-Helper | Remove-Module -Force +[Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSReviewUnusedParameter', '', Justification = 'Mock/callback parameters must match function signatures')] +param() + +Get-Module Github-Helper | Remove-Module -Force Import-Module (Join-Path $PSScriptRoot '..\Actions\Github-Helper.psm1' -Resolve) Get-Module TestActionsHelper | Remove-Module -Force Import-Module (Join-Path $PSScriptRoot 'TestActionsHelper.psm1') @@ -15,6 +18,16 @@ Describe 'DetermineArtifactsForRelease Tests' { $scriptPath = Join-Path $scriptRoot $scriptName [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseDeclaredVarsMoreThanAssignments', 'actionScript', Justification = 'False positive.')] $actionScript = GetActionScript -scriptRoot $scriptRoot -scriptName $scriptName + + function MockArtifact { + param([string] $name, [string] $sha = 'abc123') + [PSCustomObject]@{ + name = $name + expired = $false + archive_download_url = "https://example.com/artifacts/$name" + workflow_run = [PSCustomObject]@{ head_sha = $sha } + } + } } It 'Compile Action' { @@ -28,4 +41,98 @@ Describe 'DetermineArtifactsForRelease Tests' { } YamlTest -scriptRoot $scriptRoot -actionName $actionName -actionScript $actionScript -outputs $outputs } + + Context 'Artifact selection behavior' { + BeforeAll { + $env:GITHUB_REPOSITORY = 'org/repo' + $env:GITHUB_API_URL = 'https://api.github.com' + $env:GITHUB_REF_NAME = 'main' + $env:Settings = '{"repoName":"repo","type":"PTE","powerPlatformSolutionFolder":""}' + } + + BeforeEach { + $script:githubOutputFile = Join-Path ([System.IO.Path]::GetTempPath()) ([System.IO.Path]::GetRandomFileName()) + New-Item -Path $script:githubOutputFile -ItemType File | Out-Null + $env:GITHUB_OUTPUT = $script:githubOutputFile + } + + AfterEach { + if ($script:githubOutputFile -and (Test-Path $script:githubOutputFile)) { + Remove-Item -Path $script:githubOutputFile -Force + } + } + + It 'Mixed projects (apps + test artifacts) produce a non-empty include list' { + $artifacts = @( + (MockArtifact 'proj1-main-Apps-1.0.0.0'), + (MockArtifact 'proj1-main-TestApps-1.0.0.0'), + (MockArtifact 'proj2-main-Apps-1.0.0.0') + ) + Mock Invoke-RestMethod -ParameterFilter { $Uri -like '*actions/artifacts*page=1*' } -MockWith { + [PSCustomObject]@{ total_count = $artifacts.Count; Artifacts = $artifacts } + } + Mock Invoke-RestMethod -ParameterFilter { $Uri -like '*actions/artifacts*page=2*' } -MockWith { + [PSCustomObject]@{ total_count = 0; Artifacts = @() } + } + Mock Invoke-RestMethod -ParameterFilter { $Uri -like '*/branches/*' } -MockWith { + [PSCustomObject]@{ commit = [PSCustomObject]@{ sha = 'abc123' } } + } + + & $scriptPath -buildVersion 'latest' -GITHUB_TOKEN 'tok' -TOKENFORPUSH 'tok' -ProjectsJson '["proj1","proj2"]' + + $output = Get-Content $env:GITHUB_OUTPUT -Raw + $output | Should -Match 'commitish=abc123' + $output | Should -Match 'proj1-main-Apps-1\.0\.0\.0' + $output | Should -Match 'proj1-main-TestApps-1\.0\.0\.0' + $output | Should -Match 'proj2-main-Apps-1\.0\.0\.0' + } + + It 'Test-only project is skipped with a warning (including build-mode test artifacts)' { + $artifacts = @( + (MockArtifact 'proj1-main-Apps-1.0.0.0'), + (MockArtifact 'proj2-main-CleanTestApps-1.0.0.0') + ) + Mock Invoke-RestMethod -ParameterFilter { $Uri -like '*actions/artifacts*page=1*' } -MockWith { + [PSCustomObject]@{ total_count = $artifacts.Count; Artifacts = $artifacts } + } + Mock Invoke-RestMethod -ParameterFilter { $Uri -like '*actions/artifacts*page=2*' } -MockWith { + [PSCustomObject]@{ total_count = 0; Artifacts = @() } + } + Mock Invoke-RestMethod -ParameterFilter { $Uri -like '*/branches/*' } -MockWith { + [PSCustomObject]@{ commit = [PSCustomObject]@{ sha = 'abc123' } } + } + Mock Write-Host { } -ParameterFilter { "$Object" -like '::Warning::*proj2*' } + + & $scriptPath -buildVersion 'latest' -GITHUB_TOKEN 'tok' -TOKENFORPUSH 'tok' -ProjectsJson '["proj1","proj2"]' + + Assert-MockCalled Write-Host -ParameterFilter { "$Object" -like '::Warning::*proj2*' } -Scope It + $output = Get-Content $env:GITHUB_OUTPUT -Raw + $output | Should -Match 'proj1-main-Apps-1\.0\.0\.0' + $output | Should -Not -Match 'proj2-main-CleanTestApps' + } + + It 'Throws a clear error when no project has releasable artifacts' { + $artifacts = @( + (MockArtifact 'proj1-main-TestApps-1.0.0.0') + ) + Mock Invoke-RestMethod -ParameterFilter { $Uri -like '*actions/artifacts*page=1*' } -MockWith { + [PSCustomObject]@{ total_count = $artifacts.Count; Artifacts = $artifacts } + } + Mock Invoke-RestMethod -ParameterFilter { $Uri -like '*actions/artifacts*page=2*' } -MockWith { + [PSCustomObject]@{ total_count = 0; Artifacts = @() } + } + + { & $scriptPath -buildVersion 'latest' -GITHUB_TOKEN 'tok' -TOKENFORPUSH 'tok' -ProjectsJson '["proj1"]' } | + Should -Throw -ExpectedMessage '*No release artifacts found for any project*' + } + + It 'Throws original error when a project has no artifacts of any kind' { + Mock Invoke-RestMethod -ParameterFilter { $Uri -like '*actions/artifacts*' } -MockWith { + [PSCustomObject]@{ total_count = 0; Artifacts = @() } + } + + { & $scriptPath -buildVersion 'latest' -GITHUB_TOKEN 'tok' -TOKENFORPUSH 'tok' -ProjectsJson '["proj1"]' } | + Should -Throw -ExpectedMessage '*No artifacts found for this project*' + } + } } diff --git a/Tests/DetermineProjectsToBuild.Test.ps1 b/Tests/DetermineProjectsToBuild.Test.ps1 index 8445567cc8..224f1e30c6 100644 --- a/Tests/DetermineProjectsToBuild.Test.ps1 +++ b/Tests/DetermineProjectsToBuild.Test.ps1 @@ -1,4 +1,4 @@ -Import-Module (Join-Path $PSScriptRoot 'TestActionsHelper.psm1') -Force +Import-Module (Join-Path $PSScriptRoot 'TestActionsHelper.psm1') -Force Describe "Get-ProjectsToBuild" { BeforeAll { @@ -503,15 +503,15 @@ Describe "Get-ProjectsToBuild" { It 'loads dependent projects correctly, if useProjectDependencies is set to false' { # Two dependent projects - $dependecyAppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } + $dependencyAppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } New-Item -Path "$baseFolder/Project1/.AL-Go/settings.json" -type File -Force - New-Item -Path "$baseFolder/Project1/app/app.json" -Value (ConvertTo-Json $dependecyAppFile -Depth 10) -type File -Force + New-Item -Path "$baseFolder/Project1/app/app.json" -Value (ConvertTo-Json $dependencyAppFile -Depth 10) -type File -Force $dependantAppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd2'; name = 'Second App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @(@{id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'} ) } New-Item -Path "$baseFolder/Project2/.AL-Go/settings.json" -type File -Force New-Item -Path "$baseFolder/Project2/app/app.json" -Value (ConvertTo-Json $dependantAppFile -Depth 10) -type File -Force - #Add settings file + # Add settings file $alGoSettings = @{ fullBuildPatterns = @(); projects = @(); powerPlatformSolutionFolder = ''; useProjectDependencies = $false } $env:Settings = ConvertTo-Json $alGoSettings -Depth 99 -Compress @@ -555,19 +555,64 @@ Describe "Get-ProjectsToBuild" { $buildOrder[0].buildDimensions[0].project | Should -BeExactly "Project1" $buildOrder[0].buildDimensions[1].buildMode | Should -BeExactly "Default" $buildOrder[0].buildDimensions[1].project | Should -BeExactly "Project2" + + # Test that setting postponeProjectInBuildOrder to true doesn't have any effect when useProjectDependencies is false + $alGoSettings = @{ fullBuildPatterns = @(); projects = @(); powerPlatformSolutionFolder = ''; postponeProjectInBuildOrder = $true; useProjectDependencies = $false } + $env:Settings = ConvertTo-Json $alGoSettings -Depth 99 -Compress + + $allProjects, $modifiedProjects, $projectsToBuild, $projectDependencies, $buildOrder = Get-ProjectsToBuild -baseFolder $baseFolder + + $allProjects | Should -BeExactly @("Project1", "Project2") + $modifiedProjects | Should -BeExactly @() + $projectsToBuild | Should -BeExactly @("Project1", "Project2") + + $projectDependencies | Should -BeOfType System.Collections.Hashtable + $projectDependencies['Project1'] | Should -BeExactly @() + $projectDependencies['Project2'] | Should -BeExactly @() + + # Build order should have the following structure: + #[ + # { + # "projects": [ + # "Project1", + # "Project2" + # ], + # "projectsCount": 2, + # "buildDimensions": [ + # { + # "buildMode": "Default", + # "project": "Project1" + # }, + # { + # "buildMode": "Default", + # "project": "Project2" + # } + # ] + # } + #] + + $buildOrder.Count | Should -BeExactly 1 + $buildOrder[0] | Should -BeOfType System.Collections.Hashtable + $buildOrder[0].projects | Should -BeExactly @("Project1", "Project2") + $buildOrder[0].projectsCount | Should -BeExactly 2 + $buildOrder[0].buildDimensions.Count | Should -BeExactly 2 + $buildOrder[0].buildDimensions[0].buildMode | Should -BeExactly "Default" + $buildOrder[0].buildDimensions[0].project | Should -BeExactly "Project1" + $buildOrder[0].buildDimensions[1].buildMode | Should -BeExactly "Default" + $buildOrder[0].buildDimensions[1].project | Should -BeExactly "Project2" } It 'loads dependent projects correctly, if useProjectDependencies is set to true' { # Two dependent projects - $dependecyAppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } + $dependencyAppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } New-Item -Path "$baseFolder/Project1/.AL-Go/settings.json" -type File -Force - New-Item -Path "$baseFolder/Project1/app/app.json" -Value (ConvertTo-Json $dependecyAppFile -Depth 10) -type File -Force + New-Item -Path "$baseFolder/Project1/app/app.json" -Value (ConvertTo-Json $dependencyAppFile -Depth 10) -type File -Force $dependantAppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd2'; name = 'Second App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @(@{id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'} ) } New-Item -Path "$baseFolder/Project2/.AL-Go/settings.json" -type File -Force New-Item -Path "$baseFolder/Project2/app/app.json" -Value (ConvertTo-Json $dependantAppFile -Depth 10) -type File -Force - #Add settings file + # Add settings file $alGoSettings = @{ fullBuildPatterns = @(); projects = @(); powerPlatformSolutionFolder = ''; useProjectDependencies = $true } New-Item -Path "$baseFolder/.github" -type Directory -Force $alGoSettings | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder ".github/AL-Go-Settings.json") -Encoding UTF8 @@ -627,6 +672,63 @@ Describe "Get-ProjectsToBuild" { $buildOrder[1].buildDimensions.Count | Should -BeExactly 1 $buildOrder[1].buildDimensions[0].buildMode | Should -BeExactly "Default" $buildOrder[1].buildDimensions[0].project | Should -BeExactly "Project2" + + # Test that setting postponeProjectInBuildOrder to true in the last project in the build order doesn't fail or change anything + $projectSettings = @{ "postponeProjectInBuildOrder" = $true } + Set-Content -Path "$baseFolder/Project2/.AL-Go/settings.json" -Value (ConvertTo-Json $projectSettings -Depth 10) + + $allProjects, $modifiedProjects, $projectsToBuild, $projectDependencies, $buildOrder = Get-ProjectsToBuild -baseFolder $baseFolder + + $allProjects | Should -BeExactly @("Project1", "Project2") + $modifiedProjects | Should -BeExactly @() + $projectsToBuild | Should -BeExactly @("Project1", "Project2") + + $projectDependencies | Should -BeOfType System.Collections.Hashtable + $projectDependencies['Project1'] | Should -BeExactly @() + $projectDependencies['Project2'] | Should -BeExactly @("Project1") + + # Build order should have the following structure: + #[ + # { + # "projects": [ + # "Project1" + # ], + # "projectsCount": 1, + # "buildDimensions": [ + # { + # "buildMode": "Default", + # "project": "Project1" + # } + # ] + # }, + # { + # "projects": [ + # "Project2" + # ], + # "projectsCount": 1, + # "buildDimensions": [ + # { + # "buildMode": "Default", + # "project": "Project2" + # } + # ] + # } + #] + + $buildOrder.Count | Should -BeExactly 2 + $buildOrder[0] | Should -BeOfType System.Collections.Hashtable + $buildOrder[0].projects | Should -BeExactly @("Project1") + $buildOrder[0].projectsCount | Should -BeExactly 1 + $buildOrder[0].buildDimensions.Count | Should -BeExactly 1 + $buildOrder[0].buildDimensions[0].buildMode | Should -BeExactly "Default" + $buildOrder[0].buildDimensions[0].project | Should -BeExactly "Project1" + + $buildOrder[1] | Should -BeOfType System.Collections.Hashtable + $buildOrder[1].projects | Should -BeExactly @("Project2") + $buildOrder[1].projectsCount | Should -BeExactly 1 + $buildOrder[1].buildDimensions.Count | Should -BeExactly 1 + $buildOrder[1].buildDimensions[0].buildMode | Should -BeExactly "Default" + $buildOrder[1].buildDimensions[0].project | Should -BeExactly "Project2" } It 'loads dependent projects correctly, if useProjectDependencies is set to false in a project setting' { @@ -634,9 +736,9 @@ Describe "Get-ProjectsToBuild" { # Project 1 # Project 2 depends on Project 1 - useProjectDependencies is set to true from the repo settings # Project 3 depends on Project 1, but has useProjectDependencies set to false in the project settings - $dependecyAppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } + $dependencyAppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } New-Item -Path "$baseFolder/Project1/.AL-Go/settings.json" -type File -Force - New-Item -Path "$baseFolder/Project1/app/app.json" -Value (ConvertTo-Json $dependecyAppFile -Depth 10) -type File -Force + New-Item -Path "$baseFolder/Project1/app/app.json" -Value (ConvertTo-Json $dependencyAppFile -Depth 10) -type File -Force $dependantAppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd2'; name = 'Second App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @(@{id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'} ) } New-Item -Path "$baseFolder/Project2/.AL-Go/settings.json" -type File -Force @@ -728,9 +830,9 @@ Describe "Get-ProjectsToBuild" { It 'throws if the calculated build depth is more than the maximum supported' { # Two dependent projects - $dependecyAppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } + $dependencyAppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } New-Item -Path "$baseFolder/Project1/.AL-Go/settings.json" -type File -Force - New-Item -Path "$baseFolder/Project1/app/app.json" -Value (ConvertTo-Json $dependecyAppFile -Depth 10) -type File -Force + New-Item -Path "$baseFolder/Project1/app/app.json" -Value (ConvertTo-Json $dependencyAppFile -Depth 10) -type File -Force $dependantAppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd2'; name = 'Second App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @(@{id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'} ) } New-Item -Path "$baseFolder/Project2/.AL-Go/settings.json" -type File -Force @@ -747,6 +849,323 @@ Describe "Get-ProjectsToBuild" { { Get-ProjectsToBuild -baseFolder $baseFolder -maxBuildDepth 1 } | Should -Throw "The build depth is too deep, the maximum build depth is 1. You need to run 'Update AL-Go System Files' to update the workflows" } + It 'postpones projects if postponeProjectInBuildOrder is set to true' { + # Add three dependent projects + # Project 1 + # Project 2 depends on Project 1, has postponeProjectInBuildOrder set to true + # Project 3 depends on Project 1, has postponeProjectInBuildOrder set to true + # Project 4 depends on Project 2 + $dependencyAppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } + New-Item -Path "$baseFolder/Project1/.AL-Go/settings.json" -type File -Force + New-Item -Path "$baseFolder/Project1/app/app.json" -Value (ConvertTo-Json $dependencyAppFile -Depth 10) -type File -Force + + $dependantAppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd2'; name = 'Second App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @(@{id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'} ) } + New-Item -Path "$baseFolder/Project2/.AL-Go/settings.json" -type File -Force + @{ postponeProjectInBuildOrder = $true } | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder "Project2/.AL-Go/settings.json") -Encoding UTF8 + New-Item -Path "$baseFolder/Project2/app/app.json" -Value (ConvertTo-Json $dependantAppFile -Depth 10) -type File -Force + + $dependantAppFile3 = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd3'; name = 'Third App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @(@{id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'} ) } + New-Item -Path "$baseFolder/Project3/.AL-Go/settings.json" -type File -Force + @{ postponeProjectInBuildOrder = $true } | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder "Project3/.AL-Go/settings.json") -Encoding UTF8 + New-Item -Path "$baseFolder/Project3/app/app.json" -Value (ConvertTo-Json $dependantAppFile3 -Depth 10) -type File -Force + + $dependantAppFile4 = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd4'; name = 'Fourth App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @(@{id = '83fb8305-4079-415d-a25d-8132f0436fd2'; name = 'Second App'; publisher = 'Contoso'; version = '1.0.0.0'} ) } + New-Item -Path "$baseFolder/Project4/.AL-Go/settings.json" -type File -Force + New-Item -Path "$baseFolder/Project4/app/app.json" -Value (ConvertTo-Json $dependantAppFile4 -Depth 10) -type File -Force + + #Add settings file + $alGoSettings = @{ fullBuildPatterns = @(); projects = @(); powerPlatformSolutionFolder = ''; useProjectDependencies = $true } + New-Item -Path "$baseFolder/.github" -type Directory -Force + $alGoSettings | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder ".github/AL-Go-Settings.json") -Encoding UTF8 + + # Add settings as environment variable to simulate we've run ReadSettings + $env:Settings = ConvertTo-Json $alGoSettings -Depth 99 -Compress + + $allProjects, $modifiedProjects, $projectsToBuild, $projectDependencies, $buildOrder = Get-ProjectsToBuild -baseFolder $baseFolder + + $allProjects | Should -BeExactly @("Project1", "Project2", "Project3", "Project4") + $modifiedProjects | Should -BeExactly @() + $projectsToBuild | Should -BeExactly @('Project1', 'Project2', 'Project3', 'Project4') + + $projectDependencies | Should -BeOfType System.Collections.Hashtable + $projectDependencies['Project1'] | Should -BeExactly @() + $projectDependencies['Project2'] | Should -BeExactly @("Project1") + $projectDependencies['Project3'] | Should -BeExactly @("Project1") + $projectDependencies['Project4'] | Should -BeExactly @("Project2", "Project1") + + # Build order should have the following structure: + #[ + #{ + # "buildDimensions": [ + # { + # "projectName": "Project1", + # "buildMode": "Default", + # "project": "Project1", + # "githubRunnerShell": "powershell", + # "gitHubRunner": "\"windows-latest\"" + # }, + # ], + # "projectsCount": 1, + # "projects": [ + # "Project1" + # ] + #}, + #{ + # "buildDimensions": [ + # { + # "projectName": "Project2", + # "buildMode": "Default", + # "project": "Project2", + # "githubRunnerShell": "powershell", + # "gitHubRunner": "\"windows-latest\"" + # } + # ], + # "projectsCount": 1, + # "projects": [ + # "Project2" + # ] + #} + #{ + # "buildDimensions": [ + # { + # "projectName": "Project3", + # "buildMode": "Default", + # "project": "Project3", + # "githubRunnerShell": "powershell", + # "gitHubRunner": "\"windows-latest\"" + # }, + # { + # "projectName": "Project4", + # "buildMode": "Default", + # "project": "Project4", + # "githubRunnerShell": "powershell", + # "gitHubRunner": "\"windows-latest\"" + # } + # ], + # "projectsCount": 2, + # "projects": [ + # "Project3", + # "Project4" + # ] + #} + #] + $buildOrder.Count | Should -BeExactly 3 + $buildOrder[0] | Should -BeOfType System.Collections.Hashtable + $buildOrder[0].projects | Should -BeExactly @("Project1") + $buildOrder[0].projectsCount | Should -BeExactly 1 + $buildOrder[0].buildDimensions.Count | Should -BeExactly 1 + $buildOrder[0].buildDimensions[0].buildMode | Should -BeExactly "Default" + $buildOrder[0].buildDimensions[0].project | Should -BeExactly "Project1" + + $buildOrder[1] | Should -BeOfType System.Collections.Hashtable + $buildOrder[1].projects | Should -BeExactly @("Project2") + $buildOrder[1].projectsCount | Should -BeExactly 1 + $buildOrder[1].buildDimensions.Count | Should -BeExactly 1 + $buildOrder[1].buildDimensions[0].buildMode | Should -BeExactly "Default" + $buildOrder[1].buildDimensions[0].project | Should -BeExactly "Project2" + + $buildOrder[2] | Should -BeOfType System.Collections.Hashtable + $buildOrder[2].projects | Should -BeExactly @("Project4", "Project3") + $buildOrder[2].projectsCount | Should -BeExactly 2 + $buildOrder[2].buildDimensions.Count | Should -BeExactly 2 + $buildOrder[2].buildDimensions[0].buildMode | Should -BeExactly "Default" + $buildOrder[2].buildDimensions[0].project | Should -BeExactly "Project4" + $buildOrder[2].buildDimensions[1].buildMode | Should -BeExactly "Default" + $buildOrder[2].buildDimensions[1].project | Should -BeExactly "Project3" + } + + It 'resolves test project dependencies from projectsToTest setting' { + # Project1 has an app + $project1AppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } + New-Item -Path "$baseFolder/Project1/.AL-Go/settings.json" -type File -Force + New-Item -Path "$baseFolder/Project1/app/app.json" -Value (ConvertTo-Json $project1AppFile -Depth 10) -type File -Force + + # TestProject is a test-only project that depends on Project1 via projectsToTest setting + New-Item -Path "$baseFolder/TestProject/.AL-Go/settings.json" -type File -Force + @{ projectsToTest = @("Project1") } | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder "TestProject/.AL-Go/settings.json") -Encoding UTF8 + + # Add settings file + $alGoSettings = @{ fullBuildPatterns = @(); projects = @(); powerPlatformSolutionFolder = ''; useProjectDependencies = $false } + New-Item -Path "$baseFolder/.github" -type Directory -Force + $alGoSettings | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder ".github/AL-Go-Settings.json") -Encoding UTF8 + + $env:Settings = ConvertTo-Json $alGoSettings -Depth 99 -Compress + + $allProjects, $modifiedProjects, $projectsToBuild, $projectDependencies, $buildOrder = Get-ProjectsToBuild -baseFolder $baseFolder + + $allProjects | Should -BeExactly @("Project1", "TestProject") + $projectsToBuild | Should -BeExactly @("Project1", "TestProject") + + $projectDependencies | Should -BeOfType System.Collections.Hashtable + $projectDependencies['Project1'] | Should -BeExactly @() + $projectDependencies['TestProject'] | Should -BeExactly @("Project1") + + # Build order: Project1 first, then TestProject + $buildOrder.Count | Should -BeExactly 2 + $buildOrder[0].projects | Should -BeExactly @("Project1") + $buildOrder[1].projects | Should -BeExactly @("TestProject") + } + + It 'resolves test project with transitive dependencies' { + # Project1 (base) -> Project2 depends on Project1 -> TestProject depends on Project2 + $project1AppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } + New-Item -Path "$baseFolder/Project1/.AL-Go/settings.json" -type File -Force + New-Item -Path "$baseFolder/Project1/app/app.json" -Value (ConvertTo-Json $project1AppFile -Depth 10) -type File -Force + + $project2AppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd2'; name = 'Second App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @(@{id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'} ) } + New-Item -Path "$baseFolder/Project2/.AL-Go/settings.json" -type File -Force + New-Item -Path "$baseFolder/Project2/app/app.json" -Value (ConvertTo-Json $project2AppFile -Depth 10) -type File -Force + + # TestProject depends on Project2 (which transitively depends on Project1) + New-Item -Path "$baseFolder/TestProject/.AL-Go/settings.json" -type File -Force + @{ projectsToTest = @("Project2") } | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder "TestProject/.AL-Go/settings.json") -Encoding UTF8 + + $alGoSettings = @{ fullBuildPatterns = @(); projects = @(); powerPlatformSolutionFolder = ''; useProjectDependencies = $true } + New-Item -Path "$baseFolder/.github" -type Directory -Force + $alGoSettings | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder ".github/AL-Go-Settings.json") -Encoding UTF8 + + $env:Settings = ConvertTo-Json $alGoSettings -Depth 99 -Compress + + $allProjects, $modifiedProjects, $projectsToBuild, $projectDependencies, $buildOrder = Get-ProjectsToBuild -baseFolder $baseFolder + + $allProjects | Should -BeExactly @("Project1", "Project2", "TestProject") + + $projectDependencies['Project1'] | Should -BeExactly @() + $projectDependencies['Project2'] | Should -BeExactly @("Project1") + # TestProject should depend on both Project2 AND Project1 (transitive) + $projectDependencies['TestProject'] | Should -Contain "Project1" + $projectDependencies['TestProject'] | Should -Contain "Project2" + + # Build order: Project1 first, Project2 second, TestProject last + $buildOrder.Count | Should -BeExactly 3 + $buildOrder[0].projects | Should -BeExactly @("Project1") + $buildOrder[1].projects | Should -BeExactly @("Project2") + $buildOrder[2].projects | Should -BeExactly @("TestProject") + } + + It 'throws error when test project has buildable app folders' { + # TestProject has both projectsToTest setting AND an app folder - this should fail + + $appFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } + New-Item -Path "$baseFolder/Project1/.AL-Go/settings.json" -type File -Force + New-Item -Path "$baseFolder/Project1/app/app.json" -Value (ConvertTo-Json $appFile -Depth 10) -type File -Force + + New-Item -Path "$baseFolder/TestProject/.AL-Go/settings.json" -type File -Force + @{ projectsToTest = @("Project1") } | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder "TestProject/.AL-Go/settings.json") -Encoding UTF8 + # Add an app folder to the test project - this should be forbidden + $testAppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd2'; name = 'Bad App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } + New-Item -Path "$baseFolder/TestProject/app/app.json" -Value (ConvertTo-Json $testAppFile -Depth 10) -type File -Force + + $alGoSettings = @{ fullBuildPatterns = @(); projects = @(); powerPlatformSolutionFolder = ''; useProjectDependencies = $false } + New-Item -Path "$baseFolder/.github" -type Directory -Force + $alGoSettings | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder ".github/AL-Go-Settings.json") -Encoding UTF8 + + $env:Settings = ConvertTo-Json $alGoSettings -Depth 99 -Compress + + { Get-ProjectsToBuild -baseFolder $baseFolder } | Should -Throw "*must not contain buildable code*" + } + + It 'throws error when test project has buildable test folders' { + # TestProject has both projectsToTest setting AND a test folder - this should fail + + $appFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } + New-Item -Path "$baseFolder/Project1/.AL-Go/settings.json" -type File -Force + New-Item -Path "$baseFolder/Project1/app/app.json" -Value (ConvertTo-Json $appFile -Depth 10) -type File -Force + + # Add a test folder with an app to the test project + $testAppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd3'; name = 'Bad Test App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } + New-Item -Path "$baseFolder/TestProject/.AL-Go/settings.json" -type File -Force + New-Item -Path "$baseFolder/TestProject/test/app.json" -Value (ConvertTo-Json $testAppFile -Depth 10) -type File -Force + @{ projectsToTest = @("Project1"); testFolders = @("test") } | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder "TestProject/.AL-Go/settings.json") -Encoding UTF8 + + $alGoSettings = @{ fullBuildPatterns = @(); projects = @(); powerPlatformSolutionFolder = ''; useProjectDependencies = $false } + New-Item -Path "$baseFolder/.github" -type Directory -Force + $alGoSettings | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder ".github/AL-Go-Settings.json") -Encoding UTF8 + + $env:Settings = ConvertTo-Json $alGoSettings -Depth 99 -Compress + + { Get-ProjectsToBuild -baseFolder $baseFolder } | Should -Throw "*must not contain buildable code*" + } + + It 'throws error when one test project depends on another test project' { + # Project1 is a normal project, TestProject1 and TestProject2 are both test projects + # TestProject2 tries to depend on TestProject1 - this should fail + Mock OutputError {} -ModuleName DetermineProjectsToBuild + + $appFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } + New-Item -Path "$baseFolder/Project1/.AL-Go/settings.json" -type File -Force + New-Item -Path "$baseFolder/Project1/app/app.json" -Value (ConvertTo-Json $appFile -Depth 10) -type File -Force + + New-Item -Path "$baseFolder/TestProject1/.AL-Go/settings.json" -type File -Force + @{ projectsToTest = @("Project1") } | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder "TestProject1/.AL-Go/settings.json") -Encoding UTF8 + + New-Item -Path "$baseFolder/TestProject2/.AL-Go/settings.json" -type File -Force + @{ projectsToTest = @("TestProject1") } | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder "TestProject2/.AL-Go/settings.json") -Encoding UTF8 + + $alGoSettings = @{ fullBuildPatterns = @(); projects = @(); powerPlatformSolutionFolder = ''; useProjectDependencies = $false } + New-Item -Path "$baseFolder/.github" -type Directory -Force + $alGoSettings | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder ".github/AL-Go-Settings.json") -Encoding UTF8 + + $env:Settings = ConvertTo-Json $alGoSettings -Depth 99 -Compress + + { Get-ProjectsToBuild -baseFolder $baseFolder } | Should -Throw + Should -Invoke OutputError -ModuleName DetermineProjectsToBuild -ParameterFilter { $message -like "*cannot depend on another test project*" } + } + + It 'throws error when projectsToTest references nonexistent project' { + # Project1 exists + Mock OutputError {} -ModuleName DetermineProjectsToBuild + + $project1AppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } + New-Item -Path "$baseFolder/Project1/.AL-Go/settings.json" -type File -Force + New-Item -Path "$baseFolder/Project1/app/app.json" -Value (ConvertTo-Json $project1AppFile -Depth 10) -type File -Force + + # TestProject references NonExistentProject + New-Item -Path "$baseFolder/TestProject/.AL-Go/settings.json" -type File -Force + @{ projectsToTest = @("NonExistentProject") } | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder "TestProject/.AL-Go/settings.json") -Encoding UTF8 + + $alGoSettings = @{ fullBuildPatterns = @(); projects = @(); powerPlatformSolutionFolder = ''; useProjectDependencies = $false } + New-Item -Path "$baseFolder/.github" -type Directory -Force + $alGoSettings | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder ".github/AL-Go-Settings.json") -Encoding UTF8 + + $env:Settings = ConvertTo-Json $alGoSettings -Depth 99 -Compress + + { Get-ProjectsToBuild -baseFolder $baseFolder } | Should -Throw + Should -Invoke OutputError -ModuleName DetermineProjectsToBuild -ParameterFilter { $message -like "*does not exist*" } + } + + It 'test project can depend on multiple upstream projects' { + # Project1 and Project2 are independent + $project1AppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd1'; name = 'First App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } + New-Item -Path "$baseFolder/Project1/.AL-Go/settings.json" -type File -Force + New-Item -Path "$baseFolder/Project1/app/app.json" -Value (ConvertTo-Json $project1AppFile -Depth 10) -type File -Force + + $project2AppFile = @{ id = '83fb8305-4079-415d-a25d-8132f0436fd2'; name = 'Second App'; publisher = 'Contoso'; version = '1.0.0.0'; dependencies = @() } + New-Item -Path "$baseFolder/Project2/.AL-Go/settings.json" -type File -Force + New-Item -Path "$baseFolder/Project2/app/app.json" -Value (ConvertTo-Json $project2AppFile -Depth 10) -type File -Force + + # TestProject depends on both Project1 and Project2 + New-Item -Path "$baseFolder/TestProject/.AL-Go/settings.json" -type File -Force + @{ projectsToTest = @("Project1", "Project2") } | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder "TestProject/.AL-Go/settings.json") -Encoding UTF8 + + $alGoSettings = @{ fullBuildPatterns = @(); projects = @(); powerPlatformSolutionFolder = ''; useProjectDependencies = $false } + New-Item -Path "$baseFolder/.github" -type Directory -Force + $alGoSettings | ConvertTo-Json -Depth 99 -Compress | Out-File (Join-Path $baseFolder ".github/AL-Go-Settings.json") -Encoding UTF8 + + $env:Settings = ConvertTo-Json $alGoSettings -Depth 99 -Compress + + $allProjects, $modifiedProjects, $projectsToBuild, $projectDependencies, $buildOrder = Get-ProjectsToBuild -baseFolder $baseFolder + + $projectDependencies['TestProject'] | Should -Contain "Project1" + $projectDependencies['TestProject'] | Should -Contain "Project2" + + # Build order: Project1 and Project2 in first layer (parallel), TestProject in second layer + $buildOrder.Count | Should -BeExactly 2 + $buildOrder[0].projects | Should -Contain "Project1" + $buildOrder[0].projects | Should -Contain "Project2" + $buildOrder[1].projects | Should -BeExactly @("TestProject") + } + AfterEach { Remove-Item $baseFolder -Force -Recurse } @@ -795,3 +1214,194 @@ Describe "Get-BuildAllProjects" { Remove-Item $baseFolder -Force -Recurse } } + +Describe "ConvertTo-RepoRelativePath" { + BeforeAll { + Import-Module (Join-Path $PSScriptRoot "../Actions/DetermineProjectsToBuild/DetermineProjectsToBuild.psm1" -Resolve) -DisableNameChecking -Force + } + + BeforeEach { + [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseDeclaredVarsMoreThanAssignments', 'baseFolder', Justification = 'False positive.')] + $baseFolder = (New-Item -ItemType Directory -Path (Join-Path $([System.IO.Path]::GetTempPath()) $([System.IO.Path]::GetRandomFileName()))).FullName + } + + It 'resolves a standard .\ prefixed folder to a repo-relative path' { + New-Item -Path "$baseFolder/app" -ItemType Directory -Force | Out-Null + $result = ConvertTo-RepoRelativePath -folder '.\app' -projectPath $baseFolder -baseFolder $baseFolder + $result | Should -Be 'app' + } + + It 'resolves a ../ relative folder to a repo-relative path' { + # Simulate: baseFolder/src/Apps/MyApp/App exists, project is at baseFolder/build/projects/Proj + New-Item -Path "$baseFolder/src/Apps/MyApp/App" -ItemType Directory -Force | Out-Null + $projectPath = (New-Item -Path "$baseFolder/build/projects/Proj" -ItemType Directory -Force).FullName + + $result = ConvertTo-RepoRelativePath -folder '..\..\..\src\Apps\MyApp\App' -projectPath $projectPath -baseFolder $baseFolder + $sep = [System.IO.Path]::DirectorySeparatorChar + $result | Should -Be "src${sep}Apps${sep}MyApp${sep}App" + } + + It 'returns $null for a folder that does not exist on disk' { + $result = ConvertTo-RepoRelativePath -folder '.\nonexistent' -projectPath $baseFolder -baseFolder $baseFolder + $result | Should -BeNullOrEmpty + } + + It 'returns $null for a folder that resolves outside the base folder' { + # Create a directory above baseFolder and try to reference it + $parentDir = Split-Path $baseFolder -Parent + $outsideDir = New-Item -Path (Join-Path $parentDir 'outside-repo') -ItemType Directory -Force + try { + $result = ConvertTo-RepoRelativePath -folder '..\outside-repo' -projectPath $baseFolder -baseFolder $baseFolder + $result | Should -BeNullOrEmpty + } + finally { + Remove-Item $outsideDir -Force -Recurse + } + } + + It 'handles baseFolder with trailing separator' { + New-Item -Path "$baseFolder/app" -ItemType Directory -Force | Out-Null + $baseFolderWithTrailing = $baseFolder + [System.IO.Path]::DirectorySeparatorChar + $result = ConvertTo-RepoRelativePath -folder '.\app' -projectPath $baseFolder -baseFolder $baseFolderWithTrailing + $result | Should -Be 'app' + } + + It 'handles deeply nested project with multiple ../ segments' { + New-Item -Path "$baseFolder/src/Common/Shared" -ItemType Directory -Force | Out-Null + $projectPath = (New-Item -Path "$baseFolder/a/b/c/d/project" -ItemType Directory -Force).FullName + + $result = ConvertTo-RepoRelativePath -folder '..\..\..\..\..\src\Common\Shared' -projectPath $projectPath -baseFolder $baseFolder + $sep = [System.IO.Path]::DirectorySeparatorChar + $result | Should -Be "src${sep}Common${sep}Shared" + } + + AfterEach { + Remove-Item $baseFolder -Force -Recurse + } +} + +Describe "Get-UnmodifiedAppsFromBaselineWorkflowRun" { + BeforeAll { + . (Join-Path -Path $PSScriptRoot -ChildPath "../Actions/AL-Go-Helper.ps1" -Resolve) + DownloadAndImportBcContainerHelper -baseFolder $([System.IO.Path]::GetTempPath()) + + Import-Module (Join-Path $PSScriptRoot "../Actions/DetermineProjectsToBuild/DetermineProjectsToBuild.psm1" -Resolve) -DisableNameChecking -Force + + # Helper to extract the download entries from captured Write-Host output for a given section + function script:Get-DownloadEntries { + param([string] $section, [System.Collections.ArrayList] $output) + $inSection = $false + $entries = @() + foreach ($line in $output) { + if ($line -eq "Download ${section}:") { + $inSection = $true + continue + } + if ($inSection) { + if ($line -match '^Download ') { break } + $entries += $line + } + } + return $entries + } + } + + BeforeEach { + [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseDeclaredVarsMoreThanAssignments', 'baseFolder', Justification = 'False positive.')] + $baseFolder = (New-Item -ItemType Directory -Path (Join-Path $([System.IO.Path]::GetTempPath()) $([System.IO.Path]::GetRandomFileName()))).FullName + + if (-not (Get-Command 'Trace-Information' -ErrorAction SilentlyContinue)) { + function global:Trace-Information { + [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSReviewUnusedParameter', 'Message', Justification = 'Stub function for testing.')] + [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSReviewUnusedParameter', 'AdditionalData', Justification = 'Stub function for testing.')] + param([string]$Message, $AdditionalData) + } + } + $env:GITHUB_API_URL = 'https://api.github.com' + $env:GITHUB_REPOSITORY = 'test/repo' + + Mock InvokeWebRequest { + $uri = $args[0] + if (-not $uri) { $uri = $Uri } + $content = if ($uri -like '*/actions/runs/*/artifacts*') { + '{"artifacts":[]}' + } else { + '{"head_branch":"main"}' + } + return [PSCustomObject]@{ Content = $content } + } -ModuleName 'Github-Helper' + + $script:capturedOutput = [System.Collections.ArrayList]::new() + Mock Write-Host { $null = $script:capturedOutput.Add($Object) } -ModuleName 'DetermineProjectsToBuild' + } + + It 'identifies unmodified apps when appFolders reference paths above the project via ../' { + $project = 'build/projects/MyProject' + $projectPath = Join-Path $baseFolder $project + + $repoSettings = @{ fullBuildPatterns = @(); projects = @($project); powerPlatformSolutionFolder = ''; useProjectDependencies = $false; incrementalBuilds = @{ onPull_Request = $true; mode = 'modifiedApps' } } + New-Item -Path "$baseFolder/.github/AL-Go-Settings.json" -Value (ConvertTo-Json $repoSettings -Depth 10) -type File -Force | Out-Null + New-Item -Path "$projectPath/.AL-Go/settings.json" -Value (ConvertTo-Json @{ appFolders = @('../../../src/Apps/*/App'); testFolders = @(); bcptTestFolders = @() } -Depth 10) -type File -Force | Out-Null + + @('AppA', 'AppB', 'AppC') | ForEach-Object { + $app = @{ id = [guid]::NewGuid().ToString(); name = $_; publisher = 'Test'; version = '1.0.0.0'; dependencies = @() } + New-Item -Path "$baseFolder/src/Apps/$_/App/app.json" -Value (ConvertTo-Json $app -Depth 10) -type File -Force | Out-Null + } + + $env:Settings = ConvertTo-Json $repoSettings -Depth 99 -Compress + + Push-Location $projectPath + $resolvedAppFolders = @(Resolve-Path '../../../src/Apps/*/App' -Relative -ErrorAction SilentlyContinue | Where-Object { Test-Path (Join-Path $_ 'app.json') }) + Pop-Location + + $buildArtifactFolder = Join-Path $projectPath '.buildartifacts' + New-Item -Path $buildArtifactFolder -ItemType Directory -Force | Out-Null + + $sep = [System.IO.Path]::DirectorySeparatorChar + Get-UnmodifiedAppsFromBaselineWorkflowRun ` + -token 'fake-token' ` + -settings @{ appFolders = $resolvedAppFolders; testFolders = @(); bcptTestFolders = @() } ` + -baseFolder $baseFolder -project $project -baselineWorkflowRunId '12345' ` + -modifiedFiles @("src${sep}Apps${sep}AppA${sep}App${sep}MyCodeunit.al") ` + -buildArtifactFolder $buildArtifactFolder -buildMode 'Default' -projectPath $projectPath + + $entries = Get-DownloadEntries -section 'appFolders' -output $script:capturedOutput + $entries | Should -Not -Contain '- None' -Because 'unmodified apps should be identified for download' + ($entries | Where-Object { $_ -like '*AppB*' }) | Should -Not -BeNullOrEmpty -Because 'AppB is unmodified' + ($entries | Where-Object { $_ -like '*AppC*' }) | Should -Not -BeNullOrEmpty -Because 'AppC is unmodified' + ($entries | Where-Object { $_ -like '*AppA*' }) | Should -BeNullOrEmpty -Because 'AppA is modified' + } + + It 'identifies unmodified apps in a standard layout with .\ prefixed folders' { + $repoSettings = @{ fullBuildPatterns = @(); projects = @(); powerPlatformSolutionFolder = ''; useProjectDependencies = $false; incrementalBuilds = @{ onPull_Request = $true; mode = 'modifiedApps' } } + New-Item -Path "$baseFolder/.github/AL-Go-Settings.json" -Value (ConvertTo-Json $repoSettings -Depth 10) -type File -Force | Out-Null + New-Item -Path "$baseFolder/.AL-Go/settings.json" -Value (ConvertTo-Json @{} -Depth 10) -type File -Force | Out-Null + + @('app1', 'app2') | ForEach-Object { + $app = @{ id = [guid]::NewGuid().ToString(); name = $_; publisher = 'Test'; version = '1.0.0.0'; dependencies = @() } + New-Item -Path "$baseFolder/$_/app.json" -Value (ConvertTo-Json $app -Depth 10) -type File -Force | Out-Null + } + + $env:Settings = ConvertTo-Json $repoSettings -Depth 99 -Compress + + $buildArtifactFolder = Join-Path $baseFolder '.buildartifacts' + New-Item -Path $buildArtifactFolder -ItemType Directory -Force | Out-Null + + $sep = [System.IO.Path]::DirectorySeparatorChar + Get-UnmodifiedAppsFromBaselineWorkflowRun ` + -token 'fake-token' ` + -settings @{ appFolders = @('.\app1', '.\app2'); testFolders = @(); bcptTestFolders = @() } ` + -baseFolder $baseFolder -project '' -baselineWorkflowRunId '12345' ` + -modifiedFiles @("app1${sep}MyCodeunit.al") ` + -buildArtifactFolder $buildArtifactFolder -buildMode 'Default' -projectPath $baseFolder + + $entries = Get-DownloadEntries -section 'appFolders' -output $script:capturedOutput + $entries | Should -Not -Contain '- None' -Because 'unmodified app2 should be identified for download' + ($entries | Where-Object { $_ -like '*app2*' }) | Should -Not -BeNullOrEmpty -Because 'app2 is unmodified' + ($entries | Where-Object { $_ -like '*app1*' }) | Should -BeNullOrEmpty -Because 'app1 is modified' + } + + AfterEach { + Remove-Item $baseFolder -Force -Recurse + } +} diff --git a/Tests/DownloadProjectDependencies.Test.ps1 b/Tests/DownloadProjectDependencies.Test.ps1 index 82f4854a4e..aa21cabbe7 100644 --- a/Tests/DownloadProjectDependencies.Test.ps1 +++ b/Tests/DownloadProjectDependencies.Test.ps1 @@ -1,3 +1,6 @@ +[Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSReviewUnusedParameter', '', Justification = 'Mock/callback parameters must match function signatures')] +param() + Get-Module TestActionsHelper | Remove-Module -Force Import-Module (Join-Path $PSScriptRoot 'TestActionsHelper.psm1') $errorActionPreference = "Stop"; $ProgressPreference = "SilentlyContinue"; Set-StrictMode -Version 2.0 @@ -527,3 +530,244 @@ Describe "DownloadProjectDependencies - Get-DependenciesFromInstallApps Tests" { { Get-DependenciesFromInstallApps -DestinationPath $downloadPath } | Should -Throw "*unknown secret 'missingSecret'*" } } + +Describe "DownloadProjectDependencies - Get-DependencyArtifactPattern Tests" { + BeforeEach { + [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseDeclaredVarsMoreThanAssignments', 'originalGitHubHeadRef', Justification = 'False positive.')] + $originalGitHubHeadRef = $ENV:GITHUB_HEAD_REF + [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseDeclaredVarsMoreThanAssignments', 'originalGitHubRefName', Justification = 'False positive.')] + $originalGitHubRefName = $ENV:GITHUB_REF_NAME + } + + AfterEach { + $ENV:GITHUB_HEAD_REF = $originalGitHubHeadRef + $ENV:GITHUB_REF_NAME = $originalGitHubRefName + } + + It 'Returns null when project is not in the dependency map' { + $ENV:GITHUB_HEAD_REF = '' + $ENV:GITHUB_REF_NAME = 'main' + + $projectDependencies = @{} + InModuleScope DownloadProjectDependencies -Parameters @{ Project = 'MyProject'; ProjectDependencies = $projectDependencies } { + param($Project, $ProjectDependencies) + $result = Get-DependencyArtifactPattern -Project $Project -ProjectDependencies $ProjectDependencies + $result | Should -BeNullOrEmpty + } + } + + It 'Returns null when project has an empty dependency array' { + $ENV:GITHUB_HEAD_REF = '' + $ENV:GITHUB_REF_NAME = 'main' + + $projectDependencies = @{ 'MyProject' = @() } + InModuleScope DownloadProjectDependencies -Parameters @{ Project = 'MyProject'; ProjectDependencies = $projectDependencies } { + param($Project, $ProjectDependencies) + $result = Get-DependencyArtifactPattern -Project $Project -ProjectDependencies $ProjectDependencies + $result | Should -BeNullOrEmpty + } + } + + It 'Returns correct pattern for a single dependency on the default branch' { + $ENV:GITHUB_HEAD_REF = '' + $ENV:GITHUB_REF_NAME = 'main' + + $projectDependencies = @{ 'App' = @('Base') } + InModuleScope DownloadProjectDependencies -Parameters @{ Project = 'App'; ProjectDependencies = $projectDependencies } { + param($Project, $ProjectDependencies) + $result = Get-DependencyArtifactPattern -Project $Project -ProjectDependencies $ProjectDependencies + $result | Should -Be '{Base-main-*Apps-*,Base-main-*Dependencies-*}' + } + } + + It 'Uses GITHUB_HEAD_REF over GITHUB_REF_NAME and sanitizes branch slashes' { + $ENV:GITHUB_HEAD_REF = 'feature/auth' + $ENV:GITHUB_REF_NAME = 'main' + + $projectDependencies = @{ 'App' = @('Base') } + InModuleScope DownloadProjectDependencies -Parameters @{ Project = 'App'; ProjectDependencies = $projectDependencies } { + param($Project, $ProjectDependencies) + $result = Get-DependencyArtifactPattern -Project $Project -ProjectDependencies $ProjectDependencies + $result | Should -Be '{Base-feature_auth-*Apps-*,Base-feature_auth-*Dependencies-*}' + } + } +} + +Describe "DownloadProjectDependencies - Get-DependencyArtifactPattern Advanced Tests" { + BeforeEach { + [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseDeclaredVarsMoreThanAssignments', 'originalHeadRef', Justification = 'False positive.')] + $originalHeadRef = $ENV:GITHUB_HEAD_REF + [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseDeclaredVarsMoreThanAssignments', 'originalRefName', Justification = 'False positive.')] + $originalRefName = $ENV:GITHUB_REF_NAME + + $ENV:GITHUB_HEAD_REF = '' + $ENV:GITHUB_REF_NAME = 'main' + } + + AfterEach { + $ENV:GITHUB_HEAD_REF = $originalHeadRef + $ENV:GITHUB_REF_NAME = $originalRefName + } + + It 'Returns pattern with 4 brace entries for two dependencies' { + $projectDependencies = @{ "App" = @("Base", "Common") } + InModuleScope DownloadProjectDependencies -Parameters @{ Project = "App"; ProjectDependencies = $projectDependencies } { + param($Project, $ProjectDependencies) + $result = Get-DependencyArtifactPattern -Project $Project -ProjectDependencies $ProjectDependencies + $result | Should -Be "{Base-main-*Apps-*,Base-main-*Dependencies-*,Common-main-*Apps-*,Common-main-*Dependencies-*}" + } + } + + It 'Returns pattern with 6 brace entries for three flattened transitive dependencies' { + $projectDependencies = @{ "App" = @("Base", "Common", "Core") } + InModuleScope DownloadProjectDependencies -Parameters @{ Project = "App"; ProjectDependencies = $projectDependencies } { + param($Project, $ProjectDependencies) + $result = Get-DependencyArtifactPattern -Project $Project -ProjectDependencies $ProjectDependencies + $result | Should -Not -BeNullOrEmpty + $result | Should -BeLike "*Base-main-*" + $result | Should -BeLike "*Common-main-*" + $result | Should -BeLike "*Core-main-*" + # 3 deps x 2 entries each = 6 entries = 5 commas + ($result.ToCharArray() | Where-Object { $_ -eq ',' }).Count | Should -Be 5 + } + } + + It 'Sanitizes forward slashes in project dependency names' { + $projectDependencies = @{ "App" = @("src/Common") } + InModuleScope DownloadProjectDependencies -Parameters @{ Project = "App"; ProjectDependencies = $projectDependencies } { + param($Project, $ProjectDependencies) + $result = Get-DependencyArtifactPattern -Project $Project -ProjectDependencies $ProjectDependencies + $result | Should -BeLike "*src_Common-main-*" + $result | Should -Not -BeLike "*src/Common*" + } + } + + It 'Sanitizes forward slashes in branch name from GITHUB_REF_NAME' { + $ENV:GITHUB_HEAD_REF = '' + $ENV:GITHUB_REF_NAME = 'release/v2.0' + $projectDependencies = @{ "App" = @("Base") } + InModuleScope DownloadProjectDependencies -Parameters @{ Project = "App"; ProjectDependencies = $projectDependencies } { + param($Project, $ProjectDependencies) + $result = Get-DependencyArtifactPattern -Project $Project -ProjectDependencies $ProjectDependencies + $result | Should -BeLike "*Base-release_v2.0-*" + $result | Should -Not -BeLike "*release/v2.0*" + } + } + + It 'Returns null when project is not in the dependencies map even if other projects exist' { + $projectDependencies = @{ "Other" = @("Base") } + InModuleScope DownloadProjectDependencies -Parameters @{ Project = "App"; ProjectDependencies = $projectDependencies } { + param($Project, $ProjectDependencies) + $result = Get-DependencyArtifactPattern -Project $Project -ProjectDependencies $ProjectDependencies + $result | Should -BeNullOrEmpty + } + } +} + +Describe "DownloadProjectDependencies - Resolve-DependencyFiles Tests" { + BeforeEach { + [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseDeclaredVarsMoreThanAssignments', 'testFolder', Justification = 'False positive.')] + $testFolder = (New-Item -ItemType Directory -Path (Join-Path $([System.IO.Path]::GetTempPath()) $([System.IO.Path]::GetRandomFileName()))).FullName + [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseDeclaredVarsMoreThanAssignments', 'destFolder', Justification = 'False positive.')] + $destFolder = (New-Item -ItemType Directory -Path (Join-Path $testFolder "dest")).FullName + } + + AfterEach { + if (Test-Path $testFolder) { + Remove-Item -Path $testFolder -Recurse -Force + } + } + + It 'Returns empty array for empty input' { + $result = Resolve-DependencyFiles -Dependencies @() -DestinationPath $destFolder + $result | Should -HaveCount 0 + } + + It 'Returns empty array for null input' { + $result = Resolve-DependencyFiles -Dependencies $null -DestinationPath $destFolder + $result | Should -HaveCount 0 + } + + It 'Passes through .app file paths unchanged' { + $appFile = Join-Path $testFolder "myapp.app" + Set-Content -Path $appFile -Value "fake app content" + + $result = @(Resolve-DependencyFiles -Dependencies @($appFile) -DestinationPath $destFolder) + + $result | Should -HaveCount 1 + $result[0] | Should -Be $appFile + } + + It 'Passes through test app markers for .app files' { + $appFile = Join-Path $testFolder "testapp.app" + Set-Content -Path $appFile -Value "fake test app" + + $result = @(Resolve-DependencyFiles -Dependencies @("($appFile)") -DestinationPath $destFolder) + + $result | Should -HaveCount 1 + $result[0] | Should -Be "($appFile)" + } + + It 'Extracts .app files from zip archives' { + $appFile = Join-Path $testFolder "Foundation_1.0.0.0.app" + Set-Content -Path $appFile -Value "fake app content" + $zipFile = Join-Path $testFolder "Foundation-main-Apps-1.0.0.0.zip" + Compress-Archive -Path $appFile -DestinationPath $zipFile + + $result = @(Resolve-DependencyFiles -Dependencies @($zipFile) -DestinationPath $destFolder) + + $result | Should -HaveCount 1 + $result[0] | Should -BeLike "*Foundation_1.0.0.0.app" + Test-Path $result[0] | Should -Be $true + } + + It 'Removes source zip after extraction' { + $appFile = Join-Path $testFolder "app.app" + Set-Content -Path $appFile -Value "fake" + $zipFile = Join-Path $testFolder "deps.zip" + Compress-Archive -Path $appFile -DestinationPath $zipFile + + Resolve-DependencyFiles -Dependencies @($zipFile) -DestinationPath $destFolder + + Test-Path $zipFile | Should -Be $false + } + + It 'Preserves test app markers when extracting zips' { + $appFile = Join-Path $testFolder "testlib.app" + Set-Content -Path $appFile -Value "fake test lib" + $zipFile = Join-Path $testFolder "TestApps-1.0.0.0.zip" + Compress-Archive -Path $appFile -DestinationPath $zipFile + + $result = @(Resolve-DependencyFiles -Dependencies @("($zipFile)") -DestinationPath $destFolder) + + $result | Should -HaveCount 1 + $result[0] | Should -Match '^\(' + $result[0] | Should -Match '\)$' + $result[0].Trim('()') | Should -BeLike "*testlib.app" + } + + It 'Handles mixed .app and .zip dependencies' { + $appFile = Join-Path $testFolder "direct.app" + Set-Content -Path $appFile -Value "direct app" + + $zippedApp = Join-Path $testFolder "zipped.app" + Set-Content -Path $zippedApp -Value "zipped app" + $zipFile = Join-Path $testFolder "deps.zip" + Compress-Archive -Path $zippedApp -DestinationPath $zipFile + + $result = @(Resolve-DependencyFiles -Dependencies @($appFile, $zipFile) -DestinationPath $destFolder) + + $result | Should -HaveCount 2 + $result[0] | Should -Be $appFile + $result[1] | Should -BeLike "*zipped.app" + } + + It 'Passes through non-existent paths unchanged' { + $nonExistentZipFile = Join-Path $testFolder "nonexistent-fake.zip" + + $result = @(Resolve-DependencyFiles -Dependencies @($nonExistentZipFile) -DestinationPath $destFolder) + + $result | Should -HaveCount 1 + $result[0] | Should -Be $nonExistentZipFile + } +} diff --git a/Tests/GetWorkflowMultiRunBranches.Test.ps1 b/Tests/GetWorkflowMultiRunBranches.Test.ps1 index 0059df4d0d..9f44e7fff3 100644 --- a/Tests/GetWorkflowMultiRunBranches.Test.ps1 +++ b/Tests/GetWorkflowMultiRunBranches.Test.ps1 @@ -39,6 +39,9 @@ Describe "GetWorkflowMultiRunBranches Action" { $env:Settings = "" $env:GITHUB_REF_NAME = "main" + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'fetch' } -MockWith { } + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'for-each-ref'} -MockWith { return @("origin/main") } + # Call the action script . (Join-Path $scriptRoot "$actionName.ps1") @@ -52,6 +55,7 @@ Describe "GetWorkflowMultiRunBranches Action" { $env:Settings = "" $env:GITHUB_REF_NAME = "main" + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'fetch' } -MockWith { } Mock -CommandName invoke-git -ParameterFilter { $command -eq 'for-each-ref'} -MockWith { return @("origin/test-branch", "origin/main", "origin/some-other-branch", "origin") } # Call the action script @@ -67,6 +71,7 @@ Describe "GetWorkflowMultiRunBranches Action" { $env:Settings = "" $env:GITHUB_REF_NAME = "main" + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'fetch' } -MockWith { } Mock -CommandName invoke-git -ParameterFilter { $command -eq 'for-each-ref'} -MockWith { return @("origin/test-branch", "origin/main", "origin/some-other-branch", "origin") } # Call the action script @@ -82,6 +87,7 @@ Describe "GetWorkflowMultiRunBranches Action" { $env:Settings = "" $env:GITHUB_REF_NAME = "main" + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'fetch' } -MockWith { } Mock -CommandName invoke-git -ParameterFilter { $command -eq 'for-each-ref'} -MockWith { return @("origin/HEAD", "origin/main", "origin/develop", "origin/feature-1") } # Call the action script with wildcard to get all branches @@ -101,6 +107,7 @@ Describe "GetWorkflowMultiRunBranches Action" { $env:Settings = "{ 'workflowSchedule': { 'includeBranches': [] } }" $env:GITHUB_REF_NAME = "default-branch" + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'fetch' } -MockWith { } Mock -CommandName invoke-git -ParameterFilter { $command -eq 'for-each-ref'} -MockWith { return @("origin/test-branch", "origin/main", "origin/default-branch", "origin") } # Call the action script @@ -116,6 +123,7 @@ Describe "GetWorkflowMultiRunBranches Action" { $env:Settings = "{ 'workflowSchedule': { 'includeBranches': ['test-branch'] } }" $env:GITHUB_REF_NAME = "main" + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'fetch' } -MockWith { } Mock -CommandName invoke-git -ParameterFilter { $command -eq 'for-each-ref'} -MockWith { return @("origin/test-branch", "origin/main", "origin/some-other-branch", "origin") } # Call the action script @@ -131,6 +139,7 @@ Describe "GetWorkflowMultiRunBranches Action" { $env:Settings = "{ 'workflowSchedule': { 'includeBranches': ['*branch*'] } }" $env:GITHUB_REF_NAME = "main" + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'fetch' } -MockWith { } Mock -CommandName invoke-git -ParameterFilter { $command -eq 'for-each-ref'} -MockWith { return @("origin/test-branch", "origin/main", "origin/some-other-branch", "origin") } # Call the action script @@ -141,4 +150,89 @@ Describe "GetWorkflowMultiRunBranches Action" { $outputValue | Should -Be "{`"branches`":[`"test-branch`",`"some-other-branch`"]}" } } + + Context 'workflow_call event' { + It 'Action sets the current branch as result when no branch patterns are specified' { + $env:GITHUB_EVENT_NAME = "workflow_call" + $env:Settings = "" + $env:GITHUB_REF_NAME = "main" + + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'fetch' } -MockWith { } + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'for-each-ref'} -MockWith { return @("origin/main") } + + # Call the action script + . (Join-Path $scriptRoot "$actionName.ps1") + + $outputName, $outputValue = (Get-Content $env:GITHUB_OUTPUT) -split '=' + $outputName | Should -Be "Result" + $outputValue | Should -Be "{`"branches`":[`"main`"]}" + } + + It 'Action sets the input branch as result when a branch pattern is specified' { + $env:GITHUB_EVENT_NAME = "workflow_call" + $env:Settings = "" + $env:GITHUB_REF_NAME = "main" + + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'fetch' } -MockWith { } + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'for-each-ref'} -MockWith { return @("origin/test-branch", "origin/main", "origin/some-other-branch", "origin") } + + # Call the action script + . (Join-Path $scriptRoot "$actionName.ps1") -includeBranches "test-branch" + + $outputName, $outputValue = (Get-Content $env:GITHUB_OUTPUT) -split '=' + $outputName | Should -Be "Result" + $outputValue | Should -Be "{`"branches`":[`"test-branch`"]}" + } + + It 'Action sets the input branch as result when a branch pattern with wild card is specified' { + $env:GITHUB_EVENT_NAME = "workflow_call" + $env:Settings = "" + $env:GITHUB_REF_NAME = "main" + + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'fetch' } -MockWith { } + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'for-each-ref'} -MockWith { return @("origin/test-branch", "origin/main", "origin/some-other-branch", "origin") } + + # Call the action script + . (Join-Path $scriptRoot "$actionName.ps1") -includeBranches "*branch*" + + $outputName, $outputValue = (Get-Content $env:GITHUB_OUTPUT) -split '=' + $outputName | Should -Be "Result" + $outputValue | Should -Be "{`"branches`":[`"test-branch`",`"some-other-branch`"]}" + } + + It 'Action filters out HEAD symbolic reference when using wildcard' { + $env:GITHUB_EVENT_NAME = "workflow_call" + $env:Settings = "" + $env:GITHUB_REF_NAME = "main" + + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'fetch' } -MockWith { } + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'for-each-ref'} -MockWith { return @("origin/HEAD", "origin/main", "origin/develop", "origin/feature-1") } + + # Call the action script with wildcard to get all branches + . (Join-Path $scriptRoot "$actionName.ps1") -includeBranches "*" + + $outputName, $outputValue = (Get-Content $env:GITHUB_OUTPUT) -split '=' + $outputName | Should -Be "Result" + # Verify that HEAD is not included in the result + $outputValue | Should -Not -Match "HEAD" + $outputValue | Should -Be "{`"branches`":[`"main`",`"develop`",`"feature-1`"]}" + } + } + + Context 'Parameter override tests' { + It 'workflowEventName parameter overrides GITHUB_EVENT_NAME environment variable' { + $env:GITHUB_EVENT_NAME = "schedule" + $env:Settings = "{ 'workflowSchedule': { 'includeBranches': ['schedule-branch'] } }" + $env:GITHUB_REF_NAME = "main" + + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'fetch' } -MockWith { } + Mock -CommandName invoke-git -ParameterFilter { $command -eq 'for-each-ref'} -MockWith { return @("origin/call-branch", "origin/schedule-branch", "origin/main") } + + # Parameter should override environment variable + . (Join-Path $scriptRoot "$actionName.ps1") -workflowEventName "workflow_call" -includeBranches "call-branch" + + $outputName, $outputValue = (Get-Content $env:GITHUB_OUTPUT) -split '=' + $outputValue | Should -Be "{`"branches`":[`"call-branch`"]}" + } + } } diff --git a/Workshop/ContinuousDelivery.md b/Workshop/ContinuousDelivery.md index d322be911f..8a337cbacd 100644 --- a/Workshop/ContinuousDelivery.md +++ b/Workshop/ContinuousDelivery.md @@ -98,6 +98,12 @@ For detailed step-by-step instructions, configuration examples, and troubleshoot Custom delivery will be handled in an advanced part of this workshop later. +## Important Note: Automatic Skip Behavior + +The Deliver step/action is skipped when no app artifacts are available. + +This skip behavior prevents delivery errors and ensures that delivery targets are only invoked when there are actual artifacts to deliver. You'll see the Deliver step/action appear as skipped in the workflow summary when this occurs. + OK, so **CD** stands for **Continuous Delivery**, I thought it was **Continuous Deployment**? Well, it is actually both, so let's talk about **Continuous Deployment**... ______________________________________________________________________ diff --git a/Workshop/ContinuousDeployment.md b/Workshop/ContinuousDeployment.md index 5c4e60d256..3fd4475577 100644 --- a/Workshop/ContinuousDeployment.md +++ b/Workshop/ContinuousDeployment.md @@ -86,6 +86,12 @@ Paste the value from the clipboard into the "Value" field of the **AuthContext** AL-Go can also be setup for custom deployment when you want to deploy to non-SaaS environments. More about this in the advanced section. +## Important Note: Automatic Skip Behavior + +Deployment jobs automatically skip execution when no app artifacts are available. + +This skip behavior prevents deployment errors and ensures that environments are only targeted when there are actual artifacts to deploy. You'll see the deployment step appear as skipped in the workflow summary when this occurs. + This section was about Continuous Deployment, but you might not want to deploy to production environments continuously - how can we publish to production on demand? ______________________________________________________________________ diff --git a/e2eTests/e2eTestHelper.psm1 b/e2eTests/e2eTestHelper.psm1 index 68a4c3b155..66addb2460 100644 --- a/e2eTests/e2eTestHelper.psm1 +++ b/e2eTests/e2eTestHelper.psm1 @@ -1,8 +1,8 @@ $githubOwner = "githubOwner" $token = "DefaultToken" $defaultRepository = "repo" -$defaultApplication = "22.0.0.0" -$defaultRuntime = "10.0" +$defaultApplication = "27.0.0.0" +$defaultRuntime = "16.0" $defaultPublisher = "MS Test" $lastTokenRefresh = 0 @@ -354,9 +354,9 @@ function CreateNewAppInFolder { "name" = $name "version" = $version "publisher" = $publisher + "runtime" = $runtime "dependencies" = $dependencies "application" = $application - "runtime" = $runtime "idRanges" = @( @{ "from" = $objID; "to" = $objID } ) "resourceExposurePolicy" = @{ "allowDebugging" = $true; "allowDownloadingSource" = $true; "includeSourceInSymbolFile" = $true } } diff --git a/e2eTests/scenarios/WorkspaceCompilation/runtest.ps1 b/e2eTests/scenarios/WorkspaceCompilation/runtest.ps1 new file mode 100644 index 0000000000..17335715e9 --- /dev/null +++ b/e2eTests/scenarios/WorkspaceCompilation/runtest.ps1 @@ -0,0 +1,151 @@ +[Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSAvoidGlobalVars', '', Justification = 'Global vars used for local test execution only.')] +[Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSReviewUnusedParameter', '', Justification = 'All scenario tests have equal parameter set.')] +[Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSAvoidUsingPlainTextForPassword', '', Justification = 'Secrets are transferred as plain text.')] +Param( + [switch] $github, + [switch] $linux, + [string] $githubOwner = $global:E2EgithubOwner, + [string] $repoName = [System.IO.Path]::GetFileNameWithoutExtension([System.IO.Path]::GetTempFileName()), + [string] $e2eAppId, + [string] $e2eAppKey, + [string] $algoauthapp = ($global:SecureALGOAUTHAPP | Get-PlainText), + [string] $pteTemplate = $global:pteTemplate, + [string] $appSourceTemplate = $global:appSourceTemplate, + [string] $adminCenterApiCredentials = ($global:SecureadminCenterApiCredentials | Get-PlainText), + [string] $azureCredentials = ($global:SecureAzureCredentials | Get-PlainText), + [string] $githubPackagesToken = ($global:SecureGitHubPackagesToken | Get-PlainText) +) + +Write-Host -ForegroundColor Yellow @' +# __ __ _ ____ _ _ _ _ +# \ \ / /__ _ __| | _____ _ __ __ _ ___ ___ / ___|___ _ __ ___ _ __ (_) | __ _| |_(_) ___ _ __ +# \ \ /\ / / _ \| '__| |/ / __| '_ \ / _` |/ __/ _ \ | | / _ \| '_ ` _ \| '_ \| | |/ _` | __| |/ _ \| '_ \ +# \ V V / (_) | | | <\__ \ |_) | (_| | (_| __/ | |__| (_) | | | | | | |_) | | | (_| | |_| | (_) | | | | +# \_/\_/ \___/|_| |_|\_\___/ .__/ \__,_|\___\___| \____\___/|_| |_| |_| .__/|_|_|\__,_|\__|_|\___/|_| |_| +# |_| |_| +# +# This test tests the following scenario: +# +# - Create a new repository based on the PTE template with two projects (P1 and P2) +# - P1 has 2 apps: app1 (base) and app2 (depends on app1) +# - P2 has 2 apps: app3 and app4 (both depend on P1/app1 — cross-project dependencies, can compile in parallel) +# - Enable workspaceCompilation with parallelism and useProjectDependencies in repo settings +# - P1 has doNotPublishApps enabled (compile-only, no container) +# - P2 publishes apps (full pipeline with testing) +# - Run the "CI/CD" workflow +# - Verify P1 compiles both apps successfully +# - Verify P2 compiles app3 and app4 using P1's compiled app1 as a dependency +# - Cleanup repositories +# +'@ + +$errorActionPreference = "Stop"; $ProgressPreference = "SilentlyContinue"; Set-StrictMode -Version 2.0 +$prevLocation = Get-Location + +if ($linux) { + Write-Host 'Workspace compilation currently doesn''t work on Linux runners, so this test is only run on Windows.' + exit +} + + +Remove-Module e2eTestHelper -ErrorAction SilentlyContinue +Import-Module (Join-Path $PSScriptRoot "..\..\e2eTestHelper.psm1") -DisableNameChecking + +$repository = "$githubOwner/$repoName" +$branch = "main" + +$template = "https://github.com/$pteTemplate" + +# Login +SetTokenAndRepository -github:$github -githubOwner $githubOwner -appId $e2eAppId -appKey $e2eAppKey -repository $repository + +if ($linux) { + $githubRunner = "ubuntu-latest" + $githubRunnerShell = "pwsh" +} +else { + $githubRunner = "windows-latest" + $githubRunnerShell = "powershell" +} + +# Create a multi-project repo with cross-project dependencies +# P1: app1 (base), app2 (depends on app1) — compile-only (doNotPublishApps) +# P2: app3 and app4 (both depend on P1/app1, can compile in parallel) — full pipeline (publish + test) +CreateAlGoRepository ` + -github:$github ` + -linux:$linux ` + -template $template ` + -repository $repository ` + -branch $branch ` + -projects @('P1', 'P2') ` + -addRepoSettings @{ + "workspaceCompilation" = @{ + "enabled" = $true + "parallelism" = -1 + } + "useProjectDependencies" = $true + "artifact" = "////nextmajor" + "githubRunner" = $githubRunner + "githubRunnerShell" = $githubRunnerShell + } ` + -contentScript { + Param([string] $path) + # P1: compile-only, no container needed + Add-PropertiesToJsonFile -path (Join-Path $path 'P1\.AL-Go\settings.json') -properties @{ + "country" = "w1" + "doNotPublishApps" = $true + "doNotRunTests" = $true + } + + # P2: full pipeline + Add-PropertiesToJsonFile -path (Join-Path $path 'P2\.AL-Go\settings.json') -properties @{ + "country" = "w1" + } + + # P1/app1 (base app) + $script:id1 = CreateNewAppInFolder -folder (Join-Path $path 'P1') -name 'app1' -objID 50001 + + # P1/app2 (depends on app1 within same project) + $script:id2 = CreateNewAppInFolder -folder (Join-Path $path 'P1') -name 'app2' -objID 50002 -dependencies @( + @{ "id" = $script:id1; "name" = "app1"; "publisher" = (GetDefaultPublisher); "version" = "1.0.0.0" } + ) + + # P2/app3 (depends on P1/app1 — cross-project dependency) + $script:id3 = CreateNewAppInFolder -folder (Join-Path $path 'P2') -name 'app3' -objID 50003 -dependencies @( + @{ "id" = $script:id1; "name" = "app1"; "publisher" = (GetDefaultPublisher); "version" = "1.0.0.0" } + ) + + # P2/app4 (depends on P1/app1 — cross-project dependency, can compile in parallel with app3) + $script:id4 = CreateNewAppInFolder -folder (Join-Path $path 'P2') -name 'app4' -objID 50004 -dependencies @( + @{ "id" = $script:id1; "name" = "app1"; "publisher" = (GetDefaultPublisher); "version" = "1.0.0.0" } + ) + } + +$repoPath = (Get-Location).Path + +# Run Update AL-Go System Files with direct commit +RunUpdateAlGoSystemFiles -directCommit -wait -templateUrl $template -ghTokenWorkflow $algoauthapp -repository $repository -branch $branch | Out-Null + +# Wait for CI/CD to complete +Start-Sleep -Seconds 60 +$runs = invoke-gh api /repos/$repository/actions/runs -silent -returnValue | ConvertFrom-Json +$run = $runs.workflow_runs | Select-Object -First 1 +WaitWorkflow -repository $repository -runid $run.id + +# Check P1 artifacts — 2 apps compiled (compile-only, no publishing) +Test-ArtifactsFromRun -runid $run.id -folder '.artifacts' -expectedArtifacts @{ + "P1-main-Apps-*.app" = 2 + "P1-main-Apps-*_app1_1.0.2.0.app" = 1 + "P1-main-Apps-*_app2_1.0.2.0.app" = 1 +} + +# Check P2 artifacts — 2 apps compiled in parallel (using P1/app1 as dependency) +Test-ArtifactsFromRun -runid $run.id -folder '.artifacts' -expectedArtifacts @{ + "P2-main-Apps-*.app" = 2 + "P2-main-Apps-*_app3_1.0.2.0.app" = 1 + "P2-main-Apps-*_app4_1.0.2.0.app" = 1 +} + +# Cleanup repositories +Set-Location $prevLocation +RemoveRepository -repository $repository -path $repoPath