Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
160 changes: 160 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
# Playwright Workspaces sample (GitHub CI + Azure static website)

This repository demonstrates running Playwright tests in GitHub Actions using Playwright Workspaces, publishing HTML reports to an Azure Storage static website, and maintaining a small JSON feed that the lightweight `index.html` viewer consumes.

At a high level:

- GitHub Actions (see `.github/workflows/playwright-workspaces.yml`) runs Playwright tests (configured in `playwright.service.config.ts`).
- The workflow assembles an HTML report (Playwright report), creates a `meta.json` describing the run, uploads the report to an Azure Storage static website (`$web`), and merges a new feed entry into `feed.json`.
- `index.html` in this repo is a viewer that reads `feed.json` and displays available test run reports (it expects `meta.json` and related report content to be present under the run folder on the static website).

This README documents how the pieces fit together and how to configure Azure and GitHub Secrets to run the sample.

---

## Files of interest

- `.github/workflows/playwright-workspaces.yml` — the GitHub Actions pipeline that runs tests and uploads the HTML report and `meta.json` to Azure. It also updates `feed.json`.
- `playwright.service.config.ts` — Playwright configuration used by the sample tests (the repo includes a Playwright test in `tests/`).
- `index.html` — static viewer that loads `feed.json` and renders the list of reports (and pulls `data/report.json` from each report URL to show stats when available).

---

## How the workflow works (high level)

1. On `push` / `pull_request`, the workflow checks out the repo and installs dependencies.
2. It runs Playwright tests (the workflow uses `npx playwright test -c playwright.service.config.ts` and expects the Playwright Service environment variables if required).

Note: the workflow uses the Playwright `blob` reporter and merges it into an HTML report in CI so an HTML report is produced and uploaded.

3. The job ensures a minimal HTML exists for `playwright-report/index.html` and then builds `meta.json` containing metadata about the run (title, subtitle, dates, commitSha, runId, reportUrl, outcome).
4. It uploads the contents of `./playwright-report` to Azure Storage static website under a run directory (e.g. `$web/run-<id>-<attempt>/...`).
5. The workflow writes `meta.json` and then merges a new feed entry into the global `feed.json` (also stored in the `$web` static website container) — `index.html` pulls this feed.

`index.html` then fetches `feed.json` at runtime, lists entries, and loads `data/report.json` from each report URL to show test stats (if present).

---

## Azure setup (short guide)

The workflow uploads generated reports and metadata into the storage account's `$web` container (static website hosting). There are two primary setup tasks:

1. Create an Azure Storage account and enable static website hosting
2. Create a service principal for `azcopy` to authenticate from GitHub Actions

Below are example `az` commands to provision resources (replace placeholders):

```bash
# create resource group
az group create --name my-playwright-rg --location eastus

# create a StorageV2 account
az storage account create \
--resource-group my-playwright-rg \
--name <STORAGE_ACCOUNT_NAME> \
--location eastus \
--sku Standard_LRS \
--kind StorageV2

# enable static website and set index / error docs
az storage blob service-properties update \
--account-name <STORAGE_ACCOUNT_NAME> \
--static-website --index-document index.html --404-document 404.html

# get the static website endpoint (use this to preview uploads)
az storage account show -n <STORAGE_ACCOUNT_NAME> -g my-playwright-rg --query primaryEndpoints.web -o tsv
```

Create a service principal used by `azcopy` in the Actions runner. Grant it at least the Storage Blob Data Contributor role on the storage account (scoping more narrowly is recommended):

```bash
az ad sp create-for-rbac \
--name "azcopy-playwright-ci" \
--role "Storage Blob Data Contributor" \
--scopes "/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/my-playwright-rg/providers/Microsoft.Storage/storageAccounts/<STORAGE_ACCOUNT_NAME>"
```

The above command prints JSON with `appId`, `password`, `tenant`. Save those and add to GitHub secrets.

---

## Playwright Workspaces resource

Create your Playwright Workspaces resource by following the Azure quickstart:

https://learn.microsoft.com/en-us/azure/app-testing/playwright-workspaces/quickstart-automate-end-to-end-testing

After the workspace is created, open the workspace in the Azure portal and copy the region-specific service endpoint from the **Get Started** page — this is the `PLAYWRIGHT_SERVICE_URL` you will store in GitHub Secrets.

---

## Secrets and where to get them

Below is a quick mapping of the GitHub secrets used by the workflow and how to obtain each value.

This sample uses Playwright Workspaces access token authentication. You only need the `PLAYWRIGHT_SERVICE_URL` and `PLAYWRIGHT_SERVICE_ACCESS_TOKEN` for the test runs. The azcopy-related secrets are required for uploading reports to the static website (see below).

- `PLAYWRIGHT_SERVICE_URL` — workspace service endpoint (Get Started page in the workspace)
- Portal: workspace → Get Started → copy the endpoint URL (region-specific)

- `PLAYWRIGHT_SERVICE_ACCESS_TOKEN` — Playwright Workspaces access token (used in this sample)
- Portal: workspace → Access / API tokens → Create a token, copy the generated token value and add it to GitHub secrets as `PLAYWRIGHT_SERVICE_ACCESS_TOKEN`.
- Treat this token like a password: do not check it into source control.

- `STORAGE_ACCOUNT_NAME` — Azure Storage account name used for hosting static website
- Portal: Storage account → Overview → Account name
- CLI: output from `az storage account create` or `az storage account list`

- `AZCOPY_SPA_APPLICATION_ID`, `AZCOPY_SPA_CLIENT_SECRET`, `AZCOPY_TENANT_ID` — credentials for `azcopy` (if you are using a service principal to upload)
- Create a service principal scoped to your storage account and save the `appId` and `password` values:
```bash
az ad sp create-for-rbac --name "azcopy-playwright-ci" \
--role "Storage Blob Data Contributor" \
--scopes "/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RG>/providers/Microsoft.Storage/storageAccounts/<STORAGE_ACCOUNT_NAME>"
```
- Use the returned `appId` as `AZCOPY_SPA_APPLICATION_ID`
- Use the returned `password` as `AZCOPY_SPA_CLIENT_SECRET`
- Use the returned `tenant` as `AZCOPY_TENANT_ID`

Notes:
- For best security, use Microsoft Entra ID + GitHub OIDC (configure a federated identity credential on the App Registration or create a user-assigned managed identity) instead of long-lived client secrets where possible. The workflow supports Entra ID authentication via the `azure/login` action (see the quickstart link).
- The Playwright workspace service endpoint (the `PLAYWRIGHT_SERVICE_URL`) is region-specific and must match the workspace you created.

---

## GitHub secrets and repository variables

Add the following repository secrets (Repository settings → Secrets → Actions). These are referenced by the workflow:

- `STORAGE_ACCOUNT_NAME` — the storage account name used for static website uploads
- `AZCOPY_SPA_APPLICATION_ID` — the service principal app id (value `appId`)
- `AZCOPY_SPA_CLIENT_SECRET` — the service principal password (value `password`)
- `AZCOPY_TENANT_ID` — the tenant id (value `tenant`)
- `PLAYWRIGHT_SERVICE_URL` — (optional) Playwright Service URL if you run tests against a Playwright test service
- `PLAYWRIGHT_SERVICE_ACCESS_TOKEN` — (optional) Playwright Service token

The workflow uses `azcopy` with service principal env vars (set as above). It also uses `jq` in the runner to build `meta.json` and to merge `feed.json` entries.

---

## Feed format and `index.html`

- The workflow writes a `meta.json` per run (contains title, date, updated, branch, commitSha, reportUrl, outcome, runId etc.).
- The workflow merges a feed entry into `feed.json` at the repository static site root. `index.html` fetches `feed.json` and renders entries; it will fetch `data/report.json` from each report URL to show stats in the viewer when available.

If you are adapting this example, keep `meta.json` fields the same (title, updated, generatedAt, reportUrl, outcome) so `index.html` can show the run properly.

---

## Troubleshooting / tips

- Ensure the service principal has `Storage Blob Data Contributor` role on the storage account resource (otherwise `azcopy` will get 403).
- The workflow ensures `meta.json` is uploaded with `--cache-control "no-cache, no-store, must-revalidate"` to reduce stale content issues.
- If `index.html` shows "No matching reports", check that `feed.json` is present at the static website root (`$web/feed.json`) and that entries have `reportUrl` pointing to the correct run folder.
- Use `az storage account show -n <ACCOUNT> -g <RG> --query primaryEndpoints.web -o tsv` to find the static website endpoint.

---

## License

This repository is a small sample. Use the code as you see fit.
60 changes: 52 additions & 8 deletions index.html
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,12 @@
.chip.warn{background:var(--warn-bg);border-color:var(--warn-bd);color:var(--warn-fg)}
.chip.mono{font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", monospace}

/* Status icon placed left of each title */
.statusIcon{display:inline-flex;align-items:center;justify-content:center;width:20px;height:20px;border-radius:6px;font-size:12px;line-height:1;border:1px solid var(--chip-bd);color:var(--muted);background:transparent}
.statusIcon.ok{color:var(--ok-fg);border-color:var(--ok-bd);background:linear-gradient(transparent, rgba(46,160,67,0.04))}
.statusIcon.err{color:var(--err-fg);border-color:var(--err-bd);background:linear-gradient(transparent, rgba(248,81,73,0.04))}
.statusIcon.warn{color:var(--warn-fg);border-color:var(--warn-bd);background:linear-gradient(transparent, rgba(210,153,34,0.04))}

.counts{color:var(--muted); font-size:.9em; margin-top:4px}
</style>

Expand Down Expand Up @@ -136,6 +142,8 @@ <h1>Playwright Reports</h1>
const branches = [...new Set(items.map(i => i.branch).filter(Boolean))].sort();
const branchSel = document.getElementById('branchFilter');
branches.forEach(b => { const o=document.createElement('option'); o.value=b; o.textContent=b; branchSel.appendChild(o); });
// Default to the 'main' branch when available
if (branches.includes('main')) branchSel.value = 'main';

const searchBox = document.getElementById('searchBox');
const ul = document.getElementById('list');
Expand Down Expand Up @@ -220,8 +228,27 @@ <h1>Playwright Reports</h1>
}

function safeUTC(ts){
const d = ts ? new Date(ts) : null;
return (d && !isNaN(d)) ? d.toUTCString() : '—';
if (!ts) return '—';
const d = (ts instanceof Date) ? ts : new Date(ts);
if (!d || isNaN(d)) return '—';
const now = new Date();
const diff = Math.floor((now - d) / 1000); // seconds
if (diff < 5) return 'now';
if (diff < 60) return diff === 1 ? '1 second ago' : `${diff} seconds ago`;
const mins = Math.floor(diff / 60);
if (mins < 60) return mins === 1 ? '1 minute ago' : `${mins} minutes ago`;
const hours = Math.floor(mins / 60);
if (hours < 24) return hours === 1 ? '1 hour ago' : `${hours} hours ago`;
const days = Math.floor(hours / 24);
if (days === 1) return 'yesterday';
if (days < 7) return `${days} days ago`;
if (days < 30) {
const weeks = Math.floor(days / 7);
return `${weeks} week${weeks === 1 ? '' : 's'} ago`;
}
const months = Math.floor(days / 30);
if (months < 12) return `${months} month${months === 1 ? '' : 's'} ago`;
return d.toUTCString();
}

async function render(){
Expand All @@ -232,12 +259,27 @@ <h1>Playwright Reports</h1>
const stats = await fetchStatsFor(d);
const statusCls = classifyStatus(stats, d.outcome);

// Decide a small icon for the row: tick for pass, X for failed, warning for flaky
const icon = statusCls === 'ok'
? `<span class="statusIcon ok" title="Passed">✓</span>`
: statusCls === 'err'
? `<span class="statusIcon err" title="Failed">✗</span>`
: statusCls === 'warn'
? `<span class="statusIcon warn" title="Flaky">⚠</span>`
: `<span class="statusIcon" title="Unknown">?</span>`;

// Filtering
if (branch && d.branch !== branch) return null;
if (statusFilter && statusCls !== statusFilter) return null;

// Title + links from new feed fields
const titleText = d.title || (d.repo ? `${d.repo}@${d.shortSha||''}` : 'Report');
// Truncate display title to at most 72 characters (including ellipsis)
const MAX_TITLE_LEN = 72;
let displayTitle = (typeof titleText === 'string') ? titleText : String(titleText || '');
if (displayTitle.length > MAX_TITLE_LEN) {
displayTitle = displayTitle.slice(0, MAX_TITLE_LEN - 1).trimEnd() + '…';
}
const reportHref = d.reportUrl || d.url || '#';

// Commit link (build if not provided)
Expand All @@ -247,6 +289,11 @@ <h1>Playwright Reports</h1>

const ts = d.updated || d.date || '';
const when = safeUTC(ts);
let fullTs = '—';
if (ts) {
const maybe = new Date(ts);
if (!isNaN(maybe)) fullTs = maybe.toUTCString();
}

// Search text
const text = [
Expand All @@ -273,16 +320,13 @@ <h1>Playwright Reports</h1>
<li>
<div class="row">
<div class="title">
<a class="folder" href="${reportHref}" rel="noopener noreferrer">${titleText}</a>
${icon}
<a class="folder" href="${reportHref}" rel="noopener noreferrer" title="${titleText}">${displayTitle}</a>
${commitHref && hashText ? `<a class="chip mono" style="padding:1px 6px" href="${commitHref}" rel="noopener noreferrer">${hashText}</a>` : ''}
<span class="ts">${when}</span>
<span class="ts" title="${fullTs}">${when}</span>
</div>
<div class="badges">${badges}</div>
</div>
<div class="counts">
${d.subtitle ? `<a href="${reportHref}" rel="noopener noreferrer">${d.subtitle}</a>` : `<a href="${reportHref}" rel="noopener noreferrer">Open report</a>`}
${counts ? counts : ''}
</div>
</li>`;
}));

Expand Down
69 changes: 6 additions & 63 deletions playwright.config.ts
Original file line number Diff line number Diff line change
@@ -1,85 +1,28 @@
import { defineConfig, devices } from '@playwright/test';

/**
* Read environment variables from file.
* https://github.com/motdotla/dotenv
*/
import dotenv from 'dotenv';
import path from 'path';

dotenv.config({ path: path.resolve(__dirname, '.env') });

/**
* See https://playwright.dev/docs/test-configuration.
*/
export default defineConfig({
testDir: './tests',
/* Run tests in files in parallel */
fullyParallel: true,
/* Fail the build on CI if you accidentally left test.only in the source code. */
forbidOnly: !!process.env.CI,
/* Retry on CI only */
retries: process.env.CI ? 2 : 0,
/* Opt out of parallel tests on CI. */
workers: process.env.CI ? 1 : undefined,
reporter: [
['list'], // any console reporter you like
['blob', { outputDir: 'blob-report' }],// raw results (robust on failures)
// Optional: also write HTML during the run (nice for local)
['blob', { outputDir: 'blob-report' }], // keep this reporter for CI
// ['html', { open: 'never', outputFolder: 'playwright-report' }],
],
/* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
use: {
/* Base URL to use in actions like `await page.goto('/')`. */
// baseURL: 'http://localhost:3000',

/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
trace: 'on-first-retry',
video: 'retain-on-failure',
screenshot: 'on'
},

/* Configure projects for major browsers */
projects: [
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},

// {
// name: 'firefox',
// use: { ...devices['Desktop Firefox'] },
// },

// {
// name: 'webkit',
// use: { ...devices['Desktop Safari'] },
// },

/* Test against mobile viewports. */
// {
// name: 'Mobile Chrome',
// use: { ...devices['Pixel 5'] },
// },
// {
// name: 'Mobile Safari',
// use: { ...devices['iPhone 12'] },
// },

/* Test against branded browsers. */
// {
// name: 'Microsoft Edge',
// use: { ...devices['Desktop Edge'], channel: 'msedge' },
// },
// {
// name: 'Google Chrome',
// use: { ...devices['Desktop Chrome'], channel: 'chrome' },
// },
{ name: 'chromium', use: { ...devices['Desktop Chrome'] } },
{ name: 'firefox', use: { ...devices['Desktop Firefox'] } },
{ name: 'webkit', use: { ...devices['Desktop Safari'] } },
{ name: 'Mobile Chrome', use: { ...devices['Pixel 5'] } },
],

/* Run your local dev server before starting the tests */
// webServer: {
// command: 'npm run start',
// url: 'http://localhost:3000',
// reuseExistingServer: !process.env.CI,
// },
});
3 changes: 0 additions & 3 deletions playwright.service.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,6 @@ import { getServiceConfig, ServiceOS, ServiceAuth } from '@azure/playwright';
import { DefaultAzureCredential } from '@azure/identity';
import config from './playwright.config';

/* Learn more about service configuration at https://aka.ms/pww/docs/config */
// generateGuid removed; use Node's crypto.randomUUID()

export default defineConfig(
config,
getServiceConfig(config, {
Expand Down
Loading
Loading