Skip to content

Use released compile-extension package for WASM builds #402

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
adamziel wants to merge 1 commit into
base: trunk
Choose a base branch
from
Draft

Use released compile-extension package for WASM builds #402

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 2 additions & 89 deletions .github/workflows/publish-wasm-extension-artifact.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,6 @@ on:
- '.github/workflows/publish-wasm-extension-artifact.yml'
workflow_dispatch:
inputs:
playground-ref:
description: 'wordpress-playground branch/tag/SHA to build against'
required: false
default: 'trunk'
retention-days:
description: 'Actions artifact retention in days'
required: false
Expand All @@ -26,47 +22,13 @@ concurrency:
cancel-in-progress: true

jobs:
base-image:
name: Build shared Playground base image
runs-on: ubuntu-latest
timeout-minutes: 20

steps:
- name: Check out wordpress-playground
uses: actions/checkout@v4
with:
repository: WordPress/wordpress-playground
ref: ${{ github.event.inputs.playground-ref || 'trunk' }}
path: wordpress-playground
sparse-checkout: |
packages/php-wasm/compile/base-image

- name: Build Playground base image
run: |
docker build \
-f wordpress-playground/packages/php-wasm/compile/base-image/Dockerfile \
--tag playground-php-wasm:base \
wordpress-playground/packages/php-wasm/compile/base-image

- name: Save Playground base image
run: docker save --output "${RUNNER_TEMP}/playground-php-wasm-base.tar" playground-php-wasm:base

- name: Upload Playground base image
uses: actions/upload-artifact@v4
with:
name: playground-php-wasm-base-image
path: ${{ runner.temp }}/playground-php-wasm-base.tar
if-no-files-found: error
retention-days: 1

build:
name: Build wp_mysql_parser.so (PHP ${{ matrix.php }})
needs: base-image
runs-on: ubuntu-latest
timeout-minutes: 60
strategy:
fail-fast: false
max-parallel: 6
max-parallel: 1
matrix:
# The Rust WASM path uses ext-php-rs 0.15, which depends on PHP 8
# Zend APIs. PHP 7.4 cannot be added here by extending the matrix.
Expand All @@ -78,66 +40,17 @@ jobs:
with:
path: sqlite-database-integration

- name: Check out wordpress-playground
uses: actions/checkout@v4
with:
repository: WordPress/wordpress-playground
ref: ${{ github.event.inputs.playground-ref || 'trunk' }}
path: wordpress-playground
sparse-checkout-cone-mode: false
sparse-checkout: |
/package.json
/package-lock.json
/nx.json
/tsconfig.base.json
/packages/meta/
/packages/nx-extensions/
/packages/php-wasm/cli-util/
/packages/php-wasm/compile-extension/
/packages/php-wasm/compile/
/packages/php-wasm/fs-journal/
/packages/php-wasm/logger/
/packages/php-wasm/node/
/packages/php-wasm/node-builds/8-0/
/packages/php-wasm/node-builds/8-1/
/packages/php-wasm/node-builds/8-2/
/packages/php-wasm/node-builds/8-3/
/packages/php-wasm/node-builds/8-4/
/packages/php-wasm/node-builds/8-5/
/packages/php-wasm/progress/
/packages/php-wasm/scopes/
/packages/php-wasm/stream-compression/
/packages/php-wasm/universal/
/packages/php-wasm/util/

- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: '24'
cache: 'npm'
cache-dependency-path: wordpress-playground/package-lock.json

- name: Download Playground base image
uses: actions/download-artifact@v4
with:
name: playground-php-wasm-base-image
path: ${{ runner.temp }}/playground-base-image

- name: Load Playground base image
run: docker load --input "${RUNNER_TEMP}/playground-base-image/playground-php-wasm-base.tar"

- name: Install Playground deps
working-directory: wordpress-playground
run: npm ci --ignore-scripts

- name: Build wp_mysql_parser side module
working-directory: sqlite-database-integration/packages/php-ext-wp-mysql-parser/wasm-spike
env:
PHP_VERSION: ${{ matrix.php }}
ASYNC_MODE: jspi
COMPILE_EXTENSION_PACKAGE: '@php-wasm/compile-extension@3.1.27'
PLAYGROUND_REPO: ${{ github.workspace }}/wordpress-playground
SKIP_BASE_IMAGE_BUILD: '1'
COMPILE_EXTENSION_PACKAGE: '@php-wasm/compile-extension@3.1.28'
run: bash build-in-docker-rust.sh

- name: Verify side module exists
Expand Down
103 changes: 9 additions & 94 deletions .github/workflows/wasm-spike.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,54 +12,15 @@ on:
- 'packages/php-ext-wp-mysql-parser/**'
- '.github/workflows/wasm-spike.yml'
workflow_dispatch:
inputs:
playground-ref:
description: 'wordpress-playground branch/tag/SHA to build against'
required: false
default: 'trunk'

jobs:
base-image:
name: Build shared Playground base image
runs-on: ubuntu-latest
timeout-minutes: 20

steps:
- name: Check out wordpress-playground
uses: actions/checkout@v4
with:
repository: WordPress/wordpress-playground
ref: ${{ github.event.inputs.playground-ref || 'trunk' }}
path: wordpress-playground
sparse-checkout: |
packages/php-wasm/compile/base-image

- name: Build Playground base image
run: |
docker build \
-f wordpress-playground/packages/php-wasm/compile/base-image/Dockerfile \
--tag playground-php-wasm:base \
wordpress-playground/packages/php-wasm/compile/base-image

- name: Save Playground base image
run: docker save --output "${RUNNER_TEMP}/playground-php-wasm-base.tar" playground-php-wasm:base

- name: Upload Playground base image
uses: actions/upload-artifact@v4
with:
name: playground-php-wasm-base-image
path: ${{ runner.temp }}/playground-php-wasm-base.tar
if-no-files-found: error
retention-days: 1

build-and-load:
name: Build wp_mysql_parser.so and load it in Playground (PHP ${{ matrix.php }})
needs: base-image
name: Build wp_mysql_parser.so and load it in PHP.wasm (PHP ${{ matrix.php }})
runs-on: ubuntu-latest
timeout-minutes: 60
strategy:
fail-fast: false
max-parallel: 6
max-parallel: 1
matrix:
# The Rust WASM path uses ext-php-rs 0.15, which depends on PHP 8
# Zend APIs. PHP 7.4 cannot be added here by extending the matrix.
Expand All @@ -72,66 +33,17 @@ jobs:
with:
path: sqlite-database-integration

- name: Check out wordpress-playground
uses: actions/checkout@v4
with:
repository: WordPress/wordpress-playground
ref: ${{ github.event.inputs.playground-ref || 'trunk' }}
path: wordpress-playground
sparse-checkout-cone-mode: false
sparse-checkout: |
/package.json
/package-lock.json
/nx.json
/tsconfig.base.json
/packages/meta/
/packages/nx-extensions/
/packages/php-wasm/cli-util/
/packages/php-wasm/compile-extension/
/packages/php-wasm/compile/
/packages/php-wasm/fs-journal/
/packages/php-wasm/logger/
/packages/php-wasm/node/
/packages/php-wasm/node-builds/8-0/
/packages/php-wasm/node-builds/8-1/
/packages/php-wasm/node-builds/8-2/
/packages/php-wasm/node-builds/8-3/
/packages/php-wasm/node-builds/8-4/
/packages/php-wasm/node-builds/8-5/
/packages/php-wasm/progress/
/packages/php-wasm/scopes/
/packages/php-wasm/stream-compression/
/packages/php-wasm/universal/
/packages/php-wasm/util/

- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: '24'
cache: 'npm'
cache-dependency-path: wordpress-playground/package-lock.json

- name: Download Playground base image
uses: actions/download-artifact@v4
with:
name: playground-php-wasm-base-image
path: ${{ runner.temp }}/playground-base-image

- name: Load Playground base image
run: docker load --input "${RUNNER_TEMP}/playground-base-image/playground-php-wasm-base.tar"

- name: Install Playground deps
working-directory: wordpress-playground
run: npm ci --ignore-scripts

- name: Build wp_mysql_parser side module
working-directory: sqlite-database-integration/packages/php-ext-wp-mysql-parser/wasm-spike
env:
PHP_VERSION: ${{ matrix.php }}
ASYNC_MODE: ${{ matrix.async-mode }}
COMPILE_EXTENSION_PACKAGE: '@php-wasm/compile-extension@3.1.27'
PLAYGROUND_REPO: ${{ github.workspace }}/wordpress-playground
SKIP_BASE_IMAGE_BUILD: '1'
COMPILE_EXTENSION_PACKAGE: '@php-wasm/compile-extension@3.1.28'
run: bash build-in-docker-rust.sh

- name: Verify build artifacts exist
Expand All @@ -153,12 +65,15 @@ jobs:
test -f "dist/$SOURCE_PATH"
node -e "const artifact = require('./dist/manifest.json').artifacts[0]; if ('file' in artifact || 'sha256' in artifact) { throw new Error('manifest uses retired artifact fields'); }"

- name: Load the extension in Playground and parse a query
- name: Install PHP.wasm runtime deps
working-directory: sqlite-database-integration/packages/php-ext-wp-mysql-parser/wasm-spike
run: npm install --no-save --package-lock=false --no-audit --no-fund @php-wasm/node@3.1.28 @php-wasm/universal@3.1.28

- name: Load the extension in PHP.wasm and parse a query
working-directory: sqlite-database-integration/packages/php-ext-wp-mysql-parser/wasm-spike
env:
PLAYGROUND_REPO: ${{ github.workspace }}/wordpress-playground
PHP_VERSION: ${{ matrix.php }}
run: node run-spike.mjs
run: node --experimental-wasm-jspi run-spike.mjs

- name: Upload artifacts on success
if: success()
Expand Down
4 changes: 2 additions & 2 deletions packages/php-ext-wp-mysql-parser/wasm-spike/Dockerfile.rust
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Layer that adds rustup + the wasm32-unknown-emscripten target on top of
# Playground's compile-extension image.
# PHP.wasm's compile-extension image.
#
# Base image is the one produced by `@php-wasm/compile-extension`'s
# Dockerfile.ext, tagged as
Expand Down Expand Up @@ -61,7 +61,7 @@ RUN set -eux; \
/usr/local/bin/php-host-info -i | grep -q "PHP API => ${HOST_PHP_API_VERSION}"

# Set PHP_CONFIG explicitly so bindgen uses the wasm PHP headers from the
# Playground compile-extension image, not any host PHP package that apt may
# PHP.wasm compile-extension image, not any host PHP package that apt may
# provide as a transitive dependency.
ENV PHP_CONFIG=/usr/local/bin/php-config \
PHP=/usr/local/bin/php-host-info
Expand Down
38 changes: 14 additions & 24 deletions packages/php-ext-wp-mysql-parser/wasm-spike/RESULT.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@

## Current status

The spike now builds and loads `wp_mysql_parser` in Playground across every
The spike now builds and loads `wp_mysql_parser` in PHP.wasm across every
PHP version supported by the current `ext-php-rs` binding layer: 8.0 through
8.5, all JSPI. CI verifies each generated side module by loading its manifest
through Playground's compile-extension test harness and running a native lexer
through the released `@php-wasm/node` runtime and running a native lexer
smoke test. PHP 7.4 is outside this Rust path because `ext-php-rs` 0.15
depends on PHP 8 Zend APIs and does not compile against PHP 7.4 headers.

Expand All @@ -19,12 +19,11 @@ struct fields.

The build uses the published `@php-wasm/compile-extension` CLI for the phpize
side-module build, static archive force-linking, wasm-opt pass, and manifest
generation. A sparse Playground checkout is still required for the
`packages/php-wasm/compile` Docker assets and for CI's Playground load test.
The CLI is installed into an isolated temporary npm prefix before it is run;
the sparse Playground workspace also contains an unbuilt local
`@php-wasm/compile-extension` package, so relying on workspace `.bin` links
would bypass the published package.
generation. The CLI is installed into an isolated temporary npm prefix before
it is run and lazily fetches the PHP.wasm Docker assets it needs from the
matching Playground npm release. CI no longer checks out
`WordPress/wordpress-playground` for Docker assets or for the loader smoke
test.
The local glue that remains is Rust-specific: build `libwp_mysql_parser.a`
with the same Emscripten/PHP.wasm ABI and patch the vendored `ext-php-rs`
registry copy so it can run as a PHP.wasm side module.
Expand Down Expand Up @@ -108,31 +107,22 @@ extension build and manifest machinery.

| Path | What it is |
| --- | --- |
| `Dockerfile.rust` | Rust/nightly/host-PHP layer on top of Playground's compile-extension image. |
| `Dockerfile.rust` | Rust/nightly/host-PHP layer on top of the PHP.wasm compile-extension image. |
| `build-in-docker-rust.sh` | Builds the Rust staticlib, then calls the published `@php-wasm/compile-extension` CLI. |
| `write-extension-manifest.mjs` | Writes the combined all-version artifact manifest for the publish workflow. |
| `shim/config.m4` | Minimal phpize wrapper. |
| `shim/wp_mysql_parser_shim.c` | Pulls the Rust `get_module()` symbol into the side-module link. |
| `run-spike.mjs` | Loads the generated manifest in Playground and verifies the native lexer. |
| `run-spike.mjs` | Loads the generated manifest in `@php-wasm/node` and verifies the native lexer. |

## Reproduce

```bash
cd packages/php-ext-wp-mysql-parser/wasm-spike

PLAYGROUND_REPO=/abs/path/to/wordpress-playground \
bash build-in-docker-rust.sh
bash build-in-docker-rust.sh

PLAYGROUND_REPO=/abs/path/to/wordpress-playground PHP_VERSION=8.4 \
node run-spike.mjs
```

The workflow still uses a sparse checkout of `WordPress/wordpress-playground`
for `packages/php-wasm/compile` Docker assets and the Playground loader smoke
test, while the extension compile itself runs through an isolated install of
the published npm CLI.
npm install --no-save --package-lock=false --no-audit --no-fund \
@php-wasm/node@3.1.28 @php-wasm/universal@3.1.28

Follow-up for Playground: make `@php-wasm/compile-extension` self-contained so
external extension projects do not need to shallow or sparse checkout
`WordPress/wordpress-playground` just to access Docker assets or test harness
files.
PHP_VERSION=8.4 node --experimental-wasm-jspi run-spike.mjs
```
Loading
Loading