Skip to content

Added template tests#249

Open
russell-stern wants to merge 9 commits into
mainfrom
template_tests
Open

Added template tests#249
russell-stern wants to merge 9 commits into
mainfrom
template_tests

Conversation

@russell-stern
Copy link
Copy Markdown

Adds cross-repo testing to catch SDK changes that break TypeScript templates in cre-templates before they reach developers.

  1. bun run test:templates

Builds and packs the local SDK, then runs typecheck and WASM compilation against every TypeScript template in cre-templates. Defaults to ../cre-templates — override with TEMPLATES_DIR=/path/to/cre-templates.

  1. GitHub Action (informational, non-blocking)

Runs on every PR. Non-blocking by design, supports a compat/ branch convention in cre-templates for coordinated breaking changes: create the branch with template fixes applied, and the action will automatically test against it instead of main.

Posts a PR comment listing failed templates and their errors when something breaks, and removes it automatically when the PR is fixed.

Comment thread .github/workflows/template-compatibility.yml Fixed
@russell-stern russell-stern marked this pull request as ready for review April 30, 2026 14:13
@russell-stern russell-stern requested review from a team as code owners April 30, 2026 14:13
Comment thread scripts/test-templates.sh
@@ -0,0 +1,324 @@
#!/bin/bash
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit since it's a typescript repo maybe making it in a TS test would be higher-level more readable and integrated with other tests for devs to run (and do not forget), but for CI I guess bash is fine too 👍

mchain0
mchain0 previously approved these changes Apr 30, 2026
@mchain0
Copy link
Copy Markdown

mchain0 commented Apr 30, 2026

I like the idea. nit comment, but non-blocking

Copy link
Copy Markdown
Collaborator

@ernest-nowacki ernest-nowacki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it and love this addon:

Image

As a way for making sure templates also include all they need to have full TS checks ready.

Locally I got some errors (example: [7/33] tokenized-asset-servicing/asset-log-trigger-workflow) but those might be legitimate templates improvements we need. Entire process works great locally! (used: [7/33] tokenized-asset-servicing/asset-log-trigger-workflow).

Leaving some small feedback below.

on:
pull_request:
types: [opened, synchronize, reopened]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also add

  concurrency:
    group: template-compat-${{ github.head_ref || github.ref }}
    cancel-in-progress: true

As template testing could take some time and upon multiple commits we would have improved efficiency.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call

- name: Post or remove PR comment
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
with:
script: |
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel it would be cleaner and easier to maintain if we put script in file together with other scripts we had and just call it here in 1-2 lines.

Comment thread scripts/test-templates.sh Outdated
info "Packing SDK..."

cd packages/cre-sdk-javy-plugin
JAVY_TARBALL=$(npm pack --quiet 2>/dev/null)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For tarballs I wonder if we could stick with bun as we already use it everywhere except publish stage. Can you try bun pm pack and see if it would work?

Non blocking - if it's causes trouble we can keep npm - if not, then lets keep using bun whenever we can.

Comment thread scripts/test-templates.sh Outdated
cp package.json package.json.bak
npm pkg set dependencies."@chainlink/cre-sdk-javy-plugin"="file:$JAVY_TARBALL_PATH"

TARBALL=$(npm pack --quiet 2>/dev/null)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Comment on lines +89 to +97
- name: Checkout cre-templates (${{ steps.detect-ref.outputs.ref }})
uses: actions/checkout@v6
with:
repository: ${{ env.TEMPLATES_REPO }}
ref: ${{ steps.detect-ref.outputs.ref }}
path: cre-templates
token: ${{ secrets.GITHUB_TOKEN }}

- name: Setup Node (for npm)
ernest-nowacki
ernest-nowacki previously approved these changes May 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants