Skip to content

GITHUB_OPERATIONS.md

Latest commit

 

History

History
460 lines (370 loc) · 20.6 KB

GITHUB_OPERATIONS.md

File metadata and controls

460 lines (370 loc) · 20.6 KB

GitHub Operations

How this repo uses GitHub: what runs on every push and PR, what runs on tag pushes, how releases become crates.io publishes and downloadable binaries, and what maintainers do by hand vs. what the automation does.

Audience: maintainers and contributors who want to know "what happens when I push", "how do I cut a release", or "where do the pre-built binaries on the Releases page come from".

Repository

Workflows

Four GitHub Actions workflows live under .github/workflows/:

Workflow File Triggers Purpose
ci ci.yml push to main, all PRs, manual fmt, clippy, full test matrix, cargo deny, cargo audit, cargo publish --dry-run
release-please release-please.yml push to main, manual open/update the release PR; on merge, create the vX.Y.Z tag + GitHub Release
release release.yml tag push matching v*.*.* / v*.*.*-rc.*, manual re-run tests, publish the 6 Rust crates to crates.io (hyperdb-api-node is published separately to npm)
npm-build-publish npm-build-publish.yml GitHub Release published, manual build npm platform packages with bundled hyperd, publish to npm registry
verify-hyperd-pin verify-hyperd-pin.yml changes to hyperdb-bootstrap/hyperd-version.toml or its source, weekly cron, manual HEAD every pinned hyperd release URL to catch Tableau yanks / typos

CI (ci.yml)

Runs on every PR and on every push to main. Jobs:

  • rustfmt — cargo fmt --all --check.
  • clippy — cargo clippy --workspace --all-targets -- -D warnings (single runner; lints are platform-independent).
  • test — full workspace test matrix on ubuntu-latest, macos-14, windows-latest.
  • publish-dry-run — cargo publish --dry-run for each publishable crate so a broken publish manifest is caught before a tag is cut.
  • cargo-deny — license and advisory policy enforcement per deny.toml.
  • cargo-audit — RustSec advisories, --deny warnings.

In-progress PR CI runs are cancelled when a new commit is pushed to the PR. Main-branch runs always complete. This is set via the concurrency block at the top of ci.yml.

Release (release.yml)

Runs on the release: published event (release-please publishes the GitHub Release after merging the release PR) or via manual workflow_dispatch with an explicit tag input (for re-runs or emergency releases). Structure:

verify          ← full test suite + hyperd URL check, single-platform
   │
   └─► publish          ← crates.io publish in dependency order

There is no per-platform binary-archive build today. Users on supported platforms install via crates.io (cargo install hyperdb-mcp) or via npm (npm install -g hyperdb-mcp, which delivers prebuilt binaries through npm-build-publish.yml). The crates themselves are architecture-independent source on crates.io.

Dependency-ordered crates.io publish (per-crate sleep 45 between each so the crates.io index has time to settle before the next crate's verification step resolves the just-published dep):

  1. hyperdb-api-salesforce (no workspace runtime deps; published first to break the optional cycle with hyperdb-api-core)
  2. hyperdb-api-core
  3. hyperdb-api
  4. hyperdb-mcp
  5. hyperdb-bootstrap
  6. sea-query-hyperdb

hyperdb-api-node is not on the crates.io list (its Cargo.toml has publish = false) — it ships as npm hyperdb-api-node through napi-rs's own pipeline, which is outside this workflow today.

Pre-release vs. stable: the GitHub Release is marked prerelease: true automatically for tags containing -rc., -alpha., or -beta. — they show up on the Releases page but are not flagged as "Latest release".

Concurrency: only one release workflow runs at a time (the concurrency: release group at the top of the file); a second tag push during a release will queue, not clobber.

npm-publish (npm-build-publish.yml)

Builds and publishes npm packages for hyperdb-mcp and hyperdb-api-node with the hyperd database engine bundled into each platform package. This lets end users run npx hyperdb-mcp or npm install hyperdb-api-node without needing Rust toolchains or manual hyperd setup.

Triggers:

  • GitHub Release published — fires automatically after release.yml creates/updates a GitHub Release.
  • Manual workflow_dispatch — tag/branch input is optional; leave empty to build from the default branch HEAD (useful for testing the pipeline without tagging).

Structure:

verify-ci       ← checks that CI passed for this commit (gh api commit status)
   │
   └─► build-npm (matrix × 4 platforms)
          build hyperdb-mcp + hyperdb-api-node native binaries
          download hyperd via curl with SHA256 verification
          assemble platform packages (binary + hyperd + LICENSE-HYPERD)
          upload as GitHub Actions artifacts (7-day retention)
              │
              └─► publish-npm
                    publish platform packages then main packages to npm

Platform matrix:

Platform Runner Rust target hyperd source
darwin-arm64 macos-14 aarch64-apple-darwin macos-arm64
linux-x64-gnu ubuntu-latest x86_64-unknown-linux-gnu linux-x86_64
win32-x64-msvc windows-latest x86_64-pc-windows-msvc windows-x86_64

darwin-x64 (Intel macOS) is currently disabled — macos-13 GHA runners have been unreliable. The matrix entry is commented out in npm-build-publish.yml and will be re-enabled when runner availability stabilizes. Until then, Intel-Mac users must build from source.

npm packages published:

Package Type Contents
hyperdb-mcp Main (bin shim) bin.js — detects platform, sets HYPERD_PATH, spawns native binary
hyperdb-mcp-darwin-arm64 Platform hyperdb-mcp + hyperd + LICENSE-HYPERD
hyperdb-mcp-linux-x64-gnu Platform same, Linux x64
hyperdb-mcp-win32-x64-msvc Platform same, Windows x64
hyperdb-api-node Main (napi-rs) JS bindings + getHyperdPath() helper
hyperdb-api-node-* Platform .node addon + hyperd + LICENSE-HYPERD

CI gate: The verify-ci job checks that the combined commit status is success before building. If CI hasn't passed (e.g., someone triggers a manual dispatch on a broken commit), the workflow aborts immediately. Note: this does not prevent tagging — git tags can be created regardless of CI status. Use GitHub Rulesets (repo Settings → Rules) to enforce tag-creation restrictions if needed.

Downloading artifacts without publishing: Since publish-npm requires NPM_TOKEN, you can trigger a manual dispatch to test the build — the build jobs will succeed and upload downloadable artifacts, while publish-npm fails harmlessly.

# Trigger build from current main (no tag needed)
gh workflow run npm-build-publish.yml

# Trigger build for a specific tag
gh workflow run npm-build-publish.yml --field tag=v0.1.0

# Download artifacts after the run completes
gh run download <run-id> --name npm-darwin-arm64

Local builds: Use make npm-pack to build the current platform's npm packages locally without CI. This produces .tgz files you can share directly:

make npm-pack
npm install -g ./hyperdb-mcp/npm/hyperdb-mcp-darwin-arm64-*.tgz \
               ./hyperdb-mcp/npm/hyperdb-mcp-*.tgz

verify-hyperd-pin (verify-hyperd-pin.yml)

Independently checks that the per-platform URLs baked into hyperdb-bootstrap/hyperd-version.toml still resolve (via hyperdb-bootstrap verify). Runs:

  • On any PR that touches the pin file or hyperdb-bootstrap/src/** (early-warn before the pin change lands).
  • On push to main for the same paths (covers the merge).
  • Every Monday at 12:00 UTC regardless of PR traffic (catches Tableau yanking a release out from under us).
  • Manually via workflow_dispatch.

Cutting a release

Releases are driven by release-please. Maintainers don't bump versions, edit changelogs, or push tags by hand — those steps are automated based on Conventional Commits.

How it flows

  1. Contributors land PRs into main with conventional-commit titles (feat:, fix:, chore:, etc. — see CONTRIBUTING.md).

  2. The release-please workflow runs on every push to main. It opens (or updates) a single release PR titled chore(main): release X.Y.Z. That PR contains:

    • All 7 workspace crates' versions bumped (Cargo.toml + package.json).
    • The optionalDependencies and inter-crate version pins updated.
    • A new dated section in each crate's CHANGELOG.md summarizing the conventional commits that landed since the last release.
    • An updated .release-please-manifest.json.

  3. A maintainer reviews the release PR. Adjust the version manually if a different bump is needed (e.g., promote a 0.x.0 patch to a minor) by editing the PR or by tagging commits with Release-As: X.Y.Z.

  4. Merge the release PR. release-please then:

    • Creates a vX.Y.Z git tag on the merge commit.
    • Creates a GitHub Release with the auto-generated changelog.

  5. Publish workflows fire automatically. Because release-please uses a PAT (RELEASE_PLEASE_TOKEN), the release: published event triggers both release.yml (crates.io) and npm-build-publish.yml (npm) automatically. The npm workflow waits for CI to pass before building.

    If a publish workflow fails and needs a re-run:

    gh workflow run release.yml -f tag=vX.Y.Z
gh workflow run npm-build-publish.yml -f tag=vX.Y.Z

    Already-published crates are skipped gracefully on re-run.

How commits drive version bumps

release-please reads the Conventional Commits prefix on each commit since the last release tag and picks the largest bump implied. Mark a commit as a breaking change by either appending ! after the type (e.g. feat!:) or by adding a BREAKING CHANGE: footer in the commit body.

Important pre-1.0 caveat: while the workspace is on a 0.x.y version, semver treats the entire 0.x line as unstable, and release-please follows suit — a breaking change bumps the minor component, not the major. The major component stays at 0 until you explicitly opt into 1.0.0.

Commit prefix on main Bump from 0.1.0 to
fix:, fix(scope): 0.1.1 (patch)
feat:, feat(scope): 0.2.0 (minor)
feat!: / fix!: / BREAKING CHANGE: footer 0.2.0 (still minor — no major bump while pre-1.0)
chore:, docs:, refactor:, test:, style:, ci:, perf:, build: no release
Manual Release-As: 1.0.0 footer 1.0.0 (forces the major bump)

After the workspace is on 1.x.y, the same prefixes follow normal semver: feat!: will bump 1.2.3 → 2.0.0 as expected. To stabilize the API and cut 1.0.0, add a Release-As: 1.0.0 footer to a conventional-commit on main:

feat: stabilize public API

Release-As: 1.0.0

Pre-releases

For an -rc.N / -alpha.N / -beta.N release, add a footer to a commit on main:

Release-As: 0.2.0-rc.1

release-please will produce a release PR with that exact version on the next run. Pre-release tags flow through release.yml and npm-build-publish.yml exactly as stable releases do; the GitHub Release is auto-flagged as prerelease: true, and the npm dist-tag is set to rc / alpha / beta instead of latest so npm install hyperdb-mcp doesn't pull a pre-release by default.

Lockstep versioning

All 7 workspace crates share a single version number, enforced by the linked-versions plugin in release-please-config.json. When any crate's commits trigger a bump, every crate moves together. This keeps cargo publish's strict inter-crate version pins (= "X.Y.Z") in sync without manual edits.

Verifying a release

Once both release.yml and npm-build-publish.yml go green:

Re-running a partial failure

The release workflow is mostly idempotent but there are two sharp edges:

  • crates.io is append-only. If the workflow publishes hyperdb-api-core v0.2.0 and then fails on hyperdb-api, you cannot republish hyperdb-api-core v0.2.0 — that version is burned. The fix is to land a follow-up fix: commit on main, let release-please open a release PR for 0.2.1, and merge that.
  • rate limits during the first publish of a brand-new crate. crates.io caps "new crate" creations at one per ~10 minutes. The first time we publish a fresh crate name, the workflow may 429 partway through the Publish in dependency order step. Wait for the cooldown printed in the error and rerun via Actions → release → "Run workflow", entering the same tag name in the tag input. Already-published crates fail loudly with "already uploaded" and the run will continue past them via the per-crate retry below.

Re-running release.yml against an existing tag

For cases where the tag already exists in origin (e.g. you want to rerun release.yml after a transient infra failure), use the Actions UI:

  1. Actions → release → "Run workflow".
  2. Enter the existing tag name in the tag input (e.g. v0.2.0).
  3. Click Run.

The workflow's regex validator rejects malformed tag names, and concurrency: release prevents racing with an in-flight run.

Secrets

Secret Used by Scope
RELEASE_PLEASE_TOKEN release-please.yml Fine-grained PAT; triggers CI on release-please PRs/tags (see below)
CARGO_REGISTRY_TOKEN release.yml publish job cargo publish to crates.io
NPM_TOKEN npm-build-publish.yml publish-npm job npm publish to npmjs.org
GITHUB_TOKEN Every workflow Auto-provided by GitHub Actions; used to post releases, download artifacts, verify CI status

Why release-please needs a PAT

GitHub Actions suppresses workflow triggers on events created by GITHUB_TOKEN (anti-recursion protection). Without a PAT, PRs opened by release-please don't trigger CI, and tags it pushes don't trigger release.yml or npm-build-publish.yml. The workaround is a fine-grained PAT stored as RELEASE_PLEASE_TOKEN.

Option A: Fine-grained PAT (current setup)

  1. Go to Settings → Developer settings → Personal access tokens → Fine-grained tokens → Generate new token.
  2. Configure:
    • Token name: release-please-hyper-api-rust
    • Expiration: 90 days (set a calendar reminder to rotate)
    • Resource owner: tableau
    • Repository access: Only select → tableau/hyper-api-rust
    • Permissions → Repository:
      • Contents: Read and write
      • Pull requests: Read and write
  3. Click "Generate token" and copy it.
  4. Add it as a repo secret: 
    gh secret set RELEASE_PLEASE_TOKEN --repo tableau/hyper-api-rust
  5. The release-please workflow references this secret via token: ${{ secrets.RELEASE_PLEASE_TOKEN }}.

Rotation: when the PAT expires, generate a new one with the same settings and update the secret. Release-please will fail with a 401 until the secret is refreshed — CI on main pushes will still show the failure clearly.

Option B: GitHub App token (recommended for larger teams)

A GitHub App token isn't tied to any individual's account and never expires (tokens are minted per-run). Preferred for org-owned repos or when multiple maintainers need the pipeline to work independently.

  1. Create a GitHub App (org-level: Settings → Developer settings → GitHub Apps → New):
    • Name: hyper-api-rust-release-please
    • Permissions → Repository:
      • Contents: Read and write
      • Pull requests: Read and write
    • No webhook URL needed (uncheck "Active" under Webhook)
    • Generate a private key and download it
  2. Install the App on tableau/hyper-api-rust (or all repos in the org if you want it shared).
  3. Store credentials as repo secrets: 
    gh secret set APP_ID --repo tableau/hyper-api-rust        # numeric App ID
gh secret set APP_PRIVATE_KEY --repo tableau/hyper-api-rust  # PEM file contents
  4. Update the workflow to mint a short-lived token each run: 
    jobs:
  release-please:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/create-github-app-token@v2
        id: app-token
        with:
          app-id: ${{ secrets.APP_ID }}
          private-key: ${{ secrets.APP_PRIVATE_KEY }}
      - uses: googleapis/release-please-action@v5
        with:
          config-file: release-please-config.json
          manifest-file: .release-please-manifest.json
          token: ${{ steps.app-token.outputs.token }}

This mints a token scoped to the installation that expires in 1 hour — no rotation needed, no personal account dependency.

Issue & PR templates

There are no .github/ISSUE_TEMPLATE/ or .github/pull_request_template.md files today; Issues and PRs use GitHub defaults. Contributors still follow the Contribution Checklist manually.

Branch protection

Branch protection rules on main are configured via GitHub's repo settings (not in this repo as config-as-code). The expected invariants:

  • All PRs require at least one approval.
  • ci must pass before merge.
  • Force-push and deletion are blocked.
  • Tags matching v*.*.* can only be pushed by maintainers (enforced via tag protection rules, separate from branch protection).

Check the actual live settings under Settings → Branches and Settings → Tags on the GitHub UI.

When something breaks

  • CI failures on main: investigate and fix forward. The cancel-on-new-push concurrency only applies to PRs; main-branch runs always complete, so a broken main is a real signal.
  • release workflow failure during verify: the tag already exists, so manual re-tagging isn't needed. Land the fix on main, then re-run release.yml against the same tag from the Actions UI (Run workflow → enter tag → run). Or, if the fix changes the tag contents, let the next release-please PR mint a fresh patch tag.
  • release workflow failure during publish: the already-published crates are burned (crates.io is append-only). Land a fix: commit on main and merge the next release-please PR. Don't try to retag the partially-published version.
  • verify-hyperd-pin failure: the pinned hyperd release URL 404'd. Check the Tableau releases page, update hyperdb-bootstrap/hyperd-version.toml with the new version + fresh SHA-256s, and open a PR.
  • Newly-flagged cargo-audit advisory on main: open a PR with a dep bump (or, if no fix is yet available, document the waiver in deny.toml with an expiration date).

Related docs