-
Notifications
You must be signed in to change notification settings - Fork 0
96 create GitHub actions sync type script definitions with database #108
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mehanana
wants to merge
4
commits into
main
Choose a base branch
from
96-Create-Github-Actions-Sync-TypeScript-Definitions-With-Database
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
b2118e3
created the yaml file for regenerating database types github actions …
mehanana 92ef2d6
small changes to workflow with kysley
mehanana 6f3120a
changed it so the lambda directories would automatically be chosen ra…
mehanana cded932
fixed peer dependency conflicts
mehanana File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,336 @@ | ||
| name: Generate Database Types For Schema Changes | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - apps/backend/db/db_setup.sql | ||
|
|
||
|
|
||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
|
|
||
| env: | ||
| NODE_VERSION: '23' | ||
| DATABASE_URL: postgresql://postgres:postgres@localhost:5432/testdb | ||
|
|
||
| jobs: | ||
| regenerate-types: | ||
| name: Auto-regenerate DB Types | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| ref: ${{ github.head_ref }} | ||
| token: ${{ secrets.GITHUB_TOKEN }} | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Setup Node.js | ||
| uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: ${{ env.NODE_VERSION }} | ||
| cache: 'npm' | ||
|
|
||
| - name: Cache npm dependencies | ||
| uses: actions/cache@v4 | ||
| with: | ||
| path: ~/.npm | ||
| key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }} | ||
| restore-keys: | | ||
| ${{ runner.os }}-node- | ||
|
|
||
| - name: Install dependencies | ||
| run: | | ||
| echo "Installing project dependencies" | ||
| npm ci --legacy-peer-deps | ||
|
|
||
| echo "Installing kysely-codegen globally" | ||
| npm install -g kysely-codegen | ||
|
|
||
| echo "Verifying kysely-codegen installation" | ||
| kysely-codegen --version || { | ||
| echo "Error: kysely-codegen installation failed" | ||
| exit 1 | ||
| } | ||
|
|
||
| - name: Setup PostgreSQL | ||
| uses: ikalnytskyi/action-setup-postgres@v6 | ||
| with: | ||
| username: postgres | ||
| password: postgres | ||
| database: testdb | ||
| port: 5432 | ||
| id: postgres | ||
|
|
||
| - name: Verify PostgreSQL connection | ||
| run: | | ||
| echo "Testing PostgreSQL connection" | ||
| psql -h localhost -U postgres -d testdb -c "SELECT version();" | ||
| env: | ||
| PGPASSWORD: postgres | ||
|
|
||
| - name: Initialize database schema | ||
| run: | | ||
| echo "Applying database schema" | ||
| psql -h localhost -U postgres -d testdb -f apps/backend/db/db_setup.sql | ||
|
|
||
| echo "Verifying schema application" | ||
| psql -h localhost -U postgres -d testdb -c "\dt" | ||
| env: | ||
| PGPASSWORD: postgres | ||
|
|
||
| - name: Generate types with kysely-codegen | ||
| run: | | ||
| echo "Starting type generation..." | ||
| cd apps/backend/db | ||
|
|
||
| # Run kysely-codegen | ||
| kysely-codegen \ | ||
| --url "$DATABASE_URL" \ | ||
| --out-file db-types.d.ts \ | ||
| --dialect postgres || { | ||
| echo "Error: kysely-codegen failed to generate types" | ||
| exit 1 | ||
| } | ||
|
|
||
| # Verify the output file was created | ||
| if [ ! -f "db-types.d.ts" ]; then | ||
| echo "Error: db-types.d.ts was not generated" | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "Type generation successful" | ||
|
|
||
| - name: Copy generated types to lambda functions | ||
| run: | | ||
| echo "Copying generated types to lambda functions" | ||
|
|
||
| SOURCE_FILE="apps/backend/db/db-types.d.ts" | ||
|
|
||
| # verify the source file exists | ||
| if [ ! -f "$SOURCE_FILE" ]; then | ||
| echo "Error: Generated types file not found at $SOURCE_FILE" | ||
| exit 1 | ||
| fi | ||
|
|
||
| if [ ! -s "$SOURCE_FILE" ]; then | ||
| echo "Error: Generated types file is empty" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Define target lambda directories | ||
| LAMBDA_DIRS=() | ||
| for dir in apps/backend/lambdas/*; do | ||
| if [ -d "$dir" ]; then | ||
| LAMBDA_DIRS+=("$dir") | ||
| echo " Found: $dir" | ||
| fi | ||
| done | ||
|
|
||
| if [ ${#LAMBDA_DIRS[@]} -eq 0 ]; then | ||
| echo "Error: No lambda directories found in apps/backend/lambdas/" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Track success and failures | ||
| SUCCESS_COUNT=0 | ||
| FAIL_COUNT=0 | ||
|
|
||
| for dir in "${LAMBDA_DIRS[@]}"; do | ||
| if [ -d "$dir" ]; then | ||
| echo "Copying to $dir/db-types.d.ts" | ||
| cp "$SOURCE_FILE" "$dir/db-types.d.ts" | ||
|
|
||
| if [ $? -eq 0 ]; then | ||
| ((SUCCESS_COUNT++)) | ||
| else | ||
| echo "Failed to copy to $dir" | ||
| ((FAIL_COUNT++)) | ||
| fi | ||
| else | ||
| echo "Warning: Directory $dir does not exist" | ||
| ((FAIL_COUNT++)) | ||
| fi | ||
| done | ||
|
|
||
| echo "" | ||
| echo "Copy summary: $SUCCESS_COUNT succeeded, $FAIL_COUNT failed" | ||
|
|
||
| if [ $SUCCESS_COUNT -eq 0 ]; then | ||
| echo "Error: Failed to copy types to any lambda function" | ||
| exit 1 | ||
| fi | ||
|
|
||
| - name: Verify TypeScript compilation | ||
| run: | | ||
| echo "Verifying TypeScript compilation" | ||
|
|
||
| # check if tsconfig.json exists | ||
|
|
||
| if [ ! -f "tsconfig.json" ]; then | ||
| echo "Warning: tsconfig.json not found in root, skipping full compilation check" | ||
| echo "Performing basic syntax check on generated types..." | ||
|
|
||
| # Ensure the file is valid TypeScript | ||
| for file in apps/backend/lambdas/*/db-types.d.ts; do | ||
| if [ -f "$file" ]; then | ||
| npx tsc --noEmit --skipLibCheck "$file" || { | ||
| echo "Error: $file contains TypeScript errors" | ||
| exit 1 | ||
| } | ||
| fi | ||
| done | ||
| else | ||
| npx tsc --noEmit --skipLibCheck || { | ||
| echo "Error: TypeScript compilation failed" | ||
| echo "The generated database types may be incompatible with the codebase." | ||
| exit 1 | ||
| } | ||
| fi | ||
|
|
||
| echo "TypeScript compilation successful" | ||
|
|
||
| - name: Check for changes | ||
| id: git-check | ||
| run: | | ||
| git add -N apps/backend/lambdas/*/db-types.d.ts | ||
|
|
||
| if git diff --exit-code apps/backend/lambdas/*/db-types.d.ts; then | ||
| echo "changes=false" >> $GITHUB_OUTPUT | ||
| echo "No changes detected in generated types" | ||
| else | ||
| echo "changes=true" >> $GITHUB_OUTPUT | ||
| echo "Changes detected in generated types" | ||
|
|
||
| echo "Changed files:" | ||
| git diff --name-only apps/backend/lambdas/*/db-types.d.ts | ||
| fi | ||
|
|
||
| - name: Get PR author information | ||
| if: steps.git-check.outputs.changes == 'true' | ||
| id: pr-author | ||
| run: | | ||
| # username | ||
| PR_AUTHOR_NAME="${{ github.event.pull_request.user.login }}" | ||
| PR_AUTHOR_EMAIL="${{ github.event.pull_request.user.login }}@users.noreply.github.com" | ||
|
|
||
| REAL_EMAIL=$(gh api /users/${{ github.event.pull_request.user.login }} --jq '.email' 2>/dev/null || echo "") | ||
|
|
||
| if [ -n "$REAL_EMAIL" ] && [ "$REAL_EMAIL" != "null" ]; then | ||
| PR_AUTHOR_EMAIL="$REAL_EMAIL" | ||
| fi | ||
|
|
||
| echo "name=$PR_AUTHOR_NAME" >> $GITHUB_OUTPUT | ||
| echo "email=$PR_AUTHOR_EMAIL" >> $GITHUB_OUTPUT | ||
|
|
||
| echo "PR Author: $PR_AUTHOR_NAME <$PR_AUTHOR_EMAIL>" | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Commit and push changes | ||
| if: steps.git-check.outputs.changes == 'true' | ||
| run: | | ||
| git config user.name "github-actions[bot]" | ||
| git config user.email "github-actions[bot]@users.noreply.github.com" | ||
|
|
||
| git add apps/backend/lambdas/*/db-types.d.ts | ||
|
|
||
| COMMIT_MESSAGE="chore: auto-regenerate database types from schema changes" | ||
|
|
||
| if [ -n "${{ steps.pr-author.outputs.name }}" ]; then | ||
| COMMIT_MESSAGE="${COMMIT_MESSAGE} | ||
|
|
||
| Co-authored-by: ${{ steps.pr-author.outputs.name }} <${{ steps.pr-author.outputs.email }}>" | ||
| fi | ||
|
|
||
| git commit -m "$COMMIT_MESSAGE" | ||
|
|
||
| git push origin HEAD:${{ github.head_ref }} | ||
|
|
||
| echo "Changes committed + pushed successfully" | ||
|
|
||
| - name: Comment on PR | ||
| if: steps.git-check.outputs.changes == 'true' | ||
| uses: actions/github-script@v7 | ||
| with: | ||
| script: | | ||
| const { execSync } = require('child_process'); | ||
| const files = execSync('git diff --name-only apps/backend/lambdas/*/db-types.d.ts') | ||
| .toString() | ||
| .trim() | ||
| .split('\n') | ||
| .filter(f => f) | ||
| .map(f => `- \`${f}\``) | ||
| .join('\n'); | ||
|
|
||
| const comment = `**Database Types Auto-Regenerated** | ||
|
|
||
| The database schema has changed and the Typescript definitions for these lambda functions were regenerated: | ||
|
|
||
| ${files} | ||
|
|
||
| The updated types are now in sync with schema changes.`; | ||
|
|
||
| github.rest.issues.createComment({ | ||
| issue_number: context.issue.number, | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| body: comment | ||
| }); | ||
|
|
||
| - name: Comment on PR if no changes | ||
| if: steps.git-check.outputs.changes != 'true' | ||
| uses: actions/github-script@v7 | ||
| with: | ||
| script: | | ||
| const comment = `**Database Types Check Complete** | ||
|
|
||
| The database schema files were modified, but the regenerated TypeScript types are identical to the existing ones. | ||
|
|
||
| No changes were needed and the type definitions are already up to date.`; | ||
|
|
||
| github.rest.issues.createComment({ | ||
| issue_number: context.issue.number, | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| body: comment | ||
| }); | ||
|
|
||
| - name: Workflow summary | ||
| if: always() | ||
| run: | | ||
| echo "## 🔄 Database Type Regeneration Summary" >> $GITHUB_STEP_SUMMARY | ||
| echo "" >> $GITHUB_STEP_SUMMARY | ||
|
|
||
| if [ "${{ steps.git-check.outputs.changes }}" == "true" ]; then | ||
| echo "### Status: Types Regenerated Successfully" >> $GITHUB_STEP_SUMMARY | ||
| echo "" >> $GITHUB_STEP_SUMMARY | ||
| echo "The following files were updated and committed:" >> $GITHUB_STEP_SUMMARY | ||
| echo "" >> $GITHUB_STEP_SUMMARY | ||
|
|
||
| for file in apps/backend/lambdas/*/db-types.d.ts; do | ||
| if [ -f "$file" ]; then | ||
| echo "- \`$file\`" >> $GITHUB_STEP_SUMMARY | ||
| fi | ||
| done | ||
|
|
||
| echo "" >> $GITHUB_STEP_SUMMARY | ||
| echo "**Next Steps:** The changes have been automatically committed to this PR. Review the updated types and merge when ready." >> $GITHUB_STEP_SUMMARY | ||
| else | ||
| echo "### Status: No Changes Needed" >> $GITHUB_STEP_SUMMARY | ||
| echo "" >> $GITHUB_STEP_SUMMARY | ||
| echo "The regenerated types are identical to the existing ones." >> $GITHUB_STEP_SUMMARY | ||
| echo "Your type definitions are already in sync with the schema." >> $GITHUB_STEP_SUMMARY | ||
| fi | ||
|
|
||
| echo "" >> $GITHUB_STEP_SUMMARY | ||
| echo "---" >> $GITHUB_STEP_SUMMARY | ||
| echo "" >> $GITHUB_STEP_SUMMARY | ||
| echo "**Workflow Trigger:** Schema change detected in:" >> $GITHUB_STEP_SUMMARY | ||
| echo "- \`apps/backend/db/db_setup.sql\`" >> $GITHUB_STEP_SUMMARY | ||
|
|
||
|
|
||
|
|
||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One thing to be careful of here: Since we have many CI things that push changes, we need to be careful of race conditions. We should ideally have one bot that manages all the commiting/pushing, but for now, can you make the Terraform readme workflow wait for this one? (Thillai has a PR up for another workflow that waits for the terraform one)