Skip to content

Add pg_lake extension via separate Debian-based image #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
pyramation wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Add pg_lake extension via separate Debian-based image #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
6c28097
Add pg_lake extension for Iceberg and data lake access
pyramation Jan 11, 2026
93f9107
Fix pg_lake build: add krb5-dev for gssapi headers
pyramation Jan 11, 2026
8909b98
Add separate postgres-plus-lake image with pg_lake extension
pyramation Jan 11, 2026
a3a9c63
Fix PostGIS build: use sequential make to avoid race condition
pyramation Jan 11, 2026
8538c9e
Fix avro build: add pkg-config for libjansson detection
pyramation Jan 11, 2026
c60447c
Build PostgreSQL from source for pg_lake internal headers
pyramation Jan 11, 2026
a7e7d9e
Fix PostgreSQL version: use 17.7 (latest available)
pyramation Jan 11, 2026
23b9c17
Fix pg_lake build: use Dockerfile default PG_VERSION (17.7)
pyramation Jan 11, 2026
4b2dff1
Fix PostgreSQL build: use libossp-uuid-dev for OSSP UUID support
pyramation Jan 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
200 changes: 181 additions & 19 deletions .github/workflows/docker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Build and Push Docker Image
name: Build and Push Docker Images

on:
push:
Expand All @@ -9,14 +9,14 @@ on:

env:
REGISTRY: ghcr.io
IMAGE_NAME: constructive-io/docker/postgres-plus
PG_VERSION: '17'

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-docker-postgres-plus
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
# Build postgres-plus (Alpine, lean image without pg_lake)
build-postgres-plus:
strategy:
fail-fast: false
Expand Down Expand Up @@ -53,28 +53,28 @@ jobs:
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
images: ${{ env.REGISTRY }}/constructive-io/docker/postgres-plus
tags: |
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/main' }}
type=raw,value=${{ env.PG_VERSION }},enable=${{ github.ref == 'refs/heads/main' }}
type=sha,format=short,prefix=
type=semver,pattern={{version}},enable=${{ startsWith(github.ref, 'refs/tags/v') }}
type=semver,pattern={{major}}.{{minor}},enable=${{ startsWith(github.ref, 'refs/tags/v') }}

- name: Build (no push, PR only, amd64)
if: github.event_name == 'pull_request' && matrix.platform == 'linux/amd64'
- name: Build (PR only)
if: github.event_name == 'pull_request'
uses: docker/build-push-action@v6
with:
context: .
file: ./Dockerfile
platforms: linux/amd64
platforms: ${{ matrix.platform }}
push: false
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-args: |
PG_VERSION=${{ env.PG_VERSION }}
cache-from: type=gha
cache-to: type=gha,mode=max
cache-from: type=gha,scope=postgres-plus-${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=postgres-plus-${{ matrix.arch }}

- name: Build & push by digest
if: github.event_name != 'pull_request'
Expand All @@ -85,26 +85,116 @@ jobs:
file: ./Dockerfile
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }},push-by-digest=true,push=true
outputs: type=image,name=${{ env.REGISTRY }}/constructive-io/docker/postgres-plus,push-by-digest=true,push=true
build-args: |
PG_VERSION=${{ env.PG_VERSION }}
cache-from: type=gha
cache-to: type=gha,mode=max
cache-from: type=gha,scope=postgres-plus-${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=postgres-plus-${{ matrix.arch }}

- name: Export digest
if: github.event_name != 'pull_request'
run: |
mkdir -p "${{ runner.temp }}/digests-postgres-plus"
digest="${{ steps.build.outputs.digest }}"
touch "${{ runner.temp }}/digests-postgres-plus/${digest#sha256:}"

- name: Upload digest
if: github.event_name != 'pull_request'
uses: actions/upload-artifact@v4
with:
name: digests-postgres-plus-${{ matrix.arch }}
path: ${{ runner.temp }}/digests-postgres-plus/*

# Build postgres-plus-lake (Debian, includes pg_lake)
build-postgres-plus-lake:
strategy:
fail-fast: false
matrix:
include:
- platform: linux/amd64
arch: amd64
runner: ubuntu-latest
- platform: linux/arm64
arch: arm64
runner: ubuntu-24.04-arm
runs-on: ${{ matrix.runner }}

permissions:
contents: read
packages: write

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Log in to Container Registry
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/constructive-io/docker/postgres-plus-lake
tags: |
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/main' }}
type=raw,value=${{ env.PG_VERSION }},enable=${{ github.ref == 'refs/heads/main' }}
type=sha,format=short,prefix=
type=semver,pattern={{version}},enable=${{ startsWith(github.ref, 'refs/tags/v') }}
type=semver,pattern={{major}}.{{minor}},enable=${{ startsWith(github.ref, 'refs/tags/v') }}

- name: Build (PR only)
if: github.event_name == 'pull_request'
uses: docker/build-push-action@v6
with:
context: .
file: ./Dockerfile.pg_lake
platforms: ${{ matrix.platform }}
push: false
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
# Note: pg_lake image uses Dockerfile default PG_VERSION (full version like 17.7)
# because it builds PostgreSQL from source and needs exact version number
cache-from: type=gha,scope=postgres-plus-lake-${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=postgres-plus-lake-${{ matrix.arch }}
Comment on lines +163 to +166
Copy link

Copilot AI Jan 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment mentions that pg_lake image uses "Dockerfile default PG_VERSION (full version like 17.7)" but there's a mismatch: the workflow sets PG_VERSION='17' (major version), while Dockerfile.pg_lake defaults to PG_VERSION=17.7 (full version). Since the build doesn't pass PG_VERSION as a build-arg, the Dockerfile will always use 17.7 regardless of the workflow env var. The comments should be clarified to explain that the pg_lake build intentionally uses its hardcoded version (17.7) because it needs to download PostgreSQL source, while the Alpine image uses the workflow's major version (17) to reference pre-built postgres:17-alpine base images.

Copilot uses AI. Check for mistakes.

- name: Build & push by digest
if: github.event_name != 'pull_request'
id: build
uses: docker/build-push-action@v6
with:
context: .
file: ./Dockerfile.pg_lake
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ env.REGISTRY }}/constructive-io/docker/postgres-plus-lake,push-by-digest=true,push=true
# Note: pg_lake image uses Dockerfile default PG_VERSION (full version like 17.7)
# because it builds PostgreSQL from source and needs exact version number
cache-from: type=gha,scope=postgres-plus-lake-${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=postgres-plus-lake-${{ matrix.arch }}

- name: Export digest
if: github.event_name != 'pull_request'
run: |
mkdir -p "${{ runner.temp }}/digests"
mkdir -p "${{ runner.temp }}/digests-postgres-plus-lake"
digest="${{ steps.build.outputs.digest }}"
touch "${{ runner.temp }}/digests/${digest#sha256:}"
touch "${{ runner.temp }}/digests-postgres-plus-lake/${digest#sha256:}"

- name: Upload digest
if: github.event_name != 'pull_request'
uses: actions/upload-artifact@v4
with:
name: digests-${{ matrix.arch }}
path: ${{ runner.temp }}/digests/*
name: digests-postgres-plus-lake-${{ matrix.arch }}
path: ${{ runner.temp }}/digests-postgres-plus-lake/*

# Publish postgres-plus manifest
publish-postgres-plus-manifest:
if: github.event_name != 'pull_request'
runs-on: ubuntu-latest
Expand All @@ -128,15 +218,87 @@ jobs:
- name: Download digests
uses: actions/download-artifact@v4
with:
pattern: digests-*
pattern: digests-postgres-plus-*
path: ${{ runner.temp }}/digests
merge-multiple: true

- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/constructive-io/docker/postgres-plus
tags: |
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/main' }}
type=raw,value=${{ env.PG_VERSION }},enable=${{ github.ref == 'refs/heads/main' }}
type=sha,format=short,prefix=
type=semver,pattern={{version}},enable=${{ startsWith(github.ref, 'refs/tags/v') }}
type=semver,pattern={{major}}.{{minor}},enable=${{ startsWith(github.ref, 'refs/tags/v') }}

- name: Create and push multi-arch manifests
run: |
set -euo pipefail

image="${{ env.REGISTRY }}/constructive-io/docker/postgres-plus"
digest_dir="${{ runner.temp }}/digests"

if [ ! -d "$digest_dir" ]; then
echo "No digests directory found at $digest_dir"
exit 1
fi

digests=""
for digest_file in "$digest_dir"/*; do
Copy link

Copilot AI Jan 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The for loop may fail if the digest directory contains unexpected files. The pattern "$digest_dir"/* will match all files including potential hidden files or non-digest files. Consider adding a check to verify the loop actually processes files: add [ ! -e "$digest_file" ] && continue after the for loop declaration, or use a more specific pattern that validates digest filenames.

Suggested change
for digest_file in "$digest_dir"/*; do
for digest_file in "$digest_dir"/*; do
[ ! -e "$digest_file" ] && continue

Copilot uses AI. Check for mistakes.
digest="$(basename "$digest_file")"
digests="$digests $image@sha256:$digest"
done

if [ -z "$digests" ]; then
echo "No digests found to create manifest"
exit 1
fi

echo "Creating manifests for tags:"
echo "${{ steps.meta.outputs.tags }}"

echo "${{ steps.meta.outputs.tags }}" | while read -r tag; do
[ -z "$tag" ] && continue
echo "Creating multi-arch manifest for $tag"
docker buildx imagetools create -t "$tag" $digests
done

# Publish postgres-plus-lake manifest
publish-postgres-plus-lake-manifest:
if: github.event_name != 'pull_request'
runs-on: ubuntu-latest
needs: build-postgres-plus-lake

permissions:
contents: read
packages: write

steps:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Log in to Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Download digests
uses: actions/download-artifact@v4
with:
pattern: digests-postgres-plus-lake-*
path: ${{ runner.temp }}/digests
merge-multiple: true

- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
images: ${{ env.REGISTRY }}/constructive-io/docker/postgres-plus-lake
tags: |
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/main' }}
type=raw,value=${{ env.PG_VERSION }},enable=${{ github.ref == 'refs/heads/main' }}
Expand All @@ -148,7 +310,7 @@ jobs:
run: |
set -euo pipefail

image="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}"
image="${{ env.REGISTRY }}/constructive-io/docker/postgres-plus-lake"
digest_dir="${{ runner.temp }}/digests"

if [ ! -d "$digest_dir" ]; then
Expand Down
Loading