libpq Version update

[0814celsus]

Hello, I am currently working on an OAuth2 Validation Module for a Postgres 18 Database which should serve an ducklake Catalog Database, I noticed that you are staticly linking libpq. And vcpkg libpq port is currently at v16.9 right? PG18 introduced the entire SASL OAUTHBEARER mechanism including PQsetAuthDataHook, PQAUTHDATA_OAUTH_BEARER_TOKEN, and the PGoauthBearerRequest struct. So none of that exists for the postgres_scanner extention. I am using PQsetAuthDataHook to inject the JWT Token in the Authenticate workflow where my Module is picking it up and sets user session environment vars according the token. This will then be the rules for RLS filtering ducklake Columns and Tables. My question, is there a plan to bump the version anytime soon. Which possibilities do I have to get my OAuth flow working with duckdb and Postgres? Thanks Christoph

[staticlibs]

Hi, thanks for raising the issue! I think it would be great to update the libpq to REL_18_3. Unfortunately it is not built with vcpkg, but is built manually instead, so updating it may require non-trivial effort (as I understand, Windows part of the build was changed significantly there). If you are interesting in bringing it into 1.5.1 update (scheduled for March 23, the freeze is coming on March 18) - I can review a PR promptly. Otherwise it is likely to be pushed to some future update.

[0814celsus]

Thank you answering, I will have a look at it, but honestly 1.5.1 is too tight, I need to admit that my C++ knowledge is limited. But I am willing to dive into it. June or July is more likely doable.

[0814celsus]

@staticlibs I have started with a local built on linux. There the changes seems not to be dramatic. A few new source files needed to be added.

• fe-auth-oauth.c added in PG16 OAuth and OIDC support)

• jsonapi.c (the json parser required by fe-auth-oauth.c.)
  must be compiled with -DJSONAPI_USE_PQEXPBUFFER to use
  frontend-safe pqexpbuffer instead of backend palloc.

• pg_prng.c (seudo-random number generator, required by fe-auth-oauth.c.)

• strlcat.c
  Was part of strlcpy.c in PG15, split into its own file in PG18.
  Added to the Linux+WIN32 conditional block alongside strlcpy.c.

Removed source files

• postgres/src/port/thread.c
  Removed in PG18. The functions (pg_get_user_name, etc.) were
  inlined into fe-connect.c. Was renamed to user.c in PG17 as
  an intermediate step, then removed entirely.

As I mentioned before, build flag additions:

• set_source_files_properties(postgres/src/common/jsonapi.c
  PROPERTIES COMPILE_DEFINITIONS "JSONAPI_USE_PQEXPBUFFER")

What else:

The CMakeLists.txt configure invocation (line 177) uses flags that no
longer exist in PG18's ./configure:

Current (broken):
./configure --without-llvm --without-icu --without-tcl --without-perl
--without-python --without-gssapi --without-pam --without-bsd-auth
--without-ldap --without-bonjour --without-selinux --without-systemd
--without-readline --without-libxml --without-libxslt --without-zlib
--without-lz4 --without-openssl

This is true for fresh builds against PG18 cuurent anti flags are

• --without-icu
• --without-readline
• --with-libedit-preferred
• --without-zlib

PG18 defaults to not building features unless explicitly opted in

so ./configure --without-icu --without-readline --without-zlib
should actually work.

I run the tests and 109 out of 111 passed! One failed because of missing postgis Extention!

On the Windows side however things get more delicate. First I am missing a proper build windows build environment.
My question to you, How can I test the builds safely against Windows? And second can you point out the things which might be a most problematic ones from your perspective. What I found out so far is that mkvcbuild.pl was removed in favor of meson in PG18

The CMakeLists.txt still calls:

execute_process(
  COMMAND perl mkvcbuild.pl
  WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/postgres/src/tools/msvc)

The windows Source files seems all to be present in PG18

libpq:

• pthread-win32.c
• win32.c

port (WIN32 conditional in CMakeLists.txt):

• win32gettimeofday.c (was gettimeofday.c in PG15, renamed in PG18 )
• inet_aton.c
• open.c
• pgsleep.c
• system.c
• dirmod.c
• win32common.c
• win32error.c
• win32ntdll.c
• win32setlocale.c
• win32stat.c

Removed from WIN32 block:

• getaddrinfo.c (removed in PG18, its's shiped with modern Windows already?)

Link libraries (stay unchanged — all still needed.)

• wsock32
• ws2_32
• wldap32
• secur32
• crypt32

From here I'm not sure how to go further?

Maybe using Meson on Windows?

[staticlibs]

@0814celsus

Thanks for the update! The details on changed files/functions are useful.

On Windows I think it would be better to use the same build setup as used for mainline Postgres release builds (so no longer perl-based AFAIU), unless that setup has some major incompatibilities with the way it is used from CMake.

[staticlibs]

@0814celsus

Hi! Just FYI, the libpq version was updated to 18.3 in #443, please let me know if you would like to continue with OAuth.