Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

This seems to work:

# syntax=docker/dockerfile:1
FROM alpine:3.20 AS builder

# Fails with 17.0 succeeds with 16.4
ARG PG=17.0

USER root
RUN apk update && apk add --no-cache --update-cache \
ca-certificates-bundle \
util-linux-dev \
execline-dev \
libedit-dev \
libxml2-dev \
clang17-dev \
llvm18-dev \
build-base \
net-tools \
zlib-dev \
autoconf \
automake \
busybox \
icu-dev \
clang17 \
llvm18 \
cmake \
bison \
meson \
ninja \
flex \
perl \
curl \
bash
WORKDIR /app

RUN curl -L https://github.com/openssl/openssl/releases/download/openssl-3.3.2/openssl-3.3.2.tar.gz

openssl.tar.gz

RUN mkdir openssl && tar --extract --file=openssl.tar.gz
--strip-components=1 --directory=openssl && rm openssl.tar.gz
RUN cd openssl && CC=clang CXX=clang++ perl ./Configure \
linux-$(uname -m) \
--prefix=/usr \
--libdir=lib \
--openssldir=/etc/ssl \
enable-ktls \
shared \
no-zlib \
no-async \
no-comp \
no-idea \
no-mdc2 \
no-rc5 \
no-ec2m \
no-sm2 \
no-sm4 \
no-ssl3 \
no-seed \
no-weak-ssl-ciphers \
-Wa,--noexecstack && \
perl configdata.pm --dump && \
make -j$(nproc) && make install

RUN curl -L https://ftp.postgresql.org/pub/source/v$PG/postgresql-$PG.tar.bz2

postgresql.tar.bz2

RUN mkdir postgresql && tar --extract --bzip2
--file=postgresql.tar.bz2 --strip-components=1 --directory=postgresql
&& rm postgresql.tar.bz2
COPY <<EOF ./pg.patch
Subject: [PATCH] Fix static build
---
Index: src/bin/initdb/Makefile
<+>UTF-8
===================================================================
diff --git a/src/bin/initdb/Makefile b/src/bin/initdb/Makefile
--- a/src/bin/initdb/Makefile (revision
c4b8a916f8a53379621825028532b038e004f891)
+++ b/src/bin/initdb/Makefile (date 1728466911177)
@@ -20,7 +20,8 @@
 # from libpq, else we have risks of version skew if we run with a libpq
 # shared library from a different PG version.  Define
 # USE_PRIVATE_ENCODING_FUNCS to ensure that that happens.
-override CPPFLAGS := -DUSE_PRIVATE_ENCODING_FUNCS -I$(libpq_srcdir)
-I$(top_srcdir)/src/timezone $(ICU_CFLAGS) $(CPPFLAGS)
+#override CPPFLAGS := -DUSE_PRIVATE_ENCODING_FUNCS -I$(libpq_srcdir)
-I$(top_srcdir)/src/timezone $(ICU_CFLAGS) $(CPPFLAGS)
+override CPPFLAGS := -I$(libpq_srcdir) -I$(top_srcdir)/src/timezone
$(ICU_CFLAGS) $(CPPFLAGS)

 # We need libpq only because fe_utils does.
 LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils
$(libpq_pgport) $(ICU_LIBS)
Index: src/common/Makefile
<+>UTF-8
===================================================================
diff --git a/src/common/Makefile b/src/common/Makefile
--- a/src/common/Makefile (revision c4b8a916f8a53379621825028532b038e004f891)
+++ b/src/common/Makefile (date 1728466354406)
@@ -143,7 +143,7 @@
 # Files in libpgcommon.a should use/export the "xxx_private" versions
 # of pg_char_to_encoding() and friends.
 #
-$(OBJS_FRONTEND): CPPFLAGS += -DUSE_PRIVATE_ENCODING_FUNCS
+#$(OBJS_FRONTEND): CPPFLAGS += -DUSE_PRIVATE_ENCODING_FUNCS

#
EOF

RUN apk update && apk add --no-cache --update-cache git
RUN cd postgresql && git apply ../pg.patch
RUN cd postgresql && export CC=clang && export CXX=clang++ && \
./configure --with-openssl --with-libedit-preferred
--with-uuid=e2fs --with-libxml --prefix=/usr/local && \
cd src/include && make && make install && \
cd ../common && make && make install && \
cd ../port && make && make install && \
cd ../interfaces/libpq && make && make install

COPY <<EOF ./main.cpp
#include <algorithm>
#include <iostream>
#include <chrono>
#include <thread>
#include <cstring>
#include <csignal>
#include <vector>

#include <libpq-fe.h>

void sig_handler(int /*_signo*/, siginfo_t * info, void * /*_ctx*/) {
raise(info->si_signo);
_exit(EXIT_FAILURE);
}

void registerSignalHandlers() {
std::vector<int> signals = {
// Signals for which the default action is "Core".
SIGABRT, // Abort signal from abort(3)
SIGBUS, // Bus error (bad memory access)
SIGFPE, // Floating point exception
SIGILL, // Illegal Instruction
SIGIOT, // IOT trap. A synonym for SIGABRT
SIGQUIT, // Quit from keyboard
SIGSEGV, // Invalid memory reference
SIGSYS, // Bad argument to routine (SVr4)
SIGTRAP, // Trace/breakpoint trap
SIGXCPU, // CPU time limit exceeded (4.2BSD)
SIGXFSZ, // File size limit exceeded (4.2BSD)
SIGTERM
};

for (size_t i = 0; i < signals.size(); ++i) {
struct sigaction action;
memset(&action, 0, sizeof action);
action.sa_flags = static_cast<int>(SA_SIGINFO | SA_ONSTACK |
SA_NODEFER | SA_RESETHAND);
sigfillset(&action.sa_mask);
sigdelset(&action.sa_mask, signals[i]);
action.sa_sigaction = &sig_handler;
sigaction(signals[i], &action, nullptr);
}
}

int main() {
registerSignalHandlers();
std::cout << "start" << std::endl;
PGconn *conn = PQconnectdb("");
if (conn == NULL) {
std::cout << "fail" << std::endl;
} else {
if (PQstatus(conn) == CONNECTION_OK) {
std::cout << "success" << std::endl;
} else {
fprintf(stderr, "fail: %s", PQerrorMessage(conn));
}
PQfinish(conn);
}
while (true) {
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
}
return 0;
}
EOF

# Fails with "-static" succeeds without
RUN CC=clang CXX=clang++ clang++ \
-std=c++20 \
-static \
-flto \
-O3 \
-march=native \
-Wpedantic \
-Wall \
-Wextra \
-Wsign-conversion \
-Wconversion \
-o main main.cpp \
-lpq \
-lpgcommon \
-lpgport \
-lssl \
-lcrypto \
-lpthread \
-ldl
ENTRYPOINT ["./main"]
CMD []

On Wed, Oct 9, 2024 at 10:10 AM Mikael Sand <msand@seaber.io> wrote:

Here is a simpler Dockerfile for making a static build of a c++ application
using libpq from postgresql 17

# syntax=docker/dockerfile:1

FROM alpine:3.20 AS builder
USER root
WORKDIR /app
RUN apk update && apk add --no-cache --update-cache \
openssl-libs-static \
e2fsprogs-dev \
readline-dev \
libxml2-dev \
openssl-dev \
zlib-dev \
clang17 \
icu-dev \
bison \
flex \
perl \
make \
curl \
git

ARG PG=17.0
RUN curl -L https://ftp.postgresql.org/pub/source/v$PG/postgresql-$PG.tar.bz2 > postgresql.tar.bz2
RUN mkdir postgresql && tar --extract --bzip2 --file=postgresql.tar.bz2 --strip-components=1 --directory=postgresql && rm postgresql.tar.bz2
COPY <<EOF ./pg.patch
Subject: [PATCH] Fix static build
---
Index: src/bin/initdb/Makefile
<+>UTF-8
===================================================================
diff --git a/src/bin/initdb/Makefile b/src/bin/initdb/Makefile
--- a/src/bin/initdb/Makefile (revision c4b8a916f8a53379621825028532b038e004f891)
+++ b/src/bin/initdb/Makefile (date 1728466911177)
@@ -20,7 +20,8 @@
# from libpq, else we have risks of version skew if we run with a libpq
# shared library from a different PG version.  Define
# USE_PRIVATE_ENCODING_FUNCS to ensure that that happens.
-override CPPFLAGS := -DUSE_PRIVATE_ENCODING_FUNCS -I$(libpq_srcdir) -I$(top_srcdir)/src/timezone $(ICU_CFLAGS) $(CPPFLAGS)
+#override CPPFLAGS := -DUSE_PRIVATE_ENCODING_FUNCS -I$(libpq_srcdir) -I$(top_srcdir)/src/timezone $(ICU_CFLAGS) $(CPPFLAGS)
+override CPPFLAGS := -I$(libpq_srcdir) -I$(top_srcdir)/src/timezone $(ICU_CFLAGS) $(CPPFLAGS)

# We need libpq only because fe_utils does.
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) $(ICU_LIBS)
Index: src/common/Makefile
<+>UTF-8
===================================================================
diff --git a/src/common/Makefile b/src/common/Makefile
--- a/src/common/Makefile (revision c4b8a916f8a53379621825028532b038e004f891)
+++ b/src/common/Makefile (date 1728466354406)
@@ -143,7 +143,7 @@
# Files in libpgcommon.a should use/export the "xxx_private" versions
# of pg_char_to_encoding() and friends.
#
-$(OBJS_FRONTEND): CPPFLAGS += -DUSE_PRIVATE_ENCODING_FUNCS
+#$(OBJS_FRONTEND): CPPFLAGS += -DUSE_PRIVATE_ENCODING_FUNCS

#
EOF
RUN cd postgresql && git apply ../pg.patch
RUN cd postgresql && export CC=clang && export CXX=clang++ && \
./configure --with-openssl --with-libedit-preferred --with-uuid=e2fs --with-libxml --prefix=/usr/local && \
cd src/include && make && make install && \
cd ../common && make && make install && \
cd ../port && make && make install && \
cd ../interfaces/libpq && make && make install

COPY <<EOF ./main.cpp
#include <algorithm>
#include <iostream>
#include <chrono>
#include <thread>
#include <cstring>
#include <csignal>
#include <vector>

#include <libpq-fe.h>

void sig_handler(int /*_signo*/, siginfo_t * info, void * /*_ctx*/) {
raise(info->si_signo);
_exit(EXIT_FAILURE);
}

void registerSignalHandlers() {
std::vector<int> signals = {
// Signals for which the default action is "Core".
SIGABRT, // Abort signal from abort(3)
SIGBUS, // Bus error (bad memory access)
SIGFPE, // Floating point exception
SIGILL, // Illegal Instruction
SIGIOT, // IOT trap. A synonym for SIGABRT
SIGQUIT, // Quit from keyboard
SIGSEGV, // Invalid memory reference
SIGSYS, // Bad argument to routine (SVr4)
SIGTRAP, // Trace/breakpoint trap
SIGXCPU, // CPU time limit exceeded (4.2BSD)
SIGXFSZ, // File size limit exceeded (4.2BSD)
SIGTERM
};

for (size_t i = 0; i < signals.size(); ++i) {
struct sigaction action;
memset(&action, 0, sizeof action);
action.sa_flags = static_cast<int>(SA_SIGINFO | SA_ONSTACK | SA_NODEFER | SA_RESETHAND);
sigfillset(&action.sa_mask);
sigdelset(&action.sa_mask, signals[i]);
action.sa_sigaction = &sig_handler;
sigaction(signals[i], &action, nullptr);
}
}

int main() {
registerSignalHandlers();
std::cout << "start" << std::endl;
PGconn *conn = PQconnectdb("");
if (conn == NULL) {
std::cout << "fail" << std::endl;
} else {
if (PQstatus(conn) == CONNECTION_OK) {
std::cout << "success" << std::endl;
} else {
fprintf(stderr, "fail: %s", PQerrorMessage(conn));
}
PQfinish(conn);
}
while (true) {
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
}
return 0;
}
EOF

RUN CC=clang CXX=clang++ clang++ \
-std=c++20 \
-static \
-flto \
-O3 \
-march=native \
-Wpedantic \
-Wall \
-Wextra \
-Wsign-conversion \
-Wconversion \
-o main main.cpp \
-lpq \
-lpgcommon \
-lpgport \
-lssl \
-lcrypto \
-lpthread \
-ldl
ENTRYPOINT ["./main"]
CMD []

And another to verify that the build is actually static, run inside wolfi
undistro made for running a single static binary (and using clang-19
instead of 17):

# syntax=docker/dockerfile:1

FROM cgr.dev/chainguard/git:latest-dev AS builder
USER root
WORKDIR /app
RUN apk update && apk add --no-cache --update-cache \
e2fsprogs-dev \
clang-19-dev \
execline-dev \
libedit-dev \
libxml2-dev \
openssl-dev \
llvm-19-dev \
build-base \
clang-19 \
zlib-dev \
icu-dev \
llvm-19 \
bison \
flex \
perl \
curl

ARG PG=17.0
RUN curl -L https://ftp.postgresql.org/pub/source/v$PG/postgresql-$PG.tar.bz2 > postgresql.tar.bz2
RUN mkdir postgresql && tar --extract --bzip2 --file=postgresql.tar.bz2 --strip-components=1 --directory=postgresql && rm postgresql.tar.bz2
COPY <<EOF ./pg.patch
Subject: [PATCH] Fix static build
---
Index: src/bin/initdb/Makefile
<+>UTF-8
===================================================================
diff --git a/src/bin/initdb/Makefile b/src/bin/initdb/Makefile
--- a/src/bin/initdb/Makefile (revision c4b8a916f8a53379621825028532b038e004f891)
+++ b/src/bin/initdb/Makefile (date 1728466911177)
@@ -20,7 +20,8 @@
# from libpq, else we have risks of version skew if we run with a libpq
# shared library from a different PG version.  Define
# USE_PRIVATE_ENCODING_FUNCS to ensure that that happens.
-override CPPFLAGS := -DUSE_PRIVATE_ENCODING_FUNCS -I$(libpq_srcdir) -I$(top_srcdir)/src/timezone $(ICU_CFLAGS) $(CPPFLAGS)
+#override CPPFLAGS := -DUSE_PRIVATE_ENCODING_FUNCS -I$(libpq_srcdir) -I$(top_srcdir)/src/timezone $(ICU_CFLAGS) $(CPPFLAGS)
+override CPPFLAGS := -I$(libpq_srcdir) -I$(top_srcdir)/src/timezone $(ICU_CFLAGS) $(CPPFLAGS)

# We need libpq only because fe_utils does.
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) $(ICU_LIBS)
Index: src/common/Makefile
<+>UTF-8
===================================================================
diff --git a/src/common/Makefile b/src/common/Makefile
--- a/src/common/Makefile (revision c4b8a916f8a53379621825028532b038e004f891)
+++ b/src/common/Makefile (date 1728466354406)
@@ -143,7 +143,7 @@
# Files in libpgcommon.a should use/export the "xxx_private" versions
# of pg_char_to_encoding() and friends.
#
-$(OBJS_FRONTEND): CPPFLAGS += -DUSE_PRIVATE_ENCODING_FUNCS
+#$(OBJS_FRONTEND): CPPFLAGS += -DUSE_PRIVATE_ENCODING_FUNCS

#
EOF
RUN cd postgresql && git apply ../pg.patch
RUN cd postgresql && export CC=clang && export CXX=clang++ && \
./configure --with-openssl --with-libedit-preferred --with-uuid=e2fs --with-libxml --prefix=/usr/local && \
cd src/include && make && make install && \
cd ../common && make && make install && \
cd ../port && make && make install && \
cd ../interfaces/libpq && make && make install

COPY <<EOF ./main.cpp
#include <algorithm>
#include <iostream>
#include <chrono>
#include <thread>
#include <cstring>
#include <csignal>
#include <vector>

#include <libpq-fe.h>

void sig_handler(int /*_signo*/, siginfo_t * info, void * /*_ctx*/) {
raise(info->si_signo);
_exit(EXIT_FAILURE);
}

void registerSignalHandlers() {
std::vector<int> signals = {
// Signals for which the default action is "Core".
SIGABRT, // Abort signal from abort(3)
SIGBUS, // Bus error (bad memory access)
SIGFPE, // Floating point exception
SIGILL, // Illegal Instruction
SIGIOT, // IOT trap. A synonym for SIGABRT
SIGQUIT, // Quit from keyboard
SIGSEGV, // Invalid memory reference
SIGSYS, // Bad argument to routine (SVr4)
SIGTRAP, // Trace/breakpoint trap
SIGXCPU, // CPU time limit exceeded (4.2BSD)
SIGXFSZ, // File size limit exceeded (4.2BSD)
SIGTERM
};

for (size_t i = 0; i < signals.size(); ++i) {
struct sigaction action;
memset(&action, 0, sizeof action);
action.sa_flags = static_cast<int>(SA_SIGINFO | SA_ONSTACK | SA_NODEFER | SA_RESETHAND);
sigfillset(&action.sa_mask);
sigdelset(&action.sa_mask, signals[i]);
action.sa_sigaction = &sig_handler;
sigaction(signals[i], &action, nullptr);
}
}

int main() {
registerSignalHandlers();
std::cout << "start" << std::endl;
PGconn *conn = PQconnectdb("");
if (conn == NULL) {
std::cout << "fail" << std::endl;
} else {
if (PQstatus(conn) == CONNECTION_OK) {
std::cout << "success" << std::endl;
} else {
fprintf(stderr, "fail: %s", PQerrorMessage(conn));
}
PQfinish(conn);
}
while (true) {
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
}
return 0;
}
EOF

RUN ln /usr/lib/llvm-19/lib/LLVMgold.so /usr/lib/LLVMgold.so
RUN CC=clang CXX=clang++ clang++ \
-std=c++20 \
-static \
-flto \
-O3 \
-march=native \
-Wpedantic \
-Wall \
-Wextra \
-Wsign-conversion \
-Wconversion \
-o main main.cpp \
-lpq \
-lpgcommon \
-lpgport \
-lssl \
-lcrypto \
-lpthread \
-ldl
ENTRYPOINT ["./main"]
CMD []

FROM cgr.dev/chainguard/static:latest AS runner
WORKDIR /app
COPY --from=builder --chown=65532:65532 /app/main .
USER 65532
ENTRYPOINT ["./main"]
CMD []

Hope this can help somebody

Br Mikael

On Wed, Oct 9, 2024 at 11:51 AM Mikael Sand <msand@seaber.io> wrote:

Sorry for not having a properly minimal reproduction to begin with, seems I
can't reduce this any further at the moment at least:

# syntax=docker/dockerfile:1

Hi Mikael,

Sorry for not having a properly minimal reproduction to begin with, seems I can't reduce this any further at the moment at least:
[...]

Is there any particular reason for using pg.patch in your Dockerfile
or not using well regarded / official [1]I don't use Docker a lot at present and I'm not sure how oficial these images really are and/or who is maintaining them.[2]see also /messages/by-id/05d86238a6e46da637c415bd261451de@postgresql.org Docker images [3]https://hub.docker.com/_/postgres ?

It seems to me that basically you are complaining that you patched
Postgres and it didn't compile after that.

[1]: I don't use Docker a lot at present and I'm not sure how oficial these images really are and/or who is maintaining them.
these images really are and/or who is maintaining them.
[2]: see also /messages/by-id/05d86238a6e46da637c415bd261451de@postgresql.org
[3]: https://hub.docker.com/_/postgres

--
Best regards,
Aleksander Alekseev

Hi Mikael,

Please use the "Reply to All" button - send your emails to the mailing
list, not to me directly.

The official packages and the original source tar both fail to compile, the patch enables it to compile.

I just run:

```
docker run -e POSTGRES_PASSWORD=foo postgres:17.0-alpine3.20
```

... on my spare Raspberry Pi and it works just fine. Also PostgreSQL
is tested on Alpine [1]https://buildfarm.postgresql.org/cgi-bin/show_status.pl.

It means that your custom Dockerfile is wrong. I suggest comparing it
with the one published on hub.docker.com [2]https://github.com/docker-library/postgres/blob/172544062d1031004b241e917f5f3f9dfebc0df5/17/alpine3.20/Dockerfile. Also I would recommend
reading the documentation section about compiling PostgreSQL from
source code [3]https://www.postgresql.org/docs/current/installation.html. I have a few scripts [4]https://github.com/afiskon/pgscripts/ that may help you correct
the Dockerfile as well [4]https://github.com/afiskon/pgscripts/.

All this of course assumes that you really want your custom Dockerfile
instead of using "official" / well regarded 17.0-alpine3.20 one.

[1]: https://buildfarm.postgresql.org/cgi-bin/show_status.pl
[2]: https://github.com/docker-library/postgres/blob/172544062d1031004b241e917f5f3f9dfebc0df5/17/alpine3.20/Dockerfile
[3]: https://www.postgresql.org/docs/current/installation.html
[4]: https://github.com/afiskon/pgscripts/

--
Best regards,
Aleksander Alekseev

Hello Aleksander

This is for compiling a c++ application that uses libpq with the -static
flag, the server compiles fine.

Br Mikael

On Thu, Oct 10, 2024 at 12:16 PM Aleksander Alekseev <
aleksander@timescale.com> wrote:

E.g. this fails to compile:

FROM chainguard/git:latest-dev AS builder
USER root
RUN apk update && apk add --no-cache --update-cache clang-19
postgresql-17-dev openssl-dev
COPY <<EOF ./main.cpp
#include<libpq-fe.h>
int main(){return PQconnectdb("")==NULL;}
EOF
RUN clang++ -static -o main main.cpp -lpq -lpgcommon -lpgport -lssl -lcrypto

On Thu, Oct 10, 2024 at 12:16 PM Aleksander Alekseev <
aleksander@timescale.com> wrote:

Hi Mikael,

This is for compiling a c++ application that uses libpq with the -static flag, the server compiles fine.

OK. I couldn't quite do this because the only Linux machine I have at
the moment runs Raspbian and there doesn't seem to be a static glibc
available for it. But here is how to achieve what you want *in
theory*.

First compile Postgres from the source code, for instance (change the
flags as needed, you probably want a release build):

```
# sudo apt install clang-16
# git clean -dfx
CFLAGS="-static" meson setup --buildtype debug -Dicu=disabled
-Dldap=disabled -Dreadline=disabled -Dzlib=disabled -Dlz4=disabled
-Dprefix=/home/eax/pginstall build
ninja -C build
```

I recommend using Meson and Ninja. Autotools and Make are still
supported but are slow and probably will be gone in a few years. The
"ninja -C build" steps fail for me with various errors about the need
for static glibc but I think it should work on other distributions and
architectures.

Next install Postgres. I use a script for this [1]https://github.com/afiskon/pgscripts/:

```
~/pgscripts/single-install-meson.sh
```

Change the script as needed - you probably don't need pg_ctl,
createdb, etc for your task.

Now in order to compile and execute your C++ program:

```
export PRFX=/home/eax/pginstall
g++ -g -Wall -o test_libpq -I$PRFX/include
-L$PRFX/lib/aarch64-linux-gnu/ test_libpq.cpp -lpq -static
LD_LIBRARY_PATH=$PRFX/lib/aarch64-linux-gnu/ ./test_libpq
```

This being said I don't recall seeing anything about the support of
static linking of libpq in the documentation [2]https://www.postgresql.org/docs/current/libpq.html. To my knowledge this
is not officially supported / tested / maintained which means you and
your colleagues are on your own with the -static flag. Since you are
already using Docker, perhaps the easiest thing to do would be to back
the application and all its dependencies (dynamically linked) in a
Docker container.

Good luck.

[1]: https://github.com/afiskon/pgscripts/
[2]: https://www.postgresql.org/docs/current/libpq.html

--
Best regards,
Aleksander Alekseev

Hmm, so is static linking of all applications that include libpq
intentionally broken?

At least googling around, it seems like a relatively common practice.

At least it seems that 2ndquadrant builds and uses libpq statically.
/messages/by-id/20327.1501536978@sss.pgh.pa.us

I don't see how your script would fix the naming of these functions, can
you elaborate?

Here is a Dockerfile demonstrating the same issue with
the postgres:17.0-alpine3.20 image:

FROM postgres:17.0-alpine3.20 AS builder
USER root
WORKDIR /app
RUN apk update && apk add --no-cache --update-cache \
openssl-libs-static \
cyrus-sasl-static \
libevent-static \
libxml2-static \
libedit-static \
libxslt-static \
sqlite-static \
openldap-dev \
libxslt-dev \
libxml2-dev \
zstd-static \
zlib-static \
libedit-dev \
openssl-dev \
lz4-static \
e2fsprogs \
zstd-dev \
keyutils \
zlib-dev \
gdbm-dev \
clang17 \
lz4-dev \
libldap \
bison \
curl \
perl \
make

COPY <<EOF ./main.cpp
#include<libpq-fe.h>
int main(){return PQconnectdb("")==NULL;}
EOF

RUN curl -L https://kerberos.org/dist/krb5/1.21/krb5-1.21.3.tar.gz >
krb5-1.21.3.tar.gz && tar xf krb5-1.21.3.tar.gz
RUN cd krb5-1.21.3/src && \
./configure && make && make install && \
./configure --disable-shared --enable-static && make && make install

RUN curl -L https://github.com/cyrusimap/cyrus-sasl/releases/download/cyrus-sasl-2.1.28/cyrus-sasl-2.1.28.tar.gz

cyrus-sasl-2.1.28.tar.gz

RUN tar xf cyrus-sasl-2.1.28.tar.gz && cd cyrus-sasl-2.1.28 &&
./configure --enable-static && make && make install

RUN clang++ -fno-common -static -o main main.cpp \
-L/usr/local/lib -lpq -L/usr/lib/llvm15/lib -L/usr/local/lib
-lpgcommon -lpgport -lgssapi_krb5 -lm -lldap -lcrypto -ldl -pthread \
-lldap -levent -lsasl2 -lssl -lcrypto -llber -levent \
-lsqlite3 \
-L/usr/local/lib -lkrb5 -lk5crypto -lcom_err -lkrb5support \
-lgdbm \
-lssl \
-lgssapi_krb5

On Thu, Oct 10, 2024 at 1:51 PM Aleksander Alekseev <
aleksander@timescale.com> wrote:

This seems to be the commit that caused this:
https://github.com/postgres/postgres/commit/b6c7cfac88c47a9194d76f3d074129da3c46545a

So it seems the commit is related to static linking.

Is it somehow possible to statically link in a version of pgcommon with
correct function names into libpq?

I don't mind having this patch in use too much, I have a functional build
and nothing to worry about, no luck necessary :)

But, I expect a lot of unnecessary churn in the community if this is not
fixed properly.
Not all applications run inside docker, and not all can use dynamic linking.
People can dislike static linking for whatever reason, but that doesn't
remove the need for it (or both) in some cases and the desire for it in
others.

Judging from the commit, it doesn't seem like static linking is
intentionally broken.
Should a test be added so as not to break this by mistake?
I must say I don't fully comprehend the context of that commit.

On Thu, Oct 10, 2024 at 5:06 PM Mikael Sand <msand@seaber.io> wrote:

E.g. this works with 16.4 but fails in 17.0:

FROM postgres:16.4-alpine3.20 AS builder
USER root
WORKDIR /app
RUN apk update && apk add --no-cache --update-cache \
openssl-libs-static \
cyrus-sasl-static \
libevent-static \
libxml2-static \
libedit-static \
libxslt-static \
sqlite-static \
openldap-dev \
libxslt-dev \
libxml2-dev \
zstd-static \
zlib-static \
libedit-dev \
openssl-dev \
lz4-static \
e2fsprogs \
zstd-dev \
keyutils \
zlib-dev \
gdbm-dev \
clang17 \
lz4-dev \
libldap \
bison \
curl \
perl \
make

COPY <<EOF ./main.cpp
#include<libpq-fe.h>
int main(){return PQconnectdb("")==NULL;}
EOF

RUN curl -L https://kerberos.org/dist/krb5/1.21/krb5-1.21.3.tar.gz >
krb5-1.21.3.tar.gz && tar xf krb5-1.21.3.tar.gz
RUN cd krb5-1.21.3/src && \
./configure && make && make install && \
./configure --disable-shared --enable-static && make && make install

RUN curl -L https://github.com/cyrusimap/cyrus-sasl/releases/download/cyrus-sasl-2.1.28/cyrus-sasl-2.1.28.tar.gz

cyrus-sasl-2.1.28.tar.gz

RUN tar xf cyrus-sasl-2.1.28.tar.gz && cd cyrus-sasl-2.1.28 &&
./configure --enable-static && make && make install

RUN clang++ -fno-common -static -o main main.cpp \
-L/usr/local/lib -lpq -lpgcommon -lpgport \
-lldap -lsasl2 -lssl -lcrypto -llber \
-lgssapi_krb5 \
-lkrb5 -lk5crypto -lcom_err -lkrb5support \
-lgdbm

On Thu, Oct 10, 2024 at 5:28 PM Mikael Sand <msand@seaber.io> wrote:

Mikael Sand <msand@seaber.io> writes:

RUN clang++ -fno-common -static -o main main.cpp \
-L/usr/local/lib -lpq -lpgcommon -lpgport \
-lldap -lsasl2 -lssl -lcrypto -llber \
-lgssapi_krb5 \
-lkrb5 -lk5crypto -lcom_err -lkrb5support \
-lgdbm

The short answer here is that your link recipe is wrong, and has been
wrong right along, though you accidentally got away with it before.
The modules within libpq expect to be linked with libpgcommon_shlib
and libpgport_shlib, not libpgcommon/libpgport.

Having external code that needs to know explicitly about every one
of a library's dependencies is one of many reasons why we discourage
static linking.

regards, tom lane

Mikael,

On Thu, Oct 10, 2024 at 8:49 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Mikael Sand <msand@seaber.io> writes:

RUN clang++ -fno-common -static -o main main.cpp \
-L/usr/local/lib -lpq -lpgcommon -lpgport \
-lldap -lsasl2 -lssl -lcrypto -llber \
-lgssapi_krb5 \
-lkrb5 -lk5crypto -lcom_err -lkrb5support \
-lgdbm

The short answer here is that your link recipe is wrong, and has been
wrong right along, though you accidentally got away with it before.
The modules within libpq expect to be linked with libpgcommon_shlib
and libpgport_shlib, not libpgcommon/libpgport.

Having external code that needs to know explicitly about every one
of a library's dependencies is one of many reasons why we discourage
static linking.

May I ask what problem you are trying to solve with static linking in
the first place?

--
Best regards,
Aleksander Alekseev

We use static linking and link time optimization to squeeze the last bits
of performance out of the code in our most performance-critical queries,
and it simplifies our security audits to have a static binary running
inside chainguard/static as the data we handle is sensitive/business
critical.

Wonderful, the build works when using -lpgcommon_shlib -lpgport_shlib
So this appears to imply that the output of `pkg-config -libs -static
libpq`is incorrect?

/app # pkg-config -libs -static libpq

-L/usr/local/lib -lpq -L/usr/lib/llvm15/lib -L/usr/local/lib -lpgcommon
-lpgport -lgssapi_krb5 -lm -lldap -lssl -lcrypto -ldl -pthread

On Thu, Oct 10, 2024 at 7:54 PM Aleksander Alekseev <
aleksander@timescale.com> wrote:

Just for reference in case anyone else who utilizes static linking for any
reason hits upon this issue, here is a working Dockerfile for libpq /
postgresql 17

FROM postgres:17.0-alpine3.20 AS builder
USER root
WORKDIR /app
RUN apk update && apk add --no-cache --update-cache \
openssl-libs-static \
libevent-static \
libxml2-static \
libedit-static \
libxslt-static \
sqlite-static \
openldap-dev \
libxslt-dev \
libxml2-dev \
libedit-dev \
openssl-dev \
zstd-static \
zlib-static \
lz4-static \
e2fsprogs \
keyutils \
zstd-dev \
zlib-dev \
gdbm-dev \
clang17 \
lz4-dev \
libldap \
bison \
curl \
perl \
make

COPY <<EOF ./main.cpp
#include<libpq-fe.h>
int main(){return PQconnectdb("")==NULL;}
EOF

ARG KRB5=1.21.3
ARG KRB5MAJMIN=1.21
RUN curl -L https://kerberos.org/dist/krb5/$KRB5MAJMIN/krb5-$KRB5.tar.gz
| tar xzf -
RUN cd krb5-$KRB5/src && \
./configure && make && make install && \
./configure --disable-shared --enable-static && make && make install

ARG SASL=2.1.28
RUN curl -L https://github.com/cyrusimap/cyrus-sasl/releases/download/cyrus-sasl-$SASL/cyrus-sasl-$SASL.tar.gz
| tar xzf -
RUN cd cyrus-sasl-$SASL && ./configure --enable-static && make && make install

RUN clang++ -static -o main main.cpp \
-L/usr/local/lib -lpq -lpgcommon_shlib -lpgport_shlib \
-lldap -lsasl2 -lssl -lcrypto -llber \
-lgssapi_krb5 \
-lkrb5 -lk5crypto -lcom_err -lkrb5support \
-lgdbm

Mikael Sand <msand@seaber.io> writes:

Wonderful, the build works when using -lpgcommon_shlib -lpgport_shlib
So this appears to imply that the output of `pkg-config -libs -static
libpq`is incorrect?

/app # pkg-config -libs -static libpq
-L/usr/local/lib -lpq -L/usr/lib/llvm15/lib -L/usr/local/lib -lpgcommon
-lpgport -lgssapi_krb5 -lm -lldap -lssl -lcrypto -ldl -pthread

Hmm ... does seem that way. Peter apparently deliberately made it so
in 55392bc5b, but there's no real justification for removing _shlib
in the commit message. Peter?

regards, tom lane

Aleksander, do you have something against static linking or am I reading
you wrong?
I've seen this sentiment in several places but never understood why anyone
would hold this position. Could you elaborate?

At least for executables intended to be run in production inside
containers, they usually run in isolation, what other executables could
they share libraries with?
Should we leave performance on the table, given the energy and resource
requirements it implies?

Certainly, for libraries intended to be distributed in e.g. Linux
distributions and be depended on in end-user executables, it makes sense to
have shared libraries and simplify mitigating security issues.

It appears to me that there are many cases where static linking and link
time optimization would be more appropriate than dynamic.
When the executables never have a chance to share libraries with any other
process and are rebuilt whenever a new version is deployed what benefits
does dynamic linking provide over static + lto?

I expect I'm missing something crucial, but I have spent some time trying
to understand this issue deeper, and still find myself mystified.
Please enlighten me.

Br Mikael

On Thu, Oct 10, 2024 at 8:29 PM Mikael Sand <msand@seaber.io> wrote:

Tom

Would this be an acceptable patch?

Subject: [PATCH] Fix static build
---
Index: src/Makefile.shlib
<+>UTF-8
===================================================================
diff --git a/src/Makefile.shlib b/src/Makefile.shlib
--- a/src/Makefile.shlib (revision fd64ed60b62697984bb69a09a3ae19fbe2905eb6)
+++ b/src/Makefile.shlib (date 1728590127913)
@@ -354,7 +354,7 @@
 # those that point inside the build or source tree.  Use sort to
 # remove duplicates.  Also record the -l flags necessary for static
 # linking, but not those already covered by Requires.private.
- echo 'Libs.private: $(sort $(filter-out -L.% -L$(top_srcdir)/%,$(filter
-L%,$(LDFLAGS) $(SHLIB_LINK)))) $(filter-out
$(PKG_CONFIG_REQUIRES_PRIVATE:lib%=-l%),$(filter
-l%,$(SHLIB_LINK_INTERNAL:%_shlib=%) $(SHLIB_LINK)))' >>$@
+ echo 'Libs.private: $(sort $(filter-out -L.% -L$(top_srcdir)/%,$(filter
-L%,$(LDFLAGS) $(SHLIB_LINK)))) $(filter-out
$(PKG_CONFIG_REQUIRES_PRIVATE:lib%=-l%),$(filter -l%,$(SHLIB_LINK_INTERNAL)
$(SHLIB_LINK)))' >>$@

##

Br Mikael

On Thu, Oct 10, 2024 at 9:08 PM Mikael Sand <msand@seaber.io> wrote: