undefined symbol when installing pgcrypto on 16.1

Michael Nolan <htfoot@gmail.com> writes:

This is on AlmaLinux 9.3, installing postgresql from source code.
In PG 16.1 when I try to install pgcrypto, the modules compile but I
get this error when running checks:

CREATE EXTENSION pgcrypto;
+ERROR: could not load library
"/home/postgres/src/postgresql-16.1/tmp_install/usr/local/pgsql/lib/
pgcrypto.so": /home/postgres/src/postgresql-16.1/tmp_install/usr/local/pgsql/lib/pgcrypto.so:
undefined symbol: EVP_cast5_cbc

That should be supplied by OpenSSL. Are you using some weird
version of openssl? Did you get any warnings during build?

regards, tom lane

Sorry for the delay in responding, network issues kept me offline for
several days.

These are the openssl packages installed from the Almalinux 9 repositories:

apr-util-openssl.x86_64 1.6.1-23.el9 @appstream
openssl.x86_64 1:3.0.7-24.el9 @anaconda
openssl-devel.x86_64 1:3.0.7-24.el9 @appstream
openssl-libs.x86_64 1:3.0.7-24.el9 @anaconda

I reinstalled the devel package, still get the same unresolved symbol error.

Mike Nolan

On 18 Jan 2024, at 00:24, Michael Nolan <htfoot@gmail.com> wrote:

Sorry for the delay in responding, network issues kept me offline for
several days.

These are the openssl packages installed from the Almalinux 9 repositories:

apr-util-openssl.x86_64 1.6.1-23.el9 @appstream
openssl.x86_64 1:3.0.7-24.el9 @anaconda
openssl-devel.x86_64 1:3.0.7-24.el9 @appstream
openssl-libs.x86_64 1:3.0.7-24.el9 @anaconda

I reinstalled the devel package, still get the same unresolved symbol error.

My memory is failing me, but isn't CAST5 only available when loading the legacy
provider in OpenSSL 3? Which providers are loaded in your openssl config?

--
Daniel Gustafsson

On Wed, Jan 17, 2024 at 5:32 PM Daniel Gustafsson <daniel@yesql.se> wrote:

On 18 Jan 2024, at 00:24, Michael Nolan <htfoot@gmail.com> wrote:

Sorry for the delay in responding, network issues kept me offline for
several days.

These are the openssl packages installed from the Almalinux 9 repositories:

apr-util-openssl.x86_64 1.6.1-23.el9 @appstream
openssl.x86_64 1:3.0.7-24.el9 @anaconda
openssl-devel.x86_64 1:3.0.7-24.el9 @appstream
openssl-libs.x86_64 1:3.0.7-24.el9 @anaconda

I reinstalled the devel package, still get the same unresolved symbol error.

My memory is failing me, but isn't CAST5 only available when loading the legacy
provider in OpenSSL 3? Which providers are loaded in your openssl config?

--
Daniel Gustafsson

The legacy providers were enabled, disabling them worked. (The build
took a lot longer, too, but there were no obvious messages about
problems with the legacy providers enabled untilI ran the check.)

Thanks, Daniel.

On 18 Jan 2024, at 00:59, Michael Nolan <htfoot@gmail.com> wrote:
On Wed, Jan 17, 2024 at 5:32 PM Daniel Gustafsson <daniel@yesql.se> wrote:

On 18 Jan 2024, at 00:24, Michael Nolan <htfoot@gmail.com> wrote:

I reinstalled the devel package, still get the same unresolved symbol error.

My memory is failing me, but isn't CAST5 only available when loading the legacy
provider in OpenSSL 3? Which providers are loaded in your openssl config?

The legacy providers were enabled, disabling them worked. (The build
took a lot longer, too, but there were no obvious messages about
problems with the legacy providers enabled untilI ran the check.)

That's surprising, I expected that it would require the legacy provider be
loaded, not the other way around. OpenSSL works in mysterious ways though.

--
Daniel Gustafsson

On Thu, Jan 18, 2024 at 5:03 AM Daniel Gustafsson <daniel@yesql.se> wrote:

On 18 Jan 2024, at 00:59, Michael Nolan <htfoot@gmail.com> wrote:
On Wed, Jan 17, 2024 at 5:32 PM Daniel Gustafsson <daniel@yesql.se> wrote:

That's surprising, I expected that it would require the legacy provider be
loaded, not the other way around. OpenSSL works in mysterious ways though.

--
Daniel Gustafsson

Turns out that was premature, when I rebuilt postgres to add other
modules I needed, like plperl, the missing symbols issue came back.
So now I'm not sure why it (apparently briefly) worked. I'm assuming
it's an issue with the AlmaLinux 9 version of openssl. I don't know if
compiling it from source would fix the problem, but I may try that
after checking with the AlmaLinux folks.

Would I be correct to assume the postgresql build farm doesn't include
AlmaLinux 9 to test it.

Mike Nolan

Michael Nolan <htfoot@gmail.com> writes:

Would I be correct to assume the postgresql build farm doesn't include
AlmaLinux 9 to test it.

No, you wouldn't:

https://buildfarm.postgresql.org/cgi-bin/show_status.pl

Looks like we have aarch64 and ppc64 machines running Alma 8 and 9.
No x86 though, which might matter for such a low-level failure
as this.

regards, tom lane

On Thu, Jan 18, 2024 at 9:51 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Looks like we have aarch64 and ppc64 machines running Alma 8 and 9.
No x86 though, which might matter for such a low-level failure
as this.

So I guess that'll be on the list to add to the build farm at some
point? (My Xanthian 'talent' of finding bugs continues, lucky me!)

I get the impression AlmaLinux 9 isn't as widely used as Centos was
yet. (I tend to follow my son's recommendation for which Linux distro
to use, he's an SRE currently at YugaByte, which uses postgresql as a
front end to their custom storage back end.)

Mike Nolan

Writing or debugging makefiles is something I haven't done much of,
but as best I can figure out the problem is that the libcrypto.so file
isn't being linked in, though this line in the Makefile in pgcrypto
seems to say should be:

SHLIB_LINK += $(filter -lcrypto -lz, $(LIBS))

I'm guessing it is supposed to pick up the LIBS list from the top
Makefile, but isn't.

If I add this:

SHLIB_LINK += -lcrypto

'make clean' now passes all tests.
--
Mike Nolan

Sorry, I meant 'make check'. :sigh:

Michael Nolan <htfoot@gmail.com> writes:

Writing or debugging makefiles is something I haven't done much of,
but as best I can figure out the problem is that the libcrypto.so file
isn't being linked in, though this line in the Makefile in pgcrypto
seems to say should be:

SHLIB_LINK += $(filter -lcrypto -lz, $(LIBS))

I'm guessing it is supposed to pick up the LIBS list from the top
Makefile, but isn't.

What that does is to select "-lcrypto" (and "-lz", but that's not
relevant now) out of the LIBS variable. The configure script
should have included that in LIBS if you specified
--with-openssl, but maybe it failed to. Can you check the
configured value of LIBS (in src/Makefile.global) to see?

If it's not there, it would be useful to see the contents of
the config.log log file.

regards, tom lane

No, it wasn't there, because I hadn't included --with-openssl in the
configure. Looking at my history, I had done that once earlier but
dropped it for the reason noted below.

Including --with-openssl does include the crypto library, but if I
don't do a 'make clean' before doing a make, I get errors when
compiling postgresql. Apparently is it always a good idea to do make
clean when reconfiguring postgresql. (Could configure do a 'make
clean'?)

Anyway, everything's apparently working now.
--
Mike Nolan

Michael Nolan <htfoot@gmail.com> writes:

Including --with-openssl does include the crypto library, but if I
don't do a 'make clean' before doing a make, I get errors when
compiling postgresql. Apparently is it always a good idea to do make
clean when reconfiguring postgresql.

Indeed. In theory, if you use --enable-depend, you needn't do
"make clean". But I don't trust that a whole lot personally.
I find that using ccache gets most of the win with fewer assumptions.

regards, tom lane