Checking for Presence of Required Libraries Fails during PostgreSQL Upgrade

On 5/1/20 3:03 PM, TalGloz wrote:

I'm trying to upgrade my PostgreSQL form 10.2 to 11.7 (and eventually to
12.x), when I run:

/usr/pgsql-11/bin/pg_upgrade --old-bindir=/usr/pgsql-10/bin/
--new-bindir=/usr/pgsql-11/bin/ --old-datadir=/var/lib/pgsql/10/data
--new-datadir=/var/lib/pgsql/11/data --check

I get:

Checking for presence of required libraries fatal
Your installation references loadable libraries that are missing from the
new installation. You can add these libraries to the new installation,
or remove the functions using them from the old installation. A list of
problem libraries is in the file:
loadable_libraries.txt"

my loadable_libraries.txt file shows:

could not load library "myfunc": ERROR: could not access file "myfunc": No
such file or directory
could not load library "randomness": ERROR: could not access file
"randomness": No such file or directory
could not load library "seal_diff_cpp": ERROR: could not access file
"seal_diff_cpp": No such file or directory
could not load library "seal_mean_cxx": ERROR: could not access file
"seal_mean_cxx": No such file or directory
could not load library "seal_mean_cxx_v2": ERROR: could not access file
"seal_mean_cxx_v2": No such file or directory
could not load library "seal_variance_cxx": ERROR: could not access file
"seal_variance_cxx": No such file or directory
could not load library "seal_diff_benchmark": ERROR: could not access file
"seal_diff_benchmark": No such file or Directory

Running:
SELECT proname, probin, pronamespace
FROM pg_proc
WHERE probin IS NOT NULL AND pronamespace = 2200

Results in:

| proname | probin | pronamespace
| bytea_size | randomness | 2200
| sum_of_numbers | myfunc | 2200
| seal_diff_cpp | seal_diff_cpp | 2200
| seal_mean_cxx_v2 | seal_mean_cxx_v2 | 2200
| seal_variance_cxx | seal_variance_cxx | 2200
| seal_diff_benchmark | seal_diff_benchmark | 2200
| seal_mean_cxx | seal_mean_cxx | 2200

All the 7 problematic libraries (.so) are manually created c extension
functions. I can't just move them to /usr/pgsql-11/lib/ because they we
compiled with PostgresSQL10 and not 11.

Why not compile them against 11?

Then follow step 5) here:

https://www.postgresql.org/docs/12/pgupgrade.html

How can I resolve this issue? How can I delete them properly if porting them
to 11 is a painful process? Deleting them directly form /usr/pgsql-10/lib/
doesn't help.

If there are objects in the 10 database that depend on the libraries
then I imagine deleting them won't help.

--
Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html

--
Adrian Klaver
adrian.klaver@aklaver.com

On Fri, May 1, 2020 at 03:03:36PM -0700, TalGloz wrote:

I'm trying to upgrade my PostgreSQL form 10.2 to 11.7 (and eventually to
12.x), when I run:

FYI, you can go from PG 10 to 12 directly, without doing PG 11.

---------------------------------------------------------------------------

/usr/pgsql-11/bin/pg_upgrade --old-bindir=/usr/pgsql-10/bin/
--new-bindir=/usr/pgsql-11/bin/ --old-datadir=/var/lib/pgsql/10/data
--new-datadir=/var/lib/pgsql/11/data --check

I get:

Checking for presence of required libraries fatal
Your installation references loadable libraries that are missing from the
new installation. You can add these libraries to the new installation,
or remove the functions using them from the old installation. A list of
problem libraries is in the file:
loadable_libraries.txt"

my loadable_libraries.txt file shows:

could not load library "myfunc": ERROR: could not access file "myfunc": No
such file or directory
could not load library "randomness": ERROR: could not access file
"randomness": No such file or directory
could not load library "seal_diff_cpp": ERROR: could not access file
"seal_diff_cpp": No such file or directory
could not load library "seal_mean_cxx": ERROR: could not access file
"seal_mean_cxx": No such file or directory
could not load library "seal_mean_cxx_v2": ERROR: could not access file
"seal_mean_cxx_v2": No such file or directory
could not load library "seal_variance_cxx": ERROR: could not access file
"seal_variance_cxx": No such file or directory
could not load library "seal_diff_benchmark": ERROR: could not access file
"seal_diff_benchmark": No such file or Directory

Running:
SELECT proname, probin, pronamespace
FROM pg_proc
WHERE probin IS NOT NULL AND pronamespace = 2200

Results in:

| proname | probin | pronamespace
| bytea_size | randomness | 2200
| sum_of_numbers | myfunc | 2200
| seal_diff_cpp | seal_diff_cpp | 2200
| seal_mean_cxx_v2 | seal_mean_cxx_v2 | 2200
| seal_variance_cxx | seal_variance_cxx | 2200
| seal_diff_benchmark | seal_diff_benchmark | 2200
| seal_mean_cxx | seal_mean_cxx | 2200

All the 7 problematic libraries (.so) are manually created c extension
functions. I can't just move them to /usr/pgsql-11/lib/ because they we
compiled with PostgresSQL10 and not 11.

How can I resolve this issue? How can I delete them properly if porting them
to 11 is a painful process? Deleting them directly form /usr/pgsql-10/lib/
doesn't help.

--
Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EnterpriseDB https://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

Adrian Klaver-4 wrote

Why not compile them against 11?

Then follow step 5) here:

https://www.postgresql.org/docs/12/pgupgrade.html

Step 5 doesn’t actually say how to compile them during the upgrade process
it just gives a warning about not to use "CREATE EXTENSION my_lib" because
this won't effect the target version. Do you mean that I'll have to compile
them manually to my target PostgreSQL Version?

--
Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html

On 5/1/20 3:39 PM, TalGloz wrote:

Adrian Klaver-4 wrote

Why not compile them against 11?

Then follow step 5) here:

https://www.postgresql.org/docs/12/pgupgrade.html

Step 5 doesn’t actually say how to compile them during the upgrade process
it just gives a warning about not to use "CREATE EXTENSION my_lib" because
this won't effect the target version. Do you mean that I'll have to compile
them manually to my target PostgreSQL Version?

You haven't said how you installed the various versions of Postgres in
the first place. Bottom line is that the libraries that existed in the
old version need to exist on the new version. If they where not included
as part of the install process them they need to be manually created/added.

--
Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html

--
Adrian Klaver
adrian.klaver@aklaver.com

Adrian Klaver-4 wrote

On 5/1/20 3:39 PM, TalGloz wrote:

Adrian Klaver-4 wrote

Why not compile them against 11?

Then follow step 5) here:

https://www.postgresql.org/docs/12/pgupgrade.html

Step 5 doesn’t actually say how to compile them during the upgrade
process
it just gives a warning about not to use "CREATE EXTENSION my_lib"
because
this won't effect the target version. Do you mean that I'll have to
compile
them manually to my target PostgreSQL Version?

You haven't said how you installed the various versions of Postgres in
the first place. Bottom line is that the libraries that existed in the
old version need to exist on the new version. If they where not included
as part of the install process them they need to be manually
created/added.

--
Sent from:
https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html

--
Adrian Klaver

adrian.klaver@

I use Fedora OS and I've a set up and running PostgreSQL10 instance that
I've installed using
*dnf install postgresql10**. Sicne I've decided to upgrade directly to 12
instead of to 11 I've installed the 12 version using *dnf install
postgresql12**. I'm currently following the steps described in:

1. UPGRADE POSTGRESQL 10 To 11
<https://dbtut.com/index.php/2018/10/30/upgrade-postgresql-10-to-11/&gt; . It
has some syntax errors in the suggested command lines that I've corrected on
my side.

2. How to upgrade PostgreSQL from 11 to 12
<https://www.kostolansky.sk/posts/upgrading-to-postgresql-12/&gt; . The steps
were written for Ubuntu / Debian, It should work the same by updating the
paths to correct Fedora related paths of PostreSQL installation.

Unfortunately, none of the tutorials describes the steps on how to port /
handle some manually created C extension functions libraries to the new
PostgreSQL instance.

TalGloz

--
Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html

On 5/2/20 12:30 AM, TalGloz wrote:

Adrian Klaver-4 wrote

On 5/1/20 3:39 PM, TalGloz wrote:

Adrian Klaver-4 wrote

Why not compile them against 11?

Then follow step 5) here:

https://www.postgresql.org/docs/12/pgupgrade.html

Step 5 doesn’t actually say how to compile them during the upgrade
process
it just gives a warning about not to use "CREATE EXTENSION my_lib"
because
this won't effect the target version. Do you mean that I'll have to
compile
them manually to my target PostgreSQL Version?

You haven't said how you installed the various versions of Postgres in
the first place. Bottom line is that the libraries that existed in the
old version need to exist on the new version. If they where not included
as part of the install process them they need to be manually
created/added.

--
Sent from:
https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html

--
Adrian Klaver

adrian.klaver@

I use Fedora OS and I've a set up and running PostgreSQL10 instance that
I've installed using
*dnf install postgresql10**. Sicne I've decided to upgrade directly to 12
instead of to 11 I've installed the 12 version using *dnf install
postgresql12**. I'm currently following the steps described in:

1. UPGRADE POSTGRESQL 10 To 11
<https://dbtut.com/index.php/2018/10/30/upgrade-postgresql-10-to-11/&gt; . It
has some syntax errors in the suggested command lines that I've corrected on
my side.

2. How to upgrade PostgreSQL from 11 to 12
<https://www.kostolansky.sk/posts/upgrading-to-postgresql-12/&gt; . The steps
were written for Ubuntu / Debian, It should work the same by updating the
paths to correct Fedora related paths of PostreSQL installation.

Unfortunately, none of the tutorials describes the steps on how to port /
handle some manually created C extension functions libraries to the new
PostgreSQL instance.

Follow the same procedure you used to build/install them for the
Postgres 10, except run against the 12 instance. Do this before you do
the pg_upgrade procedure.

TalGloz

--
Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html

--
Adrian Klaver
adrian.klaver@aklaver.com

On Sat, May 2, 2020 at 12:31 AM TalGloz <glozmantal@gmail.com> wrote:

Adrian Klaver-4 wrote

On 5/1/20 3:39 PM, TalGloz wrote:

Adrian Klaver-4 wrote

Why not compile them against 11?

Then follow step 5) here:

https://www.postgresql.org/docs/12/pgupgrade.html

2. How to upgrade PostgreSQL from 11 to 12
<https://www.kostolansky.sk/posts/upgrading-to-postgresql-12/&gt; . The
steps
were written for Ubuntu / Debian, It should work the same by updating the
paths to correct Fedora related paths of PostreSQL installation.

Unfortunately, none of the tutorials describes the steps on how to port /
handle some manually created C extension functions libraries to the new
PostgreSQL instance.

Yes, the but guide for upgrading in the documentation does describe the
step of porting extension libraries. It says you need to do that before
you run pg_upgrade and ensure that whatever is being installed onto the new
server is binary compatible. It even says explicitly that pg_upgrade
cannot do this for you.

https://www.postgresql.org/docs/12/pgupgrade.html

David J.