pg_upgrade fails with a config-only datadir and no --new-bindir

Started by Heikki Linnakangasabout 1 month ago6 messageshackers
Jump to latest
#1Heikki Linnakangas
hlinnaka@iki.fi

While reviewing Bohyun's pg_upgrade --initdb patch, I noticed this small
existing bug:

----
$ mkdir data-new-cfg
$ touch data-new-cfg/postgresql.conf
$ ./bin/pg_upgrade --old-bindir=./bin -D ./data-new-cfg -d ./data
Finding the real data directory for the target cluster sh: 1:
(null)/postgres: not found

could not get data directory using "(null)/postgres" -D "data-new-cfg"
-C data_directory: No such file or directory
Failure, exiting
----

In this scenario, we have a combination of no --new-bindir option, so
that it is to be derived from the path of pg_upgrade itself, and a
--new-datadir (-D) option that points to a configuration-only directory.

That command was doomed to fail as the new datadir was empty except for
the config file, but a "(null)" is a sign of passing a NULL to
sprintf(), and we shouldn't be doing that.

I think the ordering of checks in main() and setup() is the culprit
here. main() calls adjust_data_dir() first, which does this:

snprintf(cmd, sizeof(cmd), "\"%s/postgres\" -D \"%s\" -C data_directory",
cluster->bindir, cluster->pgconfig);

to get the path to the data directory from the config. But
cluster->bindir isn't initialized yet.

Attached patch reorders the checks so that the binary directory is
resolved and checked before adjust_data_dir(). I think this bug is
present on all supported versions, but I'm not sure this is worth
backpatching. Although it doesn't seem very risky to backpatch either.

- Heikki

Attachments:

t254185_1
0001-Fix-order-of-bin-and-data-dir-checks-in-pg_upgrade.patchtext/x-patch; charset=UTF-8; name=0001-Fix-order-of-bin-and-data-dir-checks-in-pg_upgrade.patchDownload+44-56
#2Chao Li
li.evan.chao@gmail.com
In reply to: Heikki Linnakangas (#1)
Re: pg_upgrade fails with a config-only datadir and no --new-bindir

On Oct 14, 2026, at 06:02, Heikki Linnakangas <hlinnaka@iki.fi> wrote:

While reviewing Bohyun's pg_upgrade --initdb patch, I noticed this small existing bug:

----
$ mkdir data-new-cfg
$ touch data-new-cfg/postgresql.conf
$ ./bin/pg_upgrade --old-bindir=./bin -D ./data-new-cfg -d ./data
Finding the real data directory for the target cluster sh: 1: (null)/postgres: not found

could not get data directory using "(null)/postgres" -D "data-new-cfg" -C data_directory: No such file or directory
Failure, exiting
----

In this scenario, we have a combination of no --new-bindir option, so that it is to be derived from the path of pg_upgrade itself, and a --new-datadir (-D) option that points to a configuration-only directory.

That command was doomed to fail as the new datadir was empty except for the config file, but a "(null)" is a sign of passing a NULL to sprintf(), and we shouldn't be doing that.

I think the ordering of checks in main() and setup() is the culprit here. main() calls adjust_data_dir() first, which does this:

snprintf(cmd, sizeof(cmd), "\"%s/postgres\" -D \"%s\" -C data_directory",
cluster->bindir, cluster->pgconfig);

to get the path to the data directory from the config. But cluster->bindir isn't initialized yet.

Attached patch reorders the checks so that the binary directory is resolved and checked before adjust_data_dir(). I think this bug is present on all supported versions, but I'm not sure this is worth backpatching. Although it doesn't seem very risky to backpatch either.

- Heikki
<0001-Fix-order-of-bin-and-data-dir-checks-in-pg_upgrade.patch>

The fix looks correct to me. It makes sense to move check_bin_dirs() before adjust_data_dir(&new_cluster), since the latter may need to execute the postgres binary.

However, the updated test in 002_pg_upgrade.pl doesn’t seem to cover the bug being fixed. I tried to ran the updated test against master, and it still passed. Is that intentional?

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

#3Heikki Linnakangas
hlinnaka@iki.fi
In reply to: Chao Li (#2)
Re: pg_upgrade fails with a config-only datadir and no --new-bindir

On 14/10/2026 04:39, Chao Li wrote:

However, the updated test in 002_pg_upgrade.pl doesn’t seem to cover the bug being fixed. I tried to ran the updated test against master, and it still passed. Is that intentional?

Yes, I didn't include a test for the bug being fixed. The test change
was just to reflect that the behavior with a bogus --old-bindir changed
slightly -- the error on that now occurs earlier, before the
pg_upgrade_output.d directory is created.

- Heikki

#4Bohyun Lee
bohyun.lee@databricks.com
In reply to: Heikki Linnakangas (#3)
Re: pg_upgrade fails with a config-only datadir and no --new-bindir

I think the ordering of checks in main() and setup() is the culprit
here. main() calls adjust_data_dir() first, which does this:

snprintf(cmd, sizeof(cmd), "\"%s/postgres\" -D \"%s\" -C

data_directory",

cluster->bindir, cluster->pgconfig);

to get the path to the data directory from the config. But
cluster->bindir isn't initialized yet.

Agreed on the ordering. The main() should first call check_bin_dirs()
to resolve the default new binary directory first and then call
adjust_data_dir()
to find the actual data directory when a configuration-only directory
is specified, and then check_data_dir().

Best regards,
Bohyun

On Wed, Oct 14, 2026 at 9:17 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:

Show quoted text

On 14/10/2026 04:39, Chao Li wrote:

However, the updated test in 002_pg_upgrade.pl doesn’t seem to cover

the bug being fixed. I tried to ran the updated test against master, and it
still passed. Is that intentional?

Yes, I didn't include a test for the bug being fixed. The test change
was just to reflect that the behavior with a bogus --old-bindir changed
slightly -- the error on that now occurs earlier, before the
pg_upgrade_output.d directory is created.

- Heikki

#5Heikki Linnakangas
hlinnaka@iki.fi
In reply to: Bohyun Lee (#4)
Re: pg_upgrade fails with a config-only datadir and no --new-bindir

On 14/10/2026 11:04, Bohyun Lee wrote:

I think the ordering of checks in main() and setup() is the culprit
here. main() calls adjust_data_dir() first, which does this:

        snprintf(cmd, sizeof(cmd), "\"%s/postgres\" -D \"%s\" -C data_directory",
                         cluster->bindir, cluster->pgconfig);

to get the path to the data directory from the config. But
cluster->bindir isn't initialized yet.

Agreed on the ordering. The main() should first call check_bin_dirs()
to resolve the default new binary directory first and then call
adjust_data_dir()
to find the actual data directory when a configuration-only directory
is specified, and then check_data_dir().

Committed, thanks for the review!

- Heikki

#6Tom Lane
tgl@sss.pgh.pa.us
In reply to: Heikki Linnakangas (#5)
Re: pg_upgrade fails with a config-only datadir and no --new-bindir

Heikki Linnakangas <hlinnaka@iki.fi> writes:

Committed, thanks for the review!

FWIW, this *has* been complained of before:

/messages/by-id/f5bb021f-8f45-4825-9b4a-39241ea27a2f@jakobs.com

Don't know if that changes your calculation on whether to back-patch.

regards, tom lane