pg_upgrade --logfile option documentation
The documentation of the pg_upgrade -l/--logfile option never made much
sense to me:
-l, --logfile=FILENAME log session activity to file
I don't know what "session" means for pg_upgrade, so I never used it.
What it actually does is log the output of all the programs that
pg_upgrade calls internally, such as pg_ctl, psql, vacuumdb,
pg_resetxlog, to the specified file, which is quite useful for analyzing
errors such as
unable to connect to new postmaster started with the command: "/usr/lib/postgresql/9.1/bin/pg_ctl" -w -l "/dev/null" -D "/var/lib/postgresql/9.1/main" -o "-p 5433 -b" start >> "/dev/null" 2>&1
where -l would have put something in the place of /dev/null.
So what might be a better wording for this option? Something like "log
output of internally called programs to file"?
On Sun, Feb 19, 2012 at 6:13 AM, Peter Eisentraut <peter_e@gmx.net> wrote:
The documentation of the pg_upgrade -l/--logfile option never made much
sense to me:-l, --logfile=FILENAME log session activity to file
I don't know what "session" means for pg_upgrade, so I never used it.
What it actually does is log the output of all the programs that
pg_upgrade calls internally, such as pg_ctl, psql, vacuumdb,
pg_resetxlog, to the specified file, which is quite useful for analyzing
errors such asunable to connect to new postmaster started with the command: "/usr/lib/postgresql/9.1/bin/pg_ctl" -w -l "/dev/null" -D "/var/lib/postgresql/9.1/main" -o "-p 5433 -b" start >> "/dev/null" 2>&1
where -l would have put something in the place of /dev/null.
So what might be a better wording for this option? Something like "log
output of internally called programs to file"?
I don't think we should be that specific, because we might someday
want pg_upgrade itself to write messages to that file as well, even if
it doesn't today. I agree that the phrase "session activity" is a bit
misleading.
As a more general comment, I think that the way pg_upgrade does
logging right now is absolutely terrible. IME, it is utterly
impossible to understand what has gone wrong with pg_upgrade without
looking at the log file. And by default, no log file is created. So
typically what happens is:
- I run pg_upgrade. It fails.
- I rename the control file from the old cluster back to its original name.
- I rerun pg_upgrade, this time with -l. It fails again.
- I read the log file, figure out what the problem is, and correct it.
- I rename the control file from the old cluster back to its original
name, again.
- I run pg_upgrade a third time.
- On a good day, it works, else go to step 5.
One pretty obvious improvement would be: if pg_upgrade fails after
renaming the control file for the old cluster out of the way - say,
while loading the schema dump into the new cluster - have it RENAME
THE OLD CONTROL FILE BACK before exiting. But I also think the
logging needs improvement. Right now, we studiously redirect both
stdout and stderr to /dev/null; maybe it would be better to redirect
stdout to /dev/null and NOT redirect stderr. If that generates too
much chatter in non-failure cases, then let's adjust the output of the
commands pg_upgrade is invoking until it doesn't. The actual cause of
the failure, rather than pg_upgrade's fairly-useless gloss on it,
ought to be visible right away, at least IMHO.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Sun, Feb 19, 2012 at 01:13:10PM +0200, Peter Eisentraut wrote:
The documentation of the pg_upgrade -l/--logfile option never made much
sense to me:-l, --logfile=FILENAME log session activity to file
I don't know what "session" means for pg_upgrade, so I never used it.
What it actually does is log the output of all the programs that
pg_upgrade calls internally, such as pg_ctl, psql, vacuumdb,
pg_resetxlog, to the specified file, which is quite useful for analyzing
errors such asunable to connect to new postmaster started with the command: "/usr/lib/postgresql/9.1/bin/pg_ctl" -w -l "/dev/null" -D "/var/lib/postgresql/9.1/main" -o "-p 5433 -b" start >> "/dev/null" 2>&1
where -l would have put something in the place of /dev/null.
So what might be a better wording for this option? Something like "log
output of internally called programs to file"?
How about?
-l, --logfile=FILENAME log internal activity to file
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On Sun, Feb 19, 2012 at 01:24:34PM -0500, Robert Haas wrote:
As a more general comment, I think that the way pg_upgrade does
logging right now is absolutely terrible. IME, it is utterly
impossible to understand what has gone wrong with pg_upgrade without
looking at the log file. And by default, no log file is created. So
typically what happens is:- I run pg_upgrade. It fails.
- I rename the control file from the old cluster back to its original name.
- I rerun pg_upgrade, this time with -l. It fails again.
- I read the log file, figure out what the problem is, and correct it.
- I rename the control file from the old cluster back to its original
name, again.
- I run pg_upgrade a third time.
- On a good day, it works, else go to step 5.One pretty obvious improvement would be: if pg_upgrade fails after
renaming the control file for the old cluster out of the way - say,
while loading the schema dump into the new cluster - have it RENAME
THE OLD CONTROL FILE BACK before exiting. But I also think the
The behavior you are seeing now is the paranoia inherent in pg_upgrade's
design. Now that pg_upgrade is being used more, perhaps that needs to
be relaxed.
However, remember we rename that control file to prevent the old cluster
from being run accidentally, which is particular important in link mode.
There might be some error cases that still would not restore the
location of that file if we have a revert behavior on error. A more
normal behavior would be for pg_upgrade to rename the control file only
when the upgrade completes successfully.
logging needs improvement. Right now, we studiously redirect both
stdout and stderr to /dev/null; maybe it would be better to redirect
stdout to /dev/null and NOT redirect stderr. If that generates too
much chatter in non-failure cases, then let's adjust the output of the
commands pg_upgrade is invoking until it doesn't. The actual cause of
the failure, rather than pg_upgrade's fairly-useless gloss on it,
ought to be visible right away, at least IMHO.
Well, we have a -d option for debug; we could modify that to have debug
levels. Also, from the command line, it is difficult to have multiple
process write into a single file, so that isn't going work to have
pg_upgrade and the server logging to the same file on Windows.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
Excerpts from Bruce Momjian's message of mié feb 22 17:01:10 -0300 2012:
On Sun, Feb 19, 2012 at 01:24:34PM -0500, Robert Haas wrote:
One pretty obvious improvement would be: if pg_upgrade fails after
renaming the control file for the old cluster out of the way - say,
while loading the schema dump into the new cluster - have it RENAME
THE OLD CONTROL FILE BACK before exiting. But I also think theThe behavior you are seeing now is the paranoia inherent in pg_upgrade's
design. Now that pg_upgrade is being used more, perhaps that needs to
be relaxed.However, remember we rename that control file to prevent the old cluster
from being run accidentally, which is particular important in link mode.
... but if the upgrade failed, clearly this shouldn't be a problem. I
agree with Robert, and was bit by this last week -- in case of any error
during the procedure, the control file should be renamed back to its
original name.
There might be some error cases that still would not restore the
location of that file if we have a revert behavior on error. A more
normal behavior would be for pg_upgrade to rename the control file only
when the upgrade completes successfully.
Not sure about this. If the upgrades completes successfully and the
file is not renamed at the last minute due to some error, that would be
a problem as well, because now the old cluster would happily run and
perhaps corrupt the data files from under the new cluster.
--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
On Wed, Feb 22, 2012 at 05:22:29PM -0300, Alvaro Herrera wrote:
Excerpts from Bruce Momjian's message of mi� feb 22 17:01:10 -0300 2012:
On Sun, Feb 19, 2012 at 01:24:34PM -0500, Robert Haas wrote:
One pretty obvious improvement would be: if pg_upgrade fails after
renaming the control file for the old cluster out of the way - say,
while loading the schema dump into the new cluster - have it RENAME
THE OLD CONTROL FILE BACK before exiting. But I also think theThe behavior you are seeing now is the paranoia inherent in pg_upgrade's
design. Now that pg_upgrade is being used more, perhaps that needs to
be relaxed.However, remember we rename that control file to prevent the old cluster
from being run accidentally, which is particular important in link mode.... but if the upgrade failed, clearly this shouldn't be a problem. I
agree with Robert, and was bit by this last week -- in case of any error
during the procedure, the control file should be renamed back to its
original name.There might be some error cases that still would not restore the
location of that file if we have a revert behavior on error. A more
normal behavior would be for pg_upgrade to rename the control file only
when the upgrade completes successfully.Not sure about this. If the upgrades completes successfully and the
file is not renamed at the last minute due to some error, that would be
a problem as well, because now the old cluster would happily run and
perhaps corrupt the data files from under the new cluster.
Well, the basic problem is that the user, before pg_upgrade started,
installed a new cluster that works. If we rename the old control, but
rename it back on failure, there are cases we will miss, kill like -9 or
a server crash, and it will not be obvious to them that the control file
was renamed.
Of course, if we only rename on success, and there is kill -9 or server
crash, the old cluster is still start-able, like the new one.
One good argument for the rename early is that on a server crash, the
system is probably going to restart the database automatically, and that
means the old server.
Right now we have a clear message that they need to rename the control
file to start the old server. Not sure what the new wording would look
like --- let me try.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On ons, 2012-02-22 at 14:38 -0500, Bruce Momjian wrote:
How about?
-l, --logfile=FILENAME log internal activity to file
That sounds better.
On sön, 2012-02-19 at 13:24 -0500, Robert Haas wrote:
But I also think the
logging needs improvement. Right now, we studiously redirect both
stdout and stderr to /dev/null; maybe it would be better to redirect
stdout to /dev/null and NOT redirect stderr. If that generates too
much chatter in non-failure cases, then let's adjust the output of the
commands pg_upgrade is invoking until it doesn't.
That should be achievable for calls to psql and vacuumdb, say, but what
would you do with the server logs?
On Wed, Feb 22, 2012 at 3:51 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
On sön, 2012-02-19 at 13:24 -0500, Robert Haas wrote:
But I also think the
logging needs improvement. Right now, we studiously redirect both
stdout and stderr to /dev/null; maybe it would be better to redirect
stdout to /dev/null and NOT redirect stderr. If that generates too
much chatter in non-failure cases, then let's adjust the output of the
commands pg_upgrade is invoking until it doesn't.That should be achievable for calls to psql and vacuumdb, say, but what
would you do with the server logs?
I don't know. It might be less of an issue, though. I mean, IME,
what typically happens is that psql fails to restore the dump, either
because it can't connect to the new database or because it's confused
by some stupid case that isn't handled well. So even if we could just
improve the error handling to report those types of failures more
transparently, I think it would be a big improvement.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Wed, Feb 22, 2012 at 05:49:26PM -0500, Robert Haas wrote:
On Wed, Feb 22, 2012 at 3:51 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
On s�n, 2012-02-19 at 13:24 -0500, Robert Haas wrote:
But I also think the
logging needs improvement. �Right now, we studiously redirect both
stdout and stderr to /dev/null; maybe it would be better to redirect
stdout to /dev/null and NOT redirect stderr. �If that generates too
much chatter in non-failure cases, then let's adjust the output of the
commands pg_upgrade is invoking until it doesn't.That should be achievable for calls to psql and vacuumdb, say, but what
would you do with the server logs?I don't know. It might be less of an issue, though. I mean, IME,
what typically happens is that psql fails to restore the dump, either
because it can't connect to the new database or because it's confused
by some stupid case that isn't handled well. So even if we could just
improve the error handling to report those types of failures more
transparently, I think it would be a big improvement.
Well, on Unix, it is easy to redirect the server logs to the same place
as the pg_upgrade logs. That doesn't help? How would we improve the
reporting of SQL restore failures?
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On Wed, Feb 22, 2012 at 10:50:07PM +0200, Peter Eisentraut wrote:
On ons, 2012-02-22 at 14:38 -0500, Bruce Momjian wrote:
How about?
-l, --logfile=FILENAME log internal activity to file
That sounds better.
Done.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On Wed, Feb 22, 2012 at 03:37:29PM -0500, Bruce Momjian wrote:
On Wed, Feb 22, 2012 at 05:22:29PM -0300, Alvaro Herrera wrote:
Not sure about this. If the upgrades completes successfully and the
file is not renamed at the last minute due to some error, that would be
a problem as well, because now the old cluster would happily run and
perhaps corrupt the data files from under the new cluster.Well, the basic problem is that the user, before pg_upgrade started,
installed a new cluster that works. If we rename the old control, but
rename it back on failure, there are cases we will miss, kill like -9 or
a server crash, and it will not be obvious to them that the control file
was renamed.Of course, if we only rename on success, and there is kill -9 or server
crash, the old cluster is still start-able, like the new one.One good argument for the rename early is that on a server crash, the
system is probably going to restart the database automatically, and that
means the old server.Right now we have a clear message that they need to rename the control
file to start the old server. Not sure what the new wording would look
like --- let me try.
I have thought about this, and feel that it would be odd to lock the old
cluster at the start of the upgrade, and then unlock it on a failure,
particularly because we can't always unlock it, e.g. operating system
crash.
A cleaner solution would be to lock it when we complete the upgrade,
which I have done in the attached patch. I have also added a warning
about restarting the old server when link mode is used, and updated the
documentation to match the new behavior.
Patch attached. I would like to apply this to 9.2/HEAD.
---------------------------------------------------------------------------
Performing Consistency Checks
-----------------------------
Checking current, bin, and data directories ok
Checking cluster versions ok
Checking database user is a superuser ok
Checking for prepared transactions ok
Checking for reg* system OID user data types ok
Checking for contrib/isn with bigint-passing mismatch ok
Creating catalog dump ok
Checking for prepared transactions ok
Checking for presence of required libraries ok
If pg_upgrade fails after this point, you must re-initdb the
new cluster before continuing.
Performing Upgrade
------------------
Analyzing all rows in the new cluster ok
Freezing all rows on the new cluster ok
Deleting new commit clogs ok
Copying old commit clogs to new server ok
Setting next transaction ID for new cluster ok
Resetting WAL archives ok
Setting frozenxid counters in new cluster ok
Creating databases in the new cluster ok
Adding support functions to new cluster ok
Restoring database schema to new cluster ok
Removing support functions from new cluster ok
Linking user relation files
ok
Setting next OID for new cluster ok
Creating script to delete old cluster ok
Adding ".old" suffix to old global/pg_control ok
If you want to start the old cluster, you will need to remove
the ".old" suffix from /u/pgsql.old/data/global/pg_control.old.
Because "link" mode was used, the old cluster cannot be safely
started once the new cluster has been started.
Upgrade complete
----------------
Optimizer statistics are not transferred by pg_upgrade so
consider running:
vacuumdb --all --analyze-only
on the newly-upgraded cluster.
Running this script will delete the old cluster's data files:
/usr/local/pgdev/pg_upgrade/delete_old_cluster.sh
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
Attachments:
pg_upgrade.difftext/x-diff; charset=us-asciiDownload
diff --git a/contrib/pg_upgrade/check.c b/contrib/pg_upgrade/check.c
new file mode 100644
index 891eb9a..a5f63eb
*** a/contrib/pg_upgrade/check.c
--- b/contrib/pg_upgrade/check.c
*************** report_clusters_compatible(void)
*** 148,156 ****
}
pg_log(PG_REPORT, "\n"
! "If pg_upgrade fails after this point, you must re-initdb the new cluster\n"
! "before continuing. You will also need to remove the \".old\" suffix from\n"
! "%s/global/pg_control.old.\n", old_cluster.pgdata);
}
--- 148,155 ----
}
pg_log(PG_REPORT, "\n"
! "If pg_upgrade fails after this point, you must re-initdb the\n"
! "new cluster before continuing.\n");
}
*************** output_completion_banner(char *deletion_
*** 198,205 ****
/* Did we copy the free space files? */
if (GET_MAJOR_VERSION(old_cluster.major_version) >= 804)
pg_log(PG_REPORT,
! "Optimizer statistics are not transferred by pg_upgrade so consider\n"
! "running:\n"
" vacuumdb --all --analyze-only\n"
"on the newly-upgraded cluster.\n\n");
else
--- 197,204 ----
/* Did we copy the free space files? */
if (GET_MAJOR_VERSION(old_cluster.major_version) >= 804)
pg_log(PG_REPORT,
! "Optimizer statistics are not transferred by pg_upgrade so\n"
! "consider running:\n"
" vacuumdb --all --analyze-only\n"
"on the newly-upgraded cluster.\n\n");
else
diff --git a/contrib/pg_upgrade/controldata.c b/contrib/pg_upgrade/controldata.c
new file mode 100644
index 8560d88..4c77ac9
*** a/contrib/pg_upgrade/controldata.c
--- b/contrib/pg_upgrade/controldata.c
*************** check_control_data(ControlData *oldctrl,
*** 516,526 ****
void
! rename_old_pg_control(void)
{
char old_path[MAXPGPATH],
new_path[MAXPGPATH];
prep_status("Adding \".old\" suffix to old global/pg_control");
snprintf(old_path, sizeof(old_path), "%s/global/pg_control", old_cluster.pgdata);
--- 516,527 ----
void
! disable_old_cluster(void)
{
char old_path[MAXPGPATH],
new_path[MAXPGPATH];
+ /* rename pg_control so old server cannot be accidentally started */
prep_status("Adding \".old\" suffix to old global/pg_control");
snprintf(old_path, sizeof(old_path), "%s/global/pg_control", old_cluster.pgdata);
*************** rename_old_pg_control(void)
*** 528,531 ****
--- 529,540 ----
if (pg_mv_file(old_path, new_path) != 0)
pg_log(PG_FATAL, "Unable to rename %s to %s.\n", old_path, new_path);
check_ok();
+
+ pg_log(PG_REPORT, "\n"
+ "If you want to start the old cluster, you will need to remove\n"
+ "the \".old\" suffix from %s/global/pg_control.old.\n", old_cluster.pgdata);
+ if (user_opts.transfer_mode == TRANSFER_MODE_LINK)
+ pg_log(PG_REPORT,
+ "Because \"link\" mode was used, the old cluster cannot be safely\n"
+ "started once the new cluster has been started.\n");
}
diff --git a/contrib/pg_upgrade/pg_upgrade.c b/contrib/pg_upgrade/pg_upgrade.c
new file mode 100644
index 15b30fc..47cc683
*** a/contrib/pg_upgrade/pg_upgrade.c
--- b/contrib/pg_upgrade/pg_upgrade.c
***************
*** 43,49 ****
#include <langinfo.h>
#endif
- static void disable_old_cluster(void);
static void prepare_new_cluster(void);
static void prepare_new_databases(void);
static void create_new_objects(void);
--- 43,48 ----
*************** main(int argc, char **argv)
*** 87,93 ****
pg_log(PG_REPORT, "\nPerforming Upgrade\n");
pg_log(PG_REPORT, "------------------\n");
- disable_old_cluster();
prepare_new_cluster();
stop_postmaster(false);
--- 86,91 ----
*************** main(int argc, char **argv)
*** 128,133 ****
--- 126,133 ----
issue_warnings(sequence_script_file_name);
+ disable_old_cluster();
+
pg_log(PG_REPORT, "\nUpgrade complete\n");
pg_log(PG_REPORT, "----------------\n");
*************** setup(char *argv0, bool live_check)
*** 176,189 ****
}
- static void
- disable_old_cluster(void)
- {
- /* rename pg_control so old server cannot be accidentally started */
- rename_old_pg_control();
- }
-
-
static void
prepare_new_cluster(void)
{
--- 176,181 ----
diff --git a/contrib/pg_upgrade/pg_upgrade.h b/contrib/pg_upgrade/pg_upgrade.h
new file mode 100644
index 58d5201..a954815
*** a/contrib/pg_upgrade/pg_upgrade.h
--- b/contrib/pg_upgrade/pg_upgrade.h
*************** void create_script_for_old_cluster_dele
*** 282,289 ****
/* controldata.c */
void get_control_data(ClusterInfo *cluster, bool live_check);
! void check_control_data(ControlData *oldctrl,
! ControlData *newctrl);
/* dump.c */
--- 282,289 ----
/* controldata.c */
void get_control_data(ClusterInfo *cluster, bool live_check);
! void check_control_data(ControlData *oldctrl, ControlData *newctrl);
! void disable_old_cluster(void);
/* dump.c */
*************** int exec_prog(bool throw_error, const ch
*** 298,304 ****
__attribute__((format(PG_PRINTF_ATTRIBUTE, 2, 3)));
void verify_directories(void);
bool is_server_running(const char *datadir);
- void rename_old_pg_control(void);
/* file.c */
--- 298,303 ----
diff --git a/doc/src/sgml/pgupgrade.sgml b/doc/src/sgml/pgupgrade.sgml
new file mode 100644
index 1373069..a27f041
*** a/doc/src/sgml/pgupgrade.sgml
--- b/doc/src/sgml/pgupgrade.sgml
***************
*** 182,188 ****
<para>
If you are using a version-specific installation directory, e.g.
! <filename>/opt/PostgreSQL/8.4</>, you do not need to move the old cluster. The
one-click installers all use version-specific installation directories.
</para>
--- 182,188 ----
<para>
If you are using a version-specific installation directory, e.g.
! <filename>/opt/PostgreSQL/9.1</>, you do not need to move the old cluster. The
one-click installers all use version-specific installation directories.
</para>
*************** gmake prefix=/usr/local/pgsql.new instal
*** 254,260 ****
<para>
Install any custom shared object files (or DLLs) used by the old cluster
! into the new cluster, e.g. <filename>pgcrypto.so</filename>, whether they are from <filename>contrib</filename>
or some other source. Do not install the schema definitions, e.g.
<filename>pgcrypto.sql</>, because these will be upgraded from the old cluster.
</para>
--- 254,261 ----
<para>
Install any custom shared object files (or DLLs) used by the old cluster
! into the new cluster, e.g. <filename>pgcrypto.so</filename>,
! whether they are from <filename>contrib</filename>
or some other source. Do not install the schema definitions, e.g.
<filename>pgcrypto.sql</>, because these will be upgraded from the old cluster.
</para>
*************** psql --username postgres --file script.s
*** 457,471 ****
If you
ran <command>pg_upgrade</command> <emphasis>without</> <option>--link</>
or did not start the new server, the old cluster was not
! modified except that an <literal>.old</> suffix was appended
! to <filename>$PGDATA/global/pg_control</> and perhaps
! tablespace directories. To reuse the old cluster, remove
! the <filename>.old</> suffix
! from <filename>$PGDATA/global/pg_control</>. and, if upgrading
! to 8.4 or earlier, remove the tablespace directories created
! by the upgrade and remove the <filename>.old</> suffix from
! the tablespace directory names; then you can restart the old
! cluster.
</para>
</listitem>
</itemizedlist>
--- 458,469 ----
If you
ran <command>pg_upgrade</command> <emphasis>without</> <option>--link</>
or did not start the new server, the old cluster was not
! modified except that, if <command>pg_upgrade</command> succeeded,
! a <literal>.old</> suffix was appended to
! <filename>$PGDATA/global/pg_control</>. To reuse the old
! cluster, possibly remove the <filename>.old</> suffix from
! <filename>$PGDATA/global/pg_control</>; you can then restart the
! old cluster.
</para>
</listitem>
</itemizedlist>
*************** psql --username postgres --file script.s
*** 582,590 ****
</para>
<para>
! If you want to use link mode and you don't want your old cluster
to be modified when the new cluster is started, make a copy of the
! old cluster and upgrade that with link mode. To make a valid copy
of the old cluster, use <command>rsync</> to create a dirty
copy of the old cluster while the server is running, then shut down
the old server and run <command>rsync</> again to update the copy with any
--- 580,588 ----
</para>
<para>
! If you want to use link mode and you do not want your old cluster
to be modified when the new cluster is started, make a copy of the
! old cluster and upgrade that in link mode. To make a valid copy
of the old cluster, use <command>rsync</> to create a dirty
copy of the old cluster while the server is running, then shut down
the old server and run <command>rsync</> again to update the copy with any
On Tue, Feb 28, 2012 at 11:21 AM, Bruce Momjian <bruce@momjian.us> wrote:
On Wed, Feb 22, 2012 at 03:37:29PM -0500, Bruce Momjian wrote:
On Wed, Feb 22, 2012 at 05:22:29PM -0300, Alvaro Herrera wrote:
Not sure about this. If the upgrades completes successfully and the
file is not renamed at the last minute due to some error, that would be
a problem as well, because now the old cluster would happily run and
perhaps corrupt the data files from under the new cluster.Well, the basic problem is that the user, before pg_upgrade started,
installed a new cluster that works. If we rename the old control, but
rename it back on failure, there are cases we will miss, kill like -9 or
a server crash, and it will not be obvious to them that the control file
was renamed.Of course, if we only rename on success, and there is kill -9 or server
crash, the old cluster is still start-able, like the new one.One good argument for the rename early is that on a server crash, the
system is probably going to restart the database automatically, and that
means the old server.Right now we have a clear message that they need to rename the control
file to start the old server. Not sure what the new wording would look
like --- let me try.I have thought about this, and feel that it would be odd to lock the old
cluster at the start of the upgrade, and then unlock it on a failure,
particularly because we can't always unlock it, e.g. operating system
crash.A cleaner solution would be to lock it when we complete the upgrade,
which I have done in the attached patch. I have also added a warning
about restarting the old server when link mode is used, and updated the
documentation to match the new behavior.Patch attached. I would like to apply this to 9.2/HEAD.
---------------------------------------------------------------------------
Performing Consistency Checks
-----------------------------
Checking current, bin, and data directories ok
Checking cluster versions ok
Checking database user is a superuser ok
Checking for prepared transactions ok
Checking for reg* system OID user data types ok
Checking for contrib/isn with bigint-passing mismatch ok
Creating catalog dump ok
Checking for prepared transactions ok
Checking for presence of required libraries okIf pg_upgrade fails after this point, you must re-initdb the
new cluster before continuing.Performing Upgrade
------------------
Analyzing all rows in the new cluster ok
Freezing all rows on the new cluster ok
Deleting new commit clogs ok
Copying old commit clogs to new server ok
Setting next transaction ID for new cluster ok
Resetting WAL archives ok
Setting frozenxid counters in new cluster ok
Creating databases in the new cluster ok
Adding support functions to new cluster ok
Restoring database schema to new cluster ok
Removing support functions from new cluster ok
Linking user relation files
ok
Setting next OID for new cluster ok
Creating script to delete old cluster ok
Adding ".old" suffix to old global/pg_control okIf you want to start the old cluster, you will need to remove
the ".old" suffix from /u/pgsql.old/data/global/pg_control.old.
Because "link" mode was used, the old cluster cannot be safely
started once the new cluster has been started.Upgrade complete
----------------
Optimizer statistics are not transferred by pg_upgrade so
consider running:
vacuumdb --all --analyze-only
on the newly-upgraded cluster.Running this script will delete the old cluster's data files:
/usr/local/pgdev/pg_upgrade/delete_old_cluster.sh
I think you should rename the old control file just before the step
that says "linking user relation files". That's the point after which
it becomes unsafe to start the old cluster, right?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Excerpts from Robert Haas's message of mar feb 28 15:24:45 -0300 2012:
I think you should rename the old control file just before the step
that says "linking user relation files". That's the point after which
it becomes unsafe to start the old cluster, right?
Also, if it's not using link mode, what is the point in doing the rename
in the first place? I was using copy mode when I did my tests and yet
it got renamed (unless link mode is now the default, which I doubt?)
--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
On Tue, Feb 28, 2012 at 01:24:45PM -0500, Robert Haas wrote:
Running this script will delete the old cluster's data files:
� �/usr/local/pgdev/pg_upgrade/delete_old_cluster.shI think you should rename the old control file just before the step
that says "linking user relation files". That's the point after which
it becomes unsafe to start the old cluster, right?
Yes, it is true that that is the danger point, and also it is much less
likely to fail at that point --- it usually happens during the schema
creation. I would have to add some more conditional wording without
clearly stating if the old suffix is present.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On Tue, Feb 28, 2012 at 03:48:03PM -0300, Alvaro Herrera wrote:
Excerpts from Robert Haas's message of mar feb 28 15:24:45 -0300 2012:
I think you should rename the old control file just before the step
that says "linking user relation files". That's the point after which
it becomes unsafe to start the old cluster, right?Also, if it's not using link mode, what is the point in doing the rename
in the first place? I was using copy mode when I did my tests and yet
it got renamed (unless link mode is now the default, which I doubt?)
You are right that we lock unconditionally. We can certainly only lock
in link mode.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On Tue, Feb 28, 2012 at 02:15:30PM -0500, Bruce Momjian wrote:
On Tue, Feb 28, 2012 at 01:24:45PM -0500, Robert Haas wrote:
Running this script will delete the old cluster's data files:
� �/usr/local/pgdev/pg_upgrade/delete_old_cluster.shI think you should rename the old control file just before the step
that says "linking user relation files". That's the point after which
it becomes unsafe to start the old cluster, right?Yes, it is true that that is the danger point, and also it is much less
likely to fail at that point --- it usually happens during the schema
creation. I would have to add some more conditional wording without
clearly stating if the old suffix is present.
OK, I have implemented both Roberts and �lvaro's ideas in my patch.
I only add the .old suffix to pg_controldata when link mode is used, and
I now do it after the schema has been created (the most common failure
case for pg_upgrade), and just before we actually link files --- both
very good ideas.
Patch attached; new pg_upgrade output with link mode below.
---------------------------------------------------------------------------
Performing Consistency Checks
-----------------------------
Checking current, bin, and data directories ok
Checking cluster versions ok
Checking database user is a superuser ok
Checking for prepared transactions ok
Checking for reg* system OID user data types ok
Checking for contrib/isn with bigint-passing mismatch ok
Creating catalog dump ok
Checking for prepared transactions ok
Checking for presence of required libraries ok
If pg_upgrade fails after this point, you must re-initdb the
new cluster before continuing.
Performing Upgrade
------------------
Analyzing all rows in the new cluster ok
Freezing all rows on the new cluster ok
Deleting new commit clogs ok
Copying old commit clogs to new server ok
Setting next transaction ID for new cluster ok
Resetting WAL archives ok
Setting frozenxid counters in new cluster ok
Creating databases in the new cluster ok
Adding support functions to new cluster ok
Restoring database schema to new cluster ok
Removing support functions from new cluster ok
Adding ".old" suffix to old global/pg_control ok
If you want to start the old cluster, you will need to remove
the ".old" suffix from /u/pgsql.old/data/global/pg_control.old.
Because "link" mode was used, the old cluster cannot be safely
started once the new cluster has been started.
Linking user relation files
ok
Setting next OID for new cluster ok
Creating script to delete old cluster ok
Upgrade complete
----------------
Optimizer statistics are not transferred by pg_upgrade so
consider running:
vacuumdb --all --analyze-only
on the newly-upgraded cluster.
Running this script will delete the old cluster's data files:
/usr/local/pgdev/pg_upgrade/delete_old_cluster.sh
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
Attachments:
pg_upgrade.difftext/x-diff; charset=us-asciiDownload
diff --git a/contrib/pg_upgrade/check.c b/contrib/pg_upgrade/check.c
new file mode 100644
index 891eb9a..a5f63eb
*** a/contrib/pg_upgrade/check.c
--- b/contrib/pg_upgrade/check.c
*************** report_clusters_compatible(void)
*** 148,156 ****
}
pg_log(PG_REPORT, "\n"
! "If pg_upgrade fails after this point, you must re-initdb the new cluster\n"
! "before continuing. You will also need to remove the \".old\" suffix from\n"
! "%s/global/pg_control.old.\n", old_cluster.pgdata);
}
--- 148,155 ----
}
pg_log(PG_REPORT, "\n"
! "If pg_upgrade fails after this point, you must re-initdb the\n"
! "new cluster before continuing.\n");
}
*************** output_completion_banner(char *deletion_
*** 198,205 ****
/* Did we copy the free space files? */
if (GET_MAJOR_VERSION(old_cluster.major_version) >= 804)
pg_log(PG_REPORT,
! "Optimizer statistics are not transferred by pg_upgrade so consider\n"
! "running:\n"
" vacuumdb --all --analyze-only\n"
"on the newly-upgraded cluster.\n\n");
else
--- 197,204 ----
/* Did we copy the free space files? */
if (GET_MAJOR_VERSION(old_cluster.major_version) >= 804)
pg_log(PG_REPORT,
! "Optimizer statistics are not transferred by pg_upgrade so\n"
! "consider running:\n"
" vacuumdb --all --analyze-only\n"
"on the newly-upgraded cluster.\n\n");
else
diff --git a/contrib/pg_upgrade/controldata.c b/contrib/pg_upgrade/controldata.c
new file mode 100644
index 8560d88..5239601
*** a/contrib/pg_upgrade/controldata.c
--- b/contrib/pg_upgrade/controldata.c
*************** check_control_data(ControlData *oldctrl,
*** 516,526 ****
void
! rename_old_pg_control(void)
{
char old_path[MAXPGPATH],
new_path[MAXPGPATH];
prep_status("Adding \".old\" suffix to old global/pg_control");
snprintf(old_path, sizeof(old_path), "%s/global/pg_control", old_cluster.pgdata);
--- 516,527 ----
void
! disable_old_cluster(void)
{
char old_path[MAXPGPATH],
new_path[MAXPGPATH];
+ /* rename pg_control so old server cannot be accidentally started */
prep_status("Adding \".old\" suffix to old global/pg_control");
snprintf(old_path, sizeof(old_path), "%s/global/pg_control", old_cluster.pgdata);
*************** rename_old_pg_control(void)
*** 528,531 ****
--- 529,538 ----
if (pg_mv_file(old_path, new_path) != 0)
pg_log(PG_FATAL, "Unable to rename %s to %s.\n", old_path, new_path);
check_ok();
+
+ pg_log(PG_REPORT, "\n"
+ "If you want to start the old cluster, you will need to remove\n"
+ "the \".old\" suffix from %s/global/pg_control.old.\n"
+ "Because \"link\" mode was used, the old cluster cannot be safely\n"
+ "started once the new cluster has been started.\n\n", old_cluster.pgdata);
}
diff --git a/contrib/pg_upgrade/pg_upgrade.c b/contrib/pg_upgrade/pg_upgrade.c
new file mode 100644
index 15b30fc..3078bcd
*** a/contrib/pg_upgrade/pg_upgrade.c
--- b/contrib/pg_upgrade/pg_upgrade.c
***************
*** 43,49 ****
#include <langinfo.h>
#endif
- static void disable_old_cluster(void);
static void prepare_new_cluster(void);
static void prepare_new_databases(void);
static void create_new_objects(void);
--- 43,48 ----
*************** main(int argc, char **argv)
*** 87,93 ****
pg_log(PG_REPORT, "\nPerforming Upgrade\n");
pg_log(PG_REPORT, "------------------\n");
- disable_old_cluster();
prepare_new_cluster();
stop_postmaster(false);
--- 86,91 ----
*************** main(int argc, char **argv)
*** 109,114 ****
--- 107,122 ----
stop_postmaster(false);
+ /*
+ * Most failures happen in create_new_objects(), which has
+ * completed at this point. We do this here because it is just
+ * before linking, which will link the old and new cluster data
+ * files, preventing the old cluster from being safely started
+ * once the new cluster is started.
+ */
+ if (user_opts.transfer_mode == TRANSFER_MODE_LINK)
+ disable_old_cluster();
+
transfer_all_new_dbs(&old_cluster.dbarr, &new_cluster.dbarr,
old_cluster.pgdata, new_cluster.pgdata);
*************** setup(char *argv0, bool live_check)
*** 176,189 ****
}
- static void
- disable_old_cluster(void)
- {
- /* rename pg_control so old server cannot be accidentally started */
- rename_old_pg_control();
- }
-
-
static void
prepare_new_cluster(void)
{
--- 184,189 ----
diff --git a/contrib/pg_upgrade/pg_upgrade.h b/contrib/pg_upgrade/pg_upgrade.h
new file mode 100644
index 58d5201..a954815
*** a/contrib/pg_upgrade/pg_upgrade.h
--- b/contrib/pg_upgrade/pg_upgrade.h
*************** void create_script_for_old_cluster_dele
*** 282,289 ****
/* controldata.c */
void get_control_data(ClusterInfo *cluster, bool live_check);
! void check_control_data(ControlData *oldctrl,
! ControlData *newctrl);
/* dump.c */
--- 282,289 ----
/* controldata.c */
void get_control_data(ClusterInfo *cluster, bool live_check);
! void check_control_data(ControlData *oldctrl, ControlData *newctrl);
! void disable_old_cluster(void);
/* dump.c */
*************** int exec_prog(bool throw_error, const ch
*** 298,304 ****
__attribute__((format(PG_PRINTF_ATTRIBUTE, 2, 3)));
void verify_directories(void);
bool is_server_running(const char *datadir);
- void rename_old_pg_control(void);
/* file.c */
--- 298,303 ----
diff --git a/doc/src/sgml/pgupgrade.sgml b/doc/src/sgml/pgupgrade.sgml
new file mode 100644
index 1373069..4f263fe
*** a/doc/src/sgml/pgupgrade.sgml
--- b/doc/src/sgml/pgupgrade.sgml
***************
*** 182,188 ****
<para>
If you are using a version-specific installation directory, e.g.
! <filename>/opt/PostgreSQL/8.4</>, you do not need to move the old cluster. The
one-click installers all use version-specific installation directories.
</para>
--- 182,188 ----
<para>
If you are using a version-specific installation directory, e.g.
! <filename>/opt/PostgreSQL/9.1</>, you do not need to move the old cluster. The
one-click installers all use version-specific installation directories.
</para>
*************** gmake prefix=/usr/local/pgsql.new instal
*** 254,260 ****
<para>
Install any custom shared object files (or DLLs) used by the old cluster
! into the new cluster, e.g. <filename>pgcrypto.so</filename>, whether they are from <filename>contrib</filename>
or some other source. Do not install the schema definitions, e.g.
<filename>pgcrypto.sql</>, because these will be upgraded from the old cluster.
</para>
--- 254,261 ----
<para>
Install any custom shared object files (or DLLs) used by the old cluster
! into the new cluster, e.g. <filename>pgcrypto.so</filename>,
! whether they are from <filename>contrib</filename>
or some other source. Do not install the schema definitions, e.g.
<filename>pgcrypto.sql</>, because these will be upgraded from the old cluster.
</para>
*************** psql --username postgres --file script.s
*** 454,471 ****
<listitem>
<para>
! If you
! ran <command>pg_upgrade</command> <emphasis>without</> <option>--link</>
! or did not start the new server, the old cluster was not
! modified except that an <literal>.old</> suffix was appended
! to <filename>$PGDATA/global/pg_control</> and perhaps
! tablespace directories. To reuse the old cluster, remove
! the <filename>.old</> suffix
! from <filename>$PGDATA/global/pg_control</>. and, if upgrading
! to 8.4 or earlier, remove the tablespace directories created
! by the upgrade and remove the <filename>.old</> suffix from
! the tablespace directory names; then you can restart the old
! cluster.
</para>
</listitem>
</itemizedlist>
--- 455,468 ----
<listitem>
<para>
! If you ran <command>pg_upgrade</command> <emphasis>without</>
! <option>--link</> or did not start the new server, the
! old cluster was not modified except that, if linking
! started, a <literal>.old</> suffix was appended to
! <filename>$PGDATA/global/pg_control</>. To reuse the old
! cluster, possibly remove the <filename>.old</> suffix from
! <filename>$PGDATA/global/pg_control</>; you can then restart the
! old cluster.
</para>
</listitem>
</itemizedlist>
*************** psql --username postgres --file script.s
*** 582,590 ****
</para>
<para>
! If you want to use link mode and you don't want your old cluster
to be modified when the new cluster is started, make a copy of the
! old cluster and upgrade that with link mode. To make a valid copy
of the old cluster, use <command>rsync</> to create a dirty
copy of the old cluster while the server is running, then shut down
the old server and run <command>rsync</> again to update the copy with any
--- 579,587 ----
</para>
<para>
! If you want to use link mode and you do not want your old cluster
to be modified when the new cluster is started, make a copy of the
! old cluster and upgrade that in link mode. To make a valid copy
of the old cluster, use <command>rsync</> to create a dirty
copy of the old cluster while the server is running, then shut down
the old server and run <command>rsync</> again to update the copy with any
On Tue, Feb 28, 2012 at 9:45 PM, Bruce Momjian <bruce@momjian.us> wrote:
OK, I have implemented both Roberts and Àlvaro's ideas in my patch.
I only add the .old suffix to pg_controldata when link mode is used, and
I now do it after the schema has been created (the most common failure
case for pg_upgrade), and just before we actually link files --- both
very good ideas.
Thanks for working on this. I think this will be a significant
usability improvement.
Any ideas about improving the error reporting more generally, so that
when reloading the dump fails, the user can easily see what went
belly-up, even if they didn't use -l?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Wed, Feb 29, 2012 at 04:34:24PM -0500, Robert Haas wrote:
On Tue, Feb 28, 2012 at 9:45 PM, Bruce Momjian <bruce@momjian.us> wrote:
OK, I have implemented both Roberts and �lvaro's ideas in my patch.
I only add the .old suffix to pg_controldata when link mode is used, and
I now do it after the schema has been created (the most common failure
case for pg_upgrade), and just before we actually link files --- both
very good ideas.Thanks for working on this. I think this will be a significant
usability improvement.
Glad I got such good feedback and ideas.
Any ideas about improving the error reporting more generally, so that
when reloading the dump fails, the user can easily see what went
belly-up, even if they didn't use -l?
The only idea I have is to write the psql log to a temporary file and
report the last X lines from the file in case of failure. Does that
help?
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On Feb 29, 2012, at 6:02 PM, Bruce Momjian wrote:
On Wed, Feb 29, 2012 at 04:34:24PM -0500, Robert Haas wrote:
On Tue, Feb 28, 2012 at 9:45 PM, Bruce Momjian <bruce@momjian.us> wrote:
OK, I have implemented both Roberts and Àlvaro's ideas in my patch.
I only add the .old suffix to pg_controldata when link mode is used, and
I now do it after the schema has been created (the most common failure
case for pg_upgrade), and just before we actually link files --- both
very good ideas.Thanks for working on this. I think this will be a significant
usability improvement.Glad I got such good feedback and ideas.
Any ideas about improving the error reporting more generally, so that
when reloading the dump fails, the user can easily see what went
belly-up, even if they didn't use -l?The only idea I have is to write the psql log to a temporary file and
report the last X lines from the file in case of failure. Does that
help?
Perhaps pg_upgrade can print the path to the temp file containing the log and instruct the user to look there for more detail.
Cheers,
M
On Wed, Feb 29, 2012 at 06:22:27PM -0500, A.M. wrote:
On Feb 29, 2012, at 6:02 PM, Bruce Momjian wrote:
On Wed, Feb 29, 2012 at 04:34:24PM -0500, Robert Haas wrote:
On Tue, Feb 28, 2012 at 9:45 PM, Bruce Momjian <bruce@momjian.us> wrote:
OK, I have implemented both Roberts and �lvaro's ideas in my patch.
I only add the .old suffix to pg_controldata when link mode is used, and
I now do it after the schema has been created (the most common failure
case for pg_upgrade), and just before we actually link files --- both
very good ideas.Thanks for working on this. I think this will be a significant
usability improvement.Glad I got such good feedback and ideas.
Any ideas about improving the error reporting more generally, so that
when reloading the dump fails, the user can easily see what went
belly-up, even if they didn't use -l?The only idea I have is to write the psql log to a temporary file and
report the last X lines from the file in case of failure. Does that
help?Perhaps pg_upgrade can print the path to the temp file containing the
log and instruct the user to look there for more detail.
Hey, that's a good idea. I would always write the pg_dump output to a
log file. If the dump succeeds, I remove the file, if not, I tell users
to read the log file for details about the failure --- good idea.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On Wed, Feb 29, 2012 at 6:02 PM, Bruce Momjian <bruce@momjian.us> wrote:
Any ideas about improving the error reporting more generally, so that
when reloading the dump fails, the user can easily see what went
belly-up, even if they didn't use -l?The only idea I have is to write the psql log to a temporary file and
report the last X lines from the file in case of failure. Does that
help?
Why not just redirect stdout but not stderr? If there are error
messages, surely we want the user to just see those.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On ons, 2012-02-29 at 22:47 -0500, Bruce Momjian wrote:
Hey, that's a good idea. I would always write the pg_dump output to a
log file. If the dump succeeds, I remove the file, if not, I tell
users to read the log file for details about the failure --- good
idea.
But we also need the server log output somewhere. So I think this temp
file would need to cover everything that -l covers.
On Thu, Mar 01, 2012 at 08:45:26AM -0500, Robert Haas wrote:
On Wed, Feb 29, 2012 at 6:02 PM, Bruce Momjian <bruce@momjian.us> wrote:
Any ideas about improving the error reporting more generally, so that
when reloading the dump fails, the user can easily see what went
belly-up, even if they didn't use -l?The only idea I have is to write the psql log to a temporary file and
report the last X lines from the file in case of failure. �Does that
help?Why not just redirect stdout but not stderr? If there are error
messages, surely we want the user to just see those.
Well, I think sending the error messages to the user but stdout to a
file will leave users confused because it will be unclear which SQL
statement generated the error.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On Thu, Mar 01, 2012 at 10:17:04PM +0200, Peter Eisentraut wrote:
On ons, 2012-02-29 at 22:47 -0500, Bruce Momjian wrote:
Hey, that's a good idea. I would always write the pg_dump output to a
log file. If the dump succeeds, I remove the file, if not, I tell
users to read the log file for details about the failure --- good
idea.But we also need the server log output somewhere. So I think this temp
file would need to cover everything that -l covers.
OK, combining your and Robert's ideas, how about I have pg_upgrade write
the server log to a file, and the pg_dump output to a file (with its
stderr), and if pg_upgrade fails, I report the failure and mention those
files. If pg_upgrade succeeds, I remove the files? pg_upgrade already
creates temporary files that it removes on completion.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On Tue, Feb 28, 2012 at 09:45:41PM -0500, Bruce Momjian wrote:
On Tue, Feb 28, 2012 at 02:15:30PM -0500, Bruce Momjian wrote:
On Tue, Feb 28, 2012 at 01:24:45PM -0500, Robert Haas wrote:
Running this script will delete the old cluster's data files:
� �/usr/local/pgdev/pg_upgrade/delete_old_cluster.shI think you should rename the old control file just before the step
that says "linking user relation files". That's the point after which
it becomes unsafe to start the old cluster, right?Yes, it is true that that is the danger point, and also it is much less
likely to fail at that point --- it usually happens during the schema
creation. I would have to add some more conditional wording without
clearly stating if the old suffix is present.OK, I have implemented both Roberts and �lvaro's ideas in my patch.
I only add the .old suffix to pg_controldata when link mode is used, and
I now do it after the schema has been created (the most common failure
case for pg_upgrade), and just before we actually link files --- both
very good ideas.Patch attached; new pg_upgrade output with link mode below.
Patch applied. I will now work on the change to keep the schema restore
and server logs around in case of a failure.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On Thu, Mar 01, 2012 at 09:06:10PM -0500, Bruce Momjian wrote:
OK, combining your and Robert's ideas, how about I have pg_upgrade write
the server log to a file, and the pg_dump output to a file (with its
stderr), and if pg_upgrade fails, I report the failure and mention those
files. If pg_upgrade succeeds, I remove the files? pg_upgrade already
creates temporary files that it removes on completion.
OK, I have completed a rework of pg_upgrade logging. pg_upgrade had 4
logging options, -g, -G, -l, and -v, and still it wasn't possible to get
useful logging. :-(
What I have done with this patch is to remove -g, -G, and -l, and
unconditionally write to 4 log files in the current directory (in
addition to the 3 SQL files I already create).
If pg_upgrade succeeds, the files are removed, but if it fails (or if
the new -r/retain option is used), the files remain. Here is a sample
failure when I create a plpgsql function in the old server, but truncate
plpgsql.so in the new server:
Performing Upgrade
------------------
Analyzing all rows in the new cluster ok
Freezing all rows on the new cluster ok
Deleting new commit clogs ok
Copying old commit clogs to new server ok
Setting next transaction ID for new cluster ok
Resetting WAL archives ok
Setting frozenxid counters in new cluster ok
Creating databases in the new cluster ok
Adding support functions to new cluster ok
Restoring database schema to new cluster
Consult the last few lines of "pg_upgrade_restore.log" for
the probable cause of the failure.
Failure, exiting
$ tail pg_upgrade_restore.log
COMMENT ON LANGUAGE plpythonu IS 'PL/PythonU untrusted procedural language';
COMMENT
SET search_path = public, pg_catalog;
SET
CREATE FUNCTION x() RETURNS integer
LANGUAGE plpgsql
AS $$begin
select pg_sleep(1);
end$$;
psql:/usr/local/pgdev/pg_upgrade/pg_upgrade_dump_db.sql:233:
ERROR: could not load library
"/usr/local/pgsql/lib/plpgsql.so":
/usr/local/pgsql/lib/plpgsql.so: file too short
That seems quite clear; I enabled --echo-queries in psql so you can see
the query that generated the error.
These changes should make pg_upgrade errors much easier to diagnose. I
hope to apply this for PG 9.2.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
Attachments:
pg_upgrade.difftext/x-diff; charset=us-asciiDownload
diff --git a/contrib/pg_upgrade/check.c b/contrib/pg_upgrade/check.c
new file mode 100644
index a5f63eb..63f6645
*** a/contrib/pg_upgrade/check.c
--- b/contrib/pg_upgrade/check.c
*************** issue_warnings(char *sequence_script_fil
*** 165,176 ****
if (sequence_script_file_name)
{
prep_status("Adjusting sequences");
! exec_prog(true,
! SYSTEMQUOTE "\"%s/psql\" --set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
"-f \"%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user,
! sequence_script_file_name, log_opts.filename2);
unlink(sequence_script_file_name);
check_ok();
}
--- 165,177 ----
if (sequence_script_file_name)
{
prep_status("Adjusting sequences");
! exec_prog(true, UTILITY_LOG_FILE,
! SYSTEMQUOTE "\"%s/psql\" --echo-queries "
! "--set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
"-f \"%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user,
! sequence_script_file_name, UTILITY_LOG_FILE);
unlink(sequence_script_file_name);
check_ok();
}
diff --git a/contrib/pg_upgrade/controldata.c b/contrib/pg_upgrade/controldata.c
new file mode 100644
index 5239601..debb444
*** a/contrib/pg_upgrade/controldata.c
--- b/contrib/pg_upgrade/controldata.c
*************** get_control_data(ClusterInfo *cluster, b
*** 126,136 ****
/* we have the result of cmd in "output". so parse it line by line now */
while (fgets(bufin, sizeof(bufin), output))
{
! if (log_opts.debug)
! fputs(bufin, log_opts.debug_fd);
#ifdef WIN32
-
/*
* Due to an installer bug, LANG=C doesn't work for PG 8.3.3, but does
* work 8.2.6 and 8.3.7, so check for non-ASCII output and suggest a
--- 126,135 ----
/* we have the result of cmd in "output". so parse it line by line now */
while (fgets(bufin, sizeof(bufin), output))
{
! if (log_opts.verbose)
! pg_log(PG_INFO, "%s", bufin);
#ifdef WIN32
/*
* Due to an installer bug, LANG=C doesn't work for PG 8.3.3, but does
* work 8.2.6 and 8.3.7, so check for non-ASCII output and suggest a
diff --git a/contrib/pg_upgrade/dump.c b/contrib/pg_upgrade/dump.c
new file mode 100644
index 772ca37..36f9a0f
*** a/contrib/pg_upgrade/dump.c
--- b/contrib/pg_upgrade/dump.c
*************** generate_old_dump(void)
*** 22,31 ****
* --binary-upgrade records the width of dropped columns in pg_class, and
* restores the frozenid's for databases and relations.
*/
! exec_prog(true,
SYSTEMQUOTE "\"%s/pg_dumpall\" --port %d --username \"%s\" "
! "--schema-only --binary-upgrade > \"%s/" ALL_DUMP_FILE "\""
! SYSTEMQUOTE, new_cluster.bindir, old_cluster.port, os_info.user, os_info.cwd);
check_ok();
}
--- 22,32 ----
* --binary-upgrade records the width of dropped columns in pg_class, and
* restores the frozenid's for databases and relations.
*/
! exec_prog(true, UTILITY_LOG_FILE,
SYSTEMQUOTE "\"%s/pg_dumpall\" --port %d --username \"%s\" "
! "--schema-only --binary-upgrade >> \"%s/%s\" 2> \"%s\""
! SYSTEMQUOTE, new_cluster.bindir, old_cluster.port, os_info.user,
! os_info.cwd, ALL_DUMP_FILE, UTILITY_LOG_FILE);
check_ok();
}
diff --git a/contrib/pg_upgrade/exec.c b/contrib/pg_upgrade/exec.c
new file mode 100644
index b870ded..2aa3cc8
*** a/contrib/pg_upgrade/exec.c
--- b/contrib/pg_upgrade/exec.c
*************** static int win32_check_directory_write_p
*** 34,40 ****
* instead of returning should an error occur.
*/
int
! exec_prog(bool throw_error, const char *fmt,...)
{
va_list args;
int result;
--- 34,40 ----
* instead of returning should an error occur.
*/
int
! exec_prog(bool throw_error, const char *log_file, const char *fmt,...)
{
va_list args;
int result;
*************** exec_prog(bool throw_error, const char *
*** 50,57 ****
if (result != 0)
{
pg_log(throw_error ? PG_FATAL : PG_INFO,
! "There were problems executing \"%s\"\n", cmd);
return 1;
}
--- 50,60 ----
if (result != 0)
{
+ pg_log(PG_INFO, "There were problems executing \"%s\"\n", cmd);
pg_log(throw_error ? PG_FATAL : PG_INFO,
! "Consult the last few lines of \"%s\" for\n"
! "the probable cause of the failure.\n",
! log_file);
return 1;
}
diff --git a/contrib/pg_upgrade/info.c b/contrib/pg_upgrade/info.c
new file mode 100644
index 692cdc2..a1f6210
*** a/contrib/pg_upgrade/info.c
--- b/contrib/pg_upgrade/info.c
*************** create_rel_filename_map(const char *old_
*** 132,150 ****
void
print_maps(FileNameMap *maps, int n_maps, const char *db_name)
{
! if (log_opts.debug)
{
int mapnum;
! pg_log(PG_DEBUG, "mappings for database \"%s\":\n", db_name);
for (mapnum = 0; mapnum < n_maps; mapnum++)
! pg_log(PG_DEBUG, "%s.%s: %u to %u\n",
maps[mapnum].nspname, maps[mapnum].relname,
maps[mapnum].old_relfilenode,
maps[mapnum].new_relfilenode);
! pg_log(PG_DEBUG, "\n\n");
}
}
--- 132,150 ----
void
print_maps(FileNameMap *maps, int n_maps, const char *db_name)
{
! if (log_opts.verbose)
{
int mapnum;
! pg_log(PG_INFO, "mappings for database \"%s\":\n", db_name);
for (mapnum = 0; mapnum < n_maps; mapnum++)
! pg_log(PG_INFO, "%s.%s: %u to %u\n",
maps[mapnum].nspname, maps[mapnum].relname,
maps[mapnum].old_relfilenode,
maps[mapnum].new_relfilenode);
! pg_log(PG_INFO, "\n\n");
}
}
*************** get_db_and_rel_infos(ClusterInfo *cluste
*** 168,176 ****
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
get_rel_infos(cluster, &cluster->dbarr.dbs[dbnum]);
! if (log_opts.debug)
{
! pg_log(PG_DEBUG, "\n%s databases:\n", CLUSTER_NAME(cluster));
print_db_infos(&cluster->dbarr);
}
}
--- 168,176 ----
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
get_rel_infos(cluster, &cluster->dbarr.dbs[dbnum]);
! if (log_opts.verbose)
{
! pg_log(PG_INFO, "\n%s databases:\n", CLUSTER_NAME(cluster));
print_db_infos(&cluster->dbarr);
}
}
*************** print_db_infos(DbInfoArr *db_arr)
*** 368,376 ****
for (dbnum = 0; dbnum < db_arr->ndbs; dbnum++)
{
! pg_log(PG_DEBUG, "Database: %s\n", db_arr->dbs[dbnum].db_name);
print_rel_infos(&db_arr->dbs[dbnum].rel_arr);
! pg_log(PG_DEBUG, "\n\n");
}
}
--- 368,378 ----
for (dbnum = 0; dbnum < db_arr->ndbs; dbnum++)
{
! if (log_opts.verbose)
! pg_log(PG_INFO, "Database: %s\n", db_arr->dbs[dbnum].db_name);
print_rel_infos(&db_arr->dbs[dbnum].rel_arr);
! if (log_opts.verbose)
! pg_log(PG_INFO, "\n\n");
}
}
*************** print_rel_infos(RelInfoArr *arr)
*** 380,387 ****
{
int relnum;
! for (relnum = 0; relnum < arr->nrels; relnum++)
! pg_log(PG_DEBUG, "relname: %s.%s: reloid: %u reltblspace: %s\n",
! arr->rels[relnum].nspname, arr->rels[relnum].relname,
! arr->rels[relnum].reloid, arr->rels[relnum].tablespace);
}
--- 382,390 ----
{
int relnum;
! if (log_opts.verbose)
! for (relnum = 0; relnum < arr->nrels; relnum++)
! pg_log(PG_INFO, "relname: %s.%s: reloid: %u reltblspace: %s\n",
! arr->rels[relnum].nspname, arr->rels[relnum].relname,
! arr->rels[relnum].reloid, arr->rels[relnum].tablespace);
}
diff --git a/contrib/pg_upgrade/option.c b/contrib/pg_upgrade/option.c
new file mode 100644
index 0a105ef..2f30625
*** a/contrib/pg_upgrade/option.c
--- b/contrib/pg_upgrade/option.c
*************** parseCommandLine(int argc, char *argv[])
*** 46,63 ****
{"user", required_argument, NULL, 'u'},
{"check", no_argument, NULL, 'c'},
- {"debug", no_argument, NULL, 'g'},
- {"debugfile", required_argument, NULL, 'G'},
{"link", no_argument, NULL, 'k'},
! {"logfile", required_argument, NULL, 'l'},
{"verbose", no_argument, NULL, 'v'},
{NULL, 0, NULL, 0}
};
int option; /* Command line option */
int optindex = 0; /* used by getopt_long */
int os_user_effective_id;
char *return_buf;
!
user_opts.transfer_mode = TRANSFER_MODE_COPY;
os_info.progname = get_progname(argv[0]);
--- 46,63 ----
{"user", required_argument, NULL, 'u'},
{"check", no_argument, NULL, 'c'},
{"link", no_argument, NULL, 'k'},
! {"retain", no_argument, NULL, 'r'},
{"verbose", no_argument, NULL, 'v'},
{NULL, 0, NULL, 0}
};
int option; /* Command line option */
int optindex = 0; /* used by getopt_long */
int os_user_effective_id;
+ FILE *fp;
char *return_buf;
! int i;
!
user_opts.transfer_mode = TRANSFER_MODE_COPY;
os_info.progname = get_progname(argv[0]);
*************** parseCommandLine(int argc, char *argv[])
*** 98,104 ****
if (return_buf == NULL)
pg_log(PG_FATAL, "Could not access current working directory: %s\n", getErrorText(errno));
! while ((option = getopt_long(argc, argv, "d:D:b:B:cgG:kl:o:O:p:P:u:v",
long_options, &optindex)) != -1)
{
switch (option)
--- 98,104 ----
if (return_buf == NULL)
pg_log(PG_FATAL, "Could not access current working directory: %s\n", getErrorText(errno));
! while ((option = getopt_long(argc, argv, "d:D:b:B:cko:O:p:P:ru:v",
long_options, &optindex)) != -1)
{
switch (option)
*************** parseCommandLine(int argc, char *argv[])
*** 125,151 ****
new_cluster.pgconfig = pg_strdup(optarg);
break;
- case 'g':
- pg_log(PG_REPORT, "Running in debug mode\n");
- log_opts.debug = true;
- break;
-
- case 'G':
- if ((log_opts.debug_fd = fopen(optarg, "w")) == NULL)
- {
- pg_log(PG_FATAL, "cannot open debug file\n");
- exit(1);
- }
- break;
-
case 'k':
user_opts.transfer_mode = TRANSFER_MODE_LINK;
break;
- case 'l':
- log_opts.filename = pg_strdup(optarg);
- break;
-
case 'o':
old_cluster.pgopts = pg_strdup(optarg);
break;
--- 125,134 ----
*************** parseCommandLine(int argc, char *argv[])
*** 175,180 ****
--- 158,167 ----
}
break;
+ case 'r':
+ log_opts.retain = true;
+ break;
+
case 'u':
pg_free(os_info.user);
os_info.user = pg_strdup(optarg);
*************** parseCommandLine(int argc, char *argv[])
*** 189,194 ****
--- 176,183 ----
case 'v':
pg_log(PG_REPORT, "Running in verbose mode\n");
log_opts.verbose = true;
+ /* renable retain too */
+ log_opts.retain = true;
break;
default:
*************** parseCommandLine(int argc, char *argv[])
*** 199,235 ****
}
}
! if (log_opts.filename != NULL)
{
! /*
! * We must use append mode so output generated by child processes via
! * ">>" will not be overwritten, and we want the file truncated on
! * start.
! */
! /* truncate */
! if ((log_opts.fd = fopen(log_opts.filename, "w")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n", log_opts.filename);
! fclose(log_opts.fd);
! if ((log_opts.fd = fopen(log_opts.filename, "a")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n", log_opts.filename);
}
- else
- log_opts.filename = pg_strdup(DEVNULL);
! /* WIN32 files do not accept writes from multiple processes */
! #ifndef WIN32
! log_opts.filename2 = pg_strdup(log_opts.filename);
! #else
! log_opts.filename2 = pg_strdup(DEVNULL);
! #endif
!
! /* if no debug file name, output to the terminal */
! if (log_opts.debug && !log_opts.debug_fd)
! {
! log_opts.debug_fd = fopen(DEVTTY, "w");
! if (!log_opts.debug_fd)
! pg_log(PG_FATAL, "cannot write to terminal\n");
! }
/* Get values from env if not already set */
check_required_directory(&old_cluster.bindir, "PGBINOLD", "-b",
--- 188,204 ----
}
}
! /* truncate log files because we append to them */
! for (i = 0; i < NUM_OUTPUT_FILES; i++)
{
! if ((fp = fopen(output_files[i], "w")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n",
! output_files[i]);
! fclose(fp);
}
! if ((log_opts.fp = fopen(INTERNAL_LOG_FILE, "w")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n", INTERNAL_LOG_FILE);
/* Get values from env if not already set */
check_required_directory(&old_cluster.bindir, "PGBINOLD", "-b",
*************** Options:\n\
*** 256,271 ****
-c, --check check clusters only, don't change any data\n\
-d, --old-datadir=OLDDATADIR old cluster data directory\n\
-D, --new-datadir=NEWDATADIR new cluster data directory\n\
- -g, --debug enable debugging\n\
- -G, --debugfile=FILENAME output debugging activity to file\n\
-k, --link link instead of copying files to new cluster\n\
- -l, --logfile=FILENAME log internal activity to file\n\
-o, --old-options=OPTIONS old cluster options to pass to the server\n\
-O, --new-options=OPTIONS new cluster options to pass to the server\n\
-p, --old-port=OLDPORT old cluster port number (default %d)\n\
-P, --new-port=NEWPORT new cluster port number (default %d)\n\
-u, --user=NAME cluster superuser (default \"%s\")\n\
! -v, --verbose enable verbose output\n\
-V, --version display version information, then exit\n\
-h, --help show this help, then exit\n\
\n\
--- 225,238 ----
-c, --check check clusters only, don't change any data\n\
-d, --old-datadir=OLDDATADIR old cluster data directory\n\
-D, --new-datadir=NEWDATADIR new cluster data directory\n\
-k, --link link instead of copying files to new cluster\n\
-o, --old-options=OPTIONS old cluster options to pass to the server\n\
-O, --new-options=OPTIONS new cluster options to pass to the server\n\
-p, --old-port=OLDPORT old cluster port number (default %d)\n\
-P, --new-port=NEWPORT new cluster port number (default %d)\n\
+ -r, --retain retain SQL and log files after success\n\
-u, --user=NAME cluster superuser (default \"%s\")\n\
! -v, --verbose enable verbose logging, enables -r/--retain\n\
-V, --version display version information, then exit\n\
-h, --help show this help, then exit\n\
\n\
*************** adjust_data_dir(ClusterInfo *cluster)
*** 354,372 ****
{
char filename[MAXPGPATH];
char cmd[MAXPGPATH], cmd_output[MAX_STRING];
! FILE *fd, *output;
/* If there is no postgresql.conf, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/postgresql.conf", cluster->pgconfig);
! if ((fd = fopen(filename, "r")) == NULL)
return;
! fclose(fd);
/* If PG_VERSION exists, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/PG_VERSION", cluster->pgconfig);
! if ((fd = fopen(filename, "r")) != NULL)
{
! fclose(fd);
return;
}
--- 321,339 ----
{
char filename[MAXPGPATH];
char cmd[MAXPGPATH], cmd_output[MAX_STRING];
! FILE *fp, *output;
/* If there is no postgresql.conf, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/postgresql.conf", cluster->pgconfig);
! if ((fp = fopen(filename, "r")) == NULL)
return;
! fclose(fp);
/* If PG_VERSION exists, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/PG_VERSION", cluster->pgconfig);
! if ((fp = fopen(filename, "r")) != NULL)
{
! fclose(fp);
return;
}
diff --git a/contrib/pg_upgrade/pg_upgrade.c b/contrib/pg_upgrade/pg_upgrade.c
new file mode 100644
index 3078bcd..3579c9d
*** a/contrib/pg_upgrade/pg_upgrade.c
--- b/contrib/pg_upgrade/pg_upgrade.c
*************** ClusterInfo old_cluster,
*** 55,60 ****
--- 55,71 ----
new_cluster;
OSInfo os_info;
+ char *output_files[NUM_OUTPUT_FILES] = {
+ ALL_DUMP_FILE,
+ GLOBALS_DUMP_FILE,
+ DB_DUMP_FILE,
+ SERVER_LOG_FILE,
+ RESTORE_LOG_FILE,
+ UTILITY_LOG_FILE,
+ INTERNAL_LOG_FILE
+ };
+
+
int
main(int argc, char **argv)
{
*************** main(int argc, char **argv)
*** 127,135 ****
* because there is no need to have the schema load use new oids.
*/
prep_status("Setting next OID for new cluster");
! exec_prog(true, SYSTEMQUOTE "\"%s/pg_resetxlog\" -o %u \"%s\" > "
! DEVNULL SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_nxtoid, new_cluster.pgdata);
check_ok();
create_script_for_old_cluster_deletion(&deletion_script_file_name);
--- 138,148 ----
* because there is no need to have the schema load use new oids.
*/
prep_status("Setting next OID for new cluster");
! exec_prog(true, UTILITY_LOG_FILE,
! SYSTEMQUOTE "\"%s/pg_resetxlog\" -o %u \"%s\" >> \"%s\" 2>&1"
! SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_nxtoid,
! new_cluster.pgdata, UTILITY_LOG_FILE);
check_ok();
create_script_for_old_cluster_deletion(&deletion_script_file_name);
*************** prepare_new_cluster(void)
*** 193,202 ****
* --analyze so autovacuum doesn't update statistics later
*/
prep_status("Analyzing all rows in the new cluster");
! exec_prog(true,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
"--all --analyze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, log_opts.filename2);
check_ok();
/*
--- 206,215 ----
* --analyze so autovacuum doesn't update statistics later
*/
prep_status("Analyzing all rows in the new cluster");
! exec_prog(true, UTILITY_LOG_FILE,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
"--all --analyze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, UTILITY_LOG_FILE);
check_ok();
/*
*************** prepare_new_cluster(void)
*** 206,215 ****
* later.
*/
prep_status("Freezing all rows on the new cluster");
! exec_prog(true,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
"--all --freeze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, log_opts.filename2);
check_ok();
get_pg_database_relfilenode(&new_cluster);
--- 219,228 ----
* later.
*/
prep_status("Freezing all rows on the new cluster");
! exec_prog(true, UTILITY_LOG_FILE,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
"--all --freeze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, UTILITY_LOG_FILE);
check_ok();
get_pg_database_relfilenode(&new_cluster);
*************** prepare_new_databases(void)
*** 243,255 ****
* support functions in template1 but pg_dumpall creates database using
* the template0 template.
*/
! exec_prog(true,
! SYSTEMQUOTE "\"%s/psql\" --set ON_ERROR_STOP=on "
! /* --no-psqlrc prevents AUTOCOMMIT=off */
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! GLOBALS_DUMP_FILE, log_opts.filename2);
check_ok();
/* we load this to get a current list of databases */
--- 256,269 ----
* support functions in template1 but pg_dumpall creates database using
* the template0 template.
*/
! exec_prog(true, RESTORE_LOG_FILE,
! SYSTEMQUOTE "\"%s/psql\" --echo-queries "
! "--set ON_ERROR_STOP=on "
! /* --no-psqlrc prevents AUTOCOMMIT=off */
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\" 2>&1" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! GLOBALS_DUMP_FILE, RESTORE_LOG_FILE);
check_ok();
/* we load this to get a current list of databases */
*************** create_new_objects(void)
*** 275,286 ****
check_ok();
prep_status("Restoring database schema to new cluster");
! exec_prog(true,
! SYSTEMQUOTE "\"%s/psql\" --set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! DB_DUMP_FILE, log_opts.filename2);
check_ok();
/* regenerate now that we have objects in the databases */
--- 289,301 ----
check_ok();
prep_status("Restoring database schema to new cluster");
! exec_prog(true, RESTORE_LOG_FILE,
! SYSTEMQUOTE "\"%s/psql\" --echo-queries "
! "--set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\" 2>&1" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! DB_DUMP_FILE, RESTORE_LOG_FILE);
check_ok();
/* regenerate now that we have objects in the databases */
*************** copy_clog_xlog_xid(void)
*** 306,334 ****
check_ok();
prep_status("Copying old commit clogs to new server");
#ifndef WIN32
! exec_prog(true, SYSTEMQUOTE "%s \"%s\" \"%s\"" SYSTEMQUOTE,
"cp -Rf",
#else
/* flags: everything, no confirm, quiet, overwrite read-only */
! exec_prog(true, SYSTEMQUOTE "%s \"%s\" \"%s\\\"" SYSTEMQUOTE,
"xcopy /e /y /q /r",
#endif
! old_clog_path, new_clog_path);
check_ok();
/* set the next transaction id of the new cluster */
prep_status("Setting next transaction ID for new cluster");
! exec_prog(true, SYSTEMQUOTE "\"%s/pg_resetxlog\" -f -x %u \"%s\" > " DEVNULL SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_nxtxid, new_cluster.pgdata);
check_ok();
/* now reset the wal archives in the new cluster */
prep_status("Resetting WAL archives");
! exec_prog(true, SYSTEMQUOTE "\"%s/pg_resetxlog\" -l %u,%u,%u \"%s\" >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_tli,
! old_cluster.controldata.logid, old_cluster.controldata.nxtlogseg,
! new_cluster.pgdata, log_opts.filename2);
check_ok();
}
--- 321,358 ----
check_ok();
prep_status("Copying old commit clogs to new server");
+ exec_prog(true, UTILITY_LOG_FILE,
#ifndef WIN32
! SYSTEMQUOTE "%s \"%s\" \"%s\" >> \"%s\" 2>&1" SYSTEMQUOTE,
"cp -Rf",
#else
/* flags: everything, no confirm, quiet, overwrite read-only */
! SYSTEMQUOTE "%s \"%s\" \"%s\\\" >> \"%s\" 2>&1" SYSTEMQUOTE,
"xcopy /e /y /q /r",
#endif
! old_clog_path, new_clog_path, UTILITY_LOG_FILE);
check_ok();
/* set the next transaction id of the new cluster */
prep_status("Setting next transaction ID for new cluster");
! exec_prog(true, UTILITY_LOG_FILE,
! SYSTEMQUOTE
! "\"%s/pg_resetxlog\" -f -x %u \"%s\" >> \"%s\" 2>&1"
! SYSTEMQUOTE, new_cluster.bindir,
! old_cluster.controldata.chkpnt_nxtxid,
! new_cluster.pgdata, UTILITY_LOG_FILE);
check_ok();
/* now reset the wal archives in the new cluster */
prep_status("Resetting WAL archives");
! exec_prog(true, UTILITY_LOG_FILE,
! SYSTEMQUOTE
! "\"%s/pg_resetxlog\" -l %u,%u,%u \"%s\" >> \"%s\" 2>&1"
! SYSTEMQUOTE, new_cluster.bindir,
! old_cluster.controldata.chkpnt_tli,
! old_cluster.controldata.logid,
! old_cluster.controldata.nxtlogseg,
! new_cluster.pgdata, UTILITY_LOG_FILE);
check_ok();
}
*************** set_frozenxids(void)
*** 421,438 ****
static void
cleanup(void)
{
! char filename[MAXPGPATH];
!
! if (log_opts.fd)
! fclose(log_opts.fd);
! if (log_opts.debug_fd)
! fclose(log_opts.debug_fd);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, ALL_DUMP_FILE);
! unlink(filename);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, GLOBALS_DUMP_FILE);
! unlink(filename);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, DB_DUMP_FILE);
! unlink(filename);
}
--- 445,463 ----
static void
cleanup(void)
{
! fclose(log_opts.fp);
! /* Remove dump and log files? */
! if (!log_opts.retain)
! {
! int i;
! char filename[MAXPGPATH];
! for (i = 0; i < NUM_OUTPUT_FILES; i++)
! {
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd,
! output_files[i]);
! unlink(filename);
! }
! }
}
diff --git a/contrib/pg_upgrade/pg_upgrade.h b/contrib/pg_upgrade/pg_upgrade.h
new file mode 100644
index a954815..2a7f7e1
*** a/contrib/pg_upgrade/pg_upgrade.h
--- b/contrib/pg_upgrade/pg_upgrade.h
***************
*** 35,40 ****
--- 35,68 ----
#define GLOBALS_DUMP_FILE "pg_upgrade_dump_globals.sql"
#define DB_DUMP_FILE "pg_upgrade_dump_db.sql"
+ #define SERVER_LOG_FILE "pg_upgrade_server.log"
+ #define RESTORE_LOG_FILE "pg_upgrade_restore.log"
+ #define UTILITY_LOG_FILE "pg_upgrade_utility.log"
+ #define INTERNAL_LOG_FILE "pg_upgrade_internal.log"
+
+ #define NUM_OUTPUT_FILES 7
+ extern char *output_files[];
+
+ /*
+ * WIN32 files do not accept writes from multiple processes
+ *
+ * On Win32, we can't send both pg_upgrade output and command output to the
+ * same file because we get the error: "The process cannot access the file
+ * because it is being used by another process." so send the pg_ctl
+ * command-line output to the utility log file on Windows, rather than
+ * into the server log file.
+ *
+ * We could use the Windows pgwin32_open() flags to allow shared file
+ * writes but is unclear how all other tools would use those flags, so
+ * we just avoid it and log a little differently on Windows; we adjust
+ * the error message appropriately.
+ */
+ #ifndef WIN32
+ #define SERVER_LOG_FILE2 SERVER_LOG_FILE
+ #else
+ #define SERVER_LOG_FILE2 UTILITY_LOG_FILE
+ #endif
+
#ifndef WIN32
#define pg_copy_file copy_file
#define pg_mv_file rename
*************** typedef enum
*** 169,176 ****
PG_INFO,
PG_REPORT,
PG_WARNING,
! PG_FATAL,
! PG_DEBUG
} eLogType;
--- 197,203 ----
PG_INFO,
PG_REPORT,
PG_WARNING,
! PG_FATAL
} eLogType;
*************** typedef struct
*** 204,227 ****
*/
typedef struct
{
! char *filename; /* name of log file (may be /dev/null) */
! /*
! * WIN32 files do not accept writes from multiple processes
! *
! * On Win32, we can't send both pg_upgrade output and command output to the
! * same file because we get the error: "The process cannot access the file
! * because it is being used by another process." so we have to send all
! * other output to 'nul'. Therefore, we set this to DEVNULL on Win32, and
! * it equals 'filename' on all other platforms.
! *
! * We could use the Windows pgwin32_open() flags to allow shared file
! * writes but is unclear how all other tools would use those flags, so
! * we just avoid it and log a little less on Windows.
! */
! char *filename2;
! FILE *fd; /* log FILE */
! bool debug; /* TRUE -> log more information */
! FILE *debug_fd; /* debug-level log FILE */
bool verbose; /* TRUE -> be verbose in messages */
} LogOpts;
--- 231,238 ----
*/
typedef struct
{
! bool retain; /* retain log files on success */
! FILE *fp; /* log FILE */
bool verbose; /* TRUE -> be verbose in messages */
} LogOpts;
*************** void split_old_dump(void);
*** 294,301 ****
/* exec.c */
! int exec_prog(bool throw_error, const char *cmd, ...)
! __attribute__((format(PG_PRINTF_ATTRIBUTE, 2, 3)));
void verify_directories(void);
bool is_server_running(const char *datadir);
--- 305,312 ----
/* exec.c */
! int exec_prog(bool throw_error, const char *log_file, const char *cmd, ...)
! __attribute__((format(PG_PRINTF_ATTRIBUTE, 3, 4)));
void verify_directories(void);
bool is_server_running(const char *datadir);
diff --git a/contrib/pg_upgrade/server.c b/contrib/pg_upgrade/server.c
new file mode 100644
index 989af63..22c4f65
*** a/contrib/pg_upgrade/server.c
--- b/contrib/pg_upgrade/server.c
*************** executeQueryOrDie(PGconn *conn, const ch
*** 80,86 ****
vsnprintf(command, sizeof(command), fmt, args);
va_end(args);
! pg_log(PG_DEBUG, "executing: %s\n", command);
result = PQexec(conn, command);
status = PQresultStatus(result);
--- 80,87 ----
vsnprintf(command, sizeof(command), fmt, args);
va_end(args);
! if (log_opts.verbose)
! pg_log(PG_INFO, "executing: %s\n", command);
result = PQexec(conn, command);
status = PQresultStatus(result);
*************** start_postmaster(ClusterInfo *cluster)
*** 161,177 ****
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" "
"-o \"-p %d %s %s\" start >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, log_opts.filename2, cluster->pgconfig, cluster->port,
(cluster->controldata.cat_ver >=
BINARY_UPGRADE_SERVER_FLAG_CAT_VER) ? "-b" :
"-c autovacuum=off -c autovacuum_freeze_max_age=2000000000",
! cluster->pgopts ? cluster->pgopts : "", log_opts.filename2);
/*
* Don't throw an error right away, let connecting throw the error because
* it might supply a reason for the failure.
*/
! pg_ctl_return = exec_prog(false, "%s", cmd);
/* Check to see if we can connect to the server; if not, report it. */
if ((conn = get_db_conn(cluster, "template1")) == NULL ||
--- 162,183 ----
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" "
"-o \"-p %d %s %s\" start >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, SERVER_LOG_FILE, cluster->pgconfig, cluster->port,
(cluster->controldata.cat_ver >=
BINARY_UPGRADE_SERVER_FLAG_CAT_VER) ? "-b" :
"-c autovacuum=off -c autovacuum_freeze_max_age=2000000000",
! cluster->pgopts ? cluster->pgopts : "", SERVER_LOG_FILE2);
/*
* Don't throw an error right away, let connecting throw the error because
* it might supply a reason for the failure.
*/
! pg_ctl_return = exec_prog(false,
! /* pass both file names if the differ */
! (strcmp(SERVER_LOG_FILE, SERVER_LOG_FILE2) == 0) ?
! SERVER_LOG_FILE :
! SERVER_LOG_FILE " or " SERVER_LOG_FILE2,
! "%s", cmd);
/* Check to see if we can connect to the server; if not, report it. */
if ((conn = get_db_conn(cluster, "template1")) == NULL ||
*************** stop_postmaster(bool fast)
*** 211,221 ****
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" -o \"%s\" "
"%s stop >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, log_opts.filename2, cluster->pgconfig,
cluster->pgopts ? cluster->pgopts : "",
! fast ? "-m fast" : "", log_opts.filename2);
! exec_prog(fast ? false : true, "%s", cmd);
os_info.running_cluster = NULL;
}
--- 217,227 ----
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" -o \"%s\" "
"%s stop >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, SERVER_LOG_FILE2, cluster->pgconfig,
cluster->pgopts ? cluster->pgopts : "",
! fast ? "-m fast" : "", SERVER_LOG_FILE2);
! exec_prog(fast ? false : true, SERVER_LOG_FILE2, "%s", cmd);
os_info.running_cluster = NULL;
}
diff --git a/contrib/pg_upgrade/util.c b/contrib/pg_upgrade/util.c
new file mode 100644
index 94eaa18..ab78e3e
*** a/contrib/pg_upgrade/util.c
--- b/contrib/pg_upgrade/util.c
*************** pg_log(eLogType type, char *fmt,...)
*** 77,89 ****
vsnprintf(message, sizeof(message), fmt, args);
va_end(args);
! if (log_opts.fd != NULL)
{
! fwrite(message, strlen(message), 1, log_opts.fd);
/* if we are using OVERWRITE_MESSAGE, add newline */
if (strchr(message, '\r') != NULL)
! fwrite("\n", 1, 1, log_opts.fd);
! fflush(log_opts.fd);
}
switch (type)
--- 77,89 ----
vsnprintf(message, sizeof(message), fmt, args);
va_end(args);
! if (type != PG_INFO || log_opts.verbose)
{
! fwrite(message, strlen(message), 1, log_opts.fp);
/* if we are using OVERWRITE_MESSAGE, add newline */
if (strchr(message, '\r') != NULL)
! fwrite("\n", 1, 1, log_opts.fp);
! fflush(log_opts.fp);
}
switch (type)
*************** pg_log(eLogType type, char *fmt,...)
*** 104,114 ****
exit(1);
break;
- case PG_DEBUG:
- if (log_opts.debug)
- fprintf(log_opts.debug_fd, "%s\n", _(message));
- break;
-
default:
break;
}
--- 104,109 ----
diff --git a/doc/src/sgml/pgupgrade.sgml b/doc/src/sgml/pgupgrade.sgml
new file mode 100644
index 4f263fe..84b01a8
*** a/doc/src/sgml/pgupgrade.sgml
--- b/doc/src/sgml/pgupgrade.sgml
***************
*** 91,120 ****
</varlistentry>
<varlistentry>
- <term><option>-g</option></term>
- <term><option>--debug</option></term>
- <listitem><para>enable debugging</para></listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-G</option> <replaceable>debug_filename</></term>
- <term><option>--debugfile=</option><replaceable>debug_filename</></term>
- <listitem><para>output debugging activity to file</para></listitem>
- </varlistentry>
-
- <varlistentry>
<term><option>-k</option></term>
<term><option>--link</option></term>
<listitem><para>use hard links instead of copying files to the new cluster</para></listitem>
</varlistentry>
<varlistentry>
- <term><option>-l</option> <replaceable>log_filename</></term>
- <term><option>--logfile=</option><replaceable>log_filename</></term>
- <listitem><para>log internal activity to file</para></listitem>
- </varlistentry>
-
- <varlistentry>
<term><option>-o</option> <replaceable class="parameter">options</replaceable></term>
<term><option>--old-options</option> <replaceable class="parameter">options</replaceable></term>
<listitem><para>options to be passed directly to the
--- 91,102 ----
***************
*** 143,148 ****
--- 125,137 ----
</varlistentry>
<varlistentry>
+ <term><option>-r</option></term>
+ <term><option>--retain</option></term>
+ <listitem><para>retain SQL and log files even after successful completion
+ </para></listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-u</option> <replaceable>user_name</></term>
<term><option>--user=</option><replaceable>user_name</></term>
<listitem><para>cluster's super user name; environment
***************
*** 152,158 ****
<varlistentry>
<term><option>-v</option></term>
<term><option>--verbose</option></term>
! <listitem><para>enable verbose output</para></listitem>
</varlistentry>
<varlistentry>
--- 141,148 ----
<varlistentry>
<term><option>-v</option></term>
<term><option>--verbose</option></term>
! <listitem><para>enable verbose internal logging, also enables
! <option>-r</option>/<option>--retain</option></para></listitem>
</varlistentry>
<varlistentry>
On Mar 7, 2012, at 11:39 PM, Bruce Momjian wrote:
On Thu, Mar 01, 2012 at 09:06:10PM -0500, Bruce Momjian wrote:
OK, combining your and Robert's ideas, how about I have pg_upgrade write
the server log to a file, and the pg_dump output to a file (with its
stderr), and if pg_upgrade fails, I report the failure and mention those
files. If pg_upgrade succeeds, I remove the files? pg_upgrade already
creates temporary files that it removes on completion.OK, I have completed a rework of pg_upgrade logging. pg_upgrade had 4
logging options, -g, -G, -l, and -v, and still it wasn't possible to get
useful logging. :-(What I have done with this patch is to remove -g, -G, and -l, and
unconditionally write to 4 log files in the current directory (in
addition to the 3 SQL files I already create).If pg_upgrade succeeds, the files are removed, but if it fails (or if
the new -r/retain option is used), the files remain. Here is a sample
failure when I create a plpgsql function in the old server, but truncate
plpgsql.so in the new server:
It looks like the patch will overwrite the logs in the current working directory, for example, if pg_upgrade is run twice in the same place. Is that intentional? I had imagined that the logs would have been dumped in the /tmp directory so that one can compare results if the first pg_upgrade run had been errant.
Cheers,
M
On Thu, Mar 08, 2012 at 08:34:53AM -0500, A.M. wrote:
On Mar 7, 2012, at 11:39 PM, Bruce Momjian wrote:
On Thu, Mar 01, 2012 at 09:06:10PM -0500, Bruce Momjian wrote:
OK, combining your and Robert's ideas, how about I have pg_upgrade write
the server log to a file, and the pg_dump output to a file (with its
stderr), and if pg_upgrade fails, I report the failure and mention those
files. If pg_upgrade succeeds, I remove the files? pg_upgrade already
creates temporary files that it removes on completion.OK, I have completed a rework of pg_upgrade logging. pg_upgrade had 4
logging options, -g, -G, -l, and -v, and still it wasn't possible to get
useful logging. :-(What I have done with this patch is to remove -g, -G, and -l, and
unconditionally write to 4 log files in the current directory (in
addition to the 3 SQL files I already create).If pg_upgrade succeeds, the files are removed, but if it fails (or if
the new -r/retain option is used), the files remain. Here is a sample
failure when I create a plpgsql function in the old server, but truncate
plpgsql.so in the new server:It looks like the patch will overwrite the logs in the current working
directory, for example, if pg_upgrade is run twice in the same place. Is
that intentional? I had imagined that the logs would have been dumped in
Yes. I was afraid that continually appending to a log file on every run
would be too confusing. I could do only appends, or number the log
files, that those seemed confusing.
the /tmp directory so that one can compare results if the first pg_upgrade
run had been errant.
You would have to copy the file to a new name before re-running
pg_upgrade.
The only reason I truncate them on start is that I am appending to them
in many places in the code, and it was easier to just truncate them on
start rather than to remember where I first write to them.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On Mar 8, 2012, at 10:00 AM, Bruce Momjian wrote:
Yes. I was afraid that continually appending to a log file on every run
would be too confusing. I could do only appends, or number the log
files, that those seemed confusing.the /tmp directory so that one can compare results if the first pg_upgrade
run had been errant.You would have to copy the file to a new name before re-running
pg_upgrade.The only reason I truncate them on start is that I am appending to them
in many places in the code, and it was easier to just truncate them on
start rather than to remember where I first write to them.
mktemps?
Bruce Momjian <bruce@momjian.us> writes:
On Thu, Mar 08, 2012 at 08:34:53AM -0500, A.M. wrote:
It looks like the patch will overwrite the logs in the current working
directory, for example, if pg_upgrade is run twice in the same place. Is
that intentional? I had imagined that the logs would have been dumped in
Yes. I was afraid that continually appending to a log file on every run
would be too confusing. I could do only appends, or number the log
files, that those seemed confusing.
Use one (set of) files, and always append, but at the beginning of each
run print "\npg_upgrade starting at [timestamp]\n\n". Should make it
reasonably clear, while not losing information.
regards, tom lane
On Thu, Mar 01, 2012 at 08:45:26AM -0500, Robert Haas wrote:
On Wed, Feb 29, 2012 at 6:02 PM, Bruce Momjian <bruce@momjian.us> wrote:
Any ideas about improving the error reporting more generally, so that
when reloading the dump fails, the user can easily see what went
belly-up, even if they didn't use -l?The only idea I have is to write the psql log to a temporary file and
report the last X lines from the file in case of failure. �Does that
help?Why not just redirect stdout but not stderr? If there are error
messages, surely we want the user to just see those.
See my patch; the problem with splitting stdout and stderr is that you
lose the correspondence between the SQL queries and the error messages.
With my patch, they will all be right next to each other at the end of
the log file.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On tor, 2012-03-08 at 10:06 -0500, A.M. wrote:
The only reason I truncate them on start is that I am appending to
them
in many places in the code, and it was easier to just truncate them
on
start rather than to remember where I first write to them.
mktemps?
I don't want to see some tool unconditionally writing files (log or
otherwise) with unpredictable names. That would make it impossible to
clean up in a wrapper script.
On Mar 8, 2012, at 4:37 PM, Peter Eisentraut wrote:
On tor, 2012-03-08 at 10:06 -0500, A.M. wrote:
The only reason I truncate them on start is that I am appending to
them
in many places in the code, and it was easier to just truncate them
on
start rather than to remember where I first write to them.
mktemps?
I don't want to see some tool unconditionally writing files (log or
otherwise) with unpredictable names. That would make it impossible to
clean up in a wrapper script.
The point of writing temp files to the /tmp/ directory is that they don't need to be cleaned up.
You really prefer having log files written to your current working directory? I don't know of any utility that pollutes the cwd like that- it seems like an easy way to forget where one left the log files.
Cheers,
M
On tor, 2012-03-08 at 16:44 -0500, A.M. wrote:
The point of writing temp files to the /tmp/ directory is that they
don't need to be cleaned up.
I don't think so. If everyone just left their junk lying around
in /tmp, it would fill up just like any other partition.
On Thu, Mar 08, 2012 at 10:19:05AM -0500, Tom Lane wrote:
Bruce Momjian <bruce@momjian.us> writes:
On Thu, Mar 08, 2012 at 08:34:53AM -0500, A.M. wrote:
It looks like the patch will overwrite the logs in the current working
directory, for example, if pg_upgrade is run twice in the same place. Is
that intentional? I had imagined that the logs would have been dumped inYes. I was afraid that continually appending to a log file on every run
would be too confusing. I could do only appends, or number the log
files, that those seemed confusing.Use one (set of) files, and always append, but at the beginning of each
run print "\npg_upgrade starting at [timestamp]\n\n". Should make it
reasonably clear, while not losing information.
OK, it seems people do care about keeping log files from multiple runs
so I went with Tom's idea and have:
-----------------------------------------------------------------
pg_upgrade run on Thu Mar 8 19:30:12 2012
-----------------------------------------------------------------
Performing Consistency Checks
-----------------------------
Updated patch attached.
FYI, in retain mode, these are the files left in the current directory:
delete_old_cluster.sh
pg_upgrade_dump_all.sql
pg_upgrade_dump_db.sql
pg_upgrade_dump_globals.sql
pg_upgrade_internal.log
pg_upgrade_restore.log
pg_upgrade_server.log
pg_upgrade_utility.log
I will address the idea of using /tmp in another email.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
Attachments:
pg_upgrade.difftext/x-diff; charset=us-asciiDownload
diff --git a/contrib/pg_upgrade/check.c b/contrib/pg_upgrade/check.c
new file mode 100644
index a5f63eb..7905534
*** a/contrib/pg_upgrade/check.c
--- b/contrib/pg_upgrade/check.c
*************** issue_warnings(char *sequence_script_fil
*** 165,176 ****
if (sequence_script_file_name)
{
prep_status("Adjusting sequences");
! exec_prog(true,
! SYSTEMQUOTE "\"%s/psql\" --set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user,
! sequence_script_file_name, log_opts.filename2);
unlink(sequence_script_file_name);
check_ok();
}
--- 165,177 ----
if (sequence_script_file_name)
{
prep_status("Adjusting sequences");
! exec_prog(true, UTILITY_LOG_FILE,
! SYSTEMQUOTE "\"%s/psql\" --echo-queries "
! "--set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s\" --dbname template1 >> \"%s\" 2>&1" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user,
! sequence_script_file_name, UTILITY_LOG_FILE);
unlink(sequence_script_file_name);
check_ok();
}
diff --git a/contrib/pg_upgrade/controldata.c b/contrib/pg_upgrade/controldata.c
new file mode 100644
index 5239601..e01280d
*** a/contrib/pg_upgrade/controldata.c
--- b/contrib/pg_upgrade/controldata.c
*************** get_control_data(ClusterInfo *cluster, b
*** 126,136 ****
/* we have the result of cmd in "output". so parse it line by line now */
while (fgets(bufin, sizeof(bufin), output))
{
! if (log_opts.debug)
! fputs(bufin, log_opts.debug_fd);
#ifdef WIN32
-
/*
* Due to an installer bug, LANG=C doesn't work for PG 8.3.3, but does
* work 8.2.6 and 8.3.7, so check for non-ASCII output and suggest a
--- 126,134 ----
/* we have the result of cmd in "output". so parse it line by line now */
while (fgets(bufin, sizeof(bufin), output))
{
! pg_log(PG_VERBOSE, "%s", bufin);
#ifdef WIN32
/*
* Due to an installer bug, LANG=C doesn't work for PG 8.3.3, but does
* work 8.2.6 and 8.3.7, so check for non-ASCII output and suggest a
diff --git a/contrib/pg_upgrade/dump.c b/contrib/pg_upgrade/dump.c
new file mode 100644
index 772ca37..7ec94ad
*** a/contrib/pg_upgrade/dump.c
--- b/contrib/pg_upgrade/dump.c
*************** generate_old_dump(void)
*** 22,31 ****
* --binary-upgrade records the width of dropped columns in pg_class, and
* restores the frozenid's for databases and relations.
*/
! exec_prog(true,
SYSTEMQUOTE "\"%s/pg_dumpall\" --port %d --username \"%s\" "
! "--schema-only --binary-upgrade > \"%s/" ALL_DUMP_FILE "\""
! SYSTEMQUOTE, new_cluster.bindir, old_cluster.port, os_info.user, os_info.cwd);
check_ok();
}
--- 22,32 ----
* --binary-upgrade records the width of dropped columns in pg_class, and
* restores the frozenid's for databases and relations.
*/
! exec_prog(true, UTILITY_LOG_FILE,
SYSTEMQUOTE "\"%s/pg_dumpall\" --port %d --username \"%s\" "
! "--schema-only --binary-upgrade > \"%s/%s\" 2>> \"%s\""
! SYSTEMQUOTE, new_cluster.bindir, old_cluster.port, os_info.user,
! os_info.cwd, ALL_DUMP_FILE, UTILITY_LOG_FILE);
check_ok();
}
diff --git a/contrib/pg_upgrade/exec.c b/contrib/pg_upgrade/exec.c
new file mode 100644
index b870ded..571ad23
*** a/contrib/pg_upgrade/exec.c
--- b/contrib/pg_upgrade/exec.c
*************** static int win32_check_directory_write_p
*** 34,40 ****
* instead of returning should an error occur.
*/
int
! exec_prog(bool throw_error, const char *fmt,...)
{
va_list args;
int result;
--- 34,40 ----
* instead of returning should an error occur.
*/
int
! exec_prog(bool throw_error, const char *log_file, const char *fmt,...)
{
va_list args;
int result;
*************** exec_prog(bool throw_error, const char *
*** 44,57 ****
vsnprintf(cmd, MAXPGPATH, fmt, args);
va_end(args);
! pg_log(PG_INFO, "%s\n", cmd);
result = system(cmd);
if (result != 0)
{
! pg_log(throw_error ? PG_FATAL : PG_INFO,
! "There were problems executing \"%s\"\n", cmd);
return 1;
}
--- 44,60 ----
vsnprintf(cmd, MAXPGPATH, fmt, args);
va_end(args);
! pg_log(PG_VERBOSE, "%s\n", cmd);
result = system(cmd);
if (result != 0)
{
! pg_log(PG_REPORT, "There were problems executing \"%s\"\n", cmd);
! pg_log(throw_error ? PG_FATAL : PG_REPORT,
! "Consult the last few lines of \"%s\" for\n"
! "the probable cause of the failure.\n",
! log_file);
return 1;
}
diff --git a/contrib/pg_upgrade/info.c b/contrib/pg_upgrade/info.c
new file mode 100644
index 692cdc2..36683fa
*** a/contrib/pg_upgrade/info.c
--- b/contrib/pg_upgrade/info.c
*************** create_rel_filename_map(const char *old_
*** 132,150 ****
void
print_maps(FileNameMap *maps, int n_maps, const char *db_name)
{
! if (log_opts.debug)
{
int mapnum;
! pg_log(PG_DEBUG, "mappings for database \"%s\":\n", db_name);
for (mapnum = 0; mapnum < n_maps; mapnum++)
! pg_log(PG_DEBUG, "%s.%s: %u to %u\n",
maps[mapnum].nspname, maps[mapnum].relname,
maps[mapnum].old_relfilenode,
maps[mapnum].new_relfilenode);
! pg_log(PG_DEBUG, "\n\n");
}
}
--- 132,150 ----
void
print_maps(FileNameMap *maps, int n_maps, const char *db_name)
{
! if (log_opts.verbose)
{
int mapnum;
! pg_log(PG_VERBOSE, "mappings for database \"%s\":\n", db_name);
for (mapnum = 0; mapnum < n_maps; mapnum++)
! pg_log(PG_VERBOSE, "%s.%s: %u to %u\n",
maps[mapnum].nspname, maps[mapnum].relname,
maps[mapnum].old_relfilenode,
maps[mapnum].new_relfilenode);
! pg_log(PG_VERBOSE, "\n\n");
}
}
*************** get_db_and_rel_infos(ClusterInfo *cluste
*** 168,178 ****
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
get_rel_infos(cluster, &cluster->dbarr.dbs[dbnum]);
! if (log_opts.debug)
! {
! pg_log(PG_DEBUG, "\n%s databases:\n", CLUSTER_NAME(cluster));
print_db_infos(&cluster->dbarr);
- }
}
--- 168,176 ----
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
get_rel_infos(cluster, &cluster->dbarr.dbs[dbnum]);
! pg_log(PG_VERBOSE, "\n%s databases:\n", CLUSTER_NAME(cluster));
! if (log_opts.verbose)
print_db_infos(&cluster->dbarr);
}
*************** print_db_infos(DbInfoArr *db_arr)
*** 368,376 ****
for (dbnum = 0; dbnum < db_arr->ndbs; dbnum++)
{
! pg_log(PG_DEBUG, "Database: %s\n", db_arr->dbs[dbnum].db_name);
print_rel_infos(&db_arr->dbs[dbnum].rel_arr);
! pg_log(PG_DEBUG, "\n\n");
}
}
--- 366,374 ----
for (dbnum = 0; dbnum < db_arr->ndbs; dbnum++)
{
! pg_log(PG_VERBOSE, "Database: %s\n", db_arr->dbs[dbnum].db_name);
print_rel_infos(&db_arr->dbs[dbnum].rel_arr);
! pg_log(PG_VERBOSE, "\n\n");
}
}
*************** print_rel_infos(RelInfoArr *arr)
*** 381,387 ****
int relnum;
for (relnum = 0; relnum < arr->nrels; relnum++)
! pg_log(PG_DEBUG, "relname: %s.%s: reloid: %u reltblspace: %s\n",
arr->rels[relnum].nspname, arr->rels[relnum].relname,
arr->rels[relnum].reloid, arr->rels[relnum].tablespace);
}
--- 379,385 ----
int relnum;
for (relnum = 0; relnum < arr->nrels; relnum++)
! pg_log(PG_VERBOSE, "relname: %s.%s: reloid: %u reltblspace: %s\n",
arr->rels[relnum].nspname, arr->rels[relnum].relname,
arr->rels[relnum].reloid, arr->rels[relnum].tablespace);
}
diff --git a/contrib/pg_upgrade/option.c b/contrib/pg_upgrade/option.c
new file mode 100644
index 0a105ef..e365ee1
*** a/contrib/pg_upgrade/option.c
--- b/contrib/pg_upgrade/option.c
***************
*** 11,18 ****
#include "pg_upgrade.h"
! #include "getopt_long.h"
!
#ifdef WIN32
#include <io.h>
#endif
--- 11,20 ----
#include "pg_upgrade.h"
! #include <getopt_long.h>
! #include <time.h>
! #include <sys/types.h>
! #include <sys/stat.h>
#ifdef WIN32
#include <io.h>
#endif
*************** parseCommandLine(int argc, char *argv[])
*** 46,63 ****
{"user", required_argument, NULL, 'u'},
{"check", no_argument, NULL, 'c'},
- {"debug", no_argument, NULL, 'g'},
- {"debugfile", required_argument, NULL, 'G'},
{"link", no_argument, NULL, 'k'},
! {"logfile", required_argument, NULL, 'l'},
{"verbose", no_argument, NULL, 'v'},
{NULL, 0, NULL, 0}
};
int option; /* Command line option */
int optindex = 0; /* used by getopt_long */
int os_user_effective_id;
char *return_buf;
!
user_opts.transfer_mode = TRANSFER_MODE_COPY;
os_info.progname = get_progname(argv[0]);
--- 48,66 ----
{"user", required_argument, NULL, 'u'},
{"check", no_argument, NULL, 'c'},
{"link", no_argument, NULL, 'k'},
! {"retain", no_argument, NULL, 'r'},
{"verbose", no_argument, NULL, 'v'},
{NULL, 0, NULL, 0}
};
int option; /* Command line option */
int optindex = 0; /* used by getopt_long */
int os_user_effective_id;
+ FILE *fp;
char *return_buf;
! int i;
! time_t run_time = time(NULL);
!
user_opts.transfer_mode = TRANSFER_MODE_COPY;
os_info.progname = get_progname(argv[0]);
*************** parseCommandLine(int argc, char *argv[])
*** 90,95 ****
--- 93,100 ----
}
}
+ umask(0077);
+
/* Allow help and version to be run as root, so do the test here. */
if (os_user_effective_id == 0)
pg_log(PG_FATAL, "%s: cannot be run as root\n", os_info.progname);
*************** parseCommandLine(int argc, char *argv[])
*** 98,104 ****
if (return_buf == NULL)
pg_log(PG_FATAL, "Could not access current working directory: %s\n", getErrorText(errno));
! while ((option = getopt_long(argc, argv, "d:D:b:B:cgG:kl:o:O:p:P:u:v",
long_options, &optindex)) != -1)
{
switch (option)
--- 103,112 ----
if (return_buf == NULL)
pg_log(PG_FATAL, "Could not access current working directory: %s\n", getErrorText(errno));
! if ((log_opts.fp = fopen(INTERNAL_LOG_FILE, "a")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n", INTERNAL_LOG_FILE);
!
! while ((option = getopt_long(argc, argv, "d:D:b:B:cko:O:p:P:ru:v",
long_options, &optindex)) != -1)
{
switch (option)
*************** parseCommandLine(int argc, char *argv[])
*** 125,151 ****
new_cluster.pgconfig = pg_strdup(optarg);
break;
- case 'g':
- pg_log(PG_REPORT, "Running in debug mode\n");
- log_opts.debug = true;
- break;
-
- case 'G':
- if ((log_opts.debug_fd = fopen(optarg, "w")) == NULL)
- {
- pg_log(PG_FATAL, "cannot open debug file\n");
- exit(1);
- }
- break;
-
case 'k':
user_opts.transfer_mode = TRANSFER_MODE_LINK;
break;
- case 'l':
- log_opts.filename = pg_strdup(optarg);
- break;
-
case 'o':
old_cluster.pgopts = pg_strdup(optarg);
break;
--- 133,142 ----
*************** parseCommandLine(int argc, char *argv[])
*** 175,180 ****
--- 166,175 ----
}
break;
+ case 'r':
+ log_opts.retain = true;
+ break;
+
case 'u':
pg_free(os_info.user);
os_info.user = pg_strdup(optarg);
*************** parseCommandLine(int argc, char *argv[])
*** 199,234 ****
}
}
! if (log_opts.filename != NULL)
! {
! /*
! * We must use append mode so output generated by child processes via
! * ">>" will not be overwritten, and we want the file truncated on
! * start.
! */
! /* truncate */
! if ((log_opts.fd = fopen(log_opts.filename, "w")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n", log_opts.filename);
! fclose(log_opts.fd);
! if ((log_opts.fd = fopen(log_opts.filename, "a")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n", log_opts.filename);
! }
! else
! log_opts.filename = pg_strdup(DEVNULL);
!
! /* WIN32 files do not accept writes from multiple processes */
! #ifndef WIN32
! log_opts.filename2 = pg_strdup(log_opts.filename);
! #else
! log_opts.filename2 = pg_strdup(DEVNULL);
! #endif
!
! /* if no debug file name, output to the terminal */
! if (log_opts.debug && !log_opts.debug_fd)
{
! log_opts.debug_fd = fopen(DEVTTY, "w");
! if (!log_opts.debug_fd)
! pg_log(PG_FATAL, "cannot write to terminal\n");
}
/* Get values from env if not already set */
--- 194,211 ----
}
}
! /* label start of upgrade in logfiles */
! for (i = 0; i < NUM_LOG_FILES; i++)
{
! if ((fp = fopen(output_files[i], "a")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n",
! output_files[i]);
! fprintf(fp, "\n"
! "-----------------------------------------------------------------\n"
! " pg_upgrade run on %s"
! "-----------------------------------------------------------------\n\n",
! ctime(&run_time));
! fclose(fp);
}
/* Get values from env if not already set */
*************** Options:\n\
*** 256,271 ****
-c, --check check clusters only, don't change any data\n\
-d, --old-datadir=OLDDATADIR old cluster data directory\n\
-D, --new-datadir=NEWDATADIR new cluster data directory\n\
- -g, --debug enable debugging\n\
- -G, --debugfile=FILENAME output debugging activity to file\n\
-k, --link link instead of copying files to new cluster\n\
- -l, --logfile=FILENAME log internal activity to file\n\
-o, --old-options=OPTIONS old cluster options to pass to the server\n\
-O, --new-options=OPTIONS new cluster options to pass to the server\n\
-p, --old-port=OLDPORT old cluster port number (default %d)\n\
-P, --new-port=NEWPORT new cluster port number (default %d)\n\
-u, --user=NAME cluster superuser (default \"%s\")\n\
! -v, --verbose enable verbose output\n\
-V, --version display version information, then exit\n\
-h, --help show this help, then exit\n\
\n\
--- 233,246 ----
-c, --check check clusters only, don't change any data\n\
-d, --old-datadir=OLDDATADIR old cluster data directory\n\
-D, --new-datadir=NEWDATADIR new cluster data directory\n\
-k, --link link instead of copying files to new cluster\n\
-o, --old-options=OPTIONS old cluster options to pass to the server\n\
-O, --new-options=OPTIONS new cluster options to pass to the server\n\
-p, --old-port=OLDPORT old cluster port number (default %d)\n\
-P, --new-port=NEWPORT new cluster port number (default %d)\n\
+ -r, --retain retain SQL and log files after success\n\
-u, --user=NAME cluster superuser (default \"%s\")\n\
! -v, --verbose enable verbose internal logging\n\
-V, --version display version information, then exit\n\
-h, --help show this help, then exit\n\
\n\
*************** adjust_data_dir(ClusterInfo *cluster)
*** 354,372 ****
{
char filename[MAXPGPATH];
char cmd[MAXPGPATH], cmd_output[MAX_STRING];
! FILE *fd, *output;
/* If there is no postgresql.conf, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/postgresql.conf", cluster->pgconfig);
! if ((fd = fopen(filename, "r")) == NULL)
return;
! fclose(fd);
/* If PG_VERSION exists, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/PG_VERSION", cluster->pgconfig);
! if ((fd = fopen(filename, "r")) != NULL)
{
! fclose(fd);
return;
}
--- 329,347 ----
{
char filename[MAXPGPATH];
char cmd[MAXPGPATH], cmd_output[MAX_STRING];
! FILE *fp, *output;
/* If there is no postgresql.conf, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/postgresql.conf", cluster->pgconfig);
! if ((fp = fopen(filename, "r")) == NULL)
return;
! fclose(fp);
/* If PG_VERSION exists, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/PG_VERSION", cluster->pgconfig);
! if ((fp = fopen(filename, "r")) != NULL)
{
! fclose(fp);
return;
}
diff --git a/contrib/pg_upgrade/pg_upgrade.c b/contrib/pg_upgrade/pg_upgrade.c
new file mode 100644
index 3078bcd..a4511bb
*** a/contrib/pg_upgrade/pg_upgrade.c
--- b/contrib/pg_upgrade/pg_upgrade.c
*************** ClusterInfo old_cluster,
*** 55,60 ****
--- 55,68 ----
new_cluster;
OSInfo os_info;
+ char *output_files[NUM_LOG_FILES] = {
+ SERVER_LOG_FILE,
+ RESTORE_LOG_FILE,
+ UTILITY_LOG_FILE,
+ INTERNAL_LOG_FILE
+ };
+
+
int
main(int argc, char **argv)
{
*************** main(int argc, char **argv)
*** 127,135 ****
* because there is no need to have the schema load use new oids.
*/
prep_status("Setting next OID for new cluster");
! exec_prog(true, SYSTEMQUOTE "\"%s/pg_resetxlog\" -o %u \"%s\" > "
! DEVNULL SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_nxtoid, new_cluster.pgdata);
check_ok();
create_script_for_old_cluster_deletion(&deletion_script_file_name);
--- 135,145 ----
* because there is no need to have the schema load use new oids.
*/
prep_status("Setting next OID for new cluster");
! exec_prog(true, UTILITY_LOG_FILE,
! SYSTEMQUOTE "\"%s/pg_resetxlog\" -o %u \"%s\" >> \"%s\" 2>&1"
! SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_nxtoid,
! new_cluster.pgdata, UTILITY_LOG_FILE);
check_ok();
create_script_for_old_cluster_deletion(&deletion_script_file_name);
*************** prepare_new_cluster(void)
*** 193,202 ****
* --analyze so autovacuum doesn't update statistics later
*/
prep_status("Analyzing all rows in the new cluster");
! exec_prog(true,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
"--all --analyze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, log_opts.filename2);
check_ok();
/*
--- 203,212 ----
* --analyze so autovacuum doesn't update statistics later
*/
prep_status("Analyzing all rows in the new cluster");
! exec_prog(true, UTILITY_LOG_FILE,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
"--all --analyze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, UTILITY_LOG_FILE);
check_ok();
/*
*************** prepare_new_cluster(void)
*** 206,215 ****
* later.
*/
prep_status("Freezing all rows on the new cluster");
! exec_prog(true,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
"--all --freeze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, log_opts.filename2);
check_ok();
get_pg_database_relfilenode(&new_cluster);
--- 216,225 ----
* later.
*/
prep_status("Freezing all rows on the new cluster");
! exec_prog(true, UTILITY_LOG_FILE,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
"--all --freeze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, UTILITY_LOG_FILE);
check_ok();
get_pg_database_relfilenode(&new_cluster);
*************** prepare_new_databases(void)
*** 243,255 ****
* support functions in template1 but pg_dumpall creates database using
* the template0 template.
*/
! exec_prog(true,
! SYSTEMQUOTE "\"%s/psql\" --set ON_ERROR_STOP=on "
! /* --no-psqlrc prevents AUTOCOMMIT=off */
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! GLOBALS_DUMP_FILE, log_opts.filename2);
check_ok();
/* we load this to get a current list of databases */
--- 253,266 ----
* support functions in template1 but pg_dumpall creates database using
* the template0 template.
*/
! exec_prog(true, RESTORE_LOG_FILE,
! SYSTEMQUOTE "\"%s/psql\" --echo-queries "
! "--set ON_ERROR_STOP=on "
! /* --no-psqlrc prevents AUTOCOMMIT=off */
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\" 2>&1" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! GLOBALS_DUMP_FILE, RESTORE_LOG_FILE);
check_ok();
/* we load this to get a current list of databases */
*************** create_new_objects(void)
*** 275,286 ****
check_ok();
prep_status("Restoring database schema to new cluster");
! exec_prog(true,
! SYSTEMQUOTE "\"%s/psql\" --set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! DB_DUMP_FILE, log_opts.filename2);
check_ok();
/* regenerate now that we have objects in the databases */
--- 286,298 ----
check_ok();
prep_status("Restoring database schema to new cluster");
! exec_prog(true, RESTORE_LOG_FILE,
! SYSTEMQUOTE "\"%s/psql\" --echo-queries "
! "--set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\" 2>&1" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! DB_DUMP_FILE, RESTORE_LOG_FILE);
check_ok();
/* regenerate now that we have objects in the databases */
*************** copy_clog_xlog_xid(void)
*** 306,334 ****
check_ok();
prep_status("Copying old commit clogs to new server");
#ifndef WIN32
! exec_prog(true, SYSTEMQUOTE "%s \"%s\" \"%s\"" SYSTEMQUOTE,
"cp -Rf",
#else
/* flags: everything, no confirm, quiet, overwrite read-only */
! exec_prog(true, SYSTEMQUOTE "%s \"%s\" \"%s\\\"" SYSTEMQUOTE,
"xcopy /e /y /q /r",
#endif
! old_clog_path, new_clog_path);
check_ok();
/* set the next transaction id of the new cluster */
prep_status("Setting next transaction ID for new cluster");
! exec_prog(true, SYSTEMQUOTE "\"%s/pg_resetxlog\" -f -x %u \"%s\" > " DEVNULL SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_nxtxid, new_cluster.pgdata);
check_ok();
/* now reset the wal archives in the new cluster */
prep_status("Resetting WAL archives");
! exec_prog(true, SYSTEMQUOTE "\"%s/pg_resetxlog\" -l %u,%u,%u \"%s\" >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_tli,
! old_cluster.controldata.logid, old_cluster.controldata.nxtlogseg,
! new_cluster.pgdata, log_opts.filename2);
check_ok();
}
--- 318,355 ----
check_ok();
prep_status("Copying old commit clogs to new server");
+ exec_prog(true, UTILITY_LOG_FILE,
#ifndef WIN32
! SYSTEMQUOTE "%s \"%s\" \"%s\" >> \"%s\" 2>&1" SYSTEMQUOTE,
"cp -Rf",
#else
/* flags: everything, no confirm, quiet, overwrite read-only */
! SYSTEMQUOTE "%s \"%s\" \"%s\\\" >> \"%s\" 2>&1" SYSTEMQUOTE,
"xcopy /e /y /q /r",
#endif
! old_clog_path, new_clog_path, UTILITY_LOG_FILE);
check_ok();
/* set the next transaction id of the new cluster */
prep_status("Setting next transaction ID for new cluster");
! exec_prog(true, UTILITY_LOG_FILE,
! SYSTEMQUOTE
! "\"%s/pg_resetxlog\" -f -x %u \"%s\" >> \"%s\" 2>&1"
! SYSTEMQUOTE, new_cluster.bindir,
! old_cluster.controldata.chkpnt_nxtxid,
! new_cluster.pgdata, UTILITY_LOG_FILE);
check_ok();
/* now reset the wal archives in the new cluster */
prep_status("Resetting WAL archives");
! exec_prog(true, UTILITY_LOG_FILE,
! SYSTEMQUOTE
! "\"%s/pg_resetxlog\" -l %u,%u,%u \"%s\" >> \"%s\" 2>&1"
! SYSTEMQUOTE, new_cluster.bindir,
! old_cluster.controldata.chkpnt_tli,
! old_cluster.controldata.logid,
! old_cluster.controldata.nxtlogseg,
! new_cluster.pgdata, UTILITY_LOG_FILE);
check_ok();
}
*************** set_frozenxids(void)
*** 421,438 ****
static void
cleanup(void)
{
! char filename[MAXPGPATH];
! if (log_opts.fd)
! fclose(log_opts.fd);
! if (log_opts.debug_fd)
! fclose(log_opts.debug_fd);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, ALL_DUMP_FILE);
! unlink(filename);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, GLOBALS_DUMP_FILE);
! unlink(filename);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, DB_DUMP_FILE);
! unlink(filename);
}
--- 442,472 ----
static void
cleanup(void)
{
!
! fclose(log_opts.fp);
! /* Remove dump and log files? */
! if (!log_opts.retain)
! {
! char filename[MAXPGPATH];
! int i;
! for (i = 0; i < NUM_LOG_FILES; i++)
! {
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd,
! output_files[i]);
! unlink(filename);
! }
! /* remove SQL files */
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd,
! ALL_DUMP_FILE);
! unlink(filename);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd,
! GLOBALS_DUMP_FILE);
! unlink(filename);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd,
! DB_DUMP_FILE);
! unlink(filename);
! }
}
diff --git a/contrib/pg_upgrade/pg_upgrade.h b/contrib/pg_upgrade/pg_upgrade.h
new file mode 100644
index a954815..cf7d78f
*** a/contrib/pg_upgrade/pg_upgrade.h
--- b/contrib/pg_upgrade/pg_upgrade.h
***************
*** 35,40 ****
--- 35,68 ----
#define GLOBALS_DUMP_FILE "pg_upgrade_dump_globals.sql"
#define DB_DUMP_FILE "pg_upgrade_dump_db.sql"
+ #define SERVER_LOG_FILE "pg_upgrade_server.log"
+ #define RESTORE_LOG_FILE "pg_upgrade_restore.log"
+ #define UTILITY_LOG_FILE "pg_upgrade_utility.log"
+ #define INTERNAL_LOG_FILE "pg_upgrade_internal.log"
+
+ #define NUM_LOG_FILES 4
+ extern char *output_files[];
+
+ /*
+ * WIN32 files do not accept writes from multiple processes
+ *
+ * On Win32, we can't send both pg_upgrade output and command output to the
+ * same file because we get the error: "The process cannot access the file
+ * because it is being used by another process." so send the pg_ctl
+ * command-line output to the utility log file on Windows, rather than
+ * into the server log file.
+ *
+ * We could use the Windows pgwin32_open() flags to allow shared file
+ * writes but is unclear how all other tools would use those flags, so
+ * we just avoid it and log a little differently on Windows; we adjust
+ * the error message appropriately.
+ */
+ #ifndef WIN32
+ #define SERVER_LOG_FILE2 SERVER_LOG_FILE
+ #else
+ #define SERVER_LOG_FILE2 UTILITY_LOG_FILE
+ #endif
+
#ifndef WIN32
#define pg_copy_file copy_file
#define pg_mv_file rename
*************** typedef enum
*** 166,176 ****
*/
typedef enum
{
! PG_INFO,
PG_REPORT,
PG_WARNING,
! PG_FATAL,
! PG_DEBUG
} eLogType;
--- 194,203 ----
*/
typedef enum
{
! PG_VERBOSE,
PG_REPORT,
PG_WARNING,
! PG_FATAL
} eLogType;
*************** typedef struct
*** 204,228 ****
*/
typedef struct
{
! char *filename; /* name of log file (may be /dev/null) */
! /*
! * WIN32 files do not accept writes from multiple processes
! *
! * On Win32, we can't send both pg_upgrade output and command output to the
! * same file because we get the error: "The process cannot access the file
! * because it is being used by another process." so we have to send all
! * other output to 'nul'. Therefore, we set this to DEVNULL on Win32, and
! * it equals 'filename' on all other platforms.
! *
! * We could use the Windows pgwin32_open() flags to allow shared file
! * writes but is unclear how all other tools would use those flags, so
! * we just avoid it and log a little less on Windows.
! */
! char *filename2;
! FILE *fd; /* log FILE */
! bool debug; /* TRUE -> log more information */
! FILE *debug_fd; /* debug-level log FILE */
bool verbose; /* TRUE -> be verbose in messages */
} LogOpts;
--- 231,239 ----
*/
typedef struct
{
! FILE *fp; /* log FILE */
bool verbose; /* TRUE -> be verbose in messages */
+ bool retain; /* retain log files on success */
} LogOpts;
*************** void split_old_dump(void);
*** 294,301 ****
/* exec.c */
! int exec_prog(bool throw_error, const char *cmd, ...)
! __attribute__((format(PG_PRINTF_ATTRIBUTE, 2, 3)));
void verify_directories(void);
bool is_server_running(const char *datadir);
--- 305,312 ----
/* exec.c */
! int exec_prog(bool throw_error, const char *log_file, const char *cmd, ...)
! __attribute__((format(PG_PRINTF_ATTRIBUTE, 3, 4)));
void verify_directories(void);
bool is_server_running(const char *datadir);
diff --git a/contrib/pg_upgrade/relfilenode.c b/contrib/pg_upgrade/relfilenode.c
new file mode 100644
index a1e30b1..45d6c54
*** a/contrib/pg_upgrade/relfilenode.c
--- b/contrib/pg_upgrade/relfilenode.c
*************** transfer_relfile(pageCnvCtx *pageConvert
*** 267,273 ****
if (user_opts.transfer_mode == TRANSFER_MODE_COPY)
{
! pg_log(PG_INFO, "copying \"%s\" to \"%s\"\n", old_file, new_file);
if ((msg = copyAndUpdateFile(pageConverter, old_file, new_file, true)) != NULL)
pg_log(PG_FATAL, "error while copying relation \"%s.%s\" (\"%s\" to \"%s\"): %s\n",
--- 267,273 ----
if (user_opts.transfer_mode == TRANSFER_MODE_COPY)
{
! pg_log(PG_VERBOSE, "copying \"%s\" to \"%s\"\n", old_file, new_file);
if ((msg = copyAndUpdateFile(pageConverter, old_file, new_file, true)) != NULL)
pg_log(PG_FATAL, "error while copying relation \"%s.%s\" (\"%s\" to \"%s\"): %s\n",
*************** transfer_relfile(pageCnvCtx *pageConvert
*** 275,281 ****
}
else
{
! pg_log(PG_INFO, "linking \"%s\" to \"%s\"\n", old_file, new_file);
if ((msg = linkAndUpdateFile(pageConverter, old_file, new_file)) != NULL)
pg_log(PG_FATAL,
--- 275,281 ----
}
else
{
! pg_log(PG_VERBOSE, "linking \"%s\" to \"%s\"\n", old_file, new_file);
if ((msg = linkAndUpdateFile(pageConverter, old_file, new_file)) != NULL)
pg_log(PG_FATAL,
diff --git a/contrib/pg_upgrade/server.c b/contrib/pg_upgrade/server.c
new file mode 100644
index 989af63..f8d7326
*** a/contrib/pg_upgrade/server.c
--- b/contrib/pg_upgrade/server.c
*************** executeQueryOrDie(PGconn *conn, const ch
*** 80,86 ****
vsnprintf(command, sizeof(command), fmt, args);
va_end(args);
! pg_log(PG_DEBUG, "executing: %s\n", command);
result = PQexec(conn, command);
status = PQresultStatus(result);
--- 80,86 ----
vsnprintf(command, sizeof(command), fmt, args);
va_end(args);
! pg_log(PG_VERBOSE, "executing: %s\n", command);
result = PQexec(conn, command);
status = PQresultStatus(result);
*************** start_postmaster(ClusterInfo *cluster)
*** 161,177 ****
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" "
"-o \"-p %d %s %s\" start >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, log_opts.filename2, cluster->pgconfig, cluster->port,
(cluster->controldata.cat_ver >=
BINARY_UPGRADE_SERVER_FLAG_CAT_VER) ? "-b" :
"-c autovacuum=off -c autovacuum_freeze_max_age=2000000000",
! cluster->pgopts ? cluster->pgopts : "", log_opts.filename2);
/*
* Don't throw an error right away, let connecting throw the error because
* it might supply a reason for the failure.
*/
! pg_ctl_return = exec_prog(false, "%s", cmd);
/* Check to see if we can connect to the server; if not, report it. */
if ((conn = get_db_conn(cluster, "template1")) == NULL ||
--- 161,182 ----
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" "
"-o \"-p %d %s %s\" start >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, SERVER_LOG_FILE, cluster->pgconfig, cluster->port,
(cluster->controldata.cat_ver >=
BINARY_UPGRADE_SERVER_FLAG_CAT_VER) ? "-b" :
"-c autovacuum=off -c autovacuum_freeze_max_age=2000000000",
! cluster->pgopts ? cluster->pgopts : "", SERVER_LOG_FILE2);
/*
* Don't throw an error right away, let connecting throw the error because
* it might supply a reason for the failure.
*/
! pg_ctl_return = exec_prog(false,
! /* pass both file names if the differ */
! (strcmp(SERVER_LOG_FILE, SERVER_LOG_FILE2) == 0) ?
! SERVER_LOG_FILE :
! SERVER_LOG_FILE " or " SERVER_LOG_FILE2,
! "%s", cmd);
/* Check to see if we can connect to the server; if not, report it. */
if ((conn = get_db_conn(cluster, "template1")) == NULL ||
*************** stop_postmaster(bool fast)
*** 211,221 ****
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" -o \"%s\" "
"%s stop >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, log_opts.filename2, cluster->pgconfig,
cluster->pgopts ? cluster->pgopts : "",
! fast ? "-m fast" : "", log_opts.filename2);
! exec_prog(fast ? false : true, "%s", cmd);
os_info.running_cluster = NULL;
}
--- 216,226 ----
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" -o \"%s\" "
"%s stop >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, SERVER_LOG_FILE2, cluster->pgconfig,
cluster->pgopts ? cluster->pgopts : "",
! fast ? "-m fast" : "", SERVER_LOG_FILE2);
! exec_prog(fast ? false : true, SERVER_LOG_FILE2, "%s", cmd);
os_info.running_cluster = NULL;
}
diff --git a/contrib/pg_upgrade/util.c b/contrib/pg_upgrade/util.c
new file mode 100644
index 94eaa18..b039269
*** a/contrib/pg_upgrade/util.c
--- b/contrib/pg_upgrade/util.c
*************** pg_log(eLogType type, char *fmt,...)
*** 77,94 ****
vsnprintf(message, sizeof(message), fmt, args);
va_end(args);
! if (log_opts.fd != NULL)
{
! fwrite(message, strlen(message), 1, log_opts.fd);
/* if we are using OVERWRITE_MESSAGE, add newline */
if (strchr(message, '\r') != NULL)
! fwrite("\n", 1, 1, log_opts.fd);
! fflush(log_opts.fd);
}
switch (type)
{
! case PG_INFO:
if (log_opts.verbose)
printf("%s", _(message));
break;
--- 77,95 ----
vsnprintf(message, sizeof(message), fmt, args);
va_end(args);
! /* PG_VERBOSE is only output in verbose mode */
! if (type != PG_VERBOSE || log_opts.verbose)
{
! fwrite(message, strlen(message), 1, log_opts.fp);
/* if we are using OVERWRITE_MESSAGE, add newline */
if (strchr(message, '\r') != NULL)
! fwrite("\n", 1, 1, log_opts.fp);
! fflush(log_opts.fp);
}
switch (type)
{
! case PG_VERBOSE:
if (log_opts.verbose)
printf("%s", _(message));
break;
*************** pg_log(eLogType type, char *fmt,...)
*** 104,114 ****
exit(1);
break;
- case PG_DEBUG:
- if (log_opts.debug)
- fprintf(log_opts.debug_fd, "%s\n", _(message));
- break;
-
default:
break;
}
--- 105,110 ----
diff --git a/doc/src/sgml/pgupgrade.sgml b/doc/src/sgml/pgupgrade.sgml
new file mode 100644
index 4f263fe..6ecdfda
*** a/doc/src/sgml/pgupgrade.sgml
--- b/doc/src/sgml/pgupgrade.sgml
***************
*** 91,120 ****
</varlistentry>
<varlistentry>
- <term><option>-g</option></term>
- <term><option>--debug</option></term>
- <listitem><para>enable debugging</para></listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-G</option> <replaceable>debug_filename</></term>
- <term><option>--debugfile=</option><replaceable>debug_filename</></term>
- <listitem><para>output debugging activity to file</para></listitem>
- </varlistentry>
-
- <varlistentry>
<term><option>-k</option></term>
<term><option>--link</option></term>
<listitem><para>use hard links instead of copying files to the new cluster</para></listitem>
</varlistentry>
<varlistentry>
- <term><option>-l</option> <replaceable>log_filename</></term>
- <term><option>--logfile=</option><replaceable>log_filename</></term>
- <listitem><para>log internal activity to file</para></listitem>
- </varlistentry>
-
- <varlistentry>
<term><option>-o</option> <replaceable class="parameter">options</replaceable></term>
<term><option>--old-options</option> <replaceable class="parameter">options</replaceable></term>
<listitem><para>options to be passed directly to the
--- 91,102 ----
***************
*** 143,148 ****
--- 125,137 ----
</varlistentry>
<varlistentry>
+ <term><option>-r</option></term>
+ <term><option>--retain</option></term>
+ <listitem><para>retain SQL and log files even after successful completion
+ </para></listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-u</option> <replaceable>user_name</></term>
<term><option>--user=</option><replaceable>user_name</></term>
<listitem><para>cluster's super user name; environment
***************
*** 152,158 ****
<varlistentry>
<term><option>-v</option></term>
<term><option>--verbose</option></term>
! <listitem><para>enable verbose output</para></listitem>
</varlistentry>
<varlistentry>
--- 141,147 ----
<varlistentry>
<term><option>-v</option></term>
<term><option>--verbose</option></term>
! <listitem><para>enable verbose internal logging</para></listitem>
</varlistentry>
<varlistentry>
On Thu, Mar 08, 2012 at 11:59:59PM +0200, Peter Eisentraut wrote:
On tor, 2012-03-08 at 16:44 -0500, A.M. wrote:
The point of writing temp files to the /tmp/ directory is that they
don't need to be cleaned up.I don't think so. If everyone just left their junk lying around
in /tmp, it would fill up just like any other partition.
/tmp has a larger problem; see:
http://lwn.net/Articles/390323/
The only proper fix is to create a directory in /tmp and write into
there.
pg_upgrade has been writing into the current directory since it was
started, and no one has complained, and it leaves these files in the
current directory only if it fails, or --retain is used, so it will
clean up after itself. Actually, I think it originally write into the
user's home directory, but that caused problems.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On Thu, Mar 08, 2012 at 08:33:48PM -0500, Bruce Momjian wrote:
OK, it seems people do care about keeping log files from multiple runs
so I went with Tom's idea and have:-----------------------------------------------------------------
pg_upgrade run on Thu Mar 8 19:30:12 2012
-----------------------------------------------------------------Performing Consistency Checks
-----------------------------Updated patch attached.
FYI, in retain mode, these are the files left in the current directory:
delete_old_cluster.sh
pg_upgrade_dump_all.sql
pg_upgrade_dump_db.sql
pg_upgrade_dump_globals.sql
pg_upgrade_internal.log
pg_upgrade_restore.log
pg_upgrade_server.log
pg_upgrade_utility.logI will address the idea of using /tmp in another email.
OK, attached pg_upgrade patch applied, with this list of improvements in
logging:
add ability to control permissions of created files
have psql echo its queries for easier debugging
output four separate log files, and delete them on success
add -r/--retain option to keep log files after success
make log files append-only
remove -g/-G/-l logging options
sugggest tailing appropriate log file on failure
enhance -v/--verbose behavior
Thanks for all the feedback.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
Attachments:
pg_upgrade.difftext/x-diff; charset=us-asciiDownload
diff --git a/contrib/pg_upgrade/check.c b/contrib/pg_upgrade/check.c
new file mode 100644
index a5f63eb..cf43384
*** a/contrib/pg_upgrade/check.c
--- b/contrib/pg_upgrade/check.c
*************** issue_warnings(char *sequence_script_fil
*** 165,176 ****
if (sequence_script_file_name)
{
prep_status("Adjusting sequences");
! exec_prog(true,
! SYSTEMQUOTE "\"%s/psql\" --set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user,
! sequence_script_file_name, log_opts.filename2);
unlink(sequence_script_file_name);
check_ok();
}
--- 165,177 ----
if (sequence_script_file_name)
{
prep_status("Adjusting sequences");
! exec_prog(true, true, UTILITY_LOG_FILE,
! SYSTEMQUOTE "\"%s/psql\" --echo-queries "
! "--set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s\" --dbname template1 >> \"%s\" 2>&1" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user,
! sequence_script_file_name, UTILITY_LOG_FILE);
unlink(sequence_script_file_name);
check_ok();
}
*************** create_script_for_old_cluster_deletion(c
*** 393,402 ****
prep_status("Creating script to delete old cluster");
! snprintf(*deletion_script_file_name, MAXPGPATH, "%s/delete_old_cluster.%s",
! os_info.cwd, SCRIPT_EXT);
! if ((script = fopen(*deletion_script_file_name, "w")) == NULL)
pg_log(PG_FATAL, "Could not open file \"%s\": %s\n",
*deletion_script_file_name, getErrorText(errno));
--- 394,403 ----
prep_status("Creating script to delete old cluster");
! snprintf(*deletion_script_file_name, MAXPGPATH, "delete_old_cluster.%s",
! SCRIPT_EXT);
! if ((script = fopen_priv(*deletion_script_file_name, "w")) == NULL)
pg_log(PG_FATAL, "Could not open file \"%s\": %s\n",
*deletion_script_file_name, getErrorText(errno));
*************** check_for_isn_and_int8_passing_mismatch(
*** 541,548 ****
return;
}
! snprintf(output_path, sizeof(output_path), "%s/contrib_isn_and_int8_pass_by_value.txt",
! os_info.cwd);
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
--- 542,549 ----
return;
}
! snprintf(output_path, sizeof(output_path),
! "contrib_isn_and_int8_pass_by_value.txt");
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
*************** check_for_isn_and_int8_passing_mismatch(
*** 569,575 ****
for (rowno = 0; rowno < ntups; rowno++)
{
found = true;
! if (script == NULL && (script = fopen(output_path, "w")) == NULL)
pg_log(PG_FATAL, "Could not open file \"%s\": %s\n",
output_path, getErrorText(errno));
if (!db_used)
--- 570,576 ----
for (rowno = 0; rowno < ntups; rowno++)
{
found = true;
! if (script == NULL && (script = fopen_priv(output_path, "w")) == NULL)
pg_log(PG_FATAL, "Could not open file \"%s\": %s\n",
output_path, getErrorText(errno));
if (!db_used)
*************** check_for_reg_data_type_usage(ClusterInf
*** 628,635 ****
prep_status("Checking for reg* system OID user data types");
! snprintf(output_path, sizeof(output_path), "%s/tables_using_reg.txt",
! os_info.cwd);
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
--- 629,635 ----
prep_status("Checking for reg* system OID user data types");
! snprintf(output_path, sizeof(output_path), "tables_using_reg.txt");
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
*************** check_for_reg_data_type_usage(ClusterInf
*** 675,681 ****
for (rowno = 0; rowno < ntups; rowno++)
{
found = true;
! if (script == NULL && (script = fopen(output_path, "w")) == NULL)
pg_log(PG_FATAL, "Could not open file \"%s\": %s\n",
output_path, getErrorText(errno));
if (!db_used)
--- 675,681 ----
for (rowno = 0; rowno < ntups; rowno++)
{
found = true;
! if (script == NULL && (script = fopen_priv(output_path, "w")) == NULL)
pg_log(PG_FATAL, "Could not open file \"%s\": %s\n",
output_path, getErrorText(errno));
if (!db_used)
diff --git a/contrib/pg_upgrade/controldata.c b/contrib/pg_upgrade/controldata.c
new file mode 100644
index 5239601..e01280d
*** a/contrib/pg_upgrade/controldata.c
--- b/contrib/pg_upgrade/controldata.c
*************** get_control_data(ClusterInfo *cluster, b
*** 126,136 ****
/* we have the result of cmd in "output". so parse it line by line now */
while (fgets(bufin, sizeof(bufin), output))
{
! if (log_opts.debug)
! fputs(bufin, log_opts.debug_fd);
#ifdef WIN32
-
/*
* Due to an installer bug, LANG=C doesn't work for PG 8.3.3, but does
* work 8.2.6 and 8.3.7, so check for non-ASCII output and suggest a
--- 126,134 ----
/* we have the result of cmd in "output". so parse it line by line now */
while (fgets(bufin, sizeof(bufin), output))
{
! pg_log(PG_VERBOSE, "%s", bufin);
#ifdef WIN32
/*
* Due to an installer bug, LANG=C doesn't work for PG 8.3.3, but does
* work 8.2.6 and 8.3.7, so check for non-ASCII output and suggest a
diff --git a/contrib/pg_upgrade/dump.c b/contrib/pg_upgrade/dump.c
new file mode 100644
index 772ca37..571792b
*** a/contrib/pg_upgrade/dump.c
--- b/contrib/pg_upgrade/dump.c
***************
*** 11,16 ****
--- 11,17 ----
#include "pg_upgrade.h"
+ #include <sys/types.h>
void
generate_old_dump(void)
*************** generate_old_dump(void)
*** 22,31 ****
* --binary-upgrade records the width of dropped columns in pg_class, and
* restores the frozenid's for databases and relations.
*/
! exec_prog(true,
SYSTEMQUOTE "\"%s/pg_dumpall\" --port %d --username \"%s\" "
! "--schema-only --binary-upgrade > \"%s/" ALL_DUMP_FILE "\""
! SYSTEMQUOTE, new_cluster.bindir, old_cluster.port, os_info.user, os_info.cwd);
check_ok();
}
--- 23,34 ----
* --binary-upgrade records the width of dropped columns in pg_class, and
* restores the frozenid's for databases and relations.
*/
! exec_prog(true, true, UTILITY_LOG_FILE,
SYSTEMQUOTE "\"%s/pg_dumpall\" --port %d --username \"%s\" "
! "--schema-only --binary-upgrade %s > \"%s\" 2>> \"%s\""
! SYSTEMQUOTE, new_cluster.bindir, old_cluster.port, os_info.user,
! log_opts.verbose ? "--verbose" : "",
! ALL_DUMP_FILE, UTILITY_LOG_FILE);
check_ok();
}
*************** split_old_dump(void)
*** 56,70 ****
char filename[MAXPGPATH];
bool suppressed_username = false;
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, ALL_DUMP_FILE);
if ((all_dump = fopen(filename, "r")) == NULL)
pg_log(PG_FATAL, "Could not open dump file \"%s\": %s\n", filename, getErrorText(errno));
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, GLOBALS_DUMP_FILE);
! if ((globals_dump = fopen(filename, "w")) == NULL)
pg_log(PG_FATAL, "Could not write to dump file \"%s\": %s\n", filename, getErrorText(errno));
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, DB_DUMP_FILE);
! if ((db_dump = fopen(filename, "w")) == NULL)
pg_log(PG_FATAL, "Could not write to dump file \"%s\": %s\n", filename, getErrorText(errno));
current_output = globals_dump;
/* patterns used to prevent our own username from being recreated */
--- 59,74 ----
char filename[MAXPGPATH];
bool suppressed_username = false;
! snprintf(filename, sizeof(filename), "%s", ALL_DUMP_FILE);
if ((all_dump = fopen(filename, "r")) == NULL)
pg_log(PG_FATAL, "Could not open dump file \"%s\": %s\n", filename, getErrorText(errno));
! snprintf(filename, sizeof(filename), "%s", GLOBALS_DUMP_FILE);
! if ((globals_dump = fopen_priv(filename, "w")) == NULL)
pg_log(PG_FATAL, "Could not write to dump file \"%s\": %s\n", filename, getErrorText(errno));
! snprintf(filename, sizeof(filename), "%s", DB_DUMP_FILE);
! if ((db_dump = fopen_priv(filename, "w")) == NULL)
pg_log(PG_FATAL, "Could not write to dump file \"%s\": %s\n", filename, getErrorText(errno));
+
current_output = globals_dump;
/* patterns used to prevent our own username from being recreated */
diff --git a/contrib/pg_upgrade/exec.c b/contrib/pg_upgrade/exec.c
new file mode 100644
index b870ded..42c5c0f
*** a/contrib/pg_upgrade/exec.c
--- b/contrib/pg_upgrade/exec.c
***************
*** 13,19 ****
#include <fcntl.h>
#include <unistd.h>
!
static void check_data_dir(const char *pg_data);
static void check_bin_dir(ClusterInfo *cluster);
--- 13,19 ----
#include <fcntl.h>
#include <unistd.h>
! #include <sys/types.h>
static void check_data_dir(const char *pg_data);
static void check_bin_dir(ClusterInfo *cluster);
*************** static int win32_check_directory_write_p
*** 34,57 ****
* instead of returning should an error occur.
*/
int
! exec_prog(bool throw_error, const char *fmt,...)
{
va_list args;
int result;
char cmd[MAXPGPATH];
va_start(args, fmt);
vsnprintf(cmd, MAXPGPATH, fmt, args);
va_end(args);
! pg_log(PG_INFO, "%s\n", cmd);
result = system(cmd);
if (result != 0)
{
! pg_log(throw_error ? PG_FATAL : PG_INFO,
! "There were problems executing \"%s\"\n", cmd);
return 1;
}
--- 34,70 ----
* instead of returning should an error occur.
*/
int
! exec_prog(bool throw_error, bool is_priv,
! const char *log_file, const char *fmt,...)
{
va_list args;
int result;
char cmd[MAXPGPATH];
+ mode_t old_umask;
+
+ if (is_priv)
+ old_umask = umask(S_IRWXG | S_IRWXO);
va_start(args, fmt);
vsnprintf(cmd, MAXPGPATH, fmt, args);
va_end(args);
! pg_log(PG_VERBOSE, "%s\n", cmd);
result = system(cmd);
+ if (is_priv)
+ umask(old_umask);
+
if (result != 0)
{
! report_status(PG_REPORT, "*failure*");
! fflush(stdout);
! pg_log(PG_VERBOSE, "There were problems executing \"%s\"\n", cmd);
! pg_log(throw_error ? PG_FATAL : PG_REPORT,
! "Consult the last few lines of \"%s\" for\n"
! "the probable cause of the failure.\n",
! log_file);
return 1;
}
diff --git a/contrib/pg_upgrade/file.c b/contrib/pg_upgrade/file.c
new file mode 100644
index fcf1c44..0276636
*** a/contrib/pg_upgrade/file.c
--- b/contrib/pg_upgrade/file.c
*************** win32_pghardlink(const char *src, const
*** 316,318 ****
--- 316,334 ----
}
#endif
+
+
+ /* fopen() file with no group/other permissions */
+ FILE *
+ fopen_priv(const char *path, const char *mode)
+ {
+ mode_t old_umask = umask(S_IRWXG | S_IRWXO);
+ FILE *fp;
+
+ fp = fopen(path, mode);
+ umask(old_umask);
+
+ return fp;
+ }
+
+
diff --git a/contrib/pg_upgrade/function.c b/contrib/pg_upgrade/function.c
new file mode 100644
index 267f291..3225039
*** a/contrib/pg_upgrade/function.c
--- b/contrib/pg_upgrade/function.c
*************** check_loadable_libraries(void)
*** 218,225 ****
prep_status("Checking for presence of required libraries");
! snprintf(output_path, sizeof(output_path), "%s/loadable_libraries.txt",
! os_info.cwd);
for (libnum = 0; libnum < os_info.num_libraries; libnum++)
{
--- 218,224 ----
prep_status("Checking for presence of required libraries");
! snprintf(output_path, sizeof(output_path), "loadable_libraries.txt");
for (libnum = 0; libnum < os_info.num_libraries; libnum++)
{
*************** check_loadable_libraries(void)
*** 257,263 ****
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
found = true;
! if (script == NULL && (script = fopen(output_path, "w")) == NULL)
pg_log(PG_FATAL, "Could not open file \"%s\": %s\n",
output_path, getErrorText(errno));
fprintf(script, "Could not load library \"%s\"\n%s\n",
--- 256,262 ----
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
found = true;
! if (script == NULL && (script = fopen_priv(output_path, "w")) == NULL)
pg_log(PG_FATAL, "Could not open file \"%s\": %s\n",
output_path, getErrorText(errno));
fprintf(script, "Could not load library \"%s\"\n%s\n",
diff --git a/contrib/pg_upgrade/info.c b/contrib/pg_upgrade/info.c
new file mode 100644
index 692cdc2..36683fa
*** a/contrib/pg_upgrade/info.c
--- b/contrib/pg_upgrade/info.c
*************** create_rel_filename_map(const char *old_
*** 132,150 ****
void
print_maps(FileNameMap *maps, int n_maps, const char *db_name)
{
! if (log_opts.debug)
{
int mapnum;
! pg_log(PG_DEBUG, "mappings for database \"%s\":\n", db_name);
for (mapnum = 0; mapnum < n_maps; mapnum++)
! pg_log(PG_DEBUG, "%s.%s: %u to %u\n",
maps[mapnum].nspname, maps[mapnum].relname,
maps[mapnum].old_relfilenode,
maps[mapnum].new_relfilenode);
! pg_log(PG_DEBUG, "\n\n");
}
}
--- 132,150 ----
void
print_maps(FileNameMap *maps, int n_maps, const char *db_name)
{
! if (log_opts.verbose)
{
int mapnum;
! pg_log(PG_VERBOSE, "mappings for database \"%s\":\n", db_name);
for (mapnum = 0; mapnum < n_maps; mapnum++)
! pg_log(PG_VERBOSE, "%s.%s: %u to %u\n",
maps[mapnum].nspname, maps[mapnum].relname,
maps[mapnum].old_relfilenode,
maps[mapnum].new_relfilenode);
! pg_log(PG_VERBOSE, "\n\n");
}
}
*************** get_db_and_rel_infos(ClusterInfo *cluste
*** 168,178 ****
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
get_rel_infos(cluster, &cluster->dbarr.dbs[dbnum]);
! if (log_opts.debug)
! {
! pg_log(PG_DEBUG, "\n%s databases:\n", CLUSTER_NAME(cluster));
print_db_infos(&cluster->dbarr);
- }
}
--- 168,176 ----
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
get_rel_infos(cluster, &cluster->dbarr.dbs[dbnum]);
! pg_log(PG_VERBOSE, "\n%s databases:\n", CLUSTER_NAME(cluster));
! if (log_opts.verbose)
print_db_infos(&cluster->dbarr);
}
*************** print_db_infos(DbInfoArr *db_arr)
*** 368,376 ****
for (dbnum = 0; dbnum < db_arr->ndbs; dbnum++)
{
! pg_log(PG_DEBUG, "Database: %s\n", db_arr->dbs[dbnum].db_name);
print_rel_infos(&db_arr->dbs[dbnum].rel_arr);
! pg_log(PG_DEBUG, "\n\n");
}
}
--- 366,374 ----
for (dbnum = 0; dbnum < db_arr->ndbs; dbnum++)
{
! pg_log(PG_VERBOSE, "Database: %s\n", db_arr->dbs[dbnum].db_name);
print_rel_infos(&db_arr->dbs[dbnum].rel_arr);
! pg_log(PG_VERBOSE, "\n\n");
}
}
*************** print_rel_infos(RelInfoArr *arr)
*** 381,387 ****
int relnum;
for (relnum = 0; relnum < arr->nrels; relnum++)
! pg_log(PG_DEBUG, "relname: %s.%s: reloid: %u reltblspace: %s\n",
arr->rels[relnum].nspname, arr->rels[relnum].relname,
arr->rels[relnum].reloid, arr->rels[relnum].tablespace);
}
--- 379,385 ----
int relnum;
for (relnum = 0; relnum < arr->nrels; relnum++)
! pg_log(PG_VERBOSE, "relname: %s.%s: reloid: %u reltblspace: %s\n",
arr->rels[relnum].nspname, arr->rels[relnum].relname,
arr->rels[relnum].reloid, arr->rels[relnum].tablespace);
}
diff --git a/contrib/pg_upgrade/option.c b/contrib/pg_upgrade/option.c
new file mode 100644
index 0a105ef..a97be28
*** a/contrib/pg_upgrade/option.c
--- b/contrib/pg_upgrade/option.c
***************
*** 11,18 ****
#include "pg_upgrade.h"
! #include "getopt_long.h"
!
#ifdef WIN32
#include <io.h>
#endif
--- 11,20 ----
#include "pg_upgrade.h"
! #include <getopt_long.h>
! #include <time.h>
! #include <sys/types.h>
! #include <sys/stat.h>
#ifdef WIN32
#include <io.h>
#endif
*************** parseCommandLine(int argc, char *argv[])
*** 46,63 ****
{"user", required_argument, NULL, 'u'},
{"check", no_argument, NULL, 'c'},
- {"debug", no_argument, NULL, 'g'},
- {"debugfile", required_argument, NULL, 'G'},
{"link", no_argument, NULL, 'k'},
! {"logfile", required_argument, NULL, 'l'},
{"verbose", no_argument, NULL, 'v'},
{NULL, 0, NULL, 0}
};
int option; /* Command line option */
int optindex = 0; /* used by getopt_long */
int os_user_effective_id;
! char *return_buf;
!
user_opts.transfer_mode = TRANSFER_MODE_COPY;
os_info.progname = get_progname(argv[0]);
--- 48,65 ----
{"user", required_argument, NULL, 'u'},
{"check", no_argument, NULL, 'c'},
{"link", no_argument, NULL, 'k'},
! {"retain", no_argument, NULL, 'r'},
{"verbose", no_argument, NULL, 'v'},
{NULL, 0, NULL, 0}
};
int option; /* Command line option */
int optindex = 0; /* used by getopt_long */
int os_user_effective_id;
! FILE *fp;
! int i;
! time_t run_time = time(NULL);
!
user_opts.transfer_mode = TRANSFER_MODE_COPY;
os_info.progname = get_progname(argv[0]);
*************** parseCommandLine(int argc, char *argv[])
*** 94,104 ****
if (os_user_effective_id == 0)
pg_log(PG_FATAL, "%s: cannot be run as root\n", os_info.progname);
! return_buf = getcwd(os_info.cwd, MAXPGPATH);
! if (return_buf == NULL)
! pg_log(PG_FATAL, "Could not access current working directory: %s\n", getErrorText(errno));
! while ((option = getopt_long(argc, argv, "d:D:b:B:cgG:kl:o:O:p:P:u:v",
long_options, &optindex)) != -1)
{
switch (option)
--- 96,105 ----
if (os_user_effective_id == 0)
pg_log(PG_FATAL, "%s: cannot be run as root\n", os_info.progname);
! if ((log_opts.internal = fopen_priv(INTERNAL_LOG_FILE, "a")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n", INTERNAL_LOG_FILE);
! while ((option = getopt_long(argc, argv, "d:D:b:B:cko:O:p:P:ru:v",
long_options, &optindex)) != -1)
{
switch (option)
*************** parseCommandLine(int argc, char *argv[])
*** 125,151 ****
new_cluster.pgconfig = pg_strdup(optarg);
break;
- case 'g':
- pg_log(PG_REPORT, "Running in debug mode\n");
- log_opts.debug = true;
- break;
-
- case 'G':
- if ((log_opts.debug_fd = fopen(optarg, "w")) == NULL)
- {
- pg_log(PG_FATAL, "cannot open debug file\n");
- exit(1);
- }
- break;
-
case 'k':
user_opts.transfer_mode = TRANSFER_MODE_LINK;
break;
- case 'l':
- log_opts.filename = pg_strdup(optarg);
- break;
-
case 'o':
old_cluster.pgopts = pg_strdup(optarg);
break;
--- 126,135 ----
*************** parseCommandLine(int argc, char *argv[])
*** 175,180 ****
--- 159,168 ----
}
break;
+ case 'r':
+ log_opts.retain = true;
+ break;
+
case 'u':
pg_free(os_info.user);
os_info.user = pg_strdup(optarg);
*************** parseCommandLine(int argc, char *argv[])
*** 199,234 ****
}
}
! if (log_opts.filename != NULL)
! {
! /*
! * We must use append mode so output generated by child processes via
! * ">>" will not be overwritten, and we want the file truncated on
! * start.
! */
! /* truncate */
! if ((log_opts.fd = fopen(log_opts.filename, "w")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n", log_opts.filename);
! fclose(log_opts.fd);
! if ((log_opts.fd = fopen(log_opts.filename, "a")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n", log_opts.filename);
! }
! else
! log_opts.filename = pg_strdup(DEVNULL);
!
! /* WIN32 files do not accept writes from multiple processes */
! #ifndef WIN32
! log_opts.filename2 = pg_strdup(log_opts.filename);
! #else
! log_opts.filename2 = pg_strdup(DEVNULL);
! #endif
!
! /* if no debug file name, output to the terminal */
! if (log_opts.debug && !log_opts.debug_fd)
{
! log_opts.debug_fd = fopen(DEVTTY, "w");
! if (!log_opts.debug_fd)
! pg_log(PG_FATAL, "cannot write to terminal\n");
}
/* Get values from env if not already set */
--- 187,204 ----
}
}
! /* label start of upgrade in logfiles */
! for (i = 0; i < NUM_LOG_FILES; i++)
{
! if ((fp = fopen_priv(output_files[i], "a")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n",
! output_files[i]);
! fprintf(fp, "\n"
! "-----------------------------------------------------------------\n"
! " pg_upgrade run on %s"
! "-----------------------------------------------------------------\n\n",
! ctime(&run_time));
! fclose(fp);
}
/* Get values from env if not already set */
*************** Options:\n\
*** 256,271 ****
-c, --check check clusters only, don't change any data\n\
-d, --old-datadir=OLDDATADIR old cluster data directory\n\
-D, --new-datadir=NEWDATADIR new cluster data directory\n\
- -g, --debug enable debugging\n\
- -G, --debugfile=FILENAME output debugging activity to file\n\
-k, --link link instead of copying files to new cluster\n\
- -l, --logfile=FILENAME log internal activity to file\n\
-o, --old-options=OPTIONS old cluster options to pass to the server\n\
-O, --new-options=OPTIONS new cluster options to pass to the server\n\
-p, --old-port=OLDPORT old cluster port number (default %d)\n\
-P, --new-port=NEWPORT new cluster port number (default %d)\n\
-u, --user=NAME cluster superuser (default \"%s\")\n\
! -v, --verbose enable verbose output\n\
-V, --version display version information, then exit\n\
-h, --help show this help, then exit\n\
\n\
--- 226,239 ----
-c, --check check clusters only, don't change any data\n\
-d, --old-datadir=OLDDATADIR old cluster data directory\n\
-D, --new-datadir=NEWDATADIR new cluster data directory\n\
-k, --link link instead of copying files to new cluster\n\
-o, --old-options=OPTIONS old cluster options to pass to the server\n\
-O, --new-options=OPTIONS new cluster options to pass to the server\n\
-p, --old-port=OLDPORT old cluster port number (default %d)\n\
-P, --new-port=NEWPORT new cluster port number (default %d)\n\
+ -r, --retain retain SQL and log files after success\n\
-u, --user=NAME cluster superuser (default \"%s\")\n\
! -v, --verbose enable verbose internal logging\n\
-V, --version display version information, then exit\n\
-h, --help show this help, then exit\n\
\n\
*************** adjust_data_dir(ClusterInfo *cluster)
*** 354,372 ****
{
char filename[MAXPGPATH];
char cmd[MAXPGPATH], cmd_output[MAX_STRING];
! FILE *fd, *output;
/* If there is no postgresql.conf, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/postgresql.conf", cluster->pgconfig);
! if ((fd = fopen(filename, "r")) == NULL)
return;
! fclose(fd);
/* If PG_VERSION exists, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/PG_VERSION", cluster->pgconfig);
! if ((fd = fopen(filename, "r")) != NULL)
{
! fclose(fd);
return;
}
--- 322,340 ----
{
char filename[MAXPGPATH];
char cmd[MAXPGPATH], cmd_output[MAX_STRING];
! FILE *fp, *output;
/* If there is no postgresql.conf, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/postgresql.conf", cluster->pgconfig);
! if ((fp = fopen(filename, "r")) == NULL)
return;
! fclose(fp);
/* If PG_VERSION exists, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/PG_VERSION", cluster->pgconfig);
! if ((fp = fopen(filename, "r")) != NULL)
{
! fclose(fp);
return;
}
diff --git a/contrib/pg_upgrade/pg_upgrade.c b/contrib/pg_upgrade/pg_upgrade.c
new file mode 100644
index 3078bcd..269f8ad
*** a/contrib/pg_upgrade/pg_upgrade.c
--- b/contrib/pg_upgrade/pg_upgrade.c
*************** ClusterInfo old_cluster,
*** 55,60 ****
--- 55,68 ----
new_cluster;
OSInfo os_info;
+ char *output_files[NUM_LOG_FILES] = {
+ SERVER_LOG_FILE,
+ RESTORE_LOG_FILE,
+ UTILITY_LOG_FILE,
+ INTERNAL_LOG_FILE
+ };
+
+
int
main(int argc, char **argv)
{
*************** main(int argc, char **argv)
*** 127,135 ****
* because there is no need to have the schema load use new oids.
*/
prep_status("Setting next OID for new cluster");
! exec_prog(true, SYSTEMQUOTE "\"%s/pg_resetxlog\" -o %u \"%s\" > "
! DEVNULL SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_nxtoid, new_cluster.pgdata);
check_ok();
create_script_for_old_cluster_deletion(&deletion_script_file_name);
--- 135,145 ----
* because there is no need to have the schema load use new oids.
*/
prep_status("Setting next OID for new cluster");
! exec_prog(true, true, UTILITY_LOG_FILE,
! SYSTEMQUOTE "\"%s/pg_resetxlog\" -o %u \"%s\" >> \"%s\" 2>&1"
! SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_nxtoid,
! new_cluster.pgdata, UTILITY_LOG_FILE);
check_ok();
create_script_for_old_cluster_deletion(&deletion_script_file_name);
*************** prepare_new_cluster(void)
*** 193,202 ****
* --analyze so autovacuum doesn't update statistics later
*/
prep_status("Analyzing all rows in the new cluster");
! exec_prog(true,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
! "--all --analyze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, log_opts.filename2);
check_ok();
/*
--- 203,213 ----
* --analyze so autovacuum doesn't update statistics later
*/
prep_status("Analyzing all rows in the new cluster");
! exec_prog(true, true, UTILITY_LOG_FILE,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
! "--all --analyze %s >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user,
! log_opts.verbose ? "--verbose" : "", UTILITY_LOG_FILE);
check_ok();
/*
*************** prepare_new_cluster(void)
*** 206,215 ****
* later.
*/
prep_status("Freezing all rows on the new cluster");
! exec_prog(true,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
! "--all --freeze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, log_opts.filename2);
check_ok();
get_pg_database_relfilenode(&new_cluster);
--- 217,227 ----
* later.
*/
prep_status("Freezing all rows on the new cluster");
! exec_prog(true, true, UTILITY_LOG_FILE,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
! "--all --freeze %s >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user,
! log_opts.verbose ? "--verbose" : "", UTILITY_LOG_FILE);
check_ok();
get_pg_database_relfilenode(&new_cluster);
*************** prepare_new_databases(void)
*** 243,255 ****
* support functions in template1 but pg_dumpall creates database using
* the template0 template.
*/
! exec_prog(true,
! SYSTEMQUOTE "\"%s/psql\" --set ON_ERROR_STOP=on "
! /* --no-psqlrc prevents AUTOCOMMIT=off */
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! GLOBALS_DUMP_FILE, log_opts.filename2);
check_ok();
/* we load this to get a current list of databases */
--- 255,268 ----
* support functions in template1 but pg_dumpall creates database using
* the template0 template.
*/
! exec_prog(true, true, RESTORE_LOG_FILE,
! SYSTEMQUOTE "\"%s/psql\" --echo-queries "
! "--set ON_ERROR_STOP=on "
! /* --no-psqlrc prevents AUTOCOMMIT=off */
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s\" --dbname template1 >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user,
! GLOBALS_DUMP_FILE, RESTORE_LOG_FILE);
check_ok();
/* we load this to get a current list of databases */
*************** create_new_objects(void)
*** 275,286 ****
check_ok();
prep_status("Restoring database schema to new cluster");
! exec_prog(true,
! SYSTEMQUOTE "\"%s/psql\" --set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! DB_DUMP_FILE, log_opts.filename2);
check_ok();
/* regenerate now that we have objects in the databases */
--- 288,300 ----
check_ok();
prep_status("Restoring database schema to new cluster");
! exec_prog(true, true, RESTORE_LOG_FILE,
! SYSTEMQUOTE "\"%s/psql\" --echo-queries "
! "--set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s\" --dbname template1 >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user,
! DB_DUMP_FILE, RESTORE_LOG_FILE);
check_ok();
/* regenerate now that we have objects in the databases */
*************** copy_clog_xlog_xid(void)
*** 306,334 ****
check_ok();
prep_status("Copying old commit clogs to new server");
#ifndef WIN32
! exec_prog(true, SYSTEMQUOTE "%s \"%s\" \"%s\"" SYSTEMQUOTE,
"cp -Rf",
#else
/* flags: everything, no confirm, quiet, overwrite read-only */
! exec_prog(true, SYSTEMQUOTE "%s \"%s\" \"%s\\\"" SYSTEMQUOTE,
"xcopy /e /y /q /r",
#endif
! old_clog_path, new_clog_path);
check_ok();
/* set the next transaction id of the new cluster */
prep_status("Setting next transaction ID for new cluster");
! exec_prog(true, SYSTEMQUOTE "\"%s/pg_resetxlog\" -f -x %u \"%s\" > " DEVNULL SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_nxtxid, new_cluster.pgdata);
check_ok();
/* now reset the wal archives in the new cluster */
prep_status("Resetting WAL archives");
! exec_prog(true, SYSTEMQUOTE "\"%s/pg_resetxlog\" -l %u,%u,%u \"%s\" >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_tli,
! old_cluster.controldata.logid, old_cluster.controldata.nxtlogseg,
! new_cluster.pgdata, log_opts.filename2);
check_ok();
}
--- 320,357 ----
check_ok();
prep_status("Copying old commit clogs to new server");
+ exec_prog(true, false, UTILITY_LOG_FILE,
#ifndef WIN32
! SYSTEMQUOTE "%s \"%s\" \"%s\" >> \"%s\" 2>&1" SYSTEMQUOTE,
"cp -Rf",
#else
/* flags: everything, no confirm, quiet, overwrite read-only */
! SYSTEMQUOTE "%s \"%s\" \"%s\\\" >> \"%s\" 2>&1" SYSTEMQUOTE,
"xcopy /e /y /q /r",
#endif
! old_clog_path, new_clog_path, UTILITY_LOG_FILE);
check_ok();
/* set the next transaction id of the new cluster */
prep_status("Setting next transaction ID for new cluster");
! exec_prog(true, true, UTILITY_LOG_FILE,
! SYSTEMQUOTE
! "\"%s/pg_resetxlog\" -f -x %u \"%s\" >> \"%s\" 2>&1"
! SYSTEMQUOTE, new_cluster.bindir,
! old_cluster.controldata.chkpnt_nxtxid,
! new_cluster.pgdata, UTILITY_LOG_FILE);
check_ok();
/* now reset the wal archives in the new cluster */
prep_status("Resetting WAL archives");
! exec_prog(true, true, UTILITY_LOG_FILE,
! SYSTEMQUOTE
! "\"%s/pg_resetxlog\" -l %u,%u,%u \"%s\" >> \"%s\" 2>&1"
! SYSTEMQUOTE, new_cluster.bindir,
! old_cluster.controldata.chkpnt_tli,
! old_cluster.controldata.logid,
! old_cluster.controldata.nxtlogseg,
! new_cluster.pgdata, UTILITY_LOG_FILE);
check_ok();
}
*************** set_frozenxids(void)
*** 421,438 ****
static void
cleanup(void)
{
! char filename[MAXPGPATH];
! if (log_opts.fd)
! fclose(log_opts.fd);
! if (log_opts.debug_fd)
! fclose(log_opts.debug_fd);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, ALL_DUMP_FILE);
! unlink(filename);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, GLOBALS_DUMP_FILE);
! unlink(filename);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, DB_DUMP_FILE);
! unlink(filename);
}
--- 444,470 ----
static void
cleanup(void)
{
!
! fclose(log_opts.internal);
! /* Remove dump and log files? */
! if (!log_opts.retain)
! {
! char filename[MAXPGPATH];
! int i;
! for (i = 0; i < NUM_LOG_FILES; i++)
! {
! snprintf(filename, sizeof(filename), "%s", output_files[i]);
! unlink(filename);
! }
! /* remove SQL files */
! snprintf(filename, sizeof(filename), "%s", ALL_DUMP_FILE);
! unlink(filename);
! snprintf(filename, sizeof(filename), "%s", GLOBALS_DUMP_FILE);
! unlink(filename);
! snprintf(filename, sizeof(filename), "%s", DB_DUMP_FILE);
! unlink(filename);
! }
}
diff --git a/contrib/pg_upgrade/pg_upgrade.h b/contrib/pg_upgrade/pg_upgrade.h
new file mode 100644
index a954815..46f9169
*** a/contrib/pg_upgrade/pg_upgrade.h
--- b/contrib/pg_upgrade/pg_upgrade.h
***************
*** 35,40 ****
--- 35,68 ----
#define GLOBALS_DUMP_FILE "pg_upgrade_dump_globals.sql"
#define DB_DUMP_FILE "pg_upgrade_dump_db.sql"
+ #define SERVER_LOG_FILE "pg_upgrade_server.log"
+ #define RESTORE_LOG_FILE "pg_upgrade_restore.log"
+ #define UTILITY_LOG_FILE "pg_upgrade_utility.log"
+ #define INTERNAL_LOG_FILE "pg_upgrade_internal.log"
+
+ #define NUM_LOG_FILES 4
+ extern char *output_files[];
+
+ /*
+ * WIN32 files do not accept writes from multiple processes
+ *
+ * On Win32, we can't send both pg_upgrade output and command output to the
+ * same file because we get the error: "The process cannot access the file
+ * because it is being used by another process." so send the pg_ctl
+ * command-line output to the utility log file on Windows, rather than
+ * into the server log file.
+ *
+ * We could use the Windows pgwin32_open() flags to allow shared file
+ * writes but is unclear how all other tools would use those flags, so
+ * we just avoid it and log a little differently on Windows; we adjust
+ * the error message appropriately.
+ */
+ #ifndef WIN32
+ #define SERVER_LOG_FILE2 SERVER_LOG_FILE
+ #else
+ #define SERVER_LOG_FILE2 UTILITY_LOG_FILE
+ #endif
+
#ifndef WIN32
#define pg_copy_file copy_file
#define pg_mv_file rename
*************** typedef enum
*** 166,176 ****
*/
typedef enum
{
! PG_INFO,
PG_REPORT,
PG_WARNING,
! PG_FATAL,
! PG_DEBUG
} eLogType;
--- 194,203 ----
*/
typedef enum
{
! PG_VERBOSE,
PG_REPORT,
PG_WARNING,
! PG_FATAL
} eLogType;
*************** typedef struct
*** 204,228 ****
*/
typedef struct
{
! char *filename; /* name of log file (may be /dev/null) */
! /*
! * WIN32 files do not accept writes from multiple processes
! *
! * On Win32, we can't send both pg_upgrade output and command output to the
! * same file because we get the error: "The process cannot access the file
! * because it is being used by another process." so we have to send all
! * other output to 'nul'. Therefore, we set this to DEVNULL on Win32, and
! * it equals 'filename' on all other platforms.
! *
! * We could use the Windows pgwin32_open() flags to allow shared file
! * writes but is unclear how all other tools would use those flags, so
! * we just avoid it and log a little less on Windows.
! */
! char *filename2;
! FILE *fd; /* log FILE */
! bool debug; /* TRUE -> log more information */
! FILE *debug_fd; /* debug-level log FILE */
bool verbose; /* TRUE -> be verbose in messages */
} LogOpts;
--- 231,239 ----
*/
typedef struct
{
! FILE *internal; /* internal log FILE */
bool verbose; /* TRUE -> be verbose in messages */
+ bool retain; /* retain log files on success */
} LogOpts;
*************** typedef struct
*** 245,251 ****
const char *progname; /* complete pathname for this program */
char *exec_path; /* full path to my executable */
char *user; /* username for clusters */
- char cwd[MAXPGPATH]; /* current working directory, used for output */
char **tablespaces; /* tablespaces */
int num_tablespaces;
char **libraries; /* loadable libraries */
--- 256,261 ----
*************** void split_old_dump(void);
*** 294,301 ****
/* exec.c */
! int exec_prog(bool throw_error, const char *cmd, ...)
! __attribute__((format(PG_PRINTF_ATTRIBUTE, 2, 3)));
void verify_directories(void);
bool is_server_running(const char *datadir);
--- 304,312 ----
/* exec.c */
! int exec_prog(bool throw_error, bool is_priv,
! const char *log_file, const char *cmd, ...)
! __attribute__((format(PG_PRINTF_ATTRIBUTE, 4, 5)));
void verify_directories(void);
bool is_server_running(const char *datadir);
*************** const char *linkAndUpdateFile(pageCnvCtx
*** 339,344 ****
--- 350,356 ----
const char *dst);
void check_hard_link(void);
+ FILE *fopen_priv(const char *path, const char *mode);
/* function.c */
diff --git a/contrib/pg_upgrade/relfilenode.c b/contrib/pg_upgrade/relfilenode.c
new file mode 100644
index a1e30b1..45d6c54
*** a/contrib/pg_upgrade/relfilenode.c
--- b/contrib/pg_upgrade/relfilenode.c
*************** transfer_relfile(pageCnvCtx *pageConvert
*** 267,273 ****
if (user_opts.transfer_mode == TRANSFER_MODE_COPY)
{
! pg_log(PG_INFO, "copying \"%s\" to \"%s\"\n", old_file, new_file);
if ((msg = copyAndUpdateFile(pageConverter, old_file, new_file, true)) != NULL)
pg_log(PG_FATAL, "error while copying relation \"%s.%s\" (\"%s\" to \"%s\"): %s\n",
--- 267,273 ----
if (user_opts.transfer_mode == TRANSFER_MODE_COPY)
{
! pg_log(PG_VERBOSE, "copying \"%s\" to \"%s\"\n", old_file, new_file);
if ((msg = copyAndUpdateFile(pageConverter, old_file, new_file, true)) != NULL)
pg_log(PG_FATAL, "error while copying relation \"%s.%s\" (\"%s\" to \"%s\"): %s\n",
*************** transfer_relfile(pageCnvCtx *pageConvert
*** 275,281 ****
}
else
{
! pg_log(PG_INFO, "linking \"%s\" to \"%s\"\n", old_file, new_file);
if ((msg = linkAndUpdateFile(pageConverter, old_file, new_file)) != NULL)
pg_log(PG_FATAL,
--- 275,281 ----
}
else
{
! pg_log(PG_VERBOSE, "linking \"%s\" to \"%s\"\n", old_file, new_file);
if ((msg = linkAndUpdateFile(pageConverter, old_file, new_file)) != NULL)
pg_log(PG_FATAL,
diff --git a/contrib/pg_upgrade/server.c b/contrib/pg_upgrade/server.c
new file mode 100644
index 989af63..b515e05
*** a/contrib/pg_upgrade/server.c
--- b/contrib/pg_upgrade/server.c
*************** executeQueryOrDie(PGconn *conn, const ch
*** 80,86 ****
vsnprintf(command, sizeof(command), fmt, args);
va_end(args);
! pg_log(PG_DEBUG, "executing: %s\n", command);
result = PQexec(conn, command);
status = PQresultStatus(result);
--- 80,86 ----
vsnprintf(command, sizeof(command), fmt, args);
va_end(args);
! pg_log(PG_VERBOSE, "executing: %s\n", command);
result = PQexec(conn, command);
status = PQresultStatus(result);
*************** start_postmaster(ClusterInfo *cluster)
*** 161,177 ****
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" "
"-o \"-p %d %s %s\" start >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, log_opts.filename2, cluster->pgconfig, cluster->port,
(cluster->controldata.cat_ver >=
BINARY_UPGRADE_SERVER_FLAG_CAT_VER) ? "-b" :
"-c autovacuum=off -c autovacuum_freeze_max_age=2000000000",
! cluster->pgopts ? cluster->pgopts : "", log_opts.filename2);
/*
* Don't throw an error right away, let connecting throw the error because
* it might supply a reason for the failure.
*/
! pg_ctl_return = exec_prog(false, "%s", cmd);
/* Check to see if we can connect to the server; if not, report it. */
if ((conn = get_db_conn(cluster, "template1")) == NULL ||
--- 161,182 ----
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" "
"-o \"-p %d %s %s\" start >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, SERVER_LOG_FILE, cluster->pgconfig, cluster->port,
(cluster->controldata.cat_ver >=
BINARY_UPGRADE_SERVER_FLAG_CAT_VER) ? "-b" :
"-c autovacuum=off -c autovacuum_freeze_max_age=2000000000",
! cluster->pgopts ? cluster->pgopts : "", SERVER_LOG_FILE2);
/*
* Don't throw an error right away, let connecting throw the error because
* it might supply a reason for the failure.
*/
! pg_ctl_return = exec_prog(false, true,
! /* pass both file names if the differ */
! (strcmp(SERVER_LOG_FILE, SERVER_LOG_FILE2) == 0) ?
! SERVER_LOG_FILE :
! SERVER_LOG_FILE " or " SERVER_LOG_FILE2,
! "%s", cmd);
/* Check to see if we can connect to the server; if not, report it. */
if ((conn = get_db_conn(cluster, "template1")) == NULL ||
*************** stop_postmaster(bool fast)
*** 211,221 ****
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" -o \"%s\" "
"%s stop >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, log_opts.filename2, cluster->pgconfig,
cluster->pgopts ? cluster->pgopts : "",
! fast ? "-m fast" : "", log_opts.filename2);
! exec_prog(fast ? false : true, "%s", cmd);
os_info.running_cluster = NULL;
}
--- 216,226 ----
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" -o \"%s\" "
"%s stop >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, SERVER_LOG_FILE2, cluster->pgconfig,
cluster->pgopts ? cluster->pgopts : "",
! fast ? "-m fast" : "", SERVER_LOG_FILE2);
! exec_prog(fast ? false : true, true, SERVER_LOG_FILE2, "%s", cmd);
os_info.running_cluster = NULL;
}
diff --git a/contrib/pg_upgrade/util.c b/contrib/pg_upgrade/util.c
new file mode 100644
index 94eaa18..6977663
*** a/contrib/pg_upgrade/util.c
--- b/contrib/pg_upgrade/util.c
*************** pg_log(eLogType type, char *fmt,...)
*** 77,94 ****
vsnprintf(message, sizeof(message), fmt, args);
va_end(args);
! if (log_opts.fd != NULL)
{
! fwrite(message, strlen(message), 1, log_opts.fd);
/* if we are using OVERWRITE_MESSAGE, add newline */
if (strchr(message, '\r') != NULL)
! fwrite("\n", 1, 1, log_opts.fd);
! fflush(log_opts.fd);
}
switch (type)
{
! case PG_INFO:
if (log_opts.verbose)
printf("%s", _(message));
break;
--- 77,95 ----
vsnprintf(message, sizeof(message), fmt, args);
va_end(args);
! /* PG_VERBOSE is only output in verbose mode */
! if (type != PG_VERBOSE || log_opts.verbose)
{
! fwrite(message, strlen(message), 1, log_opts.internal);
/* if we are using OVERWRITE_MESSAGE, add newline */
if (strchr(message, '\r') != NULL)
! fwrite("\n", 1, 1, log_opts.internal);
! fflush(log_opts.internal);
}
switch (type)
{
! case PG_VERBOSE:
if (log_opts.verbose)
printf("%s", _(message));
break;
*************** pg_log(eLogType type, char *fmt,...)
*** 104,114 ****
exit(1);
break;
- case PG_DEBUG:
- if (log_opts.debug)
- fprintf(log_opts.debug_fd, "%s\n", _(message));
- break;
-
default:
break;
}
--- 105,110 ----
diff --git a/contrib/pg_upgrade/version.c b/contrib/pg_upgrade/version.c
new file mode 100644
index e8799a4..5d790a0
*** a/contrib/pg_upgrade/version.c
--- b/contrib/pg_upgrade/version.c
*************** new_9_0_populate_pg_largeobject_metadata
*** 28,35 ****
prep_status("Checking for large objects");
! snprintf(output_path, sizeof(output_path), "%s/pg_largeobject.sql",
! os_info.cwd);
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
--- 28,34 ----
prep_status("Checking for large objects");
! snprintf(output_path, sizeof(output_path), "pg_largeobject.sql");
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
*************** new_9_0_populate_pg_largeobject_metadata
*** 49,55 ****
found = true;
if (!check_mode)
{
! if (script == NULL && (script = fopen(output_path, "w")) == NULL)
pg_log(PG_FATAL, "could not open file \"%s\": %s\n", output_path, getErrorText(errno));
fprintf(script, "\\connect %s\n",
quote_identifier(active_db->db_name));
--- 48,54 ----
found = true;
if (!check_mode)
{
! if (script == NULL && (script = fopen_priv(output_path, "w")) == NULL)
pg_log(PG_FATAL, "could not open file \"%s\": %s\n", output_path, getErrorText(errno));
fprintf(script, "\\connect %s\n",
quote_identifier(active_db->db_name));
diff --git a/contrib/pg_upgrade/version_old_8_3.c b/contrib/pg_upgrade/version_old_8_3.c
new file mode 100644
index a864107..c60374e
*** a/contrib/pg_upgrade/version_old_8_3.c
--- b/contrib/pg_upgrade/version_old_8_3.c
*************** old_8_3_check_for_name_data_type_usage(C
*** 30,37 ****
prep_status("Checking for invalid \"name\" user columns");
! snprintf(output_path, sizeof(output_path), "%s/tables_using_name.txt",
! os_info.cwd);
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
--- 30,36 ----
prep_status("Checking for invalid \"name\" user columns");
! snprintf(output_path, sizeof(output_path), "tables_using_name.txt");
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
*************** old_8_3_check_for_name_data_type_usage(C
*** 73,79 ****
for (rowno = 0; rowno < ntups; rowno++)
{
found = true;
! if (script == NULL && (script = fopen(output_path, "w")) == NULL)
pg_log(PG_FATAL, "could not open file \"%s\": %s\n", output_path, getErrorText(errno));
if (!db_used)
{
--- 72,78 ----
for (rowno = 0; rowno < ntups; rowno++)
{
found = true;
! if (script == NULL && (script = fopen_priv(output_path, "w")) == NULL)
pg_log(PG_FATAL, "could not open file \"%s\": %s\n", output_path, getErrorText(errno));
if (!db_used)
{
*************** old_8_3_check_for_tsquery_usage(ClusterI
*** 126,133 ****
prep_status("Checking for tsquery user columns");
! snprintf(output_path, sizeof(output_path), "%s/tables_using_tsquery.txt",
! os_info.cwd);
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
--- 125,131 ----
prep_status("Checking for tsquery user columns");
! snprintf(output_path, sizeof(output_path), "tables_using_tsquery.txt");
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
*************** old_8_3_check_for_tsquery_usage(ClusterI
*** 164,170 ****
for (rowno = 0; rowno < ntups; rowno++)
{
found = true;
! if (script == NULL && (script = fopen(output_path, "w")) == NULL)
pg_log(PG_FATAL, "could not open file \"%s\": %s\n", output_path, getErrorText(errno));
if (!db_used)
{
--- 162,168 ----
for (rowno = 0; rowno < ntups; rowno++)
{
found = true;
! if (script == NULL && (script = fopen_priv(output_path, "w")) == NULL)
pg_log(PG_FATAL, "could not open file \"%s\": %s\n", output_path, getErrorText(errno));
if (!db_used)
{
*************** old_8_3_check_ltree_usage(ClusterInfo *c
*** 216,223 ****
prep_status("Checking for contrib/ltree");
! snprintf(output_path, sizeof(output_path), "%s/contrib_ltree.txt",
! os_info.cwd);
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
--- 214,220 ----
prep_status("Checking for contrib/ltree");
! snprintf(output_path, sizeof(output_path), "contrib_ltree.txt");
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
*************** old_8_3_check_ltree_usage(ClusterInfo *c
*** 244,250 ****
for (rowno = 0; rowno < ntups; rowno++)
{
found = true;
! if (script == NULL && (script = fopen(output_path, "w")) == NULL)
pg_log(PG_FATAL, "Could not open file \"%s\": %s\n",
output_path, getErrorText(errno));
if (!db_used)
--- 241,247 ----
for (rowno = 0; rowno < ntups; rowno++)
{
found = true;
! if (script == NULL && (script = fopen_priv(output_path, "w")) == NULL)
pg_log(PG_FATAL, "Could not open file \"%s\": %s\n",
output_path, getErrorText(errno));
if (!db_used)
*************** old_8_3_rebuild_tsvector_tables(ClusterI
*** 304,311 ****
prep_status("Checking for tsvector user columns");
! snprintf(output_path, sizeof(output_path), "%s/rebuild_tsvector_tables.sql",
! os_info.cwd);
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
--- 301,307 ----
prep_status("Checking for tsvector user columns");
! snprintf(output_path, sizeof(output_path), "rebuild_tsvector_tables.sql");
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
*************** old_8_3_rebuild_tsvector_tables(ClusterI
*** 364,370 ****
found = true;
if (!check_mode)
{
! if (script == NULL && (script = fopen(output_path, "w")) == NULL)
pg_log(PG_FATAL, "could not open file \"%s\": %s\n", output_path, getErrorText(errno));
if (!db_used)
{
--- 360,366 ----
found = true;
if (!check_mode)
{
! if (script == NULL && (script = fopen_priv(output_path, "w")) == NULL)
pg_log(PG_FATAL, "could not open file \"%s\": %s\n", output_path, getErrorText(errno));
if (!db_used)
{
*************** old_8_3_invalidate_hash_gin_indexes(Clus
*** 446,453 ****
prep_status("Checking for hash and GIN indexes");
! snprintf(output_path, sizeof(output_path), "%s/reindex_hash_and_gin.sql",
! os_info.cwd);
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
--- 442,448 ----
prep_status("Checking for hash and GIN indexes");
! snprintf(output_path, sizeof(output_path), "reindex_hash_and_gin.sql");
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
*************** old_8_3_invalidate_hash_gin_indexes(Clus
*** 481,487 ****
found = true;
if (!check_mode)
{
! if (script == NULL && (script = fopen(output_path, "w")) == NULL)
pg_log(PG_FATAL, "could not open file \"%s\": %s\n", output_path, getErrorText(errno));
if (!db_used)
{
--- 476,482 ----
found = true;
if (!check_mode)
{
! if (script == NULL && (script = fopen_priv(output_path, "w")) == NULL)
pg_log(PG_FATAL, "could not open file \"%s\": %s\n", output_path, getErrorText(errno));
if (!db_used)
{
*************** old_8_3_invalidate_bpchar_pattern_ops_in
*** 556,563 ****
prep_status("Checking for bpchar_pattern_ops indexes");
! snprintf(output_path, sizeof(output_path), "%s/reindex_bpchar_ops.sql",
! os_info.cwd);
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
--- 551,557 ----
prep_status("Checking for bpchar_pattern_ops indexes");
! snprintf(output_path, sizeof(output_path), "reindex_bpchar_ops.sql");
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
{
*************** old_8_3_invalidate_bpchar_pattern_ops_in
*** 601,607 ****
found = true;
if (!check_mode)
{
! if (script == NULL && (script = fopen(output_path, "w")) == NULL)
pg_log(PG_FATAL, "could not open file \"%s\": %s\n", output_path, getErrorText(errno));
if (!db_used)
{
--- 595,601 ----
found = true;
if (!check_mode)
{
! if (script == NULL && (script = fopen_priv(output_path, "w")) == NULL)
pg_log(PG_FATAL, "could not open file \"%s\": %s\n", output_path, getErrorText(errno));
if (!db_used)
{
*************** old_8_3_create_sequence_script(ClusterIn
*** 683,689 ****
bool found = false;
char *output_path = pg_malloc(MAXPGPATH);
! snprintf(output_path, MAXPGPATH, "%s/adjust_sequences.sql", os_info.cwd);
prep_status("Creating script to adjust sequences");
--- 677,683 ----
bool found = false;
char *output_path = pg_malloc(MAXPGPATH);
! snprintf(output_path, MAXPGPATH, "adjust_sequences.sql");
prep_status("Creating script to adjust sequences");
*************** old_8_3_create_sequence_script(ClusterIn
*** 723,729 ****
found = true;
! if (script == NULL && (script = fopen(output_path, "w")) == NULL)
pg_log(PG_FATAL, "could not open file \"%s\": %s\n", output_path, getErrorText(errno));
if (!db_used)
{
--- 717,723 ----
found = true;
! if (script == NULL && (script = fopen_priv(output_path, "w")) == NULL)
pg_log(PG_FATAL, "could not open file \"%s\": %s\n", output_path, getErrorText(errno));
if (!db_used)
{
diff --git a/doc/src/sgml/pgupgrade.sgml b/doc/src/sgml/pgupgrade.sgml
new file mode 100644
index 4f263fe..6ecdfda
*** a/doc/src/sgml/pgupgrade.sgml
--- b/doc/src/sgml/pgupgrade.sgml
***************
*** 91,120 ****
</varlistentry>
<varlistentry>
- <term><option>-g</option></term>
- <term><option>--debug</option></term>
- <listitem><para>enable debugging</para></listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-G</option> <replaceable>debug_filename</></term>
- <term><option>--debugfile=</option><replaceable>debug_filename</></term>
- <listitem><para>output debugging activity to file</para></listitem>
- </varlistentry>
-
- <varlistentry>
<term><option>-k</option></term>
<term><option>--link</option></term>
<listitem><para>use hard links instead of copying files to the new cluster</para></listitem>
</varlistentry>
<varlistentry>
- <term><option>-l</option> <replaceable>log_filename</></term>
- <term><option>--logfile=</option><replaceable>log_filename</></term>
- <listitem><para>log internal activity to file</para></listitem>
- </varlistentry>
-
- <varlistentry>
<term><option>-o</option> <replaceable class="parameter">options</replaceable></term>
<term><option>--old-options</option> <replaceable class="parameter">options</replaceable></term>
<listitem><para>options to be passed directly to the
--- 91,102 ----
***************
*** 143,148 ****
--- 125,137 ----
</varlistentry>
<varlistentry>
+ <term><option>-r</option></term>
+ <term><option>--retain</option></term>
+ <listitem><para>retain SQL and log files even after successful completion
+ </para></listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-u</option> <replaceable>user_name</></term>
<term><option>--user=</option><replaceable>user_name</></term>
<listitem><para>cluster's super user name; environment
***************
*** 152,158 ****
<varlistentry>
<term><option>-v</option></term>
<term><option>--verbose</option></term>
! <listitem><para>enable verbose output</para></listitem>
</varlistentry>
<varlistentry>
--- 141,147 ----
<varlistentry>
<term><option>-v</option></term>
<term><option>--verbose</option></term>
! <listitem><para>enable verbose internal logging</para></listitem>
</varlistentry>
<varlistentry>