[PATCH] Make pg_basebackup configure and start standby [Review]
On Sun, 01 Jul 2012 13:02:17 +0200 Boszormenyi Zoltan wrote:
attached is a patch that does $SUBJECT.
It's a usability enhancement, to take a backup, write
a minimalistic recovery.conf and start the streaming
standby in one go.
Comments?
[Review of Patch]
Basic stuff:
----------------------
- Patch applies OK
- Compiles cleanly with no warnings
What it does:
-------------------------
The pg_basebackup tool does the backup of Cluster from server to the
specified location.
This new functionality will also writes the recovery.conf in the database
directory and start the standby server based on options passed to
pg_basebackup.
Usability
----------------
For usability aspect, I am not very sure how many users would like to start
the standby server using basebackup.
According to me it can be useful for users who have automated scripts to
start server after backup can use this feature.
Feature Testing:
-----------------------------
1. Test pg_basebackup with option -R to check that the recovery.conf file is
written to data directory.
--recovery.conf file is created in data directory.
2. Test pg_basebackup with option -R to check that the recovery.conf file is
not able to create because of disk full.
--Error is given as recovery.conf file is not able to create.
3. Test pg_basebackup with option -S to check the standby server start on
the same/different machine.
--Starting standby server is success in if pg_basebackup is taken in
different machine.
4. Test pg_basebackup with both options -S and -R to check the standby
server start on same/different machine.
--Starting standby server is success in if pg_basebackup is taken in
different machine.
5. Test pg_basebackup with option -S including -h, -U, -p, -w and -W to
check the standy server start
and verify the recovery.conf which is created in data directory.
--Except password, rest of the primary connection info parameters are
working fine.
6. Test pg_basebackup with conflict options (-x or -X and -R or -S).
--Error is given when the conflict options are provided to
pg_basebackup.
7. Test pg_basebackup with option -S where pg_ctl/postmaster binaries are
not present in the path.
--Error is given as not able to execute.
8. Test pg_basebackup with option -S by connecting to a standby server.
--standby server is started successfully when pg_basebackup is made from
a standby server also.
Code Review:
----------------------------
1. In function WriteRecoveryConf(), un-initialized filename is used.
due to which it can print junk for below line in code
printf("add password to primary_conninfo in %s if needed\n", filename);
2. In function WriteRecoveryConf(), in below code if fopen fails (due to
disk full or any other file related error) it will print the error and
exits.
So now it can be confusing to user, in respect to can he consider
backup as successfull and proceed. IMO, either error meesage or
documentation
can suggest the for such error user can proceed with backup to write
his own recovery.conf and start the standby.
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("cannot create %s"), filename);
+ exit(1);
+ }
3. In function main,
instead of the following code it can be changed in two different ways,
if (startstandby)
writerecoveryconf = true;
change1:
case 'R':
writerecoveryconf = true;
break;
case 'S':
startstandby = true;
writerecoveryconf = true;
break;
change2:
case 'S':
startstandby = true;
case 'R':
writerecoveryconf = true;
break;
4. The password is not written to primary_conninfo even if the dbpassword is
present because of this reason
connecting to the primary is failing because of authentication failure.
5. write the function header for the newly added functions.
6. execvp function is deprecated beginning in Visual C++ 2005. which is used
to fork the pg_ctl process.
http://msdn.microsoft.com/en-us/library/ms235414.aspx
7. In StartStandby function, it is better to free the memory allocated for
path (path = xstrdup(command);)
Defects:
-------------
1. If the pg_basebackup is used in the same machine with the option of -S,
the standby server start
will fail as the port already in use because of using the same
postgresql.conf.
2. If the hot_standby=off in master conf file, the same is copied to
subscriber and starts the server. with that
no client connections are allowed to the server.
Documentation issues:
--------------------------------
1. For -R option,
Conflicts with <option>--xlog
I think it is better to explain the reason of conflict.
2. For -S option,
"Start the standby database server. Implies -R option."
I think the above can be improved to
"Writes the recovery.conf and start the standby database server. There
is no need for user to specify -R option explicitly."
or something similar.
With Regards,
Amit Kapila.
Hi,
first, thanks for the review. Comments are below.
2012-09-20 12:30 keltezéssel, Amit Kapila írta:
On Sun, 01 Jul 2012 13:02:17 +0200 Boszormenyi Zoltan wrote:
attached is a patch that does $SUBJECT.
It's a usability enhancement, to take a backup, write
a minimalistic recovery.conf and start the streaming
standby in one go.
Comments?
*[Review of Patch]*
*Basic stuff:*
----------------------
- Patch applies OK
This is not true anymore with a newer GIT version.
Some chunk for pg_basebackup.c was rejected.
- Compiles cleanly with no warnings
*What it does:*
-------------------------
The pg_basebackup tool does the backup of Cluster from server to the specified location.
This new functionality will also writes the recovery.conf in the database directory and
start the standby server based on options passed to pg_basebackup.*Usability*
*----------------*
For usability aspect, I am not very sure how many users would like to start the standby
server using basebackup.
Also, Magnus raised the point that it wouldn't really work on MS Windows
where you *have to* start the service via OS facilities. This part of the patch
was killed.
According to me it can be useful for users who have automated scripts to start server
after backup can use this feature.
Well, scripting is not portable across UNIXes and Windows,
you have to spell out starting the server differently.
*Feature Testing:*
-----------------------------1. Test pg_basebackup with option -R to check that the recovery.conf file is written to
data directory.
--recovery.conf file is created in data directory.2. Test pg_basebackup with option -R to check that the recovery.conf file is not able to
create because of disk full.
--Error is given as recovery.conf file is not able to create.3. Test pg_basebackup with option -S to check the standby server start on the
same/different machine.
--Starting standby server is success in if pg_basebackup is taken in different machine.4. Test pg_basebackup with both options -S and -R to check the standby server start on
same/different machine.
--Starting standby server is success in if pg_basebackup is taken in different machine.5. Test pg_basebackup with option -S including -h, -U, -p, -w and -W to check the standy
server start
and verify the recovery.conf which is created in data directory.
--Except password, rest of the primary connection info parameters are working fine.
The password part is now fixed.
6. Test pg_basebackup with conflict options (-x or -X and -R or -S).
--Error is given when the conflict options are provided to pg_basebackup.7. Test pg_basebackup with option -S where pg_ctl/postmaster binaries are not present in
the path.
--Error is given as not able to execute.8. Test pg_basebackup with option -S by connecting to a standby server.
--standby server is started successfully when pg_basebackup is made from a standby
server also.*Code Review:*
----------------------------
1. In function WriteRecoveryConf(), un-initialized filename is used.
due to which it can print junk for below line in code
printf("add password to primary_conninfo in %s if needed\n", filename);
Fixed.
2. In function WriteRecoveryConf(), in below code if fopen fails (due to disk full or
any other file related error) it will print the error and exits.
So now it can be confusing to user, in respect to can he consider backup as
successfull and proceed. IMO, either error meesage or documentation
can suggest the for such error user can proceed with backup to write his own
recovery.conf and start the standby.+ cf = fopen(filename, "w"); + if (cf == NULL) + { + fprintf(stderr, _("cannot create %s"), filename); + exit(1); + }
But BaseBackup() function did indicate already that it has
finished successfully with
if (verbose)
fprintf(stderr, "%s: base backup completed\n", progname);
Would it be an expected (as in: not confusing) behaviour to return 0
from pg_basebackup if the backup itself has finished, but failed to write
the recovery.conf or start the standby if those were requested?
I have modified my WriteRecoveryConf() to do exit(2) instead of exit(1)
to indicate a different error. exit(1) seems to be for reporting configuration
or connection errors. (I may be mistaken though.)
3. In function main,
instead of the following code it can be changed in two different ways,if (startstandby)
writerecoveryconf = true;change1:
case 'R':
writerecoveryconf = true;
break;
case 'S':
startstandby = true;
writerecoveryconf = true;
break;change2:
case 'S':
startstandby = true;
case 'R':
writerecoveryconf = true;
break;
I went with your second variant at first but it's not needed anymore
as only "-R" exists.
4. The password is not written to primary_conninfo even if the dbpassword is present
because of this reason
connecting to the primary is failing because of authentication failure.
Fixed.
5. write the function header for the newly added functions.
Fixed.
6. execvp function is deprecated beginning in Visual C++ 2005. which is used to fork the
pg_ctl process.
http://msdn.microsoft.com/en-us/library/ms235414.aspx
This issue is now irrelevant as the standby is not started, there is no "-S" option.
7. In StartStandby function, it is better to free the memory allocated for path (path =
xstrdup(command);)
Same as above.
*Defects:*
*-------------*
1. If the pg_basebackup is used in the same machine with the option of -S, the standby
server start
will fail as the port already in use because of using the same postgresql.conf.
Well, running initdb twice on the same machine with different data directories
would also cause the second server fail to start because of the same issue
and it's not called a bug. I think this is irrelevant as is and also because there
is no "-S" now.
2. If the hot_standby=off in master conf file, the same is copied to subscriber and
starts the server. with that
no client connections are allowed to the server.
Well, it simply copies the source server behaviour, which can also be a
replication standby. PostgreSQL has cascading replication, you know.
*Documentation issues:*
*--------------------------------*
1. For -R option,
Conflicts with <option>--xlog
I think it is better to explain the reason of conflict.
Fixed.
2. For -S option,
"Start the standby database server. Implies -R option."
I think the above can be improved to
"Writes the recovery.conf and start the standby database server. There is no need for
user to specify -R option explicitly."
or something similar.
Not relevant anymore.
Again, thanks for the review.
The second generation of this work is now attached and contains a new
feature as was discussed and suggested by Magnus Hagander, Fujii Masao
and Peter Eisentraut. So libpq has grown a new function:
+/* return the connection options used by a live connection */
+extern PQconninfoOption *PQconninfo(PGconn *conn);
This copies all the connection parameters back from the live PGconn itself
so everything that's needed to connect is already validated.
This is used by the second patch which makes the changes in pg_basebackup
simpler and not hardcoded.
Please, review.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
02-pg_basebackup-v2.patchtext/x-patch; name=02-pg_basebackup-v2.patchDownload
diff -durpN postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml
--- postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml 2012-08-24 09:49:22.960530329 +0200
+++ postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml 2012-10-03 18:08:44.940909214 +0200
@@ -189,6 +189,27 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--write-recovery-conf</option></term>
+ <listitem>
+ <para>
+ Write a minimal recovery.conf into the output directory using
+ the connection parameters from the command line to ease
+ setting up the standby. Since creating a backup for a standalone
+ server with <option>-x</option> or <option>-X</option> and adding
+ a recovery.conf to it wouldn't make a working standby, these options
+ naturally conflict.
+ </para>
+ <para>
+ When this option is specified and taking the base backup succeeded,
+ failing to write the <filename>recovery.conf</filename> results
+ in the error code 2. This way, scripts can distinguish between different
+ failure cases of <application>pg_basebackup</application>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-x</option></term>
<term><option>--xlog</option></term>
<listitem>
diff -durpN postgresql.1/src/bin/pg_basebackup/pg_basebackup.c postgresql.2/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.1/src/bin/pg_basebackup/pg_basebackup.c 2012-10-03 10:40:48.297207389 +0200
+++ postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2012-10-03 17:42:41.530270627 +0200
@@ -46,6 +46,7 @@ int compresslevel = 0;
bool includewal = false;
bool streamwal = false;
bool fastcheckpoint = false;
+bool writerecoveryconf = false;
int standby_message_timeout = 10 * 1000; /* 10 sec = default */
/* Progress counters */
@@ -70,14 +71,18 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
+static PQconninfoOption *connOptions = NULL;
+
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
static void progress_report(int tablespacenum, const char *filename);
+static void stderr_write_error(FILE *cf, char *filename);
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
static void BaseBackup(void);
+static void WriteRecoveryConf(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
bool segment_finished);
@@ -107,6 +112,8 @@ usage(void)
printf(_("\nOptions controlling the output:\n"));
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
+ printf(_(" -R, --write-recovery-conf\n"
+ " write recovery.conf after backup\n"));
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
printf(_(" -X, --xlog-method=fetch|stream\n"
" include required WAL files with specified method\n"));
@@ -960,6 +967,10 @@ BaseBackup(void)
/* Error message already written in GetConnection() */
exit(1);
+ /* If recovery.conf is to be written, keep the connection parameters for later usage */
+ if (writerecoveryconf)
+ connOptions = PQconninfo(conn);
+
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
@@ -1234,6 +1245,67 @@ BaseBackup(void)
}
+static void
+stderr_write_error(FILE *cf, char *filename)
+{
+ fprintf(stderr, _("cannot write to %s: %s"), filename, strerror(errno));
+ fclose(cf);
+ unlink(filename);
+ exit(2);
+}
+
+
+/*
+ * Attempt to create recovery.conf and write the expected contents to it.
+ */
+static void
+WriteRecoveryConf(void)
+{
+ char filename[MAXPGPATH];
+ PQconninfoOption *option;
+ FILE *cf;
+
+ if (!writerecoveryconf)
+ return;
+
+ sprintf(filename, "%s/recovery.conf", basedir);
+
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("cannot create %s: %s"), filename, strerror(errno));
+ exit(2);
+ }
+
+ if (fprintf(cf, "standby_mode = 'on'\n") < 0)
+ stderr_write_error(cf, filename);
+
+ if (fprintf(cf, "primary_conninfo = '") < 0)
+ stderr_write_error(cf, filename);
+
+ for (option = connOptions; option && option->keyword; option++)
+ {
+ /* Do not emit this setting if not set, empty or default */
+ if (option->val == NULL ||
+ (option->val != NULL && option->val[0] == '\0') ||
+ (option->val &&
+ option->compiled &&
+ strcmp(option->val, option->compiled) == 0))
+ continue;
+
+ /* write "keyword='value'" pieces, single quotes doubled */
+ if (fprintf(cf, "%s=''%s'' ", option->keyword, option->val) < 0)
+ stderr_write_error(cf, filename);
+ }
+
+ if (fprintf(cf, "'\n") < 0)
+ stderr_write_error(cf, filename);
+
+ PQconninfoFree(connOptions);
+
+ fclose(cf);
+}
+
int
main(int argc, char **argv)
{
@@ -1243,6 +1315,7 @@ main(int argc, char **argv)
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
+ {"write-recovery-conf", no_argument, NULL, 'R'},
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
{"gzip", no_argument, NULL, 'z'},
@@ -1280,7 +1353,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:xX:l:zZ:c:h:p:U:s:wWvP",
+ while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
long_options, &option_index)) != -1)
{
switch (c)
@@ -1301,6 +1374,9 @@ main(int argc, char **argv)
exit(1);
}
break;
+ case 'R':
+ writerecoveryconf = true;
+ break;
case 'x':
if (includewal)
{
@@ -1466,6 +1542,13 @@ main(int argc, char **argv)
}
#endif
+ if (writerecoveryconf && includewal)
+ {
+ fprintf(stderr,
+ _("--xlog and --writerecoveryconf are mutually exclusive\n"));
+ exit(1);
+ }
+
/*
* Verify that the target directory exists, or create it. For plaintext
* backups, always require the directory. For tar backups, require it
@@ -1476,5 +1559,7 @@ main(int argc, char **argv)
BaseBackup();
+ WriteRecoveryConf();
+
return 0;
}
01-PQconninfo-v3.patchtext/x-patch; name=01-PQconninfo-v3.patchDownload
diff -durpN postgresql/src/interfaces/libpq/exports.txt postgresql.1/src/interfaces/libpq/exports.txt
--- postgresql/src/interfaces/libpq/exports.txt 2012-08-03 09:39:30.118266598 +0200
+++ postgresql.1/src/interfaces/libpq/exports.txt 2012-10-03 17:17:06.344766325 +0200
@@ -161,3 +161,4 @@ PQping 158
PQpingParams 159
PQlibVersion 160
PQsetSingleRowMode 161
+PQconninfo 162
diff -durpN postgresql/src/interfaces/libpq/fe-connect.c postgresql.1/src/interfaces/libpq/fe-connect.c
--- postgresql/src/interfaces/libpq/fe-connect.c 2012-09-09 08:11:09.470401480 +0200
+++ postgresql.1/src/interfaces/libpq/fe-connect.c 2012-10-03 15:46:44.357491440 +0200
@@ -318,6 +318,8 @@ static char *conninfo_uri_decode(const c
static bool get_hexdigit(char digit, int *value);
static const char *conninfo_getval(PQconninfoOption *connOptions,
const char *keyword);
+static void conninfo_setval(PQconninfoOption *connOptions,
+ const char *keyword, const char *val);
static PQconninfoOption *conninfo_storeval(PQconninfoOption *connOptions,
const char *keyword, const char *value,
PQExpBuffer errorMessage, bool ignoreMissing, bool uri_decode);
@@ -4985,6 +4987,24 @@ conninfo_getval(PQconninfoOption *connOp
}
/*
+ * Set an option value corresponding to the keyword in the connOptions array.
+ */
+static void
+conninfo_setval(PQconninfoOption *connOptions, const char *keyword,
+ const char *val)
+{
+ PQconninfoOption *option;
+
+ option = conninfo_find(connOptions, keyword);
+ if (option)
+ {
+ if (option->val)
+ free(option->val);
+ option->val = val ? strdup(val) : NULL;
+ }
+}
+
+/*
* Store a (new) value for an option corresponding to the keyword in
* connOptions array.
*
@@ -5066,6 +5086,69 @@ conninfo_find(PQconninfoOption *connOpti
}
+/*
+ * Return the connection options used for the connections
+ */
+PQconninfoOption *
+PQconninfo(PGconn *conn)
+{
+ PQExpBufferData errorBuf;
+ PQconninfoOption *connOptions;
+
+ if (conn == NULL)
+ return NULL;
+
+ /* We don't actually report any errors here, but callees want a buffer */
+ initPQExpBuffer(&errorBuf);
+ if (PQExpBufferDataBroken(errorBuf))
+ return NULL; /* out of memory already :-( */
+
+ connOptions = conninfo_init(&errorBuf);
+
+ termPQExpBuffer(&errorBuf);
+
+ /*
+ * Move conn values into option structure
+ */
+ conninfo_setval(connOptions, "hostaddr", conn->pghostaddr);
+ conninfo_setval(connOptions, "host", conn->pghost);
+ conninfo_setval(connOptions, "port", conn->pgport);
+ conninfo_setval(connOptions, "tty", conn->pgtty);
+ conninfo_setval(connOptions, "options", conn->pgoptions);
+ conninfo_setval(connOptions, "application_name", conn->appname);
+ conninfo_setval(connOptions, "fallback_application_name", conn->fbappname);
+ conninfo_setval(connOptions, "dbname", conn->dbName);
+ conninfo_setval(connOptions, "user", conn->pguser);
+ conninfo_setval(connOptions, "password", conn->pgpass);
+ conninfo_setval(connOptions, "connect_timeout", conn->connect_timeout);
+ conninfo_setval(connOptions, "client_encoding", conn->client_encoding_initial);
+ conninfo_setval(connOptions, "keepalives", conn->keepalives);
+ conninfo_setval(connOptions, "keepalives_idle", conn->keepalives_idle);
+ conninfo_setval(connOptions, "keepalives_interval", conn->keepalives_interval);
+ conninfo_setval(connOptions, "keepalives_count", conn->keepalives_count);
+ conninfo_setval(connOptions, "sslmode", conn->sslmode);
+ conninfo_setval(connOptions, "sslcompression", conn->sslcompression);
+ conninfo_setval(connOptions, "sslkey", conn->sslkey);
+ conninfo_setval(connOptions, "sslcert", conn->sslcert);
+ conninfo_setval(connOptions, "sslrootcert", conn->sslrootcert);
+ conninfo_setval(connOptions, "sslcrl", conn->sslcrl);
+#ifdef USE_SSL
+ conninfo_setval(connOptions, "requiressl",
+ (conn->sslmode && strcmp(conn->sslmode, "require") == 0 ? "1" : "0"));
+#endif
+ conninfo_setval(connOptions, "requirepeer", conn->requirepeer);
+#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
+ conninfo_setval(connOptions, "krbsrvname", conn->krbsrvname);
+#endif
+#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
+ conninfo_setval(connOptions, "gsslib", conn->gsslib);
+#endif
+ conninfo_setval(connOptions, "replication", conn->replication);
+
+ return connOptions;
+}
+
+
void
PQconninfoFree(PQconninfoOption *connOptions)
{
diff -durpN postgresql/src/interfaces/libpq/libpq-fe.h postgresql.1/src/interfaces/libpq/libpq-fe.h
--- postgresql/src/interfaces/libpq/libpq-fe.h 2012-08-03 09:39:30.122266626 +0200
+++ postgresql.1/src/interfaces/libpq/libpq-fe.h 2012-10-03 15:00:45.458355068 +0200
@@ -262,6 +262,9 @@ extern PQconninfoOption *PQconndefaults(
/* parse connection options in same way as PQconnectdb */
extern PQconninfoOption *PQconninfoParse(const char *conninfo, char **errmsg);
+/* return the connection options used by a live connection */
+extern PQconninfoOption *PQconninfo(PGconn *conn);
+
/* free the data structure returned by PQconndefaults() or PQconninfoParse() */
extern void PQconninfoFree(PQconninfoOption *connOptions);
On Wed, 2012-10-03 at 18:15 +0200, Boszormenyi Zoltan wrote:
The second generation of this work is now attached and contains a new
feature as was discussed and suggested by Magnus Hagander, Fujii Masao
and Peter Eisentraut. So libpq has grown a new function:+/* return the connection options used by a live connection */ +extern PQconninfoOption *PQconninfo(PGconn *conn);This copies all the connection parameters back from the live PGconn
itself
so everything that's needed to connect is already validated.
I don't like that this code maintains a second list of all possible
libpq connection parameters. The parameters to add to the connection
string should be driven off the authoritative list in PQconninfoOptions.
2012-10-04 05:24 keltezéssel, Peter Eisentraut írta:
On Wed, 2012-10-03 at 18:15 +0200, Boszormenyi Zoltan wrote:
The second generation of this work is now attached and contains a new
feature as was discussed and suggested by Magnus Hagander, Fujii Masao
and Peter Eisentraut. So libpq has grown a new function:+/* return the connection options used by a live connection */ +extern PQconninfoOption *PQconninfo(PGconn *conn);This copies all the connection parameters back from the live PGconn
itself
so everything that's needed to connect is already validated.I don't like that this code maintains a second list of all possible
libpq connection parameters.
Where does it do that? In PQconninfo() itself? Why is it a problem?
Or to put it bluntly: the same problem is in fillPGconn() too, as it also
has the same set of parameters listed. So there is already code
that you don't like. :-)
How about a static mapping between option names and
offsetof(struct pg_conn, member) values? This way both fillPGconn()
and PQconninfo() can avoid maintaining the list of parameter names.
The parameters to add to the connection
string should be driven off the authoritative list in PQconninfoOptions.
So, should I add a second flag to PQconninfoOption to indicate that
certain options should not be used for primary_conninfo?
Thanks and best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
2012-10-04 06:47 keltezéssel, Boszormenyi Zoltan írta:
2012-10-04 05:24 keltezéssel, Peter Eisentraut írta:
On Wed, 2012-10-03 at 18:15 +0200, Boszormenyi Zoltan wrote:
The second generation of this work is now attached and contains a new
feature as was discussed and suggested by Magnus Hagander, Fujii Masao
and Peter Eisentraut. So libpq has grown a new function:+/* return the connection options used by a live connection */ +extern PQconninfoOption *PQconninfo(PGconn *conn);This copies all the connection parameters back from the live PGconn
itself
so everything that's needed to connect is already validated.I don't like that this code maintains a second list of all possible
libpq connection parameters.Where does it do that? In PQconninfo() itself? Why is it a problem?
Or to put it bluntly: the same problem is in fillPGconn() too, as it also
has the same set of parameters listed. So there is already code
that you don't like. :-)How about a static mapping between option names and
offsetof(struct pg_conn, member) values? This way both fillPGconn()
and PQconninfo() can avoid maintaining the list of parameter names.
Did you think about something like the attached code?
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
01-PQconninfo-v7.patchtext/x-patch; name=01-PQconninfo-v7.patchDownload
diff -durpN postgresql/src/interfaces/libpq/exports.txt postgresql.1/src/interfaces/libpq/exports.txt
--- postgresql/src/interfaces/libpq/exports.txt 2012-08-03 09:39:30.118266598 +0200
+++ postgresql.1/src/interfaces/libpq/exports.txt 2012-10-04 11:29:24.864309200 +0200
@@ -161,3 +161,5 @@ PQping 158
PQpingParams 159
PQlibVersion 160
PQsetSingleRowMode 161
+PQconninfo 162
+PQconninfoForReplication 163
diff -durpN postgresql/src/interfaces/libpq/fe-connect.c postgresql.1/src/interfaces/libpq/fe-connect.c
--- postgresql/src/interfaces/libpq/fe-connect.c 2012-09-09 08:11:09.470401480 +0200
+++ postgresql.1/src/interfaces/libpq/fe-connect.c 2012-10-04 12:30:56.705605588 +0200
@@ -131,12 +131,17 @@ static int ldapServiceLookup(const char
* "" Normal input field
* "*" Password field - hide value
* "D" Debug option - don't show by default
+ * A special "usable for replication connections" ("R") flag is
+ * added to Disp-Char in a backwards compatible fashion.
*
* PQconninfoOptions[] is a constant static array that we use to initialize
* a dynamically allocated working copy. All the "val" fields in
* PQconninfoOptions[] *must* be NULL. In a working copy, non-null "val"
* fields point to malloc'd strings that should be freed when the working
* array is freed (see PQconninfoFree).
+ *
+ * If you add a new connection option to this list, remember to add it to
+ * PQconninfoMappings[] below.
* ----------
*/
static const PQconninfoOption PQconninfoOptions[] = {
@@ -146,62 +151,62 @@ static const PQconninfoOption PQconninfo
* still try to set it.
*/
{"authtype", "PGAUTHTYPE", DefaultAuthtype, NULL,
- "Database-Authtype", "D", 20},
+ "Database-Authtype", "D\0", 20},
{"service", "PGSERVICE", NULL, NULL,
- "Database-Service", "", 20},
+ "Database-Service", "\0", 20},
{"user", "PGUSER", NULL, NULL,
- "Database-User", "", 20},
+ "Database-User", "\0R", 20},
{"password", "PGPASSWORD", NULL, NULL,
- "Database-Password", "*", 20},
+ "Database-Password", "*\0R", 20},
{"connect_timeout", "PGCONNECT_TIMEOUT", NULL, NULL,
- "Connect-timeout", "", 10}, /* strlen(INT32_MAX) == 10 */
+ "Connect-timeout", "\0R", 10}, /* strlen(INT32_MAX) == 10 */
{"dbname", "PGDATABASE", NULL, NULL,
- "Database-Name", "", 20},
+ "Database-Name", "\0", 20},
{"host", "PGHOST", NULL, NULL,
- "Database-Host", "", 40},
+ "Database-Host", "\0R", 40},
{"hostaddr", "PGHOSTADDR", NULL, NULL,
- "Database-Host-IP-Address", "", 45},
+ "Database-Host-IP-Address", "\0R", 45},
{"port", "PGPORT", DEF_PGPORT_STR, NULL,
- "Database-Port", "", 6},
+ "Database-Port", "\0R", 6},
{"client_encoding", "PGCLIENTENCODING", NULL, NULL,
- "Client-Encoding", "", 10},
+ "Client-Encoding", "\0", 10},
/*
* "tty" is no longer used either, but keep it present for backwards
* compatibility.
*/
{"tty", "PGTTY", DefaultTty, NULL,
- "Backend-Debug-TTY", "D", 40},
+ "Backend-Debug-TTY", "D\0", 40},
{"options", "PGOPTIONS", DefaultOption, NULL,
- "Backend-Debug-Options", "D", 40},
+ "Backend-Debug-Options", "D\0R", 40},
{"application_name", "PGAPPNAME", NULL, NULL,
- "Application-Name", "", 64},
+ "Application-Name", "\0", 64},
{"fallback_application_name", NULL, NULL, NULL,
- "Fallback-Application-Name", "", 64},
+ "Fallback-Application-Name", "\0", 64},
{"keepalives", NULL, NULL, NULL,
- "TCP-Keepalives", "", 1}, /* should be just '0' or '1' */
+ "TCP-Keepalives", "\0R", 1}, /* should be just '0' or '1' */
{"keepalives_idle", NULL, NULL, NULL,
- "TCP-Keepalives-Idle", "", 10}, /* strlen(INT32_MAX) == 10 */
+ "TCP-Keepalives-Idle", "\0R", 10}, /* strlen(INT32_MAX) == 10 */
{"keepalives_interval", NULL, NULL, NULL,
- "TCP-Keepalives-Interval", "", 10}, /* strlen(INT32_MAX) == 10 */
+ "TCP-Keepalives-Interval", "\0R", 10}, /* strlen(INT32_MAX) == 10 */
{"keepalives_count", NULL, NULL, NULL,
- "TCP-Keepalives-Count", "", 10}, /* strlen(INT32_MAX) == 10 */
+ "TCP-Keepalives-Count", "\0R", 10}, /* strlen(INT32_MAX) == 10 */
#ifdef USE_SSL
@@ -210,7 +215,7 @@ static const PQconninfoOption PQconninfo
* "sslmode". It remains for backwards compatibility.
*/
{"requiressl", "PGREQUIRESSL", "0", NULL,
- "Require-SSL", "D", 1},
+ "Require-SSL", "D\0", 1},
#endif
/*
@@ -220,30 +225,30 @@ static const PQconninfoOption PQconninfo
* to exclude them since none of them are mandatory.
*/
{"sslmode", "PGSSLMODE", DefaultSSLMode, NULL,
- "SSL-Mode", "", 8}, /* sizeof("disable") == 8 */
+ "SSL-Mode", "\0R", 8}, /* sizeof("disable") == 8 */
{"sslcompression", "PGSSLCOMPRESSION", "1", NULL,
- "SSL-Compression", "", 1},
+ "SSL-Compression", "\0R", 1},
{"sslcert", "PGSSLCERT", NULL, NULL,
- "SSL-Client-Cert", "", 64},
+ "SSL-Client-Cert", "\0R", 64},
{"sslkey", "PGSSLKEY", NULL, NULL,
- "SSL-Client-Key", "", 64},
+ "SSL-Client-Key", "\0R", 64},
{"sslrootcert", "PGSSLROOTCERT", NULL, NULL,
- "SSL-Root-Certificate", "", 64},
+ "SSL-Root-Certificate", "\0R", 64},
{"sslcrl", "PGSSLCRL", NULL, NULL,
- "SSL-Revocation-List", "", 64},
+ "SSL-Revocation-List", "\0R", 64},
{"requirepeer", "PGREQUIREPEER", NULL, NULL,
- "Require-Peer", "", 10},
+ "Require-Peer", "\0R", 10},
#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
/* Kerberos and GSSAPI authentication support specifying the service name */
{"krbsrvname", "PGKRBSRVNAME", PG_KRB_SRVNAM, NULL,
- "Kerberos-service-name", "", 20},
+ "Kerberos-service-name", "\0R", 20},
#endif
#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
@@ -253,17 +258,72 @@ static const PQconninfoOption PQconninfo
* default
*/
{"gsslib", "PGGSSLIB", NULL, NULL,
- "GSS-library", "", 7}, /* sizeof("gssapi") = 7 */
+ "GSS-library", "\0R", 7}, /* sizeof("gssapi") = 7 */
#endif
{"replication", NULL, NULL, NULL,
- "Replication", "D", 5},
+ "Replication", "D\0", 5},
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
};
+/*
+ * We need a mapping between the PQconninfoOptions[] array
+ * and PGconn members. We have to keep it separate from
+ * PQconninfoOptions[] to not leak info about PGconn members
+ * to clients.
+ */
+typedef struct PQconninfoMapping {
+ char *keyword;
+ size_t member_offset;
+ char *conninfoValue; /* Special value mapping */
+ char *connValue;
+} PQconninfoMapping;
+#define PGCONNMEMBERADDR(conn, mapping) ((char **)((char *)conn + mapping->member_offset))
+
+static const PQconninfoMapping PQconninfoMappings[] =
+{
+ /* "authtype" is not used anymore, there is no mapping to PGconn */
+ /* there is no mapping of "service" to PGconn */
+ { "user", offsetof(struct pg_conn, pguser), NULL, NULL },
+ { "password", offsetof(struct pg_conn, pgpass), NULL, NULL },
+ { "connect_timeout", offsetof(struct pg_conn, connect_timeout), NULL, NULL },
+ { "dbname", offsetof(struct pg_conn, dbName), NULL, NULL },
+ { "host", offsetof(struct pg_conn, pghost), NULL, NULL },
+ { "hostaddr", offsetof(struct pg_conn, pghostaddr), NULL, NULL },
+ { "port", offsetof(struct pg_conn, pgport), NULL, NULL },
+ { "client_encoding", offsetof(struct pg_conn, client_encoding_initial), NULL, NULL },
+ { "tty", offsetof(struct pg_conn, pgtty), NULL, NULL },
+ { "options", offsetof(struct pg_conn, pgoptions), NULL, NULL },
+ { "application_name", offsetof(struct pg_conn, appname), NULL, NULL },
+ { "fallback_application_name", offsetof(struct pg_conn, fbappname), NULL, NULL },
+ { "keepalives", offsetof(struct pg_conn, keepalives), NULL, NULL },
+ { "keepalives_idle", offsetof(struct pg_conn, keepalives_idle), NULL, NULL },
+ { "keepalives_interval", offsetof(struct pg_conn, keepalives_interval), NULL, NULL },
+ { "keepalives_count", offsetof(struct pg_conn, keepalives_count), NULL, NULL },
+#ifdef USE_SSL
+ { "requiressl", offsetof(struct pg_conn, sslmode), "1", "require" },
+#endif
+ { "sslmode", offsetof(struct pg_conn, sslmode), NULL, NULL },
+ { "sslcompression", offsetof(struct pg_conn, sslcompression), NULL, NULL },
+ { "sslcert", offsetof(struct pg_conn, sslcert), NULL, NULL },
+ { "sslkey", offsetof(struct pg_conn, sslkey), NULL, NULL },
+ { "sslrootcert", offsetof(struct pg_conn, sslrootcert), NULL, NULL },
+ { "sslcrl", offsetof(struct pg_conn, sslcrl), NULL, NULL },
+ { "requirepeer", offsetof(struct pg_conn, requirepeer), NULL, NULL },
+#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
+ { "krbsrvname", offsetof(struct pg_conn, krbsrvname), NULL, NULL },
+#endif
+#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
+ { "gsslib", offsetof(struct pg_conn, requirepeer), NULL, NULL },
+#endif
+ { "replication", offsetof(struct pg_conn, replication), NULL, NULL },
+ /* Terminating entry --- MUST BE LAST */
+ { NULL, 0, NULL, NULL }
+};
+
static const PQEnvironmentOption EnvironmentOptions[] =
{
/* common user-interface settings */
@@ -318,6 +378,8 @@ static char *conninfo_uri_decode(const c
static bool get_hexdigit(char digit, int *value);
static const char *conninfo_getval(PQconninfoOption *connOptions,
const char *keyword);
+static void conninfo_setval(PQconninfoOption *connOptions,
+ const char *keyword, const char *val);
static PQconninfoOption *conninfo_storeval(PQconninfoOption *connOptions,
const char *keyword, const char *value,
PQExpBuffer errorMessage, bool ignoreMissing, bool uri_decode);
@@ -627,7 +689,9 @@ PQconnectStart(const char *conninfo)
static void
fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
{
+ const PQconninfoMapping *mapping;
const char *tmp;
+ char **memberaddr;
/*
* Move option values into conn structure
@@ -637,72 +701,24 @@ fillPGconn(PGconn *conn, PQconninfoOptio
*
* XXX: probably worth checking strdup() return value here...
*/
- tmp = conninfo_getval(connOptions, "hostaddr");
- conn->pghostaddr = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "host");
- conn->pghost = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "port");
- conn->pgport = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "tty");
- conn->pgtty = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "options");
- conn->pgoptions = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "application_name");
- conn->appname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "fallback_application_name");
- conn->fbappname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "dbname");
- conn->dbName = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "user");
- conn->pguser = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "password");
- conn->pgpass = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "connect_timeout");
- conn->connect_timeout = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "client_encoding");
- conn->client_encoding_initial = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives");
- conn->keepalives = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_idle");
- conn->keepalives_idle = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_interval");
- conn->keepalives_interval = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_count");
- conn->keepalives_count = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslmode");
- conn->sslmode = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcompression");
- conn->sslcompression = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslkey");
- conn->sslkey = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcert");
- conn->sslcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslrootcert");
- conn->sslrootcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcrl");
- conn->sslcrl = tmp ? strdup(tmp) : NULL;
-#ifdef USE_SSL
- tmp = conninfo_getval(connOptions, "requiressl");
- if (tmp && tmp[0] == '1')
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
{
- /* here warn that the requiressl option is deprecated? */
- if (conn->sslmode)
- free(conn->sslmode);
- conn->sslmode = strdup("require");
+ tmp = conninfo_getval(connOptions, mapping->keyword);
+ memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ if (mapping->conninfoValue && mapping->connValue)
+ {
+ size_t len = strlen(mapping->conninfoValue);
+ if (tmp && strncmp(tmp, mapping->conninfoValue, len) == 0)
+ {
+ if (*memberaddr)
+ free(*memberaddr);
+ *memberaddr = strdup(mapping->connValue);
+ }
+ }
+ else
+ *memberaddr = tmp ? strdup(tmp) : NULL;
}
-#endif
- tmp = conninfo_getval(connOptions, "requirepeer");
- conn->requirepeer = tmp ? strdup(tmp) : NULL;
-#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "krbsrvname");
- conn->krbsrvname = tmp ? strdup(tmp) : NULL;
-#endif
-#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "gsslib");
- conn->gsslib = tmp ? strdup(tmp) : NULL;
-#endif
- tmp = conninfo_getval(connOptions, "replication");
- conn->replication = tmp ? strdup(tmp) : NULL;
}
/*
@@ -4985,6 +5001,24 @@ conninfo_getval(PQconninfoOption *connOp
}
/*
+ * Set an option value corresponding to the keyword in the connOptions array.
+ */
+static void
+conninfo_setval(PQconninfoOption *connOptions, const char *keyword,
+ const char *val)
+{
+ PQconninfoOption *option;
+
+ option = conninfo_find(connOptions, keyword);
+ if (option)
+ {
+ if (option->val)
+ free(option->val);
+ option->val = val ? strdup(val) : NULL;
+ }
+}
+
+/*
* Store a (new) value for an option corresponding to the keyword in
* connOptions array.
*
@@ -5066,6 +5100,56 @@ conninfo_find(PQconninfoOption *connOpti
}
+/*
+ * Return the connection options used for the connections
+ */
+PQconninfoOption *
+PQconninfo(PGconn *conn)
+{
+ PQExpBufferData errorBuf;
+ PQconninfoOption *connOptions;
+
+ if (conn == NULL)
+ return NULL;
+
+ /* We don't actually report any errors here, but callees want a buffer */
+ initPQExpBuffer(&errorBuf);
+ if (PQExpBufferDataBroken(errorBuf))
+ return NULL; /* out of memory already :-( */
+
+ connOptions = conninfo_init(&errorBuf);
+
+ termPQExpBuffer(&errorBuf);
+
+ if (connOptions != NULL)
+ {
+ const PQconninfoMapping *mapping;
+
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
+ {
+ char **memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ conninfo_setval(connOptions, mapping->keyword, *memberaddr);
+ }
+ }
+
+ return connOptions;
+}
+
+
+bool
+PQconninfoForReplication(PQconninfoOption *option)
+{
+ char *tmp;
+
+ if (option == NULL)
+ return false;
+
+ tmp = option->dispchar + strlen(option->dispchar) + 1;
+ return (*tmp == 'R');
+}
+
+
void
PQconninfoFree(PQconninfoOption *connOptions)
{
diff -durpN postgresql/src/interfaces/libpq/libpq-fe.h postgresql.1/src/interfaces/libpq/libpq-fe.h
--- postgresql/src/interfaces/libpq/libpq-fe.h 2012-08-03 09:39:30.122266626 +0200
+++ postgresql.1/src/interfaces/libpq/libpq-fe.h 2012-10-04 11:29:16.112252964 +0200
@@ -193,7 +193,8 @@ typedef struct _PQconninfoOption
* connect dialog. Values are: "" Display
* entered value as is "*" Password field -
* hide value "D" Debug option - don't show
- * by default */
+ * by default. Optionally a hidden "R" character
+ * after the first '\0' character. */
int dispsize; /* Field size in characters for dialog */
} PQconninfoOption;
@@ -262,6 +263,12 @@ extern PQconninfoOption *PQconndefaults(
/* parse connection options in same way as PQconnectdb */
extern PQconninfoOption *PQconninfoParse(const char *conninfo, char **errmsg);
+/* return the connection options used by a live connection */
+extern PQconninfoOption *PQconninfo(PGconn *conn);
+
+/* return whether the connection option is valid for a replication connection */
+extern bool PQconninfoForReplication(PQconninfoOption *option);
+
/* free the data structure returned by PQconndefaults() or PQconninfoParse() */
extern void PQconninfoFree(PQconninfoOption *connOptions);
02-pg_basebackup-v5.patchtext/x-patch; name=02-pg_basebackup-v5.patchDownload
diff -durpN postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml
--- postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml 2012-08-24 09:49:22.960530329 +0200
+++ postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml 2012-10-04 11:32:21.004441824 +0200
@@ -189,6 +189,27 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--write-recovery-conf</option></term>
+ <listitem>
+ <para>
+ Write a minimal recovery.conf into the output directory using
+ the connection parameters from the command line to ease
+ setting up the standby. Since creating a backup for a standalone
+ server with <option>-x</option> or <option>-X</option> and adding
+ a recovery.conf to it wouldn't make a working standby, these options
+ naturally conflict.
+ </para>
+ <para>
+ When this option is specified and taking the base backup succeeded,
+ failing to write the <filename>recovery.conf</filename> results
+ in the error code 2. This way, scripts can distinguish between different
+ failure cases of <application>pg_basebackup</application>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-x</option></term>
<term><option>--xlog</option></term>
<listitem>
diff -durpN postgresql.1/src/bin/pg_basebackup/pg_basebackup.c postgresql.2/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.1/src/bin/pg_basebackup/pg_basebackup.c 2012-10-03 10:40:48.297207389 +0200
+++ postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2012-10-04 11:34:19.137201718 +0200
@@ -46,6 +46,7 @@ int compresslevel = 0;
bool includewal = false;
bool streamwal = false;
bool fastcheckpoint = false;
+bool writerecoveryconf = false;
int standby_message_timeout = 10 * 1000; /* 10 sec = default */
/* Progress counters */
@@ -70,14 +71,18 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
+static PQconninfoOption *connOptions = NULL;
+
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
static void progress_report(int tablespacenum, const char *filename);
+static void stderr_write_error(FILE *cf, char *filename);
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
static void BaseBackup(void);
+static void WriteRecoveryConf(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
bool segment_finished);
@@ -107,6 +112,8 @@ usage(void)
printf(_("\nOptions controlling the output:\n"));
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
+ printf(_(" -R, --write-recovery-conf\n"
+ " write recovery.conf after backup\n"));
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
printf(_(" -X, --xlog-method=fetch|stream\n"
" include required WAL files with specified method\n"));
@@ -960,6 +967,10 @@ BaseBackup(void)
/* Error message already written in GetConnection() */
exit(1);
+ /* If recovery.conf is to be written, keep the connection parameters for later usage */
+ if (writerecoveryconf)
+ connOptions = PQconninfo(conn);
+
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
@@ -1234,6 +1245,71 @@ BaseBackup(void)
}
+static void
+stderr_write_error(FILE *cf, char *filename)
+{
+ fprintf(stderr, _("cannot write to %s: %s"), filename, strerror(errno));
+ fclose(cf);
+ unlink(filename);
+ exit(2);
+}
+
+
+/*
+ * Attempt to create recovery.conf and write the expected contents to it.
+ */
+static void
+WriteRecoveryConf(void)
+{
+ char filename[MAXPGPATH];
+ PQconninfoOption *option;
+ FILE *cf;
+
+ if (!writerecoveryconf)
+ return;
+
+ sprintf(filename, "%s/recovery.conf", basedir);
+
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("cannot create %s: %s"), filename, strerror(errno));
+ exit(2);
+ }
+
+ if (fprintf(cf, "standby_mode = 'on'\n") < 0)
+ stderr_write_error(cf, filename);
+
+ if (fprintf(cf, "primary_conninfo = '") < 0)
+ stderr_write_error(cf, filename);
+
+ for (option = connOptions; option && option->keyword; option++)
+ {
+ /*
+ * Do not emit this setting if not for replication,
+ * not set, empty or default.
+ */
+ if (!PQconninfoForReplication(option) ||
+ option->val == NULL ||
+ (option->val != NULL && option->val[0] == '\0') ||
+ (option->val &&
+ option->compiled &&
+ strcmp(option->val, option->compiled) == 0))
+ continue;
+
+ /* write "keyword='value'" pieces, single quotes doubled */
+ if (fprintf(cf, "%s=''%s'' ", option->keyword, option->val) < 0)
+ stderr_write_error(cf, filename);
+ }
+
+ if (fprintf(cf, "'\n") < 0)
+ stderr_write_error(cf, filename);
+
+ PQconninfoFree(connOptions);
+
+ fclose(cf);
+}
+
int
main(int argc, char **argv)
{
@@ -1243,6 +1319,7 @@ main(int argc, char **argv)
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
+ {"write-recovery-conf", no_argument, NULL, 'R'},
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
{"gzip", no_argument, NULL, 'z'},
@@ -1280,7 +1357,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:xX:l:zZ:c:h:p:U:s:wWvP",
+ while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
long_options, &option_index)) != -1)
{
switch (c)
@@ -1301,6 +1378,9 @@ main(int argc, char **argv)
exit(1);
}
break;
+ case 'R':
+ writerecoveryconf = true;
+ break;
case 'x':
if (includewal)
{
@@ -1466,6 +1546,13 @@ main(int argc, char **argv)
}
#endif
+ if (writerecoveryconf && includewal)
+ {
+ fprintf(stderr,
+ _("--xlog and --writerecoveryconf are mutually exclusive\n"));
+ exit(1);
+ }
+
/*
* Verify that the target directory exists, or create it. For plaintext
* backups, always require the directory. For tar backups, require it
@@ -1476,5 +1563,7 @@ main(int argc, char **argv)
BaseBackup();
+ WriteRecoveryConf();
+
return 0;
}
2012-10-04 12:42 keltezéssel, Boszormenyi Zoltan írta:
2012-10-04 06:47 keltezéssel, Boszormenyi Zoltan írta:
2012-10-04 05:24 keltezéssel, Peter Eisentraut írta:
On Wed, 2012-10-03 at 18:15 +0200, Boszormenyi Zoltan wrote:
The second generation of this work is now attached and contains a new
feature as was discussed and suggested by Magnus Hagander, Fujii Masao
and Peter Eisentraut. So libpq has grown a new function:+/* return the connection options used by a live connection */ +extern PQconninfoOption *PQconninfo(PGconn *conn);This copies all the connection parameters back from the live PGconn
itself
so everything that's needed to connect is already validated.I don't like that this code maintains a second list of all possible
libpq connection parameters.Where does it do that? In PQconninfo() itself? Why is it a problem?
Or to put it bluntly: the same problem is in fillPGconn() too, as it also
has the same set of parameters listed. So there is already code
that you don't like. :-)How about a static mapping between option names and
offsetof(struct pg_conn, member) values? This way both fillPGconn()
and PQconninfo() can avoid maintaining the list of parameter names.Did you think about something like the attached code?
Or rather this one, which fixes a bug so fillPGconn() and PQconninfo() are
symmetric and work for "requiressl".
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
01-PQconninfo-v8.patchtext/x-patch; name=01-PQconninfo-v8.patchDownload
diff -durpN postgresql/src/interfaces/libpq/exports.txt postgresql.1/src/interfaces/libpq/exports.txt
--- postgresql/src/interfaces/libpq/exports.txt 2012-08-03 09:39:30.118266598 +0200
+++ postgresql.1/src/interfaces/libpq/exports.txt 2012-10-04 11:29:24.864309200 +0200
@@ -161,3 +161,5 @@ PQping 158
PQpingParams 159
PQlibVersion 160
PQsetSingleRowMode 161
+PQconninfo 162
+PQconninfoForReplication 163
diff -durpN postgresql/src/interfaces/libpq/fe-connect.c postgresql.1/src/interfaces/libpq/fe-connect.c
--- postgresql/src/interfaces/libpq/fe-connect.c 2012-09-09 08:11:09.470401480 +0200
+++ postgresql.1/src/interfaces/libpq/fe-connect.c 2012-10-04 13:10:58.737418824 +0200
@@ -131,12 +131,17 @@ static int ldapServiceLookup(const char
* "" Normal input field
* "*" Password field - hide value
* "D" Debug option - don't show by default
+ * A special "usable for replication connections" ("R") flag is
+ * added to Disp-Char in a backwards compatible fashion.
*
* PQconninfoOptions[] is a constant static array that we use to initialize
* a dynamically allocated working copy. All the "val" fields in
* PQconninfoOptions[] *must* be NULL. In a working copy, non-null "val"
* fields point to malloc'd strings that should be freed when the working
* array is freed (see PQconninfoFree).
+ *
+ * If you add a new connection option to this list, remember to add it to
+ * PQconninfoMappings[] below.
* ----------
*/
static const PQconninfoOption PQconninfoOptions[] = {
@@ -146,62 +151,62 @@ static const PQconninfoOption PQconninfo
* still try to set it.
*/
{"authtype", "PGAUTHTYPE", DefaultAuthtype, NULL,
- "Database-Authtype", "D", 20},
+ "Database-Authtype", "D\0", 20},
{"service", "PGSERVICE", NULL, NULL,
- "Database-Service", "", 20},
+ "Database-Service", "\0", 20},
{"user", "PGUSER", NULL, NULL,
- "Database-User", "", 20},
+ "Database-User", "\0R", 20},
{"password", "PGPASSWORD", NULL, NULL,
- "Database-Password", "*", 20},
+ "Database-Password", "*\0R", 20},
{"connect_timeout", "PGCONNECT_TIMEOUT", NULL, NULL,
- "Connect-timeout", "", 10}, /* strlen(INT32_MAX) == 10 */
+ "Connect-timeout", "\0R", 10}, /* strlen(INT32_MAX) == 10 */
{"dbname", "PGDATABASE", NULL, NULL,
- "Database-Name", "", 20},
+ "Database-Name", "\0", 20},
{"host", "PGHOST", NULL, NULL,
- "Database-Host", "", 40},
+ "Database-Host", "\0R", 40},
{"hostaddr", "PGHOSTADDR", NULL, NULL,
- "Database-Host-IP-Address", "", 45},
+ "Database-Host-IP-Address", "\0R", 45},
{"port", "PGPORT", DEF_PGPORT_STR, NULL,
- "Database-Port", "", 6},
+ "Database-Port", "\0R", 6},
{"client_encoding", "PGCLIENTENCODING", NULL, NULL,
- "Client-Encoding", "", 10},
+ "Client-Encoding", "\0", 10},
/*
* "tty" is no longer used either, but keep it present for backwards
* compatibility.
*/
{"tty", "PGTTY", DefaultTty, NULL,
- "Backend-Debug-TTY", "D", 40},
+ "Backend-Debug-TTY", "D\0", 40},
{"options", "PGOPTIONS", DefaultOption, NULL,
- "Backend-Debug-Options", "D", 40},
+ "Backend-Debug-Options", "D\0R", 40},
{"application_name", "PGAPPNAME", NULL, NULL,
- "Application-Name", "", 64},
+ "Application-Name", "\0", 64},
{"fallback_application_name", NULL, NULL, NULL,
- "Fallback-Application-Name", "", 64},
+ "Fallback-Application-Name", "\0", 64},
{"keepalives", NULL, NULL, NULL,
- "TCP-Keepalives", "", 1}, /* should be just '0' or '1' */
+ "TCP-Keepalives", "\0R", 1}, /* should be just '0' or '1' */
{"keepalives_idle", NULL, NULL, NULL,
- "TCP-Keepalives-Idle", "", 10}, /* strlen(INT32_MAX) == 10 */
+ "TCP-Keepalives-Idle", "\0R", 10}, /* strlen(INT32_MAX) == 10 */
{"keepalives_interval", NULL, NULL, NULL,
- "TCP-Keepalives-Interval", "", 10}, /* strlen(INT32_MAX) == 10 */
+ "TCP-Keepalives-Interval", "\0R", 10}, /* strlen(INT32_MAX) == 10 */
{"keepalives_count", NULL, NULL, NULL,
- "TCP-Keepalives-Count", "", 10}, /* strlen(INT32_MAX) == 10 */
+ "TCP-Keepalives-Count", "\0R", 10}, /* strlen(INT32_MAX) == 10 */
#ifdef USE_SSL
@@ -210,7 +215,7 @@ static const PQconninfoOption PQconninfo
* "sslmode". It remains for backwards compatibility.
*/
{"requiressl", "PGREQUIRESSL", "0", NULL,
- "Require-SSL", "D", 1},
+ "Require-SSL", "D\0", 1},
#endif
/*
@@ -220,30 +225,30 @@ static const PQconninfoOption PQconninfo
* to exclude them since none of them are mandatory.
*/
{"sslmode", "PGSSLMODE", DefaultSSLMode, NULL,
- "SSL-Mode", "", 8}, /* sizeof("disable") == 8 */
+ "SSL-Mode", "\0R", 8}, /* sizeof("disable") == 8 */
{"sslcompression", "PGSSLCOMPRESSION", "1", NULL,
- "SSL-Compression", "", 1},
+ "SSL-Compression", "\0R", 1},
{"sslcert", "PGSSLCERT", NULL, NULL,
- "SSL-Client-Cert", "", 64},
+ "SSL-Client-Cert", "\0R", 64},
{"sslkey", "PGSSLKEY", NULL, NULL,
- "SSL-Client-Key", "", 64},
+ "SSL-Client-Key", "\0R", 64},
{"sslrootcert", "PGSSLROOTCERT", NULL, NULL,
- "SSL-Root-Certificate", "", 64},
+ "SSL-Root-Certificate", "\0R", 64},
{"sslcrl", "PGSSLCRL", NULL, NULL,
- "SSL-Revocation-List", "", 64},
+ "SSL-Revocation-List", "\0R", 64},
{"requirepeer", "PGREQUIREPEER", NULL, NULL,
- "Require-Peer", "", 10},
+ "Require-Peer", "\0R", 10},
#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
/* Kerberos and GSSAPI authentication support specifying the service name */
{"krbsrvname", "PGKRBSRVNAME", PG_KRB_SRVNAM, NULL,
- "Kerberos-service-name", "", 20},
+ "Kerberos-service-name", "\0R", 20},
#endif
#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
@@ -253,17 +258,72 @@ static const PQconninfoOption PQconninfo
* default
*/
{"gsslib", "PGGSSLIB", NULL, NULL,
- "GSS-library", "", 7}, /* sizeof("gssapi") = 7 */
+ "GSS-library", "\0R", 7}, /* sizeof("gssapi") = 7 */
#endif
{"replication", NULL, NULL, NULL,
- "Replication", "D", 5},
+ "Replication", "D\0", 5},
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
};
+/*
+ * We need a mapping between the PQconninfoOptions[] array
+ * and PGconn members. We have to keep it separate from
+ * PQconninfoOptions[] to not leak info about PGconn members
+ * to clients.
+ */
+typedef struct PQconninfoMapping {
+ char *keyword;
+ size_t member_offset;
+ char *conninfoValue; /* Special value mapping */
+ char *connValue;
+} PQconninfoMapping;
+#define PGCONNMEMBERADDR(conn, mapping) ((char **)((char *)conn + mapping->member_offset))
+
+static const PQconninfoMapping PQconninfoMappings[] =
+{
+ /* "authtype" is not used anymore, there is no mapping to PGconn */
+ /* there is no mapping of "service" to PGconn */
+ { "user", offsetof(struct pg_conn, pguser), NULL, NULL },
+ { "password", offsetof(struct pg_conn, pgpass), NULL, NULL },
+ { "connect_timeout", offsetof(struct pg_conn, connect_timeout), NULL, NULL },
+ { "dbname", offsetof(struct pg_conn, dbName), NULL, NULL },
+ { "host", offsetof(struct pg_conn, pghost), NULL, NULL },
+ { "hostaddr", offsetof(struct pg_conn, pghostaddr), NULL, NULL },
+ { "port", offsetof(struct pg_conn, pgport), NULL, NULL },
+ { "client_encoding", offsetof(struct pg_conn, client_encoding_initial), NULL, NULL },
+ { "tty", offsetof(struct pg_conn, pgtty), NULL, NULL },
+ { "options", offsetof(struct pg_conn, pgoptions), NULL, NULL },
+ { "application_name", offsetof(struct pg_conn, appname), NULL, NULL },
+ { "fallback_application_name", offsetof(struct pg_conn, fbappname), NULL, NULL },
+ { "keepalives", offsetof(struct pg_conn, keepalives), NULL, NULL },
+ { "keepalives_idle", offsetof(struct pg_conn, keepalives_idle), NULL, NULL },
+ { "keepalives_interval", offsetof(struct pg_conn, keepalives_interval), NULL, NULL },
+ { "keepalives_count", offsetof(struct pg_conn, keepalives_count), NULL, NULL },
+#ifdef USE_SSL
+ { "requiressl", offsetof(struct pg_conn, sslmode), "1", "require" },
+#endif
+ { "sslmode", offsetof(struct pg_conn, sslmode), NULL, NULL },
+ { "sslcompression", offsetof(struct pg_conn, sslcompression), NULL, NULL },
+ { "sslcert", offsetof(struct pg_conn, sslcert), NULL, NULL },
+ { "sslkey", offsetof(struct pg_conn, sslkey), NULL, NULL },
+ { "sslrootcert", offsetof(struct pg_conn, sslrootcert), NULL, NULL },
+ { "sslcrl", offsetof(struct pg_conn, sslcrl), NULL, NULL },
+ { "requirepeer", offsetof(struct pg_conn, requirepeer), NULL, NULL },
+#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
+ { "krbsrvname", offsetof(struct pg_conn, krbsrvname), NULL, NULL },
+#endif
+#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
+ { "gsslib", offsetof(struct pg_conn, requirepeer), NULL, NULL },
+#endif
+ { "replication", offsetof(struct pg_conn, replication), NULL, NULL },
+ /* Terminating entry --- MUST BE LAST */
+ { NULL, 0, NULL, NULL }
+};
+
static const PQEnvironmentOption EnvironmentOptions[] =
{
/* common user-interface settings */
@@ -318,6 +378,8 @@ static char *conninfo_uri_decode(const c
static bool get_hexdigit(char digit, int *value);
static const char *conninfo_getval(PQconninfoOption *connOptions,
const char *keyword);
+static void conninfo_setval(PQconninfoOption *connOptions,
+ const char *keyword, const char *val);
static PQconninfoOption *conninfo_storeval(PQconninfoOption *connOptions,
const char *keyword, const char *value,
PQExpBuffer errorMessage, bool ignoreMissing, bool uri_decode);
@@ -627,7 +689,9 @@ PQconnectStart(const char *conninfo)
static void
fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
{
+ const PQconninfoMapping *mapping;
const char *tmp;
+ char **memberaddr;
/*
* Move option values into conn structure
@@ -637,72 +701,24 @@ fillPGconn(PGconn *conn, PQconninfoOptio
*
* XXX: probably worth checking strdup() return value here...
*/
- tmp = conninfo_getval(connOptions, "hostaddr");
- conn->pghostaddr = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "host");
- conn->pghost = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "port");
- conn->pgport = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "tty");
- conn->pgtty = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "options");
- conn->pgoptions = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "application_name");
- conn->appname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "fallback_application_name");
- conn->fbappname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "dbname");
- conn->dbName = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "user");
- conn->pguser = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "password");
- conn->pgpass = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "connect_timeout");
- conn->connect_timeout = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "client_encoding");
- conn->client_encoding_initial = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives");
- conn->keepalives = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_idle");
- conn->keepalives_idle = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_interval");
- conn->keepalives_interval = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_count");
- conn->keepalives_count = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslmode");
- conn->sslmode = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcompression");
- conn->sslcompression = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslkey");
- conn->sslkey = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcert");
- conn->sslcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslrootcert");
- conn->sslrootcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcrl");
- conn->sslcrl = tmp ? strdup(tmp) : NULL;
-#ifdef USE_SSL
- tmp = conninfo_getval(connOptions, "requiressl");
- if (tmp && tmp[0] == '1')
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
{
- /* here warn that the requiressl option is deprecated? */
- if (conn->sslmode)
- free(conn->sslmode);
- conn->sslmode = strdup("require");
+ tmp = conninfo_getval(connOptions, mapping->keyword);
+ memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ if (mapping->conninfoValue && mapping->connValue)
+ {
+ size_t len = strlen(mapping->conninfoValue);
+ if (tmp && strncmp(tmp, mapping->conninfoValue, len) == 0)
+ {
+ if (*memberaddr)
+ free(*memberaddr);
+ *memberaddr = strdup(mapping->connValue);
+ }
+ }
+ else
+ *memberaddr = tmp ? strdup(tmp) : NULL;
}
-#endif
- tmp = conninfo_getval(connOptions, "requirepeer");
- conn->requirepeer = tmp ? strdup(tmp) : NULL;
-#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "krbsrvname");
- conn->krbsrvname = tmp ? strdup(tmp) : NULL;
-#endif
-#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "gsslib");
- conn->gsslib = tmp ? strdup(tmp) : NULL;
-#endif
- tmp = conninfo_getval(connOptions, "replication");
- conn->replication = tmp ? strdup(tmp) : NULL;
}
/*
@@ -4985,6 +5001,24 @@ conninfo_getval(PQconninfoOption *connOp
}
/*
+ * Set an option value corresponding to the keyword in the connOptions array.
+ */
+static void
+conninfo_setval(PQconninfoOption *connOptions, const char *keyword,
+ const char *val)
+{
+ PQconninfoOption *option;
+
+ option = conninfo_find(connOptions, keyword);
+ if (option)
+ {
+ if (option->val)
+ free(option->val);
+ option->val = val ? strdup(val) : NULL;
+ }
+}
+
+/*
* Store a (new) value for an option corresponding to the keyword in
* connOptions array.
*
@@ -5066,6 +5100,63 @@ conninfo_find(PQconninfoOption *connOpti
}
+/*
+ * Return the connection options used for the connections
+ */
+PQconninfoOption *
+PQconninfo(PGconn *conn)
+{
+ PQExpBufferData errorBuf;
+ PQconninfoOption *connOptions;
+
+ if (conn == NULL)
+ return NULL;
+
+ /* We don't actually report any errors here, but callees want a buffer */
+ initPQExpBuffer(&errorBuf);
+ if (PQExpBufferDataBroken(errorBuf))
+ return NULL; /* out of memory already :-( */
+
+ connOptions = conninfo_init(&errorBuf);
+
+ termPQExpBuffer(&errorBuf);
+
+ if (connOptions != NULL)
+ {
+ const PQconninfoMapping *mapping;
+
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
+ {
+ char **memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ if (mapping->conninfoValue && mapping->connValue)
+ {
+ size_t len = strlen(mapping->connValue);
+ if (*memberaddr && strcmp(*memberaddr, mapping->connValue, len) == 0)
+ conninfo_setval(connOptions, mapping->keyword, mapping->conninfoValue)
+ }
+ else
+ conninfo_setval(connOptions, mapping->keyword, *memberaddr);
+ }
+ }
+
+ return connOptions;
+}
+
+
+bool
+PQconninfoForReplication(PQconninfoOption *option)
+{
+ char *tmp;
+
+ if (option == NULL)
+ return false;
+
+ tmp = option->dispchar + strlen(option->dispchar) + 1;
+ return (*tmp == 'R');
+}
+
+
void
PQconninfoFree(PQconninfoOption *connOptions)
{
diff -durpN postgresql/src/interfaces/libpq/libpq-fe.h postgresql.1/src/interfaces/libpq/libpq-fe.h
--- postgresql/src/interfaces/libpq/libpq-fe.h 2012-08-03 09:39:30.122266626 +0200
+++ postgresql.1/src/interfaces/libpq/libpq-fe.h 2012-10-04 11:29:16.112252964 +0200
@@ -193,7 +193,8 @@ typedef struct _PQconninfoOption
* connect dialog. Values are: "" Display
* entered value as is "*" Password field -
* hide value "D" Debug option - don't show
- * by default */
+ * by default. Optionally a hidden "R" character
+ * after the first '\0' character. */
int dispsize; /* Field size in characters for dialog */
} PQconninfoOption;
@@ -262,6 +263,12 @@ extern PQconninfoOption *PQconndefaults(
/* parse connection options in same way as PQconnectdb */
extern PQconninfoOption *PQconninfoParse(const char *conninfo, char **errmsg);
+/* return the connection options used by a live connection */
+extern PQconninfoOption *PQconninfo(PGconn *conn);
+
+/* return whether the connection option is valid for a replication connection */
+extern bool PQconninfoForReplication(PQconninfoOption *option);
+
/* free the data structure returned by PQconndefaults() or PQconninfoParse() */
extern void PQconninfoFree(PQconninfoOption *connOptions);
02-pg_basebackup-v5.patchtext/x-patch; name=02-pg_basebackup-v5.patchDownload
diff -durpN postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml
--- postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml 2012-08-24 09:49:22.960530329 +0200
+++ postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml 2012-10-04 11:32:21.004441824 +0200
@@ -189,6 +189,27 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--write-recovery-conf</option></term>
+ <listitem>
+ <para>
+ Write a minimal recovery.conf into the output directory using
+ the connection parameters from the command line to ease
+ setting up the standby. Since creating a backup for a standalone
+ server with <option>-x</option> or <option>-X</option> and adding
+ a recovery.conf to it wouldn't make a working standby, these options
+ naturally conflict.
+ </para>
+ <para>
+ When this option is specified and taking the base backup succeeded,
+ failing to write the <filename>recovery.conf</filename> results
+ in the error code 2. This way, scripts can distinguish between different
+ failure cases of <application>pg_basebackup</application>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-x</option></term>
<term><option>--xlog</option></term>
<listitem>
diff -durpN postgresql.1/src/bin/pg_basebackup/pg_basebackup.c postgresql.2/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.1/src/bin/pg_basebackup/pg_basebackup.c 2012-10-03 10:40:48.297207389 +0200
+++ postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2012-10-04 11:34:19.137201718 +0200
@@ -46,6 +46,7 @@ int compresslevel = 0;
bool includewal = false;
bool streamwal = false;
bool fastcheckpoint = false;
+bool writerecoveryconf = false;
int standby_message_timeout = 10 * 1000; /* 10 sec = default */
/* Progress counters */
@@ -70,14 +71,18 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
+static PQconninfoOption *connOptions = NULL;
+
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
static void progress_report(int tablespacenum, const char *filename);
+static void stderr_write_error(FILE *cf, char *filename);
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
static void BaseBackup(void);
+static void WriteRecoveryConf(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
bool segment_finished);
@@ -107,6 +112,8 @@ usage(void)
printf(_("\nOptions controlling the output:\n"));
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
+ printf(_(" -R, --write-recovery-conf\n"
+ " write recovery.conf after backup\n"));
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
printf(_(" -X, --xlog-method=fetch|stream\n"
" include required WAL files with specified method\n"));
@@ -960,6 +967,10 @@ BaseBackup(void)
/* Error message already written in GetConnection() */
exit(1);
+ /* If recovery.conf is to be written, keep the connection parameters for later usage */
+ if (writerecoveryconf)
+ connOptions = PQconninfo(conn);
+
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
@@ -1234,6 +1245,71 @@ BaseBackup(void)
}
+static void
+stderr_write_error(FILE *cf, char *filename)
+{
+ fprintf(stderr, _("cannot write to %s: %s"), filename, strerror(errno));
+ fclose(cf);
+ unlink(filename);
+ exit(2);
+}
+
+
+/*
+ * Attempt to create recovery.conf and write the expected contents to it.
+ */
+static void
+WriteRecoveryConf(void)
+{
+ char filename[MAXPGPATH];
+ PQconninfoOption *option;
+ FILE *cf;
+
+ if (!writerecoveryconf)
+ return;
+
+ sprintf(filename, "%s/recovery.conf", basedir);
+
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("cannot create %s: %s"), filename, strerror(errno));
+ exit(2);
+ }
+
+ if (fprintf(cf, "standby_mode = 'on'\n") < 0)
+ stderr_write_error(cf, filename);
+
+ if (fprintf(cf, "primary_conninfo = '") < 0)
+ stderr_write_error(cf, filename);
+
+ for (option = connOptions; option && option->keyword; option++)
+ {
+ /*
+ * Do not emit this setting if not for replication,
+ * not set, empty or default.
+ */
+ if (!PQconninfoForReplication(option) ||
+ option->val == NULL ||
+ (option->val != NULL && option->val[0] == '\0') ||
+ (option->val &&
+ option->compiled &&
+ strcmp(option->val, option->compiled) == 0))
+ continue;
+
+ /* write "keyword='value'" pieces, single quotes doubled */
+ if (fprintf(cf, "%s=''%s'' ", option->keyword, option->val) < 0)
+ stderr_write_error(cf, filename);
+ }
+
+ if (fprintf(cf, "'\n") < 0)
+ stderr_write_error(cf, filename);
+
+ PQconninfoFree(connOptions);
+
+ fclose(cf);
+}
+
int
main(int argc, char **argv)
{
@@ -1243,6 +1319,7 @@ main(int argc, char **argv)
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
+ {"write-recovery-conf", no_argument, NULL, 'R'},
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
{"gzip", no_argument, NULL, 'z'},
@@ -1280,7 +1357,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:xX:l:zZ:c:h:p:U:s:wWvP",
+ while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
long_options, &option_index)) != -1)
{
switch (c)
@@ -1301,6 +1378,9 @@ main(int argc, char **argv)
exit(1);
}
break;
+ case 'R':
+ writerecoveryconf = true;
+ break;
case 'x':
if (includewal)
{
@@ -1466,6 +1546,13 @@ main(int argc, char **argv)
}
#endif
+ if (writerecoveryconf && includewal)
+ {
+ fprintf(stderr,
+ _("--xlog and --writerecoveryconf are mutually exclusive\n"));
+ exit(1);
+ }
+
/*
* Verify that the target directory exists, or create it. For plaintext
* backups, always require the directory. For tar backups, require it
@@ -1476,5 +1563,7 @@ main(int argc, char **argv)
BaseBackup();
+ WriteRecoveryConf();
+
return 0;
}
Boszormenyi Zoltan <zb@cybertec.at> writes:
Did you think about something like the attached code?
Or rather this one, which fixes a bug so fillPGconn() and PQconninfo() are
symmetric and work for "requiressl".
That's incredibly ugly. I'm not sure where we should keep the "R"
information, but shoehorning it into the existing PQconninfoOption
struct like that seems totally unacceptable. Either we're willing to
break backwards compatibility on the Disp-Char strings, or we need
to put that info somewhere else.
regards, tom lane
2012-10-04 16:43 keltezéssel, Tom Lane írta:
Boszormenyi Zoltan <zb@cybertec.at> writes:
Did you think about something like the attached code?
Or rather this one, which fixes a bug so fillPGconn() and PQconninfo() are
symmetric and work for "requiressl".That's incredibly ugly. I'm not sure where we should keep the "R"
information, but shoehorning it into the existing PQconninfoOption
struct like that seems totally unacceptable. Either we're willing to
break backwards compatibility on the Disp-Char strings, or we need
to put that info somewhere else.
I hope only handling the new flag part is ugly. It can be hidden
in the PQconninfoMapping[] array and PQconninfo(conn, true)
pre-filters the list as in the attached version.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
01-PQconninfo-v9.patchtext/x-patch; name=01-PQconninfo-v9.patchDownload
diff -durpN postgresql/doc/src/sgml/libpq.sgml postgresql.1/doc/src/sgml/libpq.sgml
--- postgresql/doc/src/sgml/libpq.sgml 2012-08-03 09:39:30.114266570 +0200
+++ postgresql.1/doc/src/sgml/libpq.sgml 2012-10-04 18:15:11.686199926 +0200
@@ -496,6 +496,37 @@ typedef struct
</listitem>
</varlistentry>
+ <varlistentry id="libpq-pqconninfo">
+ <term><function>PQconninfo</function><indexterm><primary>PQconninfo</></></term>
+ <listitem>
+ <para>
+ Returns the connection options used by a live connection.
+<synopsis>
+PQconninfoOption *PQconninfo(PGconn *conn, bool for_replication);
+</synopsis>
+ </para>
+
+ <para>
+ Returns a connection options array. This can be used to determine
+ all possible <function>PQconnectdb</function> options and their
+ current values that were used to connect to the server. The return
+ value points to an array of <structname>PQconninfoOption</structname>
+ structures, which ends with an entry having a null <structfield>keyword</>
+ pointer. Every notes above for <function>PQconndefaults</function> also apply.
+ </para>
+
+ <para>
+ The <literal>for_replication</> argument can be used to exclude some
+ options from the list which are used by the walreceiver module.
+ <application>pg_basebackup</application> uses this pre-filtered list
+ to construct <literal>primary_conninfo</> in the automatically generated
+ recovery.conf file.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+
<varlistentry id="libpq-pqconninfoparse">
<term><function>PQconninfoParse</function><indexterm><primary>PQconninfoParse</></></term>
<listitem>
diff -durpN postgresql/src/interfaces/libpq/exports.txt postgresql.1/src/interfaces/libpq/exports.txt
--- postgresql/src/interfaces/libpq/exports.txt 2012-08-03 09:39:30.118266598 +0200
+++ postgresql.1/src/interfaces/libpq/exports.txt 2012-10-04 17:10:11.483654334 +0200
@@ -161,3 +161,4 @@ PQping 158
PQpingParams 159
PQlibVersion 160
PQsetSingleRowMode 161
+PQconninfo 162
diff -durpN postgresql/src/interfaces/libpq/fe-connect.c postgresql.1/src/interfaces/libpq/fe-connect.c
--- postgresql/src/interfaces/libpq/fe-connect.c 2012-09-09 08:11:09.470401480 +0200
+++ postgresql.1/src/interfaces/libpq/fe-connect.c 2012-10-04 17:53:26.205928500 +0200
@@ -137,6 +137,9 @@ static int ldapServiceLookup(const char
* PQconninfoOptions[] *must* be NULL. In a working copy, non-null "val"
* fields point to malloc'd strings that should be freed when the working
* array is freed (see PQconninfoFree).
+ *
+ * If you add a new connection option to this list, remember to add it to
+ * PQconninfoMappings[] below.
* ----------
*/
static const PQconninfoOption PQconninfoOptions[] = {
@@ -264,6 +267,62 @@ static const PQconninfoOption PQconninfo
NULL, NULL, 0}
};
+/*
+ * We need a mapping between the PQconninfoOptions[] array
+ * and PGconn members. We have to keep it separate from
+ * PQconninfoOptions[] to not leak info about PGconn members
+ * to clients.
+ */
+typedef struct PQconninfoMapping {
+ char *keyword;
+ size_t member_offset;
+ bool for_replication;
+ char *conninfoValue; /* Special value mapping */
+ char *connValue;
+} PQconninfoMapping;
+#define PGCONNMEMBERADDR(conn, mapping) ((char **)((char *)conn + mapping->member_offset))
+
+static const PQconninfoMapping PQconninfoMappings[] =
+{
+ /* "authtype" is not used anymore, there is no mapping to PGconn */
+ /* there is no mapping of "service" to PGconn */
+ { "user", offsetof(struct pg_conn, pguser), true, NULL, NULL },
+ { "password", offsetof(struct pg_conn, pgpass), true, NULL, NULL },
+ { "connect_timeout", offsetof(struct pg_conn, connect_timeout), true, NULL, NULL },
+ { "dbname", offsetof(struct pg_conn, dbName), false, NULL, NULL },
+ { "host", offsetof(struct pg_conn, pghost), true, NULL, NULL },
+ { "hostaddr", offsetof(struct pg_conn, pghostaddr), true, NULL, NULL },
+ { "port", offsetof(struct pg_conn, pgport), true, NULL, NULL },
+ { "client_encoding", offsetof(struct pg_conn, client_encoding_initial), false, NULL, NULL },
+ { "tty", offsetof(struct pg_conn, pgtty), false, NULL, NULL },
+ { "options", offsetof(struct pg_conn, pgoptions), true, NULL, NULL },
+ { "application_name", offsetof(struct pg_conn, appname), false, NULL, NULL },
+ { "fallback_application_name", offsetof(struct pg_conn, fbappname), false, NULL, NULL },
+ { "keepalives", offsetof(struct pg_conn, keepalives), true, NULL, NULL },
+ { "keepalives_idle", offsetof(struct pg_conn, keepalives_idle), true, NULL, NULL },
+ { "keepalives_interval", offsetof(struct pg_conn, keepalives_interval), true, NULL, NULL },
+ { "keepalives_count", offsetof(struct pg_conn, keepalives_count), true, NULL, NULL },
+#ifdef USE_SSL
+ { "requiressl", offsetof(struct pg_conn, sslmode), false, "1", "require" },
+#endif
+ { "sslmode", offsetof(struct pg_conn, sslmode), true, NULL, NULL },
+ { "sslcompression", offsetof(struct pg_conn, sslcompression), true, NULL, NULL },
+ { "sslcert", offsetof(struct pg_conn, sslcert), true, NULL, NULL },
+ { "sslkey", offsetof(struct pg_conn, sslkey), true, NULL, NULL },
+ { "sslrootcert", offsetof(struct pg_conn, sslrootcert), true, NULL, NULL },
+ { "sslcrl", offsetof(struct pg_conn, sslcrl), true, NULL, NULL },
+ { "requirepeer", offsetof(struct pg_conn, requirepeer), true, NULL, NULL },
+#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
+ { "krbsrvname", offsetof(struct pg_conn, krbsrvname), true, NULL, NULL },
+#endif
+#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
+ { "gsslib", offsetof(struct pg_conn, requirepeer), true, NULL, NULL },
+#endif
+ { "replication", offsetof(struct pg_conn, replication), false, NULL, NULL },
+ /* Terminating entry --- MUST BE LAST */
+ { NULL, 0, false, NULL, NULL }
+};
+
static const PQEnvironmentOption EnvironmentOptions[] =
{
/* common user-interface settings */
@@ -295,7 +354,8 @@ static PGconn *makeEmptyPGconn(void);
static void fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
-static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage);
+static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage,
+ bool for_replication);
static PQconninfoOption *parse_connection_string(const char *conninfo,
PQExpBuffer errorMessage, bool use_defaults);
static int uri_prefix_length(const char *connstr);
@@ -318,6 +378,8 @@ static char *conninfo_uri_decode(const c
static bool get_hexdigit(char digit, int *value);
static const char *conninfo_getval(PQconninfoOption *connOptions,
const char *keyword);
+static void conninfo_setval(PQconninfoOption *connOptions,
+ const char *keyword, const char *val);
static PQconninfoOption *conninfo_storeval(PQconninfoOption *connOptions,
const char *keyword, const char *value,
PQExpBuffer errorMessage, bool ignoreMissing, bool uri_decode);
@@ -627,7 +689,9 @@ PQconnectStart(const char *conninfo)
static void
fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
{
+ const PQconninfoMapping *mapping;
const char *tmp;
+ char **memberaddr;
/*
* Move option values into conn structure
@@ -637,72 +701,24 @@ fillPGconn(PGconn *conn, PQconninfoOptio
*
* XXX: probably worth checking strdup() return value here...
*/
- tmp = conninfo_getval(connOptions, "hostaddr");
- conn->pghostaddr = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "host");
- conn->pghost = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "port");
- conn->pgport = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "tty");
- conn->pgtty = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "options");
- conn->pgoptions = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "application_name");
- conn->appname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "fallback_application_name");
- conn->fbappname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "dbname");
- conn->dbName = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "user");
- conn->pguser = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "password");
- conn->pgpass = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "connect_timeout");
- conn->connect_timeout = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "client_encoding");
- conn->client_encoding_initial = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives");
- conn->keepalives = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_idle");
- conn->keepalives_idle = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_interval");
- conn->keepalives_interval = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_count");
- conn->keepalives_count = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslmode");
- conn->sslmode = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcompression");
- conn->sslcompression = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslkey");
- conn->sslkey = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcert");
- conn->sslcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslrootcert");
- conn->sslrootcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcrl");
- conn->sslcrl = tmp ? strdup(tmp) : NULL;
-#ifdef USE_SSL
- tmp = conninfo_getval(connOptions, "requiressl");
- if (tmp && tmp[0] == '1')
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
{
- /* here warn that the requiressl option is deprecated? */
- if (conn->sslmode)
- free(conn->sslmode);
- conn->sslmode = strdup("require");
+ tmp = conninfo_getval(connOptions, mapping->keyword);
+ memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ if (mapping->conninfoValue && mapping->connValue)
+ {
+ size_t len = strlen(mapping->conninfoValue);
+ if (tmp && strncmp(tmp, mapping->conninfoValue, len) == 0)
+ {
+ if (*memberaddr)
+ free(*memberaddr);
+ *memberaddr = strdup(mapping->connValue);
+ }
+ }
+ else
+ *memberaddr = tmp ? strdup(tmp) : NULL;
}
-#endif
- tmp = conninfo_getval(connOptions, "requirepeer");
- conn->requirepeer = tmp ? strdup(tmp) : NULL;
-#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "krbsrvname");
- conn->krbsrvname = tmp ? strdup(tmp) : NULL;
-#endif
-#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "gsslib");
- conn->gsslib = tmp ? strdup(tmp) : NULL;
-#endif
- tmp = conninfo_getval(connOptions, "replication");
- conn->replication = tmp ? strdup(tmp) : NULL;
}
/*
@@ -884,7 +900,7 @@ PQconndefaults(void)
if (PQExpBufferDataBroken(errorBuf))
return NULL; /* out of memory already :-( */
- connOptions = conninfo_init(&errorBuf);
+ connOptions = conninfo_init(&errorBuf, false);
if (connOptions != NULL)
{
if (!conninfo_add_defaults(connOptions, &errorBuf))
@@ -4006,9 +4022,11 @@ PQconninfoParse(const char *conninfo, ch
/*
* Build a working copy of the constant PQconninfoOptions array.
+ * If for_replication is true, only return the options that are
+ * not added by libpqwalreceiver.
*/
static PQconninfoOption *
-conninfo_init(PQExpBuffer errorMessage)
+conninfo_init(PQExpBuffer errorMessage, bool for_replication)
{
PQconninfoOption *options;
@@ -4019,7 +4037,27 @@ conninfo_init(PQExpBuffer errorMessage)
libpq_gettext("out of memory\n"));
return NULL;
}
- memcpy(options, PQconninfoOptions, sizeof(PQconninfoOptions));
+ if (for_replication)
+ {
+ const PQconninfoMapping *mapping = PQconninfoMappings;
+ PQconninfoOption *opt_dest = options;
+
+ while (mapping->keyword)
+ {
+ const PQconninfoOption *opt_src = conninfo_find(PQconninfoOptions, mapping->keyword);
+
+ if (opt_src && mapping->for_replication)
+ {
+ memcpy(opt_dest, opt_src, sizeof(PQconninfoOption));
+ opt_dest++;
+ }
+
+ opt_src++;
+ mapping++;
+ }
+ }
+ else
+ memcpy(options, PQconninfoOptions, sizeof(PQconninfoOptions));
return options;
}
@@ -4095,7 +4133,7 @@ conninfo_parse(const char *conninfo, PQE
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
return NULL;
@@ -4295,7 +4333,7 @@ conninfo_array_parse(const char *const *
}
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
{
PQconninfoFree(dbname_options);
@@ -4485,7 +4523,7 @@ conninfo_uri_parse(const char *uri, PQEx
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
return NULL;
@@ -4985,6 +5023,24 @@ conninfo_getval(PQconninfoOption *connOp
}
/*
+ * Set an option value corresponding to the keyword in the connOptions array.
+ */
+static void
+conninfo_setval(PQconninfoOption *connOptions, const char *keyword,
+ const char *val)
+{
+ PQconninfoOption *option;
+
+ option = conninfo_find(connOptions, keyword);
+ if (option)
+ {
+ if (option->val)
+ free(option->val);
+ option->val = val ? strdup(val) : NULL;
+ }
+}
+
+/*
* Store a (new) value for an option corresponding to the keyword in
* connOptions array.
*
@@ -5066,6 +5122,50 @@ conninfo_find(PQconninfoOption *connOpti
}
+/*
+ * Return the connection options used for the connections
+ */
+PQconninfoOption *
+PQconninfo(PGconn *conn, bool for_replication)
+{
+ PQExpBufferData errorBuf;
+ PQconninfoOption *connOptions;
+
+ if (conn == NULL)
+ return NULL;
+
+ /* We don't actually report any errors here, but callees want a buffer */
+ initPQExpBuffer(&errorBuf);
+ if (PQExpBufferDataBroken(errorBuf))
+ return NULL; /* out of memory already :-( */
+
+ connOptions = conninfo_init(&errorBuf, for_replication);
+
+ termPQExpBuffer(&errorBuf);
+
+ if (connOptions != NULL)
+ {
+ const PQconninfoMapping *mapping;
+
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
+ {
+ char **memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ if (mapping->conninfoValue && mapping->connValue)
+ {
+ size_t len = strlen(mapping->connValue);
+ if (*memberaddr && strncmp(*memberaddr, mapping->connValue, len) == 0)
+ conninfo_setval(connOptions, mapping->keyword, mapping->conninfoValue);
+ }
+ else
+ conninfo_setval(connOptions, mapping->keyword, *memberaddr);
+ }
+ }
+
+ return connOptions;
+}
+
+
void
PQconninfoFree(PQconninfoOption *connOptions)
{
diff -durpN postgresql/src/interfaces/libpq/libpq-fe.h postgresql.1/src/interfaces/libpq/libpq-fe.h
--- postgresql/src/interfaces/libpq/libpq-fe.h 2012-08-03 09:39:30.122266626 +0200
+++ postgresql.1/src/interfaces/libpq/libpq-fe.h 2012-10-04 17:56:58.126076978 +0200
@@ -262,6 +262,9 @@ extern PQconninfoOption *PQconndefaults(
/* parse connection options in same way as PQconnectdb */
extern PQconninfoOption *PQconninfoParse(const char *conninfo, char **errmsg);
+/* return the connection options used by a live connection */
+extern PQconninfoOption *PQconninfo(PGconn *conn, bool for_replication);
+
/* free the data structure returned by PQconndefaults() or PQconninfoParse() */
extern void PQconninfoFree(PQconninfoOption *connOptions);
02-pg_basebackup-v6.patchtext/x-patch; name=02-pg_basebackup-v6.patchDownload
diff -durpN postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml
--- postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml 2012-08-24 09:49:22.960530329 +0200
+++ postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml 2012-10-04 17:32:59.880177443 +0200
@@ -189,6 +189,27 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--write-recovery-conf</option></term>
+ <listitem>
+ <para>
+ Write a minimal recovery.conf into the output directory using
+ the connection parameters from the command line to ease
+ setting up the standby. Since creating a backup for a standalone
+ server with <option>-x</option> or <option>-X</option> and adding
+ a recovery.conf to it wouldn't make a working standby, these options
+ naturally conflict.
+ </para>
+ <para>
+ When this option is specified and taking the base backup succeeded,
+ failing to write the <filename>recovery.conf</filename> results
+ in the error code 2. This way, scripts can distinguish between different
+ failure cases of <application>pg_basebackup</application>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-x</option></term>
<term><option>--xlog</option></term>
<listitem>
diff -durpN postgresql.1/src/bin/pg_basebackup/pg_basebackup.c postgresql.2/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.1/src/bin/pg_basebackup/pg_basebackup.c 2012-10-03 10:40:48.297207389 +0200
+++ postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2012-10-04 17:34:59.462033508 +0200
@@ -46,6 +46,7 @@ int compresslevel = 0;
bool includewal = false;
bool streamwal = false;
bool fastcheckpoint = false;
+bool writerecoveryconf = false;
int standby_message_timeout = 10 * 1000; /* 10 sec = default */
/* Progress counters */
@@ -70,14 +71,18 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
+static PQconninfoOption *connOptions = NULL;
+
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
static void progress_report(int tablespacenum, const char *filename);
+static void stderr_write_error(FILE *cf, char *filename);
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
static void BaseBackup(void);
+static void WriteRecoveryConf(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
bool segment_finished);
@@ -107,6 +112,8 @@ usage(void)
printf(_("\nOptions controlling the output:\n"));
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
+ printf(_(" -R, --write-recovery-conf\n"
+ " write recovery.conf after backup\n"));
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
printf(_(" -X, --xlog-method=fetch|stream\n"
" include required WAL files with specified method\n"));
@@ -960,6 +967,10 @@ BaseBackup(void)
/* Error message already written in GetConnection() */
exit(1);
+ /* If recovery.conf is to be written, keep the connection parameters for later usage */
+ if (writerecoveryconf)
+ connOptions = PQconninfo(conn, true);
+
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
@@ -1234,6 +1245,71 @@ BaseBackup(void)
}
+static void
+stderr_write_error(FILE *cf, char *filename)
+{
+ fprintf(stderr, _("cannot write to %s: %s"), filename, strerror(errno));
+ fclose(cf);
+ unlink(filename);
+ exit(2);
+}
+
+
+/*
+ * Attempt to create recovery.conf and write the expected contents to it.
+ */
+static void
+WriteRecoveryConf(void)
+{
+ char filename[MAXPGPATH];
+ PQconninfoOption *option;
+ FILE *cf;
+
+ if (!writerecoveryconf)
+ return;
+
+ sprintf(filename, "%s/recovery.conf", basedir);
+
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("cannot create %s: %s"), filename, strerror(errno));
+ exit(2);
+ }
+
+ if (fprintf(cf, "standby_mode = 'on'\n") < 0)
+ stderr_write_error(cf, filename);
+
+ if (fprintf(cf, "primary_conninfo = '") < 0)
+ stderr_write_error(cf, filename);
+
+ for (option = connOptions; option && option->keyword; option++)
+ {
+ /*
+ * Do not emit this setting if not set, empty or default.
+ * The list of options was already pre-filtered for options
+ * usable for replication with PQconninfo(conn, true).
+ */
+ if (option->val == NULL ||
+ (option->val != NULL && option->val[0] == '\0') ||
+ (option->val &&
+ option->compiled &&
+ strcmp(option->val, option->compiled) == 0))
+ continue;
+
+ /* write "keyword='value'" pieces, single quotes doubled */
+ if (fprintf(cf, "%s=''%s'' ", option->keyword, option->val) < 0)
+ stderr_write_error(cf, filename);
+ }
+
+ if (fprintf(cf, "'\n") < 0)
+ stderr_write_error(cf, filename);
+
+ PQconninfoFree(connOptions);
+
+ fclose(cf);
+}
+
int
main(int argc, char **argv)
{
@@ -1243,6 +1319,7 @@ main(int argc, char **argv)
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
+ {"write-recovery-conf", no_argument, NULL, 'R'},
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
{"gzip", no_argument, NULL, 'z'},
@@ -1280,7 +1357,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:xX:l:zZ:c:h:p:U:s:wWvP",
+ while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
long_options, &option_index)) != -1)
{
switch (c)
@@ -1301,6 +1378,9 @@ main(int argc, char **argv)
exit(1);
}
break;
+ case 'R':
+ writerecoveryconf = true;
+ break;
case 'x':
if (includewal)
{
@@ -1466,6 +1546,13 @@ main(int argc, char **argv)
}
#endif
+ if (writerecoveryconf && includewal)
+ {
+ fprintf(stderr,
+ _("--xlog and --writerecoveryconf are mutually exclusive\n"));
+ exit(1);
+ }
+
/*
* Verify that the target directory exists, or create it. For plaintext
* backups, always require the directory. For tar backups, require it
@@ -1476,5 +1563,7 @@ main(int argc, char **argv)
BaseBackup();
+ WriteRecoveryConf();
+
return 0;
}
On Thursday, October 04, 2012 9:50 PM Boszormenyi Zoltan
2012-10-04 16:43 keltezéssel, Tom Lane írta:
Boszormenyi Zoltan <zb@cybertec.at> writes:
Did you think about something like the attached code?
Or rather this one, which fixes a bug so fillPGconn() and
PQconninfo() are symmetric and work for "requiressl".That's incredibly ugly. I'm not sure where we should keep the "R"
information, but shoehorning it into the existing PQconninfoOption
struct like that seems totally unacceptable. Either we're willing to
break backwards compatibility on the Disp-Char strings, or we need to
put that info somewhere else.I hope only handling the new flag part is ugly. It can be hidden in the
PQconninfoMapping[] array and PQconninfo(conn, true) pre-filters the
list as in the attached version.
Please find the review of the patch.
Basic stuff:
------------
- patch apply failed at exports.txt file. Needs rebase to the current
master.
- Compiles cleanly with no warnings
What it does:
--------------
pg_basebackup does the base backup from the primary machine and also writes
the recovery.conf file with the option -R,
which is used for the standby to connect to primary for streaming
replication.
Testing:
---------
1. Test pg_basebackup with option -R to check that the recovery.conf file is
written to data directory.
--recovery.conf file is created in data directory.
2. Test pg_basebackup with option -R to check that the recovery.conf file is
not able to create because of disk full.
--Error is given as recovery.conf file is not able to create.
3. Test pg_basebackup with option -R including -h, -U, -p, -w and -W.
verify the recovery.conf which is created in data directory.
--All the primary connection info parameters are working fine.
4. Test pg_basebackup with conflict options (-x or -X and -R).
--Error is given when the conflict options are provided to
pg_basebackup.
5. Test pg_basebackup with option -R from a standby server.
--recovery.conf file is created in data directory.
Code Review:
-------------
1.
typedef struct PQconninfoMapping {
+ char *keyword;
+ size_t member_offset;
+ bool for_replication;
+ char *conninfoValue; /* Special value mapping
*/
+ char *connValue;
+} PQconninfoMapping;
Provide the better explanation of conninfoValue and connValue, how and where
these are used?
2. if (tmp && strncmp(tmp, mapping->conninfoValue, len) == 0)
In any case if the above condition is not satisfied then the PGconn data is
not filled with PQconninfoOption.
Is it correct?
Documentation:
-------------
1.
+ <para>
+ The <literal>for_replication</> argument can be used to exclude some
+ options from the list which are used by the walreceiver module.
+ <application>pg_basebackup</application> uses this pre-filtered list
+ to construct <literal>primary_conninfo</> in the automatically
generated
+ recovery.conf file.
+ </para>
I feel the explanation should be as follows,
exclude some options from the list which are not used by the walreceiver
module?
Observations/Issues:
-------------------
1. If the password contains any escape sequence characters, It is leading to
problems while walreceiver connecting to primary
by using the primary conninfo from recovery.conf
please log an warning message or a note in document to handle such a case
manually by the user.
With Regards,
Amit Kapila.
Hi,
thanks for the new review.
2012-10-10 08:58 keltezéssel, Amit Kapila írta:
On Thursday, October 04, 2012 9:50 PM Boszormenyi Zoltan
2012-10-04 16:43 keltezéssel, Tom Lane írta:
Boszormenyi Zoltan <zb@cybertec.at> writes:
Did you think about something like the attached code?
Or rather this one, which fixes a bug so fillPGconn() and
PQconninfo() are symmetric and work for "requiressl".That's incredibly ugly. I'm not sure where we should keep the "R"
information, but shoehorning it into the existing PQconninfoOption
struct like that seems totally unacceptable. Either we're willing to
break backwards compatibility on the Disp-Char strings, or we need to
put that info somewhere else.I hope only handling the new flag part is ugly. It can be hidden in the
PQconninfoMapping[] array and PQconninfo(conn, true) pre-filters the
list as in the attached version.Please find the review of the patch.
Basic stuff:
------------
- patch apply failed at exports.txt file. Needs rebase to the current
master.
Done.
- Compiles cleanly with no warnings
What it does:
--------------
pg_basebackup does the base backup from the primary machine and also writes
the recovery.conf file with the option -R,
which is used for the standby to connect to primary for streaming
replication.Testing:
---------
1. Test pg_basebackup with option -R to check that the recovery.conf file is
written to data directory.
--recovery.conf file is created in data directory.2. Test pg_basebackup with option -R to check that the recovery.conf file is
not able to create because of disk full.
--Error is given as recovery.conf file is not able to create.3. Test pg_basebackup with option -R including -h, -U, -p, -w and -W.
verify the recovery.conf which is created in data directory.
--All the primary connection info parameters are working fine.4. Test pg_basebackup with conflict options (-x or -X and -R).
--Error is given when the conflict options are provided to
pg_basebackup.5. Test pg_basebackup with option -R from a standby server.
--recovery.conf file is created in data directory.Code Review: ------------- 1. typedef struct PQconninfoMapping { + char *keyword; + size_t member_offset; + bool for_replication; + char *conninfoValue; /* Special value mapping */ + char *connValue; +} PQconninfoMapping;Provide the better explanation of conninfoValue and connValue, how and where
these are used?
OK. This is only used for " requiressl='1' " (in the connection string)
and if '1' is set, PGconn.sslmode will be set to "require". All other
values are treated as it's being unset. This simplistic mapping
is used because there is only one such setting where different values
are used on the conninfo and the PGconn sides.
2. if (tmp && strncmp(tmp, mapping->conninfoValue, len) == 0)
In any case if the above condition is not satisfied then the PGconn data is
not filled with PQconninfoOption.
Is it correct?
Yes, it stays NULL as makeEmptyPGconn() initialized it. This case only happens
with the "requiressl" setting with its special mapping. If you set " requiressl = '0' "
then it means that " requiressl='1' " was not set so the PGconn side stays as NULL.
The special casing was there in the old code too and behaves the same.
Documentation: ------------- 1. + <para> + The <literal>for_replication</> argument can be used to exclude some+ options from the list which are used by the walreceiver module. + <application>pg_basebackup</application> uses this pre-filtered list+ to construct <literal>primary_conninfo</> in the automatically generated + recovery.conf file. + </para>I feel the explanation should be as follows,
exclude some options from the list which are not used by the walreceiver
module?
Err, no. The list excludes those[1]Actually, more than these 3 options are excluded. The deprecated ones are also excluded. that *are* used (would be
overridden) by the walreceiver module:
----8<--------8<--------8<--------8<--------8<----
static bool
libpqrcv_connect(char *conninfo, XLogRecPtr startpoint)
{
...
snprintf(conninfo_repl, sizeof(conninfo_repl),
"%s dbname=replication replication=true
fallback_application_name=walreceiver",
conninfo);
----8<--------8<--------8<--------8<--------8<----
[1]: Actually, more than these 3 options are excluded. The deprecated ones are also excluded.
ones are also excluded.
Observations/Issues:
-------------------
1. If the password contains any escape sequence characters, It is leading to
problems while walreceiver connecting to primary
by using the primary conninfo from recovery.confplease log an warning message or a note in document to handle such a case
manually by the user.
Done at both places.
Also, to adapt to the style of other error messages, now all my fprintf() statements
are prefixed with: "%s: ...", progname.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
01-PQconninfo-v11.patchtext/x-patch; name=01-PQconninfo-v11.patchDownload
diff -durpN postgresql/doc/src/sgml/libpq.sgml postgresql.1/doc/src/sgml/libpq.sgml
--- postgresql/doc/src/sgml/libpq.sgml 2012-08-03 09:39:30.114266570 +0200
+++ postgresql.1/doc/src/sgml/libpq.sgml 2012-10-10 14:09:31.924890230 +0200
@@ -496,6 +496,37 @@ typedef struct
</listitem>
</varlistentry>
+ <varlistentry id="libpq-pqconninfo">
+ <term><function>PQconninfo</function><indexterm><primary>PQconninfo</></></term>
+ <listitem>
+ <para>
+ Returns the connection options used by a live connection.
+<synopsis>
+PQconninfoOption *PQconninfo(PGconn *conn, bool for_replication);
+</synopsis>
+ </para>
+
+ <para>
+ Returns a connection options array. This can be used to determine
+ all possible <function>PQconnectdb</function> options and their
+ current values that were used to connect to the server. The return
+ value points to an array of <structname>PQconninfoOption</structname>
+ structures, which ends with an entry having a null <structfield>keyword</>
+ pointer. Every notes above for <function>PQconndefaults</function> also apply.
+ </para>
+
+ <para>
+ The <literal>for_replication</> argument can be used to exclude some
+ options from the list which are used by the walreceiver module.
+ <application>pg_basebackup</application> uses this pre-filtered list
+ to construct <literal>primary_conninfo</> in the automatically generated
+ recovery.conf file.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+
<varlistentry id="libpq-pqconninfoparse">
<term><function>PQconninfoParse</function><indexterm><primary>PQconninfoParse</></></term>
<listitem>
diff -durpN postgresql/src/interfaces/libpq/exports.txt postgresql.1/src/interfaces/libpq/exports.txt
--- postgresql/src/interfaces/libpq/exports.txt 2012-10-09 09:58:14.342782974 +0200
+++ postgresql.1/src/interfaces/libpq/exports.txt 2012-10-10 14:09:56.052057835 +0200
@@ -164,3 +164,4 @@ PQsetSingleRowMode 161
lo_lseek64 162
lo_tell64 163
lo_truncate64 164
+PQconninfo 165
diff -durpN postgresql/src/interfaces/libpq/fe-connect.c postgresql.1/src/interfaces/libpq/fe-connect.c
--- postgresql/src/interfaces/libpq/fe-connect.c 2012-09-09 08:11:09.470401480 +0200
+++ postgresql.1/src/interfaces/libpq/fe-connect.c 2012-10-10 15:05:01.715399605 +0200
@@ -137,6 +137,9 @@ static int ldapServiceLookup(const char
* PQconninfoOptions[] *must* be NULL. In a working copy, non-null "val"
* fields point to malloc'd strings that should be freed when the working
* array is freed (see PQconninfoFree).
+ *
+ * If you add a new connection option to this list, remember to add it to
+ * PQconninfoMappings[] below.
* ----------
*/
static const PQconninfoOption PQconninfoOptions[] = {
@@ -264,6 +267,66 @@ static const PQconninfoOption PQconninfo
NULL, NULL, 0}
};
+/*
+ * We need a mapping between the PQconninfoOptions[] array
+ * and PGconn members. We have to keep it separate from
+ * PQconninfoOptions[] to not leak info about PGconn members
+ * to clients.
+ */
+typedef struct PQconninfoMapping {
+ char *keyword;
+ size_t member_offset;
+ bool for_replication;
+ /*
+ * Special and simplistic value mapping between
+ * PQconninfoOption and PGconn. Only used by "requiressl".
+ */
+ char *conninfoValue;
+ char *connValue;
+} PQconninfoMapping;
+#define PGCONNMEMBERADDR(conn, mapping) ((char **)((char *)conn + mapping->member_offset))
+
+static const PQconninfoMapping PQconninfoMappings[] =
+{
+ /* "authtype" is not used anymore, there is no mapping to PGconn */
+ /* there is no mapping of "service" to PGconn */
+ { "user", offsetof(struct pg_conn, pguser), true, NULL, NULL },
+ { "password", offsetof(struct pg_conn, pgpass), true, NULL, NULL },
+ { "connect_timeout", offsetof(struct pg_conn, connect_timeout), true, NULL, NULL },
+ { "dbname", offsetof(struct pg_conn, dbName), false, NULL, NULL },
+ { "host", offsetof(struct pg_conn, pghost), true, NULL, NULL },
+ { "hostaddr", offsetof(struct pg_conn, pghostaddr), true, NULL, NULL },
+ { "port", offsetof(struct pg_conn, pgport), true, NULL, NULL },
+ { "client_encoding", offsetof(struct pg_conn, client_encoding_initial), false, NULL, NULL },
+ { "tty", offsetof(struct pg_conn, pgtty), false, NULL, NULL },
+ { "options", offsetof(struct pg_conn, pgoptions), true, NULL, NULL },
+ { "application_name", offsetof(struct pg_conn, appname), false, NULL, NULL },
+ { "fallback_application_name", offsetof(struct pg_conn, fbappname), false, NULL, NULL },
+ { "keepalives", offsetof(struct pg_conn, keepalives), true, NULL, NULL },
+ { "keepalives_idle", offsetof(struct pg_conn, keepalives_idle), true, NULL, NULL },
+ { "keepalives_interval", offsetof(struct pg_conn, keepalives_interval), true, NULL, NULL },
+ { "keepalives_count", offsetof(struct pg_conn, keepalives_count), true, NULL, NULL },
+#ifdef USE_SSL
+ { "requiressl", offsetof(struct pg_conn, sslmode), false, "1", "require" },
+#endif
+ { "sslmode", offsetof(struct pg_conn, sslmode), true, NULL, NULL },
+ { "sslcompression", offsetof(struct pg_conn, sslcompression), true, NULL, NULL },
+ { "sslcert", offsetof(struct pg_conn, sslcert), true, NULL, NULL },
+ { "sslkey", offsetof(struct pg_conn, sslkey), true, NULL, NULL },
+ { "sslrootcert", offsetof(struct pg_conn, sslrootcert), true, NULL, NULL },
+ { "sslcrl", offsetof(struct pg_conn, sslcrl), true, NULL, NULL },
+ { "requirepeer", offsetof(struct pg_conn, requirepeer), true, NULL, NULL },
+#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
+ { "krbsrvname", offsetof(struct pg_conn, krbsrvname), true, NULL, NULL },
+#endif
+#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
+ { "gsslib", offsetof(struct pg_conn, requirepeer), true, NULL, NULL },
+#endif
+ { "replication", offsetof(struct pg_conn, replication), false, NULL, NULL },
+ /* Terminating entry --- MUST BE LAST */
+ { NULL, 0, false, NULL, NULL }
+};
+
static const PQEnvironmentOption EnvironmentOptions[] =
{
/* common user-interface settings */
@@ -295,7 +358,8 @@ static PGconn *makeEmptyPGconn(void);
static void fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
-static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage);
+static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage,
+ bool for_replication);
static PQconninfoOption *parse_connection_string(const char *conninfo,
PQExpBuffer errorMessage, bool use_defaults);
static int uri_prefix_length(const char *connstr);
@@ -318,6 +382,8 @@ static char *conninfo_uri_decode(const c
static bool get_hexdigit(char digit, int *value);
static const char *conninfo_getval(PQconninfoOption *connOptions,
const char *keyword);
+static void conninfo_setval(PQconninfoOption *connOptions,
+ const char *keyword, const char *val);
static PQconninfoOption *conninfo_storeval(PQconninfoOption *connOptions,
const char *keyword, const char *value,
PQExpBuffer errorMessage, bool ignoreMissing, bool uri_decode);
@@ -627,7 +693,9 @@ PQconnectStart(const char *conninfo)
static void
fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
{
+ const PQconninfoMapping *mapping;
const char *tmp;
+ char **memberaddr;
/*
* Move option values into conn structure
@@ -637,72 +705,24 @@ fillPGconn(PGconn *conn, PQconninfoOptio
*
* XXX: probably worth checking strdup() return value here...
*/
- tmp = conninfo_getval(connOptions, "hostaddr");
- conn->pghostaddr = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "host");
- conn->pghost = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "port");
- conn->pgport = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "tty");
- conn->pgtty = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "options");
- conn->pgoptions = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "application_name");
- conn->appname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "fallback_application_name");
- conn->fbappname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "dbname");
- conn->dbName = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "user");
- conn->pguser = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "password");
- conn->pgpass = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "connect_timeout");
- conn->connect_timeout = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "client_encoding");
- conn->client_encoding_initial = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives");
- conn->keepalives = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_idle");
- conn->keepalives_idle = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_interval");
- conn->keepalives_interval = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_count");
- conn->keepalives_count = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslmode");
- conn->sslmode = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcompression");
- conn->sslcompression = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslkey");
- conn->sslkey = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcert");
- conn->sslcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslrootcert");
- conn->sslrootcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcrl");
- conn->sslcrl = tmp ? strdup(tmp) : NULL;
-#ifdef USE_SSL
- tmp = conninfo_getval(connOptions, "requiressl");
- if (tmp && tmp[0] == '1')
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
{
- /* here warn that the requiressl option is deprecated? */
- if (conn->sslmode)
- free(conn->sslmode);
- conn->sslmode = strdup("require");
+ tmp = conninfo_getval(connOptions, mapping->keyword);
+ memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ if (mapping->conninfoValue && mapping->connValue)
+ {
+ size_t len = strlen(mapping->conninfoValue);
+ if (tmp && strncmp(tmp, mapping->conninfoValue, len) == 0)
+ {
+ if (*memberaddr)
+ free(*memberaddr);
+ *memberaddr = strdup(mapping->connValue);
+ }
+ }
+ else
+ *memberaddr = tmp ? strdup(tmp) : NULL;
}
-#endif
- tmp = conninfo_getval(connOptions, "requirepeer");
- conn->requirepeer = tmp ? strdup(tmp) : NULL;
-#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "krbsrvname");
- conn->krbsrvname = tmp ? strdup(tmp) : NULL;
-#endif
-#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "gsslib");
- conn->gsslib = tmp ? strdup(tmp) : NULL;
-#endif
- tmp = conninfo_getval(connOptions, "replication");
- conn->replication = tmp ? strdup(tmp) : NULL;
}
/*
@@ -884,7 +904,7 @@ PQconndefaults(void)
if (PQExpBufferDataBroken(errorBuf))
return NULL; /* out of memory already :-( */
- connOptions = conninfo_init(&errorBuf);
+ connOptions = conninfo_init(&errorBuf, false);
if (connOptions != NULL)
{
if (!conninfo_add_defaults(connOptions, &errorBuf))
@@ -4006,9 +4026,11 @@ PQconninfoParse(const char *conninfo, ch
/*
* Build a working copy of the constant PQconninfoOptions array.
+ * If for_replication is true, only return the options that are
+ * not added by libpqwalreceiver.
*/
static PQconninfoOption *
-conninfo_init(PQExpBuffer errorMessage)
+conninfo_init(PQExpBuffer errorMessage, bool for_replication)
{
PQconninfoOption *options;
@@ -4019,7 +4041,27 @@ conninfo_init(PQExpBuffer errorMessage)
libpq_gettext("out of memory\n"));
return NULL;
}
- memcpy(options, PQconninfoOptions, sizeof(PQconninfoOptions));
+ if (for_replication)
+ {
+ const PQconninfoMapping *mapping = PQconninfoMappings;
+ PQconninfoOption *opt_dest = options;
+
+ while (mapping->keyword)
+ {
+ PQconninfoOption *opt_src = conninfo_find((PQconninfoOption *)PQconninfoOptions, mapping->keyword);
+
+ if (opt_src && mapping->for_replication)
+ {
+ memcpy(opt_dest, opt_src, sizeof(PQconninfoOption));
+ opt_dest++;
+ }
+
+ opt_src++;
+ mapping++;
+ }
+ }
+ else
+ memcpy(options, PQconninfoOptions, sizeof(PQconninfoOptions));
return options;
}
@@ -4095,7 +4137,7 @@ conninfo_parse(const char *conninfo, PQE
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
return NULL;
@@ -4295,7 +4337,7 @@ conninfo_array_parse(const char *const *
}
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
{
PQconninfoFree(dbname_options);
@@ -4485,7 +4527,7 @@ conninfo_uri_parse(const char *uri, PQEx
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
return NULL;
@@ -4985,6 +5027,24 @@ conninfo_getval(PQconninfoOption *connOp
}
/*
+ * Set an option value corresponding to the keyword in the connOptions array.
+ */
+static void
+conninfo_setval(PQconninfoOption *connOptions, const char *keyword,
+ const char *val)
+{
+ PQconninfoOption *option;
+
+ option = conninfo_find(connOptions, keyword);
+ if (option)
+ {
+ if (option->val)
+ free(option->val);
+ option->val = val ? strdup(val) : NULL;
+ }
+}
+
+/*
* Store a (new) value for an option corresponding to the keyword in
* connOptions array.
*
@@ -5066,6 +5126,50 @@ conninfo_find(PQconninfoOption *connOpti
}
+/*
+ * Return the connection options used for the connections
+ */
+PQconninfoOption *
+PQconninfo(PGconn *conn, bool for_replication)
+{
+ PQExpBufferData errorBuf;
+ PQconninfoOption *connOptions;
+
+ if (conn == NULL)
+ return NULL;
+
+ /* We don't actually report any errors here, but callees want a buffer */
+ initPQExpBuffer(&errorBuf);
+ if (PQExpBufferDataBroken(errorBuf))
+ return NULL; /* out of memory already :-( */
+
+ connOptions = conninfo_init(&errorBuf, for_replication);
+
+ termPQExpBuffer(&errorBuf);
+
+ if (connOptions != NULL)
+ {
+ const PQconninfoMapping *mapping;
+
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
+ {
+ char **memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ if (mapping->conninfoValue && mapping->connValue)
+ {
+ size_t len = strlen(mapping->connValue);
+ if (*memberaddr && strncmp(*memberaddr, mapping->connValue, len) == 0)
+ conninfo_setval(connOptions, mapping->keyword, mapping->conninfoValue);
+ }
+ else
+ conninfo_setval(connOptions, mapping->keyword, *memberaddr);
+ }
+ }
+
+ return connOptions;
+}
+
+
void
PQconninfoFree(PQconninfoOption *connOptions)
{
diff -durpN postgresql/src/interfaces/libpq/libpq-fe.h postgresql.1/src/interfaces/libpq/libpq-fe.h
--- postgresql/src/interfaces/libpq/libpq-fe.h 2012-10-09 09:58:14.343782980 +0200
+++ postgresql.1/src/interfaces/libpq/libpq-fe.h 2012-10-10 14:09:31.936890314 +0200
@@ -262,6 +262,9 @@ extern PQconninfoOption *PQconndefaults(
/* parse connection options in same way as PQconnectdb */
extern PQconninfoOption *PQconninfoParse(const char *conninfo, char **errmsg);
+/* return the connection options used by a live connection */
+extern PQconninfoOption *PQconninfo(PGconn *conn, bool for_replication);
+
/* free the data structure returned by PQconndefaults() or PQconninfoParse() */
extern void PQconninfoFree(PQconninfoOption *connOptions);
02-pg_basebackup-v8.patchtext/x-patch; name=02-pg_basebackup-v8.patchDownload
diff -durpN postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml
--- postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml 2012-08-24 09:49:22.960530329 +0200
+++ postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml 2012-10-10 14:49:40.869336237 +0200
@@ -189,6 +189,36 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--write-recovery-conf</option></term>
+ <listitem>
+
+ <para>
+ Write a minimal recovery.conf into the output directory using
+ the connection parameters from the command line to ease
+ setting up the standby. Since creating a backup for a standalone
+ server with <option>-x</option> or <option>-X</option> and adding
+ a recovery.conf to it wouldn't make a working standby, these options
+ naturally conflict.
+ </para>
+
+ <para>
+ The password written into recovery.conf is not escaped even if special
+ characters appear in it. The administrator must review recovery.conf
+ to ensure proper escaping.
+ </para>
+
+ <para>
+ When this option is specified and taking the base backup succeeded,
+ failing to write the <filename>recovery.conf</filename> results
+ in the error code 2. This way, scripts can distinguish between different
+ failure cases of <application>pg_basebackup</application>.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-x</option></term>
<term><option>--xlog</option></term>
<listitem>
diff -durpN postgresql.1/src/bin/pg_basebackup/pg_basebackup.c postgresql.2/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.1/src/bin/pg_basebackup/pg_basebackup.c 2012-10-03 10:40:48.297207389 +0200
+++ postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2012-10-10 14:53:18.717764755 +0200
@@ -46,6 +46,7 @@ int compresslevel = 0;
bool includewal = false;
bool streamwal = false;
bool fastcheckpoint = false;
+bool writerecoveryconf = false;
int standby_message_timeout = 10 * 1000; /* 10 sec = default */
/* Progress counters */
@@ -70,14 +71,18 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
+static PQconninfoOption *connOptions = NULL;
+
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
static void progress_report(int tablespacenum, const char *filename);
+static void stderr_write_error(FILE *cf, char *filename);
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
static void BaseBackup(void);
+static void WriteRecoveryConf(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
bool segment_finished);
@@ -107,6 +112,8 @@ usage(void)
printf(_("\nOptions controlling the output:\n"));
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
+ printf(_(" -R, --write-recovery-conf\n"
+ " write recovery.conf after backup\n"));
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
printf(_(" -X, --xlog-method=fetch|stream\n"
" include required WAL files with specified method\n"));
@@ -960,6 +967,10 @@ BaseBackup(void)
/* Error message already written in GetConnection() */
exit(1);
+ /* If recovery.conf is to be written, keep the connection parameters for later usage */
+ if (writerecoveryconf)
+ connOptions = PQconninfo(conn, true);
+
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
@@ -1234,6 +1245,73 @@ BaseBackup(void)
}
+static void
+stderr_write_error(FILE *cf, char *filename)
+{
+ fprintf(stderr, _("%s: cannot write to %s: %s"), progname, filename, strerror(errno));
+ fclose(cf);
+ unlink(filename);
+ exit(2);
+}
+
+
+/*
+ * Attempt to create recovery.conf and write the expected contents to it.
+ */
+static void
+WriteRecoveryConf(void)
+{
+ char filename[MAXPGPATH];
+ PQconninfoOption *option;
+ FILE *cf;
+
+ if (!writerecoveryconf)
+ return;
+
+ sprintf(filename, "%s/recovery.conf", basedir);
+
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("%s: cannot create %s: %s"), progname, filename, strerror(errno));
+ exit(2);
+ }
+
+ if (fprintf(cf, "standby_mode = 'on'\n") < 0)
+ stderr_write_error(cf, filename);
+
+ if (fprintf(cf, "primary_conninfo = '") < 0)
+ stderr_write_error(cf, filename);
+
+ for (option = connOptions; option && option->keyword; option++)
+ {
+ /*
+ * Do not emit this setting if not set, empty or default.
+ * The list of options was already pre-filtered for options
+ * usable for replication with PQconninfo(conn, true).
+ */
+ if (option->val == NULL ||
+ (option->val != NULL && option->val[0] == '\0') ||
+ (option->val &&
+ option->compiled &&
+ strcmp(option->val, option->compiled) == 0))
+ continue;
+
+ /* write "keyword='value'" pieces, single quotes doubled */
+ if (fprintf(cf, "%s=''%s'' ", option->keyword, option->val) < 0)
+ stderr_write_error(cf, filename);
+ }
+
+ if (fprintf(cf, "'\n") < 0)
+ stderr_write_error(cf, filename);
+
+ PQconninfoFree(connOptions);
+
+ fclose(cf);
+
+ fprintf(stderr, _("%s: recovery.conf written. Please, add escaping to the password field in \"primary_conninfo\" if needed.\n"), progname);
+}
+
int
main(int argc, char **argv)
{
@@ -1243,6 +1321,7 @@ main(int argc, char **argv)
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
+ {"write-recovery-conf", no_argument, NULL, 'R'},
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
{"gzip", no_argument, NULL, 'z'},
@@ -1280,7 +1359,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:xX:l:zZ:c:h:p:U:s:wWvP",
+ while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
long_options, &option_index)) != -1)
{
switch (c)
@@ -1301,6 +1380,9 @@ main(int argc, char **argv)
exit(1);
}
break;
+ case 'R':
+ writerecoveryconf = true;
+ break;
case 'x':
if (includewal)
{
@@ -1466,6 +1548,13 @@ main(int argc, char **argv)
}
#endif
+ if (writerecoveryconf && includewal)
+ {
+ fprintf(stderr,
+ _("--xlog and --writerecoveryconf are mutually exclusive\n"));
+ exit(1);
+ }
+
/*
* Verify that the target directory exists, or create it. For plaintext
* backups, always require the directory. For tar backups, require it
@@ -1476,5 +1565,7 @@ main(int argc, char **argv)
BaseBackup();
+ WriteRecoveryConf();
+
return 0;
}
On Wed, Oct 10, 2012 at 10:12 PM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
Hi,
thanks for the new review.
2012-10-10 08:58 keltezéssel, Amit Kapila írta:
On Thursday, October 04, 2012 9:50 PM Boszormenyi Zoltan
2012-10-04 16:43 keltezéssel, Tom Lane írta:
Boszormenyi Zoltan <zb@cybertec.at> writes:
Did you think about something like the attached code?
Or rather this one, which fixes a bug so fillPGconn() and
PQconninfo() are symmetric and work for "requiressl".That's incredibly ugly. I'm not sure where we should keep the "R"
information, but shoehorning it into the existing PQconninfoOption
struct like that seems totally unacceptable. Either we're willing to
break backwards compatibility on the Disp-Char strings, or we need to
put that info somewhere else.I hope only handling the new flag part is ugly. It can be hidden in the
PQconninfoMapping[] array and PQconninfo(conn, true) pre-filters the
list as in the attached version.Please find the review of the patch.
Basic stuff:
------------
- patch apply failed at exports.txt file. Needs rebase to the current
master.Done.
- Compiles cleanly with no warnings
What it does:
--------------
pg_basebackup does the base backup from the primary machine and also
writes
the recovery.conf file with the option -R,
which is used for the standby to connect to primary for streaming
replication.Testing:
---------
1. Test pg_basebackup with option -R to check that the recovery.conf file
is
written to data directory.
--recovery.conf file is created in data directory.
2. Test pg_basebackup with option -R to check that the recovery.conf
file is
not able to create because of disk full.
--Error is given as recovery.conf file is not able to create.
3. Test pg_basebackup with option -R including -h, -U, -p, -w and -W.verify the recovery.conf which is created in data directory.
--All the primary connection info parameters are working fine.
4. Test pg_basebackup with conflict options (-x or -X and -R).--Error is given when the conflict options are provided to
pg_basebackup.5. Test pg_basebackup with option -R from a standby server.
--recovery.conf file is created in data directory.Code Review: ------------- 1. typedef struct PQconninfoMapping { + char *keyword; + size_t member_offset; + bool for_replication; + char *conninfoValue; /* Special value mapping */ + char *connValue; +} PQconninfoMapping;Provide the better explanation of conninfoValue and connValue, how and
where
these are used?OK. This is only used for " requiressl='1' " (in the connection string)
and if '1' is set, PGconn.sslmode will be set to "require". All other
values are treated as it's being unset. This simplistic mapping
is used because there is only one such setting where different values
are used on the conninfo and the PGconn sides.2. if (tmp && strncmp(tmp, mapping->conninfoValue, len) == 0)
In any case if the above condition is not satisfied then the PGconn data
is
not filled with PQconninfoOption.
Is it correct?Yes, it stays NULL as makeEmptyPGconn() initialized it. This case only
happens
with the "requiressl" setting with its special mapping. If you set "
requiressl = '0' "
then it means that " requiressl='1' " was not set so the PGconn side stays
as NULL.The special casing was there in the old code too and behaves the same.
Documentation: ------------- 1. + <para> + The <literal>for_replication</> argument can be used to exclude some+ options from the list which are used by the walreceiver module. + <application>pg_basebackup</application> uses this pre-filtered list+ to construct <literal>primary_conninfo</> in the automatically generated + recovery.conf file. + </para>I feel the explanation should be as follows,
exclude some options from the list which are not used by the walreceiver
module?Err, no. The list excludes those[1] that *are* used (would be
overridden) by the walreceiver module:----8<--------8<--------8<--------8<--------8<----
static bool
libpqrcv_connect(char *conninfo, XLogRecPtr startpoint)
{
...
snprintf(conninfo_repl, sizeof(conninfo_repl),
"%s dbname=replication replication=true
fallback_application_name=walreceiver",
conninfo);
----8<--------8<--------8<--------8<--------8<----[1] Actually, more than these 3 options are excluded. The deprecated
ones are also excluded.Observations/Issues:
-------------------
1. If the password contains any escape sequence characters, It is leading
to
problems while walreceiver connecting to primary
by using the primary conninfo from recovery.confplease log an warning message or a note in document to handle such a
case
manually by the user.Done at both places.
Also, to adapt to the style of other error messages, now all my fprintf()
statements
are prefixed with: "%s: ...", progname.
In new patches, when I ran "pg_basebackup -D hoge -c fast -R" on MacOS,
I got the following error message. BTW, I compiled the patched PostgreSQL
with --enable-debug and --enable-cassert options.
pg_basebackup(41751) malloc: *** error for object 0x106001af0: pointer
being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6
$ uname -a
Darwin hrk.local 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23
16:25:48 PDT 2012; root:xnu-1699.32.7~1/RELEASE_X86_64 x86_64
When tar output format is specified together with -R option, recovery.conf is
not included in base.tar. I think it should.
+ setting up the standby. Since creating a backup for a standalone
+ server with <option>-x</option> or <option>-X</option> and adding
+ a recovery.conf to it wouldn't make a working standby, these options
+ naturally conflict.
Is this right? ISTM that basically we can use pg_basebackup -x to take
a base backup for starting the standby for now. No?
Regards,
--
Fujii Masao
2012-10-10 18:23 keltezéssel, Fujii Masao írta:
On Wed, Oct 10, 2012 at 10:12 PM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
Hi,
thanks for the new review.
2012-10-10 08:58 keltezéssel, Amit Kapila írta:
On Thursday, October 04, 2012 9:50 PM Boszormenyi Zoltan
2012-10-04 16:43 keltezéssel, Tom Lane írta:
Boszormenyi Zoltan <zb@cybertec.at> writes:
Did you think about something like the attached code?
Or rather this one, which fixes a bug so fillPGconn() and
PQconninfo() are symmetric and work for "requiressl".That's incredibly ugly. I'm not sure where we should keep the "R"
information, but shoehorning it into the existing PQconninfoOption
struct like that seems totally unacceptable. Either we're willing to
break backwards compatibility on the Disp-Char strings, or we need to
put that info somewhere else.I hope only handling the new flag part is ugly. It can be hidden in the
PQconninfoMapping[] array and PQconninfo(conn, true) pre-filters the
list as in the attached version.Please find the review of the patch.
Basic stuff:
------------
- patch apply failed at exports.txt file. Needs rebase to the current
master.Done.
- Compiles cleanly with no warnings
What it does:
--------------
pg_basebackup does the base backup from the primary machine and also
writes
the recovery.conf file with the option -R,
which is used for the standby to connect to primary for streaming
replication.Testing:
---------
1. Test pg_basebackup with option -R to check that the recovery.conf file
is
written to data directory.
--recovery.conf file is created in data directory.
2. Test pg_basebackup with option -R to check that the recovery.conf
file is
not able to create because of disk full.
--Error is given as recovery.conf file is not able to create.
3. Test pg_basebackup with option -R including -h, -U, -p, -w and -W.verify the recovery.conf which is created in data directory.
--All the primary connection info parameters are working fine.
4. Test pg_basebackup with conflict options (-x or -X and -R).--Error is given when the conflict options are provided to
pg_basebackup.5. Test pg_basebackup with option -R from a standby server.
--recovery.conf file is created in data directory.Code Review: ------------- 1. typedef struct PQconninfoMapping { + char *keyword; + size_t member_offset; + bool for_replication; + char *conninfoValue; /* Special value mapping */ + char *connValue; +} PQconninfoMapping;Provide the better explanation of conninfoValue and connValue, how and
where
these are used?OK. This is only used for " requiressl='1' " (in the connection string)
and if '1' is set, PGconn.sslmode will be set to "require". All other
values are treated as it's being unset. This simplistic mapping
is used because there is only one such setting where different values
are used on the conninfo and the PGconn sides.2. if (tmp && strncmp(tmp, mapping->conninfoValue, len) == 0)
In any case if the above condition is not satisfied then the PGconn data
is
not filled with PQconninfoOption.
Is it correct?Yes, it stays NULL as makeEmptyPGconn() initialized it. This case only
happens
with the "requiressl" setting with its special mapping. If you set "
requiressl = '0' "
then it means that " requiressl='1' " was not set so the PGconn side stays
as NULL.The special casing was there in the old code too and behaves the same.
Documentation: ------------- 1. + <para> + The <literal>for_replication</> argument can be used to exclude some+ options from the list which are used by the walreceiver module. + <application>pg_basebackup</application> uses this pre-filtered list+ to construct <literal>primary_conninfo</> in the automatically generated + recovery.conf file. + </para>I feel the explanation should be as follows,
exclude some options from the list which are not used by the walreceiver
module?Err, no. The list excludes those[1] that *are* used (would be
overridden) by the walreceiver module:----8<--------8<--------8<--------8<--------8<----
static bool
libpqrcv_connect(char *conninfo, XLogRecPtr startpoint)
{
...
snprintf(conninfo_repl, sizeof(conninfo_repl),
"%s dbname=replication replication=true
fallback_application_name=walreceiver",
conninfo);
----8<--------8<--------8<--------8<--------8<----[1] Actually, more than these 3 options are excluded. The deprecated
ones are also excluded.Observations/Issues:
-------------------
1. If the password contains any escape sequence characters, It is leading
to
problems while walreceiver connecting to primary
by using the primary conninfo from recovery.confplease log an warning message or a note in document to handle such a
case
manually by the user.Done at both places.
Also, to adapt to the style of other error messages, now all my fprintf()
statements
are prefixed with: "%s: ...", progname.In new patches, when I ran "pg_basebackup -D hoge -c fast -R" on MacOS,
I got the following error message. BTW, I compiled the patched PostgreSQL
with --enable-debug and --enable-cassert options.pg_basebackup(41751) malloc: *** error for object 0x106001af0: pointer
being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6
Can you show a backtrace? I compiled it on Fedora 17/x86_64 with
--enable-depend --enable-debug --enable-cassert. GLIBC is also smart
enough to catch improper free() calls, too.
$ uname -a
Darwin hrk.local 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23
16:25:48 PDT 2012; root:xnu-1699.32.7~1/RELEASE_X86_64 x86_64When tar output format is specified together with -R option, recovery.conf is
not included in base.tar. I think it should.
Why? This patch only promises to write the recovery.conf into the
directory specified with -D.
+ setting up the standby. Since creating a backup for a standalone + server with <option>-x</option> or <option>-X</option> and adding + a recovery.conf to it wouldn't make a working standby, these options + naturally conflict.Is this right? ISTM that basically we can use pg_basebackup -x to take
a base backup for starting the standby for now. No?
I don't know. Pointers?
Thanks,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
2012-10-10 20:36 keltezéssel, Boszormenyi Zoltan írta:
2012-10-10 18:23 keltezéssel, Fujii Masao írta:
On Wed, Oct 10, 2012 at 10:12 PM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
Hi,
thanks for the new review.
2012-10-10 08:58 keltezéssel, Amit Kapila írta:
On Thursday, October 04, 2012 9:50 PM Boszormenyi Zoltan
2012-10-04 16:43 keltezéssel, Tom Lane írta:
Boszormenyi Zoltan <zb@cybertec.at> writes:
Did you think about something like the attached code?
Or rather this one, which fixes a bug so fillPGconn() and
PQconninfo() are symmetric and work for "requiressl".That's incredibly ugly. I'm not sure where we should keep the "R"
information, but shoehorning it into the existing PQconninfoOption
struct like that seems totally unacceptable. Either we're willing to
break backwards compatibility on the Disp-Char strings, or we need to
put that info somewhere else.I hope only handling the new flag part is ugly. It can be hidden in the
PQconninfoMapping[] array and PQconninfo(conn, true) pre-filters the
list as in the attached version.Please find the review of the patch.
Basic stuff:
------------
- patch apply failed at exports.txt file. Needs rebase to the current
master.Done.
- Compiles cleanly with no warnings
What it does:
--------------
pg_basebackup does the base backup from the primary machine and also
writes
the recovery.conf file with the option -R,
which is used for the standby to connect to primary for streaming
replication.Testing:
---------
1. Test pg_basebackup with option -R to check that the recovery.conf file
is
written to data directory.
--recovery.conf file is created in data directory.
2. Test pg_basebackup with option -R to check that the recovery.conf
file is
not able to create because of disk full.
--Error is given as recovery.conf file is not able to create.
3. Test pg_basebackup with option -R including -h, -U, -p, -w and -W.verify the recovery.conf which is created in data directory.
--All the primary connection info parameters are working fine.
4. Test pg_basebackup with conflict options (-x or -X and -R).--Error is given when the conflict options are provided to
pg_basebackup.5. Test pg_basebackup with option -R from a standby server.
--recovery.conf file is created in data directory.Code Review: ------------- 1. typedef struct PQconninfoMapping { + char *keyword; + size_t member_offset; + bool for_replication; + char *conninfoValue; /* Special value mapping */ + char *connValue; +} PQconninfoMapping;Provide the better explanation of conninfoValue and connValue, how and
where
these are used?OK. This is only used for " requiressl='1' " (in the connection string)
and if '1' is set, PGconn.sslmode will be set to "require". All other
values are treated as it's being unset. This simplistic mapping
is used because there is only one such setting where different values
are used on the conninfo and the PGconn sides.2. if (tmp && strncmp(tmp, mapping->conninfoValue, len) == 0)
In any case if the above condition is not satisfied then the PGconn data
is
not filled with PQconninfoOption.
Is it correct?Yes, it stays NULL as makeEmptyPGconn() initialized it. This case only
happens
with the "requiressl" setting with its special mapping. If you set "
requiressl = '0' "
then it means that " requiressl='1' " was not set so the PGconn side stays
as NULL.The special casing was there in the old code too and behaves the same.
Documentation: ------------- 1. + <para> + The <literal>for_replication</> argument can be used to exclude some+ options from the list which are used by the walreceiver module. + <application>pg_basebackup</application> uses this pre-filtered list+ to construct <literal>primary_conninfo</> in the automatically generated + recovery.conf file. + </para>I feel the explanation should be as follows,
exclude some options from the list which are not used by the walreceiver
module?Err, no. The list excludes those[1] that *are* used (would be
overridden) by the walreceiver module:----8<--------8<--------8<--------8<--------8<----
static bool
libpqrcv_connect(char *conninfo, XLogRecPtr startpoint)
{
...
snprintf(conninfo_repl, sizeof(conninfo_repl),
"%s dbname=replication replication=true
fallback_application_name=walreceiver",
conninfo);
----8<--------8<--------8<--------8<--------8<----[1] Actually, more than these 3 options are excluded. The deprecated
ones are also excluded.Observations/Issues:
-------------------
1. If the password contains any escape sequence characters, It is leading
to
problems while walreceiver connecting to primary
by using the primary conninfo from recovery.confplease log an warning message or a note in document to handle such a
case
manually by the user.Done at both places.
Also, to adapt to the style of other error messages, now all my fprintf()
statements
are prefixed with: "%s: ...", progname.In new patches, when I ran "pg_basebackup -D hoge -c fast -R" on MacOS,
I got the following error message. BTW, I compiled the patched PostgreSQL
with --enable-debug and --enable-cassert options.pg_basebackup(41751) malloc: *** error for object 0x106001af0: pointer
being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6Can you show a backtrace? I compiled it on Fedora 17/x86_64 with
--enable-depend --enable-debug --enable-cassert. GLIBC is also smart
enough to catch improper free() calls, too.
I was able to test it on OSX and I found my bug. Attached is the new code.
The problem was in conninfo_init(), the last entry in the filtered list was
not initialized to 0. It seems that for some reason, my Linux machine gave
me pre-initialized memory.
$ uname -a
Darwin hrk.local 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23
16:25:48 PDT 2012; root:xnu-1699.32.7~1/RELEASE_X86_64 x86_64When tar output format is specified together with -R option, recovery.conf is
not included in base.tar. I think it should.Why? This patch only promises to write the recovery.conf into the
directory specified with -D.+ setting up the standby. Since creating a backup for a standalone + server with <option>-x</option> or <option>-X</option> and adding + a recovery.conf to it wouldn't make a working standby, these options + naturally conflict.Is this right? ISTM that basically we can use pg_basebackup -x to take
a base backup for starting the standby for now. No?I don't know. Pointers?
Thanks,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
01-PQconninfo-v12.patchtext/x-patch; name=01-PQconninfo-v12.patchDownload
diff -durpN postgresql/doc/src/sgml/libpq.sgml postgresql.1/doc/src/sgml/libpq.sgml
--- postgresql/doc/src/sgml/libpq.sgml 2012-08-03 09:39:30.114266570 +0200
+++ postgresql.1/doc/src/sgml/libpq.sgml 2012-10-10 14:09:31.924890230 +0200
@@ -496,6 +496,37 @@ typedef struct
</listitem>
</varlistentry>
+ <varlistentry id="libpq-pqconninfo">
+ <term><function>PQconninfo</function><indexterm><primary>PQconninfo</></></term>
+ <listitem>
+ <para>
+ Returns the connection options used by a live connection.
+<synopsis>
+PQconninfoOption *PQconninfo(PGconn *conn, bool for_replication);
+</synopsis>
+ </para>
+
+ <para>
+ Returns a connection options array. This can be used to determine
+ all possible <function>PQconnectdb</function> options and their
+ current values that were used to connect to the server. The return
+ value points to an array of <structname>PQconninfoOption</structname>
+ structures, which ends with an entry having a null <structfield>keyword</>
+ pointer. Every notes above for <function>PQconndefaults</function> also apply.
+ </para>
+
+ <para>
+ The <literal>for_replication</> argument can be used to exclude some
+ options from the list which are used by the walreceiver module.
+ <application>pg_basebackup</application> uses this pre-filtered list
+ to construct <literal>primary_conninfo</> in the automatically generated
+ recovery.conf file.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+
<varlistentry id="libpq-pqconninfoparse">
<term><function>PQconninfoParse</function><indexterm><primary>PQconninfoParse</></></term>
<listitem>
diff -durpN postgresql/src/interfaces/libpq/exports.txt postgresql.1/src/interfaces/libpq/exports.txt
--- postgresql/src/interfaces/libpq/exports.txt 2012-10-09 09:58:14.342782974 +0200
+++ postgresql.1/src/interfaces/libpq/exports.txt 2012-10-10 14:09:56.052057835 +0200
@@ -164,3 +164,4 @@ PQsetSingleRowMode 161
lo_lseek64 162
lo_tell64 163
lo_truncate64 164
+PQconninfo 165
diff -durpN postgresql/src/interfaces/libpq/fe-connect.c postgresql.1/src/interfaces/libpq/fe-connect.c
--- postgresql/src/interfaces/libpq/fe-connect.c 2012-09-09 08:11:09.470401480 +0200
+++ postgresql.1/src/interfaces/libpq/fe-connect.c 2012-10-10 21:53:05.502966398 +0200
@@ -137,6 +137,9 @@ static int ldapServiceLookup(const char
* PQconninfoOptions[] *must* be NULL. In a working copy, non-null "val"
* fields point to malloc'd strings that should be freed when the working
* array is freed (see PQconninfoFree).
+ *
+ * If you add a new connection option to this list, remember to add it to
+ * PQconninfoMappings[] below.
* ----------
*/
static const PQconninfoOption PQconninfoOptions[] = {
@@ -264,6 +267,66 @@ static const PQconninfoOption PQconninfo
NULL, NULL, 0}
};
+/*
+ * We need a mapping between the PQconninfoOptions[] array
+ * and PGconn members. We have to keep it separate from
+ * PQconninfoOptions[] to not leak info about PGconn members
+ * to clients.
+ */
+typedef struct PQconninfoMapping {
+ char *keyword;
+ size_t member_offset;
+ bool for_replication;
+ /*
+ * Special and simplistic value mapping between
+ * PQconninfoOption and PGconn. Only used by "requiressl".
+ */
+ char *conninfoValue;
+ char *connValue;
+} PQconninfoMapping;
+#define PGCONNMEMBERADDR(conn, mapping) ((char **)((char *)conn + mapping->member_offset))
+
+static const PQconninfoMapping PQconninfoMappings[] =
+{
+ /* "authtype" is not used anymore, there is no mapping to PGconn */
+ /* there is no mapping of "service" to PGconn */
+ { "user", offsetof(struct pg_conn, pguser), true, NULL, NULL },
+ { "password", offsetof(struct pg_conn, pgpass), true, NULL, NULL },
+ { "connect_timeout", offsetof(struct pg_conn, connect_timeout), true, NULL, NULL },
+ { "dbname", offsetof(struct pg_conn, dbName), false, NULL, NULL },
+ { "host", offsetof(struct pg_conn, pghost), true, NULL, NULL },
+ { "hostaddr", offsetof(struct pg_conn, pghostaddr), true, NULL, NULL },
+ { "port", offsetof(struct pg_conn, pgport), true, NULL, NULL },
+ { "client_encoding", offsetof(struct pg_conn, client_encoding_initial), false, NULL, NULL },
+ { "tty", offsetof(struct pg_conn, pgtty), false, NULL, NULL },
+ { "options", offsetof(struct pg_conn, pgoptions), true, NULL, NULL },
+ { "application_name", offsetof(struct pg_conn, appname), false, NULL, NULL },
+ { "fallback_application_name", offsetof(struct pg_conn, fbappname), false, NULL, NULL },
+ { "keepalives", offsetof(struct pg_conn, keepalives), true, NULL, NULL },
+ { "keepalives_idle", offsetof(struct pg_conn, keepalives_idle), true, NULL, NULL },
+ { "keepalives_interval", offsetof(struct pg_conn, keepalives_interval), true, NULL, NULL },
+ { "keepalives_count", offsetof(struct pg_conn, keepalives_count), true, NULL, NULL },
+#ifdef USE_SSL
+ { "requiressl", offsetof(struct pg_conn, sslmode), false, "1", "require" },
+#endif
+ { "sslmode", offsetof(struct pg_conn, sslmode), true, NULL, NULL },
+ { "sslcompression", offsetof(struct pg_conn, sslcompression), true, NULL, NULL },
+ { "sslcert", offsetof(struct pg_conn, sslcert), true, NULL, NULL },
+ { "sslkey", offsetof(struct pg_conn, sslkey), true, NULL, NULL },
+ { "sslrootcert", offsetof(struct pg_conn, sslrootcert), true, NULL, NULL },
+ { "sslcrl", offsetof(struct pg_conn, sslcrl), true, NULL, NULL },
+ { "requirepeer", offsetof(struct pg_conn, requirepeer), true, NULL, NULL },
+#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
+ { "krbsrvname", offsetof(struct pg_conn, krbsrvname), true, NULL, NULL },
+#endif
+#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
+ { "gsslib", offsetof(struct pg_conn, requirepeer), true, NULL, NULL },
+#endif
+ { "replication", offsetof(struct pg_conn, replication), false, NULL, NULL },
+ /* Terminating entry --- MUST BE LAST */
+ { NULL, 0, false, NULL, NULL }
+};
+
static const PQEnvironmentOption EnvironmentOptions[] =
{
/* common user-interface settings */
@@ -295,7 +358,8 @@ static PGconn *makeEmptyPGconn(void);
static void fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
-static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage);
+static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage,
+ bool for_replication);
static PQconninfoOption *parse_connection_string(const char *conninfo,
PQExpBuffer errorMessage, bool use_defaults);
static int uri_prefix_length(const char *connstr);
@@ -318,6 +382,8 @@ static char *conninfo_uri_decode(const c
static bool get_hexdigit(char digit, int *value);
static const char *conninfo_getval(PQconninfoOption *connOptions,
const char *keyword);
+static void conninfo_setval(PQconninfoOption *connOptions,
+ const char *keyword, const char *val);
static PQconninfoOption *conninfo_storeval(PQconninfoOption *connOptions,
const char *keyword, const char *value,
PQExpBuffer errorMessage, bool ignoreMissing, bool uri_decode);
@@ -627,7 +693,9 @@ PQconnectStart(const char *conninfo)
static void
fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
{
+ const PQconninfoMapping *mapping;
const char *tmp;
+ char **memberaddr;
/*
* Move option values into conn structure
@@ -637,72 +705,24 @@ fillPGconn(PGconn *conn, PQconninfoOptio
*
* XXX: probably worth checking strdup() return value here...
*/
- tmp = conninfo_getval(connOptions, "hostaddr");
- conn->pghostaddr = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "host");
- conn->pghost = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "port");
- conn->pgport = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "tty");
- conn->pgtty = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "options");
- conn->pgoptions = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "application_name");
- conn->appname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "fallback_application_name");
- conn->fbappname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "dbname");
- conn->dbName = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "user");
- conn->pguser = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "password");
- conn->pgpass = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "connect_timeout");
- conn->connect_timeout = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "client_encoding");
- conn->client_encoding_initial = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives");
- conn->keepalives = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_idle");
- conn->keepalives_idle = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_interval");
- conn->keepalives_interval = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_count");
- conn->keepalives_count = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslmode");
- conn->sslmode = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcompression");
- conn->sslcompression = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslkey");
- conn->sslkey = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcert");
- conn->sslcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslrootcert");
- conn->sslrootcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcrl");
- conn->sslcrl = tmp ? strdup(tmp) : NULL;
-#ifdef USE_SSL
- tmp = conninfo_getval(connOptions, "requiressl");
- if (tmp && tmp[0] == '1')
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
{
- /* here warn that the requiressl option is deprecated? */
- if (conn->sslmode)
- free(conn->sslmode);
- conn->sslmode = strdup("require");
+ tmp = conninfo_getval(connOptions, mapping->keyword);
+ memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ if (mapping->conninfoValue && mapping->connValue)
+ {
+ size_t len = strlen(mapping->conninfoValue);
+ if (tmp && strncmp(tmp, mapping->conninfoValue, len) == 0)
+ {
+ if (*memberaddr)
+ free(*memberaddr);
+ *memberaddr = strdup(mapping->connValue);
+ }
+ }
+ else
+ *memberaddr = tmp ? strdup(tmp) : NULL;
}
-#endif
- tmp = conninfo_getval(connOptions, "requirepeer");
- conn->requirepeer = tmp ? strdup(tmp) : NULL;
-#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "krbsrvname");
- conn->krbsrvname = tmp ? strdup(tmp) : NULL;
-#endif
-#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "gsslib");
- conn->gsslib = tmp ? strdup(tmp) : NULL;
-#endif
- tmp = conninfo_getval(connOptions, "replication");
- conn->replication = tmp ? strdup(tmp) : NULL;
}
/*
@@ -884,7 +904,7 @@ PQconndefaults(void)
if (PQExpBufferDataBroken(errorBuf))
return NULL; /* out of memory already :-( */
- connOptions = conninfo_init(&errorBuf);
+ connOptions = conninfo_init(&errorBuf, false);
if (connOptions != NULL)
{
if (!conninfo_add_defaults(connOptions, &errorBuf))
@@ -4006,9 +4026,11 @@ PQconninfoParse(const char *conninfo, ch
/*
* Build a working copy of the constant PQconninfoOptions array.
+ * If for_replication is true, only return the options that are
+ * not added by libpqwalreceiver.
*/
static PQconninfoOption *
-conninfo_init(PQExpBuffer errorMessage)
+conninfo_init(PQExpBuffer errorMessage, bool for_replication)
{
PQconninfoOption *options;
@@ -4019,7 +4041,28 @@ conninfo_init(PQExpBuffer errorMessage)
libpq_gettext("out of memory\n"));
return NULL;
}
- memcpy(options, PQconninfoOptions, sizeof(PQconninfoOptions));
+ if (for_replication)
+ {
+ const PQconninfoMapping *mapping = PQconninfoMappings;
+ PQconninfoOption *opt_dest = options;
+
+ while (mapping->keyword)
+ {
+ PQconninfoOption *opt_src = conninfo_find((PQconninfoOption *)PQconninfoOptions, mapping->keyword);
+
+ if (opt_src && mapping->for_replication)
+ {
+ memcpy(opt_dest, opt_src, sizeof(PQconninfoOption));
+ opt_dest++;
+ }
+
+ opt_src++;
+ mapping++;
+ }
+ MemSet(opt_dest, 0, sizeof(PQconninfoOption));
+ }
+ else
+ memcpy(options, PQconninfoOptions, sizeof(PQconninfoOptions));
return options;
}
@@ -4095,7 +4138,7 @@ conninfo_parse(const char *conninfo, PQE
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
return NULL;
@@ -4295,7 +4338,7 @@ conninfo_array_parse(const char *const *
}
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
{
PQconninfoFree(dbname_options);
@@ -4485,7 +4528,7 @@ conninfo_uri_parse(const char *uri, PQEx
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
return NULL;
@@ -4985,6 +5028,24 @@ conninfo_getval(PQconninfoOption *connOp
}
/*
+ * Set an option value corresponding to the keyword in the connOptions array.
+ */
+static void
+conninfo_setval(PQconninfoOption *connOptions, const char *keyword,
+ const char *val)
+{
+ PQconninfoOption *option;
+
+ option = conninfo_find(connOptions, keyword);
+ if (option)
+ {
+ if (option->val)
+ free(option->val);
+ option->val = val ? strdup(val) : NULL;
+ }
+}
+
+/*
* Store a (new) value for an option corresponding to the keyword in
* connOptions array.
*
@@ -5066,6 +5127,50 @@ conninfo_find(PQconninfoOption *connOpti
}
+/*
+ * Return the connection options used for the connections
+ */
+PQconninfoOption *
+PQconninfo(PGconn *conn, bool for_replication)
+{
+ PQExpBufferData errorBuf;
+ PQconninfoOption *connOptions;
+
+ if (conn == NULL)
+ return NULL;
+
+ /* We don't actually report any errors here, but callees want a buffer */
+ initPQExpBuffer(&errorBuf);
+ if (PQExpBufferDataBroken(errorBuf))
+ return NULL; /* out of memory already :-( */
+
+ connOptions = conninfo_init(&errorBuf, for_replication);
+
+ termPQExpBuffer(&errorBuf);
+
+ if (connOptions != NULL)
+ {
+ const PQconninfoMapping *mapping;
+
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
+ {
+ char **memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ if (mapping->conninfoValue && mapping->connValue)
+ {
+ size_t len = strlen(mapping->connValue);
+ if (*memberaddr && strncmp(*memberaddr, mapping->connValue, len) == 0)
+ conninfo_setval(connOptions, mapping->keyword, mapping->conninfoValue);
+ }
+ else
+ conninfo_setval(connOptions, mapping->keyword, *memberaddr);
+ }
+ }
+
+ return connOptions;
+}
+
+
void
PQconninfoFree(PQconninfoOption *connOptions)
{
diff -durpN postgresql/src/interfaces/libpq/libpq-fe.h postgresql.1/src/interfaces/libpq/libpq-fe.h
--- postgresql/src/interfaces/libpq/libpq-fe.h 2012-10-09 09:58:14.343782980 +0200
+++ postgresql.1/src/interfaces/libpq/libpq-fe.h 2012-10-10 14:09:31.936890314 +0200
@@ -262,6 +262,9 @@ extern PQconninfoOption *PQconndefaults(
/* parse connection options in same way as PQconnectdb */
extern PQconninfoOption *PQconninfoParse(const char *conninfo, char **errmsg);
+/* return the connection options used by a live connection */
+extern PQconninfoOption *PQconninfo(PGconn *conn, bool for_replication);
+
/* free the data structure returned by PQconndefaults() or PQconninfoParse() */
extern void PQconninfoFree(PQconninfoOption *connOptions);
02-pg_basebackup-v9.patchtext/x-patch; name=02-pg_basebackup-v9.patchDownload
diff -durpN postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml
--- postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml 2012-08-24 09:49:22.960530329 +0200
+++ postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml 2012-10-10 14:49:40.869336237 +0200
@@ -189,6 +189,36 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--write-recovery-conf</option></term>
+ <listitem>
+
+ <para>
+ Write a minimal recovery.conf into the output directory using
+ the connection parameters from the command line to ease
+ setting up the standby. Since creating a backup for a standalone
+ server with <option>-x</option> or <option>-X</option> and adding
+ a recovery.conf to it wouldn't make a working standby, these options
+ naturally conflict.
+ </para>
+
+ <para>
+ The password written into recovery.conf is not escaped even if special
+ characters appear in it. The administrator must review recovery.conf
+ to ensure proper escaping.
+ </para>
+
+ <para>
+ When this option is specified and taking the base backup succeeded,
+ failing to write the <filename>recovery.conf</filename> results
+ in the error code 2. This way, scripts can distinguish between different
+ failure cases of <application>pg_basebackup</application>.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-x</option></term>
<term><option>--xlog</option></term>
<listitem>
diff -durpN postgresql.1/src/bin/pg_basebackup/pg_basebackup.c postgresql.2/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.1/src/bin/pg_basebackup/pg_basebackup.c 2012-10-03 10:40:48.297207389 +0200
+++ postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2012-10-10 14:53:18.717764755 +0200
@@ -46,6 +46,7 @@ int compresslevel = 0;
bool includewal = false;
bool streamwal = false;
bool fastcheckpoint = false;
+bool writerecoveryconf = false;
int standby_message_timeout = 10 * 1000; /* 10 sec = default */
/* Progress counters */
@@ -70,14 +71,18 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
+static PQconninfoOption *connOptions = NULL;
+
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
static void progress_report(int tablespacenum, const char *filename);
+static void stderr_write_error(FILE *cf, char *filename);
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
static void BaseBackup(void);
+static void WriteRecoveryConf(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
bool segment_finished);
@@ -107,6 +112,8 @@ usage(void)
printf(_("\nOptions controlling the output:\n"));
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
+ printf(_(" -R, --write-recovery-conf\n"
+ " write recovery.conf after backup\n"));
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
printf(_(" -X, --xlog-method=fetch|stream\n"
" include required WAL files with specified method\n"));
@@ -960,6 +967,10 @@ BaseBackup(void)
/* Error message already written in GetConnection() */
exit(1);
+ /* If recovery.conf is to be written, keep the connection parameters for later usage */
+ if (writerecoveryconf)
+ connOptions = PQconninfo(conn, true);
+
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
@@ -1234,6 +1245,73 @@ BaseBackup(void)
}
+static void
+stderr_write_error(FILE *cf, char *filename)
+{
+ fprintf(stderr, _("%s: cannot write to %s: %s"), progname, filename, strerror(errno));
+ fclose(cf);
+ unlink(filename);
+ exit(2);
+}
+
+
+/*
+ * Attempt to create recovery.conf and write the expected contents to it.
+ */
+static void
+WriteRecoveryConf(void)
+{
+ char filename[MAXPGPATH];
+ PQconninfoOption *option;
+ FILE *cf;
+
+ if (!writerecoveryconf)
+ return;
+
+ sprintf(filename, "%s/recovery.conf", basedir);
+
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("%s: cannot create %s: %s"), progname, filename, strerror(errno));
+ exit(2);
+ }
+
+ if (fprintf(cf, "standby_mode = 'on'\n") < 0)
+ stderr_write_error(cf, filename);
+
+ if (fprintf(cf, "primary_conninfo = '") < 0)
+ stderr_write_error(cf, filename);
+
+ for (option = connOptions; option && option->keyword; option++)
+ {
+ /*
+ * Do not emit this setting if not set, empty or default.
+ * The list of options was already pre-filtered for options
+ * usable for replication with PQconninfo(conn, true).
+ */
+ if (option->val == NULL ||
+ (option->val != NULL && option->val[0] == '\0') ||
+ (option->val &&
+ option->compiled &&
+ strcmp(option->val, option->compiled) == 0))
+ continue;
+
+ /* write "keyword='value'" pieces, single quotes doubled */
+ if (fprintf(cf, "%s=''%s'' ", option->keyword, option->val) < 0)
+ stderr_write_error(cf, filename);
+ }
+
+ if (fprintf(cf, "'\n") < 0)
+ stderr_write_error(cf, filename);
+
+ PQconninfoFree(connOptions);
+
+ fclose(cf);
+
+ fprintf(stderr, _("%s: recovery.conf written. Please, add escaping to the password field in \"primary_conninfo\" if needed.\n"), progname);
+}
+
int
main(int argc, char **argv)
{
@@ -1243,6 +1321,7 @@ main(int argc, char **argv)
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
+ {"write-recovery-conf", no_argument, NULL, 'R'},
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
{"gzip", no_argument, NULL, 'z'},
@@ -1280,7 +1359,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:xX:l:zZ:c:h:p:U:s:wWvP",
+ while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
long_options, &option_index)) != -1)
{
switch (c)
@@ -1301,6 +1380,9 @@ main(int argc, char **argv)
exit(1);
}
break;
+ case 'R':
+ writerecoveryconf = true;
+ break;
case 'x':
if (includewal)
{
@@ -1466,6 +1548,13 @@ main(int argc, char **argv)
}
#endif
+ if (writerecoveryconf && includewal)
+ {
+ fprintf(stderr,
+ _("--xlog and --writerecoveryconf are mutually exclusive\n"));
+ exit(1);
+ }
+
/*
* Verify that the target directory exists, or create it. For plaintext
* backups, always require the directory. For tar backups, require it
@@ -1476,5 +1565,7 @@ main(int argc, char **argv)
BaseBackup();
+ WriteRecoveryConf();
+
return 0;
}
On Thu, Oct 11, 2012 at 3:36 AM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
2012-10-10 18:23 keltezéssel, Fujii Masao írta:
When tar output format is specified together with -R option, recovery.conf
is
not included in base.tar. I think it should.Why? This patch only promises to write the recovery.conf into the
directory specified with -D.
Because it's more user-friendly. If recovery.conf is not included in base.tar,
when base.tar is extracted to disk to use the backup, a user always needs
to copy recovery.conf to the extracted directory. OTOH if it's included in
base.tar, such copy operation is not required and we can simplify the
procedures to use the backup a bit.
+ setting up the standby. Since creating a backup for a standalone + server with <option>-x</option> or <option>-X</option> and adding + a recovery.conf to it wouldn't make a working standby, these options + naturally conflict.Is this right? ISTM that basically we can use pg_basebackup -x to take
a base backup for starting the standby for now. No?I don't know. Pointers?
There is no restriction that the backup which was taken by using
pg_basebackup -x cannot be used to start the standby. So I wonder
why -R option cannot work together with -x. It's useful if recovery.conf
is automatically written even with pg_basebackup -x.
And I found another problem: when -(stdout) is specified in -D option,
recovery.conf fails to be written.
$ pg_basebackup -D- -F t -c fast -R > hoge.tar
NOTICE: WAL archiving is not enabled; you must ensure that all
required WAL segments are copied through other means to complete the
backup
pg_basebackup: cannot create -/recovery.conf: No such file or directory
Regards,
--
Fujii Masao
On Thu, Oct 11, 2012 at 4:58 AM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
I was able to test it on OSX and I found my bug. Attached is the new code.
The problem was in conninfo_init(), the last entry in the filtered list was
not initialized to 0. It seems that for some reason, my Linux machine gave
me pre-initialized memory.
Thanks. Will test.
Regards,
--
Fujii Masao
On Wed, Oct 10, 2012 at 8:02 PM, Fujii Masao <masao.fujii@gmail.com> wrote:
On Thu, Oct 11, 2012 at 3:36 AM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
2012-10-10 18:23 keltezéssel, Fujii Masao írta:
When tar output format is specified together with -R option, recovery.conf
is
not included in base.tar. I think it should.Why? This patch only promises to write the recovery.conf into the
directory specified with -D.Because it's more user-friendly. If recovery.conf is not included in base.tar,
when base.tar is extracted to disk to use the backup, a user always needs
to copy recovery.conf to the extracted directory. OTOH if it's included in
base.tar, such copy operation is not required and we can simplify the
procedures to use the backup a bit.
+1.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
2012-10-12 22:21 keltezéssel, Robert Haas írta:
On Wed, Oct 10, 2012 at 8:02 PM, Fujii Masao <masao.fujii@gmail.com> wrote:
On Thu, Oct 11, 2012 at 3:36 AM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
2012-10-10 18:23 keltezéssel, Fujii Masao írta:
When tar output format is specified together with -R option, recovery.conf
is
not included in base.tar. I think it should.Why? This patch only promises to write the recovery.conf into the
directory specified with -D.Because it's more user-friendly. If recovery.conf is not included in base.tar,
when base.tar is extracted to disk to use the backup, a user always needs
to copy recovery.conf to the extracted directory. OTOH if it's included in
base.tar, such copy operation is not required and we can simplify the
procedures to use the backup a bit.+1.
OK, out of popular demand, I implemented writing into the base.tar
if both -R and -Ft was specified.
The code to create a tar header and the checksum inside the header
was copied from bin/pg_dump/pg_backup_tar.c.
I tested these scenarios ("server" can be either a master and standby):
- backup a server with -Fp
(no recovery.conf in the target directory was written)
- backup a server with -Ftar
(no recovery.conf was written into the target directory or base.tar)
- backup a server with -Ftar -z
(no recovery.conf was written into the target directory or base.tar.gz)
- backup a server with -R -Fp
(the new recovery.conf was written into the target directory)
- backup a server with -R -Ftar
(the new recovery.conf was written into the base.tar)
- backup a server with -R -Ftar -z
(the new recovery.conf was written into the base.tar.gz)
Backing up a standby server without -R preserves the original recovery.conf of the
standby, it points to the standby's source server.
Backing up a standby server with -R overwrites the original recovery.conf with the new
one pointing to the standby instead of the standby's source server. Without -Ft, it is
obvious. With -Ft, there are two recovery.conf files in the tar file and upon extracting it,
the last written one (the one generated via -R) overwrites the original.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
01-PQconninfo-v13.patchtext/x-patch; name=01-PQconninfo-v13.patchDownload
diff -durpN postgresql/doc/src/sgml/libpq.sgml postgresql.1/doc/src/sgml/libpq.sgml
--- postgresql/doc/src/sgml/libpq.sgml 2012-08-03 09:39:30.114266570 +0200
+++ postgresql.1/doc/src/sgml/libpq.sgml 2012-10-14 11:12:07.049196259 +0200
@@ -496,6 +496,37 @@ typedef struct
</listitem>
</varlistentry>
+ <varlistentry id="libpq-pqconninfo">
+ <term><function>PQconninfo</function><indexterm><primary>PQconninfo</></></term>
+ <listitem>
+ <para>
+ Returns the connection options used by a live connection.
+<synopsis>
+PQconninfoOption *PQconninfo(PGconn *conn, bool for_replication);
+</synopsis>
+ </para>
+
+ <para>
+ Returns a connection options array. This can be used to determine
+ all possible <function>PQconnectdb</function> options and their
+ current values that were used to connect to the server. The return
+ value points to an array of <structname>PQconninfoOption</structname>
+ structures, which ends with an entry having a null <structfield>keyword</>
+ pointer. Every notes above for <function>PQconndefaults</function> also apply.
+ </para>
+
+ <para>
+ The <literal>for_replication</> argument can be used to exclude some
+ options from the list which are used by the walreceiver module.
+ <application>pg_basebackup</application> uses this pre-filtered list
+ to construct <literal>primary_conninfo</> in the automatically generated
+ recovery.conf file.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+
<varlistentry id="libpq-pqconninfoparse">
<term><function>PQconninfoParse</function><indexterm><primary>PQconninfoParse</></></term>
<listitem>
diff -durpN postgresql/src/interfaces/libpq/exports.txt postgresql.1/src/interfaces/libpq/exports.txt
--- postgresql/src/interfaces/libpq/exports.txt 2012-10-09 09:58:14.342782974 +0200
+++ postgresql.1/src/interfaces/libpq/exports.txt 2012-10-14 11:12:07.049196259 +0200
@@ -164,3 +164,4 @@ PQsetSingleRowMode 161
lo_lseek64 162
lo_tell64 163
lo_truncate64 164
+PQconninfo 165
diff -durpN postgresql/src/interfaces/libpq/fe-connect.c postgresql.1/src/interfaces/libpq/fe-connect.c
--- postgresql/src/interfaces/libpq/fe-connect.c 2012-09-09 08:11:09.470401480 +0200
+++ postgresql.1/src/interfaces/libpq/fe-connect.c 2012-10-14 11:12:07.053196284 +0200
@@ -137,6 +137,9 @@ static int ldapServiceLookup(const char
* PQconninfoOptions[] *must* be NULL. In a working copy, non-null "val"
* fields point to malloc'd strings that should be freed when the working
* array is freed (see PQconninfoFree).
+ *
+ * If you add a new connection option to this list, remember to add it to
+ * PQconninfoMappings[] below.
* ----------
*/
static const PQconninfoOption PQconninfoOptions[] = {
@@ -264,6 +267,66 @@ static const PQconninfoOption PQconninfo
NULL, NULL, 0}
};
+/*
+ * We need a mapping between the PQconninfoOptions[] array
+ * and PGconn members. We have to keep it separate from
+ * PQconninfoOptions[] to not leak info about PGconn members
+ * to clients.
+ */
+typedef struct PQconninfoMapping {
+ char *keyword;
+ size_t member_offset;
+ bool for_replication;
+ /*
+ * Special and simplistic value mapping between
+ * PQconninfoOption and PGconn. Only used by "requiressl".
+ */
+ char *conninfoValue;
+ char *connValue;
+} PQconninfoMapping;
+#define PGCONNMEMBERADDR(conn, mapping) ((char **)((char *)conn + mapping->member_offset))
+
+static const PQconninfoMapping PQconninfoMappings[] =
+{
+ /* "authtype" is not used anymore, there is no mapping to PGconn */
+ /* there is no mapping of "service" to PGconn */
+ { "user", offsetof(struct pg_conn, pguser), true, NULL, NULL },
+ { "password", offsetof(struct pg_conn, pgpass), true, NULL, NULL },
+ { "connect_timeout", offsetof(struct pg_conn, connect_timeout), true, NULL, NULL },
+ { "dbname", offsetof(struct pg_conn, dbName), false, NULL, NULL },
+ { "host", offsetof(struct pg_conn, pghost), true, NULL, NULL },
+ { "hostaddr", offsetof(struct pg_conn, pghostaddr), true, NULL, NULL },
+ { "port", offsetof(struct pg_conn, pgport), true, NULL, NULL },
+ { "client_encoding", offsetof(struct pg_conn, client_encoding_initial), false, NULL, NULL },
+ { "tty", offsetof(struct pg_conn, pgtty), false, NULL, NULL },
+ { "options", offsetof(struct pg_conn, pgoptions), true, NULL, NULL },
+ { "application_name", offsetof(struct pg_conn, appname), false, NULL, NULL },
+ { "fallback_application_name", offsetof(struct pg_conn, fbappname), false, NULL, NULL },
+ { "keepalives", offsetof(struct pg_conn, keepalives), true, NULL, NULL },
+ { "keepalives_idle", offsetof(struct pg_conn, keepalives_idle), true, NULL, NULL },
+ { "keepalives_interval", offsetof(struct pg_conn, keepalives_interval), true, NULL, NULL },
+ { "keepalives_count", offsetof(struct pg_conn, keepalives_count), true, NULL, NULL },
+#ifdef USE_SSL
+ { "requiressl", offsetof(struct pg_conn, sslmode), false, "1", "require" },
+#endif
+ { "sslmode", offsetof(struct pg_conn, sslmode), true, NULL, NULL },
+ { "sslcompression", offsetof(struct pg_conn, sslcompression), true, NULL, NULL },
+ { "sslcert", offsetof(struct pg_conn, sslcert), true, NULL, NULL },
+ { "sslkey", offsetof(struct pg_conn, sslkey), true, NULL, NULL },
+ { "sslrootcert", offsetof(struct pg_conn, sslrootcert), true, NULL, NULL },
+ { "sslcrl", offsetof(struct pg_conn, sslcrl), true, NULL, NULL },
+ { "requirepeer", offsetof(struct pg_conn, requirepeer), true, NULL, NULL },
+#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
+ { "krbsrvname", offsetof(struct pg_conn, krbsrvname), true, NULL, NULL },
+#endif
+#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
+ { "gsslib", offsetof(struct pg_conn, requirepeer), true, NULL, NULL },
+#endif
+ { "replication", offsetof(struct pg_conn, replication), false, NULL, NULL },
+ /* Terminating entry --- MUST BE LAST */
+ { NULL, 0, false, NULL, NULL }
+};
+
static const PQEnvironmentOption EnvironmentOptions[] =
{
/* common user-interface settings */
@@ -295,7 +358,8 @@ static PGconn *makeEmptyPGconn(void);
static void fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
-static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage);
+static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage,
+ bool for_replication);
static PQconninfoOption *parse_connection_string(const char *conninfo,
PQExpBuffer errorMessage, bool use_defaults);
static int uri_prefix_length(const char *connstr);
@@ -318,6 +382,8 @@ static char *conninfo_uri_decode(const c
static bool get_hexdigit(char digit, int *value);
static const char *conninfo_getval(PQconninfoOption *connOptions,
const char *keyword);
+static void conninfo_setval(PQconninfoOption *connOptions,
+ const char *keyword, const char *val);
static PQconninfoOption *conninfo_storeval(PQconninfoOption *connOptions,
const char *keyword, const char *value,
PQExpBuffer errorMessage, bool ignoreMissing, bool uri_decode);
@@ -627,7 +693,9 @@ PQconnectStart(const char *conninfo)
static void
fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
{
+ const PQconninfoMapping *mapping;
const char *tmp;
+ char **memberaddr;
/*
* Move option values into conn structure
@@ -637,72 +705,24 @@ fillPGconn(PGconn *conn, PQconninfoOptio
*
* XXX: probably worth checking strdup() return value here...
*/
- tmp = conninfo_getval(connOptions, "hostaddr");
- conn->pghostaddr = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "host");
- conn->pghost = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "port");
- conn->pgport = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "tty");
- conn->pgtty = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "options");
- conn->pgoptions = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "application_name");
- conn->appname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "fallback_application_name");
- conn->fbappname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "dbname");
- conn->dbName = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "user");
- conn->pguser = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "password");
- conn->pgpass = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "connect_timeout");
- conn->connect_timeout = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "client_encoding");
- conn->client_encoding_initial = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives");
- conn->keepalives = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_idle");
- conn->keepalives_idle = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_interval");
- conn->keepalives_interval = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_count");
- conn->keepalives_count = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslmode");
- conn->sslmode = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcompression");
- conn->sslcompression = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslkey");
- conn->sslkey = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcert");
- conn->sslcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslrootcert");
- conn->sslrootcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcrl");
- conn->sslcrl = tmp ? strdup(tmp) : NULL;
-#ifdef USE_SSL
- tmp = conninfo_getval(connOptions, "requiressl");
- if (tmp && tmp[0] == '1')
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
{
- /* here warn that the requiressl option is deprecated? */
- if (conn->sslmode)
- free(conn->sslmode);
- conn->sslmode = strdup("require");
+ tmp = conninfo_getval(connOptions, mapping->keyword);
+ memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ if (mapping->conninfoValue && mapping->connValue)
+ {
+ size_t len = strlen(mapping->conninfoValue);
+ if (tmp && strncmp(tmp, mapping->conninfoValue, len) == 0)
+ {
+ if (*memberaddr)
+ free(*memberaddr);
+ *memberaddr = strdup(mapping->connValue);
+ }
+ }
+ else
+ *memberaddr = tmp ? strdup(tmp) : NULL;
}
-#endif
- tmp = conninfo_getval(connOptions, "requirepeer");
- conn->requirepeer = tmp ? strdup(tmp) : NULL;
-#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "krbsrvname");
- conn->krbsrvname = tmp ? strdup(tmp) : NULL;
-#endif
-#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "gsslib");
- conn->gsslib = tmp ? strdup(tmp) : NULL;
-#endif
- tmp = conninfo_getval(connOptions, "replication");
- conn->replication = tmp ? strdup(tmp) : NULL;
}
/*
@@ -884,7 +904,7 @@ PQconndefaults(void)
if (PQExpBufferDataBroken(errorBuf))
return NULL; /* out of memory already :-( */
- connOptions = conninfo_init(&errorBuf);
+ connOptions = conninfo_init(&errorBuf, false);
if (connOptions != NULL)
{
if (!conninfo_add_defaults(connOptions, &errorBuf))
@@ -4006,9 +4026,11 @@ PQconninfoParse(const char *conninfo, ch
/*
* Build a working copy of the constant PQconninfoOptions array.
+ * If for_replication is true, only return the options that are
+ * not added by libpqwalreceiver.
*/
static PQconninfoOption *
-conninfo_init(PQExpBuffer errorMessage)
+conninfo_init(PQExpBuffer errorMessage, bool for_replication)
{
PQconninfoOption *options;
@@ -4019,7 +4041,28 @@ conninfo_init(PQExpBuffer errorMessage)
libpq_gettext("out of memory\n"));
return NULL;
}
- memcpy(options, PQconninfoOptions, sizeof(PQconninfoOptions));
+ if (for_replication)
+ {
+ const PQconninfoMapping *mapping = PQconninfoMappings;
+ PQconninfoOption *opt_dest = options;
+
+ while (mapping->keyword)
+ {
+ PQconninfoOption *opt_src = conninfo_find((PQconninfoOption *)PQconninfoOptions, mapping->keyword);
+
+ if (opt_src && mapping->for_replication)
+ {
+ memcpy(opt_dest, opt_src, sizeof(PQconninfoOption));
+ opt_dest++;
+ }
+
+ opt_src++;
+ mapping++;
+ }
+ MemSet(opt_dest, 0, sizeof(PQconninfoOption));
+ }
+ else
+ memcpy(options, PQconninfoOptions, sizeof(PQconninfoOptions));
return options;
}
@@ -4095,7 +4138,7 @@ conninfo_parse(const char *conninfo, PQE
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
return NULL;
@@ -4295,7 +4338,7 @@ conninfo_array_parse(const char *const *
}
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
{
PQconninfoFree(dbname_options);
@@ -4485,7 +4528,7 @@ conninfo_uri_parse(const char *uri, PQEx
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
return NULL;
@@ -4985,6 +5028,24 @@ conninfo_getval(PQconninfoOption *connOp
}
/*
+ * Set an option value corresponding to the keyword in the connOptions array.
+ */
+static void
+conninfo_setval(PQconninfoOption *connOptions, const char *keyword,
+ const char *val)
+{
+ PQconninfoOption *option;
+
+ option = conninfo_find(connOptions, keyword);
+ if (option)
+ {
+ if (option->val)
+ free(option->val);
+ option->val = val ? strdup(val) : NULL;
+ }
+}
+
+/*
* Store a (new) value for an option corresponding to the keyword in
* connOptions array.
*
@@ -5066,6 +5127,50 @@ conninfo_find(PQconninfoOption *connOpti
}
+/*
+ * Return the connection options used for the connections
+ */
+PQconninfoOption *
+PQconninfo(PGconn *conn, bool for_replication)
+{
+ PQExpBufferData errorBuf;
+ PQconninfoOption *connOptions;
+
+ if (conn == NULL)
+ return NULL;
+
+ /* We don't actually report any errors here, but callees want a buffer */
+ initPQExpBuffer(&errorBuf);
+ if (PQExpBufferDataBroken(errorBuf))
+ return NULL; /* out of memory already :-( */
+
+ connOptions = conninfo_init(&errorBuf, for_replication);
+
+ termPQExpBuffer(&errorBuf);
+
+ if (connOptions != NULL)
+ {
+ const PQconninfoMapping *mapping;
+
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
+ {
+ char **memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ if (mapping->conninfoValue && mapping->connValue)
+ {
+ size_t len = strlen(mapping->connValue);
+ if (*memberaddr && strncmp(*memberaddr, mapping->connValue, len) == 0)
+ conninfo_setval(connOptions, mapping->keyword, mapping->conninfoValue);
+ }
+ else
+ conninfo_setval(connOptions, mapping->keyword, *memberaddr);
+ }
+ }
+
+ return connOptions;
+}
+
+
void
PQconninfoFree(PQconninfoOption *connOptions)
{
diff -durpN postgresql/src/interfaces/libpq/libpq-fe.h postgresql.1/src/interfaces/libpq/libpq-fe.h
--- postgresql/src/interfaces/libpq/libpq-fe.h 2012-10-09 09:58:14.343782980 +0200
+++ postgresql.1/src/interfaces/libpq/libpq-fe.h 2012-10-14 11:12:07.054196291 +0200
@@ -262,6 +262,9 @@ extern PQconninfoOption *PQconndefaults(
/* parse connection options in same way as PQconnectdb */
extern PQconninfoOption *PQconninfoParse(const char *conninfo, char **errmsg);
+/* return the connection options used by a live connection */
+extern PQconninfoOption *PQconninfo(PGconn *conn, bool for_replication);
+
/* free the data structure returned by PQconndefaults() or PQconninfoParse() */
extern void PQconninfoFree(PQconninfoOption *connOptions);
02-pg_basebackup-v10.patchtext/x-patch; name=02-pg_basebackup-v10.patchDownload
diff -durpN postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml
--- postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml 2012-08-24 09:49:22.960530329 +0200
+++ postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml 2012-10-14 13:09:58.913513268 +0200
@@ -189,6 +189,37 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--write-recovery-conf</option></term>
+ <listitem>
+
+ <para>
+ Write a minimal recovery.conf into the output directory (or into
+ the base archive file if <option>--format=tar</option> was specified)
+ using the connection parameters from the command line to ease
+ setting up the standby. Since creating a backup for a standalone
+ server with <option>-x</option> or <option>-X</option> and adding
+ a recovery.conf to it wouldn't make a working standby, these options
+ naturally conflict.
+ </para>
+
+ <para>
+ The password written into recovery.conf is not escaped even if special
+ characters appear in it. The administrator must review recovery.conf
+ to ensure proper escaping.
+ </para>
+
+ <para>
+ When this option is specified and taking the base backup succeeded,
+ failing to write the <filename>recovery.conf</filename> results
+ in the error code 2. This way, scripts can distinguish between different
+ failure cases of <application>pg_basebackup</application>.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-x</option></term>
<term><option>--xlog</option></term>
<listitem>
diff -durpN postgresql.1/src/bin/pg_basebackup/pg_basebackup.c postgresql.2/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.1/src/bin/pg_basebackup/pg_basebackup.c 2012-10-03 10:40:48.297207389 +0200
+++ postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2012-10-14 13:07:10.303472940 +0200
@@ -25,6 +25,7 @@
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
+#include <time.h>
#ifdef HAVE_LIBZ
#include <zlib.h>
@@ -46,6 +47,7 @@ int compresslevel = 0;
bool includewal = false;
bool streamwal = false;
bool fastcheckpoint = false;
+bool writerecoveryconf = false;
int standby_message_timeout = 10 * 1000; /* 10 sec = default */
/* Progress counters */
@@ -70,6 +72,10 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
+#define RCBUFSZ (1024)
+static size_t recoveryconf_length = 0;
+static char *recoveryconf = NULL;
+
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
@@ -77,6 +83,13 @@ static void progress_report(int tablespa
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
+static void _reallocRCBuffer(char **buf, int *bufsz, int currsz,
+ PQconninfoOption *option, int extrasz);
+static void CreateRecoveryConf(PGconn *conn);
+static void WriteRecoveryConf(void);
+static int _tarChecksum(char *header);
+static void print_val(char *s, uint64 val, unsigned int base, size_t len);
+static void _tarCreateHeader(char *header);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -107,6 +120,8 @@ usage(void)
printf(_("\nOptions controlling the output:\n"));
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
+ printf(_(" -R, --write-recovery-conf\n"
+ " write recovery.conf after backup\n"));
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
printf(_(" -X, --xlog-method=fetch|stream\n"
" include required WAL files with specified method\n"));
@@ -598,13 +613,66 @@ ReceiveTarFile(PGconn *conn, PGresult *r
if (r == -1)
{
/*
- * End of chunk. Close file (but not stdout).
+ * End of chunk. Write recovery.conf into the tar file (if it
+ * was requested) and close file (but not stdout).
*
* Also, write two completely empty blocks at the end of the tar
* file, as required by some tar programs.
*/
char zerobuf[1024];
+ if (rownum == 0 && writerecoveryconf)
+ {
+ char header[512];
+
+ _tarCreateHeader(header);
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (gzwrite(ztarfile, header, sizeof(header)) !=
+ sizeof(header))
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, filename, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+
+ if (gzwrite(ztarfile, recoveryconf, recoveryconf_length) !=
+ recoveryconf_length)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, filename, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (fwrite(header, sizeof(header), 1, tarfile) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ if (fwrite(recoveryconf, recoveryconf_length, 1, tarfile) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ }
+ fprintf(stderr, _("%s: recovery.conf written in to the base backup. "
+ "Please, add escaping to the password field in "
+ "\"primary_conninfo\" if needed.\n"), progname);
+
+ free(recoveryconf);
+ }
+
MemSet(zerobuf, 0, sizeof(zerobuf));
#ifdef HAVE_LIBZ
if (ztarfile != NULL)
@@ -937,6 +1005,264 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
if (copybuf != NULL)
PQfreemem(copybuf);
+
+ if (rownum == 0)
+ WriteRecoveryConf();
+}
+
+static int
+_tarChecksum(char *header)
+{
+ int i,
+ sum;
+
+ /*
+ * Per POSIX, the checksum is the simple sum of all bytes in the header,
+ * treating the bytes as unsigned, and treating the checksum field (at
+ * offset 148) as though it contained 8 spaces.
+ */
+ sum = 8 * ' '; /* presumed value for checksum field */
+ for (i = 0; i < 512; i++)
+ if (i < 148 || i >= 156)
+ sum += 0xFF & header[i];
+ return sum;
+}
+
+
+/*
+ * Utility routine to print possibly larger than 32 bit integers in a
+ * portable fashion. Filled with zeros.
+ */
+static void
+print_val(char *s, uint64 val, unsigned int base, size_t len)
+{
+ int i;
+
+ for (i = len; i > 0; i--)
+ {
+ int digit = val % base;
+
+ s[i - 1] = '0' + digit;
+ val = val / base;
+ }
+}
+
+
+static void
+_tarCreateHeader(char *header)
+{
+ /*
+ * Note: most of the fields in a tar header are not supposed to be
+ * null-terminated. We use sprintf, which will write a null after the
+ * required bytes; that null goes into the first byte of the next field.
+ * This is okay as long as we fill the fields in order.
+ */
+ memset(header, 0, 512 /* sizeof the tar header */);
+
+ /* Name 100 */
+ sprintf(&header[0], "%.99s", "recovery.conf");
+
+ /* Mode 8 */
+ sprintf(&header[100], "0000600 ");
+
+ /* User ID 8 */
+ sprintf(&header[108], "0004000 ");
+
+ /* Group 8 */
+ sprintf(&header[116], "0002000 ");
+
+ /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
+ print_val(&header[124], recoveryconf_length, 8, 11);
+ sprintf(&header[135], " ");
+
+ /* Mod Time 12 */
+ sprintf(&header[136], "%011o ", (int) time(NULL));
+
+ /* Checksum 8 cannot be calculated until we've filled all other fields */
+
+ /* Type - regular file */
+ sprintf(&header[156], "0");
+
+ /* Link Name 100 (leave as nulls) */
+
+ /* Magic 6 */
+ sprintf(&header[257], "ustar");
+
+ /* Version 2 */
+ sprintf(&header[263], "00");
+
+ /* User 32 */
+ /* XXX: Do we need to care about setting correct username? */
+ sprintf(&header[265], "%.31s", "postgres");
+
+ /* Group 32 */
+ /* XXX: Do we need to care about setting correct group name? */
+ sprintf(&header[297], "%.31s", "postgres");
+
+ /* Major Dev 8 */
+ sprintf(&header[329], "%07o ", 0);
+
+ /* Minor Dev 8 */
+ sprintf(&header[337], "%07o ", 0);
+
+ /* Prefix 155 - not used, leave as nulls */
+
+ /*
+ * We mustn't overwrite the next field while inserting the checksum.
+ * Fortunately, the checksum can't exceed 6 octal digits, so we just write
+ * 6 digits, a space, and a null, which is legal per POSIX.
+ */
+ sprintf(&header[148], "%06o ", _tarChecksum(header));
+}
+
+/*
+ * Reallocate the buffer for recovery.conf if needed
+ */
+static void
+_reallocRCBuffer(char **buf, int *bufsz, int currsz, PQconninfoOption *option, int extrasz)
+{
+ int kw_val_len;
+
+ if (option != NULL)
+ kw_val_len = strlen(option->keyword) + strlen(option->val);
+ else
+ kw_val_len = 0;
+
+ if (*bufsz < currsz + kw_val_len + extrasz)
+ {
+ char *tmp;
+
+ tmp = realloc(*buf, *bufsz + RCBUFSZ);
+ if (tmp == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1)
+ }
+ *buf = tmp;
+ *bufsz += RCBUFSZ;
+ }
+}
+
+/*
+ * Try to create recovery.conf in memory and set the length to write later.
+ */
+static void
+CreateRecoveryConf(PGconn *conn)
+{
+ char *buf = NULL;
+ PQconninfoOption *connOptions;
+ PQconninfoOption *option;
+ int bufsz;
+ int total = 0;
+ int written;
+
+ if (!writerecoveryconf)
+ return;
+
+ connOptions = PQconninfo(conn, true);
+ if (connOptions == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+
+ bufsz = RCBUFSZ;
+ buf = malloc(bufsz);
+ if (buf == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ written = sprintf(&buf[total], "standby_mode = 'on'\n");
+ if (written < 0)
+ {
+ fprintf(stderr, _("%s: cannot write to string: %s"), progname, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ total += written;
+
+ written = sprintf(&buf[total], "primary_conninfo = '");
+ if (written < 0)
+ {
+ fprintf(stderr, _("%s: cannot write to string: %s"), progname, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ total += written;
+
+ for (option = connOptions; option && option->keyword; option++)
+ {
+ /*
+ * Do not emit this setting if not set, empty or default.
+ * The list of options was already pre-filtered for options
+ * usable for replication with PQconninfo(conn, true).
+ */
+ if (option->val == NULL ||
+ (option->val != NULL && option->val[0] == '\0') ||
+ (option->val &&
+ option->compiled &&
+ strcmp(option->val, option->compiled) == 0))
+ continue;
+
+ /* Write "keyword='value'" pieces, single quotes doubled */
+ _reallocRCBuffer(&buf, &bufsz, total, option, 6);
+ written = sprintf(&buf[total], "%s=''%s'' ", option->keyword, option->val);
+ if (written < 0)
+ {
+ fprintf(stderr, _("%s: cannot write to string: %s"), progname, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ total += written;
+ }
+
+ _reallocRCBuffer(&buf, &bufsz, total, NULL, 2);
+ written = sprintf(&buf[total], "'\n");
+ if (written < 0)
+ {
+ fprintf(stderr, _("%s: cannot write to string: %s"), progname, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ total += written;
+
+ recoveryconf = buf;
+ recoveryconf_length = total;
+
+ PQconninfoFree(connOptions);
+}
+
+
+static void
+WriteRecoveryConf(void)
+{
+ char filename[MAXPGPATH];
+ FILE *cf;
+
+ if (!writerecoveryconf)
+ return;
+
+ sprintf(filename, "%s/recovery.conf", basedir);
+
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("%s: cannot create %s: %s"), progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ if (fwrite(recoveryconf, recoveryconf_length, 1, cf) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ fclose(cf);
+
+ fprintf(stderr, _("%s: recovery.conf written. "
+ "Please, add escaping to the password field in "
+ "\"primary_conninfo\" if needed.\n"), progname);
}
@@ -960,6 +1286,8 @@ BaseBackup(void)
/* Error message already written in GetConnection() */
exit(1);
+ CreateRecoveryConf(conn);
+
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
@@ -1243,6 +1571,7 @@ main(int argc, char **argv)
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
+ {"write-recovery-conf", no_argument, NULL, 'R'},
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
{"gzip", no_argument, NULL, 'z'},
@@ -1280,7 +1609,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:xX:l:zZ:c:h:p:U:s:wWvP",
+ while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
long_options, &option_index)) != -1)
{
switch (c)
@@ -1301,6 +1630,9 @@ main(int argc, char **argv)
exit(1);
}
break;
+ case 'R':
+ writerecoveryconf = true;
+ break;
case 'x':
if (includewal)
{
@@ -1466,6 +1798,13 @@ main(int argc, char **argv)
}
#endif
+ if (writerecoveryconf && includewal)
+ {
+ fprintf(stderr,
+ _("--xlog and --writerecoveryconf are mutually exclusive\n"));
+ exit(1);
+ }
+
/*
* Verify that the target directory exists, or create it. For plaintext
* backups, always require the directory. For tar backups, require it
2012-10-11 02:02 keltezéssel, Fujii Masao írta:
On Thu, Oct 11, 2012 at 3:36 AM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
2012-10-10 18:23 keltezéssel, Fujii Masao írta:
When tar output format is specified together with -R option, recovery.conf
is
not included in base.tar. I think it should.Why? This patch only promises to write the recovery.conf into the
directory specified with -D.Because it's more user-friendly. If recovery.conf is not included in base.tar,
when base.tar is extracted to disk to use the backup, a user always needs
to copy recovery.conf to the extracted directory. OTOH if it's included in
base.tar, such copy operation is not required and we can simplify the
procedures to use the backup a bit.
It's implemented now.
+ setting up the standby. Since creating a backup for a standalone + server with <option>-x</option> or <option>-X</option> and adding + a recovery.conf to it wouldn't make a working standby, these options + naturally conflict.Is this right? ISTM that basically we can use pg_basebackup -x to take
a base backup for starting the standby for now. No?I don't know. Pointers?
There is no restriction that the backup which was taken by using
pg_basebackup -x cannot be used to start the standby. So I wonder
why -R option cannot work together with -x. It's useful if recovery.conf
is automatically written even with pg_basebackup -x.
There was a problem with 9.0.x (maybe even with 9.1) that the backup
failed to come up as a standby if -x or -X was specified. I don't know
if it was a bug, limitation or intended behaviour.
Before removing the check for conflicting options, I would like to ask:
is there such a known conflict with --xlog-method=stream?
And I found another problem: when -(stdout) is specified in -D option,
recovery.conf fails to be written.$ pg_basebackup -D- -F t -c fast -R > hoge.tar
NOTICE: WAL archiving is not enabled; you must ensure that all
required WAL segments are copied through other means to complete the
backup
pg_basebackup: cannot create -/recovery.conf: No such file or directory
Now it works with recovery.conf written into the tar itself. I also tried
$ pg_basebackup -D- -Ft -R
and the directory named "-" was created and of course the recovery.conf
inside it. Is this the intended behaviour regarding "stdout is to be treated
as a directory"?
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Thanks for updating the patch!
On Sun, Oct 14, 2012 at 8:41 PM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
Backing up a standby server without -R preserves the original recovery.conf
of the
standby, it points to the standby's source server.Backing up a standby server with -R overwrites the original recovery.conf
with the new
one pointing to the standby instead of the standby's source server. Without
-Ft, it is
obvious. With -Ft, there are two recovery.conf files in the tar file and
upon extracting it,
the last written one (the one generated via -R) overwrites the original.
The tar file is always extracted such way in all platform which PostgreSQL
supports? I'm just concerned about that some tool in some platform might
prefer the original recovery.conf when extracting tar file. If the spec of tar
format specifies such behavior (i.e., the last written file of the same name
is always preferred), it's OK.
I found the bug that recovery.conf is included in the tar file of the tablespace
instead of base.tar, when there are tablespaces in the server.
Maybe this is nitpicky problem,,,, but...
If port number is not explicitly specified in pg_basebackup, the port
number is not
included to primary_conninfo in recovery.conf which is created during
the backup.
That is, the standby server using such recovery.conf tries to connect
to the default
port number because the port number is not supplied in primary_conninfo. This
assumes that the default port number is the same between the master and standby.
But this is not true. The default port number can be changed in --with-pgport
configure option, so the default port number might be different
between the master
and standby. To avoid this uncertainty, pg_basebackup -R should always include
the port number in primary_conninfo?
When the password is required to connect to the server, pg_basebackup -R
always writes the password setting into primary_conninfo in recovery.conf.
But if the password is supplied from .pgpass, ISTM that the password setting
doesn't need to be written into primary_conninfo. Right?
+ The password written into recovery.conf is not escaped even if special
+ characters appear in it. The administrator must review recovery.conf
+ to ensure proper escaping.
Is it difficult to make pg_basebackup escape the special characters in the
password? It's better if we can remove this restriction.
I've not reviewed PQconninfo patch yet. Will review.
Regards,
--
Fujii Masao
On Mon, Oct 15, 2012 at 12:57 AM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
2012-10-11 02:02 keltezéssel, Fujii Masao írta:
On Thu, Oct 11, 2012 at 3:36 AM, Boszormenyi Zoltan <zb@cybertec.at>
wrote:2012-10-10 18:23 keltezéssel, Fujii Masao írta:
When tar output format is specified together with -R option,
recovery.conf
is
not included in base.tar. I think it should.Why? This patch only promises to write the recovery.conf into the
directory specified with -D.Because it's more user-friendly. If recovery.conf is not included in
base.tar,
when base.tar is extracted to disk to use the backup, a user always needs
to copy recovery.conf to the extracted directory. OTOH if it's included in
base.tar, such copy operation is not required and we can simplify the
procedures to use the backup a bit.It's implemented now.
Thanks a lot!
+ setting up the standby. Since creating a backup for a standalone + server with <option>-x</option> or <option>-X</option> and adding + a recovery.conf to it wouldn't make a working standby, these options + naturally conflict.Is this right? ISTM that basically we can use pg_basebackup -x to take
a base backup for starting the standby for now. No?I don't know. Pointers?
There is no restriction that the backup which was taken by using
pg_basebackup -x cannot be used to start the standby. So I wonder
why -R option cannot work together with -x. It's useful if recovery.conf
is automatically written even with pg_basebackup -x.There was a problem with 9.0.x (maybe even with 9.1) that the backup
failed to come up as a standby if -x or -X was specified. I don't know
if it was a bug, limitation or intended behaviour.
It sounds a bug to me. It's helpful if you provide the self-contained test case.
Before removing the check for conflicting options, I would like to ask:
is there such a known conflict with --xlog-method=stream?
AFAIK, No, we can use the backup which pg_basebackup --xlog-method=stream
took, to start the standby. BTW, --xlog-method=stream cannot be specified
together with -F tar.
And I found another problem: when -(stdout) is specified in -D option,
recovery.conf fails to be written.$ pg_basebackup -D- -F t -c fast -R > hoge.tar
NOTICE: WAL archiving is not enabled; you must ensure that all
required WAL segments are copied through other means to complete the
backup
pg_basebackup: cannot create -/recovery.conf: No such file or directoryNow it works with recovery.conf written into the tar itself. I also tried
$ pg_basebackup -D- -Ft -R
and the directory named "-" was created and of course the recovery.conf
inside it. Is this the intended behaviour regarding "stdout is to be treated
as a directory"?
Yes. Thanks.
Regards,
--
Fujii Masao
2012-10-14 18:02 keltezéssel, Fujii Masao írta:
Thanks for updating the patch!
On Sun, Oct 14, 2012 at 8:41 PM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
Backing up a standby server without -R preserves the original recovery.conf
of the
standby, it points to the standby's source server.Backing up a standby server with -R overwrites the original recovery.conf
with the new
one pointing to the standby instead of the standby's source server. Without
-Ft, it is
obvious. With -Ft, there are two recovery.conf files in the tar file and
upon extracting it,
the last written one (the one generated via -R) overwrites the original.The tar file is always extracted such way in all platform which PostgreSQL
supports? I'm just concerned about that some tool in some platform might
prefer the original recovery.conf when extracting tar file. If the spec of tar
format specifies such behavior (i.e., the last written file of the same name
is always preferred), it's OK.
Since tar is a sequential archive format, I think this is the behaviour of
every tar extractor. But I will look at adding code to skip the original
recovery.conf if it exists in the tar file.
I found the bug that recovery.conf is included in the tar file of the tablespace
instead of base.tar, when there are tablespaces in the server.
You are right, I am looking into this. But I don't know how it got there,
I check for (rownum == 0 && writerecoveryconf) and rownum == 0
supposedly means that it's the base.tar. Looking again.
Maybe this is nitpicky problem,,,, but...
If port number is not explicitly specified in pg_basebackup, the port
number is not
included to primary_conninfo in recovery.conf which is created during
the backup.
That is, the standby server using such recovery.conf tries to connect
to the default
port number because the port number is not supplied in primary_conninfo. This
assumes that the default port number is the same between the master and standby.
But this is not true. The default port number can be changed in --with-pgport
configure option, so the default port number might be different
between the master
and standby. To avoid this uncertainty, pg_basebackup -R should always include
the port number in primary_conninfo?
I think you are right. But, I wouldn't restrict it only to the port setting.
Any of the values that are set and equal to the compiled-in default,
it should be written into recovery.conf.
When the password is required to connect to the server, pg_basebackup -R
always writes the password setting into primary_conninfo in recovery.conf.
But if the password is supplied from .pgpass, ISTM that the password setting
doesn't need to be written into primary_conninfo. Right?
How can you deduce it from the PQconninfoOption structure?
Also, if the machine you take the base backup on is different
from the one where you actually use the backup on, it can be
different not only in the --with-pgport compilation option but
in the presence of .pgpass or the PGPASSWORD envvar, too.
The administrator is there for a reason or there is no .pgpass
or PGPASSWORD at all.
+ The password written into recovery.conf is not escaped even if special + characters appear in it. The administrator must review recovery.conf + to ensure proper escaping.Is it difficult to make pg_basebackup escape the special characters in the
password? It's better if we can remove this restriction.
It's not difficult. What other characters need to be escaped besides single quotes?
I've not reviewed PQconninfo patch yet. Will review.
Thanks in advance.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Hi,
2012-10-14 18:41 keltezéssel, Boszormenyi Zoltan írta:
2012-10-14 18:02 keltezéssel, Fujii Masao írta:
Thanks for updating the patch!
On Sun, Oct 14, 2012 at 8:41 PM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
Backing up a standby server without -R preserves the original recovery.conf
of the
standby, it points to the standby's source server.Backing up a standby server with -R overwrites the original recovery.conf
with the new
one pointing to the standby instead of the standby's source server. Without
-Ft, it is
obvious. With -Ft, there are two recovery.conf files in the tar file and
upon extracting it,
the last written one (the one generated via -R) overwrites the original.The tar file is always extracted such way in all platform which PostgreSQL
supports? I'm just concerned about that some tool in some platform might
prefer the original recovery.conf when extracting tar file. If the spec of tar
format specifies such behavior (i.e., the last written file of the same name
is always preferred), it's OK.Since tar is a sequential archive format, I think this is the behaviour of
every tar extractor. But I will look at adding code to skip the original
recovery.conf if it exists in the tar file.I found the bug that recovery.conf is included in the tar file of the tablespace
instead of base.tar, when there are tablespaces in the server.You are right, I am looking into this. But I don't know how it got there,
I check for (rownum == 0 && writerecoveryconf) and rownum == 0
supposedly means that it's the base.tar. Looking again.
I made a mistake in the previous check, rownum is not reliable.
The tablespaces are sent first and base backup as the last.
Now recovery.conf is written into base.tar.
Maybe this is nitpicky problem,,,, but...
If port number is not explicitly specified in pg_basebackup, the port
number is not
included to primary_conninfo in recovery.conf which is created during
the backup.
That is, the standby server using such recovery.conf tries to connect
to the default
port number because the port number is not supplied in primary_conninfo. This
assumes that the default port number is the same between the master and standby.
But this is not true. The default port number can be changed in --with-pgport
configure option, so the default port number might be different
between the master
and standby. To avoid this uncertainty, pg_basebackup -R should always include
the port number in primary_conninfo?I think you are right. But, I wouldn't restrict it only to the port setting.
Any of the values that are set and equal to the compiled-in default,
it should be written into recovery.conf.
Now all values that are set (even those being equal to the compiled-in default)
are put into recovery.conf.
When the password is required to connect to the server, pg_basebackup -R
always writes the password setting into primary_conninfo in recovery.conf.
But if the password is supplied from .pgpass, ISTM that the password setting
doesn't need to be written into primary_conninfo. Right?How can you deduce it from the PQconninfoOption structure?
Also, if the machine you take the base backup on is different
from the one where you actually use the backup on, it can be
different not only in the --with-pgport compilation option but
in the presence of .pgpass or the PGPASSWORD envvar, too.
The administrator is there for a reason or there is no .pgpass
or PGPASSWORD at all.+ The password written into recovery.conf is not escaped even if special + characters appear in it. The administrator must review recovery.conf + to ensure proper escaping.Is it difficult to make pg_basebackup escape the special characters in the
password? It's better if we can remove this restriction.It's not difficult. What other characters need to be escaped besides single quotes?
All written values are escaped.
Other changes: the recovery.conf in base.tar is correctly skipped if it exists
and -R is given. The new recovery.conf is written with padding to round up to
512, the TAR chunk size.
The PQconninfo patch is also attached but didn't change since the last mail.
I've not reviewed PQconninfo patch yet. Will review.
Thanks in advance.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
01-PQconninfo-v13.patchtext/x-patch; name=01-PQconninfo-v13.patchDownload
diff -durpN postgresql/doc/src/sgml/libpq.sgml postgresql.1/doc/src/sgml/libpq.sgml
--- postgresql/doc/src/sgml/libpq.sgml 2012-08-03 09:39:30.114266570 +0200
+++ postgresql.1/doc/src/sgml/libpq.sgml 2012-10-14 11:12:07.049196259 +0200
@@ -496,6 +496,37 @@ typedef struct
</listitem>
</varlistentry>
+ <varlistentry id="libpq-pqconninfo">
+ <term><function>PQconninfo</function><indexterm><primary>PQconninfo</></></term>
+ <listitem>
+ <para>
+ Returns the connection options used by a live connection.
+<synopsis>
+PQconninfoOption *PQconninfo(PGconn *conn, bool for_replication);
+</synopsis>
+ </para>
+
+ <para>
+ Returns a connection options array. This can be used to determine
+ all possible <function>PQconnectdb</function> options and their
+ current values that were used to connect to the server. The return
+ value points to an array of <structname>PQconninfoOption</structname>
+ structures, which ends with an entry having a null <structfield>keyword</>
+ pointer. Every notes above for <function>PQconndefaults</function> also apply.
+ </para>
+
+ <para>
+ The <literal>for_replication</> argument can be used to exclude some
+ options from the list which are used by the walreceiver module.
+ <application>pg_basebackup</application> uses this pre-filtered list
+ to construct <literal>primary_conninfo</> in the automatically generated
+ recovery.conf file.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+
<varlistentry id="libpq-pqconninfoparse">
<term><function>PQconninfoParse</function><indexterm><primary>PQconninfoParse</></></term>
<listitem>
diff -durpN postgresql/src/interfaces/libpq/exports.txt postgresql.1/src/interfaces/libpq/exports.txt
--- postgresql/src/interfaces/libpq/exports.txt 2012-10-09 09:58:14.342782974 +0200
+++ postgresql.1/src/interfaces/libpq/exports.txt 2012-10-14 11:12:07.049196259 +0200
@@ -164,3 +164,4 @@ PQsetSingleRowMode 161
lo_lseek64 162
lo_tell64 163
lo_truncate64 164
+PQconninfo 165
diff -durpN postgresql/src/interfaces/libpq/fe-connect.c postgresql.1/src/interfaces/libpq/fe-connect.c
--- postgresql/src/interfaces/libpq/fe-connect.c 2012-09-09 08:11:09.470401480 +0200
+++ postgresql.1/src/interfaces/libpq/fe-connect.c 2012-10-14 11:12:07.053196284 +0200
@@ -137,6 +137,9 @@ static int ldapServiceLookup(const char
* PQconninfoOptions[] *must* be NULL. In a working copy, non-null "val"
* fields point to malloc'd strings that should be freed when the working
* array is freed (see PQconninfoFree).
+ *
+ * If you add a new connection option to this list, remember to add it to
+ * PQconninfoMappings[] below.
* ----------
*/
static const PQconninfoOption PQconninfoOptions[] = {
@@ -264,6 +267,66 @@ static const PQconninfoOption PQconninfo
NULL, NULL, 0}
};
+/*
+ * We need a mapping between the PQconninfoOptions[] array
+ * and PGconn members. We have to keep it separate from
+ * PQconninfoOptions[] to not leak info about PGconn members
+ * to clients.
+ */
+typedef struct PQconninfoMapping {
+ char *keyword;
+ size_t member_offset;
+ bool for_replication;
+ /*
+ * Special and simplistic value mapping between
+ * PQconninfoOption and PGconn. Only used by "requiressl".
+ */
+ char *conninfoValue;
+ char *connValue;
+} PQconninfoMapping;
+#define PGCONNMEMBERADDR(conn, mapping) ((char **)((char *)conn + mapping->member_offset))
+
+static const PQconninfoMapping PQconninfoMappings[] =
+{
+ /* "authtype" is not used anymore, there is no mapping to PGconn */
+ /* there is no mapping of "service" to PGconn */
+ { "user", offsetof(struct pg_conn, pguser), true, NULL, NULL },
+ { "password", offsetof(struct pg_conn, pgpass), true, NULL, NULL },
+ { "connect_timeout", offsetof(struct pg_conn, connect_timeout), true, NULL, NULL },
+ { "dbname", offsetof(struct pg_conn, dbName), false, NULL, NULL },
+ { "host", offsetof(struct pg_conn, pghost), true, NULL, NULL },
+ { "hostaddr", offsetof(struct pg_conn, pghostaddr), true, NULL, NULL },
+ { "port", offsetof(struct pg_conn, pgport), true, NULL, NULL },
+ { "client_encoding", offsetof(struct pg_conn, client_encoding_initial), false, NULL, NULL },
+ { "tty", offsetof(struct pg_conn, pgtty), false, NULL, NULL },
+ { "options", offsetof(struct pg_conn, pgoptions), true, NULL, NULL },
+ { "application_name", offsetof(struct pg_conn, appname), false, NULL, NULL },
+ { "fallback_application_name", offsetof(struct pg_conn, fbappname), false, NULL, NULL },
+ { "keepalives", offsetof(struct pg_conn, keepalives), true, NULL, NULL },
+ { "keepalives_idle", offsetof(struct pg_conn, keepalives_idle), true, NULL, NULL },
+ { "keepalives_interval", offsetof(struct pg_conn, keepalives_interval), true, NULL, NULL },
+ { "keepalives_count", offsetof(struct pg_conn, keepalives_count), true, NULL, NULL },
+#ifdef USE_SSL
+ { "requiressl", offsetof(struct pg_conn, sslmode), false, "1", "require" },
+#endif
+ { "sslmode", offsetof(struct pg_conn, sslmode), true, NULL, NULL },
+ { "sslcompression", offsetof(struct pg_conn, sslcompression), true, NULL, NULL },
+ { "sslcert", offsetof(struct pg_conn, sslcert), true, NULL, NULL },
+ { "sslkey", offsetof(struct pg_conn, sslkey), true, NULL, NULL },
+ { "sslrootcert", offsetof(struct pg_conn, sslrootcert), true, NULL, NULL },
+ { "sslcrl", offsetof(struct pg_conn, sslcrl), true, NULL, NULL },
+ { "requirepeer", offsetof(struct pg_conn, requirepeer), true, NULL, NULL },
+#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
+ { "krbsrvname", offsetof(struct pg_conn, krbsrvname), true, NULL, NULL },
+#endif
+#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
+ { "gsslib", offsetof(struct pg_conn, requirepeer), true, NULL, NULL },
+#endif
+ { "replication", offsetof(struct pg_conn, replication), false, NULL, NULL },
+ /* Terminating entry --- MUST BE LAST */
+ { NULL, 0, false, NULL, NULL }
+};
+
static const PQEnvironmentOption EnvironmentOptions[] =
{
/* common user-interface settings */
@@ -295,7 +358,8 @@ static PGconn *makeEmptyPGconn(void);
static void fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
-static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage);
+static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage,
+ bool for_replication);
static PQconninfoOption *parse_connection_string(const char *conninfo,
PQExpBuffer errorMessage, bool use_defaults);
static int uri_prefix_length(const char *connstr);
@@ -318,6 +382,8 @@ static char *conninfo_uri_decode(const c
static bool get_hexdigit(char digit, int *value);
static const char *conninfo_getval(PQconninfoOption *connOptions,
const char *keyword);
+static void conninfo_setval(PQconninfoOption *connOptions,
+ const char *keyword, const char *val);
static PQconninfoOption *conninfo_storeval(PQconninfoOption *connOptions,
const char *keyword, const char *value,
PQExpBuffer errorMessage, bool ignoreMissing, bool uri_decode);
@@ -627,7 +693,9 @@ PQconnectStart(const char *conninfo)
static void
fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
{
+ const PQconninfoMapping *mapping;
const char *tmp;
+ char **memberaddr;
/*
* Move option values into conn structure
@@ -637,72 +705,24 @@ fillPGconn(PGconn *conn, PQconninfoOptio
*
* XXX: probably worth checking strdup() return value here...
*/
- tmp = conninfo_getval(connOptions, "hostaddr");
- conn->pghostaddr = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "host");
- conn->pghost = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "port");
- conn->pgport = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "tty");
- conn->pgtty = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "options");
- conn->pgoptions = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "application_name");
- conn->appname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "fallback_application_name");
- conn->fbappname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "dbname");
- conn->dbName = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "user");
- conn->pguser = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "password");
- conn->pgpass = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "connect_timeout");
- conn->connect_timeout = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "client_encoding");
- conn->client_encoding_initial = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives");
- conn->keepalives = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_idle");
- conn->keepalives_idle = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_interval");
- conn->keepalives_interval = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_count");
- conn->keepalives_count = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslmode");
- conn->sslmode = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcompression");
- conn->sslcompression = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslkey");
- conn->sslkey = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcert");
- conn->sslcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslrootcert");
- conn->sslrootcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcrl");
- conn->sslcrl = tmp ? strdup(tmp) : NULL;
-#ifdef USE_SSL
- tmp = conninfo_getval(connOptions, "requiressl");
- if (tmp && tmp[0] == '1')
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
{
- /* here warn that the requiressl option is deprecated? */
- if (conn->sslmode)
- free(conn->sslmode);
- conn->sslmode = strdup("require");
+ tmp = conninfo_getval(connOptions, mapping->keyword);
+ memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ if (mapping->conninfoValue && mapping->connValue)
+ {
+ size_t len = strlen(mapping->conninfoValue);
+ if (tmp && strncmp(tmp, mapping->conninfoValue, len) == 0)
+ {
+ if (*memberaddr)
+ free(*memberaddr);
+ *memberaddr = strdup(mapping->connValue);
+ }
+ }
+ else
+ *memberaddr = tmp ? strdup(tmp) : NULL;
}
-#endif
- tmp = conninfo_getval(connOptions, "requirepeer");
- conn->requirepeer = tmp ? strdup(tmp) : NULL;
-#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "krbsrvname");
- conn->krbsrvname = tmp ? strdup(tmp) : NULL;
-#endif
-#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "gsslib");
- conn->gsslib = tmp ? strdup(tmp) : NULL;
-#endif
- tmp = conninfo_getval(connOptions, "replication");
- conn->replication = tmp ? strdup(tmp) : NULL;
}
/*
@@ -884,7 +904,7 @@ PQconndefaults(void)
if (PQExpBufferDataBroken(errorBuf))
return NULL; /* out of memory already :-( */
- connOptions = conninfo_init(&errorBuf);
+ connOptions = conninfo_init(&errorBuf, false);
if (connOptions != NULL)
{
if (!conninfo_add_defaults(connOptions, &errorBuf))
@@ -4006,9 +4026,11 @@ PQconninfoParse(const char *conninfo, ch
/*
* Build a working copy of the constant PQconninfoOptions array.
+ * If for_replication is true, only return the options that are
+ * not added by libpqwalreceiver.
*/
static PQconninfoOption *
-conninfo_init(PQExpBuffer errorMessage)
+conninfo_init(PQExpBuffer errorMessage, bool for_replication)
{
PQconninfoOption *options;
@@ -4019,7 +4041,28 @@ conninfo_init(PQExpBuffer errorMessage)
libpq_gettext("out of memory\n"));
return NULL;
}
- memcpy(options, PQconninfoOptions, sizeof(PQconninfoOptions));
+ if (for_replication)
+ {
+ const PQconninfoMapping *mapping = PQconninfoMappings;
+ PQconninfoOption *opt_dest = options;
+
+ while (mapping->keyword)
+ {
+ PQconninfoOption *opt_src = conninfo_find((PQconninfoOption *)PQconninfoOptions, mapping->keyword);
+
+ if (opt_src && mapping->for_replication)
+ {
+ memcpy(opt_dest, opt_src, sizeof(PQconninfoOption));
+ opt_dest++;
+ }
+
+ opt_src++;
+ mapping++;
+ }
+ MemSet(opt_dest, 0, sizeof(PQconninfoOption));
+ }
+ else
+ memcpy(options, PQconninfoOptions, sizeof(PQconninfoOptions));
return options;
}
@@ -4095,7 +4138,7 @@ conninfo_parse(const char *conninfo, PQE
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
return NULL;
@@ -4295,7 +4338,7 @@ conninfo_array_parse(const char *const *
}
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
{
PQconninfoFree(dbname_options);
@@ -4485,7 +4528,7 @@ conninfo_uri_parse(const char *uri, PQEx
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
return NULL;
@@ -4985,6 +5028,24 @@ conninfo_getval(PQconninfoOption *connOp
}
/*
+ * Set an option value corresponding to the keyword in the connOptions array.
+ */
+static void
+conninfo_setval(PQconninfoOption *connOptions, const char *keyword,
+ const char *val)
+{
+ PQconninfoOption *option;
+
+ option = conninfo_find(connOptions, keyword);
+ if (option)
+ {
+ if (option->val)
+ free(option->val);
+ option->val = val ? strdup(val) : NULL;
+ }
+}
+
+/*
* Store a (new) value for an option corresponding to the keyword in
* connOptions array.
*
@@ -5066,6 +5127,50 @@ conninfo_find(PQconninfoOption *connOpti
}
+/*
+ * Return the connection options used for the connections
+ */
+PQconninfoOption *
+PQconninfo(PGconn *conn, bool for_replication)
+{
+ PQExpBufferData errorBuf;
+ PQconninfoOption *connOptions;
+
+ if (conn == NULL)
+ return NULL;
+
+ /* We don't actually report any errors here, but callees want a buffer */
+ initPQExpBuffer(&errorBuf);
+ if (PQExpBufferDataBroken(errorBuf))
+ return NULL; /* out of memory already :-( */
+
+ connOptions = conninfo_init(&errorBuf, for_replication);
+
+ termPQExpBuffer(&errorBuf);
+
+ if (connOptions != NULL)
+ {
+ const PQconninfoMapping *mapping;
+
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
+ {
+ char **memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ if (mapping->conninfoValue && mapping->connValue)
+ {
+ size_t len = strlen(mapping->connValue);
+ if (*memberaddr && strncmp(*memberaddr, mapping->connValue, len) == 0)
+ conninfo_setval(connOptions, mapping->keyword, mapping->conninfoValue);
+ }
+ else
+ conninfo_setval(connOptions, mapping->keyword, *memberaddr);
+ }
+ }
+
+ return connOptions;
+}
+
+
void
PQconninfoFree(PQconninfoOption *connOptions)
{
diff -durpN postgresql/src/interfaces/libpq/libpq-fe.h postgresql.1/src/interfaces/libpq/libpq-fe.h
--- postgresql/src/interfaces/libpq/libpq-fe.h 2012-10-09 09:58:14.343782980 +0200
+++ postgresql.1/src/interfaces/libpq/libpq-fe.h 2012-10-14 11:12:07.054196291 +0200
@@ -262,6 +262,9 @@ extern PQconninfoOption *PQconndefaults(
/* parse connection options in same way as PQconnectdb */
extern PQconninfoOption *PQconninfoParse(const char *conninfo, char **errmsg);
+/* return the connection options used by a live connection */
+extern PQconninfoOption *PQconninfo(PGconn *conn, bool for_replication);
+
/* free the data structure returned by PQconndefaults() or PQconninfoParse() */
extern void PQconninfoFree(PQconninfoOption *connOptions);
02-pg_basebackup-v12.patchtext/x-patch; name=02-pg_basebackup-v12.patchDownload
diff -durpN postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml
--- postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml 2012-08-24 09:49:22.960530329 +0200
+++ postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml 2012-10-14 13:09:58.913513268 +0200
@@ -189,6 +189,37 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--write-recovery-conf</option></term>
+ <listitem>
+
+ <para>
+ Write a minimal recovery.conf into the output directory (or into
+ the base archive file if <option>--format=tar</option> was specified)
+ using the connection parameters from the command line to ease
+ setting up the standby. Since creating a backup for a standalone
+ server with <option>-x</option> or <option>-X</option> and adding
+ a recovery.conf to it wouldn't make a working standby, these options
+ naturally conflict.
+ </para>
+
+ <para>
+ The password written into recovery.conf is not escaped even if special
+ characters appear in it. The administrator must review recovery.conf
+ to ensure proper escaping.
+ </para>
+
+ <para>
+ When this option is specified and taking the base backup succeeded,
+ failing to write the <filename>recovery.conf</filename> results
+ in the error code 2. This way, scripts can distinguish between different
+ failure cases of <application>pg_basebackup</application>.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-x</option></term>
<term><option>--xlog</option></term>
<listitem>
diff -durpN postgresql.1/src/bin/pg_basebackup/pg_basebackup.c postgresql.2/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.1/src/bin/pg_basebackup/pg_basebackup.c 2012-10-03 10:40:48.297207389 +0200
+++ postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2012-10-14 22:05:36.681471631 +0200
@@ -25,6 +25,7 @@
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
+#include <time.h>
#ifdef HAVE_LIBZ
#include <zlib.h>
@@ -46,6 +47,7 @@ int compresslevel = 0;
bool includewal = false;
bool streamwal = false;
bool fastcheckpoint = false;
+bool writerecoveryconf = false;
int standby_message_timeout = 10 * 1000; /* 10 sec = default */
/* Progress counters */
@@ -70,6 +72,11 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
+#define TARCHUNKSZ (512)
+#define RCBUFSZ TARCHUNKSZ
+static char *recoveryconf = NULL;
+static size_t recoveryconflen = 0;
+
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
@@ -77,6 +84,15 @@ static void progress_report(int tablespa
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
+static void _reallocRCBuffer(char **buf, int *bufsz, int currsz,
+ char *keyword, char *value,
+ int extrasz);
+static void CreateRecoveryConf(PGconn *conn);
+static void WriteRecoveryConf(void);
+static int _tarChecksum(char *header);
+static void print_val(char *s, uint64 val, unsigned int base, size_t len);
+static void scan_val(char *s, uint64 *val, unsigned int base, size_t len);
+static void _tarCreateHeader(char *header);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -107,6 +123,8 @@ usage(void)
printf(_("\nOptions controlling the output:\n"));
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
+ printf(_(" -R, --write-recovery-conf\n"
+ " write recovery.conf after backup\n"));
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
printf(_(" -X, --xlog-method=fetch|stream\n"
" include required WAL files with specified method\n"));
@@ -467,6 +485,11 @@ ReceiveTarFile(PGconn *conn, PGresult *r
char filename[MAXPGPATH];
char *copybuf = NULL;
FILE *tarfile = NULL;
+ char tarhdr[512];
+ bool basebackup;
+ bool in_tarhdr, skip_file;
+ size_t tarhdrsz;
+ uint64 filesz;
#ifdef HAVE_LIBZ
gzFile ztarfile = NULL;
@@ -519,6 +542,8 @@ ReceiveTarFile(PGconn *conn, PGresult *r
tarfile = fopen(filename, "wb");
}
}
+
+ basebackup = true;
}
else
{
@@ -547,6 +572,8 @@ ReceiveTarFile(PGconn *conn, PGresult *r
PQgetvalue(res, rownum, 0));
tarfile = fopen(filename, "wb");
}
+
+ basebackup = false;
}
#ifdef HAVE_LIBZ
@@ -584,6 +611,10 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = filesz = 0;
+
while (1)
{
int r;
@@ -598,13 +629,69 @@ ReceiveTarFile(PGconn *conn, PGresult *r
if (r == -1)
{
/*
- * End of chunk. Close file (but not stdout).
+ * End of chunk. Write recovery.conf into the tar file (if it
+ * was requested) and close file (but not stdout).
*
* Also, write two completely empty blocks at the end of the tar
* file, as required by some tar programs.
*/
char zerobuf[1024];
+ if (basebackup && writerecoveryconf)
+ {
+ char header[512];
+ int lastchunk;
+
+ _tarCreateHeader(header);
+
+ lastchunk = recoveryconflen % TARCHUNKSZ;
+ if (lastchunk > 0)
+ recoveryconflen += (TARCHUNKSZ - lastchunk);
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (gzwrite(ztarfile, header, sizeof(header)) !=
+ sizeof(header))
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, filename, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+
+ if (gzwrite(ztarfile, recoveryconf, recoveryconflen) !=
+ recoveryconflen)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, filename, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (fwrite(header, sizeof(header), 1, tarfile) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ if (fwrite(recoveryconf, recoveryconflen, 1, tarfile) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ }
+ fprintf(stderr, _("%s: recovery.conf written into '%s'\n"), progname, filename);
+
+ free(recoveryconf);
+ }
+
MemSet(zerobuf, 0, sizeof(zerobuf));
#ifdef HAVE_LIBZ
if (ztarfile != NULL)
@@ -665,25 +752,130 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
-#ifdef HAVE_LIBZ
- if (ztarfile != NULL)
+ if (basebackup && writerecoveryconf)
{
- if (gzwrite(ztarfile, copybuf, r) != r)
+ int rr = r;
+ int pos = 0;
+
+ while (rr > 0)
{
- fprintf(stderr,
- _("%s: could not write to compressed file \"%s\": %s\n"),
- progname, filename, get_gz_error(ztarfile));
- disconnect_and_exit(1);
+ if (in_tarhdr)
+ {
+ if (tarhdrsz < 512)
+ {
+ int hdrleft, bytes2copy;
+
+ hdrleft = TARCHUNKSZ - tarhdrsz;
+ bytes2copy = (rr > hdrleft ? hdrleft : rr);
+
+ memcpy(&tarhdr[tarhdrsz], copybuf + pos, bytes2copy);
+
+ rr -= bytes2copy;
+ pos += bytes2copy;
+ tarhdrsz += bytes2copy;
+ }
+ else
+ {
+ int lastchunk;
+
+ skip_file = (strcmp(&tarhdr[0], "recovery.conf") == 0);
+
+ scan_val(&tarhdr[124], &filesz, 8, 11);
+ lastchunk = filesz % TARCHUNKSZ;
+ if (lastchunk > 0)
+ filesz += (TARCHUNKSZ - lastchunk);
+
+ in_tarhdr = false;
+ tarhdrsz = 0;
+
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (!skip_file && gzwrite(ztarfile, tarhdr, TARCHUNKSZ) != TARCHUNKSZ)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, filename, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (!skip_file && fwrite(tarhdr, TARCHUNKSZ, 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ }
+ }
+ }
+ else /* inside a file */
+ {
+ if (filesz > 0)
+ {
+ int bytes2write;
+
+ bytes2write = (filesz > rr ? rr : filesz);
+
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (!skip_file && gzwrite(ztarfile, copybuf + pos, bytes2write) != bytes2write)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, filename, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (!skip_file && fwrite(copybuf + pos, bytes2write, 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ }
+
+ rr -= bytes2write;
+ pos += bytes2write;
+ filesz -= bytes2write;
+ }
+ else
+ {
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = filesz = 0;
+ }
+ }
}
}
else
-#endif
{
- if (fwrite(copybuf, r, 1, tarfile) != 1)
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
{
- fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
- progname, filename, strerror(errno));
- disconnect_and_exit(1);
+ if (gzwrite(ztarfile, copybuf, r) != r)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, filename, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (fwrite(copybuf, r, 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
}
}
totaldone += r;
@@ -712,13 +904,20 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
char filename[MAXPGPATH];
int current_len_left;
int current_padding = 0;
+ bool basebackup;
char *copybuf = NULL;
FILE *file = NULL;
if (PQgetisnull(res, rownum, 0))
+ {
strcpy(current_path, basedir);
+ basebackup = true;
+ }
else
+ {
strcpy(current_path, PQgetvalue(res, rownum, 1));
+ basebackup = false;
+ }
/*
* Get the COPY data
@@ -937,6 +1136,328 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
if (copybuf != NULL)
PQfreemem(copybuf);
+
+ if (basebackup)
+ WriteRecoveryConf();
+}
+
+static int
+_tarChecksum(char *header)
+{
+ int i,
+ sum;
+
+ /*
+ * Per POSIX, the checksum is the simple sum of all bytes in the header,
+ * treating the bytes as unsigned, and treating the checksum field (at
+ * offset 148) as though it contained 8 spaces.
+ */
+ sum = 8 * ' '; /* presumed value for checksum field */
+ for (i = 0; i < 512; i++)
+ if (i < 148 || i >= 156)
+ sum += 0xFF & header[i];
+ return sum;
+}
+
+
+/*
+ * Utility routine to print possibly larger than 32 bit integers in a
+ * portable fashion. Filled with zeros.
+ */
+static void
+print_val(char *s, uint64 val, unsigned int base, size_t len)
+{
+ int i;
+
+ for (i = len; i > 0; i--)
+ {
+ int digit = val % base;
+
+ s[i - 1] = '0' + digit;
+ val = val / base;
+ }
+}
+
+
+/*
+ * Inverse for print_val()
+ */
+static void
+scan_val(char *s, uint64 *val, unsigned int base, size_t len)
+{
+ uint64 tmp = 0;
+ int i;
+
+ for (i = 0; i < len; i++)
+ {
+ int digit = s[i] - '0';
+
+ tmp = tmp * base + digit;
+ }
+
+ *val = tmp;
+}
+
+
+static void
+_tarCreateHeader(char *header)
+{
+ /*
+ * Note: most of the fields in a tar header are not supposed to be
+ * null-terminated. We use sprintf, which will write a null after the
+ * required bytes; that null goes into the first byte of the next field.
+ * This is okay as long as we fill the fields in order.
+ */
+ memset(header, 0, 512 /* sizeof the tar header */);
+
+ /* Name 100 */
+ sprintf(&header[0], "%.99s", "recovery.conf");
+
+ /* Mode 8 */
+ sprintf(&header[100], "0000600 ");
+
+ /* User ID 8 */
+ sprintf(&header[108], "0004000 ");
+
+ /* Group 8 */
+ sprintf(&header[116], "0002000 ");
+
+ /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
+ print_val(&header[124], recoveryconflen, 8, 11);
+ sprintf(&header[135], " ");
+
+ /* Mod Time 12 */
+ sprintf(&header[136], "%011o ", (int) time(NULL));
+
+ /* Checksum 8 cannot be calculated until we've filled all other fields */
+
+ /* Type - regular file */
+ sprintf(&header[156], "0");
+
+ /* Link Name 100 (leave as nulls) */
+
+ /* Magic 6 */
+ sprintf(&header[257], "ustar");
+
+ /* Version 2 */
+ sprintf(&header[263], "00");
+
+ /* User 32 */
+ /* XXX: Do we need to care about setting correct username? */
+ sprintf(&header[265], "%.31s", "postgres");
+
+ /* Group 32 */
+ /* XXX: Do we need to care about setting correct group name? */
+ sprintf(&header[297], "%.31s", "postgres");
+
+ /* Major Dev 8 */
+ sprintf(&header[329], "%07o ", 0);
+
+ /* Minor Dev 8 */
+ sprintf(&header[337], "%07o ", 0);
+
+ /* Prefix 155 - not used, leave as nulls */
+
+ /*
+ * We mustn't overwrite the next field while inserting the checksum.
+ * Fortunately, the checksum can't exceed 6 octal digits, so we just write
+ * 6 digits, a space, and a null, which is legal per POSIX.
+ */
+ sprintf(&header[148], "%06o ", _tarChecksum(header));
+}
+
+/*
+ * Escape single quotes in a string
+ */
+static char *
+escape_string(char *value)
+{
+ char *tmp, *escaped, *esc;
+ int num_quotes;
+
+ tmp = value;
+ num_quotes = 0;
+ while (*tmp)
+ {
+ if (*tmp == '\'')
+ num_quotes++;
+ tmp++;
+ }
+
+ escaped = malloc((tmp - value) + num_quotes + 1);
+ if (escaped == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1)
+ }
+
+ tmp = value;
+ esc = escaped;
+ while (*tmp)
+ {
+ if (*tmp == '\'')
+ *esc++ = '\\';
+ *esc++ = *tmp++;
+ }
+ *esc = '\0';
+
+ return escaped;
+}
+
+/*
+ * Reallocate the buffer for recovery.conf if needed
+ */
+static void
+_reallocRCBuffer(char **buf, int *bufsz, int currsz, char *keyword, char *value, int extrasz)
+{
+ int len = extrasz;
+
+ if (keyword != NULL)
+ len += strlen(keyword);
+ if (value != NULL)
+ len += strlen(value);
+
+ if (*bufsz < currsz + len)
+ {
+ char *tmp;
+
+ tmp = realloc(*buf, *bufsz + RCBUFSZ);
+ if (tmp == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1)
+ }
+ *buf = tmp;
+ *bufsz += RCBUFSZ;
+ memset(tmp + currsz, 0, *bufsz - currsz);
+ }
+}
+
+/*
+ * Try to create recovery.conf in memory and set the length to write later.
+ */
+static void
+CreateRecoveryConf(PGconn *conn)
+{
+ char *buf = NULL;
+ PQconninfoOption *connOptions;
+ PQconninfoOption *option;
+ int bufsz;
+ int total = 0;
+ int written;
+
+ if (!writerecoveryconf)
+ return;
+
+ connOptions = PQconninfo(conn, true);
+ if (connOptions == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+
+ bufsz = RCBUFSZ;
+ buf = malloc(bufsz);
+ if (buf == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ written = sprintf(&buf[total], "standby_mode = 'on'\n");
+ if (written < 0)
+ {
+ fprintf(stderr, _("%s: cannot write to string: %s"), progname, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ total += written;
+
+ written = sprintf(&buf[total], "primary_conninfo = '");
+ if (written < 0)
+ {
+ fprintf(stderr, _("%s: cannot write to string: %s"), progname, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ total += written;
+
+ for (option = connOptions; option && option->keyword; option++)
+ {
+ char *escaped;
+ /*
+ * Do not emit this setting if not set or empty.
+ * The list of options was already pre-filtered for options
+ * usable for replication with PQconninfo(conn, true).
+ */
+ if ((option->val == NULL) ||
+ (option->val != NULL && option->val[0] == '\0'))
+ continue;
+
+ /*
+ * Write "keyword='value'" pieces, the value string is escaped
+ * if necessary and doubled single quotes around the value string.
+ */
+ escaped = escape_string(option->val);
+
+ _reallocRCBuffer(&buf, &bufsz, total, option->keyword, escaped, 6);
+
+ written = sprintf(&buf[total], "%s=''%s'' ", option->keyword, option->val);
+
+ free(escaped);
+
+ if (written < 0)
+ {
+ fprintf(stderr, _("%s: cannot write to string: %s"), progname, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ total += written;
+ }
+
+ _reallocRCBuffer(&buf, &bufsz, total, NULL, NULL, 2);
+ written = sprintf(&buf[total], "'\n");
+ if (written < 0)
+ {
+ fprintf(stderr, _("%s: cannot write to string: %s"), progname, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ total += written;
+
+ recoveryconf = buf;
+ recoveryconflen = total;
+
+ PQconninfoFree(connOptions);
+}
+
+
+static void
+WriteRecoveryConf(void)
+{
+ char filename[MAXPGPATH];
+ FILE *cf;
+
+ if (!writerecoveryconf)
+ return;
+
+ sprintf(filename, "%s/recovery.conf", basedir);
+
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("%s: cannot create %s: %s"), progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ if (fwrite(recoveryconf, recoveryconflen, 1, cf) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ fclose(cf);
+
+ fprintf(stderr, _("%s: recovery.conf written.\n"), progname);
}
@@ -960,6 +1481,8 @@ BaseBackup(void)
/* Error message already written in GetConnection() */
exit(1);
+ CreateRecoveryConf(conn);
+
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
@@ -1243,6 +1766,7 @@ main(int argc, char **argv)
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
+ {"write-recovery-conf", no_argument, NULL, 'R'},
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
{"gzip", no_argument, NULL, 'z'},
@@ -1280,7 +1804,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:xX:l:zZ:c:h:p:U:s:wWvP",
+ while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
long_options, &option_index)) != -1)
{
switch (c)
@@ -1301,6 +1825,9 @@ main(int argc, char **argv)
exit(1);
}
break;
+ case 'R':
+ writerecoveryconf = true;
+ break;
case 'x':
if (includewal)
{
2012-10-14 22:23 keltezéssel, Boszormenyi Zoltan írta:
Hi,
2012-10-14 18:41 keltezéssel, Boszormenyi Zoltan írta:
2012-10-14 18:02 keltezéssel, Fujii Masao írta:
Thanks for updating the patch!
On Sun, Oct 14, 2012 at 8:41 PM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
Backing up a standby server without -R preserves the original recovery.conf
of the
standby, it points to the standby's source server.Backing up a standby server with -R overwrites the original recovery.conf
with the new
one pointing to the standby instead of the standby's source server. Without
-Ft, it is
obvious. With -Ft, there are two recovery.conf files in the tar file and
upon extracting it,
the last written one (the one generated via -R) overwrites the original.The tar file is always extracted such way in all platform which PostgreSQL
supports? I'm just concerned about that some tool in some platform might
prefer the original recovery.conf when extracting tar file. If the spec of tar
format specifies such behavior (i.e., the last written file of the same name
is always preferred), it's OK.Since tar is a sequential archive format, I think this is the behaviour of
every tar extractor. But I will look at adding code to skip the original
recovery.conf if it exists in the tar file.I found the bug that recovery.conf is included in the tar file of the tablespace
instead of base.tar, when there are tablespaces in the server.You are right, I am looking into this. But I don't know how it got there,
I check for (rownum == 0 && writerecoveryconf) and rownum == 0
supposedly means that it's the base.tar. Looking again.I made a mistake in the previous check, rownum is not reliable.
The tablespaces are sent first and base backup as the last.
Now recovery.conf is written into base.tar.Maybe this is nitpicky problem,,,, but...
If port number is not explicitly specified in pg_basebackup, the port
number is not
included to primary_conninfo in recovery.conf which is created during
the backup.
That is, the standby server using such recovery.conf tries to connect
to the default
port number because the port number is not supplied in primary_conninfo. This
assumes that the default port number is the same between the master and standby.
But this is not true. The default port number can be changed in --with-pgport
configure option, so the default port number might be different
between the master
and standby. To avoid this uncertainty, pg_basebackup -R should always include
the port number in primary_conninfo?I think you are right. But, I wouldn't restrict it only to the port setting.
Any of the values that are set and equal to the compiled-in default,
it should be written into recovery.conf.Now all values that are set (even those being equal to the compiled-in default)
are put into recovery.conf.When the password is required to connect to the server, pg_basebackup -R
always writes the password setting into primary_conninfo in recovery.conf.
But if the password is supplied from .pgpass, ISTM that the password setting
doesn't need to be written into primary_conninfo. Right?How can you deduce it from the PQconninfoOption structure?
Also, if the machine you take the base backup on is different
from the one where you actually use the backup on, it can be
different not only in the --with-pgport compilation option but
in the presence of .pgpass or the PGPASSWORD envvar, too.
The administrator is there for a reason or there is no .pgpass
or PGPASSWORD at all.+ The password written into recovery.conf is not escaped even if special + characters appear in it. The administrator must review recovery.conf + to ensure proper escaping.Is it difficult to make pg_basebackup escape the special characters in the
password? It's better if we can remove this restriction.It's not difficult. What other characters need to be escaped besides single quotes?
All written values are escaped.
Other changes: the recovery.conf in base.tar is correctly skipped if it exists
and -R is given. The new recovery.conf is written with padding to round up to
512, the TAR chunk size.
Also, the check for conflict between -R and -x/-X is now removed.
The PQconninfo patch is also attached but didn't change since the last mail.
I've not reviewed PQconninfo patch yet. Will review.
Thanks in advance.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
2012-10-14 22:26 keltezéssel, Boszormenyi Zoltan írta:
2012-10-14 22:23 keltezéssel, Boszormenyi Zoltan írta:
Hi,
2012-10-14 18:41 keltezéssel, Boszormenyi Zoltan írta:
2012-10-14 18:02 keltezéssel, Fujii Masao írta:
Thanks for updating the patch!
On Sun, Oct 14, 2012 at 8:41 PM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
Backing up a standby server without -R preserves the original recovery.conf
of the
standby, it points to the standby's source server.Backing up a standby server with -R overwrites the original recovery.conf
with the new
one pointing to the standby instead of the standby's source server. Without
-Ft, it is
obvious. With -Ft, there are two recovery.conf files in the tar file and
upon extracting it,
the last written one (the one generated via -R) overwrites the original.The tar file is always extracted such way in all platform which PostgreSQL
supports? I'm just concerned about that some tool in some platform might
prefer the original recovery.conf when extracting tar file. If the spec of tar
format specifies such behavior (i.e., the last written file of the same name
is always preferred), it's OK.Since tar is a sequential archive format, I think this is the behaviour of
every tar extractor. But I will look at adding code to skip the original
recovery.conf if it exists in the tar file.I found the bug that recovery.conf is included in the tar file of the tablespace
instead of base.tar, when there are tablespaces in the server.You are right, I am looking into this. But I don't know how it got there,
I check for (rownum == 0 && writerecoveryconf) and rownum == 0
supposedly means that it's the base.tar. Looking again.I made a mistake in the previous check, rownum is not reliable.
The tablespaces are sent first and base backup as the last.
Now recovery.conf is written into base.tar.Maybe this is nitpicky problem,,,, but...
If port number is not explicitly specified in pg_basebackup, the port
number is not
included to primary_conninfo in recovery.conf which is created during
the backup.
That is, the standby server using such recovery.conf tries to connect
to the default
port number because the port number is not supplied in primary_conninfo. This
assumes that the default port number is the same between the master and standby.
But this is not true. The default port number can be changed in --with-pgport
configure option, so the default port number might be different
between the master
and standby. To avoid this uncertainty, pg_basebackup -R should always include
the port number in primary_conninfo?I think you are right. But, I wouldn't restrict it only to the port setting.
Any of the values that are set and equal to the compiled-in default,
it should be written into recovery.conf.Now all values that are set (even those being equal to the compiled-in default)
are put into recovery.conf.When the password is required to connect to the server, pg_basebackup -R
always writes the password setting into primary_conninfo in recovery.conf.
But if the password is supplied from .pgpass, ISTM that the password setting
doesn't need to be written into primary_conninfo. Right?How can you deduce it from the PQconninfoOption structure?
Also, if the machine you take the base backup on is different
from the one where you actually use the backup on, it can be
different not only in the --with-pgport compilation option but
in the presence of .pgpass or the PGPASSWORD envvar, too.
The administrator is there for a reason or there is no .pgpass
or PGPASSWORD at all.+ The password written into recovery.conf is not escaped even if special + characters appear in it. The administrator must review recovery.conf + to ensure proper escaping.Is it difficult to make pg_basebackup escape the special characters in the
password? It's better if we can remove this restriction.It's not difficult. What other characters need to be escaped besides single quotes?
All written values are escaped.
Other changes: the recovery.conf in base.tar is correctly skipped if it exists
and -R is given. The new recovery.conf is written with padding to round up to
512, the TAR chunk size.Also, the check for conflict between -R and -x/-X is now removed.
Really the last one, for today at least. The buffer for recovery.conf is freed
in both the -Fp and -Ft cases.
The PQconninfo patch is also attached but didn't change since the last mail.
I've not reviewed PQconninfo patch yet. Will review.
Thanks in advance.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
01-PQconninfo-v13.patchtext/x-patch; name=01-PQconninfo-v13.patchDownload
diff -durpN postgresql/doc/src/sgml/libpq.sgml postgresql.1/doc/src/sgml/libpq.sgml
--- postgresql/doc/src/sgml/libpq.sgml 2012-08-03 09:39:30.114266570 +0200
+++ postgresql.1/doc/src/sgml/libpq.sgml 2012-10-14 11:12:07.049196259 +0200
@@ -496,6 +496,37 @@ typedef struct
</listitem>
</varlistentry>
+ <varlistentry id="libpq-pqconninfo">
+ <term><function>PQconninfo</function><indexterm><primary>PQconninfo</></></term>
+ <listitem>
+ <para>
+ Returns the connection options used by a live connection.
+<synopsis>
+PQconninfoOption *PQconninfo(PGconn *conn, bool for_replication);
+</synopsis>
+ </para>
+
+ <para>
+ Returns a connection options array. This can be used to determine
+ all possible <function>PQconnectdb</function> options and their
+ current values that were used to connect to the server. The return
+ value points to an array of <structname>PQconninfoOption</structname>
+ structures, which ends with an entry having a null <structfield>keyword</>
+ pointer. Every notes above for <function>PQconndefaults</function> also apply.
+ </para>
+
+ <para>
+ The <literal>for_replication</> argument can be used to exclude some
+ options from the list which are used by the walreceiver module.
+ <application>pg_basebackup</application> uses this pre-filtered list
+ to construct <literal>primary_conninfo</> in the automatically generated
+ recovery.conf file.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+
<varlistentry id="libpq-pqconninfoparse">
<term><function>PQconninfoParse</function><indexterm><primary>PQconninfoParse</></></term>
<listitem>
diff -durpN postgresql/src/interfaces/libpq/exports.txt postgresql.1/src/interfaces/libpq/exports.txt
--- postgresql/src/interfaces/libpq/exports.txt 2012-10-09 09:58:14.342782974 +0200
+++ postgresql.1/src/interfaces/libpq/exports.txt 2012-10-14 11:12:07.049196259 +0200
@@ -164,3 +164,4 @@ PQsetSingleRowMode 161
lo_lseek64 162
lo_tell64 163
lo_truncate64 164
+PQconninfo 165
diff -durpN postgresql/src/interfaces/libpq/fe-connect.c postgresql.1/src/interfaces/libpq/fe-connect.c
--- postgresql/src/interfaces/libpq/fe-connect.c 2012-09-09 08:11:09.470401480 +0200
+++ postgresql.1/src/interfaces/libpq/fe-connect.c 2012-10-14 11:12:07.053196284 +0200
@@ -137,6 +137,9 @@ static int ldapServiceLookup(const char
* PQconninfoOptions[] *must* be NULL. In a working copy, non-null "val"
* fields point to malloc'd strings that should be freed when the working
* array is freed (see PQconninfoFree).
+ *
+ * If you add a new connection option to this list, remember to add it to
+ * PQconninfoMappings[] below.
* ----------
*/
static const PQconninfoOption PQconninfoOptions[] = {
@@ -264,6 +267,66 @@ static const PQconninfoOption PQconninfo
NULL, NULL, 0}
};
+/*
+ * We need a mapping between the PQconninfoOptions[] array
+ * and PGconn members. We have to keep it separate from
+ * PQconninfoOptions[] to not leak info about PGconn members
+ * to clients.
+ */
+typedef struct PQconninfoMapping {
+ char *keyword;
+ size_t member_offset;
+ bool for_replication;
+ /*
+ * Special and simplistic value mapping between
+ * PQconninfoOption and PGconn. Only used by "requiressl".
+ */
+ char *conninfoValue;
+ char *connValue;
+} PQconninfoMapping;
+#define PGCONNMEMBERADDR(conn, mapping) ((char **)((char *)conn + mapping->member_offset))
+
+static const PQconninfoMapping PQconninfoMappings[] =
+{
+ /* "authtype" is not used anymore, there is no mapping to PGconn */
+ /* there is no mapping of "service" to PGconn */
+ { "user", offsetof(struct pg_conn, pguser), true, NULL, NULL },
+ { "password", offsetof(struct pg_conn, pgpass), true, NULL, NULL },
+ { "connect_timeout", offsetof(struct pg_conn, connect_timeout), true, NULL, NULL },
+ { "dbname", offsetof(struct pg_conn, dbName), false, NULL, NULL },
+ { "host", offsetof(struct pg_conn, pghost), true, NULL, NULL },
+ { "hostaddr", offsetof(struct pg_conn, pghostaddr), true, NULL, NULL },
+ { "port", offsetof(struct pg_conn, pgport), true, NULL, NULL },
+ { "client_encoding", offsetof(struct pg_conn, client_encoding_initial), false, NULL, NULL },
+ { "tty", offsetof(struct pg_conn, pgtty), false, NULL, NULL },
+ { "options", offsetof(struct pg_conn, pgoptions), true, NULL, NULL },
+ { "application_name", offsetof(struct pg_conn, appname), false, NULL, NULL },
+ { "fallback_application_name", offsetof(struct pg_conn, fbappname), false, NULL, NULL },
+ { "keepalives", offsetof(struct pg_conn, keepalives), true, NULL, NULL },
+ { "keepalives_idle", offsetof(struct pg_conn, keepalives_idle), true, NULL, NULL },
+ { "keepalives_interval", offsetof(struct pg_conn, keepalives_interval), true, NULL, NULL },
+ { "keepalives_count", offsetof(struct pg_conn, keepalives_count), true, NULL, NULL },
+#ifdef USE_SSL
+ { "requiressl", offsetof(struct pg_conn, sslmode), false, "1", "require" },
+#endif
+ { "sslmode", offsetof(struct pg_conn, sslmode), true, NULL, NULL },
+ { "sslcompression", offsetof(struct pg_conn, sslcompression), true, NULL, NULL },
+ { "sslcert", offsetof(struct pg_conn, sslcert), true, NULL, NULL },
+ { "sslkey", offsetof(struct pg_conn, sslkey), true, NULL, NULL },
+ { "sslrootcert", offsetof(struct pg_conn, sslrootcert), true, NULL, NULL },
+ { "sslcrl", offsetof(struct pg_conn, sslcrl), true, NULL, NULL },
+ { "requirepeer", offsetof(struct pg_conn, requirepeer), true, NULL, NULL },
+#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
+ { "krbsrvname", offsetof(struct pg_conn, krbsrvname), true, NULL, NULL },
+#endif
+#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
+ { "gsslib", offsetof(struct pg_conn, requirepeer), true, NULL, NULL },
+#endif
+ { "replication", offsetof(struct pg_conn, replication), false, NULL, NULL },
+ /* Terminating entry --- MUST BE LAST */
+ { NULL, 0, false, NULL, NULL }
+};
+
static const PQEnvironmentOption EnvironmentOptions[] =
{
/* common user-interface settings */
@@ -295,7 +358,8 @@ static PGconn *makeEmptyPGconn(void);
static void fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
-static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage);
+static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage,
+ bool for_replication);
static PQconninfoOption *parse_connection_string(const char *conninfo,
PQExpBuffer errorMessage, bool use_defaults);
static int uri_prefix_length(const char *connstr);
@@ -318,6 +382,8 @@ static char *conninfo_uri_decode(const c
static bool get_hexdigit(char digit, int *value);
static const char *conninfo_getval(PQconninfoOption *connOptions,
const char *keyword);
+static void conninfo_setval(PQconninfoOption *connOptions,
+ const char *keyword, const char *val);
static PQconninfoOption *conninfo_storeval(PQconninfoOption *connOptions,
const char *keyword, const char *value,
PQExpBuffer errorMessage, bool ignoreMissing, bool uri_decode);
@@ -627,7 +693,9 @@ PQconnectStart(const char *conninfo)
static void
fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
{
+ const PQconninfoMapping *mapping;
const char *tmp;
+ char **memberaddr;
/*
* Move option values into conn structure
@@ -637,72 +705,24 @@ fillPGconn(PGconn *conn, PQconninfoOptio
*
* XXX: probably worth checking strdup() return value here...
*/
- tmp = conninfo_getval(connOptions, "hostaddr");
- conn->pghostaddr = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "host");
- conn->pghost = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "port");
- conn->pgport = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "tty");
- conn->pgtty = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "options");
- conn->pgoptions = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "application_name");
- conn->appname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "fallback_application_name");
- conn->fbappname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "dbname");
- conn->dbName = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "user");
- conn->pguser = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "password");
- conn->pgpass = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "connect_timeout");
- conn->connect_timeout = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "client_encoding");
- conn->client_encoding_initial = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives");
- conn->keepalives = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_idle");
- conn->keepalives_idle = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_interval");
- conn->keepalives_interval = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_count");
- conn->keepalives_count = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslmode");
- conn->sslmode = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcompression");
- conn->sslcompression = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslkey");
- conn->sslkey = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcert");
- conn->sslcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslrootcert");
- conn->sslrootcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcrl");
- conn->sslcrl = tmp ? strdup(tmp) : NULL;
-#ifdef USE_SSL
- tmp = conninfo_getval(connOptions, "requiressl");
- if (tmp && tmp[0] == '1')
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
{
- /* here warn that the requiressl option is deprecated? */
- if (conn->sslmode)
- free(conn->sslmode);
- conn->sslmode = strdup("require");
+ tmp = conninfo_getval(connOptions, mapping->keyword);
+ memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ if (mapping->conninfoValue && mapping->connValue)
+ {
+ size_t len = strlen(mapping->conninfoValue);
+ if (tmp && strncmp(tmp, mapping->conninfoValue, len) == 0)
+ {
+ if (*memberaddr)
+ free(*memberaddr);
+ *memberaddr = strdup(mapping->connValue);
+ }
+ }
+ else
+ *memberaddr = tmp ? strdup(tmp) : NULL;
}
-#endif
- tmp = conninfo_getval(connOptions, "requirepeer");
- conn->requirepeer = tmp ? strdup(tmp) : NULL;
-#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "krbsrvname");
- conn->krbsrvname = tmp ? strdup(tmp) : NULL;
-#endif
-#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "gsslib");
- conn->gsslib = tmp ? strdup(tmp) : NULL;
-#endif
- tmp = conninfo_getval(connOptions, "replication");
- conn->replication = tmp ? strdup(tmp) : NULL;
}
/*
@@ -884,7 +904,7 @@ PQconndefaults(void)
if (PQExpBufferDataBroken(errorBuf))
return NULL; /* out of memory already :-( */
- connOptions = conninfo_init(&errorBuf);
+ connOptions = conninfo_init(&errorBuf, false);
if (connOptions != NULL)
{
if (!conninfo_add_defaults(connOptions, &errorBuf))
@@ -4006,9 +4026,11 @@ PQconninfoParse(const char *conninfo, ch
/*
* Build a working copy of the constant PQconninfoOptions array.
+ * If for_replication is true, only return the options that are
+ * not added by libpqwalreceiver.
*/
static PQconninfoOption *
-conninfo_init(PQExpBuffer errorMessage)
+conninfo_init(PQExpBuffer errorMessage, bool for_replication)
{
PQconninfoOption *options;
@@ -4019,7 +4041,28 @@ conninfo_init(PQExpBuffer errorMessage)
libpq_gettext("out of memory\n"));
return NULL;
}
- memcpy(options, PQconninfoOptions, sizeof(PQconninfoOptions));
+ if (for_replication)
+ {
+ const PQconninfoMapping *mapping = PQconninfoMappings;
+ PQconninfoOption *opt_dest = options;
+
+ while (mapping->keyword)
+ {
+ PQconninfoOption *opt_src = conninfo_find((PQconninfoOption *)PQconninfoOptions, mapping->keyword);
+
+ if (opt_src && mapping->for_replication)
+ {
+ memcpy(opt_dest, opt_src, sizeof(PQconninfoOption));
+ opt_dest++;
+ }
+
+ opt_src++;
+ mapping++;
+ }
+ MemSet(opt_dest, 0, sizeof(PQconninfoOption));
+ }
+ else
+ memcpy(options, PQconninfoOptions, sizeof(PQconninfoOptions));
return options;
}
@@ -4095,7 +4138,7 @@ conninfo_parse(const char *conninfo, PQE
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
return NULL;
@@ -4295,7 +4338,7 @@ conninfo_array_parse(const char *const *
}
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
{
PQconninfoFree(dbname_options);
@@ -4485,7 +4528,7 @@ conninfo_uri_parse(const char *uri, PQEx
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
return NULL;
@@ -4985,6 +5028,24 @@ conninfo_getval(PQconninfoOption *connOp
}
/*
+ * Set an option value corresponding to the keyword in the connOptions array.
+ */
+static void
+conninfo_setval(PQconninfoOption *connOptions, const char *keyword,
+ const char *val)
+{
+ PQconninfoOption *option;
+
+ option = conninfo_find(connOptions, keyword);
+ if (option)
+ {
+ if (option->val)
+ free(option->val);
+ option->val = val ? strdup(val) : NULL;
+ }
+}
+
+/*
* Store a (new) value for an option corresponding to the keyword in
* connOptions array.
*
@@ -5066,6 +5127,50 @@ conninfo_find(PQconninfoOption *connOpti
}
+/*
+ * Return the connection options used for the connections
+ */
+PQconninfoOption *
+PQconninfo(PGconn *conn, bool for_replication)
+{
+ PQExpBufferData errorBuf;
+ PQconninfoOption *connOptions;
+
+ if (conn == NULL)
+ return NULL;
+
+ /* We don't actually report any errors here, but callees want a buffer */
+ initPQExpBuffer(&errorBuf);
+ if (PQExpBufferDataBroken(errorBuf))
+ return NULL; /* out of memory already :-( */
+
+ connOptions = conninfo_init(&errorBuf, for_replication);
+
+ termPQExpBuffer(&errorBuf);
+
+ if (connOptions != NULL)
+ {
+ const PQconninfoMapping *mapping;
+
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
+ {
+ char **memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ if (mapping->conninfoValue && mapping->connValue)
+ {
+ size_t len = strlen(mapping->connValue);
+ if (*memberaddr && strncmp(*memberaddr, mapping->connValue, len) == 0)
+ conninfo_setval(connOptions, mapping->keyword, mapping->conninfoValue);
+ }
+ else
+ conninfo_setval(connOptions, mapping->keyword, *memberaddr);
+ }
+ }
+
+ return connOptions;
+}
+
+
void
PQconninfoFree(PQconninfoOption *connOptions)
{
diff -durpN postgresql/src/interfaces/libpq/libpq-fe.h postgresql.1/src/interfaces/libpq/libpq-fe.h
--- postgresql/src/interfaces/libpq/libpq-fe.h 2012-10-09 09:58:14.343782980 +0200
+++ postgresql.1/src/interfaces/libpq/libpq-fe.h 2012-10-14 11:12:07.054196291 +0200
@@ -262,6 +262,9 @@ extern PQconninfoOption *PQconndefaults(
/* parse connection options in same way as PQconnectdb */
extern PQconninfoOption *PQconninfoParse(const char *conninfo, char **errmsg);
+/* return the connection options used by a live connection */
+extern PQconninfoOption *PQconninfo(PGconn *conn, bool for_replication);
+
/* free the data structure returned by PQconndefaults() or PQconninfoParse() */
extern void PQconninfoFree(PQconninfoOption *connOptions);
02-pg_basebackup-v13.patchtext/x-patch; name=02-pg_basebackup-v13.patchDownload
diff -durpN postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml
--- postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml 2012-08-24 09:49:22.960530329 +0200
+++ postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml 2012-10-14 13:09:58.913513268 +0200
@@ -189,6 +189,37 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--write-recovery-conf</option></term>
+ <listitem>
+
+ <para>
+ Write a minimal recovery.conf into the output directory (or into
+ the base archive file if <option>--format=tar</option> was specified)
+ using the connection parameters from the command line to ease
+ setting up the standby. Since creating a backup for a standalone
+ server with <option>-x</option> or <option>-X</option> and adding
+ a recovery.conf to it wouldn't make a working standby, these options
+ naturally conflict.
+ </para>
+
+ <para>
+ The password written into recovery.conf is not escaped even if special
+ characters appear in it. The administrator must review recovery.conf
+ to ensure proper escaping.
+ </para>
+
+ <para>
+ When this option is specified and taking the base backup succeeded,
+ failing to write the <filename>recovery.conf</filename> results
+ in the error code 2. This way, scripts can distinguish between different
+ failure cases of <application>pg_basebackup</application>.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-x</option></term>
<term><option>--xlog</option></term>
<listitem>
diff -durpN postgresql.1/src/bin/pg_basebackup/pg_basebackup.c postgresql.2/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.1/src/bin/pg_basebackup/pg_basebackup.c 2012-10-03 10:40:48.297207389 +0200
+++ postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2012-10-14 22:29:07.243890976 +0200
@@ -25,6 +25,7 @@
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
+#include <time.h>
#ifdef HAVE_LIBZ
#include <zlib.h>
@@ -46,6 +47,7 @@ int compresslevel = 0;
bool includewal = false;
bool streamwal = false;
bool fastcheckpoint = false;
+bool writerecoveryconf = false;
int standby_message_timeout = 10 * 1000; /* 10 sec = default */
/* Progress counters */
@@ -70,6 +72,11 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
+#define TARCHUNKSZ (512)
+#define RCBUFSZ TARCHUNKSZ
+static char *recoveryconf = NULL;
+static size_t recoveryconflen = 0;
+
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
@@ -77,6 +84,15 @@ static void progress_report(int tablespa
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
+static void _reallocRCBuffer(char **buf, int *bufsz, int currsz,
+ char *keyword, char *value,
+ int extrasz);
+static void CreateRecoveryConf(PGconn *conn);
+static void WriteRecoveryConf(void);
+static int _tarChecksum(char *header);
+static void print_val(char *s, uint64 val, unsigned int base, size_t len);
+static void scan_val(char *s, uint64 *val, unsigned int base, size_t len);
+static void _tarCreateHeader(char *header);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -107,6 +123,8 @@ usage(void)
printf(_("\nOptions controlling the output:\n"));
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
+ printf(_(" -R, --write-recovery-conf\n"
+ " write recovery.conf after backup\n"));
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
printf(_(" -X, --xlog-method=fetch|stream\n"
" include required WAL files with specified method\n"));
@@ -467,6 +485,11 @@ ReceiveTarFile(PGconn *conn, PGresult *r
char filename[MAXPGPATH];
char *copybuf = NULL;
FILE *tarfile = NULL;
+ char tarhdr[512];
+ bool basebackup;
+ bool in_tarhdr, skip_file;
+ size_t tarhdrsz;
+ uint64 filesz;
#ifdef HAVE_LIBZ
gzFile ztarfile = NULL;
@@ -519,6 +542,8 @@ ReceiveTarFile(PGconn *conn, PGresult *r
tarfile = fopen(filename, "wb");
}
}
+
+ basebackup = true;
}
else
{
@@ -547,6 +572,8 @@ ReceiveTarFile(PGconn *conn, PGresult *r
PQgetvalue(res, rownum, 0));
tarfile = fopen(filename, "wb");
}
+
+ basebackup = false;
}
#ifdef HAVE_LIBZ
@@ -584,6 +611,10 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = filesz = 0;
+
while (1)
{
int r;
@@ -598,13 +629,69 @@ ReceiveTarFile(PGconn *conn, PGresult *r
if (r == -1)
{
/*
- * End of chunk. Close file (but not stdout).
+ * End of chunk. Write recovery.conf into the tar file (if it
+ * was requested) and close file (but not stdout).
*
* Also, write two completely empty blocks at the end of the tar
* file, as required by some tar programs.
*/
char zerobuf[1024];
+ if (basebackup && writerecoveryconf)
+ {
+ char header[512];
+ int lastchunk;
+
+ _tarCreateHeader(header);
+
+ lastchunk = recoveryconflen % TARCHUNKSZ;
+ if (lastchunk > 0)
+ recoveryconflen += (TARCHUNKSZ - lastchunk);
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (gzwrite(ztarfile, header, sizeof(header)) !=
+ sizeof(header))
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, filename, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+
+ if (gzwrite(ztarfile, recoveryconf, recoveryconflen) !=
+ recoveryconflen)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, filename, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (fwrite(header, sizeof(header), 1, tarfile) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ if (fwrite(recoveryconf, recoveryconflen, 1, tarfile) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ }
+ fprintf(stderr, _("%s: recovery.conf written into '%s'\n"), progname, filename);
+
+ free(recoveryconf);
+ }
+
MemSet(zerobuf, 0, sizeof(zerobuf));
#ifdef HAVE_LIBZ
if (ztarfile != NULL)
@@ -665,25 +752,130 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
-#ifdef HAVE_LIBZ
- if (ztarfile != NULL)
+ if (basebackup && writerecoveryconf)
{
- if (gzwrite(ztarfile, copybuf, r) != r)
+ int rr = r;
+ int pos = 0;
+
+ while (rr > 0)
{
- fprintf(stderr,
- _("%s: could not write to compressed file \"%s\": %s\n"),
- progname, filename, get_gz_error(ztarfile));
- disconnect_and_exit(1);
+ if (in_tarhdr)
+ {
+ if (tarhdrsz < 512)
+ {
+ int hdrleft, bytes2copy;
+
+ hdrleft = TARCHUNKSZ - tarhdrsz;
+ bytes2copy = (rr > hdrleft ? hdrleft : rr);
+
+ memcpy(&tarhdr[tarhdrsz], copybuf + pos, bytes2copy);
+
+ rr -= bytes2copy;
+ pos += bytes2copy;
+ tarhdrsz += bytes2copy;
+ }
+ else
+ {
+ int lastchunk;
+
+ skip_file = (strcmp(&tarhdr[0], "recovery.conf") == 0);
+
+ scan_val(&tarhdr[124], &filesz, 8, 11);
+ lastchunk = filesz % TARCHUNKSZ;
+ if (lastchunk > 0)
+ filesz += (TARCHUNKSZ - lastchunk);
+
+ in_tarhdr = false;
+ tarhdrsz = 0;
+
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (!skip_file && gzwrite(ztarfile, tarhdr, TARCHUNKSZ) != TARCHUNKSZ)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, filename, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (!skip_file && fwrite(tarhdr, TARCHUNKSZ, 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ }
+ }
+ }
+ else /* inside a file */
+ {
+ if (filesz > 0)
+ {
+ int bytes2write;
+
+ bytes2write = (filesz > rr ? rr : filesz);
+
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (!skip_file && gzwrite(ztarfile, copybuf + pos, bytes2write) != bytes2write)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, filename, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (!skip_file && fwrite(copybuf + pos, bytes2write, 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ }
+
+ rr -= bytes2write;
+ pos += bytes2write;
+ filesz -= bytes2write;
+ }
+ else
+ {
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = filesz = 0;
+ }
+ }
}
}
else
-#endif
{
- if (fwrite(copybuf, r, 1, tarfile) != 1)
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
{
- fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
- progname, filename, strerror(errno));
- disconnect_and_exit(1);
+ if (gzwrite(ztarfile, copybuf, r) != r)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, filename, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (fwrite(copybuf, r, 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
}
}
totaldone += r;
@@ -712,13 +904,20 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
char filename[MAXPGPATH];
int current_len_left;
int current_padding = 0;
+ bool basebackup;
char *copybuf = NULL;
FILE *file = NULL;
if (PQgetisnull(res, rownum, 0))
+ {
strcpy(current_path, basedir);
+ basebackup = true;
+ }
else
+ {
strcpy(current_path, PQgetvalue(res, rownum, 1));
+ basebackup = false;
+ }
/*
* Get the COPY data
@@ -937,6 +1136,330 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
if (copybuf != NULL)
PQfreemem(copybuf);
+
+ if (basebackup)
+ WriteRecoveryConf();
+}
+
+static int
+_tarChecksum(char *header)
+{
+ int i,
+ sum;
+
+ /*
+ * Per POSIX, the checksum is the simple sum of all bytes in the header,
+ * treating the bytes as unsigned, and treating the checksum field (at
+ * offset 148) as though it contained 8 spaces.
+ */
+ sum = 8 * ' '; /* presumed value for checksum field */
+ for (i = 0; i < 512; i++)
+ if (i < 148 || i >= 156)
+ sum += 0xFF & header[i];
+ return sum;
+}
+
+
+/*
+ * Utility routine to print possibly larger than 32 bit integers in a
+ * portable fashion. Filled with zeros.
+ */
+static void
+print_val(char *s, uint64 val, unsigned int base, size_t len)
+{
+ int i;
+
+ for (i = len; i > 0; i--)
+ {
+ int digit = val % base;
+
+ s[i - 1] = '0' + digit;
+ val = val / base;
+ }
+}
+
+
+/*
+ * Inverse for print_val()
+ */
+static void
+scan_val(char *s, uint64 *val, unsigned int base, size_t len)
+{
+ uint64 tmp = 0;
+ int i;
+
+ for (i = 0; i < len; i++)
+ {
+ int digit = s[i] - '0';
+
+ tmp = tmp * base + digit;
+ }
+
+ *val = tmp;
+}
+
+
+static void
+_tarCreateHeader(char *header)
+{
+ /*
+ * Note: most of the fields in a tar header are not supposed to be
+ * null-terminated. We use sprintf, which will write a null after the
+ * required bytes; that null goes into the first byte of the next field.
+ * This is okay as long as we fill the fields in order.
+ */
+ memset(header, 0, 512 /* sizeof the tar header */);
+
+ /* Name 100 */
+ sprintf(&header[0], "%.99s", "recovery.conf");
+
+ /* Mode 8 */
+ sprintf(&header[100], "0000600 ");
+
+ /* User ID 8 */
+ sprintf(&header[108], "0004000 ");
+
+ /* Group 8 */
+ sprintf(&header[116], "0002000 ");
+
+ /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
+ print_val(&header[124], recoveryconflen, 8, 11);
+ sprintf(&header[135], " ");
+
+ /* Mod Time 12 */
+ sprintf(&header[136], "%011o ", (int) time(NULL));
+
+ /* Checksum 8 cannot be calculated until we've filled all other fields */
+
+ /* Type - regular file */
+ sprintf(&header[156], "0");
+
+ /* Link Name 100 (leave as nulls) */
+
+ /* Magic 6 */
+ sprintf(&header[257], "ustar");
+
+ /* Version 2 */
+ sprintf(&header[263], "00");
+
+ /* User 32 */
+ /* XXX: Do we need to care about setting correct username? */
+ sprintf(&header[265], "%.31s", "postgres");
+
+ /* Group 32 */
+ /* XXX: Do we need to care about setting correct group name? */
+ sprintf(&header[297], "%.31s", "postgres");
+
+ /* Major Dev 8 */
+ sprintf(&header[329], "%07o ", 0);
+
+ /* Minor Dev 8 */
+ sprintf(&header[337], "%07o ", 0);
+
+ /* Prefix 155 - not used, leave as nulls */
+
+ /*
+ * We mustn't overwrite the next field while inserting the checksum.
+ * Fortunately, the checksum can't exceed 6 octal digits, so we just write
+ * 6 digits, a space, and a null, which is legal per POSIX.
+ */
+ sprintf(&header[148], "%06o ", _tarChecksum(header));
+}
+
+/*
+ * Escape single quotes in a string
+ */
+static char *
+escape_string(char *value)
+{
+ char *tmp, *escaped, *esc;
+ int num_quotes;
+
+ tmp = value;
+ num_quotes = 0;
+ while (*tmp)
+ {
+ if (*tmp == '\'')
+ num_quotes++;
+ tmp++;
+ }
+
+ escaped = malloc((tmp - value) + num_quotes + 1);
+ if (escaped == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1)
+ }
+
+ tmp = value;
+ esc = escaped;
+ while (*tmp)
+ {
+ if (*tmp == '\'')
+ *esc++ = '\\';
+ *esc++ = *tmp++;
+ }
+ *esc = '\0';
+
+ return escaped;
+}
+
+/*
+ * Reallocate the buffer for recovery.conf if needed
+ */
+static void
+_reallocRCBuffer(char **buf, int *bufsz, int currsz, char *keyword, char *value, int extrasz)
+{
+ int len = extrasz;
+
+ if (keyword != NULL)
+ len += strlen(keyword);
+ if (value != NULL)
+ len += strlen(value);
+
+ if (*bufsz < currsz + len)
+ {
+ char *tmp;
+
+ tmp = realloc(*buf, *bufsz + RCBUFSZ);
+ if (tmp == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1)
+ }
+ *buf = tmp;
+ *bufsz += RCBUFSZ;
+ memset(tmp + currsz, 0, *bufsz - currsz);
+ }
+}
+
+/*
+ * Try to create recovery.conf in memory and set the length to write later.
+ */
+static void
+CreateRecoveryConf(PGconn *conn)
+{
+ char *buf = NULL;
+ PQconninfoOption *connOptions;
+ PQconninfoOption *option;
+ int bufsz;
+ int total = 0;
+ int written;
+
+ if (!writerecoveryconf)
+ return;
+
+ connOptions = PQconninfo(conn, true);
+ if (connOptions == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+
+ bufsz = RCBUFSZ;
+ buf = malloc(bufsz);
+ if (buf == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ written = sprintf(&buf[total], "standby_mode = 'on'\n");
+ if (written < 0)
+ {
+ fprintf(stderr, _("%s: cannot write to string: %s"), progname, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ total += written;
+
+ written = sprintf(&buf[total], "primary_conninfo = '");
+ if (written < 0)
+ {
+ fprintf(stderr, _("%s: cannot write to string: %s"), progname, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ total += written;
+
+ for (option = connOptions; option && option->keyword; option++)
+ {
+ char *escaped;
+ /*
+ * Do not emit this setting if not set or empty.
+ * The list of options was already pre-filtered for options
+ * usable for replication with PQconninfo(conn, true).
+ */
+ if ((option->val == NULL) ||
+ (option->val != NULL && option->val[0] == '\0'))
+ continue;
+
+ /*
+ * Write "keyword='value'" pieces, the value string is escaped
+ * if necessary and doubled single quotes around the value string.
+ */
+ escaped = escape_string(option->val);
+
+ _reallocRCBuffer(&buf, &bufsz, total, option->keyword, escaped, 6);
+
+ written = sprintf(&buf[total], "%s=''%s'' ", option->keyword, option->val);
+
+ free(escaped);
+
+ if (written < 0)
+ {
+ fprintf(stderr, _("%s: cannot write to string: %s"), progname, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ total += written;
+ }
+
+ _reallocRCBuffer(&buf, &bufsz, total, NULL, NULL, 2);
+ written = sprintf(&buf[total], "'\n");
+ if (written < 0)
+ {
+ fprintf(stderr, _("%s: cannot write to string: %s"), progname, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ total += written;
+
+ recoveryconf = buf;
+ recoveryconflen = total;
+
+ PQconninfoFree(connOptions);
+}
+
+
+static void
+WriteRecoveryConf(void)
+{
+ char filename[MAXPGPATH];
+ FILE *cf;
+
+ if (!writerecoveryconf)
+ return;
+
+ sprintf(filename, "%s/recovery.conf", basedir);
+
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("%s: cannot create %s: %s"), progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ if (fwrite(recoveryconf, recoveryconflen, 1, cf) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ fclose(cf);
+
+ fprintf(stderr, _("%s: recovery.conf written.\n"), progname);
+
+ free(recoveryconf);
}
@@ -960,6 +1483,8 @@ BaseBackup(void)
/* Error message already written in GetConnection() */
exit(1);
+ CreateRecoveryConf(conn);
+
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
@@ -1243,6 +1768,7 @@ main(int argc, char **argv)
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
+ {"write-recovery-conf", no_argument, NULL, 'R'},
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
{"gzip", no_argument, NULL, 'z'},
@@ -1280,7 +1806,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:xX:l:zZ:c:h:p:U:s:wWvP",
+ while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
long_options, &option_index)) != -1)
{
switch (c)
@@ -1301,6 +1827,9 @@ main(int argc, char **argv)
exit(1);
}
break;
+ case 'R':
+ writerecoveryconf = true;
+ break;
case 'x':
if (includewal)
{
2012-10-14 22:31 keltezéssel, Boszormenyi Zoltan írta:
2012-10-14 22:26 keltezéssel, Boszormenyi Zoltan írta:
2012-10-14 22:23 keltezéssel, Boszormenyi Zoltan írta:
Hi,
2012-10-14 18:41 keltezéssel, Boszormenyi Zoltan írta:
2012-10-14 18:02 keltezéssel, Fujii Masao írta:
Thanks for updating the patch!
On Sun, Oct 14, 2012 at 8:41 PM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
Backing up a standby server without -R preserves the original recovery.conf
of the
standby, it points to the standby's source server.Backing up a standby server with -R overwrites the original recovery.conf
with the new
one pointing to the standby instead of the standby's source server. Without
-Ft, it is
obvious. With -Ft, there are two recovery.conf files in the tar file and
upon extracting it,
the last written one (the one generated via -R) overwrites the original.The tar file is always extracted such way in all platform which PostgreSQL
supports? I'm just concerned about that some tool in some platform might
prefer the original recovery.conf when extracting tar file. If the spec of tar
format specifies such behavior (i.e., the last written file of the same name
is always preferred), it's OK.Since tar is a sequential archive format, I think this is the behaviour of
every tar extractor. But I will look at adding code to skip the original
recovery.conf if it exists in the tar file.I found the bug that recovery.conf is included in the tar file of the tablespace
instead of base.tar, when there are tablespaces in the server.You are right, I am looking into this. But I don't know how it got there,
I check for (rownum == 0 && writerecoveryconf) and rownum == 0
supposedly means that it's the base.tar. Looking again.I made a mistake in the previous check, rownum is not reliable.
The tablespaces are sent first and base backup as the last.
Now recovery.conf is written into base.tar.Maybe this is nitpicky problem,,,, but...
If port number is not explicitly specified in pg_basebackup, the port
number is not
included to primary_conninfo in recovery.conf which is created during
the backup.
That is, the standby server using such recovery.conf tries to connect
to the default
port number because the port number is not supplied in primary_conninfo. This
assumes that the default port number is the same between the master and standby.
But this is not true. The default port number can be changed in --with-pgport
configure option, so the default port number might be different
between the master
and standby. To avoid this uncertainty, pg_basebackup -R should always include
the port number in primary_conninfo?I think you are right. But, I wouldn't restrict it only to the port setting.
Any of the values that are set and equal to the compiled-in default,
it should be written into recovery.conf.Now all values that are set (even those being equal to the compiled-in default)
are put into recovery.conf.When the password is required to connect to the server, pg_basebackup -R
always writes the password setting into primary_conninfo in recovery.conf.
But if the password is supplied from .pgpass, ISTM that the password setting
doesn't need to be written into primary_conninfo. Right?How can you deduce it from the PQconninfoOption structure?
Also, if the machine you take the base backup on is different
from the one where you actually use the backup on, it can be
different not only in the --with-pgport compilation option but
in the presence of .pgpass or the PGPASSWORD envvar, too.
The administrator is there for a reason or there is no .pgpass
or PGPASSWORD at all.+ The password written into recovery.conf is not escaped even if special + characters appear in it. The administrator must review recovery.conf + to ensure proper escaping.Is it difficult to make pg_basebackup escape the special characters in the
password? It's better if we can remove this restriction.It's not difficult. What other characters need to be escaped besides single quotes?
All written values are escaped.
Other changes: the recovery.conf in base.tar is correctly skipped if it exists
and -R is given. The new recovery.conf is written with padding to round up to
512, the TAR chunk size.Also, the check for conflict between -R and -x/-X is now removed.
The documentation for option -R has changed to reflect this and
there is no different error code 2 now: it would make the behaviour
inconsistent between -Fp and -Ft.
The PQconninfo patch is also attached but didn't change since the last mail.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
01-PQconninfo-v13.patchtext/x-patch; name=01-PQconninfo-v13.patchDownload
diff -durpN postgresql/doc/src/sgml/libpq.sgml postgresql.1/doc/src/sgml/libpq.sgml
--- postgresql/doc/src/sgml/libpq.sgml 2012-08-03 09:39:30.114266570 +0200
+++ postgresql.1/doc/src/sgml/libpq.sgml 2012-10-14 11:12:07.049196259 +0200
@@ -496,6 +496,37 @@ typedef struct
</listitem>
</varlistentry>
+ <varlistentry id="libpq-pqconninfo">
+ <term><function>PQconninfo</function><indexterm><primary>PQconninfo</></></term>
+ <listitem>
+ <para>
+ Returns the connection options used by a live connection.
+<synopsis>
+PQconninfoOption *PQconninfo(PGconn *conn, bool for_replication);
+</synopsis>
+ </para>
+
+ <para>
+ Returns a connection options array. This can be used to determine
+ all possible <function>PQconnectdb</function> options and their
+ current values that were used to connect to the server. The return
+ value points to an array of <structname>PQconninfoOption</structname>
+ structures, which ends with an entry having a null <structfield>keyword</>
+ pointer. Every notes above for <function>PQconndefaults</function> also apply.
+ </para>
+
+ <para>
+ The <literal>for_replication</> argument can be used to exclude some
+ options from the list which are used by the walreceiver module.
+ <application>pg_basebackup</application> uses this pre-filtered list
+ to construct <literal>primary_conninfo</> in the automatically generated
+ recovery.conf file.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+
<varlistentry id="libpq-pqconninfoparse">
<term><function>PQconninfoParse</function><indexterm><primary>PQconninfoParse</></></term>
<listitem>
diff -durpN postgresql/src/interfaces/libpq/exports.txt postgresql.1/src/interfaces/libpq/exports.txt
--- postgresql/src/interfaces/libpq/exports.txt 2012-10-09 09:58:14.342782974 +0200
+++ postgresql.1/src/interfaces/libpq/exports.txt 2012-10-14 11:12:07.049196259 +0200
@@ -164,3 +164,4 @@ PQsetSingleRowMode 161
lo_lseek64 162
lo_tell64 163
lo_truncate64 164
+PQconninfo 165
diff -durpN postgresql/src/interfaces/libpq/fe-connect.c postgresql.1/src/interfaces/libpq/fe-connect.c
--- postgresql/src/interfaces/libpq/fe-connect.c 2012-09-09 08:11:09.470401480 +0200
+++ postgresql.1/src/interfaces/libpq/fe-connect.c 2012-10-14 11:12:07.053196284 +0200
@@ -137,6 +137,9 @@ static int ldapServiceLookup(const char
* PQconninfoOptions[] *must* be NULL. In a working copy, non-null "val"
* fields point to malloc'd strings that should be freed when the working
* array is freed (see PQconninfoFree).
+ *
+ * If you add a new connection option to this list, remember to add it to
+ * PQconninfoMappings[] below.
* ----------
*/
static const PQconninfoOption PQconninfoOptions[] = {
@@ -264,6 +267,66 @@ static const PQconninfoOption PQconninfo
NULL, NULL, 0}
};
+/*
+ * We need a mapping between the PQconninfoOptions[] array
+ * and PGconn members. We have to keep it separate from
+ * PQconninfoOptions[] to not leak info about PGconn members
+ * to clients.
+ */
+typedef struct PQconninfoMapping {
+ char *keyword;
+ size_t member_offset;
+ bool for_replication;
+ /*
+ * Special and simplistic value mapping between
+ * PQconninfoOption and PGconn. Only used by "requiressl".
+ */
+ char *conninfoValue;
+ char *connValue;
+} PQconninfoMapping;
+#define PGCONNMEMBERADDR(conn, mapping) ((char **)((char *)conn + mapping->member_offset))
+
+static const PQconninfoMapping PQconninfoMappings[] =
+{
+ /* "authtype" is not used anymore, there is no mapping to PGconn */
+ /* there is no mapping of "service" to PGconn */
+ { "user", offsetof(struct pg_conn, pguser), true, NULL, NULL },
+ { "password", offsetof(struct pg_conn, pgpass), true, NULL, NULL },
+ { "connect_timeout", offsetof(struct pg_conn, connect_timeout), true, NULL, NULL },
+ { "dbname", offsetof(struct pg_conn, dbName), false, NULL, NULL },
+ { "host", offsetof(struct pg_conn, pghost), true, NULL, NULL },
+ { "hostaddr", offsetof(struct pg_conn, pghostaddr), true, NULL, NULL },
+ { "port", offsetof(struct pg_conn, pgport), true, NULL, NULL },
+ { "client_encoding", offsetof(struct pg_conn, client_encoding_initial), false, NULL, NULL },
+ { "tty", offsetof(struct pg_conn, pgtty), false, NULL, NULL },
+ { "options", offsetof(struct pg_conn, pgoptions), true, NULL, NULL },
+ { "application_name", offsetof(struct pg_conn, appname), false, NULL, NULL },
+ { "fallback_application_name", offsetof(struct pg_conn, fbappname), false, NULL, NULL },
+ { "keepalives", offsetof(struct pg_conn, keepalives), true, NULL, NULL },
+ { "keepalives_idle", offsetof(struct pg_conn, keepalives_idle), true, NULL, NULL },
+ { "keepalives_interval", offsetof(struct pg_conn, keepalives_interval), true, NULL, NULL },
+ { "keepalives_count", offsetof(struct pg_conn, keepalives_count), true, NULL, NULL },
+#ifdef USE_SSL
+ { "requiressl", offsetof(struct pg_conn, sslmode), false, "1", "require" },
+#endif
+ { "sslmode", offsetof(struct pg_conn, sslmode), true, NULL, NULL },
+ { "sslcompression", offsetof(struct pg_conn, sslcompression), true, NULL, NULL },
+ { "sslcert", offsetof(struct pg_conn, sslcert), true, NULL, NULL },
+ { "sslkey", offsetof(struct pg_conn, sslkey), true, NULL, NULL },
+ { "sslrootcert", offsetof(struct pg_conn, sslrootcert), true, NULL, NULL },
+ { "sslcrl", offsetof(struct pg_conn, sslcrl), true, NULL, NULL },
+ { "requirepeer", offsetof(struct pg_conn, requirepeer), true, NULL, NULL },
+#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
+ { "krbsrvname", offsetof(struct pg_conn, krbsrvname), true, NULL, NULL },
+#endif
+#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
+ { "gsslib", offsetof(struct pg_conn, requirepeer), true, NULL, NULL },
+#endif
+ { "replication", offsetof(struct pg_conn, replication), false, NULL, NULL },
+ /* Terminating entry --- MUST BE LAST */
+ { NULL, 0, false, NULL, NULL }
+};
+
static const PQEnvironmentOption EnvironmentOptions[] =
{
/* common user-interface settings */
@@ -295,7 +358,8 @@ static PGconn *makeEmptyPGconn(void);
static void fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
-static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage);
+static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage,
+ bool for_replication);
static PQconninfoOption *parse_connection_string(const char *conninfo,
PQExpBuffer errorMessage, bool use_defaults);
static int uri_prefix_length(const char *connstr);
@@ -318,6 +382,8 @@ static char *conninfo_uri_decode(const c
static bool get_hexdigit(char digit, int *value);
static const char *conninfo_getval(PQconninfoOption *connOptions,
const char *keyword);
+static void conninfo_setval(PQconninfoOption *connOptions,
+ const char *keyword, const char *val);
static PQconninfoOption *conninfo_storeval(PQconninfoOption *connOptions,
const char *keyword, const char *value,
PQExpBuffer errorMessage, bool ignoreMissing, bool uri_decode);
@@ -627,7 +693,9 @@ PQconnectStart(const char *conninfo)
static void
fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
{
+ const PQconninfoMapping *mapping;
const char *tmp;
+ char **memberaddr;
/*
* Move option values into conn structure
@@ -637,72 +705,24 @@ fillPGconn(PGconn *conn, PQconninfoOptio
*
* XXX: probably worth checking strdup() return value here...
*/
- tmp = conninfo_getval(connOptions, "hostaddr");
- conn->pghostaddr = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "host");
- conn->pghost = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "port");
- conn->pgport = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "tty");
- conn->pgtty = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "options");
- conn->pgoptions = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "application_name");
- conn->appname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "fallback_application_name");
- conn->fbappname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "dbname");
- conn->dbName = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "user");
- conn->pguser = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "password");
- conn->pgpass = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "connect_timeout");
- conn->connect_timeout = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "client_encoding");
- conn->client_encoding_initial = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives");
- conn->keepalives = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_idle");
- conn->keepalives_idle = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_interval");
- conn->keepalives_interval = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_count");
- conn->keepalives_count = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslmode");
- conn->sslmode = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcompression");
- conn->sslcompression = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslkey");
- conn->sslkey = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcert");
- conn->sslcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslrootcert");
- conn->sslrootcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcrl");
- conn->sslcrl = tmp ? strdup(tmp) : NULL;
-#ifdef USE_SSL
- tmp = conninfo_getval(connOptions, "requiressl");
- if (tmp && tmp[0] == '1')
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
{
- /* here warn that the requiressl option is deprecated? */
- if (conn->sslmode)
- free(conn->sslmode);
- conn->sslmode = strdup("require");
+ tmp = conninfo_getval(connOptions, mapping->keyword);
+ memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ if (mapping->conninfoValue && mapping->connValue)
+ {
+ size_t len = strlen(mapping->conninfoValue);
+ if (tmp && strncmp(tmp, mapping->conninfoValue, len) == 0)
+ {
+ if (*memberaddr)
+ free(*memberaddr);
+ *memberaddr = strdup(mapping->connValue);
+ }
+ }
+ else
+ *memberaddr = tmp ? strdup(tmp) : NULL;
}
-#endif
- tmp = conninfo_getval(connOptions, "requirepeer");
- conn->requirepeer = tmp ? strdup(tmp) : NULL;
-#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "krbsrvname");
- conn->krbsrvname = tmp ? strdup(tmp) : NULL;
-#endif
-#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "gsslib");
- conn->gsslib = tmp ? strdup(tmp) : NULL;
-#endif
- tmp = conninfo_getval(connOptions, "replication");
- conn->replication = tmp ? strdup(tmp) : NULL;
}
/*
@@ -884,7 +904,7 @@ PQconndefaults(void)
if (PQExpBufferDataBroken(errorBuf))
return NULL; /* out of memory already :-( */
- connOptions = conninfo_init(&errorBuf);
+ connOptions = conninfo_init(&errorBuf, false);
if (connOptions != NULL)
{
if (!conninfo_add_defaults(connOptions, &errorBuf))
@@ -4006,9 +4026,11 @@ PQconninfoParse(const char *conninfo, ch
/*
* Build a working copy of the constant PQconninfoOptions array.
+ * If for_replication is true, only return the options that are
+ * not added by libpqwalreceiver.
*/
static PQconninfoOption *
-conninfo_init(PQExpBuffer errorMessage)
+conninfo_init(PQExpBuffer errorMessage, bool for_replication)
{
PQconninfoOption *options;
@@ -4019,7 +4041,28 @@ conninfo_init(PQExpBuffer errorMessage)
libpq_gettext("out of memory\n"));
return NULL;
}
- memcpy(options, PQconninfoOptions, sizeof(PQconninfoOptions));
+ if (for_replication)
+ {
+ const PQconninfoMapping *mapping = PQconninfoMappings;
+ PQconninfoOption *opt_dest = options;
+
+ while (mapping->keyword)
+ {
+ PQconninfoOption *opt_src = conninfo_find((PQconninfoOption *)PQconninfoOptions, mapping->keyword);
+
+ if (opt_src && mapping->for_replication)
+ {
+ memcpy(opt_dest, opt_src, sizeof(PQconninfoOption));
+ opt_dest++;
+ }
+
+ opt_src++;
+ mapping++;
+ }
+ MemSet(opt_dest, 0, sizeof(PQconninfoOption));
+ }
+ else
+ memcpy(options, PQconninfoOptions, sizeof(PQconninfoOptions));
return options;
}
@@ -4095,7 +4138,7 @@ conninfo_parse(const char *conninfo, PQE
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
return NULL;
@@ -4295,7 +4338,7 @@ conninfo_array_parse(const char *const *
}
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
{
PQconninfoFree(dbname_options);
@@ -4485,7 +4528,7 @@ conninfo_uri_parse(const char *uri, PQEx
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, false);
if (options == NULL)
return NULL;
@@ -4985,6 +5028,24 @@ conninfo_getval(PQconninfoOption *connOp
}
/*
+ * Set an option value corresponding to the keyword in the connOptions array.
+ */
+static void
+conninfo_setval(PQconninfoOption *connOptions, const char *keyword,
+ const char *val)
+{
+ PQconninfoOption *option;
+
+ option = conninfo_find(connOptions, keyword);
+ if (option)
+ {
+ if (option->val)
+ free(option->val);
+ option->val = val ? strdup(val) : NULL;
+ }
+}
+
+/*
* Store a (new) value for an option corresponding to the keyword in
* connOptions array.
*
@@ -5066,6 +5127,50 @@ conninfo_find(PQconninfoOption *connOpti
}
+/*
+ * Return the connection options used for the connections
+ */
+PQconninfoOption *
+PQconninfo(PGconn *conn, bool for_replication)
+{
+ PQExpBufferData errorBuf;
+ PQconninfoOption *connOptions;
+
+ if (conn == NULL)
+ return NULL;
+
+ /* We don't actually report any errors here, but callees want a buffer */
+ initPQExpBuffer(&errorBuf);
+ if (PQExpBufferDataBroken(errorBuf))
+ return NULL; /* out of memory already :-( */
+
+ connOptions = conninfo_init(&errorBuf, for_replication);
+
+ termPQExpBuffer(&errorBuf);
+
+ if (connOptions != NULL)
+ {
+ const PQconninfoMapping *mapping;
+
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
+ {
+ char **memberaddr = PGCONNMEMBERADDR(conn, mapping);
+
+ if (mapping->conninfoValue && mapping->connValue)
+ {
+ size_t len = strlen(mapping->connValue);
+ if (*memberaddr && strncmp(*memberaddr, mapping->connValue, len) == 0)
+ conninfo_setval(connOptions, mapping->keyword, mapping->conninfoValue);
+ }
+ else
+ conninfo_setval(connOptions, mapping->keyword, *memberaddr);
+ }
+ }
+
+ return connOptions;
+}
+
+
void
PQconninfoFree(PQconninfoOption *connOptions)
{
diff -durpN postgresql/src/interfaces/libpq/libpq-fe.h postgresql.1/src/interfaces/libpq/libpq-fe.h
--- postgresql/src/interfaces/libpq/libpq-fe.h 2012-10-09 09:58:14.343782980 +0200
+++ postgresql.1/src/interfaces/libpq/libpq-fe.h 2012-10-14 11:12:07.054196291 +0200
@@ -262,6 +262,9 @@ extern PQconninfoOption *PQconndefaults(
/* parse connection options in same way as PQconnectdb */
extern PQconninfoOption *PQconninfoParse(const char *conninfo, char **errmsg);
+/* return the connection options used by a live connection */
+extern PQconninfoOption *PQconninfo(PGconn *conn, bool for_replication);
+
/* free the data structure returned by PQconndefaults() or PQconninfoParse() */
extern void PQconninfoFree(PQconninfoOption *connOptions);
02-pg_basebackup-v14.patchtext/x-patch; name=02-pg_basebackup-v14.patchDownload
diff -durpN postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml
--- postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml 2012-08-24 09:49:22.960530329 +0200
+++ postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml 2012-10-15 09:42:35.293396090 +0200
@@ -189,6 +189,21 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--write-recovery-conf</option></term>
+ <listitem>
+
+ <para>
+ Write a minimal recovery.conf into the output directory (or into
+ the base archive file if <option>--format=tar</option> was specified)
+ using the connection parameters from the command line to ease
+ setting up the standby.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-x</option></term>
<term><option>--xlog</option></term>
<listitem>
diff -durpN postgresql.1/src/bin/pg_basebackup/pg_basebackup.c postgresql.2/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.1/src/bin/pg_basebackup/pg_basebackup.c 2012-10-03 10:40:48.297207389 +0200
+++ postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2012-10-15 09:42:39.078417914 +0200
@@ -25,6 +25,7 @@
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
+#include <time.h>
#ifdef HAVE_LIBZ
#include <zlib.h>
@@ -46,6 +47,7 @@ int compresslevel = 0;
bool includewal = false;
bool streamwal = false;
bool fastcheckpoint = false;
+bool writerecoveryconf = false;
int standby_message_timeout = 10 * 1000; /* 10 sec = default */
/* Progress counters */
@@ -70,6 +72,11 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
+#define TARCHUNKSZ (512)
+#define RCBUFSZ TARCHUNKSZ
+static char *recoveryconf = NULL;
+static size_t recoveryconflen = 0;
+
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
@@ -77,6 +84,15 @@ static void progress_report(int tablespa
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
+static void _reallocRCBuffer(char **buf, int *bufsz, int currsz,
+ char *keyword, char *value,
+ int extrasz);
+static void CreateRecoveryConf(PGconn *conn);
+static void WriteRecoveryConf(void);
+static int _tarChecksum(char *header);
+static void print_val(char *s, uint64 val, unsigned int base, size_t len);
+static void scan_val(char *s, uint64 *val, unsigned int base, size_t len);
+static void _tarCreateHeader(char *header);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -107,6 +123,8 @@ usage(void)
printf(_("\nOptions controlling the output:\n"));
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
+ printf(_(" -R, --write-recovery-conf\n"
+ " write recovery.conf after backup\n"));
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
printf(_(" -X, --xlog-method=fetch|stream\n"
" include required WAL files with specified method\n"));
@@ -467,6 +485,11 @@ ReceiveTarFile(PGconn *conn, PGresult *r
char filename[MAXPGPATH];
char *copybuf = NULL;
FILE *tarfile = NULL;
+ char tarhdr[512];
+ bool basebackup;
+ bool in_tarhdr, skip_file;
+ size_t tarhdrsz;
+ uint64 filesz;
#ifdef HAVE_LIBZ
gzFile ztarfile = NULL;
@@ -519,6 +542,8 @@ ReceiveTarFile(PGconn *conn, PGresult *r
tarfile = fopen(filename, "wb");
}
}
+
+ basebackup = true;
}
else
{
@@ -547,6 +572,8 @@ ReceiveTarFile(PGconn *conn, PGresult *r
PQgetvalue(res, rownum, 0));
tarfile = fopen(filename, "wb");
}
+
+ basebackup = false;
}
#ifdef HAVE_LIBZ
@@ -584,6 +611,10 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = filesz = 0;
+
while (1)
{
int r;
@@ -598,13 +629,69 @@ ReceiveTarFile(PGconn *conn, PGresult *r
if (r == -1)
{
/*
- * End of chunk. Close file (but not stdout).
+ * End of chunk. Write recovery.conf into the tar file (if it
+ * was requested) and close file (but not stdout).
*
* Also, write two completely empty blocks at the end of the tar
* file, as required by some tar programs.
*/
char zerobuf[1024];
+ if (basebackup && writerecoveryconf)
+ {
+ char header[512];
+ int lastchunk;
+
+ _tarCreateHeader(header);
+
+ lastchunk = recoveryconflen % TARCHUNKSZ;
+ if (lastchunk > 0)
+ recoveryconflen += (TARCHUNKSZ - lastchunk);
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (gzwrite(ztarfile, header, sizeof(header)) !=
+ sizeof(header))
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, filename, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+
+ if (gzwrite(ztarfile, recoveryconf, recoveryconflen) !=
+ recoveryconflen)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, filename, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (fwrite(header, sizeof(header), 1, tarfile) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ if (fwrite(recoveryconf, recoveryconflen, 1, tarfile) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ }
+ fprintf(stderr, _("%s: recovery.conf written into '%s'\n"), progname, filename);
+
+ free(recoveryconf);
+ }
+
MemSet(zerobuf, 0, sizeof(zerobuf));
#ifdef HAVE_LIBZ
if (ztarfile != NULL)
@@ -665,25 +752,130 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
-#ifdef HAVE_LIBZ
- if (ztarfile != NULL)
+ if (basebackup && writerecoveryconf)
{
- if (gzwrite(ztarfile, copybuf, r) != r)
+ int rr = r;
+ int pos = 0;
+
+ while (rr > 0)
{
- fprintf(stderr,
- _("%s: could not write to compressed file \"%s\": %s\n"),
- progname, filename, get_gz_error(ztarfile));
- disconnect_and_exit(1);
+ if (in_tarhdr)
+ {
+ if (tarhdrsz < 512)
+ {
+ int hdrleft, bytes2copy;
+
+ hdrleft = TARCHUNKSZ - tarhdrsz;
+ bytes2copy = (rr > hdrleft ? hdrleft : rr);
+
+ memcpy(&tarhdr[tarhdrsz], copybuf + pos, bytes2copy);
+
+ rr -= bytes2copy;
+ pos += bytes2copy;
+ tarhdrsz += bytes2copy;
+ }
+ else
+ {
+ int lastchunk;
+
+ skip_file = (strcmp(&tarhdr[0], "recovery.conf") == 0);
+
+ scan_val(&tarhdr[124], &filesz, 8, 11);
+ lastchunk = filesz % TARCHUNKSZ;
+ if (lastchunk > 0)
+ filesz += (TARCHUNKSZ - lastchunk);
+
+ in_tarhdr = false;
+ tarhdrsz = 0;
+
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (!skip_file && gzwrite(ztarfile, tarhdr, TARCHUNKSZ) != TARCHUNKSZ)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, filename, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (!skip_file && fwrite(tarhdr, TARCHUNKSZ, 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ }
+ }
+ }
+ else /* inside a file */
+ {
+ if (filesz > 0)
+ {
+ int bytes2write;
+
+ bytes2write = (filesz > rr ? rr : filesz);
+
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (!skip_file && gzwrite(ztarfile, copybuf + pos, bytes2write) != bytes2write)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, filename, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (!skip_file && fwrite(copybuf + pos, bytes2write, 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ }
+
+ rr -= bytes2write;
+ pos += bytes2write;
+ filesz -= bytes2write;
+ }
+ else
+ {
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = filesz = 0;
+ }
+ }
}
}
else
-#endif
{
- if (fwrite(copybuf, r, 1, tarfile) != 1)
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
{
- fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
- progname, filename, strerror(errno));
- disconnect_and_exit(1);
+ if (gzwrite(ztarfile, copybuf, r) != r)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, filename, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (fwrite(copybuf, r, 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
}
}
totaldone += r;
@@ -712,13 +904,20 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
char filename[MAXPGPATH];
int current_len_left;
int current_padding = 0;
+ bool basebackup;
char *copybuf = NULL;
FILE *file = NULL;
if (PQgetisnull(res, rownum, 0))
+ {
strcpy(current_path, basedir);
+ basebackup = true;
+ }
else
+ {
strcpy(current_path, PQgetvalue(res, rownum, 1));
+ basebackup = false;
+ }
/*
* Get the COPY data
@@ -937,6 +1136,330 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
if (copybuf != NULL)
PQfreemem(copybuf);
+
+ if (basebackup)
+ WriteRecoveryConf();
+}
+
+static int
+_tarChecksum(char *header)
+{
+ int i,
+ sum;
+
+ /*
+ * Per POSIX, the checksum is the simple sum of all bytes in the header,
+ * treating the bytes as unsigned, and treating the checksum field (at
+ * offset 148) as though it contained 8 spaces.
+ */
+ sum = 8 * ' '; /* presumed value for checksum field */
+ for (i = 0; i < 512; i++)
+ if (i < 148 || i >= 156)
+ sum += 0xFF & header[i];
+ return sum;
+}
+
+
+/*
+ * Utility routine to print possibly larger than 32 bit integers in a
+ * portable fashion. Filled with zeros.
+ */
+static void
+print_val(char *s, uint64 val, unsigned int base, size_t len)
+{
+ int i;
+
+ for (i = len; i > 0; i--)
+ {
+ int digit = val % base;
+
+ s[i - 1] = '0' + digit;
+ val = val / base;
+ }
+}
+
+
+/*
+ * Inverse for print_val()
+ */
+static void
+scan_val(char *s, uint64 *val, unsigned int base, size_t len)
+{
+ uint64 tmp = 0;
+ int i;
+
+ for (i = 0; i < len; i++)
+ {
+ int digit = s[i] - '0';
+
+ tmp = tmp * base + digit;
+ }
+
+ *val = tmp;
+}
+
+
+static void
+_tarCreateHeader(char *header)
+{
+ /*
+ * Note: most of the fields in a tar header are not supposed to be
+ * null-terminated. We use sprintf, which will write a null after the
+ * required bytes; that null goes into the first byte of the next field.
+ * This is okay as long as we fill the fields in order.
+ */
+ memset(header, 0, 512 /* sizeof the tar header */);
+
+ /* Name 100 */
+ sprintf(&header[0], "%.99s", "recovery.conf");
+
+ /* Mode 8 */
+ sprintf(&header[100], "0000600 ");
+
+ /* User ID 8 */
+ sprintf(&header[108], "0004000 ");
+
+ /* Group 8 */
+ sprintf(&header[116], "0002000 ");
+
+ /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
+ print_val(&header[124], recoveryconflen, 8, 11);
+ sprintf(&header[135], " ");
+
+ /* Mod Time 12 */
+ sprintf(&header[136], "%011o ", (int) time(NULL));
+
+ /* Checksum 8 cannot be calculated until we've filled all other fields */
+
+ /* Type - regular file */
+ sprintf(&header[156], "0");
+
+ /* Link Name 100 (leave as nulls) */
+
+ /* Magic 6 */
+ sprintf(&header[257], "ustar");
+
+ /* Version 2 */
+ sprintf(&header[263], "00");
+
+ /* User 32 */
+ /* XXX: Do we need to care about setting correct username? */
+ sprintf(&header[265], "%.31s", "postgres");
+
+ /* Group 32 */
+ /* XXX: Do we need to care about setting correct group name? */
+ sprintf(&header[297], "%.31s", "postgres");
+
+ /* Major Dev 8 */
+ sprintf(&header[329], "%07o ", 0);
+
+ /* Minor Dev 8 */
+ sprintf(&header[337], "%07o ", 0);
+
+ /* Prefix 155 - not used, leave as nulls */
+
+ /*
+ * We mustn't overwrite the next field while inserting the checksum.
+ * Fortunately, the checksum can't exceed 6 octal digits, so we just write
+ * 6 digits, a space, and a null, which is legal per POSIX.
+ */
+ sprintf(&header[148], "%06o ", _tarChecksum(header));
+}
+
+/*
+ * Escape single quotes in a string
+ */
+static char *
+escape_string(char *value)
+{
+ char *tmp, *escaped, *esc;
+ int num_quotes;
+
+ tmp = value;
+ num_quotes = 0;
+ while (*tmp)
+ {
+ if (*tmp == '\'')
+ num_quotes++;
+ tmp++;
+ }
+
+ escaped = malloc((tmp - value) + num_quotes + 1);
+ if (escaped == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1)
+ }
+
+ tmp = value;
+ esc = escaped;
+ while (*tmp)
+ {
+ if (*tmp == '\'')
+ *esc++ = '\\';
+ *esc++ = *tmp++;
+ }
+ *esc = '\0';
+
+ return escaped;
+}
+
+/*
+ * Reallocate the buffer for recovery.conf if needed
+ */
+static void
+_reallocRCBuffer(char **buf, int *bufsz, int currsz, char *keyword, char *value, int extrasz)
+{
+ int len = extrasz;
+
+ if (keyword != NULL)
+ len += strlen(keyword);
+ if (value != NULL)
+ len += strlen(value);
+
+ if (*bufsz < currsz + len)
+ {
+ char *tmp;
+
+ tmp = realloc(*buf, *bufsz + RCBUFSZ);
+ if (tmp == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1)
+ }
+ *buf = tmp;
+ *bufsz += RCBUFSZ;
+ memset(tmp + currsz, 0, *bufsz - currsz);
+ }
+}
+
+/*
+ * Try to create recovery.conf in memory and set the length to write later.
+ */
+static void
+CreateRecoveryConf(PGconn *conn)
+{
+ char *buf = NULL;
+ PQconninfoOption *connOptions;
+ PQconninfoOption *option;
+ int bufsz;
+ int total = 0;
+ int written;
+
+ if (!writerecoveryconf)
+ return;
+
+ connOptions = PQconninfo(conn, true);
+ if (connOptions == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+
+ bufsz = RCBUFSZ;
+ buf = malloc(bufsz);
+ if (buf == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ written = sprintf(&buf[total], "standby_mode = 'on'\n");
+ if (written < 0)
+ {
+ fprintf(stderr, _("%s: cannot write to string: %s"), progname, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ total += written;
+
+ written = sprintf(&buf[total], "primary_conninfo = '");
+ if (written < 0)
+ {
+ fprintf(stderr, _("%s: cannot write to string: %s"), progname, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ total += written;
+
+ for (option = connOptions; option && option->keyword; option++)
+ {
+ char *escaped;
+ /*
+ * Do not emit this setting if not set or empty.
+ * The list of options was already pre-filtered for options
+ * usable for replication with PQconninfo(conn, true).
+ */
+ if ((option->val == NULL) ||
+ (option->val != NULL && option->val[0] == '\0'))
+ continue;
+
+ /*
+ * Write "keyword='value'" pieces, the value string is escaped
+ * if necessary and doubled single quotes around the value string.
+ */
+ escaped = escape_string(option->val);
+
+ _reallocRCBuffer(&buf, &bufsz, total, option->keyword, escaped, 6);
+
+ written = sprintf(&buf[total], "%s=''%s'' ", option->keyword, option->val);
+
+ free(escaped);
+
+ if (written < 0)
+ {
+ fprintf(stderr, _("%s: cannot write to string: %s"), progname, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ total += written;
+ }
+
+ _reallocRCBuffer(&buf, &bufsz, total, NULL, NULL, 2);
+ written = sprintf(&buf[total], "'\n");
+ if (written < 0)
+ {
+ fprintf(stderr, _("%s: cannot write to string: %s"), progname, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ total += written;
+
+ recoveryconf = buf;
+ recoveryconflen = total;
+
+ PQconninfoFree(connOptions);
+}
+
+
+static void
+WriteRecoveryConf(void)
+{
+ char filename[MAXPGPATH];
+ FILE *cf;
+
+ if (!writerecoveryconf)
+ return;
+
+ sprintf(filename, "%s/recovery.conf", basedir);
+
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("%s: cannot create %s: %s"), progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ if (fwrite(recoveryconf, recoveryconflen, 1, cf) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ fclose(cf);
+
+ free(recoveryconf);
+
+ fprintf(stderr, _("%s: recovery.conf written.\n"), progname);
}
@@ -960,6 +1483,8 @@ BaseBackup(void)
/* Error message already written in GetConnection() */
exit(1);
+ CreateRecoveryConf(conn);
+
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
@@ -1243,6 +1768,7 @@ main(int argc, char **argv)
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
+ {"write-recovery-conf", no_argument, NULL, 'R'},
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
{"gzip", no_argument, NULL, 'z'},
@@ -1280,7 +1806,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:xX:l:zZ:c:h:p:U:s:wWvP",
+ while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
long_options, &option_index)) != -1)
{
switch (c)
@@ -1301,6 +1827,9 @@ main(int argc, char **argv)
exit(1);
}
break;
+ case 'R':
+ writerecoveryconf = true;
+ break;
case 'x':
if (includewal)
{
Boszormenyi Zoltan escribió:
Also, the check for conflict between -R and -x/-X is now removed.
The documentation for option -R has changed to reflect this and
there is no different error code 2 now: it would make the behaviour
inconsistent between -Fp and -Ft.The PQconninfo patch is also attached but didn't change since the last mail.
Magnus,
This patch is all yours to handle. I'm guessing nothing will happen
until pgconf.eu is done and over, but hopefully you can share a few
beers with Zoltan over the whole subject (and maybe with Peter about the
PQconninfo stuff?)
I'm not closing this just yet, but if you're not able to handle this
soon, maybe it'd be better to punt it to the November commitfest.
--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Oct 23, 2012 4:52 PM, "Alvaro Herrera" <alvherre@2ndquadrant.com> wrote:
Boszormenyi Zoltan escribió:
Also, the check for conflict between -R and -x/-X is now removed.
The documentation for option -R has changed to reflect this and
there is no different error code 2 now: it would make the behaviour
inconsistent between -Fp and -Ft.The PQconninfo patch is also attached but didn't change since the
last mail.
Magnus,
This patch is all yours to handle. I'm guessing nothing will happen
until pgconf.eu is done and over, but hopefully you can share a few
beers with Zoltan over the whole subject (and maybe with Peter about the
PQconninfo stuff?)I'm not closing this just yet, but if you're not able to handle this
soon, maybe it'd be better to punt it to the November commitfest.
It's on my to do list for when I get back, but correct, won't get to it
until after the conference.
/Magnus
On Tue, Oct 23, 2012 at 5:08 PM, Magnus Hagander <magnus@hagander.net> wrote:
On Oct 23, 2012 4:52 PM, "Alvaro Herrera" <alvherre@2ndquadrant.com> wrote:
Boszormenyi Zoltan escribió:
Also, the check for conflict between -R and -x/-X is now removed.
The documentation for option -R has changed to reflect this and
there is no different error code 2 now: it would make the behaviour
inconsistent between -Fp and -Ft.The PQconninfo patch is also attached but didn't change since the
last mail.Magnus,
This patch is all yours to handle. I'm guessing nothing will happen
until pgconf.eu is done and over, but hopefully you can share a few
beers with Zoltan over the whole subject (and maybe with Peter about the
PQconninfo stuff?)I'm not closing this just yet, but if you're not able to handle this
soon, maybe it'd be better to punt it to the November commitfest.It's on my to do list for when I get back, but correct, won't get to it
until after the conference.
I finally got around to looking at this patch now. Sorry about the way
too long delay.
A few thoughts:
First, on the libpq patch:
I'm not sure I like the for_replication flag to PQconninfo(). It seems
we're it's a quite limited API, and not very future proof. What's to
say that an app would only be interested in filtering based on
for_replication? One idea might be to have a bitmap field there, and
assign *all* conninfo options to a category. We could then have
categories for NORMAL and REPLICATION. But we could also for example
have a category for PASSWORD (or similar), so that you could get with
and without those. Passing in a 32-bit integer would allow us to have
32 different categories, and we could then use a bitmask to pick
things out.
It might sound a bit like overengineering, but it's also an API and
it's a PITA to change it in the future if more needs show up..
Second, I wonder if we really need to add the code for requiressl=1,
or if we should just remove it. The spelling requiressl=1 was
deprecated back in 7.4 - which has obviously been out of support for a
long time now.
Third, in fillPGconn. If mapping has both conninfoValue and connvalue,
it does a free() on the old value in memberaddr, but if it doesn't it
just overwrites memberaddr with strdup(). Shouldn't those two paths be
the same, meaning shouldn't the if (*memberaddr) free(*memberaddr);
check be outside the if block?
Fourth, I'm not sure I like the "memberaddr" term. It's not wrong, but
it threw me off a couple of times while reading it. It's not all that
important, and I'm not sure about another idea for it though - maybe
just "connmember"?
Then, about the pg_basebackup patch:
What's the reason for the () around 512 for TARCHUNKSZ?
We have a lot of hardcoded entries of the 512 for tar in that file. We
should either keep using 512 as a constant, or change all those to
*also* use the #define. Right now, the code will obviously break if
you change the #define (for example, it compares to 512, but then uses
hdrleft = TARCHUNKSZ - tarhdrsz; to do calculation).
The name choice of "basebackup" for the bool in ReceiveTarFile() is
unfortunate, since we use the term base backup to refer to the
complete thing, not just the main tablespace. Probably
"basetablespcae" instead. And it should then be assigned at the top of
the function (to the result of PQgetisnull()), and used in the main
if() branch as well.
Should we really print the status message even if not in verbose mode?
We only print the "base backup complete" messages in verbose mode, for
example.
It seems wrong to free() recoveryconf in ReceiveTarFile(). It's
allocated globally at the beginning. While that loop should only be
called once (since only one tablespace can be the main one), it's a
confusing location for the free.
The whole tar writing part of the code needs a lot more comments. It's
entirely unclear what the code does there. Why does the recovery.conf
writing code need to be split up in multiple locations inside
ReceiveTarFile(), for example? It either needs to be simplified, or
explained why it can't be simplified in comments.
_tarCreateHeader() is really badly named, since it specifically
creates a tar header for recovery.conf only. Either that needs to be a
parameter, or it needs to have a name that indicates this is the only
thing it does. The former is probably better.
Much of the tar stuff is very similar (I haven't looked to see if it's
identical) to the stuff in backend/replication/basebackup.c. Perhaps
we should have a src/port/tarutil.c?
escape_string() - already exists as escape_quotes() in initdb, AFAICT.
We should either move it to src/port/, or at least copy it so it's
exactly the same.
CreateRecoveryConf() should just use PQExpBuffer (in libpq), I think -
that does away with a lot of code. We already use this from e.g.
pg_dump, so there's a precedent for using internal code from libpq in
frontends.
Again, my apologies for this review taking so long. I will try to be
more attentive to the next round :S
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
Hi,
2012-11-18 17:20 keltezéssel, Magnus Hagander írta:
On Tue, Oct 23, 2012 at 5:08 PM, Magnus Hagander <magnus@hagander.net> wrote:
On Oct 23, 2012 4:52 PM, "Alvaro Herrera" <alvherre@2ndquadrant.com> wrote:
Boszormenyi Zoltan escribió:
Also, the check for conflict between -R and -x/-X is now removed.
The documentation for option -R has changed to reflect this and
there is no different error code 2 now: it would make the behaviour
inconsistent between -Fp and -Ft.The PQconninfo patch is also attached but didn't change since the
last mail.Magnus,
This patch is all yours to handle. I'm guessing nothing will happen
until pgconf.eu is done and over, but hopefully you can share a few
beers with Zoltan over the whole subject (and maybe with Peter about the
PQconninfo stuff?)I'm not closing this just yet, but if you're not able to handle this
soon, maybe it'd be better to punt it to the November commitfest.It's on my to do list for when I get back, but correct, won't get to it
until after the conference.I finally got around to looking at this patch now. Sorry about the way
too long delay.
No problem, thanks for looking at it.
A few thoughts:
First, on the libpq patch:
I'm not sure I like the for_replication flag to PQconninfo(). It seems
we're it's a quite limited API, and not very future proof. What's to
say that an app would only be interested in filtering based on
for_replication? One idea might be to have a bitmap field there, and
assign *all* conninfo options to a category. We could then have
categories for NORMAL and REPLICATION. But we could also for example
have a category for PASSWORD (or similar), so that you could get with
and without those. Passing in a 32-bit integer would allow us to have
32 different categories, and we could then use a bitmask to pick
things out.It might sound a bit like overengineering, but it's also an API and
it's a PITA to change it in the future if more needs show up..
You are right, I will change this accordingly.
Second, I wonder if we really need to add the code for requiressl=1,
or if we should just remove it. The spelling requiressl=1 was
deprecated back in 7.4 - which has obviously been out of support for a
long time now.
This needs opinions from more people, I am not the one to decide it.
The code would be definitely cleaner without processing this extra
non-1:1 mapping.
Third, in fillPGconn. If mapping has both conninfoValue and connvalue,
it does a free() on the old value in memberaddr, but if it doesn't it
just overwrites memberaddr with strdup(). Shouldn't those two paths be
the same, meaning shouldn't the if (*memberaddr) free(*memberaddr);
check be outside the if block?
Yes, and set it to NULL too. Otherwise there might be a case when
the free() leaves a stale pointer value if the extra mapping fails
the strcmp() check. This is all unnecessary if the extra mapping
for requiressl=1 is removed, the code would be straight.
Fourth, I'm not sure I like the "memberaddr" term. It's not wrong, but
it threw me off a couple of times while reading it. It's not all that
important, and I'm not sure about another idea for it though - maybe
just "connmember"?
I am not attached to this variable name, I will change it.
Then, about the pg_basebackup patch:
What's the reason for the () around 512 for TARCHUNKSZ?
It's simply a habit, to not forget it for more complex macros.
We have a lot of hardcoded entries of the 512 for tar in that file. We
should either keep using 512 as a constant, or change all those to
*also* use the #define. Right now, the code will obviously break if
you change the #define (for example, it compares to 512, but then uses
hdrleft = TARCHUNKSZ - tarhdrsz; to do calculation).
Yes, I left 5 pieces of the hardcoded value of 512, because
I (maybe erroneously) distinguished between a file header
and file "chunks" inside a TAR file, they are all 512.
Is it okay to change every hardcoded 512 to TARCHUNKSZ,
maybe adding a comment to the #define that it must not
be modified ever?
The name choice of "basebackup" for the bool in ReceiveTarFile() is
unfortunate, since we use the term base backup to refer to the
complete thing, not just the main tablespace. Probably
"basetablespcae" instead. And it should then be assigned at the top of
the function (to the result of PQgetisnull()), and used in the main
if() branch as well.
Will change it.
Should we really print the status message even if not in verbose mode?
We only print the "base backup complete" messages in verbose mode, for
example.
OK.
It seems wrong to free() recoveryconf in ReceiveTarFile(). It's
allocated globally at the beginning. While that loop should only be
called once (since only one tablespace can be the main one), it's a
confusing location for the free.The whole tar writing part of the code needs a lot more comments. It's
entirely unclear what the code does there. Why does the recovery.conf
writing code need to be split up in multiple locations inside
ReceiveTarFile(), for example? It either needs to be simplified, or
explained why it can't be simplified in comments._tarCreateHeader() is really badly named, since it specifically
creates a tar header for recovery.conf only. Either that needs to be a
parameter, or it needs to have a name that indicates this is the only
thing it does. The former is probably better.Much of the tar stuff is very similar (I haven't looked to see if it's
identical) to the stuff in backend/replication/basebackup.c. Perhaps
we should have a src/port/tarutil.c?
I copied the tar stuff from bin/pg_dump/pg_backup_tar.c,
so there are at least two copies of this already.
I will look into unifying them.
escape_string() - already exists as escape_quotes() in initdb, AFAICT.
We should either move it to src/port/, or at least copy it so it's
exactly the same.
OK. I can copy it, too. ;-)
CreateRecoveryConf() should just use PQExpBuffer (in libpq), I think -
that does away with a lot of code. We already use this from e.g.
pg_dump, so there's a precedent for using internal code from libpq in
frontends.
OK.
Again, my apologies for this review taking so long. I will try to be
more attentive to the next round :S
No problem. I will try to update the patches according to
your comments as soon as possible.
Thanks and best regards,
Zoltán Böszörményi
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
2012-11-18 17:20 keltezéssel, Magnus Hagander írta:
On Tue, Oct 23, 2012 at 5:08 PM, Magnus Hagander <magnus@hagander.net> wrote:
On Oct 23, 2012 4:52 PM, "Alvaro Herrera" <alvherre@2ndquadrant.com> wrote:
Boszormenyi Zoltan escribió:
Also, the check for conflict between -R and -x/-X is now removed.
The documentation for option -R has changed to reflect this and
there is no different error code 2 now: it would make the behaviour
inconsistent between -Fp and -Ft.The PQconninfo patch is also attached but didn't change since the
last mail.Magnus,
This patch is all yours to handle. I'm guessing nothing will happen
until pgconf.eu is done and over, but hopefully you can share a few
beers with Zoltan over the whole subject (and maybe with Peter about the
PQconninfo stuff?)I'm not closing this just yet, but if you're not able to handle this
soon, maybe it'd be better to punt it to the November commitfest.It's on my to do list for when I get back, but correct, won't get to it
until after the conference.I finally got around to looking at this patch now. Sorry about the way
too long delay.A few thoughts:
First, on the libpq patch:
I'm not sure I like the for_replication flag to PQconninfo(). It seems
we're it's a quite limited API, and not very future proof. What's to
say that an app would only be interested in filtering based on
for_replication? One idea might be to have a bitmap field there, and
assign *all* conninfo options to a category. We could then have
categories for NORMAL and REPLICATION. But we could also for example
have a category for PASSWORD (or similar), so that you could get with
and without those. Passing in a 32-bit integer would allow us to have
32 different categories, and we could then use a bitmask to pick
things out.It might sound a bit like overengineering, but it's also an API and
it's a PITA to change it in the future if more needs show up..
Check.
Second, I wonder if we really need to add the code for requiressl=1,
or if we should just remove it. The spelling requiressl=1 was
deprecated back in 7.4 - which has obviously been out of support for a
long time now.
I removed this option, the code is simpler, thanks to this.
Third, in fillPGconn. If mapping has both conninfoValue and connvalue,
it does a free() on the old value in memberaddr, but if it doesn't it
just overwrites memberaddr with strdup(). Shouldn't those two paths be
the same, meaning shouldn't the if (*memberaddr) free(*memberaddr);
check be outside the if block?
This point is now moot, see above.
Fourth, I'm not sure I like the "memberaddr" term. It's not wrong, but
it threw me off a couple of times while reading it. It's not all that
important, and I'm not sure about another idea for it though - maybe
just "connmember"?
The variable is now "connmember".
Also, I noticed that there was already a conninfo_storeval(),
the new patch uses it and there's no need to introduce a
new conninfo_setval() function.
Then, about the pg_basebackup patch:
What's the reason for the () around 512 for TARCHUNKSZ?
Removed the () from around the value.
We have a lot of hardcoded entries of the 512 for tar in that file. We
should either keep using 512 as a constant, or change all those to
*also* use the #define. Right now, the code will obviously break if
you change the #define (for example, it compares to 512, but then uses
hdrleft = TARCHUNKSZ - tarhdrsz; to do calculation).
All 512 constants are now using the #define.
The name choice of "basebackup" for the bool in ReceiveTarFile() is
unfortunate, since we use the term base backup to refer to the
complete thing, not just the main tablespace. Probably
"basetablespcae" instead. And it should then be assigned at the top of
the function (to the result of PQgetisnull()), and used in the main
if() branch as well.
Done without your typo, so the variable is "basetablespace". ;-)
Should we really print the status message even if not in verbose mode?
We only print the "base backup complete" messages in verbose mode, for
example.
The message is written only in verbose mode now.
It seems wrong to free() recoveryconf in ReceiveTarFile(). It's
allocated globally at the beginning. While that loop should only be
called once (since only one tablespace can be the main one), it's a
confusing location for the free.
See below.
The whole tar writing part of the code needs a lot more comments. It's
entirely unclear what the code does there. Why does the recovery.conf
writing code need to be split up in multiple locations inside
ReceiveTarFile(), for example? It either needs to be simplified, or
explained why it can't be simplified in comments._tarCreateHeader() is really badly named, since it specifically
creates a tar header for recovery.conf only. Either that needs to be a
parameter, or it needs to have a name that indicates this is the only
thing it does. The former is probably better.
_tarCreateHeader() now accepts the file name and the file size arguments.
Much of the tar stuff is very similar (I haven't looked to see if it's
identical) to the stuff in backend/replication/basebackup.c. Perhaps
we should have a src/port/tarutil.c?
I will implement it as a separate patch.
escape_string() - already exists as escape_quotes() in initdb, AFAICT.
We should either move it to src/port/, or at least copy it so it's
exactly the same.
A copy of escape_quotes() is now in pg_basebackup.c and is used.
I will also unify the copies of it in a separate patch.
CreateRecoveryConf() should just use PQExpBuffer (in libpq), I think -
that does away with a lot of code. We already use this from e.g.
pg_dump, so there's a precedent for using internal code from libpq in
frontends.
PQexpBuffer is used now and it's created and destroyed inside BaseBackup().
Again, my apologies for this review taking so long. I will try to be
more attentive to the next round :S
Please, review the new patches.
Best regards,
Zoltán Böszörményi
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
01-PQconninfo-v15.patchtext/x-patch; name=01-PQconninfo-v15.patchDownload
diff -durpN postgresql/doc/src/sgml/libpq.sgml postgresql.1/doc/src/sgml/libpq.sgml
--- postgresql/doc/src/sgml/libpq.sgml 2012-08-03 09:39:30.114266570 +0200
+++ postgresql.1/doc/src/sgml/libpq.sgml 2012-11-20 11:57:41.359859195 +0100
@@ -496,6 +496,49 @@ typedef struct
</listitem>
</varlistentry>
+ <varlistentry id="libpq-pqconninfo">
+ <term><function>PQconninfo</function><indexterm><primary>PQconninfo</></></term>
+ <listitem>
+ <para>
+ Returns the connection options used by a live connection.
+<synopsis>
+/*
+ * Option flags for PQconninfo
+ */
+#define PG_CONNINFO_NORMAL 0x01
+#define PG_CONNINFO_PASSWORD 0x02
+#define PG_CONNINFO_REPLICATION 0x04
+
+PQconninfoOption *PQconninfo(PGconn *conn, int flags);
+</synopsis>
+ </para>
+
+ <para>
+ Returns a connection options array. This can be used to determine
+ all possible <function>PQconnectdb</function> options and their
+ current values that were used to connect to the server. The return
+ value points to an array of <structname>PQconninfoOption</structname>
+ structures, which ends with an entry having a null <structfield>keyword</>
+ pointer. Every notes above for <function>PQconndefaults</function> also apply.
+ An application may present a dialog using the previous settings by:
+<programlisting>
+ PQconninfoOption *options = PQconninfo(conn, PG_CONNINFO_NORMAL | PG_CONNINFO_PASSWORD);
+</programlisting>
+ </para>
+
+ <para>
+ The array returned by the <literal>PG_CONNINFO_REPLICATION</> flag is
+ a subset of <literal>PG_CONNINFO_NORMAL | PG_CONNINFO_PASSWORD</>.
+ This subset excludes some options from the array which are used by the
+ walreceiver module. <application>pg_basebackup</application> uses this
+ pre-filtered list of options to construct <literal>primary_conninfo</>
+ in the automatically generated recovery.conf file.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+
<varlistentry id="libpq-pqconninfoparse">
<term><function>PQconninfoParse</function><indexterm><primary>PQconninfoParse</></></term>
<listitem>
diff -durpN postgresql/src/interfaces/libpq/exports.txt postgresql.1/src/interfaces/libpq/exports.txt
--- postgresql/src/interfaces/libpq/exports.txt 2012-10-09 09:58:14.342782974 +0200
+++ postgresql.1/src/interfaces/libpq/exports.txt 2012-11-20 10:55:03.396145749 +0100
@@ -164,3 +164,4 @@ PQsetSingleRowMode 161
lo_lseek64 162
lo_tell64 163
lo_truncate64 164
+PQconninfo 165
diff -durpN postgresql/src/interfaces/libpq/fe-connect.c postgresql.1/src/interfaces/libpq/fe-connect.c
--- postgresql/src/interfaces/libpq/fe-connect.c 2012-09-09 08:11:09.470401480 +0200
+++ postgresql.1/src/interfaces/libpq/fe-connect.c 2012-11-20 12:53:49.728643547 +0100
@@ -137,6 +137,9 @@ static int ldapServiceLookup(const char
* PQconninfoOptions[] *must* be NULL. In a working copy, non-null "val"
* fields point to malloc'd strings that should be freed when the working
* array is freed (see PQconninfoFree).
+ *
+ * If you add a new connection option to this list, remember to add it to
+ * PQconninfoMappings[] below.
* ----------
*/
static const PQconninfoOption PQconninfoOptions[] = {
@@ -203,16 +206,6 @@ static const PQconninfoOption PQconninfo
{"keepalives_count", NULL, NULL, NULL,
"TCP-Keepalives-Count", "", 10}, /* strlen(INT32_MAX) == 10 */
-#ifdef USE_SSL
-
- /*
- * "requiressl" is deprecated, its purpose having been taken over by
- * "sslmode". It remains for backwards compatibility.
- */
- {"requiressl", "PGREQUIRESSL", "0", NULL,
- "Require-SSL", "D", 1},
-#endif
-
/*
* ssl options are allowed even without client SSL support because the
* client can still handle SSL modes "disable" and "allow". Other
@@ -264,6 +257,83 @@ static const PQconninfoOption PQconninfo
NULL, NULL, 0}
};
+/*
+ * We need a mapping between the PQconninfoOptions[] array
+ * and PGconn members. We have to keep it separate from
+ * PQconninfoOptions[] to not leak info about PGconn members
+ * to clients.
+ */
+typedef struct PQconninfoMapping {
+ char *keyword;
+ size_t member_offset;
+ int flags;
+} PQconninfoMapping;
+#define PGCONNMEMBERADDR(conn, mapping) ((char **)((char *)conn + mapping->member_offset))
+
+static const PQconninfoMapping PQconninfoMappings[] =
+{
+ /* "authtype" is not used anymore, there is no mapping to PGconn */
+ /* there is no mapping of "service" to PGconn */
+ { "user", offsetof(struct pg_conn, pguser),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "password", offsetof(struct pg_conn, pgpass),
+ PG_CONNINFO_PASSWORD | PG_CONNINFO_REPLICATION },
+ { "connect_timeout", offsetof(struct pg_conn, connect_timeout),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "dbname", offsetof(struct pg_conn, dbName),
+ PG_CONNINFO_NORMAL },
+ { "host", offsetof(struct pg_conn, pghost),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "hostaddr", offsetof(struct pg_conn, pghostaddr),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "port", offsetof(struct pg_conn, pgport),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "client_encoding", offsetof(struct pg_conn, client_encoding_initial),
+ PG_CONNINFO_NORMAL },
+ { "tty", offsetof(struct pg_conn, pgtty),
+ PG_CONNINFO_NORMAL },
+ { "options", offsetof(struct pg_conn, pgoptions),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "application_name", offsetof(struct pg_conn, appname),
+ PG_CONNINFO_NORMAL },
+ { "fallback_application_name", offsetof(struct pg_conn, fbappname),
+ PG_CONNINFO_NORMAL },
+ { "keepalives", offsetof(struct pg_conn, keepalives),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "keepalives_idle", offsetof(struct pg_conn, keepalives_idle),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "keepalives_interval", offsetof(struct pg_conn, keepalives_interval),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "keepalives_count", offsetof(struct pg_conn, keepalives_count),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "sslmode", offsetof(struct pg_conn, sslmode),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "sslcompression", offsetof(struct pg_conn, sslcompression),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "sslcert", offsetof(struct pg_conn, sslcert),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "sslkey", offsetof(struct pg_conn, sslkey),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "sslrootcert", offsetof(struct pg_conn, sslrootcert),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "sslcrl", offsetof(struct pg_conn, sslcrl),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "requirepeer", offsetof(struct pg_conn, requirepeer),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
+ { "krbsrvname", offsetof(struct pg_conn, krbsrvname),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+#endif
+#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
+ { "gsslib", offsetof(struct pg_conn, requirepeer),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+#endif
+ { "replication", offsetof(struct pg_conn, replication),
+ PG_CONNINFO_NORMAL },
+ /* Terminating entry --- MUST BE LAST */
+ { NULL, 0, 0 }
+};
+
static const PQEnvironmentOption EnvironmentOptions[] =
{
/* common user-interface settings */
@@ -295,7 +365,8 @@ static PGconn *makeEmptyPGconn(void);
static void fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
-static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage);
+static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage,
+ int flags);
static PQconninfoOption *parse_connection_string(const char *conninfo,
PQExpBuffer errorMessage, bool use_defaults);
static int uri_prefix_length(const char *connstr);
@@ -627,7 +698,7 @@ PQconnectStart(const char *conninfo)
static void
fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
{
- const char *tmp;
+ const PQconninfoMapping *mapping;
/*
* Move option values into conn structure
@@ -637,72 +708,19 @@ fillPGconn(PGconn *conn, PQconninfoOptio
*
* XXX: probably worth checking strdup() return value here...
*/
- tmp = conninfo_getval(connOptions, "hostaddr");
- conn->pghostaddr = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "host");
- conn->pghost = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "port");
- conn->pgport = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "tty");
- conn->pgtty = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "options");
- conn->pgoptions = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "application_name");
- conn->appname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "fallback_application_name");
- conn->fbappname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "dbname");
- conn->dbName = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "user");
- conn->pguser = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "password");
- conn->pgpass = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "connect_timeout");
- conn->connect_timeout = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "client_encoding");
- conn->client_encoding_initial = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives");
- conn->keepalives = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_idle");
- conn->keepalives_idle = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_interval");
- conn->keepalives_interval = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_count");
- conn->keepalives_count = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslmode");
- conn->sslmode = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcompression");
- conn->sslcompression = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslkey");
- conn->sslkey = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcert");
- conn->sslcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslrootcert");
- conn->sslrootcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcrl");
- conn->sslcrl = tmp ? strdup(tmp) : NULL;
-#ifdef USE_SSL
- tmp = conninfo_getval(connOptions, "requiressl");
- if (tmp && tmp[0] == '1')
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
{
- /* here warn that the requiressl option is deprecated? */
- if (conn->sslmode)
- free(conn->sslmode);
- conn->sslmode = strdup("require");
+ const char *tmp = conninfo_getval(connOptions, mapping->keyword);
+
+ if (tmp)
+ {
+ char **connmember = PGCONNMEMBERADDR(conn, mapping);
+
+ if (*connmember)
+ free(*connmember);
+ *connmember = tmp ? strdup(tmp) : NULL;
+ }
}
-#endif
- tmp = conninfo_getval(connOptions, "requirepeer");
- conn->requirepeer = tmp ? strdup(tmp) : NULL;
-#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "krbsrvname");
- conn->krbsrvname = tmp ? strdup(tmp) : NULL;
-#endif
-#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "gsslib");
- conn->gsslib = tmp ? strdup(tmp) : NULL;
-#endif
- tmp = conninfo_getval(connOptions, "replication");
- conn->replication = tmp ? strdup(tmp) : NULL;
}
/*
@@ -884,7 +902,7 @@ PQconndefaults(void)
if (PQExpBufferDataBroken(errorBuf))
return NULL; /* out of memory already :-( */
- connOptions = conninfo_init(&errorBuf);
+ connOptions = conninfo_init(&errorBuf, PG_CONNINFO_NORMAL | PG_CONNINFO_PASSWORD);
if (connOptions != NULL)
{
if (!conninfo_add_defaults(connOptions, &errorBuf))
@@ -4008,9 +4026,11 @@ PQconninfoParse(const char *conninfo, ch
* Build a working copy of the constant PQconninfoOptions array.
*/
static PQconninfoOption *
-conninfo_init(PQExpBuffer errorMessage)
+conninfo_init(PQExpBuffer errorMessage, int flags)
{
+ const PQconninfoMapping *mapping;
PQconninfoOption *options;
+ PQconninfoOption *opt_dest;
options = (PQconninfoOption *) malloc(sizeof(PQconninfoOptions));
if (options == NULL)
@@ -4019,7 +4039,23 @@ conninfo_init(PQExpBuffer errorMessage)
libpq_gettext("out of memory\n"));
return NULL;
}
- memcpy(options, PQconninfoOptions, sizeof(PQconninfoOptions));
+
+ opt_dest = options;
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
+ {
+ PQconninfoOption *opt_src;
+
+ if (!(mapping->flags & flags))
+ continue;
+
+ opt_src = conninfo_find((PQconninfoOption *)PQconninfoOptions, mapping->keyword);
+ if (opt_src)
+ {
+ memcpy(opt_dest, opt_src, sizeof(PQconninfoOption));
+ opt_dest++;
+ }
+ }
+ MemSet(opt_dest, 0, sizeof(PQconninfoOption));
return options;
}
@@ -4095,7 +4131,7 @@ conninfo_parse(const char *conninfo, PQE
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, PG_CONNINFO_NORMAL | PG_CONNINFO_PASSWORD);
if (options == NULL)
return NULL;
@@ -4295,7 +4331,7 @@ conninfo_array_parse(const char *const *
}
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, PG_CONNINFO_NORMAL | PG_CONNINFO_PASSWORD);
if (options == NULL)
{
PQconninfoFree(dbname_options);
@@ -4485,7 +4521,7 @@ conninfo_uri_parse(const char *uri, PQEx
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, PG_CONNINFO_NORMAL | PG_CONNINFO_PASSWORD);
if (options == NULL)
return NULL;
@@ -5066,6 +5102,44 @@ conninfo_find(PQconninfoOption *connOpti
}
+/*
+ * Return the connection options used for the connections
+ */
+PQconninfoOption *
+PQconninfo(PGconn *conn, int flags)
+{
+ PQExpBufferData errorBuf;
+ PQconninfoOption *connOptions;
+
+ if (conn == NULL)
+ return NULL;
+
+ /* We don't actually report any errors here, but callees want a buffer */
+ initPQExpBuffer(&errorBuf);
+ if (PQExpBufferDataBroken(errorBuf))
+ return NULL; /* out of memory already :-( */
+
+ connOptions = conninfo_init(&errorBuf, flags);
+
+ if (connOptions != NULL)
+ {
+ const PQconninfoMapping *mapping;
+
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
+ {
+ char **connmember = PGCONNMEMBERADDR(conn, mapping);
+
+ conninfo_storeval(connOptions, mapping->keyword, *connmember,
+ &errorBuf, false, false);
+ }
+ }
+
+ termPQExpBuffer(&errorBuf);
+
+ return connOptions;
+}
+
+
void
PQconninfoFree(PQconninfoOption *connOptions)
{
diff -durpN postgresql/src/interfaces/libpq/libpq-fe.h postgresql.1/src/interfaces/libpq/libpq-fe.h
--- postgresql/src/interfaces/libpq/libpq-fe.h 2012-10-09 09:58:14.343782980 +0200
+++ postgresql.1/src/interfaces/libpq/libpq-fe.h 2012-11-20 11:49:36.747692399 +0100
@@ -36,6 +36,13 @@ extern "C"
#define PG_COPYRES_EVENTS 0x04
#define PG_COPYRES_NOTICEHOOKS 0x08
+/*
+ * Option flags for PQconninfo
+ */
+#define PG_CONNINFO_NORMAL 0x01
+#define PG_CONNINFO_PASSWORD 0x02
+#define PG_CONNINFO_REPLICATION 0x04
+
/* Application-visible enum types */
/*
@@ -262,6 +269,9 @@ extern PQconninfoOption *PQconndefaults(
/* parse connection options in same way as PQconnectdb */
extern PQconninfoOption *PQconninfoParse(const char *conninfo, char **errmsg);
+/* return the connection options used by a live connection */
+extern PQconninfoOption *PQconninfo(PGconn *conn, int flags);
+
/* free the data structure returned by PQconndefaults() or PQconninfoParse() */
extern void PQconninfoFree(PQconninfoOption *connOptions);
02-pg_basebackup-v15.patchtext/x-patch; name=02-pg_basebackup-v15.patchDownload
diff -durpN postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml
--- postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml 2012-11-08 13:13:04.151630632 +0100
+++ postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml 2012-11-20 12:56:32.891843707 +0100
@@ -189,6 +189,21 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--write-recovery-conf</option></term>
+ <listitem>
+
+ <para>
+ Write a minimal recovery.conf into the output directory (or into
+ the base archive file if <option>--format=tar</option> was specified)
+ using the connection parameters from the command line to ease
+ setting up the standby.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-x</option></term>
<term><option>--xlog</option></term>
<listitem>
diff -durpN postgresql.1/src/bin/pg_basebackup/pg_basebackup.c postgresql.2/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.1/src/bin/pg_basebackup/pg_basebackup.c 2012-10-03 10:40:48.297207389 +0200
+++ postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2012-11-20 16:40:52.329063335 +0100
@@ -19,12 +19,14 @@
#define FRONTEND 1
#include "postgres.h"
#include "libpq-fe.h"
+#include "pqexpbuffer.h"
#include <unistd.h>
#include <dirent.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
+#include <time.h>
#ifdef HAVE_LIBZ
#include <zlib.h>
@@ -46,6 +48,7 @@ int compresslevel = 0;
bool includewal = false;
bool streamwal = false;
bool fastcheckpoint = false;
+bool writerecoveryconf = false;
int standby_message_timeout = 10 * 1000; /* 10 sec = default */
/* Progress counters */
@@ -70,6 +73,10 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
+/* Don't ever change this value, the TAR file format requires it. */
+#define TARCHUNKSZ 512
+PQExpBuffer rcExpBuf = NULL;
+
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
@@ -77,6 +84,13 @@ static void progress_report(int tablespa
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
+static char *escape_quotes(const char *src);
+static void CreateRecoveryConf(PGconn *conn);
+static void WriteRecoveryConf(void);
+static int _tarChecksum(char *header);
+static void print_val(char *s, uint64 val, unsigned int base, size_t len);
+static void scan_val(char *s, uint64 *val, unsigned int base, size_t len);
+static void _tarCreateHeader(char *header, char *filename, size_t filesize);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -107,6 +121,8 @@ usage(void)
printf(_("\nOptions controlling the output:\n"));
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
+ printf(_(" -R, --write-recovery-conf\n"
+ " write recovery.conf after backup\n"));
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
printf(_(" -X, --xlog-method=fetch|stream\n"
" include required WAL files with specified method\n"));
@@ -452,6 +468,45 @@ progress_report(int tablespacenum, const
/*
+ * Write a piece of tar data
+ */
+static void
+writeTarData(
+#ifdef HAVE_LIBZ
+ gzFile ztarfile,
+#endif
+ FILE *tarfile, char *buf, int r, char *current_file)
+{
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (gzwrite(ztarfile, buf, r) != r)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, current_file, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (fwrite(buf, r, 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
+ progname, current_file, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ }
+}
+
+#ifdef HAVE_LIBZ
+#define WRITE_TAR_DATA(buf, sz) writeTarData(ztarfile, tarfile, buf, sz, filename)
+#else
+#define WRITE_TAR_DATA(buf, sz) writeTarData(tarfile, buf, sz, filename)
+#endif
+
+/*
* Receive a tar format file from the connection to the server, and write
* the data from this file directly into a tar file. If compression is
* enabled, the data will be compressed while written to the file.
@@ -467,12 +522,17 @@ ReceiveTarFile(PGconn *conn, PGresult *r
char filename[MAXPGPATH];
char *copybuf = NULL;
FILE *tarfile = NULL;
+ char tarhdr[TARCHUNKSZ];
+ bool basetablespace = PQgetisnull(res, rownum, 0);
+ bool in_tarhdr, skip_file;
+ size_t tarhdrsz;
+ uint64 filesz;
#ifdef HAVE_LIBZ
gzFile ztarfile = NULL;
#endif
- if (PQgetisnull(res, rownum, 0))
+ if (basetablespace)
{
/*
* Base tablespaces
@@ -584,6 +644,16 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
+ /*
+ * Initialize our variables for tracking
+ * individual files inside the TAR stream.
+ * For more detailed explanation, see below.
+ */
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = 0;
+ filesz = 0;
+
while (1)
{
int r;
@@ -598,38 +668,36 @@ ReceiveTarFile(PGconn *conn, PGresult *r
if (r == -1)
{
/*
- * End of chunk. Close file (but not stdout).
+ * End of chunk. Write recovery.conf into the tar file (if it
+ * was requested) and close file (but not stdout).
*
* Also, write two completely empty blocks at the end of the tar
* file, as required by some tar programs.
*/
- char zerobuf[1024];
+ char zerobuf[2*TARCHUNKSZ];
MemSet(zerobuf, 0, sizeof(zerobuf));
-#ifdef HAVE_LIBZ
- if (ztarfile != NULL)
- {
- if (gzwrite(ztarfile, zerobuf, sizeof(zerobuf)) !=
- sizeof(zerobuf))
- {
- fprintf(stderr,
- _("%s: could not write to compressed file \"%s\": %s\n"),
- progname, filename, get_gz_error(ztarfile));
- disconnect_and_exit(1);
- }
- }
- else
-#endif
+
+ if (basetablespace && writerecoveryconf)
{
- if (fwrite(zerobuf, sizeof(zerobuf), 1, tarfile) != 1)
- {
- fprintf(stderr,
- _("%s: could not write to file \"%s\": %s\n"),
- progname, filename, strerror(errno));
- disconnect_and_exit(1);
- }
+ char header[TARCHUNKSZ];
+ int padding;
+
+ _tarCreateHeader(header, "recovery.conf", rcExpBuf->len);
+
+ padding = ((rcExpBuf->len + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - rcExpBuf->len;
+
+ WRITE_TAR_DATA(header, sizeof(header));
+ WRITE_TAR_DATA(rcExpBuf->data, rcExpBuf->len);
+ if (padding)
+ WRITE_TAR_DATA(zerobuf, padding);
+
+ if (verbose)
+ fprintf(stderr, _("%s: recovery.conf written into '%s'\n"), progname, filename);
}
+ WRITE_TAR_DATA(zerobuf, sizeof(zerobuf));
+
#ifdef HAVE_LIBZ
if (ztarfile != NULL)
{
@@ -665,25 +733,124 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
-#ifdef HAVE_LIBZ
- if (ztarfile != NULL)
+ if (!writerecoveryconf || !basetablespace)
{
- if (gzwrite(ztarfile, copybuf, r) != r)
- {
- fprintf(stderr,
- _("%s: could not write to compressed file \"%s\": %s\n"),
- progname, filename, get_gz_error(ztarfile));
- disconnect_and_exit(1);
- }
+ /*
+ * If --write-recovery-conf was not requested or this
+ * is not the base tablespace, simply pass the received
+ * data into the TAR file, either compressed or not.
+ */
+
+ WRITE_TAR_DATA(copybuf, r);
}
else
-#endif
{
- if (fwrite(copybuf, r, 1, tarfile) != 1)
+ /*
+ * If --write-recovery-conf was requested AND this
+ * is the base tablespace, the TAR stream may contain
+ * a recovery.conf file if the backup is coming from
+ * a standby server. We have to skip this file in
+ * the stream and add a new one constructed by
+ * CreateRecoveryConf() at the end of the stream.
+ *
+ * To do this, we have to process the individual files
+ * inside the TAR stream. The stream consists of a header
+ * and zero or more chunks, all 512 bytes long. The stream
+ * from the server is broken up into smaller pieces, so
+ * we have to track the size of the files to find the next
+ * header structure.
+ */
+ int rr = r; /* Save the value returned by PQgetCopyData */
+ int pos = 0;
+
+ while (rr > 0)
{
- fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
- progname, filename, strerror(errno));
- disconnect_and_exit(1);
+ if (in_tarhdr)
+ {
+ /*
+ * We're currently reading a header structure
+ * inside the TAR stream, i.e. the file metadata.
+ */
+ if (tarhdrsz < TARCHUNKSZ)
+ {
+ /*
+ * Copy the header structure into tarhdr[]
+ * in case the header is not aligned to 512 bytes
+ * or it's not returned in whole by the last
+ * PQgetCopyData call.
+ */
+ int hdrleft, bytes2copy;
+
+ hdrleft = TARCHUNKSZ - tarhdrsz;
+ bytes2copy = (rr > hdrleft ? hdrleft : rr);
+
+ memcpy(&tarhdr[tarhdrsz], copybuf + pos, bytes2copy);
+
+ rr -= bytes2copy;
+ pos += bytes2copy;
+ tarhdrsz += bytes2copy;
+ }
+ else
+ {
+ /*
+ * We have the whole header structure in tarhdr[],
+ * look at the file metadata:
+ * - the subsequent file contents have to be skipped
+ * if the filename is recovery.conf
+ * - find out the size of the file padded to the next
+ * multiple of 512
+ */
+ int64 padding;
+
+ skip_file = (strcmp(&tarhdr[0], "recovery.conf") == 0);
+
+ scan_val(&tarhdr[124], &filesz, 8, 11);
+
+ padding = ((filesz + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - filesz;
+ filesz += padding;
+
+ /* Indicate that the subsequent data is the file content. */
+ in_tarhdr = false;
+
+ if (!skip_file)
+ WRITE_TAR_DATA(tarhdr, TARCHUNKSZ);
+ }
+ }
+ else
+ {
+ /*
+ * We're processing a file's contents.
+ */
+ if (filesz > 0)
+ {
+ /*
+ * We still have data to read (and possibly write).
+ */
+ int bytes2write;
+
+ bytes2write = (filesz > rr ? rr : filesz);
+
+ if (!skip_file)
+ WRITE_TAR_DATA(copybuf + pos, bytes2write);
+
+ rr -= bytes2write;
+ pos += bytes2write;
+ filesz -= bytes2write;
+ }
+ else
+ {
+ /*
+ * No more data in the current file,
+ * the next piece of data (if any) will
+ * be a new file header structure.
+ * Reinitialize all our variables.
+ */
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = 0;
+ filesz = 0;
+ }
+ }
}
}
totaldone += r;
@@ -712,10 +879,11 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
char filename[MAXPGPATH];
int current_len_left;
int current_padding = 0;
+ bool basetablespace = PQgetisnull(res, rownum, 0);
char *copybuf = NULL;
FILE *file = NULL;
- if (PQgetisnull(res, rownum, 0))
+ if (basetablespace)
strcpy(current_path, basedir);
else
strcpy(current_path, PQgetvalue(res, rownum, 1));
@@ -767,13 +935,13 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
/*
* No current file, so this must be the header for a new file
*/
- if (r != 512)
+ if (r != TARCHUNKSZ)
{
fprintf(stderr, _("%s: invalid tar block header size: %d\n"),
progname, r);
disconnect_and_exit(1);
}
- totaldone += 512;
+ totaldone += TARCHUNKSZ;
if (sscanf(copybuf + 124, "%11o", ¤t_len_left) != 1)
{
@@ -794,7 +962,7 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
* All files are padded up to 512 bytes
*/
current_padding =
- ((current_len_left + 511) & ~511) - current_len_left;
+ ((current_len_left + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - current_len_left;
/*
* First part of header is zero terminated filename
@@ -937,6 +1105,258 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
if (copybuf != NULL)
PQfreemem(copybuf);
+
+ if (basetablespace)
+ WriteRecoveryConf();
+}
+
+static int
+_tarChecksum(char *header)
+{
+ int i,
+ sum;
+
+ /*
+ * Per POSIX, the checksum is the simple sum of all bytes in the header,
+ * treating the bytes as unsigned, and treating the checksum field (at
+ * offset 148) as though it contained 8 spaces.
+ */
+ sum = 8 * ' '; /* presumed value for checksum field */
+ for (i = 0; i < TARCHUNKSZ; i++)
+ if (i < 148 || i >= 156)
+ sum += 0xFF & header[i];
+ return sum;
+}
+
+
+/*
+ * Utility routine to print possibly larger than 32 bit integers in a
+ * portable fashion. Filled with zeros.
+ */
+static void
+print_val(char *s, uint64 val, unsigned int base, size_t len)
+{
+ int i;
+
+ for (i = len; i > 0; i--)
+ {
+ int digit = val % base;
+
+ s[i - 1] = '0' + digit;
+ val = val / base;
+ }
+}
+
+
+/*
+ * Inverse for print_val()
+ */
+static void
+scan_val(char *s, uint64 *val, unsigned int base, size_t len)
+{
+ uint64 tmp = 0;
+ int i;
+
+ for (i = 0; i < len; i++)
+ {
+ int digit = s[i] - '0';
+
+ tmp = tmp * base + digit;
+ }
+
+ *val = tmp;
+}
+
+
+static void
+_tarCreateHeader(char *header, char *filename, size_t filesize)
+{
+ /*
+ * Note: most of the fields in a tar header are not supposed to be
+ * null-terminated. We use sprintf, which will write a null after the
+ * required bytes; that null goes into the first byte of the next field.
+ * This is okay as long as we fill the fields in order.
+ */
+ memset(header, 0, TARCHUNKSZ /* sizeof the tar header */);
+
+ /* Name 100 */
+ sprintf(&header[0], "%.99s", filename);
+
+ /* Mode 8 */
+ sprintf(&header[100], "0000600 ");
+
+ /* User ID 8 */
+ sprintf(&header[108], "0004000 ");
+
+ /* Group 8 */
+ sprintf(&header[116], "0002000 ");
+
+ /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
+ print_val(&header[124], filesize, 8, 11);
+ sprintf(&header[135], " ");
+
+ /* Mod Time 12 */
+ sprintf(&header[136], "%011o ", (int) time(NULL));
+
+ /* Checksum 8 cannot be calculated until we've filled all other fields */
+
+ /* Type - regular file */
+ sprintf(&header[156], "0");
+
+ /* Link Name 100 (leave as nulls) */
+
+ /* Magic 6 */
+ sprintf(&header[257], "ustar");
+
+ /* Version 2 */
+ sprintf(&header[263], "00");
+
+ /* User 32 */
+ /* XXX: Do we need to care about setting correct username? */
+ sprintf(&header[265], "%.31s", "postgres");
+
+ /* Group 32 */
+ /* XXX: Do we need to care about setting correct group name? */
+ sprintf(&header[297], "%.31s", "postgres");
+
+ /* Major Dev 8 */
+ sprintf(&header[329], "%07o ", 0);
+
+ /* Minor Dev 8 */
+ sprintf(&header[337], "%07o ", 0);
+
+ /* Prefix 155 - not used, leave as nulls */
+
+ /*
+ * We mustn't overwrite the next field while inserting the checksum.
+ * Fortunately, the checksum can't exceed 6 octal digits, so we just write
+ * 6 digits, a space, and a null, which is legal per POSIX.
+ */
+ sprintf(&header[148], "%06o ", _tarChecksum(header));
+}
+
+/*
+ * Escape single quotes in a string
+ */
+static char *
+escape_quotes(const char *src)
+{
+ int len = strlen(src),
+ i,
+ j;
+ char *result = pg_malloc(len * 2 + 1);
+
+ for (i = 0, j = 0; i < len; i++)
+ {
+ if (SQL_STR_DOUBLE(src[i], true))
+ result[j++] = src[i];
+ result[j++] = src[i];
+ }
+ result[j] = '\0';
+ return result;
+}
+
+/*
+ * Try to create recovery.conf in memory and set the length to write later.
+ */
+static void
+CreateRecoveryConf(PGconn *conn)
+{
+ PQconninfoOption *connOptions;
+ PQconninfoOption *option;
+
+ if (!writerecoveryconf)
+ return;
+
+ connOptions = PQconninfo(conn, PG_CONNINFO_REPLICATION);
+ if (connOptions == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "standby_mode = 'on'\n");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "primary_conninfo = '");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ for (option = connOptions; option && option->keyword; option++)
+ {
+ char *escaped;
+
+ /*
+ * Do not emit this setting if not set or empty.
+ */
+ if ((option->val == NULL) ||
+ (option->val != NULL && option->val[0] == '\0'))
+ continue;
+
+ /*
+ * Write "keyword='value'" pieces, the value string is escaped
+ * if necessary and doubled single quotes around the value string.
+ */
+ escaped = escape_quotes(option->val);
+
+ appendPQExpBuffer(rcExpBuf, "%s=''%s'' ", option->keyword, escaped);
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ free(escaped);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "'\n");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ PQconninfoFree(connOptions);
+}
+
+
+static void
+WriteRecoveryConf(void)
+{
+ char filename[MAXPGPATH];
+ FILE *cf;
+
+ if (!writerecoveryconf)
+ return;
+
+ sprintf(filename, "%s/recovery.conf", basedir);
+
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("%s: cannot create %s: %s"), progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ if (fwrite(rcExpBuf->data, rcExpBuf->len, 1, cf) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ fclose(cf);
+
+ if (verbose)
+ fprintf(stderr, _("%s: recovery.conf written.\n"), progname);
}
@@ -960,6 +1380,15 @@ BaseBackup(void)
/* Error message already written in GetConnection() */
exit(1);
+ rcExpBuf = createPQExpBuffer();
+ if (!rcExpBuf)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ CreateRecoveryConf(conn);
+
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
@@ -1223,6 +1652,9 @@ BaseBackup(void)
#endif
}
+ /* Free the recovery.conf contents */
+ destroyPQExpBuffer(rcExpBuf);
+
/*
* End of copy data. Final result is already checked inside the loop.
*/
@@ -1243,6 +1675,7 @@ main(int argc, char **argv)
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
+ {"write-recovery-conf", no_argument, NULL, 'R'},
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
{"gzip", no_argument, NULL, 'z'},
@@ -1280,7 +1713,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:xX:l:zZ:c:h:p:U:s:wWvP",
+ while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
long_options, &option_index)) != -1)
{
switch (c)
@@ -1301,6 +1734,9 @@ main(int argc, char **argv)
exit(1);
}
break;
+ case 'R':
+ writerecoveryconf = true;
+ break;
case 'x':
if (includewal)
{
diff -durpN postgresql.1/src/bin/pg_basebackup/streamutil.c postgresql.2/src/bin/pg_basebackup/streamutil.c
--- postgresql.1/src/bin/pg_basebackup/streamutil.c 2012-10-03 10:40:48.298207395 +0200
+++ postgresql.2/src/bin/pg_basebackup/streamutil.c 2012-11-20 14:29:11.815346687 +0100
@@ -44,13 +44,13 @@ pg_strdup(const char *s)
if (!result)
{
fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
+ disconnect_and_exit(1);
}
return result;
}
void *
-pg_malloc0(size_t size)
+pg_malloc(size_t size)
{
void *result;
@@ -61,13 +61,24 @@ pg_malloc0(size_t size)
if (!result)
{
fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
+ disconnect_and_exit(1);
}
MemSet(result, 0, size);
return result;
}
+void *
+pg_malloc0(size_t size)
+{
+ void *tmp;
+
+ tmp = pg_malloc(size);
+ MemSet(tmp, 0, size);
+ return tmp;
+}
+
+
/*
* Connect to the server. Returns a valid PGconn pointer if connected,
* or NULL on non-permanent error. On permanent error, the function will
diff -durpN postgresql.1/src/bin/pg_basebackup/streamutil.h postgresql.2/src/bin/pg_basebackup/streamutil.h
--- postgresql.1/src/bin/pg_basebackup/streamutil.h 2012-10-03 10:40:48.299207401 +0200
+++ postgresql.2/src/bin/pg_basebackup/streamutil.h 2012-11-20 14:24:29.517573163 +0100
@@ -17,6 +17,7 @@ extern PGconn *conn;
extern char *pg_strdup(const char *s);
+extern void *pg_malloc(size_t size);
extern void *pg_malloc0(size_t size);
extern PGconn *GetConnection(void);
2012-11-20 17:03 keltezéssel, Boszormenyi Zoltan írta:
2012-11-18 17:20 keltezéssel, Magnus Hagander írta:
The whole tar writing part of the code needs a lot more comments. It's
entirely unclear what the code does there. Why does the recovery.conf
writing code need to be split up in multiple locations inside
ReceiveTarFile(), for example? It either needs to be simplified, or
explained why it can't be simplified in comments.
I also simplified (the multiple #ifdef blocks are moved out into a function
to make the code shorter) and added comments to this function.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
2012-11-20 17:03 keltezéssel, Boszormenyi Zoltan írta:
2012-11-18 17:20 keltezéssel, Magnus Hagander írta:
Much of the tar stuff is very similar (I haven't looked to see if it's
identical) to the stuff in backend/replication/basebackup.c. Perhaps
we should have a src/port/tarutil.c?I will implement it as a separate patch.
I implemented it, all 3 patches are attached now. Use this patchset,
the previously sent 1st patch had a bug, it called conninit_storeval()
with value == NULL arguments and it crashes on strdup(NULL).
escape_string() - already exists as escape_quotes() in initdb, AFAICT.
We should either move it to src/port/, or at least copy it so it's
exactly the same.A copy of escape_quotes() is now in pg_basebackup.c and is used.
I will also unify the copies of it in a separate patch.
This one is not done yet, but a suggestion on which existing file
it can fit into or for a new src/port/filename is welcome.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
01-PQconninfo-v16.patchtext/x-patch; name=01-PQconninfo-v16.patchDownload
diff -durpN postgresql/doc/src/sgml/libpq.sgml postgresql.1/doc/src/sgml/libpq.sgml
--- postgresql/doc/src/sgml/libpq.sgml 2012-08-03 09:39:30.114266570 +0200
+++ postgresql.1/doc/src/sgml/libpq.sgml 2012-11-20 11:57:41.359859195 +0100
@@ -496,6 +496,49 @@ typedef struct
</listitem>
</varlistentry>
+ <varlistentry id="libpq-pqconninfo">
+ <term><function>PQconninfo</function><indexterm><primary>PQconninfo</></></term>
+ <listitem>
+ <para>
+ Returns the connection options used by a live connection.
+<synopsis>
+/*
+ * Option flags for PQconninfo
+ */
+#define PG_CONNINFO_NORMAL 0x01
+#define PG_CONNINFO_PASSWORD 0x02
+#define PG_CONNINFO_REPLICATION 0x04
+
+PQconninfoOption *PQconninfo(PGconn *conn, int flags);
+</synopsis>
+ </para>
+
+ <para>
+ Returns a connection options array. This can be used to determine
+ all possible <function>PQconnectdb</function> options and their
+ current values that were used to connect to the server. The return
+ value points to an array of <structname>PQconninfoOption</structname>
+ structures, which ends with an entry having a null <structfield>keyword</>
+ pointer. Every notes above for <function>PQconndefaults</function> also apply.
+ An application may present a dialog using the previous settings by:
+<programlisting>
+ PQconninfoOption *options = PQconninfo(conn, PG_CONNINFO_NORMAL | PG_CONNINFO_PASSWORD);
+</programlisting>
+ </para>
+
+ <para>
+ The array returned by the <literal>PG_CONNINFO_REPLICATION</> flag is
+ a subset of <literal>PG_CONNINFO_NORMAL | PG_CONNINFO_PASSWORD</>.
+ This subset excludes some options from the array which are used by the
+ walreceiver module. <application>pg_basebackup</application> uses this
+ pre-filtered list of options to construct <literal>primary_conninfo</>
+ in the automatically generated recovery.conf file.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+
<varlistentry id="libpq-pqconninfoparse">
<term><function>PQconninfoParse</function><indexterm><primary>PQconninfoParse</></></term>
<listitem>
diff -durpN postgresql/src/interfaces/libpq/exports.txt postgresql.1/src/interfaces/libpq/exports.txt
--- postgresql/src/interfaces/libpq/exports.txt 2012-10-09 09:58:14.342782974 +0200
+++ postgresql.1/src/interfaces/libpq/exports.txt 2012-11-20 10:55:03.396145749 +0100
@@ -164,3 +164,4 @@ PQsetSingleRowMode 161
lo_lseek64 162
lo_tell64 163
lo_truncate64 164
+PQconninfo 165
diff -durpN postgresql/src/interfaces/libpq/fe-connect.c postgresql.1/src/interfaces/libpq/fe-connect.c
--- postgresql/src/interfaces/libpq/fe-connect.c 2012-09-09 08:11:09.470401480 +0200
+++ postgresql.1/src/interfaces/libpq/fe-connect.c 2012-11-20 16:25:11.722295729 +0100
@@ -137,6 +137,9 @@ static int ldapServiceLookup(const char
* PQconninfoOptions[] *must* be NULL. In a working copy, non-null "val"
* fields point to malloc'd strings that should be freed when the working
* array is freed (see PQconninfoFree).
+ *
+ * If you add a new connection option to this list, remember to add it to
+ * PQconninfoMappings[] below.
* ----------
*/
static const PQconninfoOption PQconninfoOptions[] = {
@@ -203,16 +206,6 @@ static const PQconninfoOption PQconninfo
{"keepalives_count", NULL, NULL, NULL,
"TCP-Keepalives-Count", "", 10}, /* strlen(INT32_MAX) == 10 */
-#ifdef USE_SSL
-
- /*
- * "requiressl" is deprecated, its purpose having been taken over by
- * "sslmode". It remains for backwards compatibility.
- */
- {"requiressl", "PGREQUIRESSL", "0", NULL,
- "Require-SSL", "D", 1},
-#endif
-
/*
* ssl options are allowed even without client SSL support because the
* client can still handle SSL modes "disable" and "allow". Other
@@ -264,6 +257,83 @@ static const PQconninfoOption PQconninfo
NULL, NULL, 0}
};
+/*
+ * We need a mapping between the PQconninfoOptions[] array
+ * and PGconn members. We have to keep it separate from
+ * PQconninfoOptions[] to not leak info about PGconn members
+ * to clients.
+ */
+typedef struct PQconninfoMapping {
+ char *keyword;
+ size_t member_offset;
+ int flags;
+} PQconninfoMapping;
+#define PGCONNMEMBERADDR(conn, mapping) ((char **)((char *)conn + mapping->member_offset))
+
+static const PQconninfoMapping PQconninfoMappings[] =
+{
+ /* "authtype" is not used anymore, there is no mapping to PGconn */
+ /* there is no mapping of "service" to PGconn */
+ { "user", offsetof(struct pg_conn, pguser),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "password", offsetof(struct pg_conn, pgpass),
+ PG_CONNINFO_PASSWORD | PG_CONNINFO_REPLICATION },
+ { "connect_timeout", offsetof(struct pg_conn, connect_timeout),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "dbname", offsetof(struct pg_conn, dbName),
+ PG_CONNINFO_NORMAL },
+ { "host", offsetof(struct pg_conn, pghost),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "hostaddr", offsetof(struct pg_conn, pghostaddr),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "port", offsetof(struct pg_conn, pgport),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "client_encoding", offsetof(struct pg_conn, client_encoding_initial),
+ PG_CONNINFO_NORMAL },
+ { "tty", offsetof(struct pg_conn, pgtty),
+ PG_CONNINFO_NORMAL },
+ { "options", offsetof(struct pg_conn, pgoptions),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "application_name", offsetof(struct pg_conn, appname),
+ PG_CONNINFO_NORMAL },
+ { "fallback_application_name", offsetof(struct pg_conn, fbappname),
+ PG_CONNINFO_NORMAL },
+ { "keepalives", offsetof(struct pg_conn, keepalives),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "keepalives_idle", offsetof(struct pg_conn, keepalives_idle),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "keepalives_interval", offsetof(struct pg_conn, keepalives_interval),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "keepalives_count", offsetof(struct pg_conn, keepalives_count),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "sslmode", offsetof(struct pg_conn, sslmode),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "sslcompression", offsetof(struct pg_conn, sslcompression),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "sslcert", offsetof(struct pg_conn, sslcert),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "sslkey", offsetof(struct pg_conn, sslkey),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "sslrootcert", offsetof(struct pg_conn, sslrootcert),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "sslcrl", offsetof(struct pg_conn, sslcrl),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+ { "requirepeer", offsetof(struct pg_conn, requirepeer),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
+ { "krbsrvname", offsetof(struct pg_conn, krbsrvname),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+#endif
+#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
+ { "gsslib", offsetof(struct pg_conn, requirepeer),
+ PG_CONNINFO_NORMAL | PG_CONNINFO_REPLICATION },
+#endif
+ { "replication", offsetof(struct pg_conn, replication),
+ PG_CONNINFO_NORMAL },
+ /* Terminating entry --- MUST BE LAST */
+ { NULL, 0, 0 }
+};
+
static const PQEnvironmentOption EnvironmentOptions[] =
{
/* common user-interface settings */
@@ -295,7 +365,8 @@ static PGconn *makeEmptyPGconn(void);
static void fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
static void freePGconn(PGconn *conn);
static void closePGconn(PGconn *conn);
-static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage);
+static PQconninfoOption *conninfo_init(PQExpBuffer errorMessage,
+ int flags);
static PQconninfoOption *parse_connection_string(const char *conninfo,
PQExpBuffer errorMessage, bool use_defaults);
static int uri_prefix_length(const char *connstr);
@@ -627,7 +698,7 @@ PQconnectStart(const char *conninfo)
static void
fillPGconn(PGconn *conn, PQconninfoOption *connOptions)
{
- const char *tmp;
+ const PQconninfoMapping *mapping;
/*
* Move option values into conn structure
@@ -637,72 +708,19 @@ fillPGconn(PGconn *conn, PQconninfoOptio
*
* XXX: probably worth checking strdup() return value here...
*/
- tmp = conninfo_getval(connOptions, "hostaddr");
- conn->pghostaddr = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "host");
- conn->pghost = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "port");
- conn->pgport = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "tty");
- conn->pgtty = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "options");
- conn->pgoptions = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "application_name");
- conn->appname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "fallback_application_name");
- conn->fbappname = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "dbname");
- conn->dbName = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "user");
- conn->pguser = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "password");
- conn->pgpass = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "connect_timeout");
- conn->connect_timeout = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "client_encoding");
- conn->client_encoding_initial = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives");
- conn->keepalives = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_idle");
- conn->keepalives_idle = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_interval");
- conn->keepalives_interval = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "keepalives_count");
- conn->keepalives_count = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslmode");
- conn->sslmode = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcompression");
- conn->sslcompression = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslkey");
- conn->sslkey = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcert");
- conn->sslcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslrootcert");
- conn->sslrootcert = tmp ? strdup(tmp) : NULL;
- tmp = conninfo_getval(connOptions, "sslcrl");
- conn->sslcrl = tmp ? strdup(tmp) : NULL;
-#ifdef USE_SSL
- tmp = conninfo_getval(connOptions, "requiressl");
- if (tmp && tmp[0] == '1')
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
{
- /* here warn that the requiressl option is deprecated? */
- if (conn->sslmode)
- free(conn->sslmode);
- conn->sslmode = strdup("require");
+ const char *tmp = conninfo_getval(connOptions, mapping->keyword);
+
+ if (tmp)
+ {
+ char **connmember = PGCONNMEMBERADDR(conn, mapping);
+
+ if (*connmember)
+ free(*connmember);
+ *connmember = tmp ? strdup(tmp) : NULL;
+ }
}
-#endif
- tmp = conninfo_getval(connOptions, "requirepeer");
- conn->requirepeer = tmp ? strdup(tmp) : NULL;
-#if defined(KRB5) || defined(ENABLE_GSS) || defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "krbsrvname");
- conn->krbsrvname = tmp ? strdup(tmp) : NULL;
-#endif
-#if defined(ENABLE_GSS) && defined(ENABLE_SSPI)
- tmp = conninfo_getval(connOptions, "gsslib");
- conn->gsslib = tmp ? strdup(tmp) : NULL;
-#endif
- tmp = conninfo_getval(connOptions, "replication");
- conn->replication = tmp ? strdup(tmp) : NULL;
}
/*
@@ -884,7 +902,7 @@ PQconndefaults(void)
if (PQExpBufferDataBroken(errorBuf))
return NULL; /* out of memory already :-( */
- connOptions = conninfo_init(&errorBuf);
+ connOptions = conninfo_init(&errorBuf, PG_CONNINFO_NORMAL | PG_CONNINFO_PASSWORD);
if (connOptions != NULL)
{
if (!conninfo_add_defaults(connOptions, &errorBuf))
@@ -4008,9 +4026,11 @@ PQconninfoParse(const char *conninfo, ch
* Build a working copy of the constant PQconninfoOptions array.
*/
static PQconninfoOption *
-conninfo_init(PQExpBuffer errorMessage)
+conninfo_init(PQExpBuffer errorMessage, int flags)
{
+ const PQconninfoMapping *mapping;
PQconninfoOption *options;
+ PQconninfoOption *opt_dest;
options = (PQconninfoOption *) malloc(sizeof(PQconninfoOptions));
if (options == NULL)
@@ -4019,7 +4039,23 @@ conninfo_init(PQExpBuffer errorMessage)
libpq_gettext("out of memory\n"));
return NULL;
}
- memcpy(options, PQconninfoOptions, sizeof(PQconninfoOptions));
+
+ opt_dest = options;
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
+ {
+ PQconninfoOption *opt_src;
+
+ if (!(mapping->flags & flags))
+ continue;
+
+ opt_src = conninfo_find((PQconninfoOption *)PQconninfoOptions, mapping->keyword);
+ if (opt_src)
+ {
+ memcpy(opt_dest, opt_src, sizeof(PQconninfoOption));
+ opt_dest++;
+ }
+ }
+ MemSet(opt_dest, 0, sizeof(PQconninfoOption));
return options;
}
@@ -4095,7 +4131,7 @@ conninfo_parse(const char *conninfo, PQE
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, PG_CONNINFO_NORMAL | PG_CONNINFO_PASSWORD);
if (options == NULL)
return NULL;
@@ -4295,7 +4331,7 @@ conninfo_array_parse(const char *const *
}
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, PG_CONNINFO_NORMAL | PG_CONNINFO_PASSWORD);
if (options == NULL)
{
PQconninfoFree(dbname_options);
@@ -4485,7 +4521,7 @@ conninfo_uri_parse(const char *uri, PQEx
PQconninfoOption *options;
/* Make a working copy of PQconninfoOptions */
- options = conninfo_init(errorMessage);
+ options = conninfo_init(errorMessage, PG_CONNINFO_NORMAL | PG_CONNINFO_PASSWORD);
if (options == NULL)
return NULL;
@@ -5066,6 +5102,45 @@ conninfo_find(PQconninfoOption *connOpti
}
+/*
+ * Return the connection options used for the connections
+ */
+PQconninfoOption *
+PQconninfo(PGconn *conn, int flags)
+{
+ PQExpBufferData errorBuf;
+ PQconninfoOption *connOptions;
+
+ if (conn == NULL)
+ return NULL;
+
+ /* We don't actually report any errors here, but callees want a buffer */
+ initPQExpBuffer(&errorBuf);
+ if (PQExpBufferDataBroken(errorBuf))
+ return NULL; /* out of memory already :-( */
+
+ connOptions = conninfo_init(&errorBuf, flags);
+
+ if (connOptions != NULL)
+ {
+ const PQconninfoMapping *mapping;
+
+ for (mapping = PQconninfoMappings; mapping->keyword; mapping++)
+ {
+ char **connmember = PGCONNMEMBERADDR(conn, mapping);
+
+ if (*connmember)
+ conninfo_storeval(connOptions, mapping->keyword, *connmember,
+ &errorBuf, false, false);
+ }
+ }
+
+ termPQExpBuffer(&errorBuf);
+
+ return connOptions;
+}
+
+
void
PQconninfoFree(PQconninfoOption *connOptions)
{
diff -durpN postgresql/src/interfaces/libpq/libpq-fe.h postgresql.1/src/interfaces/libpq/libpq-fe.h
--- postgresql/src/interfaces/libpq/libpq-fe.h 2012-10-09 09:58:14.343782980 +0200
+++ postgresql.1/src/interfaces/libpq/libpq-fe.h 2012-11-20 11:49:36.747692399 +0100
@@ -36,6 +36,13 @@ extern "C"
#define PG_COPYRES_EVENTS 0x04
#define PG_COPYRES_NOTICEHOOKS 0x08
+/*
+ * Option flags for PQconninfo
+ */
+#define PG_CONNINFO_NORMAL 0x01
+#define PG_CONNINFO_PASSWORD 0x02
+#define PG_CONNINFO_REPLICATION 0x04
+
/* Application-visible enum types */
/*
@@ -262,6 +269,9 @@ extern PQconninfoOption *PQconndefaults(
/* parse connection options in same way as PQconnectdb */
extern PQconninfoOption *PQconninfoParse(const char *conninfo, char **errmsg);
+/* return the connection options used by a live connection */
+extern PQconninfoOption *PQconninfo(PGconn *conn, int flags);
+
/* free the data structure returned by PQconndefaults() or PQconninfoParse() */
extern void PQconninfoFree(PQconninfoOption *connOptions);
02-pg_basebackup-v16.patchtext/x-patch; name=02-pg_basebackup-v16.patchDownload
diff -durpN postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml
--- postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml 2012-11-08 13:13:04.151630632 +0100
+++ postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml 2012-11-20 12:56:32.891843707 +0100
@@ -189,6 +189,21 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--write-recovery-conf</option></term>
+ <listitem>
+
+ <para>
+ Write a minimal recovery.conf into the output directory (or into
+ the base archive file if <option>--format=tar</option> was specified)
+ using the connection parameters from the command line to ease
+ setting up the standby.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-x</option></term>
<term><option>--xlog</option></term>
<listitem>
diff -durpN postgresql.1/src/bin/pg_basebackup/pg_basebackup.c postgresql.2/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.1/src/bin/pg_basebackup/pg_basebackup.c 2012-10-03 10:40:48.297207389 +0200
+++ postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2012-11-20 16:40:52.329063335 +0100
@@ -19,12 +19,14 @@
#define FRONTEND 1
#include "postgres.h"
#include "libpq-fe.h"
+#include "pqexpbuffer.h"
#include <unistd.h>
#include <dirent.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
+#include <time.h>
#ifdef HAVE_LIBZ
#include <zlib.h>
@@ -46,6 +48,7 @@ int compresslevel = 0;
bool includewal = false;
bool streamwal = false;
bool fastcheckpoint = false;
+bool writerecoveryconf = false;
int standby_message_timeout = 10 * 1000; /* 10 sec = default */
/* Progress counters */
@@ -70,6 +73,10 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
+/* Don't ever change this value, the TAR file format requires it. */
+#define TARCHUNKSZ 512
+PQExpBuffer rcExpBuf = NULL;
+
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
@@ -77,6 +84,13 @@ static void progress_report(int tablespa
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
+static char *escape_quotes(const char *src);
+static void CreateRecoveryConf(PGconn *conn);
+static void WriteRecoveryConf(void);
+static int _tarChecksum(char *header);
+static void print_val(char *s, uint64 val, unsigned int base, size_t len);
+static void scan_val(char *s, uint64 *val, unsigned int base, size_t len);
+static void _tarCreateHeader(char *header, char *filename, size_t filesize);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -107,6 +121,8 @@ usage(void)
printf(_("\nOptions controlling the output:\n"));
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
+ printf(_(" -R, --write-recovery-conf\n"
+ " write recovery.conf after backup\n"));
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
printf(_(" -X, --xlog-method=fetch|stream\n"
" include required WAL files with specified method\n"));
@@ -452,6 +468,45 @@ progress_report(int tablespacenum, const
/*
+ * Write a piece of tar data
+ */
+static void
+writeTarData(
+#ifdef HAVE_LIBZ
+ gzFile ztarfile,
+#endif
+ FILE *tarfile, char *buf, int r, char *current_file)
+{
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (gzwrite(ztarfile, buf, r) != r)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, current_file, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (fwrite(buf, r, 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
+ progname, current_file, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ }
+}
+
+#ifdef HAVE_LIBZ
+#define WRITE_TAR_DATA(buf, sz) writeTarData(ztarfile, tarfile, buf, sz, filename)
+#else
+#define WRITE_TAR_DATA(buf, sz) writeTarData(tarfile, buf, sz, filename)
+#endif
+
+/*
* Receive a tar format file from the connection to the server, and write
* the data from this file directly into a tar file. If compression is
* enabled, the data will be compressed while written to the file.
@@ -467,12 +522,17 @@ ReceiveTarFile(PGconn *conn, PGresult *r
char filename[MAXPGPATH];
char *copybuf = NULL;
FILE *tarfile = NULL;
+ char tarhdr[TARCHUNKSZ];
+ bool basetablespace = PQgetisnull(res, rownum, 0);
+ bool in_tarhdr, skip_file;
+ size_t tarhdrsz;
+ uint64 filesz;
#ifdef HAVE_LIBZ
gzFile ztarfile = NULL;
#endif
- if (PQgetisnull(res, rownum, 0))
+ if (basetablespace)
{
/*
* Base tablespaces
@@ -584,6 +644,16 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
+ /*
+ * Initialize our variables for tracking
+ * individual files inside the TAR stream.
+ * For more detailed explanation, see below.
+ */
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = 0;
+ filesz = 0;
+
while (1)
{
int r;
@@ -598,38 +668,36 @@ ReceiveTarFile(PGconn *conn, PGresult *r
if (r == -1)
{
/*
- * End of chunk. Close file (but not stdout).
+ * End of chunk. Write recovery.conf into the tar file (if it
+ * was requested) and close file (but not stdout).
*
* Also, write two completely empty blocks at the end of the tar
* file, as required by some tar programs.
*/
- char zerobuf[1024];
+ char zerobuf[2*TARCHUNKSZ];
MemSet(zerobuf, 0, sizeof(zerobuf));
-#ifdef HAVE_LIBZ
- if (ztarfile != NULL)
- {
- if (gzwrite(ztarfile, zerobuf, sizeof(zerobuf)) !=
- sizeof(zerobuf))
- {
- fprintf(stderr,
- _("%s: could not write to compressed file \"%s\": %s\n"),
- progname, filename, get_gz_error(ztarfile));
- disconnect_and_exit(1);
- }
- }
- else
-#endif
+
+ if (basetablespace && writerecoveryconf)
{
- if (fwrite(zerobuf, sizeof(zerobuf), 1, tarfile) != 1)
- {
- fprintf(stderr,
- _("%s: could not write to file \"%s\": %s\n"),
- progname, filename, strerror(errno));
- disconnect_and_exit(1);
- }
+ char header[TARCHUNKSZ];
+ int padding;
+
+ _tarCreateHeader(header, "recovery.conf", rcExpBuf->len);
+
+ padding = ((rcExpBuf->len + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - rcExpBuf->len;
+
+ WRITE_TAR_DATA(header, sizeof(header));
+ WRITE_TAR_DATA(rcExpBuf->data, rcExpBuf->len);
+ if (padding)
+ WRITE_TAR_DATA(zerobuf, padding);
+
+ if (verbose)
+ fprintf(stderr, _("%s: recovery.conf written into '%s'\n"), progname, filename);
}
+ WRITE_TAR_DATA(zerobuf, sizeof(zerobuf));
+
#ifdef HAVE_LIBZ
if (ztarfile != NULL)
{
@@ -665,25 +733,124 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
-#ifdef HAVE_LIBZ
- if (ztarfile != NULL)
+ if (!writerecoveryconf || !basetablespace)
{
- if (gzwrite(ztarfile, copybuf, r) != r)
- {
- fprintf(stderr,
- _("%s: could not write to compressed file \"%s\": %s\n"),
- progname, filename, get_gz_error(ztarfile));
- disconnect_and_exit(1);
- }
+ /*
+ * If --write-recovery-conf was not requested or this
+ * is not the base tablespace, simply pass the received
+ * data into the TAR file, either compressed or not.
+ */
+
+ WRITE_TAR_DATA(copybuf, r);
}
else
-#endif
{
- if (fwrite(copybuf, r, 1, tarfile) != 1)
+ /*
+ * If --write-recovery-conf was requested AND this
+ * is the base tablespace, the TAR stream may contain
+ * a recovery.conf file if the backup is coming from
+ * a standby server. We have to skip this file in
+ * the stream and add a new one constructed by
+ * CreateRecoveryConf() at the end of the stream.
+ *
+ * To do this, we have to process the individual files
+ * inside the TAR stream. The stream consists of a header
+ * and zero or more chunks, all 512 bytes long. The stream
+ * from the server is broken up into smaller pieces, so
+ * we have to track the size of the files to find the next
+ * header structure.
+ */
+ int rr = r; /* Save the value returned by PQgetCopyData */
+ int pos = 0;
+
+ while (rr > 0)
{
- fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
- progname, filename, strerror(errno));
- disconnect_and_exit(1);
+ if (in_tarhdr)
+ {
+ /*
+ * We're currently reading a header structure
+ * inside the TAR stream, i.e. the file metadata.
+ */
+ if (tarhdrsz < TARCHUNKSZ)
+ {
+ /*
+ * Copy the header structure into tarhdr[]
+ * in case the header is not aligned to 512 bytes
+ * or it's not returned in whole by the last
+ * PQgetCopyData call.
+ */
+ int hdrleft, bytes2copy;
+
+ hdrleft = TARCHUNKSZ - tarhdrsz;
+ bytes2copy = (rr > hdrleft ? hdrleft : rr);
+
+ memcpy(&tarhdr[tarhdrsz], copybuf + pos, bytes2copy);
+
+ rr -= bytes2copy;
+ pos += bytes2copy;
+ tarhdrsz += bytes2copy;
+ }
+ else
+ {
+ /*
+ * We have the whole header structure in tarhdr[],
+ * look at the file metadata:
+ * - the subsequent file contents have to be skipped
+ * if the filename is recovery.conf
+ * - find out the size of the file padded to the next
+ * multiple of 512
+ */
+ int64 padding;
+
+ skip_file = (strcmp(&tarhdr[0], "recovery.conf") == 0);
+
+ scan_val(&tarhdr[124], &filesz, 8, 11);
+
+ padding = ((filesz + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - filesz;
+ filesz += padding;
+
+ /* Indicate that the subsequent data is the file content. */
+ in_tarhdr = false;
+
+ if (!skip_file)
+ WRITE_TAR_DATA(tarhdr, TARCHUNKSZ);
+ }
+ }
+ else
+ {
+ /*
+ * We're processing a file's contents.
+ */
+ if (filesz > 0)
+ {
+ /*
+ * We still have data to read (and possibly write).
+ */
+ int bytes2write;
+
+ bytes2write = (filesz > rr ? rr : filesz);
+
+ if (!skip_file)
+ WRITE_TAR_DATA(copybuf + pos, bytes2write);
+
+ rr -= bytes2write;
+ pos += bytes2write;
+ filesz -= bytes2write;
+ }
+ else
+ {
+ /*
+ * No more data in the current file,
+ * the next piece of data (if any) will
+ * be a new file header structure.
+ * Reinitialize all our variables.
+ */
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = 0;
+ filesz = 0;
+ }
+ }
}
}
totaldone += r;
@@ -712,10 +879,11 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
char filename[MAXPGPATH];
int current_len_left;
int current_padding = 0;
+ bool basetablespace = PQgetisnull(res, rownum, 0);
char *copybuf = NULL;
FILE *file = NULL;
- if (PQgetisnull(res, rownum, 0))
+ if (basetablespace)
strcpy(current_path, basedir);
else
strcpy(current_path, PQgetvalue(res, rownum, 1));
@@ -767,13 +935,13 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
/*
* No current file, so this must be the header for a new file
*/
- if (r != 512)
+ if (r != TARCHUNKSZ)
{
fprintf(stderr, _("%s: invalid tar block header size: %d\n"),
progname, r);
disconnect_and_exit(1);
}
- totaldone += 512;
+ totaldone += TARCHUNKSZ;
if (sscanf(copybuf + 124, "%11o", ¤t_len_left) != 1)
{
@@ -794,7 +962,7 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
* All files are padded up to 512 bytes
*/
current_padding =
- ((current_len_left + 511) & ~511) - current_len_left;
+ ((current_len_left + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - current_len_left;
/*
* First part of header is zero terminated filename
@@ -937,6 +1105,258 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
if (copybuf != NULL)
PQfreemem(copybuf);
+
+ if (basetablespace)
+ WriteRecoveryConf();
+}
+
+static int
+_tarChecksum(char *header)
+{
+ int i,
+ sum;
+
+ /*
+ * Per POSIX, the checksum is the simple sum of all bytes in the header,
+ * treating the bytes as unsigned, and treating the checksum field (at
+ * offset 148) as though it contained 8 spaces.
+ */
+ sum = 8 * ' '; /* presumed value for checksum field */
+ for (i = 0; i < TARCHUNKSZ; i++)
+ if (i < 148 || i >= 156)
+ sum += 0xFF & header[i];
+ return sum;
+}
+
+
+/*
+ * Utility routine to print possibly larger than 32 bit integers in a
+ * portable fashion. Filled with zeros.
+ */
+static void
+print_val(char *s, uint64 val, unsigned int base, size_t len)
+{
+ int i;
+
+ for (i = len; i > 0; i--)
+ {
+ int digit = val % base;
+
+ s[i - 1] = '0' + digit;
+ val = val / base;
+ }
+}
+
+
+/*
+ * Inverse for print_val()
+ */
+static void
+scan_val(char *s, uint64 *val, unsigned int base, size_t len)
+{
+ uint64 tmp = 0;
+ int i;
+
+ for (i = 0; i < len; i++)
+ {
+ int digit = s[i] - '0';
+
+ tmp = tmp * base + digit;
+ }
+
+ *val = tmp;
+}
+
+
+static void
+_tarCreateHeader(char *header, char *filename, size_t filesize)
+{
+ /*
+ * Note: most of the fields in a tar header are not supposed to be
+ * null-terminated. We use sprintf, which will write a null after the
+ * required bytes; that null goes into the first byte of the next field.
+ * This is okay as long as we fill the fields in order.
+ */
+ memset(header, 0, TARCHUNKSZ /* sizeof the tar header */);
+
+ /* Name 100 */
+ sprintf(&header[0], "%.99s", filename);
+
+ /* Mode 8 */
+ sprintf(&header[100], "0000600 ");
+
+ /* User ID 8 */
+ sprintf(&header[108], "0004000 ");
+
+ /* Group 8 */
+ sprintf(&header[116], "0002000 ");
+
+ /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
+ print_val(&header[124], filesize, 8, 11);
+ sprintf(&header[135], " ");
+
+ /* Mod Time 12 */
+ sprintf(&header[136], "%011o ", (int) time(NULL));
+
+ /* Checksum 8 cannot be calculated until we've filled all other fields */
+
+ /* Type - regular file */
+ sprintf(&header[156], "0");
+
+ /* Link Name 100 (leave as nulls) */
+
+ /* Magic 6 */
+ sprintf(&header[257], "ustar");
+
+ /* Version 2 */
+ sprintf(&header[263], "00");
+
+ /* User 32 */
+ /* XXX: Do we need to care about setting correct username? */
+ sprintf(&header[265], "%.31s", "postgres");
+
+ /* Group 32 */
+ /* XXX: Do we need to care about setting correct group name? */
+ sprintf(&header[297], "%.31s", "postgres");
+
+ /* Major Dev 8 */
+ sprintf(&header[329], "%07o ", 0);
+
+ /* Minor Dev 8 */
+ sprintf(&header[337], "%07o ", 0);
+
+ /* Prefix 155 - not used, leave as nulls */
+
+ /*
+ * We mustn't overwrite the next field while inserting the checksum.
+ * Fortunately, the checksum can't exceed 6 octal digits, so we just write
+ * 6 digits, a space, and a null, which is legal per POSIX.
+ */
+ sprintf(&header[148], "%06o ", _tarChecksum(header));
+}
+
+/*
+ * Escape single quotes in a string
+ */
+static char *
+escape_quotes(const char *src)
+{
+ int len = strlen(src),
+ i,
+ j;
+ char *result = pg_malloc(len * 2 + 1);
+
+ for (i = 0, j = 0; i < len; i++)
+ {
+ if (SQL_STR_DOUBLE(src[i], true))
+ result[j++] = src[i];
+ result[j++] = src[i];
+ }
+ result[j] = '\0';
+ return result;
+}
+
+/*
+ * Try to create recovery.conf in memory and set the length to write later.
+ */
+static void
+CreateRecoveryConf(PGconn *conn)
+{
+ PQconninfoOption *connOptions;
+ PQconninfoOption *option;
+
+ if (!writerecoveryconf)
+ return;
+
+ connOptions = PQconninfo(conn, PG_CONNINFO_REPLICATION);
+ if (connOptions == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "standby_mode = 'on'\n");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "primary_conninfo = '");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ for (option = connOptions; option && option->keyword; option++)
+ {
+ char *escaped;
+
+ /*
+ * Do not emit this setting if not set or empty.
+ */
+ if ((option->val == NULL) ||
+ (option->val != NULL && option->val[0] == '\0'))
+ continue;
+
+ /*
+ * Write "keyword='value'" pieces, the value string is escaped
+ * if necessary and doubled single quotes around the value string.
+ */
+ escaped = escape_quotes(option->val);
+
+ appendPQExpBuffer(rcExpBuf, "%s=''%s'' ", option->keyword, escaped);
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ free(escaped);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "'\n");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ PQconninfoFree(connOptions);
+}
+
+
+static void
+WriteRecoveryConf(void)
+{
+ char filename[MAXPGPATH];
+ FILE *cf;
+
+ if (!writerecoveryconf)
+ return;
+
+ sprintf(filename, "%s/recovery.conf", basedir);
+
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("%s: cannot create %s: %s"), progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ if (fwrite(rcExpBuf->data, rcExpBuf->len, 1, cf) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ fclose(cf);
+
+ if (verbose)
+ fprintf(stderr, _("%s: recovery.conf written.\n"), progname);
}
@@ -960,6 +1380,15 @@ BaseBackup(void)
/* Error message already written in GetConnection() */
exit(1);
+ rcExpBuf = createPQExpBuffer();
+ if (!rcExpBuf)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ CreateRecoveryConf(conn);
+
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
@@ -1223,6 +1652,9 @@ BaseBackup(void)
#endif
}
+ /* Free the recovery.conf contents */
+ destroyPQExpBuffer(rcExpBuf);
+
/*
* End of copy data. Final result is already checked inside the loop.
*/
@@ -1243,6 +1675,7 @@ main(int argc, char **argv)
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
+ {"write-recovery-conf", no_argument, NULL, 'R'},
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
{"gzip", no_argument, NULL, 'z'},
@@ -1280,7 +1713,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:xX:l:zZ:c:h:p:U:s:wWvP",
+ while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
long_options, &option_index)) != -1)
{
switch (c)
@@ -1301,6 +1734,9 @@ main(int argc, char **argv)
exit(1);
}
break;
+ case 'R':
+ writerecoveryconf = true;
+ break;
case 'x':
if (includewal)
{
diff -durpN postgresql.1/src/bin/pg_basebackup/streamutil.c postgresql.2/src/bin/pg_basebackup/streamutil.c
--- postgresql.1/src/bin/pg_basebackup/streamutil.c 2012-10-03 10:40:48.298207395 +0200
+++ postgresql.2/src/bin/pg_basebackup/streamutil.c 2012-11-20 14:29:11.815346687 +0100
@@ -44,13 +44,13 @@ pg_strdup(const char *s)
if (!result)
{
fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
+ disconnect_and_exit(1);
}
return result;
}
void *
-pg_malloc0(size_t size)
+pg_malloc(size_t size)
{
void *result;
@@ -61,13 +61,24 @@ pg_malloc0(size_t size)
if (!result)
{
fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
+ disconnect_and_exit(1);
}
MemSet(result, 0, size);
return result;
}
+void *
+pg_malloc0(size_t size)
+{
+ void *tmp;
+
+ tmp = pg_malloc(size);
+ MemSet(tmp, 0, size);
+ return tmp;
+}
+
+
/*
* Connect to the server. Returns a valid PGconn pointer if connected,
* or NULL on non-permanent error. On permanent error, the function will
diff -durpN postgresql.1/src/bin/pg_basebackup/streamutil.h postgresql.2/src/bin/pg_basebackup/streamutil.h
--- postgresql.1/src/bin/pg_basebackup/streamutil.h 2012-10-03 10:40:48.299207401 +0200
+++ postgresql.2/src/bin/pg_basebackup/streamutil.h 2012-11-20 14:24:29.517573163 +0100
@@ -17,6 +17,7 @@ extern PGconn *conn;
extern char *pg_strdup(const char *s);
+extern void *pg_malloc(size_t size);
extern void *pg_malloc0(size_t size);
extern PGconn *GetConnection(void);
03-tarutils.patchtext/x-patch; name=03-tarutils.patchDownload
diff -durpN postgresql.2/src/backend/replication/basebackup.c postgresql.3/src/backend/replication/basebackup.c
--- postgresql.2/src/backend/replication/basebackup.c 2012-10-06 19:25:21.002166521 +0200
+++ postgresql.3/src/backend/replication/basebackup.c 2012-11-20 20:00:14.025523523 +0100
@@ -721,48 +721,12 @@ sendDir(char *path, int basepathlen, boo
/*
- * Utility routine to print possibly larger than 32 bit integers in a
- * portable fashion. Filled with zeros.
- */
-static void
-print_val(char *s, uint64 val, unsigned int base, size_t len)
-{
- int i;
-
- for (i = len; i > 0; i--)
- {
- int digit = val % base;
-
- s[i - 1] = '0' + digit;
- val = val / base;
- }
-}
-
-/*
* Maximum file size for a tar member: The limit inherent in the
* format is 2^33-1 bytes (nearly 8 GB). But we don't want to exceed
* what we can represent in pgoff_t.
*/
#define MAX_TAR_MEMBER_FILELEN (((int64) 1 << Min(33, sizeof(pgoff_t)*8 - 1)) - 1)
-static int
-_tarChecksum(char *header)
-{
- int i,
- sum;
-
- /*
- * Per POSIX, the checksum is the simple sum of all bytes in the header,
- * treating the bytes as unsigned, and treating the checksum field (at
- * offset 148) as though it contained 8 spaces.
- */
- sum = 8 * ' '; /* presumed value for checksum field */
- for (i = 0; i < 512; i++)
- if (i < 148 || i >= 156)
- sum += 0xFF & header[i];
- return sum;
-}
-
/* Given the member, write the TAR header & send the file */
static void
sendFile(char *readfilename, char *tarfilename, struct stat * statbuf)
@@ -840,94 +804,12 @@ _tarWriteHeader(const char *filename, co
{
char h[512];
- /*
- * Note: most of the fields in a tar header are not supposed to be
- * null-terminated. We use sprintf, which will write a null after the
- * required bytes; that null goes into the first byte of the next field.
- * This is okay as long as we fill the fields in order.
- */
- memset(h, 0, sizeof(h));
-
- /* Name 100 */
- sprintf(&h[0], "%.99s", filename);
- if (linktarget != NULL || S_ISDIR(statbuf->st_mode))
- {
- /*
- * We only support symbolic links to directories, and this is
- * indicated in the tar format by adding a slash at the end of the
- * name, the same as for regular directories.
- */
- int flen = strlen(filename);
-
- flen = Min(flen, 99);
- h[flen] = '/';
- h[flen + 1] = '\0';
- }
-
- /* Mode 8 */
- sprintf(&h[100], "%07o ", (int) statbuf->st_mode);
-
- /* User ID 8 */
- sprintf(&h[108], "%07o ", statbuf->st_uid);
-
- /* Group 8 */
- sprintf(&h[116], "%07o ", statbuf->st_gid);
-
- /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
- if (linktarget != NULL || S_ISDIR(statbuf->st_mode))
- /* Symbolic link or directory has size zero */
- print_val(&h[124], 0, 8, 11);
- else
- print_val(&h[124], statbuf->st_size, 8, 11);
- sprintf(&h[135], " ");
-
- /* Mod Time 12 */
- sprintf(&h[136], "%011o ", (int) statbuf->st_mtime);
-
- /* Checksum 8 cannot be calculated until we've filled all other fields */
-
- if (linktarget != NULL)
- {
- /* Type - Symbolic link */
- sprintf(&h[156], "2");
- /* Link Name 100 */
- sprintf(&h[157], "%.99s", linktarget);
- }
- else if (S_ISDIR(statbuf->st_mode))
- /* Type - directory */
- sprintf(&h[156], "5");
- else
- /* Type - regular file */
- sprintf(&h[156], "0");
-
- /* Magic 6 */
- sprintf(&h[257], "ustar");
-
- /* Version 2 */
- sprintf(&h[263], "00");
-
- /* User 32 */
- /* XXX: Do we need to care about setting correct username? */
- sprintf(&h[265], "%.31s", "postgres");
-
- /* Group 32 */
- /* XXX: Do we need to care about setting correct group name? */
- sprintf(&h[297], "%.31s", "postgres");
-
- /* Major Dev 8 */
- sprintf(&h[329], "%07o ", 0);
-
- /* Minor Dev 8 */
- sprintf(&h[337], "%07o ", 0);
-
- /* Prefix 155 - not used, leave as nulls */
-
- /*
- * We mustn't overwrite the next field while inserting the checksum.
- * Fortunately, the checksum can't exceed 6 octal digits, so we just write
- * 6 digits, a space, and a null, which is legal per POSIX.
- */
- sprintf(&h[148], "%06o ", _tarChecksum(h));
+ _tarCreateHeader(h, filename, linktarget,
+ statbuf->st_size,
+ statbuf->st_mode,
+ statbuf->st_uid,
+ statbuf->st_gid,
+ statbuf->st_mtime);
/* Now send the completed header. */
pq_putmessage('d', h, 512);
diff -durpN postgresql.2/src/bin/pg_basebackup/pg_basebackup.c postgresql.3/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2012-11-20 16:40:52.329063335 +0100
+++ postgresql.3/src/bin/pg_basebackup/pg_basebackup.c 2012-11-20 20:18:38.312376749 +0100
@@ -26,7 +26,6 @@
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
-#include <time.h>
#ifdef HAVE_LIBZ
#include <zlib.h>
@@ -73,8 +72,6 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
-/* Don't ever change this value, the TAR file format requires it. */
-#define TARCHUNKSZ 512
PQExpBuffer rcExpBuf = NULL;
/* Function headers */
@@ -87,10 +84,6 @@ static void ReceiveAndUnpackTarFile(PGco
static char *escape_quotes(const char *src);
static void CreateRecoveryConf(PGconn *conn);
static void WriteRecoveryConf(void);
-static int _tarChecksum(char *header);
-static void print_val(char *s, uint64 val, unsigned int base, size_t len);
-static void scan_val(char *s, uint64 *val, unsigned int base, size_t len);
-static void _tarCreateHeader(char *header, char *filename, size_t filesize);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -683,7 +676,10 @@ ReceiveTarFile(PGconn *conn, PGresult *r
char header[TARCHUNKSZ];
int padding;
- _tarCreateHeader(header, "recovery.conf", rcExpBuf->len);
+ _tarCreateHeader(header, "recovery.conf", NULL,
+ rcExpBuf->len,
+ 0600, 04000, 02000,
+ time(NULL));
padding = ((rcExpBuf->len + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - rcExpBuf->len;
@@ -804,7 +800,7 @@ ReceiveTarFile(PGconn *conn, PGresult *r
skip_file = (strcmp(&tarhdr[0], "recovery.conf") == 0);
- scan_val(&tarhdr[124], &filesz, 8, 11);
+ _tarScanVal(&tarhdr[124], &filesz, 8, 11);
padding = ((filesz + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - filesz;
filesz += padding;
@@ -1110,131 +1106,6 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
WriteRecoveryConf();
}
-static int
-_tarChecksum(char *header)
-{
- int i,
- sum;
-
- /*
- * Per POSIX, the checksum is the simple sum of all bytes in the header,
- * treating the bytes as unsigned, and treating the checksum field (at
- * offset 148) as though it contained 8 spaces.
- */
- sum = 8 * ' '; /* presumed value for checksum field */
- for (i = 0; i < TARCHUNKSZ; i++)
- if (i < 148 || i >= 156)
- sum += 0xFF & header[i];
- return sum;
-}
-
-
-/*
- * Utility routine to print possibly larger than 32 bit integers in a
- * portable fashion. Filled with zeros.
- */
-static void
-print_val(char *s, uint64 val, unsigned int base, size_t len)
-{
- int i;
-
- for (i = len; i > 0; i--)
- {
- int digit = val % base;
-
- s[i - 1] = '0' + digit;
- val = val / base;
- }
-}
-
-
-/*
- * Inverse for print_val()
- */
-static void
-scan_val(char *s, uint64 *val, unsigned int base, size_t len)
-{
- uint64 tmp = 0;
- int i;
-
- for (i = 0; i < len; i++)
- {
- int digit = s[i] - '0';
-
- tmp = tmp * base + digit;
- }
-
- *val = tmp;
-}
-
-
-static void
-_tarCreateHeader(char *header, char *filename, size_t filesize)
-{
- /*
- * Note: most of the fields in a tar header are not supposed to be
- * null-terminated. We use sprintf, which will write a null after the
- * required bytes; that null goes into the first byte of the next field.
- * This is okay as long as we fill the fields in order.
- */
- memset(header, 0, TARCHUNKSZ /* sizeof the tar header */);
-
- /* Name 100 */
- sprintf(&header[0], "%.99s", filename);
-
- /* Mode 8 */
- sprintf(&header[100], "0000600 ");
-
- /* User ID 8 */
- sprintf(&header[108], "0004000 ");
-
- /* Group 8 */
- sprintf(&header[116], "0002000 ");
-
- /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
- print_val(&header[124], filesize, 8, 11);
- sprintf(&header[135], " ");
-
- /* Mod Time 12 */
- sprintf(&header[136], "%011o ", (int) time(NULL));
-
- /* Checksum 8 cannot be calculated until we've filled all other fields */
-
- /* Type - regular file */
- sprintf(&header[156], "0");
-
- /* Link Name 100 (leave as nulls) */
-
- /* Magic 6 */
- sprintf(&header[257], "ustar");
-
- /* Version 2 */
- sprintf(&header[263], "00");
-
- /* User 32 */
- /* XXX: Do we need to care about setting correct username? */
- sprintf(&header[265], "%.31s", "postgres");
-
- /* Group 32 */
- /* XXX: Do we need to care about setting correct group name? */
- sprintf(&header[297], "%.31s", "postgres");
-
- /* Major Dev 8 */
- sprintf(&header[329], "%07o ", 0);
-
- /* Minor Dev 8 */
- sprintf(&header[337], "%07o ", 0);
-
- /* Prefix 155 - not used, leave as nulls */
-
- /*
- * We mustn't overwrite the next field while inserting the checksum.
- * Fortunately, the checksum can't exceed 6 octal digits, so we just write
- * 6 digits, a space, and a null, which is legal per POSIX.
- */
- sprintf(&header[148], "%06o ", _tarChecksum(header));
-}
-
/*
* Escape single quotes in a string
*/
diff -durpN postgresql.2/src/bin/pg_dump/pg_backup_tar.c postgresql.3/src/bin/pg_dump/pg_backup_tar.c
--- postgresql.2/src/bin/pg_dump/pg_backup_tar.c 2012-10-03 10:40:48.304207436 +0200
+++ postgresql.3/src/bin/pg_dump/pg_backup_tar.c 2012-11-20 20:16:05.951430114 +0100
@@ -115,7 +115,6 @@ static char *tarGets(char *buf, size_t l
static int tarPrintf(ArchiveHandle *AH, TAR_MEMBER *th, const char *fmt,...) __attribute__((format(PG_PRINTF_ATTRIBUTE, 3, 4)));
static void _tarAddFile(ArchiveHandle *AH, TAR_MEMBER *th);
-static int _tarChecksum(char *th);
static TAR_MEMBER *_tarPositionTo(ArchiveHandle *AH, const char *filename);
static size_t tarRead(void *buf, size_t len, TAR_MEMBER *th);
static size_t tarWrite(const void *buf, size_t len, TAR_MEMBER *th);
@@ -1016,24 +1015,6 @@ tarPrintf(ArchiveHandle *AH, TAR_MEMBER
return cnt;
}
-static int
-_tarChecksum(char *header)
-{
- int i,
- sum;
-
- /*
- * Per POSIX, the checksum is the simple sum of all bytes in the header,
- * treating the bytes as unsigned, and treating the checksum field (at
- * offset 148) as though it contained 8 spaces.
- */
- sum = 8 * ' '; /* presumed value for checksum field */
- for (i = 0; i < 512; i++)
- if (i < 148 || i >= 156)
- sum += 0xFF & header[i];
- return sum;
-}
-
bool
isValidTarHeader(char *header)
{
@@ -1305,94 +1286,16 @@ _tarGetHeader(ArchiveHandle *AH, TAR_MEM
}
-/*
- * Utility routine to print possibly larger than 32 bit integers in a
- * portable fashion. Filled with zeros.
- */
-static void
-print_val(char *s, uint64 val, unsigned int base, size_t len)
-{
- int i;
-
- for (i = len; i > 0; i--)
- {
- int digit = val % base;
-
- s[i - 1] = '0' + digit;
- val = val / base;
- }
-}
-
-
static void
_tarWriteHeader(TAR_MEMBER *th)
{
char h[512];
- /*
- * Note: most of the fields in a tar header are not supposed to be
- * null-terminated. We use sprintf, which will write a null after the
- * required bytes; that null goes into the first byte of the next field.
- * This is okay as long as we fill the fields in order.
- */
- memset(h, 0, sizeof(h));
-
- /* Name 100 */
- sprintf(&h[0], "%.99s", th->targetFile);
-
- /* Mode 8 */
- sprintf(&h[100], "0000600 ");
-
- /* User ID 8 */
- sprintf(&h[108], "0004000 ");
-
- /* Group 8 */
- sprintf(&h[116], "0002000 ");
-
- /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
- print_val(&h[124], th->fileLen, 8, 11);
- sprintf(&h[135], " ");
-
- /* Mod Time 12 */
- sprintf(&h[136], "%011o ", (int) time(NULL));
-
- /* Checksum 8 cannot be calculated until we've filled all other fields */
-
- /* Type - regular file */
- sprintf(&h[156], "0");
-
- /* Link Name 100 (leave as nulls) */
-
- /* Magic 6 */
- sprintf(&h[257], "ustar");
-
- /* Version 2 */
- sprintf(&h[263], "00");
-
- /* User 32 */
- /* XXX: Do we need to care about setting correct username? */
- sprintf(&h[265], "%.31s", "postgres");
-
- /* Group 32 */
- /* XXX: Do we need to care about setting correct group name? */
- sprintf(&h[297], "%.31s", "postgres");
-
- /* Major Dev 8 */
- sprintf(&h[329], "%07o ", 0);
-
- /* Minor Dev 8 */
- sprintf(&h[337], "%07o ", 0);
-
- /* Prefix 155 - not used, leave as nulls */
-
- /*
- * We mustn't overwrite the next field while inserting the checksum.
- * Fortunately, the checksum can't exceed 6 octal digits, so we just write
- * 6 digits, a space, and a null, which is legal per POSIX.
- */
- sprintf(&h[148], "%06o ", _tarChecksum(h));
+ _tarCreateHeader(h, th->targetFile, NULL,
+ th->fileLen,
+ 0600, 04000, 02000,
+ time(NULL));
- /* Now write the completed header. */
if (fwrite(h, 1, 512, th->tarFH) != 512)
exit_horribly(modulename, "could not write to output file: %s\n", strerror(errno));
}
diff -durpN postgresql.2/src/include/port.h postgresql.3/src/include/port.h
--- postgresql.2/src/include/port.h 2012-07-22 16:48:48.548857900 +0200
+++ postgresql.3/src/include/port.h 2012-11-20 19:59:44.745328643 +0100
@@ -16,6 +16,7 @@
#include <ctype.h>
#include <netdb.h>
#include <pwd.h>
+#include <time.h>
/* socket has a different definition on WIN32 */
#ifndef WIN32
@@ -465,4 +466,19 @@ extern int pg_check_dir(const char *dir)
/* port/pgmkdirp.c */
extern int pg_mkdir_p(char *path, int omode);
+/* port/tarutil.c */
+
+/* Don't ever change this value, the TAR file format requires it. */
+#define TARCHUNKSZ 512
+
+extern int _tarChecksum(char *header);
+extern void _tarPrintVal(char *s, uint64 val, unsigned int base, size_t len);
+extern void _tarScanVal(char *s, uint64 *val, unsigned int base, size_t len);
+extern void _tarCreateHeader(char *header,
+ const char *filename,
+ const char *linktarget,
+ int64 filesize, int st_mode,
+ int st_uid, int st_gid,
+ time_t mtime);
+
#endif /* PG_PORT_H */
diff -durpN postgresql.2/src/port/Makefile postgresql.3/src/port/Makefile
--- postgresql.2/src/port/Makefile 2012-05-09 12:28:11.755496856 +0200
+++ postgresql.3/src/port/Makefile 2012-11-20 17:30:06.199236922 +0100
@@ -32,7 +32,7 @@ LIBS += $(PTHREAD_LIBS)
OBJS = $(LIBOBJS) chklocale.o dirmod.o erand48.o exec.o fls.o inet_net_ntop.o \
noblock.o path.o pgcheckdir.o pg_crc.o pgmkdirp.o pgsleep.o \
- pgstrcasecmp.o qsort.o qsort_arg.o sprompt.o thread.o
+ pgstrcasecmp.o qsort.o qsort_arg.o sprompt.o thread.o tarutil.o
# foo_srv.o and foo.o are both built from foo.c, but only foo.o has -DFRONTEND
OBJS_SRV = $(OBJS:%.o=%_srv.o)
diff -durpN postgresql.2/src/port/tarutil.c postgresql.3/src/port/tarutil.c
--- postgresql.2/src/port/tarutil.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.3/src/port/tarutil.c 2012-11-20 19:59:46.832342223 +0100
@@ -0,0 +1,174 @@
+/*-------------------------------------------------------------------------
+ *
+ * tarutil.c
+ * common functions for handling the TAR format
+ *
+ * IDENTIFICATION
+ * src/port/tarutil.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include <sys/stat.h>
+
+int
+_tarChecksum(char *header)
+{
+ int i,
+ sum;
+
+ /*
+ * Per POSIX, the checksum is the simple sum of all bytes in the header,
+ * treating the bytes as unsigned, and treating the checksum field (at
+ * offset 148) as though it contained 8 spaces.
+ */
+ sum = 8 * ' '; /* presumed value for checksum field */
+ for (i = 0; i < TARCHUNKSZ; i++)
+ if (i < 148 || i >= 156)
+ sum += 0xFF & header[i];
+ return sum;
+}
+
+
+/*
+ * Utility routine to print possibly larger than 32 bit integers in a
+ * portable fashion. Filled with zeros.
+ */
+void
+_tarPrintVal(char *s, uint64 val, unsigned int base, size_t len)
+{
+ int i;
+
+ for (i = len; i > 0; i--)
+ {
+ int digit = val % base;
+
+ s[i - 1] = '0' + digit;
+ val = val / base;
+ }
+}
+
+
+/*
+ * Inverse of _tarPrintVal()
+ */
+void
+_tarScanVal(char *s, uint64 *val, unsigned int base, size_t len)
+{
+ uint64 tmp = 0;
+ int i;
+
+ for (i = 0; i < len; i++)
+ {
+ int digit = s[i] - '0';
+
+ tmp = tmp * base + digit;
+ }
+
+ *val = tmp;
+}
+
+
+void
+_tarCreateHeader(char *header, const char *filename,
+ const char *linktarget,
+ int64 filesize, int st_mode,
+ int st_uid, int st_gid,
+ time_t mtime)
+{
+ /*
+ * Note: most of the fields in a tar header are not supposed to be
+ * null-terminated. We use sprintf, which will write a null after the
+ * required bytes; that null goes into the first byte of the next field.
+ * This is okay as long as we fill the fields in order.
+ */
+ memset(header, 0, TARCHUNKSZ /* sizeof the tar header */);
+
+ /* Name 100 */
+ sprintf(&header[0], "%.99s", filename);
+ if (linktarget != NULL || S_ISDIR(st_mode))
+ {
+ /*
+ * We only support symbolic links to directories, and this is
+ * indicated in the tar format by adding a slash at the end of the
+ * name, the same as for regular directories.
+ */
+ int flen = strlen(filename);
+
+ flen = Min(flen, 99);
+ header[flen] = '/';
+ header[flen + 1] = '\0';
+ }
+
+ /* Mode 8 */
+ sprintf(&header[100], "%07o ", st_mode);
+
+ /* User ID 8 */
+ sprintf(&header[108], "%07o ", st_uid);
+
+ /* Group 8 */
+ sprintf(&header[116], "%07o ", st_gid);
+
+ /* File size 12 - 11 digits, 1 space; use _tarPrintVal for 64 bit support */
+ if (linktarget != NULL || S_ISDIR(st_mode))
+ /* Symbolic link or directory has size zero */
+ _tarPrintVal(&header[124], 0, 8, 11);
+ else
+ _tarPrintVal(&header[124], filesize, 8, 11);
+ sprintf(&header[135], " ");
+
+ /* Mod Time 12 - 11 digits, 1 space; use _tarPrintVal for 64 bit support */
+ _tarPrintVal(&header[136], mtime, 8, 11);
+ sprintf(&header[147], " ");
+
+ /* Checksum 8 cannot be calculated until we've filled all other fields */
+
+ if (linktarget != NULL)
+ {
+ /* Type - Symbolic link */
+ sprintf(&header[156], "2");
+ /* Link Name 100 */
+ sprintf(&header[157], "%.99s", linktarget);
+ }
+ else if (S_ISDIR(st_mode))
+ /* Type - directory */
+ sprintf(&header[156], "5");
+ else
+ /* Type - regular file */
+ sprintf(&header[156], "0");
+
+ /* Magic 6 */
+ sprintf(&header[257], "ustar");
+
+ /* Version 2 */
+ sprintf(&header[263], "00");
+
+ /* User 32 */
+ /* XXX: Do we need to care about setting correct username? */
+ sprintf(&header[265], "%.31s", "postgres");
+
+ /* Group 32 */
+ /* XXX: Do we need to care about setting correct group name? */
+ sprintf(&header[297], "%.31s", "postgres");
+
+ /* Major Dev 8 */
+ sprintf(&header[329], "%07o ", 0);
+
+ /* Minor Dev 8 */
+ sprintf(&header[337], "%07o ", 0);
+
+ /* Prefix 155 - not used, leave as nulls */
+
+ /*
+ * We mustn't overwrite the next field while inserting the checksum.
+ * Fortunately, the checksum can't exceed 6 octal digits, so we just write
+ * 6 digits, a space, and a null, which is legal per POSIX.
+ */
+ sprintf(&header[148], "%06o ", _tarChecksum(header));
+}
+
2012-11-20 20:32 keltezéssel, Boszormenyi Zoltan írta:
2012-11-20 17:03 keltezéssel, Boszormenyi Zoltan írta:
2012-11-18 17:20 keltezéssel, Magnus Hagander írta:
Much of the tar stuff is very similar (I haven't looked to see if it's
identical) to the stuff in backend/replication/basebackup.c. Perhaps
we should have a src/port/tarutil.c?I will implement it as a separate patch.
I implemented it, all 3 patches are attached now. Use this patchset,
the previously sent 1st patch had a bug, it called conninit_storeval()
with value == NULL arguments and it crashes on strdup(NULL).escape_string() - already exists as escape_quotes() in initdb, AFAICT.
We should either move it to src/port/, or at least copy it so it's
exactly the same.A copy of escape_quotes() is now in pg_basebackup.c and is used.
I will also unify the copies of it in a separate patch.
This one is not done yet, but a suggestion on which existing file
it can fit into or for a new src/port/filename is welcome.
I experimented with unifying escape_quotes() shortly.
The problem is that it calls pg_malloc() which is an executable-specific
function. Some of the bin/* executables define it as calling exit(1)
when malloc() returns NULL, some call it with exit(EXIT_FAILURE)
which happens to be 1 but still can be different from the constant 1.
Some executables only define pg_malloc0() but not pg_malloc().
pg_basebackup needs pg_malloc() to call disconnect_and_exit(1)
instead to quit cleanly and not leave an "unexpected EOF from client"
message in the server log. Which is a macro at the moment, but
has to be turned into a real function for the reasons below.
To unify escape_quotes(), pg_malloc() need to be also unified.
But the diverse requirements for pg_malloc() from different
executables means that both the escape_quotes() and the pg_malloc()
interface need to be extended with the exit function they have to call
and the argument to pass to the exit function. Unless we don't care
about the bug reports about "unexpected EOF from client" messages.
Frankly, it doesn't worth the effort. Let's just keep the two separate
copies of escape_quotes().
BTW, it seems to me that this "unify even the least used functions"
mentality was not considered to be a great feature during the
introduction of pg_malloc(), which is used in far more code than
the TAR functions.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Boszormenyi Zoltan wrote:
The problem is that it calls pg_malloc() which is an executable-specific
function. Some of the bin/* executables define it as calling exit(1)
when malloc() returns NULL, some call it with exit(EXIT_FAILURE)
which happens to be 1 but still can be different from the constant 1.
Some executables only define pg_malloc0() but not pg_malloc().
It seems simpler to have the escape_quotes utility function in port just
not use pg_malloc at all, have it return NULL or similar failure
indicator when malloc() fails, and then the caller decides what to do.
--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
2012-11-21 14:19 keltezéssel, Alvaro Herrera írta:
Boszormenyi Zoltan wrote:
The problem is that it calls pg_malloc() which is an executable-specific
function. Some of the bin/* executables define it as calling exit(1)
when malloc() returns NULL, some call it with exit(EXIT_FAILURE)
which happens to be 1 but still can be different from the constant 1.
Some executables only define pg_malloc0() but not pg_malloc().It seems simpler to have the escape_quotes utility function in port just
not use pg_malloc at all, have it return NULL or similar failure
indicator when malloc() fails, and then the caller decides what to do.
$ find . -name "*.c" | xargs grep pg_malloc | grep -v "^pg_malloc" | wc -l
277
Too much work for little gain.
Also:
- pg_upgrade calls pg_log(PG_FATAL, ...)
- pgbench logs a line then calls exit(1)
What I can imagine is to introduce a new src/port/ function,
InitPostgresFrontend() or something like that. Every executable
then calls this function first inside their main() it they want to use
pg_malloc() from libpgport.a. This function would accept a pointer
to a structure, and the struct contains the pointer to the exit
function and the argument too call it with. Other data for different
use cases can be added later if needed.
This way, the pg_malloc() and friends can be unified and their call
interfaces don't have to change. InitPostgresFrontend() needs
to be added to only 9 places instead of changing 277 callers of
pg_malloc() or pg_malloc0().
BTW, the unified pg_malloc() and friends must be inside
#ifdef FRONTEND ... #endif to not leak into the backend code.
Opinions?
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Boszormenyi Zoltan wrote:
2012-11-21 14:19 keltezéssel, Alvaro Herrera írta:
Boszormenyi Zoltan wrote:
The problem is that it calls pg_malloc() which is an executable-specific
function. Some of the bin/* executables define it as calling exit(1)
when malloc() returns NULL, some call it with exit(EXIT_FAILURE)
which happens to be 1 but still can be different from the constant 1.
Some executables only define pg_malloc0() but not pg_malloc().It seems simpler to have the escape_quotes utility function in port just
not use pg_malloc at all, have it return NULL or similar failure
indicator when malloc() fails, and then the caller decides what to do.$ find . -name "*.c" | xargs grep pg_malloc | grep -v "^pg_malloc" | wc -l
277Too much work for little gain.
I probably wrote the above in a confusing way. I am not suggesting that
pg_malloc is changed in any way.
--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Boszormenyi Zoltan <zb@cybertec.at> writes:
pg_basebackup needs pg_malloc() to call disconnect_and_exit(1)
instead to quit cleanly and not leave an "unexpected EOF from client"
message in the server log. Which is a macro at the moment, but
has to be turned into a real function for the reasons below.
man 2 atexit
regards, tom lane
2012-11-21 15:29 keltezéssel, Tom Lane írta:
Boszormenyi Zoltan <zb@cybertec.at> writes:
pg_basebackup needs pg_malloc() to call disconnect_and_exit(1)
instead to quit cleanly and not leave an "unexpected EOF from client"
message in the server log. Which is a macro at the moment, but
has to be turned into a real function for the reasons below.man 2 atexit
Aww, crap. I knew I forgot about something. :-)
Thanks.
regards, tom lane
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
2012-11-21 15:29 keltezéssel, Tom Lane írta:
Boszormenyi Zoltan <zb@cybertec.at> writes:
pg_basebackup needs pg_malloc() to call disconnect_and_exit(1)
instead to quit cleanly and not leave an "unexpected EOF from client"
message in the server log. Which is a macro at the moment, but
has to be turned into a real function for the reasons below.man 2 atexit
It doesn't really help to make it easier. E.g: pg_strdup() in psql does this:
char *
pg_strdup(const char *string)
{
char *tmp;
if (!string)
{
psql_error("%s: pg_strdup: cannot duplicate null pointer (internal error)\n",
pset.progname);
exit(EXIT_FAILURE);
}
tmp = strdup(string);
if (!tmp)
{
psql_error("out of memory\n");
exit(EXIT_FAILURE);
}
return tmp;
}
The function passed to atexit() still needs to know which string to print
and this needs an internal variable. Also, the actions taken on pg_malloc/pg_strdup
errors don't produce the exact same messages.
This is why I suggest the attached 03-pg_malloc-unified-v2.patch.
The 04-tarutils-v2.patch unifies the common tar functions and the string functions
like print_val() used by _tarCreateHeader (previously _tarWriteHeader) and
escape_quotes().
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
03-pg_malloc-unified-v2.patchtext/x-patch; name=03-pg_malloc-unified-v2.patchDownload
diff -durpN postgresql.2/contrib/oid2name/oid2name.c postgresql.3/contrib/oid2name/oid2name.c
--- postgresql.2/contrib/oid2name/oid2name.c 2012-10-03 10:40:48.241207023 +0200
+++ postgresql.3/contrib/oid2name/oid2name.c 2012-11-21 18:00:58.906546432 +0100
@@ -50,9 +50,6 @@ struct options
/* function prototypes */
static void help(const char *progname);
void get_opts(int, char **, struct options *);
-void *pg_malloc(size_t size);
-void *pg_realloc(void *ptr, size_t size);
-char *pg_strdup(const char *str);
void add_one_elt(char *eltname, eary *eary);
char *get_comma_elts(eary *eary);
PGconn *sql_conn(struct options *);
@@ -201,53 +198,6 @@ help(const char *progname)
progname, progname);
}
-void *
-pg_malloc(size_t size)
-{
- void *ptr;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- ptr = malloc(size);
- if (!ptr)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return ptr;
-}
-
-void *
-pg_realloc(void *ptr, size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- result = realloc(ptr, size);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-char *
-pg_strdup(const char *str)
-{
- char *result = strdup(str);
-
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
/*
* add_one_elt
*
@@ -598,6 +548,8 @@ main(int argc, char **argv)
struct options *my_opts;
PGconn *pgconn;
+ InitPostgresFrontend(NULL);
+
my_opts = (struct options *) pg_malloc(sizeof(struct options));
my_opts->oids = (eary *) pg_malloc(sizeof(eary));
diff -durpN postgresql.2/contrib/pgbench/pgbench.c postgresql.3/contrib/pgbench/pgbench.c
--- postgresql.2/contrib/pgbench/pgbench.c 2012-10-21 10:56:15.358945296 +0200
+++ postgresql.3/contrib/pgbench/pgbench.c 2012-11-21 18:01:33.795785484 +0100
@@ -296,58 +296,6 @@ static void setalarm(int seconds);
static void *threadRun(void *arg);
-/*
- * routines to check mem allocations and fail noisily.
- */
-static void *
-pg_malloc(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-static void *
-pg_realloc(void *ptr, size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- result = realloc(ptr, size);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-static char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-
static void
usage(void)
{
@@ -1956,6 +1904,8 @@ main(int argc, char **argv)
char val[64];
+ InitPostgresFrontend(NULL);
+
progname = get_progname(argv[0]);
if (argc > 1)
diff -durpN postgresql.2/contrib/pg_upgrade/pg_upgrade.c postgresql.3/contrib/pg_upgrade/pg_upgrade.c
--- postgresql.2/contrib/pg_upgrade/pg_upgrade.c 2012-09-04 11:02:05.935828363 +0200
+++ postgresql.3/contrib/pg_upgrade/pg_upgrade.c 2012-11-21 18:02:43.531265024 +0100
@@ -67,6 +67,21 @@ char *output_files[] = {
NULL
};
+static void
+pg_upgrade_malloc_oom(void)
+{
+ pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
+}
+
+static void
+pg_upgrade_strdup_null(void)
+{
+ pg_log(PG_FATAL, "%s: pg_strdup: cannot duplicate null pointer (internal error)\n", os_info.progname);
+}
+
+static PostgresFrontendData pg_upgrade_fe_data = {
+ pg_upgrade_malloc_oom, pg_upgrade_strdup_null
+};
int
main(int argc, char **argv)
@@ -76,6 +91,8 @@ main(int argc, char **argv)
char *deletion_script_file_name = NULL;
bool live_check = false;
+ InitPostgresFrontend(&pg_upgrade_fe_data);
+
parseCommandLine(argc, argv);
adjust_data_dir(&old_cluster);
diff -durpN postgresql.2/contrib/pg_upgrade/pg_upgrade.h postgresql.3/contrib/pg_upgrade/pg_upgrade.h
--- postgresql.2/contrib/pg_upgrade/pg_upgrade.h 2012-11-15 13:39:47.132060180 +0100
+++ postgresql.3/contrib/pg_upgrade/pg_upgrade.h 2012-11-21 16:54:30.995832875 +0100
@@ -437,10 +437,6 @@ void
prep_status(const char *fmt,...)
__attribute__((format(PG_PRINTF_ATTRIBUTE, 1, 2)));
void check_ok(void);
-char *pg_strdup(const char *s);
-void *pg_malloc(size_t size);
-void *pg_realloc(void *ptr, size_t size);
-void pg_free(void *ptr);
const char *getErrorText(int errNum);
unsigned int str2uint(const char *str);
void pg_putenv(const char *var, const char *val);
diff -durpN postgresql.2/contrib/pg_upgrade/util.c postgresql.3/contrib/pg_upgrade/util.c
--- postgresql.2/contrib/pg_upgrade/util.c 2012-10-03 10:40:48.243207034 +0200
+++ postgresql.3/contrib/pg_upgrade/util.c 2012-11-21 16:52:54.636175667 +0100
@@ -191,55 +191,6 @@ get_user_info(char **user_name)
}
-void *
-pg_malloc(size_t size)
-{
- void *p;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- p = malloc(size);
- if (p == NULL)
- pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
- return p;
-}
-
-void *
-pg_realloc(void *ptr, size_t size)
-{
- void *p;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- p = realloc(ptr, size);
- if (p == NULL)
- pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
- return p;
-}
-
-
-void
-pg_free(void *ptr)
-{
- if (ptr != NULL)
- free(ptr);
-}
-
-
-char *
-pg_strdup(const char *s)
-{
- char *result = strdup(s);
-
- if (result == NULL)
- pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
-
- return result;
-}
-
-
/*
* getErrorText()
*
diff -durpN postgresql.2/src/bin/initdb/initdb.c postgresql.3/src/bin/initdb/initdb.c
--- postgresql.2/src/bin/initdb/initdb.c 2012-10-15 10:44:54.640429361 +0200
+++ postgresql.3/src/bin/initdb/initdb.c 2012-11-21 18:03:54.547749709 +0100
@@ -177,8 +177,6 @@ static const char *backend_options = "--
static char bin_path[MAXPGPATH];
static char backend_exec[MAXPGPATH];
-static void *pg_malloc(size_t size);
-static char *pg_strdup(const char *s);
static char **replace_token(char **lines,
const char *token, const char *replacement);
@@ -284,43 +282,6 @@ do { \
#endif
/*
- * routines to check mem allocations and fail noisily.
- *
- * Note that we can't call exit_nicely() on a memory failure, as it calls
- * rmtree() which needs memory allocation. So we just exit with a bang.
- */
-static void *
-pg_malloc(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-static char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-/*
* make a copy of the array of lines, with token replaced by replacement
* the first time it occurs on each line.
*
@@ -2823,6 +2784,22 @@ check_need_password(const char *authmeth
}
}
+static void
+initdb_malloc_oom(void)
+{
+ fprintf(stderr, _("%s: out of memory\n"), progname);
+}
+
+static void
+initdb_strdup_null(void)
+{
+ fprintf(stderr, _("%s: pg_strdup: cannot duplicate null pointer (internal error)\n"), progname);
+}
+
+static PostgresFrontendData initdb_fe_data = {
+ initdb_malloc_oom, initdb_strdup_null
+};
+
int
main(int argc, char *argv[])
{
@@ -2890,6 +2867,8 @@ main(int argc, char *argv[])
"pg_stat_tmp"
};
+ InitPostgresFrontend(&initdb_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("initdb"));
diff -durpN postgresql.2/src/bin/pg_basebackup/pg_basebackup.c postgresql.3/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2012-11-21 15:14:27.364562199 +0100
+++ postgresql.3/src/bin/pg_basebackup/pg_basebackup.c 2012-11-21 18:05:17.308315854 +0100
@@ -1666,6 +1666,28 @@ BaseBackup(void)
}
+static void
+pgbb_malloc_oom(void)
+{
+ fprintf(stderr, _("%s: out of memory\n"), progname);
+}
+
+static void
+pgbb_strdup_null(void)
+{
+ fprintf(stderr, _("%s: pg_strdup: cannot duplicate null pointer (internal error)\n"), progname);
+}
+
+static void
+pgbb_app_atexit(void)
+{
+ PQfinish(conn);
+}
+
+static PostgresFrontendData pgbb_fe_data = {
+ pgbb_malloc_oom, pgbb_strdup_null, pgbb_app_atexit
+};
+
int
main(int argc, char **argv)
{
@@ -1695,6 +1717,8 @@ main(int argc, char **argv)
int option_index;
+ InitPostgresFrontend(&pgbb_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_basebackup"));
diff -durpN postgresql.2/src/bin/pg_basebackup/streamutil.c postgresql.3/src/bin/pg_basebackup/streamutil.c
--- postgresql.2/src/bin/pg_basebackup/streamutil.c 2012-11-21 15:14:27.365562205 +0100
+++ postgresql.3/src/bin/pg_basebackup/streamutil.c 2012-11-21 16:48:00.810168593 +0100
@@ -32,54 +32,6 @@ static char *dbpassword = NULL;
PGconn *conn = NULL;
/*
- * strdup() and malloc() replacements that print an error and exit
- * if something goes wrong. Can never return NULL.
- */
-char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- disconnect_and_exit(1);
- }
- return result;
-}
-
-void *
-pg_malloc(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- disconnect_and_exit(1);
- }
- MemSet(result, 0, size);
- return result;
-}
-
-
-void *
-pg_malloc0(size_t size)
-{
- void *tmp;
-
- tmp = pg_malloc(size);
- MemSet(tmp, 0, size);
- return tmp;
-}
-
-
-/*
* Connect to the server. Returns a valid PGconn pointer if connected,
* or NULL on non-permanent error. On permanent error, the function will
* call exit(1) directly.
diff -durpN postgresql.2/src/bin/pg_basebackup/streamutil.h postgresql.3/src/bin/pg_basebackup/streamutil.h
--- postgresql.2/src/bin/pg_basebackup/streamutil.h 2012-11-21 15:14:27.365562205 +0100
+++ postgresql.3/src/bin/pg_basebackup/streamutil.h 2012-11-21 16:55:31.796250379 +0100
@@ -16,8 +16,4 @@ extern PGconn *conn;
}
-extern char *pg_strdup(const char *s);
-extern void *pg_malloc(size_t size);
-extern void *pg_malloc0(size_t size);
-
extern PGconn *GetConnection(void);
diff -durpN postgresql.2/src/bin/pg_ctl/pg_ctl.c postgresql.3/src/bin/pg_ctl/pg_ctl.c
--- postgresql.2/src/bin/pg_ctl/pg_ctl.c 2012-10-19 19:12:00.307691010 +0200
+++ postgresql.3/src/bin/pg_ctl/pg_ctl.c 2012-11-21 18:06:04.997642034 +0100
@@ -118,8 +118,6 @@ write_stderr(const char *fmt,...)
/* This extension allows gcc to check the format string for consistency with
the supplied arguments. */
__attribute__((format(PG_PRINTF_ATTRIBUTE, 1, 2)));
-static void *pg_malloc(size_t size);
-static char *pg_strdup(const char *s);
static void do_advice(void);
static void do_help(void);
static void set_mode(char *modeopt);
@@ -226,42 +224,6 @@ write_stderr(const char *fmt,...)
}
/*
- * routines to check memory allocations and fail noisily.
- */
-
-static void *
-pg_malloc(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- write_stderr(_("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-
-static char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- write_stderr(_("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-/*
* Given an already-localized string, print it to stdout unless the
* user has specified that no messages should be printed.
*/
@@ -1994,6 +1956,22 @@ adjust_data_dir(void)
}
+static void
+pgctl_malloc_oom(void)
+{
+ write_stderr(_("%s: out of memory\n"), progname);
+}
+
+static void
+pgctl_strdup_null(void)
+{
+ write_stderr(_("%s: pg_strdup: cannot duplicate null pointer (internal error)\n"), progname);
+}
+
+static PostgresFrontendData pgctl_fe_data = {
+ pgctl_malloc_oom, pgctl_strdup_null
+};
+
int
main(int argc, char **argv)
{
@@ -2013,6 +1991,8 @@ main(int argc, char **argv)
int c;
pgpid_t killproc = 0;
+ InitPostgresFrontend(&pgctl_fe_data);
+
#if defined(WIN32) || defined(__CYGWIN__)
setvbuf(stderr, NULL, _IONBF, 0);
#endif
diff -durpN postgresql.2/src/bin/pg_dump/common.c postgresql.3/src/bin/pg_dump/common.c
--- postgresql.2/src/bin/pg_dump/common.c 2012-07-22 16:48:48.536857822 +0200
+++ postgresql.3/src/bin/pg_dump/common.c 2012-11-21 16:58:49.782639540 +0100
@@ -18,7 +18,6 @@
#include <ctype.h>
#include "catalog/pg_class.h"
-#include "dumpmem.h"
#include "dumputils.h"
diff -durpN postgresql.2/src/bin/pg_dump/compress_io.c postgresql.3/src/bin/pg_dump/compress_io.c
--- postgresql.2/src/bin/pg_dump/compress_io.c 2012-10-03 10:40:48.301207415 +0200
+++ postgresql.3/src/bin/pg_dump/compress_io.c 2012-11-21 17:03:26.135545975 +0100
@@ -53,7 +53,6 @@
*/
#include "compress_io.h"
-#include "dumpmem.h"
#include "dumputils.h"
/*----------------------
diff -durpN postgresql.2/src/bin/pg_dump/dumpmem.c postgresql.3/src/bin/pg_dump/dumpmem.c
--- postgresql.2/src/bin/pg_dump/dumpmem.c 2012-10-03 10:40:48.301207415 +0200
+++ postgresql.3/src/bin/pg_dump/dumpmem.c 1970-01-01 01:00:00.000000000 +0100
@@ -1,76 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * dumpmem.c
- * Memory allocation routines used by pg_dump, pg_dumpall, and pg_restore
- *
- * Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
- *
- *
- * IDENTIFICATION
- * src/bin/pg_dump/dumpmem.c
- *
- *-------------------------------------------------------------------------
- */
-#include "postgres_fe.h"
-
-#include "dumputils.h"
-#include "dumpmem.h"
-
-
-/*
- * Safer versions of some standard C library functions. If an
- * out-of-memory condition occurs, these functions will bail out via exit();
- *therefore, their return value is guaranteed to be non-NULL.
- */
-
-char *
-pg_strdup(const char *string)
-{
- char *tmp;
-
- if (!string)
- exit_horribly(NULL, "cannot duplicate null pointer\n");
- tmp = strdup(string);
- if (!tmp)
- exit_horribly(NULL, "out of memory\n");
- return tmp;
-}
-
-void *
-pg_malloc(size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- tmp = malloc(size);
- if (!tmp)
- exit_horribly(NULL, "out of memory\n");
- return tmp;
-}
-
-void *
-pg_malloc0(size_t size)
-{
- void *tmp;
-
- tmp = pg_malloc(size);
- MemSet(tmp, 0, size);
- return tmp;
-}
-
-void *
-pg_realloc(void *ptr, size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- tmp = realloc(ptr, size);
- if (!tmp)
- exit_horribly(NULL, "out of memory\n");
- return tmp;
-}
diff -durpN postgresql.2/src/bin/pg_dump/dumpmem.h postgresql.3/src/bin/pg_dump/dumpmem.h
--- postgresql.2/src/bin/pg_dump/dumpmem.h 2012-10-03 10:40:48.301207415 +0200
+++ postgresql.3/src/bin/pg_dump/dumpmem.h 1970-01-01 01:00:00.000000000 +0100
@@ -1,22 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * dumpmem.h
- * Memory allocation routines used by pg_dump, pg_dumpall, and pg_restore
- *
- * Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
- *
- * src/bin/pg_dump/dumpmem.h
- *
- *-------------------------------------------------------------------------
- */
-
-#ifndef DUMPMEM_H
-#define DUMPMEM_H
-
-extern char *pg_strdup(const char *string);
-extern void *pg_malloc(size_t size);
-extern void *pg_malloc0(size_t size);
-extern void *pg_realloc(void *ptr, size_t size);
-
-#endif /* DUMPMEM_H */
diff -durpN postgresql.2/src/bin/pg_dump/Makefile postgresql.3/src/bin/pg_dump/Makefile
--- postgresql.2/src/bin/pg_dump/Makefile 2012-04-16 19:57:22.595917312 +0200
+++ postgresql.3/src/bin/pg_dump/Makefile 2012-11-21 17:07:12.746122542 +0100
@@ -20,7 +20,7 @@ override CPPFLAGS := -I$(libpq_srcdir) $
OBJS= pg_backup_archiver.o pg_backup_db.o pg_backup_custom.o \
pg_backup_null.o pg_backup_tar.o \
- pg_backup_directory.o dumpmem.o dumputils.o compress_io.o $(WIN32RES)
+ pg_backup_directory.o dumputils.o compress_io.o $(WIN32RES)
KEYWRDOBJS = keywords.o kwlookup.o
@@ -35,8 +35,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort
pg_restore: pg_restore.o $(OBJS) $(KEYWRDOBJS) | submake-libpq submake-libpgport
$(CC) $(CFLAGS) pg_restore.o $(KEYWRDOBJS) $(OBJS) $(libpq_pgport) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
-pg_dumpall: pg_dumpall.o dumputils.o dumpmem.o $(KEYWRDOBJS) | submake-libpq submake-libpgport
- $(CC) $(CFLAGS) pg_dumpall.o dumputils.o dumpmem.o $(KEYWRDOBJS) $(WIN32RES) $(libpq_pgport) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o $(KEYWRDOBJS) | submake-libpq submake-libpgport
+ $(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(KEYWRDOBJS) $(WIN32RES) $(libpq_pgport) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
install: all installdirs
$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff -durpN postgresql.2/src/bin/pg_dump/nls.mk postgresql.3/src/bin/pg_dump/nls.mk
--- postgresql.2/src/bin/pg_dump/nls.mk 2012-04-16 19:57:22.596917323 +0200
+++ postgresql.3/src/bin/pg_dump/nls.mk 2012-11-21 17:07:46.449355315 +0100
@@ -3,7 +3,7 @@ CATALOG_NAME = pg_dump
AVAIL_LANGUAGES = de es fr it ja ko pt_BR sv tr zh_CN zh_TW
GETTEXT_FILES = pg_backup_archiver.c pg_backup_db.c pg_backup_custom.c \
pg_backup_null.c pg_backup_tar.c \
- pg_backup_directory.c dumpmem.c dumputils.c compress_io.c \
+ pg_backup_directory.c dumputils.c compress_io.c \
pg_dump.c common.c pg_dump_sort.c \
pg_restore.c pg_dumpall.c \
../../port/exec.c
diff -durpN postgresql.2/src/bin/pg_dump/pg_backup_archiver.c postgresql.3/src/bin/pg_dump/pg_backup_archiver.c
--- postgresql.2/src/bin/pg_dump/pg_backup_archiver.c 2012-10-21 10:56:15.398945610 +0200
+++ postgresql.3/src/bin/pg_dump/pg_backup_archiver.c 2012-11-21 17:02:24.913122881 +0100
@@ -21,7 +21,6 @@
*/
#include "pg_backup_db.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <ctype.h>
diff -durpN postgresql.2/src/bin/pg_dump/pg_backup_custom.c postgresql.3/src/bin/pg_dump/pg_backup_custom.c
--- postgresql.2/src/bin/pg_dump/pg_backup_custom.c 2012-10-03 10:40:48.303207429 +0200
+++ postgresql.3/src/bin/pg_dump/pg_backup_custom.c 2012-11-21 16:57:43.446182808 +0100
@@ -26,7 +26,6 @@
#include "compress_io.h"
#include "dumputils.h"
-#include "dumpmem.h"
/*--------
* Routines in the format interface
diff -durpN postgresql.2/src/bin/pg_dump/pg_backup_db.c postgresql.3/src/bin/pg_dump/pg_backup_db.c
--- postgresql.2/src/bin/pg_dump/pg_backup_db.c 2012-07-25 10:51:14.973507544 +0200
+++ postgresql.3/src/bin/pg_dump/pg_backup_db.c 2012-11-21 17:02:38.216215418 +0100
@@ -11,7 +11,6 @@
*/
#include "pg_backup_db.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <unistd.h>
diff -durpN postgresql.2/src/bin/pg_dump/pg_backup_directory.c postgresql.3/src/bin/pg_dump/pg_backup_directory.c
--- postgresql.2/src/bin/pg_dump/pg_backup_directory.c 2012-10-03 10:40:48.303207429 +0200
+++ postgresql.3/src/bin/pg_dump/pg_backup_directory.c 2012-11-21 16:59:26.446892844 +0100
@@ -34,7 +34,6 @@
*/
#include "compress_io.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <dirent.h>
diff -durpN postgresql.2/src/bin/pg_dump/pg_backup_null.c postgresql.3/src/bin/pg_dump/pg_backup_null.c
--- postgresql.2/src/bin/pg_dump/pg_backup_null.c 2012-04-16 19:57:22.598917347 +0200
+++ postgresql.3/src/bin/pg_dump/pg_backup_null.c 2012-11-21 17:00:16.119238575 +0100
@@ -23,7 +23,6 @@
*/
#include "pg_backup_archiver.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <unistd.h> /* for dup */
diff -durpN postgresql.2/src/bin/pg_dump/pg_backup_tar.c postgresql.3/src/bin/pg_dump/pg_backup_tar.c
--- postgresql.2/src/bin/pg_dump/pg_backup_tar.c 2012-10-03 10:40:48.304207436 +0200
+++ postgresql.3/src/bin/pg_dump/pg_backup_tar.c 2012-11-21 16:59:40.614991367 +0100
@@ -28,7 +28,6 @@
#include "pg_backup.h"
#include "pg_backup_archiver.h"
#include "pg_backup_tar.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <sys/stat.h>
diff -durpN postgresql.2/src/bin/pg_dump/pg_dumpall.c postgresql.3/src/bin/pg_dump/pg_dumpall.c
--- postgresql.2/src/bin/pg_dump/pg_dumpall.c 2012-10-14 11:09:08.143146185 +0200
+++ postgresql.3/src/bin/pg_dump/pg_dumpall.c 2012-11-21 17:03:45.879682211 +0100
@@ -23,7 +23,6 @@
#include "getopt_long.h"
#include "dumputils.h"
-#include "dumpmem.h"
#include "pg_backup.h"
/* version string we expect back from pg_dump */
diff -durpN postgresql.2/src/bin/pg_dump/pg_dump.c postgresql.3/src/bin/pg_dump/pg_dump.c
--- postgresql.2/src/bin/pg_dump/pg_dump.c 2012-10-28 07:41:12.051532033 +0100
+++ postgresql.3/src/bin/pg_dump/pg_dump.c 2012-11-21 16:58:32.384518598 +0100
@@ -59,7 +59,6 @@
#include "pg_backup_archiver.h"
#include "pg_backup_db.h"
-#include "dumpmem.h"
#include "dumputils.h"
extern char *optarg;
diff -durpN postgresql.2/src/bin/pg_dump/pg_dump_sort.c postgresql.3/src/bin/pg_dump/pg_dump_sort.c
--- postgresql.2/src/bin/pg_dump/pg_dump_sort.c 2012-10-19 19:12:00.334691199 +0200
+++ postgresql.3/src/bin/pg_dump/pg_dump_sort.c 2012-11-21 16:58:08.053352426 +0100
@@ -15,7 +15,6 @@
*/
#include "pg_backup_archiver.h"
#include "dumputils.h"
-#include "dumpmem.h"
/* translator: this is a module name */
static const char *modulename = gettext_noop("sorter");
diff -durpN postgresql.2/src/bin/pg_dump/pg_restore.c postgresql.3/src/bin/pg_dump/pg_restore.c
--- postgresql.2/src/bin/pg_dump/pg_restore.c 2012-10-14 11:09:08.143146185 +0200
+++ postgresql.3/src/bin/pg_dump/pg_restore.c 2012-11-21 17:03:35.040607345 +0100
@@ -41,7 +41,6 @@
#include "pg_backup_archiver.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <ctype.h>
diff -durpN postgresql.2/src/bin/psql/common.c postgresql.3/src/bin/psql/common.c
--- postgresql.2/src/bin/psql/common.c 2012-10-03 10:40:48.322207553 +0200
+++ postgresql.3/src/bin/psql/common.c 2012-11-21 16:35:09.299927138 +0100
@@ -33,56 +33,6 @@ static bool command_no_begin(const char
static bool is_select_command(const char *query);
/*
- * "Safe" wrapper around strdup()
- */
-char *
-pg_strdup(const char *string)
-{
- char *tmp;
-
- if (!string)
- {
- psql_error("%s: pg_strdup: cannot duplicate null pointer (internal error)\n",
- pset.progname);
- exit(EXIT_FAILURE);
- }
- tmp = strdup(string);
- if (!tmp)
- {
- psql_error("out of memory\n");
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc(size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- tmp = malloc(size);
- if (!tmp)
- {
- psql_error("out of memory\n");
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc0(size_t size)
-{
- void *tmp;
-
- tmp = pg_malloc(size);
- MemSet(tmp, 0, size);
- return tmp;
-}
-
-/*
* setQFout
* -- handler for -o command line option and \o command
*
diff -durpN postgresql.2/src/bin/psql/common.h postgresql.3/src/bin/psql/common.h
--- postgresql.2/src/bin/psql/common.h 2012-10-03 10:40:48.322207553 +0200
+++ postgresql.3/src/bin/psql/common.h 2012-11-21 16:55:06.557076835 +0100
@@ -21,15 +21,6 @@
#define atooid(x) ((Oid) strtoul((x), NULL, 10))
-/*
- * Safer versions of some standard C library functions. If an
- * out-of-memory condition occurs, these functions will bail out
- * safely; therefore, their return value is guaranteed to be non-NULL.
- */
-extern char *pg_strdup(const char *string);
-extern void *pg_malloc(size_t size);
-extern void *pg_malloc0(size_t size);
-
extern bool setQFout(const char *fname);
extern void
diff -durpN postgresql.2/src/bin/psql/startup.c postgresql.3/src/bin/psql/startup.c
--- postgresql.2/src/bin/psql/startup.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.3/src/bin/psql/startup.c 2012-11-21 18:07:48.773361102 +0100
@@ -77,6 +77,21 @@ static void process_psqlrc_file(char *fi
static void showVersion(void);
static void EstablishVariableSpace(void);
+static void psql_malloc_oom(void)
+{
+ psql_error("out of memory\n");
+}
+
+static void psql_strdup_null(void)
+{
+ psql_error("%s: pg_strdup: cannot duplicate null pointer (internal error)\n",
+ pset.progname);
+}
+
+static PostgresFrontendData psql_fe_data = {
+ psql_malloc_oom, psql_strdup_null
+};
+
/*
*
* main
@@ -91,6 +106,8 @@ main(int argc, char *argv[])
char *password_prompt = NULL;
bool new_pass;
+ InitPostgresFrontend(&psql_fe_data);
+
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("psql"));
if (argc > 1)
diff -durpN postgresql.2/src/bin/scripts/clusterdb.c postgresql.3/src/bin/scripts/clusterdb.c
--- postgresql.2/src/bin/scripts/clusterdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.3/src/bin/scripts/clusterdb.c 2012-11-21 18:10:25.326438629 +0100
@@ -61,6 +61,8 @@ main(int argc, char *argv[])
char *table = NULL;
bool verbose = false;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.2/src/bin/scripts/common.c postgresql.3/src/bin/scripts/common.c
--- postgresql.2/src/bin/scripts/common.c 2012-10-03 10:40:48.324207567 +0200
+++ postgresql.3/src/bin/scripts/common.c 2012-11-21 18:10:04.686298291 +0100
@@ -31,6 +31,25 @@ static CRITICAL_SECTION cancelConnLock;
#endif
/*
+ * Helper functions to report error in pg_malloc and pg_strdup
+ */
+static void
+common_malloc_oom(void)
+{
+ fprintf(stderr, _("out of memory\n"));
+}
+
+static void
+common_strdup_null(void)
+{
+ fprintf(stderr, _("pg_strdup: cannot duplicate null pointer (internal error)\n"));
+}
+
+PostgresFrontendData common_fe_data = {
+ common_malloc_oom, common_strdup_null
+};
+
+/*
* Returns the current user name.
*/
const char *
@@ -278,55 +297,6 @@ executeMaintenanceCommand(PGconn *conn,
}
/*
- * "Safe" wrapper around strdup(). Pulled from psql/common.c
- */
-char *
-pg_strdup(const char *string)
-{
- char *tmp;
-
- if (!string)
- {
- fprintf(stderr, _("pg_strdup: cannot duplicate null pointer (internal error)\n"));
- exit(EXIT_FAILURE);
- }
- tmp = strdup(string);
- if (!tmp)
- {
- fprintf(stderr, _("out of memory\n"));
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc(size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- tmp = malloc(size);
- if (!tmp)
- {
- fprintf(stderr, _("out of memory\n"));
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc0(size_t size)
-{
- void *tmp;
-
- tmp = pg_malloc(size);
- MemSet(tmp, 0, size);
- return tmp;
-}
-
-/*
* Check yes/no answer in a localized way. 1=yes, 0=no, -1=neither.
*/
diff -durpN postgresql.2/src/bin/scripts/common.h postgresql.3/src/bin/scripts/common.h
--- postgresql.2/src/bin/scripts/common.h 2012-10-03 10:40:48.324207567 +0200
+++ postgresql.3/src/bin/scripts/common.h 2012-11-21 18:09:57.334247576 +0100
@@ -22,6 +22,8 @@ enum trivalue
typedef void (*help_handler) (const char *progname);
+extern PostgresFrontendData common_fe_data;
+
extern const char *get_user_name(const char *progname);
extern void handle_help_version_opts(int argc, char *argv[],
@@ -50,8 +52,4 @@ extern bool yesno_prompt(const char *que
extern void setup_cancel_handler(void);
-extern char *pg_strdup(const char *string);
-extern void *pg_malloc(size_t size);
-extern void *pg_malloc0(size_t size);
-
#endif /* COMMON_H */
diff -durpN postgresql.2/src/bin/scripts/createdb.c postgresql.3/src/bin/scripts/createdb.c
--- postgresql.2/src/bin/scripts/createdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.3/src/bin/scripts/createdb.c 2012-11-21 18:10:30.790475639 +0100
@@ -64,6 +64,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.2/src/bin/scripts/createlang.c postgresql.3/src/bin/scripts/createlang.c
--- postgresql.2/src/bin/scripts/createlang.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.3/src/bin/scripts/createlang.c 2012-11-21 18:10:34.070498052 +0100
@@ -52,6 +52,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.2/src/bin/scripts/createuser.c postgresql.3/src/bin/scripts/createuser.c
--- postgresql.2/src/bin/scripts/createuser.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.3/src/bin/scripts/createuser.c 2012-11-21 18:10:37.638522326 +0100
@@ -78,6 +78,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.2/src/bin/scripts/dropdb.c postgresql.3/src/bin/scripts/dropdb.c
--- postgresql.2/src/bin/scripts/dropdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.3/src/bin/scripts/dropdb.c 2012-11-21 18:10:41.590549265 +0100
@@ -54,6 +54,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.2/src/bin/scripts/droplang.c postgresql.3/src/bin/scripts/droplang.c
--- postgresql.2/src/bin/scripts/droplang.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.3/src/bin/scripts/droplang.c 2012-11-21 18:10:44.726570630 +0100
@@ -51,6 +51,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.2/src/bin/scripts/dropuser.c postgresql.3/src/bin/scripts/dropuser.c
--- postgresql.2/src/bin/scripts/dropuser.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.3/src/bin/scripts/dropuser.c 2012-11-21 18:10:48.110593633 +0100
@@ -52,6 +52,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.2/src/bin/scripts/reindexdb.c postgresql.3/src/bin/scripts/reindexdb.c
--- postgresql.2/src/bin/scripts/reindexdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.3/src/bin/scripts/reindexdb.c 2012-11-21 18:10:50.870612385 +0100
@@ -67,6 +67,8 @@ main(int argc, char *argv[])
const char *table = NULL;
const char *index = NULL;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.2/src/bin/scripts/vacuumdb.c postgresql.3/src/bin/scripts/vacuumdb.c
--- postgresql.2/src/bin/scripts/vacuumdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.3/src/bin/scripts/vacuumdb.c 2012-11-21 18:10:54.166634846 +0100
@@ -72,6 +72,8 @@ main(int argc, char *argv[])
bool full = false;
bool verbose = false;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.2/src/include/port.h postgresql.3/src/include/port.h
--- postgresql.2/src/include/port.h 2012-07-22 16:48:48.548857900 +0200
+++ postgresql.3/src/include/port.h 2012-11-21 18:02:18.860095054 +0100
@@ -17,6 +17,19 @@
#include <netdb.h>
#include <pwd.h>
+/* port/init.c */
+typedef void (*fe_func)(void);
+
+typedef struct PostgresFrontendData
+{
+ fe_func pg_malloc_oom;
+ fe_func pg_strdup_null;
+ fe_func app_atexit;
+} PostgresFrontendData, *PostgresFrontend;
+extern PostgresFrontend pg_fedata;
+
+extern void InitPostgresFrontend(PostgresFrontend fe_data);
+
/* socket has a different definition on WIN32 */
#ifndef WIN32
typedef int pgsocket;
@@ -462,6 +475,19 @@ extern char *inet_net_ntop(int af, const
/* port/pgcheckdir.c */
extern int pg_check_dir(const char *dir);
+/* port/pgmalloc.c */
+extern void pgmalloc_atexit(void);
+
+extern char *pg_strdup(const char *string);
+
+extern void *pg_malloc(size_t size);
+
+extern void *pg_malloc0(size_t size);
+
+extern void *pg_realloc(void *ptr, size_t size);
+
+extern void pg_free(void *ptr);
+
/* port/pgmkdirp.c */
extern int pg_mkdir_p(char *path, int omode);
diff -durpN postgresql.2/src/port/init.c postgresql.3/src/port/init.c
--- postgresql.2/src/port/init.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.3/src/port/init.c 2012-11-21 16:31:50.370535889 +0100
@@ -0,0 +1,32 @@
+/*-------------------------------------------------------------------------
+ *
+ * init.c
+ * Functions for initializing frontend functions
+ *
+ *
+ * Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/init.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef FRONTEND
+#include "postgres_fe.h"
+
+PostgresFrontend pg_fedata = NULL;
+
+void InitPostgresFrontend(PostgresFrontend ptr)
+{
+ pg_fedata = ptr;
+
+ atexit(pgmalloc_atexit);
+
+ if (ptr && ptr->app_atexit)
+ atexit(ptr->app_atexit);
+}
+
+#endif
diff -durpN postgresql.2/src/port/Makefile postgresql.3/src/port/Makefile
--- postgresql.2/src/port/Makefile 2012-05-09 12:28:11.755496856 +0200
+++ postgresql.3/src/port/Makefile 2012-11-21 16:18:29.251969612 +0100
@@ -30,9 +30,10 @@ include $(top_builddir)/src/Makefile.glo
override CPPFLAGS := -I$(top_builddir)/src/port -DFRONTEND $(CPPFLAGS)
LIBS += $(PTHREAD_LIBS)
-OBJS = $(LIBOBJS) chklocale.o dirmod.o erand48.o exec.o fls.o inet_net_ntop.o \
- noblock.o path.o pgcheckdir.o pg_crc.o pgmkdirp.o pgsleep.o \
- pgstrcasecmp.o qsort.o qsort_arg.o sprompt.o thread.o
+OBJS = $(LIBOBJS) chklocale.o dirmod.o erand48.o exec.o fls.o init.o \
+ inet_net_ntop.o noblock.o path.o pgcheckdir.o pg_crc.o pgmalloc.o \
+ pgmkdirp.o pgsleep.o pgstrcasecmp.o qsort.o qsort_arg.o sprompt.o \
+ thread.o
# foo_srv.o and foo.o are both built from foo.c, but only foo.o has -DFRONTEND
OBJS_SRV = $(OBJS:%.o=%_srv.o)
diff -durpN postgresql.2/src/port/pgmalloc.c postgresql.3/src/port/pgmalloc.c
--- postgresql.2/src/port/pgmalloc.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.3/src/port/pgmalloc.c 2012-11-21 17:59:52.107088562 +0100
@@ -0,0 +1,125 @@
+/*-------------------------------------------------------------------------
+ *
+ * pgmalloc.c
+ * Functions for allocating memory and
+ * exiting on out-of-memory
+ *
+ *
+ * Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pgmalloc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef FRONTEND
+#include "postgres_fe.h"
+
+#include <stdio.h>
+
+typedef enum {
+ PG_MALLOC_NOP,
+ PG_MALLOC_OOM,
+ PG_STRDUP_NULL
+} PgMallocAction;
+
+static PgMallocAction pgmalloc_action = PG_MALLOC_NOP;
+
+void pgmalloc_atexit(void)
+{
+ switch (pgmalloc_action)
+ {
+ case PG_MALLOC_OOM:
+ if (pg_fedata && pg_fedata->pg_malloc_oom)
+ pg_fedata->pg_malloc_oom();
+ else
+ fprintf(stderr, "out of memory\n");
+ break;
+ case PG_STRDUP_NULL:
+ if (pg_fedata && pg_fedata->pg_strdup_null)
+ pg_fedata->pg_strdup_null();
+ else
+ fprintf(stderr, "pg_strdup: cannot duplicate null pointer (internal error)\n");
+ break;
+ case PG_MALLOC_NOP:
+ break;
+ }
+}
+
+/*
+ * "Safe" wrapper around strdup()
+ */
+char *
+pg_strdup(const char *string)
+{
+ char *tmp;
+
+ if (!string)
+ {
+ pgmalloc_action = PG_STRDUP_NULL;
+ exit(1);
+ }
+ tmp = strdup(string);
+ if (!tmp)
+ {
+ pgmalloc_action = PG_MALLOC_OOM;
+ exit(1);
+ }
+ return tmp;
+}
+
+void *
+pg_malloc(size_t size)
+{
+ void *tmp;
+
+ /* Avoid unportable behavior of malloc(0) */
+ if (size == 0)
+ size = 1;
+ tmp = malloc(size);
+ if (!tmp)
+ {
+ pgmalloc_action = PG_MALLOC_OOM;
+ exit(1);
+ }
+ return tmp;
+}
+
+void *
+pg_malloc0(size_t size)
+{
+ void *tmp;
+
+ tmp = pg_malloc(size);
+ MemSet(tmp, 0, size);
+ return tmp;
+}
+
+void *
+pg_realloc(void *ptr, size_t size)
+{
+ void *tmp;
+
+ /* Avoid unportable behavior of realloc(NULL, 0) */
+ if (ptr == NULL && size == 0)
+ size = 1;
+ tmp = realloc(ptr, size);
+ if (!tmp)
+ {
+ pgmalloc_action = PG_MALLOC_OOM;
+ exit(1);
+ }
+ return tmp;
+}
+
+void
+pg_free(void *ptr)
+{
+ if (ptr != NULL)
+ free(ptr);
+}
+
+#endif
04-tarutils-v2.patchtext/x-patch; name=04-tarutils-v2.patchDownload
diff -durpN postgresql.3/src/backend/replication/basebackup.c postgresql.4/src/backend/replication/basebackup.c
--- postgresql.3/src/backend/replication/basebackup.c 2012-10-06 19:25:21.002166521 +0200
+++ postgresql.4/src/backend/replication/basebackup.c 2012-11-21 18:15:21.764511960 +0100
@@ -721,48 +721,12 @@ sendDir(char *path, int basepathlen, boo
/*
- * Utility routine to print possibly larger than 32 bit integers in a
- * portable fashion. Filled with zeros.
- */
-static void
-print_val(char *s, uint64 val, unsigned int base, size_t len)
-{
- int i;
-
- for (i = len; i > 0; i--)
- {
- int digit = val % base;
-
- s[i - 1] = '0' + digit;
- val = val / base;
- }
-}
-
-/*
* Maximum file size for a tar member: The limit inherent in the
* format is 2^33-1 bytes (nearly 8 GB). But we don't want to exceed
* what we can represent in pgoff_t.
*/
#define MAX_TAR_MEMBER_FILELEN (((int64) 1 << Min(33, sizeof(pgoff_t)*8 - 1)) - 1)
-static int
-_tarChecksum(char *header)
-{
- int i,
- sum;
-
- /*
- * Per POSIX, the checksum is the simple sum of all bytes in the header,
- * treating the bytes as unsigned, and treating the checksum field (at
- * offset 148) as though it contained 8 spaces.
- */
- sum = 8 * ' '; /* presumed value for checksum field */
- for (i = 0; i < 512; i++)
- if (i < 148 || i >= 156)
- sum += 0xFF & header[i];
- return sum;
-}
-
/* Given the member, write the TAR header & send the file */
static void
sendFile(char *readfilename, char *tarfilename, struct stat * statbuf)
@@ -840,94 +804,12 @@ _tarWriteHeader(const char *filename, co
{
char h[512];
- /*
- * Note: most of the fields in a tar header are not supposed to be
- * null-terminated. We use sprintf, which will write a null after the
- * required bytes; that null goes into the first byte of the next field.
- * This is okay as long as we fill the fields in order.
- */
- memset(h, 0, sizeof(h));
-
- /* Name 100 */
- sprintf(&h[0], "%.99s", filename);
- if (linktarget != NULL || S_ISDIR(statbuf->st_mode))
- {
- /*
- * We only support symbolic links to directories, and this is
- * indicated in the tar format by adding a slash at the end of the
- * name, the same as for regular directories.
- */
- int flen = strlen(filename);
-
- flen = Min(flen, 99);
- h[flen] = '/';
- h[flen + 1] = '\0';
- }
-
- /* Mode 8 */
- sprintf(&h[100], "%07o ", (int) statbuf->st_mode);
-
- /* User ID 8 */
- sprintf(&h[108], "%07o ", statbuf->st_uid);
-
- /* Group 8 */
- sprintf(&h[116], "%07o ", statbuf->st_gid);
-
- /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
- if (linktarget != NULL || S_ISDIR(statbuf->st_mode))
- /* Symbolic link or directory has size zero */
- print_val(&h[124], 0, 8, 11);
- else
- print_val(&h[124], statbuf->st_size, 8, 11);
- sprintf(&h[135], " ");
-
- /* Mod Time 12 */
- sprintf(&h[136], "%011o ", (int) statbuf->st_mtime);
-
- /* Checksum 8 cannot be calculated until we've filled all other fields */
-
- if (linktarget != NULL)
- {
- /* Type - Symbolic link */
- sprintf(&h[156], "2");
- /* Link Name 100 */
- sprintf(&h[157], "%.99s", linktarget);
- }
- else if (S_ISDIR(statbuf->st_mode))
- /* Type - directory */
- sprintf(&h[156], "5");
- else
- /* Type - regular file */
- sprintf(&h[156], "0");
-
- /* Magic 6 */
- sprintf(&h[257], "ustar");
-
- /* Version 2 */
- sprintf(&h[263], "00");
-
- /* User 32 */
- /* XXX: Do we need to care about setting correct username? */
- sprintf(&h[265], "%.31s", "postgres");
-
- /* Group 32 */
- /* XXX: Do we need to care about setting correct group name? */
- sprintf(&h[297], "%.31s", "postgres");
-
- /* Major Dev 8 */
- sprintf(&h[329], "%07o ", 0);
-
- /* Minor Dev 8 */
- sprintf(&h[337], "%07o ", 0);
-
- /* Prefix 155 - not used, leave as nulls */
-
- /*
- * We mustn't overwrite the next field while inserting the checksum.
- * Fortunately, the checksum can't exceed 6 octal digits, so we just write
- * 6 digits, a space, and a null, which is legal per POSIX.
- */
- sprintf(&h[148], "%06o ", _tarChecksum(h));
+ _tarCreateHeader(h, filename, linktarget,
+ statbuf->st_size,
+ statbuf->st_mode,
+ statbuf->st_uid,
+ statbuf->st_gid,
+ statbuf->st_mtime);
/* Now send the completed header. */
pq_putmessage('d', h, 512);
diff -durpN postgresql.3/src/bin/initdb/initdb.c postgresql.4/src/bin/initdb/initdb.c
--- postgresql.3/src/bin/initdb/initdb.c 2012-11-21 18:03:54.547749709 +0100
+++ postgresql.4/src/bin/initdb/initdb.c 2012-11-21 18:27:34.223704650 +0100
@@ -218,7 +218,6 @@ static void make_postgres(void);
static void perform_fsync(void);
static void trapsig(int signum);
static void check_ok(void);
-static char *escape_quotes(const char *src);
static int locale_date_order(const char *locale);
static bool check_locale_name(int category, const char *locale,
char **canonname);
@@ -2342,35 +2341,6 @@ check_ok(void)
}
}
-/*
- * Escape (by doubling) any single quotes or backslashes in given string
- *
- * Note: this is used to process both postgresql.conf entries and SQL
- * string literals. Since postgresql.conf strings are defined to treat
- * backslashes as escapes, we have to double backslashes here. Hence,
- * when using this for a SQL string literal, use E'' syntax.
- *
- * We do not need to worry about encoding considerations because all
- * valid backend encodings are ASCII-safe.
- */
-static char *
-escape_quotes(const char *src)
-{
- int len = strlen(src),
- i,
- j;
- char *result = pg_malloc(len * 2 + 1);
-
- for (i = 0, j = 0; i < len; i++)
- {
- if (SQL_STR_DOUBLE(src[i], true))
- result[j++] = src[i];
- result[j++] = src[i];
- }
- result[j] = '\0';
- return result;
-}
-
/* Hack to suppress a warning about %x from some versions of gcc */
static inline size_t
my_strftime(char *s, size_t max, const char *fmt, const struct tm * tm)
diff -durpN postgresql.3/src/bin/pg_basebackup/pg_basebackup.c postgresql.4/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.3/src/bin/pg_basebackup/pg_basebackup.c 2012-11-21 18:05:17.308315854 +0100
+++ postgresql.4/src/bin/pg_basebackup/pg_basebackup.c 2012-11-21 18:28:02.647904139 +0100
@@ -26,7 +26,6 @@
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
-#include <time.h>
#ifdef HAVE_LIBZ
#include <zlib.h>
@@ -73,8 +72,6 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
-/* Don't ever change this value, the TAR file format requires it. */
-#define TARCHUNKSZ 512
PQExpBuffer rcExpBuf = NULL;
/* Function headers */
@@ -84,13 +81,8 @@ static void progress_report(int tablespa
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
-static char *escape_quotes(const char *src);
static void CreateRecoveryConf(PGconn *conn);
static void WriteRecoveryConf(void);
-static int _tarChecksum(char *header);
-static void print_val(char *s, uint64 val, unsigned int base, size_t len);
-static void scan_val(char *s, uint64 *val, unsigned int base, size_t len);
-static void _tarCreateHeader(char *header, char *filename, size_t filesize);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -683,7 +675,10 @@ ReceiveTarFile(PGconn *conn, PGresult *r
char header[TARCHUNKSZ];
int padding;
- _tarCreateHeader(header, "recovery.conf", rcExpBuf->len);
+ _tarCreateHeader(header, "recovery.conf", NULL,
+ rcExpBuf->len,
+ 0600, 04000, 02000,
+ time(NULL));
padding = ((rcExpBuf->len + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - rcExpBuf->len;
@@ -1110,152 +1105,6 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
WriteRecoveryConf();
}
-static int
-_tarChecksum(char *header)
-{
- int i,
- sum;
-
- /*
- * Per POSIX, the checksum is the simple sum of all bytes in the header,
- * treating the bytes as unsigned, and treating the checksum field (at
- * offset 148) as though it contained 8 spaces.
- */
- sum = 8 * ' '; /* presumed value for checksum field */
- for (i = 0; i < TARCHUNKSZ; i++)
- if (i < 148 || i >= 156)
- sum += 0xFF & header[i];
- return sum;
-}
-
-
-/*
- * Utility routine to print possibly larger than 32 bit integers in a
- * portable fashion. Filled with zeros.
- */
-static void
-print_val(char *s, uint64 val, unsigned int base, size_t len)
-{
- int i;
-
- for (i = len; i > 0; i--)
- {
- int digit = val % base;
-
- s[i - 1] = '0' + digit;
- val = val / base;
- }
-}
-
-
-/*
- * Inverse for print_val()
- */
-static void
-scan_val(char *s, uint64 *val, unsigned int base, size_t len)
-{
- uint64 tmp = 0;
- int i;
-
- for (i = 0; i < len; i++)
- {
- int digit = s[i] - '0';
-
- tmp = tmp * base + digit;
- }
-
- *val = tmp;
-}
-
-
-static void
-_tarCreateHeader(char *header, char *filename, size_t filesize)
-{
- /*
- * Note: most of the fields in a tar header are not supposed to be
- * null-terminated. We use sprintf, which will write a null after the
- * required bytes; that null goes into the first byte of the next field.
- * This is okay as long as we fill the fields in order.
- */
- memset(header, 0, TARCHUNKSZ /* sizeof the tar header */);
-
- /* Name 100 */
- sprintf(&header[0], "%.99s", filename);
-
- /* Mode 8 */
- sprintf(&header[100], "0000600 ");
-
- /* User ID 8 */
- sprintf(&header[108], "0004000 ");
-
- /* Group 8 */
- sprintf(&header[116], "0002000 ");
-
- /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
- print_val(&header[124], filesize, 8, 11);
- sprintf(&header[135], " ");
-
- /* Mod Time 12 */
- sprintf(&header[136], "%011o ", (int) time(NULL));
-
- /* Checksum 8 cannot be calculated until we've filled all other fields */
-
- /* Type - regular file */
- sprintf(&header[156], "0");
-
- /* Link Name 100 (leave as nulls) */
-
- /* Magic 6 */
- sprintf(&header[257], "ustar");
-
- /* Version 2 */
- sprintf(&header[263], "00");
-
- /* User 32 */
- /* XXX: Do we need to care about setting correct username? */
- sprintf(&header[265], "%.31s", "postgres");
-
- /* Group 32 */
- /* XXX: Do we need to care about setting correct group name? */
- sprintf(&header[297], "%.31s", "postgres");
-
- /* Major Dev 8 */
- sprintf(&header[329], "%07o ", 0);
-
- /* Minor Dev 8 */
- sprintf(&header[337], "%07o ", 0);
-
- /* Prefix 155 - not used, leave as nulls */
-
- /*
- * We mustn't overwrite the next field while inserting the checksum.
- * Fortunately, the checksum can't exceed 6 octal digits, so we just write
- * 6 digits, a space, and a null, which is legal per POSIX.
- */
- sprintf(&header[148], "%06o ", _tarChecksum(header));
-}
-
-/*
- * Escape single quotes in a string
- */
-static char *
-escape_quotes(const char *src)
-{
- int len = strlen(src),
- i,
- j;
- char *result = pg_malloc(len * 2 + 1);
-
- for (i = 0, j = 0; i < len; i++)
- {
- if (SQL_STR_DOUBLE(src[i], true))
- result[j++] = src[i];
- result[j++] = src[i];
- }
- result[j] = '\0';
- return result;
-}
-
/*
* Try to create recovery.conf in memory and set the length to write later.
*/
diff -durpN postgresql.3/src/bin/pg_dump/pg_backup_tar.c postgresql.4/src/bin/pg_dump/pg_backup_tar.c
--- postgresql.3/src/bin/pg_dump/pg_backup_tar.c 2012-11-21 16:59:40.614991367 +0100
+++ postgresql.4/src/bin/pg_dump/pg_backup_tar.c 2012-11-21 18:15:21.767511985 +0100
@@ -114,7 +114,6 @@ static char *tarGets(char *buf, size_t l
static int tarPrintf(ArchiveHandle *AH, TAR_MEMBER *th, const char *fmt,...) __attribute__((format(PG_PRINTF_ATTRIBUTE, 3, 4)));
static void _tarAddFile(ArchiveHandle *AH, TAR_MEMBER *th);
-static int _tarChecksum(char *th);
static TAR_MEMBER *_tarPositionTo(ArchiveHandle *AH, const char *filename);
static size_t tarRead(void *buf, size_t len, TAR_MEMBER *th);
static size_t tarWrite(const void *buf, size_t len, TAR_MEMBER *th);
@@ -1015,24 +1014,6 @@ tarPrintf(ArchiveHandle *AH, TAR_MEMBER
return cnt;
}
-static int
-_tarChecksum(char *header)
-{
- int i,
- sum;
-
- /*
- * Per POSIX, the checksum is the simple sum of all bytes in the header,
- * treating the bytes as unsigned, and treating the checksum field (at
- * offset 148) as though it contained 8 spaces.
- */
- sum = 8 * ' '; /* presumed value for checksum field */
- for (i = 0; i < 512; i++)
- if (i < 148 || i >= 156)
- sum += 0xFF & header[i];
- return sum;
-}
-
bool
isValidTarHeader(char *header)
{
@@ -1304,94 +1285,16 @@ _tarGetHeader(ArchiveHandle *AH, TAR_MEM
}
-/*
- * Utility routine to print possibly larger than 32 bit integers in a
- * portable fashion. Filled with zeros.
- */
-static void
-print_val(char *s, uint64 val, unsigned int base, size_t len)
-{
- int i;
-
- for (i = len; i > 0; i--)
- {
- int digit = val % base;
-
- s[i - 1] = '0' + digit;
- val = val / base;
- }
-}
-
-
static void
_tarWriteHeader(TAR_MEMBER *th)
{
char h[512];
- /*
- * Note: most of the fields in a tar header are not supposed to be
- * null-terminated. We use sprintf, which will write a null after the
- * required bytes; that null goes into the first byte of the next field.
- * This is okay as long as we fill the fields in order.
- */
- memset(h, 0, sizeof(h));
-
- /* Name 100 */
- sprintf(&h[0], "%.99s", th->targetFile);
-
- /* Mode 8 */
- sprintf(&h[100], "0000600 ");
-
- /* User ID 8 */
- sprintf(&h[108], "0004000 ");
-
- /* Group 8 */
- sprintf(&h[116], "0002000 ");
-
- /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
- print_val(&h[124], th->fileLen, 8, 11);
- sprintf(&h[135], " ");
-
- /* Mod Time 12 */
- sprintf(&h[136], "%011o ", (int) time(NULL));
-
- /* Checksum 8 cannot be calculated until we've filled all other fields */
-
- /* Type - regular file */
- sprintf(&h[156], "0");
-
- /* Link Name 100 (leave as nulls) */
-
- /* Magic 6 */
- sprintf(&h[257], "ustar");
-
- /* Version 2 */
- sprintf(&h[263], "00");
-
- /* User 32 */
- /* XXX: Do we need to care about setting correct username? */
- sprintf(&h[265], "%.31s", "postgres");
-
- /* Group 32 */
- /* XXX: Do we need to care about setting correct group name? */
- sprintf(&h[297], "%.31s", "postgres");
-
- /* Major Dev 8 */
- sprintf(&h[329], "%07o ", 0);
-
- /* Minor Dev 8 */
- sprintf(&h[337], "%07o ", 0);
-
- /* Prefix 155 - not used, leave as nulls */
-
- /*
- * We mustn't overwrite the next field while inserting the checksum.
- * Fortunately, the checksum can't exceed 6 octal digits, so we just write
- * 6 digits, a space, and a null, which is legal per POSIX.
- */
- sprintf(&h[148], "%06o ", _tarChecksum(h));
+ _tarCreateHeader(h, th->targetFile, NULL,
+ th->fileLen,
+ 0600, 04000, 02000,
+ time(NULL));
- /* Now write the completed header. */
if (fwrite(h, 1, 512, th->tarFH) != 512)
exit_horribly(modulename, "could not write to output file: %s\n", strerror(errno));
}
diff -durpN postgresql.3/src/include/port.h postgresql.4/src/include/port.h
--- postgresql.3/src/include/port.h 2012-11-21 18:02:18.860095054 +0100
+++ postgresql.4/src/include/port.h 2012-11-21 18:27:22.006619619 +0100
@@ -16,6 +16,7 @@
#include <ctype.h>
#include <netdb.h>
#include <pwd.h>
+#include <time.h>
/* port/init.c */
typedef void (*fe_func)(void);
@@ -491,4 +492,22 @@ extern void pg_free(void *ptr);
/* port/pgmkdirp.c */
extern int pg_mkdir_p(char *path, int omode);
+/* port/strutil.c */
+extern void print_val(char *s, uint64 val, unsigned int base, size_t len);
+extern void scan_val(char *s, uint64 *val, unsigned int base, size_t len);
+extern char *escape_quotes(const char *src);
+
+/* port/tarutil.c */
+
+/* Don't ever change this value, the TAR file format requires it. */
+#define TARCHUNKSZ 512
+
+extern int _tarChecksum(char *header);
+extern void _tarCreateHeader(char *header,
+ const char *filename,
+ const char *linktarget,
+ int64 filesize, int st_mode,
+ int st_uid, int st_gid,
+ time_t mtime);
+
#endif /* PG_PORT_H */
diff -durpN postgresql.3/src/port/Makefile postgresql.4/src/port/Makefile
--- postgresql.3/src/port/Makefile 2012-11-21 16:18:29.251969612 +0100
+++ postgresql.4/src/port/Makefile 2012-11-21 18:18:11.002685027 +0100
@@ -33,7 +33,7 @@ LIBS += $(PTHREAD_LIBS)
OBJS = $(LIBOBJS) chklocale.o dirmod.o erand48.o exec.o fls.o init.o \
inet_net_ntop.o noblock.o path.o pgcheckdir.o pg_crc.o pgmalloc.o \
pgmkdirp.o pgsleep.o pgstrcasecmp.o qsort.o qsort_arg.o sprompt.o \
- thread.o
+ strutil.o tarutil.o thread.o
# foo_srv.o and foo.o are both built from foo.c, but only foo.o has -DFRONTEND
OBJS_SRV = $(OBJS:%.o=%_srv.o)
diff -durpN postgresql.3/src/port/strutil.c postgresql.4/src/port/strutil.c
--- postgresql.3/src/port/strutil.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.4/src/port/strutil.c 2012-11-21 18:33:28.833248501 +0100
@@ -0,0 +1,87 @@
+/*-------------------------------------------------------------------------
+ *
+ * strutil.c
+ * common string functions
+ *
+ * IDENTIFICATION
+ * src/port/strutil.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FRONTEND
+#include "postgres.h"
+#include "utils/palloc.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+/*
+ * Utility routine to print possibly larger than 32 bit integers in a
+ * portable fashion. Filled with zeros.
+ */
+void
+print_val(char *s, uint64 val, unsigned int base, size_t len)
+{
+ int i;
+
+ for (i = len; i > 0; i--)
+ {
+ int digit = val % base;
+
+ s[i - 1] = '0' + digit;
+ val = val / base;
+ }
+}
+
+
+/*
+ * Inverse of print_val()
+ */
+void
+scan_val(char *s, uint64 *val, unsigned int base, size_t len)
+{
+ uint64 tmp = 0;
+ int i;
+
+ for (i = 0; i < len; i++)
+ {
+ int digit = s[i] - '0';
+
+ tmp = tmp * base + digit;
+ }
+
+ *val = tmp;
+}
+
+/*
+ * Escape (by doubling) any single quotes or backslashes in given string
+ *
+ * Note: this is used to process both postgresql.conf entries and SQL
+ * string literals. Since postgresql.conf strings are defined to treat
+ * backslashes as escapes, we have to double backslashes here. Hence,
+ * when using this for a SQL string literal, use E'' syntax.
+ *
+ * We do not need to worry about encoding considerations because all
+ * valid backend encodings are ASCII-safe.
+ */
+char *
+escape_quotes(const char *src)
+{
+ int len = strlen(src),
+ i,
+ j;
+#ifdef FRONTEND
+ char *result = pg_malloc(len * 2 + 1);
+#else
+ char *result = palloc(len * 2 + 1);
+#endif
+
+ for (i = 0, j = 0; i < len; i++)
+ {
+ if (SQL_STR_DOUBLE(src[i], true))
+ result[j++] = src[i];
+ result[j++] = src[i];
+ }
+ result[j] = '\0';
+ return result;
+}
diff -durpN postgresql.3/src/port/tarutil.c postgresql.4/src/port/tarutil.c
--- postgresql.3/src/port/tarutil.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.4/src/port/tarutil.c 2012-11-21 18:16:25.537953568 +0100
@@ -0,0 +1,135 @@
+/*-------------------------------------------------------------------------
+ *
+ * tarutil.c
+ * common functions for handling the TAR format
+ *
+ * IDENTIFICATION
+ * src/port/tarutil.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include <sys/stat.h>
+
+int
+_tarChecksum(char *header)
+{
+ int i,
+ sum;
+
+ /*
+ * Per POSIX, the checksum is the simple sum of all bytes in the header,
+ * treating the bytes as unsigned, and treating the checksum field (at
+ * offset 148) as though it contained 8 spaces.
+ */
+ sum = 8 * ' '; /* presumed value for checksum field */
+ for (i = 0; i < TARCHUNKSZ; i++)
+ if (i < 148 || i >= 156)
+ sum += 0xFF & header[i];
+ return sum;
+}
+
+
+void
+_tarCreateHeader(char *header, const char *filename,
+ const char *linktarget,
+ int64 filesize, int st_mode,
+ int st_uid, int st_gid,
+ time_t mtime)
+{
+ /*
+ * Note: most of the fields in a tar header are not supposed to be
+ * null-terminated. We use sprintf, which will write a null after the
+ * required bytes; that null goes into the first byte of the next field.
+ * This is okay as long as we fill the fields in order.
+ */
+ memset(header, 0, TARCHUNKSZ /* sizeof the tar header */);
+
+ /* Name 100 */
+ sprintf(&header[0], "%.99s", filename);
+ if (linktarget != NULL || S_ISDIR(st_mode))
+ {
+ /*
+ * We only support symbolic links to directories, and this is
+ * indicated in the tar format by adding a slash at the end of the
+ * name, the same as for regular directories.
+ */
+ int flen = strlen(filename);
+
+ flen = Min(flen, 99);
+ header[flen] = '/';
+ header[flen + 1] = '\0';
+ }
+
+ /* Mode 8 */
+ sprintf(&header[100], "%07o ", st_mode);
+
+ /* User ID 8 */
+ sprintf(&header[108], "%07o ", st_uid);
+
+ /* Group 8 */
+ sprintf(&header[116], "%07o ", st_gid);
+
+ /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
+ if (linktarget != NULL || S_ISDIR(st_mode))
+ /* Symbolic link or directory has size zero */
+ print_val(&header[124], 0, 8, 11);
+ else
+ print_val(&header[124], filesize, 8, 11);
+ sprintf(&header[135], " ");
+
+ /* Mod Time 12 - 11 digits, 1 space; use print_val for 64 bit support */
+ print_val(&header[136], mtime, 8, 11);
+ sprintf(&header[147], " ");
+
+ /* Checksum 8 cannot be calculated until we've filled all other fields */
+
+ if (linktarget != NULL)
+ {
+ /* Type - Symbolic link */
+ sprintf(&header[156], "2");
+ /* Link Name 100 */
+ sprintf(&header[157], "%.99s", linktarget);
+ }
+ else if (S_ISDIR(st_mode))
+ /* Type - directory */
+ sprintf(&header[156], "5");
+ else
+ /* Type - regular file */
+ sprintf(&header[156], "0");
+
+ /* Magic 6 */
+ sprintf(&header[257], "ustar");
+
+ /* Version 2 */
+ sprintf(&header[263], "00");
+
+ /* User 32 */
+ /* XXX: Do we need to care about setting correct username? */
+ sprintf(&header[265], "%.31s", "postgres");
+
+ /* Group 32 */
+ /* XXX: Do we need to care about setting correct group name? */
+ sprintf(&header[297], "%.31s", "postgres");
+
+ /* Major Dev 8 */
+ sprintf(&header[329], "%07o ", 0);
+
+ /* Minor Dev 8 */
+ sprintf(&header[337], "%07o ", 0);
+
+ /* Prefix 155 - not used, leave as nulls */
+
+ /*
+ * We mustn't overwrite the next field while inserting the checksum.
+ * Fortunately, the checksum can't exceed 6 octal digits, so we just write
+ * 6 digits, a space, and a null, which is legal per POSIX.
+ */
+ sprintf(&header[148], "%06o ", _tarChecksum(header));
+}
+
Hi,
2012-11-21 18:39 keltezéssel, Boszormenyi Zoltan írta:
2012-11-21 15:29 keltezéssel, Tom Lane írta:
Boszormenyi Zoltan <zb@cybertec.at> writes:
pg_basebackup needs pg_malloc() to call disconnect_and_exit(1)
instead to quit cleanly and not leave an "unexpected EOF from client"
message in the server log. Which is a macro at the moment, but
has to be turned into a real function for the reasons below.man 2 atexit
It doesn't really help to make it easier. E.g: pg_strdup() in psql does this:
char *
pg_strdup(const char *string)
{
char *tmp;if (!string)
{
psql_error("%s: pg_strdup: cannot duplicate null pointer (internal
error)\n",
pset.progname);
exit(EXIT_FAILURE);
}
tmp = strdup(string);
if (!tmp)
{
psql_error("out of memory\n");
exit(EXIT_FAILURE);
}
return tmp;
}The function passed to atexit() still needs to know which string to print
and this needs an internal variable. Also, the actions taken on pg_malloc/pg_strdup
errors don't produce the exact same messages.This is why I suggest the attached 03-pg_malloc-unified-v2.patch.
The 04-tarutils-v2.patch unifies the common tar functions and the string functions
like print_val() used by _tarCreateHeader (previously _tarWriteHeader) and
escape_quotes().
The PQconninfo patch grew its own mail thread (this is why
there's no 1st patch in this series now) and I reworked the order
of the patches to make it more logical. This made the pg_basebackup
patch 5K smaller.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
02-pg_malloc-unified-v3.patchtext/x-patch; name=02-pg_malloc-unified-v3.patchDownload
diff -durpN postgresql.1/contrib/oid2name/oid2name.c postgresql.2/contrib/oid2name/oid2name.c
--- postgresql.1/contrib/oid2name/oid2name.c 2012-10-03 10:40:48.241207023 +0200
+++ postgresql.2/contrib/oid2name/oid2name.c 2012-11-22 10:16:53.670767439 +0100
@@ -50,9 +50,6 @@ struct options
/* function prototypes */
static void help(const char *progname);
void get_opts(int, char **, struct options *);
-void *pg_malloc(size_t size);
-void *pg_realloc(void *ptr, size_t size);
-char *pg_strdup(const char *str);
void add_one_elt(char *eltname, eary *eary);
char *get_comma_elts(eary *eary);
PGconn *sql_conn(struct options *);
@@ -201,53 +198,6 @@ help(const char *progname)
progname, progname);
}
-void *
-pg_malloc(size_t size)
-{
- void *ptr;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- ptr = malloc(size);
- if (!ptr)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return ptr;
-}
-
-void *
-pg_realloc(void *ptr, size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- result = realloc(ptr, size);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-char *
-pg_strdup(const char *str)
-{
- char *result = strdup(str);
-
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
/*
* add_one_elt
*
@@ -598,6 +548,8 @@ main(int argc, char **argv)
struct options *my_opts;
PGconn *pgconn;
+ InitPostgresFrontend(NULL);
+
my_opts = (struct options *) pg_malloc(sizeof(struct options));
my_opts->oids = (eary *) pg_malloc(sizeof(eary));
diff -durpN postgresql.1/contrib/pgbench/pgbench.c postgresql.2/contrib/pgbench/pgbench.c
--- postgresql.1/contrib/pgbench/pgbench.c 2012-10-21 10:56:15.358945296 +0200
+++ postgresql.2/contrib/pgbench/pgbench.c 2012-11-22 10:16:53.672767451 +0100
@@ -296,58 +296,6 @@ static void setalarm(int seconds);
static void *threadRun(void *arg);
-/*
- * routines to check mem allocations and fail noisily.
- */
-static void *
-pg_malloc(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-static void *
-pg_realloc(void *ptr, size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- result = realloc(ptr, size);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-static char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-
static void
usage(void)
{
@@ -1956,6 +1904,8 @@ main(int argc, char **argv)
char val[64];
+ InitPostgresFrontend(NULL);
+
progname = get_progname(argv[0]);
if (argc > 1)
diff -durpN postgresql.1/contrib/pg_upgrade/pg_upgrade.c postgresql.2/contrib/pg_upgrade/pg_upgrade.c
--- postgresql.1/contrib/pg_upgrade/pg_upgrade.c 2012-09-04 11:02:05.935828363 +0200
+++ postgresql.2/contrib/pg_upgrade/pg_upgrade.c 2012-11-22 10:16:53.673767458 +0100
@@ -67,6 +67,21 @@ char *output_files[] = {
NULL
};
+static void
+pg_upgrade_malloc_oom(void)
+{
+ pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
+}
+
+static void
+pg_upgrade_strdup_null(void)
+{
+ pg_log(PG_FATAL, "%s: pg_strdup: cannot duplicate null pointer (internal error)\n", os_info.progname);
+}
+
+static PostgresFrontendData pg_upgrade_fe_data = {
+ pg_upgrade_malloc_oom, pg_upgrade_strdup_null
+};
int
main(int argc, char **argv)
@@ -76,6 +91,8 @@ main(int argc, char **argv)
char *deletion_script_file_name = NULL;
bool live_check = false;
+ InitPostgresFrontend(&pg_upgrade_fe_data);
+
parseCommandLine(argc, argv);
adjust_data_dir(&old_cluster);
diff -durpN postgresql.1/contrib/pg_upgrade/pg_upgrade.h postgresql.2/contrib/pg_upgrade/pg_upgrade.h
--- postgresql.1/contrib/pg_upgrade/pg_upgrade.h 2012-11-15 13:39:47.132060180 +0100
+++ postgresql.2/contrib/pg_upgrade/pg_upgrade.h 2012-11-22 10:16:53.673767458 +0100
@@ -437,10 +437,6 @@ void
prep_status(const char *fmt,...)
__attribute__((format(PG_PRINTF_ATTRIBUTE, 1, 2)));
void check_ok(void);
-char *pg_strdup(const char *s);
-void *pg_malloc(size_t size);
-void *pg_realloc(void *ptr, size_t size);
-void pg_free(void *ptr);
const char *getErrorText(int errNum);
unsigned int str2uint(const char *str);
void pg_putenv(const char *var, const char *val);
diff -durpN postgresql.1/contrib/pg_upgrade/util.c postgresql.2/contrib/pg_upgrade/util.c
--- postgresql.1/contrib/pg_upgrade/util.c 2012-10-03 10:40:48.243207034 +0200
+++ postgresql.2/contrib/pg_upgrade/util.c 2012-11-22 10:16:53.674767465 +0100
@@ -191,55 +191,6 @@ get_user_info(char **user_name)
}
-void *
-pg_malloc(size_t size)
-{
- void *p;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- p = malloc(size);
- if (p == NULL)
- pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
- return p;
-}
-
-void *
-pg_realloc(void *ptr, size_t size)
-{
- void *p;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- p = realloc(ptr, size);
- if (p == NULL)
- pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
- return p;
-}
-
-
-void
-pg_free(void *ptr)
-{
- if (ptr != NULL)
- free(ptr);
-}
-
-
-char *
-pg_strdup(const char *s)
-{
- char *result = strdup(s);
-
- if (result == NULL)
- pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
-
- return result;
-}
-
-
/*
* getErrorText()
*
diff -durpN postgresql.1/src/bin/initdb/initdb.c postgresql.2/src/bin/initdb/initdb.c
--- postgresql.1/src/bin/initdb/initdb.c 2012-10-15 10:44:54.640429361 +0200
+++ postgresql.2/src/bin/initdb/initdb.c 2012-11-22 10:16:53.675767471 +0100
@@ -177,8 +177,6 @@ static const char *backend_options = "--
static char bin_path[MAXPGPATH];
static char backend_exec[MAXPGPATH];
-static void *pg_malloc(size_t size);
-static char *pg_strdup(const char *s);
static char **replace_token(char **lines,
const char *token, const char *replacement);
@@ -284,43 +282,6 @@ do { \
#endif
/*
- * routines to check mem allocations and fail noisily.
- *
- * Note that we can't call exit_nicely() on a memory failure, as it calls
- * rmtree() which needs memory allocation. So we just exit with a bang.
- */
-static void *
-pg_malloc(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-static char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-/*
* make a copy of the array of lines, with token replaced by replacement
* the first time it occurs on each line.
*
@@ -2823,6 +2784,22 @@ check_need_password(const char *authmeth
}
}
+static void
+initdb_malloc_oom(void)
+{
+ fprintf(stderr, _("%s: out of memory\n"), progname);
+}
+
+static void
+initdb_strdup_null(void)
+{
+ fprintf(stderr, _("%s: pg_strdup: cannot duplicate null pointer (internal error)\n"), progname);
+}
+
+static PostgresFrontendData initdb_fe_data = {
+ initdb_malloc_oom, initdb_strdup_null
+};
+
int
main(int argc, char *argv[])
{
@@ -2890,6 +2867,8 @@ main(int argc, char *argv[])
"pg_stat_tmp"
};
+ InitPostgresFrontend(&initdb_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("initdb"));
diff -durpN postgresql.1/src/bin/pg_basebackup/pg_basebackup.c postgresql.2/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.1/src/bin/pg_basebackup/pg_basebackup.c 2012-10-03 10:40:48.297207389 +0200
+++ postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2012-11-22 10:27:33.854681991 +0100
@@ -1234,6 +1234,29 @@ BaseBackup(void)
}
+static void
+pgbb_malloc_oom(void)
+{
+ fprintf(stderr, _("%s: out of memory\n"), progname);
+}
+
+static void
+pgbb_strdup_null(void)
+{
+ fprintf(stderr, _("%s: pg_strdup: cannot duplicate null pointer (internal error)\n"), progname);
+}
+
+static void
+pgbb_app_atexit(void)
+{
+ if (conn != NULL)
+ PQfinish(conn);
+}
+
+static PostgresFrontendData pgbb_fe_data = {
+ pgbb_malloc_oom, pgbb_strdup_null, pgbb_app_atexit
+};
+
int
main(int argc, char **argv)
{
@@ -1262,6 +1285,8 @@ main(int argc, char **argv)
int option_index;
+ InitPostgresFrontend(&pgbb_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_basebackup"));
diff -durpN postgresql.1/src/bin/pg_basebackup/streamutil.c postgresql.2/src/bin/pg_basebackup/streamutil.c
--- postgresql.1/src/bin/pg_basebackup/streamutil.c 2012-10-03 10:40:48.298207395 +0200
+++ postgresql.2/src/bin/pg_basebackup/streamutil.c 2012-11-22 10:26:06.694165253 +0100
@@ -32,43 +32,6 @@ static char *dbpassword = NULL;
PGconn *conn = NULL;
/*
- * strdup() and malloc() replacements that print an error and exit
- * if something goes wrong. Can never return NULL.
- */
-char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-void *
-pg_malloc0(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- MemSet(result, 0, size);
- return result;
-}
-
-
-/*
* Connect to the server. Returns a valid PGconn pointer if connected,
* or NULL on non-permanent error. On permanent error, the function will
* call exit(1) directly.
diff -durpN postgresql.1/src/bin/pg_basebackup/streamutil.h postgresql.2/src/bin/pg_basebackup/streamutil.h
--- postgresql.1/src/bin/pg_basebackup/streamutil.h 2012-10-03 10:40:48.299207401 +0200
+++ postgresql.2/src/bin/pg_basebackup/streamutil.h 2012-11-22 10:26:03.390145318 +0100
@@ -16,7 +16,4 @@ extern PGconn *conn;
}
-extern char *pg_strdup(const char *s);
-extern void *pg_malloc0(size_t size);
-
extern PGconn *GetConnection(void);
diff -durpN postgresql.1/src/bin/pg_ctl/pg_ctl.c postgresql.2/src/bin/pg_ctl/pg_ctl.c
--- postgresql.1/src/bin/pg_ctl/pg_ctl.c 2012-10-19 19:12:00.307691010 +0200
+++ postgresql.2/src/bin/pg_ctl/pg_ctl.c 2012-11-22 10:17:15.227895622 +0100
@@ -118,8 +118,6 @@ write_stderr(const char *fmt,...)
/* This extension allows gcc to check the format string for consistency with
the supplied arguments. */
__attribute__((format(PG_PRINTF_ATTRIBUTE, 1, 2)));
-static void *pg_malloc(size_t size);
-static char *pg_strdup(const char *s);
static void do_advice(void);
static void do_help(void);
static void set_mode(char *modeopt);
@@ -226,42 +224,6 @@ write_stderr(const char *fmt,...)
}
/*
- * routines to check memory allocations and fail noisily.
- */
-
-static void *
-pg_malloc(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- write_stderr(_("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-
-static char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- write_stderr(_("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-/*
* Given an already-localized string, print it to stdout unless the
* user has specified that no messages should be printed.
*/
@@ -1994,6 +1956,22 @@ adjust_data_dir(void)
}
+static void
+pgctl_malloc_oom(void)
+{
+ write_stderr(_("%s: out of memory\n"), progname);
+}
+
+static void
+pgctl_strdup_null(void)
+{
+ write_stderr(_("%s: pg_strdup: cannot duplicate null pointer (internal error)\n"), progname);
+}
+
+static PostgresFrontendData pgctl_fe_data = {
+ pgctl_malloc_oom, pgctl_strdup_null
+};
+
int
main(int argc, char **argv)
{
@@ -2013,6 +1991,8 @@ main(int argc, char **argv)
int c;
pgpid_t killproc = 0;
+ InitPostgresFrontend(&pgctl_fe_data);
+
#if defined(WIN32) || defined(__CYGWIN__)
setvbuf(stderr, NULL, _IONBF, 0);
#endif
diff -durpN postgresql.1/src/bin/pg_dump/common.c postgresql.2/src/bin/pg_dump/common.c
--- postgresql.1/src/bin/pg_dump/common.c 2012-07-22 16:48:48.536857822 +0200
+++ postgresql.2/src/bin/pg_dump/common.c 2012-11-22 10:17:15.228895628 +0100
@@ -18,7 +18,6 @@
#include <ctype.h>
#include "catalog/pg_class.h"
-#include "dumpmem.h"
#include "dumputils.h"
diff -durpN postgresql.1/src/bin/pg_dump/compress_io.c postgresql.2/src/bin/pg_dump/compress_io.c
--- postgresql.1/src/bin/pg_dump/compress_io.c 2012-10-03 10:40:48.301207415 +0200
+++ postgresql.2/src/bin/pg_dump/compress_io.c 2012-11-22 10:17:15.229895635 +0100
@@ -53,7 +53,6 @@
*/
#include "compress_io.h"
-#include "dumpmem.h"
#include "dumputils.h"
/*----------------------
diff -durpN postgresql.1/src/bin/pg_dump/dumpmem.c postgresql.2/src/bin/pg_dump/dumpmem.c
--- postgresql.1/src/bin/pg_dump/dumpmem.c 2012-10-03 10:40:48.301207415 +0200
+++ postgresql.2/src/bin/pg_dump/dumpmem.c 1970-01-01 01:00:00.000000000 +0100
@@ -1,76 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * dumpmem.c
- * Memory allocation routines used by pg_dump, pg_dumpall, and pg_restore
- *
- * Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
- *
- *
- * IDENTIFICATION
- * src/bin/pg_dump/dumpmem.c
- *
- *-------------------------------------------------------------------------
- */
-#include "postgres_fe.h"
-
-#include "dumputils.h"
-#include "dumpmem.h"
-
-
-/*
- * Safer versions of some standard C library functions. If an
- * out-of-memory condition occurs, these functions will bail out via exit();
- *therefore, their return value is guaranteed to be non-NULL.
- */
-
-char *
-pg_strdup(const char *string)
-{
- char *tmp;
-
- if (!string)
- exit_horribly(NULL, "cannot duplicate null pointer\n");
- tmp = strdup(string);
- if (!tmp)
- exit_horribly(NULL, "out of memory\n");
- return tmp;
-}
-
-void *
-pg_malloc(size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- tmp = malloc(size);
- if (!tmp)
- exit_horribly(NULL, "out of memory\n");
- return tmp;
-}
-
-void *
-pg_malloc0(size_t size)
-{
- void *tmp;
-
- tmp = pg_malloc(size);
- MemSet(tmp, 0, size);
- return tmp;
-}
-
-void *
-pg_realloc(void *ptr, size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- tmp = realloc(ptr, size);
- if (!tmp)
- exit_horribly(NULL, "out of memory\n");
- return tmp;
-}
diff -durpN postgresql.1/src/bin/pg_dump/dumpmem.h postgresql.2/src/bin/pg_dump/dumpmem.h
--- postgresql.1/src/bin/pg_dump/dumpmem.h 2012-10-03 10:40:48.301207415 +0200
+++ postgresql.2/src/bin/pg_dump/dumpmem.h 1970-01-01 01:00:00.000000000 +0100
@@ -1,22 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * dumpmem.h
- * Memory allocation routines used by pg_dump, pg_dumpall, and pg_restore
- *
- * Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
- *
- * src/bin/pg_dump/dumpmem.h
- *
- *-------------------------------------------------------------------------
- */
-
-#ifndef DUMPMEM_H
-#define DUMPMEM_H
-
-extern char *pg_strdup(const char *string);
-extern void *pg_malloc(size_t size);
-extern void *pg_malloc0(size_t size);
-extern void *pg_realloc(void *ptr, size_t size);
-
-#endif /* DUMPMEM_H */
diff -durpN postgresql.1/src/bin/pg_dump/Makefile postgresql.2/src/bin/pg_dump/Makefile
--- postgresql.1/src/bin/pg_dump/Makefile 2012-04-16 19:57:22.595917312 +0200
+++ postgresql.2/src/bin/pg_dump/Makefile 2012-11-22 10:17:15.234895667 +0100
@@ -20,7 +20,7 @@ override CPPFLAGS := -I$(libpq_srcdir) $
OBJS= pg_backup_archiver.o pg_backup_db.o pg_backup_custom.o \
pg_backup_null.o pg_backup_tar.o \
- pg_backup_directory.o dumpmem.o dumputils.o compress_io.o $(WIN32RES)
+ pg_backup_directory.o dumputils.o compress_io.o $(WIN32RES)
KEYWRDOBJS = keywords.o kwlookup.o
@@ -35,8 +35,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort
pg_restore: pg_restore.o $(OBJS) $(KEYWRDOBJS) | submake-libpq submake-libpgport
$(CC) $(CFLAGS) pg_restore.o $(KEYWRDOBJS) $(OBJS) $(libpq_pgport) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
-pg_dumpall: pg_dumpall.o dumputils.o dumpmem.o $(KEYWRDOBJS) | submake-libpq submake-libpgport
- $(CC) $(CFLAGS) pg_dumpall.o dumputils.o dumpmem.o $(KEYWRDOBJS) $(WIN32RES) $(libpq_pgport) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o $(KEYWRDOBJS) | submake-libpq submake-libpgport
+ $(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(KEYWRDOBJS) $(WIN32RES) $(libpq_pgport) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
install: all installdirs
$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff -durpN postgresql.1/src/bin/pg_dump/nls.mk postgresql.2/src/bin/pg_dump/nls.mk
--- postgresql.1/src/bin/pg_dump/nls.mk 2012-04-16 19:57:22.596917323 +0200
+++ postgresql.2/src/bin/pg_dump/nls.mk 2012-11-22 10:17:15.235895673 +0100
@@ -3,7 +3,7 @@ CATALOG_NAME = pg_dump
AVAIL_LANGUAGES = de es fr it ja ko pt_BR sv tr zh_CN zh_TW
GETTEXT_FILES = pg_backup_archiver.c pg_backup_db.c pg_backup_custom.c \
pg_backup_null.c pg_backup_tar.c \
- pg_backup_directory.c dumpmem.c dumputils.c compress_io.c \
+ pg_backup_directory.c dumputils.c compress_io.c \
pg_dump.c common.c pg_dump_sort.c \
pg_restore.c pg_dumpall.c \
../../port/exec.c
diff -durpN postgresql.1/src/bin/pg_dump/pg_backup_archiver.c postgresql.2/src/bin/pg_dump/pg_backup_archiver.c
--- postgresql.1/src/bin/pg_dump/pg_backup_archiver.c 2012-10-21 10:56:15.398945610 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_archiver.c 2012-11-22 10:17:15.236895680 +0100
@@ -21,7 +21,6 @@
*/
#include "pg_backup_db.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <ctype.h>
diff -durpN postgresql.1/src/bin/pg_dump/pg_backup_custom.c postgresql.2/src/bin/pg_dump/pg_backup_custom.c
--- postgresql.1/src/bin/pg_dump/pg_backup_custom.c 2012-10-03 10:40:48.303207429 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_custom.c 2012-11-22 10:17:15.237895687 +0100
@@ -26,7 +26,6 @@
#include "compress_io.h"
#include "dumputils.h"
-#include "dumpmem.h"
/*--------
* Routines in the format interface
diff -durpN postgresql.1/src/bin/pg_dump/pg_backup_db.c postgresql.2/src/bin/pg_dump/pg_backup_db.c
--- postgresql.1/src/bin/pg_dump/pg_backup_db.c 2012-07-25 10:51:14.973507544 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_db.c 2012-11-22 10:17:15.238895693 +0100
@@ -11,7 +11,6 @@
*/
#include "pg_backup_db.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <unistd.h>
diff -durpN postgresql.1/src/bin/pg_dump/pg_backup_directory.c postgresql.2/src/bin/pg_dump/pg_backup_directory.c
--- postgresql.1/src/bin/pg_dump/pg_backup_directory.c 2012-10-03 10:40:48.303207429 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_directory.c 2012-11-22 10:17:15.238895693 +0100
@@ -34,7 +34,6 @@
*/
#include "compress_io.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <dirent.h>
diff -durpN postgresql.1/src/bin/pg_dump/pg_backup_null.c postgresql.2/src/bin/pg_dump/pg_backup_null.c
--- postgresql.1/src/bin/pg_dump/pg_backup_null.c 2012-04-16 19:57:22.598917347 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_null.c 2012-11-22 10:17:15.239895699 +0100
@@ -23,7 +23,6 @@
*/
#include "pg_backup_archiver.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <unistd.h> /* for dup */
diff -durpN postgresql.1/src/bin/pg_dump/pg_backup_tar.c postgresql.2/src/bin/pg_dump/pg_backup_tar.c
--- postgresql.1/src/bin/pg_dump/pg_backup_tar.c 2012-10-03 10:40:48.304207436 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_tar.c 2012-11-22 10:17:15.240895705 +0100
@@ -28,7 +28,6 @@
#include "pg_backup.h"
#include "pg_backup_archiver.h"
#include "pg_backup_tar.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <sys/stat.h>
diff -durpN postgresql.1/src/bin/pg_dump/pg_dumpall.c postgresql.2/src/bin/pg_dump/pg_dumpall.c
--- postgresql.1/src/bin/pg_dump/pg_dumpall.c 2012-10-14 11:09:08.143146185 +0200
+++ postgresql.2/src/bin/pg_dump/pg_dumpall.c 2012-11-22 10:17:15.241895712 +0100
@@ -23,7 +23,6 @@
#include "getopt_long.h"
#include "dumputils.h"
-#include "dumpmem.h"
#include "pg_backup.h"
/* version string we expect back from pg_dump */
diff -durpN postgresql.1/src/bin/pg_dump/pg_dump.c postgresql.2/src/bin/pg_dump/pg_dump.c
--- postgresql.1/src/bin/pg_dump/pg_dump.c 2012-10-28 07:41:12.051532033 +0100
+++ postgresql.2/src/bin/pg_dump/pg_dump.c 2012-11-22 10:17:15.247895751 +0100
@@ -59,7 +59,6 @@
#include "pg_backup_archiver.h"
#include "pg_backup_db.h"
-#include "dumpmem.h"
#include "dumputils.h"
extern char *optarg;
diff -durpN postgresql.1/src/bin/pg_dump/pg_dump_sort.c postgresql.2/src/bin/pg_dump/pg_dump_sort.c
--- postgresql.1/src/bin/pg_dump/pg_dump_sort.c 2012-10-19 19:12:00.334691199 +0200
+++ postgresql.2/src/bin/pg_dump/pg_dump_sort.c 2012-11-22 10:17:15.247895751 +0100
@@ -15,7 +15,6 @@
*/
#include "pg_backup_archiver.h"
#include "dumputils.h"
-#include "dumpmem.h"
/* translator: this is a module name */
static const char *modulename = gettext_noop("sorter");
diff -durpN postgresql.1/src/bin/pg_dump/pg_restore.c postgresql.2/src/bin/pg_dump/pg_restore.c
--- postgresql.1/src/bin/pg_dump/pg_restore.c 2012-10-14 11:09:08.143146185 +0200
+++ postgresql.2/src/bin/pg_dump/pg_restore.c 2012-11-22 10:17:15.248895757 +0100
@@ -41,7 +41,6 @@
#include "pg_backup_archiver.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <ctype.h>
diff -durpN postgresql.1/src/bin/psql/common.c postgresql.2/src/bin/psql/common.c
--- postgresql.1/src/bin/psql/common.c 2012-10-03 10:40:48.322207553 +0200
+++ postgresql.2/src/bin/psql/common.c 2012-11-22 10:17:15.248895757 +0100
@@ -33,56 +33,6 @@ static bool command_no_begin(const char
static bool is_select_command(const char *query);
/*
- * "Safe" wrapper around strdup()
- */
-char *
-pg_strdup(const char *string)
-{
- char *tmp;
-
- if (!string)
- {
- psql_error("%s: pg_strdup: cannot duplicate null pointer (internal error)\n",
- pset.progname);
- exit(EXIT_FAILURE);
- }
- tmp = strdup(string);
- if (!tmp)
- {
- psql_error("out of memory\n");
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc(size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- tmp = malloc(size);
- if (!tmp)
- {
- psql_error("out of memory\n");
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc0(size_t size)
-{
- void *tmp;
-
- tmp = pg_malloc(size);
- MemSet(tmp, 0, size);
- return tmp;
-}
-
-/*
* setQFout
* -- handler for -o command line option and \o command
*
diff -durpN postgresql.1/src/bin/psql/common.h postgresql.2/src/bin/psql/common.h
--- postgresql.1/src/bin/psql/common.h 2012-10-03 10:40:48.322207553 +0200
+++ postgresql.2/src/bin/psql/common.h 2012-11-22 10:17:15.248895757 +0100
@@ -21,15 +21,6 @@
#define atooid(x) ((Oid) strtoul((x), NULL, 10))
-/*
- * Safer versions of some standard C library functions. If an
- * out-of-memory condition occurs, these functions will bail out
- * safely; therefore, their return value is guaranteed to be non-NULL.
- */
-extern char *pg_strdup(const char *string);
-extern void *pg_malloc(size_t size);
-extern void *pg_malloc0(size_t size);
-
extern bool setQFout(const char *fname);
extern void
diff -durpN postgresql.1/src/bin/psql/startup.c postgresql.2/src/bin/psql/startup.c
--- postgresql.1/src/bin/psql/startup.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/psql/startup.c 2012-11-22 10:17:15.249895764 +0100
@@ -77,6 +77,21 @@ static void process_psqlrc_file(char *fi
static void showVersion(void);
static void EstablishVariableSpace(void);
+static void psql_malloc_oom(void)
+{
+ psql_error("out of memory\n");
+}
+
+static void psql_strdup_null(void)
+{
+ psql_error("%s: pg_strdup: cannot duplicate null pointer (internal error)\n",
+ pset.progname);
+}
+
+static PostgresFrontendData psql_fe_data = {
+ psql_malloc_oom, psql_strdup_null
+};
+
/*
*
* main
@@ -91,6 +106,8 @@ main(int argc, char *argv[])
char *password_prompt = NULL;
bool new_pass;
+ InitPostgresFrontend(&psql_fe_data);
+
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("psql"));
if (argc > 1)
diff -durpN postgresql.1/src/bin/scripts/clusterdb.c postgresql.2/src/bin/scripts/clusterdb.c
--- postgresql.1/src/bin/scripts/clusterdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/clusterdb.c 2012-11-22 10:17:15.249895764 +0100
@@ -61,6 +61,8 @@ main(int argc, char *argv[])
char *table = NULL;
bool verbose = false;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/bin/scripts/common.c postgresql.2/src/bin/scripts/common.c
--- postgresql.1/src/bin/scripts/common.c 2012-10-03 10:40:48.324207567 +0200
+++ postgresql.2/src/bin/scripts/common.c 2012-11-22 10:17:15.249895764 +0100
@@ -31,6 +31,25 @@ static CRITICAL_SECTION cancelConnLock;
#endif
/*
+ * Helper functions to report error in pg_malloc and pg_strdup
+ */
+static void
+common_malloc_oom(void)
+{
+ fprintf(stderr, _("out of memory\n"));
+}
+
+static void
+common_strdup_null(void)
+{
+ fprintf(stderr, _("pg_strdup: cannot duplicate null pointer (internal error)\n"));
+}
+
+PostgresFrontendData common_fe_data = {
+ common_malloc_oom, common_strdup_null
+};
+
+/*
* Returns the current user name.
*/
const char *
@@ -278,55 +297,6 @@ executeMaintenanceCommand(PGconn *conn,
}
/*
- * "Safe" wrapper around strdup(). Pulled from psql/common.c
- */
-char *
-pg_strdup(const char *string)
-{
- char *tmp;
-
- if (!string)
- {
- fprintf(stderr, _("pg_strdup: cannot duplicate null pointer (internal error)\n"));
- exit(EXIT_FAILURE);
- }
- tmp = strdup(string);
- if (!tmp)
- {
- fprintf(stderr, _("out of memory\n"));
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc(size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- tmp = malloc(size);
- if (!tmp)
- {
- fprintf(stderr, _("out of memory\n"));
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc0(size_t size)
-{
- void *tmp;
-
- tmp = pg_malloc(size);
- MemSet(tmp, 0, size);
- return tmp;
-}
-
-/*
* Check yes/no answer in a localized way. 1=yes, 0=no, -1=neither.
*/
diff -durpN postgresql.1/src/bin/scripts/common.h postgresql.2/src/bin/scripts/common.h
--- postgresql.1/src/bin/scripts/common.h 2012-10-03 10:40:48.324207567 +0200
+++ postgresql.2/src/bin/scripts/common.h 2012-11-22 10:17:15.250895771 +0100
@@ -22,6 +22,8 @@ enum trivalue
typedef void (*help_handler) (const char *progname);
+extern PostgresFrontendData common_fe_data;
+
extern const char *get_user_name(const char *progname);
extern void handle_help_version_opts(int argc, char *argv[],
@@ -50,8 +52,4 @@ extern bool yesno_prompt(const char *que
extern void setup_cancel_handler(void);
-extern char *pg_strdup(const char *string);
-extern void *pg_malloc(size_t size);
-extern void *pg_malloc0(size_t size);
-
#endif /* COMMON_H */
diff -durpN postgresql.1/src/bin/scripts/createdb.c postgresql.2/src/bin/scripts/createdb.c
--- postgresql.1/src/bin/scripts/createdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/createdb.c 2012-11-22 10:17:15.250895771 +0100
@@ -64,6 +64,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/bin/scripts/createlang.c postgresql.2/src/bin/scripts/createlang.c
--- postgresql.1/src/bin/scripts/createlang.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/createlang.c 2012-11-22 10:17:15.250895771 +0100
@@ -52,6 +52,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/bin/scripts/createuser.c postgresql.2/src/bin/scripts/createuser.c
--- postgresql.1/src/bin/scripts/createuser.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/createuser.c 2012-11-22 10:17:15.251895777 +0100
@@ -78,6 +78,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/bin/scripts/dropdb.c postgresql.2/src/bin/scripts/dropdb.c
--- postgresql.1/src/bin/scripts/dropdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/dropdb.c 2012-11-22 10:17:15.251895777 +0100
@@ -54,6 +54,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/bin/scripts/droplang.c postgresql.2/src/bin/scripts/droplang.c
--- postgresql.1/src/bin/scripts/droplang.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/droplang.c 2012-11-22 10:17:15.252895783 +0100
@@ -51,6 +51,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/bin/scripts/dropuser.c postgresql.2/src/bin/scripts/dropuser.c
--- postgresql.1/src/bin/scripts/dropuser.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/dropuser.c 2012-11-22 10:17:15.253895789 +0100
@@ -52,6 +52,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/bin/scripts/reindexdb.c postgresql.2/src/bin/scripts/reindexdb.c
--- postgresql.1/src/bin/scripts/reindexdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/reindexdb.c 2012-11-22 10:17:15.253895789 +0100
@@ -67,6 +67,8 @@ main(int argc, char *argv[])
const char *table = NULL;
const char *index = NULL;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/bin/scripts/vacuumdb.c postgresql.2/src/bin/scripts/vacuumdb.c
--- postgresql.1/src/bin/scripts/vacuumdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/vacuumdb.c 2012-11-22 10:17:15.254895795 +0100
@@ -72,6 +72,8 @@ main(int argc, char *argv[])
bool full = false;
bool verbose = false;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/include/port.h postgresql.2/src/include/port.h
--- postgresql.1/src/include/port.h 2012-07-22 16:48:48.548857900 +0200
+++ postgresql.2/src/include/port.h 2012-11-22 10:17:15.255895802 +0100
@@ -17,6 +17,19 @@
#include <netdb.h>
#include <pwd.h>
+/* port/init.c */
+typedef void (*fe_func)(void);
+
+typedef struct PostgresFrontendData
+{
+ fe_func pg_malloc_oom;
+ fe_func pg_strdup_null;
+ fe_func app_atexit;
+} PostgresFrontendData, *PostgresFrontend;
+extern PostgresFrontend pg_fedata;
+
+extern void InitPostgresFrontend(PostgresFrontend fe_data);
+
/* socket has a different definition on WIN32 */
#ifndef WIN32
typedef int pgsocket;
@@ -462,6 +475,19 @@ extern char *inet_net_ntop(int af, const
/* port/pgcheckdir.c */
extern int pg_check_dir(const char *dir);
+/* port/pgmalloc.c */
+extern void pgmalloc_atexit(void);
+
+extern char *pg_strdup(const char *string);
+
+extern void *pg_malloc(size_t size);
+
+extern void *pg_malloc0(size_t size);
+
+extern void *pg_realloc(void *ptr, size_t size);
+
+extern void pg_free(void *ptr);
+
/* port/pgmkdirp.c */
extern int pg_mkdir_p(char *path, int omode);
diff -durpN postgresql.1/src/port/init.c postgresql.2/src/port/init.c
--- postgresql.1/src/port/init.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.2/src/port/init.c 2012-11-22 10:17:15.255895802 +0100
@@ -0,0 +1,32 @@
+/*-------------------------------------------------------------------------
+ *
+ * init.c
+ * Functions for initializing frontend functions
+ *
+ *
+ * Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/init.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef FRONTEND
+#include "postgres_fe.h"
+
+PostgresFrontend pg_fedata = NULL;
+
+void InitPostgresFrontend(PostgresFrontend ptr)
+{
+ pg_fedata = ptr;
+
+ atexit(pgmalloc_atexit);
+
+ if (ptr && ptr->app_atexit)
+ atexit(ptr->app_atexit);
+}
+
+#endif
diff -durpN postgresql.1/src/port/Makefile postgresql.2/src/port/Makefile
--- postgresql.1/src/port/Makefile 2012-05-09 12:28:11.755496856 +0200
+++ postgresql.2/src/port/Makefile 2012-11-22 10:17:15.256895809 +0100
@@ -30,9 +30,10 @@ include $(top_builddir)/src/Makefile.glo
override CPPFLAGS := -I$(top_builddir)/src/port -DFRONTEND $(CPPFLAGS)
LIBS += $(PTHREAD_LIBS)
-OBJS = $(LIBOBJS) chklocale.o dirmod.o erand48.o exec.o fls.o inet_net_ntop.o \
- noblock.o path.o pgcheckdir.o pg_crc.o pgmkdirp.o pgsleep.o \
- pgstrcasecmp.o qsort.o qsort_arg.o sprompt.o thread.o
+OBJS = $(LIBOBJS) chklocale.o dirmod.o erand48.o exec.o fls.o init.o \
+ inet_net_ntop.o noblock.o path.o pgcheckdir.o pg_crc.o pgmalloc.o \
+ pgmkdirp.o pgsleep.o pgstrcasecmp.o qsort.o qsort_arg.o sprompt.o \
+ thread.o
# foo_srv.o and foo.o are both built from foo.c, but only foo.o has -DFRONTEND
OBJS_SRV = $(OBJS:%.o=%_srv.o)
diff -durpN postgresql.1/src/port/pgmalloc.c postgresql.2/src/port/pgmalloc.c
--- postgresql.1/src/port/pgmalloc.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.2/src/port/pgmalloc.c 2012-11-22 10:17:15.256895809 +0100
@@ -0,0 +1,125 @@
+/*-------------------------------------------------------------------------
+ *
+ * pgmalloc.c
+ * Functions for allocating memory and
+ * exiting on out-of-memory
+ *
+ *
+ * Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pgmalloc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef FRONTEND
+#include "postgres_fe.h"
+
+#include <stdio.h>
+
+typedef enum {
+ PG_MALLOC_NOP,
+ PG_MALLOC_OOM,
+ PG_STRDUP_NULL
+} PgMallocAction;
+
+static PgMallocAction pgmalloc_action = PG_MALLOC_NOP;
+
+void pgmalloc_atexit(void)
+{
+ switch (pgmalloc_action)
+ {
+ case PG_MALLOC_OOM:
+ if (pg_fedata && pg_fedata->pg_malloc_oom)
+ pg_fedata->pg_malloc_oom();
+ else
+ fprintf(stderr, "out of memory\n");
+ break;
+ case PG_STRDUP_NULL:
+ if (pg_fedata && pg_fedata->pg_strdup_null)
+ pg_fedata->pg_strdup_null();
+ else
+ fprintf(stderr, "pg_strdup: cannot duplicate null pointer (internal error)\n");
+ break;
+ case PG_MALLOC_NOP:
+ break;
+ }
+}
+
+/*
+ * "Safe" wrapper around strdup()
+ */
+char *
+pg_strdup(const char *string)
+{
+ char *tmp;
+
+ if (!string)
+ {
+ pgmalloc_action = PG_STRDUP_NULL;
+ exit(1);
+ }
+ tmp = strdup(string);
+ if (!tmp)
+ {
+ pgmalloc_action = PG_MALLOC_OOM;
+ exit(1);
+ }
+ return tmp;
+}
+
+void *
+pg_malloc(size_t size)
+{
+ void *tmp;
+
+ /* Avoid unportable behavior of malloc(0) */
+ if (size == 0)
+ size = 1;
+ tmp = malloc(size);
+ if (!tmp)
+ {
+ pgmalloc_action = PG_MALLOC_OOM;
+ exit(1);
+ }
+ return tmp;
+}
+
+void *
+pg_malloc0(size_t size)
+{
+ void *tmp;
+
+ tmp = pg_malloc(size);
+ MemSet(tmp, 0, size);
+ return tmp;
+}
+
+void *
+pg_realloc(void *ptr, size_t size)
+{
+ void *tmp;
+
+ /* Avoid unportable behavior of realloc(NULL, 0) */
+ if (ptr == NULL && size == 0)
+ size = 1;
+ tmp = realloc(ptr, size);
+ if (!tmp)
+ {
+ pgmalloc_action = PG_MALLOC_OOM;
+ exit(1);
+ }
+ return tmp;
+}
+
+void
+pg_free(void *ptr)
+{
+ if (ptr != NULL)
+ free(ptr);
+}
+
+#endif
03-tarutils-strutils-v3.patchtext/x-patch; name=03-tarutils-strutils-v3.patchDownload
diff -durpN postgresql.2/src/backend/replication/basebackup.c postgresql.3/src/backend/replication/basebackup.c
--- postgresql.2/src/backend/replication/basebackup.c 2012-10-06 19:25:21.002166521 +0200
+++ postgresql.3/src/backend/replication/basebackup.c 2012-11-22 10:31:29.501038356 +0100
@@ -721,48 +721,12 @@ sendDir(char *path, int basepathlen, boo
/*
- * Utility routine to print possibly larger than 32 bit integers in a
- * portable fashion. Filled with zeros.
- */
-static void
-print_val(char *s, uint64 val, unsigned int base, size_t len)
-{
- int i;
-
- for (i = len; i > 0; i--)
- {
- int digit = val % base;
-
- s[i - 1] = '0' + digit;
- val = val / base;
- }
-}
-
-/*
* Maximum file size for a tar member: The limit inherent in the
* format is 2^33-1 bytes (nearly 8 GB). But we don't want to exceed
* what we can represent in pgoff_t.
*/
#define MAX_TAR_MEMBER_FILELEN (((int64) 1 << Min(33, sizeof(pgoff_t)*8 - 1)) - 1)
-static int
-_tarChecksum(char *header)
-{
- int i,
- sum;
-
- /*
- * Per POSIX, the checksum is the simple sum of all bytes in the header,
- * treating the bytes as unsigned, and treating the checksum field (at
- * offset 148) as though it contained 8 spaces.
- */
- sum = 8 * ' '; /* presumed value for checksum field */
- for (i = 0; i < 512; i++)
- if (i < 148 || i >= 156)
- sum += 0xFF & header[i];
- return sum;
-}
-
/* Given the member, write the TAR header & send the file */
static void
sendFile(char *readfilename, char *tarfilename, struct stat * statbuf)
@@ -840,94 +804,12 @@ _tarWriteHeader(const char *filename, co
{
char h[512];
- /*
- * Note: most of the fields in a tar header are not supposed to be
- * null-terminated. We use sprintf, which will write a null after the
- * required bytes; that null goes into the first byte of the next field.
- * This is okay as long as we fill the fields in order.
- */
- memset(h, 0, sizeof(h));
-
- /* Name 100 */
- sprintf(&h[0], "%.99s", filename);
- if (linktarget != NULL || S_ISDIR(statbuf->st_mode))
- {
- /*
- * We only support symbolic links to directories, and this is
- * indicated in the tar format by adding a slash at the end of the
- * name, the same as for regular directories.
- */
- int flen = strlen(filename);
-
- flen = Min(flen, 99);
- h[flen] = '/';
- h[flen + 1] = '\0';
- }
-
- /* Mode 8 */
- sprintf(&h[100], "%07o ", (int) statbuf->st_mode);
-
- /* User ID 8 */
- sprintf(&h[108], "%07o ", statbuf->st_uid);
-
- /* Group 8 */
- sprintf(&h[116], "%07o ", statbuf->st_gid);
-
- /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
- if (linktarget != NULL || S_ISDIR(statbuf->st_mode))
- /* Symbolic link or directory has size zero */
- print_val(&h[124], 0, 8, 11);
- else
- print_val(&h[124], statbuf->st_size, 8, 11);
- sprintf(&h[135], " ");
-
- /* Mod Time 12 */
- sprintf(&h[136], "%011o ", (int) statbuf->st_mtime);
-
- /* Checksum 8 cannot be calculated until we've filled all other fields */
-
- if (linktarget != NULL)
- {
- /* Type - Symbolic link */
- sprintf(&h[156], "2");
- /* Link Name 100 */
- sprintf(&h[157], "%.99s", linktarget);
- }
- else if (S_ISDIR(statbuf->st_mode))
- /* Type - directory */
- sprintf(&h[156], "5");
- else
- /* Type - regular file */
- sprintf(&h[156], "0");
-
- /* Magic 6 */
- sprintf(&h[257], "ustar");
-
- /* Version 2 */
- sprintf(&h[263], "00");
-
- /* User 32 */
- /* XXX: Do we need to care about setting correct username? */
- sprintf(&h[265], "%.31s", "postgres");
-
- /* Group 32 */
- /* XXX: Do we need to care about setting correct group name? */
- sprintf(&h[297], "%.31s", "postgres");
-
- /* Major Dev 8 */
- sprintf(&h[329], "%07o ", 0);
-
- /* Minor Dev 8 */
- sprintf(&h[337], "%07o ", 0);
-
- /* Prefix 155 - not used, leave as nulls */
-
- /*
- * We mustn't overwrite the next field while inserting the checksum.
- * Fortunately, the checksum can't exceed 6 octal digits, so we just write
- * 6 digits, a space, and a null, which is legal per POSIX.
- */
- sprintf(&h[148], "%06o ", _tarChecksum(h));
+ _tarCreateHeader(h, filename, linktarget,
+ statbuf->st_size,
+ statbuf->st_mode,
+ statbuf->st_uid,
+ statbuf->st_gid,
+ statbuf->st_mtime);
/* Now send the completed header. */
pq_putmessage('d', h, 512);
diff -durpN postgresql.2/src/bin/initdb/initdb.c postgresql.3/src/bin/initdb/initdb.c
--- postgresql.2/src/bin/initdb/initdb.c 2012-11-22 10:16:53.675767471 +0100
+++ postgresql.3/src/bin/initdb/initdb.c 2012-11-22 10:31:29.503038369 +0100
@@ -218,7 +218,6 @@ static void make_postgres(void);
static void perform_fsync(void);
static void trapsig(int signum);
static void check_ok(void);
-static char *escape_quotes(const char *src);
static int locale_date_order(const char *locale);
static bool check_locale_name(int category, const char *locale,
char **canonname);
@@ -2342,35 +2341,6 @@ check_ok(void)
}
}
-/*
- * Escape (by doubling) any single quotes or backslashes in given string
- *
- * Note: this is used to process both postgresql.conf entries and SQL
- * string literals. Since postgresql.conf strings are defined to treat
- * backslashes as escapes, we have to double backslashes here. Hence,
- * when using this for a SQL string literal, use E'' syntax.
- *
- * We do not need to worry about encoding considerations because all
- * valid backend encodings are ASCII-safe.
- */
-static char *
-escape_quotes(const char *src)
-{
- int len = strlen(src),
- i,
- j;
- char *result = pg_malloc(len * 2 + 1);
-
- for (i = 0, j = 0; i < len; i++)
- {
- if (SQL_STR_DOUBLE(src[i], true))
- result[j++] = src[i];
- result[j++] = src[i];
- }
- result[j] = '\0';
- return result;
-}
-
/* Hack to suppress a warning about %x from some versions of gcc */
static inline size_t
my_strftime(char *s, size_t max, const char *fmt, const struct tm * tm)
diff -durpN postgresql.2/src/bin/pg_dump/pg_backup_tar.c postgresql.3/src/bin/pg_dump/pg_backup_tar.c
--- postgresql.2/src/bin/pg_dump/pg_backup_tar.c 2012-11-22 10:17:15.240895705 +0100
+++ postgresql.3/src/bin/pg_dump/pg_backup_tar.c 2012-11-22 10:31:32.050052844 +0100
@@ -114,7 +114,6 @@ static char *tarGets(char *buf, size_t l
static int tarPrintf(ArchiveHandle *AH, TAR_MEMBER *th, const char *fmt,...) __attribute__((format(PG_PRINTF_ATTRIBUTE, 3, 4)));
static void _tarAddFile(ArchiveHandle *AH, TAR_MEMBER *th);
-static int _tarChecksum(char *th);
static TAR_MEMBER *_tarPositionTo(ArchiveHandle *AH, const char *filename);
static size_t tarRead(void *buf, size_t len, TAR_MEMBER *th);
static size_t tarWrite(const void *buf, size_t len, TAR_MEMBER *th);
@@ -1015,24 +1014,6 @@ tarPrintf(ArchiveHandle *AH, TAR_MEMBER
return cnt;
}
-static int
-_tarChecksum(char *header)
-{
- int i,
- sum;
-
- /*
- * Per POSIX, the checksum is the simple sum of all bytes in the header,
- * treating the bytes as unsigned, and treating the checksum field (at
- * offset 148) as though it contained 8 spaces.
- */
- sum = 8 * ' '; /* presumed value for checksum field */
- for (i = 0; i < 512; i++)
- if (i < 148 || i >= 156)
- sum += 0xFF & header[i];
- return sum;
-}
-
bool
isValidTarHeader(char *header)
{
@@ -1304,94 +1285,16 @@ _tarGetHeader(ArchiveHandle *AH, TAR_MEM
}
-/*
- * Utility routine to print possibly larger than 32 bit integers in a
- * portable fashion. Filled with zeros.
- */
-static void
-print_val(char *s, uint64 val, unsigned int base, size_t len)
-{
- int i;
-
- for (i = len; i > 0; i--)
- {
- int digit = val % base;
-
- s[i - 1] = '0' + digit;
- val = val / base;
- }
-}
-
-
static void
_tarWriteHeader(TAR_MEMBER *th)
{
char h[512];
- /*
- * Note: most of the fields in a tar header are not supposed to be
- * null-terminated. We use sprintf, which will write a null after the
- * required bytes; that null goes into the first byte of the next field.
- * This is okay as long as we fill the fields in order.
- */
- memset(h, 0, sizeof(h));
-
- /* Name 100 */
- sprintf(&h[0], "%.99s", th->targetFile);
-
- /* Mode 8 */
- sprintf(&h[100], "0000600 ");
-
- /* User ID 8 */
- sprintf(&h[108], "0004000 ");
-
- /* Group 8 */
- sprintf(&h[116], "0002000 ");
-
- /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
- print_val(&h[124], th->fileLen, 8, 11);
- sprintf(&h[135], " ");
-
- /* Mod Time 12 */
- sprintf(&h[136], "%011o ", (int) time(NULL));
-
- /* Checksum 8 cannot be calculated until we've filled all other fields */
-
- /* Type - regular file */
- sprintf(&h[156], "0");
-
- /* Link Name 100 (leave as nulls) */
-
- /* Magic 6 */
- sprintf(&h[257], "ustar");
-
- /* Version 2 */
- sprintf(&h[263], "00");
-
- /* User 32 */
- /* XXX: Do we need to care about setting correct username? */
- sprintf(&h[265], "%.31s", "postgres");
-
- /* Group 32 */
- /* XXX: Do we need to care about setting correct group name? */
- sprintf(&h[297], "%.31s", "postgres");
-
- /* Major Dev 8 */
- sprintf(&h[329], "%07o ", 0);
-
- /* Minor Dev 8 */
- sprintf(&h[337], "%07o ", 0);
-
- /* Prefix 155 - not used, leave as nulls */
-
- /*
- * We mustn't overwrite the next field while inserting the checksum.
- * Fortunately, the checksum can't exceed 6 octal digits, so we just write
- * 6 digits, a space, and a null, which is legal per POSIX.
- */
- sprintf(&h[148], "%06o ", _tarChecksum(h));
+ _tarCreateHeader(h, th->targetFile, NULL,
+ th->fileLen,
+ 0600, 04000, 02000,
+ time(NULL));
- /* Now write the completed header. */
if (fwrite(h, 1, 512, th->tarFH) != 512)
exit_horribly(modulename, "could not write to output file: %s\n", strerror(errno));
}
diff -durpN postgresql.2/src/include/port.h postgresql.3/src/include/port.h
--- postgresql.2/src/include/port.h 2012-11-22 10:17:15.255895802 +0100
+++ postgresql.3/src/include/port.h 2012-11-22 10:31:32.050052844 +0100
@@ -16,6 +16,7 @@
#include <ctype.h>
#include <netdb.h>
#include <pwd.h>
+#include <time.h>
/* port/init.c */
typedef void (*fe_func)(void);
@@ -491,4 +492,22 @@ extern void pg_free(void *ptr);
/* port/pgmkdirp.c */
extern int pg_mkdir_p(char *path, int omode);
+/* port/strutil.c */
+extern void print_val(char *s, uint64 val, unsigned int base, size_t len);
+extern void scan_val(char *s, uint64 *val, unsigned int base, size_t len);
+extern char *escape_quotes(const char *src);
+
+/* port/tarutil.c */
+
+/* Don't ever change this value, the TAR file format requires it. */
+#define TARCHUNKSZ 512
+
+extern int _tarChecksum(char *header);
+extern void _tarCreateHeader(char *header,
+ const char *filename,
+ const char *linktarget,
+ int64 filesize, int st_mode,
+ int st_uid, int st_gid,
+ time_t mtime);
+
#endif /* PG_PORT_H */
diff -durpN postgresql.2/src/port/Makefile postgresql.3/src/port/Makefile
--- postgresql.2/src/port/Makefile 2012-11-22 10:17:15.256895809 +0100
+++ postgresql.3/src/port/Makefile 2012-11-22 10:31:32.051052850 +0100
@@ -33,7 +33,7 @@ LIBS += $(PTHREAD_LIBS)
OBJS = $(LIBOBJS) chklocale.o dirmod.o erand48.o exec.o fls.o init.o \
inet_net_ntop.o noblock.o path.o pgcheckdir.o pg_crc.o pgmalloc.o \
pgmkdirp.o pgsleep.o pgstrcasecmp.o qsort.o qsort_arg.o sprompt.o \
- thread.o
+ strutil.o tarutil.o thread.o
# foo_srv.o and foo.o are both built from foo.c, but only foo.o has -DFRONTEND
OBJS_SRV = $(OBJS:%.o=%_srv.o)
diff -durpN postgresql.2/src/port/strutil.c postgresql.3/src/port/strutil.c
--- postgresql.2/src/port/strutil.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.3/src/port/strutil.c 2012-11-22 10:31:32.051052850 +0100
@@ -0,0 +1,87 @@
+/*-------------------------------------------------------------------------
+ *
+ * strutil.c
+ * common string functions
+ *
+ * IDENTIFICATION
+ * src/port/strutil.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FRONTEND
+#include "postgres.h"
+#include "utils/palloc.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+/*
+ * Utility routine to print possibly larger than 32 bit integers in a
+ * portable fashion. Filled with zeros.
+ */
+void
+print_val(char *s, uint64 val, unsigned int base, size_t len)
+{
+ int i;
+
+ for (i = len; i > 0; i--)
+ {
+ int digit = val % base;
+
+ s[i - 1] = '0' + digit;
+ val = val / base;
+ }
+}
+
+
+/*
+ * Inverse of print_val()
+ */
+void
+scan_val(char *s, uint64 *val, unsigned int base, size_t len)
+{
+ uint64 tmp = 0;
+ int i;
+
+ for (i = 0; i < len; i++)
+ {
+ int digit = s[i] - '0';
+
+ tmp = tmp * base + digit;
+ }
+
+ *val = tmp;
+}
+
+/*
+ * Escape (by doubling) any single quotes or backslashes in given string
+ *
+ * Note: this is used to process both postgresql.conf entries and SQL
+ * string literals. Since postgresql.conf strings are defined to treat
+ * backslashes as escapes, we have to double backslashes here. Hence,
+ * when using this for a SQL string literal, use E'' syntax.
+ *
+ * We do not need to worry about encoding considerations because all
+ * valid backend encodings are ASCII-safe.
+ */
+char *
+escape_quotes(const char *src)
+{
+ int len = strlen(src),
+ i,
+ j;
+#ifdef FRONTEND
+ char *result = pg_malloc(len * 2 + 1);
+#else
+ char *result = palloc(len * 2 + 1);
+#endif
+
+ for (i = 0, j = 0; i < len; i++)
+ {
+ if (SQL_STR_DOUBLE(src[i], true))
+ result[j++] = src[i];
+ result[j++] = src[i];
+ }
+ result[j] = '\0';
+ return result;
+}
diff -durpN postgresql.2/src/port/tarutil.c postgresql.3/src/port/tarutil.c
--- postgresql.2/src/port/tarutil.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.3/src/port/tarutil.c 2012-11-22 10:31:32.052052856 +0100
@@ -0,0 +1,135 @@
+/*-------------------------------------------------------------------------
+ *
+ * tarutil.c
+ * common functions for handling the TAR format
+ *
+ * IDENTIFICATION
+ * src/port/tarutil.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include <sys/stat.h>
+
+int
+_tarChecksum(char *header)
+{
+ int i,
+ sum;
+
+ /*
+ * Per POSIX, the checksum is the simple sum of all bytes in the header,
+ * treating the bytes as unsigned, and treating the checksum field (at
+ * offset 148) as though it contained 8 spaces.
+ */
+ sum = 8 * ' '; /* presumed value for checksum field */
+ for (i = 0; i < TARCHUNKSZ; i++)
+ if (i < 148 || i >= 156)
+ sum += 0xFF & header[i];
+ return sum;
+}
+
+
+void
+_tarCreateHeader(char *header, const char *filename,
+ const char *linktarget,
+ int64 filesize, int st_mode,
+ int st_uid, int st_gid,
+ time_t mtime)
+{
+ /*
+ * Note: most of the fields in a tar header are not supposed to be
+ * null-terminated. We use sprintf, which will write a null after the
+ * required bytes; that null goes into the first byte of the next field.
+ * This is okay as long as we fill the fields in order.
+ */
+ memset(header, 0, TARCHUNKSZ /* sizeof the tar header */);
+
+ /* Name 100 */
+ sprintf(&header[0], "%.99s", filename);
+ if (linktarget != NULL || S_ISDIR(st_mode))
+ {
+ /*
+ * We only support symbolic links to directories, and this is
+ * indicated in the tar format by adding a slash at the end of the
+ * name, the same as for regular directories.
+ */
+ int flen = strlen(filename);
+
+ flen = Min(flen, 99);
+ header[flen] = '/';
+ header[flen + 1] = '\0';
+ }
+
+ /* Mode 8 */
+ sprintf(&header[100], "%07o ", st_mode);
+
+ /* User ID 8 */
+ sprintf(&header[108], "%07o ", st_uid);
+
+ /* Group 8 */
+ sprintf(&header[116], "%07o ", st_gid);
+
+ /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
+ if (linktarget != NULL || S_ISDIR(st_mode))
+ /* Symbolic link or directory has size zero */
+ print_val(&header[124], 0, 8, 11);
+ else
+ print_val(&header[124], filesize, 8, 11);
+ sprintf(&header[135], " ");
+
+ /* Mod Time 12 - 11 digits, 1 space; use print_val for 64 bit support */
+ print_val(&header[136], mtime, 8, 11);
+ sprintf(&header[147], " ");
+
+ /* Checksum 8 cannot be calculated until we've filled all other fields */
+
+ if (linktarget != NULL)
+ {
+ /* Type - Symbolic link */
+ sprintf(&header[156], "2");
+ /* Link Name 100 */
+ sprintf(&header[157], "%.99s", linktarget);
+ }
+ else if (S_ISDIR(st_mode))
+ /* Type - directory */
+ sprintf(&header[156], "5");
+ else
+ /* Type - regular file */
+ sprintf(&header[156], "0");
+
+ /* Magic 6 */
+ sprintf(&header[257], "ustar");
+
+ /* Version 2 */
+ sprintf(&header[263], "00");
+
+ /* User 32 */
+ /* XXX: Do we need to care about setting correct username? */
+ sprintf(&header[265], "%.31s", "postgres");
+
+ /* Group 32 */
+ /* XXX: Do we need to care about setting correct group name? */
+ sprintf(&header[297], "%.31s", "postgres");
+
+ /* Major Dev 8 */
+ sprintf(&header[329], "%07o ", 0);
+
+ /* Minor Dev 8 */
+ sprintf(&header[337], "%07o ", 0);
+
+ /* Prefix 155 - not used, leave as nulls */
+
+ /*
+ * We mustn't overwrite the next field while inserting the checksum.
+ * Fortunately, the checksum can't exceed 6 octal digits, so we just write
+ * 6 digits, a space, and a null, which is legal per POSIX.
+ */
+ sprintf(&header[148], "%06o ", _tarChecksum(header));
+}
+
04-pg_basebackup-v17.patchtext/x-patch; name=04-pg_basebackup-v17.patchDownload
diff -durpN postgresql.3/doc/src/sgml/ref/pg_basebackup.sgml postgresql.4/doc/src/sgml/ref/pg_basebackup.sgml
--- postgresql.3/doc/src/sgml/ref/pg_basebackup.sgml 2012-11-08 13:13:04.151630632 +0100
+++ postgresql.4/doc/src/sgml/ref/pg_basebackup.sgml 2012-11-22 10:44:09.232437587 +0100
@@ -189,6 +189,21 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--write-recovery-conf</option></term>
+ <listitem>
+
+ <para>
+ Write a minimal recovery.conf into the output directory (or into
+ the base archive file if <option>--format=tar</option> was specified)
+ using the connection parameters from the command line to ease
+ setting up the standby.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-x</option></term>
<term><option>--xlog</option></term>
<listitem>
diff -durpN postgresql.3/src/bin/pg_basebackup/pg_basebackup.c postgresql.4/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.3/src/bin/pg_basebackup/pg_basebackup.c 2012-11-22 10:27:33.854681991 +0100
+++ postgresql.4/src/bin/pg_basebackup/pg_basebackup.c 2012-11-22 10:57:35.291135926 +0100
@@ -19,6 +19,7 @@
#define FRONTEND 1
#include "postgres.h"
#include "libpq-fe.h"
+#include "pqexpbuffer.h"
#include <unistd.h>
#include <dirent.h>
@@ -46,6 +47,7 @@ int compresslevel = 0;
bool includewal = false;
bool streamwal = false;
bool fastcheckpoint = false;
+bool writerecoveryconf = false;
int standby_message_timeout = 10 * 1000; /* 10 sec = default */
/* Progress counters */
@@ -70,6 +72,8 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
+PQExpBuffer rcExpBuf = NULL;
+
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
@@ -77,6 +81,8 @@ static void progress_report(int tablespa
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
+static void CreateRecoveryConf(PGconn *conn);
+static void WriteRecoveryConf(void);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -107,6 +113,8 @@ usage(void)
printf(_("\nOptions controlling the output:\n"));
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
+ printf(_(" -R, --write-recovery-conf\n"
+ " write recovery.conf after backup\n"));
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
printf(_(" -X, --xlog-method=fetch|stream\n"
" include required WAL files with specified method\n"));
@@ -452,6 +460,45 @@ progress_report(int tablespacenum, const
/*
+ * Write a piece of tar data
+ */
+static void
+writeTarData(
+#ifdef HAVE_LIBZ
+ gzFile ztarfile,
+#endif
+ FILE *tarfile, char *buf, int r, char *current_file)
+{
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (gzwrite(ztarfile, buf, r) != r)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, current_file, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (fwrite(buf, r, 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
+ progname, current_file, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ }
+}
+
+#ifdef HAVE_LIBZ
+#define WRITE_TAR_DATA(buf, sz) writeTarData(ztarfile, tarfile, buf, sz, filename)
+#else
+#define WRITE_TAR_DATA(buf, sz) writeTarData(tarfile, buf, sz, filename)
+#endif
+
+/*
* Receive a tar format file from the connection to the server, and write
* the data from this file directly into a tar file. If compression is
* enabled, the data will be compressed while written to the file.
@@ -467,12 +514,17 @@ ReceiveTarFile(PGconn *conn, PGresult *r
char filename[MAXPGPATH];
char *copybuf = NULL;
FILE *tarfile = NULL;
+ char tarhdr[TARCHUNKSZ];
+ bool basetablespace = PQgetisnull(res, rownum, 0);
+ bool in_tarhdr, skip_file;
+ size_t tarhdrsz;
+ uint64 filesz;
#ifdef HAVE_LIBZ
gzFile ztarfile = NULL;
#endif
- if (PQgetisnull(res, rownum, 0))
+ if (basetablespace)
{
/*
* Base tablespaces
@@ -584,6 +636,16 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
+ /*
+ * Initialize our variables for tracking
+ * individual files inside the TAR stream.
+ * For more detailed explanation, see below.
+ */
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = 0;
+ filesz = 0;
+
while (1)
{
int r;
@@ -598,38 +660,39 @@ ReceiveTarFile(PGconn *conn, PGresult *r
if (r == -1)
{
/*
- * End of chunk. Close file (but not stdout).
+ * End of chunk. Write recovery.conf into the tar file (if it
+ * was requested) and close file (but not stdout).
*
* Also, write two completely empty blocks at the end of the tar
* file, as required by some tar programs.
*/
- char zerobuf[1024];
+ char zerobuf[2*TARCHUNKSZ];
MemSet(zerobuf, 0, sizeof(zerobuf));
-#ifdef HAVE_LIBZ
- if (ztarfile != NULL)
- {
- if (gzwrite(ztarfile, zerobuf, sizeof(zerobuf)) !=
- sizeof(zerobuf))
- {
- fprintf(stderr,
- _("%s: could not write to compressed file \"%s\": %s\n"),
- progname, filename, get_gz_error(ztarfile));
- disconnect_and_exit(1);
- }
- }
- else
-#endif
+
+ if (basetablespace && writerecoveryconf)
{
- if (fwrite(zerobuf, sizeof(zerobuf), 1, tarfile) != 1)
- {
- fprintf(stderr,
- _("%s: could not write to file \"%s\": %s\n"),
- progname, filename, strerror(errno));
- disconnect_and_exit(1);
- }
+ char header[TARCHUNKSZ];
+ int padding;
+
+ _tarCreateHeader(header, "recovery.conf", NULL,
+ rcExpBuf->len,
+ 0600, 04000, 02000,
+ time(NULL));
+
+ padding = ((rcExpBuf->len + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - rcExpBuf->len;
+
+ WRITE_TAR_DATA(header, sizeof(header));
+ WRITE_TAR_DATA(rcExpBuf->data, rcExpBuf->len);
+ if (padding)
+ WRITE_TAR_DATA(zerobuf, padding);
+
+ if (verbose)
+ fprintf(stderr, _("%s: recovery.conf written into '%s'\n"), progname, filename);
}
+ WRITE_TAR_DATA(zerobuf, sizeof(zerobuf));
+
#ifdef HAVE_LIBZ
if (ztarfile != NULL)
{
@@ -665,25 +728,124 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
-#ifdef HAVE_LIBZ
- if (ztarfile != NULL)
+ if (!writerecoveryconf || !basetablespace)
{
- if (gzwrite(ztarfile, copybuf, r) != r)
- {
- fprintf(stderr,
- _("%s: could not write to compressed file \"%s\": %s\n"),
- progname, filename, get_gz_error(ztarfile));
- disconnect_and_exit(1);
- }
+ /*
+ * If --write-recovery-conf was not requested or this
+ * is not the base tablespace, simply pass the received
+ * data into the TAR file, either compressed or not.
+ */
+
+ WRITE_TAR_DATA(copybuf, r);
}
else
-#endif
{
- if (fwrite(copybuf, r, 1, tarfile) != 1)
+ /*
+ * If --write-recovery-conf was requested AND this
+ * is the base tablespace, the TAR stream may contain
+ * a recovery.conf file if the backup is coming from
+ * a standby server. We have to skip this file in
+ * the stream and add a new one constructed by
+ * CreateRecoveryConf() at the end of the stream.
+ *
+ * To do this, we have to process the individual files
+ * inside the TAR stream. The stream consists of a header
+ * and zero or more chunks, all 512 bytes long. The stream
+ * from the server is broken up into smaller pieces, so
+ * we have to track the size of the files to find the next
+ * header structure.
+ */
+ int rr = r; /* Save the value returned by PQgetCopyData */
+ int pos = 0;
+
+ while (rr > 0)
{
- fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
- progname, filename, strerror(errno));
- disconnect_and_exit(1);
+ if (in_tarhdr)
+ {
+ /*
+ * We're currently reading a header structure
+ * inside the TAR stream, i.e. the file metadata.
+ */
+ if (tarhdrsz < TARCHUNKSZ)
+ {
+ /*
+ * Copy the header structure into tarhdr[]
+ * in case the header is not aligned to 512 bytes
+ * or it's not returned in whole by the last
+ * PQgetCopyData call.
+ */
+ int hdrleft, bytes2copy;
+
+ hdrleft = TARCHUNKSZ - tarhdrsz;
+ bytes2copy = (rr > hdrleft ? hdrleft : rr);
+
+ memcpy(&tarhdr[tarhdrsz], copybuf + pos, bytes2copy);
+
+ rr -= bytes2copy;
+ pos += bytes2copy;
+ tarhdrsz += bytes2copy;
+ }
+ else
+ {
+ /*
+ * We have the whole header structure in tarhdr[],
+ * look at the file metadata:
+ * - the subsequent file contents have to be skipped
+ * if the filename is recovery.conf
+ * - find out the size of the file padded to the next
+ * multiple of 512
+ */
+ int64 padding;
+
+ skip_file = (strcmp(&tarhdr[0], "recovery.conf") == 0);
+
+ scan_val(&tarhdr[124], &filesz, 8, 11);
+
+ padding = ((filesz + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - filesz;
+ filesz += padding;
+
+ /* Indicate that the subsequent data is the file content. */
+ in_tarhdr = false;
+
+ if (!skip_file)
+ WRITE_TAR_DATA(tarhdr, TARCHUNKSZ);
+ }
+ }
+ else
+ {
+ /*
+ * We're processing a file's contents.
+ */
+ if (filesz > 0)
+ {
+ /*
+ * We still have data to read (and possibly write).
+ */
+ int bytes2write;
+
+ bytes2write = (filesz > rr ? rr : filesz);
+
+ if (!skip_file)
+ WRITE_TAR_DATA(copybuf + pos, bytes2write);
+
+ rr -= bytes2write;
+ pos += bytes2write;
+ filesz -= bytes2write;
+ }
+ else
+ {
+ /*
+ * No more data in the current file,
+ * the next piece of data (if any) will
+ * be a new file header structure.
+ * Reinitialize all our variables.
+ */
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = 0;
+ filesz = 0;
+ }
+ }
}
}
totaldone += r;
@@ -712,10 +874,11 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
char filename[MAXPGPATH];
int current_len_left;
int current_padding = 0;
+ bool basetablespace = PQgetisnull(res, rownum, 0);
char *copybuf = NULL;
FILE *file = NULL;
- if (PQgetisnull(res, rownum, 0))
+ if (basetablespace)
strcpy(current_path, basedir);
else
strcpy(current_path, PQgetvalue(res, rownum, 1));
@@ -767,13 +930,13 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
/*
* No current file, so this must be the header for a new file
*/
- if (r != 512)
+ if (r != TARCHUNKSZ)
{
fprintf(stderr, _("%s: invalid tar block header size: %d\n"),
progname, r);
disconnect_and_exit(1);
}
- totaldone += 512;
+ totaldone += TARCHUNKSZ;
if (sscanf(copybuf + 124, "%11o", ¤t_len_left) != 1)
{
@@ -794,7 +957,7 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
* All files are padded up to 512 bytes
*/
current_padding =
- ((current_len_left + 511) & ~511) - current_len_left;
+ ((current_len_left + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - current_len_left;
/*
* First part of header is zero terminated filename
@@ -937,6 +1100,112 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
if (copybuf != NULL)
PQfreemem(copybuf);
+
+ if (basetablespace)
+ WriteRecoveryConf();
+}
+
+/*
+ * Create the new recovery.conf in memory
+ */
+static void
+CreateRecoveryConf(PGconn *conn)
+{
+ PQconninfoOption *connOptions;
+ PQconninfoOption *option;
+
+ if (!writerecoveryconf)
+ return;
+
+ connOptions = PQconninfo(conn, PG_CONNINFO_REPLICATION);
+ if (connOptions == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "standby_mode = 'on'\n");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "primary_conninfo = '");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ for (option = connOptions; option && option->keyword; option++)
+ {
+ char *escaped;
+
+ /*
+ * Do not emit this setting if not set or empty.
+ */
+ if ((option->val == NULL) ||
+ (option->val != NULL && option->val[0] == '\0'))
+ continue;
+
+ /*
+ * Write "keyword='value'" pieces, the value string is escaped
+ * if necessary and doubled single quotes around the value string.
+ */
+ escaped = escape_quotes(option->val);
+
+ appendPQExpBuffer(rcExpBuf, "%s=''%s'' ", option->keyword, escaped);
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ free(escaped);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "'\n");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ PQconninfoFree(connOptions);
+}
+
+
+static void
+WriteRecoveryConf(void)
+{
+ char filename[MAXPGPATH];
+ FILE *cf;
+
+ if (!writerecoveryconf)
+ return;
+
+ sprintf(filename, "%s/recovery.conf", basedir);
+
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("%s: cannot create %s: %s"), progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ if (fwrite(rcExpBuf->data, rcExpBuf->len, 1, cf) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ fclose(cf);
+
+ if (verbose)
+ fprintf(stderr, _("%s: recovery.conf written.\n"), progname);
}
@@ -960,6 +1229,15 @@ BaseBackup(void)
/* Error message already written in GetConnection() */
exit(1);
+ rcExpBuf = createPQExpBuffer();
+ if (!rcExpBuf)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ CreateRecoveryConf(conn);
+
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
@@ -1223,6 +1501,9 @@ BaseBackup(void)
#endif
}
+ /* Free the recovery.conf contents */
+ destroyPQExpBuffer(rcExpBuf);
+
/*
* End of copy data. Final result is already checked inside the loop.
*/
@@ -1266,6 +1547,7 @@ main(int argc, char **argv)
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
+ {"write-recovery-conf", no_argument, NULL, 'R'},
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
{"gzip", no_argument, NULL, 'z'},
@@ -1305,7 +1587,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:xX:l:zZ:c:h:p:U:s:wWvP",
+ while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
long_options, &option_index)) != -1)
{
switch (c)
@@ -1326,6 +1608,9 @@ main(int argc, char **argv)
exit(1);
}
break;
+ case 'R':
+ writerecoveryconf = true;
+ break;
case 'x':
if (includewal)
{
2012-11-22 11:06 keltezéssel, Boszormenyi Zoltan írta:
Hi,
2012-11-21 18:39 keltezéssel, Boszormenyi Zoltan írta:
2012-11-21 15:29 keltezéssel, Tom Lane írta:
Boszormenyi Zoltan <zb@cybertec.at> writes:
pg_basebackup needs pg_malloc() to call disconnect_and_exit(1)
instead to quit cleanly and not leave an "unexpected EOF from client"
message in the server log. Which is a macro at the moment, but
has to be turned into a real function for the reasons below.man 2 atexit
It doesn't really help to make it easier. E.g: pg_strdup() in psql does this:
char *
pg_strdup(const char *string)
{
char *tmp;if (!string)
{
psql_error("%s: pg_strdup: cannot duplicate null pointer (internal
error)\n",
pset.progname);
exit(EXIT_FAILURE);
}
tmp = strdup(string);
if (!tmp)
{
psql_error("out of memory\n");
exit(EXIT_FAILURE);
}
return tmp;
}The function passed to atexit() still needs to know which string to print
and this needs an internal variable. Also, the actions taken on pg_malloc/pg_strdup
errors don't produce the exact same messages.This is why I suggest the attached 03-pg_malloc-unified-v2.patch.
The 04-tarutils-v2.patch unifies the common tar functions and the string functions
like print_val() used by _tarCreateHeader (previously _tarWriteHeader) and
escape_quotes().The PQconninfo patch grew its own mail thread (this is why
there's no 1st patch in this series now) and I reworked the order
of the patches to make it more logical. This made the pg_basebackup
patch 5K smaller.
The changes here are:
02-pg_malloc-unified-v4.1.patch: fixed crashing PQfinish() at exit time
in pg_basebackup, added the atexit() initializer to pg_receivexlog
03-tarutils-strutils-v3.1.patch: rebased to avoid fuzz in patching
04-pg_basebackup-v18.1.patch: use Magnus' PQconninfo patch
and filter the PQconninfoOption array internally, also rebased
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
02-pg_malloc-unified-v4.1.patchtext/x-patch; name=02-pg_malloc-unified-v4.1.patchDownload
diff -durpN postgresql.1/contrib/oid2name/oid2name.c postgresql.2/contrib/oid2name/oid2name.c
--- postgresql.1/contrib/oid2name/oid2name.c 2012-10-03 10:40:48.241207023 +0200
+++ postgresql.2/contrib/oid2name/oid2name.c 2012-11-22 14:55:52.194178400 +0100
@@ -50,9 +50,6 @@ struct options
/* function prototypes */
static void help(const char *progname);
void get_opts(int, char **, struct options *);
-void *pg_malloc(size_t size);
-void *pg_realloc(void *ptr, size_t size);
-char *pg_strdup(const char *str);
void add_one_elt(char *eltname, eary *eary);
char *get_comma_elts(eary *eary);
PGconn *sql_conn(struct options *);
@@ -201,53 +198,6 @@ help(const char *progname)
progname, progname);
}
-void *
-pg_malloc(size_t size)
-{
- void *ptr;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- ptr = malloc(size);
- if (!ptr)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return ptr;
-}
-
-void *
-pg_realloc(void *ptr, size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- result = realloc(ptr, size);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-char *
-pg_strdup(const char *str)
-{
- char *result = strdup(str);
-
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
/*
* add_one_elt
*
@@ -598,6 +548,8 @@ main(int argc, char **argv)
struct options *my_opts;
PGconn *pgconn;
+ InitPostgresFrontend(NULL);
+
my_opts = (struct options *) pg_malloc(sizeof(struct options));
my_opts->oids = (eary *) pg_malloc(sizeof(eary));
diff -durpN postgresql.1/contrib/pgbench/pgbench.c postgresql.2/contrib/pgbench/pgbench.c
--- postgresql.1/contrib/pgbench/pgbench.c 2012-10-21 10:56:15.358945296 +0200
+++ postgresql.2/contrib/pgbench/pgbench.c 2012-11-22 14:55:52.195178407 +0100
@@ -296,58 +296,6 @@ static void setalarm(int seconds);
static void *threadRun(void *arg);
-/*
- * routines to check mem allocations and fail noisily.
- */
-static void *
-pg_malloc(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-static void *
-pg_realloc(void *ptr, size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- result = realloc(ptr, size);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-static char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-
static void
usage(void)
{
@@ -1956,6 +1904,8 @@ main(int argc, char **argv)
char val[64];
+ InitPostgresFrontend(NULL);
+
progname = get_progname(argv[0]);
if (argc > 1)
diff -durpN postgresql.1/contrib/pg_upgrade/pg_upgrade.c postgresql.2/contrib/pg_upgrade/pg_upgrade.c
--- postgresql.1/contrib/pg_upgrade/pg_upgrade.c 2012-09-04 11:02:05.935828363 +0200
+++ postgresql.2/contrib/pg_upgrade/pg_upgrade.c 2012-11-22 14:55:52.196178414 +0100
@@ -67,6 +67,21 @@ char *output_files[] = {
NULL
};
+static void
+pg_upgrade_malloc_oom(void)
+{
+ pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
+}
+
+static void
+pg_upgrade_strdup_null(void)
+{
+ pg_log(PG_FATAL, "%s: pg_strdup: cannot duplicate null pointer (internal error)\n", os_info.progname);
+}
+
+static PostgresFrontendData pg_upgrade_fe_data = {
+ pg_upgrade_malloc_oom, pg_upgrade_strdup_null
+};
int
main(int argc, char **argv)
@@ -76,6 +91,8 @@ main(int argc, char **argv)
char *deletion_script_file_name = NULL;
bool live_check = false;
+ InitPostgresFrontend(&pg_upgrade_fe_data);
+
parseCommandLine(argc, argv);
adjust_data_dir(&old_cluster);
diff -durpN postgresql.1/contrib/pg_upgrade/pg_upgrade.h postgresql.2/contrib/pg_upgrade/pg_upgrade.h
--- postgresql.1/contrib/pg_upgrade/pg_upgrade.h 2012-11-15 13:39:47.132060180 +0100
+++ postgresql.2/contrib/pg_upgrade/pg_upgrade.h 2012-11-22 14:55:52.197178420 +0100
@@ -437,10 +437,6 @@ void
prep_status(const char *fmt,...)
__attribute__((format(PG_PRINTF_ATTRIBUTE, 1, 2)));
void check_ok(void);
-char *pg_strdup(const char *s);
-void *pg_malloc(size_t size);
-void *pg_realloc(void *ptr, size_t size);
-void pg_free(void *ptr);
const char *getErrorText(int errNum);
unsigned int str2uint(const char *str);
void pg_putenv(const char *var, const char *val);
diff -durpN postgresql.1/contrib/pg_upgrade/util.c postgresql.2/contrib/pg_upgrade/util.c
--- postgresql.1/contrib/pg_upgrade/util.c 2012-10-03 10:40:48.243207034 +0200
+++ postgresql.2/contrib/pg_upgrade/util.c 2012-11-22 14:55:52.197178420 +0100
@@ -191,55 +191,6 @@ get_user_info(char **user_name)
}
-void *
-pg_malloc(size_t size)
-{
- void *p;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- p = malloc(size);
- if (p == NULL)
- pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
- return p;
-}
-
-void *
-pg_realloc(void *ptr, size_t size)
-{
- void *p;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- p = realloc(ptr, size);
- if (p == NULL)
- pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
- return p;
-}
-
-
-void
-pg_free(void *ptr)
-{
- if (ptr != NULL)
- free(ptr);
-}
-
-
-char *
-pg_strdup(const char *s)
-{
- char *result = strdup(s);
-
- if (result == NULL)
- pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
-
- return result;
-}
-
-
/*
* getErrorText()
*
diff -durpN postgresql.1/src/bin/initdb/initdb.c postgresql.2/src/bin/initdb/initdb.c
--- postgresql.1/src/bin/initdb/initdb.c 2012-10-15 10:44:54.640429361 +0200
+++ postgresql.2/src/bin/initdb/initdb.c 2012-11-22 14:55:52.198178426 +0100
@@ -177,8 +177,6 @@ static const char *backend_options = "--
static char bin_path[MAXPGPATH];
static char backend_exec[MAXPGPATH];
-static void *pg_malloc(size_t size);
-static char *pg_strdup(const char *s);
static char **replace_token(char **lines,
const char *token, const char *replacement);
@@ -284,43 +282,6 @@ do { \
#endif
/*
- * routines to check mem allocations and fail noisily.
- *
- * Note that we can't call exit_nicely() on a memory failure, as it calls
- * rmtree() which needs memory allocation. So we just exit with a bang.
- */
-static void *
-pg_malloc(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-static char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-/*
* make a copy of the array of lines, with token replaced by replacement
* the first time it occurs on each line.
*
@@ -2823,6 +2784,22 @@ check_need_password(const char *authmeth
}
}
+static void
+initdb_malloc_oom(void)
+{
+ fprintf(stderr, _("%s: out of memory\n"), progname);
+}
+
+static void
+initdb_strdup_null(void)
+{
+ fprintf(stderr, _("%s: pg_strdup: cannot duplicate null pointer (internal error)\n"), progname);
+}
+
+static PostgresFrontendData initdb_fe_data = {
+ initdb_malloc_oom, initdb_strdup_null
+};
+
int
main(int argc, char *argv[])
{
@@ -2890,6 +2867,8 @@ main(int argc, char *argv[])
"pg_stat_tmp"
};
+ InitPostgresFrontend(&initdb_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("initdb"));
diff -durpN postgresql.1/src/bin/pg_basebackup/pg_basebackup.c postgresql.2/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.1/src/bin/pg_basebackup/pg_basebackup.c 2012-10-03 10:40:48.297207389 +0200
+++ postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2012-11-22 14:55:52.199178432 +0100
@@ -1262,6 +1262,8 @@ main(int argc, char **argv)
int option_index;
+ InitPostgresFrontend(&streamutil_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_basebackup"));
diff -durpN postgresql.1/src/bin/pg_basebackup/pg_receivexlog.c postgresql.2/src/bin/pg_basebackup/pg_receivexlog.c
--- postgresql.1/src/bin/pg_basebackup/pg_receivexlog.c 2012-10-03 10:40:48.298207395 +0200
+++ postgresql.2/src/bin/pg_basebackup/pg_receivexlog.c 2012-11-22 14:55:52.199178432 +0100
@@ -283,6 +283,7 @@ StreamLog(void)
stop_streaming, standby_message_timeout, false);
PQfinish(conn);
+ conn = NULL; /* don't crash on the atexit function */
}
/*
@@ -318,6 +319,8 @@ main(int argc, char **argv)
int c;
int option_index;
+ InitPostgresFrontend(&streamutil_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_receivexlog"));
diff -durpN postgresql.1/src/bin/pg_basebackup/streamutil.c postgresql.2/src/bin/pg_basebackup/streamutil.c
--- postgresql.1/src/bin/pg_basebackup/streamutil.c 2012-11-22 14:55:35.345073179 +0100
+++ postgresql.2/src/bin/pg_basebackup/streamutil.c 2012-11-22 14:55:52.199178432 +0100
@@ -31,42 +31,29 @@ int dbgetpassword = 0; /* 0=auto, -1=n
static char *dbpassword = NULL;
PGconn *conn = NULL;
-/*
- * strdup() and malloc() replacements that print an error and exit
- * if something goes wrong. Can never return NULL.
- */
-char *
-pg_strdup(const char *s)
+static void
+streamutil_malloc_oom(void)
{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
+ fprintf(stderr, _("%s: out of memory\n"), progname);
}
-void *
-pg_malloc0(size_t size)
+static void
+streamutil_strdup_null(void)
{
- void *result;
+ fprintf(stderr, _("%s: pg_strdup: cannot duplicate null pointer (internal error)\n"), progname);
+}
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- MemSet(result, 0, size);
- return result;
+static void
+streamutil_app_atexit(void)
+{
+ if (conn != NULL)
+ PQfinish(conn);
}
+PostgresFrontendData streamutil_fe_data = {
+ streamutil_malloc_oom, streamutil_strdup_null, streamutil_app_atexit
+};
+
/*
* Connect to the server. Returns a valid PGconn pointer if connected,
diff -durpN postgresql.1/src/bin/pg_basebackup/streamutil.h postgresql.2/src/bin/pg_basebackup/streamutil.h
--- postgresql.1/src/bin/pg_basebackup/streamutil.h 2012-10-03 10:40:48.299207401 +0200
+++ postgresql.2/src/bin/pg_basebackup/streamutil.h 2012-11-22 14:55:52.199178432 +0100
@@ -15,8 +15,6 @@ extern PGconn *conn;
exit(code); \
}
-
-extern char *pg_strdup(const char *s);
-extern void *pg_malloc0(size_t size);
+extern PostgresFrontendData streamutil_fe_data;
extern PGconn *GetConnection(void);
diff -durpN postgresql.1/src/bin/pg_ctl/pg_ctl.c postgresql.2/src/bin/pg_ctl/pg_ctl.c
--- postgresql.1/src/bin/pg_ctl/pg_ctl.c 2012-10-19 19:12:00.307691010 +0200
+++ postgresql.2/src/bin/pg_ctl/pg_ctl.c 2012-11-22 14:55:52.200178439 +0100
@@ -118,8 +118,6 @@ write_stderr(const char *fmt,...)
/* This extension allows gcc to check the format string for consistency with
the supplied arguments. */
__attribute__((format(PG_PRINTF_ATTRIBUTE, 1, 2)));
-static void *pg_malloc(size_t size);
-static char *pg_strdup(const char *s);
static void do_advice(void);
static void do_help(void);
static void set_mode(char *modeopt);
@@ -226,42 +224,6 @@ write_stderr(const char *fmt,...)
}
/*
- * routines to check memory allocations and fail noisily.
- */
-
-static void *
-pg_malloc(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- write_stderr(_("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-
-static char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- write_stderr(_("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-/*
* Given an already-localized string, print it to stdout unless the
* user has specified that no messages should be printed.
*/
@@ -1994,6 +1956,22 @@ adjust_data_dir(void)
}
+static void
+pgctl_malloc_oom(void)
+{
+ write_stderr(_("%s: out of memory\n"), progname);
+}
+
+static void
+pgctl_strdup_null(void)
+{
+ write_stderr(_("%s: pg_strdup: cannot duplicate null pointer (internal error)\n"), progname);
+}
+
+static PostgresFrontendData pgctl_fe_data = {
+ pgctl_malloc_oom, pgctl_strdup_null
+};
+
int
main(int argc, char **argv)
{
@@ -2013,6 +1991,8 @@ main(int argc, char **argv)
int c;
pgpid_t killproc = 0;
+ InitPostgresFrontend(&pgctl_fe_data);
+
#if defined(WIN32) || defined(__CYGWIN__)
setvbuf(stderr, NULL, _IONBF, 0);
#endif
diff -durpN postgresql.1/src/bin/pg_dump/common.c postgresql.2/src/bin/pg_dump/common.c
--- postgresql.1/src/bin/pg_dump/common.c 2012-07-22 16:48:48.536857822 +0200
+++ postgresql.2/src/bin/pg_dump/common.c 2012-11-22 14:55:52.200178439 +0100
@@ -18,7 +18,6 @@
#include <ctype.h>
#include "catalog/pg_class.h"
-#include "dumpmem.h"
#include "dumputils.h"
diff -durpN postgresql.1/src/bin/pg_dump/compress_io.c postgresql.2/src/bin/pg_dump/compress_io.c
--- postgresql.1/src/bin/pg_dump/compress_io.c 2012-10-03 10:40:48.301207415 +0200
+++ postgresql.2/src/bin/pg_dump/compress_io.c 2012-11-22 14:55:52.201178446 +0100
@@ -53,7 +53,6 @@
*/
#include "compress_io.h"
-#include "dumpmem.h"
#include "dumputils.h"
/*----------------------
diff -durpN postgresql.1/src/bin/pg_dump/dumpmem.c postgresql.2/src/bin/pg_dump/dumpmem.c
--- postgresql.1/src/bin/pg_dump/dumpmem.c 2012-10-03 10:40:48.301207415 +0200
+++ postgresql.2/src/bin/pg_dump/dumpmem.c 1970-01-01 01:00:00.000000000 +0100
@@ -1,76 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * dumpmem.c
- * Memory allocation routines used by pg_dump, pg_dumpall, and pg_restore
- *
- * Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
- *
- *
- * IDENTIFICATION
- * src/bin/pg_dump/dumpmem.c
- *
- *-------------------------------------------------------------------------
- */
-#include "postgres_fe.h"
-
-#include "dumputils.h"
-#include "dumpmem.h"
-
-
-/*
- * Safer versions of some standard C library functions. If an
- * out-of-memory condition occurs, these functions will bail out via exit();
- *therefore, their return value is guaranteed to be non-NULL.
- */
-
-char *
-pg_strdup(const char *string)
-{
- char *tmp;
-
- if (!string)
- exit_horribly(NULL, "cannot duplicate null pointer\n");
- tmp = strdup(string);
- if (!tmp)
- exit_horribly(NULL, "out of memory\n");
- return tmp;
-}
-
-void *
-pg_malloc(size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- tmp = malloc(size);
- if (!tmp)
- exit_horribly(NULL, "out of memory\n");
- return tmp;
-}
-
-void *
-pg_malloc0(size_t size)
-{
- void *tmp;
-
- tmp = pg_malloc(size);
- MemSet(tmp, 0, size);
- return tmp;
-}
-
-void *
-pg_realloc(void *ptr, size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- tmp = realloc(ptr, size);
- if (!tmp)
- exit_horribly(NULL, "out of memory\n");
- return tmp;
-}
diff -durpN postgresql.1/src/bin/pg_dump/dumpmem.h postgresql.2/src/bin/pg_dump/dumpmem.h
--- postgresql.1/src/bin/pg_dump/dumpmem.h 2012-10-03 10:40:48.301207415 +0200
+++ postgresql.2/src/bin/pg_dump/dumpmem.h 1970-01-01 01:00:00.000000000 +0100
@@ -1,22 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * dumpmem.h
- * Memory allocation routines used by pg_dump, pg_dumpall, and pg_restore
- *
- * Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
- *
- * src/bin/pg_dump/dumpmem.h
- *
- *-------------------------------------------------------------------------
- */
-
-#ifndef DUMPMEM_H
-#define DUMPMEM_H
-
-extern char *pg_strdup(const char *string);
-extern void *pg_malloc(size_t size);
-extern void *pg_malloc0(size_t size);
-extern void *pg_realloc(void *ptr, size_t size);
-
-#endif /* DUMPMEM_H */
diff -durpN postgresql.1/src/bin/pg_dump/Makefile postgresql.2/src/bin/pg_dump/Makefile
--- postgresql.1/src/bin/pg_dump/Makefile 2012-04-16 19:57:22.595917312 +0200
+++ postgresql.2/src/bin/pg_dump/Makefile 2012-11-22 14:55:52.203178458 +0100
@@ -20,7 +20,7 @@ override CPPFLAGS := -I$(libpq_srcdir) $
OBJS= pg_backup_archiver.o pg_backup_db.o pg_backup_custom.o \
pg_backup_null.o pg_backup_tar.o \
- pg_backup_directory.o dumpmem.o dumputils.o compress_io.o $(WIN32RES)
+ pg_backup_directory.o dumputils.o compress_io.o $(WIN32RES)
KEYWRDOBJS = keywords.o kwlookup.o
@@ -35,8 +35,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort
pg_restore: pg_restore.o $(OBJS) $(KEYWRDOBJS) | submake-libpq submake-libpgport
$(CC) $(CFLAGS) pg_restore.o $(KEYWRDOBJS) $(OBJS) $(libpq_pgport) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
-pg_dumpall: pg_dumpall.o dumputils.o dumpmem.o $(KEYWRDOBJS) | submake-libpq submake-libpgport
- $(CC) $(CFLAGS) pg_dumpall.o dumputils.o dumpmem.o $(KEYWRDOBJS) $(WIN32RES) $(libpq_pgport) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o $(KEYWRDOBJS) | submake-libpq submake-libpgport
+ $(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(KEYWRDOBJS) $(WIN32RES) $(libpq_pgport) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
install: all installdirs
$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff -durpN postgresql.1/src/bin/pg_dump/nls.mk postgresql.2/src/bin/pg_dump/nls.mk
--- postgresql.1/src/bin/pg_dump/nls.mk 2012-04-16 19:57:22.596917323 +0200
+++ postgresql.2/src/bin/pg_dump/nls.mk 2012-11-22 14:55:52.203178458 +0100
@@ -3,7 +3,7 @@ CATALOG_NAME = pg_dump
AVAIL_LANGUAGES = de es fr it ja ko pt_BR sv tr zh_CN zh_TW
GETTEXT_FILES = pg_backup_archiver.c pg_backup_db.c pg_backup_custom.c \
pg_backup_null.c pg_backup_tar.c \
- pg_backup_directory.c dumpmem.c dumputils.c compress_io.c \
+ pg_backup_directory.c dumputils.c compress_io.c \
pg_dump.c common.c pg_dump_sort.c \
pg_restore.c pg_dumpall.c \
../../port/exec.c
diff -durpN postgresql.1/src/bin/pg_dump/pg_backup_archiver.c postgresql.2/src/bin/pg_dump/pg_backup_archiver.c
--- postgresql.1/src/bin/pg_dump/pg_backup_archiver.c 2012-10-21 10:56:15.398945610 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_archiver.c 2012-11-22 14:55:52.206178477 +0100
@@ -21,7 +21,6 @@
*/
#include "pg_backup_db.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <ctype.h>
diff -durpN postgresql.1/src/bin/pg_dump/pg_backup_custom.c postgresql.2/src/bin/pg_dump/pg_backup_custom.c
--- postgresql.1/src/bin/pg_dump/pg_backup_custom.c 2012-10-03 10:40:48.303207429 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_custom.c 2012-11-22 14:55:52.207178484 +0100
@@ -26,7 +26,6 @@
#include "compress_io.h"
#include "dumputils.h"
-#include "dumpmem.h"
/*--------
* Routines in the format interface
diff -durpN postgresql.1/src/bin/pg_dump/pg_backup_db.c postgresql.2/src/bin/pg_dump/pg_backup_db.c
--- postgresql.1/src/bin/pg_dump/pg_backup_db.c 2012-07-25 10:51:14.973507544 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_db.c 2012-11-22 14:55:52.207178484 +0100
@@ -11,7 +11,6 @@
*/
#include "pg_backup_db.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <unistd.h>
diff -durpN postgresql.1/src/bin/pg_dump/pg_backup_directory.c postgresql.2/src/bin/pg_dump/pg_backup_directory.c
--- postgresql.1/src/bin/pg_dump/pg_backup_directory.c 2012-10-03 10:40:48.303207429 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_directory.c 2012-11-22 14:55:52.208178490 +0100
@@ -34,7 +34,6 @@
*/
#include "compress_io.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <dirent.h>
diff -durpN postgresql.1/src/bin/pg_dump/pg_backup_null.c postgresql.2/src/bin/pg_dump/pg_backup_null.c
--- postgresql.1/src/bin/pg_dump/pg_backup_null.c 2012-04-16 19:57:22.598917347 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_null.c 2012-11-22 14:55:52.209178496 +0100
@@ -23,7 +23,6 @@
*/
#include "pg_backup_archiver.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <unistd.h> /* for dup */
diff -durpN postgresql.1/src/bin/pg_dump/pg_backup_tar.c postgresql.2/src/bin/pg_dump/pg_backup_tar.c
--- postgresql.1/src/bin/pg_dump/pg_backup_tar.c 2012-10-03 10:40:48.304207436 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_tar.c 2012-11-22 14:55:52.210178502 +0100
@@ -28,7 +28,6 @@
#include "pg_backup.h"
#include "pg_backup_archiver.h"
#include "pg_backup_tar.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <sys/stat.h>
diff -durpN postgresql.1/src/bin/pg_dump/pg_dumpall.c postgresql.2/src/bin/pg_dump/pg_dumpall.c
--- postgresql.1/src/bin/pg_dump/pg_dumpall.c 2012-10-14 11:09:08.143146185 +0200
+++ postgresql.2/src/bin/pg_dump/pg_dumpall.c 2012-11-22 14:55:52.211178508 +0100
@@ -23,7 +23,6 @@
#include "getopt_long.h"
#include "dumputils.h"
-#include "dumpmem.h"
#include "pg_backup.h"
/* version string we expect back from pg_dump */
diff -durpN postgresql.1/src/bin/pg_dump/pg_dump.c postgresql.2/src/bin/pg_dump/pg_dump.c
--- postgresql.1/src/bin/pg_dump/pg_dump.c 2012-10-28 07:41:12.051532033 +0100
+++ postgresql.2/src/bin/pg_dump/pg_dump.c 2012-11-22 14:55:52.217178546 +0100
@@ -59,7 +59,6 @@
#include "pg_backup_archiver.h"
#include "pg_backup_db.h"
-#include "dumpmem.h"
#include "dumputils.h"
extern char *optarg;
diff -durpN postgresql.1/src/bin/pg_dump/pg_dump_sort.c postgresql.2/src/bin/pg_dump/pg_dump_sort.c
--- postgresql.1/src/bin/pg_dump/pg_dump_sort.c 2012-10-19 19:12:00.334691199 +0200
+++ postgresql.2/src/bin/pg_dump/pg_dump_sort.c 2012-11-22 14:55:52.218178553 +0100
@@ -15,7 +15,6 @@
*/
#include "pg_backup_archiver.h"
#include "dumputils.h"
-#include "dumpmem.h"
/* translator: this is a module name */
static const char *modulename = gettext_noop("sorter");
diff -durpN postgresql.1/src/bin/pg_dump/pg_restore.c postgresql.2/src/bin/pg_dump/pg_restore.c
--- postgresql.1/src/bin/pg_dump/pg_restore.c 2012-10-14 11:09:08.143146185 +0200
+++ postgresql.2/src/bin/pg_dump/pg_restore.c 2012-11-22 14:55:52.218178553 +0100
@@ -41,7 +41,6 @@
#include "pg_backup_archiver.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <ctype.h>
diff -durpN postgresql.1/src/bin/psql/common.c postgresql.2/src/bin/psql/common.c
--- postgresql.1/src/bin/psql/common.c 2012-10-03 10:40:48.322207553 +0200
+++ postgresql.2/src/bin/psql/common.c 2012-11-22 14:55:52.219178560 +0100
@@ -33,56 +33,6 @@ static bool command_no_begin(const char
static bool is_select_command(const char *query);
/*
- * "Safe" wrapper around strdup()
- */
-char *
-pg_strdup(const char *string)
-{
- char *tmp;
-
- if (!string)
- {
- psql_error("%s: pg_strdup: cannot duplicate null pointer (internal error)\n",
- pset.progname);
- exit(EXIT_FAILURE);
- }
- tmp = strdup(string);
- if (!tmp)
- {
- psql_error("out of memory\n");
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc(size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- tmp = malloc(size);
- if (!tmp)
- {
- psql_error("out of memory\n");
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc0(size_t size)
-{
- void *tmp;
-
- tmp = pg_malloc(size);
- MemSet(tmp, 0, size);
- return tmp;
-}
-
-/*
* setQFout
* -- handler for -o command line option and \o command
*
diff -durpN postgresql.1/src/bin/psql/common.h postgresql.2/src/bin/psql/common.h
--- postgresql.1/src/bin/psql/common.h 2012-10-03 10:40:48.322207553 +0200
+++ postgresql.2/src/bin/psql/common.h 2012-11-22 14:55:52.219178560 +0100
@@ -21,15 +21,6 @@
#define atooid(x) ((Oid) strtoul((x), NULL, 10))
-/*
- * Safer versions of some standard C library functions. If an
- * out-of-memory condition occurs, these functions will bail out
- * safely; therefore, their return value is guaranteed to be non-NULL.
- */
-extern char *pg_strdup(const char *string);
-extern void *pg_malloc(size_t size);
-extern void *pg_malloc0(size_t size);
-
extern bool setQFout(const char *fname);
extern void
diff -durpN postgresql.1/src/bin/psql/startup.c postgresql.2/src/bin/psql/startup.c
--- postgresql.1/src/bin/psql/startup.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/psql/startup.c 2012-11-22 14:55:52.219178560 +0100
@@ -77,6 +77,21 @@ static void process_psqlrc_file(char *fi
static void showVersion(void);
static void EstablishVariableSpace(void);
+static void psql_malloc_oom(void)
+{
+ psql_error("out of memory\n");
+}
+
+static void psql_strdup_null(void)
+{
+ psql_error("%s: pg_strdup: cannot duplicate null pointer (internal error)\n",
+ pset.progname);
+}
+
+static PostgresFrontendData psql_fe_data = {
+ psql_malloc_oom, psql_strdup_null
+};
+
/*
*
* main
@@ -91,6 +106,8 @@ main(int argc, char *argv[])
char *password_prompt = NULL;
bool new_pass;
+ InitPostgresFrontend(&psql_fe_data);
+
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("psql"));
if (argc > 1)
diff -durpN postgresql.1/src/bin/scripts/clusterdb.c postgresql.2/src/bin/scripts/clusterdb.c
--- postgresql.1/src/bin/scripts/clusterdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/clusterdb.c 2012-11-22 14:55:52.219178560 +0100
@@ -61,6 +61,8 @@ main(int argc, char *argv[])
char *table = NULL;
bool verbose = false;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/bin/scripts/common.c postgresql.2/src/bin/scripts/common.c
--- postgresql.1/src/bin/scripts/common.c 2012-10-03 10:40:48.324207567 +0200
+++ postgresql.2/src/bin/scripts/common.c 2012-11-22 14:55:52.220178566 +0100
@@ -31,6 +31,25 @@ static CRITICAL_SECTION cancelConnLock;
#endif
/*
+ * Helper functions to report error in pg_malloc and pg_strdup
+ */
+static void
+common_malloc_oom(void)
+{
+ fprintf(stderr, _("out of memory\n"));
+}
+
+static void
+common_strdup_null(void)
+{
+ fprintf(stderr, _("pg_strdup: cannot duplicate null pointer (internal error)\n"));
+}
+
+PostgresFrontendData common_fe_data = {
+ common_malloc_oom, common_strdup_null
+};
+
+/*
* Returns the current user name.
*/
const char *
@@ -278,55 +297,6 @@ executeMaintenanceCommand(PGconn *conn,
}
/*
- * "Safe" wrapper around strdup(). Pulled from psql/common.c
- */
-char *
-pg_strdup(const char *string)
-{
- char *tmp;
-
- if (!string)
- {
- fprintf(stderr, _("pg_strdup: cannot duplicate null pointer (internal error)\n"));
- exit(EXIT_FAILURE);
- }
- tmp = strdup(string);
- if (!tmp)
- {
- fprintf(stderr, _("out of memory\n"));
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc(size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- tmp = malloc(size);
- if (!tmp)
- {
- fprintf(stderr, _("out of memory\n"));
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc0(size_t size)
-{
- void *tmp;
-
- tmp = pg_malloc(size);
- MemSet(tmp, 0, size);
- return tmp;
-}
-
-/*
* Check yes/no answer in a localized way. 1=yes, 0=no, -1=neither.
*/
diff -durpN postgresql.1/src/bin/scripts/common.h postgresql.2/src/bin/scripts/common.h
--- postgresql.1/src/bin/scripts/common.h 2012-10-03 10:40:48.324207567 +0200
+++ postgresql.2/src/bin/scripts/common.h 2012-11-22 14:55:52.220178566 +0100
@@ -22,6 +22,8 @@ enum trivalue
typedef void (*help_handler) (const char *progname);
+extern PostgresFrontendData common_fe_data;
+
extern const char *get_user_name(const char *progname);
extern void handle_help_version_opts(int argc, char *argv[],
@@ -50,8 +52,4 @@ extern bool yesno_prompt(const char *que
extern void setup_cancel_handler(void);
-extern char *pg_strdup(const char *string);
-extern void *pg_malloc(size_t size);
-extern void *pg_malloc0(size_t size);
-
#endif /* COMMON_H */
diff -durpN postgresql.1/src/bin/scripts/createdb.c postgresql.2/src/bin/scripts/createdb.c
--- postgresql.1/src/bin/scripts/createdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/createdb.c 2012-11-22 14:55:52.220178566 +0100
@@ -64,6 +64,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/bin/scripts/createlang.c postgresql.2/src/bin/scripts/createlang.c
--- postgresql.1/src/bin/scripts/createlang.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/createlang.c 2012-11-22 14:55:52.221178572 +0100
@@ -52,6 +52,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/bin/scripts/createuser.c postgresql.2/src/bin/scripts/createuser.c
--- postgresql.1/src/bin/scripts/createuser.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/createuser.c 2012-11-22 14:55:52.221178572 +0100
@@ -78,6 +78,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/bin/scripts/dropdb.c postgresql.2/src/bin/scripts/dropdb.c
--- postgresql.1/src/bin/scripts/dropdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/dropdb.c 2012-11-22 14:55:52.222178578 +0100
@@ -54,6 +54,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/bin/scripts/droplang.c postgresql.2/src/bin/scripts/droplang.c
--- postgresql.1/src/bin/scripts/droplang.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/droplang.c 2012-11-22 14:55:52.222178578 +0100
@@ -51,6 +51,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/bin/scripts/dropuser.c postgresql.2/src/bin/scripts/dropuser.c
--- postgresql.1/src/bin/scripts/dropuser.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/dropuser.c 2012-11-22 14:55:52.222178578 +0100
@@ -52,6 +52,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/bin/scripts/reindexdb.c postgresql.2/src/bin/scripts/reindexdb.c
--- postgresql.1/src/bin/scripts/reindexdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/reindexdb.c 2012-11-22 14:55:52.223178584 +0100
@@ -67,6 +67,8 @@ main(int argc, char *argv[])
const char *table = NULL;
const char *index = NULL;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/bin/scripts/vacuumdb.c postgresql.2/src/bin/scripts/vacuumdb.c
--- postgresql.1/src/bin/scripts/vacuumdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/vacuumdb.c 2012-11-22 14:55:52.223178584 +0100
@@ -72,6 +72,8 @@ main(int argc, char *argv[])
bool full = false;
bool verbose = false;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql.1/src/include/port.h postgresql.2/src/include/port.h
--- postgresql.1/src/include/port.h 2012-07-22 16:48:48.548857900 +0200
+++ postgresql.2/src/include/port.h 2012-11-22 14:55:52.223178584 +0100
@@ -17,6 +17,19 @@
#include <netdb.h>
#include <pwd.h>
+/* port/init.c */
+typedef void (*fe_func)(void);
+
+typedef struct PostgresFrontendData
+{
+ fe_func pg_malloc_oom;
+ fe_func pg_strdup_null;
+ fe_func app_atexit;
+} PostgresFrontendData, *PostgresFrontend;
+extern PostgresFrontend pg_fedata;
+
+extern void InitPostgresFrontend(PostgresFrontend fe_data);
+
/* socket has a different definition on WIN32 */
#ifndef WIN32
typedef int pgsocket;
@@ -462,6 +475,19 @@ extern char *inet_net_ntop(int af, const
/* port/pgcheckdir.c */
extern int pg_check_dir(const char *dir);
+/* port/pgmalloc.c */
+extern void pgmalloc_atexit(void);
+
+extern char *pg_strdup(const char *string);
+
+extern void *pg_malloc(size_t size);
+
+extern void *pg_malloc0(size_t size);
+
+extern void *pg_realloc(void *ptr, size_t size);
+
+extern void pg_free(void *ptr);
+
/* port/pgmkdirp.c */
extern int pg_mkdir_p(char *path, int omode);
diff -durpN postgresql.1/src/port/init.c postgresql.2/src/port/init.c
--- postgresql.1/src/port/init.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.2/src/port/init.c 2012-11-22 14:55:56.942208075 +0100
@@ -0,0 +1,32 @@
+/*-------------------------------------------------------------------------
+ *
+ * init.c
+ * Functions for initializing frontend functions
+ *
+ *
+ * Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/init.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef FRONTEND
+#include "postgres_fe.h"
+
+PostgresFrontend pg_fedata = NULL;
+
+void InitPostgresFrontend(PostgresFrontend ptr)
+{
+ pg_fedata = ptr;
+
+ atexit(pgmalloc_atexit);
+
+ if (ptr && ptr->app_atexit)
+ atexit(ptr->app_atexit);
+}
+
+#endif
diff -durpN postgresql.1/src/port/Makefile postgresql.2/src/port/Makefile
--- postgresql.1/src/port/Makefile 2012-05-09 12:28:11.755496856 +0200
+++ postgresql.2/src/port/Makefile 2012-11-22 14:55:56.942208075 +0100
@@ -30,9 +30,10 @@ include $(top_builddir)/src/Makefile.glo
override CPPFLAGS := -I$(top_builddir)/src/port -DFRONTEND $(CPPFLAGS)
LIBS += $(PTHREAD_LIBS)
-OBJS = $(LIBOBJS) chklocale.o dirmod.o erand48.o exec.o fls.o inet_net_ntop.o \
- noblock.o path.o pgcheckdir.o pg_crc.o pgmkdirp.o pgsleep.o \
- pgstrcasecmp.o qsort.o qsort_arg.o sprompt.o thread.o
+OBJS = $(LIBOBJS) chklocale.o dirmod.o erand48.o exec.o fls.o init.o \
+ inet_net_ntop.o noblock.o path.o pgcheckdir.o pg_crc.o pgmalloc.o \
+ pgmkdirp.o pgsleep.o pgstrcasecmp.o qsort.o qsort_arg.o sprompt.o \
+ thread.o
# foo_srv.o and foo.o are both built from foo.c, but only foo.o has -DFRONTEND
OBJS_SRV = $(OBJS:%.o=%_srv.o)
diff -durpN postgresql.1/src/port/pgmalloc.c postgresql.2/src/port/pgmalloc.c
--- postgresql.1/src/port/pgmalloc.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.2/src/port/pgmalloc.c 2012-11-22 14:55:56.943208082 +0100
@@ -0,0 +1,125 @@
+/*-------------------------------------------------------------------------
+ *
+ * pgmalloc.c
+ * Functions for allocating memory and
+ * exiting on out-of-memory
+ *
+ *
+ * Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pgmalloc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef FRONTEND
+#include "postgres_fe.h"
+
+#include <stdio.h>
+
+typedef enum {
+ PG_MALLOC_NOP,
+ PG_MALLOC_OOM,
+ PG_STRDUP_NULL
+} PgMallocAction;
+
+static PgMallocAction pgmalloc_action = PG_MALLOC_NOP;
+
+void pgmalloc_atexit(void)
+{
+ switch (pgmalloc_action)
+ {
+ case PG_MALLOC_OOM:
+ if (pg_fedata && pg_fedata->pg_malloc_oom)
+ pg_fedata->pg_malloc_oom();
+ else
+ fprintf(stderr, "out of memory\n");
+ break;
+ case PG_STRDUP_NULL:
+ if (pg_fedata && pg_fedata->pg_strdup_null)
+ pg_fedata->pg_strdup_null();
+ else
+ fprintf(stderr, "pg_strdup: cannot duplicate null pointer (internal error)\n");
+ break;
+ case PG_MALLOC_NOP:
+ break;
+ }
+}
+
+/*
+ * "Safe" wrapper around strdup()
+ */
+char *
+pg_strdup(const char *string)
+{
+ char *tmp;
+
+ if (!string)
+ {
+ pgmalloc_action = PG_STRDUP_NULL;
+ exit(1);
+ }
+ tmp = strdup(string);
+ if (!tmp)
+ {
+ pgmalloc_action = PG_MALLOC_OOM;
+ exit(1);
+ }
+ return tmp;
+}
+
+void *
+pg_malloc(size_t size)
+{
+ void *tmp;
+
+ /* Avoid unportable behavior of malloc(0) */
+ if (size == 0)
+ size = 1;
+ tmp = malloc(size);
+ if (!tmp)
+ {
+ pgmalloc_action = PG_MALLOC_OOM;
+ exit(1);
+ }
+ return tmp;
+}
+
+void *
+pg_malloc0(size_t size)
+{
+ void *tmp;
+
+ tmp = pg_malloc(size);
+ MemSet(tmp, 0, size);
+ return tmp;
+}
+
+void *
+pg_realloc(void *ptr, size_t size)
+{
+ void *tmp;
+
+ /* Avoid unportable behavior of realloc(NULL, 0) */
+ if (ptr == NULL && size == 0)
+ size = 1;
+ tmp = realloc(ptr, size);
+ if (!tmp)
+ {
+ pgmalloc_action = PG_MALLOC_OOM;
+ exit(1);
+ }
+ return tmp;
+}
+
+void
+pg_free(void *ptr)
+{
+ if (ptr != NULL)
+ free(ptr);
+}
+
+#endif
03-tarutils-strutils-v3.1.patchtext/x-patch; name=03-tarutils-strutils-v3.1.patchDownload
diff -durpN postgresql.2/src/backend/replication/basebackup.c postgresql.3/src/backend/replication/basebackup.c
--- postgresql.2/src/backend/replication/basebackup.c 2012-10-06 19:25:21.002166521 +0200
+++ postgresql.3/src/backend/replication/basebackup.c 2012-11-22 14:56:59.186587142 +0100
@@ -721,48 +721,12 @@ sendDir(char *path, int basepathlen, boo
/*
- * Utility routine to print possibly larger than 32 bit integers in a
- * portable fashion. Filled with zeros.
- */
-static void
-print_val(char *s, uint64 val, unsigned int base, size_t len)
-{
- int i;
-
- for (i = len; i > 0; i--)
- {
- int digit = val % base;
-
- s[i - 1] = '0' + digit;
- val = val / base;
- }
-}
-
-/*
* Maximum file size for a tar member: The limit inherent in the
* format is 2^33-1 bytes (nearly 8 GB). But we don't want to exceed
* what we can represent in pgoff_t.
*/
#define MAX_TAR_MEMBER_FILELEN (((int64) 1 << Min(33, sizeof(pgoff_t)*8 - 1)) - 1)
-static int
-_tarChecksum(char *header)
-{
- int i,
- sum;
-
- /*
- * Per POSIX, the checksum is the simple sum of all bytes in the header,
- * treating the bytes as unsigned, and treating the checksum field (at
- * offset 148) as though it contained 8 spaces.
- */
- sum = 8 * ' '; /* presumed value for checksum field */
- for (i = 0; i < 512; i++)
- if (i < 148 || i >= 156)
- sum += 0xFF & header[i];
- return sum;
-}
-
/* Given the member, write the TAR header & send the file */
static void
sendFile(char *readfilename, char *tarfilename, struct stat * statbuf)
@@ -840,94 +804,12 @@ _tarWriteHeader(const char *filename, co
{
char h[512];
- /*
- * Note: most of the fields in a tar header are not supposed to be
- * null-terminated. We use sprintf, which will write a null after the
- * required bytes; that null goes into the first byte of the next field.
- * This is okay as long as we fill the fields in order.
- */
- memset(h, 0, sizeof(h));
-
- /* Name 100 */
- sprintf(&h[0], "%.99s", filename);
- if (linktarget != NULL || S_ISDIR(statbuf->st_mode))
- {
- /*
- * We only support symbolic links to directories, and this is
- * indicated in the tar format by adding a slash at the end of the
- * name, the same as for regular directories.
- */
- int flen = strlen(filename);
-
- flen = Min(flen, 99);
- h[flen] = '/';
- h[flen + 1] = '\0';
- }
-
- /* Mode 8 */
- sprintf(&h[100], "%07o ", (int) statbuf->st_mode);
-
- /* User ID 8 */
- sprintf(&h[108], "%07o ", statbuf->st_uid);
-
- /* Group 8 */
- sprintf(&h[116], "%07o ", statbuf->st_gid);
-
- /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
- if (linktarget != NULL || S_ISDIR(statbuf->st_mode))
- /* Symbolic link or directory has size zero */
- print_val(&h[124], 0, 8, 11);
- else
- print_val(&h[124], statbuf->st_size, 8, 11);
- sprintf(&h[135], " ");
-
- /* Mod Time 12 */
- sprintf(&h[136], "%011o ", (int) statbuf->st_mtime);
-
- /* Checksum 8 cannot be calculated until we've filled all other fields */
-
- if (linktarget != NULL)
- {
- /* Type - Symbolic link */
- sprintf(&h[156], "2");
- /* Link Name 100 */
- sprintf(&h[157], "%.99s", linktarget);
- }
- else if (S_ISDIR(statbuf->st_mode))
- /* Type - directory */
- sprintf(&h[156], "5");
- else
- /* Type - regular file */
- sprintf(&h[156], "0");
-
- /* Magic 6 */
- sprintf(&h[257], "ustar");
-
- /* Version 2 */
- sprintf(&h[263], "00");
-
- /* User 32 */
- /* XXX: Do we need to care about setting correct username? */
- sprintf(&h[265], "%.31s", "postgres");
-
- /* Group 32 */
- /* XXX: Do we need to care about setting correct group name? */
- sprintf(&h[297], "%.31s", "postgres");
-
- /* Major Dev 8 */
- sprintf(&h[329], "%07o ", 0);
-
- /* Minor Dev 8 */
- sprintf(&h[337], "%07o ", 0);
-
- /* Prefix 155 - not used, leave as nulls */
-
- /*
- * We mustn't overwrite the next field while inserting the checksum.
- * Fortunately, the checksum can't exceed 6 octal digits, so we just write
- * 6 digits, a space, and a null, which is legal per POSIX.
- */
- sprintf(&h[148], "%06o ", _tarChecksum(h));
+ _tarCreateHeader(h, filename, linktarget,
+ statbuf->st_size,
+ statbuf->st_mode,
+ statbuf->st_uid,
+ statbuf->st_gid,
+ statbuf->st_mtime);
/* Now send the completed header. */
pq_putmessage('d', h, 512);
diff -durpN postgresql.2/src/bin/initdb/initdb.c postgresql.3/src/bin/initdb/initdb.c
--- postgresql.2/src/bin/initdb/initdb.c 2012-11-22 14:55:52.198178426 +0100
+++ postgresql.3/src/bin/initdb/initdb.c 2012-11-22 14:56:59.188587156 +0100
@@ -218,7 +218,6 @@ static void make_postgres(void);
static void perform_fsync(void);
static void trapsig(int signum);
static void check_ok(void);
-static char *escape_quotes(const char *src);
static int locale_date_order(const char *locale);
static bool check_locale_name(int category, const char *locale,
char **canonname);
@@ -2342,35 +2341,6 @@ check_ok(void)
}
}
-/*
- * Escape (by doubling) any single quotes or backslashes in given string
- *
- * Note: this is used to process both postgresql.conf entries and SQL
- * string literals. Since postgresql.conf strings are defined to treat
- * backslashes as escapes, we have to double backslashes here. Hence,
- * when using this for a SQL string literal, use E'' syntax.
- *
- * We do not need to worry about encoding considerations because all
- * valid backend encodings are ASCII-safe.
- */
-static char *
-escape_quotes(const char *src)
-{
- int len = strlen(src),
- i,
- j;
- char *result = pg_malloc(len * 2 + 1);
-
- for (i = 0, j = 0; i < len; i++)
- {
- if (SQL_STR_DOUBLE(src[i], true))
- result[j++] = src[i];
- result[j++] = src[i];
- }
- result[j] = '\0';
- return result;
-}
-
/* Hack to suppress a warning about %x from some versions of gcc */
static inline size_t
my_strftime(char *s, size_t max, const char *fmt, const struct tm * tm)
diff -durpN postgresql.2/src/bin/pg_dump/pg_backup_tar.c postgresql.3/src/bin/pg_dump/pg_backup_tar.c
--- postgresql.2/src/bin/pg_dump/pg_backup_tar.c 2012-11-22 14:55:52.210178502 +0100
+++ postgresql.3/src/bin/pg_dump/pg_backup_tar.c 2012-11-22 14:56:59.189587163 +0100
@@ -114,7 +114,6 @@ static char *tarGets(char *buf, size_t l
static int tarPrintf(ArchiveHandle *AH, TAR_MEMBER *th, const char *fmt,...) __attribute__((format(PG_PRINTF_ATTRIBUTE, 3, 4)));
static void _tarAddFile(ArchiveHandle *AH, TAR_MEMBER *th);
-static int _tarChecksum(char *th);
static TAR_MEMBER *_tarPositionTo(ArchiveHandle *AH, const char *filename);
static size_t tarRead(void *buf, size_t len, TAR_MEMBER *th);
static size_t tarWrite(const void *buf, size_t len, TAR_MEMBER *th);
@@ -1015,24 +1014,6 @@ tarPrintf(ArchiveHandle *AH, TAR_MEMBER
return cnt;
}
-static int
-_tarChecksum(char *header)
-{
- int i,
- sum;
-
- /*
- * Per POSIX, the checksum is the simple sum of all bytes in the header,
- * treating the bytes as unsigned, and treating the checksum field (at
- * offset 148) as though it contained 8 spaces.
- */
- sum = 8 * ' '; /* presumed value for checksum field */
- for (i = 0; i < 512; i++)
- if (i < 148 || i >= 156)
- sum += 0xFF & header[i];
- return sum;
-}
-
bool
isValidTarHeader(char *header)
{
@@ -1304,94 +1285,16 @@ _tarGetHeader(ArchiveHandle *AH, TAR_MEM
}
-/*
- * Utility routine to print possibly larger than 32 bit integers in a
- * portable fashion. Filled with zeros.
- */
-static void
-print_val(char *s, uint64 val, unsigned int base, size_t len)
-{
- int i;
-
- for (i = len; i > 0; i--)
- {
- int digit = val % base;
-
- s[i - 1] = '0' + digit;
- val = val / base;
- }
-}
-
-
static void
_tarWriteHeader(TAR_MEMBER *th)
{
char h[512];
- /*
- * Note: most of the fields in a tar header are not supposed to be
- * null-terminated. We use sprintf, which will write a null after the
- * required bytes; that null goes into the first byte of the next field.
- * This is okay as long as we fill the fields in order.
- */
- memset(h, 0, sizeof(h));
-
- /* Name 100 */
- sprintf(&h[0], "%.99s", th->targetFile);
-
- /* Mode 8 */
- sprintf(&h[100], "0000600 ");
-
- /* User ID 8 */
- sprintf(&h[108], "0004000 ");
-
- /* Group 8 */
- sprintf(&h[116], "0002000 ");
-
- /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
- print_val(&h[124], th->fileLen, 8, 11);
- sprintf(&h[135], " ");
-
- /* Mod Time 12 */
- sprintf(&h[136], "%011o ", (int) time(NULL));
-
- /* Checksum 8 cannot be calculated until we've filled all other fields */
-
- /* Type - regular file */
- sprintf(&h[156], "0");
-
- /* Link Name 100 (leave as nulls) */
-
- /* Magic 6 */
- sprintf(&h[257], "ustar");
-
- /* Version 2 */
- sprintf(&h[263], "00");
-
- /* User 32 */
- /* XXX: Do we need to care about setting correct username? */
- sprintf(&h[265], "%.31s", "postgres");
-
- /* Group 32 */
- /* XXX: Do we need to care about setting correct group name? */
- sprintf(&h[297], "%.31s", "postgres");
-
- /* Major Dev 8 */
- sprintf(&h[329], "%07o ", 0);
-
- /* Minor Dev 8 */
- sprintf(&h[337], "%07o ", 0);
-
- /* Prefix 155 - not used, leave as nulls */
-
- /*
- * We mustn't overwrite the next field while inserting the checksum.
- * Fortunately, the checksum can't exceed 6 octal digits, so we just write
- * 6 digits, a space, and a null, which is legal per POSIX.
- */
- sprintf(&h[148], "%06o ", _tarChecksum(h));
+ _tarCreateHeader(h, th->targetFile, NULL,
+ th->fileLen,
+ 0600, 04000, 02000,
+ time(NULL));
- /* Now write the completed header. */
if (fwrite(h, 1, 512, th->tarFH) != 512)
exit_horribly(modulename, "could not write to output file: %s\n", strerror(errno));
}
diff -durpN postgresql.2/src/include/port.h postgresql.3/src/include/port.h
--- postgresql.2/src/include/port.h 2012-11-22 14:55:52.223178584 +0100
+++ postgresql.3/src/include/port.h 2012-11-22 14:56:59.190587169 +0100
@@ -16,6 +16,7 @@
#include <ctype.h>
#include <netdb.h>
#include <pwd.h>
+#include <time.h>
/* port/init.c */
typedef void (*fe_func)(void);
@@ -491,4 +492,22 @@ extern void pg_free(void *ptr);
/* port/pgmkdirp.c */
extern int pg_mkdir_p(char *path, int omode);
+/* port/strutil.c */
+extern void print_val(char *s, uint64 val, unsigned int base, size_t len);
+extern void scan_val(char *s, uint64 *val, unsigned int base, size_t len);
+extern char *escape_quotes(const char *src);
+
+/* port/tarutil.c */
+
+/* Don't ever change this value, the TAR file format requires it. */
+#define TARCHUNKSZ 512
+
+extern int _tarChecksum(char *header);
+extern void _tarCreateHeader(char *header,
+ const char *filename,
+ const char *linktarget,
+ int64 filesize, int st_mode,
+ int st_uid, int st_gid,
+ time_t mtime);
+
#endif /* PG_PORT_H */
diff -durpN postgresql.2/src/port/Makefile postgresql.3/src/port/Makefile
--- postgresql.2/src/port/Makefile 2012-11-22 14:55:56.942208075 +0100
+++ postgresql.3/src/port/Makefile 2012-11-22 14:56:59.190587169 +0100
@@ -33,7 +33,7 @@ LIBS += $(PTHREAD_LIBS)
OBJS = $(LIBOBJS) chklocale.o dirmod.o erand48.o exec.o fls.o init.o \
inet_net_ntop.o noblock.o path.o pgcheckdir.o pg_crc.o pgmalloc.o \
pgmkdirp.o pgsleep.o pgstrcasecmp.o qsort.o qsort_arg.o sprompt.o \
- thread.o
+ strutil.o tarutil.o thread.o
# foo_srv.o and foo.o are both built from foo.c, but only foo.o has -DFRONTEND
OBJS_SRV = $(OBJS:%.o=%_srv.o)
diff -durpN postgresql.2/src/port/strutil.c postgresql.3/src/port/strutil.c
--- postgresql.2/src/port/strutil.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.3/src/port/strutil.c 2012-11-22 14:56:59.191587175 +0100
@@ -0,0 +1,87 @@
+/*-------------------------------------------------------------------------
+ *
+ * strutil.c
+ * common string functions
+ *
+ * IDENTIFICATION
+ * src/port/strutil.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FRONTEND
+#include "postgres.h"
+#include "utils/palloc.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+/*
+ * Utility routine to print possibly larger than 32 bit integers in a
+ * portable fashion. Filled with zeros.
+ */
+void
+print_val(char *s, uint64 val, unsigned int base, size_t len)
+{
+ int i;
+
+ for (i = len; i > 0; i--)
+ {
+ int digit = val % base;
+
+ s[i - 1] = '0' + digit;
+ val = val / base;
+ }
+}
+
+
+/*
+ * Inverse of print_val()
+ */
+void
+scan_val(char *s, uint64 *val, unsigned int base, size_t len)
+{
+ uint64 tmp = 0;
+ int i;
+
+ for (i = 0; i < len; i++)
+ {
+ int digit = s[i] - '0';
+
+ tmp = tmp * base + digit;
+ }
+
+ *val = tmp;
+}
+
+/*
+ * Escape (by doubling) any single quotes or backslashes in given string
+ *
+ * Note: this is used to process both postgresql.conf entries and SQL
+ * string literals. Since postgresql.conf strings are defined to treat
+ * backslashes as escapes, we have to double backslashes here. Hence,
+ * when using this for a SQL string literal, use E'' syntax.
+ *
+ * We do not need to worry about encoding considerations because all
+ * valid backend encodings are ASCII-safe.
+ */
+char *
+escape_quotes(const char *src)
+{
+ int len = strlen(src),
+ i,
+ j;
+#ifdef FRONTEND
+ char *result = pg_malloc(len * 2 + 1);
+#else
+ char *result = palloc(len * 2 + 1);
+#endif
+
+ for (i = 0, j = 0; i < len; i++)
+ {
+ if (SQL_STR_DOUBLE(src[i], true))
+ result[j++] = src[i];
+ result[j++] = src[i];
+ }
+ result[j] = '\0';
+ return result;
+}
diff -durpN postgresql.2/src/port/tarutil.c postgresql.3/src/port/tarutil.c
--- postgresql.2/src/port/tarutil.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.3/src/port/tarutil.c 2012-11-22 14:56:59.191587175 +0100
@@ -0,0 +1,135 @@
+/*-------------------------------------------------------------------------
+ *
+ * tarutil.c
+ * common functions for handling the TAR format
+ *
+ * IDENTIFICATION
+ * src/port/tarutil.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include <sys/stat.h>
+
+int
+_tarChecksum(char *header)
+{
+ int i,
+ sum;
+
+ /*
+ * Per POSIX, the checksum is the simple sum of all bytes in the header,
+ * treating the bytes as unsigned, and treating the checksum field (at
+ * offset 148) as though it contained 8 spaces.
+ */
+ sum = 8 * ' '; /* presumed value for checksum field */
+ for (i = 0; i < TARCHUNKSZ; i++)
+ if (i < 148 || i >= 156)
+ sum += 0xFF & header[i];
+ return sum;
+}
+
+
+void
+_tarCreateHeader(char *header, const char *filename,
+ const char *linktarget,
+ int64 filesize, int st_mode,
+ int st_uid, int st_gid,
+ time_t mtime)
+{
+ /*
+ * Note: most of the fields in a tar header are not supposed to be
+ * null-terminated. We use sprintf, which will write a null after the
+ * required bytes; that null goes into the first byte of the next field.
+ * This is okay as long as we fill the fields in order.
+ */
+ memset(header, 0, TARCHUNKSZ /* sizeof the tar header */);
+
+ /* Name 100 */
+ sprintf(&header[0], "%.99s", filename);
+ if (linktarget != NULL || S_ISDIR(st_mode))
+ {
+ /*
+ * We only support symbolic links to directories, and this is
+ * indicated in the tar format by adding a slash at the end of the
+ * name, the same as for regular directories.
+ */
+ int flen = strlen(filename);
+
+ flen = Min(flen, 99);
+ header[flen] = '/';
+ header[flen + 1] = '\0';
+ }
+
+ /* Mode 8 */
+ sprintf(&header[100], "%07o ", st_mode);
+
+ /* User ID 8 */
+ sprintf(&header[108], "%07o ", st_uid);
+
+ /* Group 8 */
+ sprintf(&header[116], "%07o ", st_gid);
+
+ /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
+ if (linktarget != NULL || S_ISDIR(st_mode))
+ /* Symbolic link or directory has size zero */
+ print_val(&header[124], 0, 8, 11);
+ else
+ print_val(&header[124], filesize, 8, 11);
+ sprintf(&header[135], " ");
+
+ /* Mod Time 12 - 11 digits, 1 space; use print_val for 64 bit support */
+ print_val(&header[136], mtime, 8, 11);
+ sprintf(&header[147], " ");
+
+ /* Checksum 8 cannot be calculated until we've filled all other fields */
+
+ if (linktarget != NULL)
+ {
+ /* Type - Symbolic link */
+ sprintf(&header[156], "2");
+ /* Link Name 100 */
+ sprintf(&header[157], "%.99s", linktarget);
+ }
+ else if (S_ISDIR(st_mode))
+ /* Type - directory */
+ sprintf(&header[156], "5");
+ else
+ /* Type - regular file */
+ sprintf(&header[156], "0");
+
+ /* Magic 6 */
+ sprintf(&header[257], "ustar");
+
+ /* Version 2 */
+ sprintf(&header[263], "00");
+
+ /* User 32 */
+ /* XXX: Do we need to care about setting correct username? */
+ sprintf(&header[265], "%.31s", "postgres");
+
+ /* Group 32 */
+ /* XXX: Do we need to care about setting correct group name? */
+ sprintf(&header[297], "%.31s", "postgres");
+
+ /* Major Dev 8 */
+ sprintf(&header[329], "%07o ", 0);
+
+ /* Minor Dev 8 */
+ sprintf(&header[337], "%07o ", 0);
+
+ /* Prefix 155 - not used, leave as nulls */
+
+ /*
+ * We mustn't overwrite the next field while inserting the checksum.
+ * Fortunately, the checksum can't exceed 6 octal digits, so we just write
+ * 6 digits, a space, and a null, which is legal per POSIX.
+ */
+ sprintf(&header[148], "%06o ", _tarChecksum(header));
+}
+
04-pg_basebackup-v18.1.patchtext/x-patch; name=04-pg_basebackup-v18.1.patchDownload
diff -durpN postgresql.3/doc/src/sgml/ref/pg_basebackup.sgml postgresql.4/doc/src/sgml/ref/pg_basebackup.sgml
--- postgresql.3/doc/src/sgml/ref/pg_basebackup.sgml 2012-11-08 13:13:04.151630632 +0100
+++ postgresql.4/doc/src/sgml/ref/pg_basebackup.sgml 2012-11-22 14:57:20.353717277 +0100
@@ -189,6 +189,21 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--write-recovery-conf</option></term>
+ <listitem>
+
+ <para>
+ Write a minimal recovery.conf into the output directory (or into
+ the base archive file if <option>--format=tar</option> was specified)
+ using the connection parameters from the command line to ease
+ setting up the standby.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-x</option></term>
<term><option>--xlog</option></term>
<listitem>
diff -durpN postgresql.3/src/bin/pg_basebackup/pg_basebackup.c postgresql.4/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.3/src/bin/pg_basebackup/pg_basebackup.c 2012-11-22 14:55:52.199178432 +0100
+++ postgresql.4/src/bin/pg_basebackup/pg_basebackup.c 2012-11-22 14:57:20.355717289 +0100
@@ -19,6 +19,7 @@
#define FRONTEND 1
#include "postgres.h"
#include "libpq-fe.h"
+#include "pqexpbuffer.h"
#include <unistd.h>
#include <dirent.h>
@@ -46,6 +47,7 @@ int compresslevel = 0;
bool includewal = false;
bool streamwal = false;
bool fastcheckpoint = false;
+bool writerecoveryconf = false;
int standby_message_timeout = 10 * 1000; /* 10 sec = default */
/* Progress counters */
@@ -70,6 +72,8 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
+PQExpBuffer rcExpBuf = NULL;
+
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
@@ -77,6 +81,8 @@ static void progress_report(int tablespa
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
+static void CreateRecoveryConf(PGconn *conn);
+static void WriteRecoveryConf(void);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -107,6 +113,8 @@ usage(void)
printf(_("\nOptions controlling the output:\n"));
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
+ printf(_(" -R, --write-recovery-conf\n"
+ " write recovery.conf after backup\n"));
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
printf(_(" -X, --xlog-method=fetch|stream\n"
" include required WAL files with specified method\n"));
@@ -452,6 +460,45 @@ progress_report(int tablespacenum, const
/*
+ * Write a piece of tar data
+ */
+static void
+writeTarData(
+#ifdef HAVE_LIBZ
+ gzFile ztarfile,
+#endif
+ FILE *tarfile, char *buf, int r, char *current_file)
+{
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (gzwrite(ztarfile, buf, r) != r)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, current_file, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (fwrite(buf, r, 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
+ progname, current_file, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ }
+}
+
+#ifdef HAVE_LIBZ
+#define WRITE_TAR_DATA(buf, sz) writeTarData(ztarfile, tarfile, buf, sz, filename)
+#else
+#define WRITE_TAR_DATA(buf, sz) writeTarData(tarfile, buf, sz, filename)
+#endif
+
+/*
* Receive a tar format file from the connection to the server, and write
* the data from this file directly into a tar file. If compression is
* enabled, the data will be compressed while written to the file.
@@ -467,12 +514,17 @@ ReceiveTarFile(PGconn *conn, PGresult *r
char filename[MAXPGPATH];
char *copybuf = NULL;
FILE *tarfile = NULL;
+ char tarhdr[TARCHUNKSZ];
+ bool basetablespace = PQgetisnull(res, rownum, 0);
+ bool in_tarhdr, skip_file;
+ size_t tarhdrsz;
+ uint64 filesz;
#ifdef HAVE_LIBZ
gzFile ztarfile = NULL;
#endif
- if (PQgetisnull(res, rownum, 0))
+ if (basetablespace)
{
/*
* Base tablespaces
@@ -584,6 +636,16 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
+ /*
+ * Initialize our variables for tracking
+ * individual files inside the TAR stream.
+ * For more detailed explanation, see below.
+ */
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = 0;
+ filesz = 0;
+
while (1)
{
int r;
@@ -598,38 +660,39 @@ ReceiveTarFile(PGconn *conn, PGresult *r
if (r == -1)
{
/*
- * End of chunk. Close file (but not stdout).
+ * End of chunk. Write recovery.conf into the tar file (if it
+ * was requested) and close file (but not stdout).
*
* Also, write two completely empty blocks at the end of the tar
* file, as required by some tar programs.
*/
- char zerobuf[1024];
+ char zerobuf[2*TARCHUNKSZ];
MemSet(zerobuf, 0, sizeof(zerobuf));
-#ifdef HAVE_LIBZ
- if (ztarfile != NULL)
- {
- if (gzwrite(ztarfile, zerobuf, sizeof(zerobuf)) !=
- sizeof(zerobuf))
- {
- fprintf(stderr,
- _("%s: could not write to compressed file \"%s\": %s\n"),
- progname, filename, get_gz_error(ztarfile));
- disconnect_and_exit(1);
- }
- }
- else
-#endif
+
+ if (basetablespace && writerecoveryconf)
{
- if (fwrite(zerobuf, sizeof(zerobuf), 1, tarfile) != 1)
- {
- fprintf(stderr,
- _("%s: could not write to file \"%s\": %s\n"),
- progname, filename, strerror(errno));
- disconnect_and_exit(1);
- }
+ char header[TARCHUNKSZ];
+ int padding;
+
+ _tarCreateHeader(header, "recovery.conf", NULL,
+ rcExpBuf->len,
+ 0600, 04000, 02000,
+ time(NULL));
+
+ padding = ((rcExpBuf->len + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - rcExpBuf->len;
+
+ WRITE_TAR_DATA(header, sizeof(header));
+ WRITE_TAR_DATA(rcExpBuf->data, rcExpBuf->len);
+ if (padding)
+ WRITE_TAR_DATA(zerobuf, padding);
+
+ if (verbose)
+ fprintf(stderr, _("%s: recovery.conf written into '%s'\n"), progname, filename);
}
+ WRITE_TAR_DATA(zerobuf, sizeof(zerobuf));
+
#ifdef HAVE_LIBZ
if (ztarfile != NULL)
{
@@ -665,25 +728,124 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
-#ifdef HAVE_LIBZ
- if (ztarfile != NULL)
+ if (!writerecoveryconf || !basetablespace)
{
- if (gzwrite(ztarfile, copybuf, r) != r)
- {
- fprintf(stderr,
- _("%s: could not write to compressed file \"%s\": %s\n"),
- progname, filename, get_gz_error(ztarfile));
- disconnect_and_exit(1);
- }
+ /*
+ * If --write-recovery-conf was not requested or this
+ * is not the base tablespace, simply pass the received
+ * data into the TAR file, either compressed or not.
+ */
+
+ WRITE_TAR_DATA(copybuf, r);
}
else
-#endif
{
- if (fwrite(copybuf, r, 1, tarfile) != 1)
+ /*
+ * If --write-recovery-conf was requested AND this
+ * is the base tablespace, the TAR stream may contain
+ * a recovery.conf file if the backup is coming from
+ * a standby server. We have to skip this file in
+ * the stream and add a new one constructed by
+ * CreateRecoveryConf() at the end of the stream.
+ *
+ * To do this, we have to process the individual files
+ * inside the TAR stream. The stream consists of a header
+ * and zero or more chunks, all 512 bytes long. The stream
+ * from the server is broken up into smaller pieces, so
+ * we have to track the size of the files to find the next
+ * header structure.
+ */
+ int rr = r; /* Save the value returned by PQgetCopyData */
+ int pos = 0;
+
+ while (rr > 0)
{
- fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
- progname, filename, strerror(errno));
- disconnect_and_exit(1);
+ if (in_tarhdr)
+ {
+ /*
+ * We're currently reading a header structure
+ * inside the TAR stream, i.e. the file metadata.
+ */
+ if (tarhdrsz < TARCHUNKSZ)
+ {
+ /*
+ * Copy the header structure into tarhdr[]
+ * in case the header is not aligned to 512 bytes
+ * or it's not returned in whole by the last
+ * PQgetCopyData call.
+ */
+ int hdrleft, bytes2copy;
+
+ hdrleft = TARCHUNKSZ - tarhdrsz;
+ bytes2copy = (rr > hdrleft ? hdrleft : rr);
+
+ memcpy(&tarhdr[tarhdrsz], copybuf + pos, bytes2copy);
+
+ rr -= bytes2copy;
+ pos += bytes2copy;
+ tarhdrsz += bytes2copy;
+ }
+ else
+ {
+ /*
+ * We have the whole header structure in tarhdr[],
+ * look at the file metadata:
+ * - the subsequent file contents have to be skipped
+ * if the filename is recovery.conf
+ * - find out the size of the file padded to the next
+ * multiple of 512
+ */
+ int64 padding;
+
+ skip_file = (strcmp(&tarhdr[0], "recovery.conf") == 0);
+
+ scan_val(&tarhdr[124], &filesz, 8, 11);
+
+ padding = ((filesz + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - filesz;
+ filesz += padding;
+
+ /* Indicate that the subsequent data is the file content. */
+ in_tarhdr = false;
+
+ if (!skip_file)
+ WRITE_TAR_DATA(tarhdr, TARCHUNKSZ);
+ }
+ }
+ else
+ {
+ /*
+ * We're processing a file's contents.
+ */
+ if (filesz > 0)
+ {
+ /*
+ * We still have data to read (and possibly write).
+ */
+ int bytes2write;
+
+ bytes2write = (filesz > rr ? rr : filesz);
+
+ if (!skip_file)
+ WRITE_TAR_DATA(copybuf + pos, bytes2write);
+
+ rr -= bytes2write;
+ pos += bytes2write;
+ filesz -= bytes2write;
+ }
+ else
+ {
+ /*
+ * No more data in the current file,
+ * the next piece of data (if any) will
+ * be a new file header structure.
+ * Reinitialize all our variables.
+ */
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = 0;
+ filesz = 0;
+ }
+ }
}
}
totaldone += r;
@@ -712,10 +874,11 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
char filename[MAXPGPATH];
int current_len_left;
int current_padding = 0;
+ bool basetablespace = PQgetisnull(res, rownum, 0);
char *copybuf = NULL;
FILE *file = NULL;
- if (PQgetisnull(res, rownum, 0))
+ if (basetablespace)
strcpy(current_path, basedir);
else
strcpy(current_path, PQgetvalue(res, rownum, 1));
@@ -767,13 +930,13 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
/*
* No current file, so this must be the header for a new file
*/
- if (r != 512)
+ if (r != TARCHUNKSZ)
{
fprintf(stderr, _("%s: invalid tar block header size: %d\n"),
progname, r);
disconnect_and_exit(1);
}
- totaldone += 512;
+ totaldone += TARCHUNKSZ;
if (sscanf(copybuf + 124, "%11o", ¤t_len_left) != 1)
{
@@ -794,7 +957,7 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
* All files are padded up to 512 bytes
*/
current_padding =
- ((current_len_left + 511) & ~511) - current_len_left;
+ ((current_len_left + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - current_len_left;
/*
* First part of header is zero terminated filename
@@ -937,6 +1100,114 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
if (copybuf != NULL)
PQfreemem(copybuf);
+
+ if (basetablespace)
+ WriteRecoveryConf();
+}
+
+/*
+ * Create the new recovery.conf in memory
+ */
+static void
+CreateRecoveryConf(PGconn *conn)
+{
+ PQconninfoOption *connOptions;
+ PQconninfoOption *option;
+
+ if (!writerecoveryconf)
+ return;
+
+ connOptions = PQconninfo(conn, PG_CONNINFO_NORMAL | PG_CONNINFO_PASSWORD);
+ if (connOptions == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "standby_mode = 'on'\n");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "primary_conninfo = '");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ for (option = connOptions; option && option->keyword; option++)
+ {
+ char *escaped;
+
+ /*
+ * Do not emit this setting if not set or empty.
+ */
+ if (strcmp(option->keyword, "dbname") != 0 ||
+ strcmp(option->keyword, "fallback_application_name") != 0 ||
+ (option->val == NULL) ||
+ (option->val != NULL && option->val[0] == '\0'))
+ continue;
+
+ /*
+ * Write "keyword='value'" pieces, the value string is escaped
+ * if necessary and doubled single quotes around the value string.
+ */
+ escaped = escape_quotes(option->val);
+
+ appendPQExpBuffer(rcExpBuf, "%s=''%s'' ", option->keyword, escaped);
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ free(escaped);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "'\n");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ PQconninfoFree(connOptions);
+}
+
+
+static void
+WriteRecoveryConf(void)
+{
+ char filename[MAXPGPATH];
+ FILE *cf;
+
+ if (!writerecoveryconf)
+ return;
+
+ sprintf(filename, "%s/recovery.conf", basedir);
+
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("%s: cannot create %s: %s"), progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ if (fwrite(rcExpBuf->data, rcExpBuf->len, 1, cf) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ fclose(cf);
+
+ if (verbose)
+ fprintf(stderr, _("%s: recovery.conf written.\n"), progname);
}
@@ -960,6 +1231,15 @@ BaseBackup(void)
/* Error message already written in GetConnection() */
exit(1);
+ rcExpBuf = createPQExpBuffer();
+ if (!rcExpBuf)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ CreateRecoveryConf(conn);
+
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
@@ -1223,6 +1503,9 @@ BaseBackup(void)
#endif
}
+ /* Free the recovery.conf contents */
+ destroyPQExpBuffer(rcExpBuf);
+
/*
* End of copy data. Final result is already checked inside the loop.
*/
@@ -1243,6 +1526,7 @@ main(int argc, char **argv)
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
+ {"write-recovery-conf", no_argument, NULL, 'R'},
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
{"gzip", no_argument, NULL, 'z'},
@@ -1282,7 +1566,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:xX:l:zZ:c:h:p:U:s:wWvP",
+ while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
long_options, &option_index)) != -1)
{
switch (c)
@@ -1303,6 +1587,9 @@ main(int argc, char **argv)
exit(1);
}
break;
+ case 'R':
+ writerecoveryconf = true;
+ break;
case 'x':
if (includewal)
{
Hi,
now that PQconninfo() was committed, I rebased the subsequent
patches. Actual code change was only in the last patch, to
conform to the committed PQconninfo() API.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
02-pg_malloc-unified-v5.patchtext/x-patch; name=02-pg_malloc-unified-v5.patchDownload
diff -durpN postgresql/contrib/oid2name/oid2name.c postgresql.2/contrib/oid2name/oid2name.c
--- postgresql/contrib/oid2name/oid2name.c 2012-10-03 10:40:48.241207023 +0200
+++ postgresql.2/contrib/oid2name/oid2name.c 2012-11-30 09:49:47.067595921 +0100
@@ -50,9 +50,6 @@ struct options
/* function prototypes */
static void help(const char *progname);
void get_opts(int, char **, struct options *);
-void *pg_malloc(size_t size);
-void *pg_realloc(void *ptr, size_t size);
-char *pg_strdup(const char *str);
void add_one_elt(char *eltname, eary *eary);
char *get_comma_elts(eary *eary);
PGconn *sql_conn(struct options *);
@@ -201,53 +198,6 @@ help(const char *progname)
progname, progname);
}
-void *
-pg_malloc(size_t size)
-{
- void *ptr;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- ptr = malloc(size);
- if (!ptr)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return ptr;
-}
-
-void *
-pg_realloc(void *ptr, size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- result = realloc(ptr, size);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-char *
-pg_strdup(const char *str)
-{
- char *result = strdup(str);
-
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
/*
* add_one_elt
*
@@ -598,6 +548,8 @@ main(int argc, char **argv)
struct options *my_opts;
PGconn *pgconn;
+ InitPostgresFrontend(NULL);
+
my_opts = (struct options *) pg_malloc(sizeof(struct options));
my_opts->oids = (eary *) pg_malloc(sizeof(eary));
diff -durpN postgresql/contrib/pgbench/pgbench.c postgresql.2/contrib/pgbench/pgbench.c
--- postgresql/contrib/pgbench/pgbench.c 2012-10-21 10:56:15.358945296 +0200
+++ postgresql.2/contrib/pgbench/pgbench.c 2012-11-30 09:49:47.068595928 +0100
@@ -296,58 +296,6 @@ static void setalarm(int seconds);
static void *threadRun(void *arg);
-/*
- * routines to check mem allocations and fail noisily.
- */
-static void *
-pg_malloc(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-static void *
-pg_realloc(void *ptr, size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- result = realloc(ptr, size);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-static char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-
static void
usage(void)
{
@@ -1956,6 +1904,8 @@ main(int argc, char **argv)
char val[64];
+ InitPostgresFrontend(NULL);
+
progname = get_progname(argv[0]);
if (argc > 1)
diff -durpN postgresql/contrib/pg_upgrade/pg_upgrade.c postgresql.2/contrib/pg_upgrade/pg_upgrade.c
--- postgresql/contrib/pg_upgrade/pg_upgrade.c 2012-09-04 11:02:05.935828363 +0200
+++ postgresql.2/contrib/pg_upgrade/pg_upgrade.c 2012-11-30 09:49:47.073595967 +0100
@@ -67,6 +67,21 @@ char *output_files[] = {
NULL
};
+static void
+pg_upgrade_malloc_oom(void)
+{
+ pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
+}
+
+static void
+pg_upgrade_strdup_null(void)
+{
+ pg_log(PG_FATAL, "%s: pg_strdup: cannot duplicate null pointer (internal error)\n", os_info.progname);
+}
+
+static PostgresFrontendData pg_upgrade_fe_data = {
+ pg_upgrade_malloc_oom, pg_upgrade_strdup_null
+};
int
main(int argc, char **argv)
@@ -76,6 +91,8 @@ main(int argc, char **argv)
char *deletion_script_file_name = NULL;
bool live_check = false;
+ InitPostgresFrontend(&pg_upgrade_fe_data);
+
parseCommandLine(argc, argv);
adjust_data_dir(&old_cluster);
diff -durpN postgresql/contrib/pg_upgrade/pg_upgrade.h postgresql.2/contrib/pg_upgrade/pg_upgrade.h
--- postgresql/contrib/pg_upgrade/pg_upgrade.h 2012-11-15 13:39:47.132060180 +0100
+++ postgresql.2/contrib/pg_upgrade/pg_upgrade.h 2012-11-30 09:49:47.073595967 +0100
@@ -437,10 +437,6 @@ void
prep_status(const char *fmt,...)
__attribute__((format(PG_PRINTF_ATTRIBUTE, 1, 2)));
void check_ok(void);
-char *pg_strdup(const char *s);
-void *pg_malloc(size_t size);
-void *pg_realloc(void *ptr, size_t size);
-void pg_free(void *ptr);
const char *getErrorText(int errNum);
unsigned int str2uint(const char *str);
void pg_putenv(const char *var, const char *val);
diff -durpN postgresql/contrib/pg_upgrade/util.c postgresql.2/contrib/pg_upgrade/util.c
--- postgresql/contrib/pg_upgrade/util.c 2012-10-03 10:40:48.243207034 +0200
+++ postgresql.2/contrib/pg_upgrade/util.c 2012-11-30 09:49:47.074595974 +0100
@@ -191,55 +191,6 @@ get_user_info(char **user_name)
}
-void *
-pg_malloc(size_t size)
-{
- void *p;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- p = malloc(size);
- if (p == NULL)
- pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
- return p;
-}
-
-void *
-pg_realloc(void *ptr, size_t size)
-{
- void *p;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- p = realloc(ptr, size);
- if (p == NULL)
- pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
- return p;
-}
-
-
-void
-pg_free(void *ptr)
-{
- if (ptr != NULL)
- free(ptr);
-}
-
-
-char *
-pg_strdup(const char *s)
-{
- char *result = strdup(s);
-
- if (result == NULL)
- pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
-
- return result;
-}
-
-
/*
* getErrorText()
*
diff -durpN postgresql/src/bin/initdb/initdb.c postgresql.2/src/bin/initdb/initdb.c
--- postgresql/src/bin/initdb/initdb.c 2012-10-15 10:44:54.640429361 +0200
+++ postgresql.2/src/bin/initdb/initdb.c 2012-11-30 09:49:47.076595989 +0100
@@ -177,8 +177,6 @@ static const char *backend_options = "--
static char bin_path[MAXPGPATH];
static char backend_exec[MAXPGPATH];
-static void *pg_malloc(size_t size);
-static char *pg_strdup(const char *s);
static char **replace_token(char **lines,
const char *token, const char *replacement);
@@ -284,43 +282,6 @@ do { \
#endif
/*
- * routines to check mem allocations and fail noisily.
- *
- * Note that we can't call exit_nicely() on a memory failure, as it calls
- * rmtree() which needs memory allocation. So we just exit with a bang.
- */
-static void *
-pg_malloc(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-static char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-/*
* make a copy of the array of lines, with token replaced by replacement
* the first time it occurs on each line.
*
@@ -2823,6 +2784,22 @@ check_need_password(const char *authmeth
}
}
+static void
+initdb_malloc_oom(void)
+{
+ fprintf(stderr, _("%s: out of memory\n"), progname);
+}
+
+static void
+initdb_strdup_null(void)
+{
+ fprintf(stderr, _("%s: pg_strdup: cannot duplicate null pointer (internal error)\n"), progname);
+}
+
+static PostgresFrontendData initdb_fe_data = {
+ initdb_malloc_oom, initdb_strdup_null
+};
+
int
main(int argc, char *argv[])
{
@@ -2890,6 +2867,8 @@ main(int argc, char *argv[])
"pg_stat_tmp"
};
+ InitPostgresFrontend(&initdb_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("initdb"));
diff -durpN postgresql/src/bin/pg_basebackup/pg_basebackup.c postgresql.2/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql/src/bin/pg_basebackup/pg_basebackup.c 2012-10-03 10:40:48.297207389 +0200
+++ postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2012-11-30 09:49:47.077595997 +0100
@@ -1262,6 +1262,8 @@ main(int argc, char **argv)
int option_index;
+ InitPostgresFrontend(&streamutil_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_basebackup"));
diff -durpN postgresql/src/bin/pg_basebackup/pg_receivexlog.c postgresql.2/src/bin/pg_basebackup/pg_receivexlog.c
--- postgresql/src/bin/pg_basebackup/pg_receivexlog.c 2012-10-03 10:40:48.298207395 +0200
+++ postgresql.2/src/bin/pg_basebackup/pg_receivexlog.c 2012-11-30 09:49:47.078596004 +0100
@@ -283,6 +283,7 @@ StreamLog(void)
stop_streaming, standby_message_timeout, false);
PQfinish(conn);
+ conn = NULL; /* don't crash on the atexit function */
}
/*
@@ -318,6 +319,8 @@ main(int argc, char **argv)
int c;
int option_index;
+ InitPostgresFrontend(&streamutil_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_receivexlog"));
diff -durpN postgresql/src/bin/pg_basebackup/streamutil.c postgresql.2/src/bin/pg_basebackup/streamutil.c
--- postgresql/src/bin/pg_basebackup/streamutil.c 2012-10-03 10:40:48.298207395 +0200
+++ postgresql.2/src/bin/pg_basebackup/streamutil.c 2012-11-30 09:49:47.078596004 +0100
@@ -31,42 +31,29 @@ int dbgetpassword = 0; /* 0=auto, -1=n
static char *dbpassword = NULL;
PGconn *conn = NULL;
-/*
- * strdup() and malloc() replacements that print an error and exit
- * if something goes wrong. Can never return NULL.
- */
-char *
-pg_strdup(const char *s)
+static void
+streamutil_malloc_oom(void)
{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
+ fprintf(stderr, _("%s: out of memory\n"), progname);
}
-void *
-pg_malloc0(size_t size)
+static void
+streamutil_strdup_null(void)
{
- void *result;
+ fprintf(stderr, _("%s: pg_strdup: cannot duplicate null pointer (internal error)\n"), progname);
+}
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- MemSet(result, 0, size);
- return result;
+static void
+streamutil_app_atexit(void)
+{
+ if (conn != NULL)
+ PQfinish(conn);
}
+PostgresFrontendData streamutil_fe_data = {
+ streamutil_malloc_oom, streamutil_strdup_null, streamutil_app_atexit
+};
+
/*
* Connect to the server. Returns a valid PGconn pointer if connected,
diff -durpN postgresql/src/bin/pg_basebackup/streamutil.h postgresql.2/src/bin/pg_basebackup/streamutil.h
--- postgresql/src/bin/pg_basebackup/streamutil.h 2012-10-03 10:40:48.299207401 +0200
+++ postgresql.2/src/bin/pg_basebackup/streamutil.h 2012-11-30 09:49:47.079596011 +0100
@@ -15,8 +15,6 @@ extern PGconn *conn;
exit(code); \
}
-
-extern char *pg_strdup(const char *s);
-extern void *pg_malloc0(size_t size);
+extern PostgresFrontendData streamutil_fe_data;
extern PGconn *GetConnection(void);
diff -durpN postgresql/src/bin/pg_ctl/pg_ctl.c postgresql.2/src/bin/pg_ctl/pg_ctl.c
--- postgresql/src/bin/pg_ctl/pg_ctl.c 2012-10-19 19:12:00.307691010 +0200
+++ postgresql.2/src/bin/pg_ctl/pg_ctl.c 2012-11-30 09:49:47.079596011 +0100
@@ -118,8 +118,6 @@ write_stderr(const char *fmt,...)
/* This extension allows gcc to check the format string for consistency with
the supplied arguments. */
__attribute__((format(PG_PRINTF_ATTRIBUTE, 1, 2)));
-static void *pg_malloc(size_t size);
-static char *pg_strdup(const char *s);
static void do_advice(void);
static void do_help(void);
static void set_mode(char *modeopt);
@@ -226,42 +224,6 @@ write_stderr(const char *fmt,...)
}
/*
- * routines to check memory allocations and fail noisily.
- */
-
-static void *
-pg_malloc(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- write_stderr(_("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-
-static char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- write_stderr(_("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-/*
* Given an already-localized string, print it to stdout unless the
* user has specified that no messages should be printed.
*/
@@ -1994,6 +1956,22 @@ adjust_data_dir(void)
}
+static void
+pgctl_malloc_oom(void)
+{
+ write_stderr(_("%s: out of memory\n"), progname);
+}
+
+static void
+pgctl_strdup_null(void)
+{
+ write_stderr(_("%s: pg_strdup: cannot duplicate null pointer (internal error)\n"), progname);
+}
+
+static PostgresFrontendData pgctl_fe_data = {
+ pgctl_malloc_oom, pgctl_strdup_null
+};
+
int
main(int argc, char **argv)
{
@@ -2013,6 +1991,8 @@ main(int argc, char **argv)
int c;
pgpid_t killproc = 0;
+ InitPostgresFrontend(&pgctl_fe_data);
+
#if defined(WIN32) || defined(__CYGWIN__)
setvbuf(stderr, NULL, _IONBF, 0);
#endif
diff -durpN postgresql/src/bin/pg_dump/common.c postgresql.2/src/bin/pg_dump/common.c
--- postgresql/src/bin/pg_dump/common.c 2012-07-22 16:48:48.536857822 +0200
+++ postgresql.2/src/bin/pg_dump/common.c 2012-11-30 09:49:47.080596019 +0100
@@ -18,7 +18,6 @@
#include <ctype.h>
#include "catalog/pg_class.h"
-#include "dumpmem.h"
#include "dumputils.h"
diff -durpN postgresql/src/bin/pg_dump/compress_io.c postgresql.2/src/bin/pg_dump/compress_io.c
--- postgresql/src/bin/pg_dump/compress_io.c 2012-10-03 10:40:48.301207415 +0200
+++ postgresql.2/src/bin/pg_dump/compress_io.c 2012-11-30 09:49:47.080596019 +0100
@@ -53,7 +53,6 @@
*/
#include "compress_io.h"
-#include "dumpmem.h"
#include "dumputils.h"
/*----------------------
diff -durpN postgresql/src/bin/pg_dump/dumpmem.c postgresql.2/src/bin/pg_dump/dumpmem.c
--- postgresql/src/bin/pg_dump/dumpmem.c 2012-10-03 10:40:48.301207415 +0200
+++ postgresql.2/src/bin/pg_dump/dumpmem.c 1970-01-01 01:00:00.000000000 +0100
@@ -1,76 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * dumpmem.c
- * Memory allocation routines used by pg_dump, pg_dumpall, and pg_restore
- *
- * Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
- *
- *
- * IDENTIFICATION
- * src/bin/pg_dump/dumpmem.c
- *
- *-------------------------------------------------------------------------
- */
-#include "postgres_fe.h"
-
-#include "dumputils.h"
-#include "dumpmem.h"
-
-
-/*
- * Safer versions of some standard C library functions. If an
- * out-of-memory condition occurs, these functions will bail out via exit();
- *therefore, their return value is guaranteed to be non-NULL.
- */
-
-char *
-pg_strdup(const char *string)
-{
- char *tmp;
-
- if (!string)
- exit_horribly(NULL, "cannot duplicate null pointer\n");
- tmp = strdup(string);
- if (!tmp)
- exit_horribly(NULL, "out of memory\n");
- return tmp;
-}
-
-void *
-pg_malloc(size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- tmp = malloc(size);
- if (!tmp)
- exit_horribly(NULL, "out of memory\n");
- return tmp;
-}
-
-void *
-pg_malloc0(size_t size)
-{
- void *tmp;
-
- tmp = pg_malloc(size);
- MemSet(tmp, 0, size);
- return tmp;
-}
-
-void *
-pg_realloc(void *ptr, size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- tmp = realloc(ptr, size);
- if (!tmp)
- exit_horribly(NULL, "out of memory\n");
- return tmp;
-}
diff -durpN postgresql/src/bin/pg_dump/dumpmem.h postgresql.2/src/bin/pg_dump/dumpmem.h
--- postgresql/src/bin/pg_dump/dumpmem.h 2012-10-03 10:40:48.301207415 +0200
+++ postgresql.2/src/bin/pg_dump/dumpmem.h 1970-01-01 01:00:00.000000000 +0100
@@ -1,22 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * dumpmem.h
- * Memory allocation routines used by pg_dump, pg_dumpall, and pg_restore
- *
- * Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
- *
- * src/bin/pg_dump/dumpmem.h
- *
- *-------------------------------------------------------------------------
- */
-
-#ifndef DUMPMEM_H
-#define DUMPMEM_H
-
-extern char *pg_strdup(const char *string);
-extern void *pg_malloc(size_t size);
-extern void *pg_malloc0(size_t size);
-extern void *pg_realloc(void *ptr, size_t size);
-
-#endif /* DUMPMEM_H */
diff -durpN postgresql/src/bin/pg_dump/Makefile postgresql.2/src/bin/pg_dump/Makefile
--- postgresql/src/bin/pg_dump/Makefile 2012-04-16 19:57:22.595917312 +0200
+++ postgresql.2/src/bin/pg_dump/Makefile 2012-11-30 09:49:47.085596058 +0100
@@ -20,7 +20,7 @@ override CPPFLAGS := -I$(libpq_srcdir) $
OBJS= pg_backup_archiver.o pg_backup_db.o pg_backup_custom.o \
pg_backup_null.o pg_backup_tar.o \
- pg_backup_directory.o dumpmem.o dumputils.o compress_io.o $(WIN32RES)
+ pg_backup_directory.o dumputils.o compress_io.o $(WIN32RES)
KEYWRDOBJS = keywords.o kwlookup.o
@@ -35,8 +35,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort
pg_restore: pg_restore.o $(OBJS) $(KEYWRDOBJS) | submake-libpq submake-libpgport
$(CC) $(CFLAGS) pg_restore.o $(KEYWRDOBJS) $(OBJS) $(libpq_pgport) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
-pg_dumpall: pg_dumpall.o dumputils.o dumpmem.o $(KEYWRDOBJS) | submake-libpq submake-libpgport
- $(CC) $(CFLAGS) pg_dumpall.o dumputils.o dumpmem.o $(KEYWRDOBJS) $(WIN32RES) $(libpq_pgport) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o $(KEYWRDOBJS) | submake-libpq submake-libpgport
+ $(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(KEYWRDOBJS) $(WIN32RES) $(libpq_pgport) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
install: all installdirs
$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff -durpN postgresql/src/bin/pg_dump/nls.mk postgresql.2/src/bin/pg_dump/nls.mk
--- postgresql/src/bin/pg_dump/nls.mk 2012-04-16 19:57:22.596917323 +0200
+++ postgresql.2/src/bin/pg_dump/nls.mk 2012-11-30 09:49:47.086596065 +0100
@@ -3,7 +3,7 @@ CATALOG_NAME = pg_dump
AVAIL_LANGUAGES = de es fr it ja ko pt_BR sv tr zh_CN zh_TW
GETTEXT_FILES = pg_backup_archiver.c pg_backup_db.c pg_backup_custom.c \
pg_backup_null.c pg_backup_tar.c \
- pg_backup_directory.c dumpmem.c dumputils.c compress_io.c \
+ pg_backup_directory.c dumputils.c compress_io.c \
pg_dump.c common.c pg_dump_sort.c \
pg_restore.c pg_dumpall.c \
../../port/exec.c
diff -durpN postgresql/src/bin/pg_dump/pg_backup_archiver.c postgresql.2/src/bin/pg_dump/pg_backup_archiver.c
--- postgresql/src/bin/pg_dump/pg_backup_archiver.c 2012-10-21 10:56:15.398945610 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_archiver.c 2012-11-30 09:49:47.088596080 +0100
@@ -21,7 +21,6 @@
*/
#include "pg_backup_db.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <ctype.h>
diff -durpN postgresql/src/bin/pg_dump/pg_backup_custom.c postgresql.2/src/bin/pg_dump/pg_backup_custom.c
--- postgresql/src/bin/pg_dump/pg_backup_custom.c 2012-10-03 10:40:48.303207429 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_custom.c 2012-11-30 09:49:47.089596088 +0100
@@ -26,7 +26,6 @@
#include "compress_io.h"
#include "dumputils.h"
-#include "dumpmem.h"
/*--------
* Routines in the format interface
diff -durpN postgresql/src/bin/pg_dump/pg_backup_db.c postgresql.2/src/bin/pg_dump/pg_backup_db.c
--- postgresql/src/bin/pg_dump/pg_backup_db.c 2012-07-25 10:51:14.973507544 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_db.c 2012-11-30 09:49:47.090596096 +0100
@@ -11,7 +11,6 @@
*/
#include "pg_backup_db.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <unistd.h>
diff -durpN postgresql/src/bin/pg_dump/pg_backup_directory.c postgresql.2/src/bin/pg_dump/pg_backup_directory.c
--- postgresql/src/bin/pg_dump/pg_backup_directory.c 2012-10-03 10:40:48.303207429 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_directory.c 2012-11-30 09:49:47.091596103 +0100
@@ -34,7 +34,6 @@
*/
#include "compress_io.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <dirent.h>
diff -durpN postgresql/src/bin/pg_dump/pg_backup_null.c postgresql.2/src/bin/pg_dump/pg_backup_null.c
--- postgresql/src/bin/pg_dump/pg_backup_null.c 2012-04-16 19:57:22.598917347 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_null.c 2012-11-30 09:49:47.092596110 +0100
@@ -23,7 +23,6 @@
*/
#include "pg_backup_archiver.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <unistd.h> /* for dup */
diff -durpN postgresql/src/bin/pg_dump/pg_backup_tar.c postgresql.2/src/bin/pg_dump/pg_backup_tar.c
--- postgresql/src/bin/pg_dump/pg_backup_tar.c 2012-10-03 10:40:48.304207436 +0200
+++ postgresql.2/src/bin/pg_dump/pg_backup_tar.c 2012-11-30 09:49:47.093596118 +0100
@@ -28,7 +28,6 @@
#include "pg_backup.h"
#include "pg_backup_archiver.h"
#include "pg_backup_tar.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <sys/stat.h>
diff -durpN postgresql/src/bin/pg_dump/pg_dumpall.c postgresql.2/src/bin/pg_dump/pg_dumpall.c
--- postgresql/src/bin/pg_dump/pg_dumpall.c 2012-10-14 11:09:08.143146185 +0200
+++ postgresql.2/src/bin/pg_dump/pg_dumpall.c 2012-11-30 09:49:47.095596134 +0100
@@ -23,7 +23,6 @@
#include "getopt_long.h"
#include "dumputils.h"
-#include "dumpmem.h"
#include "pg_backup.h"
/* version string we expect back from pg_dump */
diff -durpN postgresql/src/bin/pg_dump/pg_dump.c postgresql.2/src/bin/pg_dump/pg_dump.c
--- postgresql/src/bin/pg_dump/pg_dump.c 2012-10-28 07:41:12.051532033 +0100
+++ postgresql.2/src/bin/pg_dump/pg_dump.c 2012-11-30 09:49:47.100596172 +0100
@@ -59,7 +59,6 @@
#include "pg_backup_archiver.h"
#include "pg_backup_db.h"
-#include "dumpmem.h"
#include "dumputils.h"
extern char *optarg;
diff -durpN postgresql/src/bin/pg_dump/pg_dump_sort.c postgresql.2/src/bin/pg_dump/pg_dump_sort.c
--- postgresql/src/bin/pg_dump/pg_dump_sort.c 2012-10-19 19:12:00.334691199 +0200
+++ postgresql.2/src/bin/pg_dump/pg_dump_sort.c 2012-11-30 09:49:47.101596179 +0100
@@ -15,7 +15,6 @@
*/
#include "pg_backup_archiver.h"
#include "dumputils.h"
-#include "dumpmem.h"
/* translator: this is a module name */
static const char *modulename = gettext_noop("sorter");
diff -durpN postgresql/src/bin/pg_dump/pg_restore.c postgresql.2/src/bin/pg_dump/pg_restore.c
--- postgresql/src/bin/pg_dump/pg_restore.c 2012-10-14 11:09:08.143146185 +0200
+++ postgresql.2/src/bin/pg_dump/pg_restore.c 2012-11-30 09:49:47.101596179 +0100
@@ -41,7 +41,6 @@
#include "pg_backup_archiver.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <ctype.h>
diff -durpN postgresql/src/bin/psql/common.c postgresql.2/src/bin/psql/common.c
--- postgresql/src/bin/psql/common.c 2012-10-03 10:40:48.322207553 +0200
+++ postgresql.2/src/bin/psql/common.c 2012-11-30 09:49:47.102596186 +0100
@@ -33,56 +33,6 @@ static bool command_no_begin(const char
static bool is_select_command(const char *query);
/*
- * "Safe" wrapper around strdup()
- */
-char *
-pg_strdup(const char *string)
-{
- char *tmp;
-
- if (!string)
- {
- psql_error("%s: pg_strdup: cannot duplicate null pointer (internal error)\n",
- pset.progname);
- exit(EXIT_FAILURE);
- }
- tmp = strdup(string);
- if (!tmp)
- {
- psql_error("out of memory\n");
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc(size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- tmp = malloc(size);
- if (!tmp)
- {
- psql_error("out of memory\n");
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc0(size_t size)
-{
- void *tmp;
-
- tmp = pg_malloc(size);
- MemSet(tmp, 0, size);
- return tmp;
-}
-
-/*
* setQFout
* -- handler for -o command line option and \o command
*
diff -durpN postgresql/src/bin/psql/common.h postgresql.2/src/bin/psql/common.h
--- postgresql/src/bin/psql/common.h 2012-10-03 10:40:48.322207553 +0200
+++ postgresql.2/src/bin/psql/common.h 2012-11-30 09:49:47.102596186 +0100
@@ -21,15 +21,6 @@
#define atooid(x) ((Oid) strtoul((x), NULL, 10))
-/*
- * Safer versions of some standard C library functions. If an
- * out-of-memory condition occurs, these functions will bail out
- * safely; therefore, their return value is guaranteed to be non-NULL.
- */
-extern char *pg_strdup(const char *string);
-extern void *pg_malloc(size_t size);
-extern void *pg_malloc0(size_t size);
-
extern bool setQFout(const char *fname);
extern void
diff -durpN postgresql/src/bin/psql/startup.c postgresql.2/src/bin/psql/startup.c
--- postgresql/src/bin/psql/startup.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/psql/startup.c 2012-11-30 09:49:47.102596186 +0100
@@ -77,6 +77,21 @@ static void process_psqlrc_file(char *fi
static void showVersion(void);
static void EstablishVariableSpace(void);
+static void psql_malloc_oom(void)
+{
+ psql_error("out of memory\n");
+}
+
+static void psql_strdup_null(void)
+{
+ psql_error("%s: pg_strdup: cannot duplicate null pointer (internal error)\n",
+ pset.progname);
+}
+
+static PostgresFrontendData psql_fe_data = {
+ psql_malloc_oom, psql_strdup_null
+};
+
/*
*
* main
@@ -91,6 +106,8 @@ main(int argc, char *argv[])
char *password_prompt = NULL;
bool new_pass;
+ InitPostgresFrontend(&psql_fe_data);
+
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("psql"));
if (argc > 1)
diff -durpN postgresql/src/bin/scripts/clusterdb.c postgresql.2/src/bin/scripts/clusterdb.c
--- postgresql/src/bin/scripts/clusterdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/clusterdb.c 2012-11-30 09:49:47.103596194 +0100
@@ -61,6 +61,8 @@ main(int argc, char *argv[])
char *table = NULL;
bool verbose = false;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql/src/bin/scripts/common.c postgresql.2/src/bin/scripts/common.c
--- postgresql/src/bin/scripts/common.c 2012-10-03 10:40:48.324207567 +0200
+++ postgresql.2/src/bin/scripts/common.c 2012-11-30 09:49:47.104596202 +0100
@@ -31,6 +31,25 @@ static CRITICAL_SECTION cancelConnLock;
#endif
/*
+ * Helper functions to report error in pg_malloc and pg_strdup
+ */
+static void
+common_malloc_oom(void)
+{
+ fprintf(stderr, _("out of memory\n"));
+}
+
+static void
+common_strdup_null(void)
+{
+ fprintf(stderr, _("pg_strdup: cannot duplicate null pointer (internal error)\n"));
+}
+
+PostgresFrontendData common_fe_data = {
+ common_malloc_oom, common_strdup_null
+};
+
+/*
* Returns the current user name.
*/
const char *
@@ -278,55 +297,6 @@ executeMaintenanceCommand(PGconn *conn,
}
/*
- * "Safe" wrapper around strdup(). Pulled from psql/common.c
- */
-char *
-pg_strdup(const char *string)
-{
- char *tmp;
-
- if (!string)
- {
- fprintf(stderr, _("pg_strdup: cannot duplicate null pointer (internal error)\n"));
- exit(EXIT_FAILURE);
- }
- tmp = strdup(string);
- if (!tmp)
- {
- fprintf(stderr, _("out of memory\n"));
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc(size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- tmp = malloc(size);
- if (!tmp)
- {
- fprintf(stderr, _("out of memory\n"));
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc0(size_t size)
-{
- void *tmp;
-
- tmp = pg_malloc(size);
- MemSet(tmp, 0, size);
- return tmp;
-}
-
-/*
* Check yes/no answer in a localized way. 1=yes, 0=no, -1=neither.
*/
diff -durpN postgresql/src/bin/scripts/common.h postgresql.2/src/bin/scripts/common.h
--- postgresql/src/bin/scripts/common.h 2012-10-03 10:40:48.324207567 +0200
+++ postgresql.2/src/bin/scripts/common.h 2012-11-30 09:49:47.104596202 +0100
@@ -22,6 +22,8 @@ enum trivalue
typedef void (*help_handler) (const char *progname);
+extern PostgresFrontendData common_fe_data;
+
extern const char *get_user_name(const char *progname);
extern void handle_help_version_opts(int argc, char *argv[],
@@ -50,8 +52,4 @@ extern bool yesno_prompt(const char *que
extern void setup_cancel_handler(void);
-extern char *pg_strdup(const char *string);
-extern void *pg_malloc(size_t size);
-extern void *pg_malloc0(size_t size);
-
#endif /* COMMON_H */
diff -durpN postgresql/src/bin/scripts/createdb.c postgresql.2/src/bin/scripts/createdb.c
--- postgresql/src/bin/scripts/createdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/createdb.c 2012-11-30 09:49:47.105596209 +0100
@@ -64,6 +64,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql/src/bin/scripts/createlang.c postgresql.2/src/bin/scripts/createlang.c
--- postgresql/src/bin/scripts/createlang.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/createlang.c 2012-11-30 09:49:47.105596209 +0100
@@ -52,6 +52,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql/src/bin/scripts/createuser.c postgresql.2/src/bin/scripts/createuser.c
--- postgresql/src/bin/scripts/createuser.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/createuser.c 2012-11-30 09:49:47.106596217 +0100
@@ -78,6 +78,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql/src/bin/scripts/dropdb.c postgresql.2/src/bin/scripts/dropdb.c
--- postgresql/src/bin/scripts/dropdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/dropdb.c 2012-11-30 09:49:47.107596225 +0100
@@ -54,6 +54,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql/src/bin/scripts/droplang.c postgresql.2/src/bin/scripts/droplang.c
--- postgresql/src/bin/scripts/droplang.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/droplang.c 2012-11-30 09:49:47.107596225 +0100
@@ -51,6 +51,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql/src/bin/scripts/dropuser.c postgresql.2/src/bin/scripts/dropuser.c
--- postgresql/src/bin/scripts/dropuser.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/dropuser.c 2012-11-30 09:49:47.108596233 +0100
@@ -52,6 +52,8 @@ main(int argc, char *argv[])
PGconn *conn;
PGresult *result;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql/src/bin/scripts/reindexdb.c postgresql.2/src/bin/scripts/reindexdb.c
--- postgresql/src/bin/scripts/reindexdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/reindexdb.c 2012-11-30 09:49:47.108596233 +0100
@@ -67,6 +67,8 @@ main(int argc, char *argv[])
const char *table = NULL;
const char *index = NULL;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql/src/bin/scripts/vacuumdb.c postgresql.2/src/bin/scripts/vacuumdb.c
--- postgresql/src/bin/scripts/vacuumdb.c 2012-10-14 11:09:08.144146192 +0200
+++ postgresql.2/src/bin/scripts/vacuumdb.c 2012-11-30 09:49:47.109596240 +0100
@@ -72,6 +72,8 @@ main(int argc, char *argv[])
bool full = false;
bool verbose = false;
+ InitPostgresFrontend(&common_fe_data);
+
progname = get_progname(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pgscripts"));
diff -durpN postgresql/src/include/port.h postgresql.2/src/include/port.h
--- postgresql/src/include/port.h 2012-07-22 16:48:48.548857900 +0200
+++ postgresql.2/src/include/port.h 2012-11-30 09:49:47.109596240 +0100
@@ -17,6 +17,19 @@
#include <netdb.h>
#include <pwd.h>
+/* port/init.c */
+typedef void (*fe_func)(void);
+
+typedef struct PostgresFrontendData
+{
+ fe_func pg_malloc_oom;
+ fe_func pg_strdup_null;
+ fe_func app_atexit;
+} PostgresFrontendData, *PostgresFrontend;
+extern PostgresFrontend pg_fedata;
+
+extern void InitPostgresFrontend(PostgresFrontend fe_data);
+
/* socket has a different definition on WIN32 */
#ifndef WIN32
typedef int pgsocket;
@@ -462,6 +475,19 @@ extern char *inet_net_ntop(int af, const
/* port/pgcheckdir.c */
extern int pg_check_dir(const char *dir);
+/* port/pgmalloc.c */
+extern void pgmalloc_atexit(void);
+
+extern char *pg_strdup(const char *string);
+
+extern void *pg_malloc(size_t size);
+
+extern void *pg_malloc0(size_t size);
+
+extern void *pg_realloc(void *ptr, size_t size);
+
+extern void pg_free(void *ptr);
+
/* port/pgmkdirp.c */
extern int pg_mkdir_p(char *path, int omode);
diff -durpN postgresql/src/port/init.c postgresql.2/src/port/init.c
--- postgresql/src/port/init.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.2/src/port/init.c 2012-11-30 09:49:47.109596240 +0100
@@ -0,0 +1,32 @@
+/*-------------------------------------------------------------------------
+ *
+ * init.c
+ * Functions for initializing frontend functions
+ *
+ *
+ * Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/init.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef FRONTEND
+#include "postgres_fe.h"
+
+PostgresFrontend pg_fedata = NULL;
+
+void InitPostgresFrontend(PostgresFrontend ptr)
+{
+ pg_fedata = ptr;
+
+ atexit(pgmalloc_atexit);
+
+ if (ptr && ptr->app_atexit)
+ atexit(ptr->app_atexit);
+}
+
+#endif
diff -durpN postgresql/src/port/Makefile postgresql.2/src/port/Makefile
--- postgresql/src/port/Makefile 2012-05-09 12:28:11.755496856 +0200
+++ postgresql.2/src/port/Makefile 2012-11-30 09:49:47.110596247 +0100
@@ -30,9 +30,10 @@ include $(top_builddir)/src/Makefile.glo
override CPPFLAGS := -I$(top_builddir)/src/port -DFRONTEND $(CPPFLAGS)
LIBS += $(PTHREAD_LIBS)
-OBJS = $(LIBOBJS) chklocale.o dirmod.o erand48.o exec.o fls.o inet_net_ntop.o \
- noblock.o path.o pgcheckdir.o pg_crc.o pgmkdirp.o pgsleep.o \
- pgstrcasecmp.o qsort.o qsort_arg.o sprompt.o thread.o
+OBJS = $(LIBOBJS) chklocale.o dirmod.o erand48.o exec.o fls.o init.o \
+ inet_net_ntop.o noblock.o path.o pgcheckdir.o pg_crc.o pgmalloc.o \
+ pgmkdirp.o pgsleep.o pgstrcasecmp.o qsort.o qsort_arg.o sprompt.o \
+ thread.o
# foo_srv.o and foo.o are both built from foo.c, but only foo.o has -DFRONTEND
OBJS_SRV = $(OBJS:%.o=%_srv.o)
diff -durpN postgresql/src/port/pgmalloc.c postgresql.2/src/port/pgmalloc.c
--- postgresql/src/port/pgmalloc.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.2/src/port/pgmalloc.c 2012-11-30 09:49:47.111596255 +0100
@@ -0,0 +1,125 @@
+/*-------------------------------------------------------------------------
+ *
+ * pgmalloc.c
+ * Functions for allocating memory and
+ * exiting on out-of-memory
+ *
+ *
+ * Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pgmalloc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef FRONTEND
+#include "postgres_fe.h"
+
+#include <stdio.h>
+
+typedef enum {
+ PG_MALLOC_NOP,
+ PG_MALLOC_OOM,
+ PG_STRDUP_NULL
+} PgMallocAction;
+
+static PgMallocAction pgmalloc_action = PG_MALLOC_NOP;
+
+void pgmalloc_atexit(void)
+{
+ switch (pgmalloc_action)
+ {
+ case PG_MALLOC_OOM:
+ if (pg_fedata && pg_fedata->pg_malloc_oom)
+ pg_fedata->pg_malloc_oom();
+ else
+ fprintf(stderr, "out of memory\n");
+ break;
+ case PG_STRDUP_NULL:
+ if (pg_fedata && pg_fedata->pg_strdup_null)
+ pg_fedata->pg_strdup_null();
+ else
+ fprintf(stderr, "pg_strdup: cannot duplicate null pointer (internal error)\n");
+ break;
+ case PG_MALLOC_NOP:
+ break;
+ }
+}
+
+/*
+ * "Safe" wrapper around strdup()
+ */
+char *
+pg_strdup(const char *string)
+{
+ char *tmp;
+
+ if (!string)
+ {
+ pgmalloc_action = PG_STRDUP_NULL;
+ exit(1);
+ }
+ tmp = strdup(string);
+ if (!tmp)
+ {
+ pgmalloc_action = PG_MALLOC_OOM;
+ exit(1);
+ }
+ return tmp;
+}
+
+void *
+pg_malloc(size_t size)
+{
+ void *tmp;
+
+ /* Avoid unportable behavior of malloc(0) */
+ if (size == 0)
+ size = 1;
+ tmp = malloc(size);
+ if (!tmp)
+ {
+ pgmalloc_action = PG_MALLOC_OOM;
+ exit(1);
+ }
+ return tmp;
+}
+
+void *
+pg_malloc0(size_t size)
+{
+ void *tmp;
+
+ tmp = pg_malloc(size);
+ MemSet(tmp, 0, size);
+ return tmp;
+}
+
+void *
+pg_realloc(void *ptr, size_t size)
+{
+ void *tmp;
+
+ /* Avoid unportable behavior of realloc(NULL, 0) */
+ if (ptr == NULL && size == 0)
+ size = 1;
+ tmp = realloc(ptr, size);
+ if (!tmp)
+ {
+ pgmalloc_action = PG_MALLOC_OOM;
+ exit(1);
+ }
+ return tmp;
+}
+
+void
+pg_free(void *ptr)
+{
+ if (ptr != NULL)
+ free(ptr);
+}
+
+#endif
03-tarutils-strutils-v4.patchtext/x-patch; name=03-tarutils-strutils-v4.patchDownload
diff -durpN postgresql.2/src/backend/replication/basebackup.c postgresql.3/src/backend/replication/basebackup.c
--- postgresql.2/src/backend/replication/basebackup.c 2012-10-06 19:25:21.002166521 +0200
+++ postgresql.3/src/backend/replication/basebackup.c 2012-11-30 09:50:22.985851922 +0100
@@ -721,48 +721,12 @@ sendDir(char *path, int basepathlen, boo
/*
- * Utility routine to print possibly larger than 32 bit integers in a
- * portable fashion. Filled with zeros.
- */
-static void
-print_val(char *s, uint64 val, unsigned int base, size_t len)
-{
- int i;
-
- for (i = len; i > 0; i--)
- {
- int digit = val % base;
-
- s[i - 1] = '0' + digit;
- val = val / base;
- }
-}
-
-/*
* Maximum file size for a tar member: The limit inherent in the
* format is 2^33-1 bytes (nearly 8 GB). But we don't want to exceed
* what we can represent in pgoff_t.
*/
#define MAX_TAR_MEMBER_FILELEN (((int64) 1 << Min(33, sizeof(pgoff_t)*8 - 1)) - 1)
-static int
-_tarChecksum(char *header)
-{
- int i,
- sum;
-
- /*
- * Per POSIX, the checksum is the simple sum of all bytes in the header,
- * treating the bytes as unsigned, and treating the checksum field (at
- * offset 148) as though it contained 8 spaces.
- */
- sum = 8 * ' '; /* presumed value for checksum field */
- for (i = 0; i < 512; i++)
- if (i < 148 || i >= 156)
- sum += 0xFF & header[i];
- return sum;
-}
-
/* Given the member, write the TAR header & send the file */
static void
sendFile(char *readfilename, char *tarfilename, struct stat * statbuf)
@@ -840,94 +804,12 @@ _tarWriteHeader(const char *filename, co
{
char h[512];
- /*
- * Note: most of the fields in a tar header are not supposed to be
- * null-terminated. We use sprintf, which will write a null after the
- * required bytes; that null goes into the first byte of the next field.
- * This is okay as long as we fill the fields in order.
- */
- memset(h, 0, sizeof(h));
-
- /* Name 100 */
- sprintf(&h[0], "%.99s", filename);
- if (linktarget != NULL || S_ISDIR(statbuf->st_mode))
- {
- /*
- * We only support symbolic links to directories, and this is
- * indicated in the tar format by adding a slash at the end of the
- * name, the same as for regular directories.
- */
- int flen = strlen(filename);
-
- flen = Min(flen, 99);
- h[flen] = '/';
- h[flen + 1] = '\0';
- }
-
- /* Mode 8 */
- sprintf(&h[100], "%07o ", (int) statbuf->st_mode);
-
- /* User ID 8 */
- sprintf(&h[108], "%07o ", statbuf->st_uid);
-
- /* Group 8 */
- sprintf(&h[116], "%07o ", statbuf->st_gid);
-
- /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
- if (linktarget != NULL || S_ISDIR(statbuf->st_mode))
- /* Symbolic link or directory has size zero */
- print_val(&h[124], 0, 8, 11);
- else
- print_val(&h[124], statbuf->st_size, 8, 11);
- sprintf(&h[135], " ");
-
- /* Mod Time 12 */
- sprintf(&h[136], "%011o ", (int) statbuf->st_mtime);
-
- /* Checksum 8 cannot be calculated until we've filled all other fields */
-
- if (linktarget != NULL)
- {
- /* Type - Symbolic link */
- sprintf(&h[156], "2");
- /* Link Name 100 */
- sprintf(&h[157], "%.99s", linktarget);
- }
- else if (S_ISDIR(statbuf->st_mode))
- /* Type - directory */
- sprintf(&h[156], "5");
- else
- /* Type - regular file */
- sprintf(&h[156], "0");
-
- /* Magic 6 */
- sprintf(&h[257], "ustar");
-
- /* Version 2 */
- sprintf(&h[263], "00");
-
- /* User 32 */
- /* XXX: Do we need to care about setting correct username? */
- sprintf(&h[265], "%.31s", "postgres");
-
- /* Group 32 */
- /* XXX: Do we need to care about setting correct group name? */
- sprintf(&h[297], "%.31s", "postgres");
-
- /* Major Dev 8 */
- sprintf(&h[329], "%07o ", 0);
-
- /* Minor Dev 8 */
- sprintf(&h[337], "%07o ", 0);
-
- /* Prefix 155 - not used, leave as nulls */
-
- /*
- * We mustn't overwrite the next field while inserting the checksum.
- * Fortunately, the checksum can't exceed 6 octal digits, so we just write
- * 6 digits, a space, and a null, which is legal per POSIX.
- */
- sprintf(&h[148], "%06o ", _tarChecksum(h));
+ _tarCreateHeader(h, filename, linktarget,
+ statbuf->st_size,
+ statbuf->st_mode,
+ statbuf->st_uid,
+ statbuf->st_gid,
+ statbuf->st_mtime);
/* Now send the completed header. */
pq_putmessage('d', h, 512);
diff -durpN postgresql.2/src/bin/initdb/initdb.c postgresql.3/src/bin/initdb/initdb.c
--- postgresql.2/src/bin/initdb/initdb.c 2012-11-30 09:49:47.076595989 +0100
+++ postgresql.3/src/bin/initdb/initdb.c 2012-11-30 09:50:22.987851938 +0100
@@ -218,7 +218,6 @@ static void make_postgres(void);
static void perform_fsync(void);
static void trapsig(int signum);
static void check_ok(void);
-static char *escape_quotes(const char *src);
static int locale_date_order(const char *locale);
static bool check_locale_name(int category, const char *locale,
char **canonname);
@@ -2342,35 +2341,6 @@ check_ok(void)
}
}
-/*
- * Escape (by doubling) any single quotes or backslashes in given string
- *
- * Note: this is used to process both postgresql.conf entries and SQL
- * string literals. Since postgresql.conf strings are defined to treat
- * backslashes as escapes, we have to double backslashes here. Hence,
- * when using this for a SQL string literal, use E'' syntax.
- *
- * We do not need to worry about encoding considerations because all
- * valid backend encodings are ASCII-safe.
- */
-static char *
-escape_quotes(const char *src)
-{
- int len = strlen(src),
- i,
- j;
- char *result = pg_malloc(len * 2 + 1);
-
- for (i = 0, j = 0; i < len; i++)
- {
- if (SQL_STR_DOUBLE(src[i], true))
- result[j++] = src[i];
- result[j++] = src[i];
- }
- result[j] = '\0';
- return result;
-}
-
/* Hack to suppress a warning about %x from some versions of gcc */
static inline size_t
my_strftime(char *s, size_t max, const char *fmt, const struct tm * tm)
diff -durpN postgresql.2/src/bin/pg_dump/pg_backup_tar.c postgresql.3/src/bin/pg_dump/pg_backup_tar.c
--- postgresql.2/src/bin/pg_dump/pg_backup_tar.c 2012-11-30 09:49:47.093596118 +0100
+++ postgresql.3/src/bin/pg_dump/pg_backup_tar.c 2012-11-30 09:50:22.988851945 +0100
@@ -114,7 +114,6 @@ static char *tarGets(char *buf, size_t l
static int tarPrintf(ArchiveHandle *AH, TAR_MEMBER *th, const char *fmt,...) __attribute__((format(PG_PRINTF_ATTRIBUTE, 3, 4)));
static void _tarAddFile(ArchiveHandle *AH, TAR_MEMBER *th);
-static int _tarChecksum(char *th);
static TAR_MEMBER *_tarPositionTo(ArchiveHandle *AH, const char *filename);
static size_t tarRead(void *buf, size_t len, TAR_MEMBER *th);
static size_t tarWrite(const void *buf, size_t len, TAR_MEMBER *th);
@@ -1015,24 +1014,6 @@ tarPrintf(ArchiveHandle *AH, TAR_MEMBER
return cnt;
}
-static int
-_tarChecksum(char *header)
-{
- int i,
- sum;
-
- /*
- * Per POSIX, the checksum is the simple sum of all bytes in the header,
- * treating the bytes as unsigned, and treating the checksum field (at
- * offset 148) as though it contained 8 spaces.
- */
- sum = 8 * ' '; /* presumed value for checksum field */
- for (i = 0; i < 512; i++)
- if (i < 148 || i >= 156)
- sum += 0xFF & header[i];
- return sum;
-}
-
bool
isValidTarHeader(char *header)
{
@@ -1304,94 +1285,16 @@ _tarGetHeader(ArchiveHandle *AH, TAR_MEM
}
-/*
- * Utility routine to print possibly larger than 32 bit integers in a
- * portable fashion. Filled with zeros.
- */
-static void
-print_val(char *s, uint64 val, unsigned int base, size_t len)
-{
- int i;
-
- for (i = len; i > 0; i--)
- {
- int digit = val % base;
-
- s[i - 1] = '0' + digit;
- val = val / base;
- }
-}
-
-
static void
_tarWriteHeader(TAR_MEMBER *th)
{
char h[512];
- /*
- * Note: most of the fields in a tar header are not supposed to be
- * null-terminated. We use sprintf, which will write a null after the
- * required bytes; that null goes into the first byte of the next field.
- * This is okay as long as we fill the fields in order.
- */
- memset(h, 0, sizeof(h));
-
- /* Name 100 */
- sprintf(&h[0], "%.99s", th->targetFile);
-
- /* Mode 8 */
- sprintf(&h[100], "0000600 ");
-
- /* User ID 8 */
- sprintf(&h[108], "0004000 ");
-
- /* Group 8 */
- sprintf(&h[116], "0002000 ");
-
- /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
- print_val(&h[124], th->fileLen, 8, 11);
- sprintf(&h[135], " ");
-
- /* Mod Time 12 */
- sprintf(&h[136], "%011o ", (int) time(NULL));
-
- /* Checksum 8 cannot be calculated until we've filled all other fields */
-
- /* Type - regular file */
- sprintf(&h[156], "0");
-
- /* Link Name 100 (leave as nulls) */
-
- /* Magic 6 */
- sprintf(&h[257], "ustar");
-
- /* Version 2 */
- sprintf(&h[263], "00");
-
- /* User 32 */
- /* XXX: Do we need to care about setting correct username? */
- sprintf(&h[265], "%.31s", "postgres");
-
- /* Group 32 */
- /* XXX: Do we need to care about setting correct group name? */
- sprintf(&h[297], "%.31s", "postgres");
-
- /* Major Dev 8 */
- sprintf(&h[329], "%07o ", 0);
-
- /* Minor Dev 8 */
- sprintf(&h[337], "%07o ", 0);
-
- /* Prefix 155 - not used, leave as nulls */
-
- /*
- * We mustn't overwrite the next field while inserting the checksum.
- * Fortunately, the checksum can't exceed 6 octal digits, so we just write
- * 6 digits, a space, and a null, which is legal per POSIX.
- */
- sprintf(&h[148], "%06o ", _tarChecksum(h));
+ _tarCreateHeader(h, th->targetFile, NULL,
+ th->fileLen,
+ 0600, 04000, 02000,
+ time(NULL));
- /* Now write the completed header. */
if (fwrite(h, 1, 512, th->tarFH) != 512)
exit_horribly(modulename, "could not write to output file: %s\n", strerror(errno));
}
diff -durpN postgresql.2/src/include/port.h postgresql.3/src/include/port.h
--- postgresql.2/src/include/port.h 2012-11-30 09:49:47.109596240 +0100
+++ postgresql.3/src/include/port.h 2012-11-30 09:50:22.989851952 +0100
@@ -16,6 +16,7 @@
#include <ctype.h>
#include <netdb.h>
#include <pwd.h>
+#include <time.h>
/* port/init.c */
typedef void (*fe_func)(void);
@@ -491,4 +492,22 @@ extern void pg_free(void *ptr);
/* port/pgmkdirp.c */
extern int pg_mkdir_p(char *path, int omode);
+/* port/strutil.c */
+extern void print_val(char *s, uint64 val, unsigned int base, size_t len);
+extern void scan_val(char *s, uint64 *val, unsigned int base, size_t len);
+extern char *escape_quotes(const char *src);
+
+/* port/tarutil.c */
+
+/* Don't ever change this value, the TAR file format requires it. */
+#define TARCHUNKSZ 512
+
+extern int _tarChecksum(char *header);
+extern void _tarCreateHeader(char *header,
+ const char *filename,
+ const char *linktarget,
+ int64 filesize, int st_mode,
+ int st_uid, int st_gid,
+ time_t mtime);
+
#endif /* PG_PORT_H */
diff -durpN postgresql.2/src/port/Makefile postgresql.3/src/port/Makefile
--- postgresql.2/src/port/Makefile 2012-11-30 09:49:47.110596247 +0100
+++ postgresql.3/src/port/Makefile 2012-11-30 09:50:22.989851952 +0100
@@ -33,7 +33,7 @@ LIBS += $(PTHREAD_LIBS)
OBJS = $(LIBOBJS) chklocale.o dirmod.o erand48.o exec.o fls.o init.o \
inet_net_ntop.o noblock.o path.o pgcheckdir.o pg_crc.o pgmalloc.o \
pgmkdirp.o pgsleep.o pgstrcasecmp.o qsort.o qsort_arg.o sprompt.o \
- thread.o
+ strutil.o tarutil.o thread.o
# foo_srv.o and foo.o are both built from foo.c, but only foo.o has -DFRONTEND
OBJS_SRV = $(OBJS:%.o=%_srv.o)
diff -durpN postgresql.2/src/port/strutil.c postgresql.3/src/port/strutil.c
--- postgresql.2/src/port/strutil.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.3/src/port/strutil.c 2012-11-30 09:50:22.990851960 +0100
@@ -0,0 +1,87 @@
+/*-------------------------------------------------------------------------
+ *
+ * strutil.c
+ * common string functions
+ *
+ * IDENTIFICATION
+ * src/port/strutil.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FRONTEND
+#include "postgres.h"
+#include "utils/palloc.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+/*
+ * Utility routine to print possibly larger than 32 bit integers in a
+ * portable fashion. Filled with zeros.
+ */
+void
+print_val(char *s, uint64 val, unsigned int base, size_t len)
+{
+ int i;
+
+ for (i = len; i > 0; i--)
+ {
+ int digit = val % base;
+
+ s[i - 1] = '0' + digit;
+ val = val / base;
+ }
+}
+
+
+/*
+ * Inverse of print_val()
+ */
+void
+scan_val(char *s, uint64 *val, unsigned int base, size_t len)
+{
+ uint64 tmp = 0;
+ int i;
+
+ for (i = 0; i < len; i++)
+ {
+ int digit = s[i] - '0';
+
+ tmp = tmp * base + digit;
+ }
+
+ *val = tmp;
+}
+
+/*
+ * Escape (by doubling) any single quotes or backslashes in given string
+ *
+ * Note: this is used to process both postgresql.conf entries and SQL
+ * string literals. Since postgresql.conf strings are defined to treat
+ * backslashes as escapes, we have to double backslashes here. Hence,
+ * when using this for a SQL string literal, use E'' syntax.
+ *
+ * We do not need to worry about encoding considerations because all
+ * valid backend encodings are ASCII-safe.
+ */
+char *
+escape_quotes(const char *src)
+{
+ int len = strlen(src),
+ i,
+ j;
+#ifdef FRONTEND
+ char *result = pg_malloc(len * 2 + 1);
+#else
+ char *result = palloc(len * 2 + 1);
+#endif
+
+ for (i = 0, j = 0; i < len; i++)
+ {
+ if (SQL_STR_DOUBLE(src[i], true))
+ result[j++] = src[i];
+ result[j++] = src[i];
+ }
+ result[j] = '\0';
+ return result;
+}
diff -durpN postgresql.2/src/port/tarutil.c postgresql.3/src/port/tarutil.c
--- postgresql.2/src/port/tarutil.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.3/src/port/tarutil.c 2012-11-30 09:50:22.991851968 +0100
@@ -0,0 +1,135 @@
+/*-------------------------------------------------------------------------
+ *
+ * tarutil.c
+ * common functions for handling the TAR format
+ *
+ * IDENTIFICATION
+ * src/port/tarutil.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef FRONTEND
+#include "postgres.h"
+#else
+#include "postgres_fe.h"
+#endif
+
+#include <sys/stat.h>
+
+int
+_tarChecksum(char *header)
+{
+ int i,
+ sum;
+
+ /*
+ * Per POSIX, the checksum is the simple sum of all bytes in the header,
+ * treating the bytes as unsigned, and treating the checksum field (at
+ * offset 148) as though it contained 8 spaces.
+ */
+ sum = 8 * ' '; /* presumed value for checksum field */
+ for (i = 0; i < TARCHUNKSZ; i++)
+ if (i < 148 || i >= 156)
+ sum += 0xFF & header[i];
+ return sum;
+}
+
+
+void
+_tarCreateHeader(char *header, const char *filename,
+ const char *linktarget,
+ int64 filesize, int st_mode,
+ int st_uid, int st_gid,
+ time_t mtime)
+{
+ /*
+ * Note: most of the fields in a tar header are not supposed to be
+ * null-terminated. We use sprintf, which will write a null after the
+ * required bytes; that null goes into the first byte of the next field.
+ * This is okay as long as we fill the fields in order.
+ */
+ memset(header, 0, TARCHUNKSZ /* sizeof the tar header */);
+
+ /* Name 100 */
+ sprintf(&header[0], "%.99s", filename);
+ if (linktarget != NULL || S_ISDIR(st_mode))
+ {
+ /*
+ * We only support symbolic links to directories, and this is
+ * indicated in the tar format by adding a slash at the end of the
+ * name, the same as for regular directories.
+ */
+ int flen = strlen(filename);
+
+ flen = Min(flen, 99);
+ header[flen] = '/';
+ header[flen + 1] = '\0';
+ }
+
+ /* Mode 8 */
+ sprintf(&header[100], "%07o ", st_mode);
+
+ /* User ID 8 */
+ sprintf(&header[108], "%07o ", st_uid);
+
+ /* Group 8 */
+ sprintf(&header[116], "%07o ", st_gid);
+
+ /* File size 12 - 11 digits, 1 space; use print_val for 64 bit support */
+ if (linktarget != NULL || S_ISDIR(st_mode))
+ /* Symbolic link or directory has size zero */
+ print_val(&header[124], 0, 8, 11);
+ else
+ print_val(&header[124], filesize, 8, 11);
+ sprintf(&header[135], " ");
+
+ /* Mod Time 12 - 11 digits, 1 space; use print_val for 64 bit support */
+ print_val(&header[136], mtime, 8, 11);
+ sprintf(&header[147], " ");
+
+ /* Checksum 8 cannot be calculated until we've filled all other fields */
+
+ if (linktarget != NULL)
+ {
+ /* Type - Symbolic link */
+ sprintf(&header[156], "2");
+ /* Link Name 100 */
+ sprintf(&header[157], "%.99s", linktarget);
+ }
+ else if (S_ISDIR(st_mode))
+ /* Type - directory */
+ sprintf(&header[156], "5");
+ else
+ /* Type - regular file */
+ sprintf(&header[156], "0");
+
+ /* Magic 6 */
+ sprintf(&header[257], "ustar");
+
+ /* Version 2 */
+ sprintf(&header[263], "00");
+
+ /* User 32 */
+ /* XXX: Do we need to care about setting correct username? */
+ sprintf(&header[265], "%.31s", "postgres");
+
+ /* Group 32 */
+ /* XXX: Do we need to care about setting correct group name? */
+ sprintf(&header[297], "%.31s", "postgres");
+
+ /* Major Dev 8 */
+ sprintf(&header[329], "%07o ", 0);
+
+ /* Minor Dev 8 */
+ sprintf(&header[337], "%07o ", 0);
+
+ /* Prefix 155 - not used, leave as nulls */
+
+ /*
+ * We mustn't overwrite the next field while inserting the checksum.
+ * Fortunately, the checksum can't exceed 6 octal digits, so we just write
+ * 6 digits, a space, and a null, which is legal per POSIX.
+ */
+ sprintf(&header[148], "%06o ", _tarChecksum(header));
+}
+
04-pg_basebackup-v19.patchtext/x-patch; name=04-pg_basebackup-v19.patchDownload
diff -durpN postgresql.3/doc/src/sgml/ref/pg_basebackup.sgml postgresql.4/doc/src/sgml/ref/pg_basebackup.sgml
--- postgresql.3/doc/src/sgml/ref/pg_basebackup.sgml 2012-11-08 13:13:04.151630632 +0100
+++ postgresql.4/doc/src/sgml/ref/pg_basebackup.sgml 2012-11-30 09:50:48.710034824 +0100
@@ -189,6 +189,21 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--write-recovery-conf</option></term>
+ <listitem>
+
+ <para>
+ Write a minimal recovery.conf into the output directory (or into
+ the base archive file if <option>--format=tar</option> was specified)
+ using the connection parameters from the command line to ease
+ setting up the standby.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-x</option></term>
<term><option>--xlog</option></term>
<listitem>
diff -durpN postgresql.3/src/bin/pg_basebackup/pg_basebackup.c postgresql.4/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.3/src/bin/pg_basebackup/pg_basebackup.c 2012-11-30 09:49:47.077595997 +0100
+++ postgresql.4/src/bin/pg_basebackup/pg_basebackup.c 2012-11-30 10:01:45.805424225 +0100
@@ -19,6 +19,7 @@
#define FRONTEND 1
#include "postgres.h"
#include "libpq-fe.h"
+#include "pqexpbuffer.h"
#include <unistd.h>
#include <dirent.h>
@@ -46,6 +47,7 @@ int compresslevel = 0;
bool includewal = false;
bool streamwal = false;
bool fastcheckpoint = false;
+bool writerecoveryconf = false;
int standby_message_timeout = 10 * 1000; /* 10 sec = default */
/* Progress counters */
@@ -70,6 +72,8 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
+PQExpBuffer rcExpBuf = NULL;
+
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
@@ -77,6 +81,8 @@ static void progress_report(int tablespa
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
+static void CreateRecoveryConf(PGconn *conn);
+static void WriteRecoveryConf(void);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -107,6 +113,8 @@ usage(void)
printf(_("\nOptions controlling the output:\n"));
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
+ printf(_(" -R, --write-recovery-conf\n"
+ " write recovery.conf after backup\n"));
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
printf(_(" -X, --xlog-method=fetch|stream\n"
" include required WAL files with specified method\n"));
@@ -452,6 +460,45 @@ progress_report(int tablespacenum, const
/*
+ * Write a piece of tar data
+ */
+static void
+writeTarData(
+#ifdef HAVE_LIBZ
+ gzFile ztarfile,
+#endif
+ FILE *tarfile, char *buf, int r, char *current_file)
+{
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (gzwrite(ztarfile, buf, r) != r)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, current_file, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (fwrite(buf, r, 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
+ progname, current_file, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ }
+}
+
+#ifdef HAVE_LIBZ
+#define WRITE_TAR_DATA(buf, sz) writeTarData(ztarfile, tarfile, buf, sz, filename)
+#else
+#define WRITE_TAR_DATA(buf, sz) writeTarData(tarfile, buf, sz, filename)
+#endif
+
+/*
* Receive a tar format file from the connection to the server, and write
* the data from this file directly into a tar file. If compression is
* enabled, the data will be compressed while written to the file.
@@ -467,12 +514,17 @@ ReceiveTarFile(PGconn *conn, PGresult *r
char filename[MAXPGPATH];
char *copybuf = NULL;
FILE *tarfile = NULL;
+ char tarhdr[TARCHUNKSZ];
+ bool basetablespace = PQgetisnull(res, rownum, 0);
+ bool in_tarhdr, skip_file;
+ size_t tarhdrsz;
+ uint64 filesz;
#ifdef HAVE_LIBZ
gzFile ztarfile = NULL;
#endif
- if (PQgetisnull(res, rownum, 0))
+ if (basetablespace)
{
/*
* Base tablespaces
@@ -584,6 +636,16 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
+ /*
+ * Initialize our variables for tracking
+ * individual files inside the TAR stream.
+ * For more detailed explanation, see below.
+ */
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = 0;
+ filesz = 0;
+
while (1)
{
int r;
@@ -598,38 +660,39 @@ ReceiveTarFile(PGconn *conn, PGresult *r
if (r == -1)
{
/*
- * End of chunk. Close file (but not stdout).
+ * End of chunk. Write recovery.conf into the tar file (if it
+ * was requested) and close file (but not stdout).
*
* Also, write two completely empty blocks at the end of the tar
* file, as required by some tar programs.
*/
- char zerobuf[1024];
+ char zerobuf[2*TARCHUNKSZ];
MemSet(zerobuf, 0, sizeof(zerobuf));
-#ifdef HAVE_LIBZ
- if (ztarfile != NULL)
- {
- if (gzwrite(ztarfile, zerobuf, sizeof(zerobuf)) !=
- sizeof(zerobuf))
- {
- fprintf(stderr,
- _("%s: could not write to compressed file \"%s\": %s\n"),
- progname, filename, get_gz_error(ztarfile));
- disconnect_and_exit(1);
- }
- }
- else
-#endif
+
+ if (basetablespace && writerecoveryconf)
{
- if (fwrite(zerobuf, sizeof(zerobuf), 1, tarfile) != 1)
- {
- fprintf(stderr,
- _("%s: could not write to file \"%s\": %s\n"),
- progname, filename, strerror(errno));
- disconnect_and_exit(1);
- }
+ char header[TARCHUNKSZ];
+ int padding;
+
+ _tarCreateHeader(header, "recovery.conf", NULL,
+ rcExpBuf->len,
+ 0600, 04000, 02000,
+ time(NULL));
+
+ padding = ((rcExpBuf->len + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - rcExpBuf->len;
+
+ WRITE_TAR_DATA(header, sizeof(header));
+ WRITE_TAR_DATA(rcExpBuf->data, rcExpBuf->len);
+ if (padding)
+ WRITE_TAR_DATA(zerobuf, padding);
+
+ if (verbose)
+ fprintf(stderr, _("%s: recovery.conf written into '%s'\n"), progname, filename);
}
+ WRITE_TAR_DATA(zerobuf, sizeof(zerobuf));
+
#ifdef HAVE_LIBZ
if (ztarfile != NULL)
{
@@ -665,25 +728,124 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
-#ifdef HAVE_LIBZ
- if (ztarfile != NULL)
+ if (!writerecoveryconf || !basetablespace)
{
- if (gzwrite(ztarfile, copybuf, r) != r)
- {
- fprintf(stderr,
- _("%s: could not write to compressed file \"%s\": %s\n"),
- progname, filename, get_gz_error(ztarfile));
- disconnect_and_exit(1);
- }
+ /*
+ * If --write-recovery-conf was not requested or this
+ * is not the base tablespace, simply pass the received
+ * data into the TAR file, either compressed or not.
+ */
+
+ WRITE_TAR_DATA(copybuf, r);
}
else
-#endif
{
- if (fwrite(copybuf, r, 1, tarfile) != 1)
+ /*
+ * If --write-recovery-conf was requested AND this
+ * is the base tablespace, the TAR stream may contain
+ * a recovery.conf file if the backup is coming from
+ * a standby server. We have to skip this file in
+ * the stream and add a new one constructed by
+ * CreateRecoveryConf() at the end of the stream.
+ *
+ * To do this, we have to process the individual files
+ * inside the TAR stream. The stream consists of a header
+ * and zero or more chunks, all 512 bytes long. The stream
+ * from the server is broken up into smaller pieces, so
+ * we have to track the size of the files to find the next
+ * header structure.
+ */
+ int rr = r; /* Save the value returned by PQgetCopyData */
+ int pos = 0;
+
+ while (rr > 0)
{
- fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
- progname, filename, strerror(errno));
- disconnect_and_exit(1);
+ if (in_tarhdr)
+ {
+ /*
+ * We're currently reading a header structure
+ * inside the TAR stream, i.e. the file metadata.
+ */
+ if (tarhdrsz < TARCHUNKSZ)
+ {
+ /*
+ * Copy the header structure into tarhdr[]
+ * in case the header is not aligned to 512 bytes
+ * or it's not returned in whole by the last
+ * PQgetCopyData call.
+ */
+ int hdrleft, bytes2copy;
+
+ hdrleft = TARCHUNKSZ - tarhdrsz;
+ bytes2copy = (rr > hdrleft ? hdrleft : rr);
+
+ memcpy(&tarhdr[tarhdrsz], copybuf + pos, bytes2copy);
+
+ rr -= bytes2copy;
+ pos += bytes2copy;
+ tarhdrsz += bytes2copy;
+ }
+ else
+ {
+ /*
+ * We have the whole header structure in tarhdr[],
+ * look at the file metadata:
+ * - the subsequent file contents have to be skipped
+ * if the filename is recovery.conf
+ * - find out the size of the file padded to the next
+ * multiple of 512
+ */
+ int64 padding;
+
+ skip_file = (strcmp(&tarhdr[0], "recovery.conf") == 0);
+
+ scan_val(&tarhdr[124], &filesz, 8, 11);
+
+ padding = ((filesz + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - filesz;
+ filesz += padding;
+
+ /* Indicate that the subsequent data is the file content. */
+ in_tarhdr = false;
+
+ if (!skip_file)
+ WRITE_TAR_DATA(tarhdr, TARCHUNKSZ);
+ }
+ }
+ else
+ {
+ /*
+ * We're processing a file's contents.
+ */
+ if (filesz > 0)
+ {
+ /*
+ * We still have data to read (and possibly write).
+ */
+ int bytes2write;
+
+ bytes2write = (filesz > rr ? rr : filesz);
+
+ if (!skip_file)
+ WRITE_TAR_DATA(copybuf + pos, bytes2write);
+
+ rr -= bytes2write;
+ pos += bytes2write;
+ filesz -= bytes2write;
+ }
+ else
+ {
+ /*
+ * No more data in the current file,
+ * the next piece of data (if any) will
+ * be a new file header structure.
+ * Reinitialize all our variables.
+ */
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = 0;
+ filesz = 0;
+ }
+ }
}
}
totaldone += r;
@@ -712,10 +874,11 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
char filename[MAXPGPATH];
int current_len_left;
int current_padding = 0;
+ bool basetablespace = PQgetisnull(res, rownum, 0);
char *copybuf = NULL;
FILE *file = NULL;
- if (PQgetisnull(res, rownum, 0))
+ if (basetablespace)
strcpy(current_path, basedir);
else
strcpy(current_path, PQgetvalue(res, rownum, 1));
@@ -767,13 +930,13 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
/*
* No current file, so this must be the header for a new file
*/
- if (r != 512)
+ if (r != TARCHUNKSZ)
{
fprintf(stderr, _("%s: invalid tar block header size: %d\n"),
progname, r);
disconnect_and_exit(1);
}
- totaldone += 512;
+ totaldone += TARCHUNKSZ;
if (sscanf(copybuf + 124, "%11o", ¤t_len_left) != 1)
{
@@ -794,7 +957,7 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
* All files are padded up to 512 bytes
*/
current_padding =
- ((current_len_left + 511) & ~511) - current_len_left;
+ ((current_len_left + (TARCHUNKSZ-1)) & ~(TARCHUNKSZ-1)) - current_len_left;
/*
* First part of header is zero terminated filename
@@ -937,6 +1100,119 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
if (copybuf != NULL)
PQfreemem(copybuf);
+
+ if (basetablespace)
+ WriteRecoveryConf();
+}
+
+/*
+ * Create the new recovery.conf in memory
+ */
+static void
+CreateRecoveryConf(PGconn *conn)
+{
+ PQconninfoOption *connOptions;
+ PQconninfoOption *option;
+
+ if (!writerecoveryconf)
+ return;
+
+ connOptions = PQconninfo(conn);
+ if (connOptions == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "standby_mode = 'on'\n");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "primary_conninfo = '");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ for (option = connOptions; option && option->keyword; option++)
+ {
+ char *escaped;
+
+ /*
+ * Do not emit this setting if:
+ * - the setting is "replication", "dbname" or
+ * "fallback_application_name", since these would be
+ * overridden by the libpqwalreceiver module anyway.
+ * - not set or empty.
+ */
+ if (strcmp(option->keyword, "replication") != 0 ||
+ strcmp(option->keyword, "dbname") != 0 ||
+ strcmp(option->keyword, "fallback_application_name") != 0 ||
+ (option->val == NULL) ||
+ (option->val != NULL && option->val[0] == '\0'))
+ continue;
+
+ /*
+ * Write "keyword='value'" pieces, the value string is escaped
+ * if necessary and doubled single quotes around the value string.
+ */
+ escaped = escape_quotes(option->val);
+
+ appendPQExpBuffer(rcExpBuf, "%s=''%s'' ", option->keyword, escaped);
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ free(escaped);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "'\n");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ PQconninfoFree(connOptions);
+}
+
+
+static void
+WriteRecoveryConf(void)
+{
+ char filename[MAXPGPATH];
+ FILE *cf;
+
+ if (!writerecoveryconf)
+ return;
+
+ sprintf(filename, "%s/recovery.conf", basedir);
+
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("%s: cannot create %s: %s"), progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ if (fwrite(rcExpBuf->data, rcExpBuf->len, 1, cf) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ fclose(cf);
+
+ if (verbose)
+ fprintf(stderr, _("%s: recovery.conf written.\n"), progname);
}
@@ -960,6 +1236,15 @@ BaseBackup(void)
/* Error message already written in GetConnection() */
exit(1);
+ rcExpBuf = createPQExpBuffer();
+ if (!rcExpBuf)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ CreateRecoveryConf(conn);
+
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
@@ -1223,6 +1508,9 @@ BaseBackup(void)
#endif
}
+ /* Free the recovery.conf contents */
+ destroyPQExpBuffer(rcExpBuf);
+
/*
* End of copy data. Final result is already checked inside the loop.
*/
@@ -1243,6 +1531,7 @@ main(int argc, char **argv)
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
+ {"write-recovery-conf", no_argument, NULL, 'R'},
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
{"gzip", no_argument, NULL, 'z'},
@@ -1282,7 +1571,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:xX:l:zZ:c:h:p:U:s:wWvP",
+ while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
long_options, &option_index)) != -1)
{
switch (c)
@@ -1303,6 +1592,9 @@ main(int argc, char **argv)
exit(1);
}
break;
+ case 'R':
+ writerecoveryconf = true;
+ break;
case 'x':
if (includewal)
{
On Fri, Nov 30, 2012 at 10:13 AM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
Hi,
now that PQconninfo() was committed, I rebased the subsequent
patches. Actual code change was only in the last patch, to
conform to the committed PQconninfo() API.
So, finally coming back to this one.
What happened to Alvaro's suggestion of:
It seems simpler to have the escape_quotes utility function in port just
not use pg_malloc at all, have it return NULL or similar failure
indicator when malloc() fails, and then the caller decides what to do.
That way we can get around the whole need for changing memory allocation
across all the frontends, no? Like the attached.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
Attachments:
quotes.patchapplication/octet-stream; name=quotes.patchDownload
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 3e05ac3..ca085c2 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -354,6 +354,18 @@ pg_strdup(const char *s)
return result;
}
+static char *
+escape_quotes(const char *src)
+{
+ char *result = escape_single_quotes(src);
+ if (!result)
+ {
+ fprintf(stderr, _("%s: out of memory\n"), progname);
+ exit(1);
+ }
+ return result;
+}
+
/*
* make a copy of the array of lines, with token replaced by replacement
* the first time it occurs on each line.
@@ -2415,35 +2427,6 @@ check_ok(void)
}
}
-/*
- * Escape (by doubling) any single quotes or backslashes in given string
- *
- * Note: this is used to process both postgresql.conf entries and SQL
- * string literals. Since postgresql.conf strings are defined to treat
- * backslashes as escapes, we have to double backslashes here. Hence,
- * when using this for a SQL string literal, use E'' syntax.
- *
- * We do not need to worry about encoding considerations because all
- * valid backend encodings are ASCII-safe.
- */
-static char *
-escape_quotes(const char *src)
-{
- int len = strlen(src),
- i,
- j;
- char *result = pg_malloc(len * 2 + 1);
-
- for (i = 0, j = 0; i < len; i++)
- {
- if (SQL_STR_DOUBLE(src[i], true))
- result[j++] = src[i];
- result[j++] = src[i];
- }
- result[j] = '\0';
- return result;
-}
-
/* Hack to suppress a warning about %x from some versions of gcc */
static inline size_t
my_strftime(char *s, size_t max, const char *fmt, const struct tm * tm)
diff --git a/src/include/port.h b/src/include/port.h
index c429c77..b5a11c3 100644
--- a/src/include/port.h
+++ b/src/include/port.h
@@ -465,4 +465,7 @@ extern int pg_check_dir(const char *dir);
/* port/pgmkdirp.c */
extern int pg_mkdir_p(char *path, int omode);
+/* port/quotes.c */
+extern char *escape_single_quotes(const char *src);
+
#endif /* PG_PORT_H */
diff --git a/src/port/Makefile b/src/port/Makefile
index bace576..04253f8 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -32,7 +32,7 @@ LIBS += $(PTHREAD_LIBS)
OBJS = $(LIBOBJS) chklocale.o dirmod.o erand48.o exec.o fls.o inet_net_ntop.o \
noblock.o path.o pgcheckdir.o pg_crc.o pgmkdirp.o pgsleep.o \
- pgstrcasecmp.o qsort.o qsort_arg.o sprompt.o thread.o
+ pgstrcasecmp.o quotes.o qsort.o qsort_arg.o sprompt.o thread.o
# foo_srv.o and foo.o are both built from foo.c, but only foo.o has -DFRONTEND
OBJS_SRV = $(OBJS:%.o=%_srv.o)
diff --git a/src/port/quotes.c b/src/port/quotes.c
new file mode 100644
index 0000000..989fea6
--- /dev/null
+++ b/src/port/quotes.c
@@ -0,0 +1,37 @@
+#include "c.h"
+
+/*
+ * Escape (by doubling) any single quotes or backslashes in given string
+ *
+ * Note: this is used to process both postgresql.conf entries and SQL
+ * string literals. Since postgresql.conf strings are defined to treat
+ * backslashes as escapes, we have to double backslashes here. Hence,
+ * when using this for a SQL string literal, use E'' syntax.
+ *
+ * We do not need to worry about encoding considerations because all
+ * valid backend encodings are ASCII-safe.
+ *
+ * Returns a malloced() string that it's the responsibility of the caller
+ * to free.
+ */
+char *
+escape_single_quotes(const char *src)
+{
+ int len = strlen(src),
+ i,
+ j;
+ char *result = malloc(len * 2 + 1);
+
+ if (!result)
+ return NULL;
+
+ for (i = 0, j = 0; i < len; i++)
+ {
+ if (SQL_STR_DOUBLE(src[i], true))
+ result[j++] = src[i];
+ result[j++] = src[i];
+ }
+ result[j] = '\0';
+ return result;
+}
+
On Fri, Nov 30, 2012 at 10:13 AM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
Hi,
now that PQconninfo() was committed, I rebased the subsequent
patches. Actual code change was only in the last patch, to
conform to the committed PQconninfo() API.
I've applied a modified version of the "tar unification" patch to master.
It didn't apply cleanly so I ended up redoing a number of the things
manually, but the end result is fairly similar. I don't think it'll cause
anything but really trivial merge conflicts against the 04 patch, but I
didn't actually check that (e.g. it's tarCreateHeader() not
_tarCreateHeader() now).
I took at look at the basebackup patch as well. Easier to get now that it's
commented :) There's quite a lot of code in there whereby it tries to
remove recovery.conf from the tar stream if it's already in there. That
seems like an ugly way to do it. I see two other options, that I think both
are better. If we do need it to be removed, we should probably pass that as
a parameter up to the walsender, and make sure the file isn't included in
the first place. But we can also leave it in there - the tar format is
perfectly happy to have multiple copies of the same file in the archive -
it'll just use whichever copy comes last.
Given that the code there is already fairly difficult to track, I think
just keeping it simpler is definitely worth doing one of those two. I would
vote for just leaving two copies of recovery.conf in there.
Comments/thoughts from others?
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
2013-01-01 18:25 keltezéssel, Magnus Hagander írta:
On Fri, Nov 30, 2012 at 10:13 AM, Boszormenyi Zoltan <zb@cybertec.at
<mailto:zb@cybertec.at>> wrote:Hi,
now that PQconninfo() was committed, I rebased the subsequent
patches. Actual code change was only in the last patch, to
conform to the committed PQconninfo() API.I've applied a modified version of the "tar unification" patch to master. It didn't
apply cleanly so I ended up redoing a number of the things manually, but the end result
is fairly similar. I don't think it'll cause anything but really trivial merge conflicts
against the 04 patch, but I didn't actually check that (e.g. it's tarCreateHeader() not
_tarCreateHeader() now).
Thanks.
I took at look at the basebackup patch as well. Easier to get now that it's commented :)
There's quite a lot of code in there whereby it tries to remove recovery.conf from the
tar stream if it's already in there. That seems like an ugly way to do it. I see two
other options, that I think both are better. If we do need it to be removed, we should
probably pass that as a parameter up to the walsender, and make sure the file isn't
included in the first place. But we can also leave it in there - the tar format is
perfectly happy to have multiple copies of the same file in the archive - it'll just use
whichever copy comes last.
IIRC, Fujii Masao's wish was to remove the recovery.conf from the tar stream.
I know that the tar format is perfectly happy with two files with the same
path name but his reasoning was that GUIs (like WinRar) may get confused
by two such files and cannot decide which one to extract. I didn't actually
test this but it is a reasonable suspicion.
Passing a parameter to the walsender may be a better solution.
It's a bad idea only because it wasn't mine. ;-)
The only problem with this idea is that currently there's nothing that stops
pg_basebackup to be a generic and backwards-compatible tool.
It simply receives a TAR stream via plain PQgetCopyData() and optionally
extracts it. The client-side solution to skip the recovery.conf file keeps this
backward compatible feature. I tested this and pg_basebackup from 9.3dev
happily backs up a 9.2.2 database...
Given that the code there is already fairly difficult to track, I think just keeping it
simpler is definitely worth doing one of those two. I would vote for just leaving two
copies of recovery.conf in there.Comments/thoughts from others?
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
Best regards,
Zoltán Böszörmény
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
On Tue, Jan 1, 2013 at 7:13 PM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
2013-01-01 18:25 keltezéssel, Magnus Hagander írta:
On Fri, Nov 30, 2012 at 10:13 AM, Boszormenyi Zoltan <zb@cybertec.at>wrote:
Hi,
now that PQconninfo() was committed, I rebased the subsequent
patches. Actual code change was only in the last patch, to
conform to the committed PQconninfo() API.I've applied a modified version of the "tar unification" patch to
master. It didn't apply cleanly so I ended up redoing a number of the
things manually, but the end result is fairly similar. I don't think it'll
cause anything but really trivial merge conflicts against the 04 patch, but
I didn't actually check that (e.g. it's tarCreateHeader() not
_tarCreateHeader() now).Thanks.
I took at look at the basebackup patch as well. Easier to get now that
it's commented :) There's quite a lot of code in there whereby it tries to
remove recovery.conf from the tar stream if it's already in there. That
seems like an ugly way to do it. I see two other options, that I think both
are better. If we do need it to be removed, we should probably pass that as
a parameter up to the walsender, and make sure the file isn't included in
the first place. But we can also leave it in there - the tar format is
perfectly happy to have multiple copies of the same file in the archive -
it'll just use whichever copy comes last.IIRC, Fujii Masao's wish was to remove the recovery.conf from the tar
stream.
I know that the tar format is perfectly happy with two files with the same
path name but his reasoning was that GUIs (like WinRar) may get confused
by two such files and cannot decide which one to extract. I didn't actually
test this but it is a reasonable suspicion.
Hmm. Somehow, I missed that part of the discussion. Sorry, didn't realize
it had been mentioned already.
Passing a parameter to the walsender may be a better solution.
It's a bad idea only because it wasn't mine. ;-)
The only problem with this idea is that currently there's nothing that
stops
pg_basebackup to be a generic and backwards-compatible tool.
It simply receives a TAR stream via plain PQgetCopyData() and optionally
extracts it. The client-side solution to skip the recovery.conf file keeps
this
backward compatible feature. I tested this and pg_basebackup from 9.3dev
happily backs up a 9.2.2 database...
I thought commit add6c3179a4d4fa3e62dd3e86a00f23303336bac at leasdt broke
that? In particular, did you test where any of those values were different
between client and server? Or maybe just didn't test in -x mode?
I actually thought there was something else that broke the compatibility,
but I can't seem to find it so perhaps that was something that was never
committed.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
2013-01-01 17:18 keltezéssel, Magnus Hagander írta:
On Fri, Nov 30, 2012 at 10:13 AM, Boszormenyi Zoltan <zb@cybertec.at
<mailto:zb@cybertec.at>> wrote:Hi,
now that PQconninfo() was committed, I rebased the subsequent
patches. Actual code change was only in the last patch, to
conform to the committed PQconninfo() API.So, finally coming back to this one.
What happened to Alvaro's suggestion of:
It seems simpler to have the escape_quotes utility function in port just
not use pg_malloc at all, have it return NULL or similar failure
indicator when malloc() fails, and then the caller decides what to do.
I seem to have skipped it while reading my mails,
I don't remember this suggestion at all. Sorry.
That way we can get around the whole need for changing memory allocation across all the
frontends, no? Like the attached.
Sure it's simpler but then the consistent look of the code is lost.
What about the other patch to unify pg_malloc and friends?
Basically all client code boils down to
fprintf(stderr, ...)
in different disguise in their error reporting, so that patch can
also be simplified but it seems that the atexit() - either explicitly
or hidden behind InitPostgresFrontend() - cannot be avoided.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Boszormenyi Zoltan <zb@cybertec.at> writes:
2013-01-01 17:18 keltez�ssel, Magnus Hagander �rta:
That way we can get around the whole need for changing memory allocation across all the
frontends, no? Like the attached.
Sure it's simpler but then the consistent look of the code is lost.
What about the other patch to unify pg_malloc and friends?
Basically all client code boils down to
fprintf(stderr, ...)
in different disguise in their error reporting, so that patch can
also be simplified but it seems that the atexit() - either explicitly
or hidden behind InitPostgresFrontend() - cannot be avoided.
Meh. I find it seriously wrongheaded that something as minor as an
escape_quotes() function should get to dictate both malloc wrappers
and error recovery handling throughout every program that might use it.
I like Magnus' version a lot better than that idea.
A bigger issue that I notice with this code is that it's only correct in
backend-safe encodings, as the comment mentions. If we're going to be
putting it into frontend programs, how safe is that going to be?
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
2013-01-02 01:24 keltezéssel, Tom Lane írta:
Boszormenyi Zoltan <zb@cybertec.at> writes:
2013-01-01 17:18 keltezéssel, Magnus Hagander írta:
That way we can get around the whole need for changing memory allocation across all the
frontends, no? Like the attached.Sure it's simpler but then the consistent look of the code is lost.
What about the other patch to unify pg_malloc and friends?
Basically all client code boils down to
fprintf(stderr, ...)
in different disguise in their error reporting, so that patch can
also be simplified but it seems that the atexit() - either explicitly
or hidden behind InitPostgresFrontend() - cannot be avoided.Meh. I find it seriously wrongheaded that something as minor as an
escape_quotes() function should get to dictate both malloc wrappers
and error recovery handling throughout every program that might use it.
Actually, the unification of pg_malloc and friends wasn't dictated
by this little code, it was just that pg_basebackup doesn't provide
a pg_malloc implementation (only pg_malloc0) that is used by
initdb's escape_quotes() function. Then I noticed how wide these
almost identical functions have spread into client apps already.
I would say this unification patch is completely orthogonal to
the patch in $SUBJECT. I will post it in a different thread if it's
wanted at all. The extra atexit() handler is not needed if a simple
fprintf(stderr, ...) error reporting is enough in all clients.
As far as I saw, all clients do exactly this but some of them hide
this behind #define's.
I like Magnus' version a lot better than that idea.
OK, I will post the core patch building on his code.
A bigger issue that I notice with this code is that it's only correct in
backend-safe encodings, as the comment mentions. If we're going to be
putting it into frontend programs, how safe is that going to be?regards, tom lane
The question in a different form is: does PostgreSQL support
non-ASCII-safe encodings at all (or on the client side)? Forgive
my ignorance and enlighten me: how many such encodings
exist besides EBCDIC? Is UTF-16 non-ASCII-safe?
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Jan 2, 2013 at 9:59 AM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
2013-01-02 01:24 keltezéssel, Tom Lane írta:
Boszormenyi Zoltan <zb@cybertec.at> writes:
2013-01-01 17:18 keltezéssel, Magnus Hagander írta:
That way we can get around the whole need for changing memory
allocation across all the
frontends, no? Like the attached.Sure it's simpler but then the consistent look of the code is lost.
What about the other patch to unify pg_malloc and friends?
Basically all client code boils down to
fprintf(stderr, ...)
in different disguise in their error reporting, so that patch can
also be simplified but it seems that the atexit() - either explicitly
or hidden behind InitPostgresFrontend() - cannot be avoided.Meh. I find it seriously wrongheaded that something as minor as an
escape_quotes() function should get to dictate both malloc wrappers
and error recovery handling throughout every program that might use it.Actually, the unification of pg_malloc and friends wasn't dictated
by this little code, it was just that pg_basebackup doesn't provide
a pg_malloc implementation (only pg_malloc0) that is used by
initdb's escape_quotes() function. Then I noticed how wide these
almost identical functions have spread into client apps already.I would say this unification patch is completely orthogonal to
the patch in $SUBJECT. I will post it in a different thread if it's
wanted at all. The extra atexit() handler is not needed if a simple
fprintf(stderr, ...) error reporting is enough in all clients.
As far as I saw, all clients do exactly this but some of them hide
this behind #define's.
Please do keep that one separate - let's avoid unnecessary feature-creep,
whether it's good or bad features.
I like Magnus' version a lot better than that idea.
OK, I will post the core patch building on his code.
Thanks.
A bigger issue that I notice with this code is that it's only correct in
backend-safe encodings, as the comment mentions. If we're going to be
putting it into frontend programs, how safe is that going to be?regards, tom lane
The question in a different form is: does PostgreSQL support
non-ASCII-safe encodings at all (or on the client side)? Forgive
my ignorance and enlighten me: how many such encodings
exist besides EBCDIC? Is UTF-16 non-ASCII-safe?
We do. See http://www.postgresql.org/docs/9.2/static/multibyte.html.
There are quite a few far-eastern encodings that aren't available as server
encondings, and I believe it's all for this reason.
That said, do we need to care *in this specific case*? We use it in initdb
to parse config strings, I believe. And we'd use it to parse a conninfo
string in pg_basebackup, correct? Perhaps all we need to do is to clearly
comment that it doesn't work with non-ascii safe encodings, or rename it to
indicate that it's limited in this?
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
2013-01-02 10:12 keltezéssel, Magnus Hagander írta:
On Wed, Jan 2, 2013 at 9:59 AM, Boszormenyi Zoltan <zb@cybertec.at
<mailto:zb@cybertec.at>> wrote:2013-01-02 01:24 keltezéssel, Tom Lane írta:
Boszormenyi Zoltan <zb@cybertec.at <mailto:zb@cybertec.at>> writes:
2013-01-01 17:18 keltezéssel, Magnus Hagander írta:
That way we can get around the whole need for changing memory allocation
across all the
frontends, no? Like the attached.Sure it's simpler but then the consistent look of the code is lost.
What about the other patch to unify pg_malloc and friends?
Basically all client code boils down to
fprintf(stderr, ...)
in different disguise in their error reporting, so that patch can
also be simplified but it seems that the atexit() - either explicitly
or hidden behind InitPostgresFrontend() - cannot be avoided.Meh. I find it seriously wrongheaded that something as minor as an
escape_quotes() function should get to dictate both malloc wrappers
and error recovery handling throughout every program that might use it.Actually, the unification of pg_malloc and friends wasn't dictated
by this little code, it was just that pg_basebackup doesn't provide
a pg_malloc implementation (only pg_malloc0) that is used by
initdb's escape_quotes() function. Then I noticed how wide these
almost identical functions have spread into client apps already.I would say this unification patch is completely orthogonal to
the patch in $SUBJECT. I will post it in a different thread if it's
wanted at all. The extra atexit() handler is not needed if a simple
fprintf(stderr, ...) error reporting is enough in all clients.
As far as I saw, all clients do exactly this but some of them hide
this behind #define's.Please do keep that one separate - let's avoid unnecessary feature-creep, whether it's
good or bad features.I like Magnus' version a lot better than that idea.
OK, I will post the core patch building on his code.
Thanks.
A bigger issue that I notice with this code is that it's only correct in
backend-safe encodings, as the comment mentions. If we're going to be
putting it into frontend programs, how safe is that going to be?regards, tom lane
The question in a different form is: does PostgreSQL support
non-ASCII-safe encodings at all (or on the client side)? Forgive
my ignorance and enlighten me: how many such encodings
exist besides EBCDIC? Is UTF-16 non-ASCII-safe?We do. See http://www.postgresql.org/docs/9.2/static/multibyte.html.
There are quite a few far-eastern encodings that aren't available as server encondings,
and I believe it's all for this reason.
I see, thanks.
That said, do we need to care *in this specific case*? We use it in initdb to parse
config strings, I believe. And we'd use it to parse a conninfo string in pg_basebackup,
correct?
Correct.
Perhaps all we need to do is to clearly comment that it doesn't work with non-ascii safe
encodings, or rename it to indicate that it's limited in this?
If you send a new patch with the function renamed accordingly, I will modify
my code to use it.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
2013-01-02 10:37 keltezéssel, Boszormenyi Zoltan írta:
2013-01-02 10:12 keltezéssel, Magnus Hagander írta:
On Wed, Jan 2, 2013 at 9:59 AM, Boszormenyi Zoltan <zb@cybertec.at
<mailto:zb@cybertec.at>> wrote:2013-01-02 01:24 keltezéssel, Tom Lane írta:
Boszormenyi Zoltan <zb@cybertec.at <mailto:zb@cybertec.at>> writes:
2013-01-01 17:18 keltezéssel, Magnus Hagander írta:
That way we can get around the whole need for changing memory
allocation across all the
frontends, no? Like the attached.Sure it's simpler but then the consistent look of the code is lost.
What about the other patch to unify pg_malloc and friends?
Basically all client code boils down to
fprintf(stderr, ...)
in different disguise in their error reporting, so that patch can
also be simplified but it seems that the atexit() - either explicitly
or hidden behind InitPostgresFrontend() - cannot be avoided.Meh. I find it seriously wrongheaded that something as minor as an
escape_quotes() function should get to dictate both malloc wrappers
and error recovery handling throughout every program that might use it.Actually, the unification of pg_malloc and friends wasn't dictated
by this little code, it was just that pg_basebackup doesn't provide
a pg_malloc implementation (only pg_malloc0) that is used by
initdb's escape_quotes() function. Then I noticed how wide these
almost identical functions have spread into client apps already.I would say this unification patch is completely orthogonal to
the patch in $SUBJECT. I will post it in a different thread if it's
wanted at all. The extra atexit() handler is not needed if a simple
fprintf(stderr, ...) error reporting is enough in all clients.
As far as I saw, all clients do exactly this but some of them hide
this behind #define's.Please do keep that one separate - let's avoid unnecessary feature-creep, whether it's
good or bad features.I like Magnus' version a lot better than that idea.
OK, I will post the core patch building on his code.
Thanks.
A bigger issue that I notice with this code is that it's only correct in
backend-safe encodings, as the comment mentions. If we're going to be
putting it into frontend programs, how safe is that going to be?regards, tom lane
The question in a different form is: does PostgreSQL support
non-ASCII-safe encodings at all (or on the client side)? Forgive
my ignorance and enlighten me: how many such encodings
exist besides EBCDIC? Is UTF-16 non-ASCII-safe?We do. See http://www.postgresql.org/docs/9.2/static/multibyte.html.
There are quite a few far-eastern encodings that aren't available as server encondings,
and I believe it's all for this reason.I see, thanks.
That said, do we need to care *in this specific case*? We use it in initdb to parse
config strings, I believe. And we'd use it to parse a conninfo string in pg_basebackup,
correct?Correct.
Perhaps all we need to do is to clearly comment that it doesn't work with non-ascii
safe encodings, or rename it to indicate that it's limited in this?If you send a new patch with the function renamed accordingly, I will modify
my code to use it.
Attached is the quotes-v2 patch, the function is renamed and
the comment is modified plus the pg_basebackup v21 patch
that uses this function.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
quotes-v2.patchtext/x-patch; name=quotes-v2.patchDownload
diff -durpN postgresql/src/bin/initdb/initdb.c postgresql.1/src/bin/initdb/initdb.c
--- postgresql/src/bin/initdb/initdb.c 2013-01-02 09:19:03.855521804 +0100
+++ postgresql.1/src/bin/initdb/initdb.c 2013-01-02 11:26:10.008494149 +0100
@@ -354,6 +354,18 @@ pg_strdup(const char *s)
return result;
}
+static char *
+escape_quotes(const char *src)
+{
+ char *result = escape_single_quotes_ascii(src);
+ if (!result)
+ {
+ fprintf(stderr, _("%s: out of memory\n"), progname);
+ exit(1);
+ }
+ return result;
+}
+
/*
* make a copy of the array of lines, with token replaced by replacement
* the first time it occurs on each line.
@@ -2415,35 +2427,6 @@ check_ok(void)
}
}
-/*
- * Escape (by doubling) any single quotes or backslashes in given string
- *
- * Note: this is used to process both postgresql.conf entries and SQL
- * string literals. Since postgresql.conf strings are defined to treat
- * backslashes as escapes, we have to double backslashes here. Hence,
- * when using this for a SQL string literal, use E'' syntax.
- *
- * We do not need to worry about encoding considerations because all
- * valid backend encodings are ASCII-safe.
- */
-static char *
-escape_quotes(const char *src)
-{
- int len = strlen(src),
- i,
- j;
- char *result = pg_malloc(len * 2 + 1);
-
- for (i = 0, j = 0; i < len; i++)
- {
- if (SQL_STR_DOUBLE(src[i], true))
- result[j++] = src[i];
- result[j++] = src[i];
- }
- result[j] = '\0';
- return result;
-}
-
/* Hack to suppress a warning about %x from some versions of gcc */
static inline size_t
my_strftime(char *s, size_t max, const char *fmt, const struct tm * tm)
diff -durpN postgresql/src/include/port.h postgresql.1/src/include/port.h
--- postgresql/src/include/port.h 2013-01-02 09:19:03.902522322 +0100
+++ postgresql.1/src/include/port.h 2013-01-02 11:26:32.695654454 +0100
@@ -469,4 +469,7 @@ extern int pg_mkdir_p(char *path, int om
void tarCreateHeader(char *h, const char *filename, const char *linktarget, size_t size, mode_t mode, uid_t uid, gid_t gid, time_t mtime);
int tarChecksum(char *header);
+/* port/quotes.c */
+extern char *escape_single_quotes_ascii(const char *src);
+
#endif /* PG_PORT_H */
diff -durpN postgresql/src/port/Makefile postgresql.1/src/port/Makefile
--- postgresql/src/port/Makefile 2013-01-02 09:19:03.935522680 +0100
+++ postgresql.1/src/port/Makefile 2013-01-02 10:43:57.358410221 +0100
@@ -32,7 +32,7 @@ LIBS += $(PTHREAD_LIBS)
OBJS = $(LIBOBJS) chklocale.o dirmod.o erand48.o exec.o fls.o inet_net_ntop.o \
noblock.o path.o pgcheckdir.o pg_crc.o pgmkdirp.o pgsleep.o \
- pgstrcasecmp.o qsort.o qsort_arg.o sprompt.o tar.o thread.o
+ pgstrcasecmp.o quotes.o qsort.o qsort_arg.o sprompt.o tar.o thread.o
# foo_srv.o and foo.o are both built from foo.c, but only foo.o has -DFRONTEND
OBJS_SRV = $(OBJS:%.o=%_srv.o)
diff -durpN postgresql/src/port/quotes.c postgresql.1/src/port/quotes.c
--- postgresql/src/port/quotes.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.1/src/port/quotes.c 2013-01-02 11:25:46.616328180 +0100
@@ -0,0 +1,37 @@
+#include "c.h"
+
+/*
+ * Escape (by doubling) any single quotes or backslashes in given string
+ *
+ * Note: this is used to process both postgresql.conf entries and SQL
+ * string literals. Since postgresql.conf strings are defined to treat
+ * backslashes as escapes, we have to double backslashes here. Hence,
+ * when using this for a SQL string literal, use E'' syntax.
+ *
+ * We do not care about encoding considerations because this function
+ * is only used for parsing or creating configuration files.
+ *
+ * Returns a malloced() string that it's the responsibility of the caller
+ * to free.
+ */
+char *
+escape_single_quotes_ascii(const char *src)
+{
+ int len = strlen(src),
+ i,
+ j;
+ char *result = malloc(len * 2 + 1);
+
+ if (!result)
+ return NULL;
+
+ for (i = 0, j = 0; i < len; i++)
+ {
+ if (SQL_STR_DOUBLE(src[i], true))
+ result[j++] = src[i];
+ result[j++] = src[i];
+ }
+ result[j] = '\0';
+ return result;
+}
+
pg_basebackup-v21.patchtext/x-patch; name=pg_basebackup-v21.patchDownload
diff -durpN postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml
--- postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml 2012-11-08 13:13:04.151630632 +0100
+++ postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml 2013-01-02 10:46:41.642608888 +0100
@@ -189,6 +189,21 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--write-recovery-conf</option></term>
+ <listitem>
+
+ <para>
+ Write a minimal recovery.conf into the output directory (or into
+ the base archive file if <option>--format=tar</option> was specified)
+ using the connection parameters from the command line to ease
+ setting up the standby.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-x</option></term>
<term><option>--xlog</option></term>
<listitem>
diff -durpN postgresql.1/src/bin/pg_basebackup/pg_basebackup.c postgresql.2/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.1/src/bin/pg_basebackup/pg_basebackup.c 2013-01-02 09:19:03.856521815 +0100
+++ postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2013-01-02 11:29:05.353701197 +0100
@@ -13,12 +13,14 @@
#include "postgres_fe.h"
#include "libpq-fe.h"
+#include "pqexpbuffer.h"
#include <unistd.h>
#include <dirent.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
+#include <time.h>
#ifdef HAVE_LIBZ
#include <zlib.h>
@@ -40,6 +42,7 @@ int compresslevel = 0;
bool includewal = false;
bool streamwal = false;
bool fastcheckpoint = false;
+bool writerecoveryconf = false;
int standby_message_timeout = 10 * 1000; /* 10 sec = default */
/* Progress counters */
@@ -64,13 +67,18 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
+PQExpBuffer rcExpBuf = NULL;
+
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
static void progress_report(int tablespacenum, const char *filename);
+static void scan_val(char *s, uint64 *val, unsigned int base, size_t len);
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
+static void CreateRecoveryConf(PGconn *conn);
+static void WriteRecoveryConf(void);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -101,6 +109,8 @@ usage(void)
printf(_("\nOptions controlling the output:\n"));
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
+ printf(_(" -R, --write-recovery-conf\n"
+ " write recovery.conf after backup\n"));
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
printf(_(" -X, --xlog-method=fetch|stream\n"
" include required WAL files with specified method\n"));
@@ -446,6 +456,65 @@ progress_report(int tablespacenum, const
/*
+ * Inverse of print_val()
+ */
+static void
+scan_val(char *s, uint64 *val, unsigned int base, size_t len)
+{
+ uint64 tmp = 0;
+ int i;
+
+ for (i = 0; i < len; i++)
+ {
+ int digit = s[i] - '0';
+
+ tmp = tmp * base + digit;
+ }
+
+ *val = tmp;
+}
+
+
+/*
+ * Write a piece of tar data
+ */
+static void
+writeTarData(
+#ifdef HAVE_LIBZ
+ gzFile ztarfile,
+#endif
+ FILE *tarfile, char *buf, int r, char *current_file)
+{
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (gzwrite(ztarfile, buf, r) != r)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, current_file, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (fwrite(buf, r, 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
+ progname, current_file, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ }
+}
+
+#ifdef HAVE_LIBZ
+#define WRITE_TAR_DATA(buf, sz) writeTarData(ztarfile, tarfile, buf, sz, filename)
+#else
+#define WRITE_TAR_DATA(buf, sz) writeTarData(tarfile, buf, sz, filename)
+#endif
+
+/*
* Receive a tar format file from the connection to the server, and write
* the data from this file directly into a tar file. If compression is
* enabled, the data will be compressed while written to the file.
@@ -461,12 +530,17 @@ ReceiveTarFile(PGconn *conn, PGresult *r
char filename[MAXPGPATH];
char *copybuf = NULL;
FILE *tarfile = NULL;
+ char tarhdr[512];
+ bool basetablespace = PQgetisnull(res, rownum, 0);
+ bool in_tarhdr, skip_file;
+ size_t tarhdrsz;
+ uint64 filesz;
#ifdef HAVE_LIBZ
gzFile ztarfile = NULL;
#endif
- if (PQgetisnull(res, rownum, 0))
+ if (basetablespace)
{
/*
* Base tablespaces
@@ -578,6 +652,16 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
+ /*
+ * Initialize our variables for tracking
+ * individual files inside the TAR stream.
+ * For more detailed explanation, see below.
+ */
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = 0;
+ filesz = 0;
+
while (1)
{
int r;
@@ -592,7 +676,8 @@ ReceiveTarFile(PGconn *conn, PGresult *r
if (r == -1)
{
/*
- * End of chunk. Close file (but not stdout).
+ * End of chunk. Write recovery.conf into the tar file (if it
+ * was requested) and close file (but not stdout).
*
* Also, write two completely empty blocks at the end of the tar
* file, as required by some tar programs.
@@ -600,30 +685,30 @@ ReceiveTarFile(PGconn *conn, PGresult *r
char zerobuf[1024];
MemSet(zerobuf, 0, sizeof(zerobuf));
-#ifdef HAVE_LIBZ
- if (ztarfile != NULL)
- {
- if (gzwrite(ztarfile, zerobuf, sizeof(zerobuf)) !=
- sizeof(zerobuf))
- {
- fprintf(stderr,
- _("%s: could not write to compressed file \"%s\": %s\n"),
- progname, filename, get_gz_error(ztarfile));
- disconnect_and_exit(1);
- }
- }
- else
-#endif
+
+ if (basetablespace && writerecoveryconf)
{
- if (fwrite(zerobuf, sizeof(zerobuf), 1, tarfile) != 1)
- {
- fprintf(stderr,
- _("%s: could not write to file \"%s\": %s\n"),
- progname, filename, strerror(errno));
- disconnect_and_exit(1);
- }
+ char header[512];
+ int padding;
+
+ tarCreateHeader(header, "recovery.conf", NULL,
+ rcExpBuf->len,
+ 0600, 04000, 02000,
+ time(NULL));
+
+ padding = ((rcExpBuf->len + 511) & ~511) - rcExpBuf->len;
+
+ WRITE_TAR_DATA(header, sizeof(header));
+ WRITE_TAR_DATA(rcExpBuf->data, rcExpBuf->len);
+ if (padding)
+ WRITE_TAR_DATA(zerobuf, padding);
+
+ if (verbose)
+ fprintf(stderr, _("%s: recovery.conf written into '%s'\n"), progname, filename);
}
+ WRITE_TAR_DATA(zerobuf, sizeof(zerobuf));
+
#ifdef HAVE_LIBZ
if (ztarfile != NULL)
{
@@ -659,25 +744,124 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
-#ifdef HAVE_LIBZ
- if (ztarfile != NULL)
+ if (!writerecoveryconf || !basetablespace)
{
- if (gzwrite(ztarfile, copybuf, r) != r)
- {
- fprintf(stderr,
- _("%s: could not write to compressed file \"%s\": %s\n"),
- progname, filename, get_gz_error(ztarfile));
- disconnect_and_exit(1);
- }
+ /*
+ * If --write-recovery-conf was not requested or this
+ * is not the base tablespace, simply pass the received
+ * data into the TAR file, either compressed or not.
+ */
+
+ WRITE_TAR_DATA(copybuf, r);
}
else
-#endif
{
- if (fwrite(copybuf, r, 1, tarfile) != 1)
+ /*
+ * If --write-recovery-conf was requested AND this
+ * is the base tablespace, the TAR stream may contain
+ * a recovery.conf file if the backup is coming from
+ * a standby server. We have to skip this file in
+ * the stream and add a new one constructed by
+ * CreateRecoveryConf() at the end of the stream.
+ *
+ * To do this, we have to process the individual files
+ * inside the TAR stream. The stream consists of a header
+ * and zero or more chunks, all 512 bytes long. The stream
+ * from the server is broken up into smaller pieces, so
+ * we have to track the size of the files to find the next
+ * header structure.
+ */
+ int rr = r; /* Save the value returned by PQgetCopyData */
+ int pos = 0;
+
+ while (rr > 0)
{
- fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
- progname, filename, strerror(errno));
- disconnect_and_exit(1);
+ if (in_tarhdr)
+ {
+ /*
+ * We're currently reading a header structure
+ * inside the TAR stream, i.e. the file metadata.
+ */
+ if (tarhdrsz < 512)
+ {
+ /*
+ * Copy the header structure into tarhdr[]
+ * in case the header is not aligned to 512 bytes
+ * or it's not returned in whole by the last
+ * PQgetCopyData call.
+ */
+ int hdrleft, bytes2copy;
+
+ hdrleft = 512 - tarhdrsz;
+ bytes2copy = (rr > hdrleft ? hdrleft : rr);
+
+ memcpy(&tarhdr[tarhdrsz], copybuf + pos, bytes2copy);
+
+ rr -= bytes2copy;
+ pos += bytes2copy;
+ tarhdrsz += bytes2copy;
+ }
+ else
+ {
+ /*
+ * We have the whole header structure in tarhdr[],
+ * look at the file metadata:
+ * - the subsequent file contents have to be skipped
+ * if the filename is recovery.conf
+ * - find out the size of the file padded to the next
+ * multiple of 512
+ */
+ int64 padding;
+
+ skip_file = (strcmp(&tarhdr[0], "recovery.conf") == 0);
+
+ scan_val(&tarhdr[124], &filesz, 8, 11);
+
+ padding = ((filesz + 511) & ~511) - filesz;
+ filesz += padding;
+
+ /* Indicate that the subsequent data is the file content. */
+ in_tarhdr = false;
+
+ if (!skip_file)
+ WRITE_TAR_DATA(tarhdr, 512);
+ }
+ }
+ else
+ {
+ /*
+ * We're processing a file's contents.
+ */
+ if (filesz > 0)
+ {
+ /*
+ * We still have data to read (and possibly write).
+ */
+ int bytes2write;
+
+ bytes2write = (filesz > rr ? rr : filesz);
+
+ if (!skip_file)
+ WRITE_TAR_DATA(copybuf + pos, bytes2write);
+
+ rr -= bytes2write;
+ pos += bytes2write;
+ filesz -= bytes2write;
+ }
+ else
+ {
+ /*
+ * No more data in the current file,
+ * the next piece of data (if any) will
+ * be a new file header structure.
+ * Reinitialize all our variables.
+ */
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = 0;
+ filesz = 0;
+ }
+ }
}
}
totaldone += r;
@@ -706,10 +890,11 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
char filename[MAXPGPATH];
int current_len_left;
int current_padding = 0;
+ bool basetablespace = PQgetisnull(res, rownum, 0);
char *copybuf = NULL;
FILE *file = NULL;
- if (PQgetisnull(res, rownum, 0))
+ if (basetablespace)
strcpy(current_path, basedir);
else
strcpy(current_path, PQgetvalue(res, rownum, 1));
@@ -931,6 +1116,131 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
if (copybuf != NULL)
PQfreemem(copybuf);
+
+ if (basetablespace)
+ WriteRecoveryConf();
+}
+
+static char *
+escape_quotes(const char *src)
+{
+ char *result = escape_single_quotes_ascii(src);
+ if (!result)
+ {
+ fprintf(stderr, _("%s: out of memory\n"), progname);
+ exit(1);
+ }
+ return result;
+}
+
+/*
+ * Create the new recovery.conf in memory
+ */
+static void
+CreateRecoveryConf(PGconn *conn)
+{
+ PQconninfoOption *connOptions;
+ PQconninfoOption *option;
+
+ if (!writerecoveryconf)
+ return;
+
+ connOptions = PQconninfo(conn);
+ if (connOptions == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "standby_mode = 'on'\n");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "primary_conninfo = '");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ for (option = connOptions; option && option->keyword; option++)
+ {
+ char *escaped;
+
+ /*
+ * Do not emit this setting if:
+ * - the setting is "replication", "dbname" or
+ * "fallback_application_name", since these would be
+ * overridden by the libpqwalreceiver module anyway.
+ * - not set or empty.
+ */
+ if (strcmp(option->keyword, "replication") == 0 ||
+ strcmp(option->keyword, "dbname") == 0 ||
+ strcmp(option->keyword, "fallback_application_name") == 0 ||
+ (option->val == NULL) ||
+ (option->val != NULL && option->val[0] == '\0'))
+ continue;
+
+ /*
+ * Write "keyword='value'" pieces, the value string is escaped
+ * if necessary and doubled single quotes around the value string.
+ */
+ escaped = escape_quotes(option->val);
+
+ appendPQExpBuffer(rcExpBuf, "%s=''%s'' ", option->keyword, escaped);
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ free(escaped);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "'\n");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ PQconninfoFree(connOptions);
+}
+
+
+static void
+WriteRecoveryConf(void)
+{
+ char filename[MAXPGPATH];
+ FILE *cf;
+
+ if (!writerecoveryconf)
+ return;
+
+ sprintf(filename, "%s/recovery.conf", basedir);
+
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("%s: cannot create %s: %s"), progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ if (fwrite(rcExpBuf->data, rcExpBuf->len, 1, cf) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ fclose(cf);
+
+ if (verbose)
+ fprintf(stderr, _("%s: recovery.conf written.\n"), progname);
}
@@ -954,6 +1264,15 @@ BaseBackup(void)
/* Error message already written in GetConnection() */
exit(1);
+ rcExpBuf = createPQExpBuffer();
+ if (!rcExpBuf)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ CreateRecoveryConf(conn);
+
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
@@ -1217,6 +1536,9 @@ BaseBackup(void)
#endif
}
+ /* Free the recovery.conf contents */
+ destroyPQExpBuffer(rcExpBuf);
+
/*
* End of copy data. Final result is already checked inside the loop.
*/
@@ -1237,6 +1559,7 @@ main(int argc, char **argv)
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
+ {"write-recovery-conf", no_argument, NULL, 'R'},
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
{"gzip", no_argument, NULL, 'z'},
@@ -1274,7 +1597,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:xX:l:zZ:c:h:p:U:s:wWvP",
+ while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
long_options, &option_index)) != -1)
{
switch (c)
@@ -1295,6 +1618,9 @@ main(int argc, char **argv)
exit(1);
}
break;
+ case 'R':
+ writerecoveryconf = true;
+ break;
case 'x':
if (includewal)
{
2013-01-02 10:12 keltezéssel, Magnus Hagander írta:
On Wed, Jan 2, 2013 at 9:59 AM, Boszormenyi Zoltan <zb@cybertec.at
<mailto:zb@cybertec.at>> wrote:2013-01-02 01:24 keltezéssel, Tom Lane írta:
Boszormenyi Zoltan <zb@cybertec.at <mailto:zb@cybertec.at>> writes:
2013-01-01 17:18 keltezéssel, Magnus Hagander írta:
That way we can get around the whole need for changing memory allocation
across all the
frontends, no? Like the attached.Sure it's simpler but then the consistent look of the code is lost.
What about the other patch to unify pg_malloc and friends?
Basically all client code boils down to
fprintf(stderr, ...)
in different disguise in their error reporting, so that patch can
also be simplified but it seems that the atexit() - either explicitly
or hidden behind InitPostgresFrontend() - cannot be avoided.Meh. I find it seriously wrongheaded that something as minor as an
escape_quotes() function should get to dictate both malloc wrappers
and error recovery handling throughout every program that might use it.Actually, the unification of pg_malloc and friends wasn't dictated
by this little code, it was just that pg_basebackup doesn't provide
a pg_malloc implementation (only pg_malloc0) that is used by
initdb's escape_quotes() function. Then I noticed how wide these
almost identical functions have spread into client apps already.I would say this unification patch is completely orthogonal to
the patch in $SUBJECT. I will post it in a different thread if it's
wanted at all. The extra atexit() handler is not needed if a simple
fprintf(stderr, ...) error reporting is enough in all clients.
As far as I saw, all clients do exactly this but some of them hide
this behind #define's.Please do keep that one separate - let's avoid unnecessary feature-creep, whether it's
good or bad features.
The patch is attached. There is no extra atexit() code in this one.
I did this over my pg_basebackup patch, there are two chunks
that gets rejected if applied without it: one in initdb.c, the other is
in src/port/Makefile. It's because the modified codes are too close
to each other.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
pg_malloc-unified-v9.patchtext/x-patch; name=pg_malloc-unified-v9.patchDownload
diff -durpN postgresql.2/contrib/oid2name/oid2name.c postgresql.3/contrib/oid2name/oid2name.c
--- postgresql.2/contrib/oid2name/oid2name.c 2012-10-03 10:40:48.241207023 +0200
+++ postgresql.3/contrib/oid2name/oid2name.c 2013-01-02 13:40:08.483260103 +0100
@@ -50,9 +50,6 @@ struct options
/* function prototypes */
static void help(const char *progname);
void get_opts(int, char **, struct options *);
-void *pg_malloc(size_t size);
-void *pg_realloc(void *ptr, size_t size);
-char *pg_strdup(const char *str);
void add_one_elt(char *eltname, eary *eary);
char *get_comma_elts(eary *eary);
PGconn *sql_conn(struct options *);
@@ -201,53 +198,6 @@ help(const char *progname)
progname, progname);
}
-void *
-pg_malloc(size_t size)
-{
- void *ptr;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- ptr = malloc(size);
- if (!ptr)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return ptr;
-}
-
-void *
-pg_realloc(void *ptr, size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- result = realloc(ptr, size);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-char *
-pg_strdup(const char *str)
-{
- char *result = strdup(str);
-
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
/*
* add_one_elt
*
diff -durpN postgresql.2/contrib/pgbench/pgbench.c postgresql.3/contrib/pgbench/pgbench.c
--- postgresql.2/contrib/pgbench/pgbench.c 2013-01-02 09:19:03.655519614 +0100
+++ postgresql.3/contrib/pgbench/pgbench.c 2013-01-02 13:40:26.539378189 +0100
@@ -296,58 +296,6 @@ static void setalarm(int seconds);
static void *threadRun(void *arg);
-/*
- * routines to check mem allocations and fail noisily.
- */
-static void *
-pg_malloc(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-static void *
-pg_realloc(void *ptr, size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- result = realloc(ptr, size);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-static char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- fprintf(stderr, "out of memory\n");
- exit(1);
- }
- return result;
-}
-
-
static void
usage(void)
{
diff -durpN postgresql.2/contrib/pg_upgrade/pg_upgrade.h postgresql.3/contrib/pg_upgrade/pg_upgrade.h
--- postgresql.2/contrib/pg_upgrade/pg_upgrade.h 2013-01-02 09:19:03.654519603 +0100
+++ postgresql.3/contrib/pg_upgrade/pg_upgrade.h 2013-01-02 12:47:14.578678309 +0100
@@ -438,10 +438,6 @@ void
prep_status(const char *fmt,...)
__attribute__((format(PG_PRINTF_ATTRIBUTE, 1, 2)));
void check_ok(void);
-char *pg_strdup(const char *s);
-void *pg_malloc(size_t size);
-void *pg_realloc(void *ptr, size_t size);
-void pg_free(void *ptr);
const char *getErrorText(int errNum);
unsigned int str2uint(const char *str);
void pg_putenv(const char *var, const char *val);
diff -durpN postgresql.2/contrib/pg_upgrade/util.c postgresql.3/contrib/pg_upgrade/util.c
--- postgresql.2/contrib/pg_upgrade/util.c 2013-01-02 09:19:03.655519614 +0100
+++ postgresql.3/contrib/pg_upgrade/util.c 2013-01-02 12:47:14.579678318 +0100
@@ -213,55 +213,6 @@ get_user_info(char **user_name)
}
-void *
-pg_malloc(size_t size)
-{
- void *p;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- p = malloc(size);
- if (p == NULL)
- pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
- return p;
-}
-
-void *
-pg_realloc(void *ptr, size_t size)
-{
- void *p;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- p = realloc(ptr, size);
- if (p == NULL)
- pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
- return p;
-}
-
-
-void
-pg_free(void *ptr)
-{
- if (ptr != NULL)
- free(ptr);
-}
-
-
-char *
-pg_strdup(const char *s)
-{
- char *result = strdup(s);
-
- if (result == NULL)
- pg_log(PG_FATAL, "%s: out of memory\n", os_info.progname);
-
- return result;
-}
-
-
/*
* getErrorText()
*
diff -durpN postgresql.2/src/bin/initdb/initdb.c postgresql.3/src/bin/initdb/initdb.c
--- postgresql.2/src/bin/initdb/initdb.c 2013-01-02 11:28:34.369505958 +0100
+++ postgresql.3/src/bin/initdb/initdb.c 2013-01-02 13:38:44.281677642 +0100
@@ -200,8 +200,6 @@ const char *subdirs[] = {
static char bin_path[MAXPGPATH];
static char backend_exec[MAXPGPATH];
-static void *pg_malloc(size_t size);
-static char *pg_strdup(const char *s);
static char **replace_token(char **lines,
const char *token, const char *replacement);
@@ -317,43 +315,6 @@ do { \
#define DIR_SEP "\\"
#endif
-/*
- * routines to check mem allocations and fail noisily.
- *
- * Note that we can't call exit_nicely() on a memory failure, as it calls
- * rmtree() which needs memory allocation. So we just exit with a bang.
- */
-static void *
-pg_malloc(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-static char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
static char *
escape_quotes(const char *src)
{
diff -durpN postgresql.2/src/bin/pg_basebackup/streamutil.c postgresql.3/src/bin/pg_basebackup/streamutil.c
--- postgresql.2/src/bin/pg_basebackup/streamutil.c 2013-01-02 09:19:03.856521815 +0100
+++ postgresql.3/src/bin/pg_basebackup/streamutil.c 2013-01-02 12:57:57.401140484 +0100
@@ -26,43 +26,6 @@ static char *dbpassword = NULL;
PGconn *conn = NULL;
/*
- * strdup() and malloc() replacements that print an error and exit
- * if something goes wrong. Can never return NULL.
- */
-char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-void *
-pg_malloc0(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- fprintf(stderr, _("%s: out of memory\n"), progname);
- exit(1);
- }
- MemSet(result, 0, size);
- return result;
-}
-
-
-/*
* Connect to the server. Returns a valid PGconn pointer if connected,
* or NULL on non-permanent error. On permanent error, the function will
* call exit(1) directly.
diff -durpN postgresql.2/src/bin/pg_basebackup/streamutil.h postgresql.3/src/bin/pg_basebackup/streamutil.h
--- postgresql.2/src/bin/pg_basebackup/streamutil.h 2012-10-03 10:40:48.299207401 +0200
+++ postgresql.3/src/bin/pg_basebackup/streamutil.h 2013-01-02 13:34:42.760112017 +0100
@@ -15,8 +15,4 @@ extern PGconn *conn;
exit(code); \
}
-
-extern char *pg_strdup(const char *s);
-extern void *pg_malloc0(size_t size);
-
extern PGconn *GetConnection(void);
diff -durpN postgresql.2/src/bin/pg_ctl/pg_ctl.c postgresql.3/src/bin/pg_ctl/pg_ctl.c
--- postgresql.2/src/bin/pg_ctl/pg_ctl.c 2013-01-02 09:19:03.857521826 +0100
+++ postgresql.3/src/bin/pg_ctl/pg_ctl.c 2013-01-02 13:36:01.760623946 +0100
@@ -118,8 +118,6 @@ write_stderr(const char *fmt,...)
/* This extension allows gcc to check the format string for consistency with
the supplied arguments. */
__attribute__((format(PG_PRINTF_ATTRIBUTE, 1, 2)));
-static void *pg_malloc(size_t size);
-static char *pg_strdup(const char *s);
static void do_advice(void);
static void do_help(void);
static void set_mode(char *modeopt);
@@ -226,42 +224,6 @@ write_stderr(const char *fmt,...)
}
/*
- * routines to check memory allocations and fail noisily.
- */
-
-static void *
-pg_malloc(size_t size)
-{
- void *result;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- result = malloc(size);
- if (!result)
- {
- write_stderr(_("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-
-static char *
-pg_strdup(const char *s)
-{
- char *result;
-
- result = strdup(s);
- if (!result)
- {
- write_stderr(_("%s: out of memory\n"), progname);
- exit(1);
- }
- return result;
-}
-
-/*
* Given an already-localized string, print it to stdout unless the
* user has specified that no messages should be printed.
*/
diff -durpN postgresql.2/src/bin/pg_dump/common.c postgresql.3/src/bin/pg_dump/common.c
--- postgresql.2/src/bin/pg_dump/common.c 2013-01-02 09:19:03.857521826 +0100
+++ postgresql.3/src/bin/pg_dump/common.c 2013-01-02 12:47:14.587678379 +0100
@@ -18,7 +18,6 @@
#include <ctype.h>
#include "catalog/pg_class.h"
-#include "dumpmem.h"
#include "dumputils.h"
diff -durpN postgresql.2/src/bin/pg_dump/compress_io.c postgresql.3/src/bin/pg_dump/compress_io.c
--- postgresql.2/src/bin/pg_dump/compress_io.c 2013-01-02 09:19:03.857521826 +0100
+++ postgresql.3/src/bin/pg_dump/compress_io.c 2013-01-02 12:47:14.587678379 +0100
@@ -53,7 +53,6 @@
*/
#include "compress_io.h"
-#include "dumpmem.h"
#include "dumputils.h"
/*----------------------
diff -durpN postgresql.2/src/bin/pg_dump/dumpmem.c postgresql.3/src/bin/pg_dump/dumpmem.c
--- postgresql.2/src/bin/pg_dump/dumpmem.c 2013-01-02 09:19:03.857521826 +0100
+++ postgresql.3/src/bin/pg_dump/dumpmem.c 1970-01-01 01:00:00.000000000 +0100
@@ -1,76 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * dumpmem.c
- * Memory allocation routines used by pg_dump, pg_dumpall, and pg_restore
- *
- * Portions Copyright (c) 1996-2013, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
- *
- *
- * IDENTIFICATION
- * src/bin/pg_dump/dumpmem.c
- *
- *-------------------------------------------------------------------------
- */
-#include "postgres_fe.h"
-
-#include "dumputils.h"
-#include "dumpmem.h"
-
-
-/*
- * Safer versions of some standard C library functions. If an
- * out-of-memory condition occurs, these functions will bail out via exit();
- *therefore, their return value is guaranteed to be non-NULL.
- */
-
-char *
-pg_strdup(const char *string)
-{
- char *tmp;
-
- if (!string)
- exit_horribly(NULL, "cannot duplicate null pointer\n");
- tmp = strdup(string);
- if (!tmp)
- exit_horribly(NULL, "out of memory\n");
- return tmp;
-}
-
-void *
-pg_malloc(size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- tmp = malloc(size);
- if (!tmp)
- exit_horribly(NULL, "out of memory\n");
- return tmp;
-}
-
-void *
-pg_malloc0(size_t size)
-{
- void *tmp;
-
- tmp = pg_malloc(size);
- MemSet(tmp, 0, size);
- return tmp;
-}
-
-void *
-pg_realloc(void *ptr, size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of realloc(NULL, 0) */
- if (ptr == NULL && size == 0)
- size = 1;
- tmp = realloc(ptr, size);
- if (!tmp)
- exit_horribly(NULL, "out of memory\n");
- return tmp;
-}
diff -durpN postgresql.2/src/bin/pg_dump/dumpmem.h postgresql.3/src/bin/pg_dump/dumpmem.h
--- postgresql.2/src/bin/pg_dump/dumpmem.h 2013-01-02 09:19:03.857521826 +0100
+++ postgresql.3/src/bin/pg_dump/dumpmem.h 1970-01-01 01:00:00.000000000 +0100
@@ -1,22 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * dumpmem.h
- * Memory allocation routines used by pg_dump, pg_dumpall, and pg_restore
- *
- * Portions Copyright (c) 1996-2013, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
- *
- * src/bin/pg_dump/dumpmem.h
- *
- *-------------------------------------------------------------------------
- */
-
-#ifndef DUMPMEM_H
-#define DUMPMEM_H
-
-extern char *pg_strdup(const char *string);
-extern void *pg_malloc(size_t size);
-extern void *pg_malloc0(size_t size);
-extern void *pg_realloc(void *ptr, size_t size);
-
-#endif /* DUMPMEM_H */
diff -durpN postgresql.2/src/bin/pg_dump/Makefile postgresql.3/src/bin/pg_dump/Makefile
--- postgresql.2/src/bin/pg_dump/Makefile 2013-01-02 09:19:03.857521826 +0100
+++ postgresql.3/src/bin/pg_dump/Makefile 2013-01-02 12:47:21.267724965 +0100
@@ -20,7 +20,7 @@ override CPPFLAGS := -I$(libpq_srcdir) $
OBJS= pg_backup_archiver.o pg_backup_db.o pg_backup_custom.o \
pg_backup_null.o pg_backup_tar.o \
- pg_backup_directory.o dumpmem.o dumputils.o compress_io.o $(WIN32RES)
+ pg_backup_directory.o dumputils.o compress_io.o $(WIN32RES)
KEYWRDOBJS = keywords.o kwlookup.o
@@ -35,8 +35,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort
pg_restore: pg_restore.o $(OBJS) $(KEYWRDOBJS) | submake-libpq submake-libpgport
$(CC) $(CFLAGS) pg_restore.o $(KEYWRDOBJS) $(OBJS) $(libpq_pgport) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
-pg_dumpall: pg_dumpall.o dumputils.o dumpmem.o $(KEYWRDOBJS) | submake-libpq submake-libpgport
- $(CC) $(CFLAGS) pg_dumpall.o dumputils.o dumpmem.o $(KEYWRDOBJS) $(WIN32RES) $(libpq_pgport) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+pg_dumpall: pg_dumpall.o dumputils.o $(KEYWRDOBJS) | submake-libpq submake-libpgport
+ $(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(KEYWRDOBJS) $(WIN32RES) $(libpq_pgport) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
install: all installdirs
$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
diff -durpN postgresql.2/src/bin/pg_dump/nls.mk postgresql.3/src/bin/pg_dump/nls.mk
--- postgresql.2/src/bin/pg_dump/nls.mk 2012-04-16 19:57:22.596917323 +0200
+++ postgresql.3/src/bin/pg_dump/nls.mk 2013-01-02 12:47:21.268724970 +0100
@@ -3,7 +3,7 @@ CATALOG_NAME = pg_dump
AVAIL_LANGUAGES = de es fr it ja ko pt_BR sv tr zh_CN zh_TW
GETTEXT_FILES = pg_backup_archiver.c pg_backup_db.c pg_backup_custom.c \
pg_backup_null.c pg_backup_tar.c \
- pg_backup_directory.c dumpmem.c dumputils.c compress_io.c \
+ pg_backup_directory.c dumputils.c compress_io.c \
pg_dump.c common.c pg_dump_sort.c \
pg_restore.c pg_dumpall.c \
../../port/exec.c
diff -durpN postgresql.2/src/bin/pg_dump/pg_backup_archiver.c postgresql.3/src/bin/pg_dump/pg_backup_archiver.c
--- postgresql.2/src/bin/pg_dump/pg_backup_archiver.c 2012-10-21 10:56:15.398945610 +0200
+++ postgresql.3/src/bin/pg_dump/pg_backup_archiver.c 2013-01-02 12:47:21.270724967 +0100
@@ -21,7 +21,6 @@
*/
#include "pg_backup_db.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <ctype.h>
diff -durpN postgresql.2/src/bin/pg_dump/pg_backup_custom.c postgresql.3/src/bin/pg_dump/pg_backup_custom.c
--- postgresql.2/src/bin/pg_dump/pg_backup_custom.c 2012-10-03 10:40:48.303207429 +0200
+++ postgresql.3/src/bin/pg_dump/pg_backup_custom.c 2013-01-02 12:47:21.270724967 +0100
@@ -26,7 +26,6 @@
#include "compress_io.h"
#include "dumputils.h"
-#include "dumpmem.h"
/*--------
* Routines in the format interface
diff -durpN postgresql.2/src/bin/pg_dump/pg_backup_db.c postgresql.3/src/bin/pg_dump/pg_backup_db.c
--- postgresql.2/src/bin/pg_dump/pg_backup_db.c 2012-07-25 10:51:14.973507544 +0200
+++ postgresql.3/src/bin/pg_dump/pg_backup_db.c 2013-01-02 12:47:21.271724969 +0100
@@ -11,7 +11,6 @@
*/
#include "pg_backup_db.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <unistd.h>
diff -durpN postgresql.2/src/bin/pg_dump/pg_backup_directory.c postgresql.3/src/bin/pg_dump/pg_backup_directory.c
--- postgresql.2/src/bin/pg_dump/pg_backup_directory.c 2013-01-02 09:19:03.858521837 +0100
+++ postgresql.3/src/bin/pg_dump/pg_backup_directory.c 2013-01-02 12:47:21.272724974 +0100
@@ -34,7 +34,6 @@
*/
#include "compress_io.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <dirent.h>
diff -durpN postgresql.2/src/bin/pg_dump/pg_backup_null.c postgresql.3/src/bin/pg_dump/pg_backup_null.c
--- postgresql.2/src/bin/pg_dump/pg_backup_null.c 2012-04-16 19:57:22.598917347 +0200
+++ postgresql.3/src/bin/pg_dump/pg_backup_null.c 2013-01-02 12:47:21.272724974 +0100
@@ -23,7 +23,6 @@
*/
#include "pg_backup_archiver.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <unistd.h> /* for dup */
diff -durpN postgresql.2/src/bin/pg_dump/pg_backup_tar.c postgresql.3/src/bin/pg_dump/pg_backup_tar.c
--- postgresql.2/src/bin/pg_dump/pg_backup_tar.c 2013-01-02 09:19:03.858521837 +0100
+++ postgresql.3/src/bin/pg_dump/pg_backup_tar.c 2013-01-02 12:47:21.273724983 +0100
@@ -28,7 +28,6 @@
#include "pg_backup.h"
#include "pg_backup_archiver.h"
#include "pg_backup_tar.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <sys/stat.h>
diff -durpN postgresql.2/src/bin/pg_dump/pg_dumpall.c postgresql.3/src/bin/pg_dump/pg_dumpall.c
--- postgresql.2/src/bin/pg_dump/pg_dumpall.c 2013-01-02 09:19:03.861521870 +0100
+++ postgresql.3/src/bin/pg_dump/pg_dumpall.c 2013-01-02 12:47:21.273724983 +0100
@@ -23,7 +23,6 @@
#include "getopt_long.h"
#include "dumputils.h"
-#include "dumpmem.h"
#include "pg_backup.h"
/* version string we expect back from pg_dump */
diff -durpN postgresql.2/src/bin/pg_dump/pg_dump.c postgresql.3/src/bin/pg_dump/pg_dump.c
--- postgresql.2/src/bin/pg_dump/pg_dump.c 2013-01-02 09:19:03.861521870 +0100
+++ postgresql.3/src/bin/pg_dump/pg_dump.c 2013-01-02 12:47:21.276725015 +0100
@@ -59,7 +59,6 @@
#include "pg_backup_archiver.h"
#include "pg_backup_db.h"
-#include "dumpmem.h"
#include "dumputils.h"
extern char *optarg;
diff -durpN postgresql.2/src/bin/pg_dump/pg_dump_sort.c postgresql.3/src/bin/pg_dump/pg_dump_sort.c
--- postgresql.2/src/bin/pg_dump/pg_dump_sort.c 2013-01-02 09:19:03.861521870 +0100
+++ postgresql.3/src/bin/pg_dump/pg_dump_sort.c 2013-01-02 12:47:21.276725015 +0100
@@ -15,7 +15,6 @@
*/
#include "pg_backup_archiver.h"
#include "dumputils.h"
-#include "dumpmem.h"
/* translator: this is a module name */
static const char *modulename = gettext_noop("sorter");
diff -durpN postgresql.2/src/bin/pg_dump/pg_restore.c postgresql.3/src/bin/pg_dump/pg_restore.c
--- postgresql.2/src/bin/pg_dump/pg_restore.c 2012-10-14 11:09:08.143146185 +0200
+++ postgresql.3/src/bin/pg_dump/pg_restore.c 2013-01-02 12:47:21.277725022 +0100
@@ -41,7 +41,6 @@
#include "pg_backup_archiver.h"
-#include "dumpmem.h"
#include "dumputils.h"
#include <ctype.h>
diff -durpN postgresql.2/src/bin/psql/common.c postgresql.3/src/bin/psql/common.c
--- postgresql.2/src/bin/psql/common.c 2013-01-02 09:19:03.863521892 +0100
+++ postgresql.3/src/bin/psql/common.c 2013-01-02 12:47:21.278725029 +0100
@@ -33,56 +33,6 @@ static bool command_no_begin(const char
static bool is_select_command(const char *query);
/*
- * "Safe" wrapper around strdup()
- */
-char *
-pg_strdup(const char *string)
-{
- char *tmp;
-
- if (!string)
- {
- psql_error("%s: pg_strdup: cannot duplicate null pointer (internal error)\n",
- pset.progname);
- exit(EXIT_FAILURE);
- }
- tmp = strdup(string);
- if (!tmp)
- {
- psql_error("out of memory\n");
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc(size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- tmp = malloc(size);
- if (!tmp)
- {
- psql_error("out of memory\n");
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc0(size_t size)
-{
- void *tmp;
-
- tmp = pg_malloc(size);
- MemSet(tmp, 0, size);
- return tmp;
-}
-
-/*
* setQFout
* -- handler for -o command line option and \o command
*
diff -durpN postgresql.2/src/bin/psql/common.h postgresql.3/src/bin/psql/common.h
--- postgresql.2/src/bin/psql/common.h 2013-01-02 09:19:03.863521892 +0100
+++ postgresql.3/src/bin/psql/common.h 2013-01-02 12:47:21.279725035 +0100
@@ -14,15 +14,6 @@
#define atooid(x) ((Oid) strtoul((x), NULL, 10))
-/*
- * Safer versions of some standard C library functions. If an
- * out-of-memory condition occurs, these functions will bail out
- * safely; therefore, their return value is guaranteed to be non-NULL.
- */
-extern char *pg_strdup(const char *string);
-extern void *pg_malloc(size_t size);
-extern void *pg_malloc0(size_t size);
-
extern bool setQFout(const char *fname);
extern void
diff -durpN postgresql.2/src/bin/scripts/common.c postgresql.3/src/bin/scripts/common.c
--- postgresql.2/src/bin/scripts/common.c 2013-01-02 09:19:03.871521980 +0100
+++ postgresql.3/src/bin/scripts/common.c 2013-01-02 13:36:57.808986089 +0100
@@ -278,55 +278,6 @@ executeMaintenanceCommand(PGconn *conn,
}
/*
- * "Safe" wrapper around strdup(). Pulled from psql/common.c
- */
-char *
-pg_strdup(const char *string)
-{
- char *tmp;
-
- if (!string)
- {
- fprintf(stderr, _("pg_strdup: cannot duplicate null pointer (internal error)\n"));
- exit(EXIT_FAILURE);
- }
- tmp = strdup(string);
- if (!tmp)
- {
- fprintf(stderr, _("out of memory\n"));
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc(size_t size)
-{
- void *tmp;
-
- /* Avoid unportable behavior of malloc(0) */
- if (size == 0)
- size = 1;
- tmp = malloc(size);
- if (!tmp)
- {
- fprintf(stderr, _("out of memory\n"));
- exit(EXIT_FAILURE);
- }
- return tmp;
-}
-
-void *
-pg_malloc0(size_t size)
-{
- void *tmp;
-
- tmp = pg_malloc(size);
- MemSet(tmp, 0, size);
- return tmp;
-}
-
-/*
* Check yes/no answer in a localized way. 1=yes, 0=no, -1=neither.
*/
diff -durpN postgresql.2/src/bin/scripts/common.h postgresql.3/src/bin/scripts/common.h
--- postgresql.2/src/bin/scripts/common.h 2013-01-02 09:19:03.871521980 +0100
+++ postgresql.3/src/bin/scripts/common.h 2013-01-02 13:36:44.504900390 +0100
@@ -50,8 +50,4 @@ extern bool yesno_prompt(const char *que
extern void setup_cancel_handler(void);
-extern char *pg_strdup(const char *string);
-extern void *pg_malloc(size_t size);
-extern void *pg_malloc0(size_t size);
-
#endif /* COMMON_H */
diff -durpN postgresql.2/src/include/port.h postgresql.3/src/include/port.h
--- postgresql.2/src/include/port.h 2013-01-02 11:28:47.256587185 +0100
+++ postgresql.3/src/include/port.h 2013-01-02 13:49:38.686060590 +0100
@@ -462,6 +462,17 @@ extern char *inet_net_ntop(int af, const
/* port/pgcheckdir.c */
extern int pg_check_dir(const char *dir);
+/* port/pgmalloc.c */
+extern char *pg_strdup(const char *string);
+
+extern void *pg_malloc(size_t size);
+
+extern void *pg_malloc0(size_t size);
+
+extern void *pg_realloc(void *ptr, size_t size);
+
+extern void pg_free(void *ptr);
+
/* port/pgmkdirp.c */
extern int pg_mkdir_p(char *path, int omode);
diff -durpN postgresql.2/src/port/Makefile postgresql.3/src/port/Makefile
--- postgresql.2/src/port/Makefile 2013-01-02 10:43:57.358410221 +0100
+++ postgresql.3/src/port/Makefile 2013-01-02 12:48:18.788120056 +0100
@@ -31,7 +31,7 @@ override CPPFLAGS := -I$(top_builddir)/s
LIBS += $(PTHREAD_LIBS)
OBJS = $(LIBOBJS) chklocale.o dirmod.o erand48.o exec.o fls.o inet_net_ntop.o \
- noblock.o path.o pgcheckdir.o pg_crc.o pgmkdirp.o pgsleep.o \
+ noblock.o path.o pgcheckdir.o pg_crc.o pgmalloc.o pgmkdirp.o pgsleep.o \
pgstrcasecmp.o quotes.o qsort.o qsort_arg.o sprompt.o tar.o thread.o
# foo_srv.o and foo.o are both built from foo.c, but only foo.o has -DFRONTEND
diff -durpN postgresql.2/src/port/pgmalloc.c postgresql.3/src/port/pgmalloc.c
--- postgresql.2/src/port/pgmalloc.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.3/src/port/pgmalloc.c 2013-01-02 12:51:05.238252844 +0100
@@ -0,0 +1,96 @@
+/*-------------------------------------------------------------------------
+ *
+ * pgmalloc.c
+ * Functions for allocating memory and
+ * exiting on out-of-memory
+ *
+ *
+ * Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pgmalloc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef FRONTEND
+#include "postgres_fe.h"
+
+#include <stdio.h>
+
+/*
+ * "Safe" wrapper around strdup()
+ */
+char *
+pg_strdup(const char *string)
+{
+ char *tmp;
+
+ if (!string)
+ {
+ fprintf(stderr, "pg_strdup: cannot duplicate null pointer (internal error)\n");
+ exit(1);
+ }
+ tmp = strdup(string);
+ if (!tmp)
+ {
+ fprintf(stderr, "pg_strdup: out of memory\n");
+ exit(1);
+ }
+ return tmp;
+}
+
+void *
+pg_malloc(size_t size)
+{
+ void *tmp;
+
+ /* Avoid unportable behavior of malloc(0) */
+ if (size == 0)
+ size = 1;
+ tmp = malloc(size);
+ if (!tmp)
+ {
+ fprintf(stderr, "pg_malloc: out of memory\n");
+ exit(1);
+ }
+ return tmp;
+}
+
+void *
+pg_malloc0(size_t size)
+{
+ void *tmp;
+
+ tmp = pg_malloc(size);
+ MemSet(tmp, 0, size);
+ return tmp;
+}
+
+void *
+pg_realloc(void *ptr, size_t size)
+{
+ void *tmp;
+
+ /* Avoid unportable behavior of realloc(NULL, 0) */
+ if (ptr == NULL && size == 0)
+ size = 1;
+ tmp = realloc(ptr, size);
+ if (!tmp)
+ {
+ fprintf(stderr, "pg_realloc: out of memory\n");
+ exit(1);
+ }
+ return tmp;
+}
+
+void
+pg_free(void *ptr)
+{
+ if (ptr != NULL)
+ free(ptr);
+}
+
+#endif
2013-01-02 11:54 keltezéssel, Boszormenyi Zoltan írta:
2013-01-02 10:37 keltezéssel, Boszormenyi Zoltan írta:
2013-01-02 10:12 keltezéssel, Magnus Hagander írta:
On Wed, Jan 2, 2013 at 9:59 AM, Boszormenyi Zoltan <zb@cybertec.at
<mailto:zb@cybertec.at>> wrote:2013-01-02 01:24 keltezéssel, Tom Lane írta:
Boszormenyi Zoltan <zb@cybertec.at <mailto:zb@cybertec.at>> writes:
2013-01-01 17:18 keltezéssel, Magnus Hagander írta:
That way we can get around the whole need for changing memory
allocation across all the
frontends, no? Like the attached.Sure it's simpler but then the consistent look of the code is lost.
What about the other patch to unify pg_malloc and friends?
Basically all client code boils down to
fprintf(stderr, ...)
in different disguise in their error reporting, so that patch can
also be simplified but it seems that the atexit() - either explicitly
or hidden behind InitPostgresFrontend() - cannot be avoided.Meh. I find it seriously wrongheaded that something as minor as an
escape_quotes() function should get to dictate both malloc wrappers
and error recovery handling throughout every program that might use it.Actually, the unification of pg_malloc and friends wasn't dictated
by this little code, it was just that pg_basebackup doesn't provide
a pg_malloc implementation (only pg_malloc0) that is used by
initdb's escape_quotes() function. Then I noticed how wide these
almost identical functions have spread into client apps already.I would say this unification patch is completely orthogonal to
the patch in $SUBJECT. I will post it in a different thread if it's
wanted at all. The extra atexit() handler is not needed if a simple
fprintf(stderr, ...) error reporting is enough in all clients.
As far as I saw, all clients do exactly this but some of them hide
this behind #define's.Please do keep that one separate - let's avoid unnecessary feature-creep, whether it's
good or bad features.I like Magnus' version a lot better than that idea.
OK, I will post the core patch building on his code.
Thanks.
A bigger issue that I notice with this code is that it's only correct in
backend-safe encodings, as the comment mentions. If we're going to be
putting it into frontend programs, how safe is that going to be?regards, tom lane
The question in a different form is: does PostgreSQL support
non-ASCII-safe encodings at all (or on the client side)? Forgive
my ignorance and enlighten me: how many such encodings
exist besides EBCDIC? Is UTF-16 non-ASCII-safe?We do. See http://www.postgresql.org/docs/9.2/static/multibyte.html.
There are quite a few far-eastern encodings that aren't available as server
encondings, and I believe it's all for this reason.I see, thanks.
That said, do we need to care *in this specific case*? We use it in initdb to parse
config strings, I believe. And we'd use it to parse a conninfo string in
pg_basebackup, correct?Correct.
Perhaps all we need to do is to clearly comment that it doesn't work with non-ascii
safe encodings, or rename it to indicate that it's limited in this?If you send a new patch with the function renamed accordingly, I will modify
my code to use it.Attached is the quotes-v2 patch, the function is renamed and
the comment is modified plus the pg_basebackup v21 patch
that uses this function.
Rebased after pgtar.h was added. The quotes.c comment was modified
even more so it doesn't say this function is used for SQL string literals.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
Attachments:
quotes-v4.patchtext/x-patch; name=quotes-v4.patchDownload
diff -durpN postgresql/src/bin/initdb/initdb.c postgresql.1/src/bin/initdb/initdb.c
--- postgresql/src/bin/initdb/initdb.c 2013-01-02 09:19:03.855521804 +0100
+++ postgresql.1/src/bin/initdb/initdb.c 2013-01-03 11:31:10.819564769 +0100
@@ -354,6 +354,18 @@ pg_strdup(const char *s)
return result;
}
+static char *
+escape_quotes(const char *src)
+{
+ char *result = escape_single_quotes_ascii(src);
+ if (!result)
+ {
+ fprintf(stderr, _("%s: out of memory\n"), progname);
+ exit(1);
+ }
+ return result;
+}
+
/*
* make a copy of the array of lines, with token replaced by replacement
* the first time it occurs on each line.
@@ -2415,35 +2427,6 @@ check_ok(void)
}
}
-/*
- * Escape (by doubling) any single quotes or backslashes in given string
- *
- * Note: this is used to process both postgresql.conf entries and SQL
- * string literals. Since postgresql.conf strings are defined to treat
- * backslashes as escapes, we have to double backslashes here. Hence,
- * when using this for a SQL string literal, use E'' syntax.
- *
- * We do not need to worry about encoding considerations because all
- * valid backend encodings are ASCII-safe.
- */
-static char *
-escape_quotes(const char *src)
-{
- int len = strlen(src),
- i,
- j;
- char *result = pg_malloc(len * 2 + 1);
-
- for (i = 0, j = 0; i < len; i++)
- {
- if (SQL_STR_DOUBLE(src[i], true))
- result[j++] = src[i];
- result[j++] = src[i];
- }
- result[j] = '\0';
- return result;
-}
-
/* Hack to suppress a warning about %x from some versions of gcc */
static inline size_t
my_strftime(char *s, size_t max, const char *fmt, const struct tm * tm)
diff -durpN postgresql/src/include/port.h postgresql.1/src/include/port.h
--- postgresql/src/include/port.h 2013-01-03 11:25:47.358179463 +0100
+++ postgresql.1/src/include/port.h 2013-01-03 11:31:10.819564769 +0100
@@ -465,4 +465,7 @@ extern int pg_check_dir(const char *dir)
/* port/pgmkdirp.c */
extern int pg_mkdir_p(char *path, int omode);
+/* port/quotes.c */
+extern char *escape_single_quotes_ascii(const char *src);
+
#endif /* PG_PORT_H */
diff -durpN postgresql/src/port/Makefile postgresql.1/src/port/Makefile
--- postgresql/src/port/Makefile 2013-01-02 09:19:03.935522680 +0100
+++ postgresql.1/src/port/Makefile 2013-01-03 11:31:10.819564769 +0100
@@ -32,7 +32,7 @@ LIBS += $(PTHREAD_LIBS)
OBJS = $(LIBOBJS) chklocale.o dirmod.o erand48.o exec.o fls.o inet_net_ntop.o \
noblock.o path.o pgcheckdir.o pg_crc.o pgmkdirp.o pgsleep.o \
- pgstrcasecmp.o qsort.o qsort_arg.o sprompt.o tar.o thread.o
+ pgstrcasecmp.o quotes.o qsort.o qsort_arg.o sprompt.o tar.o thread.o
# foo_srv.o and foo.o are both built from foo.c, but only foo.o has -DFRONTEND
OBJS_SRV = $(OBJS:%.o=%_srv.o)
diff -durpN postgresql/src/port/quotes.c postgresql.1/src/port/quotes.c
--- postgresql/src/port/quotes.c 1970-01-01 01:00:00.000000000 +0100
+++ postgresql.1/src/port/quotes.c 2013-01-03 13:28:45.092437852 +0100
@@ -0,0 +1,37 @@
+#include "c.h"
+
+/*
+ * Escape (by doubling) any single quotes or backslashes in given string
+ *
+ * Note: this is used to process postgresql.conf entries and to quote
+ * string literals in pg_basebackup for creating recovery.conf.
+ * Since postgresql.conf strings are defined to treat backslashes as escapes,
+ * we have to double backslashes here.
+ *
+ * Since this function is only used for parsing or creating configuration
+ * files, we do not care about encoding considerations.
+ *
+ * Returns a malloced() string that it's the responsibility of the caller
+ * to free.
+ */
+char *
+escape_single_quotes_ascii(const char *src)
+{
+ int len = strlen(src),
+ i,
+ j;
+ char *result = malloc(len * 2 + 1);
+
+ if (!result)
+ return NULL;
+
+ for (i = 0, j = 0; i < len; i++)
+ {
+ if (SQL_STR_DOUBLE(src[i], true))
+ result[j++] = src[i];
+ result[j++] = src[i];
+ }
+ result[j] = '\0';
+ return result;
+}
+
pg_basebackup-v22.patchtext/x-patch; name=pg_basebackup-v22.patchDownload
diff -durpN postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml
--- postgresql.1/doc/src/sgml/ref/pg_basebackup.sgml 2012-11-08 13:13:04.151630632 +0100
+++ postgresql.2/doc/src/sgml/ref/pg_basebackup.sgml 2013-01-03 11:31:57.216907223 +0100
@@ -189,6 +189,21 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--write-recovery-conf</option></term>
+ <listitem>
+
+ <para>
+ Write a minimal recovery.conf into the output directory (or into
+ the base archive file if <option>--format=tar</option> was specified)
+ using the connection parameters from the command line to ease
+ setting up the standby.
+ </para>
+
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-x</option></term>
<term><option>--xlog</option></term>
<listitem>
diff -durpN postgresql.1/src/bin/pg_basebackup/pg_basebackup.c postgresql.2/src/bin/pg_basebackup/pg_basebackup.c
--- postgresql.1/src/bin/pg_basebackup/pg_basebackup.c 2013-01-02 09:19:03.856521815 +0100
+++ postgresql.2/src/bin/pg_basebackup/pg_basebackup.c 2013-01-03 11:32:10.520005418 +0100
@@ -13,12 +13,15 @@
#include "postgres_fe.h"
#include "libpq-fe.h"
+#include "pqexpbuffer.h"
+#include "pgtar.h"
#include <unistd.h>
#include <dirent.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
+#include <time.h>
#ifdef HAVE_LIBZ
#include <zlib.h>
@@ -40,6 +43,7 @@ int compresslevel = 0;
bool includewal = false;
bool streamwal = false;
bool fastcheckpoint = false;
+bool writerecoveryconf = false;
int standby_message_timeout = 10 * 1000; /* 10 sec = default */
/* Progress counters */
@@ -64,13 +68,18 @@ static int has_xlogendptr = 0;
static volatile LONG has_xlogendptr = 0;
#endif
+PQExpBuffer rcExpBuf = NULL;
+
/* Function headers */
static void usage(void);
static void verify_dir_is_empty_or_create(char *dirname);
static void progress_report(int tablespacenum, const char *filename);
+static void scan_val(char *s, uint64 *val, unsigned int base, size_t len);
static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
+static void CreateRecoveryConf(PGconn *conn);
+static void WriteRecoveryConf(void);
static void BaseBackup(void);
static bool reached_end_position(XLogRecPtr segendpos, uint32 timeline,
@@ -101,6 +110,8 @@ usage(void)
printf(_("\nOptions controlling the output:\n"));
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain (default), tar)\n"));
+ printf(_(" -R, --write-recovery-conf\n"
+ " write recovery.conf after backup\n"));
printf(_(" -x, --xlog include required WAL files in backup (fetch mode)\n"));
printf(_(" -X, --xlog-method=fetch|stream\n"
" include required WAL files with specified method\n"));
@@ -446,6 +457,65 @@ progress_report(int tablespacenum, const
/*
+ * Inverse of print_val()
+ */
+static void
+scan_val(char *s, uint64 *val, unsigned int base, size_t len)
+{
+ uint64 tmp = 0;
+ int i;
+
+ for (i = 0; i < len; i++)
+ {
+ int digit = s[i] - '0';
+
+ tmp = tmp * base + digit;
+ }
+
+ *val = tmp;
+}
+
+
+/*
+ * Write a piece of tar data
+ */
+static void
+writeTarData(
+#ifdef HAVE_LIBZ
+ gzFile ztarfile,
+#endif
+ FILE *tarfile, char *buf, int r, char *current_file)
+{
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (gzwrite(ztarfile, buf, r) != r)
+ {
+ fprintf(stderr,
+ _("%s: could not write to compressed file \"%s\": %s\n"),
+ progname, current_file, get_gz_error(ztarfile));
+ disconnect_and_exit(1);
+ }
+ }
+ else
+#endif
+ {
+ if (fwrite(buf, r, 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
+ progname, current_file, strerror(errno));
+ disconnect_and_exit(1);
+ }
+ }
+}
+
+#ifdef HAVE_LIBZ
+#define WRITE_TAR_DATA(buf, sz) writeTarData(ztarfile, tarfile, buf, sz, filename)
+#else
+#define WRITE_TAR_DATA(buf, sz) writeTarData(tarfile, buf, sz, filename)
+#endif
+
+/*
* Receive a tar format file from the connection to the server, and write
* the data from this file directly into a tar file. If compression is
* enabled, the data will be compressed while written to the file.
@@ -461,12 +531,17 @@ ReceiveTarFile(PGconn *conn, PGresult *r
char filename[MAXPGPATH];
char *copybuf = NULL;
FILE *tarfile = NULL;
+ char tarhdr[512];
+ bool basetablespace = PQgetisnull(res, rownum, 0);
+ bool in_tarhdr, skip_file;
+ size_t tarhdrsz;
+ uint64 filesz;
#ifdef HAVE_LIBZ
gzFile ztarfile = NULL;
#endif
- if (PQgetisnull(res, rownum, 0))
+ if (basetablespace)
{
/*
* Base tablespaces
@@ -578,6 +653,16 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
+ /*
+ * Initialize our variables for tracking
+ * individual files inside the TAR stream.
+ * For more detailed explanation, see below.
+ */
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = 0;
+ filesz = 0;
+
while (1)
{
int r;
@@ -592,7 +677,8 @@ ReceiveTarFile(PGconn *conn, PGresult *r
if (r == -1)
{
/*
- * End of chunk. Close file (but not stdout).
+ * End of chunk. Write recovery.conf into the tar file (if it
+ * was requested) and close file (but not stdout).
*
* Also, write two completely empty blocks at the end of the tar
* file, as required by some tar programs.
@@ -600,30 +686,30 @@ ReceiveTarFile(PGconn *conn, PGresult *r
char zerobuf[1024];
MemSet(zerobuf, 0, sizeof(zerobuf));
-#ifdef HAVE_LIBZ
- if (ztarfile != NULL)
- {
- if (gzwrite(ztarfile, zerobuf, sizeof(zerobuf)) !=
- sizeof(zerobuf))
- {
- fprintf(stderr,
- _("%s: could not write to compressed file \"%s\": %s\n"),
- progname, filename, get_gz_error(ztarfile));
- disconnect_and_exit(1);
- }
- }
- else
-#endif
+
+ if (basetablespace && writerecoveryconf)
{
- if (fwrite(zerobuf, sizeof(zerobuf), 1, tarfile) != 1)
- {
- fprintf(stderr,
- _("%s: could not write to file \"%s\": %s\n"),
- progname, filename, strerror(errno));
- disconnect_and_exit(1);
- }
+ char header[512];
+ int padding;
+
+ tarCreateHeader(header, "recovery.conf", NULL,
+ rcExpBuf->len,
+ 0600, 04000, 02000,
+ time(NULL));
+
+ padding = ((rcExpBuf->len + 511) & ~511) - rcExpBuf->len;
+
+ WRITE_TAR_DATA(header, sizeof(header));
+ WRITE_TAR_DATA(rcExpBuf->data, rcExpBuf->len);
+ if (padding)
+ WRITE_TAR_DATA(zerobuf, padding);
+
+ if (verbose)
+ fprintf(stderr, _("%s: recovery.conf written into '%s'\n"), progname, filename);
}
+ WRITE_TAR_DATA(zerobuf, sizeof(zerobuf));
+
#ifdef HAVE_LIBZ
if (ztarfile != NULL)
{
@@ -659,25 +745,124 @@ ReceiveTarFile(PGconn *conn, PGresult *r
disconnect_and_exit(1);
}
-#ifdef HAVE_LIBZ
- if (ztarfile != NULL)
+ if (!writerecoveryconf || !basetablespace)
{
- if (gzwrite(ztarfile, copybuf, r) != r)
- {
- fprintf(stderr,
- _("%s: could not write to compressed file \"%s\": %s\n"),
- progname, filename, get_gz_error(ztarfile));
- disconnect_and_exit(1);
- }
+ /*
+ * If --write-recovery-conf was not requested or this
+ * is not the base tablespace, simply pass the received
+ * data into the TAR file, either compressed or not.
+ */
+
+ WRITE_TAR_DATA(copybuf, r);
}
else
-#endif
{
- if (fwrite(copybuf, r, 1, tarfile) != 1)
+ /*
+ * If --write-recovery-conf was requested AND this
+ * is the base tablespace, the TAR stream may contain
+ * a recovery.conf file if the backup is coming from
+ * a standby server. We have to skip this file in
+ * the stream and add a new one constructed by
+ * CreateRecoveryConf() at the end of the stream.
+ *
+ * To do this, we have to process the individual files
+ * inside the TAR stream. The stream consists of a header
+ * and zero or more chunks, all 512 bytes long. The stream
+ * from the server is broken up into smaller pieces, so
+ * we have to track the size of the files to find the next
+ * header structure.
+ */
+ int rr = r; /* Save the value returned by PQgetCopyData */
+ int pos = 0;
+
+ while (rr > 0)
{
- fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"),
- progname, filename, strerror(errno));
- disconnect_and_exit(1);
+ if (in_tarhdr)
+ {
+ /*
+ * We're currently reading a header structure
+ * inside the TAR stream, i.e. the file metadata.
+ */
+ if (tarhdrsz < 512)
+ {
+ /*
+ * Copy the header structure into tarhdr[]
+ * in case the header is not aligned to 512 bytes
+ * or it's not returned in whole by the last
+ * PQgetCopyData call.
+ */
+ int hdrleft, bytes2copy;
+
+ hdrleft = 512 - tarhdrsz;
+ bytes2copy = (rr > hdrleft ? hdrleft : rr);
+
+ memcpy(&tarhdr[tarhdrsz], copybuf + pos, bytes2copy);
+
+ rr -= bytes2copy;
+ pos += bytes2copy;
+ tarhdrsz += bytes2copy;
+ }
+ else
+ {
+ /*
+ * We have the whole header structure in tarhdr[],
+ * look at the file metadata:
+ * - the subsequent file contents have to be skipped
+ * if the filename is recovery.conf
+ * - find out the size of the file padded to the next
+ * multiple of 512
+ */
+ int64 padding;
+
+ skip_file = (strcmp(&tarhdr[0], "recovery.conf") == 0);
+
+ scan_val(&tarhdr[124], &filesz, 8, 11);
+
+ padding = ((filesz + 511) & ~511) - filesz;
+ filesz += padding;
+
+ /* Indicate that the subsequent data is the file content. */
+ in_tarhdr = false;
+
+ if (!skip_file)
+ WRITE_TAR_DATA(tarhdr, 512);
+ }
+ }
+ else
+ {
+ /*
+ * We're processing a file's contents.
+ */
+ if (filesz > 0)
+ {
+ /*
+ * We still have data to read (and possibly write).
+ */
+ int bytes2write;
+
+ bytes2write = (filesz > rr ? rr : filesz);
+
+ if (!skip_file)
+ WRITE_TAR_DATA(copybuf + pos, bytes2write);
+
+ rr -= bytes2write;
+ pos += bytes2write;
+ filesz -= bytes2write;
+ }
+ else
+ {
+ /*
+ * No more data in the current file,
+ * the next piece of data (if any) will
+ * be a new file header structure.
+ * Reinitialize all our variables.
+ */
+ in_tarhdr = true;
+ skip_file = false;
+ tarhdrsz = 0;
+ filesz = 0;
+ }
+ }
}
}
totaldone += r;
@@ -706,10 +891,11 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
char filename[MAXPGPATH];
int current_len_left;
int current_padding = 0;
+ bool basetablespace = PQgetisnull(res, rownum, 0);
char *copybuf = NULL;
FILE *file = NULL;
- if (PQgetisnull(res, rownum, 0))
+ if (basetablespace)
strcpy(current_path, basedir);
else
strcpy(current_path, PQgetvalue(res, rownum, 1));
@@ -931,6 +1117,131 @@ ReceiveAndUnpackTarFile(PGconn *conn, PG
if (copybuf != NULL)
PQfreemem(copybuf);
+
+ if (basetablespace)
+ WriteRecoveryConf();
+}
+
+static char *
+escape_quotes(const char *src)
+{
+ char *result = escape_single_quotes_ascii(src);
+ if (!result)
+ {
+ fprintf(stderr, _("%s: out of memory\n"), progname);
+ exit(1);
+ }
+ return result;
+}
+
+/*
+ * Create the new recovery.conf in memory
+ */
+static void
+CreateRecoveryConf(PGconn *conn)
+{
+ PQconninfoOption *connOptions;
+ PQconninfoOption *option;
+
+ if (!writerecoveryconf)
+ return;
+
+ connOptions = PQconninfo(conn);
+ if (connOptions == NULL)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "standby_mode = 'on'\n");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "primary_conninfo = '");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ for (option = connOptions; option && option->keyword; option++)
+ {
+ char *escaped;
+
+ /*
+ * Do not emit this setting if:
+ * - the setting is "replication", "dbname" or
+ * "fallback_application_name", since these would be
+ * overridden by the libpqwalreceiver module anyway.
+ * - not set or empty.
+ */
+ if (strcmp(option->keyword, "replication") == 0 ||
+ strcmp(option->keyword, "dbname") == 0 ||
+ strcmp(option->keyword, "fallback_application_name") == 0 ||
+ (option->val == NULL) ||
+ (option->val != NULL && option->val[0] == '\0'))
+ continue;
+
+ /*
+ * Write "keyword='value'" pieces, the value string is escaped
+ * if necessary and doubled single quotes around the value string.
+ */
+ escaped = escape_quotes(option->val);
+
+ appendPQExpBuffer(rcExpBuf, "%s=''%s'' ", option->keyword, escaped);
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ free(escaped);
+ }
+
+ appendPQExpBufferStr(rcExpBuf, "'\n");
+ if (PQExpBufferBroken(rcExpBuf))
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ PQconninfoFree(connOptions);
+}
+
+
+static void
+WriteRecoveryConf(void)
+{
+ char filename[MAXPGPATH];
+ FILE *cf;
+
+ if (!writerecoveryconf)
+ return;
+
+ sprintf(filename, "%s/recovery.conf", basedir);
+
+ cf = fopen(filename, "w");
+ if (cf == NULL)
+ {
+ fprintf(stderr, _("%s: cannot create %s: %s"), progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ if (fwrite(rcExpBuf->data, rcExpBuf->len, 1, cf) != 1)
+ {
+ fprintf(stderr,
+ _("%s: could not write to file \"%s\": %s\n"),
+ progname, filename, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
+ fclose(cf);
+
+ if (verbose)
+ fprintf(stderr, _("%s: recovery.conf written.\n"), progname);
}
@@ -954,6 +1265,15 @@ BaseBackup(void)
/* Error message already written in GetConnection() */
exit(1);
+ rcExpBuf = createPQExpBuffer();
+ if (!rcExpBuf)
+ {
+ fprintf(stderr, _("%s: out of memory"), progname);
+ disconnect_and_exit(1);
+ }
+
+ CreateRecoveryConf(conn);
+
/*
* Run IDENTIFY_SYSTEM so we can get the timeline
*/
@@ -1217,6 +1537,9 @@ BaseBackup(void)
#endif
}
+ /* Free the recovery.conf contents */
+ destroyPQExpBuffer(rcExpBuf);
+
/*
* End of copy data. Final result is already checked inside the loop.
*/
@@ -1237,6 +1560,7 @@ main(int argc, char **argv)
{"pgdata", required_argument, NULL, 'D'},
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
+ {"write-recovery-conf", no_argument, NULL, 'R'},
{"xlog", no_argument, NULL, 'x'},
{"xlog-method", required_argument, NULL, 'X'},
{"gzip", no_argument, NULL, 'z'},
@@ -1274,7 +1598,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:xX:l:zZ:c:h:p:U:s:wWvP",
+ while ((c = getopt_long(argc, argv, "D:F:RxX:l:zZ:c:h:p:U:s:wWvP",
long_options, &option_index)) != -1)
{
switch (c)
@@ -1295,6 +1619,9 @@ main(int argc, char **argv)
exit(1);
}
break;
+ case 'R':
+ writerecoveryconf = true;
+ break;
case 'x':
if (includewal)
{
On Thu, Jan 3, 2013 at 1:33 PM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
2013-01-02 11:54 keltezéssel, Boszormenyi Zoltan írta:
2013-01-02 10:37 keltezéssel, Boszormenyi Zoltan írta:
2013-01-02 10:12 keltezéssel, Magnus Hagander írta:
Attached is the quotes-v2 patch, the function is renamed and
the comment is modified plus the pg_basebackup v21 patch
that uses this function.Rebased after pgtar.h was added. The quotes.c comment was modified
even more so it doesn't say this function is used for SQL string literals.
Quotes patch applied with a few small changes:
1) Support for the msvc build (needs an entry added for new files that
go in src/port if they should be used on Windows)
2) Makefile entries should be alphabetical (yes, that's really trivial
nitpicking :D)
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sat, Jan 5, 2013 at 3:41 PM, Magnus Hagander <magnus@hagander.net> wrote:
On Thu, Jan 3, 2013 at 1:33 PM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
2013-01-02 11:54 keltezéssel, Boszormenyi Zoltan írta:
2013-01-02 10:37 keltezéssel, Boszormenyi Zoltan írta:
2013-01-02 10:12 keltezéssel, Magnus Hagander írta:
Attached is the quotes-v2 patch, the function is renamed and
the comment is modified plus the pg_basebackup v21 patch
that uses this function.Rebased after pgtar.h was added. The quotes.c comment was modified
even more so it doesn't say this function is used for SQL string literals.Quotes patch applied with a few small changes:
1) Support for the msvc build (needs an entry added for new files that
go in src/port if they should be used on Windows)
2) Makefile entries should be alphabetical (yes, that's really trivial
nitpicking :D)
After some further modifications, I've applied the pg_basebackup patch as well.
Thanks! And again, apologies for dragging the process out longer than
should've been necessary.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
2013-01-05 16:58 keltezéssel, Magnus Hagander írta:
On Sat, Jan 5, 2013 at 3:41 PM, Magnus Hagander <magnus@hagander.net> wrote:
On Thu, Jan 3, 2013 at 1:33 PM, Boszormenyi Zoltan <zb@cybertec.at> wrote:
2013-01-02 11:54 keltezéssel, Boszormenyi Zoltan írta:
2013-01-02 10:37 keltezéssel, Boszormenyi Zoltan írta:
2013-01-02 10:12 keltezéssel, Magnus Hagander írta:
Attached is the quotes-v2 patch, the function is renamed and
the comment is modified plus the pg_basebackup v21 patch
that uses this function.Rebased after pgtar.h was added. The quotes.c comment was modified
even more so it doesn't say this function is used for SQL string literals.Quotes patch applied with a few small changes:
1) Support for the msvc build (needs an entry added for new files that
go in src/port if they should be used on Windows)
2) Makefile entries should be alphabetical (yes, that's really trivial
nitpicking :D)After some further modifications, I've applied the pg_basebackup patch as well.
Thank you very much!
Thanks! And again, apologies for dragging the process out longer than
should've been necessary.
I blamed it on me not having done reviews earlier in the commitfest,
which I finally did last week. Now, if only Tom could find some time
to review the lock_timeout patch... ;-)
Thanks again and best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Boszormenyi Zoltan wrote:
2013-01-02 10:12 keltezéssel, Magnus Hagander írta:
<mailto:zb@cybertec.at>> wrote:
Actually, the unification of pg_malloc and friends wasn't dictated
by this little code, it was just that pg_basebackup doesn't provide
a pg_malloc implementation (only pg_malloc0) that is used by
initdb's escape_quotes() function. Then I noticed how wide these
almost identical functions have spread into client apps already.
Please do keep that one separate - let's avoid unnecessary
feature-creep, whether it's good or bad features.The patch is attached. There is no extra atexit() code in this one.
I have applied a patch similar in spirit to this one.
--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers