pg_archive_bypass
Hi,
I tend to consider it a bug that there's no known way under windows to
use the same trick as under Unix by using '/usr/bin/true' as your
archive command. And this Unix trick itself does feel like a hack.
Also I'd very much like to be able to recommend (even if not change the
official defaults) to setup wal_level to archive, archive_mode=on and
archive_command=pg_archive_bypass, so that the day you have a HA budget
ain't the day you're going to restart the server to enable the fault
tolerance settings…
So please find attached a very simple "let's see about it" patch to
implement an internal archive_command that just returns true and is
called pg_archive_bypass. It's missing documentation, which I'll provide
if needed (meaning there's some will to consider applying such a patch).
Regards,
--
dim
Attachments:
pg_archive_bypass.difftext/x-diffDownload
*** a/src/backend/postmaster/pgarch.c
--- b/src/backend/postmaster/pgarch.c
***************
*** 475,481 **** pgarch_ArchiverCopyLoop(void)
/*
* pgarch_archiveXlog
*
! * Invokes system(3) to copy one archive file to wherever it should go
*
* Returns true if successful
*/
--- 475,485 ----
/*
* pgarch_archiveXlog
*
! * Invokes system(3) to copy one archive file to wherever it should go, or
! * just return true if set to the internal command "pg_archive_bypass". That
! * allows windows users to easily have the "/usr/bin/true", and that allows
! * for setting up a archiving only when you need it by changing the command
! * and issuing a reload.
*
* Returns true if successful
*/
***************
*** 490,495 **** pgarch_archiveXlog(char *xlog)
--- 494,509 ----
const char *sp;
int rc;
+ /*
+ * If the command is "pg_archive_bypass", just return true
+ */
+ if( strcmp(XLogArchiveCommand, "pg_archive_bypass") == 0 )
+ {
+ ereport(DEBUG3,
+ (errmsg_internal("bypassing archive command")));
+ return true;
+ }
+
snprintf(pathname, MAXPGPATH, XLOGDIR "/%s", xlog);
/*
On Mon, 2010-06-14 at 12:39 +0200, Dimitri Fontaine wrote:
I tend to consider it a bug that there's no known way under windows to
use the same trick as under Unix by using '/usr/bin/true' as your
archive command. And this Unix trick itself does feel like a hack.Also I'd very much like to be able to recommend (even if not change the
official defaults) to setup wal_level to archive, archive_mode=on and
archive_command=pg_archive_bypass, so that the day you have a HA budget
ain't the day you're going to restart the server to enable the fault
tolerance settings…So please find attached a very simple "let's see about it" patch to
implement an internal archive_command that just returns true and is
called pg_archive_bypass. It's missing documentation, which I'll provide
if needed (meaning there's some will to consider applying such a patch).
ISTM like a good idea to have a couple of more obvious commands provided
purely as "internal commands". If we do this on restore_command as well,
we can skip pg_archivecleanup completely, for example.
So, not for 9.0.
But I like concept for 9.1. Would need to be coded as some kind of
escape phrase, followed by other command. So not just a quick hack with
this one exception.
Something like
archive_command = 'pg_archive_internal:true'
Wouldn't rush to another patch though, needs agreement first.
--
Simon Riggs www.2ndQuadrant.com
On 14/06/10 13:39, Dimitri Fontaine wrote:
I tend to consider it a bug that there's no known way under windows to
use the same trick as under Unix by using '/usr/bin/true' as your
archive command. And this Unix trick itself does feel like a hack.Also I'd very much like to be able to recommend (even if not change the
official defaults) to setup wal_level to archive, archive_mode=on and
archive_command=pg_archive_bypass, so that the day you have a HA budget
ain't the day you're going to restart the server to enable the fault
tolerance settings…
That particular use case is better handled by making archive_mode
changeable on-the-fly. Now that we have wal_level as a separate GUC, it
shouldn't be too hard.
I like internal commands as a solution in general, but it's still in
search of a problem..
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:
On 14/06/10 13:39, Dimitri Fontaine wrote:
I tend to consider it a bug that there's no known way under windows to
use the same trick as under Unix by using '/usr/bin/true' as your
archive command. And this Unix trick itself does feel like a hack.Also I'd very much like to be able to recommend (even if not change the
official defaults) to setup wal_level to archive, archive_mode=on and
archive_command=pg_archive_bypass, so that the day you have a HA budget
ain't the day you're going to restart the server to enable the fault
tolerance settings…That particular use case is better handled by making archive_mode changeable
on-the-fly. Now that we have wal_level as a separate GUC, it shouldn't be
too hard.
Ok.
I like internal commands as a solution in general, but it's still in search
of a problem..
What about /usr/bin/true, or a simple archive where you cp in a given
location (which could happen to be a remote server thanks to unix
network file systems or windows shares), etc. It seems to me those are
existing problem that we solve poorly: let each user face the same
pitfalls (error management).
I also like Simon's approach to have options to internal commands
too. What about this syntax :
guc_name = "pg_internal function_name arg1 %r ... argn"
Then function_name is any SQL callable function and you have the %x
substitution still happening there for you. So you can mix hard-coded
arguments (some path) and dynamic arguments. And you can script your
commands in PL/WhateverU.
It's more involved a patch, of course, but if we do that and provide
some simple commands in -core, then most installations will only use
that and stop bothering writing scripts to handle their simple
cases. Which is what I'm after.
Regards,
--
dim
On Mon, Jun 14, 2010 at 1:55 PM, Dimitri Fontaine
<dfontaine@hi-media.com> wrote:
What about /usr/bin/true, or a simple archive where you cp in a given
location (which could happen to be a remote server thanks to unix
network file systems or windows shares), etc. It seems to me those are
existing problem that we solve poorly: let each user face the same
pitfalls (error management).
I would like to see the case where the archive is just a mounted
directory accessible through the filesystem be handled internally.
Ideally if I could just specify the archive location on both the
master and slave using the same parameter just containing a filesystem
path then I could have the same configuration on both machines except
for the actual parameter which decides whether they're a master or
slave. That would make me much more confident I've configured both
machines properly. All the nodes would have the same information and
they would be deciding for themselves whether to push or pull archives
based on whether they're the master or slave.
--
greg
Dimitri Fontaine wrote:
Hi,
I tend to consider it a bug that there's no known way under windows to
use the same trick as under Unix by using '/usr/bin/true' as your
archive command. And this Unix trick itself does feel like a hack.Also I'd very much like to be able to recommend (even if not change the
official defaults) to setup wal_level to archive, archive_mode=on and
archive_command=pg_archive_bypass, so that the day you have a HA budget
ain't the day you're going to restart the server to enable the fault
tolerance settings?So please find attached a very simple "let's see about it" patch to
implement an internal archive_command that just returns true and is
called pg_archive_bypass. It's missing documentation, which I'll provide
if needed (meaning there's some will to consider applying such a patch).
Turn out 'REM' acts like /bin/true on Windows. I have documented that
fact in the attached, applied patch.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ None of us is going to be here forever. +
Attachments:
/rtmp/difftext/x-diffDownload
Index: doc/src/sgml/config.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/config.sgml,v
retrieving revision 1.287
diff -c -c -r1.287 config.sgml
*** doc/src/sgml/config.sgml 29 Jun 2010 22:29:13 -0000 1.287
--- doc/src/sgml/config.sgml 30 Jun 2010 02:40:40 -0000
***************
*** 1793,1799 ****
disabled, but the server continues to accumulate WAL segment files in
the expectation that a command will soon be provided. Setting
<varname>archive_command</> to a command that does nothing but
! return true, e.g. <literal>/bin/true</>, effectively disables
archiving, but also breaks the chain of WAL files needed for
archive recovery, so it should only be used in unusual circumstances.
</para>
--- 1793,1800 ----
disabled, but the server continues to accumulate WAL segment files in
the expectation that a command will soon be provided. Setting
<varname>archive_command</> to a command that does nothing but
! return true, e.g. <literal>/bin/true</> (<literal>REM</> on
! Windows), effectively disables
archiving, but also breaks the chain of WAL files needed for
archive recovery, so it should only be used in unusual circumstances.
</para>
Bruce Momjian <bruce@momjian.us> writes:
Turn out 'REM' acts like /bin/true on Windows. I have documented that
fact in the attached, applied patch.
I guess that kills it for the "pg_archive_bypass" internal command, but
in a good way. Thanks!
Regards,
--
Dimitri Fontaine
PostgreSQL DBA, Architecte
Damn, I still baffled that we can escape the internal commands
altogether.