Online backup vs Continuous backup

Bruce Momjian <pgman@candle.pha.pa.us> writes:

I suggest the following patch to rename our capability "Continuous
Backup".

This doesn't seem like an improvement. "Online backup" is the standard
terminology AFAIK.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

I suggest the following patch to rename our capability "Continuous
Backup".

This doesn't seem like an improvement. "Online backup" is the standard
terminology AFAIK.

But why is it the standard terminology? It doesn't seem logical.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:

I noticed that we are using the term "Online Backup" in our
documentation when we are talking about continuous backup and PITR.

To me, "online backup" is doing a backup while the system is online
(online-backup), and that is accomplished by pg_dump. I know a lot of
databases us "Online Backup" but I assume this is for historical reasons
because at some time in the past their full backups didn't work while
the database was online. Other systems use the term "Continuous
Logging", but I think that is too easily confused with the server
activity logs.

I suggest the following patch to rename our capability "Continuous
Backup".

I have never heard the term Continuous backup. Although I have heard
online backup. The problem is that when I hear the term online backup
I think Hot backup which is what we do with pg_dump.

I would just call the other Point in time recovery :)

Joshua D. Drake wrote:

Bruce Momjian wrote:

I noticed that we are using the term "Online Backup" in our
documentation when we are talking about continuous backup and PITR.

To me, "online backup" is doing a backup while the system is online
(online-backup), and that is accomplished by pg_dump. I know a lot of
databases us "Online Backup" but I assume this is for historical reasons
because at some time in the past their full backups didn't work while
the database was online. Other systems use the term "Continuous
Logging", but I think that is too easily confused with the server
activity logs.

I suggest the following patch to rename our capability "Continuous
Backup".

I have never heard the term Continuous backup. Although I have heard
online backup. The problem is that when I hear the term online backup
I think Hot backup which is what we do with pg_dump.

Yes, that is my problem too.

I would just call the other Point in time recovery :)

The problem there is that it is "recovery", and the documentation talks
about the "backup" step. I am thinking we are going to have to add
something like "Continuous backup, also called Online Backup" or
something like that.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

On Mon, 26 Dec 2005, Bruce Momjian wrote:

Joshua D. Drake wrote:

I have never heard the term Continuous backup. Although I have heard
online backup. The problem is that when I hear the term online backup
I think Hot backup which is what we do with pg_dump.

Yes, that is my problem too.

I would just call the other Point in time recovery :)

The problem there is that it is "recovery", and the documentation talks
about the "backup" step. I am thinking we are going to have to add
something like "Continuous backup, also called Online Backup" or
something like that.

FWIW, I believe "log archival" is a common term for that.

- Heikki

Am Montag, 26. Dezember 2005 20:46 schrieb Heikki Linnakangas:

FWIW, I believe "log archival" is a common term for that.

I like that better, although there are a lot of logs, so maybe "transaction
log archival".

Considering the list originally referenced:

- File system backup
- SQL dump
- (Transaction log archival?)

This addresses the technical workings of the various backup systems. The fact
that it is or isn't on-line, hot, continuous, fast, or flexible is secondary.

Peter Eisentraut wrote:

Am Montag, 26. Dezember 2005 20:46 schrieb Heikki Linnakangas:

FWIW, I believe "log archival" is a common term for that.

I like that better, although there are a lot of logs, so maybe
"transaction log archival".

Yep, when I think of log archiving, I think of saving the server logs,
which thanks to log file rotation, we kind of support.

The problem is that when I think of the transaction log, I think of clog
because it is the transaction status log. :-(

I am thinking anything that says "log" is going to be confusing. How about
"Transaction Archiving"?

Considering the list originally referenced:

- File system backup - SQL dump - (Transaction log archival?)

This addresses the technical workings of the various backup systems.
The fact that it is or isn't on-line, hot, continuous, fast, or flexible
is secondary.

To me, the continuous activity is the significant feature of that backup
method. I chose "Continuous Backup" because it is the continual
activity that is significant. "Continuous Archiving"?

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

I noticed that we are using the term "Online Backup" in our
documentation when we are talking about continuous backup and PITR.

To me, "online backup" is doing a backup while the system is online
(online-backup), and that is accomplished by pg_dump.

Well fwiw what pg_dump does isn't even considered a "backup" at all in other
systems. It's a "logical export" or something of that sort.

It's not considered a "backup" because it's saving something different than
the actual physical database. When you restore you get something (hopefully)
logically equivalent but still physically different.

Hot backups which is what you read "online backup" to mean is pretty closely
tied to PITR log archiving backups. One isn't very useful without the other.
Specifically, you can have all the archived logs in the world but if you don't
have a backup to start from they're useless. And in postgres I gather hot
backups aren't very useful if you don't have the transaction logs necessary to
fix any torn pages.

--
greg

Bruce Momjian <pgman@candle.pha.pa.us> writes:

This addresses the technical workings of the various backup systems.
The fact that it is or isn't on-line, hot, continuous, fast, or flexible
is secondary.

To me, the continuous activity is the significant feature of that backup
method. I chose "Continuous Backup" because it is the continual
activity that is significant. "Continuous Archiving"?

I like Peter's suggestion as-is; or "WAL Archiving". I don't like
applying adjectives that are not specific to the particular method.
That will just create more confusion down the road if we add another
feature that could also be called "continuous archiving".

regards, tom lane

Greg Stark said:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

I noticed that we are using the term "Online Backup" in our
documentation when we are talking about continuous backup and PITR.

To me, "online backup" is doing a backup while the system is online
(online-backup), and that is accomplished by pg_dump.

Well fwiw what pg_dump does isn't even considered a "backup" at all in
other systems. It's a "logical export" or something of that sort.

It's not considered a "backup" because it's saving something different
than the actual physical database. When you restore you get something
(hopefully) logically equivalent but still physically different.

This seems fairly arbitrary. On that basis anything on a higher level than
dd is not a backup method, ISTM.

cheers

andrew

"Andrew Dunstan" <andrew@dunslane.net> writes:

Greg Stark said:

Well fwiw what pg_dump does isn't even considered a "backup" at all in
other systems. It's a "logical export" or something of that sort.

It's not considered a "backup" because it's saving something different
than the actual physical database. When you restore you get something
(hopefully) logically equivalent but still physically different.

This seems fairly arbitrary. On that basis anything on a higher level than
dd is not a backup method, ISTM.

Well most of other options *aren't* any higher level than dd.

Even with PITR you're just storing the transaction logs byte for byte. When
you replay it works at slightly higher level but it's still far from
reinterpreting your data and you'll get (almost) exactly the same bytes back.

I'm not saying pg_dump is a bad thing, but now that Postgres has real backups
there isn't much of a use case for using pg_dump as a stand-in for backups. If
your system crashes and you need to bring up a new system quickly you want to
bring it up in exactly the state the one that crashed was in. You don't want
to pass all the data through a program that interprets the ascii
representation again and hope it comes up with the same data.

There are other reasons to need pg_dump of course. Just not for backups.

--
greg

Hi there,

How can i send mail form postgresql.

any suggestion.

thanx & regards
aftab

Code from perl cookbook, wrapped in plperlu wrapper

not very portable, Net::SMTP would probably be better, and result in more
portable perl code.

Didn't test this - but it is copied straight from perl cookbook via
google: http://www.unix.org.ua/orelly/perl/cookbook/ch18_04.htm

CREATE OR REPLACE FUNCTION send_email(from_address text, to_address text,
subject text, body text) RETURNS void AS $$

use Mail::Mailer;
my ($from_address, $to_address, $subject, $body) = @_;

my $mailer = Mail::Mailer->new("sendmail");
$mailer->open({ From => $from_address,
To => $to_address,
Subject => $subject,
})
or die "Can't open: $!";
print $mailer $body;
$mailer->close();

$$ LANGUAGE plperlu VOLATILE STRICT;

On Wed, 28 Dec 2005, Aftab Alam wrote:

Hi there,

How can i send mail form postgresql.

any suggestion.

thanx & regards
aftab

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

--
Don't steal; thou'lt never thus compete successfully in business. Cheat.
-- Ambrose Bierce

Bruce Momjian <pgman@candle.pha.pa.us> writes:

I suggest the following patch to rename our capability "Continuous
Backup".

This doesn't seem like an improvement. "Online backup" is the standard
terminology AFAIK.

There's something ambiguous about it, in view that there are two
useful points in time for a backup to be "as of," namely:
a) The start time (which is what pg_dump does), and
b) The end time (which I believe is not uncommon with other systems).

PITR is obviously more like b)...
--
wm(X,Y):-write(X),write('@'),write(Y). wm('cbbrowne','gmail.com').
http://linuxfinances.info/info/slony.html
Keeping instructions and operands in different memories saves .20
(.09) microseconds.

Hello,

it depend on your possibilities. Simply, use PL/Perl or PL/sh.

Regards
Pavel Stehule

or use PgSendMail
http://sourceforge.net/project/showfiles.php?group_id=35804

Hi there,

How can i send mail form postgresql.

any suggestion.

thanx & regards
aftab

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

_________________________________________________________________
Citite se osamele? Poznejte nekoho vyjmecneho diky Match.com.
http://www.msn.cz/

On Wed, Dec 28, 2005 at 08:52:41AM +0530, Aftab Alam wrote:

Hi there,

How can i send mail form postgresql.

Generally, it is a better idea either to poll your PostgreSQL database
and send mail based on that, or use LISTEN/NOTIFY, rather than
directly sending mail from your database.

Cheers,
D
--
David Fetter david@fetter.org http://fetter.org/
phone: +1 415 235 3778

Remember to vote!

On Mon, 2005-12-26 at 13:46 -0500, Bruce Momjian wrote:

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

I suggest the following patch to rename our capability "Continuous
Backup".

This doesn't seem like an improvement. "Online backup" is the standard
terminology AFAIK.

But why is it the standard terminology? It doesn't seem logical.

Well, as Greg says its a physical backup that can be done on-line, so
online backup makes perfect sense to me. I've never had somebody say
"that makes no sense" before. Nomenclature is different everywhere, I
accept.

I generally describe it like this:

Logical Backup
- use pg_dump - must be done on-line
Physical Backup
All file copy only
- must be Cold/Off-line backup
All file copy + WAL archiving
- allows Hot/Online or Cold/Offline backup

People understand those terms...

When do I mention PITR? Well, I describe this as Archive Recovery, with
an option to go to end-of-logs, or to a point-in-time.
[In the code, the mode variable is InArchiveRecovery.]

I do think that saying "do you use PITR?" makes little sense. We should
be talking about the backup mode, not the potential future recovery
mode.

I think it would all make more sense if we described the use of
archive_command = something as being in "WAL Archive Mode". That would
then allow us to say:
"You can only take Online Backups while in WAL Archive Mode".
"If you ever wish to perform PITR, you must use WAL Archive Mode".
"If you backed-up in WAL Archive Mode, you can perform an Archive
Recovery".

Best Regards, Simon Riggs

How about:

use "Online backup" or "Hot backup" to refer to either method of back
since they are both done while the system is online or hot.

If you want to get specific refer to doing a "sql dump" etc for using
pg_dump
Then use "Incremental backup" to refer to the whole process of the
WAL archival etc
Refer to the actual log files themselves as transaction logs.

That all seems to be pretty intuitive and non-ambiguous non-confusing
to me.

On Dec 26, 2005, at 11:44 AM, Tom Lane wrote:

I think it would all make more sense if we described the use of
archive_command = something as being in "WAL Archive Mode". That would
then allow us to say:
"You can only take Online Backups while in WAL Archive Mode".
"If you ever wish to perform PITR, you must use WAL Archive Mode".
"If you backed-up in WAL Archive Mode, you can perform an Archive
Recovery".

It seems to me there are two different context in which one would be
making statements like this. And what we are "allowed to say"
depends greatly on context. These contexts are as follows:

1) Explaining the feature set of postgres to a potential user.
2) Explaining to an actual postgres user how to actually do something.

In the first case it makes the most sense to me to use industry
standard or very intuitive terminology to the extend that it exists.
ie (Transaction Logs vs. WAL). Incremental Backup and Point in Time
Recovery seem to be fairly commonly used and understood database
buzzwords for someone to investigate the feature set of an RDBMS.

In the second case it seems to me that the most important thing is
that you pick terminology that is consistent, unambiguous and clearly
defined. Log archival, PITR, etc are not point and click operations
like they are in say MS SQL Server. This gives us more flexibility
but it also requires a deeper understanding. If someone is unwilling
or unable to to learn whatever terminology you happen to come up with
then it seems to me they shouldn't even be attempting to set up one
of those features. At the same time if the terminology you uses
changes all the time (is not consistent), or if you can't figure out
what any of the terms mean (they are not clearly defined) or if you
use terms like "online backup" to mean both types of backup but then
use it once in a specific circumstance where only one usage is
appropriate (you are using the terms ambiguously) then users will be
confused and it will be your fault not theirs.

Just my 2 cents

Rick Gigger

I used your suggestion and renamed "online backup" to "incremental
backup", and added a mention that many database vendors call it "online
backup".

Patch attached.

---------------------------------------------------------------------------

Rick Gigger wrote:

How about:

use "Online backup" or "Hot backup" to refer to either method of back
since they are both done while the system is online or hot.

If you want to get specific refer to doing a "sql dump" etc for using
pg_dump
Then use "Incremental backup" to refer to the whole process of the
WAL archival etc
Refer to the actual log files themselves as transaction logs.

That all seems to be pretty intuitive and non-ambiguous non-confusing
to me.

On Dec 26, 2005, at 11:44 AM, Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

I suggest the following patch to rename our capability "Continuous
Backup".

This doesn't seem like an improvement. "Online backup" is the
standard
terminology AFAIK.

regards, tom lane

---------------------------(end of
broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:

I used your suggestion and renamed "online backup" to "incremental
backup", and added a mention that many database vendors call it
"online backup".

Consistency would then demand that the other two be renamed to "full
backup". I think we had better suggestions earlier.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:

Bruce Momjian wrote:

I used your suggestion and renamed "online backup" to "incremental
backup", and added a mention that many database vendors call it
"online backup".

Consistency would then demand that the other two be renamed to "full
backup". I think we had better suggestions earlier.

I am also now thinking the word "incremental" is wrong because it
implies something that happens in parts, like "backup everything that
changed from last night until now" which not how this feature works.

I am thinking "Continuous Archiving" is the proper wording, and it
avoids the "backup" word.

Updated patch attached.

--
Bruce Momjian http://candle.pha.pa.us
SRA OSS, Inc. http://www.sraoss.com

+ If your life is a hard drive, Christ can be your backup. +

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

This addresses the technical workings of the various backup systems.
The fact that it is or isn't on-line, hot, continuous, fast, or flexible
is secondary.

To me, the continuous activity is the significant feature of that backup
method. I chose "Continuous Backup" because it is the continual
activity that is significant. "Continuous Archiving"?

I like Peter's suggestion as-is; or "WAL Archiving". I don't like
applying adjectives that are not specific to the particular method.
That will just create more confusion down the road if we add another
feature that could also be called "continuous archiving".

I do like "WAL Archiving" but I am concerned that it references a
specific internal implementation detail (WAL). Is that OK? Also, I am
concerned that is specificially talks about archiving the wal files and
is not general enough to include the file system backup that is also
necessary.

--
Bruce Momjian http://candle.pha.pa.us
SRA OSS, Inc. http://www.sraoss.com

+ If your life is a hard drive, Christ can be your backup. +

Applied.

---------------------------------------------------------------------------

Bruce Momjian wrote:

Peter Eisentraut wrote:

Bruce Momjian wrote:

I used your suggestion and renamed "online backup" to "incremental
backup", and added a mention that many database vendors call it
"online backup".

Consistency would then demand that the other two be renamed to "full
backup". I think we had better suggestions earlier.

I am also now thinking the word "incremental" is wrong because it
implies something that happens in parts, like "backup everything that
changed from last night until now" which not how this feature works.

I am thinking "Continuous Archiving" is the proper wording, and it
avoids the "backup" word.

Updated patch attached.

--
Bruce Momjian http://candle.pha.pa.us
SRA OSS, Inc. http://www.sraoss.com

+ If your life is a hard drive, Christ can be your backup. +

Index: doc/src/sgml/backup.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/backup.sgml,v
retrieving revision 2.76
diff -c -c -r2.76 backup.sgml
*** doc/src/sgml/backup.sgml	7 Nov 2005 17:36:44 -0000	2.76
--- doc/src/sgml/backup.sgml	14 Feb 2006 04:00:50 -0000
***************
*** 19,25 ****
<itemizedlist>
<listitem><para><acronym>SQL</> dump</para></listitem>
<listitem><para>File system level backup</para></listitem>
!    <listitem><para>On-line backup</para></listitem>
</itemizedlist>
Each has its own strengths and weaknesses.
</para>
--- 19,25 ----
<itemizedlist>
<listitem><para><acronym>SQL</> dump</para></listitem>
<listitem><para>File system level backup</para></listitem>
!    <listitem><para>Continuous Archiving</para></listitem>
</itemizedlist>
Each has its own strengths and weaknesses.
</para>
***************
*** 372,382 ****
</para>
</sect1>

! <sect1 id="backup-online">
! <title>On-line backup and point-in-time recovery (PITR)</title>

<indexterm zone="backup">
! <primary>on-line backup</primary>
</indexterm>

<indexterm zone="backup">
--- 372,382 ----
</para>
</sect1>

! <sect1 id="continuous-archiving">
! <title>Continuous Archiving and Point-In-Time Recovery (PITR)</title>

<indexterm zone="backup">
! <primary>continuous archiving</primary>
</indexterm>

<indexterm zone="backup">
***************
*** 452,458 ****
</para>

<para>
!    To recover successfully using an on-line backup, you need a continuous
sequence of archived WAL files that extends back at least as far as the
start time of your backup.  So to get started, you should set up and test
your procedure for archiving WAL files <emphasis>before</> you take your
--- 452,459 ----
</para>

<para>
! To recover successfully using continuous archiving (also called "online
! backup" by many database vendors), you need a continuous
sequence of archived WAL files that extends back at least as far as the
start time of your backup. So to get started, you should set up and test
your procedure for archiving WAL files <emphasis>before</> you take your
***************
*** 782,793 ****
<function>pg_start_backup</> or <function>pg_stop_backup</>, and
you will therefore be left to your own devices to keep track of which
backup dump is which and how far back the associated WAL files go.
! It is generally better to follow the on-line backup procedure above.
</para>
</sect2>

<sect2 id="backup-pitr-recovery">
! <title>Recovering with an On-line Backup</title>

<para>
Okay, the worst has happened and you need to recover from your backup.
--- 783,794 ----
<function>pg_start_backup</> or <function>pg_stop_backup</>, and
you will therefore be left to your own devices to keep track of which
backup dump is which and how far back the associated WAL files go.
!     It is generally better to follow the continuous archiving procedure above.
</para>
</sect2>

<sect2 id="backup-pitr-recovery">
! <title>Recovering using a Continuous Archive Backup</title>

<para>
Okay, the worst has happened and you need to recover from your backup.
***************
*** 1119,1129 ****
</para>
</sect2>

! <sect2 id="backup-online-caveats">
<title>Caveats</title>

<para>
! At this writing, there are several limitations of the on-line backup
technique. These will probably be fixed in future releases:

<itemizedlist>
--- 1120,1130 ----
</para>
</sect2>

! <sect2 id="continuous-archiving-caveats">
<title>Caveats</title>

<para>
! At this writing, there are several limitations of the continuous archiving
technique. These will probably be fixed in future releases:

<itemizedlist>
Index: doc/src/sgml/config.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/config.sgml,v
retrieving revision 1.47
diff -c -c -r1.47 config.sgml
*** doc/src/sgml/config.sgml	5 Feb 2006 18:19:14 -0000	1.47
--- doc/src/sgml/config.sgml	14 Feb 2006 04:00:53 -0000
***************
*** 1387,1393 ****
<para>
Turning off this parameter does not affect use of
WAL archiving for point-in-time recovery (PITR)
!         (see <xref linkend="backup-online">).
</para>

<para>
--- 1387,1393 ----
<para>
Turning off this parameter does not affect use of
WAL archiving for point-in-time recovery (PITR)
!         (see <xref linkend="continuous-archiving">).
</para>

<para>
Index: doc/src/sgml/func.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/func.sgml,v
retrieving revision 1.306
diff -c -c -r1.306 func.sgml
*** doc/src/sgml/func.sgml	12 Feb 2006 04:44:15 -0000	1.306
--- doc/src/sgml/func.sgml	14 Feb 2006 04:00:59 -0000
***************
*** 9784,9790 ****

<para>
For details about proper usage of these functions, see
! <xref linkend="backup-online">.
</para>

<para>
--- 9784,9790 ----

<para>
For details about proper usage of these functions, see
! <xref linkend="continuous-archiving">.
</para>

<para>
Index: doc/src/sgml/wal.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/wal.sgml,v
retrieving revision 1.38
diff -c -c -r1.38 wal.sgml
*** doc/src/sgml/wal.sgml	4 Nov 2005 23:14:02 -0000	1.38
--- doc/src/sgml/wal.sgml	14 Feb 2006 04:01:00 -0000
***************
*** 136,142 ****
<para>
<acronym>WAL</acronym> also makes it possible to support on-line
backup and point-in-time recovery, as described in <xref
!     linkend="backup-online">.  By archiving the WAL data we can support
reverting to any time instant covered by the available WAL data:
we simply install a prior physical backup of the database, and
replay the WAL log just as far as the desired time.  What's more,
--- 136,142 ----
<para>
<acronym>WAL</acronym> also makes it possible to support on-line
backup and point-in-time recovery, as described in <xref
!     linkend="continuous-archiving">.  By archiving the WAL data we can support
reverting to any time instant covered by the available WAL data:
we simply install a prior physical backup of the database, and
replay the WAL log just as far as the desired time.  What's more,

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

--
Bruce Momjian http://candle.pha.pa.us
SRA OSS, Inc. http://www.sraoss.com

+ If your life is a hard drive, Christ can be your backup. +