WAL archive on slave

James Sewell wrote:

My understanding is that WAL archiving can not be enabled on the slave in a streaming replication
pair.

It can be enabled. Did you try it?

If this is correct, is there a reason behind it? I can see logs showing up in pg_xlog, so could they
not be archived?

These are files containing the WAL data replicated from the master.
They won't be archived.

The reason I ask is if this happened it would allow the following with a streaming replication pair
(A,B):

1. Start A as master
2. Attach B as slave using basebackup
3. work ....
4. Promote B to master

5. Restore A from a scheduled backup to a time before promotion
6. Attach A as slave pointing at B's WAL archive

If we used A's WAL archive in this case and A had writes after the promotion then we would get
timeline errors.

You shouldn't with 9.3, because in that case A would follow the
timeline switch introduced by B's promotion rather than its old timelime.
/messages/by-id/E1TjCRc-00084r-1H@gemulon.postgresql.org

I may be missing something there since I have never tried it.

As far as I can tell, using the WAL archive from B would resolve this issue.

That should work in any event.

Yours,
Laurenz Albe

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Thanks for the reply Albe.

I have WAL archiving enabled on both my servers, but it only happens on the
master.

When I promote the slave to be the new master then it will start archiving
automatically, which suggests that my configuration is correct.

Can you think of anything else which might cause this?

Cheers,

James Sewell,
PostgreSQL Team Lead / Solutions Architect
______________________________________

Level 2, 50 Queen St, Melbourne VIC 3000

*P *(+61) 3 8370 8000 *W* www.lisasoft.com *F *(+61) 3 8370 8099

On Fri, Feb 7, 2014 at 7:30 PM, Albe Laurenz <laurenz.albe@wien.gv.at>wrote:

James Sewell wrote:

My understanding is that WAL archiving can not be enabled on the slave

in a streaming replication

pair.

It can be enabled. Did you try it?

If this is correct, is there a reason behind it? I can see logs showing

up in pg_xlog, so could they

not be archived?

These are files containing the WAL data replicated from the master.
They won't be archived.

The reason I ask is if this happened it would allow the following with a

streaming replication pair

(A,B):

1. Start A as master
2. Attach B as slave using basebackup
3. work ....
4. Promote B to master

5. Restore A from a scheduled backup to a time before promotion
6. Attach A as slave pointing at B's WAL archive

If we used A's WAL archive in this case and A had writes after the

promotion then we would get

timeline errors.

You shouldn't with 9.3, because in that case A would follow the
timeline switch introduced by B's promotion rather than its old timelime.

/messages/by-id/E1TjCRc-00084r-1H@gemulon.postgresql.org

I may be missing something there since I have never tried it.

As far as I can tell, using the WAL archive from B would resolve this

issue.

That should work in any event.

Yours,
Laurenz Albe

--

------------------------------
The contents of this email are confidential and may be subject to legal or
professional privilege and copyright. No representation is made that this
email is free of viruses or other defects. If you have received this
communication in error, you may not copy or distribute any part of it or
otherwise disclose its contents to anyone. Please advise the sender of your
incorrect receipt of this correspondence.

Hi again,

I just realised I asked this question around a year ago (for a different
reason actually) and you answered me then! Amazing!

Your answer then was:

It works fine, only the server will not generate WAL while

it is in recovery. As soon as you promote the standby,
it will archive ist WALs.

Is this still correct? If it is the the only way that I could achieve what
I wanted would be to set wal_keep_segments high enough then they will all
be archived on promotion?

I'm still not sure why they wouldn't be archived on the slave, seen as they
show up in the directory? Is there a limitation I'm not thinking about here?

Cheers,

James Sewell,
PostgreSQL Team Lead / Solutions Architect
______________________________________

Level 2, 50 Queen St, Melbourne VIC 3000

*P *(+61) 3 8370 8000 *W* www.lisasoft.com *F *(+61) 3 8370 8099

On Mon, Feb 10, 2014 at 11:16 AM, James Sewell <james.sewell@lisasoft.com>wrote:

Thanks for the reply Albe.

I have WAL archiving enabled on both my servers, but it only happens on
the master.

When I promote the slave to be the new master then it will start archiving
automatically, which suggests that my configuration is correct.

Can you think of anything else which might cause this?

Cheers,

James Sewell,
PostgreSQL Team Lead / Solutions Architect
______________________________________

Level 2, 50 Queen St, Melbourne VIC 3000

*P *(+61) 3 8370 8000 *W* www.lisasoft.com *F *(+61) 3 8370 8099

On Fri, Feb 7, 2014 at 7:30 PM, Albe Laurenz <laurenz.albe@wien.gv.at>wrote:

James Sewell wrote:

My understanding is that WAL archiving can not be enabled on the slave

in a streaming replication

pair.

It can be enabled. Did you try it?

If this is correct, is there a reason behind it? I can see logs showing

up in pg_xlog, so could they

not be archived?

These are files containing the WAL data replicated from the master.
They won't be archived.

The reason I ask is if this happened it would allow the following with

a streaming replication pair

(A,B):

1. Start A as master
2. Attach B as slave using basebackup
3. work ....
4. Promote B to master

5. Restore A from a scheduled backup to a time before promotion
6. Attach A as slave pointing at B's WAL archive

If we used A's WAL archive in this case and A had writes after the

promotion then we would get

timeline errors.

You shouldn't with 9.3, because in that case A would follow the
timeline switch introduced by B's promotion rather than its old timelime.

/messages/by-id/E1TjCRc-00084r-1H@gemulon.postgresql.org

I may be missing something there since I have never tried it.

As far as I can tell, using the WAL archive from B would resolve this

issue.

That should work in any event.

Yours,
Laurenz Albe

--

------------------------------
The contents of this email are confidential and may be subject to legal or
professional privilege and copyright. No representation is made that this
email is free of viruses or other defects. If you have received this
communication in error, you may not copy or distribute any part of it or
otherwise disclose its contents to anyone. Please advise the sender of your
incorrect receipt of this correspondence.

On Mon, Feb 10, 2014 at 9:43 AM, James Sewell <james.sewell@lisasoft.com>wrote:

Hi again,

I just realised I asked this question around a year ago (for a different
reason actually) and you answered me then! Amazing!

Your answer then was:

It works fine, only the server will not generate WAL while

it is in recovery. As soon as you promote the standby,
it will archive ist WALs.

Is this still correct?

Yes, it is. A slave will begin to archive WALs after getting promoted and
dumping to a new timeline.

I'm still not sure why they wouldn't be archived on the slave, seen as
they show up in the directory?

WAL files are only replayed on a slave, not archived.
Regards
--
Michael

James Sewell wrote:

If it is the the only way that I could achieve what I wanted would be to set
wal_keep_segments high enough then they will all be archived on promotion?

Even if you set wal_keep_segments high I don't think that the replayed
WAL will be archived.

I'm still not sure why they wouldn't be archived on the slave, seen as they show up in the directory?
Is there a limitation I'm not thinking about here?

I guess that the idea is that a cluster will only archive the WAL it
generates. Typically, the primary would archive these (if you don't archive
WAL on the primary, you're probably doing something wrong).
So this would only lead to duplicates.

Yours,
Laurenz Albe

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

I can understand it will create duplicates, but it would also allow for
recovery from backups.

If a backup was taken at midnight, and promotion happened at 6am then
having archiving on the slave would allow log replay from the backup.

Log replay from the old master would potentially end up in the incorrect
timeline.

Although potentially merging the archive logs favoring the new master's
logs might fix this?

Either way, it seems the answer to my question is no.

Thanks for the help!

-James Sewell

On Monday, 10 February 2014, Albe Laurenz <laurenz.albe@wien.gv.at> wrote:

James Sewell wrote:

If it is the the only way that I could achieve what I wanted would be to

set

wal_keep_segments high enough then they will all be archived on

promotion?

Even if you set wal_keep_segments high I don't think that the replayed
WAL will be archived.

I'm still not sure why they wouldn't be archived on the slave, seen as

they show up in the directory?

Is there a limitation I'm not thinking about here?

I guess that the idea is that a cluster will only archive the WAL it
generates. Typically, the primary would archive these (if you don't
archive
WAL on the primary, you're probably doing something wrong).
So this would only lead to duplicates.

Yours,
Laurenz Albe

--

James Sewell,
PostgreSQL Team Lead / Solutions Architect
______________________________________

Level 2, 50 Queen St, Melbourne VIC 3000

*P *(+61) 3 8370 8000 *W* www.lisasoft.com *F *(+61) 3 8370 8099

--

------------------------------
The contents of this email are confidential and may be subject to legal or
professional privilege and copyright. No representation is made that this
email is free of viruses or other defects. If you have received this
communication in error, you may not copy or distribute any part of it or
otherwise disclose its contents to anyone. Please advise the sender of your
incorrect receipt of this correspondence.