Add -c to rsync commands on SR tutorial wiki page

On 8/17/16 5:34 PM, Jim Nasby wrote:

https://wiki.postgresql.org/wiki/Binary_Replication_Tutorial does not
specify -c for any of the rsync commands. That's maybe safe for WAL, but
I don't think it's safe for any of the other uses, right? I'd like
someone to confirm before I just change the page... my intention is to
just stick -c in all the commands.

Why do you think that?

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

* Jim Nasby (Jim.Nasby@BlueTreble.com) wrote:

https://wiki.postgresql.org/wiki/Binary_Replication_Tutorial does
not specify -c for any of the rsync commands. That's maybe safe for
WAL, but I don't think it's safe for any of the other uses, right?
I'd like someone to confirm before I just change the page... my
intention is to just stick -c in all the commands.

-c is only relevant when you are doing an incremental copy, but on a
quick look, all those rsync commands appear to be doing full copies?

You would want -c if you were taking a backup and then doing an update
of it using rsync. or something along those lines, as you can't really
trust rsync's time/size based comparison as it only has a 1 second level
granularity.

Thanks!

Stephen

On 8/17/16 9:46 PM, Stephen Frost wrote:

* Jim Nasby (Jim.Nasby@BlueTreble.com) wrote:

https://wiki.postgresql.org/wiki/Binary_Replication_Tutorial does
not specify -c for any of the rsync commands. That's maybe safe for
WAL, but I don't think it's safe for any of the other uses, right?
I'd like someone to confirm before I just change the page... my
intention is to just stick -c in all the commands.

-c is only relevant when you are doing an incremental copy, but on a
quick look, all those rsync commands appear to be doing full copies?

You would want -c if you were taking a backup and then doing an update
of it using rsync. or something along those lines, as you can't really
trust rsync's time/size based comparison as it only has a 1 second level
granularity.

I don't think it's any great leap for someone to think they can use
those commands incrementally. It's certainly one of the first things you
think of when using rsync. AFAIK there's no downside at all to using -c
when it is a brand new copy, so I'm thinking we should just put it in
there, especially considering what the potential downside is.

--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532) mobile: 512-569-9461

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

* Jim Nasby (Jim.Nasby@BlueTreble.com) wrote:

On 8/17/16 9:46 PM, Stephen Frost wrote:

* Jim Nasby (Jim.Nasby@BlueTreble.com) wrote:

https://wiki.postgresql.org/wiki/Binary_Replication_Tutorial does
not specify -c for any of the rsync commands. That's maybe safe for
WAL, but I don't think it's safe for any of the other uses, right?
I'd like someone to confirm before I just change the page... my
intention is to just stick -c in all the commands.

-c is only relevant when you are doing an incremental copy, but on a
quick look, all those rsync commands appear to be doing full copies?

You would want -c if you were taking a backup and then doing an update
of it using rsync. or something along those lines, as you can't really
trust rsync's time/size based comparison as it only has a 1 second level
granularity.

I don't think it's any great leap for someone to think they can use
those commands incrementally. It's certainly one of the first things
you think of when using rsync. AFAIK there's no downside at all to
using -c when it is a brand new copy, so I'm thinking we should just
put it in there, especially considering what the potential downside
is.

To have proper incremental backups done requires a lot more than just
throwing "-c" into the rsyncs. For my 2c, I'm at the point where I'd
prefer we discourage people from using rsync, cp, or generally try to
set up their own hand-rolled backup system with PG. Those examples
simply encourage poor setups that risk losing data and introducing
corruption.

Thanks!

Stephen

On 8/18/16 1:31 PM, Stephen Frost wrote:

To have proper incremental backups done requires a lot more than just
throwing "-c" into the rsyncs. For my 2c, I'm at the point where I'd
prefer we discourage people from using rsync, cp, or generally try to
set up their own hand-rolled backup system with PG. Those examples
simply encourage poor setups that risk losing data and introducing
corruption.

I agree (and the only reason I was looking at that page is because I
client was following the directions on it!)

So, what would this mean in terms of the wiki? Should we nuke that page
and point at something in the docs? Something else?
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532) mobile: 512-569-9461

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Thu, Aug 18, 2016 at 2:27 PM, Jim Nasby <Jim.Nasby@bluetreble.com> wrote:

I don't think it's any great leap for someone to think they can use those
commands incrementally. It's certainly one of the first things you think of
when using rsync. AFAIK there's no downside at all to using -c when it is a
brand new copy, so I'm thinking we should just put it in there, especially
considering what the potential downside is.

I think that's right.

The suggestion that people might use some backup tool is a good one,
but that's not a reason not to add -c here.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers