Streaming replication, loose ends

Heikki Linnakangas wrote:

I've now committed streaming replication. I moved the files around a
bit, and put the walreceiver/walsender stuff in a new
src/backend/replication subdirectory. There's enough stuff there already
to deserve a new subdirectory, and if we add the capability for
streaming base backups etc. that has been discussed, we will have more
code in there.

But it's not time to party yet. There's still a few loose ends we need
to address:

Documentation. The patch originally moved around some sections, but I
didn't include that in the committed version, to make it clear in the
diff what exactly was added/changed. But I do agree with the original
thought of adding a new "Replication" chapter, and moving all the
replication and standby related stuff there from the "Backup and
Restore" chapter, so let's start working on that.

Uh, do we really want to call this "replication" rather than archive log
streaming or something. It seems "replication" is a generic term and
will confuse people who are using other replication solutions like
Slony.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

2010/1/15 Bruce Momjian <bruce@momjian.us>:

Heikki Linnakangas wrote:

I've now committed streaming replication. I moved the files around a
bit, and put the walreceiver/walsender stuff in a new
src/backend/replication subdirectory. There's enough stuff there already
to deserve a new subdirectory, and if we add the capability for
streaming base backups etc. that has been discussed, we will have more
code in there.

But it's not time to party yet. There's still a few loose ends we need
to address:

Documentation. The patch originally moved around some sections, but I
didn't include that in the committed version, to make it clear in the
diff what exactly was added/changed. But I do agree with the original
thought of adding a new "Replication" chapter, and moving all the
replication and standby related stuff there from the "Backup and
Restore" chapter, so let's start working on that.

Uh, do we really want to call this "replication" rather than archive log
streaming or something.  It seems "replication" is a generic term and
will confuse people who are using other replication solutions like
Slony.

+1

Pavel

Hi,

Heikki Linnakangas �rta:

I've now committed streaming replication. I moved the files around a
bit, and put the walreceiver/walsender stuff in a new
src/backend/replication subdirectory. [snip]

there's one loose end indeed.
"make maintainer-clean" doesn't delete these:

src/backend/replication/walreceiver/walreceiver.o
src/backend/replication/walreceiver/walreceiver.so
src/backend/replication/walreceiver/.deps/

Best regards,
Zolt�n B�sz�rm�nyi

--
Bible has answers for everything. Proof:
"But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
than these cometh of evil." (Matthew 5:37) - basics of digital technology.
"May your kingdom come" - superficial description of plate tectonics

----------------------------------
Zolt�n B�sz�rm�nyi
Cybertec Sch�nig & Sch�nig GmbH
http://www.postgresql.at/

Boszormenyi Zoltan wrote:

Heikki Linnakangas �rta:

I've now committed streaming replication. I moved the files around a
bit, and put the walreceiver/walsender stuff in a new
src/backend/replication subdirectory. [snip]

there's one loose end indeed.
"make maintainer-clean" doesn't delete these:

src/backend/replication/walreceiver/walreceiver.o
src/backend/replication/walreceiver/walreceiver.so
src/backend/replication/walreceiver/.deps/

Hmm, I think I'm going to need some help with the Makefiles. Clearly the
way I hooked that directory to the build system was wrong. I think the
attached patch will fix that, but I wonder if there's a trick I'm missing.

Do the MSVC scripts need adjusting? 'red_bat' compiled fine, so I guess not.

Also, I'm seeing a failure in buildfarm member 'colugos':

/opt/local/bin/ccache /Developer/usr/bin/llvm-gcc-4.2 -no-cpp-precomp
-I/opt/local/include -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing
-fwrapv -g -bundle -multiply_defined suppress walreceiver.o
-bundle_loader ../../../../src/backend/postgres
-L../../../../src/interfaces/libpq -L../../../../src/port
-L/opt/local/lib -lpq -o walreceiver.so
ld: library not found for -lpq
collect2: ld returned 1 exit status
make[2]: *** [walreceiver.so] Error 1
make[2]: *** Waiting for unfinished jobs....

I suspect that's because libpq isn't built yet. I have this:

all: submake-libpq all-shared-lib

in src/backend/replication/walreceiver/Makefile, but is that not enough?

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Heikki Linnakangas wrote:

Also, I'm seeing a failure in buildfarm member 'colugos':

/opt/local/bin/ccache /Developer/usr/bin/llvm-gcc-4.2 -no-cpp-precomp
-I/opt/local/include -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing
-fwrapv -g -bundle -multiply_defined suppress walreceiver.o
-bundle_loader ../../../../src/backend/postgres
-L../../../../src/interfaces/libpq -L../../../../src/port
-L/opt/local/lib -lpq -o walreceiver.so
ld: library not found for -lpq
collect2: ld returned 1 exit status
make[2]: *** [walreceiver.so] Error 1
make[2]: *** Waiting for unfinished jobs....

I suspect that's because libpq isn't built yet. I have this:

all: submake-libpq all-shared-lib

in src/backend/replication/walreceiver/Makefile, but is that not enough?

Yep. What's happening is that "make -j" starts building libpq and
walreceiver.so simultaneously, because of the above line in the
Makefile. We actually have the same problem in src/bin/*/Makefile, but
we don't notice it there because src/interfaces is listed before src/bin
in src/Makefile, so when you do "make -j" at the top-level, libpq is
built first. You get the same error if you do "make clean" at the
top-level, and then e.g "cd src/bin/scripts/; make -j"

So the simple fix would be to reorder the lines in src/Makefile, so that
src/interfaces is built before src/backend. Alternatively we could do this:

*** src/backend/replication/walreceiver/Makefile	15 Jan 2010 09:19:03
-0000	1.1
--- src/backend/replication/walreceiver/Makefile	15 Jan 2010 13:57:24 -0000
***************
*** 18,24 ****
  SHLIB_LINK = $(libpq)
  NAME = walreceiver

! all: submake-libpq all-shared-lib

include $(top_srcdir)/src/Makefile.shlib

--- 18,28 ----
  SHLIB_LINK = $(libpq)
  NAME = walreceiver

! all: all-shared-lib
!
! # Compiling walreceiver.o doesn't really need libpq library,
! # only linking it does. But there's no easy way to specify that.
! walreceiver.o: submake-libpq

include $(top_srcdir)/src/Makefile.shlib

And I guess all the other uses of submake-libpq should be changed similarly.

Am I missing a trick?

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:

Yep. What's happening is that "make -j" starts building libpq and
walreceiver.so simultaneously, because of the above line in the
Makefile. We actually have the same problem in src/bin/*/Makefile, but
we don't notice it there because src/interfaces is listed before src/bin
in src/Makefile, so when you do "make -j" at the top-level, libpq is
built first.

I'm actually fairly uncomfortable with the notion that something buried
deep within the src/backend tree is going to reach over and cause libpq
to get built. Maybe the real answer is that you put walreceiver in the
wrong place, and it ought to be under src/bin/.

regards, tom lane

Tom Lane wrote:

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:

Yep. What's happening is that "make -j" starts building libpq and
walreceiver.so simultaneously, because of the above line in the
Makefile. We actually have the same problem in src/bin/*/Makefile, but
we don't notice it there because src/interfaces is listed before src/bin
in src/Makefile, so when you do "make -j" at the top-level, libpq is
built first.

I'm actually fairly uncomfortable with the notion that something buried
deep within the src/backend tree is going to reach over and cause libpq
to get built. Maybe the real answer is that you put walreceiver in the
wrong place, and it ought to be under src/bin/.

That feels even more wrong to me. Walreceiver is a postmaster
subprocess, tightly integrated with the rest of the backend.

One can argue that it shouldn't be, and walreceiver process should call
libpq through some new API, and the builtin implementation of that API
which uses libpq would be a loadable module that could be in src/bin/ or
contrib. Greg Stark requested that earlier. But I don't want to start
designing such an API at this point.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Fri, Jan 15, 2010 at 11:47 AM, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:

Tom Lane wrote:

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:

Yep. What's happening is that "make -j" starts building libpq and
walreceiver.so simultaneously, because of the above line in the
Makefile. We actually have the same problem in src/bin/*/Makefile, but
we don't notice it there because src/interfaces is listed before src/bin
in src/Makefile, so when you do "make -j" at the top-level, libpq is
built first.

I'm actually fairly uncomfortable with the notion that something buried
deep within the src/backend tree is going to reach over and cause libpq
to get built.  Maybe the real answer is that you put walreceiver in the
wrong place, and it ought to be under src/bin/.

That feels even more wrong to me. Walreceiver is a postmaster
subprocess, tightly integrated with the rest of the backend.

The major problem with having one part of the tree depend on a
completely different part of the tree is that it's easy for the
dependencies to be wrong. If the backend depends on libpq, then it
depends implicitly on all the things on which libpq depends. If
something that libpq depends on, but that the backend does not depend
on directly, gets updated, does the backend get rebuilt? It's easy to
get this wrong. On the other hand, it's also possible to get it
right. If we can decide what we want to happen, I'm willing to take a
crack at it, though if you or Tom or Peter prefer to do it that is
certainly OK with me too.

...Robert

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:

Tom Lane wrote:

I'm actually fairly uncomfortable with the notion that something buried
deep within the src/backend tree is going to reach over and cause libpq
to get built. Maybe the real answer is that you put walreceiver in the
wrong place, and it ought to be under src/bin/.

That feels even more wrong to me. Walreceiver is a postmaster
subprocess, tightly integrated with the rest of the backend.

[ shrug... ] pg_dump, to take one example, is considerably more
"tightly integrated" with the backend than walreceiver is.

regards, tom lane

Robert Haas wrote:

On Fri, Jan 15, 2010 at 11:47 AM, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:

Tom Lane wrote:

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:

Yep. What's happening is that "make -j" starts building libpq and
walreceiver.so simultaneously, because of the above line in the
Makefile. We actually have the same problem in src/bin/*/Makefile, but
we don't notice it there because src/interfaces is listed before src/bin
in src/Makefile, so when you do "make -j" at the top-level, libpq is
built first.

I'm actually fairly uncomfortable with the notion that something buried
deep within the src/backend tree is going to reach over and cause libpq
to get built. Maybe the real answer is that you put walreceiver in the
wrong place, and it ought to be under src/bin/.

That feels even more wrong to me. Walreceiver is a postmaster
subprocess, tightly integrated with the rest of the backend.

The major problem with having one part of the tree depend on a
completely different part of the tree is that it's easy for the
dependencies to be wrong. If the backend depends on libpq, then it
depends implicitly on all the things on which libpq depends. If
something that libpq depends on, but that the backend does not depend
on directly, gets updated, does the backend get rebuilt?

The backend doesn't get rebuilt, and it doesn't need to be. The fact
that walreceiver is a dynamically loaded module should isolate changes
in libpq or its dependencies from affecting the rest of the backend.

I moved the line for src/backend/replication/walreceiver in src/Makefile
further down, after src/interfaces. That should fix the build failures
for now, but I'm all ears if there's better suggestions.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:

I moved the line for src/backend/replication/walreceiver in src/Makefile
further down, after src/interfaces. That should fix the build failures
for now, but I'm all ears if there's better suggestions.

Yeah, I saw. Seems like a reasonable solution for now.

regards, tom lane

On Fri, 2010-01-15 at 05:36 -0500, Bruce Momjian wrote:

Heikki Linnakangas wrote:

I've now committed streaming replication. I moved the files around a
bit, and put the walreceiver/walsender stuff in a new
src/backend/replication subdirectory. There's enough stuff there already
to deserve a new subdirectory, and if we add the capability for
streaming base backups etc. that has been discussed, we will have more
code in there.

But it's not time to party yet. There's still a few loose ends we need
to address:

Documentation. The patch originally moved around some sections, but I
didn't include that in the committed version, to make it clear in the
diff what exactly was added/changed. But I do agree with the original
thought of adding a new "Replication" chapter, and moving all the
replication and standby related stuff there from the "Backup and
Restore" chapter, so let's start working on that.

Uh, do we really want to call this "replication" rather than archive log
streaming or something. It seems "replication" is a generic term and
will confuse people who are using other replication solutions like
Slony.

+1, it is not replication. I would call it something like continuous
archiving or streaming pitr

Joshua D. Drake

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

--
PostgreSQL.org Major Contributor
Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564
Consulting, Training, Support, Custom Development, Engineering
Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.

On Fri, Jan 15, 2010 at 12:17 PM, Joshua D. Drake <jd@commandprompt.com> wrote:

Uh, do we really want to call this "replication" rather than archive log
streaming or something.  It seems "replication" is a generic term and
will confuse people who are using other replication solutions like
Slony.

+1, it is not replication. I would call it something like continuous
archiving or streaming pitr

Of course "PITR" does stand for "point-in-time replication"...

...Robert

Bruce Momjian wrote:

Heikki Linnakangas wrote:

I've now committed streaming replication. I moved the files around a
bit, and put the walreceiver/walsender stuff in a new
src/backend/replication subdirectory. There's enough stuff there already
to deserve a new subdirectory, and if we add the capability for
streaming base backups etc. that has been discussed, we will have more
code in there.

But it's not time to party yet. There's still a few loose ends we need
to address:

Documentation. The patch originally moved around some sections, but I
didn't include that in the committed version, to make it clear in the
diff what exactly was added/changed. But I do agree with the original
thought of adding a new "Replication" chapter, and moving all the
replication and standby related stuff there from the "Backup and
Restore" chapter, so let's start working on that.

Uh, do we really want to call this "replication" rather than archive log
streaming or something. It seems "replication" is a generic term and
will confuse people who are using other replication solutions like
Slony.

Good question. OTOH, if we move the sections about setting up a
file-shipping based standby with pg_standby, that's not streaming.

What we have now is:

Server Administration
...
Backup and Restore
SQL Dump
File System Level Backup
Continuous Archiving and Point-In-Time Recovery (PITR)
Warm Standby Servers for High Availability
Planning
Implementation
Failover
Record-based Log Shipping
Streaming Replication
Incrementally Updated Backups
Hot Standby
User's Overview
Handling query conflicts
Administrator's Overview
Hot Standby Parameter Reference
Caveats
High Availability, Load Balancing, and Replication

I propose:

Server Administration
Backup and Restore
SQL Dump
File System Level Backup
Continuous Archiving and Point-In-Time Recovery (PITR)
High Availability, Load Balancing, and Replication
Introduction
Comparison of different solutions (*)
File-based Log Shipping
Planning
Implementation
Streaming Replication
Setting up
Failover
Hot Standby
User's Overview
Handling query conflicts
Administrator's Overview
Hot Standby Parameter Reference
Caveats
Incrementally Updated Backups

(*) Current content of "High Availability, Load Balancing, and
Replication" chapter goes here

Note that I propose to remove "Record-based Log Shipping" section
altogether. We can briefly mention that method under Streaming
Replication, but I consider that obsolete with streaming replication.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Robert Haas wrote:

On Fri, Jan 15, 2010 at 12:17 PM, Joshua D. Drake <jd@commandprompt.com> wrote:

Uh, do we really want to call this "replication" rather than archive log
streaming or something. It seems "replication" is a generic term and
will confuse people who are using other replication solutions like
Slony.

+1, it is not replication. I would call it something like continuous
archiving or streaming pitr

Of course "PITR" does stand for "point-in-time replication"...

I'm not sure if you're joking, but PITR actually stands for
Point-In-Time *Recovery*.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Fri, 2010-01-15 at 19:30 +0200, Heikki Linnakangas wrote:

Robert Haas wrote:

On Fri, Jan 15, 2010 at 12:17 PM, Joshua D. Drake <jd@commandprompt.com> wrote:

Uh, do we really want to call this "replication" rather than archive log
streaming or something. It seems "replication" is a generic term and
will confuse people who are using other replication solutions like
Slony.

+1, it is not replication. I would call it something like continuous
archiving or streaming pitr

Of course "PITR" does stand for "point-in-time replication"...

I'm not sure if you're joking, but PITR actually stands for
Point-In-Time *Recovery*.

Right.

Joshua D. Drake

--
PostgreSQL.org Major Contributor
Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564
Consulting, Training, Support, Custom Development, Engineering
Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.

On Fri, Jan 15, 2010 at 12:30 PM, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:

Robert Haas wrote:

On Fri, Jan 15, 2010 at 12:17 PM, Joshua D. Drake <jd@commandprompt.com> wrote:

Uh, do we really want to call this "replication" rather than archive log
streaming or something.  It seems "replication" is a generic term and
will confuse people who are using other replication solutions like
Slony.

+1, it is not replication. I would call it something like continuous
archiving or streaming pitr

Of course "PITR" does stand for "point-in-time replication"...

I'm not sure if you're joking, but PITR actually stands for
Point-In-Time *Recovery*.

Oops. No, not joking, just wrong.

But I'm still wondering why this isn't replication.

...Robert

On Jan 15, 2010, at 9:37 AM, Robert Haas wrote:

But I'm still wondering why this isn't replication.

I was wondering the same thing. ISTM that the docs could reference third-party replication solutions, too (or a wiki page listing them, since they'll change often).

Anyway, I think Heikki's proposed chapter name covers it:

High Availability, Load Balancing, and Replication

Works for me.

Best,

David

On Fri, Jan 15, 2010 at 12:11:01PM -0500, Tom Lane wrote:

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:

I moved the line for src/backend/replication/walreceiver in
src/Makefile further down, after src/interfaces. That should fix
the build failures for now, but I'm all ears if there's better
suggestions.

Yeah, I saw. Seems like a reasonable solution for now.

We can always cvs mv...oh, wait! ;)

Cheers,
David.
--
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

+1, it is not replication. I would call it something like continuous
archiving or streaming pitr

I agree we should consider a different name.

But I'm still wondering why this isn't replication.

Because replication is an ambiguous and overloaded term.

On the other hand, it's a great buzzword, so we should use
the phrase as much as possible in the press releases. :)

- --
Greg Sabino Mullane greg@turnstep.com
End Point Corporation http://www.endpoint.com/
PGP Key: 0x14964AC8 201001151257
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAktQrNYACgkQvJuQZxSWSsj8NACgwIQVq/GGQoY/4U6pAYyR5IeS
EsoAoLjoYlLErv4g3Vy65rbA9u9W0vww
=u2Hk
-----END PGP SIGNATURE-----

On Fri, 2010-01-15 at 19:29 +0200, Heikki Linnakangas wrote:

File-based Log Shipping
Planning
Implementation
Streaming Replication
Setting up

How about "Log Streaming Replication"?

So its a particular kind of replication, which seems correct to me.

--
Simon Riggs www.2ndQuadrant.com

On Fri, Jan 15, 2010 at 6:39 PM, Simon Riggs <simon@2ndquadrant.com> wrote:

On Fri, 2010-01-15 at 19:29 +0200, Heikki Linnakangas wrote:
How about "Log Streaming Replication"?

So its a particular kind of replication, which seems correct to me.

I thought the whole point of this effort was to be able to bill it as
a built-in easy replication which was perceived as a major hole in
Postgres's feature set.

--
greg

On Fri, 2010-01-15 at 22:38 +0000, Greg Stark wrote:

On Fri, Jan 15, 2010 at 6:39 PM, Simon Riggs <simon@2ndquadrant.com> wrote:

On Fri, 2010-01-15 at 19:29 +0200, Heikki Linnakangas wrote:
How about "Log Streaming Replication"?

So its a particular kind of replication, which seems correct to me.

I thought the whole point of this effort was to be able to bill it as
a built-in easy replication which was perceived as a major hole in
Postgres's feature set.

How does that affect my proposal?

Heikki called it Streaming Replication; I have just added "Log" to it,
to make clear that it is similar in many ways to File-based Log
Shipping, which was a section heading earlier in docs.

--
Simon Riggs www.2ndQuadrant.com

Heikki Linnakangas wrote:

Uh, do we really want to call this "replication" rather than archive log
streaming or something. It seems "replication" is a generic term and
will confuse people who are using other replication solutions like
Slony.

Good question. OTOH, if we move the sections about setting up a
file-shipping based standby with pg_standby, that's not streaming.

What we have now is:

Frankly, I am concerned we now have a "replication" CVS subdirectory;
it looks more like a 'wal_streaming' directory to me.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Simon Riggs wrote:

On Fri, 2010-01-15 at 19:29 +0200, Heikki Linnakangas wrote:

File-based Log Shipping
Planning
Implementation
Streaming Replication
Setting up

How about "Log Streaming Replication"?

So its a particular kind of replication, which seems correct to me.

Yea, I like that.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +