Fine-grained replication?

Started by Paul Jonesalmost 14 years ago7 messagesgeneral
Jump to latest
#1Paul Jones
pbj@cmicdo.com

Is anyone aware of other non-trigger-based, fine-grained replication tools for PostgreSQL along the lines of the XReader <http://wiki.postgresql.org/wiki/XReader&gt; or pgreplay <http://pgreplay.projects.postgresql.org/&gt;?

Thanks,
Paul Jones

#2Sergey Konoplev
sergey.konoplev@postgresql-consulting.com
In reply to: Paul Jones (#1)
Re: Fine-grained replication?

On Tue, Jun 19, 2012 at 1:49 AM, Paul Jones <pbj@cmicdo.com> wrote:

Is anyone aware of other non-trigger-based, fine-grained replication tools for PostgreSQL along the lines of the XReader <http://wiki.postgresql.org/wiki/XReader&gt; or pgreplay <http://pgreplay.projects.postgresql.org/&gt;?

Binary Replication? http://wiki.postgresql.org/wiki/Binary_Replication_Tutorial

Thanks,
Paul Jones

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--
Sergey Konoplev

a database architect, software developer at PostgreSQL-Consulting.com
http://www.postgresql-consulting.com

Jabber: gray.ru@gmail.com Skype: gray-hemp Phone: +79160686204

#3Achilleas Mantzios
achill@smadev.internal.net
In reply to: Sergey Konoplev (#2)
Re: Fine-grained replication?

On Τρι 19 Ιουν 2012 12:00:49 Sergey Konoplev wrote:

On Tue, Jun 19, 2012 at 1:49 AM, Paul Jones <pbj@cmicdo.com> wrote:

Is anyone aware of other non-trigger-based, fine-grained replication
tools for PostgreSQL along the lines of the XReader
<http://wiki.postgresql.org/wiki/XReader&gt; or pgreplay
<http://pgreplay.projects.postgresql.org/&gt;?

Binary Replication?
http://wiki.postgresql.org/wiki/Binary_Replication_Tutorial

Alright, but the OP wrote about "fine-grained". IMHO log-based and fine-grained
seem a liitle but mutually conflicting.

Thanks,
Paul Jones

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

-
Achilleas Mantzios
IT DEPT

#4Guillaume Lelarge
guillaume@lelarge.info
In reply to: Paul Jones (#1)
Re: Fine-grained replication?

On Mon, 18 Jun 2012 14:49:32 -0700 (PDT), Paul Jones <pbj@cmicdo.com>
wrote:

Is anyone aware of other non-trigger-based, fine-grained replication

tools

for PostgreSQL along the lines of the XReader
<http://wiki.postgresql.org/wiki/XReader&gt; or pgreplay
<http://pgreplay.projects.postgresql.org/&gt;?

If you want fine grained replication, it means you won't be able to use
log shipping and streaming replication. If you don't want trigger based
replication, you won't have many options still available. pgPool comes to
mind but, AFAIK, it isn't fine grained (and, to be honest, I would say it
isn't really replication).

So, nope, sorry.

BTW, what's the issue with trigger-based replication? it really helps a
lot in many cases.

--
Guillaume
http://www.postgresql.fr
http://dalibo.com

#5Achilleas Mantzios
achill@smadev.internal.net
In reply to: Guillaume Lelarge (#4)
Re: Fine-grained replication?

On Τρι 19 Ιουν 2012 13:09:28 Guillaume Lelarge wrote:

On Mon, 18 Jun 2012 14:49:32 -0700 (PDT), Paul Jones <pbj@cmicdo.com>

wrote:

Is anyone aware of other non-trigger-based, fine-grained replication

tools

for PostgreSQL along the lines of the XReader
<http://wiki.postgresql.org/wiki/XReader&gt; or pgreplay
<http://pgreplay.projects.postgresql.org/&gt;?

If you want fine grained replication, it means you won't be able to use
log shipping and streaming replication. If you don't want trigger based
replication, you won't have many options still available. pgPool comes to
mind but, AFAIK, it isn't fine grained (and, to be honest, I would say it
isn't really replication).

So, nope, sorry.

BTW, what's the issue with trigger-based replication? it really helps a
lot in many cases.

Totally agree, for us, a heavily modified version of (what initially was) DBmirror
has made the job more than well.
Also, our replication is the most "fine-grained" situation i can think of.
(even rows being lazily replicated based on the FK dependency graph, and many other nice tricks)

-
Achilleas Mantzios
IT DEPT

#6Laurenz Albe
laurenz.albe@cybertec.at
In reply to: Paul Jones (#1)
Re: Fine-grained replication?

Paul Jones wrote:

Is anyone aware of other non-trigger-based, fine-grained replication

tools for PostgreSQL along the

lines of the XReader <http://wiki.postgresql.org/wiki/XReader&gt; or

pgreplay

<http://pgreplay.projects.postgresql.org/&gt;?

I'm not sure what you want, because "pgreplay" is certainly not a
replication
tool, unless "replication" means something quite different to you than
it
does to me.

Can you be more specific?

Yours,
Laurenz Albe

#7Paul Jones
pbj@cmicdo.com
In reply to: Sergey Konoplev (#2)
Re: Fine-grained replication?

I should have been more specific about why I'm interested in this.

The idea is not so much the replication but being able to use the
replication information to make business decisions based on what is
observed happening to the data, or being able to selectively repair
data if there are problems.

XReader certainly shows a lot of promise, but it's not quite here yet.

Specific answers to respondents below...

From: Sergey Konoplev <sergey.konoplev@postgresql-consulting.com>
Date: Tue, 19 Jun 2012 13:00:49 +0400

On Tue, Jun 19, 2012 at 1:49 AM, Paul Jones <pbj@cmicdo.com> wrote:

Is anyone aware of other non-trigger-based, fine-grained replication tools for PostgreSQL along the lines of the XReader <http://wiki.postgresql.org/wiki/XReader&gt; or pgreplay <http://pgreplay.projects.postgresql.org/&gt;?

Binary Replication? http://wiki.postgresql.org/wiki/Binary_Replication_Tutorial
Sergey Konoplev

Thanks!  This is good info.

On Tue, Jun 19, 2012 at 12:34:02PM +0200, Albe Laurenz wrote:

I'm not sure what you want, because "pgreplay" is certainly not a
replication
tool, unless "replication" means something quite different to you than
it
does to me.

Can you be more specific?

Yes, as explained above.  So pgreplay in this context would be more likely
to be used to repair something.

Date: Tue, 19 Jun 2012 12:09:28 +0200
From: Guillaume Lelarge <guillaume@lelarge.info>

If you want fine grained replication, it means you won't be able to use
log shipping and streaming replication. If you don't want trigger based
replication, you won't have many options still available. pgPool comes to
mind but, AFAIK, it isn't fine grained (and, to be honest, I would say it
isn't really replication).

So, nope, sorry.

BTW, what's the issue with trigger-based replication? it really helps a
lot in many cases.

The worry with trigger-based replication is that it could become a
performance bottleneck for heavy loads.  We tried out Slony just to see
how it worked and it did its job just fine for what we did with it.

Paul Jones