Seeded Replication

Hi Lou,

Did you restore data on the subscriber during the first time. If yes this
error is likely to occur.

As a remedy, you need to restore only structural dump during initial
building of subscription.

Earlier the same issue was faced by me later it got resolved following the
above said approach.

Kindly revert back if any queries.

Regards,
Pavan Teja,
9841380956

On Wed, 3 Apr, 2019, 5:15 PM Lou Tseng, <ltseng@advancedpricing.com> wrote:

Pavan,

This is the steps I took. I think I did it right:

[Master]

pg_dump -s -f schema.sql
pg_dump -a -f data.sql
CREATE PUBLICATION

[Client/Slave]

psql \i schema.sql
CREATE SUBSCRIPTION ... WITH (enabled = false)
SET session_replication_role = replica; -- turn off trigger
psql \i data.sql
ALTER SUBSCRIPTION ... ENABLE

--> errors occur here

Thoughts?

Lou Tseng

ltseng@advancedpricing.com<mailto:ltseng@advancedpricing.com>
[http://advancedpricing.com/wp-content/uploads/logo_AMPS_email.png]&lt;http://www.advancedpricing.com/&gt;
Advanced Medical Pricing Solutions<http://advancedpricing.com/&gt;
35 Technology Parkway South, Suite. 100
Peachtree Corners, GA 30092

________________________________
From: Pavan Teja <pavan.postgresdba@gmail.com>
Sent: Wednesday, April 3, 2019 6:48 AM
To: Lou Tseng
Cc: pgsql-general@postgresql.org >> PG-General Mailing List
Subject: Re: Seeded Replication

Hi Lou,

Did you restore data on the subscriber during the first time. If yes this error is likely to occur.

As a remedy, you need to restore only structural dump during initial building of subscription.

Earlier the same issue was faced by me later it got resolved following the above said approach.

Kindly revert back if any queries.

Regards,
Pavan Teja,
9841380956

On Wed, 3 Apr, 2019, 5:15 PM Lou Tseng, <ltseng@advancedpricing.com<mailto:ltseng@advancedpricing.com>> wrote:
Hi,

We are working on a project to set up replication from on premises Postgresql 10.4 to Azure Postgresql services. However, our database is about 200G and it will take a long time to complete the initial data copy. We would like to manually seed the subscriber database with data dump and then turn on the subscription like depesz showed in this post https://www.depesz.com/2017/02/07/waiting-for-postgresql-10-logical-replication/ . It works for the small testing database but when I applied same steps with entire database, I am getting "ERROR: duplicate key value violates unique constraint" errors. Basically postgresql does not recognize the primary key.

Any thought / suggestion is helpful!

Lou Tseng

ltseng@advancedpricing.com<mailto:ltseng@advancedpricing.com>
[http://advancedpricing.com/wp-content/uploads/logo_AMPS_email.png]&lt;http://www.advancedpricing.com/&gt;
Advanced Medical Pricing Solutions<http://advancedpricing.com/&gt;
35 Technology Parkway South, Suite. 100
Peachtree Corners, GA 30092

On 4/3/19 4:44 AM, Lou Tseng wrote:

Hi,

We are working on a project to set up replication from on premises
Postgresql 10.4 to Azure Postgresql services. However, our database is
about 200G and it will take a long time to complete the initial data
copy.� We would like to manually seed the subscriber database with data
dump and then turn on the subscription like depesz showed in this post
https://www.depesz.com/2017/02/07/waiting-for-postgresql-10-logical-replication/�.
It works for the small testing database but when I applied same steps
with entire database, I am getting "ERROR: �duplicate key value violates
unique constraint" errors. Basically postgresql does not recognize the
primary key.

Actually it does which why you get the error, which by the way is the
same error depesz got in his demo:)

From your subsequent post:

CREATE SUBSCRIPTION ... WITH (enabled = false)

Did the above also have?:

https://www.postgresql.org/docs/11/sql-createsubscription.html

copy_data = false

If not then you would copying over the data again and that would trigger
the duplicate key error.

Any thought / suggestion is helpful!

Lou Tseng

ltseng@advancedpricing.com <mailto:ltseng@advancedpricing.com>
<http://www.advancedpricing.com/&gt;
Advanced Medical Pricing Solutions <http://advancedpricing.com/&gt;
35 Technology Parkway South, Suite. 100
Peachtree Corners, GA 30092

--
Adrian Klaver
adrian.klaver@aklaver.com

Thank you Adrian. copy_data = false did the trick!

Lou Tseng

ltseng@advancedpricing.com<mailto:ltseng@advancedpricing.com>
[http://advancedpricing.com/wp-content/uploads/logo_AMPS_email.png]&lt;http://www.advancedpricing.com/&gt;
Advanced Medical Pricing Solutions<http://advancedpricing.com/&gt;
35 Technology Parkway South, Suite. 100
Peachtree Corners, GA 30092

________________________________
From: Adrian Klaver <adrian.klaver@aklaver.com>
Sent: Wednesday, April 3, 2019 7:10 PM
To: Lou Tseng; pgsql-general@postgresql.org
Subject: Re: Seeded Replication

On 4/3/19 4:44 AM, Lou Tseng wrote:

Hi,

We are working on a project to set up replication from on premises
Postgresql 10.4 to Azure Postgresql services. However, our database is
about 200G and it will take a long time to complete the initial data
copy. We would like to manually seed the subscriber database with data
dump and then turn on the subscription like depesz showed in this post
https://www.depesz.com/2017/02/07/waiting-for-postgresql-10-logical-replication/ .
It works for the small testing database but when I applied same steps
with entire database, I am getting "ERROR: duplicate key value violates
unique constraint" errors. Basically postgresql does not recognize the
primary key.

Actually it does which why you get the error, which by the way is the
same error depesz got in his demo:)

From your subsequent post:

CREATE SUBSCRIPTION ... WITH (enabled = false)

Did the above also have?:

https://www.postgresql.org/docs/11/sql-createsubscription.html

copy_data = false

If not then you would copying over the data again and that would trigger
the duplicate key error.

Any thought / suggestion is helpful!

Lou Tseng

ltseng@advancedpricing.com <mailto:ltseng@advancedpricing.com>
<http://www.advancedpricing.com/&gt;
Advanced Medical Pricing Solutions <http://advancedpricing.com/&gt;
35 Technology Parkway South, Suite. 100
Peachtree Corners, GA 30092

--
Adrian Klaver
adrian.klaver@aklaver.com

On 2019-04-03 13:44, Lou Tseng wrote:

However, our database is about 200G and it will take a long time to
complete the initial data copy.� We would like to manually seed the
subscriber database with data dump and then turn on the subscription
like depesz showed in this
post�https://www.depesz.com/2017/02/07/waiting-for-postgresql-10-logical-replication/�.

I doubt you will get a huge speedup, since the dump and the initial data
copy use the same COPY command internally.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services