BDR Error restarted

Started by deansover 11 years ago14 messagesgeneral

Jump to latest

deans

dean.song888@gmail.com

over 11 years ago

Hi Guys.

First of all, BDR is cool, should have tried it earlier.

Environment: CentOS 6.5, PostgreSQL 9.4.0 with BDR from yum repository

Done the PostgreSQL 9.4 with BDR setup successfully by following the User
Guide and Admin Doc, but got a issue when tried to do postgresql service
restart on any of the cluster nodes, it always lost items in the relation
"pg_stat_replication" after the service restarted, means lost replication
connections, like normally we have 26 items(13 databases and 3 servers in
cluster, so 13x2 items in pg_stat_replication on each node, yes, lots dbs on
one cluster), but after the restart, got some random count of item in the
relation, like 20, or 16 or 4, but not 26 as expected, i.e. not all the
replication connections running well. And the logs showing replication is
"wait until the node has caught up", but they never catch up again.

BDR Settings(replaced the real db name here):
1. on node 01, the replication src one:

2. on node 02, mostly like 03

Error logs:

Detail on the pgsql relation said above:

Thanks for any help/suggestion on this!
-Dean

--
View this message in context: http://postgresql.nabble.com/BDR-Error-restarted-tp5833139.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Craig Ringer

craig@2ndquadrant.com

over 11 years ago

In reply to: deans (#1)

Re: BDR Error restarted

On 01/07/2015 11:28 AM, deans wrote:

Hi Guys.

First of all, BDR is cool, should have tried it earlier.

Environment: CentOS 6.5, PostgreSQL 9.4.0 with BDR from yum repository

Done the PostgreSQL 9.4 with BDR setup successfully by following the User
Guide and Admin Doc, but got a issue when tried to do postgresql service
restart on any of the cluster nodes, it always lost items in the relation
"pg_stat_replication" after the service restarted

It'd be weird if it didn't. That's not a table, it's a view over a
function that reports server status information.

It's like pg_stat_activity for replication.

How exactly do you restart the node?

means lost replication
connections, like normally we have 26 items(13 databases and 3 servers in
cluster, so 13x2 items in pg_stat_replication on each node, yes, lots dbs on
one cluster), but after the restart, got some random count of item in the
relation, like 20, or 16 or 4, but not 26 as expected, i.e. not all the
replication connections running well.

That's normal, in that it can take time for peers to notice and
re-connect. This is true whether it's BDR, other logical replication, or
conventional streaming replication.

And the logs showing replication is
"wait until the node has caught up", but they never catch up again.

Please show the full log line, not just an excerpt, along with the lines
around it.

If possible run with log_error_verbosity=verbose and show the extra
line(s) printed after each log line too, please.

BDR Settings(replaced the real db name here):
1. on node 01, the replication src one:

2. on node 02, mostly like 03

You forgot these.

Error logs:

and these.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

deans

dean.song888@gmail.com

over 11 years ago

In reply to: Craig Ringer (#2)

Re: BDR Error restarted

Thanks Craig.

How exactly do you restart the node?

Using "service postgresql-9.4 restart" command to do the service restart.

Please show the full log line, not just an excerpt, along with the lines around

it.

If possible run with log_error_verbosity=verbose and show the extra line(s)

printed after each log line too, please.

See the full logs during the restart process on one node:
http://tny.cz/e1831e2a

Thanks,
Dean

On Wed, Jan 7, 2015 at 12:56 PM, Craig Ringer-3 [via PostgreSQL] <
ml-node+s1045698n5833173h75@n5.nabble.com> wrote:

On 01/07/2015 11:28 AM, deans wrote:

Hi Guys.

First of all, BDR is cool, should have tried it earlier.

Environment: CentOS 6.5, PostgreSQL 9.4.0 with BDR from yum repository

Done the PostgreSQL 9.4 with BDR setup successfully by following the

User

Guide and Admin Doc, but got a issue when tried to do postgresql service
restart on any of the cluster nodes, it always lost items in the

relation

"pg_stat_replication" after the service restarted

It'd be weird if it didn't. That's not a table, it's a view over a
function that reports server status information.

It's like pg_stat_activity for replication.

How exactly do you restart the node?

means lost replication
connections, like normally we have 26 items(13 databases and 3 servers

in

cluster, so 13x2 items in pg_stat_replication on each node, yes, lots

dbs on

one cluster), but after the restart, got some random count of item in

the

relation, like 20, or 16 or 4, but not 26 as expected, i.e. not all the
replication connections running well.

That's normal, in that it can take time for peers to notice and
re-connect. This is true whether it's BDR, other logical replication, or
conventional streaming replication.

And the logs showing replication is
"wait until the node has caught up", but they never catch up again.

Please show the full log line, not just an excerpt, along with the lines
around it.

If possible run with log_error_verbosity=verbose and show the extra
line(s) printed after each log line too, please.

BDR Settings(replaced the real db name here):
1. on node 01, the replication src one:

2. on node 02, mostly like 03

You forgot these.

Error logs:

and these.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-general mailing list ([hidden email]
<http:///user/SendEmail.jtp?type=node&node=5833173&i=0>)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

------------------------------
If you reply to this email, your message will be added to the discussion
below:
http://postgresql.nabble.com/BDR-Error-restarted-tp5833139p5833173.html
To unsubscribe from BDR Error restarted, click here
<http://postgresql.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5833139&code=ZGVhbi5zb25nODg4QGdtYWlsLmNvbXw1ODMzMTM5fDE4MDgxODY4ODg=>
.
NAML
<http://postgresql.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>

--
Thanks,
Dean.song

--
View this message in context: http://postgresql.nabble.com/BDR-Error-restarted-tp5833139p5833298.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

deans

dean.song888@gmail.com

over 11 years ago

In reply to: deans (#3)

Re: BDR Error restarted

Hi Craig, any insight on this issue? :)

Thanks,
Dean

--
View this message in context: http://postgresql.nabble.com/BDR-Error-restarted-tp5833139p5833722.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

agent

zipho.m@gmail.com

about 11 years ago

In reply to: deans (#4)

Re: BDR Error restarted

Hi Dean

Did you manage do get any help. I am having the exact issue at the moment.

--
View this message in context: http://postgresql.nabble.com/BDR-Error-restarted-tp5833139p5834722.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

deans

dean.song888@gmail.com

about 11 years ago

In reply to: agent (#5)

Re: BDR Error restarted

:( not yet, still having the issue.

--
View this message in context: http://postgresql.nabble.com/BDR-Error-restarted-tp5833139p5834821.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Andres Freund

andres@anarazel.de

about 11 years ago

In reply to: deans (#4)

Re: BDR Error restarted

Hi,

On 2015-01-13 02:43:26 -0700, deans wrote:

Hi Craig, any insight on this issue? :)

You've, afaics, not posted your configuration. Could you do so?

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

deans

dean.song888@gmail.com

about 11 years ago

In reply to: Andres Freund (#7)

Re: BDR Error restarted

Hi Andres,

Any other conf should I list in addition to the bdr settings below?

BDR Settings(replaced the real db name here):

1. on node 01, the replication src one:
# Generic settings required for BDR
max_replication_slots = 60
max_wal_senders = 60
wal_level = 'logical'
track_commit_timestamp = on
shared_preload_libraries = 'bdr'
max_worker_processes = 100

# These aren't required, but are useful for diagnosing problems
#log_error_verbosity = verbose
#log_min_messages = debug1
#log_line_prefix = 'd=%d p=%p a=%a%q '

# Useful options for playing with conflicts
#bdr.default_apply_delay=2000 # milliseconds
#bdr.log_conflicts_to_table=on

bdr.connections =
'bdr02db1,bdr02db2,bdr02db3,bdr02db4,bdr02db5,bdr02db6,bdr02db7,bdr02db8,bdr02db9,bdr02db10,bdr02db11,bdr02db12,bdr02db13,bdr03db1,bdr03db2,bdr03db3,bdr03db4,bdr03db5,bdr03db6,bdr03db7,bdr03db8,bdr03db9,bdr03db10,bdr03db11,bdr03db12,bdr03db13'

bdr.bdr02db1_dsn = 'dbname=db1 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db2_dsn = 'dbname=db2 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db3_dsn = 'dbname=db3 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db4_dsn = 'dbname=db4 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db5_dsn = 'dbname=db5 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db6_dsn = 'dbname=db6 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db7_dsn = 'dbname=db7 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db8_dsn = 'dbname=db8 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db8_dsn = 'dbname=db9 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db10_dsn = 'dbname=db10 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db11_dsn = 'dbname=db11 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db12_dsn = 'dbname=db12 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db13_dsn = 'dbname=db13 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr03db1_dsn = 'dbname=db1 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db2_dsn = 'dbname=db2 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db3_dsn = 'dbname=db3 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db4_dsn = 'dbname=db4 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db5_dsn = 'dbname=db5 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db6_dsn = 'dbname=db6 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db7_dsn = 'dbname=db7 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db8_dsn = 'dbname=db8 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db9_dsn = 'dbname=db9 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db10_dsn = 'dbname=db10 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db11_dsn = 'dbname=db11 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db12_dsn = 'dbname=db12 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db13_dsn = 'dbname=db13 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'

2. on node 02
# Generic settings required for BDR
max_replication_slots = 60
max_wal_senders = 60
wal_level = 'logical'
track_commit_timestamp = on
shared_preload_libraries = 'bdr'
max_worker_processes = 100

# These aren't required, but are useful for diagnosing problems
#log_error_verbosity = verbose
#log_min_messages = debug1
#log_line_prefix = 'd=%d p=%p a=%a%q '

# Useful options for playing with conflicts
#bdr.default_apply_delay=2000 # milliseconds
#bdr.log_conflicts_to_table=on

bdr.connections =
'bdr01db1,bdr01db2,bdr01db3,bdr01db4,bdr01db5,bdr01db6,bdr01db7,bdr01db8,bdr01db9,bdr01db10,bdr01db11,bdr01db12,bdr01db13,bdr03db1,bdr03db2,bdr03db3,bdr03db4,bdr03db5,bdr03db6,bdr03db7,bdr03db8,bdr03db9,bdr03db10,bdr03db11,bdr03db12,bdr03db13'

bdr.bdr01db1_dsn = 'dbname=db1 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db2_dsn = 'dbname=db2 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db3_dsn = 'dbname=db3 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db4_dsn = 'dbname=db4 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db5_dsn = 'dbname=db5 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db6_dsn = 'dbname=db6 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db7_dsn = 'dbname=db7 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db8_dsn = 'dbname=db8 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db8_dsn = 'dbname=db9 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db10_dsn = 'dbname=db10 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db11_dsn = 'dbname=db11 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db12_dsn = 'dbname=db12 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db13_dsn = 'dbname=db13 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db1_dsn = 'dbname=db1 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db2_dsn = 'dbname=db2 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db3_dsn = 'dbname=db3 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db4_dsn = 'dbname=db4 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db5_dsn = 'dbname=db5 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db6_dsn = 'dbname=db6 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db7_dsn = 'dbname=db7 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db8_dsn = 'dbname=db8 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db9_dsn = 'dbname=db9 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db10_dsn = 'dbname=db10 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db11_dsn = 'dbname=db11 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db12_dsn = 'dbname=db12 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db13_dsn = 'dbname=db13 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'

bdr.bdr01db1_init_replica = on
bdr.bbdr01db1_replica_local_dsn = 'dbname=db1 user=postgres'
bdr.bdr01db2_init_replica = on
bdr.bbdr01db2_replica_local_dsn = 'dbname=db2 user=postgres'
bdr.bdr01db3_init_replica = on
bdr.bbdr01db3_replica_local_dsn = 'dbname=db3 user=postgres'
bdr.bdr01db4_init_replica = on
bdr.bbdr01db4_replica_local_dsn = 'dbname=db4 user=postgres'
bdr.bdr01db5_init_replica = on
bdr.bbdr01db5_replica_local_dsn = 'dbname=db5 user=postgres'
bdr.bdr01db6_init_replica = on
bdr.bbdr01db6_replica_local_dsn = 'dbname=db6 user=postgres'
bdr.bdr01db7_init_replica = on
bdr.bbdr01db7_replica_local_dsn = 'dbname=db7 user=postgres'
bdr.bdr01db8_init_replica = on
bdr.bbdr01db8_replica_local_dsn = 'dbname=db8 user=postgres'
bdr.bdr01db9_init_replica = on
bdr.bbdr01db9_replica_local_dsn = 'dbname=db9 user=postgres'
bdr.bdr01db10_init_replica = on
bdr.bbdr01db10_replica_local_dsn = 'dbname=db10 user=postgres'
bdr.bdr01db11_init_replica = on
bdr.bbdr01db11_replica_local_dsn = 'dbname=db11 user=postgres'
bdr.bdr01db12_init_replica = on
bdr.bbdr01db12_replica_local_dsn = 'dbname=db12 user=postgres'
bdr.bdr01db13_init_replica = on
bdr.bbdr01db13_replica_local_dsn = 'dbname=db13 user=postgres'

--
View this message in context: http://postgresql.nabble.com/BDR-Error-restarted-tp5833139p5834859.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Craig Ringer

craig@2ndquadrant.com

about 11 years ago

In reply to: deans (#8)

Re: BDR Error restarted

That's a lot of databases and explains why the logs were so busy.

For the purpose of testing it would be interesting to simplify this case
down, if you can, to the minimum required to reproduce the issue. It's
awfully hard to keep track of what's going on with this many concurrent
operations - especially since at the moment background workers don't
respect the %d format parameter in log_line_prefix .

The logs of the same time on the node you didn't restart would be useful
too, i.e. a matching pair.

The ideal is to determine what sequence of actions creates this situation
so we can reproduce what you're seeing.

On 21 January 2015 at 19:33, deans <dean.song888@gmail.com> wrote:

Hi Andres,

Any other conf should I list in addition to the bdr settings below?

BDR Settings(replaced the real db name here):

1. on node 01, the replication src one:
# Generic settings required for BDR
max_replication_slots = 60
max_wal_senders = 60
wal_level = 'logical'
track_commit_timestamp = on
shared_preload_libraries = 'bdr'
max_worker_processes = 100

# These aren't required, but are useful for diagnosing problems
#log_error_verbosity = verbose
#log_min_messages = debug1
#log_line_prefix = 'd=%d p=%p a=%a%q '

# Useful options for playing with conflicts
#bdr.default_apply_delay=2000 # milliseconds
#bdr.log_conflicts_to_table=on

bdr.connections =

'bdr02db1,bdr02db2,bdr02db3,bdr02db4,bdr02db5,bdr02db6,bdr02db7,bdr02db8,bdr02db9,bdr02db10,bdr02db11,bdr02db12,bdr02db13,bdr03db1,bdr03db2,bdr03db3,bdr03db4,bdr03db5,bdr03db6,bdr03db7,bdr03db8,bdr03db9,bdr03db10,bdr03db11,bdr03db12,bdr03db13'

bdr.bdr02db1_dsn = 'dbname=db1 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db2_dsn = 'dbname=db2 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db3_dsn = 'dbname=db3 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db4_dsn = 'dbname=db4 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db5_dsn = 'dbname=db5 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db6_dsn = 'dbname=db6 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db7_dsn = 'dbname=db7 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db8_dsn = 'dbname=db8 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db8_dsn = 'dbname=db9 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db10_dsn = 'dbname=db10 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db11_dsn = 'dbname=db11 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db12_dsn = 'dbname=db12 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr02db13_dsn = 'dbname=db13 host=pgsql-bdr-cluster-02.ol.xxx.net
user=postgres'
bdr.bdr03db1_dsn = 'dbname=db1 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db2_dsn = 'dbname=db2 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db3_dsn = 'dbname=db3 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db4_dsn = 'dbname=db4 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db5_dsn = 'dbname=db5 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db6_dsn = 'dbname=db6 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db7_dsn = 'dbname=db7 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db8_dsn = 'dbname=db8 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db9_dsn = 'dbname=db9 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db10_dsn = 'dbname=db10 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db11_dsn = 'dbname=db11 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db12_dsn = 'dbname=db12 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db13_dsn = 'dbname=db13 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'

2. on node 02
# Generic settings required for BDR
max_replication_slots = 60
max_wal_senders = 60
wal_level = 'logical'
track_commit_timestamp = on
shared_preload_libraries = 'bdr'
max_worker_processes = 100

# These aren't required, but are useful for diagnosing problems
#log_error_verbosity = verbose
#log_min_messages = debug1
#log_line_prefix = 'd=%d p=%p a=%a%q '

# Useful options for playing with conflicts
#bdr.default_apply_delay=2000 # milliseconds
#bdr.log_conflicts_to_table=on

bdr.connections =

'bdr01db1,bdr01db2,bdr01db3,bdr01db4,bdr01db5,bdr01db6,bdr01db7,bdr01db8,bdr01db9,bdr01db10,bdr01db11,bdr01db12,bdr01db13,bdr03db1,bdr03db2,bdr03db3,bdr03db4,bdr03db5,bdr03db6,bdr03db7,bdr03db8,bdr03db9,bdr03db10,bdr03db11,bdr03db12,bdr03db13'

bdr.bdr01db1_dsn = 'dbname=db1 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db2_dsn = 'dbname=db2 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db3_dsn = 'dbname=db3 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db4_dsn = 'dbname=db4 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db5_dsn = 'dbname=db5 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db6_dsn = 'dbname=db6 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db7_dsn = 'dbname=db7 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db8_dsn = 'dbname=db8 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db8_dsn = 'dbname=db9 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db10_dsn = 'dbname=db10 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db11_dsn = 'dbname=db11 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db12_dsn = 'dbname=db12 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db13_dsn = 'dbname=db13 host=pgsql-bdr-cluster-01.ol.xxx.net
user=postgres'
bdr.bdr01db1_dsn = 'dbname=db1 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db2_dsn = 'dbname=db2 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db3_dsn = 'dbname=db3 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db4_dsn = 'dbname=db4 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db5_dsn = 'dbname=db5 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db6_dsn = 'dbname=db6 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db7_dsn = 'dbname=db7 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db8_dsn = 'dbname=db8 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db9_dsn = 'dbname=db9 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db10_dsn = 'dbname=db10 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db11_dsn = 'dbname=db11 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db12_dsn = 'dbname=db12 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'
bdr.bdr03db13_dsn = 'dbname=db13 host=pgsql-bdr-cluster-03.ol.xxx.net
user=postgres'

bdr.bdr01db1_init_replica = on
bdr.bbdr01db1_replica_local_dsn = 'dbname=db1 user=postgres'
bdr.bdr01db2_init_replica = on
bdr.bbdr01db2_replica_local_dsn = 'dbname=db2 user=postgres'
bdr.bdr01db3_init_replica = on
bdr.bbdr01db3_replica_local_dsn = 'dbname=db3 user=postgres'
bdr.bdr01db4_init_replica = on
bdr.bbdr01db4_replica_local_dsn = 'dbname=db4 user=postgres'
bdr.bdr01db5_init_replica = on
bdr.bbdr01db5_replica_local_dsn = 'dbname=db5 user=postgres'
bdr.bdr01db6_init_replica = on
bdr.bbdr01db6_replica_local_dsn = 'dbname=db6 user=postgres'
bdr.bdr01db7_init_replica = on
bdr.bbdr01db7_replica_local_dsn = 'dbname=db7 user=postgres'
bdr.bdr01db8_init_replica = on
bdr.bbdr01db8_replica_local_dsn = 'dbname=db8 user=postgres'
bdr.bdr01db9_init_replica = on
bdr.bbdr01db9_replica_local_dsn = 'dbname=db9 user=postgres'
bdr.bdr01db10_init_replica = on
bdr.bbdr01db10_replica_local_dsn = 'dbname=db10 user=postgres'
bdr.bdr01db11_init_replica = on
bdr.bbdr01db11_replica_local_dsn = 'dbname=db11 user=postgres'
bdr.bdr01db12_init_replica = on
bdr.bbdr01db12_replica_local_dsn = 'dbname=db12 user=postgres'
bdr.bdr01db13_init_replica = on
bdr.bbdr01db13_replica_local_dsn = 'dbname=db13 user=postgres'

--
View this message in context:
http://postgresql.nabble.com/BDR-Error-restarted-tp5833139p5834859.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

#10

agent

zipho.m@gmail.com

about 11 years ago

In reply to: Craig Ringer (#9)

Re: BDR Error restarted

Hi CraigI have a similar issue with a rather small number of servers
involved.*Node 1:*max_replication_slots = 4max_wal_senders = 6wal_level =
'logical'track_commit_timestamp = onshared_preload_libraries =
'bdr'max_worker_processes = 10log_error_verbosity = verboselog_min_messages
= debug1log_line_prefix = 'd=%d p=%p a=%a%q 'bdr.default_apply_delay =
2000bdr.log_conflicts_to_table = onbdr.connections =
'bdrnode3,bdrnode3'bdr.bdrnode2_dsn = 'dbname=db user=postgres host=host2
port=5599'bdr.bdrnode3_dsn = 'dbname=db user=postgres host=host3
port=5599'*Node 2:*max_replication_slots = 4max_wal_senders = 6wal_level =
'logical'track_commit_timestamp = onshared_preload_libraries =
'bdr'max_worker_processes = 10log_error_verbosity = verboselog_min_messages
= debug1log_line_prefix = 'd=%d p=%p a=%a%q 'bdr.default_apply_delay =
2000bdr.log_conflicts_to_table = onbdr.connections =
'bdrnode1,bdrnode3'bdr.bdrnode1_dsn = 'dbname=db user=postgres host=host1
port=5599'bdr.bdrnode3_dsn = 'dbname=db user=postgres host=host3
port=5599'bdr.bdrnode1_init_replica = onbdr.bdrnode1_replica_local_dsn =
'dbname=db user=postgres host=localhost port=5599'*Node
3:*max_replication_slots = 4max_wal_senders = 6wal_level =
'logical'track_commit_timestamp = onshared_preload_libraries =
'bdr'max_worker_processes = 10log_error_verbosity = verboselog_min_messages
= debug1log_line_prefix = 'd=%d p=%p a=%a%q 'bdr.default_apply_delay =
2000bdr.log_conflicts_to_table = onbdr.connections =
'bdrnode1,bdrnode2'bdr.bdrnode1_dsn = 'dbname=db user=postgres host=host1
port=5599'bdr.bdrnode2_dsn = 'dbname=db user=postgres host=host2
port=5599'bdr.bdrnode1_init_replica = onbdr.bdrnode1_replica_local_dsn =
'dbname=db user=postgres host=localhost port=5599'The above setup is an
attempt for a "3-remote site simple Multi-Master Plex" configuration.Node1
and Node2 are replicating, but Node3 does not work, and the following data
is is from Node3 logs:d=db_name p=3173 a=bdr (6106457559585933042,1,16398,):
receive CONTEXT: slot "bdr_16396_6106457559585933042_1_16398__", output
plugin "bdr", in the startup callbackd=db_name p=3173 a=bdr
(6106457559585933042,1,16398,): receive LOCATION: pg_decode_startup,
bdr_output.c:450d=db_name p=3173 a=bdr (6106457559585933042,1,16398,):
receive ERROR: 55000: bdr output plugin: slot creation rejected,
bdr.bdr_nodes entry for local node (sysid=6107134089288229725, timelineid=1,
dboid=16396): status='i', bdr still starting up: applying initial dump of
remote noded=db_name p=3173 a=bdr (6106457559585933042,1,16398,): receive
HINT: Monitor pg_stat_activity and the logs, wait until the node has caught
upd=db_name p=3173 a=bdr (6106457559585933042,1,16398,): receive CONTEXT:
slot "bdr_16396_6106457559585933042_1_16398__", output plugin "bdr", in the
startup callbackd=db_name p=3173 a=bdr (6106457559585933042,1,16398,):
receive LOCATION: bdr_ensure_node_ready, bdr_output.c:268d=db_name p=3173
a=bdr (6106457559585933042,1,16398,): receive LOG: 08006: could not receive
data from client: Connection reset by peerd=db_name p=3173 a=bdr
(6106457559585933042,1,16398,): receive LOCATION: pq_recvbuf,
pqcomm.c:869d=db_name p=3173 a=bdr (6106457559585933042,1,16398,): receive
DEBUG: 08003: unexpected EOF on client connectiond=db_name p=3173 a=bdr
(6106457559585933042,1,16398,): receive LOCATION: SocketBackend,
postgres.c:353d=db_name p=3175 a=bdr (6106458869483394081,1,16396,): receive
DEBUG: 00000: received replication command: IDENTIFY_SYSTEMd=db_name p=3175
a=bdr (6106458869483394081,1,16396,): receive LOCATION:
exec_replication_command, walsender.c:1292d=db_name p=3175 a=bdr
(6106458869483394081,1,16396,): receive DEBUG: 00000: received replication
command: START_REPLICATION SLOT "bdr_16396_6106458869483394081_1_16396__"
LOGICAL 0/0 (pg_version '90400', pg_catversion '201408161', bdr_version
'701', min_bdr_version '700', sizeof_int '4', sizeof_long '8', sizeof_datum
'8', maxalign '8', float4_byval '1', float8_byval '1', integer_datetimes
'1', bigendian '0', db_encoding 'UTF8')d=db_name p=3175 a=bdr
(6106458869483394081,1,16396,): receive LOCATION: exec_replication_command,
walsender.c:1292d=db_name p=3175 a=bdr (6106458869483394081,1,16396,):
receive DEBUG: 00000: bdr.bdr_conflict_handlers OID set to 16991d=db_name
p=3175 a=bdr (6106458869483394081,1,16396,): receive CONTEXT: slot
"bdr_16396_6106458869483394081_1_16396__", output plugin "bdr", in the
startup callbackd=db_name p=3175 a=bdr (6106458869483394081,1,16396,):
receive LOCATION: pg_decode_startup, bdr_output.c:450d=db_name p=3175 a=bdr
(6106458869483394081,1,16396,): receive ERROR: 55000: bdr output plugin:
slot creation rejected, bdr.bdr_nodes entry for local node
(sysid=6107134089288229725, timelineid=1, dboid=16396): status='i', bdr
still starting up: applying initial dump of remote noded=db_name p=3175
a=bdr (6106458869483394081,1,16396,): receive HINT: Monitor
pg_stat_activity and the logs, wait until the node has caught upd=db_name
p=3175 a=bdr (6106458869483394081,1,16396,): receive CONTEXT: slot
"bdr_16396_6106458869483394081_1_16396__", output plugin "bdr", in the
startup callbackd=db_name p=3175 a=bdr (6106458869483394081,1,16396,):
receive LOCATION: bdr_ensure_node_ready, bdr_output.c:268d=db_name p=3175
a=bdr (6106458869483394081,1,16396,): receive LOG: 08006: could not receive
data from client: Connection reset by peerd=db_name p=3175 a=bdr
(6106458869483394081,1,16396,): receive LOCATION: pq_recvbuf,
pqcomm.c:869d=db_name p=3175 a=bdr (6106458869483394081,1,16396,): receive
DEBUG: 08003: unexpected EOF on client connectiond=db_name p=3175 a=bdr
(6106458869483394081,1,16396,): receive LOCATION: SocketBackend,
postgres.c:353d=db_name p=3189 a=bdr (6106457559585933042,1,16398,): receive
DEBUG: 00000: received replication command: IDENTIFY_SYSTEMd=db_name p=3189
a=bdr (6106457559585933042,1,16398,): receive LOCATION:
exec_replication_command, walsender.c:1292d=db_name p=3189 a=bdr
(6106457559585933042,1,16398,): receive DEBUG: 00000: received replication
command: START_REPLICATION SLOT "bdr_16396_6106457559585933042_1_16398__"
LOGICAL 0/0 (pg_version '90400', pg_catversion '201408161', bdr_version
'701', min_bdr_version '700', sizeof_int '4', sizeof_long '8', sizeof_datum
'8', maxalign '8', float4_byval '1', float8_byval '1', integer_datetimes
'1', bigendian '0', db_encoding 'UTF8')d=db_name p=3189 a=bdr
(6106457559585933042,1,16398,): receive LOCATION: exec_replication_command,
walsender.c:1292d=db_name p=3189 a=bdr (6106457559585933042,1,16398,):
receive DEBUG: 00000: bdr.bdr_conflict_handlers OID set to 16991d=db_name
p=3189 a=bdr (6106457559585933042,1,16398,): receive CONTEXT: slot
"bdr_16396_6106457559585933042_1_16398__", output plugin "bdr", in the
startup callbackd=db_name p=3189 a=bdr (6106457559585933042,1,16398,):
receive LOCATION: pg_decode_startup, bdr_output.c:450d=db_name p=3189 a=bdr
(6106457559585933042,1,16398,): receive ERROR: 55000: bdr output plugin:
slot creation rejected, bdr.bdr_nodes entry for local node
(sysid=6107134089288229725, timelineid=1, dboid=16396): status='i', bdr
still starting up: applying initial dump of remote noded=db_name p=3189
a=bdr (6106457559585933042,1,16398,): receive HINT: Monitor
pg_stat_activity and the logs, wait until the node has caught upd=db_name
p=3189 a=bdr (6106457559585933042,1,16398,): receive CONTEXT: slot
"bdr_16396_6106457559585933042_1_16398__", output plugin "bdr", in the
startup callbackd=db_name p=3189 a=bdr (6106457559585933042,1,16398,):
receive LOCATION: bdr_ensure_node_ready, bdr_output.c:268d=db_name p=3189
a=bdr (6106457559585933042,1,16398,): receive LOG: 08006: could not receive
data from client: Connection reset by peerd=db_name p=3189 a=bdr
(6106457559585933042,1,16398,): receive LOCATION: pq_recvbuf,
pqcomm.c:869d=db_name p=3189 a=bdr (6106457559585933042,1,16398,): receive
DEBUG: 08003: unexpected EOF on client connectiond=db_name p=3189 a=bdr
(6106457559585933042,1,16398,): receive LOCATION: SocketBackend,
postgres.c:353d= p=3190 a=DEBUG: 00000: autovacuum: processing database
"db_name"d= p=3190 a=LOCATION: AutoVacWorkerMain,
autovacuum.c:1670d=db_name p=3191 a=bdr (6106458869483394081,1,16396,):
receive DEBUG: 00000: received replication command:
IDENTIFY_SYSTEMd=db_name p=3191 a=bdr (6106458869483394081,1,16396,):
receive LOCATION: exec_replication_command, walsender.c:1292d=db_name
p=3191 a=bdr (6106458869483394081,1,16396,): receive DEBUG: 00000: received
replication command: START_REPLICATION SLOT
"bdr_16396_6106458869483394081_1_16396__" LOGICAL 0/0 (pg_version '90400',
pg_catversion '201408161', bdr_version '701', min_bdr_version '700',
sizeof_int '4', sizeof_long '8', sizeof_datum '8', maxalign '8',
float4_byval '1', float8_byval '1', integer_datetimes '1', bigendian '0',
db_encoding 'UTF8')d=db_name p=3191 a=bdr (6106458869483394081,1,16396,):
receive LOCATION: exec_replication_command, walsender.c:1292d=db_name
p=3191 a=bdr (6106458869483394081,1,16396,): receive DEBUG: 00000:
bdr.bdr_conflict_handlers OID set to 16991d=db_name p=3191 a=bdr
(6106458869483394081,1,16396,): receive CONTEXT: slot
"bdr_16396_6106458869483394081_1_16396__", output plugin "bdr", in the
startup callbackd=db_name p=3191 a=bdr (6106458869483394081,1,16396,):
receive LOCATION: pg_decode_startup, bdr_output.c:450d=db_name p=3191 a=bdr
(6106458869483394081,1,16396,): receive ERROR: 55000: bdr output plugin:
slot creation rejected, bdr.bdr_nodes entry for local node
(sysid=6107134089288229725, timelineid=1, dboid=16396): status='i', bdr
still starting up: applying initial dump of remote noded=db_name p=3191
a=bdr (6106458869483394081,1,16396,): receive HINT: Monitor
pg_stat_activity and the logs, wait until the node has caught upd=db_name
p=3191 a=bdr (6106458869483394081,1,16396,): receive CONTEXT: slot
"bdr_16396_6106458869483394081_1_16396__", output plugin "bdr", in the
startup callbackd=db_name p=3191 a=bdr (6106458869483394081,1,16396,):
receive LOCATION: bdr_ensure_node_ready, bdr_output.c:268d=db_name p=3191
a=bdr (6106458869483394081,1,16396,): receive LOG: 08006: could not receive
data from client: Connection reset by peerd=db_name p=3191 a=bdr
(6106458869483394081,1,16396,): receive LOCATION: pq_recvbuf,
pqcomm.c:869d=db_name p=3191 a=bdr (6106458869483394081,1,16396,): receive
DEBUG: 08003: unexpected EOF on client connectiond=db_name p=3191 a=bdr
(6106458869483394081,1,16396,): receive LOCATION: SocketBackend,
postgres.c:353d=db_name p=3199 a=bdr (6106457559585933042,1,16398,): receive
DEBUG: 00000: received replication command: IDENTIFY_SYSTEMd=db_name p=3199
a=bdr (6106457559585933042,1,16398,): receive LOCATION:
exec_replication_command, walsender.c:1292d=db_name p=3199 a=bdr
(6106457559585933042,1,16398,): receive DEBUG: 00000: received replication
command: START_REPLICATION SLOT "bdr_16396_6106457559585933042_1_16398__"
LOGICAL 0/0 (pg_version '90400', pg_catversion '201408161', bdr_version
'701', min_bdr_version '700', sizeof_int '4', sizeof_long '8', sizeof_datum
'8', maxalign '8', float4_byval '1', float8_byval '1', integer_datetimes
'1', bigendian '0', db_encoding 'UTF8')d=db_name p=3199 a=bdr
(6106457559585933042,1,16398,): receive LOCATION: exec_replication_command,
walsender.c:1292d=db_name p=3199 a=bdr (6106457559585933042,1,16398,):
receive DEBUG: 00000: bdr.bdr_conflict_handlers OID set to 16991d=db_name
p=3199 a=bdr (6106457559585933042,1,16398,): receive CONTEXT: slot
"bdr_16396_6106457559585933042_1_16398__", output plugin "bdr", in the
startup callbackd=db_name p=3199 a=bdr (6106457559585933042,1,16398,):
receive LOCATION: pg_decode_startup, bdr_output.c:450d=db_name p=3199 a=bdr
(6106457559585933042,1,16398,): receive ERROR: 55000: bdr output plugin:
slot creation rejected, bdr.bdr_nodes entry for local node
(sysid=6107134089288229725, timelineid=1, dboid=16396): status='i', bdr
still starting up: applying initial dump of remote noded=db_name p=3199
a=bdr (6106457559585933042,1,16398,): receive HINT: Monitor
pg_stat_activity and the logs, wait until the node has caught upd=db_name
p=3199 a=bdr (6106457559585933042,1,16398,): receive CONTEXT: slot
"bdr_16396_6106457559585933042_1_16398__", output plugin "bdr", in the
startup callbackd=db_name p=3199 a=bdr (6106457559585933042,1,16398,):
receive LOCATION: bdr_ensure_node_ready, bdr_output.c:268d=db_name p=3199
a=bdr (6106457559585933042,1,16398,): receive LOG: 08006: could not receive
data from client: Connection reset by peerd=db_name p=3199 a=bdr
(6106457559585933042,1,16398,): receive LOCATION: pq_recvbuf,
pqcomm.c:869d=db_name p=3199 a=bdr (6106457559585933042,1,16398,): receive
DEBUG: 08003: unexpected EOF on client connectiond=db_name p=3199 a=bdr
(6106457559585933042,1,16398,): receive LOCATION: SocketBackend,
postgres.c:353*The persisting error is:* 55000: bdr output plugin: slot
creation rejected, bdr.bdr_nodes entry for local node
(sysid=6107134089288229725, timelineid=1, dboid=16396): status='i', bdr
still starting up: applying initial dump of remote nodeI hope I have
provided enough detail and that I am posting at the right forum.Regards

--
View this message in context: http://postgresql.nabble.com/BDR-Error-restarted-tp5833139p5835137.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

#11

Craig Ringer

craig@2ndquadrant.com

about 11 years ago

In reply to: agent (#10)

Re: BDR Error restarted

On 23 January 2015 at 08:22, agent <zipho.m@gmail.com> wrote:

Hi Craig I have a similar issue with a rather small number of servers
involved.

OK. That looks odd.

What revision are you running exactly?

git rev-parse --short HEAD

please.

Also, correponding logs from the other two nodes please.

#12

agent

zipho.m@gmail.com

about 11 years ago

In reply to: Craig Ringer (#11)

Re: BDR Error restarted

Hi Chris

When running: git rev-parse --short HEAD

Output: a63674e

The source package I got from:

git clone git://git.postgresql.org/git/2ndquadrant_bdr.git
git clean -d -fx ""
git checkout bdr/0.7.1

This is the logs from node 1:

d= p=28495 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28495 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=28495 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28495 a=DEBUG: 00000: bdr (6106457559585933042,1,16398,): node3: apply
initialized on db_name
d= p=28495 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=28495 a=DEBUG: 00000: found valid replication identifier 4
d= p=28495 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=28495 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=28495 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=28495 a=ERROR: XX000: data stream ended
d= p=28495 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=31417 a=LOG: 00000: worker process: bdr
(6106457559585933042,1,16398,): node3: apply (PID 28495) exited with exit
code 1
d= p=31417 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=31417 a=LOG: 00000: starting background worker process "bdr
(6106457559585933042,1,16398,): node3: apply"
d= p=31417 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=28515 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=28515 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28515 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=28515 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28515 a=DEBUG: 00000: bdr (6106457559585933042,1,16398,): node3: apply
initialized on db_name
d= p=28515 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=28515 a=DEBUG: 00000: found valid replication identifier 4
d= p=28515 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=28515 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=28515 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=28515 a=ERROR: XX000: data stream ended
d= p=28515 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=31417 a=LOG: 00000: worker process: bdr
(6106457559585933042,1,16398,): node3: apply (PID 28515) exited with exit
code 1
d= p=31417 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=28519 a=DEBUG: 00000: autovacuum: processing database "postgres"
d= p=28519 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=31417 a=LOG: 00000: starting background worker process "bdr
(6106457559585933042,1,16398,): node3: apply"
d= p=31417 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=28520 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=28520 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28520 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=28520 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28520 a=DEBUG: 00000: bdr (6106457559585933042,1,16398,): node3: apply
initialized on db_name
d= p=28520 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=28520 a=DEBUG: 00000: found valid replication identifier 4
d= p=28520 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=28520 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=28520 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=28520 a=ERROR: XX000: data stream ended
d= p=28520 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=31417 a=LOG: 00000: worker process: bdr
(6106457559585933042,1,16398,): node3: apply (PID 28520) exited with exit
code 1
d= p=31417 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=31417 a=LOG: 00000: starting background worker process "bdr
(6106457559585933042,1,16398,): node3: apply"
d= p=31417 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=28524 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=28524 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28524 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=28524 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28524 a=DEBUG: 00000: bdr (6106457559585933042,1,16398,): node3: apply
initialized on db_name
d= p=28524 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=28524 a=DEBUG: 00000: found valid replication identifier 4
d= p=28524 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=28524 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=28524 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=28524 a=ERROR: XX000: data stream ended
d= p=28524 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=31417 a=LOG: 00000: worker process: bdr
(6106457559585933042,1,16398,): node3: apply (PID 28524) exited with exit
code 1
d= p=31417 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=31417 a=LOG: 00000: starting background worker process "bdr
(6106457559585933042,1,16398,): node3: apply"
d= p=31417 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=28550 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=28550 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28550 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=28550 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28550 a=DEBUG: 00000: bdr (6106457559585933042,1,16398,): node3: apply
initialized on db_name
d= p=28550 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=28550 a=DEBUG: 00000: found valid replication identifier 4
d= p=28550 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=28550 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=28550 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=28550 a=ERROR: XX000: data stream ended
d= p=28550 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=31417 a=LOG: 00000: worker process: bdr
(6106457559585933042,1,16398,): node3: apply (PID 28550) exited with exit
code 1
d= p=31417 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=28566 a=DEBUG: 00000: autovacuum: processing database "template1"
d= p=28566 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=31417 a=LOG: 00000: starting background worker process "bdr
(6106457559585933042,1,16398,): node3: apply"
d= p=31417 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=28591 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=28591 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28591 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=28591 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28591 a=DEBUG: 00000: bdr (6106457559585933042,1,16398,): node3: apply
initialized on db_name
d= p=28591 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=28591 a=DEBUG: 00000: found valid replication identifier 4
d= p=28591 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=28591 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=28591 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=28591 a=ERROR: XX000: data stream ended
d= p=28591 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=31417 a=LOG: 00000: worker process: bdr
(6106457559585933042,1,16398,): node3: apply (PID 28591) exited with exit
code 1
d= p=31417 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=31417 a=LOG: 00000: starting background worker process "bdr
(6106457559585933042,1,16398,): node3: apply"
d= p=31417 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=28595 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=28595 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28595 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=28595 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28595 a=DEBUG: 00000: bdr (6106457559585933042,1,16398,): node3: apply
initialized on db_name
d= p=28595 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=28595 a=DEBUG: 00000: found valid replication identifier 4
d= p=28595 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=28595 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=28595 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=28595 a=ERROR: XX000: data stream ended
d= p=28595 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=31417 a=LOG: 00000: worker process: bdr
(6106457559585933042,1,16398,): node3: apply (PID 28595) exited with exit
code 1
d= p=31417 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=31417 a=LOG: 00000: starting background worker process "bdr
(6106457559585933042,1,16398,): node3: apply"
d= p=31417 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=28599 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=28599 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28599 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=28599 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28599 a=DEBUG: 00000: bdr (6106457559585933042,1,16398,): node3: apply
initialized on db_name
d= p=28599 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=28599 a=DEBUG: 00000: found valid replication identifier 4
d= p=28599 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=28599 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=28599 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=28599 a=ERROR: XX000: data stream ended
d= p=28599 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=31417 a=LOG: 00000: worker process: bdr
(6106457559585933042,1,16398,): node3: apply (PID 28599) exited with exit
code 1
d= p=31417 a=LOCATION: LogChildExit, postmaster.c:3287

node2:

d= p=4813 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 4813) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4823 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4823 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4823 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4823 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4823 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4823 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4823 a=DEBUG: 00000: found valid replication identifier 4
d= p=4823 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4823 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4823 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4823 a=ERROR: XX000: data stream ended
d= p=4823 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 4823) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=4827 a=DEBUG: 00000: autovacuum: processing database
"trustpay_null_production"
d= p=4827 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4850 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4850 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4850 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4850 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4850 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4850 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4850 a=DEBUG: 00000: found valid replication identifier 4
d= p=4850 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4850 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4850 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4850 a=ERROR: XX000: data stream ended
d= p=4850 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 4850) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4854 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4854 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4854 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4854 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4854 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4854 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4854 a=DEBUG: 00000: found valid replication identifier 4
d= p=4854 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4854 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4854 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4854 a=ERROR: XX000: data stream ended
d= p=4854 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 4854) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4868 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4868 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4868 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4868 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4868 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4868 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4868 a=DEBUG: 00000: found valid replication identifier 4
d= p=4868 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4868 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4868 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4868 a=ERROR: XX000: data stream ended
d= p=4868 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 4868) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=4876 a=DEBUG: 00000: autovacuum: processing database "db_name"
d= p=4876 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4880 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4880 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4880 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4880 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4880 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4880 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4880 a=DEBUG: 00000: found valid replication identifier 4
d= p=4880 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4880 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4880 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4880 a=ERROR: XX000: data stream ended
d= p=4880 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 4880) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4898 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4898 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4898 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4898 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4898 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4898 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4898 a=DEBUG: 00000: found valid replication identifier 4
d= p=4898 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4898 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4898 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4898 a=ERROR: XX000: data stream ended
d= p=4898 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 4898) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4907 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4907 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4907 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4907 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4907 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4907 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4907 a=DEBUG: 00000: found valid replication identifier 4
d= p=4907 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4907 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4907 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4907 a=ERROR: XX000: data stream ended
d= p=4907 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 4907) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=4912 a=DEBUG: 00000: autovacuum: processing database "postgres"
d= p=4912 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4935 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4935 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4935 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4935 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4935 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4935 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4935 a=DEBUG: 00000: found valid replication identifier 4
d= p=4935 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4935 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4935 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4935 a=ERROR: XX000: data stream ended
d= p=4935 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 4935) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4940 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4940 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4940 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4940 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4940 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4940 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4940 a=DEBUG: 00000: found valid replication identifier 4
d= p=4940 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4940 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4940 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4940 a=ERROR: XX000: data stream ended
d= p=4940 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 4940) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4954 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4954 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4954 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4954 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4954 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4954 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4954 a=DEBUG: 00000: found valid replication identifier 4
d= p=4954 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4954 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4954 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4954 a=ERROR: XX000: data stream ended
d= p=4954 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 4954) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6707 a=DEBUG: 00000: started 0 votes
d= p=6707 a=LOCATION: bdr_sequencer_vote, bdr_seq.c:715
d= p=6707 a=DEBUG: 00000: tallied 0 elections
d= p=6707 a=LOCATION: bdr_sequencer_tally, bdr_seq.c:823
d= p=6707 a=DEBUG: 00000: started 0 elections
d= p=6707 a=LOCATION: bdr_sequencer_start_elections, bdr_seq.c:766
d= p=4964 a=DEBUG: 00000: autovacuum: processing database "template1"
d= p=4964 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4965 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4965 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4965 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4965 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4965 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4965 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4965 a=DEBUG: 00000: found valid replication identifier 4
d= p=4965 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4965 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4965 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4965 a=ERROR: XX000: data stream ended
d= p=4965 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 4965) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4970 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4970 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4970 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4970 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4970 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4970 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4970 a=DEBUG: 00000: found valid replication identifier 4
d= p=4970 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4970 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4970 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4970 a=ERROR: XX000: data stream ended
d= p=4970 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 4970) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4980 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4980 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4980 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4980 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4980 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4980 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4980 a=DEBUG: 00000: found valid replication identifier 4
d= p=4980 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4980 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4980 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4980 a=ERROR: XX000: data stream ended
d= p=4980 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 4980) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=4984 a=DEBUG: 00000: autovacuum: processing database
"trustpay_null_production"
d= p=4984 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5007 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5007 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5007 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5007 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5007 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5007 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5007 a=DEBUG: 00000: found valid replication identifier 4
d= p=5007 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5007 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5007 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5007 a=ERROR: XX000: data stream ended
d= p=5007 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5007) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5011 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5011 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5011 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5011 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5011 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5011 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5011 a=DEBUG: 00000: found valid replication identifier 4
d= p=5011 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5011 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5011 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5011 a=ERROR: XX000: data stream ended
d= p=5011 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5011) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5025 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5025 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5025 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5025 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5025 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5025 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5025 a=DEBUG: 00000: found valid replication identifier 4
d= p=5025 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5025 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5025 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5025 a=ERROR: XX000: data stream ended
d= p=5025 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5025) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=5033 a=DEBUG: 00000: autovacuum: processing database "db_name"
d= p=5033 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5037 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5037 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5037 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5037 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5037 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5037 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5037 a=DEBUG: 00000: found valid replication identifier 4
d= p=5037 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5037 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5037 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5037 a=ERROR: XX000: data stream ended
d= p=5037 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5037) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5083 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5083 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5083 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5083 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5083 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5083 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5083 a=DEBUG: 00000: found valid replication identifier 4
d= p=5083 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5083 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5083 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5083 a=ERROR: XX000: data stream ended
d= p=5083 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5083) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5092 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5092 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5092 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5092 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5092 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5092 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5092 a=DEBUG: 00000: found valid replication identifier 4
d= p=5092 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5092 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5092 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5092 a=ERROR: XX000: data stream ended
d= p=5092 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5092) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=5097 a=DEBUG: 00000: autovacuum: processing database "postgres"
d= p=5097 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5120 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5120 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5120 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5120 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5120 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5120 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5120 a=DEBUG: 00000: found valid replication identifier 4
d= p=5120 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5120 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5120 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5120 a=ERROR: XX000: data stream ended
d= p=5120 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5120) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5125 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5125 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5125 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5125 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5125 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5125 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5125 a=DEBUG: 00000: found valid replication identifier 4
d= p=5125 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5125 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5125 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5125 a=ERROR: XX000: data stream ended
d= p=5125 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5125) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5139 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5139 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5139 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5139 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5139 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5139 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5139 a=DEBUG: 00000: found valid replication identifier 4
d= p=5139 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5139 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5139 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5139 a=ERROR: XX000: data stream ended
d= p=5139 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5139) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=5149 a=DEBUG: 00000: autovacuum: processing database "template1"
d= p=5149 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5150 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5150 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5150 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5150 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5150 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5150 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5150 a=DEBUG: 00000: found valid replication identifier 4
d= p=5150 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5150 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5150 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5150 a=ERROR: XX000: data stream ended
d= p=5150 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5150) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5155 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5155 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5155 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5155 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5155 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5155 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5155 a=DEBUG: 00000: found valid replication identifier 4
d= p=5155 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5155 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5155 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5155 a=ERROR: XX000: data stream ended
d= p=5155 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5155) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5165 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5165 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5165 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5165 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5165 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5165 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5165 a=DEBUG: 00000: found valid replication identifier 4
d= p=5165 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5165 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5165 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5165 a=ERROR: XX000: data stream ended
d= p=5165 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5165) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=5169 a=DEBUG: 00000: autovacuum: processing database
"trustpay_null_production"
d= p=5169 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5192 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5192 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5192 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5192 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5192 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5192 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5192 a=DEBUG: 00000: found valid replication identifier 4
d= p=5192 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5192 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5192 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5192 a=ERROR: XX000: data stream ended
d= p=5192 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5192) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5196 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5196 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5196 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5196 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5196 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5196 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5196 a=DEBUG: 00000: found valid replication identifier 4
d= p=5196 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5196 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5196 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5196 a=ERROR: XX000: data stream ended
d= p=5196 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5196) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5210 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5210 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5210 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5210 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5210 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5210 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5210 a=DEBUG: 00000: found valid replication identifier 4
d= p=5210 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5210 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5210 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5210 a=ERROR: XX000: data stream ended
d= p=5210 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5210) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=5218 a=DEBUG: 00000: autovacuum: processing database "db_name"
d= p=5218 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5229 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5229 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5229 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5229 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5229 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5229 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5229 a=DEBUG: 00000: found valid replication identifier 4
d= p=5229 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5229 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5229 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5229 a=ERROR: XX000: data stream ended
d= p=5229 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5229) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5240 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5240 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5240 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5240 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5240 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5240 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5240 a=DEBUG: 00000: found valid replication identifier 4
d= p=5240 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5240 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5240 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5240 a=ERROR: XX000: data stream ended
d= p=5240 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5240) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5249 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5249 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5249 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5249 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5249 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5249 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5249 a=DEBUG: 00000: found valid replication identifier 4
d= p=5249 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5249 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5249 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5249 a=ERROR: XX000: data stream ended
d= p=5249 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr (6106458869483394081,1,16396,):
node3: apply (PID 5249) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=5254 a=DEBUG: 00000: autovacuum: processing database "postgres"
d= p=5254 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670

--
View this message in context: http://postgresql.nabble.com/BDR-Error-restarted-tp5833139p5835190.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#13

Craig Ringer

craig@2ndquadrant.com

about 11 years ago

In reply to: agent (#12)

Re: BDR Error restarted

OK, so you're on 0.7.1.

Before we pursue much more, can you please update to bdr-plugin/next ? The
0.8.0 release is pending shortly, and contains a lot of changes since
0.7.1, some of which pertain to init behaviour.

After 0.7.1 the structure was changed a bit. So you will need to first
check out and compile bdr-pg/REL9_4_STABLE . Then you'll need to check out
bdr-plugin/next and compile it against your new Pg. Details are on the BDR
wiki: https://wiki.postgresql.org/wiki/BDR_Administration#From_source . Or
if you're on Fedora or RHEL/CentOS you can install the packages.

On 23 January 2015 at 19:24, agent <zipho.m@gmail.com> wrote:

Hi Chris

When running: git rev-parse --short HEAD

Output: a63674e

The source package I got from:

git clone git://git.postgresql.org/git/2ndquadrant_bdr.git
git clean -d -fx ""
git checkout bdr/0.7.1

This is the logs from node 1:

d= p=28495 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28495 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=28495 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28495 a=DEBUG: 00000: bdr (6106457559585933042,1,16398,): node3:
apply
initialized on db_name
d= p=28495 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=28495 a=DEBUG: 00000: found valid replication identifier 4
d= p=28495 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=28495 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=28495 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=28495 a=ERROR: XX000: data stream ended
d= p=28495 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=31417 a=LOG: 00000: worker process: bdr
(6106457559585933042,1,16398,): node3: apply (PID 28495) exited with exit
code 1
d= p=31417 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=31417 a=LOG: 00000: starting background worker process "bdr
(6106457559585933042,1,16398,): node3: apply"
d= p=31417 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=28515 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=28515 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28515 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=28515 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28515 a=DEBUG: 00000: bdr (6106457559585933042,1,16398,): node3:
apply
initialized on db_name
d= p=28515 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=28515 a=DEBUG: 00000: found valid replication identifier 4
d= p=28515 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=28515 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=28515 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=28515 a=ERROR: XX000: data stream ended
d= p=28515 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=31417 a=LOG: 00000: worker process: bdr
(6106457559585933042,1,16398,): node3: apply (PID 28515) exited with exit
code 1
d= p=31417 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=28519 a=DEBUG: 00000: autovacuum: processing database "postgres"
d= p=28519 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=31417 a=LOG: 00000: starting background worker process "bdr
(6106457559585933042,1,16398,): node3: apply"
d= p=31417 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=28520 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=28520 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28520 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=28520 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28520 a=DEBUG: 00000: bdr (6106457559585933042,1,16398,): node3:
apply
initialized on db_name
d= p=28520 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=28520 a=DEBUG: 00000: found valid replication identifier 4
d= p=28520 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=28520 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=28520 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=28520 a=ERROR: XX000: data stream ended
d= p=28520 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=31417 a=LOG: 00000: worker process: bdr
(6106457559585933042,1,16398,): node3: apply (PID 28520) exited with exit
code 1
d= p=31417 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=31417 a=LOG: 00000: starting background worker process "bdr
(6106457559585933042,1,16398,): node3: apply"
d= p=31417 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=28524 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=28524 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28524 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=28524 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28524 a=DEBUG: 00000: bdr (6106457559585933042,1,16398,): node3:
apply
initialized on db_name
d= p=28524 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=28524 a=DEBUG: 00000: found valid replication identifier 4
d= p=28524 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=28524 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=28524 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=28524 a=ERROR: XX000: data stream ended
d= p=28524 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=31417 a=LOG: 00000: worker process: bdr
(6106457559585933042,1,16398,): node3: apply (PID 28524) exited with exit
code 1
d= p=31417 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=31417 a=LOG: 00000: starting background worker process "bdr
(6106457559585933042,1,16398,): node3: apply"
d= p=31417 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=28550 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=28550 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28550 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=28550 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28550 a=DEBUG: 00000: bdr (6106457559585933042,1,16398,): node3:
apply
initialized on db_name
d= p=28550 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=28550 a=DEBUG: 00000: found valid replication identifier 4
d= p=28550 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=28550 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=28550 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=28550 a=ERROR: XX000: data stream ended
d= p=28550 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=31417 a=LOG: 00000: worker process: bdr
(6106457559585933042,1,16398,): node3: apply (PID 28550) exited with exit
code 1
d= p=31417 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=28566 a=DEBUG: 00000: autovacuum: processing database "template1"
d= p=28566 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=31417 a=LOG: 00000: starting background worker process "bdr
(6106457559585933042,1,16398,): node3: apply"
d= p=31417 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=28591 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=28591 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28591 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=28591 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28591 a=DEBUG: 00000: bdr (6106457559585933042,1,16398,): node3:
apply
initialized on db_name
d= p=28591 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=28591 a=DEBUG: 00000: found valid replication identifier 4
d= p=28591 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=28591 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=28591 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=28591 a=ERROR: XX000: data stream ended
d= p=28591 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=31417 a=LOG: 00000: worker process: bdr
(6106457559585933042,1,16398,): node3: apply (PID 28591) exited with exit
code 1
d= p=31417 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=31417 a=LOG: 00000: starting background worker process "bdr
(6106457559585933042,1,16398,): node3: apply"
d= p=31417 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=28595 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=28595 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28595 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=28595 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28595 a=DEBUG: 00000: bdr (6106457559585933042,1,16398,): node3:
apply
initialized on db_name
d= p=28595 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=28595 a=DEBUG: 00000: found valid replication identifier 4
d= p=28595 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=28595 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=28595 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=28595 a=ERROR: XX000: data stream ended
d= p=28595 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=31417 a=LOG: 00000: worker process: bdr
(6106457559585933042,1,16398,): node3: apply (PID 28595) exited with exit
code 1
d= p=31417 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=31417 a=LOG: 00000: starting background worker process "bdr
(6106457559585933042,1,16398,): node3: apply"
d= p=31417 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=28599 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=28599 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28599 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=28599 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=28599 a=DEBUG: 00000: bdr (6106457559585933042,1,16398,): node3:
apply
initialized on db_name
d= p=28599 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=28599 a=DEBUG: 00000: found valid replication identifier 4
d= p=28599 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=28599 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=28599 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=28599 a=ERROR: XX000: data stream ended
d= p=28599 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=31417 a=LOG: 00000: worker process: bdr
(6106457559585933042,1,16398,): node3: apply (PID 28599) exited with exit
code 1
d= p=31417 a=LOCATION: LogChildExit, postmaster.c:3287

node2:

d= p=4813 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 4813) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4823 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4823 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4823 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4823 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4823 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4823 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4823 a=DEBUG: 00000: found valid replication identifier 4
d= p=4823 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4823 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4823 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4823 a=ERROR: XX000: data stream ended
d= p=4823 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 4823) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=4827 a=DEBUG: 00000: autovacuum: processing database
"trustpay_null_production"
d= p=4827 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4850 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4850 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4850 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4850 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4850 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4850 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4850 a=DEBUG: 00000: found valid replication identifier 4
d= p=4850 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4850 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4850 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4850 a=ERROR: XX000: data stream ended
d= p=4850 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 4850) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4854 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4854 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4854 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4854 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4854 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4854 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4854 a=DEBUG: 00000: found valid replication identifier 4
d= p=4854 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4854 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4854 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4854 a=ERROR: XX000: data stream ended
d= p=4854 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 4854) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4868 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4868 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4868 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4868 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4868 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4868 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4868 a=DEBUG: 00000: found valid replication identifier 4
d= p=4868 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4868 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4868 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4868 a=ERROR: XX000: data stream ended
d= p=4868 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 4868) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=4876 a=DEBUG: 00000: autovacuum: processing database "db_name"
d= p=4876 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4880 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4880 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4880 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4880 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4880 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4880 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4880 a=DEBUG: 00000: found valid replication identifier 4
d= p=4880 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4880 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4880 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4880 a=ERROR: XX000: data stream ended
d= p=4880 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 4880) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4898 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4898 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4898 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4898 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4898 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4898 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4898 a=DEBUG: 00000: found valid replication identifier 4
d= p=4898 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4898 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4898 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4898 a=ERROR: XX000: data stream ended
d= p=4898 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 4898) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4907 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4907 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4907 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4907 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4907 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4907 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4907 a=DEBUG: 00000: found valid replication identifier 4
d= p=4907 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4907 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4907 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4907 a=ERROR: XX000: data stream ended
d= p=4907 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 4907) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=4912 a=DEBUG: 00000: autovacuum: processing database "postgres"
d= p=4912 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4935 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4935 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4935 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4935 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4935 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4935 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4935 a=DEBUG: 00000: found valid replication identifier 4
d= p=4935 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4935 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4935 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4935 a=ERROR: XX000: data stream ended
d= p=4935 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 4935) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4940 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4940 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4940 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4940 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4940 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4940 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4940 a=DEBUG: 00000: found valid replication identifier 4
d= p=4940 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4940 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4940 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4940 a=ERROR: XX000: data stream ended
d= p=4940 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 4940) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4954 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4954 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4954 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4954 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4954 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4954 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4954 a=DEBUG: 00000: found valid replication identifier 4
d= p=4954 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4954 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4954 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4954 a=ERROR: XX000: data stream ended
d= p=4954 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 4954) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6707 a=DEBUG: 00000: started 0 votes
d= p=6707 a=LOCATION: bdr_sequencer_vote, bdr_seq.c:715
d= p=6707 a=DEBUG: 00000: tallied 0 elections
d= p=6707 a=LOCATION: bdr_sequencer_tally, bdr_seq.c:823
d= p=6707 a=DEBUG: 00000: started 0 elections
d= p=6707 a=LOCATION: bdr_sequencer_start_elections, bdr_seq.c:766
d= p=4964 a=DEBUG: 00000: autovacuum: processing database "template1"
d= p=4964 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4965 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4965 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4965 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4965 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4965 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4965 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4965 a=DEBUG: 00000: found valid replication identifier 4
d= p=4965 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4965 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4965 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4965 a=ERROR: XX000: data stream ended
d= p=4965 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 4965) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4970 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4970 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4970 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4970 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4970 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4970 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4970 a=DEBUG: 00000: found valid replication identifier 4
d= p=4970 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4970 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4970 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4970 a=ERROR: XX000: data stream ended
d= p=4970 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 4970) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=4980 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=4980 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4980 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=4980 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=4980 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=4980 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=4980 a=DEBUG: 00000: found valid replication identifier 4
d= p=4980 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=4980 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=4980 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=4980 a=ERROR: XX000: data stream ended
d= p=4980 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 4980) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=4984 a=DEBUG: 00000: autovacuum: processing database
"trustpay_null_production"
d= p=4984 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5007 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5007 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5007 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5007 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5007 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5007 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5007 a=DEBUG: 00000: found valid replication identifier 4
d= p=5007 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5007 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5007 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5007 a=ERROR: XX000: data stream ended
d= p=5007 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5007) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5011 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5011 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5011 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5011 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5011 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5011 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5011 a=DEBUG: 00000: found valid replication identifier 4
d= p=5011 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5011 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5011 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5011 a=ERROR: XX000: data stream ended
d= p=5011 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5011) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5025 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5025 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5025 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5025 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5025 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5025 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5025 a=DEBUG: 00000: found valid replication identifier 4
d= p=5025 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5025 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5025 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5025 a=ERROR: XX000: data stream ended
d= p=5025 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5025) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=5033 a=DEBUG: 00000: autovacuum: processing database "db_name"
d= p=5033 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5037 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5037 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5037 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5037 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5037 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5037 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5037 a=DEBUG: 00000: found valid replication identifier 4
d= p=5037 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5037 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5037 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5037 a=ERROR: XX000: data stream ended
d= p=5037 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5037) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5083 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5083 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5083 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5083 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5083 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5083 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5083 a=DEBUG: 00000: found valid replication identifier 4
d= p=5083 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5083 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5083 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5083 a=ERROR: XX000: data stream ended
d= p=5083 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5083) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5092 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5092 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5092 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5092 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5092 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5092 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5092 a=DEBUG: 00000: found valid replication identifier 4
d= p=5092 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5092 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5092 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5092 a=ERROR: XX000: data stream ended
d= p=5092 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5092) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=5097 a=DEBUG: 00000: autovacuum: processing database "postgres"
d= p=5097 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5120 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5120 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5120 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5120 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5120 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5120 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5120 a=DEBUG: 00000: found valid replication identifier 4
d= p=5120 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5120 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5120 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5120 a=ERROR: XX000: data stream ended
d= p=5120 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5120) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5125 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5125 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5125 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5125 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5125 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5125 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5125 a=DEBUG: 00000: found valid replication identifier 4
d= p=5125 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5125 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5125 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5125 a=ERROR: XX000: data stream ended
d= p=5125 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5125) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5139 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5139 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5139 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5139 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5139 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5139 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5139 a=DEBUG: 00000: found valid replication identifier 4
d= p=5139 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5139 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5139 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5139 a=ERROR: XX000: data stream ended
d= p=5139 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5139) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=5149 a=DEBUG: 00000: autovacuum: processing database "template1"
d= p=5149 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5150 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5150 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5150 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5150 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5150 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5150 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5150 a=DEBUG: 00000: found valid replication identifier 4
d= p=5150 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5150 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5150 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5150 a=ERROR: XX000: data stream ended
d= p=5150 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5150) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5155 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5155 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5155 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5155 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5155 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5155 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5155 a=DEBUG: 00000: found valid replication identifier 4
d= p=5155 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5155 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5155 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5155 a=ERROR: XX000: data stream ended
d= p=5155 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5155) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5165 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5165 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5165 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5165 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5165 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5165 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5165 a=DEBUG: 00000: found valid replication identifier 4
d= p=5165 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5165 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5165 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5165 a=ERROR: XX000: data stream ended
d= p=5165 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5165) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=5169 a=DEBUG: 00000: autovacuum: processing database
"trustpay_null_production"
d= p=5169 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5192 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5192 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5192 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5192 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5192 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5192 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5192 a=DEBUG: 00000: found valid replication identifier 4
d= p=5192 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5192 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5192 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5192 a=ERROR: XX000: data stream ended
d= p=5192 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5192) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5196 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5196 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5196 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5196 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5196 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5196 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5196 a=DEBUG: 00000: found valid replication identifier 4
d= p=5196 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5196 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5196 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5196 a=ERROR: XX000: data stream ended
d= p=5196 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5196) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5210 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5210 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5210 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5210 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5210 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5210 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5210 a=DEBUG: 00000: found valid replication identifier 4
d= p=5210 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5210 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5210 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5210 a=ERROR: XX000: data stream ended
d= p=5210 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5210) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=5218 a=DEBUG: 00000: autovacuum: processing database "db_name"
d= p=5218 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5229 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5229 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5229 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5229 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5229 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5229 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5229 a=DEBUG: 00000: found valid replication identifier 4
d= p=5229 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5229 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5229 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5229 a=ERROR: XX000: data stream ended
d= p=5229 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5229) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5240 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5240 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5240 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5240 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5240 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5240 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5240 a=DEBUG: 00000: found valid replication identifier 4
d= p=5240 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5240 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5240 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5240 a=ERROR: XX000: data stream ended
d= p=5240 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5240) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=6700 a=LOG: 00000: starting background worker process "bdr
(6106458869483394081,1,16396,): node3: apply"
d= p=6700 a=LOCATION: do_start_bgworker, postmaster.c:5354
d= p=5249 a=NOTICE: 00000: version "1.0" of extension "btree_gist" is
already installed
d= p=5249 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5249 a=NOTICE: 00000: version "0.7.1" of extension "bdr" is already
installed
d= p=5249 a=LOCATION: ExecAlterExtensionStmt, extension.c:2700
d= p=5249 a=DEBUG: 00000: bdr (6106458869483394081,1,16396,): node3: apply
initialized on db_name
d= p=5249 a=LOCATION: bdr_apply_main, bdr.c:563
d= p=5249 a=DEBUG: 00000: found valid replication identifier 4
d= p=5249 a=LOCATION: bdr_establish_connection_and_slot, bdr.c:502
d= p=5249 a=INFO: 00000: starting up replication from 4 at 0/0
d= p=5249 a=LOCATION: bdr_apply_main, bdr.c:614
d= p=5249 a=ERROR: XX000: data stream ended
d= p=5249 a=LOCATION: bdr_apply_work, bdr_apply.c:2240
d= p=6700 a=LOG: 00000: worker process: bdr
(6106458869483394081,1,16396,):
node3: apply (PID 5249) exited with exit code 1
d= p=6700 a=LOCATION: LogChildExit, postmaster.c:3287
d= p=5254 a=DEBUG: 00000: autovacuum: processing database "postgres"
d= p=5254 a=LOCATION: AutoVacWorkerMain, autovacuum.c:1670

--
View this message in context:
http://postgresql.nabble.com/BDR-Error-restarted-tp5833139p5835190.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

#14

theftp

theftp.ru@gmail.com

about 11 years ago

In reply to: Craig Ringer (#13)

Re: BDR Error restarted

Hi all,

I have a similar problem.

OS Red Hat Enterprise Linux Server release 6.6.

Postgresql packets were installed from repository as described in
https://wiki.postgresql.org/wiki/BDR_Packages

postgresql-bdr94-bdr-0.8.0beta1-1_2ndQuadrant.el6.x86_64
postgresql-bdr94-contrib-9.4.0_bdr1-1_2ndQuadrant.el6.x86_64
postgresql-bdr94-9.4.0_bdr1-1_2ndQuadrant.el6.x86_64
postgresql-bdr94-libs-9.4.0_bdr1-1_2ndQuadrant.el6.x86_64
postgresql-bdr94-server-9.4.0_bdr1-1_2ndQuadrant.el6.x86_64

There are two servers: 192.168.214.169 (main), 192.168.214.163

On each server run commands:
su - postgres -c "/usr/pgsql-9.4/bin/initdb
--pgdata=/var/lib/pgsql/9.4-bdr/data --auth='trust' --username=test_root"
su - postgres -c "/usr/pgsql-9.4/bin/pg_ctl -l
/var/lib/pgsql/9.4-bdr/data/postgresql-02.log -D
/var/lib/pgsql/9.4-bdr/data -o '-p 5432' -w start"
createdb -p 5432 -U test_root testdb

On the first server I've created a schema and a table in it.

Then configs were updated and postgresql servers were restarted.
iptables is switched off.

//postgrsql.conf [192.168.214.169]

listen_addresses = '192.168.214.169'
port=5432

max_replication_slots = 3
max_wal_senders = 4
wal_level = 'logical'
track_commit_timestamp = on
shared_preload_libraries = 'bdr'
max_worker_processes = 10

bdr.connections = 'second'
bdr.second_dsn = 'host=192.168.214.163 port=5432 dbname=testdb
user=test_root'

// pg_hba.conf [192.168.214.169]

host all test_root 192.168.214.163/32 trust
host replication test_root 192.168.214.163/32 trust
+ uncomment existing replication lines

//postgrsql.conf [192.168.214.163]

listen_addresses = '192.168.214.163'
port=5432

max_replication_slots = 3
max_wal_senders = 4
wal_level = 'logical'
track_commit_timestamp = on
shared_preload_libraries = 'bdr'
max_worker_processes = 10

bdr.connections = 'first'
bdr.first_dsn = 'host=192.168.214.169 port=5432 dbname=testdb
user=test_root'
bdr.first_init_replica = on
bdr.first_replica_local_dsn = 'port=5432 dbname=testdb user=test_root'

// pg_hba.conf [192.168.214.163]

host all test_root 192.168.214.169/32 trust
host replication test_root 192.168.214.169/32 trust
+ uncomment existing replication lines

Replication works only on that created table and only from the first to the
second server.
When I try to create one more table, the sql command 'CREATE TABLE ...'
hangs.

The followng errors are added each second in postgresql logs:

// 192.168.214.169

LOG: starting background worker process "bdr
(6109689945671396351,1,16384,): second: apply"
NOTICE: version "1.0" of extension "btree_gist" is already installed
CONTEXT: slot "bdr_16384_6109689945671396351_1_16384__", output plugin
"bdr", in the startup callback
NOTICE: version "0.8.0.6" of extension "bdr" is already installed
CONTEXT: slot "bdr_16384_6109689945671396351_1_16384__", output plugin
"bdr", in the startup callback
ERROR: data stream ended
LOG: worker process: bdr (6109689945671396351,1,16384,): second: apply (PID
13549) exited with exit code 1

// 192.168.214.163

ERROR: bdr output plugin: slot creation rejected, bdr.bdr_nodes entry for
local node (sysid=6109702913719211129, timelineid=1, dboid=16384):
status='i', bdr still starting up: applying initial dump of remote node
HINT: Monitor pg_stat_activity and the logs, wait until the node has caught
up
CONTEXT: slot "bdr_16384_6109689945671396351_1_16384__", output plugin
"bdr", in the startup callback
LOG: could not receive data from client: Connection reset by peer

In attachment - debug logs of postgresql.

How to fix it?

first_postgres.log
<http://postgresql.nabble.com/file/n5836407/first_postgres.log>
second_postgres.log
<http://postgresql.nabble.com/file/n5836407/second_postgres.log>

--
View this message in context: http://postgresql.nabble.com/BDR-Error-restarted-tp5833139p5836407.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general