BUG #7534: walreceiver takes long time to detect n/w breakdown

On Wed, Sep 12, 2012 at 1:54 PM, <amit.kapila@huawei.com> wrote:

The following bug has been logged on the website:

Bug reference: 7534
Logged by: Amit Kapila
Email address: amit.kapila@huawei.com
PostgreSQL version: 9.2.0
Operating system: Suse 10
Description:

1. Both master and standby machine are connected normally,
2. then you use the command: ifconfig ip down; make the network card of
master and standby down,

Observation
master can detect connect abnormal, but the standby can't detect connect
abnormal and show a connected channel long time.

The master will detect it quicker, because it will get an error when
it tries to send something.

But the standby should detect it either when sending the feedback
message (what's your wal_receiver_status_interval set to?) or when
ythe kernel does (have you configured the tcp keepalive on the slave
somehow?)

Oh, and what do you actually mean by "long time"?

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

On Wed, Sep 12, 2012 at 8:54 PM, <amit.kapila@huawei.com> wrote:

The following bug has been logged on the website:

Bug reference: 7534
Logged by: Amit Kapila
Email address: amit.kapila@huawei.com
PostgreSQL version: 9.2.0
Operating system: Suse 10
Description:

1. Both master and standby machine are connected normally,
2. then you use the command: ifconfig ip down; make the network card of
master and standby down,

Observation
master can detect connect abnormal, but the standby can't detect connect
abnormal and show a connected channel long time.

What about setting keepalives_xxx libpq parameters?
http://www.postgresql.org/docs/devel/static/libpq-connect.html#LIBPQ-PARAMKEYWORDS

Keepalives are not a perfect solution for the termination of connection, but
it would help to a certain extent. If you need something like
walreceiver-version
of replication_timeout, such feature has not been implemented yet. Please feel
free to implement that!

Regards,

--
Fujii Masao

On Wednesday, September 12, 2012 10:12 PM Magnus Hagander wrote:
On Wed, Sep 12, 2012 at 1:54 PM, <amit.kapila@huawei.com> wrote:

The following bug has been logged on the website:

Bug reference: 7534
Logged by: Amit Kapila
Email address: amit.kapila@huawei.com
PostgreSQL version: 9.2.0
Operating system: Suse 10
Description:

1. Both master and standby machine are connected normally,
2. then you use the command: ifconfig ip down; make the network card of
master and standby down,

Observation
master can detect connect abnormal, but the standby can't detect connect
abnormal and show a connected channel long time.

The master will detect it quicker, because it will get an error when
it tries to send something.

But the standby should detect it either when sending the feedback
message (what's your wal_receiver_status_interval set to?) or when
ythe kernel does (have you configured the tcp keepalive on the slave
somehow?)

wal_receiver_status_interval - 10s (we have not changed this. Used as
default).
We have tried by using tcp keepalive as well, it might not be able to
detect as receiver is anyway trying to send
Receiver status.
It fails during send socket call from XLogWalRcvSendReply() after calling
the same many times as internally might be in
send() until the sockets internal buffer is full, it keeps accumulating
even if other side recv has not received the
data.
Also in walsender, it is failing to replication_timeout parameter not due
to send failure.
So in my opinion, the full-proof solution would be to have mechanism
(replication_timeout) similar to walsender in
walreceiver.

Oh, and what do you actually mean by "long time"?

15-20 mins.

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

On Wednesday, September 12, 2012 10:15 PM Fujii Masao
On Wed, Sep 12, 2012 at 8:54 PM, <amit.kapila@huawei.com> wrote:

The following bug has been logged on the website:

Bug reference: 7534
Logged by: Amit Kapila
Email address: amit.kapila@huawei.com
PostgreSQL version: 9.2.0
Operating system: Suse 10
Description:

1. Both master and standby machine are connected normally,
2. then you use the command: ifconfig ip down; make the network card of
master and standby down,

Observation
master can detect connect abnormal, but the standby can't detect connect
abnormal and show a connected channel long time.

What about setting keepalives_xxx libpq parameters?

http://www.postgresql.org/docs/devel/static/libpq-connect.html#LIBPQ-PARAMKE
YWORDS

Keepalives are not a perfect solution for the termination of connection,

but

it would help to a certain extent.

We have tried by enabling keepalive, but it didn't worked maybe because
walreceiver is trying to send reveiver status.
It fails in sending that after many attempts of same.

If you need something like walreceiver-version of replication_timeout,

such feature has not been implemented yet.

Please feel free to implement that!

I would like to implement such feature for walreceiver, but there is one
confusion that whether to use
same configuration parameter(replication_timeout) for walrecevier as for
master or introduce a new
configuration parameter (receiver_replication_timeout).

The only point in having different timeout parameters for walsender and
walreceiver is for the case of standby which
has both walsender and walreceiver to send logs to cascaded standby, in
such case somebody might want to have different timeout parameters for
walsender and walreceiver.
OTOH it will create confusion to have too many parameters. My opinion is to
have one timeout parameter for both walsender and walrecevier.

Let me know your suggestion/opinion about same.

Note- I am marking cc to pgsql-hackers, as it will be a feature request.

With Regards,
Amit Kapila.

On Thu, Sep 13, 2012 at 1:22 PM, Amit Kapila <amit.kapila@huawei.com> wrote:

On Wednesday, September 12, 2012 10:15 PM Fujii Masao
On Wed, Sep 12, 2012 at 8:54 PM, <amit.kapila@huawei.com> wrote:

The following bug has been logged on the website:

Bug reference: 7534
Logged by: Amit Kapila
Email address: amit.kapila@huawei.com
PostgreSQL version: 9.2.0
Operating system: Suse 10
Description:

1. Both master and standby machine are connected normally,
2. then you use the command: ifconfig ip down; make the network card of
master and standby down,

Observation
master can detect connect abnormal, but the standby can't detect connect
abnormal and show a connected channel long time.

What about setting keepalives_xxx libpq parameters?

http://www.postgresql.org/docs/devel/static/libpq-connect.html#LIBPQ-PARAMKE
YWORDS

Keepalives are not a perfect solution for the termination of connection,

but

it would help to a certain extent.

We have tried by enabling keepalive, but it didn't worked maybe because
walreceiver is trying to send reveiver status.
It fails in sending that after many attempts of same.

If you need something like walreceiver-version of replication_timeout,

such feature has not been implemented yet.

Please feel free to implement that!

I would like to implement such feature for walreceiver, but there is one
confusion that whether to use
same configuration parameter(replication_timeout) for walrecevier as for
master or introduce a new
configuration parameter (receiver_replication_timeout).

I like the latter. I believe some users want to set the different
timeout values,
for example, in the case where the master and standby servers are placed in
the same room, but cascaded standby is placed in other continent.

Regards,

--
Fujii Masao

On Thursday, September 13, 2012 10:57 PM Fujii Masao
On Thu, Sep 13, 2012 at 1:22 PM, Amit Kapila <amit.kapila@huawei.com> wrote:

On Wednesday, September 12, 2012 10:15 PM Fujii Masao
On Wed, Sep 12, 2012 at 8:54 PM, <amit.kapila@huawei.com> wrote:

The following bug has been logged on the website:

Bug reference: 7534
Logged by: Amit Kapila
Email address: amit.kapila@huawei.com
PostgreSQL version: 9.2.0
Operating system: Suse 10
Description:

1. Both master and standby machine are connected normally,
2. then you use the command: ifconfig ip down; make the network card of
master and standby down,

Observation
master can detect connect abnormal, but the standby can't detect connect
abnormal and show a connected channel long time.

I would like to implement such feature for walreceiver, but there is one
confusion that whether to use
same configuration parameter(replication_timeout) for walrecevier as for
master or introduce a new
configuration parameter (receiver_replication_timeout).

I like the latter. I believe some users want to set the different
timeout values,
for example, in the case where the master and standby servers are placed in
the same room, but cascaded standby is placed in other continent.

Thank you for your suggestion. I have implemented as per your suggestion to have separate timeout parameter for walreceiver.
The main changes are:
1. Introduce a new configuration parameter wal_receiver_replication_timeout for walreceiver.
2. In function WalReceiverMain(), check if there is no communication till wal_receiver_replication_timeout, exit the walreceiver.
This is same as walsender functionality.

As this is a feature, So I am uploading the attached patch in coming CommitFest.

Suggestions/Comments?

With Regards,
Amit Kapila.

On Fri, Sep 14, 2012 at 10:01 PM, Amit kapila <amit.kapila@huawei.com> wrote:

On Thursday, September 13, 2012 10:57 PM Fujii Masao
On Thu, Sep 13, 2012 at 1:22 PM, Amit Kapila <amit.kapila@huawei.com> wrote:

On Wednesday, September 12, 2012 10:15 PM Fujii Masao
On Wed, Sep 12, 2012 at 8:54 PM, <amit.kapila@huawei.com> wrote:

The following bug has been logged on the website:

Bug reference: 7534
Logged by: Amit Kapila
Email address: amit.kapila@huawei.com
PostgreSQL version: 9.2.0
Operating system: Suse 10
Description:

1. Both master and standby machine are connected normally,
2. then you use the command: ifconfig ip down; make the network card of
master and standby down,

Observation
master can detect connect abnormal, but the standby can't detect connect
abnormal and show a connected channel long time.

I would like to implement such feature for walreceiver, but there is one
confusion that whether to use
same configuration parameter(replication_timeout) for walrecevier as for
master or introduce a new
configuration parameter (receiver_replication_timeout).

I like the latter. I believe some users want to set the different
timeout values,
for example, in the case where the master and standby servers are placed in
the same room, but cascaded standby is placed in other continent.

Thank you for your suggestion. I have implemented as per your suggestion to have separate timeout parameter for walreceiver.
The main changes are:
1. Introduce a new configuration parameter wal_receiver_replication_timeout for walreceiver.
2. In function WalReceiverMain(), check if there is no communication till wal_receiver_replication_timeout, exit the walreceiver.
This is same as walsender functionality.

As this is a feature, So I am uploading the attached patch in coming CommitFest.

Suggestions/Comments?

You also need to change walsender so that it periodically sends the heartbeat
message, like walreceiver does each wal_receiver_status_interval. Otherwise,
walreceiver will detect the timeout wrongly whenever there is no traffic in the
master.

Regards,

--
Fujii Masao

On Saturday, September 15, 2012 11:27 AM Fujii Masao wrote:
On Fri, Sep 14, 2012 at 10:01 PM, Amit kapila <amit.kapila@huawei.com> wrote:

On Thursday, September 13, 2012 10:57 PM Fujii Masao
On Thu, Sep 13, 2012 at 1:22 PM, Amit Kapila <amit.kapila@huawei.com> wrote:

On Wednesday, September 12, 2012 10:15 PM Fujii Masao
On Wed, Sep 12, 2012 at 8:54 PM, <amit.kapila@huawei.com> wrote:

The following bug has been logged on the website:

I would like to implement such feature for walreceiver, but there is one
confusion that whether to use
same configuration parameter(replication_timeout) for walrecevier as for
master or introduce a new
configuration parameter (receiver_replication_timeout).

I like the latter. I believe some users want to set the different
timeout values,
for example, in the case where the master and standby servers are placed in
the same room, but cascaded standby is placed in other continent.

Thank you for your suggestion. I have implemented as per your suggestion to have separate timeout parameter for walreceiver.
The main changes are:
1. Introduce a new configuration parameter wal_receiver_replication_timeout for walreceiver.
2. In function WalReceiverMain(), check if there is no communication till wal_receiver_replication_timeout, exit the walreceiver.
This is same as walsender functionality.

As this is a feature, So I am uploading the attached patch in coming CommitFest.

Suggestions/Comments?

You also need to change walsender so that it periodically sends the heartbeat
message, like walreceiver does each wal_receiver_status_interval. Otherwise,
walreceiver will detect the timeout wrongly whenever there is no traffic in the
master.

Doesn't current keepalive message from walsender will suffice that need?

With Regards,
Amit Kapila.

On Sat, Sep 15, 2012 at 4:26 PM, Amit kapila <amit.kapila@huawei.com> wrote:

On Saturday, September 15, 2012 11:27 AM Fujii Masao wrote:
On Fri, Sep 14, 2012 at 10:01 PM, Amit kapila <amit.kapila@huawei.com> wrote:

On Thursday, September 13, 2012 10:57 PM Fujii Masao
On Thu, Sep 13, 2012 at 1:22 PM, Amit Kapila <amit.kapila@huawei.com> wrote:

On Wednesday, September 12, 2012 10:15 PM Fujii Masao
On Wed, Sep 12, 2012 at 8:54 PM, <amit.kapila@huawei.com> wrote:

The following bug has been logged on the website:

I would like to implement such feature for walreceiver, but there is one
confusion that whether to use
same configuration parameter(replication_timeout) for walrecevier as for
master or introduce a new
configuration parameter (receiver_replication_timeout).

I like the latter. I believe some users want to set the different
timeout values,
for example, in the case where the master and standby servers are placed in
the same room, but cascaded standby is placed in other continent.

Thank you for your suggestion. I have implemented as per your suggestion to have separate timeout parameter for walreceiver.
The main changes are:
1. Introduce a new configuration parameter wal_receiver_replication_timeout for walreceiver.
2. In function WalReceiverMain(), check if there is no communication till wal_receiver_replication_timeout, exit the walreceiver.
This is same as walsender functionality.

As this is a feature, So I am uploading the attached patch in coming CommitFest.

Suggestions/Comments?

You also need to change walsender so that it periodically sends the heartbeat
message, like walreceiver does each wal_receiver_status_interval. Otherwise,
walreceiver will detect the timeout wrongly whenever there is no traffic in the
master.

Doesn't current keepalive message from walsender will suffice that need?

No. Though the keepalive interval should be smaller than the timeout,
IIRC there is
no way to specify the keepalive interval now.

Regards,

--
Fujii Masao

On Sunday, September 16, 2012 12:14 AM Fujii Masao wrote:
On Sat, Sep 15, 2012 at 4:26 PM, Amit kapila <amit.kapila@huawei.com> wrote:

On Saturday, September 15, 2012 11:27 AM Fujii Masao wrote:
On Fri, Sep 14, 2012 at 10:01 PM, Amit kapila <amit.kapila@huawei.com> wrote:

On Thursday, September 13, 2012 10:57 PM Fujii Masao
On Thu, Sep 13, 2012 at 1:22 PM, Amit Kapila <amit.kapila@huawei.com> wrote:

On Wednesday, September 12, 2012 10:15 PM Fujii Masao
On Wed, Sep 12, 2012 at 8:54 PM, <amit.kapila@huawei.com> wrote:

The following bug has been logged on the website:

I would like to implement such feature for walreceiver, but there is one
confusion that whether to use
same configuration parameter(replication_timeout) for walrecevier as for
master or introduce a new
configuration parameter (receiver_replication_timeout).

I like the latter. I believe some users want to set the different
timeout values,
for example, in the case where the master and standby servers are placed in
the same room, but cascaded standby is placed in other continent.

Thank you for your suggestion. I have implemented as per your suggestion to have separate timeout parameter for walreceiver.
The main changes are:
1. Introduce a new configuration parameter wal_receiver_replication_timeout for walreceiver.
2. In function WalReceiverMain(), check if there is no communication till wal_receiver_replication_timeout, exit the walreceiver.
This is same as walsender functionality.

As this is a feature, So I am uploading the attached patch in coming CommitFest.

Suggestions/Comments?

You also need to change walsender so that it periodically sends the heartbeat
message, like walreceiver does each wal_receiver_status_interval. Otherwise,
walreceiver will detect the timeout wrongly whenever there is no traffic in the
master.

Doesn't current keepalive message from walsender will suffice that need?

No. Though the keepalive interval should be smaller than the timeout,
IIRC there is
no way to specify the keepalive interval now.

Currently AFAICS in the code on idle system, it should send keepalive after 10s which is hardcoded value as sleeptime.
You are right that if its not configurable, and somebody configures replication_timeout as value lower than 10s then the logic will fail.

So is it okay if a new config parameter similar to wal_receiver_status_interval be added and map it directly to sleeptime in the current code.
There will be no need for any new heartbeat message, existing keepalive will sufice that purpose.

With Regards,
Amit Kapila.

On Sunday, September 16, 2012 12:14 AM Fujii Masao wrote:
On Sat, Sep 15, 2012 at 4:26 PM, Amit kapila <amit.kapila@huawei.com> wrote:

On Saturday, September 15, 2012 11:27 AM Fujii Masao wrote:
On Fri, Sep 14, 2012 at 10:01 PM, Amit kapila <amit.kapila@huawei.com>

wrote:

On Thursday, September 13, 2012 10:57 PM Fujii Masao
On Thu, Sep 13, 2012 at 1:22 PM, Amit Kapila <amit.kapila@huawei.com>

wrote:

On Wednesday, September 12, 2012 10:15 PM Fujii Masao
On Wed, Sep 12, 2012 at 8:54 PM, <amit.kapila@huawei.com> wrote:

The following bug has been logged on the website:

I would like to implement such feature for walreceiver, but there is

one

confusion that whether to use
same configuration parameter(replication_timeout) for walrecevier as

for

master or introduce a new
configuration parameter (receiver_replication_timeout).

I like the latter. I believe some users want to set the different
timeout values,
for example, in the case where the master and standby servers are

placed in

the same room, but cascaded standby is placed in other continent.

Thank you for your suggestion. I have implemented as per your

suggestion to have separate timeout parameter for walreceiver.

The main changes are:
1. Introduce a new configuration parameter

wal_receiver_replication_timeout for walreceiver.

2. In function WalReceiverMain(), check if there is no communication

till wal_receiver_replication_timeout, exit the walreceiver.

This is same as walsender functionality.

As this is a feature, So I am uploading the attached patch in coming

CommitFest.

Suggestions/Comments?

You also need to change walsender so that it periodically sends the

heartbeat

message, like walreceiver does each wal_receiver_status_interval.

Otherwise,

walreceiver will detect the timeout wrongly whenever there is no traffic

in the

master.

Doesn't current keepalive message from walsender will suffice that need?

No. Though the keepalive interval should be smaller than the timeout,
IIRC there is
no way to specify the keepalive interval now.

To define the behavior correctly, according to me there are 2 options now:

Approach-1 :
Document that both(sender and receiver) the timeout parameters should be
greater than wal_receiver_status_interval.
If both are greater, then I think it might never timeout due to Idle.

Approach-2 :
Provide a variable wal_send_status_interval, such that if this is 0, then
the current behavior would prevail and if its non-zero then KeepAlive
message would be send maximum after that time.
The modified code of WALSendLoop will be as follows:

TimestampTz timeout = 0;
long sleeptime = 10000; /* 10 s */
int wakeEvents;

/* sleeptime should be equal to wal send interval if
it is not zero otherwise default as 10 sec*/
if (wal_send_status_interval > 0)
{
sleeptime = wal_send_status_interval;
}

wakeEvents = WL_LATCH_SET | WL_POSTMASTER_DEATH |
WL_SOCKET_READABLE | WL_TIMEOUT;

if (pq_is_send_pending())
wakeEvents |= WL_SOCKET_WRITEABLE;
else if (wal_send_status_interval > 0)
{
WalSndKeepalive(output_message);
/* Try to flush pending output to the client
*/
if (pq_flush_if_writable() != 0)
break;
}

/* Determine time until replication timeout */
if (replication_timeout > 0)
{
timeout =
TimestampTzPlusMilliseconds(last_reply_timestamp,

replication_timeout);

if (wal_send_status_interval <= 0)
{
sleeptime = 1 + (replication_timeout
/ 10);
}
}

/* Sleep until something happens or replication
timeout */
WaitLatchOrSocket(&MyWalSnd->latch, wakeEvents,
MyProcPort->sock,
sleeptime);

/*
* Check for replication timeout. Note we ignore
the corner case
* possibility that the client replied just as we
reached the
* timeout ... he's supposed to reply *before* that.

*/
if (replication_timeout > 0 &&
GetCurrentTimestamp() >= timeout)
{
/*
* Since typically expiration of replication
timeout means
* communication problem, we don't send the
error message to
* the standby.
*/
ereport(COMMERROR,
(errmsg("terminating
walsender process due to replication timeout")));
break;
}
}

Which way you think is better or you have any other idea to handle.

With Regards,
Amit Kapila.

On Mon, Sep 17, 2012 at 4:03 PM, Amit Kapila <amit.kapila@huawei.com> wrote:

To define the behavior correctly, according to me there are 2 options now:

Approach-1 :
Document that both(sender and receiver) the timeout parameters should be
greater than wal_receiver_status_interval.
If both are greater, then I think it might never timeout due to Idle.

In this approach, keepalive messages are sent each wal_receiver_status_interval?

Approach-2 :
Provide a variable wal_send_status_interval, such that if this is 0, then
the current behavior would prevail and if its non-zero then KeepAlive
message would be send maximum after that time.
The modified code of WALSendLoop will be as follows:

<snip>

Which way you think is better or you have any other idea to handle.

I think #2 is better because it's more intuitive to a user.

Regards,

--
Fujii Masao

On Tuesday, September 18, 2012 6:03 PM Fujii Masao wrote:
On Mon, Sep 17, 2012 at 4:03 PM, Amit Kapila <amit.kapila@huawei.com> wrote:

To define the behavior correctly, according to me there are 2 options

now:

Approach-1 :
Document that both(sender and receiver) the timeout parameters should be
greater than wal_receiver_status_interval.
If both are greater, then I think it might never timeout due to Idle.

In this approach, keepalive messages are sent each

wal_receiver_status_interval?
wal_receiver_status_interval or sleeptime whichever is smaller.

Approach-2 :
Provide a variable wal_send_status_interval, such that if this is 0, then
the current behavior would prevail and if its non-zero then KeepAlive
message would be send maximum after that time.
The modified code of WALSendLoop will be as follows:

<snip>

Which way you think is better or you have any other idea to handle.

I think #2 is better because it's more intuitive to a user.

I shall update the Patch as per Approach-2 and upload the same.

With Regards,
Amit Kapila.

On Tuesday, September 18, 2012 6:02 PM Fujii Masao wrote:
On Mon, Sep 17, 2012 at 4:03 PM, Amit Kapila <amit.kapila@huawei.com> wrote:

Approach-2 :
Provide a variable wal_send_status_interval, such that if this is 0, then
the current behavior would prevail and if its non-zero then KeepAlive
message would be send maximum after that time.
The modified code of WALSendLoop will be as follows:

<snip>

Which way you think is better or you have any other idea to handle.

I think #2 is better because it's more intuitive to a user.

Please find a patch attached for implementation of Approach-2.

With Regards,
Amit Kapila.

On 21.09.2012 14:18, Amit kapila wrote:

On Tuesday, September 18, 2012 6:02 PM Fujii Masao wrote:
On Mon, Sep 17, 2012 at 4:03 PM, Amit Kapila<amit.kapila@huawei.com> wrote:

Approach-2 :
Provide a variable wal_send_status_interval, such that if this is 0, then
the current behavior would prevail and if its non-zero then KeepAlive
message would be send maximum after that time.
The modified code of WALSendLoop will be as follows:

<snip>

Which way you think is better or you have any other idea to handle.

I think #2 is better because it's more intuitive to a user.

Please find a patch attached for implementation of Approach-2.

Hmm, I think we need to step back a bit. I've never liked the way
replication_timeout works, where it's the user's responsibility to set
wal_receiver_status_interval < replication_timeout. It's not very
user-friendly. I'd rather not copy that same design to this walreceiver
timeout. If there's two different timeouts like that, it's even worse,
because it's easy to confuse the two.

So let's think how this should ideally work from a user's point of view.
I think there should be just two settings: walsender_timeout and
walreceiver_timeout. walsender_timeout specifies how long a walsender
will keep a connection open if it doesn't hear from the walreceiver, and
walreceiver_timeout is the same for walreceiver. The system should
figure out itself how often to send keepalive messages so that those
timeouts are not reached.

In walsender, after half of walsender_timeout has elapsed and we haven't
received anything from the client, the walsender process should send a
"ping" message to the client. Whenever the client receives a Ping, it
replies. The walreceiver does the same; when half of walreceiver_timeout
has elapsed, send a Ping message to the server. Each Ping-Pong roundtrip
resets the timer in both ends, regardless of which side initiated it, so
if e.g walsender_timeout < walreceiver_timeout, the client will never
have to initiate a Ping message, because walsender will always reach the
walsender_timeout/2 point first and initiate the heartbeat message.

The Ping/Pong messages don't necessarily need to be new message types,
we can use the message types we currently have, perhaps with an
additional flag attached to them, to request the other side to reply
immediately.

- Heikki

On Mon, Oct 1, 2012 at 6:38 AM, Heikki Linnakangas
<hlinnakangas@vmware.com> wrote:

Hmm, I think we need to step back a bit. I've never liked the way
replication_timeout works, where it's the user's responsibility to set
wal_receiver_status_interval < replication_timeout. It's not very
user-friendly. I'd rather not copy that same design to this walreceiver
timeout. If there's two different timeouts like that, it's even worse,
because it's easy to confuse the two.

I agree, but also note that wal_receiver_status_interval serves
another user-visible purpose as well.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Oct 1, 2012 at 7:38 PM, Heikki Linnakangas
<hlinnakangas@vmware.com> wrote:

Hmm, I think we need to step back a bit. I've never liked the way
replication_timeout works, where it's the user's responsibility to set
wal_receiver_status_interval < replication_timeout. It's not very
user-friendly. I'd rather not copy that same design to this walreceiver
timeout. If there's two different timeouts like that, it's even worse,
because it's easy to confuse the two.

Agreed.

I'd like to specify the replication timeout like we do TCP keepalives, i.e.,
what about introducing something like following parameters?

walsender_keepalives_idle
walsender_keepalives_interval
walsender_keeaplives_count
walreceiver_keepalives_idle
walreceiver_keepalives_interval
walreceiver_keepalives_count

I believe many users are basically familiar with TCP keepalives and how to
specify it. So I think that this approach would be intuitive to users. Also
this approach includes your proposal. If you specify

walsender_keepalives_idle = walsender_timeout / 2
walsender_keepalives_interval = -1 (disable; Ping is never sent
again if there is no reply after first Ping is sent)
walsender_keepalives_count = 1

the replication timeout works as you proposed. But of course the downside
of this approach is that the number of parameter for replication timeout is
increased from two (replication_timeout and
wal_receiver_status_interval) to six,
and those parameters are confusingly similar to existing
tcp_keepalives parameters,
which might cause another confusion to users. One idea to solve this problem is
to use existing tcp_keepalives paramters values for the replication timeout.

Regards,

--
Fujii Masao

On Mon, Oct 1, 2012 at 12:57 PM, Fujii Masao <masao.fujii@gmail.com> wrote:

I believe many users are basically familiar with TCP keepalives and how to
specify it. So I think that this approach would be intuitive to users.

My experience is that many users are unfamiliar with TCP keepalives
and that when given the options they tend to do it wrong. I think a
simpler system would be better.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Excerpts from Robert Haas's message of lun oct 01 21:02:54 -0300 2012:

On Mon, Oct 1, 2012 at 12:57 PM, Fujii Masao <masao.fujii@gmail.com> wrote:

I believe many users are basically familiar with TCP keepalives and how to
specify it. So I think that this approach would be intuitive to users.

My experience is that many users are unfamiliar with TCP keepalives
and that when given the options they tend to do it wrong. I think a
simpler system would be better.

+1

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services