Quorum commit for multiple synchronous replication.

Started by Masahiko Sawadaover 9 years ago128 messageshackers
Jump to latest
#1Masahiko Sawada
sawada.mshk@gmail.com

Hi all,

In 9.6 development cycle, we had been discussed about configuration
syntax for a long time while considering expanding.
As a result, we had new dedicated language for multiple synchronous
replication, but it supports only priority method.
We know that quorum commit is very useful for many users and can
expand dedicated language easily for quorum commit.
So I'd like to propose quorum commit for multiple synchronous replication here.

The followings are changes attached patches made.
- Add new syntax 'Any N ( node1, node2, ... )' to
synchornous_standby_names for quorum commit.
- In quorum commit, the master can return commit to client after
received ACK from *at least* any N servers of listed standbys.
- sync_priority of all listed servers are same, 1.
- Add regression test for quorum commit.

I was thinking that the syntax for quorum method would use '[ ... ]'
but it will be confused with '( ... )' priority method used.
001 patch adds 'Any N ( ... )' style syntax but I know that we still
might need to discuss about better syntax, discussion is very welcome.
Attached draft patch, please give me feedback.

Regards,

--
Masahiko Sawada

Attachments:

000_quorum_commit_v1.patchtext/x-patch; charset=US-ASCII; name=000_quorum_commit_v1.patchDownload+224-31
001_add_regression_test_v1.patchtext/x-patch; charset=US-ASCII; name=001_add_regression_test_v1.patchDownload+23-1
#2Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#1)
Re: Quorum commit for multiple synchronous replication.

On Wed, Aug 3, 2016 at 2:52 PM, Masahiko Sawada <sawada.mshk@gmail.com> wrote:

I was thinking that the syntax for quorum method would use '[ ... ]'
but it will be confused with '( ... )' priority method used.
001 patch adds 'Any N ( ... )' style syntax but I know that we still
might need to discuss about better syntax, discussion is very welcome.
Attached draft patch, please give me feedback.

I am +1 for using either "{}" or "[]" to define a quorum set, and -1
for the addition of a keyword in front of the integer defining for how
many nodes server need to wait for.

-    foreach(cell, sync_standbys)
+    foreach (cell, sync_standbys)
     {
-        WalSnd       *walsnd = &WalSndCtl->walsnds[lfirst_int(cell)];
+        WalSnd *walsnd = &WalSndCtl->walsnds[lfirst_int(cell)];
This patch has some noise.
-- 
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Michael Paquier (#2)
Re: Quorum commit for multiple synchronous replication.

On Wed, Aug 3, 2016 at 3:05 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:

On Wed, Aug 3, 2016 at 2:52 PM, Masahiko Sawada <sawada.mshk@gmail.com> wrote:

I was thinking that the syntax for quorum method would use '[ ... ]'
but it will be confused with '( ... )' priority method used.
001 patch adds 'Any N ( ... )' style syntax but I know that we still
might need to discuss about better syntax, discussion is very welcome.
Attached draft patch, please give me feedback.

I am +1 for using either "{}" or "[]" to define a quorum set, and -1
for the addition of a keyword in front of the integer defining for how
many nodes server need to wait for.

Thank you for reply.
"{}" or "[]" are not bad but because these are not intuitive, I
thought that it will be hard for uses to use different method for each
purpose.

-    foreach(cell, sync_standbys)
+    foreach (cell, sync_standbys)
{
-        WalSnd       *walsnd = &WalSndCtl->walsnds[lfirst_int(cell)];
+        WalSnd *walsnd = &WalSndCtl->walsnds[lfirst_int(cell)];
This patch has some noise.

Will fix.

--
Regards,

--
Masahiko Sawada

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4Petr Jelinek
petr@2ndquadrant.com
In reply to: Masahiko Sawada (#3)
Re: Quorum commit for multiple synchronous replication.

On 04/08/16 06:40, Masahiko Sawada wrote:

On Wed, Aug 3, 2016 at 3:05 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:

On Wed, Aug 3, 2016 at 2:52 PM, Masahiko Sawada <sawada.mshk@gmail.com> wrote:

I was thinking that the syntax for quorum method would use '[ ... ]'
but it will be confused with '( ... )' priority method used.
001 patch adds 'Any N ( ... )' style syntax but I know that we still
might need to discuss about better syntax, discussion is very welcome.
Attached draft patch, please give me feedback.

I am +1 for using either "{}" or "[]" to define a quorum set, and -1
for the addition of a keyword in front of the integer defining for how
many nodes server need to wait for.

Thank you for reply.
"{}" or "[]" are not bad but because these are not intuitive, I
thought that it will be hard for uses to use different method for each
purpose.

I think the "any" keyword is more explicit and understandable, also
closer to SQL. So I would be in favor of doing that.

--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#5Fujii Masao
masao.fujii@gmail.com
In reply to: Petr Jelinek (#4)
Re: Quorum commit for multiple synchronous replication.

On Sat, Aug 6, 2016 at 6:36 PM, Petr Jelinek <petr@2ndquadrant.com> wrote:

On 04/08/16 06:40, Masahiko Sawada wrote:

On Wed, Aug 3, 2016 at 3:05 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:

On Wed, Aug 3, 2016 at 2:52 PM, Masahiko Sawada <sawada.mshk@gmail.com>
wrote:

I was thinking that the syntax for quorum method would use '[ ... ]'
but it will be confused with '( ... )' priority method used.
001 patch adds 'Any N ( ... )' style syntax but I know that we still
might need to discuss about better syntax, discussion is very welcome.
Attached draft patch, please give me feedback.

I am +1 for using either "{}" or "[]" to define a quorum set, and -1
for the addition of a keyword in front of the integer defining for how
many nodes server need to wait for.

Thank you for reply.
"{}" or "[]" are not bad but because these are not intuitive, I
thought that it will be hard for uses to use different method for each
purpose.

I think the "any" keyword is more explicit and understandable, also closer
to SQL. So I would be in favor of doing that.

+1

Also I like the following Simon's idea.

/messages/by-id/CANP8+jLHfBVv_pW6grASNUpW+bdk5DcTu7GWpNAP-+-ZWvKT6w@mail.gmail.com
-----------------------
* first k (n1, n2, n3) – does the same as k (n1, n2, n3) does now
* any k (n1, n2, n3) – would release waiters as soon as we have the
responses from k out of N standbys. “any k” would be faster, so is
desirable for performance and resilience
-----------------------

Regards,

--
Fujii Masao

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6Simon Riggs
simon@2ndQuadrant.com
In reply to: Fujii Masao (#5)
Re: Quorum commit for multiple synchronous replication.

On 29 August 2016 at 14:52, Fujii Masao <masao.fujii@gmail.com> wrote:

On Sat, Aug 6, 2016 at 6:36 PM, Petr Jelinek <petr@2ndquadrant.com> wrote:

On 04/08/16 06:40, Masahiko Sawada wrote:

On Wed, Aug 3, 2016 at 3:05 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:

On Wed, Aug 3, 2016 at 2:52 PM, Masahiko Sawada <sawada.mshk@gmail.com>
wrote:

I was thinking that the syntax for quorum method would use '[ ... ]'
but it will be confused with '( ... )' priority method used.
001 patch adds 'Any N ( ... )' style syntax but I know that we still
might need to discuss about better syntax, discussion is very welcome.
Attached draft patch, please give me feedback.

I am +1 for using either "{}" or "[]" to define a quorum set, and -1
for the addition of a keyword in front of the integer defining for how
many nodes server need to wait for.

Thank you for reply.
"{}" or "[]" are not bad but because these are not intuitive, I
thought that it will be hard for uses to use different method for each
purpose.

I think the "any" keyword is more explicit and understandable, also closer
to SQL. So I would be in favor of doing that.

+1

Also I like the following Simon's idea.

/messages/by-id/CANP8+jLHfBVv_pW6grASNUpW+bdk5DcTu7GWpNAP-+-ZWvKT6w@mail.gmail.com
-----------------------
* first k (n1, n2, n3) – does the same as k (n1, n2, n3) does now
* any k (n1, n2, n3) – would release waiters as soon as we have the
responses from k out of N standbys. “any k” would be faster, so is
desirable for performance and resilience
-----------------------

+1

"synchronous_method" -> "synchronization_method"

I'm concerned about the performance of this code. Can we work out a
way of measuring it, so we can judge how successful we are at
releasing waiters quickly? Thanks

For 9.6 we implemented something that allows the DBA to define how
slow programs are. Previously, since 9.1 this was something specified
on the application side. I would like to put it back that way, so we
end up with a parameter on client e.g. commit_quorum = k. Forget the
exact parameters/user API for now, but I'd like to allow the code to
work with user defined settings. Thanks.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#7Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Simon Riggs (#6)
Re: Quorum commit for multiple synchronous replication.

On Tue, Sep 6, 2016 at 11:08 PM, Simon Riggs <simon@2ndquadrant.com> wrote:

On 29 August 2016 at 14:52, Fujii Masao <masao.fujii@gmail.com> wrote:

On Sat, Aug 6, 2016 at 6:36 PM, Petr Jelinek <petr@2ndquadrant.com> wrote:

On 04/08/16 06:40, Masahiko Sawada wrote:

On Wed, Aug 3, 2016 at 3:05 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:

On Wed, Aug 3, 2016 at 2:52 PM, Masahiko Sawada <sawada.mshk@gmail.com>
wrote:

I was thinking that the syntax for quorum method would use '[ ... ]'
but it will be confused with '( ... )' priority method used.
001 patch adds 'Any N ( ... )' style syntax but I know that we still
might need to discuss about better syntax, discussion is very welcome.
Attached draft patch, please give me feedback.

I am +1 for using either "{}" or "[]" to define a quorum set, and -1
for the addition of a keyword in front of the integer defining for how
many nodes server need to wait for.

Thank you for reply.
"{}" or "[]" are not bad but because these are not intuitive, I
thought that it will be hard for uses to use different method for each
purpose.

I think the "any" keyword is more explicit and understandable, also closer
to SQL. So I would be in favor of doing that.

+1

Also I like the following Simon's idea.

/messages/by-id/CANP8+jLHfBVv_pW6grASNUpW+bdk5DcTu7GWpNAP-+-ZWvKT6w@mail.gmail.com
-----------------------
* first k (n1, n2, n3) – does the same as k (n1, n2, n3) does now
* any k (n1, n2, n3) – would release waiters as soon as we have the
responses from k out of N standbys. “any k” would be faster, so is
desirable for performance and resilience
-----------------------

+1

"synchronous_method" -> "synchronization_method"

Thanks, will fix.

I'm concerned about the performance of this code. Can we work out a
way of measuring it, so we can judge how successful we are at
releasing waiters quickly? Thanks

I will measure the performance effect of this code.
I'm expecting that performances are,
'first 1 (n1, n2)' > 'any 1(n1, n2)' > 'first 2(n1, n2)'
'first 1 (n1, n2)' will be highest throughput.

For 9.6 we implemented something that allows the DBA to define how
slow programs are. Previously, since 9.1 this was something specified
on the application side. I would like to put it back that way, so we
end up with a parameter on client e.g. commit_quorum = k. Forget the
exact parameters/user API for now, but I'd like to allow the code to
work with user defined settings. Thanks.

I see. The parameter on client should effect for priority method as well.
And similar to synchronous_commit, the client can specify the how much
standbys the master waits to commit for according to synchronization
method, even if s_s_names is defined.

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#8Josh Berkus
josh@agliodbs.com
In reply to: Masahiko Sawada (#1)
Re: Quorum commit for multiple synchronous replication.

On 08/29/2016 06:52 AM, Fujii Masao wrote:

Also I like the following Simon's idea.

/messages/by-id/CANP8+jLHfBVv_pW6grASNUpW+bdk5DcTu7GWpNAP-+-ZWvKT6w@mail.gmail.com
-----------------------
* first k (n1, n2, n3) – does the same as k (n1, n2, n3) does now
* any k (n1, n2, n3) – would release waiters as soon as we have the
responses from k out of N standbys. “any k” would be faster, so is
desirable for performance and resilience

What are we going to do for backwards compatibility, here?

So, here's the dilemma:

If we want to keep backwards compatibility with 9.6, then:

"k (n1, n2, n3)" == "first k (n1, n2, n3)"

However, "first k" is not what most users will want, most of the time;
users of version 13, years from now, will be getting constantly confused
by "first k" behavior when they wanted quorum. So the sensible default
would be:

"k (n1, n2, n3)" == "any k (n1, n2, n3)"

... however, that will break backwards compatibility. Thoughts?

My $0.02 is that we break backwards compat somehow and document the heck
out of it.

--
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#9Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Masahiko Sawada (#7)
Re: Quorum commit for multiple synchronous replication.

On Wed, Sep 7, 2016 at 12:47 AM, Masahiko Sawada <sawada.mshk@gmail.com> wrote:

On Tue, Sep 6, 2016 at 11:08 PM, Simon Riggs <simon@2ndquadrant.com> wrote:

On 29 August 2016 at 14:52, Fujii Masao <masao.fujii@gmail.com> wrote:

On Sat, Aug 6, 2016 at 6:36 PM, Petr Jelinek <petr@2ndquadrant.com> wrote:

On 04/08/16 06:40, Masahiko Sawada wrote:

On Wed, Aug 3, 2016 at 3:05 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:

On Wed, Aug 3, 2016 at 2:52 PM, Masahiko Sawada <sawada.mshk@gmail.com>
wrote:

I was thinking that the syntax for quorum method would use '[ ... ]'
but it will be confused with '( ... )' priority method used.
001 patch adds 'Any N ( ... )' style syntax but I know that we still
might need to discuss about better syntax, discussion is very welcome.
Attached draft patch, please give me feedback.

I am +1 for using either "{}" or "[]" to define a quorum set, and -1
for the addition of a keyword in front of the integer defining for how
many nodes server need to wait for.

Thank you for reply.
"{}" or "[]" are not bad but because these are not intuitive, I
thought that it will be hard for uses to use different method for each
purpose.

I think the "any" keyword is more explicit and understandable, also closer
to SQL. So I would be in favor of doing that.

+1

Also I like the following Simon's idea.

/messages/by-id/CANP8+jLHfBVv_pW6grASNUpW+bdk5DcTu7GWpNAP-+-ZWvKT6w@mail.gmail.com
-----------------------
* first k (n1, n2, n3) – does the same as k (n1, n2, n3) does now
* any k (n1, n2, n3) – would release waiters as soon as we have the
responses from k out of N standbys. “any k” would be faster, so is
desirable for performance and resilience
-----------------------

+1

"synchronous_method" -> "synchronization_method"

Thanks, will fix.

I'm concerned about the performance of this code. Can we work out a
way of measuring it, so we can judge how successful we are at
releasing waiters quickly? Thanks

I will measure the performance effect of this code.
I'm expecting that performances are,
'first 1 (n1, n2)' > 'any 1(n1, n2)' > 'first 2(n1, n2)'
'first 1 (n1, n2)' will be highest throughput.

Sorry, that's wrong.
'any 1(n1, n2)' will be highest throughput or same as 'first 1(n1, n2)'.

For 9.6 we implemented something that allows the DBA to define how
slow programs are. Previously, since 9.1 this was something specified
on the application side. I would like to put it back that way, so we
end up with a parameter on client e.g. commit_quorum = k. Forget the
exact parameters/user API for now, but I'd like to allow the code to
work with user defined settings. Thanks.

I see. The parameter on client should effect for priority method as well.
And similar to synchronous_commit, the client can specify the how much
standbys the master waits to commit for according to synchronization
method, even if s_s_names is defined.

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#10Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Josh Berkus (#8)
Re: Quorum commit for multiple synchronous replication.

On Wed, Sep 7, 2016 at 4:03 AM, Josh Berkus <josh@agliodbs.com> wrote:

On 08/29/2016 06:52 AM, Fujii Masao wrote:

Also I like the following Simon's idea.

/messages/by-id/CANP8+jLHfBVv_pW6grASNUpW+bdk5DcTu7GWpNAP-+-ZWvKT6w@mail.gmail.com
-----------------------
* first k (n1, n2, n3) – does the same as k (n1, n2, n3) does now
* any k (n1, n2, n3) – would release waiters as soon as we have the
responses from k out of N standbys. “any k” would be faster, so is
desirable for performance and resilience

What are we going to do for backwards compatibility, here?

So, here's the dilemma:

If we want to keep backwards compatibility with 9.6, then:

"k (n1, n2, n3)" == "first k (n1, n2, n3)"

However, "first k" is not what most users will want, most of the time;
users of version 13, years from now, will be getting constantly confused
by "first k" behavior when they wanted quorum. So the sensible default
would be:

"k (n1, n2, n3)" == "any k (n1, n2, n3)"

+1.

"k (n1, n2, n3)" == "first k (n1, n2, n3)" doesn't break backward
compatibility but most users would think "k(n1, n2, n3)" as quorum
after introduced quorum.
I wish we can change the s_s_names syntax of 9.6 to "first k(n1, n2,
n3)" style before 9.6 releasing if we got consensus.

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#11Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#10)
Re: Quorum commit for multiple synchronous replication.

On Thu, Sep 8, 2016 at 6:26 PM, Masahiko Sawada <sawada.mshk@gmail.com> wrote:

"k (n1, n2, n3)" == "first k (n1, n2, n3)" doesn't break backward
compatibility but most users would think "k(n1, n2, n3)" as quorum
after introduced quorum.
I wish we can change the s_s_names syntax of 9.6 to "first k(n1, n2,
n3)" style before 9.6 releasing if we got consensus.

Considering breaking backward-compatibility in the next release does
not sound like a good idea to me for a new feature that is going to be
GA soon.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#12Vik Fearing
vik@postgresfriends.org
In reply to: Michael Paquier (#11)
Re: Quorum commit for multiple synchronous replication.

On 09/09/2016 03:28 AM, Michael Paquier wrote:

On Thu, Sep 8, 2016 at 6:26 PM, Masahiko Sawada <sawada.mshk@gmail.com> wrote:

"k (n1, n2, n3)" == "first k (n1, n2, n3)" doesn't break backward
compatibility but most users would think "k(n1, n2, n3)" as quorum
after introduced quorum.
I wish we can change the s_s_names syntax of 9.6 to "first k(n1, n2,
n3)" style before 9.6 releasing if we got consensus.

Considering breaking backward-compatibility in the next release does
not sound like a good idea to me for a new feature that is going to be
GA soon.

Indeed. I'll vote for pulling a fast one on 9.6 for this.
--
Vik Fearing +33 6 46 75 15 36
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#13Petr Jelinek
petr@2ndquadrant.com
In reply to: Vik Fearing (#12)
Re: Quorum commit for multiple synchronous replication.

On 09/09/16 08:23, Vik Fearing wrote:

On 09/09/2016 03:28 AM, Michael Paquier wrote:

On Thu, Sep 8, 2016 at 6:26 PM, Masahiko Sawada <sawada.mshk@gmail.com> wrote:

"k (n1, n2, n3)" == "first k (n1, n2, n3)" doesn't break backward
compatibility but most users would think "k(n1, n2, n3)" as quorum
after introduced quorum.
I wish we can change the s_s_names syntax of 9.6 to "first k(n1, n2,
n3)" style before 9.6 releasing if we got consensus.

Considering breaking backward-compatibility in the next release does
not sound like a good idea to me for a new feature that is going to be
GA soon.

Indeed. I'll vote for pulling a fast one on 9.6 for this.

+1

--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#14Simon Riggs
simon@2ndQuadrant.com
In reply to: Masahiko Sawada (#10)
Re: Quorum commit for multiple synchronous replication.

On 8 September 2016 at 10:26, Masahiko Sawada <sawada.mshk@gmail.com> wrote:

"k (n1, n2, n3)" == "first k (n1, n2, n3)" doesn't break backward
compatibility but most users would think "k(n1, n2, n3)" as quorum
after introduced quorum.
I wish we can change the s_s_names syntax of 9.6 to "first k(n1, n2,
n3)" style before 9.6 releasing if we got consensus.

Let's see the proposed patch, so we can evaluate the proposal.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#15Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Simon Riggs (#14)
Re: Quorum commit for multiple synchronous replication.

On Fri, Sep 9, 2016 at 6:23 PM, Simon Riggs <simon@2ndquadrant.com> wrote:

On 8 September 2016 at 10:26, Masahiko Sawada <sawada.mshk@gmail.com> wrote:

"k (n1, n2, n3)" == "first k (n1, n2, n3)" doesn't break backward
compatibility but most users would think "k(n1, n2, n3)" as quorum
after introduced quorum.
I wish we can change the s_s_names syntax of 9.6 to "first k(n1, n2,
n3)" style before 9.6 releasing if we got consensus.

Let's see the proposed patch, so we can evaluate the proposal.

Attached 2 patches.
000 patch changes syntax of s_s_names from 'k(n1, n2, n3)' to 'First k
(n1, n2,n3)' for PG9.6.
001 patch adds the quorum commit using syntax 'Any k (n1, n2,n3)' for PG10.

Since we already released 9.6RC1, I understand that it's quite hard to
change syntax of 9.6.
But considering that we support the quorum commit, this could be one
of the solutions in order to avoid breaking backward compatibility and
to provide useful user interface.
So I attached these patches.

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

Attachments:

000_change_syntax_96.patchtext/x-patch; charset=US-ASCII; name=000_change_syntax_96.patchDownload+14-7
001_quorum_commit_v2.patchtext/x-patch; charset=US-ASCII; name=001_quorum_commit_v2.patchDownload+284-56
#16Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#15)
Re: Quorum commit for multiple synchronous replication.

On Sat, Sep 17, 2016 at 2:04 AM, Masahiko Sawada <sawada.mshk@gmail.com> wrote:

Since we already released 9.6RC1, I understand that it's quite hard to
change syntax of 9.6.
But considering that we support the quorum commit, this could be one
of the solutions in order to avoid breaking backward compatibility and
to provide useful user interface.
So I attached these patches.

 standby_config:
-        standby_list                { $$ = create_syncrep_config("1", $1); }
-        | FIRST NUM '(' standby_list ')'    { $$ =
create_syncrep_config($1, $4); }
+        standby_list                        { $$ =
create_syncrep_config("1", $1, SYNC_REP_PRIORITY); }
+        | ANY NUM '(' standby_list ')'        { $$ =
create_syncrep_config($2, $4, SYNC_REP_QUORUM); }
+        | FIRST NUM '(' standby_list ')'    { $$ =
create_syncrep_config($2, $4, SYNC_REP_PRIORITY); }

Reading again the thread, it seems that my previous post [1]/messages/by-id/CAB7nPqRDvJn18e54ccNpOP1A2_iUN6-iU=4nJgmMgiAgvcSDKA@mail.gmail.com -- Michael was a bit
misunderstood. My position is to not introduce any new behavior
changes in 9.6, so we could just make the FIRST NUM grammar equivalent
to NUM.

[1]: /messages/by-id/CAB7nPqRDvJn18e54ccNpOP1A2_iUN6-iU=4nJgmMgiAgvcSDKA@mail.gmail.com -- Michael
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#17Vik Fearing
vik@postgresfriends.org
In reply to: Michael Paquier (#16)
Re: Quorum commit for multiple synchronous replication.

On 09/21/2016 08:30 AM, Michael Paquier wrote:

On Sat, Sep 17, 2016 at 2:04 AM, Masahiko Sawada <sawada.mshk@gmail.com> wrote:

Since we already released 9.6RC1, I understand that it's quite hard to
change syntax of 9.6.
But considering that we support the quorum commit, this could be one
of the solutions in order to avoid breaking backward compatibility and
to provide useful user interface.
So I attached these patches.

standby_config:
-        standby_list                { $$ = create_syncrep_config("1", $1); }
-        | FIRST NUM '(' standby_list ')'    { $$ =
create_syncrep_config($1, $4); }
+        standby_list                        { $$ =
create_syncrep_config("1", $1, SYNC_REP_PRIORITY); }
+        | ANY NUM '(' standby_list ')'        { $$ =
create_syncrep_config($2, $4, SYNC_REP_QUORUM); }
+        | FIRST NUM '(' standby_list ')'    { $$ =
create_syncrep_config($2, $4, SYNC_REP_PRIORITY); }

Reading again the thread, it seems that my previous post [1] was a bit
misunderstood. My position is to not introduce any new behavior
changes in 9.6, so we could just make the FIRST NUM grammar equivalent
to NUM.

[1]: /messages/by-id/CAB7nPqRDvJn18e54ccNpOP1A2_iUN6-iU=4nJgmMgiAgvcSDKA@mail.gmail.com

I misunderstood your intent, then. But I still stand by what I did
understand, namely that 'k (...)' should mean 'any k (...)'. It's much
more natural than having it mean 'first k (...)' and I also think it
will be more frequent in practice.
--
Vik Fearing +33 6 46 75 15 36
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#18Petr Jelinek
petr@2ndquadrant.com
In reply to: Vik Fearing (#17)
Re: Quorum commit for multiple synchronous replication.

On 21/09/16 09:18, Vik Fearing wrote:

On 09/21/2016 08:30 AM, Michael Paquier wrote:

On Sat, Sep 17, 2016 at 2:04 AM, Masahiko Sawada <sawada.mshk@gmail.com> wrote:

Since we already released 9.6RC1, I understand that it's quite hard to
change syntax of 9.6.
But considering that we support the quorum commit, this could be one
of the solutions in order to avoid breaking backward compatibility and
to provide useful user interface.
So I attached these patches.

standby_config:
-        standby_list                { $$ = create_syncrep_config("1", $1); }
-        | FIRST NUM '(' standby_list ')'    { $$ =
create_syncrep_config($1, $4); }
+        standby_list                        { $$ =
create_syncrep_config("1", $1, SYNC_REP_PRIORITY); }
+        | ANY NUM '(' standby_list ')'        { $$ =
create_syncrep_config($2, $4, SYNC_REP_QUORUM); }
+        | FIRST NUM '(' standby_list ')'    { $$ =
create_syncrep_config($2, $4, SYNC_REP_PRIORITY); }

Reading again the thread, it seems that my previous post [1] was a bit
misunderstood. My position is to not introduce any new behavior
changes in 9.6, so we could just make the FIRST NUM grammar equivalent
to NUM.

[1]: /messages/by-id/CAB7nPqRDvJn18e54ccNpOP1A2_iUN6-iU=4nJgmMgiAgvcSDKA@mail.gmail.com

I misunderstood your intent, then. But I still stand by what I did
understand, namely that 'k (...)' should mean 'any k (...)'. It's much
more natural than having it mean 'first k (...)' and I also think it
will be more frequent in practice.

I think so as well.

--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#19Robert Haas
robertmhaas@gmail.com
In reply to: Petr Jelinek (#18)
Re: Quorum commit for multiple synchronous replication.

On Wed, Sep 21, 2016 at 5:54 AM, Petr Jelinek <petr@2ndquadrant.com> wrote:

Reading again the thread, it seems that my previous post [1] was a bit
misunderstood. My position is to not introduce any new behavior
changes in 9.6, so we could just make the FIRST NUM grammar equivalent
to NUM.

[1]: /messages/by-id/CAB7nPqRDvJn18e54ccNpOP1A2_iUN6-iU=4nJgmMgiAgvcSDKA@mail.gmail.com

I misunderstood your intent, then. But I still stand by what I did
understand, namely that 'k (...)' should mean 'any k (...)'. It's much
more natural than having it mean 'first k (...)' and I also think it
will be more frequent in practice.

I think so as well.

Well, I agree, but I think making behavior changes after rc1 is a
non-starter. It's better to live with the incompatibility than to
change the behavior so close to release. At least, that's my
position. Getting the release out on time with a minimal bug count is
more important to me than a minor incompatibility in the meaning of
one GUC.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#20Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Robert Haas (#19)
Re: Quorum commit for multiple synchronous replication.

On Wed, Sep 21, 2016 at 11:22 PM, Robert Haas <robertmhaas@gmail.com> wrote:

On Wed, Sep 21, 2016 at 5:54 AM, Petr Jelinek <petr@2ndquadrant.com> wrote:

Reading again the thread, it seems that my previous post [1] was a bit
misunderstood. My position is to not introduce any new behavior
changes in 9.6, so we could just make the FIRST NUM grammar equivalent
to NUM.

[1]: /messages/by-id/CAB7nPqRDvJn18e54ccNpOP1A2_iUN6-iU=4nJgmMgiAgvcSDKA@mail.gmail.com

I misunderstood your intent, then. But I still stand by what I did
understand, namely that 'k (...)' should mean 'any k (...)'. It's much
more natural than having it mean 'first k (...)' and I also think it
will be more frequent in practice.

I think so as well.

Well, I agree, but I think making behavior changes after rc1 is a
non-starter. It's better to live with the incompatibility than to
change the behavior so close to release. At least, that's my
position. Getting the release out on time with a minimal bug count is
more important to me than a minor incompatibility in the meaning of
one GUC.

As the release team announced, it's better to postpone changing the
syntax of existing s_s_name.
I still vote for changing behaviour of existing syntax 'k (n1, n2)' to
quorum commit.
That is,
1. 'First k (n1, n2, n3)' means that the master server waits for ACKs
from k standby servers whose name appear earlier in the list.
2. 'Any k (n1, n2, n3)' means that the master server waits for ACKs
from any k listed standby servers.
3. 'n1, n2, n3' is the same as #1 with k=1.
4. '(n1, n2, n3)' is the same as #2 with k=1.

Attached updated patch.

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

Attachments:

quorum_commit_v3.patchapplication/octet-stream; name=quorum_commit_v3.patchDownload+316-62
#21Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#20)
#22Michael Paquier
michael@paquier.xyz
In reply to: Michael Paquier (#21)
#23Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Michael Paquier (#21)
#24Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#23)
#25Michael Paquier
michael@paquier.xyz
In reply to: Michael Paquier (#24)
#26Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Masahiko Sawada (#23)
#27Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#26)
#28Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Michael Paquier (#27)
#29Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#28)
#30Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Michael Paquier (#29)
#31Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Masahiko Sawada (#30)
#32Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#31)
#33Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Michael Paquier (#32)
#34Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#33)
#35Fujii Masao
masao.fujii@gmail.com
In reply to: Masahiko Sawada (#33)
#36Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Fujii Masao (#35)
#37Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#36)
#38Fujii Masao
masao.fujii@gmail.com
In reply to: Masahiko Sawada (#36)
#39Michael Paquier
michael@paquier.xyz
In reply to: Fujii Masao (#38)
#40Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Michael Paquier (#39)
#41Michael Paquier
michael@paquier.xyz
In reply to: Kyotaro Horiguchi (#40)
#42Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Michael Paquier (#41)
#43Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#42)
#44Robert Haas
robertmhaas@gmail.com
In reply to: Michael Paquier (#39)
#45Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Michael Paquier (#43)
#46Michael Paquier
michael@paquier.xyz
In reply to: Robert Haas (#44)
#47Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Michael Paquier (#46)
#48Amit Kapila
amit.kapila16@gmail.com
In reply to: Masahiko Sawada (#47)
#49Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Amit Kapila (#48)
#50Fujii Masao
masao.fujii@gmail.com
In reply to: Masahiko Sawada (#49)
#51Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Fujii Masao (#50)
#52Amit Kapila
amit.kapila16@gmail.com
In reply to: Masahiko Sawada (#51)
#53Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Amit Kapila (#52)
#54Fujii Masao
masao.fujii@gmail.com
In reply to: Kyotaro Horiguchi (#53)
#55Michael Paquier
michael@paquier.xyz
In reply to: Fujii Masao (#54)
#56Fujii Masao
masao.fujii@gmail.com
In reply to: Michael Paquier (#55)
#57Michael Paquier
michael@paquier.xyz
In reply to: Fujii Masao (#56)
#58Amit Kapila
amit.kapila16@gmail.com
In reply to: Michael Paquier (#57)
#59Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Michael Paquier (#57)
#60Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Masahiko Sawada (#59)
#61Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Kyotaro Horiguchi (#60)
#62Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#61)
#63Fujii Masao
masao.fujii@gmail.com
In reply to: Michael Paquier (#62)
#64Fujii Masao
masao.fujii@gmail.com
In reply to: Fujii Masao (#63)
#65Michael Paquier
michael@paquier.xyz
In reply to: Fujii Masao (#64)
#66Fujii Masao
masao.fujii@gmail.com
In reply to: Michael Paquier (#65)
#67Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Fujii Masao (#66)
#68Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Fujii Masao (#66)
#69Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#68)
#70Fujii Masao
masao.fujii@gmail.com
In reply to: Alvaro Herrera (#67)
#71Fujii Masao
masao.fujii@gmail.com
In reply to: Michael Paquier (#69)
#72Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Fujii Masao (#71)
#73Fujii Masao
masao.fujii@gmail.com
In reply to: Kyotaro Horiguchi (#72)
#74Noah Misch
noah@leadboat.com
In reply to: Fujii Masao (#66)
#75Fujii Masao
masao.fujii@gmail.com
In reply to: Noah Misch (#74)
#76Noah Misch
noah@leadboat.com
In reply to: Fujii Masao (#75)
#77Petr Jelinek
petr@2ndquadrant.com
In reply to: Noah Misch (#76)
#78Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Noah Misch (#76)
#79Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Masahiko Sawada (#78)
#80Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Masahiko Sawada (#78)
#81Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Kyotaro Horiguchi (#80)
#82Fujii Masao
masao.fujii@gmail.com
In reply to: Masahiko Sawada (#81)
#83Michael Paquier
michael@paquier.xyz
In reply to: Fujii Masao (#82)
#84Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Michael Paquier (#83)
#85Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Masahiko Sawada (#84)
#86Simon Riggs
simon@2ndQuadrant.com
In reply to: Fujii Masao (#82)
#87Noah Misch
noah@leadboat.com
In reply to: Noah Misch (#76)
#88Noah Misch
noah@leadboat.com
In reply to: Noah Misch (#87)
#89Fujii Masao
masao.fujii@gmail.com
In reply to: Noah Misch (#88)
#90Fujii Masao
masao.fujii@gmail.com
In reply to: Masahiko Sawada (#79)
#91Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Fujii Masao (#90)
#92Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Masahiko Sawada (#91)
#93Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Kyotaro Horiguchi (#92)
#94Fujii Masao
masao.fujii@gmail.com
In reply to: Masahiko Sawada (#93)
#95Noah Misch
noah@leadboat.com
In reply to: Fujii Masao (#89)
#96Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Noah Misch (#95)
#97Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#96)
#98Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Fujii Masao (#94)
#99Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Kyotaro Horiguchi (#98)
#100Noah Misch
noah@leadboat.com
In reply to: Masahiko Sawada (#96)
#101Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Noah Misch (#100)
#102Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Masahiko Sawada (#101)
#103Noah Misch
noah@leadboat.com
In reply to: Masahiko Sawada (#101)
#104Noah Misch
noah@leadboat.com
In reply to: Noah Misch (#103)
#105Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Kyotaro Horiguchi (#99)
#106Fujii Masao
masao.fujii@gmail.com
In reply to: Noah Misch (#104)
#107Fujii Masao
masao.fujii@gmail.com
In reply to: Masahiko Sawada (#105)
#108Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Fujii Masao (#106)
#109Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Fujii Masao (#107)
#110Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Masahiko Sawada (#108)
#111Amit Kapila
amit.kapila16@gmail.com
In reply to: Kyotaro Horiguchi (#109)
#112Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Amit Kapila (#111)
#113Fujii Masao
masao.fujii@gmail.com
In reply to: Kyotaro Horiguchi (#110)
#114Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Masahiko Sawada (#112)
#115Noah Misch
noah@leadboat.com
In reply to: Petr Jelinek (#77)
#116Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Noah Misch (#115)
#117Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#116)
#118Josh Berkus
josh@agliodbs.com
In reply to: Michael Paquier (#117)
#119Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Josh Berkus (#118)
#120Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#119)
#121Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Michael Paquier (#120)
#122Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#121)
#123Robert Haas
robertmhaas@gmail.com
In reply to: Michael Paquier (#122)
#124Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Robert Haas (#123)
#125Michael Paquier
michael@paquier.xyz
In reply to: Masahiko Sawada (#124)
#126Josh Berkus
josh@agliodbs.com
In reply to: Masahiko Sawada (#124)
#127Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Josh Berkus (#126)
#128Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Masahiko Sawada (#127)