pgsql: Add new replication mode synchronous_commit = 'write'.
Add new replication mode synchronous_commit = 'write'.
Replication occurs only to memory on standby, not to disk,
so provides additional performance if user wishes to
reduce durability level slightly. Adds concept of multiple
independent sync rep queues.
Fujii Masao and Simon Riggs
Branch
------
master
Details
-------
http://git.postgresql.org/pg/commitdiff/443b4821f1649bc617c5ce1f6f3ffc65842a8930
Modified Files
--------------
doc/src/sgml/config.sgml | 18 +++-
doc/src/sgml/high-availability.sgml | 16 +++-
src/backend/replication/syncrep.c | 112 ++++++++++++++++++---------
src/backend/replication/walsender.c | 3 +-
src/backend/utils/misc/guc.c | 5 +-
src/include/access/xact.h | 1 +
src/include/replication/syncrep.h | 13 +++-
src/include/replication/walsender_private.h | 8 +-
8 files changed, 124 insertions(+), 52 deletions(-)
On Tue, Jan 24, 2012 at 3:22 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
Add new replication mode synchronous_commit = 'write'.
Replication occurs only to memory on standby, not to disk,
so provides additional performance if user wishes to
reduce durability level slightly. Adds concept of multiple
independent sync rep queues.Fujii Masao and Simon Riggs
i guess, you should add the new value in postgresql.conf too.
just a question, why not add a flush value and make it behave like on?
actually sync rep is on in both cases and having the different names
makes more easy to unserstand what is happening.
i only want to keep on for compatibility but i wouldn't mind if we
remove it in favor of more descriptive names.
or there's any difference between on and flush that i'm failing to see?
--
Jaime Casanova www.2ndQuadrant.com
Professional PostgreSQL: Soporte 24x7 y capacitación
On Wed, Jan 25, 2012 at 5:35 AM, Jaime Casanova <jaime@2ndquadrant.com> wrote:
On Tue, Jan 24, 2012 at 3:22 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
Add new replication mode synchronous_commit = 'write'.
Replication occurs only to memory on standby, not to disk,
so provides additional performance if user wishes to
reduce durability level slightly. Adds concept of multiple
independent sync rep queues.Fujii Masao and Simon Riggs
i guess, you should add the new value in postgresql.conf too.
Yes, I forgot to do that. Patch attached.
just a question, why not add a flush value and make it behave like on?
actually sync rep is on in both cases and having the different names
makes more easy to unserstand what is happening.
i only want to keep on for compatibility but i wouldn't mind if we
remove it in favor of more descriptive names.
Also we should add "async" as an alias for "off"?
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
Attachments:
nonameapplication/octet-stream; name=nonameDownload
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 400c52b..8fab4c3 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -158,7 +158,7 @@
#wal_level = minimal # minimal, archive, or hot_standby
# (change requires restart)
#fsync = on # turns forced synchronization on or off
-#synchronous_commit = on # synchronization level; on, off, or local
+#synchronous_commit = on # synchronization level; on, off, write, or local
#wal_sync_method = fsync # the default is the first option
# supported by the operating system:
# open_datasync
On Wed, Jan 25, 2012 at 1:23 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
On Wed, Jan 25, 2012 at 5:35 AM, Jaime Casanova <jaime@2ndquadrant.com> wrote:
On Tue, Jan 24, 2012 at 3:22 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
Add new replication mode synchronous_commit = 'write'.
Replication occurs only to memory on standby, not to disk,
so provides additional performance if user wishes to
reduce durability level slightly. Adds concept of multiple
independent sync rep queues.Fujii Masao and Simon Riggs
i guess, you should add the new value in postgresql.conf too.
Yes, I forgot to do that. Patch attached.
I think that this would be a lot more clear if we described this as
synchronous_commit = remote_write rather than just synchronous_commit
= write. Actually, the internal constant is named that way already,
but it's not exposed as such to the user.
There's a logical hierarchy here:
fully async commit ("off") < local flush only ("local") < local flush
+ remote write (currently "write") < local flush + remote flush
(currently "on") < local flush + remote apply
All of the levels except for "off" involve waiting for local flush;
the higher levels also involve waiting for something on the remote
side. But the name of the variable is just synchronous_commit, so I
thik that if the word "remote" doesn't appear anywhere in the
user-visible parameter name, it's not as clear as it could be. In
addition to renaming "write" to "remote_write", I think we might also
want to add "remote_flush" as an alias for "on".
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Wed, Jan 25, 2012 at 1:57 PM, Robert Haas <robertmhaas@gmail.com> wrote:
I think that this would be a lot more clear if we described this as
synchronous_commit = remote_write rather than just synchronous_commit
= write. Actually, the internal constant is named that way already,
but it's not exposed as such to the user.
That's something to discuss at the end of the CF when people are less
busy and we get more input.
It's an easy change whatever we decide to do.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Wed, Jan 25, 2012 at 11:12 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
On Wed, Jan 25, 2012 at 1:57 PM, Robert Haas <robertmhaas@gmail.com> wrote:
I think that this would be a lot more clear if we described this as
synchronous_commit = remote_write rather than just synchronous_commit
= write. Actually, the internal constant is named that way already,
but it's not exposed as such to the user.That's something to discuss at the end of the CF when people are less
busy and we get more input.It's an easy change whatever we decide to do.
Added this to 9.2 Open Items.
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
On Thu, Jan 26, 2012 at 1:21 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
On Wed, Jan 25, 2012 at 11:12 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
On Wed, Jan 25, 2012 at 1:57 PM, Robert Haas <robertmhaas@gmail.com> wrote:
I think that this would be a lot more clear if we described this as
synchronous_commit = remote_write rather than just synchronous_commit
= write. Actually, the internal constant is named that way already,
but it's not exposed as such to the user.That's something to discuss at the end of the CF when people are less
busy and we get more input.It's an easy change whatever we decide to do.
Added this to 9.2 Open Items.
OK, so I think it's time to decide what we want to do here. In my
view, remote_write seems a lot more clear than write (since someone
might otherwise think we were talking about a local write) and it has
the additional advantage of matching the internal naming convention -
surely, if it's write to call it SYNCHRONOUS_COMMIT_REMOTE_WRITE
inside the system, then there's not really much reason to drop the
word "remote" on the user-visible side of things. However, I just
work here. Does anyone want to make a counter-argument?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Robert Haas <robertmhaas@gmail.com> wrote:
In my view, remote_write seems a lot more clear than write
+1
I sure didn't understand it to mean remote_write when I read the
subject line.
-Kevin
On 13 April 2012 19:15, Kevin Grittner <Kevin.Grittner@wicourts.gov> wrote:
Robert Haas <robertmhaas@gmail.com> wrote:
In my view, remote_write seems a lot more clear than write
+1
I sure didn't understand it to mean remote_write when I read the
subject line.
Whatever this option value is named, it needs to be referenced in the
postgresql.conf comment for this option, as it isn't currently.
I have a question though. What happens when this is set to "write"
(or "remote_write" as proposed) but it's being used on a standalone
primary? At the moment it's not documented what level of guarantee
this would provide.
--
Thom
On 04/13/2012 08:15 PM, Kevin Grittner wrote:
Robert Haas<robertmhaas@gmail.com> wrote:
In my view, remote_write seems a lot more clear than write
+1
I sure didn't understand it to mean remote_write when I read the
subject line.
Neither did I. So definitely +1.
--
Guillaume
http://www.postgresql.fr
http://dalibo.com
On Sat, Apr 14, 2012 at 4:57 AM, Guillaume Lelarge
<guillaume@lelarge.info> wrote:
On 04/13/2012 08:15 PM, Kevin Grittner wrote:
Robert Haas<robertmhaas@gmail.com> wrote:
In my view, remote_write seems a lot more clear than write
+1
I sure didn't understand it to mean remote_write when I read the
subject line.Neither did I. So definitely +1.
+1
Patch attached.
Regards,
--
Fujii Masao
Attachments:
remote_write_v1.patchapplication/octet-stream; name=remote_write_v1.patchDownload
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
***************
*** 1626,1632 **** SET ENABLE_SEQSCAN TO OFF;
<para>
Specifies whether transaction commit will wait for WAL records
to be written to disk before the command returns a <quote>success</>
! indication to the client. Valid values are <literal>on</>, <literal>write</>,
<literal>local</>, and <literal>off</>. The default, and safe, value
is <literal>on</>. When <literal>off</>, there can be a delay between
when success is reported to the client and when the transaction is
--- 1626,1632 ----
<para>
Specifies whether transaction commit will wait for WAL records
to be written to disk before the command returns a <quote>success</>
! indication to the client. Valid values are <literal>on</>, <literal>remote_write</>,
<literal>local</>, and <literal>off</>. The default, and safe, value
is <literal>on</>. When <literal>off</>, there can be a delay between
when success is reported to the client and when the transaction is
***************
*** 1646,1652 **** SET ENABLE_SEQSCAN TO OFF;
If <xref linkend="guc-synchronous-standby-names"> is set, this
parameter also controls whether or not transaction commit will wait
for the transaction's WAL records to be flushed to disk and replicated
! to the standby server. When <literal>write</>, the commit wait will
last until a reply from the current synchronous standby indicates
it has received the commit record of the transaction to memory.
Normally this causes no data loss at the time of failover. However,
--- 1646,1652 ----
If <xref linkend="guc-synchronous-standby-names"> is set, this
parameter also controls whether or not transaction commit will wait
for the transaction's WAL records to be flushed to disk and replicated
! to the standby server. When <literal>remote_write</>, the commit wait will
last until a reply from the current synchronous standby indicates
it has received the commit record of the transaction to memory.
Normally this causes no data loss at the time of failover. However,
*** a/doc/src/sgml/high-availability.sgml
--- b/doc/src/sgml/high-availability.sgml
***************
*** 1011,1017 **** primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
</para>
<para>
! Setting <varname>synchronous_commit</> to <literal>write</> will
cause each commit to wait for confirmation that the standby has received
the commit record to memory. This provides a lower level of durability
than <literal>on</> does. However, it's a practically useful setting
--- 1011,1017 ----
</para>
<para>
! Setting <varname>synchronous_commit</> to <literal>remote_write</> will
cause each commit to wait for confirmation that the standby has received
the commit record to memory. This provides a lower level of durability
than <literal>on</> does. However, it's a practically useful setting
***************
*** 1075,1081 **** primary_conninfo = 'host=192.168.1.50 port=5432 user=foo password=foopass'
<para>
Commits made when <varname>synchronous_commit</> is set to <literal>on</>
! or <literal>write</> will wait until the synchronous standby responds. The response
may never occur if the last, or only, standby should crash.
</para>
--- 1075,1081 ----
<para>
Commits made when <varname>synchronous_commit</> is set to <literal>on</>
! or <literal>remote_write</> will wait until the synchronous standby responds. The response
may never occur if the last, or only, standby should crash.
</para>
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
***************
*** 370,381 **** static const struct config_enum_entry constraint_exclusion_options[] = {
};
/*
! * Although only "on", "off", "write", and "local" are documented, we
* accept all the likely variants of "on" and "off".
*/
static const struct config_enum_entry synchronous_commit_options[] = {
{"local", SYNCHRONOUS_COMMIT_LOCAL_FLUSH, false},
! {"write", SYNCHRONOUS_COMMIT_REMOTE_WRITE, false},
{"on", SYNCHRONOUS_COMMIT_ON, false},
{"off", SYNCHRONOUS_COMMIT_OFF, false},
{"true", SYNCHRONOUS_COMMIT_ON, true},
--- 370,381 ----
};
/*
! * Although only "on", "off", "remote_write", and "local" are documented, we
* accept all the likely variants of "on" and "off".
*/
static const struct config_enum_entry synchronous_commit_options[] = {
{"local", SYNCHRONOUS_COMMIT_LOCAL_FLUSH, false},
! {"remote_write", SYNCHRONOUS_COMMIT_REMOTE_WRITE, false},
{"on", SYNCHRONOUS_COMMIT_ON, false},
{"off", SYNCHRONOUS_COMMIT_OFF, false},
{"true", SYNCHRONOUS_COMMIT_ON, true},
*** a/src/backend/utils/misc/postgresql.conf.sample
--- b/src/backend/utils/misc/postgresql.conf.sample
***************
*** 162,168 ****
#wal_level = minimal # minimal, archive, or hot_standby
# (change requires restart)
#fsync = on # turns forced synchronization on or off
! #synchronous_commit = on # synchronization level; on, off, or local
#wal_sync_method = fsync # the default is the first option
# supported by the operating system:
# open_datasync
--- 162,169 ----
#wal_level = minimal # minimal, archive, or hot_standby
# (change requires restart)
#fsync = on # turns forced synchronization on or off
! #synchronous_commit = on # synchronization level;
! # off, local, remote_write, or on
#wal_sync_method = fsync # the default is the first option
# supported by the operating system:
# open_datasync
On Sat, Apr 14, 2012 at 10:28 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
On Sat, Apr 14, 2012 at 4:57 AM, Guillaume Lelarge
<guillaume@lelarge.info> wrote:On 04/13/2012 08:15 PM, Kevin Grittner wrote:
Robert Haas<robertmhaas@gmail.com> wrote:
In my view, remote_write seems a lot more clear than write
+1
I sure didn't understand it to mean remote_write when I read the
subject line.Neither did I. So definitely +1.
+1
Patch attached.
Thanks. Committed.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Sat, Apr 14, 2012 at 4:16 AM, Thom Brown <thom@linux.com> wrote:
On 13 April 2012 19:15, Kevin Grittner <Kevin.Grittner@wicourts.gov> wrote:
Robert Haas <robertmhaas@gmail.com> wrote:
In my view, remote_write seems a lot more clear than write
+1
I sure didn't understand it to mean remote_write when I read the
subject line.Whatever this option value is named, it needs to be referenced in the
postgresql.conf comment for this option, as it isn't currently.
Yes. The patch I've posted does this.
I have a question though. What happens when this is set to "write"
(or "remote_write" as proposed) but it's being used on a standalone
primary? At the moment it's not documented what level of guarantee
this would provide.
http://www.postgresql.org/docs/devel/static/warm-standby.html#SYNCHRONOUS-REPLICATION-HA
-----------------
Commits made when synchronous_commit is set to on or write will
wait until the synchronous standby responds. The response may
never occur if the last, or only, standby should crash.
-----------------
Is this description not enough? If not enough, how should we change
the document?
Regards,
--
Fujii Masao
On 14 April 2012 15:58, Fujii Masao <masao.fujii@gmail.com> wrote:
On Sat, Apr 14, 2012 at 4:16 AM, Thom Brown <thom@linux.com> wrote:
I have a question though. What happens when this is set to "write"
(or "remote_write" as proposed) but it's being used on a standalone
primary? At the moment it's not documented what level of guarantee
this would provide.http://www.postgresql.org/docs/devel/static/warm-standby.html#SYNCHRONOUS-REPLICATION-HA
-----------------
Commits made when synchronous_commit is set to on or write will
wait until the synchronous standby responds. The response may
never occur if the last, or only, standby should crash.
-----------------Is this description not enough? If not enough, how should we change
the document?
No, that's not what I was referring to. If you don't have a standby
(i.e. a single, isolated database cluster with no replication), and
its synchronous_commit is set to 'remote_write', what effect does that
have?
--
Thom
On Sun, Apr 15, 2012 at 12:13 AM, Thom Brown <thom@linux.com> wrote:
On 14 April 2012 15:58, Fujii Masao <masao.fujii@gmail.com> wrote:
On Sat, Apr 14, 2012 at 4:16 AM, Thom Brown <thom@linux.com> wrote:
I have a question though. What happens when this is set to "write"
(or "remote_write" as proposed) but it's being used on a standalone
primary? At the moment it's not documented what level of guarantee
this would provide.http://www.postgresql.org/docs/devel/static/warm-standby.html#SYNCHRONOUS-REPLICATION-HA
-----------------
Commits made when synchronous_commit is set to on or write will
wait until the synchronous standby responds. The response may
never occur if the last, or only, standby should crash.
-----------------Is this description not enough? If not enough, how should we change
the document?No, that's not what I was referring to. If you don't have a standby
(i.e. a single, isolated database cluster with no replication), and
its synchronous_commit is set to 'remote_write', what effect does that
have?
It's the same effect as 'on' and 'local' do, i.e., transaction commit waits
for only local WAL flush. This behavior is not documented explicitly...
How should we change the document? What about adding the following
into the explanation of synchronous_commit parameter (maybe the end
of second paragraph of that)?
-----------------
If synchronous_standby_names is not set, on, remote_write and local
provide the same synchronization level; transaction commit only waits for
local flush.
-----------------
Regards,
--
Fujii Masao
On 16 April 2012 17:21, Fujii Masao <masao.fujii@gmail.com> wrote:
On Sun, Apr 15, 2012 at 12:13 AM, Thom Brown <thom@linux.com> wrote:
No, that's not what I was referring to. If you don't have a standby
(i.e. a single, isolated database cluster with no replication), and
its synchronous_commit is set to 'remote_write', what effect does that
have?It's the same effect as 'on' and 'local' do, i.e., transaction commit waits
for only local WAL flush. This behavior is not documented explicitly...
How should we change the document? What about adding the following
into the explanation of synchronous_commit parameter (maybe the end
of second paragraph of that)?-----------------
If synchronous_standby_names is not set, on, remote_write and local
provide the same synchronization level; transaction commit only waits for
local flush.
-----------------
Yes, that sounds fine.
--
Thom
On Tue, Apr 17, 2012 at 9:52 PM, Thom Brown <thom@linux.com> wrote:
On 16 April 2012 17:21, Fujii Masao <masao.fujii@gmail.com> wrote:
On Sun, Apr 15, 2012 at 12:13 AM, Thom Brown <thom@linux.com> wrote:
No, that's not what I was referring to. If you don't have a standby
(i.e. a single, isolated database cluster with no replication), and
its synchronous_commit is set to 'remote_write', what effect does that
have?It's the same effect as 'on' and 'local' do, i.e., transaction commit waits
for only local WAL flush. This behavior is not documented explicitly...
How should we change the document? What about adding the following
into the explanation of synchronous_commit parameter (maybe the end
of second paragraph of that)?-----------------
If synchronous_standby_names is not set, on, remote_write and local
provide the same synchronization level; transaction commit only waits for
local flush.
-----------------Yes, that sounds fine.
Okay, patch attached.
Regards,
--
Fujii Masao
Attachments:
synchronous_commit_doc_v1.patchapplication/octet-stream; name=synchronous_commit_doc_v1.patchDownload
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
***************
*** 1662,1667 **** SET ENABLE_SEQSCAN TO OFF;
--- 1662,1670 ----
to allow the transaction to commit asynchronously. However, the
special value <literal>local</> is available for transactions that
wish to wait for local flush to disk, but not synchronous replication.
+ If <varname>synchronous_standby_names</> is not set, <literal>on</>,
+ <literal>remote_write</> and <literal>local</> provide the same
+ synchronization level; transaction commit only waits for local flush.
</para>
<para>
This parameter can be changed at any time; the behavior for any
On Tue, Apr 17, 2012 at 3:29 PM, Fujii Masao <masao.fujii@gmail.com> wrote:
Okay, patch attached.
Committed.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company