Function to know last log write timestamp

On Mon, Aug 11, 2014 at 9:23 AM, Tatsuo Ishii <ishii@postgresql.org> wrote:

We can know the LSN of last committed WAL record on primary by using
pg_current_xlog_location(). It seems there's no API to know the time
when the WAL record was created. I would like to know standby delay by
using pg_last_xact_replay_timestamp() and such that API.

If there's no such a API, it would be useful to invent usch an API IMO.

+1

I proposed that function before, but unfortunately it failed to be applied.
But I still think that function is useful to calculate the replication delay.
The past discussion is
/messages/by-id/CAHGQGwF3ZjfuNEj5ka683KU5rQUBtSWtqFq7g1X0g34o+JXWBw@mail.gmail.com

Regards,

--
Fujii Masao

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Mon, Aug 11, 2014 at 9:23 AM, Tatsuo Ishii <ishii@postgresql.org> wrote:

We can know the LSN of last committed WAL record on primary by using
pg_current_xlog_location(). It seems there's no API to know the time
when the WAL record was created. I would like to know standby delay by
using pg_last_xact_replay_timestamp() and such that API.

If there's no such a API, it would be useful to invent usch an API IMO.

+1

I proposed that function before, but unfortunately it failed to be applied.
But I still think that function is useful to calculate the replication delay.
The past discussion is
/messages/by-id/CAHGQGwF3ZjfuNEj5ka683KU5rQUBtSWtqFq7g1X0g34o+JXWBw@mail.gmail.com

I looked into the thread briefly and found Simon and Robert gave -1
for this because of performance concern. I'm not sure if it's a actual
performance penalty or not. Maybe we need to major the penalty?

However I still think that kind of API is very useful because
replication delay is one of the big DBA's concern. Why don't we have a
switch to enable the API for DBAs who think the priority is
replication delay, over small performance penalty?

Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese:http://www.sraoss.co.jp

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Mon, Aug 11, 2014 at 10:48 AM, Tatsuo Ishii <ishii@postgresql.org> wrote:

On Mon, Aug 11, 2014 at 9:23 AM, Tatsuo Ishii <ishii@postgresql.org> wrote:

We can know the LSN of last committed WAL record on primary by using
pg_current_xlog_location(). It seems there's no API to know the time
when the WAL record was created. I would like to know standby delay by
using pg_last_xact_replay_timestamp() and such that API.

If there's no such a API, it would be useful to invent usch an API IMO.

+1

I proposed that function before, but unfortunately it failed to be applied.
But I still think that function is useful to calculate the replication delay.
The past discussion is
/messages/by-id/CAHGQGwF3ZjfuNEj5ka683KU5rQUBtSWtqFq7g1X0g34o+JXWBw@mail.gmail.com

I looked into the thread briefly and found Simon and Robert gave -1
for this because of performance concern. I'm not sure if it's a actual
performance penalty or not. Maybe we need to major the penalty?

I think that the performance penalty is negligible small because the patch
I posted before added only three stores to shared memory per commit/abort.
No time-consuming operations like lock, gettimeofday, etc were added.
Of course, it's worth checking whether the penalty is actually small or not,
though.

Regards,

--
Fujii Masao

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2014-08-11 12:42:06 +0900, Fujii Masao wrote:

On Mon, Aug 11, 2014 at 10:48 AM, Tatsuo Ishii <ishii@postgresql.org> wrote:

On Mon, Aug 11, 2014 at 9:23 AM, Tatsuo Ishii <ishii@postgresql.org> wrote:

We can know the LSN of last committed WAL record on primary by using
pg_current_xlog_location(). It seems there's no API to know the time
when the WAL record was created. I would like to know standby delay by
using pg_last_xact_replay_timestamp() and such that API.

If there's no such a API, it would be useful to invent usch an API IMO.

+1

I proposed that function before, but unfortunately it failed to be applied.
But I still think that function is useful to calculate the replication delay.
The past discussion is
/messages/by-id/CAHGQGwF3ZjfuNEj5ka683KU5rQUBtSWtqFq7g1X0g34o+JXWBw@mail.gmail.com

I looked into the thread briefly and found Simon and Robert gave -1
for this because of performance concern. I'm not sure if it's a actual
performance penalty or not. Maybe we need to major the penalty?

I think that the performance penalty is negligible small because the patch
I posted before added only three stores to shared memory per
commit/abort.

Uh. It adds another atomic operation (the spinlock) to the commit
path. That's surely *not* insignificant. At the very least the
concurrency approach needs to be rethought.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Mon, Aug 11, 2014 at 3:54 PM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2014-08-11 12:42:06 +0900, Fujii Masao wrote:

On Mon, Aug 11, 2014 at 10:48 AM, Tatsuo Ishii <ishii@postgresql.org> wrote:

On Mon, Aug 11, 2014 at 9:23 AM, Tatsuo Ishii <ishii@postgresql.org> wrote:

We can know the LSN of last committed WAL record on primary by using
pg_current_xlog_location(). It seems there's no API to know the time
when the WAL record was created. I would like to know standby delay by
using pg_last_xact_replay_timestamp() and such that API.

If there's no such a API, it would be useful to invent usch an API IMO.

+1

I proposed that function before, but unfortunately it failed to be applied.
But I still think that function is useful to calculate the replication delay.
The past discussion is
/messages/by-id/CAHGQGwF3ZjfuNEj5ka683KU5rQUBtSWtqFq7g1X0g34o+JXWBw@mail.gmail.com

I looked into the thread briefly and found Simon and Robert gave -1
for this because of performance concern. I'm not sure if it's a actual
performance penalty or not. Maybe we need to major the penalty?

I think that the performance penalty is negligible small because the patch
I posted before added only three stores to shared memory per
commit/abort.

Uh. It adds another atomic operation (the spinlock) to the commit
path. That's surely *not* insignificant. At the very least the
concurrency approach needs to be rethought.

No, the patch doesn't add the spinlock at all. What the commit path
additionally does are

1. increment the counter in shared memory
2. set the timestamp of last commit record to shared memory
3. increment the counter in shared memory

There is no extra spinlock.

OTOH, when pg_last_xact_insert_timestamp reads the timestamp from
the shared memory, it checks whether the counter values are the same
or not before and after reading the timestamp. If they are not the same,
it tries to read the timesetamp again. This logic is necessary for reading
the consistent timestamp value there.

Regards,

--
Fujii Masao

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2014-08-11 16:20:41 +0900, Fujii Masao wrote:

On Mon, Aug 11, 2014 at 3:54 PM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2014-08-11 12:42:06 +0900, Fujii Masao wrote:

On Mon, Aug 11, 2014 at 10:48 AM, Tatsuo Ishii <ishii@postgresql.org> wrote:

On Mon, Aug 11, 2014 at 9:23 AM, Tatsuo Ishii <ishii@postgresql.org> wrote:

We can know the LSN of last committed WAL record on primary by using
pg_current_xlog_location(). It seems there's no API to know the time
when the WAL record was created. I would like to know standby delay by
using pg_last_xact_replay_timestamp() and such that API.

If there's no such a API, it would be useful to invent usch an API IMO.

+1

I proposed that function before, but unfortunately it failed to be applied.
But I still think that function is useful to calculate the replication delay.
The past discussion is
/messages/by-id/CAHGQGwF3ZjfuNEj5ka683KU5rQUBtSWtqFq7g1X0g34o+JXWBw@mail.gmail.com

I looked into the thread briefly and found Simon and Robert gave -1
for this because of performance concern. I'm not sure if it's a actual
performance penalty or not. Maybe we need to major the penalty?

I think that the performance penalty is negligible small because the patch
I posted before added only three stores to shared memory per
commit/abort.

Uh. It adds another atomic operation (the spinlock) to the commit
path. That's surely *not* insignificant. At the very least the
concurrency approach needs to be rethought.

No, the patch doesn't add the spinlock at all. What the commit path
additionally does are

1. increment the counter in shared memory
2. set the timestamp of last commit record to shared memory
3. increment the counter in shared memory

There is no extra spinlock.

Ah, I see. There's another patch somewhere down that thread
(CAHGQGwG4xFZjfyzaBn5v__d3qpyNNsGBpH3nAr6p40eLivkW5w@mail.gmail.com). The
patch in the message you linked to *does* use a spinlock though.

OTOH, when pg_last_xact_insert_timestamp reads the timestamp from
the shared memory, it checks whether the counter values are the same
or not before and after reading the timestamp. If they are not the same,
it tries to read the timesetamp again. This logic is necessary for reading
the consistent timestamp value there.

Yea, that approach then just touches a cacheline that should already be
local. I doubt that the implementation is correct on some more lenient
platforms (missing write memory barrier), but that's not "your fault".

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Mon, Aug 11, 2014 at 3:20 AM, Fujii Masao <masao.fujii@gmail.com> wrote:

There is no extra spinlock.

The version I reviewed had one; that's what I was objecting to.

Might need to add some pg_read_barrier() and pg_write_barrier() calls,
since we have those now.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Aug 15, 2014 at 1:55 AM, Robert Haas <robertmhaas@gmail.com> wrote:

On Mon, Aug 11, 2014 at 3:20 AM, Fujii Masao <masao.fujii@gmail.com> wrote:

There is no extra spinlock.

The version I reviewed had one; that's what I was objecting to.

Sorry for confusing you. I posted the latest patch to other thread.
This version doesn't use any spinlock.

/messages/by-id/CAHGQGwEwuh5CC3ib6wd0fxs9LAWme=kO09S4MOXnYnAfn7N5Bg@mail.gmail.com

Might need to add some pg_read_barrier() and pg_write_barrier() calls,
since we have those now.

Yep, memory barries might be needed as follows.

* Set the commit timestamp to shared memory.

shmem->counter++;
pg_write_barrier();
shmem->timestamp = my_timestamp;
pg_write_barrier();
shmem->count++;

* Read the commit timestamp from shared memory

my_count = shmem->counter;
pg_read_barrier();
my_timestamp = shmem->timestamp;
pg_read_barrier();
my_count = shmem->counter;

Is this way to use memory barriers right?

Regards,

--
Fujii Masao

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Thu, Aug 14, 2014 at 1:51 PM, Fujii Masao <masao.fujii@gmail.com> wrote:

On Fri, Aug 15, 2014 at 1:55 AM, Robert Haas <robertmhaas@gmail.com> wrote:

On Mon, Aug 11, 2014 at 3:20 AM, Fujii Masao <masao.fujii@gmail.com> wrote:

There is no extra spinlock.

The version I reviewed had one; that's what I was objecting to.

Sorry for confusing you. I posted the latest patch to other thread.
This version doesn't use any spinlock.

/messages/by-id/CAHGQGwEwuh5CC3ib6wd0fxs9LAWme=kO09S4MOXnYnAfn7N5Bg@mail.gmail.com

Might need to add some pg_read_barrier() and pg_write_barrier() calls,
since we have those now.

Yep, memory barries might be needed as follows.

* Set the commit timestamp to shared memory.

shmem->counter++;
pg_write_barrier();
shmem->timestamp = my_timestamp;
pg_write_barrier();
shmem->count++;

* Read the commit timestamp from shared memory

my_count = shmem->counter;
pg_read_barrier();
my_timestamp = shmem->timestamp;
pg_read_barrier();
my_count = shmem->counter;

Is this way to use memory barriers right?

That's about the idea. However, what you've got there is actually
unsafe, because shmem->counter++ is not an atomic operation. It reads
the counter (possibly even as two separate 4-byte loads if the counter
is an 8-byte value), increments it inside the CPU, and then writes the
resulting value back to memory. If two backends do this concurrently,
one of the updates might be lost.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2014-08-14 14:19:13 -0400, Robert Haas wrote:

On Thu, Aug 14, 2014 at 1:51 PM, Fujii Masao <masao.fujii@gmail.com> wrote:

On Fri, Aug 15, 2014 at 1:55 AM, Robert Haas <robertmhaas@gmail.com> wrote:

On Mon, Aug 11, 2014 at 3:20 AM, Fujii Masao <masao.fujii@gmail.com> wrote:

There is no extra spinlock.

The version I reviewed had one; that's what I was objecting to.

Sorry for confusing you. I posted the latest patch to other thread.
This version doesn't use any spinlock.

/messages/by-id/CAHGQGwEwuh5CC3ib6wd0fxs9LAWme=kO09S4MOXnYnAfn7N5Bg@mail.gmail.com

Might need to add some pg_read_barrier() and pg_write_barrier() calls,
since we have those now.

Yep, memory barries might be needed as follows.

* Set the commit timestamp to shared memory.

shmem->counter++;
pg_write_barrier();
shmem->timestamp = my_timestamp;
pg_write_barrier();
shmem->count++;

* Read the commit timestamp from shared memory

my_count = shmem->counter;
pg_read_barrier();
my_timestamp = shmem->timestamp;
pg_read_barrier();
my_count = shmem->counter;

Is this way to use memory barriers right?

That's about the idea. However, what you've got there is actually
unsafe, because shmem->counter++ is not an atomic operation. It reads
the counter (possibly even as two separate 4-byte loads if the counter
is an 8-byte value), increments it inside the CPU, and then writes the
resulting value back to memory. If two backends do this concurrently,
one of the updates might be lost.

All these are only written by one backend, so it should be safe. Note
that that coding pattern, just without memory barriers, is all over
pgstat.c

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Thu, Aug 14, 2014 at 2:21 PM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2014-08-14 14:19:13 -0400, Robert Haas wrote:

On Thu, Aug 14, 2014 at 1:51 PM, Fujii Masao <masao.fujii@gmail.com> wrote:

On Fri, Aug 15, 2014 at 1:55 AM, Robert Haas <robertmhaas@gmail.com> wrote:

On Mon, Aug 11, 2014 at 3:20 AM, Fujii Masao <masao.fujii@gmail.com> wrote:

There is no extra spinlock.

The version I reviewed had one; that's what I was objecting to.

Sorry for confusing you. I posted the latest patch to other thread.
This version doesn't use any spinlock.

/messages/by-id/CAHGQGwEwuh5CC3ib6wd0fxs9LAWme=kO09S4MOXnYnAfn7N5Bg@mail.gmail.com

Might need to add some pg_read_barrier() and pg_write_barrier() calls,
since we have those now.

Yep, memory barries might be needed as follows.

* Set the commit timestamp to shared memory.

shmem->counter++;
pg_write_barrier();
shmem->timestamp = my_timestamp;
pg_write_barrier();
shmem->count++;

* Read the commit timestamp from shared memory

my_count = shmem->counter;
pg_read_barrier();
my_timestamp = shmem->timestamp;
pg_read_barrier();
my_count = shmem->counter;

Is this way to use memory barriers right?

That's about the idea. However, what you've got there is actually
unsafe, because shmem->counter++ is not an atomic operation. It reads
the counter (possibly even as two separate 4-byte loads if the counter
is an 8-byte value), increments it inside the CPU, and then writes the
resulting value back to memory. If two backends do this concurrently,
one of the updates might be lost.

All these are only written by one backend, so it should be safe. Note
that that coding pattern, just without memory barriers, is all over
pgstat.c

Ah, OK. If there's a separate slot for each backend, I agree that it's safe.

We should probably add barriers to pgstat.c, too.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2014-08-14 14:37:22 -0400, Robert Haas wrote:

On Thu, Aug 14, 2014 at 2:21 PM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2014-08-14 14:19:13 -0400, Robert Haas wrote:

That's about the idea. However, what you've got there is actually
unsafe, because shmem->counter++ is not an atomic operation. It reads
the counter (possibly even as two separate 4-byte loads if the counter
is an 8-byte value), increments it inside the CPU, and then writes the
resulting value back to memory. If two backends do this concurrently,
one of the updates might be lost.

All these are only written by one backend, so it should be safe. Note
that that coding pattern, just without memory barriers, is all over
pgstat.c

Ah, OK. If there's a separate slot for each backend, I agree that it's safe.

We should probably add barriers to pgstat.c, too.

Yea, definitely. I think this is rather borked on "weaker"
architectures. It's just that the consequences of an out of date/torn
value are rather low, so it's unlikely to be noticed.

Imo we should encapsulate the changecount modifications/checks somehow
instead of repeating the barriers, Asserts, comments et al everywhere.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Aug 15, 2014 at 3:40 AM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2014-08-14 14:37:22 -0400, Robert Haas wrote:

On Thu, Aug 14, 2014 at 2:21 PM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2014-08-14 14:19:13 -0400, Robert Haas wrote:

That's about the idea. However, what you've got there is actually
unsafe, because shmem->counter++ is not an atomic operation. It reads
the counter (possibly even as two separate 4-byte loads if the counter
is an 8-byte value), increments it inside the CPU, and then writes the
resulting value back to memory. If two backends do this concurrently,
one of the updates might be lost.

All these are only written by one backend, so it should be safe. Note
that that coding pattern, just without memory barriers, is all over
pgstat.c

Ah, OK. If there's a separate slot for each backend, I agree that it's safe.

We should probably add barriers to pgstat.c, too.

Yea, definitely. I think this is rather borked on "weaker"
architectures. It's just that the consequences of an out of date/torn
value are rather low, so it's unlikely to be noticed.

Imo we should encapsulate the changecount modifications/checks somehow
instead of repeating the barriers, Asserts, comments et al everywhere.

So what about applying the attached patch first, which adds the macros
to load and store the changecount with the memory barries, and changes
pgstat.c use them. Maybe this patch needs to be back-patch to at least 9.4?

After applying the patch, I will rebase the pg_last_xact_insert_timestamp
patch and post it again.

Regards,

--
Fujii Masao

On Fri, Aug 15, 2014 at 7:17 AM, Fujii Masao <masao.fujii@gmail.com> wrote:

On Fri, Aug 15, 2014 at 3:40 AM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2014-08-14 14:37:22 -0400, Robert Haas wrote:

On Thu, Aug 14, 2014 at 2:21 PM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2014-08-14 14:19:13 -0400, Robert Haas wrote:

That's about the idea. However, what you've got there is actually
unsafe, because shmem->counter++ is not an atomic operation. It reads
the counter (possibly even as two separate 4-byte loads if the counter
is an 8-byte value), increments it inside the CPU, and then writes the
resulting value back to memory. If two backends do this concurrently,
one of the updates might be lost.

All these are only written by one backend, so it should be safe. Note
that that coding pattern, just without memory barriers, is all over
pgstat.c

Ah, OK. If there's a separate slot for each backend, I agree that it's safe.

We should probably add barriers to pgstat.c, too.

Yea, definitely. I think this is rather borked on "weaker"
architectures. It's just that the consequences of an out of date/torn
value are rather low, so it's unlikely to be noticed.

Imo we should encapsulate the changecount modifications/checks somehow
instead of repeating the barriers, Asserts, comments et al everywhere.

So what about applying the attached patch first, which adds the macros
to load and store the changecount with the memory barries, and changes
pgstat.c use them. Maybe this patch needs to be back-patch to at least 9.4?

After applying the patch, I will rebase the pg_last_xact_insert_timestamp
patch and post it again.

That looks OK to me on a relatively-quick read-through. I was
initially a bit worried about this part:

do
{
! pgstat_increment_changecount_before(beentry);
} while ((beentry->st_changecount & 1) == 0);

pgstat_increment_changecount_before is an increment followed by a
write barrier. This seemed like funny coding to me at first because
while-test isn't protected by any sort of barrier. But now I think
it's correct, because there's only one process that can possibly write
to that data, and that's the one that is making the test, and it had
certainly better see its own modifications in program order no matter
what.

I wouldn't object to back-patching this to 9.4 if we were earlier in
the beta cycle, but at this point I'm more inclined to just put it in
9.5. If we get an actual bug report about any of this, we can always
back-patch the fix at that time. But so far that seems mostly
hypothetical, so I think the less-risky course of action is to give
this a longer time to bake before it hits an official release.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Tue, Aug 19, 2014 at 1:07 AM, Robert Haas <robertmhaas@gmail.com> wrote:

On Fri, Aug 15, 2014 at 7:17 AM, Fujii Masao <masao.fujii@gmail.com> wrote:

On Fri, Aug 15, 2014 at 3:40 AM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2014-08-14 14:37:22 -0400, Robert Haas wrote:

On Thu, Aug 14, 2014 at 2:21 PM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2014-08-14 14:19:13 -0400, Robert Haas wrote:

That's about the idea. However, what you've got there is actually
unsafe, because shmem->counter++ is not an atomic operation. It reads
the counter (possibly even as two separate 4-byte loads if the counter
is an 8-byte value), increments it inside the CPU, and then writes the
resulting value back to memory. If two backends do this concurrently,
one of the updates might be lost.

All these are only written by one backend, so it should be safe. Note
that that coding pattern, just without memory barriers, is all over
pgstat.c

Ah, OK. If there's a separate slot for each backend, I agree that it's safe.

We should probably add barriers to pgstat.c, too.

Yea, definitely. I think this is rather borked on "weaker"
architectures. It's just that the consequences of an out of date/torn
value are rather low, so it's unlikely to be noticed.

Imo we should encapsulate the changecount modifications/checks somehow
instead of repeating the barriers, Asserts, comments et al everywhere.

So what about applying the attached patch first, which adds the macros
to load and store the changecount with the memory barries, and changes
pgstat.c use them. Maybe this patch needs to be back-patch to at least 9.4?

After applying the patch, I will rebase the pg_last_xact_insert_timestamp
patch and post it again.

That looks OK to me on a relatively-quick read-through. I was
initially a bit worried about this part:

do
{
! pgstat_increment_changecount_before(beentry);
} while ((beentry->st_changecount & 1) == 0);

pgstat_increment_changecount_before is an increment followed by a
write barrier. This seemed like funny coding to me at first because
while-test isn't protected by any sort of barrier. But now I think
it's correct, because there's only one process that can possibly write
to that data, and that's the one that is making the test, and it had
certainly better see its own modifications in program order no matter
what.

I wouldn't object to back-patching this to 9.4 if we were earlier in
the beta cycle, but at this point I'm more inclined to just put it in
9.5. If we get an actual bug report about any of this, we can always
back-patch the fix at that time. But so far that seems mostly
hypothetical, so I think the less-risky course of action is to give
this a longer time to bake before it hits an official release.

Sounds reasonable. So, barring any objection, I will apply the patch
only to the master branch.

Regards,

--
Fujii Masao

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 8/27/14, 7:33 AM, Fujii Masao wrote:

On Tue, Aug 19, 2014 at 1:07 AM, Robert Haas <robertmhaas@gmail.com> wrote:

On Fri, Aug 15, 2014 at 7:17 AM, Fujii Masao <masao.fujii@gmail.com> wrote:

On Fri, Aug 15, 2014 at 3:40 AM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2014-08-14 14:37:22 -0400, Robert Haas wrote:

On Thu, Aug 14, 2014 at 2:21 PM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2014-08-14 14:19:13 -0400, Robert Haas wrote:

That's about the idea. However, what you've got there is actually
unsafe, because shmem->counter++ is not an atomic operation. It reads
the counter (possibly even as two separate 4-byte loads if the counter
is an 8-byte value), increments it inside the CPU, and then writes the
resulting value back to memory. If two backends do this concurrently,
one of the updates might be lost.

All these are only written by one backend, so it should be safe. Note
that that coding pattern, just without memory barriers, is all over
pgstat.c

Ah, OK. If there's a separate slot for each backend, I agree that it's safe.

We should probably add barriers to pgstat.c, too.

Yea, definitely. I think this is rather borked on "weaker"
architectures. It's just that the consequences of an out of date/torn
value are rather low, so it's unlikely to be noticed.

Imo we should encapsulate the changecount modifications/checks somehow
instead of repeating the barriers, Asserts, comments et al everywhere.

So what about applying the attached patch first, which adds the macros
to load and store the changecount with the memory barries, and changes
pgstat.c use them. Maybe this patch needs to be back-patch to at least 9.4?

After applying the patch, I will rebase the pg_last_xact_insert_timestamp
patch and post it again.

That looks OK to me on a relatively-quick read-through. I was
initially a bit worried about this part:

do
{
! pgstat_increment_changecount_before(beentry);
} while ((beentry->st_changecount & 1) == 0);

pgstat_increment_changecount_before is an increment followed by a
write barrier. This seemed like funny coding to me at first because
while-test isn't protected by any sort of barrier. But now I think
it's correct, because there's only one process that can possibly write
to that data, and that's the one that is making the test, and it had
certainly better see its own modifications in program order no matter
what.

I wouldn't object to back-patching this to 9.4 if we were earlier in
the beta cycle, but at this point I'm more inclined to just put it in
9.5. If we get an actual bug report about any of this, we can always
back-patch the fix at that time. But so far that seems mostly
hypothetical, so I think the less-risky course of action is to give
this a longer time to bake before it hits an official release.

Sounds reasonable. So, barring any objection, I will apply the patch
only to the master branch.

It's probably worth adding a comment explaining why it's safe to do this without a barrier...
--
Jim C. Nasby, Data Architect jim@nasby.net
512.569.9461 (cell) http://jim.nasby.net

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Thu, Aug 28, 2014 at 2:44 AM, Jim Nasby <jim@nasby.net> wrote:

On 8/27/14, 7:33 AM, Fujii Masao wrote:

On Tue, Aug 19, 2014 at 1:07 AM, Robert Haas <robertmhaas@gmail.com>
wrote:

On Fri, Aug 15, 2014 at 7:17 AM, Fujii Masao <masao.fujii@gmail.com>
wrote:

On Fri, Aug 15, 2014 at 3:40 AM, Andres Freund <andres@2ndquadrant.com>
wrote:

On 2014-08-14 14:37:22 -0400, Robert Haas wrote:

On Thu, Aug 14, 2014 at 2:21 PM, Andres Freund
<andres@2ndquadrant.com> wrote:

On 2014-08-14 14:19:13 -0400, Robert Haas wrote:

That's about the idea. However, what you've got there is actually
unsafe, because shmem->counter++ is not an atomic operation. It
reads
the counter (possibly even as two separate 4-byte loads if the
counter
is an 8-byte value), increments it inside the CPU, and then writes
the
resulting value back to memory. If two backends do this
concurrently,
one of the updates might be lost.

All these are only written by one backend, so it should be safe. Note
that that coding pattern, just without memory barriers, is all over
pgstat.c

Ah, OK. If there's a separate slot for each backend, I agree that
it's safe.

We should probably add barriers to pgstat.c, too.

Yea, definitely. I think this is rather borked on "weaker"
architectures. It's just that the consequences of an out of date/torn
value are rather low, so it's unlikely to be noticed.

Imo we should encapsulate the changecount modifications/checks somehow
instead of repeating the barriers, Asserts, comments et al everywhere.

So what about applying the attached patch first, which adds the macros
to load and store the changecount with the memory barries, and changes
pgstat.c use them. Maybe this patch needs to be back-patch to at least
9.4?

After applying the patch, I will rebase the
pg_last_xact_insert_timestamp
patch and post it again.

That looks OK to me on a relatively-quick read-through. I was
initially a bit worried about this part:

do
{
! pgstat_increment_changecount_before(beentry);
} while ((beentry->st_changecount & 1) == 0);

pgstat_increment_changecount_before is an increment followed by a
write barrier. This seemed like funny coding to me at first because
while-test isn't protected by any sort of barrier. But now I think
it's correct, because there's only one process that can possibly write
to that data, and that's the one that is making the test, and it had
certainly better see its own modifications in program order no matter
what.

I wouldn't object to back-patching this to 9.4 if we were earlier in
the beta cycle, but at this point I'm more inclined to just put it in
9.5. If we get an actual bug report about any of this, we can always
back-patch the fix at that time. But so far that seems mostly
hypothetical, so I think the less-risky course of action is to give
this a longer time to bake before it hits an official release.

Sounds reasonable. So, barring any objection, I will apply the patch
only to the master branch.

It's probably worth adding a comment explaining why it's safe to do this
without a barrier...

s/without/with ?

Theoretically it's not safe without a barrier on a machine with weak
memory ordering. No?

Regards,

--
Fujii Masao

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Thu, Aug 28, 2014 at 3:34 AM, Fujii Masao <masao.fujii@gmail.com> wrote:

Theoretically it's not safe without a barrier on a machine with weak
memory ordering. No?

Why not?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Aug 15, 2014 at 8:17 PM, Fujii Masao <masao.fujii@gmail.com> wrote:

On Fri, Aug 15, 2014 at 3:40 AM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2014-08-14 14:37:22 -0400, Robert Haas wrote:

On Thu, Aug 14, 2014 at 2:21 PM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2014-08-14 14:19:13 -0400, Robert Haas wrote:

That's about the idea. However, what you've got there is actually
unsafe, because shmem->counter++ is not an atomic operation. It reads
the counter (possibly even as two separate 4-byte loads if the counter
is an 8-byte value), increments it inside the CPU, and then writes the
resulting value back to memory. If two backends do this concurrently,
one of the updates might be lost.

All these are only written by one backend, so it should be safe. Note
that that coding pattern, just without memory barriers, is all over
pgstat.c

Ah, OK. If there's a separate slot for each backend, I agree that it's safe.

We should probably add barriers to pgstat.c, too.

Yea, definitely. I think this is rather borked on "weaker"
architectures. It's just that the consequences of an out of date/torn
value are rather low, so it's unlikely to be noticed.

Imo we should encapsulate the changecount modifications/checks somehow
instead of repeating the barriers, Asserts, comments et al everywhere.

So what about applying the attached patch first, which adds the macros
to load and store the changecount with the memory barries, and changes
pgstat.c use them. Maybe this patch needs to be back-patch to at least 9.4?

After applying the patch, I will rebase the pg_last_xact_insert_timestamp
patch and post it again.

Hm, what's the status on this patch? The addition of those macros to
control count increment with a memory barrier seems like a good thing
at least. The 2nd patch has not been rebased but still..
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers