Should pg_stat_bgwriter.buffers_backend_fsync be removed?

Peter Geoghegan <pg@heroku.com> writes:

Backend fsyncs are theoretically still possible after the fsync
request queue compaction patch (which was subsequently back-patched to
all supported release branches). However, I'm reasonably confident
that that patch was so effective as to make a backend fsync all but
impossible.

What's your evidence for that claim?

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Sat, Apr 26, 2014 at 9:16 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Peter Geoghegan <pg@heroku.com> writes:

Backend fsyncs are theoretically still possible after the fsync
request queue compaction patch (which was subsequently back-patched to
all supported release branches). However, I'm reasonably confident
that that patch was so effective as to make a backend fsync all but
impossible.

What's your evidence for that claim?

I don't have any evidence, but I think it unlikely that this is an
occurrence that is seen in the real world. I was not able to see any
instances of it on the entire Heroku fleet at one point a few months
back, for one thing. For another, I have never observed this with any
benchmark, even though pgbench-tools presents buffers_backend_fsync
for each test run. The queue compaction patch completely fixed Greg
Smith's original test case. Even then, I believe it was considered
more of a patch addressing an edge case than anything else. Even the
comments above ForwardFsyncRequest() consider the occurance of backend
fsyncs to be only "theoretically possible".

If anyone is aware of any cases where this is still actually known to
happen in production, I'd like to hear about them. However, ISTM that
if this actually does still happen, those cases would be better served
by surfacing the problem in the logs.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

--On 26. April 2014 19:42:47 -0700 Peter Geoghegan <pg@heroku.com> wrote:

I suggest removing it for 9.5, and instead logging individual
occurrences of backend fsync requests within ForwardFsyncRequest(). It
seems fair to treat that as an anomaly to draw particular attention
to.

But wouldn't that make it more complicated/unlikely to discover cases,
where it still doesn't work?

--
Thanks

Bernd

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Tue, Apr 29, 2014 at 2:51 AM, Bernd Helmle <mailings@oopsware.de> wrote:

I suggest removing it for 9.5, and instead logging individual
occurrences of backend fsync requests within ForwardFsyncRequest(). It
seems fair to treat that as an anomaly to draw particular attention
to.

But wouldn't that make it more complicated/unlikely to discover cases, where
it still doesn't work?

I don't think so, no.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Tue, Apr 29, 2014 at 5:54 AM, Peter Geoghegan <pg@heroku.com> wrote:

On Tue, Apr 29, 2014 at 2:51 AM, Bernd Helmle <mailings@oopsware.de> wrote:

I suggest removing it for 9.5, and instead logging individual
occurrences of backend fsync requests within ForwardFsyncRequest(). It
seems fair to treat that as an anomaly to draw particular attention
to.

But wouldn't that make it more complicated/unlikely to discover cases, where
it still doesn't work?

I don't think so, no.

I think it just depends. For people who are running a log scraper
anyway, a message would be better than a statistics counter, because
it's one less thing to check. For people who are running something
that monitors the stats views anyway, but perhaps not a log scraper,
the counter is better.

Overall, I don't see much reason to tinker with this. If we had no
reporting at all of this condition now, I'd probably be mildly more
supportive of adding a log message than a counter. But since we've
already got something and there's no real problem with it, I'm
disinclined to make a change.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Robert Haas <robertmhaas@gmail.com> writes:

Overall, I don't see much reason to tinker with this. If we had no
reporting at all of this condition now, I'd probably be mildly more
supportive of adding a log message than a counter. But since we've
already got something and there's no real problem with it, I'm
disinclined to make a change.

+1 ... if it ain't broke, why fix it?

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 4/26/14, 9:42 PM, Peter Geoghegan wrote:

Backend fsyncs are theoretically still possible after the fsync
request queue compaction patch (which was subsequently back-patched to
all supported release branches). However, I'm reasonably confident
that that patch was so effective as to make a backend fsync all but
impossible. As such, it seems like the buffers_backend_fsync column in
the pg_stat_bgwriter view is more or less obsolete.

I suggest removing it for 9.5, and instead logging individual
occurrences of backend fsync requests within ForwardFsyncRequest(). It
seems fair to treat that as an anomaly to draw particular attention
to.

All else equal, I don't like the idea of removing this from pg_stat_bgwriter. Being able to look there and see if this is occurring since last stats reset is much easier than grepping logfiles.

I don't have an issue with logging it, though I think we need to be careful to ensure we don't go crazy if something happens in the system where suddenly all backends are fsyncing. If that happens you're going to have major IO problems and trying to log thousands (or more) of extra entries is just going to make it worse.
--
Jim C. Nasby, Data Architect jim@nasby.net
512.569.9461 (cell) http://jim.nasby.net

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Tue, Apr 29, 2014 at 12:02 PM, Jim Nasby <jim@nasby.net> wrote:

All else equal, I don't like the idea of removing this from
pg_stat_bgwriter. Being able to look there and see if this is occurring
since last stats reset is much easier than grepping logfiles.

Have you ever actually seen it at a non-zero value after the
compaction patch went in?

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers