pgsql: Checkpoint sorting and balancing.
Checkpoint sorting and balancing.
Up to now checkpoints were written in the order they're in the
BufferDescriptors. That's nearly random in a lot of cases, which
performs badly on rotating media, but even on SSDs it causes slowdowns.
To avoid that, sort checkpoints before writing them out. We currently
sort by tablespace, relfilenode, fork and block number.
One of the major reasons that previously wasn't done, was fear of
imbalance between tablespaces. To address that balance writes between
tablespaces.
The other prime concern was that the relatively large allocation to sort
the buffers in might fail, preventing checkpoints from happening. Thus
pre-allocate the required memory in shared memory, at server startup.
This particularly makes it more efficient to have checkpoint flushing
enabled, because that'll often result in a lot of writes that can be
coalesced into one flush.
Discussion: alpine.DEB.2.10.1506011320000.28433@sto
Author: Fabien Coelho and Andres Freund
Branch
------
master
Details
-------
http://git.postgresql.org/pg/commitdiff/9cd00c457e6a1ebb984167ac556a9961812a683c
Modified Files
--------------
src/backend/storage/buffer/README | 5 -
src/backend/storage/buffer/buf_init.c | 22 ++-
src/backend/storage/buffer/bufmgr.c | 274 +++++++++++++++++++++++++++++-----
src/include/storage/buf_internals.h | 18 +++
src/tools/pgindent/typedefs.list | 2 +
5 files changed, 277 insertions(+), 44 deletions(-)
--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers
On 3/10/16 8:29 PM, Andres Freund wrote:
Checkpoint sorting and balancing.
This patch appears to have introduced a change that makes the build
with dtrace fail:
In file included from ../../../../src/include/pg_trace.h:15:0,
from bufmgr.c:42:
bufmgr.c: In function 'BufferSync':
bufmgr.c:1939:59: error: 'num_to_write' undeclared (first use in this function)
TRACE_POSTGRESQL_BUFFER_SYNC_DONE(NBuffers, num_written, num_to_write);
^
../../../../src/include/utils/probes.h:93:83: note: in definition of macro 'TRACE_POSTGRESQL_BUFFER_SYNC_DONE'
__dtrace_probe$postgresql$buffer__sync__done$v1$696e74$696e74$696e74(arg0, arg1, arg2); \
^~~~
bufmgr.c:1939:59: note: each undeclared identifier is reported only once for each function it appears in
TRACE_POSTGRESQL_BUFFER_SYNC_DONE(NBuffers, num_written, num_to_write);
^
../../../../src/include/utils/probes.h:93:83: note: in definition of macro 'TRACE_POSTGRESQL_BUFFER_SYNC_DONE'
__dtrace_probe$postgresql$buffer__sync__done$v1$696e74$696e74$696e74(arg0, arg1, arg2); \
^~~~
--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers
On 2016-03-10 20:47:10 -0500, Peter Eisentraut wrote:
On 3/10/16 8:29 PM, Andres Freund wrote:
Checkpoint sorting and balancing.
This patch appears to have introduced a change that makes the build
with dtrace fail:In file included from ../../../../src/include/pg_trace.h:15:0,
from bufmgr.c:42:
bufmgr.c: In function 'BufferSync':
bufmgr.c:1939:59: error: 'num_to_write' undeclared (first use in this function)
TRACE_POSTGRESQL_BUFFER_SYNC_DONE(NBuffers, num_written, num_to_write);
^
../../../../src/include/utils/probes.h:93:83: note: in definition of macro 'TRACE_POSTGRESQL_BUFFER_SYNC_DONE'
__dtrace_probe$postgresql$buffer__sync__done$v1$696e74$696e74$696e74(arg0, arg1, arg2); \
^~~~
bufmgr.c:1939:59: note: each undeclared identifier is reported only once for each function it appears in
TRACE_POSTGRESQL_BUFFER_SYNC_DONE(NBuffers, num_written, num_to_write);
^
../../../../src/include/utils/probes.h:93:83: note: in definition of macro 'TRACE_POSTGRESQL_BUFFER_SYNC_DONE'
__dtrace_probe$postgresql$buffer__sync__done$v1$696e74$696e74$696e74(arg0, arg1, arg2); \
^~~~
Hrmpf. I tried to fix the issue; could you quickly verify?
These probes are a bit annoying; and I seriously doubt anybody actually
uses them. But obviously we should keep them working as long as we have
them.
Thanks!
Andres
--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers
On 3/10/16 8:52 PM, Andres Freund wrote:
Hrmpf. I tried to fix the issue; could you quickly verify?
looks good
--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers
On Fri, Mar 11, 2016 at 2:29 AM, Andres Freund <andres@anarazel.de> wrote:
Checkpoint sorting and balancing.
+1.
--
Michael--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers