setproctitle_fast()
Hello hackers,
FreeBSD's setproctitle() is a bit slow because it contains a syscall
or two, so people often run PostgreSQL with update_process_title set
to off on that OS. That makes the user experience not quite as nice
as Linux. As a weekend learn-me-some-kernel-hacking project I fixed
that and got the patch committed to FreeBSD 12, though I was asked to
use a new libc entry point _fast(). Here's a patch to teach
PostgreSQL about that. It doesn't have much effect on small systems,
but it makes "pgbench -c 40 -j 40 -S -M prepared" do ~10% more
transactions per second on an AWS m4.10xlarge instance.
I'll park this patch here until the FreeBSD feature escapes in a
RELEASE version in a few months.
For anyone interested, https://wiki.postgresql.org/wiki/FreeBSD has
some notes on this and other such things.
--
Thomas Munro
http://www.enterprisedb.com
Attachments:
0001-Use-setproctitle_fast-to-update-the-ps-status-if-ava.patchapplication/octet-stream; name=0001-Use-setproctitle_fast-to-update-the-ps-status-if-ava.patchDownload
From b24cf4acf6c1eec7dc9b79ae5120ee58975ade78 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Wed, 4 Jul 2018 22:21:47 +1200
Subject: [PATCH] Use setproctitle_fast() to update the ps status, if
available.
FreeBSD 12 introduced a faster variant of setproctitle(). Use it if
possible.
Author: Thomas Munro
Reviewed-by:
Discussion:
---
configure | 2 +-
configure.in | 2 +-
src/backend/utils/misc/ps_status.c | 11 +++++++++--
src/include/pg_config.h.in | 3 +++
4 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 1bc465b942..5fcce9f6dc 100755
--- a/configure
+++ b/configure
@@ -14915,7 +14915,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l
+for ac_func in cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
diff --git a/configure.in b/configure.in
index a6b3b88cfa..75a2099f80 100644
--- a/configure.in
+++ b/configure.in
@@ -1540,7 +1540,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/backend/utils/misc/ps_status.c b/src/backend/utils/misc/ps_status.c
index 83a93d2405..55521c19af 100644
--- a/src/backend/utils/misc/ps_status.c
+++ b/src/backend/utils/misc/ps_status.c
@@ -38,6 +38,9 @@ bool update_process_title = true;
/*
* Alternative ways of updating ps display:
*
+ * PS_USE_SETPROCTITLE_FAST
+ * use the function setproctitle_fast(const char *, ...)
+ * (newer FreeBSD systems)
* PS_USE_SETPROCTITLE
* use the function setproctitle(const char *, ...)
* (newer BSD systems)
@@ -59,7 +62,9 @@ bool update_process_title = true;
* don't update ps display
* (This is the default, as it is safest.)
*/
-#if defined(HAVE_SETPROCTITLE)
+#if defined(HAVE_SETPROCTITLE_FAST)
+#define PS_USE_SETPROCTITLE_FAST
+#elif defined(HAVE_SETPROCTITLE)
#define PS_USE_SETPROCTITLE
#elif defined(HAVE_PSTAT) && defined(PSTAT_SETCMD)
#define PS_USE_PSTAT
@@ -286,7 +291,7 @@ init_ps_display(const char *username, const char *dbname,
* Make fixed prefix of ps display.
*/
-#ifdef PS_USE_SETPROCTITLE
+#if defined(PS_USE_SETPROCTITLE) || defined(PS_USE_SETPROCTITLE_FAST)
/*
* apparently setproctitle() already adds a `progname:' prefix to the ps
@@ -349,6 +354,8 @@ set_ps_display(const char *activity, bool force)
#ifdef PS_USE_SETPROCTITLE
setproctitle("%s", ps_buffer);
+#elif defined(PS_USE_SETPROCTITLE_FAST)
+ setproctitle_fast("%s", ps_buffer);
#endif
#ifdef PS_USE_PSTAT
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 89b8804251..9c26cea973 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -488,6 +488,9 @@
/* Define to 1 if you have the `setproctitle' function. */
#undef HAVE_SETPROCTITLE
+/* Define to 1 if you have the `setproctitle_fast' function. */
+#undef HAVE_SETPROCTITLE_FAST
+
/* Define to 1 if you have the `setsid' function. */
#undef HAVE_SETSID
--
2.17.1
FreeBSD's setproctitle() is a bit slow because it contains a syscall
or two, so people often run PostgreSQL with update_process_title set
to off on that OS. That makes the user experience not quite as nice
as Linux. As a weekend learn-me-some-kernel-hacking project I fixed
that and got the patch committed to FreeBSD 12, though I was asked to
use a new libc entry point _fast(). Here's a patch to teach
PostgreSQL about that. It doesn't have much effect on small systems,
but it makes "pgbench -c 40 -j 40 -S -M prepared" do ~10% more
transactions per second on an AWS m4.10xlarge instance.
I am curious why they asked you to use a new libc entry point. The
function signatures are the same. More people could benefit from
making the existing function faster.
On Sat, Jul 7, 2018 at 11:57 PM, Emre Hasegeli <emre@hasegeli.com> wrote:
FreeBSD's setproctitle() is a bit slow because it contains a syscall
or two, so people often run PostgreSQL with update_process_title set
to off on that OS. That makes the user experience not quite as nice
as Linux. As a weekend learn-me-some-kernel-hacking project I fixed
that and got the patch committed to FreeBSD 12, though I was asked to
use a new libc entry point _fast(). Here's a patch to teach
PostgreSQL about that. It doesn't have much effect on small systems,
but it makes "pgbench -c 40 -j 40 -S -M prepared" do ~10% more
transactions per second on an AWS m4.10xlarge instance.I am curious why they asked you to use a new libc entry point. The
function signatures are the same. More people could benefit from
making the existing function faster.
There is a small trade-off: ps/top/htop do a bit more work (the kernel
has to go and pull the data out of the source process's memory instead
of keeping its own copy maintained via syscalls). Originally I
proposed to make setproctitle() adaptive, figuring out who were the
frequent updaters, but the heuristics I proposed were rejected and it
looked hard to come up with better ones[1]https://lists.freebsd.org/pipermail/freebsd-hackers/2018-July/052975.html. There may also be an
argument that you should have to opt into the fast and loose
consistency of this approach explicitly (though no one seems to mind
about that on Linux).
Anyway, I guess there aren't too many other applications that want to
do this kind of thing at 1MHz or whatever!
[1]: https://lists.freebsd.org/pipermail/freebsd-hackers/2018-July/052975.html
--
Thomas Munro
http://www.enterprisedb.com
On Thu, Jul 5, 2018 at 10:09 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
I'll park this patch here until the FreeBSD feature escapes in a
RELEASE version in a few months.
It has now been merged into FreeBSD stable/11, so it's expected in
FreeBSD 11.3. It is supported by buildfarm animal peripatus. So I
figured I might as well push this to master now, to increase the time
it gets tested on both sides. Done.
I won't back-patch it, per project policy, but I've proposed local
patches for FreeBSD's ports over here:
https://reviews.freebsd.org/D16234
--
Thomas Munro
http://www.enterprisedb.com