pgsql: Add PSQL_WATCH_PAGER for psql's \watch command.

Thomas Munro <tmunro@postgresql.org> writes:

To make \watch react quickly when the user quits the pager or presses
^C, and also to increase the accuracy of its timing and decrease the
rate of useless context switches, change the main loop of the \watch
command to use sigwait() rather than a sleeping/polling loop, on Unix.

I think this is going to fall over on gaur, which doesn't have POSIX-style
sigwait. We've escaped dealing with that so far because our only existing
use of sigwait is hidden under

#if defined(ENABLE_THREAD_SAFETY) && !defined(WIN32)

Perhaps the easiest answer is to make the #if conditional for this
code look like that too.

regards, tom lane

On Tue, Jul 13, 2021 at 12:30 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Thomas Munro <tmunro@postgresql.org> writes:

To make \watch react quickly when the user quits the pager or presses
^C, and also to increase the accuracy of its timing and decrease the
rate of useless context switches, change the main loop of the \watch
command to use sigwait() rather than a sleeping/polling loop, on Unix.

I think this is going to fall over on gaur, which doesn't have POSIX-style
sigwait. We've escaped dealing with that so far because our only existing
use of sigwait is hidden under

#if defined(ENABLE_THREAD_SAFETY) && !defined(WIN32)

Perhaps the easiest answer is to make the #if conditional for this
code look like that too.

Oh, thanks for the advance warning. Wouldn't HAVE_SIGWAIT be better? Like so.

Thomas Munro <thomas.munro@gmail.com> writes:

On Tue, Jul 13, 2021 at 12:30 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

I think this is going to fall over on gaur, which doesn't have POSIX-style
sigwait.

Oh, thanks for the advance warning. Wouldn't HAVE_SIGWAIT be better? Like so.

That won't help as-is, because it *does* have sigwait, just not with
the POSIX API. thread_test.c does this:

/* Test for POSIX.1c 2-arg sigwait() and fail on single-arg version */
#include <signal.h>
int sigwait(const sigset_t *set, int *sig);

which perhaps should be pulled out of there and moved to the
configure script proper.

regards, tom lane

On Tue, Jul 13, 2021 at 1:09 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Thomas Munro <thomas.munro@gmail.com> writes:

Oh, thanks for the advance warning. Wouldn't HAVE_SIGWAIT be better? Like so.

That won't help as-is, because it *does* have sigwait, just not with
the POSIX API. thread_test.c does this:

/* Test for POSIX.1c 2-arg sigwait() and fail on single-arg version */
#include <signal.h>
int sigwait(const sigset_t *set, int *sig);

which perhaps should be pulled out of there and moved to the
configure script proper.

Ah, I see. I'll have a crack at that after lunch.

Thomas Munro <thomas.munro@gmail.com> writes:

On Tue, Jul 13, 2021 at 1:09 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

That won't help as-is, because it *does* have sigwait, just not with
the POSIX API. thread_test.c does this:

/* Test for POSIX.1c 2-arg sigwait() and fail on single-arg version */
#include <signal.h>
int sigwait(const sigset_t *set, int *sig);

which perhaps should be pulled out of there and moved to the
configure script proper.

Ah, I see. I'll have a crack at that after lunch.

Huh ... gaur did this:

/usr/ccs/bin/ld: Unsatisfied symbols:
sigwait (code)

which is not what I was expecting, because there is a definition for
sigwait in /usr/include (though not in a header file we use). It must
be in some add-on library instead of libc.

However, wrasse did this:

"/export/home/nm/farm/studio64v12_6/HEAD/pgsql.build/../pgsql/src/bin/psql/command.c", line 5062: prototype mismatch: 2 args passed, 1 expected
cc: acomp failed for /export/home/nm/farm/studio64v12_6/HEAD/pgsql.build/../pgsql/src/bin/psql/command.c

which I was expecting even less. Evidently, prehistoric HPUX is not the
only platform still using the pre-POSIX API for this function. So we
really do need the full configure check.

regards, tom lane

On Tue, Jul 13, 2021 at 4:15 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Huh ... gaur did this:

/usr/ccs/bin/ld: Unsatisfied symbols:
sigwait (code)

which is not what I was expecting, because there is a definition for
sigwait in /usr/include (though not in a header file we use). It must
be in some add-on library instead of libc.

Could be part of "DCE threads" (= "draft 4"), from last time I googled
this topic when gaur blew up.

However, wrasse did this:

"/export/home/nm/farm/studio64v12_6/HEAD/pgsql.build/../pgsql/src/bin/psql/command.c", line 5062: prototype mismatch: 2 args passed, 1 expected
cc: acomp failed for /export/home/nm/farm/studio64v12_6/HEAD/pgsql.build/../pgsql/src/bin/psql/command.c

which I was expecting even less. Evidently, prehistoric HPUX is not the
only platform still using the pre-POSIX API for this function. So we
really do need the full configure check.

Huh, that is surprising. Apparently it has "draft 6" and also final,
depending on a macro:

https://illumos.org/man/2/sigwait
https://docs.oracle.com/cd/E36784_01/html/E36872/sigwait-2.html

Here's a first swing at a configure check. Could probably be improved
a bit, as I haven't yet tried to share the check with thread_test.c (I
was going to ask if it might be entirely removable since draft 4
systems like gaur surely can't compile the rest of thread_test.c due
to other differences we know about, but now that I know that there are
also draft 6 systems out there...). Note also the errno change, which
I (ironically) did the pre-standard way by mistake.

On Tue, Jul 13, 2021 at 5:18 PM Thomas Munro <thomas.munro@gmail.com> wrote:

https://illumos.org/man/2/sigwait
https://docs.oracle.com/cd/E36784_01/html/E36872/sigwait-2.html

Oh, hmm, that's something we already have to do elsewhere to get
standard conforming interfaces on that platform:

# Some platforms use these, so just define them. They can't hurt if they
# are not supported. For example, on Solaris -D_POSIX_PTHREAD_SEMANTICS
# enables 5-arg getpwuid_r, among other things.
PTHREAD_CFLAGS="$PTHREAD_CFLAGS -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS"

Here's a better patch. I check if there is any declaration at all,
which ancient HPUX should fail based on:

command.c:5062:4: warning: implicit declaration of function 'sigwait'

Then I also check that there isn't an incompatible declaration with
the technique from thread_test.c, which Solaris should fail, based on:

command.c:5062:8: error: too many arguments to function 'sigwait'

A well placed -D_POSIX_PTHREAD_SEMANTICS might allow it work on
Solaris, but I'm not sure if it's OK to do that without other
threading options and I don't have access to test.

Thomas Munro <thomas.munro@gmail.com> writes:

Here's a better patch.

I didn't like the way you'd done this as two independent tests;
the second one falsely reports success if there isn't actually
any declaration of sigwait. I see you hacked around that by
checking both results in c.h, but the printed result is still
very misleading. Plus it's a waste of time to run the second
test at all, if the first one fails.

Here's a revised patch that I've tested (albeit lightly) on
both HPUX and Solaris. I also got rid of the independent
check in thread_test.c, since that's always been a hack
that delivers a misleading complaint when it fails.

Personally, I'd skip adding the stanza in c.h in favor of just
making configure set USE_SIGWAIT directly, but perhaps you
see it differently.

regards, tom lane

I wrote:

Here's a revised patch that I've tested (albeit lightly) on
both HPUX and Solaris.

Hm, I'd verified the configure results, but I didn't wait around
for the builds to finish, which was a mistake. On Solaris 11:

command.c: In function 'do_watch':
command.c:5062:8: error: too many arguments to function 'sigwait'
if (sigwait(&sigalrm_sigchld_sigint, &signal_received) < 0)
^
In file included from ../../../src/include/fe_utils/print.h:16:0,
from command.h:12,
from command.c:28:
/usr/include/signal.h:233:12: note: declared here
extern int sigwait(sigset_t *);
^
gmake[3]: *** [<builtin>: command.o] Error 1
gmake[2]: *** [Makefile:43: all-psql-recurse] Error 2
gmake[1]: *** [Makefile:42: all-bin-recurse] Error 2
gmake: *** [GNUmakefile:11: all-src-recurse] Error 2

What is happening is obvious in retrospect. Per my patch, configure
checks how sigwait is declared with -D_POSIX_PTHREAD_SEMANTICS, and
that's also how libpq's reference is compiled ... but psql's reference
is not compiled that way. It seems to work if we force command.c
to be compiled with PTHREAD_CFLAGS added, but that is really quite
grotty. I think it'd be safer to push the sigwait reference out
into a separate file to minimize the scope of those flags.

Just how badly did you want to use sigwait here? I'm having
considerable second thoughts about the value of that change
versus the hoops we're going to have to jump through to use it.

(I suppose a hacky solution might be to never define USE_SIGWAIT
on Solaris.)

regards, tom lane

On Wed, Jul 14, 2021 at 6:17 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Just how badly did you want to use sigwait here? I'm having
considerable second thoughts about the value of that change
versus the hoops we're going to have to jump through to use it.

I'm sure I could find another way if it comes to it. The original
proposal had a sleep(0.1) loop... I thought this was more elegant, but
obviously I didn't know about this portability problem and there are
surely other ways. (Searching for sigwait in the archives brings up
some fun discussion of other long forgotten dinosaurs that also had
this problem.)

(I suppose a hacky solution might be to never define USE_SIGWAIT
on Solaris.)

Yeah I was thinking about that. I've just got a shell on an illumos
VM and will try a couple of ideas out if I can remember how to drive
this thing... more soon.

On Wed, Jul 14, 2021 at 4:08 PM Thomas Munro <thomas.munro@gmail.com> wrote:

On Wed, Jul 14, 2021 at 6:17 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:

(I suppose a hacky solution might be to never define USE_SIGWAIT
on Solaris.)

Yeah I was thinking about that. I've just got a shell on an illumos
VM and will try a couple of ideas out if I can remember how to drive
this thing... more soon.

I decided to try to make it work properly on that OS instead of the
hacky solution now that I have access. I took your last patch, moved
-D_POSIX_PTHREAD_SEMANTICS into CFLAGS in src/template/solaris,
removed the special flags and libs from the new configure check where
you had them, and then used HAVE_POSIX_SIGWAIT directly without the
c.h change. I can't find any evidence on the 'net that any other OS
cares about _POSIX_PTHREAD_SEMANTICS (every reference says it's a
Solaris thing) so this should just remove some useless clutter, and
it's not like we want POSIX draft 6 behaviour anywhere. (It's a bit
weird to define a macro that has PTHREAD in the name when building
things that aren't multi-threaded, but that was their choice).

Do you see any problems with this approach? The main thing I can
think of is that people who are using a custom CFLAGS on that OS will
need to update the value they're using, but that seems OK (they'll
likely see the new error and be alerted to the change). Another thing
I noticed is that config/ax_pthread.m4 (something from autoconf that
we don't want to modify) will add another copy of
-D_POSIX_PTHREAD_SEMANTICS, but that's already the case.

Thomas Munro <thomas.munro@gmail.com> writes:

I decided to try to make it work properly on that OS instead of the
hacky solution now that I have access. I took your last patch, moved
-D_POSIX_PTHREAD_SEMANTICS into CFLAGS in src/template/solaris,

Aha! Yeah, just defining _POSIX_PTHREAD_SEMANTICS unconditionally
on Solaris seems like a good idea. However, messing with the template
file is not the best way to inject that, because if the user specifies
CFLAGS then the template file's setting is ignored. It's better to
handle it in the part of configure.ac that automatically adds flags
onto whatever the user or template file gave. Also, being a -D
switch, it really oughta go into CPPFLAGS not CFLAGS.

I've tested the attached on the GCC farm's Solaris machine (which
I believe is the same machine wrasse runs on), and it seems OK.
I think it's ready to go.

I can't find any evidence on the 'net that any other OS
cares about _POSIX_PTHREAD_SEMANTICS (every reference says it's a
Solaris thing) so this should just remove some useless clutter,

Agreed; if we do find any other platforms that need it, we can extend
the PORTNAME check used here.

Another thing
I noticed is that config/ax_pthread.m4 (something from autoconf that
we don't want to modify) will add another copy of
-D_POSIX_PTHREAD_SEMANTICS, but that's already the case.

Ah, yeah, I see:

$ grep POSIX src/Makefile.global
PTHREAD_CFLAGS = -D_POSIX_PTHREAD_SEMANTICS -pthread -D_REENTRANT -D_THREAD_SAFE
CPPFLAGS = -D_POSIX_PTHREAD_SEMANTICS
Seems pretty harmless, and anyway we have to make sure this gets in there
whether or not --enable-thread-safety is given.

regards, tom lane

On Thu, Jul 15, 2021 at 4:07 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:

I've tested the attached on the GCC farm's Solaris machine (which
I believe is the same machine wrasse runs on), and it seems OK.
I think it's ready to go.

Pushed. Thanks!