pgkill on win32

James Mansion wrote:

I'm wondering if the mechanism used for sending signals between
postmaster processes on Win32 is much more heavyweight that is
necessary.

Is there a reason not to call OpenThread on the target postmaster's
thread id, and then use QueueUserAPC to execute a 'signal handler'
method on it? (Or Terminate Thread for 'extreme' cases).

Yes. We used to use APCs, but touching anything remotely related to
Winsock from an APC is not supported... We had a lot of trouble with it
initially, and it took a long support case with Microsoft PSS to figure
out what was broken, because this being unsupported was not properly
documented.

Haven't tried it - but I can't help thinking that the named pipe
service is a bit heavyweight.

It certainly is ;-) We could probably find something more efficient,
but APCs are not the one.

//Magnus

Magnus Hagander wrote:

Yes. We used to use APCs, but touching anything remotely related to
Winsock from an APC is not supported... We had a lot of trouble with it

By implication you'd be doing socket'y stuff from the signal handler on
UNIX? Scary.
I was assuming it would be used to signal an event that would release
the main
thread and then do the real work there.

I suppose by implication you can't knock a thread out of select from the
APC?

Though, presumably, you could hand over the request to a waiting
application thread
and *that* would have full WinSock access.

I can help feeling that the eventselect mechanism is the one to use, not
the crappy
bsd select emulation.

initially, and it took a long support case with Microsoft PSS to figure
out what was broken, because this being unsupported was not properly
documented.

Indeed. And its a very odd limitation given the way that APCs are used
with completion ports.
Did any sort of reference get written to the knowledge base for this, do
you know?

It certainly is ;-) We could probably find something more efficient,
but APCs are not the one.

The concern I have isn't so much that the mechanism might be a bit ugly,
but the synchronous
delivery and the scheduling gymnastics implied..

James Mansion wrote:

Magnus Hagander wrote:

Yes. We used to use APCs, but touching anything remotely related to
Winsock from an APC is not supported... We had a lot of trouble
with it

By implication you'd be doing socket'y stuff from the signal handler
on UNIX? Scary.

Uh, sorry, got that explained backwards.
The problem is when winsock operations are interrupted by APCs.

See:
http://archives.postgresql.org/pgsql-hackers-win32/2004-04/msg00013.php

I was assuming it would be used to signal an event that would release
the main
thread and then do the real work there.

I suppose by implication you can't knock a thread out of select from
the APC?

Not only that, but you may corrupt internal winsock structures if you
try.

As stated, one major issue is that we have third-party libs calling the
winsock functions directly (SSL and Kerberos come to mind - the GSSAPI
code specifically does not do this). For SSL, I'd like to eventually
rewrite it to make sure *we* call the send and receive functions and
just use the library to encrypt buffers, which should make things a
bit safer in that regard.

I can help feeling that the eventselect mechanism is the one to use,
not the crappy bsd select emulation.

The original design, which is what is there now, was written to make
sure we made as little change as possible to the backend code itself,
and instead implemented portability layers. I think now that the port
is more mature, if we need to make some smaller changes around say the
select loop processing, we can do that - as long as we're careful. But
we certainly don't want to do anything that'll require more or less
global changes around the code.

initially, and it took a long support case with Microsoft PSS to
figure out what was broken, because this being unsupported was not
properly documented.

Indeed. And its a very odd limitation given the way that APCs are
used with completion ports.
Did any sort of reference get written to the knowledge base for this,
do you know?

There was one written, AFIAK, but I don't remember the number or can't
find it right now. It may well be that it's premier only. Also, it only
applied to traditional sockets API calls, and not "native" Winsock2
calls. But see above about SSL.

It certainly is ;-) We could probably find something more efficient,
but APCs are not the one.

The concern I have isn't so much that the mechanism might be a bit
ugly, but the synchronous
delivery and the scheduling gymnastics implied..

Right.

//Magnus

Magnus Hagander wrote:

The problem is when winsock operations are interrupted by APCs.

See:
http://archives.postgresql.org/pgsql-hackers-win32/2004-04/msg00013.php

Whoa! Indeed, that's a bit sucky because they really aren't documented
as interruptible.

In this case though I see not material problem with going back to APCs.
At the moment you
deliver to a pipe server thread with a pipe RPC. I can't see why you
cannot deliver to a
signal handling thread with the APC - the published {thread-id,function}
tuple does not need
to refer to the main thread for the process. This would de-synchronize
the delivery but make
a relatively small change since that background thread could deliver to
the main thread the
same way it does now.

If there were any desire to provide a MT-aware postmaster, the same
technique of masking
signals except on a signal thread might apply.

James

On Mon, Apr 21, 2008 at 09:47:02AM +0200, Magnus Hagander wrote:

Uh, sorry, got that explained backwards.
The problem is when winsock operations are interrupted by APCs.

See:
http://archives.postgresql.org/pgsql-hackers-win32/2004-04/msg00013.php

WRT this bit:

(1) carries a big problem - if we do this, we can *not* support SSL
on win32, because it uses recv() and send() internally (inside the
openssl library). Therefor, I don't think (1) is really an option -
at least not in the long run.

openssl does not have to use recv/send, that's what the BIO
infrastructure is for. The current backend code already constructs BIOs
for reading to support interrupting (see my_sock_read), so you could
replace the read with any functions you wanted.

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

Martijn van Oosterhout wrote:

On Mon, Apr 21, 2008 at 09:47:02AM +0200, Magnus Hagander wrote:

Uh, sorry, got that explained backwards.
The problem is when winsock operations are interrupted by APCs.

See:
http://archives.postgresql.org/pgsql-hackers-win32/2004-04/msg00013.php

WRT this bit:

(1) carries a big problem - if we do this, we can *not* support SSL
on win32, because it uses recv() and send() internally (inside the
openssl library). Therefor, I don't think (1) is really an option -
at least not in the long run.

openssl does not have to use recv/send, that's what the BIO
infrastructure is for. The current backend code already constructs
BIOs for reading to support interrupting (see my_sock_read), so you
could replace the read with any functions you wanted.

Yes, it's on my TODO list to do just that. Unless this means you are
volunteering to do it? ;)

//Magnus

James Mansion wrote:

Magnus Hagander wrote:

The problem is when winsock operations are interrupted by APCs.

See:
http://archives.postgresql.org/pgsql-hackers-win32/2004-04/msg00013.php

Whoa! Indeed, that's a bit sucky because they really aren't
documented as interruptible.

In this case though I see not material problem with going back to
APCs. At the moment you
deliver to a pipe server thread with a pipe RPC. I can't see why you
cannot deliver to a
signal handling thread with the APC - the published
{thread-id,function} tuple does not need
to refer to the main thread for the process. This would
de-synchronize the delivery but make
a relatively small change since that background thread could deliver
to the main thread the
same way it does now.

Good point. I see no reason not to be able to do that, agreed. Once we
accepted using a second thread for it, that opportunity is certainly
there.

You interested in trying to code up a patch to verify that? ;)

If there were any desire to provide a MT-aware postmaster, the same
technique of masking
signals except on a signal thread might apply.

Define MT-aware :-) It's certainly MT-aware in the fact that it's
already MT... But there is no interest in making the actual backends
launch as threads in the postmaster - at least not currently.

//Magnus

Magnus Hagander wrote:

You interested in trying to code up a patch to verify that? ;)

Practical reality says that I won't get to this before the next version
of Windows is released.
I don't want to promise something I can't deliver.

If there were any desire to provide a MT-aware postmaster, the same
technique of masking
signals except on a signal thread might apply.

Define MT-aware :-) It's certainly MT-aware in the fact that it's
already MT... But there is no interest in making the actual backends
launch as threads in the postmaster - at least not currently.

I seem to remember being slapped about for daring to suggest using a
threaded embedded
language even if only one thread calls into the core, on the ground that
the signals might not
go to the right thread. So I'm assuming that a thread-aware build would
generally mask async
signals and wait for them in a specific thread in sigwait, which would
effectively match the
Win32 model (for a threaded build).

On the other hand, I'd normally regard signals as the work of the devil
and prefer to send
a wakeup via some other IPC, for pretty much that reason, but there you go.

James

James Mansion wrote:

Magnus Hagander wrote:

You interested in trying to code up a patch to verify that? ;)

Practical reality says that I won't get to this before the next
version of Windows is released.
I don't want to promise something I can't deliver.

:-)
If you want to throw me some code-snippet-ideas off-list that's not
ready for an actual patch, be my guest - and maybe I can put something
together.

If there were any desire to provide a MT-aware postmaster, the
same technique of masking
signals except on a signal thread might apply.

Define MT-aware :-) It's certainly MT-aware in the fact that it's
already MT... But there is no interest in making the actual backends
launch as threads in the postmaster - at least not currently.

I seem to remember being slapped about for daring to suggest using a
threaded embedded
language even if only one thread calls into the core, on the ground
that the signals might not
go to the right thread. So I'm assuming that a thread-aware build
would generally mask async
signals and wait for them in a specific thread in sigwait, which
would effectively match the
Win32 model (for a threaded build).

That is something different than a threaded build, though ;-) You're
probably more likely to get that to happen - though maybe not by much...

//Magnus