Query cancel and OOB data

Bruce Momjian wrote:

OK, thanks to Tom Lane's many patches, I have query cancel working on my
machine. However, it is not working with Unix domain sockets. I get:

Cannot send cancel request:
PQrequestCancel() -- couldn't send OOB data: errno=45
Operation not supported

This is under BSDI 3.1.

Do Unix Domain sockets support OOB(out-of-band) data?

Unix domain sockets don't support OOB (Stevens, Unix Network Programming).

Yea, I found that too, late last night, Section 6.14, page 332.

I basically need some way to 'signal' the backend of a cancellation
request. Polling the socket is not an option because it would impose
too great a performance penalty. Maybe async-io on a read(), but that
is not going to be very portable.

I could pass the backend pid to the front end, and send a kill(SIG_URG)
to that pid on a cancel, but the frontend can be running as a different
user than the backend. Problem is, the only communcation channel is
that unix domain socket.

We basically need some way to get the attention of the backend,
hopefully via some signal.

Any ideas?

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Bruce Momjian wrote:

OK, thanks to Tom Lane's many patches, I have query cancel working on my
machine. However, it is not working with Unix domain sockets. I get:

Cannot send cancel request:
PQrequestCancel() -- couldn't send OOB data: errno=45
Operation not supported

This is under BSDI 3.1.

Do Unix Domain sockets support OOB(out-of-band) data?

Unix domain sockets don't support OOB (Stevens, Unix Network Programming).

Yea, I found that too, late last night, Section 6.14, page 332.

I basically need some way to 'signal' the backend of a cancellation
request. Polling the socket is not an option because it would impose
too great a performance penalty. Maybe async-io on a read(), but that
is not going to be very portable.

I could pass the backend pid to the front end, and send a kill(SIG_URG)
to that pid on a cancel, but the frontend can be running as a different
user than the backend. Problem is, the only communcation channel is
that unix domain socket.

We basically need some way to get the attention of the backend,
hopefully via some signal.

Any ideas?

Use TCP. On most modern systems (eg Linux ;-) ), TCP especially on the local
machine is very efficient. Not quite as efficient as a Unix domain socket,
but close enough that no one will notice.

To investigate this, see Larry McVoy's wonderful lmbench suite...

-dg

David Gould dg@illustra.com 510.628.3783 or 510.305.9468
Informix Software (No, really) 300 Lakeside Drive Oakland, CA 94612
"Of course, someone who knows more about this will correct me if I'm wrong,
and someone who knows less will correct me if I'm right."
--David Palmer (palmer@tybalt.caltech.edu)

Bruce Momjian wrote:

OK, thanks to Tom Lane's many patches, I have query cancel working on my
machine. However, it is not working with Unix domain sockets. I get:

Cannot send cancel request:
PQrequestCancel() -- couldn't send OOB data: errno=45
Operation not supported

This is under BSDI 3.1.

Do Unix Domain sockets support OOB(out-of-band) data?

Unix domain sockets don't support OOB (Stevens, Unix Network Programming).

Yea, I found that too, late last night, Section 6.14, page 332.

I basically need some way to 'signal' the backend of a cancellation
request. Polling the socket is not an option because it would impose
too great a performance penalty. Maybe async-io on a read(), but that
is not going to be very portable.

I could pass the backend pid to the front end, and send a kill(SIG_URG)
to that pid on a cancel, but the frontend can be running as a different
user than the backend. Problem is, the only communcation channel is
that unix domain socket.

We basically need some way to get the attention of the backend,
hopefully via some signal.

Any ideas?

Use TCP. On most modern systems (eg Linux ;-) ), TCP especially on the local
machine is very efficient. Not quite as efficient as a Unix domain socket,
but close enough that no one will notice.

To investigate this, see Larry McVoy's wonderful lmbench suite...

We implemented Unix domain sockets for performance, and security. Hard
to beat a Unix domain socket's security. I need the SIB_URG signal.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Use TCP. On most modern systems (eg Linux ;-) ), TCP especially on the

local

machine is very efficient. Not quite as efficient as a Unix domain socket,
but close enough that no one will notice.

Unix domain sockets are not only more efficient. They also are less of a
security
risk IMO.

With regards from Maurice.

Any ideas?

Use TCP. On most modern systems (eg Linux ;-) ), TCP especially on the local
machine is very efficient. Not quite as efficient as a Unix domain socket,
but close enough that no one will notice.

I have not experienced noteable performance differences on my AIX box either.
Of course when doing client server communication I always suggest PVM3.
It can be configured to do nearly all current communication methods,
including hardware based ones on MPP/NUMA systems.

Andreas

Bruce Momjian wrote:

Bruce Momjian wrote:

Yea, I found that too, late last night, Section 6.14, page 332.

I basically need some way to 'signal' the backend of a cancellation
request. Polling the socket is not an option because it would impose
too great a performance penalty. Maybe async-io on a read(), but that
is not going to be very portable.

I could pass the backend pid to the front end, and send a kill(SIG_URG)
to that pid on a cancel, but the frontend can be running as a different
user than the backend. Problem is, the only communcation channel is
that unix domain socket.

We basically need some way to get the attention of the backend,
hopefully via some signal.

Any ideas?

postmaster could be listening (adding to select()) on a "signal socket"
for cancel request and shot down its children on request.

how do we make such a scheme secure ??

terveiset,
--
---------------------------------------------
G�ran Thyni, sysadm, JMS Bildbasen, Kiruna

We basically need some way to get the attention of the backend,
hopefully via some signal.

Any ideas?

postmaster could be listening (adding to select()) on a "signal socket"
for cancel request and shot down its children on request.

how do we make such a scheme secure ??

I think that is the big question.
-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

I basically need some way to 'signal' the backend of a cancellation
request. Polling the socket is not an option because it would impose
too great a performance penalty. Maybe async-io on a read(), but that
is not going to be very portable.

I could pass the backend pid to the front end, and send a kill(SIG_URG)
to that pid on a cancel, but the frontend can be running as a different
user than the backend. Problem is, the only communcation channel is
that unix domain socket.

We basically need some way to get the attention of the backend,
hopefully via some signal.

OK, I think I have a solution. I recommend we pass the backend pid to
the client as part of connection startup. Then, when the client wants
to cancel a query, it sends a cancel packet to its backend (new packet
type), and then sends that pid to the postmaster with a new packet type.

When the postmaster receives the packet with the pid, it sends a signal
to that pid/backend. The backend does a recv(MSG_PEEK) to see if it has
a pending packet with a cancel request. If it does, it cancels, if not,
it ignores it. In the read loop of the backend, all cancel requests are
ignored.

So the cancel packet to the postmaster only causes the backend to look
for a pending cancel packet.

This does a few things for us. It allows us to use cancel in unix
domain sockets, and in Java or anything that can't support OOB. In
fact, I would recommend discarding OOB in favor of this method.

Also, it does not require the postmaster to authenticate the cancel
request. This could be hard, especially if the user has to type in a
password. No one wants to type in a password to cancel a query.

Comments?

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Bruce Momjian <maillist@candle.pha.pa.us> writes:

OK, I think I have a solution. I recommend we pass the backend pid to
the client as part of connection startup. Then, when the client wants
to cancel a query, it sends a cancel packet to its backend (new packet
type), and then sends that pid to the postmaster with a new packet type.

When the postmaster receives the packet with the pid, it sends a signal
to that pid/backend. The backend does a recv(MSG_PEEK) to see if it has
a pending packet with a cancel request. If it does, it cancels, if not,
it ignores it. In the read loop of the backend, all cancel requests are
ignored.

OK, I guess the point of sending the normal-channel packet is to
authenticate the cancel request? Otherwise anyone could send a cancel
request to the postmaster, if they know the backend PID.

I see a few flaws however:

1. What if the postmaster/signal/backend path is completed before the
normal-channel cancel packet arrives? The backend looks, sees no
packet, and ignores the request. Oops. This scenario is not at all
implausible across a remote connection, since the first transmission
of the normal-channel packet might be lost to a data glitch. By the
time the client-side TCP stack decides to retransmit, it's too late.

2. I don't think you could use this to abort out of a COPY IN transfer,
because the confirmation packet would be impossible to distinguish
from data reliably. In general there's a risk of confusion if the
server might be looking for the confirmation packet when the client
thinks it's in the middle of sending a regular request.

3. There's still a possibility of a denial-of-service attack.
A bad guy could send a flood of cancel requests with the right PID,
and he'd slow down the server substantially even if nothing ever gets
cancelled. (Also, because of point 2, some of the forged cancels
might succeed...)

This does a few things for us. It allows us to use cancel in unix
domain sockets, and in Java or anything that can't support OOB. In
fact, I would recommend discarding OOB in favor of this method.

The real advantage of OOB for this purpose is that there's no
possibility of confusing the cancel request with normal data.

I still like the idea I floated a couple days ago: have the initial
handshake provide both the PID of the backend and a "secret code"
randomly generated by the server for that connection. The client
must transmit both the PID and the code to the postmaster for the
cancel request to be accepted. That method has all the advantages:

1. The client doesn't have to supply a password; libpq will retain
all the necessary info internally.

2. The probability of defeating the scheme can be made arbitrarily
small (much smaller than guessing a password, say) with a long enough
secret code. 8 or so random bytes ought to do.

3. There's no problem with synchronization between the client/postmaster
and client/backend data paths, because no data need be sent across the
client/backend path. This is just as good as using OOB to keep the
cancel separate from normal traffic.

4. Don't have to depend on having OOB facility.

The only disadvantage I can see is having to open a new postmaster
connection every time you want to cancel; but hopefully that won't
be very often, so performance shouldn't be much of an issue.

regards, tom lane

I have taken some time to think about this some more.

Bruce Momjian <maillist@candle.pha.pa.us> writes:

OK, I think I have a solution. I recommend we pass the backend pid to
the client as part of connection startup. Then, when the client wants
to cancel a query, it sends a cancel packet to its backend (new packet
type), and then sends that pid to the postmaster with a new packet type.

When the postmaster receives the packet with the pid, it sends a signal
to that pid/backend. The backend does a recv(MSG_PEEK) to see if it has
a pending packet with a cancel request. If it does, it cancels, if not,
it ignores it. In the read loop of the backend, all cancel requests are
ignored.

OK, I guess the point of sending the normal-channel packet is to
authenticate the cancel request? Otherwise anyone could send a cancel
request to the postmaster, if they know the backend PID.

Yes, that is the intent of the normal-channel packet.

I see a few flaws however:

1. What if the postmaster/signal/backend path is completed before the
normal-channel cancel packet arrives? The backend looks, sees no
packet, and ignores the request. Oops. This scenario is not at all
implausible across a remote connection, since the first transmission
of the normal-channel packet might be lost to a data glitch. By the
time the client-side TCP stack decides to retransmit, it's too late.

Yes, this could happen, but it is only a cancel request. Another way to
proceed is to have the server query the client after it receives the
request from the postmaster, but that seems odd.

2. I don't think you could use this to abort out of a COPY IN transfer,
because the confirmation packet would be impossible to distinguish
from data reliably. In general there's a risk of confusion if the
server might be looking for the confirmation packet when the client
thinks it's in the middle of sending a regular request.

Yes, that is a good point.

3. There's still a possibility of a denial-of-service attack.
A bad guy could send a flood of cancel requests with the right PID,
and he'd slow down the server substantially even if nothing ever gets
cancelled. (Also, because of point 2, some of the forged cancels
might succeed...)

Yes, but does this increase our denial-of-service vulnerability, or just
give the person one more way to slow things down?

This does a few things for us. It allows us to use cancel in unix
domain sockets, and in Java or anything that can't support OOB. In
fact, I would recommend discarding OOB in favor of this method.

The real advantage of OOB for this purpose is that there's no
possibility of confusing the cancel request with normal data.

Yes, that is true. I just am grasping for a unix domain and
java/non-oob solution.

I still like the idea I floated a couple days ago: have the initial
handshake provide both the PID of the backend and a "secret code"
randomly generated by the server for that connection. The client
must transmit both the PID and the code to the postmaster for the
cancel request to be accepted. That method has all the advantages:

1. The client doesn't have to supply a password; libpq will retain
all the necessary info internally.

2. The probability of defeating the scheme can be made arbitrarily
small (much smaller than guessing a password, say) with a long enough
secret code. 8 or so random bytes ought to do.

3. There's no problem with synchronization between the client/postmaster
and client/backend data paths, because no data need be sent across the
client/backend path. This is just as good as using OOB to keep the
cancel separate from normal traffic.

4. Don't have to depend on having OOB facility.

The only disadvantage I can see is having to open a new postmaster
connection every time you want to cancel; but hopefully that won't
be very often, so performance shouldn't be much of an issue.

Yes, the overhead of opening a new postmaster connection is very small,
especially because no backend is started. I was trying to avoid the
'magic cookie' solution for a few reasons:

1) generating a random secret codes can be slow (I may be wrong)

2) the random key is sent across the network with a cancel
request, so once it is used, it can be used by a malcontent to cancel
any query for that backend. He doesn't need to spoof any packets to
insert it into the TCP/IP stream, he just connects to the postmaster and
sends the secret key. For long-running queries, that may be a problem.
Not sure how much of a vulnerability that is.

3) I hesitate to add the bookkeeping in the postmaster and libpq
of that pid/secret key combination. Seems like some bloat we could do
without.

4) You have to store the secret key in the client address space,
possibly open to snooping.

However, in thinking about it, I don't think there is any way to avoid
your solution of pid/secret key. The postmaster, on receiving the
secret key, can send a signal to the backend, and the query will be
cancelled. Nothing will be sent along the backend/client channel. All
other interfaces that want cancel handling will have to add some code
for this too.

This basically simulates OOB by sending a message to the postmaster,
which is always listening, and having it send a signal, which is
possible because they are owned by the same user.

Actually, in:

ConnCreate()

I see a call to:

RandomSalt(port->salt);

Any idea what that is used for? Maybe we can use that? And why is it
being generated for non-crypt connections? Seems like a waste if
random() is an expensive function. He calls it twice, once for each of
the two salt characters. Looks like a cheap function on BSDI from the
looks of the library code.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

However, in thinking about it, I don't think there is any way to avoid
your solution of pid/secret key. The postmaster, on receiving the
secret key, can send a signal to the backend, and the query will be
cancelled. Nothing will be sent along the backend/client channel. All
other interfaces that want cancel handling will have to add some code
for this too.

Assuming that every user has a password which is known by both the client
and the server, it seem to me like using a one-way function based on the
clientuser password as the secret key (refered to above) is appropiate.
This avoids the need for introducing "yet another shared secret into the
system".

A one-way function is expected to make it computationaly infeasible to
deduce the password given the secretkey. One-way functions (SHA1, MD5) are
also quite fast. (If I'm not mistaken these functions are allowed to be
exported
from the US. )

By including a cancel request id (together with the user password) with the
information being hashed (by the one-way function) it is also possible to
detect (and avoid) denial of service attacks
(which are based on replaying the cancel request secret keys).

This does however imply that a certain amount of extra booking is needed.

With regards from Maurice.

Bruce Momjian <maillist@candle.pha.pa.us> writes:

I was trying to avoid the
'magic cookie' solution for a few reasons:

1) generating a random secret codes can be slow (I may be wrong)

Not really. A typical system rand() subroutine is a multiply and an
add. For the moment I'd recommend generating an 8-byte random key with
something like

for (i=0; i<8; i++)
key[i] = rand() & 0xFF;

which isn't going to take enough time to notice.

The above isn't cryptographically secure (which means that a person who
receives a "random" key generated this way might be able to predict the
next one you generate). But it will do to get the protocol debugged,
and we can improve it later. I have Schneier's "Applied Cryptography"
and will study its chapter on secure random number generators.

2) the random key is sent across the network with a cancel
request, so once it is used, it can be used by a malcontent to cancel
any query for that backend.

True, if you have a packet sniffer then you've got big troubles ---
on the other hand, a packet sniffer can also grab your password,
make his own connection to the server, and wreak much more havoc
than just issuing a cancel. I don't see that this adds any
vulnerability that wasn't there before.

3) I hesitate to add the bookkeeping in the postmaster and libpq
of that pid/secret key combination. Seems like some bloat we could do
without.

The libpq-side bookkeeping is trivial. I'm not sure about the
postmaster though. Does the postmaster currently keep track of
all operating backend processes, or not? If it does, then another
field per process doesn't seem like a problem.

4) You have to store the secret key in the client address space,
possibly open to snooping.

See password. In any case, someone with access to the client address
space can probably manage to send packets from the client, too. So
"security" based on access to the client/backend connection isn't any
better.

This basically simulates OOB by sending a message to the postmaster,
which is always listening, and having it send a signal, which is
possible because they are owned by the same user.

Right.

Maybe we should look at this as a fallback that libpq uses if it
tries OOB and that doesn't work? Or is it not a good idea to have
two mechanisms?

regards, tom lane

"Maurice Gittens" <mgittens@gits.nl> writes:

Assuming that every user has a password which is known by both the client
and the server, it seem to me like using a one-way function based on the
clientuser password as the secret key (refered to above) is appropiate.
This avoids the need for introducing "yet another shared secret into the
system".

Well, I think that the cancel security mechanism ought to be per backend
process, not per user. That is, simply being the same "Postgres user"
should not give you the ability to issue a cancel; you ought to be
required to have some direct association with a particular client/backend
session. Access to the client/backend connection channel is one way;
knowledge of a per-connection secret is another.

Also, isn't it true that not all the supported authentication mechanisms
use a password? Taking this approach would mean we have to design a new
cancel security mechanism for each authentication protocol.

regards, tom lane

-----Original Message-----
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Maurice Gittens <mgittens@gits.nl>
Cc: hackers@postgreSQL.org <hackers@postgreSQL.org>
Date: zondag 24 mei 1998 23:52
Subject: Re: [HACKERS] Query cancel and OOB data

"Maurice Gittens" <mgittens@gits.nl> writes:

Assuming that every user has a password which is known by both the client
and the server, it seem to me like using a one-way function based on the
clientuser password as the secret key (refered to above) is appropiate.
This avoids the need for introducing "yet another shared secret into the
system".

Well, I think that the cancel security mechanism ought to be per backend
process, not per user.

I assumed that this was understood.

That is, simply being the same "Postgres user"
should not give you the ability to issue a cancel; you ought to be
required to have some direct association with a particular client/backend
session. Access to the client/backend connection channel is one way;
knowledge of a per-connection secret is another.

Also, isn't it true that not all the supported authentication mechanisms
use a password? Taking this approach would mean we have to design a new
cancel security mechanism for each authentication protocol.

This may be true. The point I'm trying to make is that using one
way-functions
together with a shared secret will make it possible to avoid denial of
service attacks
which rely on replaying the "magic token".

Again I assumed it to be understood that the pid of the particular backend
would exchanged with the client during the initial handshake. It would also
be included (together with the shared secret e.g. the password and
and some form of a sequence id) in the one-way hash.

regards, tom lane

Regards, Maurice.

Bruce Momjian <maillist@candle.pha.pa.us> writes:

I was trying to avoid the
'magic cookie' solution for a few reasons:

1) generating a random secret codes can be slow (I may be wrong)

Not really. A typical system rand() subroutine is a multiply and an
add. For the moment I'd recommend generating an 8-byte random key with
something like

for (i=0; i<8; i++)
key[i] = rand() & 0xFF;

which isn't going to take enough time to notice.

Actually, just sending a random int as returned from random() is enough.
random() returns a long here, but just cast it to int.

The above isn't cryptographically secure (which means that a person who
receives a "random" key generated this way might be able to predict the
next one you generate). But it will do to get the protocol debugged,
and we can improve it later. I have Schneier's "Applied Cryptography"
and will study its chapter on secure random number generators.

Yes, that may be true. Not sure if having a single random() value can
predict the next one. If we just use on random() return value, I don't
think that is possible.

2) the random key is sent across the network with a cancel
request, so once it is used, it can be used by a malcontent to cancel
any query for that backend.

True, if you have a packet sniffer then you've got big troubles ---
on the other hand, a packet sniffer can also grab your password,
make his own connection to the server, and wreak much more havoc
than just issuing a cancel. I don't see that this adds any
vulnerability that wasn't there before.

Yes.

3) I hesitate to add the bookkeeping in the postmaster and libpq
of that pid/secret key combination. Seems like some bloat we could do
without.

The libpq-side bookkeeping is trivial. I'm not sure about the
postmaster though. Does the postmaster currently keep track of
all operating backend processes, or not? If it does, then another
field per process doesn't seem like a problem.

Yes. The backend does already have such a per-connection structure, so
adding it is trivial too.

4) You have to store the secret key in the client address space,
possibly open to snooping.

See password. In any case, someone with access to the client address
space can probably manage to send packets from the client, too. So
"security" based on access to the client/backend connection isn't any
better.

Yep.

This basically simulates OOB by sending a message to the postmaster,
which is always listening, and having it send a signal, which is
possible because they are owned by the same user.

Right.

Maybe we should look at this as a fallback that libpq uses if it
tries OOB and that doesn't work? Or is it not a good idea to have
two mechanisms?

You have convinced me. Let's bag OOB, and use this new machanism. I
can do the backend changes, I think.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

"Maurice Gittens" <mgittens@gits.nl> writes:

Assuming that every user has a password which is known by both the client
and the server, it seem to me like using a one-way function based on the
clientuser password as the secret key (refered to above) is appropiate.
This avoids the need for introducing "yet another shared secret into the
system".

Well, I think that the cancel security mechanism ought to be per backend
process, not per user. That is, simply being the same "Postgres user"
should not give you the ability to issue a cancel; you ought to be
required to have some direct association with a particular client/backend
session. Access to the client/backend connection channel is one way;
knowledge of a per-connection secret is another.

Also, isn't it true that not all the supported authentication mechanisms
use a password? Taking this approach would mean we have to design a new
cancel security mechanism for each authentication protocol.

Yes, most connections don't have passwords. Better to keep cancel
separate.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Tom Lane <tgl@sss.pgh.pa.us> writes:

on the other hand, a packet sniffer can also grab your password,
make his own connection to the server, and wreak much more havoc
than just issuing a cancel. I don't see that this adds any
vulnerability that wasn't there before.

Ahem. Not true for those of us who use Kerberos authentication.
We never send our passwords over the network, instead using them
as (part of) a key that's used to encrypt other data.

-tih
--
Popularity is the hallmark of mediocrity. --Niles Crane, "Frasier"

Tom Lane <tgl@sss.pgh.pa.us> writes:

on the other hand, a packet sniffer can also grab your password,
make his own connection to the server, and wreak much more havoc
than just issuing a cancel. I don't see that this adds any
vulnerability that wasn't there before.

Ahem. Not true for those of us who use Kerberos authentication.
We never send our passwords over the network, instead using them
as (part of) a key that's used to encrypt other data.

OK, lets review this, with thought about our various authentication
options:

trust, password, ident, crypt, krb4, krb5

As far as I know, they all transmit queries and results as clear text
across the network. They encrypt the passwords and tickets, but not the
data. [Even kerberos does not encrypt the data stream, does it?]

So, if someone snoops the network, they will see the query and results,
and see the cancel secret key. Of course, once they see the cancel
secret key, it is trivial for them to send that to the postmaster to
cancel a query. However, if they are already snooping, how much harder
is it for them to insert their own query into the tcp stream? If it is
as easy as sending the cancel secret key, then the additional
vulnerability of being able to replay the cancel packet is trivial
compared to the ability to send your own query, so we don't loose
anything by using a non-encrypted cancel secret key.

Of course, if the stream were encrypted, they could not see the secret key
needs to be accepted and sent in an encrypted format.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Bruce Momjian <maillist@candle.pha.pa.us> writes:

OK, lets review this, with thought about our various authentication
options:

trust, password, ident, crypt, krb4, krb5

As far as I know, they all transmit queries and results as clear text
across the network. They encrypt the passwords and tickets, but not the
data. [Even kerberos does not encrypt the data stream, does it?]

True. Encrypted communication should be an option, though. With
Kerberos, the ability to do this securely is already there in the
library, so it would be natural to use it. Adding encryption to the
communication between client and postmaster is probably a good thing
even if we don't (yet) encrypt that between client and backend, and
would also be a good, simple way to start implementing it.

-tih
--
Popularity is the hallmark of mediocrity. --Niles Crane, "Frasier"

"Maurice Gittens" <mgittens@gits.nl> writes:

This may be true. The point I'm trying to make is that using one
way-functions together with a shared secret will make it possible to
avoid denial of service attacks which rely on replaying the "magic
token".

Again I assumed it to be understood that the pid of the particular backend
would exchanged with the client during the initial handshake. It would also
be included (together with the shared secret e.g. the password and
and some form of a sequence id) in the one-way hash.

Ah, now I think I see your point: you want to encrypt the cancel request
so that even a packet sniffer could not generate additional cancel
requests after seeing the first one. That seems like a good idea, but
there is still the problem of what to use for the encryption key (the
"shared secret"). A password would work in those authentication schemes
that have a password, but what about those that don't?

More generally, I think we risk overdesigning the cancel authorization
mechanism while failing to deal with systemic security issues. Above
we are blithely assuming that a user's Postgres password is secret ...
which it is hardly likely to be against an attacker with packet-sniffing
capability. I don't think it's worth trying to make the cancel mechanism
(alone) proof against attacks that really need to be dealt with by
using a secure transport method.

regards, tom lane

Bruce Momjian <maillist@candle.pha.pa.us> writes:

Yes, that may be true. Not sure if having a single random() value can
predict the next one. If we just use on random() return value, I don't
think that is possible.

In typical rand() implementations, having the whole of one output value
is sufficient to give you all future outputs. That's why I recommended
using only 8 bits from each of several outputs. I believe that is still
breakable, but less trivially so. (I will be going on vacation
Wednesday morning and don't have time to research better methods before
then, but I do know they exist.)

The real question we need to ask here is not the details of generating
a one-time secret key, but what attacks we need to defend against and
how to do that. A simple secret code per my original proposal is clearly
not proof against a packet-sniffing attacker. Should we beef up the
coding, or consider that such an attacker must be met directly by
encrypting communications? If the latter, how do we encrypt the first
packet sent to or from the postmaster?

regards, tom lane

Tom Lane wrote:

Bruce Momjian <maillist@candle.pha.pa.us> writes:

I was trying to avoid the
'magic cookie' solution for a few reasons:

1) generating a random secret codes can be slow (I may be wrong)

Not really. A typical system rand() subroutine is a multiply and an
add. For the moment I'd recommend generating an 8-byte random key with
something like

for (i=0; i<8; i++)
key[i] = rand() & 0xFF;

which isn't going to take enough time to notice.

The above isn't cryptographically secure (which means that a person who
receives a "random" key generated this way might be able to predict the
next one you generate). But it will do to get the protocol debugged,
and we can improve it later. I have Schneier's "Applied Cryptography"
and will study its chapter on secure random number generators.

A neat feature of linux is that it has a kernel random number
generator which is fed random data from interrupt times. The only
drawback is that this is sort of a "pool", so whn the pool is full,
drawing 8 bytes from it is not a problem, but when the pool is
drained, it can take some time to generate more data. At any rate, it
might be a good starting point for a postgres random number generator
-- sample usage of shared memory and perform a hash on this. From
"applied cryptography":

"In effect, the system degrades gracefully from perfect to practical
randomness when the demand exceeds the supply. In this case it
becomes theoretically possible .. to determine a previous or
subsequent result. But this requires inverting MD5, which is
computationally infeasible"

applied cryptography, 2nd eddition, p427.

This is sort of what we want. As random as the key can be, but able
to generate a pseudo-random key if we're short on time.

Ocie

/dev/urandom performs a similar function without the wait. of course
not all of the data is new, but it should still be pretty secure.

On Tue, 26 May 1998, at 14:00:57, ocie@paracel.com wrote:

Tom Lane wrote:

"Maurice Gittens" <mgittens@gits.nl> writes:

This may be true. The point I'm trying to make is that using one
way-functions together with a shared secret will make it possible to
avoid denial of service attacks which rely on replaying the "magic
token".

Again I assumed it to be understood that the pid of the particular backend
would exchanged with the client during the initial handshake. It would also
be included (together with the shared secret e.g. the password and
and some form of a sequence id) in the one-way hash.

Ah, now I think I see your point: you want to encrypt the cancel request
so that even a packet sniffer could not generate additional cancel
requests after seeing the first one. That seems like a good idea, but
there is still the problem of what to use for the encryption key (the
"shared secret"). A password would work in those authentication schemes
that have a password, but what about those that don't?

Aha!

I'm slowly working through back emails, so I apologize if someone else
already posted this. If we want to create a shared secret between the
postmaster and the client, we should think about the Diffe-Helman
algorithm.

For those unfamiliar with this, we start by picking large numbers b
and m. The client picks a number k and then sends K=b^k%m, while the
server picks a number l and sends L=b^l%m. The client calculates
L^k%m and the server calculates K^l%m, and these numbers are
identical. A third party eavesdropping on the conversation would only
get K and L, and would have no idea what the shared number is, unless
they can calculate the computationally infeasible discrete logarithm.

Anyway, something to think about.

Ocie

OK, lets review this, with thought about our various authentication
options:

trust, password, ident, crypt, krb4, krb5

As far as I know, they all transmit queries and results as clear text
across the network. They encrypt the passwords and tickets, but not the
data. [Even kerberos does not encrypt the data stream, does it?]

So, if someone snoops the network, they will see the query and results,
and see the cancel secret key. Of course, once they see the cancel
secret key, it is trivial for them to send that to the postmaster to
cancel a query. However, if they are already snooping, how much harder
is it for them to insert their own query into the tcp stream? If it is
as easy as sending the cancel secret key, then the additional
vulnerability of being able to replay the cancel packet is trivial
compared to the ability to send your own query, so we don't loose
anything by using a non-encrypted cancel secret key.

Can someone answer this for me?

Of course, if the stream were encrypted, they could not see the secret key
needs to be accepted and sent in an encrypted format.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
+  If your life is a hard drive,     |  (610) 353-9879(w)
+  Christ can be your backup.        |  (610) 853-3000(h)

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Bruce Momjian <maillist@candle.pha.pa.us> writes:

However, if they are already snooping, how much harder
is it for them to insert their own query into the tcp stream?

Can someone answer this for me?

Well, that depends entirely on what your threat model is --- for
example, someone with read access on /dev/kmem on a relay machine
might be able to watch packets going by, yet not be able to inject
more. On the other hand, someone with root privileges on another
machine on your local LAN could likely do both.

My guess is that most of the plausible cases that allow one also
allow the other. But it's only a guess.

regards, tom lane

ocie@paracel.com writes:

If we want to create a shared secret between the
postmaster and the client, we should think about the Diffe-Helman
[ discrete logarithm ] algorithm.

I used Diffie-Hellman for that purpose years ago, and perhaps could
still dig up the code for it. But I thought discrete logarithm had been
broken since then, or at least shown to be far less intractable than
people thought. In any case, D-H is pretty slow --- are we prepared to
add seconds to the backend startup time in the name of security?

regards, tom lane

On Tue, 26 May 1998, Tom Lane wrote:

I used Diffie-Hellman for that purpose years ago, and perhaps could
still dig up the code for it. But I thought discrete logarithm had been
broken since then, or at least shown to be far less intractable than
people thought. In any case, D-H is pretty slow --- are we prepared to
add seconds to the backend startup time in the name of security?

I think everyone is thinking too hard on this issue.

Transport security should be just that.

Use SSL or Kerberos encryption if you wish thoe entire session to be (more
or less) unsnoopable/unspoofable.

Trying to hack things in will only result in an incomplete and/or ugly
solution.

The way I see it people have several choices:

- Run with no network listeners and therefore no network clients to expose
to snooping/spoofing attacks.

- Require SSLed or Kerberized connections, incuring longer startup times
but insuring a secure channel.

- Use SKIP or some other IP level encryption system to provide a secure
'virtual lan' insuring a secure channel.

- Isolate communication across secure, private networks insuring a secure
channel.

So long as we make people aware of the risks they are exposing themselves
to, adding 'security features' in places better left to lower level
protocols is unnecessary.

/*
Matthew N. Dodd | A memory retaining a love you had for life
winter@jurai.net | As cruel as it seems nothing ever seems to
http://www.jurai.net/~winter | go right - FLA M 3.1:53
*/

Matthew N. Dodd wrote:

On Tue, 26 May 1998, Tom Lane wrote:

I used Diffie-Hellman for that purpose years ago, and perhaps could
still dig up the code for it. But I thought discrete logarithm had been
broken since then, or at least shown to be far less intractable than
people thought. In any case, D-H is pretty slow --- are we prepared to
add seconds to the backend startup time in the name of security?

I think everyone is thinking too hard on this issue.

Transport security should be just that.

Use SSL or Kerberos encryption if you wish thoe entire session to be (more
or less) unsnoopable/unspoofable.

Trying to hack things in will only result in an incomplete and/or ugly
solution.

The way I see it people have several choices:

- Run with no network listeners and therefore no network clients to expose
to snooping/spoofing attacks.

- Require SSLed or Kerberized connections, incuring longer startup times
but insuring a secure channel.

- Use SKIP or some other IP level encryption system to provide a secure
'virtual lan' insuring a secure channel.

- Isolate communication across secure, private networks insuring a secure
channel.

So long as we make people aware of the risks they are exposing themselves
to, adding 'security features' in places better left to lower level
protocols is unnecessary.

HMM, you do make a convincing argument. As one of my H.S. teachers
used to say, we are putting "Descartes before Horace". Probably
better to just have the postmaster generate and issue a random number
to the client.

It would be nice if this can be done in a forward/backward-compatible
way. I.E. old clients that don't know ablout cancelling should be
able to work with servers that can cancel, and newer clients that can
cancel should be able to disable this feature if talking with an older
server. A rolling database gathers no development community :)

Ocie

Bruce Momjian <maillist@candle.pha.pa.us> writes:

However, if they are already snooping, how much harder
is it for them to insert their own query into the tcp stream?

Can someone answer this for me?

Well, that depends entirely on what your threat model is --- for
example, someone with read access on /dev/kmem on a relay machine
might be able to watch packets going by, yet not be able to inject
more. On the other hand, someone with root privileges on another
machine on your local LAN could likely do both.

My guess is that most of the plausible cases that allow one also
allow the other. But it's only a guess.

Ok, I agree. If someone sees the cancel secret going over the wire,
they almost just as easily can send their own query on the wire.
Remember, the cancel is going directly to the postmaster as a separate
connection, so it is a little easier than spoofing a packet.

So, with that decided, the only issue is how the postmaster should
generate the random key. Currently, to get the password salt, which
does not have to be un-guessable, RandomSalt() seeds the random number
generator with the current time, and then just continues to call random.

If we continue that practice, someone can guess the first cancel
password by finding out when the first postgres backend needed a random
number, and use that time in seconds to figure out the new random
number. We could load/save the seed on postmaster start/exit, and
somehow seed that value during install or initdb. Perhaps the
completion time of initdb can be used. Maybe a:

'date +%s' >/pgsql/data/pg_random

and have the postmaster load it on startup, and write it on exit.
Because initdb takes some time to run, we could put it in between two of
the initdb commands that take some time to run. Their timestamp of
activity is burried in pgsql/data and only postgres read-able.

Comments?

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Bruce Momjian <maillist@candle.pha.pa.us> writes:

However, if they are already snooping, how much harder
is it for them to insert their own query into the tcp stream?

Can someone answer this for me?

Well, that depends entirely on what your threat model is --- for
example, someone with read access on /dev/kmem on a relay machine
might be able to watch packets going by, yet not be able to inject
more. On the other hand, someone with root privileges on another
machine on your local LAN could likely do both.

My guess is that most of the plausible cases that allow one also
allow the other. But it's only a guess.

Oh, yes, one more thing. When generating the cancel key, We will have
to call random twice and return part of each so users will not see our
current random values.

When I remove the exec(), people will be able to call random() in the
backend to see the random value. May need to reset the seed on
backend startup.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Matthew N. Dodd writes:

I think everyone is thinking too hard on this issue.

Transport security should be just that.

Use SSL or Kerberos encryption if you wish thoe entire session to be (more
or less) unsnoopable/unspoofable.

Trying to hack things in will only result in an incomplete and/or ugly
solution.

The way I see it people have several choices:

- Run with no network listeners and therefore no network clients to expose
to snooping/spoofing attacks.

- Require SSLed or Kerberized connections, incuring longer startup times
but insuring a secure channel.

- Use SKIP or some other IP level encryption system to provide a secure
'virtual lan' insuring a secure channel.

- Isolate communication across secure, private networks insuring a secure
channel.

So long as we make people aware of the risks they are exposing themselves
to, adding 'security features' in places better left to lower level
protocols is unnecessary.

Right on. I have been following this discussion about securing the
cancel channel hoping for it to come back to earth and now it has.

All the major systems I am familiar with (Sybase, Informix, Illustra,
MS SQL Server) use TCP as their primary client/server transport and do not
use encryption (most even send cleartext passwords over the wire). Some of
these systems support only TCP.

The assumption is that the dbms and clients are on a private network and not
exposed to the internet at large except through gateways of some kind.
As I have not heard any horror stories about breakins, denial of service
etc at customer sites in my ten years working with this stuff, I assume
that while it may happen, it does not happen often enough for the customers
to complain to their db vendors about.

The other thing is that security is hard. It is hard to make a system
secure, and it is even harder to make it usable after you make it secure.
And if you don't make it usable, then you find the office and dumpsters filled
with post-its with passwords on them.

Likewise, most environments are not really secure anyway, it will usually be
easier to hack a root shell and kill the postmaster or copy out the data
base files than to fool around figuring out the postgres on the wire traffic.

-dg

David Gould dg@illustra.com 510.628.3783 or 510.305.9468
Informix Software 300 Lakeside Drive Oakland, CA 94612
- A child of five could understand this! Fetch me a child of five.

Bruce Momjian <maillist@candle.pha.pa.us> writes:

However, if they are already snooping, how much harder
is it for them to insert their own query into the tcp stream?

Can someone answer this for me?

Well, that depends entirely on what your threat model is --- for
example, someone with read access on /dev/kmem on a relay machine
might be able to watch packets going by, yet not be able to inject
more. On the other hand, someone with root privileges on another
machine on your local LAN could likely do both.

My guess is that most of the plausible cases that allow one also
allow the other. But it's only a guess.

Ok, I agree. If someone sees the cancel secret going over the wire,
they almost just as easily can send their own query on the wire.
Remember, the cancel is going directly to the postmaster as a separate
connection, so it is a little easier than spoofing a packet.

So, with that decided, the only issue is how the postmaster should
generate the random key. Currently, to get the password salt, which
does not have to be un-guessable, RandomSalt() seeds the random number
generator with the current time, and then just continues to call random.

Just do time and pid. But get the time from gettimeofday so it will be
down to the millisecond or so. Anything more is overkill for this application.

If we continue that practice, someone can guess the first cancel
password by finding out when the first postgres backend needed a random
number, and use that time in seconds to figure out the new random
number. We could load/save the seed on postmaster start/exit, and
somehow seed that value during install or initdb. Perhaps the
completion time of initdb can be used. Maybe a:

'date +%s' >/pgsql/data/pg_random

and have the postmaster load it on startup, and write it on exit.
Because initdb takes some time to run, we could put it in between two of
the initdb commands that take some time to run. Their timestamp of
activity is burried in pgsql/data and only postgres read-able.

Comments?

See Mr Dodds excellent post. This is getting too elaborate.

One possibility, make it configurable to allow cancels at all. Then if
someone really had a spurious cancel problem they could work around it by
turning cancels off.

But hey, I think we should just use TCP only and then we could count on OOB.

Btw, on my P166 at work, lmbench says Linux 2.1.101 can do round trip tcp
in 125 microseconds. That is pretty quick.

-dg

David Gould dg@illustra.com 510.628.3783 or 510.305.9468
Informix Software 300 Lakeside Drive Oakland, CA 94612
- A child of five could understand this! Fetch me a child of five.

Just do time and pid. But get the time from gettimeofday so it will be
down to the millisecond or so. Anything more is overkill for this application.

You have given me exactly what I needed. If I run gettimeofday() on
postmaster startup, and run it when the first backend is started, I can
use the microseconds from both calls to generate a truely random seed.
Even if the clock is only accurate to 10 ms, I still get a 10,000 random
keyspace. I can mix the values by taking/swapping the low and high
16-bit parts so even with poor resolution, both get used.

The micro-second times are not visible via ps, or probably even kept in
the kernel, so these values will work fine.

Once random is seeded, for each backend we call random twice and return
a merge of the two random values. I wonder if we just call random once,
and XOR it with our randeom seed, if that would be just as good or
better? Cryptologists?

Comments? Sounds like a plan. The thought of giving the users yet
another option to disable cancel just made me squirm.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

From: Bruce Momjian <maillist@candle.pha.pa.us>
Date: Mon, 1 Jun 1998 00:53:21 -0400 (EDT)

...

Just do time and pid. But get the time from gettimeofday so it will be
down to the millisecond or so. Anything more is overkill for this application.

You have given me exactly what I needed. If I run gettimeofday() on
postmaster startup, and run it when the first backend is started, I can
use the microseconds from both calls to generate a truely random seed.
Even if the clock is only accurate to 10 ms, I still get a 10,000 random
keyspace. I can mix the values by taking/swapping the low and high
16-bit parts so even with poor resolution, both get used.

The micro-second times are not visible via ps, or probably even kept in
the kernel, so these values will work fine.

Once random is seeded, for each backend we call random twice and return
a merge of the two random values. I wonder if we just call random once,
and XOR it with our randeom seed, if that would be just as good or
better? Cryptologists?

Comments? Sounds like a plan. The thought of giving the users yet
another option to disable cancel just made me squirm.

For FreeBSD and Linux, isn't /dev/urandom the method of choice for
getting random bits? [I've been away from this thread awhile - please
excuse if this option was already discussed].

From: Bruce Momjian <maillist@candle.pha.pa.us>
Date: Mon, 1 Jun 1998 00:53:21 -0400 (EDT)

...

Just do time and pid. But get the time from gettimeofday so it will be
down to the millisecond or so. Anything more is overkill for this application.

You have given me exactly what I needed. If I run gettimeofday() on
postmaster startup, and run it when the first backend is started, I can
use the microseconds from both calls to generate a truely random seed.
Even if the clock is only accurate to 10 ms, I still get a 10,000 random
keyspace. I can mix the values by taking/swapping the low and high
16-bit parts so even with poor resolution, both get used.

The micro-second times are not visible via ps, or probably even kept in
the kernel, so these values will work fine.

Once random is seeded, for each backend we call random twice and return
a merge of the two random values. I wonder if we just call random once,
and XOR it with our randeom seed, if that would be just as good or
better? Cryptologists?

Comments? Sounds like a plan. The thought of giving the users yet
another option to disable cancel just made me squirm.

For FreeBSD and Linux, isn't /dev/urandom the method of choice for
getting random bits? [I've been away from this thread awhile - please
excuse if this option was already discussed].

Not available on most/all platforms.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Hal Snyder wrote:

From: Bruce Momjian <maillist@candle.pha.pa.us>
Date: Mon, 1 Jun 1998 00:53:21 -0400 (EDT)

...

Just do time and pid. But get the time from gettimeofday so it will be
down to the millisecond or so. Anything more is overkill for this application.

You have given me exactly what I needed. If I run gettimeofday() on
postmaster startup, and run it when the first backend is started, I can
use the microseconds from both calls to generate a truely random seed.
Even if the clock is only accurate to 10 ms, I still get a 10,000 random
keyspace. I can mix the values by taking/swapping the low and high
16-bit parts so even with poor resolution, both get used.

The micro-second times are not visible via ps, or probably even kept in
the kernel, so these values will work fine.

Once random is seeded, for each backend we call random twice and return
a merge of the two random values. I wonder if we just call random once,
and XOR it with our randeom seed, if that would be just as good or
better? Cryptologists?

Comments? Sounds like a plan. The thought of giving the users yet
another option to disable cancel just made me squirm.

For FreeBSD and Linux, isn't /dev/urandom the method of choice for
getting random bits? [I've been away from this thread awhile - please
excuse if this option was already discussed].

I believe /dev/random is guaranteed to be "random", while /dev/urandom
is guaranteed to return a certain number of psuedorandom bytes in a
given time. These are not universally available though. Seeding with
bits taken from the pid and hi-res time should be OK.

Something I did on a similar task was to set up a max-keys-per-key and
max-elapsed-time-per-key. Basically, seed the random number generator
when the postmaster starts, and reseed after every 10 keys, or if 10
minutes have elapsed since the random number generator was las seeded.
This way, the would be attacker can't know for sure when the random
number generator was last reseeded.

Ocie Mitchell