Still wondering about random numbers...
I am running on OpenBSD and Linux, both of which have
cryptographic-quality RNGs built in. When I call RANDOM() in PG, do I
get the old C library random numbers, which are not very random, or do
I get high-quality random numbers from the crypto-RNG that's built in?
Any sugestions for getting high-quality random numbers?
I am running on OpenBSD and Linux, both of which have
cryptographic-quality RNGs built in. When I call RANDOM() in PG, do I
get the old C library random numbers, which are not very random, or do
I get high-quality random numbers from the crypto-RNG that's built in?
Any sugestions for getting high-quality random numbers?
Looking through the source, I find:
/*
* drandom - returns a random number
*/
Datum
drandom(PG_FUNCTION_ARGS)
{
float8 result;
/* result 0.0-1.0 */
result = ((double) random()) / ((double) MAX_RANDOM_VALUE);
PG_RETURN_FLOAT8(result);
}
Looks like plain old C library random numbers. If you want to use
/dev/random or /dev/urandom, best bet would be to write your own C function.
You should also check through the contrib stuff -- I don't recall seeing RNG
functions in there, but I'm constantly amazed at what is in contrib that I
didn't notice before ;-)
HTH,
Joe
"Joe Conway" <joseph.conway@home.com> writes:
Looks like plain old C library random numbers. If you want to use
/dev/random or /dev/urandom, best bet would be to write your own C function.
You should also check through the contrib stuff -- I don't recall seeing RNG
functions in there,
I don't see any in there either, but this seems like a fine candidate for
a contrib item. I doubt we'd accept it into the mainstream for lack of
portability, but as a contrib item, why not?
regards, tom lane
Looks like plain old C library random numbers. If you want to use
/dev/random or /dev/urandom, best bet would be to write your own C function.
You should also check through the contrib stuff -- I don't recall seeing RNG
functions in there,I don't see any in there either, but this seems like a fine candidate for
a contrib item. I doubt we'd accept it into the mainstream for lack of
portability, but as a contrib item, why not?
configure is our friend. A better random number generator is always
welcome imho, and if it can be reliably supported through autoconf then
it would stand a chance to be included in the main tree. Especially
since the usage of random() seems to be very isolated in the code.
contrib/ is always a good first step though...
- Thomas
Joe Conway wrote:
I am running on OpenBSD and Linux, both of which have
cryptographic-quality RNGs built in. When I call RANDOM() in PG, do I
get the old C library random numbers, which are not very random, or do
I get high-quality random numbers from the crypto-RNG that's built in?
Any sugestions for getting high-quality random numbers?Looking through the source, I find:
result = ((double) random()) / ((double) MAX_RANDOM_VALUE);
The random() function returns better (good?)
random numbers. The rand() function returns poor
random numbers. random() also depends on how
much state it is initialized with (and may or may
not be based on /dev/random, I don't know).
j
Looks like plain old C library random numbers. If you want to use
/dev/random or /dev/urandom, best bet would be to write your own C
function.
You should also check through the contrib stuff -- I don't recall
seeing RNG
functions in there,
I don't see any in there either, but this seems like a fine candidate
for
a contrib item. I doubt we'd accept it into the mainstream for lack of
portability, but as a contrib item, why not?configure is our friend. A better random number generator is always
welcome imho, and if it can be reliably supported through autoconf then
it would stand a chance to be included in the main tree. Especially
since the usage of random() seems to be very isolated in the code.
contrib/ is always a good first step though...- Thomas
Sounds like there's some interest in this. I just wrote a function for
generating random initialization vectors from /dev/urandom within the past
week -- should be easy to make a pgsql contrib out of it. I'll send
something to patches in a couple of days or so.
-- Joe
On Tue, Aug 07, 2001 at 04:36:23PM +0000,
Thomas Lockhart <lockhart@fourpalms.org> wrote:
configure is our friend. A better random number generator is always
welcome imho, and if it can be reliably supported through autoconf then
it would stand a chance to be included in the main tree. Especially
since the usage of random() seems to be very isolated in the code.
contrib/ is always a good first step though...
'Better' is in the eye of the beholder. /dev/random can stall for
very long periods of time waiting for entropy. Even /dev/urandom may
be overkill for some uses of random numbers. There is also the question
of how many bytes the function is going to return.
While it would be nice to have some access to /dev/random and /dev/urandom,
I don't know that replacing the current random function is the best way
to do this.
'Better' is in the eye of the beholder. /dev/random can stall for
very long periods of time waiting for entropy. Even /dev/urandom may
be overkill for some uses of random numbers. There is also the question
of how many bytes the function is going to return.While it would be nice to have some access to /dev/random and /dev/urandom,
I don't know that replacing the current random function is the best way
to do this.
I'm the one who originally posted this question, and I agree, you have
a very good point. /dev/random relies on a pool of entropy, which is
limited. Applications which require a large volume of low-quality
random numbers (perhaps for rendering in a video game) should
definitely not use /dev/random, but applications which require a small
amount of high-quality randomness (generating cryptographic session
keys perhaps) should use it. PG should have both calls available. If
anyone has written a C function which calls the crypto-random
generator, which I could link in, if you would mail it to me, I would
be most appreciative, because I am using this to generate
cryptographic challenges, session keys and the like, which really do
need crypto-quality random numbers.
As an intermediate solution, I can use SHA1 to generate a sequence of
hashes, which does work as a fairly good crypto-RNG. If anyone wants,
I can post the SHA1 code for PG. It should really be in the
distribution, I think. But then again, so should AES and RSA and
OpenPGP...
Bruno Wolff III wrote:
On Tue, Aug 07, 2001 at 04:36:23PM +0000,
Thomas Lockhart <lockhart@fourpalms.org> wrote:configure is our friend. A better random number generator is always
welcome imho, and if it can be reliably supported through autoconf then
it would stand a chance to be included in the main tree. Especially
since the usage of random() seems to be very isolated in the code.
contrib/ is always a good first step though...'Better' is in the eye of the beholder. /dev/random can stall for
very long periods of time waiting for entropy.
On Intel i8x0 motherboards you can set the CONFIG_INTEL_RNG kernel parameter and access the hardware random entropy generator.
On other motherboards, reading from /dev/random can stall indefinitely. This is not a Good Thing. /dev/urandom is fine, but not rally better than rand(3) or random(3).
Is it actually possible to replace the built-in function? I haven't checked.
If it is, then contrib is a fine place for the improvement. If it isn't, then configure is probably our friend.
Just my �0.02.
Allan.
Bruno Wolff III wrote:
On Tue, Aug 07, 2001 at 04:36:23PM +0000,
Thomas Lockhart <lockhart@fourpalms.org> wrote:configure is our friend. A better random number generator is always
welcome imho, and if it can be reliably supported through autoconf then
it would stand a chance to be included in the main tree. Especially
since the usage of random() seems to be very isolated in the code.
contrib/ is always a good first step though...'Better' is in the eye of the beholder. /dev/random can stall for
very long periods of time waiting for entropy.
On Intel i8x0 motherboards you can set the CONFIG_INTEL_RNG kernel parameter and access the hardware random entropy generator.
On other motherboards, reading from /dev/random can stall indefinitely. This is not a Good Thing. /dev/urandom is fine, but not rally better than rand(3) or random(3).
Is it actually possible to replace the built-in function? I haven't checked.
If it is, then contrib is a fine place for the improvement. If it isn't, then configure is probably our friend.
Just my �0.02.
Allan.
I'm the one who originally posted this question, and I agree, you have
a very good point. /dev/random relies on a pool of entropy, which is
limited. Applications which require a large volume of low-quality
random numbers (perhaps for rendering in a video game) should
definitely not use /dev/random, but applications which require a small
amount of high-quality randomness (generating cryptographic session
keys perhaps) should use it. PG should have both calls available. If
anyone has written a C function which calls the crypto-random
generator, which I could link in, if you would mail it to me, I would
be most appreciative, because I am using this to generate
cryptographic challenges, session keys and the like, which really do
need crypto-quality random numbers.
Isn't /dev/random best used for seeding the random number generator,
rather than for getting random number?
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026Isn't /dev/random best used for seeding the random number generator,
rather than for getting random number?
It is best used for any situation when quality random numbers are
needed. This includes seeding, and in some cases it may also include
generating session keys and other things. It's good to have a
choice. Right now I have a lot of pl/pgsql which calls plain old
RANDOM() to generate session keys, and that is not good. Before I
launch this thing, I will need to find a way of getting better random
numbers. If anyone has created a function like that for PG, please
mail me, because I need it.
Isn't /dev/random best used for seeding the random number generator,
rather than for getting random number?It is best used for any situation when quality random numbers are
needed. This includes seeding, and in some cases it may also include
generating session keys and other things. It's good to have a
choice. Right now I have a lot of pl/pgsql which calls plain old
RANDOM() to generate session keys, and that is not good. Before I
launch this thing, I will need to find a way of getting better random
numbers. If anyone has created a function like that for PG, please
mail me, because I need it.
As I said earlier, I just recently created a function for something else I'm
working on which returns a binary string from /dev/urandom of whatever
length you request. I'm using it for initialization vectors (IVs) for 3des
cbc and for session keys. It should be relatively easy to turn into a pgsql
contrib function, so I'll post something to patches within the next couple
of days. It would not be intended to replace the standard RANDOM() function,
just compliment it. Biggest question in my mind is the return type -- should
it return pure binary as a bytea type, or hex as a text type? Any thoughts
on this?
I agree with other comments regarding /dev/random. During testing I found
that it can block for significant periods of time, especially when used
repetitively.
-- Joe
Allan Engelhardt <allane@cybaea.com> writes:
On other motherboards, reading from /dev/random can stall
indefinitely. This is not a Good Thing. /dev/urandom is fine, but
not rally better than rand(3) or random(3).
Wrong; it's still a lot better, especially if you have a reasonable
amount of entropy coming in--/dev/urandom uses the same entropy pool
as /dev/random and generates its data using a cryptographically secure
hash function. This is still a lot better (for crypto purposes) than
the simple LCGs used in the standard C library functions.
See the random(4) manpage on your Linux system for more details.
-Doug
--
Free Dmitry Sklyarov!
http://www.freesklyarov.org/
We will return to our regularly scheduled signature shortly.
Import Notes
Reply to msg id not found: AllanEngelhardt'smessageofTue07Aug2001220759+0100
Doug McNaught wrote:
Allan Engelhardt <allane@cybaea.com> writes:
On other motherboards, reading from /dev/random can stall
indefinitely. This is not a Good Thing. /dev/urandom is fine, but
not rally better than rand(3) or random(3).Wrong; it's still a lot better, especially if you have a reasonable
amount of entropy coming in--/dev/urandom uses the same entropy pool
as /dev/random and generates its data using a cryptographically secure
hash function. This is still a lot better (for crypto purposes) than
the simple LCGs used in the standard C library functions.
Absolutely! I had minor brain damage when I wrote the paragraph. What I meant was:
"/dev/urandom is not really better than rand(3) or random(3) *in this situation* [i.e. when reads from /dev/random stalls and there is no system entropy]"
You don't get a lot of entropy from the standard /dev/random drivers on a system without users (pressing a key gives 10 bytes of entropy, moving the mouse ~8), but you do get a some so it is better.
As you said.
Allan.
I'm almost dead certain (although I've been too lazy to actually look,)
that
network activity also fills the entropy pool (on Linux boxen). I
conjecture
this partly based on my headless "home services" box which acts as
firewall,
web server, game server, pg server, blah, blah, and blah on my LAN. It
has
never wanted for entropy, and has actually never blocked on reads to
/dev/random
for any (human) appreciable lengths of time. It drinks from the entropy
pool
for key generation and suchlike frequently and deeply.
This may require netfilter.
-- Matt
-----Original Message-----
From: pgsql-general-owner@postgresql.org
[mailto:pgsql-general-owner@postgresql.org] On Behalf Of Allan
Engelhardt
Sent: Wednesday, August 08, 2001 6:23 AM
To: Doug McNaught
Cc: Bruno Wolff III; pgsql-general@postgresql.org
Subject: Re: [GENERAL] Re: Still wondering about random numbers...
Doug McNaught wrote:
Allan Engelhardt <allane@cybaea.com> writes:
On other motherboards, reading from /dev/random can stall
indefinitely. This is not a Good Thing. /dev/urandom is fine, but
not rally better than rand(3) or random(3).Wrong; it's still a lot better, especially if you have a reasonable
amount of entropy coming in--/dev/urandom uses the same entropy pool
as /dev/random and generates its data using a cryptographically secure
hash function. This is still a lot better (for crypto purposes) than
the simple LCGs used in the standard C library functions.
Absolutely! I had minor brain damage when I wrote the paragraph. What
I meant was:
"/dev/urandom is not really better than rand(3) or random(3) *in this
situation* [i.e. when reads from /dev/random stalls and there is no
system entropy]"
You don't get a lot of entropy from the standard /dev/random drivers on
a system without users (pressing a key gives 10 bytes of entropy, moving
the mouse ~8), but you do get a some so it is better.
As you said.
Allan.
---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
Allan Engelhardt <allane@cybaea.com> writes:
"/dev/urandom is not really better than rand(3) or random(3) *in
this situation* [i.e. when reads from /dev/random stalls and there
is no system entropy]"
I would still disagree. The difference for crypto purposes between a
CRNG seeded with real entropy (/dev/urandom) and an LCG (libc
functions) is huge. The former is useful (with caveats); the latter
is trivially breakable.
-Doug
--
Free Dmitry Sklyarov!
http://www.freesklyarov.org/
We will return to our regularly scheduled signature shortly.
Import Notes
Reply to msg id not found: AllanEngelhardt'smessageofWed08Aug2001112231+0100
generating session keys and other things. It's good to have a
choice. Right now I have a lot of pl/pgsql which calls plain old
RANDOM() to generate session keys, and that is not good. Before I
launch this thing, I will need to find a way of getting better random
numbers. If anyone has created a function like that for PG, please
mail me, because I need it.As I said earlier, I just recently created a function for something else
I'm
working on which returns a binary string from /dev/urandom of whatever
length you request. I'm using it for initialization vectors (IVs) for 3des
cbc and for session keys. It should be relatively easy to turn into a
pgsql
contrib function, so I'll post something to patches within the next couple
of days. It would not be intended to replace the standard RANDOM()
function,
just compliment it. Biggest question in my mind is the return type --
should
it return pure binary as a bytea type, or hex as a text type? Any thoughts
on this?
Per the discussion yesterday, here's a patch. There are two versions of
essentially the same function. They both take an int as the number of
requested random bytes, and generate a random binary string of the requested
length from /dev/urandom. The first one (randomstr_hex) converts the binary
to hex and returns it as text, and the other (randomstr_bytea) does the
needed escaping of special characters and returns bytea.
Let me know if you have any other thoughts on how this can be improved. I
didn't bother to create a function to return a random 0 - 1 interval float
(ala random()) because in most cases where cryptographic quality random
values are needed, the provided forms are the most useful ones (I think).
-- Joe
Attachments:
randomstr_r01.diffapplication/octet-stream; name=randomstr_r01.diffDownload+391-0
Is there any reason why integer values should *not* be quoted in statements
passed to pgsql? it seems like there are few benefits of this practice,
curios what the drawbacks may be.
TIA!
Ryan Mahoney
Doug McNaught wrote:
Allan Engelhardt <allane@cybaea.com> writes:
"/dev/urandom is not really better than rand(3) or random(3) *in
this situation* [i.e. when reads from /dev/random stalls and there
is no system entropy]"I would still disagree. The difference for crypto purposes between a
CRNG seeded with real entropy (/dev/urandom) and an LCG (libc
functions) is huge. The former is useful (with caveats); the latter
is trivially breakable.
Fair comment. I agree. Thanks for the clarification.
Allan.
