pgcrypto 3des failure, OpenSSL 0.9.8, Solaris 9/sparc
On my Solaris 9/sparc box with OpenSSL 0.9.8-beta6, the pgcrypto
regression tests fail the 3des test. I haven't checked against
older versions of OpenSSL; I'll do so when I get a chance.
I haven't dug into the pgcrypto code yet -- is it doing anything
that might be platform-specific? Or is this more likely a problem
with OpenSSL?
regression.diffs attached.
--
Michael Fuhr
http://www.fuhr.org/~mfuhr/
Attachments:
regression.diffstext/plain; charset=us-asciiDownload
*** ./expected/3des.out Sun Mar 20 22:24:51 2005
--- ./results/3des.out Tue Jul 5 08:20:26 2005
***************
*** 17,23 ****
select encode( encrypt('', 'foo', '3des'), 'hex');
encode
------------------
! 9b641a6936249eb4
(1 row)
-- 10 bytes key
--- 17,23 ----
select encode( encrypt('', 'foo', '3des'), 'hex');
encode
------------------
! 4d91b5c9cc64e1c9
(1 row)
-- 10 bytes key
***************
*** 24,30 ****
select encode( encrypt('foo', '0123456789', '3des'), 'hex');
encode
------------------
! 6f02b7076a366504
(1 row)
-- 22 bytes key
--- 24,30 ----
select encode( encrypt('foo', '0123456789', '3des'), 'hex');
encode
------------------
! d2fb8baa1717cb02
(1 row)
-- 22 bytes key
***************
*** 37,44 ****
-- decrypt
select decrypt(encrypt('foo', '0123456', '3des'), '0123456', '3des');
decrypt
! ---------
! foo
(1 row)
-- iv
--- 37,44 ----
-- decrypt
select decrypt(encrypt('foo', '0123456', '3des'), '0123456', '3des');
decrypt
! ----------------------------
! \327\007C\202\334|\232\310
(1 row)
-- iv
***************
*** 45,56 ****
select encode(encrypt_iv('foo', '0123456', 'abcd', '3des'), 'hex');
encode
------------------
! df27c264fb24ed7a
(1 row)
select decrypt_iv(decode('df27c264fb24ed7a', 'hex'), '0123456', 'abcd', '3des');
decrypt_iv
! ------------
! foo
(1 row)
--- 45,56 ----
select encode(encrypt_iv('foo', '0123456', 'abcd', '3des'), 'hex');
encode
------------------
! a1917305c1df08f8
(1 row)
select decrypt_iv(decode('df27c264fb24ed7a', 'hex'), '0123456', 'abcd', '3des');
decrypt_iv
! -------------------------
! \325c\311\365\223\312E_
(1 row)
======================================================================
Michael Fuhr <mike@fuhr.org> writes:
On my Solaris 9/sparc box with OpenSSL 0.9.8-beta6, the pgcrypto
regression tests fail the 3des test. I haven't checked against
older versions of OpenSSL; I'll do so when I get a chance.
I haven't dug into the pgcrypto code yet -- is it doing anything
that might be platform-specific? Or is this more likely a problem
with OpenSSL?
I don't see a problem with the pgcrypto regression tests in CVS HEAD,
either locally or on the buildfarm board. However probably none of
these machines are running a beta version of OpenSSL ...
regards, tom lane
On Tue, Jul 05, 2005 at 08:40:08AM -0600, Michael Fuhr wrote:
On my Solaris 9/sparc box with OpenSSL 0.9.8-beta6, the pgcrypto
regression tests fail the 3des test. I haven't checked against
older versions of OpenSSL; I'll do so when I get a chance.I haven't dug into the pgcrypto code yet -- is it doing anything
that might be platform-specific? Or is this more likely a problem
with OpenSSL?
It is a bug in pgcrypto. I can only excuse it with my strong antipathy
towards 3des.
Could you test it with newer OpenSSL?
--
marko
Attachments:
fix-3des.difftext/plain; charset=us-asciiDownload
Index: contrib/pgcrypto/openssl.c
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/openssl.c,v
retrieving revision 1.19
diff -u -c -r1.19 openssl.c
*** contrib/pgcrypto/openssl.c 4 Jul 2005 02:02:01 -0000 1.19
--- contrib/pgcrypto/openssl.c 5 Jul 2005 16:01:37 -0000
***************
*** 393,399 ****
memset(&xkey1, 0, sizeof(xkey1));
memset(&xkey2, 0, sizeof(xkey2));
! memset(&xkey2, 0, sizeof(xkey2));
memcpy(&xkey1, key, klen > 8 ? 8 : klen);
if (klen > 8)
memcpy(&xkey2, key + 8, (klen - 8) > 8 ? 8 : (klen - 8));
--- 393,399 ----
memset(&xkey1, 0, sizeof(xkey1));
memset(&xkey2, 0, sizeof(xkey2));
! memset(&xkey3, 0, sizeof(xkey3));
memcpy(&xkey1, key, klen > 8 ? 8 : klen);
if (klen > 8)
memcpy(&xkey2, key + 8, (klen - 8) > 8 ? 8 : (klen - 8));
Index: contrib/pgcrypto/sql/3des.sql
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/sql/3des.sql,v
retrieving revision 1.1
diff -u -c -r1.1 3des.sql
*** contrib/pgcrypto/sql/3des.sql 21 Mar 2005 05:24:52 -0000 1.1
--- contrib/pgcrypto/sql/3des.sql 5 Jul 2005 16:02:26 -0000
***************
*** 22,26 ****
-- iv
select encode(encrypt_iv('foo', '0123456', 'abcd', '3des'), 'hex');
! select decrypt_iv(decode('df27c264fb24ed7a', 'hex'), '0123456', 'abcd', '3des');
--- 22,26 ----
-- iv
select encode(encrypt_iv('foo', '0123456', 'abcd', '3des'), 'hex');
! select decrypt_iv(decode('50735067b073bb93', 'hex'), '0123456', 'abcd', '3des');
Index: contrib/pgcrypto/expected/3des.out
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/expected/3des.out,v
retrieving revision 1.1
diff -u -c -r1.1 3des.out
*** contrib/pgcrypto/expected/3des.out 21 Mar 2005 05:24:51 -0000 1.1
--- contrib/pgcrypto/expected/3des.out 5 Jul 2005 16:03:23 -0000
***************
*** 17,30 ****
select encode( encrypt('', 'foo', '3des'), 'hex');
encode
------------------
! 9b641a6936249eb4
(1 row)
-- 10 bytes key
select encode( encrypt('foo', '0123456789', '3des'), 'hex');
encode
------------------
! 6f02b7076a366504
(1 row)
-- 22 bytes key
--- 17,30 ----
select encode( encrypt('', 'foo', '3des'), 'hex');
encode
------------------
! 752111e37a2d7ac3
(1 row)
-- 10 bytes key
select encode( encrypt('foo', '0123456789', '3des'), 'hex');
encode
------------------
! d2fb8baa1717cb02
(1 row)
-- 22 bytes key
***************
*** 45,54 ****
select encode(encrypt_iv('foo', '0123456', 'abcd', '3des'), 'hex');
encode
------------------
! df27c264fb24ed7a
(1 row)
! select decrypt_iv(decode('df27c264fb24ed7a', 'hex'), '0123456', 'abcd', '3des');
decrypt_iv
------------
foo
--- 45,54 ----
select encode(encrypt_iv('foo', '0123456', 'abcd', '3des'), 'hex');
encode
------------------
! 50735067b073bb93
(1 row)
! select decrypt_iv(decode('50735067b073bb93', 'hex'), '0123456', 'abcd', '3des');
decrypt_iv
------------
foo
On Tue, Jul 05, 2005 at 07:21:17PM +0300, Marko Kreen wrote:
It is a bug in pgcrypto. I can only excuse it with my strong antipathy
towards 3des.Could you test it with newer OpenSSL?
Looks good. After applying the patch, all pgcrypto regression tests
pass on my box running Solaris 9/sparc, OpenSSL 0.9.8-beta6, and HEAD.
I expect you'll need to submit the patch to pgsql-patches so it'll
get put in the queue.
Thanks.
--
Michael Fuhr
http://www.fuhr.org/~mfuhr/
Michael Fuhr <mike@fuhr.org> writes:
On Tue, Jul 05, 2005 at 07:21:17PM +0300, Marko Kreen wrote:
It is a bug in pgcrypto. I can only excuse it with my strong antipathy
towards 3des.
Looks good. After applying the patch, all pgcrypto regression tests
pass on my box running Solaris 9/sparc, OpenSSL 0.9.8-beta6, and HEAD.
I expect you'll need to submit the patch to pgsql-patches so it'll
get put in the queue.
No need, we'll apply it.
regards, tom lane
Marko Kreen <marko@l-t.ee> writes:
On Tue, Jul 05, 2005 at 08:40:08AM -0600, Michael Fuhr wrote:
On my Solaris 9/sparc box with OpenSSL 0.9.8-beta6, the pgcrypto
regression tests fail the 3des test. I haven't checked against
older versions of OpenSSL; I'll do so when I get a chance.
It is a bug in pgcrypto. I can only excuse it with my strong antipathy
towards 3des.
Patch applied.
Now that I look, the reason the buildfarm failed to find this is of
course that the pgcrypto Makefile is configured to never build or test
this code. Would it be reasonable to fix the makefile to follow the
toplevel --with-openssl choice?
regards, tom lane
Tom Lane wrote:
Now that I look, the reason the buildfarm failed to find this is of
course that the pgcrypto Makefile is configured to never build or test
this code. Would it be reasonable to fix the makefile to follow the
toplevel --with-openssl choice?
Yes, please! good catch!
andrew
On Tue, Jul 05, 2005 at 02:18:00PM -0400, Tom Lane wrote:
Marko Kreen <marko@l-t.ee> writes:
On Tue, Jul 05, 2005 at 08:40:08AM -0600, Michael Fuhr wrote:
On my Solaris 9/sparc box with OpenSSL 0.9.8-beta6, the pgcrypto
regression tests fail the 3des test. I haven't checked against
older versions of OpenSSL; I'll do so when I get a chance.It is a bug in pgcrypto. I can only excuse it with my strong antipathy
towards 3des.Patch applied.
Now that I look, the reason the buildfarm failed to find this is of
course that the pgcrypto Makefile is configured to never build or test
this code. Would it be reasonable to fix the makefile to follow the
toplevel --with-openssl choice?
Heh. As it happens, I just researched the same thing. This
will be especially imortant with the new PGP code, which simply
does not work without OpenSSL.
I see 2 variants:
1) put @with_openssl@ and @with_zlib@ variables into
contrib/pgcrypto/Makefile.in and let configure process it.
2) put them in some other makefile fragment under src/
and let pgcrypto include it.
First I did the simple thing and put them into Makefile.global.in,
but this does not work, as it needs to be included _after_ all
module variables are set. So 2) needs a new file.
--
marko
Marko Kreen <marko@l-t.ee> writes:
I see 2 variants:
1) put @with_openssl@ and @with_zlib@ variables into
contrib/pgcrypto/Makefile.in and let configure process it.
2) put them in some other makefile fragment under src/
and let pgcrypto include it.
First I did the simple thing and put them into Makefile.global.in,
but this does not work, as it needs to be included _after_ all
module variables are set. So 2) needs a new file.
Hm ... libpq manages to build code that requires openssl without
needing a generated Makefile, so why do we need it here?
I'd prefer #1 of these two, but seeing that no other contrib module
has a Makefile.in, not having to do either would be even better.
This is particularly true if you aren't going to completely eliminate
the hand-configuration options in the Makefile, because having to edit
a generated Makefile or having to edit Makefile.in and then remember
to reconfigure both suck.
regards, tom lane
On Tue, Jul 05, 2005 at 02:55:07PM -0400, Tom Lane wrote:
Marko Kreen <marko@l-t.ee> writes:
I see 2 variants:
1) put @with_openssl@ and @with_zlib@ variables into
contrib/pgcrypto/Makefile.in and let configure process it.
2) put them in some other makefile fragment under src/
and let pgcrypto include it.First I did the simple thing and put them into Makefile.global.in,
but this does not work, as it needs to be included _after_ all
module variables are set. So 2) needs a new file.Hm ... libpq manages to build code that requires openssl without
needing a generated Makefile, so why do we need it here?
Now, looking more into it, it indeed does work.
But it breaks pgcrypto build for USE_PGXS case.
I'd prefer #1 of these two, but seeing that no other contrib module
has a Makefile.in, not having to do either would be even better.
Can I break USE_PGXS? Otherwise I need pre-processed
Makefile.in.
This is particularly true if you aren't going to completely eliminate
the hand-configuration options in the Makefile, because having to edit
a generated Makefile or having to edit Makefile.in and then remember
to reconfigure both suck.
I'd like to eliminate hand-config. Current code does not need
hand-config - you will miss only couple of algorithms without
OpenSSL. And there is no code that needs strong randomness.
With new PGP code, if you dont have OpenSSL you won't have
public-key code anyway - you can't configure bignum support.
That leaves randomness for pgp_sym_encrypt - which can be
solved with including a strong PRNG with pgcrypto. I think
this can be done in 8.1 timeframe, as it will be pretty
small piece of code.
There is also zlib - no need to configure it if I can get
it from main config.
--
marko
Marko Kreen <marko@l-t.ee> writes:
On Tue, Jul 05, 2005 at 02:55:07PM -0400, Tom Lane wrote:
Hm ... libpq manages to build code that requires openssl without
needing a generated Makefile, so why do we need it here?
Now, looking more into it, it indeed does work.
But it breaks pgcrypto build for USE_PGXS case.
How exactly?
Can I break USE_PGXS?
Nope, that's right out. If we have to use a Makefile.in we will,
but I'd like to understand why we have to.
I'd like to eliminate hand-config.
That would be very good. In practice no one is likely to do any manual
configuration down inside a contrib module ... certainly none of the RPM
distributions do any such thing.
regards, tom lane
On Tue, Jul 05, 2005 at 03:36:06PM -0400, Tom Lane wrote:
Marko Kreen <marko@l-t.ee> writes:
On Tue, Jul 05, 2005 at 02:55:07PM -0400, Tom Lane wrote:
Hm ... libpq manages to build code that requires openssl without
needing a generated Makefile, so why do we need it here?Now, looking more into it, it indeed does work.
But it breaks pgcrypto build for USE_PGXS case.How exactly?
ifdef USE_PGXS
PGXS = $(shell pg_config --pgxs)
include $(PGXS)
else
.....
The "include $(PGXS)" includes parts that need initialized
local variables, but the variable initializing is the part
that needs settings from main makefiles...
And including Makefile.global twice gives errors.
Can I break USE_PGXS?
Nope, that's right out. If we have to use a Makefile.in we will,
but I'd like to understand why we have to.I'd like to eliminate hand-config.
That would be very good. In practice no one is likely to do any manual
configuration down inside a contrib module ... certainly none of the RPM
distributions do any such thing.
It was unimportant thus far, but now it's life-or-death ;)
--
marko
Marko Kreen <marko@l-t.ee> writes:
Oh, ofcourse I would not need to break it, if the interesting
settings can be put into eg. src/Makefile.config (.in), and
I include that one.
You mean Makefile.global.in, no? That seems fine to me.
But that case would break if top_srcdir is not ../..
No it wouldn't.
regards, tom lane
Import Notes
Reply to msg id not found: 20050705195520.GA30992@l-t.ee
On Tue, Jul 05, 2005 at 03:58:43PM -0400, Tom Lane wrote:
Marko Kreen <marko@l-t.ee> writes:
Oh, ofcourse I would not need to break it, if the interesting
settings can be put into eg. src/Makefile.config (.in), and
I include that one.You mean Makefile.global.in, no? That seems fine to me.
No, thats the point - the PGXS include also includes
Makefile.global, and including it twice does not work.
But that case would break if top_srcdir is not ../..
No it wouldn't.
How can I find the top dir?
Hm, actually, I can do $(dir $(PGXS))../Makefile.config
So, yes it would work.
So here's the variants:
1) contrib/pgcrypto/Makefile.in
2) src/Makefile.config.in
I think I like 2) more, thus pgcrypto would not be
special among contrib modules. It could be useful for
other modules aswell.
Also 1) would break postgresql-base build, no? It
could be probably avoided with some shell scripting
but it would be still inelegant.
Comments?
--
marko
Marko Kreen <marko@l-t.ee> writes:
On Tue, Jul 05, 2005 at 03:58:43PM -0400, Tom Lane wrote:
You mean Makefile.global.in, no? That seems fine to me.
No, thats the point - the PGXS include also includes
Makefile.global, and including it twice does not work.
So? pgxs.mk includes Makefile.global before doing anything interesting.
Seems to me you could do what you need to given make's rules about
delayed evaluation of variables.
regards, tom lane
On Tue, Jul 05, 2005 at 04:16:04PM -0400, Tom Lane wrote:
Marko Kreen <marko@l-t.ee> writes:
On Tue, Jul 05, 2005 at 03:58:43PM -0400, Tom Lane wrote:
You mean Makefile.global.in, no? That seems fine to me.
No, thats the point - the PGXS include also includes
Makefile.global, and including it twice does not work.So? pgxs.mk includes Makefile.global before doing anything interesting.
Seems to me you could do what you need to given make's rules about
delayed evaluation of variables.
How? Current config is done using make ifdef/ifeq which are
executed immidately. I don't know make that well, maybe
there is a way to put if's inside variables, but that would
get rather messy, no?
--
marko
Marko Kreen <marko@l-t.ee> writes:
Ok Tom, you win. It is indeed possible to make it work, and the
resulting makefile is even cleaner than before.
Following patch thus autoconfigures pgcrypto. It drops the
possibility to use libc's crypt, which was pointless.
Applied with a little extra hacking (completely overriding the original
values of PG_CPPFLAGS and SHLIB_LINK was unwise, and in fact guaranteed
to fail on machines where OpenSSL isn't in the default location).
regards, tom lane
Import Notes
Reply to msg id not found: 20050705211616.GA32629@l-t.ee
On Tue, Jul 05, 2005 at 07:20:48PM -0400, Tom Lane wrote:
Marko Kreen <marko@l-t.ee> writes:
Ok Tom, you win. It is indeed possible to make it work, and the
resulting makefile is even cleaner than before.
Following patch thus autoconfigures pgcrypto. It drops the
possibility to use libc's crypt, which was pointless.Applied with a little extra hacking (completely overriding the original
values of PG_CPPFLAGS and SHLIB_LINK was unwise, and in fact guaranteed
to fail on machines where OpenSSL isn't in the default location).
Your PG_CPPFLAGS change does not work - it gets always RAND_SILLY.
If I change it from := to =, make complains about recursion.
--
marko
Attachments:
fix-cflags.difftext/plain; charset=us-asciiDownload
Index: contrib/pgcrypto/Makefile
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/Makefile,v
retrieving revision 1.16
diff -u -c -r1.16 Makefile
*** contrib/pgcrypto/Makefile 5 Jul 2005 23:18:44 -0000 1.16
--- contrib/pgcrypto/Makefile 6 Jul 2005 10:37:26 -0000
***************
*** 16,22 ****
CF_TESTS = $(if $(subst no,,$(with_openssl)), $(OSSL_TESTS))
CF_CFLAGS = $(if $(subst no,,$(with_openssl)), $(OSSL_CFLAGS), $(INT_CFLAGS))
! PG_CPPFLAGS := $(CF_CFLAGS) -I$(srcdir) $(PG_CPPFLAGS)
SRCS = pgcrypto.c px.c px-hmac.c px-crypt.c misc.c random.c \
crypt-gensalt.c crypt-blowfish.c crypt-des.c \
--- 16,22 ----
CF_TESTS = $(if $(subst no,,$(with_openssl)), $(OSSL_TESTS))
CF_CFLAGS = $(if $(subst no,,$(with_openssl)), $(OSSL_CFLAGS), $(INT_CFLAGS))
! PG_CPPFLAGS = $(CF_CFLAGS) -I$(srcdir)
SRCS = pgcrypto.c px.c px-hmac.c px-crypt.c misc.c random.c \
crypt-gensalt.c crypt-blowfish.c crypt-des.c \
Marko Kreen <marko@l-t.ee> writes:
! PG_CPPFLAGS := $(CF_CFLAGS) -I$(srcdir) $(PG_CPPFLAGS)
...
! PG_CPPFLAGS = $(CF_CFLAGS) -I$(srcdir)
Ah, right. Actually we don't need -I$(srcdir) either, since pgxs.mk
adds that automatically.
regards, tom lane