4 pgcrypto regressions failures - 1 unsolved

Started by Marko Kreenover 20 years ago23 messages
#1Marko Kreen
marko@l-t.ee
2 attachment(s)

http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=potorooo&dt=2005-07-10%2022:30:03

New sha2 code on Solaris 2.8 / SPARC. Seems like it has
problems memcpy'ing to a non-8-byte-aligned uint64 *.

Attached patch fixes it by simplifying the _Final code and
getting rid of the pointer.

(I redefined bzero and bcopy but now I think they should be
replaced directly - patch later.)

http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=goose&dt=2005-07-11%2006:00:04
http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=ferret&dt=2005-07-10%2018:25:11

The new sha2.c checks not only whether BYTE_ENDIAN is
LITTLE_ENDIAN or BIG_ENDIAN but also whether it is set.
And the test fails on both Cygwin and MINGW.

As gcc evaluates "#if UNDEF1 == UNDEF2" as true and there
were no compile failures reported with current code, that
means currently internal AES, SHA1 and MD5 used randomly
big-endian, little-endian or both variants of code.

If there was no regression failures on those platforms,
it must be only by dumb luck.

Attached patch includes sys/param.h, where I found them on
MINGW, and puts stricter checks into all files.

After I see successful run in pgbuildfarm, I send it for
stable branches too.

http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=canary&dt=2005-07-11%2002:30:00

NetBSD 1.6 with older OpenSSL. OpenSSL < 0.9.7 does not have
AES, but most of PGP tests use it as it is the preferred cipher.
And the AES tests fails anyway. I guess it can stay as expected
failure.

http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=dragonfly&amp;dt=2005-07-11%2003:30:04

Linking problem with zlib on Solaris 9/x86. I am clueless about
this. I can anyone look into it?

Error message:
ld: fatal: relocations remain against allocatable but non-writable sections

--
marko

Attachments:

sha2-sparc-fix.difftext/plain; charset=us-asciiDownload
Index: pgsql/contrib/pgcrypto/sha2.c
===================================================================
*** pgsql.orig/contrib/pgcrypto/sha2.c
--- pgsql/contrib/pgcrypto/sha2.c
*************** SHA256_Update(SHA256_CTX *context, const
*** 496,502 ****
  void
  SHA256_Final(uint8 digest[], SHA256_CTX *context)
  {
- 	uint32	*d = (uint32 *)digest;
  	unsigned int	usedspace;
  
  	/* If no digest buffer is passed, we don't bother doing this: */
--- 496,501 ----
*************** SHA256_Final(uint8 digest[], SHA256_CTX 
*** 542,553 ****
  			int	j;
  			for (j = 0; j < 8; j++) {
  				REVERSE32(context->state[j],context->state[j]);
- 				*d++ = context->state[j];
  			}
  		}
- #else
- 		bcopy(context->state, d, SHA256_DIGEST_LENGTH);
  #endif
  	}
  
  	/* Clean up state data: */
--- 541,550 ----
  			int	j;
  			for (j = 0; j < 8; j++) {
  				REVERSE32(context->state[j],context->state[j]);
  			}
  		}
  #endif
+ 		bcopy(context->state, digest, SHA256_DIGEST_LENGTH);
  	}
  
  	/* Clean up state data: */
*************** SHA512_Last(SHA512_CTX *context)
*** 823,830 ****
  void
  SHA512_Final(uint8 digest[], SHA512_CTX *context)
  {
- 	uint64	*d = (uint64 *)digest;
- 
  	/* If no digest buffer is passed, we don't bother doing this: */
  	if (digest != NULL) {
  		SHA512_Last(context);
--- 820,825 ----
*************** SHA512_Final(uint8 digest[], SHA512_CTX 
*** 836,847 ****
  			int	j;
  			for (j = 0; j < 8; j++) {
  				REVERSE64(context->state[j],context->state[j]);
- 				*d++ = context->state[j];
  			}
  		}
- #else
- 		bcopy(context->state, d, SHA512_DIGEST_LENGTH);
  #endif
  	}
  
  	/* Zero out state data */
--- 831,840 ----
  			int	j;
  			for (j = 0; j < 8; j++) {
  				REVERSE64(context->state[j],context->state[j]);
  			}
  		}
  #endif
+ 		bcopy(context->state, digest, SHA512_DIGEST_LENGTH);
  	}
  
  	/* Zero out state data */
*************** SHA384_Update(SHA384_CTX *context, const
*** 869,876 ****
  void
  SHA384_Final(uint8 digest[], SHA384_CTX *context)
  {
- 	uint64	*d = (uint64 *)digest;
- 
  	/* If no digest buffer is passed, we don't bother doing this: */
  	if (digest != NULL) {
  		SHA512_Last((SHA512_CTX *)context);
--- 862,867 ----
*************** SHA384_Final(uint8 digest[], SHA384_CTX 
*** 882,893 ****
  			int	j;
  			for (j = 0; j < 6; j++) {
  				REVERSE64(context->state[j],context->state[j]);
- 				*d++ = context->state[j];
  			}
  		}
- #else
- 		bcopy(context->state, d, SHA384_DIGEST_LENGTH);
  #endif
  	}
  
  	/* Zero out state data */
--- 873,882 ----
  			int	j;
  			for (j = 0; j < 6; j++) {
  				REVERSE64(context->state[j],context->state[j]);
  			}
  		}
  #endif
+ 		bcopy(context->state, digest, SHA384_DIGEST_LENGTH);
  	}
  
  	/* Zero out state data */
byteorder-fix.difftext/plain; charset=us-asciiDownload
Index: pgsql/contrib/pgcrypto/md5.c
===================================================================
*** pgsql.orig/contrib/pgcrypto/md5.c
--- pgsql/contrib/pgcrypto/md5.c
***************
*** 30,40 ****
   * SUCH DAMAGE.
   */
  
! #include "postgres.h"
  #include "px.h"
  
  #include "md5.h"
  
  #define SHIFT(X, s) (((X) << (s)) | ((X) >> (32 - (s))))
  
  #define F(X, Y, Z) (((X) & (Y)) | ((~X) & (Z)))
--- 30,47 ----
   * SUCH DAMAGE.
   */
  
! #include <postgres.h>
! #include <sys/param.h>
! 
  #include "px.h"
  
  #include "md5.h"
  
+ /* sanity check */
+ #if !defined(BYTE_ORDER) || (BYTE_ORDER != LITTLE_ENDIAN && BYTE_ORDER != BIG_ENDIAN)
+ #error Define BYTE_ORDER to be equal to either LITTLE_ENDIAN or BIG_ENDIAN
+ #endif
+ 
  #define SHIFT(X, s) (((X) << (s)) | ((X) >> (32 - (s))))
  
  #define F(X, Y, Z) (((X) & (Y)) | ((~X) & (Z)))
Index: pgsql/contrib/pgcrypto/rijndael.c
===================================================================
*** pgsql.orig/contrib/pgcrypto/rijndael.c
--- pgsql/contrib/pgcrypto/rijndael.c
*************** Mean:		   500 cycles =    51.2 mbits/sec
*** 39,48 ****
--- 39,56 ----
  */
  
  #include <postgres.h>
+ #include <sys/param.h>
+ 
  #include "px.h"
  
  #include "rijndael.h"
  
+ /* sanity check */
+ #if !defined(BYTE_ORDER) || (BYTE_ORDER != LITTLE_ENDIAN && BYTE_ORDER != BIG_ENDIAN)
+ #error Define BYTE_ORDER to be equal to either LITTLE_ENDIAN or BIG_ENDIAN
+ #endif
+ 
+ 
  #define PRE_CALC_TABLES
  #define LARGE_TABLES
  
Index: pgsql/contrib/pgcrypto/sha1.c
===================================================================
*** pgsql.orig/contrib/pgcrypto/sha1.c
--- pgsql/contrib/pgcrypto/sha1.c
***************
*** 35,53 ****
   * implemented by Jun-ichiro itojun Itoh <itojun@itojun.org>
   */
  
! #include "postgres.h"
! #include "px.h"
  
  #include "sha1.h"
  
  /* sanity check */
! #if BYTE_ORDER != BIG_ENDIAN
! #if BYTE_ORDER != LITTLE_ENDIAN
! #define unsupported 1
  #endif
- #endif
- 
- #ifndef unsupported
  
  /* constant table */
  static uint32 _K[] = {0x5a827999, 0x6ed9eba1, 0x8f1bbcdc, 0xca62c1d6};
--- 35,50 ----
   * implemented by Jun-ichiro itojun Itoh <itojun@itojun.org>
   */
  
! #include <postgres.h>
! #include <sys/param.h>
  
+ #include "px.h"
  #include "sha1.h"
  
  /* sanity check */
! #if !defined(BYTE_ORDER) || (BYTE_ORDER != LITTLE_ENDIAN && BYTE_ORDER != BIG_ENDIAN)
! #error Define BYTE_ORDER to be equal to either LITTLE_ENDIAN or BIG_ENDIAN
  #endif
  
  /* constant table */
  static uint32 _K[] = {0x5a827999, 0x6ed9eba1, 0x8f1bbcdc, 0xca62c1d6};
*************** sha1_result(struct sha1_ctxt * ctxt, uin
*** 347,350 ****
  #endif
  }
  
- #endif   /* unsupported */
--- 344,346 ----
Index: pgsql/contrib/pgcrypto/sha2.c
===================================================================
*** pgsql.orig/contrib/pgcrypto/sha2.c
--- pgsql/contrib/pgcrypto/sha2.c
***************
*** 36,41 ****
--- 36,42 ----
   */
  
  #include <postgres.h>
+ #include <sys/param.h>
  
  #include "sha2.h"
  
#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Kreen (#1)
Re: 4 pgcrypto regressions failures - 1 unsolved

Marko Kreen <marko@l-t.ee> writes:

(I redefined bzero and bcopy but now I think they should be
replaced directly - patch later.)

Please. We do not use those old functions in the Postgres code;
memcpy, memmove, memset, etc are the project standard.

regards, tom lane

#3Marko Kreen
marko@l-t.ee
In reply to: Tom Lane (#2)
Re: 4 pgcrypto regressions failures - 1 unsolved

On Mon, Jul 11, 2005 at 10:13:22AM -0400, Tom Lane wrote:

Marko Kreen <marko@l-t.ee> writes:

(I redefined bzero and bcopy but now I think they should be
replaced directly - patch later.)

Please. We do not use those old functions in the Postgres code;
memcpy, memmove, memset, etc are the project standard.

Indeed. But I'll wait until the previous sha2 patch is applied
as they would conflict.

--
marko

#4Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Kreen (#1)
Re: 4 pgcrypto regressions failures - 1 unsolved

Marko Kreen <marko@l-t.ee> writes:

New sha2 code on Solaris 2.8 / SPARC. Seems like it has
problems memcpy'ing to a non-8-byte-aligned uint64 *.
...
Attached patch includes sys/param.h, where I found them on
MINGW, and puts stricter checks into all files.

Applied.

regards, tom lane

#5Marko Kreen
marko@l-t.ee
In reply to: Tom Lane (#4)
1 attachment(s)
Re: 4 pgcrypto regressions failures - 1 unsolved

On Mon, Jul 11, 2005 at 11:09:06AM -0400, Tom Lane wrote:

Marko Kreen <marko@l-t.ee> writes:

New sha2 code on Solaris 2.8 / SPARC. Seems like it has
problems memcpy'ing to a non-8-byte-aligned uint64 *.
...
Attached patch includes sys/param.h, where I found them on
MINGW, and puts stricter checks into all files.

Applied.

I see you also cleaned the includes. Thanks.

Here is the bcopy, bzero removal patch.

--
marko

Attachments:

use-memcpy.difftext/plain; charset=us-asciiDownload
Index: contrib/pgcrypto/sha2.c
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/sha2.c,v
retrieving revision 1.2
diff -u -c -r1.2 sha2.c
*** contrib/pgcrypto/sha2.c	11 Jul 2005 15:07:59 -0000	1.2
--- contrib/pgcrypto/sha2.c	11 Jul 2005 15:20:33 -0000
***************
*** 42,52 ****
  
  #include "sha2.h"
  
- #undef bcopy
- #undef bzero
- #define bcopy(src, dst, len)	memcpy((dst), (src), (len))
- #define bzero(ptr, len)			memset((ptr), 0, (len))
- 
  /*
   * UNROLLED TRANSFORM LOOP NOTE:
   * You can define SHA2_UNROLL_TRANSFORM to use the unrolled transform
--- 42,47 ----
***************
*** 281,288 ****
  {
  	if (context == NULL)
  		return;
! 	bcopy(sha256_initial_hash_value, context->state, SHA256_DIGEST_LENGTH);
! 	bzero(context->buffer, SHA256_BLOCK_LENGTH);
  	context->bitcount = 0;
  }
  
--- 276,283 ----
  {
  	if (context == NULL)
  		return;
! 	memcpy(context->state, sha256_initial_hash_value, SHA256_DIGEST_LENGTH);
! 	memset(context->buffer, 0, SHA256_BLOCK_LENGTH);
  	context->bitcount = 0;
  }
  
***************
*** 466,479 ****
  
  		if (len >= freespace) {
  			/* Fill the buffer completely and process it */
! 			bcopy(data, &context->buffer[usedspace], freespace);
  			context->bitcount += freespace << 3;
  			len -= freespace;
  			data += freespace;
  			SHA256_Transform(context, context->buffer);
  		} else {
  			/* The buffer is not yet full */
! 			bcopy(data, &context->buffer[usedspace], len);
  			context->bitcount += len << 3;
  			/* Clean up: */
  			usedspace = freespace = 0;
--- 461,474 ----
  
  		if (len >= freespace) {
  			/* Fill the buffer completely and process it */
! 			memcpy(&context->buffer[usedspace], data, freespace);
  			context->bitcount += freespace << 3;
  			len -= freespace;
  			data += freespace;
  			SHA256_Transform(context, context->buffer);
  		} else {
  			/* The buffer is not yet full */
! 			memcpy(&context->buffer[usedspace], data, len);
  			context->bitcount += len << 3;
  			/* Clean up: */
  			usedspace = freespace = 0;
***************
*** 489,495 ****
  	}
  	if (len > 0) {
  		/* There's left-overs, so save 'em */
! 		bcopy(data, context->buffer, len);
  		context->bitcount += len << 3;
  	}
  	/* Clean up: */
--- 484,490 ----
  	}
  	if (len > 0) {
  		/* There's left-overs, so save 'em */
! 		memcpy(context->buffer, data, len);
  		context->bitcount += len << 3;
  	}
  	/* Clean up: */
***************
*** 514,533 ****
  
  			if (usedspace <= SHA256_SHORT_BLOCK_LENGTH) {
  				/* Set-up for the last transform: */
! 				bzero(&context->buffer[usedspace], SHA256_SHORT_BLOCK_LENGTH - usedspace);
  			} else {
  				if (usedspace < SHA256_BLOCK_LENGTH) {
! 					bzero(&context->buffer[usedspace], SHA256_BLOCK_LENGTH - usedspace);
  				}
  				/* Do second-to-last transform: */
  				SHA256_Transform(context, context->buffer);
  
  				/* And set-up for the last transform: */
! 				bzero(context->buffer, SHA256_SHORT_BLOCK_LENGTH);
  			}
  		} else {
  			/* Set-up for the last transform: */
! 			bzero(context->buffer, SHA256_SHORT_BLOCK_LENGTH);
  
  			/* Begin padding with a 1 bit: */
  			*context->buffer = 0x80;
--- 509,528 ----
  
  			if (usedspace <= SHA256_SHORT_BLOCK_LENGTH) {
  				/* Set-up for the last transform: */
! 				memset(&context->buffer[usedspace], 0, SHA256_SHORT_BLOCK_LENGTH - usedspace);
  			} else {
  				if (usedspace < SHA256_BLOCK_LENGTH) {
! 					memset(&context->buffer[usedspace], 0, SHA256_BLOCK_LENGTH - usedspace);
  				}
  				/* Do second-to-last transform: */
  				SHA256_Transform(context, context->buffer);
  
  				/* And set-up for the last transform: */
! 				memset(context->buffer, 0, SHA256_SHORT_BLOCK_LENGTH);
  			}
  		} else {
  			/* Set-up for the last transform: */
! 			memset(context->buffer, 0, SHA256_SHORT_BLOCK_LENGTH);
  
  			/* Begin padding with a 1 bit: */
  			*context->buffer = 0x80;
***************
*** 547,557 ****
  			}
  		}
  #endif
! 		bcopy(context->state, digest, SHA256_DIGEST_LENGTH);
  	}
  
  	/* Clean up state data: */
! 	bzero(context, sizeof(*context));
  	usedspace = 0;
  }
  
--- 542,552 ----
  			}
  		}
  #endif
! 		memcpy(digest, context->state, SHA256_DIGEST_LENGTH);
  	}
  
  	/* Clean up state data: */
! 	memset(context, 0, sizeof(*context));
  	usedspace = 0;
  }
  
***************
*** 562,569 ****
  {
  	if (context == NULL)
  		return;
! 	bcopy(sha512_initial_hash_value, context->state, SHA512_DIGEST_LENGTH);
! 	bzero(context->buffer, SHA512_BLOCK_LENGTH);
  	context->bitcount[0] = context->bitcount[1] =  0;
  }
  
--- 557,564 ----
  {
  	if (context == NULL)
  		return;
! 	memcpy(context->state, sha512_initial_hash_value, SHA512_DIGEST_LENGTH);
! 	memset(context->buffer, 0, SHA512_BLOCK_LENGTH);
  	context->bitcount[0] = context->bitcount[1] =  0;
  }
  
***************
*** 747,760 ****
  
  		if (len >= freespace) {
  			/* Fill the buffer completely and process it */
! 			bcopy(data, &context->buffer[usedspace], freespace);
  			ADDINC128(context->bitcount, freespace << 3);
  			len -= freespace;
  			data += freespace;
  			SHA512_Transform(context, context->buffer);
  		} else {
  			/* The buffer is not yet full */
! 			bcopy(data, &context->buffer[usedspace], len);
  			ADDINC128(context->bitcount, len << 3);
  			/* Clean up: */
  			usedspace = freespace = 0;
--- 742,755 ----
  
  		if (len >= freespace) {
  			/* Fill the buffer completely and process it */
! 			memcpy(&context->buffer[usedspace], data, freespace);
  			ADDINC128(context->bitcount, freespace << 3);
  			len -= freespace;
  			data += freespace;
  			SHA512_Transform(context, context->buffer);
  		} else {
  			/* The buffer is not yet full */
! 			memcpy(&context->buffer[usedspace], data, len);
  			ADDINC128(context->bitcount, len << 3);
  			/* Clean up: */
  			usedspace = freespace = 0;
***************
*** 770,776 ****
  	}
  	if (len > 0) {
  		/* There's left-overs, so save 'em */
! 		bcopy(data, context->buffer, len);
  		ADDINC128(context->bitcount, len << 3);
  	}
  	/* Clean up: */
--- 765,771 ----
  	}
  	if (len > 0) {
  		/* There's left-overs, so save 'em */
! 		memcpy(context->buffer, data, len);
  		ADDINC128(context->bitcount, len << 3);
  	}
  	/* Clean up: */
***************
*** 794,813 ****
  
  		if (usedspace <= SHA512_SHORT_BLOCK_LENGTH) {
  			/* Set-up for the last transform: */
! 			bzero(&context->buffer[usedspace], SHA512_SHORT_BLOCK_LENGTH - usedspace);
  		} else {
  			if (usedspace < SHA512_BLOCK_LENGTH) {
! 				bzero(&context->buffer[usedspace], SHA512_BLOCK_LENGTH - usedspace);
  			}
  			/* Do second-to-last transform: */
  			SHA512_Transform(context, context->buffer);
  
  			/* And set-up for the last transform: */
! 			bzero(context->buffer, SHA512_BLOCK_LENGTH - 2);
  		}
  	} else {
  		/* Prepare for final transform: */
! 		bzero(context->buffer, SHA512_SHORT_BLOCK_LENGTH);
  
  		/* Begin padding with a 1 bit: */
  		*context->buffer = 0x80;
--- 789,808 ----
  
  		if (usedspace <= SHA512_SHORT_BLOCK_LENGTH) {
  			/* Set-up for the last transform: */
! 			memset(&context->buffer[usedspace], 0, SHA512_SHORT_BLOCK_LENGTH - usedspace);
  		} else {
  			if (usedspace < SHA512_BLOCK_LENGTH) {
! 				memset(&context->buffer[usedspace], 0, SHA512_BLOCK_LENGTH - usedspace);
  			}
  			/* Do second-to-last transform: */
  			SHA512_Transform(context, context->buffer);
  
  			/* And set-up for the last transform: */
! 			memset(context->buffer, 0, SHA512_BLOCK_LENGTH - 2);
  		}
  	} else {
  		/* Prepare for final transform: */
! 		memset(context->buffer, 0, SHA512_SHORT_BLOCK_LENGTH);
  
  		/* Begin padding with a 1 bit: */
  		*context->buffer = 0x80;
***************
*** 837,847 ****
  			}
  		}
  #endif
! 		bcopy(context->state, digest, SHA512_DIGEST_LENGTH);
  	}
  
  	/* Zero out state data */
! 	bzero(context, sizeof(*context));
  }
  
  
--- 832,842 ----
  			}
  		}
  #endif
! 		memcpy(digest, context->state, SHA512_DIGEST_LENGTH);
  	}
  
  	/* Zero out state data */
! 	memset(context, 0, sizeof(*context));
  }
  
  
***************
*** 851,858 ****
  {
  	if (context == NULL)
  		return;
! 	bcopy(sha384_initial_hash_value, context->state, SHA512_DIGEST_LENGTH);
! 	bzero(context->buffer, SHA384_BLOCK_LENGTH);
  	context->bitcount[0] = context->bitcount[1] = 0;
  }
  
--- 846,853 ----
  {
  	if (context == NULL)
  		return;
! 	memcpy(context->state, sha384_initial_hash_value, SHA512_DIGEST_LENGTH);
! 	memset(context->buffer, 0, SHA384_BLOCK_LENGTH);
  	context->bitcount[0] = context->bitcount[1] = 0;
  }
  
***************
*** 879,887 ****
  			}
  		}
  #endif
! 		bcopy(context->state, digest, SHA384_DIGEST_LENGTH);
  	}
  
  	/* Zero out state data */
! 	bzero(context, sizeof(*context));
  }
--- 874,882 ----
  			}
  		}
  #endif
! 		memcpy(digest, context->state, SHA384_DIGEST_LENGTH);
  	}
  
  	/* Zero out state data */
! 	memset(context, 0, sizeof(*context));
  }
#6Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Kreen (#5)
Re: 4 pgcrypto regressions failures - 1 unsolved

Marko Kreen <marko@l-t.ee> writes:

Here is the bcopy, bzero removal patch.

Applied.

I'm seeing the following build failure on HPUX:

/usr/ccs/bin/ld +h libpgcrypto.sl.0 -b +b /home/postgres/testversion/lib pgcrypto.o px.o px-hmac.o px-crypt.o misc.o crypt-gensalt.o crypt-blowfish.o crypt-des.o crypt-md5.o md5.o sha1.o sha2.o internal.o blf.o rijndael.o fortuna.o random.o pgp-mpi-internal.o mbuf.o pgp.o pgp-armor.o pgp-cfb.o pgp-compress.o pgp-decrypt.o pgp-encrypt.o pgp-info.o pgp-mpi.o pgp-pubdec.o pgp-pubenc.o pgp-pubkey.o pgp-s2k.o pgp-pgsql.o -L../../src/port `gcc -L../../src/port -Wl,-z -Wl,+b -Wl,/home/postgres/testversion/lib -print-libgcc-file-name` -lz -o libpgcrypto.sl.0
/usr/ccs/bin/ld: Can't find library for -lz
make: *** [libpgcrypto.sl.0] Error 1

I believe the issue is that libz.sl is in /usr/local/lib/, which is not
searched by default by HP's linker. It *is* searched by default by gcc,
which is why -lz works without any explicit -L in the pg_dump/pg_restore
builds. But here we are invoking a different tool with a different
default search path.

Possibly there's something similar happening on that Solaris buildfarm
machine?

regards, tom lane

#7Marko Kreen
marko@l-t.ee
In reply to: Tom Lane (#6)
Re: 4 pgcrypto regressions failures - 1 unsolved

On Mon, Jul 11, 2005 at 11:46:29AM -0400, Tom Lane wrote:

Marko Kreen <marko@l-t.ee> writes:

Here is the bcopy, bzero removal patch.

Applied.

I'm seeing the following build failure on HPUX:

/usr/ccs/bin/ld +h libpgcrypto.sl.0 -b +b /home/postgres/testversion/lib pgcrypto.o px.o px-hmac.o px-crypt.o misc.o crypt-gensalt.o crypt-blowfish.o crypt-des.o crypt-md5.o md5.o sha1.o sha2.o internal.o blf.o rijndael.o fortuna.o random.o pgp-mpi-internal.o mbuf.o pgp.o pgp-armor.o pgp-cfb.o pgp-compress.o pgp-decrypt.o pgp-encrypt.o pgp-info.o pgp-mpi.o pgp-pubdec.o pgp-pubenc.o pgp-pubkey.o pgp-s2k.o pgp-pgsql.o -L../../src/port `gcc -L../../src/port -Wl,-z -Wl,+b -Wl,/home/postgres/testversion/lib -print-libgcc-file-name` -lz -o libpgcrypto.sl.0
/usr/ccs/bin/ld: Can't find library for -lz
make: *** [libpgcrypto.sl.0] Error 1

I believe the issue is that libz.sl is in /usr/local/lib/, which is not
searched by default by HP's linker. It *is* searched by default by gcc,
which is why -lz works without any explicit -L in the pg_dump/pg_restore
builds. But here we are invoking a different tool with a different
default search path.

Possibly there's something similar happening on that Solaris buildfarm
machine?

No, there it finds the libz (in /usr/local/lib - the link command
is gcc with -L/usr/local/lib), but in not satisfied with .. something.

--
marko

#8Marko Kreen
marko@l-t.ee
In reply to: Tom Lane (#6)
1 attachment(s)
Re: 4 pgcrypto regressions failures - 1 unsolved

Another build failure from buildfarm. Seems like
I forgot to update win32 code when doing a renaming
in random.c

--
marko

Attachments:

winrnd.difftext/plain; charset=us-asciiDownload
Index: contrib/pgcrypto/random.c
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/random.c,v
retrieving revision 1.12
diff -u -c -r1.12 random.c
*** contrib/pgcrypto/random.c	11 Jul 2005 15:07:59 -0000	1.12
--- contrib/pgcrypto/random.c	11 Jul 2005 16:40:16 -0000
***************
*** 126,132 ****
  	if (!res)
  		return dst;
  	
! 	res = CryptGenRandom(h, NUM_BYTES, dst);
  	if (res == TRUE)
  		dst += len;
  
--- 126,132 ----
  	if (!res)
  		return dst;
  	
! 	res = CryptGenRandom(h, RND_BYTES, dst);
  	if (res == TRUE)
  		dst += len;
  
#9Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Kreen (#8)
Re: 4 pgcrypto regressions failures - 1 unsolved

Marko Kreen <marko@l-t.ee> writes:

Another build failure from buildfarm. Seems like
I forgot to update win32 code when doing a renaming
in random.c

Applied.

regards, tom lane

#10Kris Jurka
books@ejurka.com
In reply to: Marko Kreen (#1)
Re: 4 pgcrypto regressions failures - 1 unsolved

Marko Kreen wrote:

http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=dragonfly&amp;dt=2005-07-11%2003:30:04

Linking problem with zlib on Solaris 9/x86. I am clueless about
this. I can anyone look into it?

It appears to be finding the static /usr/local/lib/libz.a instead of the
dynamic /usr/lib/libz.so.

Kris Jurka

#11Marko Kreen
marko@l-t.ee
In reply to: Kris Jurka (#10)
Re: 4 pgcrypto regressions failures - 1 unsolved

On Mon, Jul 11, 2005 at 04:47:18PM -0700, Kris Jurka wrote:

Marko Kreen wrote:

http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=dragonfly&amp;dt=2005-07-11%2003:30:04

Linking problem with zlib on Solaris 9/x86. I am clueless about
this. I can anyone look into it?

It appears to be finding the static /usr/local/lib/libz.a instead of the
dynamic /usr/lib/libz.so.

So it is a local problem? I see that the configure line contains:
"--with-includes=/usr/local/include --with-libs=/usr/local/lib"
explicitly.

--
marko

#12Marko Kreen
marko@l-t.ee
In reply to: Marko Kreen (#1)
1 attachment(s)
Re: 4 pgcrypto regressions failures - 1 unsolved

Hopefully the last regression failure.

- openssl.c used EVP_MAX_KEY_LENGTH / EVP_MAX_IV_LENGTH
constants for buffers, which are small in case of
OpenSSL 0.9.6x and internal AES. (I tested it with
0.9.7 only, so I didn't notice...)
- Also I noticed that the wrapper macros for CBC mode
do not update IV buffer.
- As the previous mistake was not picked up by current
regression tests, I added a 'longer than a block'
test to all ciphers.

--
marko

Attachments:

oldssl.fix.difftext/plain; charset=us-asciiDownload
Index: contrib/pgcrypto/openssl.c
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/openssl.c,v
retrieving revision 1.24
diff -u -c -r1.24 openssl.c
*** contrib/pgcrypto/openssl.c	11 Jul 2005 15:07:59 -0000	1.24
--- contrib/pgcrypto/openssl.c	12 Jul 2005 09:27:59 -0000
***************
*** 40,45 ****
--- 40,50 ----
  #include <openssl/rand.h>
  #include <openssl/err.h>
  
+ /*
+  * Max lengths we might want to handle.
+  */
+ #define MAX_KEY		(512/8)
+ #define MAX_IV		(128/8)
  
  /*
   * Does OpenSSL support AES? 
***************
*** 78,87 ****
  #define AES_cbc_encrypt(src, dst, len, ctx, iv, enc) \
  	do { \
  		memcpy((dst), (src), (len)); \
! 		if (enc) \
  			aes_cbc_encrypt((ctx), (iv), (dst), (len)); \
! 		else \
  			aes_cbc_decrypt((ctx), (iv), (dst), (len)); \
  	} while (0)
  
  #endif	/* old OPENSSL */
--- 83,95 ----
  #define AES_cbc_encrypt(src, dst, len, ctx, iv, enc) \
  	do { \
  		memcpy((dst), (src), (len)); \
! 		if (enc) { \
  			aes_cbc_encrypt((ctx), (iv), (dst), (len)); \
! 			memcpy((iv), (dst) + (len) - 16, 16); \
! 		} else { \
  			aes_cbc_decrypt((ctx), (iv), (dst), (len)); \
+ 			memcpy(iv, (src) + (len) - 16, 16); \
+ 		} \
  	} while (0)
  
  #endif	/* old OPENSSL */
***************
*** 243,250 ****
  		CAST_KEY	cast_key;
  		AES_KEY		aes_key;
  	}			u;
! 	uint8		key[EVP_MAX_KEY_LENGTH];
! 	uint8		iv[EVP_MAX_IV_LENGTH];
  	unsigned	klen;
  	unsigned	init;
  	const struct ossl_cipher *ciph;
--- 251,258 ----
  		CAST_KEY	cast_key;
  		AES_KEY		aes_key;
  	}			u;
! 	uint8		key[MAX_KEY];
! 	uint8		iv[MAX_IV];
  	unsigned	klen;
  	unsigned	init;
  	const struct ossl_cipher *ciph;
Index: contrib/pgcrypto/expected/3des.out
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/expected/3des.out,v
retrieving revision 1.2
diff -u -c -r1.2 3des.out
*** contrib/pgcrypto/expected/3des.out	5 Jul 2005 18:15:36 -0000	1.2
--- contrib/pgcrypto/expected/3des.out	12 Jul 2005 09:41:16 -0000
***************
*** 54,56 ****
--- 54,69 ----
   foo
  (1 row)
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '0123456789012345678901', '3des'), 'hex');
+                               encode                              
+ ------------------------------------------------------------------
+  b71e3422269d0ded19468f33d65cd663c28e0871984792a7b3ba0ddcecec8d2c
+ (1 row)
+ 
+ select decrypt(encrypt('Lets try a longer message.', '0123456789012345678901', '3des'), '0123456789012345678901', '3des');
+           decrypt           
+ ----------------------------
+  Lets try a longer message.
+ (1 row)
+ 
Index: contrib/pgcrypto/expected/blowfish.out
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/expected/blowfish.out,v
retrieving revision 1.4
diff -u -c -r1.4 blowfish.out
*** contrib/pgcrypto/expected/blowfish.out	21 Mar 2005 05:24:51 -0000	1.4
--- contrib/pgcrypto/expected/blowfish.out	12 Jul 2005 09:32:13 -0000
***************
*** 158,160 ****
--- 158,173 ----
   foo
  (1 row)
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '0123456789', 'bf'), 'hex');
+                               encode                              
+ ------------------------------------------------------------------
+  a76059f7a1b627b5b84080d9beb337714c7a7f8b70300023e5feb6dfa6813536
+ (1 row)
+ 
+ select decrypt(encrypt('Lets try a longer message.', '0123456789', 'bf'), '0123456789', 'bf');
+           decrypt           
+ ----------------------------
+  Lets try a longer message.
+ (1 row)
+ 
Index: contrib/pgcrypto/expected/cast5.out
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/expected/cast5.out,v
retrieving revision 1.1
diff -u -c -r1.1 cast5.out
*** contrib/pgcrypto/expected/cast5.out	21 Mar 2005 05:24:51 -0000	1.1
--- contrib/pgcrypto/expected/cast5.out	12 Jul 2005 09:41:22 -0000
***************
*** 71,73 ****
--- 71,86 ----
   foo
  (1 row)
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '0123456789', 'cast5'), 'hex');
+                               encode                              
+ ------------------------------------------------------------------
+  04fcffc91533e1505dadcb10766d9fed0937818e663e402384e049942ba60fff
+ (1 row)
+ 
+ select decrypt(encrypt('Lets try a longer message.', '0123456789', 'cast5'), '0123456789', 'cast5');
+           decrypt           
+ ----------------------------
+  Lets try a longer message.
+ (1 row)
+ 
Index: contrib/pgcrypto/expected/des.out
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/expected/des.out,v
retrieving revision 1.1
diff -u -c -r1.1 des.out
*** contrib/pgcrypto/expected/des.out	21 Mar 2005 05:24:51 -0000	1.1
--- contrib/pgcrypto/expected/des.out	12 Jul 2005 09:41:19 -0000
***************
*** 46,48 ****
--- 46,61 ----
   foo
  (1 row)
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '01234567', 'des'), 'hex');
+                               encode                              
+ ------------------------------------------------------------------
+  5ad146043e5f30967e06a0fcbae602daf4ff2a5fd0ed12d6c5913cf85f1e36ca
+ (1 row)
+ 
+ select decrypt(encrypt('Lets try a longer message.', '01234567', 'des'), '01234567', 'des');
+           decrypt           
+ ----------------------------
+  Lets try a longer message.
+ (1 row)
+ 
Index: contrib/pgcrypto/expected/rijndael.out
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/expected/rijndael.out,v
retrieving revision 1.4
diff -u -c -r1.4 rijndael.out
*** contrib/pgcrypto/expected/rijndael.out	21 Mar 2005 05:24:51 -0000	1.4
--- contrib/pgcrypto/expected/rijndael.out	12 Jul 2005 09:27:59 -0000
***************
*** 109,111 ****
--- 109,124 ----
   foo
  (1 row)
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '0123456789', 'aes'), 'hex');
+                               encode                              
+ ------------------------------------------------------------------
+  d9beb785dd5403ed02f66b755bb191b93ed93ca54930153f2c3b9ec7785056ad
+ (1 row)
+ 
+ select decrypt(encrypt('Lets try a longer message.', '0123456789', 'aes'), '0123456789', 'aes');
+           decrypt           
+ ----------------------------
+  Lets try a longer message.
+ (1 row)
+ 
Index: contrib/pgcrypto/sql/3des.sql
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/sql/3des.sql,v
retrieving revision 1.2
diff -u -c -r1.2 3des.sql
*** contrib/pgcrypto/sql/3des.sql	5 Jul 2005 18:15:36 -0000	1.2
--- contrib/pgcrypto/sql/3des.sql	12 Jul 2005 09:40:00 -0000
***************
*** 24,26 ****
--- 24,30 ----
  select encode(encrypt_iv('foo', '0123456', 'abcd', '3des'), 'hex');
  select decrypt_iv(decode('50735067b073bb93', 'hex'), '0123456', 'abcd', '3des');
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '0123456789012345678901', '3des'), 'hex');
+ select decrypt(encrypt('Lets try a longer message.', '0123456789012345678901', '3des'), '0123456789012345678901', '3des');
+ 
Index: contrib/pgcrypto/sql/blowfish.sql
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/sql/blowfish.sql,v
retrieving revision 1.4
diff -u -c -r1.4 blowfish.sql
*** contrib/pgcrypto/sql/blowfish.sql	21 Mar 2005 05:24:52 -0000	1.4
--- contrib/pgcrypto/sql/blowfish.sql	12 Jul 2005 09:31:51 -0000
***************
*** 85,87 ****
--- 85,91 ----
  select encode(encrypt_iv('foo', '0123456', 'abcd', 'bf'), 'hex');
  select decrypt_iv(decode('95c7e89322525d59', 'hex'), '0123456', 'abcd', 'bf');
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '0123456789', 'bf'), 'hex');
+ select decrypt(encrypt('Lets try a longer message.', '0123456789', 'bf'), '0123456789', 'bf');
+ 
Index: contrib/pgcrypto/sql/cast5.sql
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/sql/cast5.sql,v
retrieving revision 1.1
diff -u -c -r1.1 cast5.sql
*** contrib/pgcrypto/sql/cast5.sql	21 Mar 2005 05:24:52 -0000	1.1
--- contrib/pgcrypto/sql/cast5.sql	12 Jul 2005 09:40:55 -0000
***************
*** 40,42 ****
--- 40,46 ----
  select decrypt_iv(decode('384a970695ce016a', 'hex'),
                  '0123456', 'abcd', 'cast5');
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '0123456789', 'cast5'), 'hex');
+ select decrypt(encrypt('Lets try a longer message.', '0123456789', 'cast5'), '0123456789', 'cast5');
+ 
Index: contrib/pgcrypto/sql/des.sql
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/sql/des.sql,v
retrieving revision 1.1
diff -u -c -r1.1 des.sql
*** contrib/pgcrypto/sql/des.sql	21 Mar 2005 05:24:52 -0000	1.1
--- contrib/pgcrypto/sql/des.sql	12 Jul 2005 09:38:27 -0000
***************
*** 22,24 ****
--- 22,28 ----
  select encode(encrypt_iv('foo', '0123456', 'abcd', 'des'), 'hex');
  select decrypt_iv(decode('50735067b073bb93', 'hex'), '0123456', 'abcd', 'des');
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '01234567', 'des'), 'hex');
+ select decrypt(encrypt('Lets try a longer message.', '01234567', 'des'), '01234567', 'des');
+ 
Index: contrib/pgcrypto/sql/rijndael.sql
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/sql/rijndael.sql,v
retrieving revision 1.4
diff -u -c -r1.4 rijndael.sql
*** contrib/pgcrypto/sql/rijndael.sql	21 Mar 2005 05:24:52 -0000	1.4
--- contrib/pgcrypto/sql/rijndael.sql	12 Jul 2005 09:27:59 -0000
***************
*** 56,58 ****
--- 56,62 ----
  select decrypt_iv(decode('2c24cb7da91d6d5699801268b0f5adad', 'hex'),
  		'0123456', 'abcd', 'aes'); 
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '0123456789', 'aes'), 'hex');
+ select decrypt(encrypt('Lets try a longer message.', '0123456789', 'aes'), '0123456789', 'aes');
+ 
#13Kris Jurka
books@ejurka.com
In reply to: Marko Kreen (#11)
Re: 4 pgcrypto regressions failures - 1 unsolved

On Tue, 12 Jul 2005, Marko Kreen wrote:

On Mon, Jul 11, 2005 at 04:47:18PM -0700, Kris Jurka wrote:

Marko Kreen wrote:

http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=dragonfly&amp;dt=2005-07-11%2003:30:04

Linking problem with zlib on Solaris 9/x86. I am clueless about
this. I can anyone look into it?

It appears to be finding the static /usr/local/lib/libz.a instead of the
dynamic /usr/lib/libz.so.

So it is a local problem? I see that the configure line contains:
"--with-includes=/usr/local/include --with-libs=/usr/local/lib"
explicitly.

Well the buildfarm machine kudu is actually the same machine just building
with the Sun compiler and it works fine. It links all of libz.a into
libpgcrypto.so while gcc refuses to.

Kris Jurka

#14Marko Kreen
marko@l-t.ee
In reply to: Marko Kreen (#12)
1 attachment(s)
Re: 4 pgcrypto regressions failures - 1 unsolved

Seems like down mail server ate first mail.

Here it is again.

Show quoted text

On Tue, Jul 12, 2005 at 12:51:44PM +0300, Marko Kreen wrote:

Hopefully the last regression failure.

- openssl.c used EVP_MAX_KEY_LENGTH / EVP_MAX_IV_LENGTH
constants for buffers, which are small in case of
OpenSSL 0.9.6x and internal AES. (I tested it with
0.9.7 only, so I didn't notice...)
- Also I noticed that the wrapper macros for CBC mode
do not update IV buffer.
- As the previous mistake was not picked up by current
regression tests, I added a 'longer than a block'
test to all ciphers.

--
marko

Attachments:

oldssl.fix.difftext/plain; charset=us-asciiDownload
Index: contrib/pgcrypto/openssl.c
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/openssl.c,v
retrieving revision 1.24
diff -u -c -r1.24 openssl.c
*** contrib/pgcrypto/openssl.c	11 Jul 2005 15:07:59 -0000	1.24
--- contrib/pgcrypto/openssl.c	12 Jul 2005 09:27:59 -0000
***************
*** 40,45 ****
--- 40,50 ----
  #include <openssl/rand.h>
  #include <openssl/err.h>
  
+ /*
+  * Max lengths we might want to handle.
+  */
+ #define MAX_KEY		(512/8)
+ #define MAX_IV		(128/8)
  
  /*
   * Does OpenSSL support AES? 
***************
*** 78,87 ****
  #define AES_cbc_encrypt(src, dst, len, ctx, iv, enc) \
  	do { \
  		memcpy((dst), (src), (len)); \
! 		if (enc) \
  			aes_cbc_encrypt((ctx), (iv), (dst), (len)); \
! 		else \
  			aes_cbc_decrypt((ctx), (iv), (dst), (len)); \
  	} while (0)
  
  #endif	/* old OPENSSL */
--- 83,95 ----
  #define AES_cbc_encrypt(src, dst, len, ctx, iv, enc) \
  	do { \
  		memcpy((dst), (src), (len)); \
! 		if (enc) { \
  			aes_cbc_encrypt((ctx), (iv), (dst), (len)); \
! 			memcpy((iv), (dst) + (len) - 16, 16); \
! 		} else { \
  			aes_cbc_decrypt((ctx), (iv), (dst), (len)); \
+ 			memcpy(iv, (src) + (len) - 16, 16); \
+ 		} \
  	} while (0)
  
  #endif	/* old OPENSSL */
***************
*** 243,250 ****
  		CAST_KEY	cast_key;
  		AES_KEY		aes_key;
  	}			u;
! 	uint8		key[EVP_MAX_KEY_LENGTH];
! 	uint8		iv[EVP_MAX_IV_LENGTH];
  	unsigned	klen;
  	unsigned	init;
  	const struct ossl_cipher *ciph;
--- 251,258 ----
  		CAST_KEY	cast_key;
  		AES_KEY		aes_key;
  	}			u;
! 	uint8		key[MAX_KEY];
! 	uint8		iv[MAX_IV];
  	unsigned	klen;
  	unsigned	init;
  	const struct ossl_cipher *ciph;
Index: contrib/pgcrypto/expected/3des.out
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/expected/3des.out,v
retrieving revision 1.2
diff -u -c -r1.2 3des.out
*** contrib/pgcrypto/expected/3des.out	5 Jul 2005 18:15:36 -0000	1.2
--- contrib/pgcrypto/expected/3des.out	12 Jul 2005 09:41:16 -0000
***************
*** 54,56 ****
--- 54,69 ----
   foo
  (1 row)
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '0123456789012345678901', '3des'), 'hex');
+                               encode                              
+ ------------------------------------------------------------------
+  b71e3422269d0ded19468f33d65cd663c28e0871984792a7b3ba0ddcecec8d2c
+ (1 row)
+ 
+ select decrypt(encrypt('Lets try a longer message.', '0123456789012345678901', '3des'), '0123456789012345678901', '3des');
+           decrypt           
+ ----------------------------
+  Lets try a longer message.
+ (1 row)
+ 
Index: contrib/pgcrypto/expected/blowfish.out
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/expected/blowfish.out,v
retrieving revision 1.4
diff -u -c -r1.4 blowfish.out
*** contrib/pgcrypto/expected/blowfish.out	21 Mar 2005 05:24:51 -0000	1.4
--- contrib/pgcrypto/expected/blowfish.out	12 Jul 2005 09:32:13 -0000
***************
*** 158,160 ****
--- 158,173 ----
   foo
  (1 row)
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '0123456789', 'bf'), 'hex');
+                               encode                              
+ ------------------------------------------------------------------
+  a76059f7a1b627b5b84080d9beb337714c7a7f8b70300023e5feb6dfa6813536
+ (1 row)
+ 
+ select decrypt(encrypt('Lets try a longer message.', '0123456789', 'bf'), '0123456789', 'bf');
+           decrypt           
+ ----------------------------
+  Lets try a longer message.
+ (1 row)
+ 
Index: contrib/pgcrypto/expected/cast5.out
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/expected/cast5.out,v
retrieving revision 1.1
diff -u -c -r1.1 cast5.out
*** contrib/pgcrypto/expected/cast5.out	21 Mar 2005 05:24:51 -0000	1.1
--- contrib/pgcrypto/expected/cast5.out	12 Jul 2005 09:41:22 -0000
***************
*** 71,73 ****
--- 71,86 ----
   foo
  (1 row)
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '0123456789', 'cast5'), 'hex');
+                               encode                              
+ ------------------------------------------------------------------
+  04fcffc91533e1505dadcb10766d9fed0937818e663e402384e049942ba60fff
+ (1 row)
+ 
+ select decrypt(encrypt('Lets try a longer message.', '0123456789', 'cast5'), '0123456789', 'cast5');
+           decrypt           
+ ----------------------------
+  Lets try a longer message.
+ (1 row)
+ 
Index: contrib/pgcrypto/expected/des.out
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/expected/des.out,v
retrieving revision 1.1
diff -u -c -r1.1 des.out
*** contrib/pgcrypto/expected/des.out	21 Mar 2005 05:24:51 -0000	1.1
--- contrib/pgcrypto/expected/des.out	12 Jul 2005 09:41:19 -0000
***************
*** 46,48 ****
--- 46,61 ----
   foo
  (1 row)
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '01234567', 'des'), 'hex');
+                               encode                              
+ ------------------------------------------------------------------
+  5ad146043e5f30967e06a0fcbae602daf4ff2a5fd0ed12d6c5913cf85f1e36ca
+ (1 row)
+ 
+ select decrypt(encrypt('Lets try a longer message.', '01234567', 'des'), '01234567', 'des');
+           decrypt           
+ ----------------------------
+  Lets try a longer message.
+ (1 row)
+ 
Index: contrib/pgcrypto/expected/rijndael.out
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/expected/rijndael.out,v
retrieving revision 1.4
diff -u -c -r1.4 rijndael.out
*** contrib/pgcrypto/expected/rijndael.out	21 Mar 2005 05:24:51 -0000	1.4
--- contrib/pgcrypto/expected/rijndael.out	12 Jul 2005 09:27:59 -0000
***************
*** 109,111 ****
--- 109,124 ----
   foo
  (1 row)
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '0123456789', 'aes'), 'hex');
+                               encode                              
+ ------------------------------------------------------------------
+  d9beb785dd5403ed02f66b755bb191b93ed93ca54930153f2c3b9ec7785056ad
+ (1 row)
+ 
+ select decrypt(encrypt('Lets try a longer message.', '0123456789', 'aes'), '0123456789', 'aes');
+           decrypt           
+ ----------------------------
+  Lets try a longer message.
+ (1 row)
+ 
Index: contrib/pgcrypto/sql/3des.sql
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/sql/3des.sql,v
retrieving revision 1.2
diff -u -c -r1.2 3des.sql
*** contrib/pgcrypto/sql/3des.sql	5 Jul 2005 18:15:36 -0000	1.2
--- contrib/pgcrypto/sql/3des.sql	12 Jul 2005 09:40:00 -0000
***************
*** 24,26 ****
--- 24,30 ----
  select encode(encrypt_iv('foo', '0123456', 'abcd', '3des'), 'hex');
  select decrypt_iv(decode('50735067b073bb93', 'hex'), '0123456', 'abcd', '3des');
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '0123456789012345678901', '3des'), 'hex');
+ select decrypt(encrypt('Lets try a longer message.', '0123456789012345678901', '3des'), '0123456789012345678901', '3des');
+ 
Index: contrib/pgcrypto/sql/blowfish.sql
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/sql/blowfish.sql,v
retrieving revision 1.4
diff -u -c -r1.4 blowfish.sql
*** contrib/pgcrypto/sql/blowfish.sql	21 Mar 2005 05:24:52 -0000	1.4
--- contrib/pgcrypto/sql/blowfish.sql	12 Jul 2005 09:31:51 -0000
***************
*** 85,87 ****
--- 85,91 ----
  select encode(encrypt_iv('foo', '0123456', 'abcd', 'bf'), 'hex');
  select decrypt_iv(decode('95c7e89322525d59', 'hex'), '0123456', 'abcd', 'bf');
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '0123456789', 'bf'), 'hex');
+ select decrypt(encrypt('Lets try a longer message.', '0123456789', 'bf'), '0123456789', 'bf');
+ 
Index: contrib/pgcrypto/sql/cast5.sql
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/sql/cast5.sql,v
retrieving revision 1.1
diff -u -c -r1.1 cast5.sql
*** contrib/pgcrypto/sql/cast5.sql	21 Mar 2005 05:24:52 -0000	1.1
--- contrib/pgcrypto/sql/cast5.sql	12 Jul 2005 09:40:55 -0000
***************
*** 40,42 ****
--- 40,46 ----
  select decrypt_iv(decode('384a970695ce016a', 'hex'),
                  '0123456', 'abcd', 'cast5');
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '0123456789', 'cast5'), 'hex');
+ select decrypt(encrypt('Lets try a longer message.', '0123456789', 'cast5'), '0123456789', 'cast5');
+ 
Index: contrib/pgcrypto/sql/des.sql
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/sql/des.sql,v
retrieving revision 1.1
diff -u -c -r1.1 des.sql
*** contrib/pgcrypto/sql/des.sql	21 Mar 2005 05:24:52 -0000	1.1
--- contrib/pgcrypto/sql/des.sql	12 Jul 2005 09:38:27 -0000
***************
*** 22,24 ****
--- 22,28 ----
  select encode(encrypt_iv('foo', '0123456', 'abcd', 'des'), 'hex');
  select decrypt_iv(decode('50735067b073bb93', 'hex'), '0123456', 'abcd', 'des');
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '01234567', 'des'), 'hex');
+ select decrypt(encrypt('Lets try a longer message.', '01234567', 'des'), '01234567', 'des');
+ 
Index: contrib/pgcrypto/sql/rijndael.sql
===================================================================
RCS file: /opt/arc/cvs2/pgsql/contrib/pgcrypto/sql/rijndael.sql,v
retrieving revision 1.4
diff -u -c -r1.4 rijndael.sql
*** contrib/pgcrypto/sql/rijndael.sql	21 Mar 2005 05:24:52 -0000	1.4
--- contrib/pgcrypto/sql/rijndael.sql	12 Jul 2005 09:27:59 -0000
***************
*** 56,58 ****
--- 56,62 ----
  select decrypt_iv(decode('2c24cb7da91d6d5699801268b0f5adad', 'hex'),
  		'0123456', 'abcd', 'aes'); 
  
+ -- long message
+ select encode(encrypt('Lets try a longer message.', '0123456789', 'aes'), 'hex');
+ select decrypt(encrypt('Lets try a longer message.', '0123456789', 'aes'), '0123456789', 'aes');
+ 
#15Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Kreen (#12)
Re: 4 pgcrypto regressions failures - 1 unsolved

Marko Kreen <marko@l-t.ee> writes:

Hopefully the last regression failure.

- openssl.c used EVP_MAX_KEY_LENGTH / EVP_MAX_IV_LENGTH
constants for buffers, which are small in case of
OpenSSL 0.9.6x and internal AES. (I tested it with
0.9.7 only, so I didn't notice...)
- Also I noticed that the wrapper macros for CBC mode
do not update IV buffer.
- As the previous mistake was not picked up by current
regression tests, I added a 'longer than a block'
test to all ciphers.

Applied, thanks.

regards, tom lane

#16Marko Kreen
marko@l-t.ee
In reply to: Kris Jurka (#13)
1 attachment(s)
Re: 4 pgcrypto regressions failures - 1 unsolved

[buildfarm machine dragonfly]

On Tue, Jul 12, 2005 at 01:06:46PM -0500, Kris Jurka wrote:

Well the buildfarm machine kudu is actually the same machine just building
with the Sun compiler and it works fine. It links all of libz.a into
libpgcrypto.so while gcc refuses to.

I googled a bit and found two suggestions:

1. http://curl.haxx.se/mail/lib-2002-01/0092.html
(Use -mimpure-text on linking line)

2. http://www.filibeto.org/pipermail/solaris-users/2004-March/000662.html
(that using GNU ld not Sun's fixes it)

The attached patch does #1. Could you try it and see if it fixes it?

--
marko

Attachments:

solgcc.difftext/plain; charset=us-asciiDownload
Index: src/Makefile.shlib
===================================================================
RCS file: /opt/arc/cvs2/pgsql/src/Makefile.shlib,v
retrieving revision 1.95
diff -u -c -r1.95 Makefile.shlib
*** src/Makefile.shlib	13 Jul 2005 17:00:44 -0000	1.95
--- src/Makefile.shlib	15 Jul 2005 17:11:57 -0000
***************
*** 188,194 ****
  
  ifeq ($(PORTNAME), solaris)
    ifeq ($(GCC), yes)
!     LINK.shared		= $(CC) -shared
    else
      LINK.shared		= $(CC) -G
    endif
--- 188,194 ----
  
  ifeq ($(PORTNAME), solaris)
    ifeq ($(GCC), yes)
!     LINK.shared		= $(CC) -shared -mimpure-text
    else
      LINK.shared		= $(CC) -G
    endif
#17Kris Jurka
books@ejurka.com
In reply to: Marko Kreen (#16)
Re: 4 pgcrypto regressions failures - 1 unsolved

On Fri, 15 Jul 2005, Marko Kreen wrote:

[buildfarm machine dragonfly]

On Tue, Jul 12, 2005 at 01:06:46PM -0500, Kris Jurka wrote:

Well the buildfarm machine kudu is actually the same machine just building
with the Sun compiler and it works fine. It links all of libz.a into
libpgcrypto.so while gcc refuses to.

I googled a bit and found two suggestions:

1. http://curl.haxx.se/mail/lib-2002-01/0092.html
(Use -mimpure-text on linking line)

The attached patch does #1. Could you try it and see if it fixes it?

This patch works, pgcrypto links and passes its installcheck test now.

Kris Jurka

#18Marko Kreen
marko@l-t.ee
In reply to: Kris Jurka (#17)
1 attachment(s)
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved

On Fri, Jul 15, 2005 at 08:06:15PM -0500, Kris Jurka wrote:

On Fri, 15 Jul 2005, Marko Kreen wrote:

[buildfarm machine dragonfly]

On Tue, Jul 12, 2005 at 01:06:46PM -0500, Kris Jurka wrote:

Well the buildfarm machine kudu is actually the same machine just building
with the Sun compiler and it works fine. It links all of libz.a into
libpgcrypto.so while gcc refuses to.

I googled a bit and found two suggestions:

1. http://curl.haxx.se/mail/lib-2002-01/0092.html
(Use -mimpure-text on linking line)

The attached patch does #1. Could you try it and see if it fixes it?

This patch works, pgcrypto links and passes its installcheck test now.

Kris Jurka

Thanks.

Here is the patch with a little comment.

It should not break anything as it just disables a extra
argument "-assert pure-text" to linker.

Linking static libraries into shared one is bad idea, as the
static parts wont be shared between processes, but erroring
out is worse, especially if another compiler for a platform
allows it.

This makes gcc act same way as Sun's cc.

--
marko

Attachments:

solgcc2.difftext/plain; charset=us-asciiDownload
Index: src/Makefile.shlib
===================================================================
RCS file: /opt/arc/cvs2/pgsql/src/Makefile.shlib,v
retrieving revision 1.95
diff -u -c -r1.95 Makefile.shlib
*** src/Makefile.shlib	13 Jul 2005 17:00:44 -0000	1.95
--- src/Makefile.shlib	16 Jul 2005 09:59:18 -0000
***************
*** 188,194 ****
  
  ifeq ($(PORTNAME), solaris)
    ifeq ($(GCC), yes)
!     LINK.shared		= $(CC) -shared
    else
      LINK.shared		= $(CC) -G
    endif
--- 188,196 ----
  
  ifeq ($(PORTNAME), solaris)
    ifeq ($(GCC), yes)
!     # -mimpure-text disables passing '-assert pure-text' to linker,
!     # to allow linking static library into shared one, like Sun's cc does.
!     LINK.shared		= $(CC) -shared -mimpure-text
    else
      LINK.shared		= $(CC) -G
    endif
#19Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Kreen (#18)
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved

Marko Kreen <marko@l-t.ee> writes:

On Tue, Jul 12, 2005 at 01:06:46PM -0500, Kris Jurka wrote:

Well the buildfarm machine kudu is actually the same machine just building
with the Sun compiler and it works fine. It links all of libz.a into
libpgcrypto.so while gcc refuses to.

I googled a bit and found two suggestions:

1. http://curl.haxx.se/mail/lib-2002-01/0092.html
(Use -mimpure-text on linking line)

The attached patch does #1. Could you try it and see if it fixes it?

This sure seems like a crude band-aid rather than an actual solution.
The bug as I see it is that gcc is choosing to link libz.a rather than
libz.so --- why is that happening?

regards, tom lane

#20Kris Jurka
books@ejurka.com
In reply to: Tom Lane (#19)
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved

On Sat, 16 Jul 2005, Tom Lane wrote:

Marko Kreen <marko@l-t.ee> writes:

I googled a bit and found two suggestions:

1. http://curl.haxx.se/mail/lib-2002-01/0092.html
(Use -mimpure-text on linking line)

This sure seems like a crude band-aid rather than an actual solution.
The bug as I see it is that gcc is choosing to link libz.a rather than
libz.so --- why is that happening?

The link line says -L/usr/local/lib -lz and libz.a is in /usr/local/lib
while libz.so is in /usr/lib.

Kris Jurka

#21Tom Lane
tgl@sss.pgh.pa.us
In reply to: Kris Jurka (#20)
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved

Kris Jurka <books@ejurka.com> writes:

On Sat, 16 Jul 2005, Tom Lane wrote:

This sure seems like a crude band-aid rather than an actual solution.
The bug as I see it is that gcc is choosing to link libz.a rather than
libz.so --- why is that happening?

The link line says -L/usr/local/lib -lz and libz.a is in /usr/local/lib
while libz.so is in /usr/lib.

Well, that is a flat-out configuration error on the local sysadmin's
part. I can't think of any good reason for the .so and .a versions of a
library to live in different places. We certainly shouldn't hack our
build process to build deliberately-inefficient object files in order to
accommodate such a setup.

regards, tom lane

#22Kris Jurka
books@ejurka.com
In reply to: Tom Lane (#21)
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved

On Sat, 16 Jul 2005, Tom Lane wrote:

Kris Jurka <books@ejurka.com> writes:

The link line says -L/usr/local/lib -lz and libz.a is in /usr/local/lib
while libz.so is in /usr/lib.

Well, that is a flat-out configuration error on the local sysadmin's
part. I can't think of any good reason for the .so and .a versions of a
library to live in different places. We certainly shouldn't hack our
build process to build deliberately-inefficient object files in order to
accommodate such a setup.

Well the OS only came with the shared library and I needed the static one
for some reason, so I installed it alone under /usr/local. This works
fine with Sun's cc and Marko's research indicates that this will also
work fine using GNU ld instead of Sun's ld. This is certainly an unusual
thing to do, but I don't believe it is a flat-out configuration error,
consider what would happen if the shared library didn't exist at all and
only a static version were available. Until this recent batch of pgcrypto
changes everything built fine.

Kris Jurka

#23Tom Lane
tgl@sss.pgh.pa.us
In reply to: Kris Jurka (#22)
Re: [HACKERS] 4 pgcrypto regressions failures - 1 unsolved

Kris Jurka <books@ejurka.com> writes:

consider what would happen if the shared library didn't exist at all and
only a static version were available. Until this recent batch of pgcrypto
changes everything built fine.

Well, the right answer to that really is that pgcrypto ought not try to
link to libz unless a shared libz is available (compare for instance the
situation with plperl and an unshared libperl). However, I'm not sure
that we could reasonably expect to make a configuration test that would
detect a situation like this --- that is, if we did look for shared
libz, we would find it, and the fact that a nonshared libz in a
different place would cause the actual link to fail seems like something
that configure would be unlikely to be able to realize.

I'm still of the opinion that your libz installation is broken; the fact
that some other products chance not to fail with it is not evidence that
it's OK. You could for instance have installed both libz.a and libz.so
from the same build in /usr/local/lib, and that would work fine,
independently of the existence of a version in /usr/lib.

Come to think of it, are you sure that the versions in /usr/lib and
/usr/local/lib are even ABI-compatible? If they are from different zlib
releases, I think you're risking trouble regardless. Really the right
way to deal with this sort of thing is that you put libz.a and libz.so
in /usr/local/lib and corresponding headers in /usr/local/include, and
then you don't need to sweat whether they are exactly compatible with
what appears in /usr/lib and /usr/include.

regards, tom lane