pgsql: Use OpenSSL EVP API for symmetric encryption in pgcrypto.

Heikki Linnakangas <heikki.linnakangas@iki.fi> writes:

Use OpenSSL EVP API for symmetric encryption in pgcrypto.

BTW, "narwhal" seems to have a problem with this.
Not very clear what, maybe an incompatibility with old openssl versions?

regards, tom lane

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

On Tue, Oct 18, 2016 at 6:28 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Heikki Linnakangas <heikki.linnakangas@iki.fi> writes:

Use OpenSSL EVP API for symmetric encryption in pgcrypto.

BTW, "narwhal" seems to have a problem with this.
Not very clear what, maybe an incompatibility with old openssl versions?

Details are here:
http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=narwhal&amp;dt=2016-10-17%2016%3A00%3A01
--
Michael

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

On 10/18/2016 12:28 AM, Tom Lane wrote:

Heikki Linnakangas <heikki.linnakangas@iki.fi> writes:

Use OpenSSL EVP API for symmetric encryption in pgcrypto.

BTW, "narwhal" seems to have a problem with this.
Not very clear what, maybe an incompatibility with old openssl versions?

Dave, what version of OpenSSL are 'narwhal' and 'baiji' using? I've
tried to reproduce this on my laptop, by compiling different versions of
OpenSSL, between 0.9.8beta1 and 0.9.8 head, but without success..

Googling for the failed test vector, I found this:

https://www.mail-archive.com/openssl-users@openssl.org/msg68108.html

It looks like the same issue, given the symptoms, but AFAICS the
pgcrypto code is correctly calling EVP_EncryptInit_ex() twice.

- Heikki

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

On Tue, Oct 18, 2016 at 9:10 AM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:

On 10/18/2016 12:28 AM, Tom Lane wrote:

Heikki Linnakangas <heikki.linnakangas@iki.fi> writes:

Use OpenSSL EVP API for symmetric encryption in pgcrypto.

BTW, "narwhal" seems to have a problem with this.
Not very clear what, maybe an incompatibility with old openssl versions?

Dave, what version of OpenSSL are 'narwhal' and 'baiji' using? I've tried to
reproduce this on my laptop, by compiling different versions of OpenSSL,
between 0.9.8beta1 and 0.9.8 head, but without success..

Insanely old ones, as reflects the age of those machines which haven't
been touched in 10 years or so...

Baiji: 0.9.8e
Narwhal: 0.9.6b

and for completeness:

Mastodon: 0.9.8e

--
Dave Page
PostgreSQL Core Team
http://www.postgresql.org/

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

On 18 October 2016 12:52:14 EEST, Dave Page <dpage@postgresql.org> wrote:

On Tue, Oct 18, 2016 at 9:10 AM, Heikki Linnakangas <hlinnaka@iki.fi>
wrote:

On 10/18/2016 12:28 AM, Tom Lane wrote:

Heikki Linnakangas <heikki.linnakangas@iki.fi> writes:

Use OpenSSL EVP API for symmetric encryption in pgcrypto.

BTW, "narwhal" seems to have a problem with this.
Not very clear what, maybe an incompatibility with old openssl

versions?

Dave, what version of OpenSSL are 'narwhal' and 'baiji' using? I've

tried to

reproduce this on my laptop, by compiling different versions of

OpenSSL,

between 0.9.8beta1 and 0.9.8 head, but without success..

Insanely old ones, as reflects the age of those machines which haven't
been touched in 10 years or so...

Baiji: 0.9.8e
Narwhal: 0.9.6b

Hang on, I removed support for OpenSSL < 0.9.8 a while ago. Narwhal shouldn't even compile with 0.9.6.

- Heikki

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

On Tue, Oct 18, 2016 at 12:18 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:

On 18 October 2016 12:52:14 EEST, Dave Page <dpage@postgresql.org> wrote:

On Tue, Oct 18, 2016 at 9:10 AM, Heikki Linnakangas <hlinnaka@iki.fi>
wrote:

On 10/18/2016 12:28 AM, Tom Lane wrote:

Heikki Linnakangas <heikki.linnakangas@iki.fi> writes:

Use OpenSSL EVP API for symmetric encryption in pgcrypto.

BTW, "narwhal" seems to have a problem with this.
Not very clear what, maybe an incompatibility with old openssl

versions?

Dave, what version of OpenSSL are 'narwhal' and 'baiji' using? I've

tried to

reproduce this on my laptop, by compiling different versions of

OpenSSL,

between 0.9.8beta1 and 0.9.8 head, but without success..

Insanely old ones, as reflects the age of those machines which haven't
been touched in 10 years or so...

Baiji: 0.9.8e
Narwhal: 0.9.6b

Hang on, I removed support for OpenSSL < 0.9.8 a while ago. Narwhal shouldn't even compile with 0.9.6.

Oops, sorry - I found another copy in the Mingw installation that it
must be using (Mingw/Msys was a mess back then). I only have the
header files to go by, but it looks like the very first 0.9.8 release.

--
Dave Page
PostgreSQL Core Team
http://www.postgresql.org/

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Dave Page <dpage@postgresql.org> writes:

On Tue, Oct 18, 2016 at 12:18 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:

On 18 October 2016 12:52:14 EEST, Dave Page <dpage@postgresql.org> wrote:

Baiji: 0.9.8e
Narwhal: 0.9.6b

Hang on, I removed support for OpenSSL < 0.9.8 a while ago. Narwhal shouldn't even compile with 0.9.6.

Oops, sorry - I found another copy in the Mingw installation that it
must be using (Mingw/Msys was a mess back then). I only have the
header files to go by, but it looks like the very first 0.9.8 release.

Is it possible that there's a header-vs-executable version mismatch
contributing to the problem here? (Although you'd think we'd have
hit it before now, if so.)

regards, tom lane

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

On Tue, Oct 18, 2016 at 1:41 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Dave Page <dpage@postgresql.org> writes:

On Tue, Oct 18, 2016 at 12:18 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:

On 18 October 2016 12:52:14 EEST, Dave Page <dpage@postgresql.org> wrote:

Baiji: 0.9.8e
Narwhal: 0.9.6b

Hang on, I removed support for OpenSSL < 0.9.8 a while ago. Narwhal shouldn't even compile with 0.9.6.

Oops, sorry - I found another copy in the Mingw installation that it
must be using (Mingw/Msys was a mess back then). I only have the
header files to go by, but it looks like the very first 0.9.8 release.

Is it possible that there's a header-vs-executable version mismatch
contributing to the problem here? (Although you'd think we'd have
hit it before now, if so.)

We're not actually calling the openssl binary are we? There's an
openssl binary only in the Msys installation (the really old version),
and headers/libraries but no executable in the Mingw installation.

--
Dave Page
PostgreSQL Core Team
http://www.postgresql.org/

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

On 10/18/2016 04:32 PM, Dave Page wrote:

On Tue, Oct 18, 2016 at 1:41 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Dave Page <dpage@postgresql.org> writes:

On Tue, Oct 18, 2016 at 12:18 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:

On 18 October 2016 12:52:14 EEST, Dave Page <dpage@postgresql.org> wrote:

Baiji: 0.9.8e
Narwhal: 0.9.6b

Hang on, I removed support for OpenSSL < 0.9.8 a while ago. Narwhal shouldn't even compile with 0.9.6.

Oops, sorry - I found another copy in the Mingw installation that it
must be using (Mingw/Msys was a mess back then). I only have the
header files to go by, but it looks like the very first 0.9.8 release.

Is it possible that there's a header-vs-executable version mismatch
contributing to the problem here? (Although you'd think we'd have
hit it before now, if so.)

We're not actually calling the openssl binary are we? There's an
openssl binary only in the Msys installation (the really old version),
and headers/libraries but no executable in the Mingw installation.

Nope.

Any chance of updating the OpenSSL on that box, to see if it helps? I'd
rather not spend time investigate this further, if it turns out to be a
bug in an ancient version of OpenSSL.

- Heikki

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Dave Page <dpage@postgresql.org> writes:

On Tue, Oct 18, 2016 at 1:41 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Is it possible that there's a header-vs-executable version mismatch
contributing to the problem here? (Although you'd think we'd have
hit it before now, if so.)

We're not actually calling the openssl binary are we? There's an
openssl binary only in the Msys installation (the really old version),
and headers/libraries but no executable in the Mingw installation.

Sorry, what I meant was header vs library mismatch.

regards, tom lane

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

On Tue, Oct 18, 2016 at 2:34 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:

On 10/18/2016 04:32 PM, Dave Page wrote:

On Tue, Oct 18, 2016 at 1:41 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Dave Page <dpage@postgresql.org> writes:

On Tue, Oct 18, 2016 at 12:18 PM, Heikki Linnakangas <hlinnaka@iki.fi>
wrote:

On 18 October 2016 12:52:14 EEST, Dave Page <dpage@postgresql.org>
wrote:

Baiji: 0.9.8e
Narwhal: 0.9.6b

Hang on, I removed support for OpenSSL < 0.9.8 a while ago. Narwhal
shouldn't even compile with 0.9.6.

Oops, sorry - I found another copy in the Mingw installation that it
must be using (Mingw/Msys was a mess back then). I only have the
header files to go by, but it looks like the very first 0.9.8 release.

Is it possible that there's a header-vs-executable version mismatch
contributing to the problem here? (Although you'd think we'd have
hit it before now, if so.)

We're not actually calling the openssl binary are we? There's an
openssl binary only in the Msys installation (the really old version),
and headers/libraries but no executable in the Mingw installation.

Nope.

Any chance of updating the OpenSSL on that box, to see if it helps? I'd
rather not spend time investigate this further, if it turns out to be a bug
in an ancient version of OpenSSL.

The latest OpenSSL version won't even configure on there. I wonder if
it's finally time to retire that old machine...

--
Dave Page
PostgreSQL Core Team
http://www.postgresql.org/

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Dave Page <dpage@postgresql.org> writes:

On Tue, Oct 18, 2016 at 9:10 AM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:

Dave, what version of OpenSSL are 'narwhal' and 'baiji' using? I've tried to
reproduce this on my laptop, by compiling different versions of OpenSSL,
between 0.9.8beta1 and 0.9.8 head, but without success..

Insanely old ones, as reflects the age of those machines which haven't
been touched in 10 years or so...
Baiji: 0.9.8e
Narwhal: 0.9.6b
and for completeness:
Mastodon: 0.9.8e

Interestingly, it's now apparent that all three of those critters are
failing identically. I begin to suspect that it's somehow
Windows-specific; if it were a bug in a particular OpenSSL version,
seems like Heikki's testing described above would have reproduced it.

The only other Windows machines in the buildfarm that are building
with openssl are Andrew's bowerbird and frogmouth; they're both passing.
I wonder what openssl version is on those.

regards, tom lane

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

I wrote:

The only other Windows machines in the buildfarm that are building
with openssl are Andrew's bowerbird and frogmouth; they're both passing.
I wonder what openssl version is on those.

What's the plan for getting the buildfarm back to green in the wake
of this patch? If there isn't one, I'd like to request that the patch
be reverted until such time as some effort can be put into it. Right
now, these failures are preventing us from telling whether the recent
postgres_fdw changes will pass everywhere.

regards, tom lane

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

On 10/22/2016 12:11 PM, Tom Lane wrote:

I wrote:

The only other Windows machines in the buildfarm that are building
with openssl are Andrew's bowerbird and frogmouth; they're both passing.
I wonder what openssl version is on those.

What's the plan for getting the buildfarm back to green in the wake
of this patch? If there isn't one, I'd like to request that the patch
be reverted until such time as some effort can be put into it. Right
now, these failures are preventing us from telling whether the recent
postgres_fdw changes will pass everywhere.

bowerbird: 1.0.1d
frogmouth: 1.0.0c

They probably both should be uodated, but are both more modern than the
failing machines by the look of it.

cheers

andrew

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

On 10/22/2016 07:11 PM, Tom Lane wrote:

I wrote:

The only other Windows machines in the buildfarm that are building
with openssl are Andrew's bowerbird and frogmouth; they're both passing.
I wonder what openssl version is on those.

What's the plan for getting the buildfarm back to green in the wake
of this patch? If there isn't one, I'd like to request that the patch
be reverted until such time as some effort can be put into it. Right
now, these failures are preventing us from telling whether the recent
postgres_fdw changes will pass everywhere.

Dave, what are we going to do about this? I cannot reproduce a similar
breakage on my system with OpenSSL 0.9.8 versions, and I don't have
access to the boxes themselves, so I'm pretty much stuck in trying to
debug the issue. And I wouldn't like to spend much time on that anyway,
given that the OpenSSL versions they use are ancient, and all other
buildfarm animals are happy.

Any chance you could update the OpenSSL versions on those systems, or
failing that, remove --with-openssl?

- Heikki

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Hi

On Sun, Oct 23, 2016 at 8:22 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:

On 10/22/2016 07:11 PM, Tom Lane wrote:

I wrote:

The only other Windows machines in the buildfarm that are building
with openssl are Andrew's bowerbird and frogmouth; they're both passing.
I wonder what openssl version is on those.

What's the plan for getting the buildfarm back to green in the wake
of this patch? If there isn't one, I'd like to request that the patch
be reverted until such time as some effort can be put into it. Right
now, these failures are preventing us from telling whether the recent
postgres_fdw changes will pass everywhere.

Dave, what are we going to do about this? I cannot reproduce a similar
breakage on my system with OpenSSL 0.9.8 versions, and I don't have access
to the boxes themselves, so I'm pretty much stuck in trying to debug the
issue. And I wouldn't like to spend much time on that anyway, given that the
OpenSSL versions they use are ancient, and all other buildfarm animals are
happy.

As I mentioned earlier in the thread, I couldn't get a modern version
to configure at all on there.

Any chance you could update the OpenSSL versions on those systems, or
failing that, remove --with-openssl?

Well I could. But really, how useful is that given that it's got to be
pretty rare that people won't use OpenSSL in a build? Those machines
are so old now that I don't think there's any merit in keeping them is
there?

--
Dave Page
PostgreSQL Core Team
http://www.postgresql.org/

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

On 24 October 2016 12:33:49 EEST, Dave Page <dpage@postgresql.org> wrote:

Hi

On Sun, Oct 23, 2016 at 8:22 PM, Heikki Linnakangas <hlinnaka@iki.fi>
wrote:

Any chance you could update the OpenSSL versions on those systems, or
failing that, remove --with-openssl?

Well I could. But really, how useful is that given that it's got to be
pretty rare that people won't use OpenSSL in a build? Those machines
are so old now that I don't think there's any merit in keeping them is
there?

I'm ok with removing them from the buildfarm, if you feel that way.

- Heikki

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

On Mon, Oct 24, 2016 at 12:30 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:

On 24 October 2016 12:33:49 EEST, Dave Page <dpage@postgresql.org> wrote:

Hi

On Sun, Oct 23, 2016 at 8:22 PM, Heikki Linnakangas <hlinnaka@iki.fi>
wrote:

Any chance you could update the OpenSSL versions on those systems, or
failing that, remove --with-openssl?

Well I could. But really, how useful is that given that it's got to be
pretty rare that people won't use OpenSSL in a build? Those machines
are so old now that I don't think there's any merit in keeping them is
there?

I'm ok with removing them from the buildfarm, if you feel that way.

Andrew, Tom - any thoughts on removing them (just from head and future
branches)?

--
Dave Page
PostgreSQL Core Team
http://www.postgresql.org/

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Dave Page <dpage@postgresql.org> writes:

On Mon, Oct 24, 2016 at 12:30 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:

I'm ok with removing them from the buildfarm, if you feel that way.

Andrew, Tom - any thoughts on removing them (just from head and future
branches)?

I'd prefer the solution of disabling --with-openssl. If you just nuke
those members, we never will find out whether the recent postgres_fdw
changes pass on them. Besides, the point of the buildfarm has never
been for all members to have the same configuration options.

regards, tom lane

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers