SSL tests failing with "ee key too small" error on Debian SID

Started by Michael Paquierover 7 years ago12 messages
#1Michael Paquier
michael@paquier.xyz

Hi all,

On a rather freshly-updated Debian SID server, I am able to see failures
for the SSL TAP tests:
2018-09-17 22:00:27.389 JST [13072] LOG: database system is shut down
2018-09-17 22:00:27.506 JST [13082] FATAL: could not load server
certificate file "server-cn-only.crt": ee key too small
2018-09-17 22:00:27.506 JST [13082] LOG: database system is shut down
2018-09-17 22:00:27.720 JST [13084] FATAL: could not load server
certificate file "server-cn-only.crt": ee key too small

Wouldn't it be better to rework the rules used to generate the different
certificates and reissue them in the tree? It seems to me that this is
just waiting to fail in other platforms as well..

Thanks,
--
Michael

#2Kyotaro HORIGUCHI
horiguchi.kyotaro@lab.ntt.co.jp
In reply to: Michael Paquier (#1)
Re: SSL tests failing with "ee key too small" error on Debian SID

Hello.

At Mon, 17 Sep 2018 22:13:40 +0900, Michael Paquier <michael@paquier.xyz> wrote in <20180917131340.GE31460@paquier.xyz>

Hi all,

On a rather freshly-updated Debian SID server, I am able to see failures
for the SSL TAP tests:
2018-09-17 22:00:27.389 JST [13072] LOG: database system is shut down
2018-09-17 22:00:27.506 JST [13082] FATAL: could not load server
certificate file "server-cn-only.crt": ee key too small
2018-09-17 22:00:27.506 JST [13082] LOG: database system is shut down
2018-09-17 22:00:27.720 JST [13084] FATAL: could not load server
certificate file "server-cn-only.crt": ee key too small

Wouldn't it be better to rework the rules used to generate the different
certificates and reissue them in the tree? It seems to me that this is
just waiting to fail in other platforms as well..

I agree that we could get into the same trouble sooner or later.

Do you mean that cert/key files are generated on-the-fly while
running 'make check'? It sounds reasonable as long as just
replaceing existing files with those with longer (2048bits?) keys
doesn't work for all supported platforms.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#3Michael Paquier
michael@paquier.xyz
In reply to: Kyotaro HORIGUCHI (#2)
Re: SSL tests failing with "ee key too small" error on Debian SID

On Tue, Sep 25, 2018 at 12:48:57PM +0900, Kyotaro HORIGUCHI wrote:

Do you mean that cert/key files are generated on-the-fly while
running 'make check'? It sounds reasonable as long as just
replaceing existing files with those with longer (2048bits?) keys
doesn't work for all supported platforms.

The files are present by default in the tree, but can be regenerated
easily by using the makefile rule "sslfiles". From what I can see, this
is caused by OpenSSL 1.1.1 which Debian SID has visibly upgraded to
recently. That's the version I have on my system. I have not dug much
into the Makefile to see if things could get done right and change the
openssl commands though..
--
Michael

#4Kyotaro HORIGUCHI
horiguchi.kyotaro@lab.ntt.co.jp
In reply to: Michael Paquier (#3)
4 attachment(s)
Re: SSL tests failing with "ee key too small" error on Debian SID

At Tue, 25 Sep 2018 14:26:42 +0900, Michael Paquier <michael@paquier.xyz> wrote in <20180925052642.GJ1354@paquier.xyz>

On Tue, Sep 25, 2018 at 12:48:57PM +0900, Kyotaro HORIGUCHI wrote:

Do you mean that cert/key files are generated on-the-fly while
running 'make check'? It sounds reasonable as long as just
replaceing existing files with those with longer (2048bits?) keys
doesn't work for all supported platforms.

The files are present by default in the tree, but can be regenerated
easily by using the makefile rule "sslfiles". From what I can see, this
is caused by OpenSSL 1.1.1 which Debian SID has visibly upgraded to
recently. That's the version I have on my system. I have not dug much
into the Makefile to see if things could get done right and change the
openssl commands though..

# I have no experience in Debian..

In Debian /etc/ssl/openssl.cnf has been changed to
"CiperString=DEFAULT@SECLEVEL=2", which implies that "RSA and DHE
keys need to be at least 2048 bit long" according to the
following page.

https://wiki.debian.org/ContinuousIntegration/TriagingTips/openssl-1.1.1

It seems to be Debian's special feature and I suppose
(differently from the previous mail..) it won't happen on other
platforms.

Instead, I managed to cause "ee key too smal" by setting
ssl_ciphers in postgresql.conf as the follows with openssl
1.1.1. With the first attached it happens during
001_ssltests_master.

ssl_ciphers = 'HIGH:MEDIUM:+3DES:!aNULL@SECLEVEL=2' # allowed SSL ciphers

The attached second patch just changes key size to 2048 bits and
"ee key too small" are eliminated in 001_ssltests_master, but
instead I got "ca md too weak" error. This is eliminated by using
sha256 instead of sha1 in cas.config. (third attached)

By the way I got (with both 1.0.2k and 1.1.1) a "tlsv1 alert
unknown ca" error from 002_scram.pl. It is fixed for me by the
forth attached, but I'm not sure why we haven't have such a
complain. (It happens only for me?)

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

Attachments:

cause_ee_key_too_small.patchtext/x-patch; charset=us-asciiDownload
diff --git a/src/test/ssl/t/001_ssltests.pl b/src/test/ssl/t/001_ssltests.pl
index 2b875a3c95..6d267f994e 100644
--- a/src/test/ssl/t/001_ssltests.pl
+++ b/src/test/ssl/t/001_ssltests.pl
@@ -43,6 +43,10 @@ chmod 0644, "ssl/client_wrongperms_tmp.key";
 note "setting up data directory";
 my $node = get_new_node('master');
 $node->init;
+#### ##### restrict cipher suites
+$node->append_conf("postgresql.conf", <<'EOF');
+ssl_ciphers = 'HIGH:MEDIUM:+3DES:!aNULL@SECLEVEL=2' # allowed SSL ciphers
+EOF
 
 # PGHOST is enforced here to set up the node, subsequent connections
 # will use a dedicated connection string.
use_rsa_2048b_keys.patchtext/x-patch; charset=us-asciiDownload
diff --git a/src/test/ssl/Makefile b/src/test/ssl/Makefile
index 97389c90f8..4b621e18b6 100644
--- a/src/test/ssl/Makefile
+++ b/src/test/ssl/Makefile
@@ -39,7 +39,7 @@ ssl/new_certs_dir:
 
 # Rule for creating private/public key pairs.
 ssl/%.key:
-	openssl genrsa -out $@ 1024
+	openssl genrsa -out $@ 2048
 	chmod 0600 $@
 
 # Root CA certificate
change_md_to_sha256.patchtext/x-patch; charset=us-asciiDownload
diff --git a/src/test/ssl/cas.config b/src/test/ssl/cas.config
index 013cebae16..8c0ef6d82b 100644
--- a/src/test/ssl/cas.config
+++ b/src/test/ssl/cas.config
@@ -13,7 +13,7 @@ basicConstraints = CA:true
 dir = ./ssl/
 database = ./ssl/root_ca-certindex
 serial = ./ssl/root_ca.srl
-default_md = sha1
+default_md = sha256
 default_days= 10000
 default_crl_days= 10000
 certificate = ./ssl/root_ca.crt
@@ -26,7 +26,7 @@ email_in_dn				= no
 [ server_ca ]
 dir = ./ssl/
 database = ./ssl/server_ca-certindex
-default_md = sha1
+default_md = sha256
 default_days= 10000
 default_crl_days= 10000
 certificate = ./ssl/server_ca.crt
@@ -42,7 +42,7 @@ crl = ./ssl/server.crl
 [ client_ca ]
 dir = ./ssl/
 database = ./ssl/client_ca-certindex
-default_md = sha1
+default_md = sha256
 default_days= 10000
 default_crl_days= 10000
 certificate = ./ssl/client_ca.crt
set_sslrootcert_in_scram_test.patchtext/x-patch; charset=us-asciiDownload
diff --git a/src/test/ssl/t/002_scram.pl b/src/test/ssl/t/002_scram.pl
index b460a7fa8a..147f51783d 100644
--- a/src/test/ssl/t/002_scram.pl
+++ b/src/test/ssl/t/002_scram.pl
@@ -39,7 +39,7 @@ configure_test_server_for_ssl($node, $SERVERHOSTADDR, "scram-sha-256",
 switch_server_cert($node, 'server-cn-only');
 $ENV{PGPASSWORD} = "pass";
 $common_connstr =
-  "user=ssltestuser dbname=trustdb sslmode=require hostaddr=$SERVERHOSTADDR";
+  "user=ssltestuser dbname=trustdb sslmode=require sslrootcert=ssl/root+server_ca.crt hostaddr=$SERVERHOSTADDR";
 
 # Default settings
 test_connect_ok($common_connstr, '',
#5Michael Paquier
michael@paquier.xyz
In reply to: Kyotaro HORIGUCHI (#4)
Re: SSL tests failing with "ee key too small" error on Debian SID

On Mon, Oct 01, 2018 at 09:18:01PM +0900, Kyotaro HORIGUCHI wrote:

In Debian /etc/ssl/openssl.cnf has been changed to
"CiperString=DEFAULT@SECLEVEL=2", which implies that "RSA and DHE
keys need to be at least 2048 bit long" according to the
following page.

https://wiki.debian.org/ContinuousIntegration/TriagingTips/openssl-1.1.1

It seems to be Debian's special feature and I suppose
(differently from the previous mail..) it won't happen on other
platforms.

Ah... Thanks for the information. I have missed that bit. Likely
other platforms would not bother much about that.

The attached second patch just changes key size to 2048 bits and
"ee key too small" are eliminated in 001_ssltests_master, but
instead I got "ca md too weak" error. This is eliminated by using
sha256 instead of sha1 in cas.config. (third attached)

I find your suggestion quite tempting at the end instead of having to
tweak the global system's configuration. That should normally work with
any configuration. This would require regenerating the certs in the
tree. Any thoughts from others?

By the way I got (with both 1.0.2k and 1.1.1) a "tlsv1 alert
unknown ca" error from 002_scram.pl. It is fixed for me by the
forth attached, but I'm not sure why we haven't have such a
complain. (It happens only for me?)

I am actually seeing that for 001_ssltests, but that's expected as there
are some cases with revoked certs, but not for 002_scram.
--
Michael

#6Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Michael Paquier (#5)
Re: SSL tests failing with "ee key too small" error on Debian SID

On Wed, Oct 3, 2018 at 1:32 PM Michael Paquier <michael@paquier.xyz> wrote:

On Mon, Oct 01, 2018 at 09:18:01PM +0900, Kyotaro HORIGUCHI wrote:

The attached second patch just changes key size to 2048 bits and
"ee key too small" are eliminated in 001_ssltests_master, but
instead I got "ca md too weak" error. This is eliminated by using
sha256 instead of sha1 in cas.config. (third attached)

I find your suggestion quite tempting at the end instead of having to
tweak the global system's configuration. That should normally work with
any configuration. This would require regenerating the certs in the
tree. Any thoughts from others?

I don't really have opinion here, but I wanted to point out that
src/test/ldap/t/001_auth.pl creates new certs on the fly, which is a
bit inconsistent with the SSL test's approach of certs-in-the-tree.
Which is better?

--
Thomas Munro
http://www.enterprisedb.com

#7Michael Paquier
michael@paquier.xyz
In reply to: Thomas Munro (#6)
Re: SSL tests failing with "ee key too small" error on Debian SID

On Mon, Nov 26, 2018 at 01:17:24PM +1300, Thomas Munro wrote:

On Wed, Oct 3, 2018 at 1:32 PM Michael Paquier <michael@paquier.xyz> wrote:

I find your suggestion quite tempting at the end instead of having to
tweak the global system's configuration. That should normally work with
any configuration. This would require regenerating the certs in the
tree. Any thoughts from others?

I don't really have opinion here, but I wanted to point out that
src/test/ldap/t/001_auth.pl creates new certs on the fly, which is a
bit inconsistent with the SSL test's approach of certs-in-the-tree.
Which is better?

When going up to 2k, it takes longer to generate the keys than to run
the tests, so keeping them in the tree looks like a pretty good gain to
me.
--
Michael

#8Peter Eisentraut
peter.eisentraut@2ndquadrant.com
In reply to: Michael Paquier (#7)
Re: SSL tests failing with "ee key too small" error on Debian SID

On 26/11/2018 01:35, Michael Paquier wrote:

When going up to 2k, it takes longer to generate the keys than to run
the tests, so keeping them in the tree looks like a pretty good gain to
me.

Another concern might be that repeatedly generating certificates might
drain entropy unnecessarily.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#9Peter Eisentraut
peter.eisentraut@2ndquadrant.com
In reply to: Kyotaro HORIGUCHI (#4)
Re: SSL tests failing with "ee key too small" error on Debian SID

On 01/10/2018 14:18, Kyotaro HORIGUCHI wrote:

The attached second patch just changes key size to 2048 bits and
"ee key too small" are eliminated in 001_ssltests_master, but
instead I got "ca md too weak" error. This is eliminated by using
sha256 instead of sha1 in cas.config. (third attached)

I have applied these configuration changes and created a new set of test
files with them.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#10Peter Eisentraut
peter.eisentraut@2ndquadrant.com
In reply to: Kyotaro HORIGUCHI (#4)
Re: SSL tests failing with "ee key too small" error on Debian SID

On 01/10/2018 14:18, Kyotaro HORIGUCHI wrote:

By the way I got (with both 1.0.2k and 1.1.1) a "tlsv1 alert
unknown ca" error from 002_scram.pl. It is fixed for me by the
forth attached, but I'm not sure why we haven't have such a
complain. (It happens only for me?)

I haven't seen it. Do the tests print that out or does it appear in the
logs? Which test complains?

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#11Tom Lane
tgl@sss.pgh.pa.us
In reply to: Peter Eisentraut (#9)
Re: SSL tests failing with "ee key too small" error on Debian SID

Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:

On 01/10/2018 14:18, Kyotaro HORIGUCHI wrote:

The attached second patch just changes key size to 2048 bits and
"ee key too small" are eliminated in 001_ssltests_master, but
instead I got "ca md too weak" error. This is eliminated by using
sha256 instead of sha1 in cas.config. (third attached)

I have applied these configuration changes and created a new set of test
files with them.

Buildfarm critters aren't going to be happy unless you back-patch that.

regards, tom lane

#12Michael Paquier
michael@paquier.xyz
In reply to: Tom Lane (#11)
Re: SSL tests failing with "ee key too small" error on Debian SID

On Tue, Nov 27, 2018 at 09:37:17AM -0500, Tom Lane wrote:

Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:

On 01/10/2018 14:18, Kyotaro HORIGUCHI wrote:

The attached second patch just changes key size to 2048 bits and
"ee key too small" are eliminated in 001_ssltests_master, but
instead I got "ca md too weak" error. This is eliminated by using
sha256 instead of sha1 in cas.config. (third attached)

I have applied these configuration changes and created a new set of test
files with them.

Buildfarm critters aren't going to be happy unless you back-patch that.

Thanks for applying that, Peter.
--
Michael