Value of Transparent Data Encryption (TDE)

On Mon, Sep 30, 2019 at 05:40:52PM -0400, Bruce Momjian wrote:

For plan for full-cluster Transparent Data Encryption (TDE) is here:

https://wiki.postgresql.org/wiki/Transparent_Data_Encryption#TODO_for_Full-Cluster_Encryption

The values it has, I think, are:

* encrypts data for anyone with read-access to the file system (but not
memory)

* I think write access would allow access to the encryption keys
by modifying postgresql.conf or other files

* This is particularly useful if the storage is remote

* encrypts non-logical/non-pg_dump-like backups

* fulfills several security compliance requirements

* encrypts storage

Maybe. I think this is approaching the problem from the wrong angle.
Encryption is more a means of achieving something. OK, for compliance
purposes it's useful to be able to tick "encryption" checkbox. But other
than that, people really care about threat models and how encryption
improves them (or does not).

So I think it'd be valuable to improve the "thread models" section on
that wiki page, with more detailed cases. We need to explain what
capabilities the attacker has (can he read files?can he interact with
the database? can he read memory? ..) and then explain how that works
with encrypted cluster.

* perhaps easier to implement than file system encryption

Not sure. IMO filesystem encryption is fairly simple to use, to the
extent that it's hard to beat. The problem is usually people can't use
it for various reasons - lack of support on their OS, no access to the
block device, problems with obtaining the privileges etc.

Having it built into the database menas you can sidestep most of those
issue (e.g. you can deploy it as a DBA, on arbitrary OS, ...).

Plus it allows features you can't easily achieve with fs encryption,
because the filesystem only sees opaque data files. So having keys per
database/user/... is easier from within the database.

regards

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Tue, Oct 1, 2019 at 03:43:05PM +0200, Tomas Vondra wrote:

On Mon, Sep 30, 2019 at 05:40:52PM -0400, Bruce Momjian wrote:
Maybe. I think this is approaching the problem from the wrong angle.
Encryption is more a means of achieving something. OK, for compliance
purposes it's useful to be able to tick "encryption" checkbox. But other
than that, people really care about threat models and how encryption
improves them (or does not).

Yes, that is what I am trying to do with this email thread.

So I think it'd be valuable to improve the "thread models" section on
that wiki page, with more detailed cases. We need to explain what
capabilities the attacker has (can he read files?can he interact with
the database? can he read memory? ..) and then explain how that works
with encrypted cluster.

* perhaps easier to implement than file system encryption

Not sure. IMO filesystem encryption is fairly simple to use, to the
extent that it's hard to beat. The problem is usually people can't use
it for various reasons - lack of support on their OS, no access to the
block device, problems with obtaining the privileges etc.

Right, that's the "perhaps easier" part of my text above.

Having it built into the database menas you can sidestep most of those
issue (e.g. you can deploy it as a DBA, on arbitrary OS, ...).

Plus it allows features you can't easily achieve with fs encryption,
because the filesystem only sees opaque data files. So having keys per
database/user/... is easier from within the database.

Yes, but we will not be doing that for the first release because of the
complexity related to handling this in WAL and requiring crash recovery
to be able to unlock all the keys for replay. I personally think that
database/user/... keys are best done at the SQL level, with proper
locking. pgcryptokey (http://momjian.us/download/pgcryptokey/) is an
example of that.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

On Tue, Oct 1, 2019 at 11:54:26AM -0400, Bruce Momjian wrote:

On Tue, Oct 1, 2019 at 03:43:05PM +0200, Tomas Vondra wrote:

Plus it allows features you can't easily achieve with fs encryption,
because the filesystem only sees opaque data files. So having keys per
database/user/... is easier from within the database.

Yes, but we will not be doing that for the first release because of the
complexity related to handling this in WAL and requiring crash recovery
to be able to unlock all the keys for replay. I personally think that
database/user/... keys are best done at the SQL level, with proper
locking. pgcryptokey (http://momjian.us/download/pgcryptokey/) is an
example of that.

Just to give more detail. Initially, there was a desire to store keys
in only one place, either in the file system or in database tables.
However, it became clear that the needs of booting the server and crash
recovery required file system keys, and per-user/db keys were best done
at the SQL level, so that indexing can be used, and logical dumps
contain the locked keys. SQL-level storage allows databases to be
completely independent of other databases in terms of key storage and
usage.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

On Tue, Oct 1, 2019 at 12:19 PM Bruce Momjian <bruce@momjian.us> wrote:

Just to give more detail. Initially, there was a desire to store keys
in only one place, either in the file system or in database tables.
However, it became clear that the needs of booting the server and crash
recovery required file system keys, and per-user/db keys were best done
at the SQL level, so that indexing can be used, and logical dumps
contain the locked keys. SQL-level storage allows databases to be
completely independent of other databases in terms of key storage and
usage.

Wait, we're going to store the encryption keys with the database? It
seems like you're debating whether to store your front door keys under
the doormat or in a fake rock by the side of the path, when what you
really ought to be doing is keeping them physically separated from the
house, like in your pocket or your purse.

It seems to me that the right design is that there's a configurable
mechanism for PostgreSQL to request keys from someplace outside the
database, and that other place is responsible for storing the keys
securely and not losing them. Probably, it's a key-server of some kind
running on another machine, but if you really want you can do
something insecure instead, like getting them from the local
filesystem.

I admit I haven't been following the threads on this topic, but this
just seems like a really strange idea.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Greetings,

* Robert Haas (robertmhaas@gmail.com) wrote:

On Tue, Oct 1, 2019 at 12:19 PM Bruce Momjian <bruce@momjian.us> wrote:

Just to give more detail. Initially, there was a desire to store keys
in only one place, either in the file system or in database tables.
However, it became clear that the needs of booting the server and crash
recovery required file system keys, and per-user/db keys were best done
at the SQL level, so that indexing can be used, and logical dumps
contain the locked keys. SQL-level storage allows databases to be
completely independent of other databases in terms of key storage and
usage.

Wait, we're going to store the encryption keys with the database? It
seems like you're debating whether to store your front door keys under
the doormat or in a fake rock by the side of the path, when what you
really ought to be doing is keeping them physically separated from the
house, like in your pocket or your purse.

This isn't news and shouldn't be shocking- databases which support TDE
all have a vaulting system for managing the keys and, yes, that's stored
with the database.

It seems to me that the right design is that there's a configurable
mechanism for PostgreSQL to request keys from someplace outside the
database, and that other place is responsible for storing the keys
securely and not losing them. Probably, it's a key-server of some kind
running on another machine, but if you really want you can do
something insecure instead, like getting them from the local
filesystem.

I support the option to have an external vault that's used, but I don't
believe that should be a requirement and I don't think that removes the
need to have a vaulting system of our own, so we can have a stand-alone
TDE solution.

I admit I haven't been following the threads on this topic, but this
just seems like a really strange idea.

It's not new and it's how TDE works in all of the other database systems
which support it.

Thanks,

Stephen

On Thu, Oct 03, 2019 at 10:43:21AM -0400, Stephen Frost wrote:

Greetings,

* Robert Haas (robertmhaas@gmail.com) wrote:

On Tue, Oct 1, 2019 at 12:19 PM Bruce Momjian <bruce@momjian.us> wrote:

Just to give more detail. Initially, there was a desire to store keys
in only one place, either in the file system or in database tables.
However, it became clear that the needs of booting the server and crash
recovery required file system keys, and per-user/db keys were best done
at the SQL level, so that indexing can be used, and logical dumps
contain the locked keys. SQL-level storage allows databases to be
completely independent of other databases in terms of key storage and
usage.

Wait, we're going to store the encryption keys with the database? It
seems like you're debating whether to store your front door keys under
the doormat or in a fake rock by the side of the path, when what you
really ought to be doing is keeping them physically separated from the
house, like in your pocket or your purse.

This isn't news and shouldn't be shocking- databases which support TDE
all have a vaulting system for managing the keys and, yes, that's stored
with the database.

Right. The important bit here is that the vault is encrypted, and has to
be unlocked using a passphrase (or something like that) when starting
the database. So it's not really as silly as a key under the doormat.

It seems to me that the right design is that there's a configurable
mechanism for PostgreSQL to request keys from someplace outside the
database, and that other place is responsible for storing the keys
securely and not losing them. Probably, it's a key-server of some kind
running on another machine, but if you really want you can do
something insecure instead, like getting them from the local
filesystem.

I support the option to have an external vault that's used, but I don't
believe that should be a requirement and I don't think that removes the
need to have a vaulting system of our own, so we can have a stand-alone
TDE solution.

Right. If anything, we need a local vault that we could use for testing.
In other cases it might be a simple wrapper for a vault/keyring provided
by the operating system (if it's good enough for gpg keys ...).

I admit I haven't been following the threads on this topic, but this
just seems like a really strange idea.

It's not new and it's how TDE works in all of the other database systems
which support it.

Yep.

regards

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Thu, Oct 03, 2019 at 10:26:15AM -0400, Robert Haas wrote:

On Tue, Oct 1, 2019 at 12:19 PM Bruce Momjian <bruce@momjian.us> wrote:

Just to give more detail. Initially, there was a desire to store
keys in only one place, either in the file system or in database
tables. However, it became clear that the needs of booting the
server and crash recovery required file system keys, and
per-user/db keys were best done at the SQL level, so that indexing
can be used, and logical dumps contain the locked keys. SQL-level
storage allows databases to be completely independent of other
databases in terms of key storage and usage.

Wait, we're going to store the encryption keys with the database?

Encryption keys are fine there so long as decryption keys are
separate.

Best,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate