Enquiry about TDE with PgSQL

I would like to enquire that based on the anecdotal experience of group
members, which TDE solution works best for PgSQL 17 databases.

Generally speaking, there is no "best". People use whatever vendor they
happen to already use. Your best solution is to avoid TDE altogether. If
you really need encryption at rest, have the OS do it. That works well
(transparently, even), is very battle-tested, and has minimal performance
impact. TDE, on the other hand, is a very complex and difficult thing to
add into Postgres. Currently it means you are using a forked version of
Postgres and are incurring overhead every time you read or write to disk.

The scenario I have is of a large number of tables (15-20K) and some

tables with 100M tuples each. The total database size is 4TB.

The size and number of tables does not really matter. How often you write
WAL, and how often things move in and out of shared buffers is what matters.

Cheers,
Greg

On Thu, Oct 16, 2025 at 6:05 PM Greg Sabino Mullane <htamfids@gmail.com>
wrote:

I would like to enquire that based on the anecdotal experience of group

members, which TDE solution works best for PgSQL 17 databases.

Generally speaking, there is no "best". People use whatever vendor they
happen to already use. Your best solution is to avoid TDE altogether. If
you really need encryption at rest, have the OS do it. That works well
(transparently, even), is very battle-tested, and has minimal performance
impact.

But filesystem encryption still means that validly logged-in users see the
unencrypted data. That's great for a laptop that might get stolen, or for
drives that are discarded without being wiped, but are no protection
against hackers who want to exfiltrate your data.

(Neither protect against ransomware, but that's a different problem.)

TDE, on the other hand, is a very complex and difficult thing to add
into Postgres.

TDE was added to SQL Server, with (to us, at least) minimally-noticed
overhead. Oracle has it, too, but I don't know the details.

The bottom line is that requirements for TDE are escalating, whether you
like it or not, as Yet Another Layer Of Defense against hackers
exfiltrating data, and then threatening to leak it to the public.

--
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

On Fri, 2025-10-17 at 00:49 -0400, Ron Johnson wrote:

On Thu, Oct 16, 2025 at 6:05 PM Greg Sabino Mullane <htamfids@gmail.com> wrote:

TDE, on the other hand, is a very complex and difficult thing to add into Postgres.

TDE was added to SQL Server, with (to us, at least) minimally-noticed overhead.
Oracle has it, too, but I don't know the details.

The bottom line is that requirements for TDE are escalating, whether you like it or
not, as Yet Another Layer Of Defense against hackers exfiltrating data, and then
threatening to leak it to the public.

Bruce Momjian has interesting things to say about that in
https://compiledconversations.com/6/ (unfortunately I don't remember where
exactly in this 84 minute piece).

It is a feature that users want (or need to comply with whatever they feel
they have to comply with). On the other hand, it has very limited technical
or security value, which hampers its acceptance into core.

Yours,
Laurenz Albe

On Fri, Oct 17, 2025 at 3:01 AM Laurenz Albe <laurenz.albe@cybertec.at>
wrote:

On Fri, 2025-10-17 at 00:49 -0400, Ron Johnson wrote:

On Thu, Oct 16, 2025 at 6:05 PM Greg Sabino Mullane <htamfids@gmail.com>

wrote:

TDE, on the other hand, is a very complex and difficult thing to add

into Postgres.

TDE was added to SQL Server, with (to us, at least) minimally-noticed

overhead.

Oracle has it, too, but I don't know the details.

The bottom line is that requirements for TDE are escalating, whether you

like it or

not, as Yet Another Layer Of Defense against hackers exfiltrating data,

and then

threatening to leak it to the public.

Bruce Momjian has interesting things to say about that in
https://compiledconversations.com/6/ (unfortunately I don't remember where
exactly in this 84 minute piece).

It is a feature that users want (or need to comply with whatever they feel
they have to comply with). On the other hand, it has very limited
technical
or security value, which hampers its acceptance into core.

I gave you a reason: "Yet Another Layer Of Defense against hackers
exfiltrating data". It's the same reason PgBackRest encrypts backups.

--
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

On Fri, Oct 17, 2025 at 12:49 AM Ron Johnson <ronljohnsonjr@gmail.com>
wrote:

But filesystem encryption still means that validly logged-in users see the

unencrypted data. That's great for a laptop that might get stolen, or for
drives that are discarded without being wiped, but are no protection
against hackers who want to exfiltrate your data.

I stand by my recommendation. If someone is logged in and has access to
your data directory (e.g. is root or postgres user), then they also have
the TDE key or some easy way to bypass it.

TDE was added to SQL Server, with (to us, at least) minimally-noticed

overhead. Oracle has it, too, but I don't know the details.
The bottom line is that requirements for TDE are escalating, whether you
like it or not

I'm not arguing against putting TDE in Postgres - indeed, I am all for
that. But it's a very tricky thing to do technically, with minimal benefits
other than "checking the box" of some security requirements document.

The bottom line is that requirements for TDE are escalating, whether you

like it or not, as Yet Another Layer Of Defense against hackers
exfiltrating data, and then threatening to leak it to the public.

I'd love to see a real-world example where TDE would have saved someone but
disk encryption could not.

--
Cheers,
Greg

--
Crunchy Data - https://www.crunchydata.com
Enterprise Postgres Software Products & Tech Support

Am 2025-10-17 15:12, schrieb Greg Sabino Mullane:

On Fri, Oct 17, 2025 at 12:49 AM Ron Johnson
<ronljohnsonjr@gmail.com> wrote:

But filesystem encryption still means that validly logged-in users
see the unencrypted data. That's great for a laptop that might get
stolen, or for drives that are discarded without being wiped, but
are no protection against hackers who want to exfiltrate your data.

I stand by my recommendation. If someone is logged in and has access
to your data directory (e.g. is root or postgres user), then they also
have the TDE key or some easy way to bypass it.

TDE was added to SQL Server, with (to us, at least)
minimally-noticed overhead. Oracle has it, too, but I don't know
the details.
The bottom line is that requirements for TDE are escalating, whether
you like it or not

I'm not arguing against putting TDE in Postgres - indeed, I am all for
that. But it's a very tricky thing to do technically, with minimal
benefits other than "checking the box" of some security requirements
document.

The bottom line is that requirements for TDE are escalating, whether
you like it or not, as Yet Another Layer Of Defense against hackers
exfiltrating data, and then threatening to leak it to the public.

I'd love to see a real-world example where TDE would have saved
someone but disk encryption could not.

At least with Oracle and TDE, the rman backups are encrypted, too.
So, you cannot just download the dumps from a file-share and import them
somewhere else (or if you get hold of the tapes, use those to restore
the files).
It's not really about protecting the files from a hacker but from a
person inside the organization who wants to sell the data.

TDE really only (IMHO) makes sense when you also have a HSM.

Now, I've seen a case where a customer had Oracle + Dataguard + TDE +
HSM.

The thing is, you rarely interact with the HSM normally. So, it's
getting a bit hazy in that department when you have to add a new
secret...plus, while Oracle and the HSM vendor will happily license you
the feature, they will happily point their fingers at each other in case
a problem arrives.

In this case, the HSM was wiped clear of all the keys in there, in an
attempt to add a new key.
Oracle closed the wallets and the database was offline.

Dang!

Now, you had fix figures worth of hardware doing exactly nothing.

Due to incredibly lucky circumstances, it was possible to recover the
database without data loss.

So, with TDE, you really need to be sure you actually know what you're
doing.

On Fri, Oct 17, 2025 at 09:01:52AM +0200, Laurenz Albe wrote:

On Fri, 2025-10-17 at 00:49 -0400, Ron Johnson wrote:

On Thu, Oct 16, 2025 at 6:05 PM Greg Sabino Mullane <htamfids@gmail.com> wrote:

TDE, on the other hand, is a very complex and difficult thing to add into Postgres.

TDE was added to SQL Server, with (to us, at least) minimally-noticed overhead.
Oracle has it, too, but I don't know the details.

The bottom line is that requirements for TDE are escalating, whether you like it or
not, as Yet Another Layer Of Defense against hackers exfiltrating data, and then
threatening to leak it to the public.

Bruce Momjian has interesting things to say about that in
https://compiledconversations.com/6/ (unfortunately I don't remember where
exactly in this 84 minute piece).

Here is my most recent blog about TDE:

https://momjian.us/main/blogs/pgblog/2025.html#February_22_2025

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Do not let urgent matters crowd out time for investment in the future.

As I personally believe, there is no real way around TDE in the future,
either by extensibility of the core (start with the storage manager and
move your way on from there), to make an extension possible, or by directly
adding it to the core, there are more reasons coming or are already on
their way.

With the PCI DSS v4.1 standard, one key rule to comply with is, that "If
PAN is stored, it must be rendered unreadable". Of course there are other
ways, like tokenization, hashing etc. but this regulation is pushing
towards at rest encryption in the long run, and not only disk encryption.
We can dislike it, but we are already seeing the need coming from large
industries and companies that they cannot work around this anymore, as the
auditors doing the checkboxes do not really care about "good alternatives",
as they do not even technically understand what this is about. They do
compare postgres simply against other already in use databases at these
orgs (MySQL or MongoDB), and as such, we are currently the only one that
cannot be used in such a use case, at least not without the willingness of
the auditor to make it happen.

On Thu, Oct 30, 2025 at 9:00 PM Bruce Momjian <bruce@momjian.us> wrote:

Hi Ashish,
If you are looking for "native" TDE in PostgreSQL, you won't find it in the
PostgreSQL community distribution.
If you search for "postgresql tde", you will find a list of vendors that
provide TDE functions in their own distributions of PostgreSQL, such as:

-
https://www.cybertec-postgresql.com/en/products/cybertec-postgresql-enterprise-edition-pgee/
- https://www.enterprisedb.com/products/database
- https://docs.percona.com/pg-tde/

What's important to understand is that if you migrate from PostgreSQL
community to one of these distributions and encrypt your tables using the
respective solution offered by the distribution, moving back to PostgreSQL
Community will require a logical backup (pg_dump) and restore.

Disclaimer: I work for Percona.

On Mon, Oct 13, 2025 at 12:55 PM Ashish Mukherjee <
ashish.mukherjee@gmail.com> wrote:

On Fri, Oct 31, 2025 at 03:01:48PM +0100, Kai Wagner wrote:

As I personally believe, there is no real way around TDE in the future, either
by extensibility of the core (start with the storage manager and move your way
on from there), to make an extension possible, or by directly adding it to the
core, there are more reasons coming or are already on their way.

With the PCI DSS v4.1 standard, one key rule to comply with is, that "If PAN is

Uh, I think you mean the 4.0.1 standard, which became active on January
1, 2025. I am surprised this is only being mentioned now:

https://blog.pcisecuritystandards.org/just-published-pci-dss-v4-0-1

When will PCI DSS v4.0 be retired?

As with all new versions of PCI DSS, there will be a period where both
the current and updated version will be active at the same time. PCI DSS
--> v4.0 will be retired on 31 December 2024. After that point, PCI DSS
--> v4.0.1 will be the only active version of the standard supported by PCI
SSC.

While it was active on Jan 1, it became effective on March 31, 2025:

Does PCI DSS v4.0.1 change the 31 March 2025 effective date for the new
requirements?

No. This limited revision does not impact the effective date of these
new requirements.

stored, it must be rendered unreadable". Of course there are other ways, like
tokenization, hashing etc. but this regulation is pushing towards at rest
encryption in the long run, and not only disk encryption. We can dislike it,
but we are already seeing the need coming from large industries and companies
that they cannot work around this anymore, as the auditors doing the checkboxes
do not really care about "good alternatives", as they do not even technically
understand what this is about. They do compare postgres simply against other
already in use databases at these orgs (MySQL or MongoDB), and as such, we are
currently the only one that cannot be used in such a use case, at least not
without the willingness of the auditor to make it happen.

I see the new specification that disk-level encryption is insufficient,
starting on page 93 (page 97 in the PDF URL):

https://www.middlebury.edu/sites/default/files/2025-01/PCI-DSS-v4_0_1.pdf#page=97

--> 3.5.1.2 If disk-level or partition-level encryption (rather than
file-, column-, or field-level database encryption) is used to
render PAN unreadable, it is implemented only as follows:

• On removable electronic media

OR

• If used for non-removable electronic media, PAN is also
--> rendered unreadable via another mechanism that meets Requirement
3.5.1.

...

While disk or partition encryption may still be present on these
types of devices, it cannot be the only mechanism used to protect
PAN stored on those systems. Any stored PAN must also be rendered
unreadable per Requirement 3.5.1—for example, through truncation
or a data-level encryption mechanism. Full disk encryption helps
to protect data in the event of physical loss of a disk and
therefore its use is appropriate only for removable electronic
media storage devices.

Purpose

Disk-level and partition-level encryption typically encrypts
the entire disk or partition using the same key, with all data
automatically decrypted when the system runs or when an authorized
--> user requests it. For this reason, disk-level encryption is not
--> appropriate to protect stored PAN on computers, laptops, servers,
storage arrays, or any other system that provides transparent
decryption upon user authentication.

PAN is:

https://www.middlebury.edu/sites/default/files/2025-01/PCI-DSS-v4_0_1.pdf#page=391

PAN Acronym for “primary account number.” Unique payment card
number (credit, debit, or prepaid cards, etc.) that identifies
the issuer and the cardholder account.

So it seems we have somewhat of a stand-off, with the Postgres project
questioning the value of TDE and the PCI writers doubling-down on
specifying disk-level encryption as insufficient.

The biggest possible downside of this standoff is that enterprises that
need to meet PCI compliance specifications are forced to use specialized
versions of Postgres or Postgres extensions that support TDE.

The fact that it has been seven months since PCI 4.0.1 was effective,
with little to no discussion or movement on adding TDE to community
Postgres means to me that we are unlikely to see TDE added to community
Postgres anytime soon. I have a small hope that adding compression to
the writing of temporary files will reduce the code changes needed to
encrypt temporary files, thus reducing the amount of TDE code changes
needed.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Do not let urgent matters crowd out time for investment in the future.

On 10/31/25 07:54, Bruce Momjian wrote:

On Fri, Oct 31, 2025 at 03:01:48PM +0100, Kai Wagner wrote:

With the PCI DSS v4.1 standard, one key rule to comply with is, that "If PAN is

Uh, I think you mean the 4.0.1 standard, which became active on January
1, 2025. I am surprised this is only being mentioned now:

So it seems we have somewhat of a stand-off, with the Postgres project
questioning the value of TDE and the PCI writers doubling-down on
specifying disk-level encryption as insufficient.

Yeah, what I would like to know is how many of the data breaches
actually grab directly from the storage versus getting it through the
database or other software above the storage? It seems to me social
engineering plays a bigger role in this.

--
Adrian Klaver
adrian.klaver@aklaver.com

On Fri, Oct 31, 2025 at 10:54 AM Bruce Momjian <bruce@momjian.us> wrote:

Disk-level and partition-level encryption typically encrypts
the entire disk or partition using the same key, with all data
automatically decrypted when the system runs or when an authorized
--> user requests it. For this reason, disk-level encryption is not
--> appropriate to protect stored PAN on computers, laptops, servers,
storage arrays, or any other system that provides transparent
decryption upon user authentication.

Hmm, I read this a few times but still not sure what the technical
objection is. Yes, the entire disk is encrypted with the same key, but why
is that insufficient to protect things? Anyone care to guess what they are
thinking here?

The biggest possible downside of this standoff is that enterprises that

need to meet PCI compliance specifications are forced to use specialized
versions of Postgres or Postgres extensions that support TDE.

Not always a downside for the companies selling those specialized versions
though.

Cheers,
Greg

--
Crunchy Data - https://www.crunchydata.com
Enterprise Postgres Software Products & Tech Support

On Fri, Oct 31, 2025 at 11:25:04AM -0400, Greg Sabino Mullane wrote:

On Fri, Oct 31, 2025 at 10:54 AM Bruce Momjian <bruce@momjian.us> wrote:

        Disk-level and partition-level encryption typically encrypts
        the entire disk or partition using the same key, with all data
        automatically decrypted when the system runs or when an authorized
-->     user requests it. For this reason, disk-level encryption is not
-->     appropriate to protect stored PAN on computers, laptops, servers,
        storage arrays, or any other system that provides transparent
        decryption upon user authentication.

Hmm, I read this a few times but still not sure what the technical objection
is. Yes, the entire disk is encrypted with the same key, but why is that
insufficient to protect things? Anyone care to guess what they are thinking
here?

This is more an argument for security using layers. With storage
encryption, the file system as visible is unencrypted, and backups of
that file system can be unencrypted.

Community Postgres relies on file system permissions to secure the data
directory. You can argue that once file system permissions are
bypassed, security is impossible, and agree with that, but some feel the
extra step needed to pull the Postgres encryption key from memory is a
security feature. And for backups, the Postgres encryption key might
not be in memory. Of course, forcing encrypted backups is a solution,
but an extra step.

The biggest possible downside of this standoff is that enterprises that
need to meet PCI compliance specifications are forced to use specialized
versions of Postgres or Postgres extensions that support TDE.

Not always a downside for the companies selling those specialized versions
though.

Yes, no question they are happy.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Do not let urgent matters crowd out time for investment in the future.

On Fri, Oct 31, 2025 at 11:25 AM Greg Sabino Mullane <htamfids@gmail.com>
wrote:

On Fri, Oct 31, 2025 at 10:54 AM Bruce Momjian <bruce@momjian.us> wrote:

Disk-level and partition-level encryption typically encrypts
the entire disk or partition using the same key, with all data
automatically decrypted when the system runs or when an authorized
--> user requests it. For this reason, disk-level encryption is not
--> appropriate to protect stored PAN on computers, laptops, servers,
storage arrays, or any other system that provides transparent
decryption upon user authentication.

Hmm, I read this a few times but still not sure what the technical
objection is. Yes, the entire disk is encrypted with the same key, but why
is that insufficient to protect things? Anyone care to guess what they are
thinking here?

Networking.

Who breaks into a DC and steals a rack of disks or SSDs? Very, very few
evil-doers.
Who hacks into networks and exfiltrates data over the wire? Many hackers.

--
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

On 10/31/25 08:25, Greg Sabino Mullane wrote:

On Fri, Oct 31, 2025 at 10:54 AM Bruce Momjian <bruce@momjian.us
<mailto:bruce@momjian.us>> wrote:

        Disk-level and partition-level encryption typically encrypts
        the entire disk or partition using the same key, with all data
        automatically decrypted when the system runs or when an
authorized
-->     user requests it. For this reason, disk-level encryption is not
-->     appropriate to protect stored PAN on computers, laptops,
servers,
        storage arrays, or any other system that provides transparent
        decryption upon user authentication.

Hmm, I read this a few times but still not sure what the technical
objection is. Yes, the entire disk is encrypted with the same key, but
why is that insufficient to protect things? Anyone care to guess what
they are thinking here?

My best guess, is that the more the storage encryption is fragmented by
different keys the less likely all the disk could be decrypted by a
single action. The weak link is '... other system that provides
transparent decryption upon user authentication.'. At some point the
data needs to be decrypted for end user use. That means the point of
attack is moved to the user and storage encryption does not really matter.

The biggest possible downside of this standoff is that enterprises
that need to meet PCI compliance specifications are forced to use
specialized versions of Postgres or Postgres extensions that support
TDE.

Not always a downside for the companies selling those specialized
versions though.

Cheers,
Greg

--
Crunchy Data - https://www.crunchydata.com <https://www.crunchydata.com&gt;
Enterprise Postgres Software Products & Tech Support

--
Adrian Klaver
adrian.klaver@aklaver.com

On Fri, 2025-10-31 at 08:21 -0700, Adrian Klaver wrote:

Yeah, what I would like to know is how many of the data breaches
actually grab directly from the storage versus getting it through the
database or other software above the storage? It seems to me social
engineering plays a bigger role in this.

This is not about actual security considerations, it is about checkboxes.
Consequently, rational arguments are missing the point.

Yours,
Laurenz Albe

On Fri, Oct 31, 2025 at 05:40:31PM +0100, Laurenz Albe wrote:

On Fri, 2025-10-31 at 08:21 -0700, Adrian Klaver wrote:

Yeah, what I would like to know is how many of the data breaches
actually grab directly from the storage versus getting it through the
database or other software above the storage? It seems to me social
engineering plays a bigger role in this.

This is not about actual security considerations, it is about checkboxes.
Consequently, rational arguments are missing the point.

I think the big question is that, now with the effective PCI spec
disallowing only storage-level encryption, can we, as a project,
continue to reject in-core TDE because it is a check-box item.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Do not let urgent matters crowd out time for investment in the future.

On 10/31/25 09:40, Laurenz Albe wrote:

On Fri, 2025-10-31 at 08:21 -0700, Adrian Klaver wrote:

Yeah, what I would like to know is how many of the data breaches
actually grab directly from the storage versus getting it through the
database or other software above the storage? It seems to me social
engineering plays a bigger role in this.

This is not about actual security considerations, it is about checkboxes.
Consequently, rational arguments are missing the point.

Alright, been there.

Years ago I used to drive a delivery truck for wholesale greenhouse and
one of my chores was to go to a remote greenhouse we operated and
pickup/deliver plants. There was a whole process for securing the key
that you used to open the entry door. I pointed out that the greenhouse
walls where two layers of plastic inflated by an air blower and then I
proceeded to pull out my pocket knife as an example of a 'universal'
key. The door key process stayed because it made people feel the
greenhouse contents where safe. FYI, things did get stolen though that
was because folks left them outside and the thieves did not have to
bother with a knife.

Yours,
Laurenz Albe

--
Adrian Klaver
adrian.klaver@aklaver.com

On Oct 31, 2025, at 08:21, Adrian Klaver <adrian.klaver@aklaver.com> wrote:
Yeah, what I would like to know is how many of the data breaches actually grab directly from the storage versus getting it through the database or other software above the storage?

Essentially zero.

PCI, like a lot of data security standards, are a magpie's assemblage of things that the authors have heard about all of which sound "secure" to them. However, since these particular magpies have machine guns (metaphorically) and can do serious damage to businesses, we must play along with the masquerade.