Supporting Encryption in Postgresql

On R, 2004-09-10 at 00:03, Murat Kantarcioglu wrote:

My questions are in order to support page level encryption(i,e encrypt
each page before writing back to disk and decrypt each page after we
read from disk.) which parts of the code should be changed?
Our more simply, is /src/backend/storage/smgr/md.c the only code
that does the disk access?

Since our design requires thread support (we will do some of the
decryption, before we see the data, therefore during disk access, we
need to continue decryption)

Why do you think that you need threads to do the (en/de)cryption?

Why is it not sufficient to just replace the page read/write functions
with ones supporting encryption ?

Or just use encrypted filesystem ;) <evil grin>

-------------
Hannu

Murat,

For our research project, I need to implement an encryption support for
Postgressql. At this current phase, I need to at least support page
level  encryption In other words, each page that belongs to a certain
sensitive table will be stored encrypted on the harddisk.

Are you planning on doing the decryption on the back-end, or on the client?
It certainly seems to me that doing it on the client would make more sense;
if the data is decrypted on the back-end, then you will still need the
overhead of an SSL connection.

In any case, I'm glad that you're looking into this; encryption-on-disk is one
of those "missing features" that we might never have gotten around to as a
project ...

--
--Josh

Josh Berkus
Aglio Database Solutions
San Francisco

Given that the client does not write pages to the disk, this would be
back-end encryption. Just out of curiosity, what threat model does this
sort of encryption protect against? Surely any attacker who can read
the files off the disk can also get the password used to encrypt them.
Or would this be provided by the client and kept in RAM only?

Paul Tillotson

Paul Tillotson <pntil@shentel.net> writes:

Given that the client does not write pages to the disk, this would be
back-end encryption. Just out of curiosity, what threat model does
this sort of encryption protect against? Surely any attacker who can
read the files off the disk can also get the password used to encrypt
them. Or would this be provided by the client and kept in RAM only?

If I have root- or postgres-level access to the machine, I can snarf
the encryption key out of RAM even if it's never written to disk.

I don't see what this (backend page-level encryption) would buy you
over just using an encrypted partition, which is already available on
most OSs...

-Doug
--
Let us cross over the river, and rest under the shade of the trees.
--T. J. Jackson, 1863

Murat Kantarcioglu wrote:

For our research project, I need to implement an encryption support for
Postgressql. At this current phase, I need to at least support page
level encryption In other words, each page that belongs to a certain
sensitive table will be stored encrypted on the harddisk.
Since we are trying to have a new design that can start the decryption
before even we see the data, I need to have some kind of thread support.

I have to say that this is becoming an important problem in the European
market. In Italy for example the law impose that if you store personal
data about your customers then this information shall be stored in an
encrypted form. The bast way I found to accomplish this is using an
encrypted file system. Google for "cryptoloop" or if you are brave enough
look for StegFS.

Regards
Gaetano Mendola

Centuries ago, Nostradamus foresaw when Murat Kantarcioglu <kanmurat@cs.purdue.edu> would write:

For our research project, I need to implement an encryption support
for Postgressql. At this current phase, I need to at least support
page level encryption In other words, each page that belongs to a
certain sensitive table will be stored encrypted on the harddisk.
Since we are trying to have a new design that can start the decryption
before even we see the data, I need to have some kind of thread
support.

My questions are in order to support page level encryption(i,e
encrypt each page before writing back to disk and decrypt each page
after we read from disk.) which parts of the code should be changed?
Our more simply, is /src/backend/storage/smgr/md.c the only code
that does the disk access?

Since our design requires thread support (we will do some of the
decryption, before we see the data, therefore during disk access, we
need to continue decryption) Can you suggest me a good thread lib
you think will work fine with postgresql ?

Thanks for your help.

You'd better step back to your threat model, and figure out what
encryption will actually get you.

I don't see any reason to think that you can actually gain _anything_
from page level encryption.

If you think you do, then you ought to either:

a) Show how you gain it using something like Linux's capability
to use encrypted loopback filesystems, which would not require
touching PostgreSQL at all, or

b) Demonstrate what are the attacks that page level encryption
would protect against, and how.

The problem with any such mechanisms is essentially the same, namely
that the encryption key has got to sit in memory in either the
database server process or in the kernel's memory.

As such, the key is vulnerable to anyone with root access that can
access /proc or its equivalent and get at process memory.

The only way for the encryption key NOT to be vulnerable in this
fashion is if the encryption key is communicated neither to the
database server nor to the OS kernel.

I'd suggest you avail yourself of the book _Translucent Databases_ by
Peter Weyner; it involves a model where the database engine is not
entrusted with cryptography at all.

Instead, cryptography is all done within the client.
--
(reverse (concatenate 'string "gro.gultn" "@" "enworbbc"))
http://www.ntlug.org/~cbbrowne/unix.html
In case you weren't aware, "ad homineum" is not latin for "the user of
this technique is a fine debater." -- Thomas F. Burdick

Thanks for the comments.

This piece will be a part of a bigger design and the problems
mentioned are very real. In the future, our goal is to design a database
system where the processing is done in a "secure coprocessor"(i.e no one
will be able to see what is inside) and
the small code inside the co-processor is verified using formal methods.
Therefore, all the problems you have mentioned will not be a issue for
our general case. We are even considering what could be revealed just
watching the disk access. Initial technical report can be found at
( http://www.cs.purdue.edu/homes/kanmurat/technical.ps).

Can you suggest me a solution to how to do
this on Postgresql backend?

I am asssuming that somewhere in the code, you are calling a function like
getPage(Page_id)
to retrieve the page(I am trying to change backend)

All I need to do is (I am not sure yet)
change such code with (ofcourse, I need to change writePage part)
getPage(Page_id)
{
ctr=Hash_Table(Page_id) //return somevalue needed for deccryption
Thread_Read(Page_id) // will call the original read code
Thread_Encryption.start(ctr, length);
when both threads are done finish the encryption
}

On Fri, Sep 10, 2004 at 11:52:26AM -0500, Murat Kantarcioglu wrote:

Can you suggest me a solution to how to do
this on Postgresql backend?

I am asssuming that somewhere in the code, you are calling a function like
getPage(Page_id)
to retrieve the page(I am trying to change backend)

Probably the code you want to modify is in src/backend/storage/smgr.
Maybe you want to add a different storage manager (they are pluggable,
sort of).

getPage(Page_id)
{
ctr=Hash_Table(Page_id) //return somevalue needed for deccryption
Thread_Read(Page_id) // will call the original read code
Thread_Encryption.start(ctr, length);
when both threads are done finish the encryption
}

I think it would need extensive, painful and unwelcome modifications to
use threads to do the job. You could just as well do it sequentially,
like in

encryptedPage = getPage(page_id);
clearPage = unencrypt(encryptedPage);
return clearPage;

And the reverse for storage. This may only need modifications to
mdread() and mdwrite() ... unless your encryption scheme returns a
different length than the original.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"Estoy de acuerdo contigo en que la verdad absoluta no existe...
El problema es que la mentira s� existe y tu est�s mintiendo" (G. Lama)

Our basic claim is to be able to do most of the encryption while we are
reading the page. That is the reason I need the threads. Any suggestion
about the threads are welcome. Thanks.
Murat

Alvaro Herrera wrote:

Murat Kantarcioglu <kanmurat@cs.purdue.edu> writes:

This piece will be a part of a bigger design and the problems
mentioned are very real. In the future, our goal is to design a database
system where the processing is done in a "secure coprocessor"(i.e no one
will be able to see what is inside) and
the small code inside the co-processor is verified using formal methods.

[ raised eyebrow... ] You think a SQL database is small code you can
verify using formal methods? I don't really see how you can expect that
the decrypted data can be held entirely within a small secured area and
still get any useful work done.

regards, tom lane

It is really hard to describe all the project in few e-mails.

Obviously, we will not try to run entire database software in that
secure hardware. Also memory limitations are not important.
For example, please check the research on "oblivious RAM" to see even
small memory on such hardware can be leveraged to execute programs with
huge memory requirements. Also please check the "Practical Private
Information Retrieval" work to see how such hardware is used for solving
PIR problem.

Anyway, I totaly understand your reservations but we are trying to have
a solution to answer your concerns and much more. Thanks for the interest.
Murat

Tom Lane wrote: