PostgreSQL Trusted Startup

On Mon, Dec 20, 2010 at 12:12 PM, Kenneth Buckler
<kenneth.buckler@gmail.com> wrote:

Hello,

I am investigating security requirements for configuring a PostgreSQL
database on a Linux system.
One of the security requirements our organization would like to implement is
"trusted startup", in that PostgreSQL would verify the authenticity of the
binaries and configuration files before making the database available to
users.  This would enable the database to detect if the system has possibly
been compromised.

But, if the script is run on the same machine as postgresql is on, the
scripts that check for changes could be compromised as well and then
you'd never know.

Since this is a Linux system, I could keep a list of known good MD5
checksums and compare the checksums prior to startup by editing the init
script.  The list would of course need to be updated any time I make a
configuration change or apply a patch.
Is there an alternative method of implementing such a requirement?  Possibly
one already incorporated into PostgreSQL?

pgsql doesn't do any of that, but I'm sure you can roll your own so to
speak. I would tend to write some kind of nagios plugin that could be
called remotely that would notify you whenever it changes so you would
know as soon as a change occurred rather than later when trying to
restart the database during a midday outage while the boss screams
"get the system back up now! We're losing money!"

Generally, if the db's been compromised, someone's already gotten to
an app server or two, and might be sniffing traffic anyway, so it's
likely a lost cause by then.

On 12/20/10 11:12 AM, Kenneth Buckler wrote:

Hello,

I am investigating security requirements for configuring a PostgreSQL
database on a Linux system.
One of the security requirements our organization would like to
implement is "trusted startup", in that PostgreSQL would verify the
authenticity of the binaries and configuration files before making the
database available to users. This would enable the database to detect
if the system has possibly been compromised.
Since this is a Linux system, I could keep a list of known good MD5
checksums and compare the checksums prior to startup by editing the
init script. The list would of course need to be updated any time I
make a configuration change or apply a patch.
Is there an alternative method of implementing such a requirement?
Possibly one already incorporated into PostgreSQL?

I would look into selinux. lock it down with this, and it will be much
harder to compromise.

On Mon, Dec 20, 2010 at 1:43 PM, John R Pierce <pierce@hogranch.com> wrote:

I would look into selinux.   lock it down with this, and it will be much
harder to compromise.

I agree. By the time you've got compromised binaries / config files
on the system, you've already lost.

On Mon, Dec 20, 2010 at 3:31 PM, Scott Marlowe <scott.marlowe@gmail.com> wrote:

But, if the script is run on the same machine as postgresql is on, the
scripts that check for changes could be compromised as well and then
you'd never know.

I agree, if the system has been compromised, nothing will prevent the
scripts from being compromised.
Hence why I am looking for alternatives. I consider the md5 script
approach a poor approach, but it does meet the letter of the
requirements set forth by the organization.

Is there an alternative method of implementing such a requirement?  Possibly
one already incorporated into PostgreSQL?

pgsql doesn't do any of that, but I'm sure you can roll your own so to
speak.  I would tend to write some kind of nagios plugin that could be
called remotely that would notify you whenever it changes so you would
know as soon as a change occurred rather than later when trying to
restart the database during a midday outage while the boss screams
"get the system back up now! We're losing money!"

Thanks for clarifying that for me. Part of the requirement I'm
working with requires "vendor documentation" stating if such a feature
exists. Since there is no vendor documentation, they'll have to
settle for my own documentation, backed up with a mailing list post.

Writing my own plugin/module hasn't been ruled out. I wanted to make
sure that I'm not re-inventing the wheel.

In any approach to this, I will be including an override which will
allow PostgreSQL to start despite failing the "trusted files" check.

Generally, if the db's been compromised, someone's already gotten to
an app server or two, and might be sniffing traffic anyway, so it's
likely a lost cause by then.

Agreed. Unfortunately, I've been given specific requirements and I am
obligated to fulfill those requirements, even if I don't agree those
requirements are necessary. This is all in addition to an extensive
OS lockdown script, as well as additional lockdown requirements for
the database.

I appreciate the help. I believe this is an excellent starting point
to try and address this requirement.

Ken

On 12/21/2010 06:12 AM, Kenneth Buckler wrote:

Hello,

I am investigating security requirements for configuring a PostgreSQL
database on a Linux system.
One of the security requirements our organization would like to
implement is "trusted startup", in that PostgreSQL would verify the
authenticity of the binaries and configuration files before making the
database available to users.

Do you have a trusted boot path from BIOS to bootloader to kernel to
init core userspace, where everything is digitally signed (by you or
someone else) and verified before execution? Do you disable kernel
module loading?

If not, you're wasting your time, because a compromise by malicious
kernel module, modified init, modified md5 command, etc will render your
precautions totally pointless.

If your BIOS can't verify the bootloader, which is likely on an x86 /
x64 system, then you can still get some protection by signing your
kernels and using a bootloader that checks signatures. If someone messes
with the bootloader you lose, but it'll help protect you against obvious
automated attacks. You might be able to use the Trusted Platform Module
(TPM) on your machine to get a fully verified chain of trust, though, by
using Trusted GRUB.

http://trousers.sourceforge.net/grub.html

If you can reasonably trust that the kernel you loaded is OK, you can
have it verify signatures on binaries before executing them. There was a
DigSig project for that (http://disec.sourceforge.net/) but it seems to
have stopped recently. I'm not sure if there's any replacement.

Without kernel-level signature verification, all you can really do is
have a custom initrd/initramfs (signed and verified by grub during boot)
that checks the signatures on init, md5, gpg, libc, etc etc (any binary
root runs, including scripts) before switching to the real root FS
during boot. Then you can have your Pg startup scripts (which you signed
on a separate, trusted machine) verify GnuPG signatures of the Pg
binaries before execution.

All in all, it's a painful, clumsy way to do things, and AFAIK there's
little support in mainline Linux systems for trusted boot and
trusted-binary systems. You might find out more with a search for "linux
trusted computing", "linux trusted boot", "linux tpm", "linux signed
binaries", etc.

Personally, I'd be using existing system- and network-level intrusion
detection tools like tripwire and snort to try to spot intrusion if and
when it happens. I'm not confident that a chain-of-trust approach is
workable on Linux systems at present, though I'd love to be proved wrong
by being pointed at existing support I've missed.

--
Craig Ringer

On Mon, Dec 20, 2010 at 8:53 PM, Craig Ringer
<craig@postnewspapers.com.au> wrote:

Do you have a trusted boot path from BIOS to bootloader to kernel to init
core userspace, where everything is digitally signed (by you or someone
else) and verified before execution? Do you disable kernel module loading?

If not, you're wasting your time, because a compromise by malicious kernel
module, modified init, modified md5 command, etc will render your
precautions totally pointless.

If your BIOS can't verify the bootloader, which is likely on an x86 / x64
system, then you can still get some protection by signing your kernels and
using a bootloader that checks signatures. If someone messes with the
bootloader you lose, but it'll help protect you against obvious automated
attacks. You might be able to use the Trusted Platform Module (TPM) on your
machine to get a fully verified chain of trust, though, by using Trusted
GRUB.

http://trousers.sourceforge.net/grub.html

If you can reasonably trust that the kernel you loaded is OK, you can have
it verify signatures on binaries before executing them. There was a DigSig
project for that (http://disec.sourceforge.net/) but it seems to have
stopped recently. I'm not sure if there's any replacement.

Without kernel-level signature verification, all you can really do is have a
custom initrd/initramfs (signed and verified by grub during boot) that
checks the signatures on init, md5, gpg, libc, etc etc (any binary root
runs, including scripts) before switching to the real root FS during boot.
Then you can have your Pg startup scripts (which you signed on a separate,
trusted machine) verify GnuPG signatures of the Pg binaries before
execution.

All in all, it's a painful, clumsy way to do things, and AFAIK there's
little support in mainline Linux systems for trusted boot and trusted-binary
systems. You might find out more with a search for "linux trusted
computing", "linux trusted boot", "linux tpm", "linux signed binaries", etc.

Personally, I'd be using existing system- and network-level intrusion
detection tools like tripwire and snort to try to spot intrusion if and when
it happens. I'm not confident that a chain-of-trust approach is workable on
Linux systems at present, though I'd love to be proved wrong by being
pointed at existing support I've missed.

--
Craig Ringer

I find it very comforting that I am not the only one who finds this
requirement a bit "out there".
Unfortunately, these requirements are set in stone, and no matter how
hard I try, can not be altered.
We live in a world where compliance is king. Nevermind if compliance
doesn't actually make the system more secure.

Unfortunately Tripwire does not meet the full requirement, as it does
not prevent the database from starting.

Ken

On 12/22/2010 02:05 AM, Kenneth Buckler wrote:

I find it very comforting that I am not the only one who finds this
requirement a bit "out there".
Unfortunately, these requirements are set in stone, and no matter how
hard I try, can not be altered.
We live in a world where compliance is king. Nevermind if compliance
doesn't actually make the system more secure.

Unfortunately Tripwire does not meet the full requirement, as it does
not prevent the database from starting.

In this case, here's what I'd do:

- Call Red Hat
- Say "I'd like to buy RHEL, but have <x> weird requirement, can you
help me"

--
Craig Ringer

We live in a world where compliance is king. Nevermind if compliance
doesn't actually make the system more secure.

Er .. re my previous post, I don't mean "lie to RH and claim to want to
buy RHEL to get free support". I mean that you should consider going to
management and getting approval for professional support and integration
work from a specialist, because you're going to need it.

Alternately you could do the dodgy Trusted GRUB + signed kernel + signed
initrd with scripted GnuPG verification hack. It'd be a lot better than
nothing if your target server has a TPM you can enable and use for
Trusted GRUB.

--
Craig Ringer