Compression (was Re: [HACKERS] varchar/char size)

My CA/Ingres Admin manual points out that there is a tradeoff between
compressing tuples to save disk storage and the extra processing work
required to uncompress for use. They suggest that the only case where you
would consider compressing on disk is when your system is very I/O bound,
and you have CPU to burn.

The default for Ingres is to not compress anything, but you can specify
compression on a table-by-table basis.

btw, char() is a bit trickier to handle correctly if you do compress it on
disk, since trailing blanks must be handled correctly all the way through.
For example, you would want 'hi' = 'hi ' to be true, which is not a
requirement for varchar().

- Tom

Anybody thought about real gzip style compression? There's a specialiased
RDBMS called Iditis (written specifically for one task) which, like
PostgreSQL stores data at the file level and uses a gzip-based library
to access the files. I gather this is transparent to the software. Has
anyone thought of anything equivalent for PG/SQL?

To be honest I haven't looked into how Iditis does it (it's a commercial
program and I don't have the source). I don't actually see how this
could be done for small writes of data - how does it build the lookup
tables for the compression? However, it might be worth considering for
use with the text field type.

Andrew
----------------------------------------------------------------------------
Dr. Andrew C.R. Martin University College London
EMAIL: (Work) martin@biochem.ucl.ac.uk (Home) andrew@stagleys.demon.co.uk
URL: http://www.biochem.ucl.ac.uk/~martin
Tel: (Work) +44(0)171 419 3890 (Home) +44(0)1372 275775