PostGreSQL and zlib

Started by Hubert FONGARNANDabout 18 years ago4 messages
#1Hubert FONGARNAND
informatique.internet@fiducial.fr

Hi everybody,

I'm looking for using zlib function directly in the database :

Example : a function "zipit" that would take a bytea as argument and
returns a compressed bytea
and an "unzipit" that would take a compressed bytea as argument and
returns a uncompressed bytea

I'm sure it's very easy to implement as a C function that would call
zlib

Is there already an implementation of such things?

Thanks!

Hubert FONGARNAND
_______________________________________________
Ce message et les éventuels documents joints peuvent contenir des informations confidentielles.
Au cas où il ne vous serait pas destiné, nous vous remercions de bien vouloir le supprimer et en aviser immédiatement l'expéditeur. Toute utilisation de ce message non conforme à sa destination, toute diffusion ou publication, totale ou partielle et quel qu'en soit le moyen est formellement interdite.
Les communications sur internet n'étant pas sécurisées, l'intégrité de ce message n'est pas assurée et la société émettrice ne peut être tenue pour responsable de son contenu.

#2Gregory Stark
stark@enterprisedb.com
In reply to: Hubert FONGARNAND (#1)
Re: PostGreSQL and zlib

"Hubert FONGARNAND" <informatique.internet@fiducial.fr> writes:

I'm sure it's very easy to implement as a C function that would call zlib

Is there already an implementation of such things?

Not that I'm aware though you might look at pgcrypto. Good crypto has to
compress first so there may be a possibility of just doing the compression.

Postgres also will automatically try to compress data like byteas if the
record is larger than 2kB (1/4 of the block size if you've changed the block
size). So you may not have to do anything if you're just looking to save space
on disk.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com

#3Hubert FONGARNAND
informatique.internet@fiducial.fr
In reply to: Gregory Stark (#2)
Re: PostGreSQL and zlib

The problem is that for TOAST columns :
"The compression technique used is a fairly simple and very fast member
of the LZ family of compression techniques. See
src/backend/utils/adt/pg_lzcompress.c for the details."
With such function you could choose you compression algorithm and
compression level...

Another question : does this TOAST code works for all type of data, like
TEXT (in our case)

Le jeudi 25 octobre 2007 à 11:03 +0100, Gregory Stark a écrit :

"Hubert FONGARNAND" <informatique.internet@fiducial.fr> writes:

I'm sure it's very easy to implement as a C function that would call zlib

Is there already an implementation of such things?

Not that I'm aware though you might look at pgcrypto. Good crypto has to
compress first so there may be a possibility of just doing the compression.

Postgres also will automatically try to compress data like byteas if the
record is larger than 2kB (1/4 of the block size if you've changed the block
size). So you may not have to do anything if you're just looking to save space
on disk.

_______________________________________________
Ce message et les éventuels documents joints peuvent contenir des informations confidentielles.
Au cas où il ne vous serait pas destiné, nous vous remercions de bien vouloir le supprimer et en aviser immédiatement l'expéditeur. Toute utilisation de ce message non conforme à sa destination, toute diffusion ou publication, totale ou partielle et quel qu'en soit le moyen est formellement interdite.
Les communications sur internet n'étant pas sécurisées, l'intégrité de ce message n'est pas assurée et la société émettrice ne peut être tenue pour responsable de son contenu.

#4Martijn van Oosterhout
kleptog@svana.org
In reply to: Hubert FONGARNAND (#3)
Re: PostGreSQL and zlib

On Thu, Oct 25, 2007 at 01:49:57PM +0200, Hubert FONGARNAND wrote:

Another question : does this TOAST code works for all type of data, like
TEXT (in our case)

Yes.

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

Show quoted text

From each according to his ability. To each according to his ability to litigate.