Simple row serialization?

Started by Ivan Vorasalmost 18 years ago13 messages
#1Ivan Voras
ivoras@freebsd.org

Hi,

I'd like to implement some simple data logging via triggers on a small
number of infrequently updated tables and I'm wondering if there are
some helpful functions, plugins or idioms that would serialize a row
(received for example in a AFTER INSERT trigger) into a string that I'd
store in the log table. There's a limited number of field types
involved: varchars, integers and booleans. I'm not looking for anything
fancy, comma-separated string result will be just fine; Even better,
something like a dictionary ("field_name":"field_value",...) would be
nice. The reason for trying to do it this way is that I don't want to
create separate log tables for every table I wish to log.

#2Pavel Stehule
pavel.stehule@gmail.com
In reply to: Ivan Voras (#1)
Re: Simple row serialization?

Hello,

use plperl.

PostgreSQL hasn't simple tool for it. Maybe this link will be usefull for you
http://www.ciselant.de/projects/pg_ci_diff/

Pavel

Show quoted text

On 26/01/2008, Ivan Voras <ivoras@freebsd.org> wrote:

Hi,

I'd like to implement some simple data logging via triggers on a small
number of infrequently updated tables and I'm wondering if there are
some helpful functions, plugins or idioms that would serialize a row
(received for example in a AFTER INSERT trigger) into a string that I'd
store in the log table. There's a limited number of field types
involved: varchars, integers and booleans. I'm not looking for anything
fancy, comma-separated string result will be just fine; Even better,
something like a dictionary ("field_name":"field_value",...) would be
nice. The reason for trying to do it this way is that I don't want to
create separate log tables for every table I wish to log.

#3Andrew Dunstan
andrew@dunslane.net
In reply to: Ivan Voras (#1)
Re: Simple row serialization?

Ivan Voras wrote:

Hi,

I'd like to implement some simple data logging via triggers on a small
number of infrequently updated tables and I'm wondering if there are
some helpful functions, plugins or idioms that would serialize a row
(received for example in a AFTER INSERT trigger) into a string that
I'd store in the log table. There's a limited number of field types
involved: varchars, integers and booleans. I'm not looking for
anything fancy, comma-separated string result will be just fine; Even
better, something like a dictionary ("field_name":"field_value",...)
would be nice. The reason for trying to do it this way is that I don't
want to create separate log tables for every table I wish to log.

Why not create the audit tables with composite types rather than strings?

create table audit_foo_table as (who text, when timestamptz, old
foo, new foo);

Or you could use an hstore (see contrib).

Or you could possibly use some of the XML support in 8.3 for serialization.

This is a usage question, so it really doesn't belong on -hackers.

cheers

andrew

#4Ivan Voras
ivoras@freebsd.org
In reply to: Pavel Stehule (#2)
Re: Simple row serialization?

Pavel Stehule wrote:

Hello,

use plperl.

I'd like something more light-weight to reduce complexity of deployment.
Something in pgplsql would be ideal. Is there a way to simply iterate
over the fields of a row and retrieve field names and values?

PostgreSQL hasn't simple tool for it. Maybe this link will be usefull for you
http://www.ciselant.de/projects/pg_ci_diff/

Thanks, this is very interesting work! It's an overkill for my current
needs but I'll keep it in mind.

#5Ivan Voras
ivoras@freebsd.org
In reply to: Andrew Dunstan (#3)
Re: Simple row serialization?

Andrew Dunstan wrote:

Why not create the audit tables with composite types rather than strings?

create table audit_foo_table as (who text, when timestamptz, old foo,
new foo);

Because this would lead to having a log/shadow/audit table for every
table I wish to log. (or is there an opaque "generic row" data type?
"record" and "any" generate syntax errors).

Or you could use an hstore (see contrib).

Doesn't seem applicable.

Or you could possibly use some of the XML support in 8.3 for serialization.

I need this for 8.1 :)

This is a usage question, so it really doesn't belong on -hackers.

Thank you - I'm reading the list through gmane and I didn't notice its
name "gmane.comp.db.postgresql.devel.general" is incorrect. I'll find a
more suitable list.

#6Tom Lane
tgl@sss.pgh.pa.us
In reply to: Ivan Voras (#5)
Re: Simple row serialization?

Ivan Voras <ivoras@freebsd.org> writes:

Andrew Dunstan wrote:

Or you could possibly use some of the XML support in 8.3 for serialization.

I need this for 8.1 :)

There's an even easier way in 8.3: just cast the rowtype to text.

regression=# select row(1,2,false)::text;
row
---------
(1,2,f)
(1 row)

Although this won't work at the SQL level in 8.1, I think you might be
able to accomplish the equivalent within plpgsql by assigning the
rowtype value to a text variable.

regards, tom lane

#7Pavel Stehule
pavel.stehule@gmail.com
In reply to: Tom Lane (#6)
Re: Simple row serialization?

On 26/01/2008, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Ivan Voras <ivoras@freebsd.org> writes:

Andrew Dunstan wrote:

Or you could possibly use some of the XML support in 8.3 for serialization.

I need this for 8.1 :)

There's an even easier way in 8.3: just cast the rowtype to text.

regression=# select row(1,2,false)::text;
row
---------
(1,2,f)
(1 row)

Although this won't work at the SQL level in 8.1, I think you might be
able to accomplish the equivalent within plpgsql by assigning the
rowtype value to a text variable.

you lost names :(. The best of will be support some like dictionary

so select ((row(1,2,3))::sometype)
{{a:1},{b:2},{c:3}}

Regards
Pavel Stehule

Show quoted text

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

#8Tom Lane
tgl@sss.pgh.pa.us
In reply to: Pavel Stehule (#7)
Re: Simple row serialization?

"Pavel Stehule" <pavel.stehule@gmail.com> writes:

On 26/01/2008, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Although this won't work at the SQL level in 8.1, I think you might be
able to accomplish the equivalent within plpgsql by assigning the
rowtype value to a text variable.

you lost names :(.

And datatype info too; but AFAICT that's exactly what the OP wants.

regards, tom lane

#9Ivan Voras
ivoras@freebsd.org
In reply to: Tom Lane (#8)
Re: Simple row serialization?

Tom Lane wrote:

"Pavel Stehule" <pavel.stehule@gmail.com> writes:

On 26/01/2008, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Although this won't work at the SQL level in 8.1, I think you might be
able to accomplish the equivalent within plpgsql by assigning the
rowtype value to a text variable.

you lost names :(.

And datatype info too; but AFAICT that's exactly what the OP wants.

Thanks! Having names would be great, but this is sufficient for now.
(I've tested it and it works!).

#10Oleg Bartunov
oleg@sai.msu.su
In reply to: Ivan Voras (#5)
Re: Simple row serialization?

Ivan,

have you seen contrib/hstore ? It's perl-like hash data type and can be
ideally suited for you.

Oleg
On Sat, 26 Jan 2008, Ivan Voras wrote:

Andrew Dunstan wrote:

Why not create the audit tables with composite types rather than strings?

create table audit_foo_table as (who text, when timestamptz, old foo,
new foo);

Because this would lead to having a log/shadow/audit table for every table I
wish to log. (or is there an opaque "generic row" data type? "record" and
"any" generate syntax errors).

Or you could use an hstore (see contrib).

Doesn't seem applicable.

Or you could possibly use some of the XML support in 8.3 for serialization.

I need this for 8.1 :)

This is a usage question, so it really doesn't belong on -hackers.

Thank you - I'm reading the list through gmane and I didn't notice its name
"gmane.comp.db.postgresql.devel.general" is incorrect. I'll find a more
suitable list.

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

#11Noname
tomas@tuxteam.de
In reply to: Ivan Voras (#5)
Re: Simple row serialization?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sat, Jan 26, 2008 at 07:32:35PM +0100, Ivan Voras wrote:

Andrew Dunstan wrote:

[...]

Or you could use an hstore (see contrib).

Doesn't seem applicable.

Have a closer look: it might :-)

regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFHnBcxBcgs9XrR2kYRAv+zAJwPux4ik4JLOHzwyNanUFNkV2yGwACfdZwl
SzA4xZxahgT7d8Z3PyHJwJ4=
=5ECG
-----END PGP SIGNATURE-----

#12Ivan Voras
ivoras@freebsd.org
In reply to: Noname (#11)
Re: Simple row serialization?

On 27/01/2008, tomas@tuxteam.de <tomas@tuxteam.de> wrote:

On Sat, Jan 26, 2008 at 07:32:35PM +0100, Ivan Voras wrote:

Andrew Dunstan wrote:

[...]

Or you could use an hstore (see contrib).

Doesn't seem applicable.

Have a closer look: it might :-)

I found these documents on hstore:

http://developer.postgresql.org/pgdocs/postgres/hstore.html
http://www.sai.msu.su/~megera/wiki/Hstore

From them, it seems "hstore" is another datatype (as in:
'a=>b'::hstore), which sounds good (though if it was me I'd have
picked a different name for it, like "dict" or "hashmap" :) ) for
storing both field names and their values, but I don't see a way to
convert a row/record passed to a trigger to a hstore.

#13Andrew Dunstan
andrew@dunslane.net
In reply to: Ivan Voras (#12)
Re: Simple row serialization?

Ivan Voras wrote:

On 27/01/2008, tomas@tuxteam.de <tomas@tuxteam.de> wrote:

On Sat, Jan 26, 2008 at 07:32:35PM +0100, Ivan Voras wrote:

Andrew Dunstan wrote:

[...]

Or you could use an hstore (see contrib).

Doesn't seem applicable.

Have a closer look: it might :-)

I found these documents on hstore:

http://developer.postgresql.org/pgdocs/postgres/hstore.html
http://www.sai.msu.su/~megera/wiki/Hstore

From them, it seems "hstore" is another datatype (as in:

'a=>b'::hstore), which sounds good (though if it was me I'd have
picked a different name for it, like "dict" or "hashmap" :) ) for
storing both field names and their values, but I don't see a way to
convert a row/record passed to a trigger to a hstore.

It's trivial to do in a plperl trigger, since it gets the new and old
records as hashes with the field names as keys.

cheers

andrew