EAV or not to EAV?

On Thu, 22 Nov 2007, Reg Me Please wrote:

Hi all.

I have to take into account the update history of every single
field into a number of tables.
Those updates can come to the future (next values) but also to the
past (history fix) and apply only to some fields, usually not to the
entire record.
Every lookup in those tables is always related to a timestamp that
normally is the current timestamp, but can also be a past timestamp.

I've come up with a "traditionally regretted" EAV design with the add of
timestamps for the history. And the related problems, some of which have
been solved by Joe Conways's crosstab contrib.

Is there a better idea than mine? I hope so.

We use contrib/hstore for this

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

Hello all!

I've a small bash script backup.sh for creating dumps on my Postgre db:

#!/bin/bash
time=`date '+%d'-'%m'-'%y'`
cd /home/swkm/services/test
pg_dump mydb > mydb_dump_$time.out

I've edited crontab and added a line:

00 4 * * * swkm /home/swkm/services/test/backup.sh

to execute the backup.sh as user swkm daily at 4 am.

The user swkm is the user I use to create backups manually. The script
itself is executed fine if run manually but run on cron scheduler I got an
mydb_dump_$time.out file empty (of 0 kb)

Do you have any idea about what's wrong?

Thanks
Sorin

On Nov 22, 2007 2:19 PM, Sorin N. Ciolofan <ciolofan@ics.forth.gr> wrote:

Hello all!

I've a small bash script backup.sh for creating dumps on my Postgre db:

#!/bin/bash
time=`date '+%d'-'%m'-'%y'`
cd /home/swkm/services/test
pg_dump mydb > mydb_dump_$time.out

I've edited crontab and added a line:

00 4 * * * swkm /home/swkm/services/test/backup.sh

to execute the backup.sh as user swkm daily at 4 am.

The user swkm is the user I use to create backups manually. The script
itself is executed fine if run manually but run on cron scheduler I got an
mydb_dump_$time.out file empty (of 0 kb)

Do you have any idea about what's wrong?

Thanks
Sorin

Hi Sorin,

why don't you add a "MAILTO=<youraddress>" at the start of your
crontab file, so that you can receive a report of the problem?

Regards
Marco

--
Marco Bizzarri
http://iliveinpisa.blogspot.com/

Hi Marco!

Thank you for the advice.

I got:

/home/swkm/services/test/backup.sh: line 4: pg_dump: command not found
updating: mydb_dump_22-11-07.out (stored 0%)

which seems strange

-----Original Message-----
From: pgsql-admin-owner@postgresql.org
[mailto:pgsql-admin-owner@postgresql.org] On Behalf Of Marco Bizzarri
Sent: Thursday, November 22, 2007 3:28 PM
To: Sorin N. Ciolofan
Cc: pgsql-admin@postgresql.org; pgsql-general@postgresql.org
Subject: Re: [ADMIN] backup of postgres scheduled with cron

On Nov 22, 2007 2:19 PM, Sorin N. Ciolofan <ciolofan@ics.forth.gr> wrote:

Hello all!

I've a small bash script backup.sh for creating dumps on my Postgre db:

#!/bin/bash
time=`date '+%d'-'%m'-'%y'`
cd /home/swkm/services/test
pg_dump mydb > mydb_dump_$time.out

I've edited crontab and added a line:

00 4 * * * swkm /home/swkm/services/test/backup.sh

to execute the backup.sh as user swkm daily at 4 am.

The user swkm is the user I use to create backups manually. The script
itself is executed fine if run manually but run on cron scheduler I got an
mydb_dump_$time.out file empty (of 0 kb)

Do you have any idea about what's wrong?

Thanks
Sorin

Hi Sorin,

why don't you add a "MAILTO=<youraddress>" at the start of your
crontab file, so that you can receive a report of the problem?

Regards
Marco

--
Marco Bizzarri
http://iliveinpisa.blogspot.com/

---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

On Thu, 22 Nov 2007, Sorin N. Ciolofan wrote:

Hello all!

I've a small bash script backup.sh for creating dumps on my Postgre db:

#!/bin/bash
time=`date '+%d'-'%m'-'%y'`
cd /home/swkm/services/test
pg_dump mydb > mydb_dump_$time.out

I've edited crontab and added a line:

00 4 * * * swkm /home/swkm/services/test/backup.sh

to execute the backup.sh as user swkm daily at 4 am.

The user swkm is the user I use to create backups manually. The script
itself is executed fine if run manually but run on cron scheduler I got an
mydb_dump_$time.out file empty (of 0 kb)

Do you have any idea about what's wrong?

you have to pass the -U argument to pg_dump, otherwise it'll try to run
the script as root

jef

On Nov 22, 2007 2:46 PM, Sorin N. Ciolofan <ciolofan@ics.forth.gr> wrote:

Hi Marco!

Thank you for the advice.

I got:

/home/swkm/services/test/backup.sh: line 4: pg_dump: command not found
updating: mydb_dump_22-11-07.out (stored 0%)

which seems strange

Try putting the full path of the pg_dump command in the script.

Regards
Marco

--
Marco Bizzarri
http://iliveinpisa.blogspot.com/

On Thu, Nov 22, 2007 at 02:28:08PM +0100, Marco Bizzarri wrote:

why don't you add a "MAILTO=<youraddress>" at the start of your
crontab file, so that you can receive a report of the problem?

Note: check that your cron accepts such an addition. Many systems now use
Vixie's cron, which does accept that, but some don't. It's a nice feature,
and good for this purpose.

A

--
Andrew Sullivan
Old sigs will return after re-constitution of blue smoke

On Nov 22, 2007 2:53 PM, Andrew Sullivan <ajs@crankycanuck.ca> wrote:

On Thu, Nov 22, 2007 at 02:28:08PM +0100, Marco Bizzarri wrote:

why don't you add a "MAILTO=<youraddress>" at the start of your
crontab file, so that you can receive a report of the problem?

Note: check that your cron accepts such an addition. Many systems now use
Vixie's cron, which does accept that, but some don't. It's a nice feature,
and good for this purpose.

Andrew, can you confirm the previous statement? I'm checking on a Debian Linux,
at it seems to be a Vixie Cron, and that feature is described in the man page...

A

--
Andrew Sullivan
Old sigs will return after re-constitution of blue smoke

Regards
Marco

--
Marco Bizzarri
http://iliveinpisa.blogspot.com/

Hello Sorin!

Sorin N. Ciolofan wrote:

#!/bin/bash
time=`date '+%d'-'%m'-'%y'`
cd /home/swkm/services/test
pg_dump mydb > mydb_dump_$time.out

You should output STDERR to some error logfile or set MAILTO in your
crontab.
I guess you then would have seen an error message saying that pg_dump
was not found because cron doesn't load the users environment and
therefore PATH variable isn't set.
I suggest you call pg_dump in your script by absolute path.

Greetings,
Frank Wittig

On Thu, Nov 22, 2007 at 02:59:33PM +0100, Marco Bizzarri wrote:

Andrew, can you confirm the previous statement? I'm checking on a Debian Linux,
at it seems to be a Vixie Cron, and that feature is described in the man page...

If the feature's in your man page, then it works on your system :) I just
wanted to warn you that this isn't an original feature of cron, so you have
to check your system always to be sure you have it.

A

--
Andrew Sullivan
Old sigs will return after re-constitution of blue smoke

Good Morning AndrewI noticed the Vixie cron responds to SIGHUP signalsDo you have any suggestions or tutorials on how Postgres would feed these event signals to Vixie cron?
Thanks/Martin ______________________________________________Disclaimer and confidentiality noteEverything in this e-mail and any attachments relates to the official business of Sender. This transmission is of a confidential nature and Sender does not endorse distribution to any party other than intended recipient. Sender does not necessarily endorse content contained within this transmission.> > On Thu, Nov 22, 2007 at 02:28:08PM +0100, Marco Bizzarri wrote:> > > > why don't you add a "MAILTO=<youraddress>" at the start of your> > crontab file, so that you can receive a report of the problem?> > Note: check that your cron accepts such an addition. Many systems now use> Vixie's cron, which does accept that, but some don't. It's a nice feature,> and good for this purpose.> > A> > -- > Andrew Sullivan> Old sigs will return after re-constitution of blue smoke> > ---------------------------(end of broadcast)---------------------------> TIP 1: if posting/reading through Usenet, please send an appropriate> subscribe-nomail command to majordomo@postgresql.org so that your> message can get through to the mailing list cleanly
_________________________________________________________________
Share life as it happens with the new Windows Live.Download today it's FREE!
http://www.windowslive.com/share.html?ocid=TXT_TAGLM_Wave2_sharelife_112007

Thank you all,

Yes, I used the absolute path in my script and now works ok :-)

Thank you again
Sorin

-----Original Message-----
From: pgsql-general-owner@postgresql.org
[mailto:pgsql-general-owner@postgresql.org] On Behalf Of Frank Wittig
Sent: Thursday, November 22, 2007 4:01 PM
To: Sorin N. Ciolofan
Cc: pgsql-admin@postgresql.org; pgsql-general@postgresql.org
Subject: Re: [GENERAL] backup of postgres scheduled with cron

Hello Sorin!

Sorin N. Ciolofan wrote:

#!/bin/bash
time=`date '+%d'-'%m'-'%y'`
cd /home/swkm/services/test
pg_dump mydb > mydb_dump_$time.out

You should output STDERR to some error logfile or set MAILTO in your
crontab.
I guess you then would have seen an error message saying that pg_dump
was not found because cron doesn't load the users environment and
therefore PATH variable isn't set.
I suggest you call pg_dump in your script by absolute path.

Greetings,
Frank Wittig

On Thu, Nov 22, 2007 at 09:14:13AM -0500, Martin Gainty wrote:

Good Morning AndrewI noticed the Vixie cron responds to SIGHUP signalsDo
you have any suggestions or tutorials on how Postgres would feed these
event signals to Vixie cron?

Why would Postgres have to tell crond to restart?

A

--
Andrew Sullivan
Old sigs will return after re-constitution of blue smoke

On Thu, 2007-11-22 at 09:48 +0100, Reg Me Please wrote:

I have to take into account the update history of every single
field into a number of tables.
Those updates can come to the future (next values) but also to the
past (history fix) and apply only to some fields, usually not to the
entire record.
Every lookup in those tables is always related to a timestamp that
normally is the current timestamp, but can also be a past timestamp.

I've come up with a "traditionally regretted" EAV design with the add of
timestamps for the history. And the related problems, some of which have
been solved by Joe Conways's crosstab contrib.

Here are two more ideas:

1) Vertically partition the tables so that each field you want to track
is in its own table, and then track the history of those individual
tables. Then join the tables on a key that doesn't change when you need
to see the whole table. This works well if you only have a few non-key
fields per table.

2) Store a normal history, but also store a bitmap of the fields that
change in each record. You might use partial indexes to be able to
quickly select only those records where a certain field has changed.

EAV will give you many problems down the line, and I don't think it will
solve anything for you.

Regards,
Jeff Davis