Re: Questions for the COPY expert

Dawn,
I'm not that much of an expert :-)
I am forwarding your message to the "pgsql-general"
mailing list, where the real experts can be found.
I'm sure some of those can provide enlightenment.
I suspect that the limitations you observe have more
to do with the performance of networks and operating
systems than of COPY itself.
Regards

--- Dawn Vu <dvu@satx.rr.com> wrote:

Hi Jeff,

Read your usingcopy.php, it looks like you have a
lot of
experience with it. Wonder if you can answer the
following
questions for me:

1. COPY [tablename] TO '/../datafile' seems to have
a
limitation when datafile is bigger than 2G. Is
this a
postgresql limitation? I can get around it but
want to
know if there is a way to let the file grow as
large as
OS filesize limit. Any idea?

2. I use COPY to transfer data from one machine to
another
over the network using stdin and stdout without
having
to write data to intermediate file. Does COPY
read and
load all data into memory at one time or does it
do a
chunk at a time and then pipeline data over the
network?
The reason why I ask this question is because I
was
transferring 41M+ records and one hour into it,
the process
got a calloc error. What seems to be the
limitation here?
What memory aspects do I need to examine? More
DB buffers? Configuration setting(s)?

-DV

__________________________________________________
Do You Yahoo!?
Send FREE video emails in Yahoo! Mail!
http://promo.yahoo.com/videomail/