COPY FROM how long should take ?

Date: 08/26/2011 05:40 PM
Subject: [GENERAL] COPY FROM how long should take ?
Sent by: pgsql-general-owner@postgresql.org

Hello,
today I try to restore on of my tables with copy from file that I made
before.
The file is 2.4 GB, only integers ... took me 3 hours and 30 min and I
hit
CTRL+C on i7 processor with 8 GB memory, sata 2 hard drive. I modify
some
psql conf file values and increase memory, work, wal, temp, check point
segments to 55
Is that normal ? Whole file is 37 mil lines. When I hit enter it was on
line
as logs says 26 million. I run it twice and second time after 45 min I
again hit
CTRL+C and in logs I see it was again on 26 million line. Well, the
line number
is approximately. Is everything normal ?

Postgresql 9.0.4

'It was on 26 million line' - does this mean there were 26 million records
in the table? I was migrating data from Oracle to PostgreSQL, using ora2pg
with COPY option and in under 3 hours it easily copied a table that was 10
GB in size. Are you seeing any errors in postgres log file? In my case,
the number of records in postgresql table stopped increasing after some
time and I realized data transfer was failing. I checked the log file for
errors,corrected the data issues in the source table (Oracle) and it
finished the load without issues.
Regards,
Jayadevan

DISCLAIMER:

"The information in this e-mail and any attachment is intended only for
the person to whom it is addressed and may contain confidential and/or
privileged material. If you have received this e-mail in error, kindly
contact the sender and destroy all copies of the original communication.
IBS makes no warranty, express or implied, nor guarantees the accuracy,
adequacy or completeness of the information contained in this email or any
attachment and is not liable for any errors, defects, omissions, viruses
or for resultant loss or damage, if any, direct or indirect."

On Fri, 26 Aug 2011 17:49:35 +0530, Jayadevan M wrote:

Date:

08/26/2011 05:40 PM

Subject: [GENERAL] COPY FROM how long should

take ?

Sent by: pgsql-general-owner@postgresql.org

Hello,

today I try to restore on of my tables with copy from file that I made

before.
The file is 2.4 GB, only integers ... took me 3 hours

and 30 min and I

hit
CTRL+C on i7 processor with 8 GB memory,

sata 2 hard drive. I modify

some
psql conf file values and

increase memory, work, wal, temp, check point

segments to 55
Is

that normal ? Whole file is 37 mil lines. When I hit enter it was on

line

as logs says 26 million. I run it twice and second time after

45 min I

again hit
CTRL+C and in logs I see it was again on 26

million line. Well, the

line number
is approximately. Is

everything normal ?

Postgresql 9.0.4

'It was on 26 million

line' - does this mean there were 26 million records in the table? I was
migrating data from Oracle to PostgreSQL, using ora2pg with COPY option
and in under 3 hours it easily copied a table that was 10 GB in size.
Are you seeing any errors in postgres log file? In my case, the number
of records in postgresql table stopped increasing after some time and I
realized data transfer was failing. I checked the log file for
errors,corrected the data issues in the source table (Oracle) and it
finished the load without issues.

Regards,
Jayadevan

Whole file
is approximately 32 million lines = 32 million records. I check log file
no any errors.

--
Regards,
Condor

You can add a trigger and send message every 100000 rows

--
------------
pasman