Status reporting for COPY

"Jim C. Nasby" <jim@nasby.net> writes:

Would it be difficult to allow COPY to output intermediate status while
it's in operation? IE: total rows copied so far, and maybe what position
in the file it's in on a copy from? This would let you know it was still
working as well as giving you some idea how much longer it will take. It
would also be handy to get a summary report after the copy was done...
rows inserted or exported, how long it took, rows/second.

IMHO this should be done by the client--use 'COPY FROM stdin' and keep
track yourself of where you are in the copy as you send the data. I'd
hate to clutter up the wire protocol with status updates that are
basically noise.

There's no reason that psql's \copy command couldn't be enhanced to do
this on the client side.

BTW, is there a more official means of submitting feature requests? I
don't see anything like a Bugzilla for PGSQL...

Posting to -hackers is probably a better place than -general.

-Doug

Moving to -hackers at Doug's suggestion...

On Wed, May 14, 2003 at 01:18:05PM -0400, Doug McNaught wrote:

"Jim C. Nasby" <jim@nasby.net> writes:

Would it be difficult to allow COPY to output intermediate status while
it's in operation? IE: total rows copied so far, and maybe what position
in the file it's in on a copy from? This would let you know it was still
working as well as giving you some idea how much longer it will take. It
would also be handy to get a summary report after the copy was done...
rows inserted or exported, how long it took, rows/second.

IMHO this should be done by the client--use 'COPY FROM stdin' and keep
track yourself of where you are in the copy as you send the data. I'd
hate to clutter up the wire protocol with status updates that are
basically noise.

There's no reason that psql's \copy command couldn't be enhanced to do
this on the client side.

If the performance gain of COPY over \copy is substantial it might still
be nice to figure something out; but I'm guessing that the gain isn't
than big... or is it? FWIW, some databases just use a seperate utility
for copy/load...

BTW, is there a more official means of submitting feature requests? I
don't see anything like a Bugzilla for PGSQL...

Posting to -hackers is probably a better place than -general.

-Doug

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

--
Jim C. Nasby (aka Decibel!) jim@nasby.net
Member: Triangle Fraternity, Sports Car Club of America
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

"Jim C. Nasby" <jim@nasby.net> writes:

If the performance gain of COPY over \copy is substantial it might still
be nice to figure something out; but I'm guessing that the gain isn't
than big... or is it? FWIW, some databases just use a seperate utility
for copy/load...

\copy just invokes COPY on the server side, reads the file on the
client and send it to the server over the DB connection. It's for
situations where the client and server are different machines, or the
server process can't see the file you're loading.

If server and client are on the same machine COPYing directly from the
file (in the server) will probably be a little faster than \copy in
psql.

Adding progress reporting to server-side COPY is still bloat IMHO.

-Doug