Status during copy [patch]

How do people feel about this patch? Currently COPY doesn't even return
a line count of the number of lines processed, while this patch would
make psql \copy produce date/time and count every 1000 rows, then print
a similar completion message.

---------------------------------------------------------------------------

Martijn van Oosterhout wrote:
-- Start of PGP signed section.

Hi,

The following (link to a) patch is a first attempt to add updates during a
\copy. This is so that one has some idea of progress. The output looks as
follows:

kleptog=# \copy website to /tmp/g
COPY OUT 0:00:00 28 lines completed.
kleptog=# \copy a from /tmp/h
ERROR: copy: line 186645, Invalid UNICODE character sequence found (0xe05f6c)
lost synchronization with server, resetting connection
kleptog=# \copy a from /tmp/i
COPY IN 0:00:42 100000 lines completed.
\.

http://svana.org/kleptog/pgsql/psql-copy.patch

File: src/bin/psql/copy.c +64 -3

The output is updated every 1000 rows. A total is listed at the end of the
copy. When there's an error, no summary is printed.

The output is sent to stderr, because it needs to be unbuffered and you
generally don't want the progress to be redirected to a file. However, if
stderr is redirected, should we just not print anything? What about
non-interactive? Should it be a command-line option?

Note it's not against current CVS since the anoncvs server is not working
for me (error below). However, the sources don't appear to have changed too
much.

cvs [update aborted]: connect to anoncvs.postgresql.org(64.49.215.9):2401
failed: Connection refused

Comments appreciated.
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

"the West won the world not by the superiority of its ideas or values or
religion but rather by its superiority in applying organized violence.
Westerners often forget this fact, non-Westerners never do."
- Samuel P. Huntington

-- End of PGP section, PGP failed!

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

How do people feel about this patch? Currently COPY doesn't even return
a line count of the number of lines processed, while this patch would
make psql \copy produce date/time and count every 1000 rows, then print
a similar completion message.

Seems much too noisy for me. That would be appropriate behavior in a
GUI, but psql is not and never will be a GUI. Martijn's concern about
hacking the behavior depending on where stderr points demonstrates
exactly why we don't want to do this.

I would not object to adding a line count to the COPY command completion
report (on the server side), though. That's arguably comparable to the
number-of-rows-processed values for INSERT/UPDATE/DELETE.

regards, tom lane

Yes, completely agree. Added to TODO:

o Return the number of rows loaded/unloaded

---------------------------------------------------------------------------

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

How do people feel about this patch? Currently COPY doesn't even return
a line count of the number of lines processed, while this patch would
make psql \copy produce date/time and count every 1000 rows, then print
a similar completion message.

Seems much too noisy for me. That would be appropriate behavior in a
GUI, but psql is not and never will be a GUI. Martijn's concern about
hacking the behavior depending on where stderr points demonstrates
exactly why we don't want to do this.

I would not object to adding a line count to the COPY command completion
report (on the server side), though. That's arguably comparable to the
number-of-rows-processed values for INSERT/UPDATE/DELETE.

regards, tom lane

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Added to TODO:

o Have COPY return number of rows loaded/unloaded

This is a backend change to return the number of rows affected as part
of the command status string returned, like we do with UPDATE/INSERT/DELETE.

---------------------------------------------------------------------------

Jim C. Nasby wrote:

On Mon, Jun 02, 2003 at 03:23:26PM -0400, Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

How do people feel about this patch? Currently COPY doesn't even return
a line count of the number of lines processed, while this patch would
make psql \copy produce date/time and count every 1000 rows, then print
a similar completion message.

Seems much too noisy for me. That would be appropriate behavior in a
GUI, but psql is not and never will be a GUI. Martijn's concern about
hacking the behavior depending on where stderr points demonstrates
exactly why we don't want to do this.

What about an option to enable it? Or a command to find out how many
rows a running copy command has imported? It would be very handy to be
able to know how far along a large copy operation is.
--
Jim C. Nasby (aka Decibel!) jim@nasby.net
Member: Triangle Fraternity, Sports Car Club of America
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073