reading row in backend
I'm just upgrading a database to v7.0, but I am rather unlucky in that in
each of 2 large (ca. 20000 rows) tables, 1 row is too big eg:
ERROR: copy: line 57552, Tuple is too big: size 8152, max size 8140
Now, it would help me if I could actually get a clue as to which row this
is. The error message comes from hio.c:118 at which point one has a
HeapTuple. Is there a way of extracting a piece of the row in the tuple to
be able to identify it?
It seems that a HeapTuple starts with a HeapTupleHeader, so the data I
suppose starts at tuple->t_data[sizeof(HeapTupleHeaderData)] and goes on for
tuple->t_len-sizeof(HeapTupleHeaderData) bytes, but how is it represented?
Any pointers appreciated! (These are huge COPY statements, so after the
error I'm left with an empty table - I'd just like to chop the 12 bytes
off!)
Cheers,
Patrick
Patrick Welche <prlw1@newn.cam.ac.uk> writes:
ERROR: copy: line 57552, Tuple is too big: size 8152, max size 8140
Now, it would help me if I could actually get a clue as to which row this
is.
Um ... doesn't that line number in the COPY source data help any?
regards, tom lane
On Mon, May 15, 2000 at 06:55:50PM -0400, Tom Lane wrote:
Patrick Welche <prlw1@newn.cam.ac.uk> writes:
ERROR: copy: line 57552, Tuple is too big: size 8152, max size 8140
Now, it would help me if I could actually get a clue as to which row this
is.Um ... doesn't that line number in the COPY source data help any?
Unfortunately not - in fact the 2nd large tuple allegedly has a smaller
line number than the first :(
Am I barking up the wrong tree if I think that the HeapTuple contains the
actual data? The first number in it would the primary key...
Cheers,
Patrick
Patrick Welche <prlw1@newn.cam.ac.uk> writes:
Um ... doesn't that line number in the COPY source data help any?
Unfortunately not - in fact the 2nd large tuple allegedly has a smaller
line number than the first :(
Hmm, are you claiming that COPY is reporting a bogus line number?
If so, that needs to be looked into.
Am I barking up the wrong tree if I think that the HeapTuple contains the
actual data? The first number in it would the primary key...
Type HeapTuple is a pointer to a HeapTupleData, which is just
administrative overhead. The t_data field of the HeapTupleData
points at the actual tuple (a HeapTupleHeaderData followed by
null-values bitmap and then the user data). See include/access/htup.h
regards, tom lane
On Mon, May 15, 2000 at 07:22:30PM -0400, Tom Lane wrote:
Patrick Welche <prlw1@newn.cam.ac.uk> writes:
Um ... doesn't that line number in the COPY source data help any?
Unfortunately not - in fact the 2nd large tuple allegedly has a smaller
line number than the first :(Hmm, are you claiming that COPY is reporting a bogus line number?
If so, that needs to be looked into.
Yup:
ERROR: copy: line 57552, Tuple is too big: size 8152, max size 8140
PQendcopy: resetting connection
ERROR: copy: line 26714, Tuple is too big: size 8164, max size 8140
PQendcopy: resetting connection
% wc /home/prlw1/db.out
3631449 19833180 186847533 /home/prlw1/db.out
and the second line is in a table that is over half way through the file.
I'll take a look after some sleep..
Am I barking up the wrong tree if I think that the HeapTuple contains the
actual data? The first number in it would the primary key...Type HeapTuple is a pointer to a HeapTupleData, which is just
administrative overhead. The t_data field of the HeapTupleData
points at the actual tuple (a HeapTupleHeaderData followed by
null-values bitmap and then the user data). See include/access/htup.h
I think my question really is, is the user data meant to be just text? When
I tried printing a block, I just got jibberish - it could well be the way I
tried to print it, but I don't know what to expect..
Cheers,
Patrick
Patrick Welche <prlw1@newn.cam.ac.uk> writes:
Hmm, are you claiming that COPY is reporting a bogus line number?
If so, that needs to be looked into.
Yup:
ERROR: copy: line 57552, Tuple is too big: size 8152, max size 8140
PQendcopy: resetting connection
ERROR: copy: line 26714, Tuple is too big: size 8164, max size 8140
PQendcopy: resetting connection
% wc /home/prlw1/db.out
3631449 19833180 186847533 /home/prlw1/db.out
and the second line is in a table that is over half way through the file.
Looks reasonable enough to me. The line numbers are within the data
being fed to that copy command --- copy has no way to know that it's
part of some larger script...
regards, tom lane
On Mon, May 15, 2000 at 08:44:03PM -0400, Tom Lane wrote:
Patrick Welche <prlw1@newn.cam.ac.uk> writes:
Hmm, are you claiming that COPY is reporting a bogus line number?
If so, that needs to be looked into.Yup:
ERROR: copy: line 57552, Tuple is too big: size 8152, max size 8140
PQendcopy: resetting connection
ERROR: copy: line 26714, Tuple is too big: size 8164, max size 8140
PQendcopy: resetting connection% wc /home/prlw1/db.out
3631449 19833180 186847533 /home/prlw1/db.outand the second line is in a table that is over half way through the file.
Looks reasonable enough to me. The line numbers are within the data
being fed to that copy command --- copy has no way to know that it's
part of some larger script...
Ah! Thank you! So it's 57552 rows down from "COPY" !
OK
Cheers,
Patrick