BUG #5959: pg_restore --use-list does not always work with a pg_restore generated TOC file

"Per-Olov Esgard" <Per-Olov.Esgard@micronic-mydata.com> writes:

Description: pg_restore --use-list does not always work with a
pg_restore generated TOC file
Details:

General
=======
In my application I am using automatically generated psm:s. These psm: are
using many arguments, in some cases over 80. This means that a TOC entry in
a TOC file generated by pg_restore --list may be very long for psm:s with
such a long signature.

Specific
========
My backup system makes a schema dump to a file and a data dump to another
file. When performing a restore I create a TOC list from the schema dump by
using pg_restore on the schema dump file. I do some filtering on the TOC
file and then I restore it using pg_restore --use-list. This does not work
since some entries in the TOC file are more than 2000 characters long. The
function SortTocFromFile in pg_backup_archiver.c has a hard coded limitation
of 1024 characters for each row in the TOC file.

Hm. The function only cares about the dump ID at the start of the line.
AFAICS the consequence of buffer overflow would be that it'd take a line
continuation as a new line; which would generally result in a harmless
"WARNING: line ignored" message. You didn't say exactly what symptom
you were seeing, but "does not work" seems like a bit of an
overstatement for that. So I'm wondering whether you're seeing some
behavior I'm missing.

regards, tom lane

I wrote:

Per-Olov Esgard <Per-Olov.Esgard@micronic-mydata.com> writes:

The third extracted buffer is not ok but interpreted as ok with id 10:
10."boolean", mydbcomndata_10."boolean", mydbcomndata_10."boolean",
mydbcomndata_10."boolean", mydbcomndata_10."boolean", character varying,
mydbcomndata_10."integer", mydbcomndata_10.componentname,
mydbcomndata_10."integer", mydbcomndata_10."integer",
mydbcomndata_10."integer", mydbcomndata_10.angle,
mydbcomndata_10."boolean", mydbcomndata_10."boolean",
mydbcomndata_10."boolean", mydbcomndata_10."boolean",
mydbcomndata_10."boolean", mydbcomndata_10."boolean",
mydbcomndata_10."boolean") postgres

So as you can see, the last part of my schema name consists of a number
that is interpreted as an id.

Ah, I see. So what we probably need to do for a real fix is to teach
that code to distinguish continuation lines from normal ones.

Fix committed --- thanks for the report!

regards, tom lane