Re: [HACKERS] pg_dump core dump, upgrading from 6.5b1 to 5/24 snapshot
Ari Halberstadt <ari@shore.net> writes:
Using pg_dump with 6.5b1 on solaris sparc crashes with a core dump. This
means I can't keep backups and I can't upgrade my data model without being
able to export the old data.
On Bruce's suggestion I tried upgrading to a recent snapshot (in this case
the one from 5/24). When I killed the old postmaster and started the new
postmaster I got the following error in psql:
Hi Ari,
I believe there's been at least one initdb-forcing change since 6.5b1,
so you're right, you can't run a newer postmaster until you have
extracted your data.
But I think what Bruce really had in mind was to run pg_dump out of
the latest snapshot against your 6.5b1 postmaster. That should work
as far as compatibility issues go. If we're really lucky, the bug has
been fixed since then --- if not, it'd be easiest to try to find it in
current sources anyway...
The coredump is happening in pg_dump, right, not in the connected
backend?
regards, tom lane
Import Notes
Reply to msg id not found: YourmessageofTue25May1999140359-0500v04003a05b370a37a10be@207.244.108.39
Tom Lane <tgl@sss.pgh.pa.us> wrote:
...
But I think what Bruce really had in mind was to run pg_dump out of
the latest snapshot against your 6.5b1 postmaster. That should work
Running the new pg_dump:
$ pg_dump -p 4001 -D -f bboard.out bboard
Segmentation Fault (core dumped)
The coredump is happening in pg_dump, right, not in the connected
backend?
Yes, it's pg_dump, not the backend.
-- Ari Halberstadt mailto:ari@shore.net <http://www.magiccookie.com/>
PGP public key available at <http://www.magiccookie.com/pgpkey.txt>
Ari Halberstadt <ari@shore.net> writes:
Running the new pg_dump:
$ pg_dump -p 4001 -D -f bboard.out bboard
Segmentation Fault (core dumped)
The coredump is happening in pg_dump, right, not in the connected
backend?
Yes, it's pg_dump, not the backend.
OK, could you see whether omitting -D and/or -f makes a difference?
regards, tom lane
Import Notes
Reply to msg id not found: YourmessageofTue25May1999152548-0500v04003a08b370b74db950@207.244.108.39 | Resolved by subject fallback
Tom Lane <tgl@sss.pgh.pa.us> wrote:
OK, could you see whether omitting -D and/or -f makes a difference?
It works with 6.5b1 and the 5/24 snapshot when omitting the -D. It crashes
with -D or -d. It works with -f. So,
pg_dump -f bboard.out bboard
works ok. Thanks for suggesting this.
-- Ari Halberstadt mailto:ari@shore.net <http://www.magiccookie.com/>
PGP public key available at <http://www.magiccookie.com/pgpkey.txt>
Import Notes
Resolved by subject fallback
Tom Lane <tgl@sss.pgh.pa.us> noted that MAXQUERYLEN's value in pg_dump is
5000. Some of my fields are the maximum length for a text field.
Using the 5/26 snapshot, I increased MAXQUERYLEN to 16384 and it completed
without crashing. I also tried it at 8192 but it still crashed at that size.
The dumped data file is 15MB (no -d or -D option) or 22MB (with -D). The
core file is 13.8MB, which sounds like a memory leak in pg_dump.
-- Ari Halberstadt mailto:ari@shore.net <http://www.magiccookie.com/>
PGP public key available at <http://www.magiccookie.com/pgpkey.txt>
Import Notes
Resolved by subject fallback
Ari Halberstadt <ari@shore.net> writes:
Tom Lane <tgl@sss.pgh.pa.us> noted that MAXQUERYLEN's value in pg_dump is
5000. Some of my fields are the maximum length for a text field.
There are two bugs here: dumpClasses_dumpData() should not be making any
assumption at all about the maximum size of a tuple field, and pg_dump's
value for MAXQUERYLEN ought to match the backend's. I hadn't realized
that it wasn't using the same query buffer size as the backend does ---
this might possibly explain some other complaints we've seen about being
unable to dump complex table or rule definitions.
Will fix both problems this evening.
The dumped data file is 15MB (no -d or -D option) or 22MB (with -D). The
core file is 13.8MB, which sounds like a memory leak in pg_dump.
Not necessarily --- are the large text fields in a multi-megabyte table?
When you're using -D, pg_dump just does a "SELECT * FROM table" and then
iterates through the returned result, which must hold the whole table.
(This is another reason why I prefer not to use -d/-D ... the COPY
method doesn't require buffering the whole table inside pg_dump.)
Some day we should enhance libpq to allow a select result to be received
and processed in chunks smaller than the whole result.
regards, tom lane
Import Notes
Reply to msg id not found: YourmessageofWed26May1999131243-0500v04003a03b371e9ff05ad@207.244.108.97 | Resolved by subject fallback
Tom Lane <tgl@sss.pgh.pa.us> wrote:
...
Will fix both problems this evening.
Thanks!
The dumped data file is 15MB (no -d or -D option) or 22MB (with -D). The
core file is 13.8MB, which sounds like a memory leak in pg_dump.Not necessarily --- are the large text fields in a multi-megabyte table?
Yes, it's a 15MB file for the table.
When you're using -D, pg_dump just does a "SELECT * FROM table" and then
iterates through the returned result, which must hold the whole table.
(This is another reason why I prefer not to use -d/-D ... the COPY
method doesn't require buffering the whole table inside pg_dump.)
The -d/-D options are out now for my nightly backups. (Foolish of me to
have used them with backups in the first place!)
-- Ari Halberstadt mailto:ari@shore.net <http://www.magiccookie.com/>
PGP public key available at <http://www.magiccookie.com/pgpkey.txt>