pg_basebackup failing

On Wed, Jan 15, 2014 at 6:53 AM, Alan Nilsson <anilsson@apple.com> wrote:

Could someone give me some insight to the following error message:

[mqsql06:/Volumes/SQL_Set] _postgres% pg_basebackup --user=replicate
--host=mqsql03 -xP -Fp --pgdata=pgsql
19439890/65873894 kB (29%), 1/1 tablespace
pg_basebackup: could not get transaction log end position from server:
ERROR: archive member "base/17040/25491" too large for tar format

I find the error message confusing since I have the format explicitly set to
plain.

pg_basebackup fetches a full backup from server using tar all the time
for performance purposes to reduce the amount of data sent through the
stream. pg_basebackup contains some code to decode this tarball and
generate a plain format from it.
The error you are seeing is triggered because this relation file
exceeds MAX_TAR_MEMBER_FILELEN or 8GB for a single tar member, which
is as well the norm for tar.

Is there a solution/workaround?

Not use a pg_basebackup but an external backup solution or a custom
script. Break this relation file into several pieces by doing some
partitions on it.

Can I break up the referenced file somehow?

Partitions on the parent table.
Regards,
--
Michael

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

The error you are seeing is triggered because this relation file
exceeds MAX_TAR_MEMBER_FILELEN or 8GB for a single tar member, which
is as well the norm for tar.

I thought PostgreSQL would break the file if it grows beyond 1GB (1GB is
segment size which one can modify while compiling). Am I missing something
or have I got it totally wrong?

Is there a solution/workaround?
Not use a pg_basebackup but an external backup solution or a custom
script. Break this relation file into several pieces by doing some
partitions on it.

Can I break up the referenced file somehow?

Partitions on the parent table.

Which might have other implications on the performance if the queries do
not use Partition key in where clause.

Best Regards,
*Sameer Kumar | Database Consultant*

*ASHNIK PTE. LTD. *101 Cecil Street, #11-11 Tong Eng Building, Singapore
069533
M : *+65 8110 0350* T: +65 6438 3504 | www.ashnik.com
www.facebook.com/ashnikbiz | www.twitter.com/ashnikbiz

[image: email patch]

This email may contain confidential, privileged or copyright material and
is solely for the use of the intended recipient(s).

On Jan 15, 2014 12:07 PM, "Sameer Kumar" <sameer.kumar@ashnik.com> wrote:

The error you are seeing is triggered because this relation file
exceeds MAX_TAR_MEMBER_FILELEN or 8GB for a single tar member, which
is as well the norm for tar.

I thought PostgreSQL would break the file if it grows beyond 1GB (1GB is

segment size which one can modify while compiling). Am I missing something
or have I got it totally wrong?

Yeah, how did you actually end up with a file that size? Do you have any
non standard compiler options set when you built your server?

/Magnus

On Wed, Jan 15, 2014 at 8:05 PM, Sameer Kumar <sameer.kumar@ashnik.com>wrote:

The error you are seeing is triggered because this relation file

exceeds MAX_TAR_MEMBER_FILELEN or 8GB for a single tar member, which
is as well the norm for tar.

I thought PostgreSQL would break the file if it grows beyond 1GB (1GB is
segment size which one can modify while compiling). Am I missing something
or have I got it totally wrong?

No, you are missing nothing. Thanks for reminding :) (This is settable with
./configure --with-segsize)
So the build has been done with some uncommon option.
Regards,
--
Michael

Thanks all

alan

On Jan 15, 2014, at 6:30 AM, Michael Paquier <michael.paquier@gmail.com> wrote: