BUG #17156: pg_restore: [custom archiver] WARNING: ftell mismatch with expected position -- ftell used

On Mon, Aug 23, 2021 at 7:31 PM PG Bug reporting form
<noreply@postgresql.org> wrote:

The following bug has been logged on the website:

Bug reference: 17156
Logged by: Ashutosh Kumar
Email address: ashutosh.kumar@morningstar.com
PostgreSQL version: 10.18
Operating system: RHEL
Description:

Getting below warning when I am restoring the backup file :

Warning Message : pg_restore: [custom archiver] WARNING: ftell mismatch with
expected position -- ftell used

Backup was taken with below command :

pg_dump -h'127.0.0.1' -p 5432 -U (username) -Fc databasename >
backupfile.sql

I am restoring it with pg_restore when i got the above warning.

Could you share the exact pg_restore command with the arguments you executed?

What's is the size of the backup? Are both Postgres servers that you
took/restored the backup from/to are RHEL?

Could you try to execute pg_dump without a pipe and restore it and see
the same error happens? That is, please execute "pg_dump -h'127.0.0.1'
-p 5432 -U (username) -Fc databasename -f backupfile.sql".

Regards,

--
Masahiko Sawada
EDB: https://www.enterprisedb.com/

At Tue, 24 Aug 2021 14:25:50 +0900, Masahiko Sawada <sawada.mshk@gmail.com> wrote in

On Mon, Aug 23, 2021 at 7:31 PM PG Bug reporting form
<noreply@postgresql.org> wrote:

The following bug has been logged on the website:

Bug reference: 17156
Logged by: Ashutosh Kumar
Email address: ashutosh.kumar@morningstar.com
PostgreSQL version: 10.18
Operating system: RHEL
Description:

Getting below warning when I am restoring the backup file :

Warning Message : pg_restore: [custom archiver] WARNING: ftell mismatch with
expected position -- ftell used

Backup was taken with below command :

pg_dump -h'127.0.0.1' -p 5432 -U (username) -Fc databasename >
backupfile.sql

I am restoring it with pg_restore when i got the above warning.

Could you share the exact pg_restore command with the arguments you executed?

What's is the size of the backup? Are both Postgres servers that you
took/restored the backup from/to are RHEL?

Could you try to execute pg_dump without a pipe and restore it and see
the same error happens? That is, please execute "pg_dump -h'127.0.0.1'
-p 5432 -U (username) -Fc databasename -f backupfile.sql".

That error means when reading a seekable source, the file pointer
ftello returns doesn't consistent with how many bytes pg_restore has
read, or ftello returned an error. Considering that the path is very
ancient, the latter likely to happen. If the failure is reproducible,
running pg_restore with strace could show something helpful.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

Restore Command :

On Prem backup(linux RHEL) to RDS PostgreSQL

Backup size around : 6.4 GB(on prem server which was restored on RDS postgres when I received the warning)

Thanks,
Ashu

-----Original Message-----
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Sent: Tuesday, August 24, 2021 2:12 PM
To: sawada.mshk@gmail.com
Cc: Ashutosh Kumar <Ashutosh.Kumar@morningstar.com>; pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #17156: pg_restore: [custom archiver] WARNING: ftell mismatch with expected position -- ftell used

[EXTERNAL EMAIL]

At Tue, 24 Aug 2021 14:25:50 +0900, Masahiko Sawada <sawada.mshk@gmail.com<mailto:sawada.mshk@gmail.com>> wrote in

On Mon, Aug 23, 2021 at 7:31 PM PG Bug reporting form
<noreply@postgresql.org<mailto:noreply@postgresql.org>> wrote:

The following bug has been logged on the website:

Bug reference: 17156
Logged by: Ashutosh Kumar
Email address: ashutosh.kumar@morningstar.com<mailto:ashutosh.kumar@morningstar.com>
PostgreSQL version: 10.18
Operating system: RHEL
Description:

Getting below warning when I am restoring the backup file :

Warning Message : pg_restore: [custom archiver] WARNING: ftell
mismatch with expected position -- ftell used

Backup was taken with below command :

pg_dump -h'127.0.0.1' -p 5432 -U (username) -Fc databasename >
backupfile.sql

I am restoring it with pg_restore when i got the above warning.

Could you share the exact pg_restore command with the arguments you executed?

What's is the size of the backup? Are both Postgres servers that you
took/restored the backup from/to are RHEL?

Could you try to execute pg_dump without a pipe and restore it and see
the same error happens? That is, please execute "pg_dump -h'127.0.0.1'
-p 5432 -U (username) -Fc databasename -f backupfile.sql".

That error means when reading a seekable source, the file pointer ftello returns doesn't consistent with how many bytes pg_restore has read, or ftello returned an error. Considering that the path is very ancient, the latter likely to happen. If the failure is reproducible, running pg_restore with strace could show something helpful.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

On Tue, Aug 24, 2021 at 5:42 PM Kyotaro Horiguchi
<horikyota.ntt@gmail.com> wrote:

At Tue, 24 Aug 2021 14:25:50 +0900, Masahiko Sawada <sawada.mshk@gmail.com> wrote in

On Mon, Aug 23, 2021 at 7:31 PM PG Bug reporting form
<noreply@postgresql.org> wrote:

The following bug has been logged on the website:

Bug reference: 17156
Logged by: Ashutosh Kumar
Email address: ashutosh.kumar@morningstar.com
PostgreSQL version: 10.18
Operating system: RHEL
Description:

Getting below warning when I am restoring the backup file :

Warning Message : pg_restore: [custom archiver] WARNING: ftell mismatch with
expected position -- ftell used

Backup was taken with below command :

pg_dump -h'127.0.0.1' -p 5432 -U (username) -Fc databasename >
backupfile.sql

I am restoring it with pg_restore when i got the above warning.

Could you share the exact pg_restore command with the arguments you executed?

What's is the size of the backup? Are both Postgres servers that you
took/restored the backup from/to are RHEL?

Could you try to execute pg_dump without a pipe and restore it and see
the same error happens? That is, please execute "pg_dump -h'127.0.0.1'
-p 5432 -U (username) -Fc databasename -f backupfile.sql".

That error means when reading a seekable source, the file pointer
ftello returns doesn't consistent with how many bytes pg_restore has
read, or ftello returned an error. Considering that the path is very
ancient, the latter likely to happen.

If ftello returned an error, i.g., -1, wouldn't pg_restore die instead
of emitting the warning? As Tom mentioned[1]/messages/by-id/3172545.1594845659@sss.pgh.pa.us, we have fseeko() calls
in some code paths but don't correct ctx->filePos afterward. This is
also likely to cause this warning.

FIWI I'd like to note that this warning is meant to be for debugging
purposes as the comment in the source code mentioned. Since if the
input source is seekable we rely on ftello() to get the current
position in the file the restoration should be performed even if we
got this warning. Actually, we removed manual tracking of file
position by commit a8d0732ac, which backpatched to v12.

Regards,

[1]: /messages/by-id/3172545.1594845659@sss.pgh.pa.us

--
Masahiko Sawada
EDB: https://www.enterprisedb.com/