ERROR: invalid memory alloc request size 18446744073709551613

On Jun 22, 2020, at 8:13 AM, Flaris Feller <flaris.feller@gmail.com> wrote:

Hello all,

When using Postgres 9.6.15 on "CentOS Linux release 7.5.1804 (Core)" on Intel x86_64 I noticed "invalid memory alloc request size" error at PostgreSQL logs.
This is the postgresq.log file's fragment of log where the error was found.

2020-06-22 00:29:18 BRT [16987]: [1-1] db=bxs,user=postgres ERRO: invalid memory alloc request size 18446744073709551613
2020-06-22 00:29:18 BRT [16987]: [2-1] db=bxs,user=postgres COMANDO: COPY public.cham_chamada....

PostgreSQL was installed using official PGDG repository through yum.
This is the list of postgresql.conf settings used in the cluster.

listen_addresses = '*'
log_destination = 'stderr'
logging_collector = on
log_directory = 'pg_log'
log_filename = 'postgresql-%a.log'
log_truncate_on_rotation = on
log_rotation_age = 1d
log_rotation_size = 0
autovacuum = off
datestyle = 'iso, mdy'
lc_messages = 'pt_BR'
lc_monetary = 'pt_BR'
lc_numeric = 'pt_BR'
lc_time = 'pt_BR'
default_text_search_config = 'pg_catalog.portuguese'
max_connections=150
shared_buffers=2GB
effective_cache_size=4GB
work_mem=13981kB
maintenance_work_mem=256MB
log_min_duration_statement = 300000
log_line_prefix = '%t [%p]: [%l-1] db=%d,user=%u '
log_lock_waits = on
timezone='America/Sao_paulo'
log_timezone = 'Brazil/East'
min_wal_size = 1GB
max_wal_size = 2GB

Looking at previous list's messages I've found this could be a data corruption issue and I've followed the recommended procedures.
But even removing the corrupted records, doing a vacuum full and re-indexing the table the problem keep going recurrently.
So I would like some guidance to find the root cause of the table corruption on the database.
Any help would be appreciated.

Thanks in advance,
Flaris Feller.

I there an application generating this value perhaps?
1111111111111111111111111111111111111111111111111111111111111101
Looks a little like a signed/un-signed mismatch

On 2020-06-22 11:13:33 -0300, Flaris Feller wrote:

When using Postgres 9.6.15 on "CentOS Linux release 7.5.1804 (Core)" on Intel
x86_64 I noticed "invalid memory alloc request size" error at PostgreSQL logs.
This is the postgresq.log file's fragment of log where the error was found.

2020-06-22 00:29:18 BRT [16987]: [1-1] db=bxs,user=postgres ERRO:  invalid
memory alloc request size 18446744073709551613

That's 2**64 - 3. So probably the size of some object is (erroneously)
computed as -3 bytes.

2020-06-22 00:29:18 BRT [16987]: [2-1] db=bxs,user=postgres COMANDO:  COPY
public.cham_chamada....

Does this always happen in conjunction with a COPY command or sometimes
with other commands, too? If the former, are you copying into the
database or out of it?

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"

Hi Peter,

The error happens with SELECT command too and the COPY command was used to
extract data out of the database.

Thanks in advance,
Flaris Feller.

Em seg., 22 de jun. de 2020 às 12:33, Peter J. Holzer <hjp-pgsql@hjp.at>
escreveu:

On 2020-06-22 11:13:33 -0300, Flaris Feller wrote:

When using Postgres 9.6.15 on "CentOS Linux release 7.5.1804 (Core)" on

Intel

x86_64 I noticed "invalid memory alloc request size" error at PostgreSQL

logs.

This is the postgresq.log file's fragment of log where the error was

found.

2020-06-22 00:29:18 BRT [16987]: [1-1] db=bxs,user=postgres ERRO:

invalid

memory alloc request size 18446744073709551613

That's 2**64 - 3. So probably the size of some object is (erroneously)
computed as -3 bytes.

2020-06-22 00:29:18 BRT [16987]: [2-1] db=bxs,user=postgres COMANDO:

COPY

public.cham_chamada....

Does this always happen in conjunction with a COPY command or sometimes
with other commands, too? If the former, are you copying into the
database or out of it?

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"

--
Flaris Roland Feller
48-999811781
https://www.linkedin.com/in/flarisfeller

Flaris Feller <flaris.feller@gmail.com> writes:

Em seg., 22 de jun. de 2020 às 12:33, Peter J. Holzer <hjp-pgsql@hjp.at>
escreveu:

On 2020-06-22 11:13:33 -0300, Flaris Feller wrote:

When using Postgres 9.6.15 on "CentOS Linux release 7.5.1804 (Core)" on
Intel x86_64 I noticed "invalid memory alloc request size" error at PostgreSQL
logs.

That's 2**64 - 3. So probably the size of some object is (erroneously)
computed as -3 bytes.

Yeah. I don't recall the bit-level details offhand, but this is an
extremely common symptom of a corrupted length word in a variable-length
field.

The usual approach is to try to isolate which row or rows contains corrupt
data and then delete it. I'm not aware of any tools for doing that
automatically, but you can usually home in on a bad row by a process of
binary search, eg testing how many rows you can fetch without seeing the
error. Also "select * from mytab where ctid = '(m,n)'" is useful for
probing individual rows, once you get close to the problem area.

regards, tom lane

Hello all,

I’ve got successfully isolate and delete the corrupted rows at the table, but what would caused that? How to prevent it?

Thanks in advance,
Flaris.

Flaris R. Feller
48-999811781
flaris.feller@gmail.com
http://linkedin.com/in/flarisfeller

Hello Rob,

You mean, an insert or update carrying this value? No, it doesn’t.

Thanks,
Flaris.

Flaris R. Feller
flaris.feller@gmail.com
http://linkedin.com/in/flarisfeller

Hello Peter,

In this case, It was a COPY for a table to out of the database, but the error occurs with other commands like a SELECT.

Thanks in advance,
Flaris.

Flaris R. Feller
flaris.feller@gmail.com
http://linkedin.com/in/flarisfeller