info on strange error messages on postgresql

Joe Maldonado <jmaldonado@webehosting.biz> writes:

I have these messages on my 7.4.7 database log...
TopMemoryContext: 87494704 total in 10676 blocks; 179400 free (61
chunks); 87315304 used
TopTransactionContext: 57344 total in 3 blocks; 648 free (5 chunks);
56696 used
DeferredTriggerXact: 0 total in 0 blocks; 0 free (0 chunks); 0 used
...

What's at the top and bottom of that?

PG prints out a memory stats dump like this when it runs out of memory.
The dump itself isn't much use to anyone but a developer; what you want
to look into is what triggered it. The error message appearing just
after (or maybe just before, I forget) should be relevant.

followed by about 16GB of the following type of entries...
pg_temp_1486707494: 2048 total in 1 blocks; 768 free (0 chunks); 1280 used

Could you have been trying to vacuum a ridiculously large number of
tables, or some such?

regards, tom lane

Tom Lane wrote:

Joe Maldonado <jmaldonado@webehosting.biz> writes:

I have these messages on my 7.4.7 database log...
TopMemoryContext: 87494704 total in 10676 blocks; 179400 free (61
chunks); 87315304 used
TopTransactionContext: 57344 total in 3 blocks; 648 free (5 chunks);
56696 used
DeferredTriggerXact: 0 total in 0 blocks; 0 free (0 chunks); 0 used
...

What's at the top and bottom of that?

Above this there is just a few expected error messages about a table
already existing, unfortunately this log happened between logrotates and
the log filled the logs partition to 100% so I do not have the post dump
messages.

The last entry in the log is ...
pg_temp_1486707604: 1024 total in 1 blocks; 640 free

PG prints out a memory stats dump like this when it runs out of memory.
The dump itself isn't much use to anyone but a developer; what you want
to look into is what triggered it. The error message appearing just
after (or maybe just before, I forget) should be relevant.

followed by about 16GB of the following type of entries...
pg_temp_1486707494: 2048 total in 1 blocks; 768 free (0 chunks); 1280 used

Could you have been trying to vacuum a ridiculously large number of
tables, or some such?

At first I suspected vacuum to be the culprit of the 16G of data though
this db is being vacuumed by pg_autovacuum with (-s 900 -d 3 -S 0 -A 0
-V 0) as options and it has recently undergone a vacuum full and reindex
operation.

regards, tom lane

One other data point is that the shared buffers are very high
max_connections = 512
shared_buffers = 100000

though there are no other high memory consumers on that machine and it
has 4G of physical RAM

-Joe

What's at the top and bottom of that?

PG prints out a memory stats dump like this when it runs out of memory.
The dump itself isn't much use to anyone but a developer; what you want
to look into is what triggered it. The error message appearing just
after (or maybe just before, I forget) should be relevant.

Sorry for the really late response but here is the bottom of one of
these...

MdSmgr: 8192 total in 1 blocks; 6096 free (20 chunks); 2096 used
DynaHash: 8192 total in 1 blocks; 6928 free (0 chunks); 1264 used
DynaHashTable: 24576 total in 2 blocks; 14280 free (6 chunks); 10296 used
DynaHashTable: 8192 total in 1 blocks; 5080 free (0 chunks); 3112 used
DynaHashTable: 8192 total in 1 blocks; 2008 free (0 chunks); 6184 used
DynaHashTable: 8192 total in 1 blocks; 952 free (0 chunks); 7240 used
DynaHashTable: 8192 total in 1 blocks; 3000 free (0 chunks); 5192 used
DynaHashTable: 24576 total in 2 blocks; 9136 free (4 chunks); 15440 used
DynaHashTable: 0 total in 0 blocks; 0 free (0 chunks); 0 used
DynaHashTable: 0 total in 0 blocks; 0 free (0 chunks); 0 used
DynaHashTable: 0 total in 0 blocks; 0 free (0 chunks); 0 used
DynaHashTable: 0 total in 0 blocks; 0 free (0 chunks); 0 used
DynaHashTable: 0 total in 0 blocks; 0 free (0 chunks); 0 used
ErrorContext: 8192 total in 1 blocks; 8176 free (3 chunks); 16 used
2005-04-21 22:30:47 [21315] ERROR: out of memory
DETAIL: Failed on request of size 268.
2005-04-21 22:30:47 [21315] ERROR: current transaction is aborted,
commands ignored until end of transac
tion block
2005-04-21 22:30:47 [21315] ERROR: current transaction is aborted,
commands ignored until end of transac
tion block
2005-04-21 22:30:47 [21315] ERROR: current transaction is aborted,
commands ignored until end of transac
tion block
2005-04-21 22:30:47 [21315] ERROR: current transaction is aborted,
commands ignored until end of transac
tion block
2005-04-21 22:30:47 [21315] ERROR: current transaction is aborted,
commands ignored until end of transac
tion block
2005-04-21 22:30:47 [21315] ERROR: current transaction is aborted,
commands ignored until end of transac
tion block
2005-04-21 22:30:47 [21315] ERROR: current transaction is aborted,
commands ignored until end of transac
tion block
2005-04-21 22:30:47 [21315] ERROR: current transaction is aborted,
commands ignored until end of transac
tion block
2005-04-21 22:30:47 [21315] ERROR: current transaction is aborted,
commands ignored until end of transac
tion block

we're suspecting pg_autovacuum as being the culprit but cannot nail it
down quite yet.

-Joe