pg_stat_statements: improve loading and saving routines for the dump file
Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.
You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:
docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t50924psql -h localhost -U postgresBuilt from patchset v3 (message #3), July 27, 2026 at 10:25 AM.
Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:
git clone --branch t50924_3 https://github.com/hackorum-dev/postgres.gitIn a checkout you already have, add the fork once:
git remote add hackorum https://github.com/hackorum-dev/postgres.gitthen, for this patchset and every later one:
git fetch hackorum t50924_3 && git checkout t50924_3Patchset v3 (message #3) is on t50924_3
Hi!
Currently in pg_stat_statements save/load routines the whole pgssEntry
entity data are written/read with its last field
slock_t mutex;
which is actually not used then.
This small patch fixes this issue. Hope, it will be useful.
Respectfully,
Mikhail Litsarev,
Postgres Professional: https://postgrespro.com
Attachments:
v1-0001-Exclude-pgssEntry-mutex-for-rw-dump-file.patchtext/x-diff; name=v1-0001-Exclude-pgssEntry-mutex-for-rw-dump-file.patchDownload+7-5
Hello, Mikhail.
1) I'd add to comment a reason, why mutex should be last.
// Mutex should be last field, as this field isn't copied to dump file
+ /* protects the counters only. Should be the very last field, as this field isn't copied to dump file
+ slock_t mutex;
} pgssEntry;2) You didn't take into account the upgrade. Saved in Postgres with this
byte and try to load in version without this byte.
On 1/20/25 16:49, m.litsarev@postgrespro.ru wrote:
Hi!
Currently in pg_stat_statements save/load routines the whole pgssEntry
entity data are written/read with its last field
slock_t mutex;
which is actually not used then.
This small patch fixes this issue. Hope, it will be useful.Respectfully,
Mikhail Litsarev,
Postgres Professional: https://postgrespro.com
--
Best wishes,
Ivan Kush
Tantor Labs LLC
// Mutex should be last field, as this field isn't copied to dump file
Updated.
2) You didn't take into account the upgrade. Saved in Postgres with
this byte and try to load in version without this byte.
The PGSS_DUMP_FILE format is defined by PGSS_FILE_HEADER magic number
(the first four bytes in the dump file). Once they does not coinside
(when the file is being read) the statistics are not loaded from the
dump file and set to zero and the irrelevant dump file is skipped. This
is the actual behaviour implemented in the code.
Respectfully,
Mikhail Litsarev,
Postgres Professional: https://postgrespro.com
What does this patch give on aglobal scale? Does it save much memory or
increase performance? How many times?
On 1/21/25 13:51, m.litsarev@postgrespro.ru wrote:
// Mutex should be last field, as this field isn't copied to dump file
Updated.
2) You didn't take into account the upgrade. Saved in Postgres with
this byte and try to load in version without this byte.The PGSS_DUMP_FILE format is defined by PGSS_FILE_HEADER magic number
(the first four bytes in the dump file). Once they does not coinside
(when the file is being read) the statistics are not loaded from the
dump file and set to zero and the irrelevant dump file is skipped.
This is the actual behaviour implemented in the code.Respectfully,
Mikhail Litsarev,
Postgres Professional: https://postgrespro.com
--
Best wishes,
Ivan Kush
Tantor Labs LLC
This will only reduce the size of the
$PGDATA/pg_stat/pg_stat_statements.txt file. Even with
100k entries, the most I have seen pg_stat_statements.max
set to, that will be less than 1 MB of disk saving. The default
config of 5k entries will be much less.
Regards,
Sami
What does this patch give on aglobal scale? Does it save much memory or
increase performance? How many times?
This patch makes the code semantically more correct and we don't lose
anything. It is obviously not about performance or memory optimisation.
This will only reduce the size of the
$PGDATA/pg_stat/pg_stat_statements.txt file. Even with
100k entries, the most I have seen pg_stat_statements.max
set to, that will be less than 1 MB of disk saving. The default
config of 5k entries will be much less.
Perfectly agree. I would just add that statistics are dropped into the
pg_stat_statements.txt file at server stop which is not a very frequent
event.
Respectfully,
Mikhail Litsarev,
Postgres Professional: https://postgrespro.com
m.litsarev@postgrespro.ru writes:
What does this patch give on aglobal scale? Does it save much memory or
increase performance? How many times?
This patch makes the code semantically more correct and we don't lose
anything. It is obviously not about performance or memory optimisation.
Indeed not: on my machine I see sizeof(pgssEntry) = 432. It's full of
int64 fields, so the alignment requirement is 8 bytes, meaning that
the mutex field accounts for 8 bytes even though it's likely just 1
or 4 bytes wide. Still, that means what you suggest is only going
to save 8/432 = 1.8% of the on-disk size of the struct. Given that
we also store the SQL query text for each entry, the net savings
fraction is even smaller.
I don't really agree that this is adding any semantic correctness
either. If we were reading directly into a live hashtable entry,
overwriting the mutex field could indeed be bad. But the fread()
call is reading into a local variable "temp" and we only copy
selected fields out of that. So it's just some bytes in a
transient stack entry.
On the whole therefore, I'm inclined to reject this on several
grounds:
* The savings is not worth the costs of bumping the stats file format
magic number (and thereby invalidating everyone's stored statistics).
* I'd rather keep the flexibility to put the mutex wherever we want in
the struct. Right now that isn't worth much either, but depending on
what fields get added in future, we might have the opportunity to move
the mutex to someplace where it fits into padding space and hence adds
nothing to sizeof(pgssEntry).
* Adding the requirement on where the mutex is makes the code more
fragile, since somebody might ignore the comment and place things
incorrectly. I'd like to think that such an error would be spotted
immediately, but we've committed sillier mistakes. The consequences
would likely be that some fields don't get stored/reloaded correctly,
which might escape notice for awhile.
regards, tom lane
Hi!
Thank you for detailed explanations.
Respectfully,
Mikhail Litsarev,
Postgres Professional: https://postgrespro.com