Remove toast_max_chunk_size from control file

Started by Michael Paquier3 months ago4 messageshackers
Jump to latest
#1Michael Paquier
michael@paquier.xyz

Hi all,

Please find attached a patch for $subject, which is something that I
had on my stack of things to look at for some time now.

My main argument regarding the removal of toast_max_chunk_size in the
control file is that it is a redundant check, due to the fact that the
definition of TOAST_OID_MAX_CHUNK_SIZE is tied to two fields that we
already track in the control file:
- BLCKSZ
- MAXALIGN

Then, I have this table, that shows the value of the max_chunk_size
depending on both parameters, for all MAXALIGN and block sizes
supported,assuming my maths are right:
BLCKSZ MAXALIGN 4_byte_id 8_byte_id
1kB 4 208 204
1kB 8 204 200
2kB 4 464 460
2kB 8 460 456
4kB 4 976 972
4kB 8 972 968
8kB 4 2000 1996
8kB 8 1996 1992
16kB 4 4048 4044
16kB 8 4044 4040
32kB 4 8144 8140
32kB 8 8140 8136

8_byte_id is assuming a OID8 TOAST value, but look just at the
4_byte_id column for the existing OID case. That's where I can see
that if either BLCKSZ or MAXALIGN is different, we would fail the
early validity checks on a cluster if trying to copy a data folder
with an incompatible set of any of (BLCKSZ,MAXALIGN), without
max_chunk_size interfering at all. In terms of pg_upgrade, we check
for got_align and got_blocksz (see pg_upgrade/controldata.c).

I won't hide that this removal offers extra benefits for the other
work I am doing now for TOAST with more external pointer types,
because it makes the max_chunk_size kind of irrelevant anyway at
cluster level, but freeing bytes from the control file is super nice
as a change of its own, because it's more bytes for more useful things
in the future. (Spoiler: I'd need these 4 bytes myself, but that's a
separate discussion.)

There may be an argument about somebody enforcing a new
TOAST_OID_MAX_CHUNK_SIZE or EXTERN_TUPLES_PER_PAGE manually, of
course, but while we claim that tweaks are possible in heaptoast.h,
I've never seen that as an officially-supported documented option
(right?), and I've never heard somebody actually doing that. So I see
nothing that prevents this removal from the control file?

Comments or opinions? Feel free to point out anything I may be
missing, of course..
--
Michael

Attachments:

t253695_1
0001-Remove-toast_max_chunk_size-from-pg_control.patchtext/plain; charset=us-asciiDownload+11-61
#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Michael Paquier (#1)
Re: Remove toast_max_chunk_size from control file

Michael Paquier <michael@paquier.xyz> writes:

My main argument regarding the removal of toast_max_chunk_size in the
control file is that it is a redundant check, due to the fact that the
definition of TOAST_OID_MAX_CHUNK_SIZE is tied to two fields that we
already track in the control file:
- BLCKSZ
- MAXALIGN

I don't think this follows. heaptoast.h says:

#define EXTERN_TUPLES_PER_PAGE 4 /* tweak only this */

#define EXTERN_TUPLE_MAX_SIZE MaximumBytesPerTuple(EXTERN_TUPLES_PER_PAGE)

#define TOAST_MAX_CHUNK_SIZE \
(EXTERN_TUPLE_MAX_SIZE - \
MAXALIGN(SizeofHeapTupleHeader) - \
sizeof(Oid) - \
sizeof(int32) - \
VARHDRSZ)

EXTERN_TUPLES_PER_PAGE is a free variable here, and it's entirely
possible that someone would wish to tweak it. So I disagree that
it's safe to remove this value from pg_control.

regards, tom lane

#3Michael Paquier
michael@paquier.xyz
In reply to: Tom Lane (#2)
Re: Remove toast_max_chunk_size from control file

On Sun, Sep 06, 2026 at 09:25:34PM -0400, Tom Lane wrote:

EXTERN_TUPLES_PER_PAGE is a free variable here, and it's entirely
possible that someone would wish to tweak it. So I disagree that
it's safe to remove this value from pg_control.

Requoting my previous message that you did not include:

There may be an argument about somebody enforcing a new
TOAST_OID_MAX_CHUNK_SIZE or EXTERN_TUPLES_PER_PAGE manually, of
course, but while we claim that tweaks are possible in heaptoast.h,

I've never seen anybody actually do that. So yes, I agree that if
somebody has the idea to change this value manually, the control file
value makes sense to keep. It's just that I doubt that anybody is
actually doing that. :)
--
Michael

#4Tom Lane
tgl@sss.pgh.pa.us
In reply to: Michael Paquier (#3)
Re: Remove toast_max_chunk_size from control file

Michael Paquier <michael@paquier.xyz> writes:

Requoting my previous message that you did not include:

There may be an argument about somebody enforcing a new
TOAST_OID_MAX_CHUNK_SIZE or EXTERN_TUPLES_PER_PAGE manually, of
course, but while we claim that tweaks are possible in heaptoast.h,

I've never seen anybody actually do that. So yes, I agree that if
somebody has the idea to change this value manually, the control file
value makes sense to keep. It's just that I doubt that anybody is
actually doing that. :)

Perhaps not, but the point of these disk-layout-compatibility fields
in pg_control is to keep people from shooting themselves in the foot.
I don't think it's that unlikely that someone would decide that four
TOAST tuples per page isn't ideal, especially if they were running
with non-default BLCKSZ.

regards, tom lane