doc: Document Linux cgroup memory limits

Hi João,

Thanks for writing this up. I tried it on a cgroup v2 host (kernel
7.2, systemd 262), running master in a systemd scope with
MemoryMax=400M and no swap.

The main behaviour checks out. With four sessions sorting with
work_mem = 300MB, one process was SIGKILLed, the postmaster
reinitialized, every session lost its connection, and nobody got an
out-of-memory error. Reading a 638 MB table twice took usage up to
memory.max without any kill, since the page cache was reclaimed.

Two things I would change:

Memory allocated from the huge page pool (see huge_pages) is by
default not counted towards the memory usage of the cgroup.

That is the kernel default, but systemd 259 and later mount cgroup2
with memory_hugetlb_accounting [1]https://github.com/systemd/systemd/blob/main/NEWS (CHANGES WITH 259), and then it is counted. Here,
with shared_buffers = 256MB and huge_pages = on, memory.stat showed
290MB under hugetlb. Perhaps say that whether it counts depends on
that mount option.

If that is a server process other than the postmaster, the
postmaster treats it as a crash

With memory.oom.group set to 1, the OOM killer kills every process in
the cgroup, the postmaster included. There is no crash recovery then,
and nothing reaches the server log: the last line was "database system
is ready to accept connections". Kubernetes sets memory.oom.group for
containers on cgroup v2 unless singleProcessOOMKill is enabled [2]https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/ (singleProcessOOMKill), so
this is probably the case most container users will see. A sentence
about it would help.

One observation, maybe not for the docs: the first process killed was
not one of the sorting backends but an io worker, with about 90MB of
shared buffers mapped and almost no private memory. The OOM killer
counts mapped shared memory, so the server log named the io worker
rather than the sessions that used the memory.

[1]: https://github.com/systemd/systemd/blob/main/NEWS (CHANGES WITH 259)
[2]: https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/ (singleProcessOOMKill)
(singleProcessOOMKill)

Regards,
Manu

Hi,

Here is v2, with these changes based on Manu's testing:

- the huge pages accounting now depends on memory_hugetlb_accounting
instead of being described as off by default
- a new paragraph covers memory.oom.group, where the whole cgroup
(postmaster included) is killed and nothing is logged

Thanks again for the review.

Regards,
João Marcelo

Em sáb., 3 de out. de 2026 às 00:08, Manu <manuelreyesbravo@gmail.com>
escreveu:

Hi João,

Thanks for v2. It applies to master, the docs build, and the
hugetlb sentence matches what I see here (systemd 262 mounts cgroup2
with memory_hugetlb_accounting).

One wording point:

In that case there is no crash recovery, and the server stops without
logging anything.

What goes away is the automatic restart, not the recovery: after a
SIGKILL of the whole process tree, which is what memory.oom.group
does, the next start logs "database system was not properly shut
down; automatic recovery in progress" and replays WAL as usual.
Perhaps:

In that case the server is not restarted automatically, and nothing
is logged about the termination.

With that, it looks ready for committer to me.

Regards,
Manu

Hi Manu,

Thanks for testing v2 and for the wording. You're right, what goes away
is the automatic restart, not the recovery. v3 attached with your
suggested sentence, slightly reworded to say the server doesn't log the
termination.

Regards,
João Marcelo

Em dom., 4 de out. de 2026 às 20:42, Manu <manuelreyesbravo@gmail.com>
escreveu:

Hi João,

Thanks for v3. The new sentence is right, and the docs build. Going
through the whole section once more, two cases behave differently
from what it says; I should have raised them with v2.

If that is a server process other than the postmaster, the
postmaster treats it as a crash and, unless restart_after_crash is
turned off, restarts all server processes, disconnecting all sessions.

Not when the server runs as a systemd service, which is a common way
to set a memory limit outside containers. systemd's OOMPolicy applies
too, and its default, stop, stops the whole service after an OOM kill.
With MemoryMax=400M and no OOMPolicy set, two of three runs ended with
"received smart shutdown request" and the unit in the oom-kill failed
state, and the third was already deactivating; with
OOMPolicy=continue, the postmaster restarted all three times. Perhaps
add:

If the server runs as a systemd service, systemd's OOMPolicy setting
also applies; its default, stop, stops the whole service instead.

If memory.oom.group is set for the cgroup, [...] the OOM killer
instead terminates all processes of the cgroup, including the
postmaster.

The kernel exempts processes whose oom_score_adj is -1000 from the
group kill, and that is what linux-memory-overcommit recommends for
the postmaster. With the postmaster at -1000 and memory.oom.group set
to 1, the group kill happened but the postmaster survived and
restarted the other processes, in three runs out of three. Inside a
container the postmaster usually cannot lower its own score, so the
Kubernetes case stays as you describe. Perhaps:

... the OOM killer instead terminates all processes of the cgroup
except those with an oom_score_adj of -1000, so the postmaster too,
unless it is protected as described in
<xref linkend="linux-memory-overcommit"/>.

Regards,
Manu

Hi Manu,

Thanks for the quick response. Attached v4 with both points: the systemd
OOMPolicy note, and the oom_score_adj -1000 exemption for the group
kill, with a pointer to the overcommit section. I also resolved the
ambiguity
that arose after this new insertion.

Regards,
João Marcelo

Em dom., 4 de out. de 2026 às 22:19, Manu <manuelreyesbravo@gmail.com>
escreveu:

Hi João,

Attached v4 with both points: the systemd OOMPolicy note, and the
oom_score_adj -1000 exemption for the group kill [...] I also resolved
the ambiguity that arose after this new insertion.

Both sentences match what I measured, and the "if the postmaster is
terminated" conditional resolves the ambiguity cleanly. It applies to
master and the docs build. I've marked it Ready for Committer.

Thanks for the quick turnaround.

Regards,
Manu

On Mon, Oct 5, 2026 at 4:17 AM Joao Detomini
<joao.detomini@enterprisedb.com> wrote:

Hi Manu,

Thanks for the quick response. Attached v4 with both points: the systemd
OOMPolicy note, and the oom_score_adj -1000 exemption for the group
kill, with a pointer to the overcommit section. I also resolved the ambiguity
that arose after this new insertion.

+1 to the documenting this, but it's more than just set couple of GUCs
to avoid OOM:

We are kind of weak on self-limiting memory usage (there's new leak every
now and then, and people are complaining about work_mem not working). IMHO
running without vm.overcommit_memory=2 and without rotating connections
every couple hours/days [to release leaks/fat from free() caching by e.g.
glibc [1]/messages/by-id/CA+T=_GV-igvf8gLRK4w22aN1qB6F1zgHD9dxnEHtK=XEoTTPEA@mail.gmail.com[2]/messages/by-id/3424675.QJadu78ljV@aivenlaptop] is basically prolonged outage sooner or later before one hits
suchleaks, dunno what people do with cgroups2 do to avoid this. So to me it's
some kind of risky to run PostgreSQL on cgroups2 in the first place. IMHO
kernel should be enhanced to implement overcommit_memory=2 within cgroups,
but I understand that stuff that some stuff is implemented during page-fault
handling (and not memory allocation - the brk() itself), so it is impossible?

The patch mentions some GUC, but do in my experience, that's not the whole
truth, there's plenty of other things: number of relations involved, libc/
malloc implementation, and so on.

Two years ago I was thinking we could simply - in theory - issue
setrlimit(RLIMIT_DATA) (per process to limit each backend to strict X MB),
but AFAIR(?) Andres told me it is not really usable there as it may cause
SIGSEGVs in case of hitting the limit when growing stack and not just
overallocating memory (if I recall correctly). Or maybe I misunderstood and
I couldn't reproducce this on modern kernels.

-J.

[1]: /messages/by-id/CA+T=_GV-igvf8gLRK4w22aN1qB6F1zgHD9dxnEHtK=XEoTTPEA@mail.gmail.com
[2]: /messages/by-id/3424675.QJadu78ljV@aivenlaptop

Hi Jakub,

Thanks for looking at it. You're right that the last paragraph read
as if choosing those settings is enough to stay under the limit, and
that isn't true: the total can be many times work_mem. Attached v5
softens that sentence and recommends leaving a safety margin below
the limit, the same way the work_mem docs put it.

Overcommit inside cgroups and RLIMIT_DATA seem like a separate
discussion from this docs patch, so I left them out.

Changes since v4: only the last paragraph of the section.

Thanks,
João Marcelo

Em seg., 5 de out. de 2026 às 03:37, Jakub Wartak <
jakub.wartak@enterprisedb.com> escreveu:

Hi João,

Attached v5 softens that sentence and recommends leaving a safety
margin below the limit, the same way the work_mem docs put it.

v5 reads correctly -- the total can indeed be many times work_mem, and
the safety-margin wording matches how work_mem is documented. Only the
last paragraph changed and the docs still build, so this stays Ready for
Committer from my side.

Regards,
Manu

Hi João

Attached v5 softens that sentence and recommends leaving a safety
margin below the limit, the same way the work_mem docs put it.

Thank you for updating a patch and adding information about cgroup might
be helpful.

Here are some comments.

I noticed some of the topics are already written in the above
'overcommit' section. For example, how to tune memory related parameters
and how postgres behaves after OOM killer terminates postgresql.
(I did small fix related to this topic in [1]/messages/by-id/ce269321-a7f4-48f4-a9ca-a4a4983b9fd6@gmail.com)

So I would do either of the followings:
1. Create a new section for cgroup as you did and only add some
references to overcommit section like "Please see the overcommit section
to decrease the risk of OOM by tuning memory parameters"

Or
2. Not create a new dedicated section for cgroup and just add "Note"
sub-section under the overcommit section, then write some peculiar
things for cgroup.

What do you think?

[1]: /messages/by-id/ce269321-a7f4-48f4-a9ca-a4a4983b9fd6@gmail.com
/messages/by-id/ce269321-a7f4-48f4-a9ca-a4a4983b9fd6@gmail.com

Regards,
Takeshi Ideriha

Hi Takeshi,

I noticed some of the topics are already written in the above
'overcommit' section. For example, how to tune memory related parameters
and how postgres behaves after OOM killer terminates postgresql.

Both are right. To give João something concrete to decide on, I applied
v5 on current master (it applies with a 6-line offset, on top of your
febd05600d7) and went through it against the overcommit section topic by
topic.

Already covered by the overcommit section:

- A server process other than the postmaster is killed: crash restart,
or shutdown with restart_after_crash off (v5 2nd paragraph).
- The postmaster is killed: no automatic restart (v5 3rd paragraph).
- Sizing shared_buffers, work_mem, hash_mem_multiplier and
max_connections (v5 4th paragraph). The overcommit section has those
four; v5 adds maintenance_work_mem, the overcommit section adds
connection pooling.

The first two only became duplicates with your fix on Oct 16. v5 is
from Oct 1, when the overcommit section still said existing connections
"will continue to function normally", so at the time v5 contradicted
it. Now the two agree.

Not in the overcommit section, and specific to cgroups:

- the limit applies regardless of vm.overcommit_memory
- memory.max (v2 only)
- the OOM killer runs inside the cgroup, so allocations usually don't
fail with an error
- memory.oom.group kills the whole cgroup except processes with
oom_score_adj -1000, and recent Kubernetes sets it by default
- shared memory and each process's private memory both count
- huge pages count only with memory_hugetlb_accounting

Not in the overcommit section, but not specific to cgroups either:
systemd's OOMPolicy, the postmaster not logging its own termination, and
the safety margin below the limit. The -1000 protection for the
postmaster is already a reference to the overcommit section rather than
a copy.

So 3 of the 14 topics overlap, in three sentences (at most 13 of the 44
lines of text). The cgroup-specific part is about 25 lines, which is a
lot for a Note. That fits your option 1: keep the section and replace
those three sentences with a reference to the overcommit section.
Option 2 also works, but then the three general topics above would fit
better in the overcommit text itself than in a cgroup Note. Either way,
the choice is João's -- I'm just putting the numbers next to your two
options.

On my side: when I said v5 stays Ready for Committer, I was reviewing
the changed paragraph and checking that the docs build. I didn't look
at how the section fits with the overcommit one, and your point about
that is fair. The content still reads right to me.

Regards,
Manu