Is the XLP_BKP_REMOVABLE flag itself removable/obsolete?
Whilst going through WAL definitions and headers, I noticed
XLP_BKP_REMOVABLE, which doesn't have any users in the PG repo. It is
still being written into WAL page headers, but never accessed or acted
on, and has been present since before the big WAL record revamp in
2014 (2c03216d83).
Is this flag actually useful and used in the more recent versions of
PostgreSQL? If not, let's remove it.
Even back in 2011 (when XLP_BKP_REMOVABLE was introduced in its
current form) there were questions about the practical uses of this
flag [0]/messages/by-id/flat/18626.1323703037@sss.pgh.pa.us.
Back then, the pg_lesslog tool (iiuc, a tool for archiving WAL, with
deep WAL introspection/reparsing for reduced storage needs) was the
only mention of a tool that uses it, but 15 years later I'm not sure
there are any tools left which use that flag - pg_lesslog seems to
have had its last release in 2010, and seems to have died a quiet
death.
In 2025, Noah sent a mail to -hackers [1]/messages/by-id/20250705001628.c3.nmisch@google.com showing a race condition
that could get this flag to be set on incorrect WAL pages, which could
cause issues for software that relied on this flag's indicated value.
Finally, I (and Michael in 2018 [2]/messages/by-id/20180330235757.GA1394@paquier.xyz) also couldn't find any code in
any of release branches >= 9.2 that used this flag, I couldn't find
any user in Debian Code Search, nor did I find any code on Github that
would indicate any real usage of that flag: There are some WAL
decoding tools that display its presence on the page, or which copy
the definition into their respective project in their language of
choice, but never any that have material logic that utilizes whatever
this flag means for more than displaying the WAL page state.
/cc Noah Misch and Michael Paquier, you both suggested at some point
to remove the flag.
Kind regards,
Matthias van de Meent
Databricks (https://www.databricks.com)
[0]: /messages/by-id/flat/18626.1323703037@sss.pgh.pa.us
[1]: /messages/by-id/20250705001628.c3.nmisch@google.com
[2]: /messages/by-id/20180330235757.GA1394@paquier.xyz
Attachments:
v1-0001-Remove-XLP_BKP_REMOVABLE.patchapplication/octet-stream; name=v1-0001-Remove-XLP_BKP_REMOVABLE.patchDownload+2-26
On Thu, Feb 26, 2026 at 03:53:54PM +0100, Matthias van de Meent wrote:
In 2025, Noah sent a mail to -hackers [1] showing a race condition
that could get this flag to be set on incorrect WAL pages, which could
cause issues for software that relied on this flag's indicated value.Finally, I (and Michael in 2018 [2]) also couldn't find any code in
any of release branches >= 9.2 that used this flag, I couldn't find
any user in Debian Code Search, nor did I find any code on Github that
would indicate any real usage of that flag: There are some WAL
decoding tools that display its presence on the page, or which copy
the definition into their respective project in their language of
choice, but never any that have material logic that utilizes whatever
this flag means for more than displaying the WAL page state.
Noah's statement is a better argument than the impression I got from
2018 (aka it's useless): even if one tries to use this flag, it may
report a different fact than what one wants to track.
Let's just remove this flag. I would have let XLP_ALL_FLAGS as it is
now at 0x000F, though. It is just used for filtering the bits we
allow to exist in this area of the header, for validation. Perhaps
some forks like this extensibility if they have their own extra custom
bits, meaning less code to modify. That's a minor point for sure, but
people like hacking on Postgres and forking this code.
Thoughts or objections from others?
--
Michael
On Fri, Feb 27, 2026 at 08:15:22AM +0900, Michael Paquier wrote:
Let's just remove this flag. I would have let XLP_ALL_FLAGS as it is
now at 0x000F, though. It is just used for filtering the bits we
allow to exist in this area of the header, for validation. Perhaps
some forks like this extensibility if they have their own extra custom
bits, meaning less code to modify. That's a minor point for sure, but
people like hacking on Postgres and forking this code.
Done. The patch missed a variable declaration not required anymore in
AdvanceXLInsertBuffer(), that I have caught with a compiler warning.
Another thing that did not feel right was to mark 0x0004 as free, so I
have moved the contrecord value one level down and updated
XLP_ALL_FLAGS accordingly, like ff9f111bce24 and 2dd9322ba6ee.
XLOG_PAGE_MAGIC also got its bump.
--
Michael
On Mon, 2 Mar 2026 at 06:20, Michael Paquier <michael@paquier.xyz> wrote:
On Fri, Feb 27, 2026 at 08:15:22AM +0900, Michael Paquier wrote:
Let's just remove this flag. I would have let XLP_ALL_FLAGS as it is
now at 0x000F, though. It is just used for filtering the bits we
allow to exist in this area of the header, for validation. Perhaps
some forks like this extensibility if they have their own extra custom
bits, meaning less code to modify. That's a minor point for sure, but
people like hacking on Postgres and forking this code.Done. The patch missed a variable declaration not required anymore in
AdvanceXLInsertBuffer(), that I have caught with a compiler warning.
Thanks for the check, clean up, and commit!
Kind regards,
Matthias van de Meent
Databricks (https://www.databricks.com)