GUCifying MAX_WAL_SEND
Hello all,
*TL; DR*
There has been a discussion about GUCifying the MAX_WAL_SEND constant in
walsender.c in here <https://commitfest.postgresql.org/13/958/> nearly 7
years ago, but resulting in nothing in the end. Today, I found out the
configurability of this parameter can be very helpful. So, I want to submit
a patch for it, but I also want to know your comments before then.
What is MAX_WAL_SEND?
It's the maximum size of WAL records that walsender reads from the disk and
then sends to the standby servers. Its current value is hardcoded as
XLOG_BLCKSZ * 16, which is 128KiB (8KiB * 16) by default.
Why do I think it can be beneficial to GUCify it?
We use Postgres in K8s along with RBD disks. Today, I found out that
because we use remote disks, it's better to read bigger chunks of data from
the disk in one operation. In our setup (which I assume is not a silly
setup), we had a query that took 4.5 seconds to execute on the primary
server and ~21 seconds to send the WAL records to our synchronous standby
server. After recompiling Postgres and setting MAX_WAL_SEND to 16MiB, that
21 seconds decreased to just 1.5 seconds, which is a 92.9% improvement.
Thank you for your comments in advance
Majid
On Thu, Jan 11, 2024 at 01:15:48AM +0330, Majid Garoosi wrote:
Why do I think it can be beneficial to GUCify it?
We use Postgres in K8s along with RBD disks. Today, I found out that
because we use remote disks, it's better to read bigger chunks of data from
the disk in one operation. In our setup (which I assume is not a silly
setup), we had a query that took 4.5 seconds to execute on the primary
server and ~21 seconds to send the WAL records to our synchronous standby
server. After recompiling Postgres and setting MAX_WAL_SEND to 16MiB, that
21 seconds decreased to just 1.5 seconds, which is a 92.9% improvement.
I did not follow the older thread, but MAX_SEND_SIZE does not stress
me as a performance bottleneck in walsender.c when we use it for
calculations with the end LSN, so I can get behind the argument of
GUC-ifying it when reading WAL data in batches, even if it can be easy
to get it wrong. The hardcoded value comes back to 40f908bdcdc7 back
in 2010, and hardware has evolved a lot since that point.
So I'd +1 your suggestion. A RBD setup sounds exotic to me for a
database, though :)
--
Michael