About default inBufSize (connection read buffer size) in libpq

Hi,

On 2023-03-05 05:42:06 +0300, Трофимов Иван wrote:

I was running some benchmarks for PG driver built on top of libpq async
functionality,
and noticed that recv syscalls issued by the application are limited by 16Kb,
which seems to
be inBufSize coming from makeEmptyPGconn in interfaces/libpq/fe-connect.c.
 
Hacking that to higher values allowed my benchmarks to issue drastically less
syscalls
when running some heavy selects, both in local and cloud environments, which
made them
significantly faster.
 
I believe there is a reason for that value to be 16Kb, but i was wondering if
it's safe to change
this default to user-provided value, and if it is - could this functionality be
added into API?

I've observed the small buffer size hurting as well - not just client side,
also on the serve.

But I don't think we necessarily need to make it configurable. From what I can
tell the pain mainly comes using the read/send buffers when they don't even
help, because the message data we're processing is bigger than the buffer
size. When we need to receive / send data that we know is bigger than the
the buffer, we should copy the portion that is still in the buffer, and then
send/receive directly from the data to be sent/received.

Greetings,

Andres Freund