assembling PGresults from multiple simultaneous queries (libpq, singlerowmode)

Started by Konstantin Izmailovabout 7 years ago7 messagesgeneral
Jump to latest
#1Konstantin Izmailov
pgfizm@gmail.com

Hi,
I'm experimenting with Postgres 10 and protocol v3. I noticed that the
Postgres allows executing multiple queries simultaneously (I basically
commented out a check that prevents sending another query in libpq while
previous result(s) reading is not complete). Things appear like working,
but I just wanted to ask if anyone else tried the same (logically separate
results from multiple simultaneous queries)?

Afaik libpq was not designed for that scenario, but it seems the Postgres
server supports it. Any thought or information will be appreciated.

Thank you!
Konstantin

#2Pavel Stehule
pavel.stehule@gmail.com
In reply to: Konstantin Izmailov (#1)
Re: assembling PGresults from multiple simultaneous queries (libpq, singlerowmode)

Hi

ne 7. 4. 2019 v 20:47 odesílatel Konstantin Izmailov <pgfizm@gmail.com>
napsal:

Hi,
I'm experimenting with Postgres 10 and protocol v3. I noticed that the
Postgres allows executing multiple queries simultaneously (I basically
commented out a check that prevents sending another query in libpq while
previous result(s) reading is not complete). Things appear like working,
but I just wanted to ask if anyone else tried the same (logically separate
results from multiple simultaneous queries)?

Postgres cannot to execute simultaneous queries in one session. So queries
should be executed in serial form every time.

Regards

Pavel

Show quoted text

Afaik libpq was not designed for that scenario, but it seems the Postgres
server supports it. Any thought or information will be appreciated.

Thank you!
Konstantin

#3Andres Freund
andres@anarazel.de
In reply to: Pavel Stehule (#2)
Re: assembling PGresults from multiple simultaneous queries (libpq, singlerowmode)

Hi,

On 2019-04-07 20:57:56 +0200, Pavel Stehule wrote:

ne 7. 4. 2019 v 20:47 odes�latel Konstantin Izmailov <pgfizm@gmail.com>
napsal:

Hi,
I'm experimenting with Postgres 10 and protocol v3. I noticed that the
Postgres allows executing multiple queries simultaneously (I basically
commented out a check that prevents sending another query in libpq while
previous result(s) reading is not complete). Things appear like working,
but I just wanted to ask if anyone else tried the same (logically separate
results from multiple simultaneous queries)?

Postgres cannot to execute simultaneous queries in one session. So queries
should be executed in serial form every time.

I think what Konstantin is really talking about is pipelining
(i.e. sending multiple queries without waiting for the results
inbetween, thereby reducing latency), and that is actually supported by
postgres. Some drivers make fairly extensive use of it (e.g. the pgjdbc
driver).

As for libpq: No, that's currently not supported. There is a patch that
I hope to get into v13 however: https://commitfest.postgresql.org/20/1317

Greetings,

Andres Freund

#4Konstantin Izmailov
pgfizm@gmail.com
In reply to: Andres Freund (#3)
Re: assembling PGresults from multiple simultaneous queries (libpq, singlerowmode)

Yes, Andres, I meant "pipelining", just couldn't choose correct word. Thank
you for the answer(s)!

I also made changes in my own copy of libpq, and they work fine. I think
the pipelining support is needed in libpq. Btw, how can I get the patch
code? I want to compare your approach with mine. I couldn't figure out how
to get the patch from the link.

Thanks so much!

On Sun, Apr 7, 2019 at 12:21 PM Andres Freund <andres@anarazel.de> wrote:

Show quoted text

Hi,

On 2019-04-07 20:57:56 +0200, Pavel Stehule wrote:

ne 7. 4. 2019 v 20:47 odesílatel Konstantin Izmailov <pgfizm@gmail.com>
napsal:

Hi,
I'm experimenting with Postgres 10 and protocol v3. I noticed that the
Postgres allows executing multiple queries simultaneously (I basically
commented out a check that prevents sending another query in libpq

while

previous result(s) reading is not complete). Things appear like

working,

but I just wanted to ask if anyone else tried the same (logically

separate

results from multiple simultaneous queries)?

Postgres cannot to execute simultaneous queries in one session. So

queries

should be executed in serial form every time.

I think what Konstantin is really talking about is pipelining
(i.e. sending multiple queries without waiting for the results
inbetween, thereby reducing latency), and that is actually supported by
postgres. Some drivers make fairly extensive use of it (e.g. the pgjdbc
driver).

As for libpq: No, that's currently not supported. There is a patch that
I hope to get into v13 however: https://commitfest.postgresql.org/20/1317

Greetings,

Andres Freund

#5Konstantin Izmailov
pgfizm@gmail.com
In reply to: Konstantin Izmailov (#4)
Re: assembling PGresults from multiple simultaneous queries (libpq, singlerowmode)

Never mind, I found the link to the github in emails from the link. Thanks
again!

Konstantin

On Sun, Apr 7, 2019 at 1:28 PM Konstantin Izmailov <pgfizm@gmail.com> wrote:

Show quoted text

Yes, Andres, I meant "pipelining", just couldn't choose correct word.
Thank you for the answer(s)!

I also made changes in my own copy of libpq, and they work fine. I think
the pipelining support is needed in libpq. Btw, how can I get the patch
code? I want to compare your approach with mine. I couldn't figure out how
to get the patch from the link.

Thanks so much!

On Sun, Apr 7, 2019 at 12:21 PM Andres Freund <andres@anarazel.de> wrote:

Hi,

On 2019-04-07 20:57:56 +0200, Pavel Stehule wrote:

ne 7. 4. 2019 v 20:47 odesílatel Konstantin Izmailov <pgfizm@gmail.com>
napsal:

Hi,
I'm experimenting with Postgres 10 and protocol v3. I noticed that the
Postgres allows executing multiple queries simultaneously (I basically
commented out a check that prevents sending another query in libpq

while

previous result(s) reading is not complete). Things appear like

working,

but I just wanted to ask if anyone else tried the same (logically

separate

results from multiple simultaneous queries)?

Postgres cannot to execute simultaneous queries in one session. So

queries

should be executed in serial form every time.

I think what Konstantin is really talking about is pipelining
(i.e. sending multiple queries without waiting for the results
inbetween, thereby reducing latency), and that is actually supported by
postgres. Some drivers make fairly extensive use of it (e.g. the pgjdbc
driver).

As for libpq: No, that's currently not supported. There is a patch that
I hope to get into v13 however: https://commitfest.postgresql.org/20/1317

Greetings,

Andres Freund

#6Andres Freund
andres@anarazel.de
In reply to: Konstantin Izmailov (#4)
Re: assembling PGresults from multiple simultaneous queries (libpq, singlerowmode)

Hi,

On postgres mailing lists please don't write your reply at the top of a
fully quoted email. We like the reply to be inline and trimmed to the
necessary parts.

On 2019-04-07 13:28:46 -0700, Konstantin Izmailov wrote:

Yes, Andres, I meant "pipelining", just couldn't choose correct word. Thank
you for the answer(s)!

I also made changes in my own copy of libpq, and they work fine. I think
the pipelining support is needed in libpq. Btw, how can I get the patch
code? I want to compare your approach with mine. I couldn't figure out how
to get the patch from the link.

Hm, odd. There's a link on the page "Latest attachment" - but for
unknown reasons that's broken. I've attached it for now, but will also
inquire with the webadmin team about what's up.

Greetings,

Andres Freund

Attachments:

0001-Pipelining-batch-support-for-libpq-code-v16.patchtext/x-diff; charset=us-asciiDownload+1186-44
#7Konstantin Izmailov
pgfizm@gmail.com
In reply to: Andres Freund (#6)
Re: assembling PGresults from multiple simultaneous queries (libpq, singlerowmode)

Got it! Thanks!