Re: Bytea and perl
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
OK. Here is my follow-up question. Why is this explicit parameter binding
necessary? When would I want to have pg_type be something other than
PG_BYTEA when inserting into a bytea column?
You wouldn't, but the trick is getting all the pieces to know that the column
is bytea. DBD::Pg has no inherent way to find out for iteslf. Nor does libpq.
The planner has an idea, but that information is not transmitted back to DBD:Pg.
The difference then becomes that the low-level calls that DDB::Pg makes to
PostgreSQL via PQexecParams and PQexecPrepared are different if any of the values
are binary. If they are, we can't simply pass a string, but have to pass a
separate array of string lengths, as we can't use \0 to indicae the end of the
data anymore.
The reason this is important is that many (read this as ALL, as far as I
know) modules built on top of DBI do not use explicit paramater binding and
rely on the sth->execute(...) quoting to do the right thing, which it does
with all column types except bytea, it seems.
Well, there are other column type cases where it will fail, but they are not
as common as bytea. Unfortunately, there is no easy solution. Hopefully these
high-level interface modules left some hooks and knobs to handle this sort
of situation. If they don't, drop them a line, because they should. :)
I guess a third option is the large object interface, which I am trying to
avoid.
I suspect that this is even less supported by the other modules, so you might
as well go with the binding at that point. Good luck: hopefully one of the
four options will work out for you.
- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 200603232139
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----
iD8DBQFEI15nvJuQZxSWSsgRAnDGAJ9CW2gb0qE53isrOfLjoALuQYetKQCgwCLQ
A1EKVpnIhjPHiqT0HTAfwjY=
=LmM9
-----END PGP SIGNATURE-----
On 3/23/06 9:50 PM, "Greg Sabino Mullane" <greg@turnstep.com> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1OK. Here is my follow-up question. Why is this explicit parameter binding
necessary? When would I want to have pg_type be something other than
PG_BYTEA when inserting into a bytea column?You wouldn't, but the trick is getting all the pieces to know that the column
is bytea. DBD::Pg has no inherent way to find out for iteslf. Nor does libpq.
The planner has an idea, but that information is not transmitted back to
DBD:Pg.
The difference then becomes that the low-level calls that DDB::Pg makes to
PostgreSQL via PQexecParams and PQexecPrepared are different if any of the
values
are binary. If they are, we can't simply pass a string, but have to pass a
separate array of string lengths, as we can't use \0 to indicae the end of the
data anymore.
Ahhh. Now things start to make sense.
The reason this is important is that many (read this as ALL, as far as I
know) modules built on top of DBI do not use explicit paramater binding and
rely on the sth->execute(...) quoting to do the right thing, which it does
with all column types except bytea, it seems.Well, there are other column type cases where it will fail, but they are not
as common as bytea. Unfortunately, there is no easy solution. Hopefully these
high-level interface modules left some hooks and knobs to handle this sort
of situation. If they don't, drop them a line, because they should. :)
I will.
I guess a third option is the large object interface, which I am trying to
avoid.I suspect that this is even less supported by the other modules, so you might
as well go with the binding at that point. Good luck: hopefully one of the
four options will work out for you.
Oh, it really isn't that big a deal. Right now, I am using base64 encoding,
which is fine.
Thanks for elaborating.
Sean
On Mar 23, 2006, at 9:50 PM, Greg Sabino Mullane wrote:
OK. Here is my follow-up question. Why is this explicit
parameter binding
necessary? When would I want to have pg_type be something other than
PG_BYTEA when inserting into a bytea column?You wouldn't, but the trick is getting all the pieces to know that
the column
is bytea. DBD::Pg has no inherent way to find out for iteslf. Nor
does libpq.
The planner has an idea, but that information is not transmitted
back to DBD:Pg.
I have not looked at libpq in any detail, but it should have access
to the type of all the parameters in the prepared statement. The
Describe (F) statement in the frontend/backend protocol identifies
the type of each parameter. I'm using this in Lisp to convert
parameters as needed for prepared statements.
http://www.postgresql.org/docs/8.1/static/protocol-message-formats.html
John DeSoi, Ph.D.
http://pgedit.com/
Power Tools for PostgreSQL
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I have not looked at libpq in any detail, but it should have access to the
type of all the parameters in the prepared statement. The Describe (F)
statement in the frontend/backend protocol identifies the type of each
parameter. I'm using this in Lisp to convert parameters as needed for
prepared statements.
It should, but it does not yet. Known limitation. It's in the libpq docs
but not sure if it is on the official TODO list...yep, it's in there.
Pardon if this has been addressed already, my mail server crashed and I'm
slowly catching up.
- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 200603282138
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----
iD8DBQFEKfMtvJuQZxSWSsgRAsJeAKCC08Isc+1Oip1MO/EPNNXZwBdgXQCgprLU
qFnAhuyoRKyV4CMQ6QFHL7Q=
=F4b8
-----END PGP SIGNATURE-----
Hi,
On Mar 25 08:47, John DeSoi wrote:
I have not looked at libpq in any detail, but it should have access
to the type of all the parameters in the prepared statement. The
Describe (F) statement in the frontend/backend protocol identifies
the type of each parameter.
I've prepared a patch for the Describe <-> ParameterDescription
messaging which is available via current extended query protocol.
Usage (and implementation) is explained in the patch's documentation
related part. (Also I tried to place informative comments in the code
too.)
But I've a problem with ereport() error calls caused by erronous
target_type entries. After an error in exec_describe_statement_message()
(or exec_describe_portal_message()) it leaves block with ereport() call
and client side stalls in PGASYNC_BUSY state while backend stalls in
PostgresMain() by calling ReadCommand(). To summerize, an error
returning pqDescribe() call causes both side to stall.
I'd be so appreciated to hear your thoughts about the patch and above
problem.
Regards.
Attachments:
libpq_desc_msg.patch.1text/plain; charset=us-asciiDownload
? src/interfaces/libpq/.deps
? src/interfaces/libpq/cscope.out
? src/interfaces/libpq/libpq.so.4.2
Index: doc/src/sgml/libpq.sgml
===================================================================
RCS file: /projects/cvsroot/pgsql/doc/src/sgml/libpq.sgml,v
retrieving revision 1.206
diff -c -r1.206 libpq.sgml
*** doc/src/sgml/libpq.sgml 10 Mar 2006 19:10:48 -0000 1.206
--- doc/src/sgml/libpq.sgml 1 Apr 2006 18:27:41 -0000
***************
*** 2045,2053 ****
</varlistentry>
<varlistentry>
! <term><function>PQprint</function><indexterm><primary>PQprint</></></term>
<listitem>
<para>
Prints out all the rows and, optionally, the
column names to the specified output stream.
<synopsis>
--- 2045,2081 ----
</varlistentry>
<varlistentry>
! <term><function>PQdescPrepared</function><indexterm><primary>PQdescPrepared</></></term>
! <term><function>PQdescPortal</function><indexterm><primary>PQdescPortal</></></term>
<listitem>
<para>
+ Describe a prepared statement or portal.
+ <synopsis>int PQdescPrepared(PGconn *conn, const char *stmt);</synopsis>
+ <synopsis>int PQdescPortal(PGconn *conn, const char *portal);</synopsis>
+ </para>
+ <para>
+ These two functions are used to describe a prepared statement or portal.
+ Functions return 1 on success and 0 on failure. (An appropriate error
+ message will be placed in an error situation.) <symbol>NULL</> values in
+ the <parameter>stmt</> and <parameter>portal</> parameters will be treated
+ as an empty string.
+ </para>
+ <para>
+ After a <function>PQdescPrepared</> or <function>PQdescPortal</> function
+ call, issuing a <function>PQgetResult</> will place backend's response
+ into a <structname>PGresult</structname> structure - that'll be
+ extractable via <function>PQftype</>.
+ </para>
+ <para>
+ These functions are available within extended query protocol which is
+ available in version 3.0 and above.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><function>PQprint</function><indexterm><primary>PQprint</></></term>
+ <para>
Prints out all the rows and, optionally, the
column names to the specified output stream.
<synopsis>
Index: src/backend/tcop/postgres.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/tcop/postgres.c,v
retrieving revision 1.482
diff -c -r1.482 postgres.c
*** src/backend/tcop/postgres.c 14 Mar 2006 22:48:21 -0000 1.482
--- src/backend/tcop/postgres.c 1 Apr 2006 18:28:08 -0000
***************
*** 3391,3396 ****
--- 3391,3398 ----
describe_type)));
break;
}
+
+ send_ready_for_query = true;
}
break;
Index: src/interfaces/libpq/fe-exec.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/interfaces/libpq/fe-exec.c,v
retrieving revision 1.182
diff -c -r1.182 fe-exec.c
*** src/interfaces/libpq/fe-exec.c 14 Mar 2006 22:48:23 -0000 1.182
--- src/interfaces/libpq/fe-exec.c 1 Apr 2006 18:28:15 -0000
***************
*** 55,60 ****
--- 55,62 ----
static void parseInput(PGconn *conn);
static bool PQexecStart(PGconn *conn);
static PGresult *PQexecFinish(PGconn *conn);
+ static int pqDescribe(PGconn *conn, const char desc_type,
+ const char *desc_target);
/* ----------------
***************
*** 2281,2286 ****
--- 2283,2400 ----
return 0;
}
+
+ /*
+ * pqDescribe - Describe given prepared statement or portal.
+ *
+ * Available options for target_type are
+ * 's' to describe a prepared statement; or
+ * 'p' to describe a portal.
+ * Returns 1 on success and 0 on failure.
+ *
+ * By issuing a PQgetResult(), response from the server will be placed
+ * in an empty PGresult that will be extractable via PQftype().
+ */
+ static int
+ pqDescribe(PGconn *conn, const char desc_type, const char *desc_target)
+ {
+ int ret;
+
+ /* This isn't gonna work on a 2.0 server. */
+ if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("function requires at least protocol version 3.0\n"));
+ return 0;
+ }
+
+ if (!conn)
+ return 0;
+
+ /* Clear the connection error message. */
+ resetPQExpBuffer(&conn->errorMessage);
+
+ /* Don't try to send if we know there's no live connection. */
+ if (conn->status != CONNECTION_OK)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("no connection to the server\n"));
+ return false;
+ }
+
+ /* Can't send while already busy, either. */
+ if (conn->asyncStatus != PGASYNC_IDLE)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("another command is already in progress\n"));
+ return false;
+ }
+
+ /* Initialize async result-accumulation state. */
+ conn->result = NULL;
+ conn->curTuple = NULL;
+
+ if (desc_target)
+ {
+ int len = strlen(desc_target) + 2;
+ char buf[len];
+
+ snprintf(buf, len, "%c%s", desc_type, desc_target);
+
+ /*
+ * Flushing stuff will be handled by pqPacketSend(). (Although
+ * this theoretically could block, it really shouldn't for
+ * these kind of small messages.)
+ */
+ ret = pqPacketSend(conn, 'D', buf, len);
+ }
+
+ /* NULL desc_target values will be treated as an empty string. */
+ else
+ {
+ int len = 2;
+ char buf[len];
+
+ buf[0] = desc_type;
+ buf[1] = '\0';
+ ret = pqPacketSend(conn, 'D', buf, len);
+ }
+
+ /* In case of a pqPacketSend() failure... */
+ if (ret != STATUS_OK)
+ {
+ pqHandleSendFailure(conn);
+ return 0;
+ }
+
+ /* Remember we are using extended query protocol. */
+ conn->queryclass = PGQUERY_EXTENDED;
+
+ /* Free last query string. */
+ if (conn->last_query)
+ {
+ free(conn->last_query);
+ conn->last_query = NULL;
+ }
+
+ /* Describe request is sent. */
+ conn->asyncStatus = PGASYNC_BUSY;
+ return 1;
+ }
+
+ int
+ PQdescPrepared(PGconn *conn, const char *stmt)
+ {
+ return pqDescribe(conn, 'S', stmt);
+ }
+
+ int
+ PQdescPortal(PGconn *conn, const char *portal)
+ {
+ return pqDescribe(conn, 'P', portal);
+ }
+
+
/* PQsetnonblocking:
* sets the PGconn's database connection non-blocking if the arg is TRUE
* or makes it non-blocking if the arg is FALSE, this will not protect
***************
*** 2346,2351 ****
--- 2460,2466 ----
free(ptr);
}
+
/*
* PQfreeNotify - free's the memory associated with a PGnotify
*
Index: src/interfaces/libpq/fe-protocol3.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/interfaces/libpq/fe-protocol3.c,v
retrieving revision 1.26
diff -c -r1.26 fe-protocol3.c
*** src/interfaces/libpq/fe-protocol3.c 14 Mar 2006 22:48:23 -0000 1.26
--- src/interfaces/libpq/fe-protocol3.c 1 Apr 2006 18:28:22 -0000
***************
*** 50,55 ****
--- 50,56 ----
static int getNotify(PGconn *conn);
static int getCopyStart(PGconn *conn, ExecStatusType copytype);
static int getReadyForQuery(PGconn *conn);
+ static int getParamDescriptions(PGconn *conn);
static void reportErrorPosition(PQExpBuffer msg, const char *query,
int loc, int encoding);
static int build_startup_packet(const PGconn *conn, char *packet,
***************
*** 350,355 ****
--- 351,375 ----
* the COPY command.
*/
break;
+ case 't': /* Parameter Description */
+ if (!conn->result)
+ {
+ if (!getParamDescriptions(conn)) /* Success. */
+ break;
+ else /* Not enough data or an error. */
+ return;
+ }
+ else
+ {
+ /*
+ * We'll use a new PGresult to place 't' input. Thus,
+ * we stop parsing until the application accepts the
+ * current result.
+ */
+ conn->asyncStatus = PGASYNC_READY;
+ return;
+ }
+ break;
default:
printfPQExpBuffer(&conn->errorMessage,
libpq_gettext(
***************
*** 1154,1159 ****
--- 1174,1249 ----
}
/*
+ * getParamDescriptions - process ParameterDescription message.
+ *
+ * We'll place parsed message information into a new PGresult. (Only
+ * parameter description attributes are going to be reliable.)
+ * Returns 0 if parsing is completed, 1 if there isn't enough data yet
+ * and -1 if there occurs an error.
+ */
+ static int
+ getParamDescriptions(PGconn *conn)
+ {
+ PGresult *res;
+ int nattr;
+ int i;
+ Oid typid;
+
+ res = PQmakeEmptyPGresult(conn, PGRES_COMMAND_OK);
+ if (!res)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ goto Error;
+ }
+
+ /*
+ * parseInput() already read the 't' label and message length.
+ * The next byte is the number of parameters used by the
+ * statement (may be zero).
+ */
+ if (pqGetInt(&nattr, 2, conn) < 0)
+ goto NotEnoughData;
+ res->numAttributes = nattr;
+
+ /* Allocate space for the attribute descriptors. */
+ if (nattr > 0)
+ {
+ int sz = nattr * sizeof(PGresAttDesc);
+
+ res->attDescs = (PGresAttDesc *) pqResultAlloc(res, sz, TRUE);
+ if (!res->attDescs)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ goto Error;
+ }
+
+ MemSet(res->attDescs, 0, sz);
+ }
+
+ /* Loop through attribute's type OIDs. */
+ for (i = 0; i < nattr; i++)
+ {
+ if (pqGetInt(&typid, 4, conn) < 0)
+ goto NotEnoughData;
+ res->attDescs[i].typid = typid;
+ }
+
+ /* Success. */
+ conn->result = res;
+ return 0;
+
+ NotEnoughData:
+ PQclear(res);
+ return 1;
+
+ Error:
+ PQclear(res);
+ return -1;
+ }
+
+ /*
* PQgetCopyData - read a row of data from the backend during COPY OUT
*
* If successful, sets *buffer to point to a malloc'd row of data, and
Index: src/interfaces/libpq/libpq-fe.h
===================================================================
RCS file: /projects/cvsroot/pgsql/src/interfaces/libpq/libpq-fe.h,v
retrieving revision 1.126
diff -c -r1.126 libpq-fe.h
*** src/interfaces/libpq/libpq-fe.h 20 Mar 2006 15:07:05 -0000 1.126
--- src/interfaces/libpq/libpq-fe.h 1 Apr 2006 18:28:25 -0000
***************
*** 414,419 ****
--- 414,423 ----
extern int PQgetlength(const PGresult *res, int tup_num, int field_num);
extern int PQgetisnull(const PGresult *res, int tup_num, int field_num);
+ /* Describe prepared statements or portals. */
+ extern int PQdescPrepared(PGconn *conn, const char *stmt);
+ extern int PQdescPortal(PGconn *conn, const char *portal);
+
/* Delete a PGresult */
extern void PQclear(PGresult *res);
Hi,
[Sending this message (first) to -hackers for discussion about the
extension and followed implementation.]
On Apr 01 09:39, Volkan YAZICI wrote:
I've prepared a patch for the Describe <-> ParameterDescription
messaging which is available via current extended query protocol.
Here's a written from scratch patch for the above mentioned extension.
It adds PQdescribePrepared() and PQdescribePortal() functions to the
libpq. New functions work as follows:
1. Issue a PQdescribePrepared() call.
2. First PQgetResult() will return a PGresult with input parameter
types of the prepared statement. (You can use PQftype() on this
PGresult to extract information.)
3. Second PQgetResult() will return another PGresult which holds the
column information for the will be returned tuples. (All PQf*()
functions can be used on this result.)
(A PQdescribePortal() call will just skip the 2nd step in the above
list.)
Patch passes regression tests and there're two examples attached for
PQdescribePrepared() and PQdescribePortal() usage.
To mention about the followed implementation, it needed some hack on
pqParseInput3() code to make it understand if a received message is
a reponse to a Describe ('D') query or to another tuple returning
query. To summarize problem, there're two possible forms of a 'D'
response:
1. Description of a prepared statement: t, T, Z
2. Description of a portal: T, Z
The problem is, AFAICS, it's not possible to distinguish between a tuple
returning query (T, ..., C, Z or T, E) and a description of a portal (T,
Z). Therefore, I've created a global flag (parsing_row_desc) which is
turned on when we receive a 'T' and turned off if we receive a 'C' or
'E'. It's a kind of ugly method but the only solution I could come up
with.
Regards.
Attachments:
libpq_desc_msg.patch.4text/plain; charset=us-asciiDownload
Index: src/backend/tcop/postgres.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/tcop/postgres.c,v
retrieving revision 1.483
diff -c -r1.483 postgres.c
*** src/backend/tcop/postgres.c 4 Apr 2006 19:35:35 -0000 1.483
--- src/backend/tcop/postgres.c 15 Apr 2006 07:39:49 -0000
***************
*** 1870,1875 ****
--- 1870,1876 ----
static void
exec_describe_statement_message(const char *stmt_name)
{
+ MemoryContext oldContext;
PreparedStatement *pstmt;
TupleDesc tupdesc;
ListCell *l;
***************
*** 1882,1888 ****
start_xact_command();
/* Switch back to message context */
! MemoryContextSwitchTo(MessageContext);
/* Find prepared statement */
if (stmt_name[0] != '\0')
--- 1883,1889 ----
start_xact_command();
/* Switch back to message context */
! oldContext = MemoryContextSwitchTo(MessageContext);
/* Find prepared statement */
if (stmt_name[0] != '\0')
***************
*** 1940,1946 ****
--- 1941,1950 ----
NULL);
else
pq_putemptymessage('n'); /* NoData */
+
+ MemoryContextSwitchTo(oldContext);
+ finish_xact_command();
}
/*
***************
*** 1951,1956 ****
--- 1955,1961 ----
static void
exec_describe_portal_message(const char *portal_name)
{
+ MemoryContext oldContext;
Portal portal;
/*
***************
*** 1960,1966 ****
start_xact_command();
/* Switch back to message context */
! MemoryContextSwitchTo(MessageContext);
portal = GetPortalByName(portal_name);
if (!PortalIsValid(portal))
--- 1965,1971 ----
start_xact_command();
/* Switch back to message context */
! oldContext = MemoryContextSwitchTo(MessageContext);
portal = GetPortalByName(portal_name);
if (!PortalIsValid(portal))
***************
*** 1992,1997 ****
--- 1997,2006 ----
portal->formats);
else
pq_putemptymessage('n'); /* NoData */
+
+ MemoryContextSwitchTo(oldContext);
+
+ finish_xact_command();
}
Index: src/interfaces/libpq/fe-exec.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/interfaces/libpq/fe-exec.c,v
retrieving revision 1.182
diff -c -r1.182 fe-exec.c
*** src/interfaces/libpq/fe-exec.c 14 Mar 2006 22:48:23 -0000 1.182
--- src/interfaces/libpq/fe-exec.c 15 Apr 2006 07:39:58 -0000
***************
*** 55,60 ****
--- 55,62 ----
static void parseInput(PGconn *conn);
static bool PQexecStart(PGconn *conn);
static PGresult *PQexecFinish(PGconn *conn);
+ static int pqDescribe(PGconn *conn, const char desc_type,
+ const char *desc_target);
/* ----------------
***************
*** 2281,2286 ****
--- 2283,2392 ----
return 0;
}
+
+ /*
+ * pqDescribe - Describe given prepared statement or portal.
+ *
+ * Available options for target_type are
+ * 'S' to describe a prepared statement; or
+ * 'P' to describe a portal.
+ * Returns 0 on success and 1 on failure.
+ *
+ * By issuing a PQgetResult(), response from the server will be placed
+ * in an empty PGresult which will be extractable via PQf*() function family.
+ */
+ static int
+ pqDescribe(PGconn *conn, const char desc_type, const char *desc_target)
+ {
+ int ret;
+
+ /* This isn't gonna work on a 2.0 server. */
+ if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("function requires at least protocol version 3.0\n"));
+ return 1;
+ }
+
+ if (!conn)
+ return 1;
+
+ /* Clear the connection error message. */
+ resetPQExpBuffer(&conn->errorMessage);
+
+ /* Don't try to send if we know there's no live connection. */
+ if (conn->status != CONNECTION_OK)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("no connection to the server\n"));
+ return 1;
+ }
+
+ /* Can't send while already busy, either. */
+ if (conn->asyncStatus != PGASYNC_IDLE)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("another command is already in progress\n"));
+ return 1;
+ }
+
+ /* Initialize async result-accumulation state. */
+ conn->result = NULL;
+ conn->curTuple = NULL;
+
+ if (desc_target)
+ ret = (pqPutMsgStart('D', false, conn) < 0 ||
+ pqPutc(desc_type, conn) < 0 ||
+ pqPuts(desc_target, conn) < 0 ||
+ pqPutMsgEnd(conn) < 0);
+
+ /* NULL desc_target values will be treated as an empty string. */
+ else
+ ret = (pqPutMsgStart('D', false, conn) < 0 ||
+ pqPutc(desc_type, conn) < 0 ||
+ pqPutc('\0', conn) < 0 ||
+ pqPutMsgEnd(conn) < 0);
+
+ if (ret)
+ goto SendFailure;
+
+ /* Extended protocol requires a Sync message. */
+ if (pqPutMsgStart('S', false, conn) < 0 ||
+ pqPutMsgEnd(conn) < 0)
+ goto SendFailure;
+
+ /* Remember we are using extended query protocol. */
+ conn->queryclass = PGQUERY_EXTENDED;
+
+ /* Free last query string. */
+ if (conn->last_query)
+ {
+ free(conn->last_query);
+ conn->last_query = NULL;
+ }
+
+ /* Describe request is sent. */
+ conn->asyncStatus = PGASYNC_BUSY;
+ return 0;
+
+ SendFailure:
+ pqHandleSendFailure(conn);
+ return 1;
+ }
+
+ int
+ PQdescribePrepared(PGconn *conn, const char *stmt)
+ {
+ return pqDescribe(conn, 'S', stmt);
+ }
+
+ int
+ PQdescribePortal(PGconn *conn, const char *portal)
+ {
+ return pqDescribe(conn, 'P', portal);
+ }
+
+
/* PQsetnonblocking:
* sets the PGconn's database connection non-blocking if the arg is TRUE
* or makes it non-blocking if the arg is FALSE, this will not protect
***************
*** 2346,2351 ****
--- 2452,2458 ----
free(ptr);
}
+
/*
* PQfreeNotify - free's the memory associated with a PGnotify
*
Index: src/interfaces/libpq/fe-protocol3.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/interfaces/libpq/fe-protocol3.c,v
retrieving revision 1.26
diff -c -r1.26 fe-protocol3.c
*** src/interfaces/libpq/fe-protocol3.c 14 Mar 2006 22:48:23 -0000 1.26
--- src/interfaces/libpq/fe-protocol3.c 15 Apr 2006 07:40:05 -0000
***************
*** 42,50 ****
--- 42,56 ----
((id) == 'T' || (id) == 'D' || (id) == 'd' || (id) == 'V' || \
(id) == 'E' || (id) == 'N' || (id) == 'A')
+ /*
+ * Flag to distinguish between an odd RowDescription and start of
+ * a tuple returning query.
+ */
+ static bool parsing_row_desc = false;
static void handleSyncLoss(PGconn *conn, char id, int msgLength);
static int getRowDescriptions(PGconn *conn);
+ static int getParamDescriptions(PGconn *conn);
static int getAnotherTuple(PGconn *conn, int msgLength);
static int getParameterStatus(PGconn *conn);
static int getNotify(PGconn *conn);
***************
*** 206,221 ****
--- 212,247 ----
}
strncpy(conn->result->cmdStatus, conn->workBuffer.data,
CMDSTATUS_LEN);
+
+ /* This cannot be a Describe ('D') response no more. */
+ parsing_row_desc = false;
+
conn->asyncStatus = PGASYNC_READY;
break;
case 'E': /* error return */
if (pqGetErrorNotice3(conn, true))
return;
conn->asyncStatus = PGASYNC_READY;
+
+ /* No further RowDescription parsing is possible. */
+ parsing_row_desc = false;
break;
case 'Z': /* backend is ready for new query */
if (getReadyForQuery(conn))
return;
+
+ if (parsing_row_desc)
+ {
+ /*
+ * This is probably received from a Describe ('D')
+ * query's response.
+ */
+ parsing_row_desc = false;
+ conn->asyncStatus = PGASYNC_READY;
+ return;
+ }
+
+ /* Backend is ready for a new query. */
conn->asyncStatus = PGASYNC_IDLE;
break;
case 'I': /* empty query */
***************
*** 268,273 ****
--- 294,302 ----
/* First 'T' in a query sequence */
if (getRowDescriptions(conn))
return;
+
+ /* Turning "RowDesc is parsed" flag to on. */
+ parsing_row_desc = true;
}
else
{
***************
*** 294,299 ****
--- 323,348 ----
conn->result = PQmakeEmptyPGresult(conn,
PGRES_COMMAND_OK);
break;
+ case 't': /* Parameter Description */
+ if (!conn->result)
+ {
+ /* A new PGresult will be created to load parsed data into. */
+ if (getParamDescriptions(conn))
+ return;
+
+ /* Result is ready. */
+ conn->asyncStatus = PGASYNC_READY;
+ }
+ else
+ {
+ /*
+ * We'll wait until the application accepts the current
+ * existing result.
+ */
+ conn->asyncStatus = PGASYNC_READY;
+ return;
+ }
+ break;
case 'D': /* Data Row */
if (conn->result != NULL &&
conn->result->resultStatus == PGRES_TUPLES_OK)
***************
*** 500,505 ****
--- 549,620 ----
}
/*
+ * parseInput subroutine to read a 't' (ParameterDescription) message.
+ * Returns 0 if parsing is completed, 1 if there isn't enough data yet
+ * and EOF if an error occurs.
+ *
+ * Parsed data will get loaded into the existing conn->result.
+ */
+ static int
+ getParamDescriptions(PGconn *conn)
+ {
+ PGresult *res;
+ int nattr;
+ int i;
+ Oid typid;
+
+ res = PQmakeEmptyPGresult(conn, PGRES_COMMAND_OK);
+ if (!res)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ EOF;
+ }
+
+ /* First extracting number of attributes from the message. */
+ if (pqGetInt(&nattr, 2, conn) < 0)
+ goto NotEnoughData;
+ res->numAttributes = nattr;
+
+ /* Allocate space for the attribute descriptors. */
+ if (nattr > 0)
+ {
+ int sz = nattr * sizeof(PGresAttDesc);
+
+ res->attDescs = (PGresAttDesc *) pqResultAlloc(res, sz, TRUE);
+ if (!res->attDescs)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ goto Error;
+ }
+
+ /* Reset attribute values. */
+ MemSet(res->attDescs, 0, sz);
+ }
+
+ /* Loop through attribute's type OIDs. */
+ for (i = 0; i < nattr; i++)
+ {
+ if (pqGetInt(&typid, 4, conn) < 0)
+ goto NotEnoughData;
+ res->attDescs[i].typid = typid;
+ }
+
+ /* Success. */
+ conn->result = res;
+ return 0;
+
+ NotEnoughData:
+ PQclear(res);
+ return 1;
+
+ Error:
+ PQclear(res);
+ return EOF;
+ }
+
+ /*
* parseInput subroutine to read a 'D' (row data) message.
* We add another tuple to the existing PGresult structure.
* Returns: 0 if completed message, EOF if error or not enough data yet.
Index: src/interfaces/libpq/libpq-fe.h
===================================================================
RCS file: /projects/cvsroot/pgsql/src/interfaces/libpq/libpq-fe.h,v
retrieving revision 1.126
diff -c -r1.126 libpq-fe.h
*** src/interfaces/libpq/libpq-fe.h 20 Mar 2006 15:07:05 -0000 1.126
--- src/interfaces/libpq/libpq-fe.h 15 Apr 2006 07:40:08 -0000
***************
*** 414,419 ****
--- 414,423 ----
extern int PQgetlength(const PGresult *res, int tup_num, int field_num);
extern int PQgetisnull(const PGresult *res, int tup_num, int field_num);
+ /* Describe prepared statements and portals. */
+ extern int PQdescribePrepared(PGconn *conn, const char *stmt);
+ extern int PQdescribePortal(PGconn *conn, const char *portal);
+
/* Delete a PGresult */
extern void PQclear(PGresult *res);
Volkan YAZICI wrote:
To mention about the followed implementation, it needed some hack on
pqParseInput3() code to make it understand if a received message is
a reponse to a Describe ('D') query or to another tuple returning
query. To summarize problem, there're two possible forms of a 'D'
response:1. Description of a prepared statement: t, T, Z
2. Description of a portal: T, ZThe problem is, AFAICS, it's not possible to distinguish between a tuple
returning query (T, ..., C, Z or T, E) and a description of a portal (T,
Z). Therefore, I've created a global flag (parsing_row_desc) which is
turned on when we receive a 'T' and turned off if we receive a 'C' or
'E'. It's a kind of ugly method but the only solution I could come up
with.
The problem with this solution is that it is not thread-safe. Perhaps
you can use a per-PGconn boolean?
--
Bruce Momjian http://candle.pha.pa.us
EnterpriseDB http://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Bruce Momjian <pgman@candle.pha.pa.us> writes:
Volkan YAZICI wrote:
The problem is, AFAICS, it's not possible to distinguish between a tuple
returning query (T, ..., C, Z or T, E) and a description of a portal (T,
Z). Therefore, I've created a global flag (parsing_row_desc) which is
turned on when we receive a 'T' and turned off if we receive a 'C' or
'E'. It's a kind of ugly method but the only solution I could come up
with.
The problem with this solution is that it is not thread-safe. Perhaps
you can use a per-PGconn boolean?
The whole thing sounds like brute force to me. Shouldn't you be adding
states to enum PGQueryClass, if you need to track what sort of Describe
you're doing?
regards, tom lane
On Jun 16 08:21, Tom Lane wrote:
Bruce Momjian <pgman@candle.pha.pa.us> writes:
Volkan YAZICI wrote:
The problem is, AFAICS, it's not possible to distinguish between a tuple
returning query (T, ..., C, Z or T, E) and a description of a portal (T,
Z). Therefore, I've created a global flag (parsing_row_desc) which is
turned on when we receive a 'T' and turned off if we receive a 'C' or
'E'. It's a kind of ugly method but the only solution I could come up
with.The problem with this solution is that it is not thread-safe. Perhaps
you can use a per-PGconn boolean?
Ie replaced the static flag with a conn->queryclass value using
PGQueryClass as Tom suggested. Also updated patch to be compatible with
exports.txt stuff.
The whole thing sounds like brute force to me. Shouldn't you be adding
states to enum PGQueryClass, if you need to track what sort of Describe
you're doing?
I totally agree with the followed ugly style. But IMHO the recursive
parsing (that is followed in pqParseInputN()) of received data is the main
problem behind this. I think, it will even get harder everytime somebody
try to to add another type of message parsing capability to that loop.
For instance, isn't pollution of PGQueryClass with state variables (like
PGQUERY_PREPARE or PGQUERY_DESCRIBE) one of the proofs of this.
While playing with pqParseInputN loops, I feel like coding Lisp recursions
using C syntax; it's quite ridiculous.
Regards.
Attachments:
libpq_desc_msg.patch.5text/plain; charset=us-asciiDownload
Index: src/backend/tcop/postgres.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/tcop/postgres.c,v
retrieving revision 1.489
diff -c -r1.489 postgres.c
*** src/backend/tcop/postgres.c 20 Jun 2006 22:52:00 -0000 1.489
--- src/backend/tcop/postgres.c 24 Jun 2006 11:31:10 -0000
***************
*** 1853,1858 ****
--- 1853,1859 ----
static void
exec_describe_statement_message(const char *stmt_name)
{
+ MemoryContext oldContext;
PreparedStatement *pstmt;
TupleDesc tupdesc;
ListCell *l;
***************
*** 1865,1871 ****
start_xact_command();
/* Switch back to message context */
! MemoryContextSwitchTo(MessageContext);
/* Find prepared statement */
if (stmt_name[0] != '\0')
--- 1866,1872 ----
start_xact_command();
/* Switch back to message context */
! oldContext = MemoryContextSwitchTo(MessageContext);
/* Find prepared statement */
if (stmt_name[0] != '\0')
***************
*** 1923,1929 ****
--- 1924,1933 ----
NULL);
else
pq_putemptymessage('n'); /* NoData */
+
+ MemoryContextSwitchTo(oldContext);
+ finish_xact_command();
}
/*
***************
*** 1934,1939 ****
--- 1938,1944 ----
static void
exec_describe_portal_message(const char *portal_name)
{
+ MemoryContext oldContext;
Portal portal;
/*
***************
*** 1943,1949 ****
start_xact_command();
/* Switch back to message context */
! MemoryContextSwitchTo(MessageContext);
portal = GetPortalByName(portal_name);
if (!PortalIsValid(portal))
--- 1948,1954 ----
start_xact_command();
/* Switch back to message context */
! oldContext = MemoryContextSwitchTo(MessageContext);
portal = GetPortalByName(portal_name);
if (!PortalIsValid(portal))
***************
*** 1975,1980 ****
--- 1980,1989 ----
portal->formats);
else
pq_putemptymessage('n'); /* NoData */
+
+ MemoryContextSwitchTo(oldContext);
+
+ finish_xact_command();
}
Index: src/interfaces/libpq/exports.txt
===================================================================
RCS file: /projects/cvsroot/pgsql/src/interfaces/libpq/exports.txt,v
retrieving revision 1.11
diff -c -r1.11 exports.txt
*** src/interfaces/libpq/exports.txt 28 May 2006 22:42:05 -0000 1.11
--- src/interfaces/libpq/exports.txt 24 Jun 2006 11:31:10 -0000
***************
*** 130,132 ****
--- 130,134 ----
PQencryptPassword 128
PQisthreadsafe 129
enlargePQExpBuffer 130
+ PQdescribePrepared 131
+ PQdescribePortal 132
Index: src/interfaces/libpq/fe-exec.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/interfaces/libpq/fe-exec.c,v
retrieving revision 1.186
diff -c -r1.186 fe-exec.c
*** src/interfaces/libpq/fe-exec.c 28 May 2006 21:13:54 -0000 1.186
--- src/interfaces/libpq/fe-exec.c 24 Jun 2006 11:31:12 -0000
***************
*** 61,66 ****
--- 61,68 ----
static void parseInput(PGconn *conn);
static bool PQexecStart(PGconn *conn);
static PGresult *PQexecFinish(PGconn *conn);
+ static int pqDescribe(PGconn *conn, const char desc_type,
+ const char *desc_target);
/* ----------------
***************
*** 2304,2309 ****
--- 2306,2415 ----
return 0;
}
+
+ /*
+ * pqDescribe - Describe given prepared statement or portal.
+ *
+ * Available options for target_type are
+ * 'S' to describe a prepared statement; or
+ * 'P' to describe a portal.
+ * Returns 0 on success and 1 on failure.
+ *
+ * By issuing a PQgetResult(), response from the server will be placed
+ * in an empty PGresult which will be extractable via PQf*() function family.
+ */
+ static int
+ pqDescribe(PGconn *conn, const char desc_type, const char *desc_target)
+ {
+ int ret;
+
+ /* This isn't gonna work on a 2.0 server. */
+ if (PG_PROTOCOL_MAJOR(conn->pversion) < 3)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("function requires at least protocol version 3.0\n"));
+ return 1;
+ }
+
+ if (!conn)
+ return 1;
+
+ /* Clear the connection error message. */
+ resetPQExpBuffer(&conn->errorMessage);
+
+ /* Don't try to send if we know there's no live connection. */
+ if (conn->status != CONNECTION_OK)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("no connection to the server\n"));
+ return 1;
+ }
+
+ /* Can't send while already busy, either. */
+ if (conn->asyncStatus != PGASYNC_IDLE)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("another command is already in progress\n"));
+ return 1;
+ }
+
+ /* Initialize async result-accumulation state. */
+ conn->result = NULL;
+ conn->curTuple = NULL;
+
+ if (desc_target)
+ ret = (pqPutMsgStart('D', false, conn) < 0 ||
+ pqPutc(desc_type, conn) < 0 ||
+ pqPuts(desc_target, conn) < 0 ||
+ pqPutMsgEnd(conn) < 0);
+
+ /* NULL desc_target values will be treated as an empty string. */
+ else
+ ret = (pqPutMsgStart('D', false, conn) < 0 ||
+ pqPutc(desc_type, conn) < 0 ||
+ pqPutc('\0', conn) < 0 ||
+ pqPutMsgEnd(conn) < 0);
+
+ if (ret)
+ goto SendFailure;
+
+ /* Extended protocol requires a Sync message. */
+ if (pqPutMsgStart('S', false, conn) < 0 ||
+ pqPutMsgEnd(conn) < 0)
+ goto SendFailure;
+
+ /* Remember we'll parse a Describe response. */
+ conn->queryclass = PGQUERY_DESCRIBE;
+
+ /* Free last query string. */
+ if (conn->last_query)
+ {
+ free(conn->last_query);
+ conn->last_query = NULL;
+ }
+
+ /* Describe request is sent. */
+ conn->asyncStatus = PGASYNC_BUSY;
+ return 0;
+
+ SendFailure:
+ pqHandleSendFailure(conn);
+ return 1;
+ }
+
+ int
+ PQdescribePrepared(PGconn *conn, const char *stmt)
+ {
+ return pqDescribe(conn, 'S', stmt);
+ }
+
+ int
+ PQdescribePortal(PGconn *conn, const char *portal)
+ {
+ return pqDescribe(conn, 'P', portal);
+ }
+
+
/* PQsetnonblocking:
* sets the PGconn's database connection non-blocking if the arg is TRUE
* or makes it non-blocking if the arg is FALSE, this will not protect
***************
*** 2381,2386 ****
--- 2487,2493 ----
free(ptr);
}
+
/*
* PQfreeNotify - free's the memory associated with a PGnotify
*
Index: src/interfaces/libpq/fe-protocol3.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/interfaces/libpq/fe-protocol3.c,v
retrieving revision 1.26
diff -c -r1.26 fe-protocol3.c
*** src/interfaces/libpq/fe-protocol3.c 14 Mar 2006 22:48:23 -0000 1.26
--- src/interfaces/libpq/fe-protocol3.c 24 Jun 2006 11:31:13 -0000
***************
*** 45,50 ****
--- 45,51 ----
static void handleSyncLoss(PGconn *conn, char id, int msgLength);
static int getRowDescriptions(PGconn *conn);
+ static int getParamDescriptions(PGconn *conn);
static int getAnotherTuple(PGconn *conn, int msgLength);
static int getParameterStatus(PGconn *conn);
static int getNotify(PGconn *conn);
***************
*** 81,87 ****
if (pqGetc(&id, conn))
return;
if (pqGetInt(&msgLength, 4, conn))
! return;
/*
* Try to validate message type/length here. A length less than 4 is
--- 82,88 ----
if (pqGetc(&id, conn))
return;
if (pqGetInt(&msgLength, 4, conn))
! return;
/*
* Try to validate message type/length here. A length less than 4 is
***************
*** 206,211 ****
--- 207,213 ----
}
strncpy(conn->result->cmdStatus, conn->workBuffer.data,
CMDSTATUS_LEN);
+
conn->asyncStatus = PGASYNC_READY;
break;
case 'E': /* error return */
***************
*** 216,221 ****
--- 218,236 ----
case 'Z': /* backend is ready for new query */
if (getReadyForQuery(conn))
return;
+
+ /*
+ * If there's any previosly available result and this is
+ * received from a Describe query's response, consume it
+ * first.
+ */
+ if (conn->result && conn->queryclass == PGQUERY_DESCRIBE)
+ {
+ conn->asyncStatus = PGASYNC_READY;
+ return;
+ }
+
+ /* Backend is ready for a new query. */
conn->asyncStatus = PGASYNC_IDLE;
break;
case 'I': /* empty query */
***************
*** 294,299 ****
--- 309,338 ----
conn->result = PQmakeEmptyPGresult(conn,
PGRES_COMMAND_OK);
break;
+ case 't': /* Parameter Description */
+ /* Parse after a pqDescribe call only. */
+ if (conn->queryclass == PGQUERY_DESCRIBE)
+ {
+ if (!conn->result)
+ {
+ /* A new PGresult will be created to load parsed data into. */
+ if (getParamDescriptions(conn))
+ return;
+
+ /* Result is ready. */
+ conn->asyncStatus = PGASYNC_READY;
+ }
+ else
+ {
+ /*
+ * We'll wait until the application accepts the current
+ * existing result.
+ */
+ conn->asyncStatus = PGASYNC_READY;
+ return;
+ }
+ }
+ break;
case 'D': /* Data Row */
if (conn->result != NULL &&
conn->result->resultStatus == PGRES_TUPLES_OK)
***************
*** 500,505 ****
--- 539,611 ----
}
/*
+ * parseInput subroutine to read a 't' (ParameterDescription) message.
+ * Returns 0 if parsing is completed, 1 if there isn't enough data yet
+ * and EOF if an error occurs.
+ *
+ * Parsed data will get loaded into the existing conn->result.
+ */
+ static int
+ getParamDescriptions(PGconn *conn)
+ {
+ PGresult *res;
+ int nattr;
+ int i;
+ Oid typid;
+
+ res = PQmakeEmptyPGresult(conn, PGRES_COMMAND_OK);
+ if (!res)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ goto Error;
+ }
+
+ /* First extracting number of attributes from the message. */
+ if (pqGetInt(&nattr, 2, conn) < 0)
+ goto NotEnoughData;
+ res->numAttributes = nattr;
+
+ /* Allocate space for the attribute descriptors. */
+ if (nattr > 0)
+ {
+ int sz = nattr * sizeof(PGresAttDesc);
+
+ res->attDescs = (PGresAttDesc *) pqResultAlloc(res, sz, TRUE);
+ if (!res->attDescs)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ goto Error;
+ }
+
+ /* Reset attribute values. */
+ MemSet(res->attDescs, 0, sz);
+ }
+
+ /* Loop through attribute's type OIDs. */
+ for (i = 0; i < nattr; i++)
+ {
+ if (pqGetInt(&typid, 4, conn) < 0)
+ goto NotEnoughData;
+ res->attDescs[i].typid = typid;
+ }
+
+ /* Success. */
+ conn->result = res;
+ return 0;
+
+ NotEnoughData:
+ PQclear(res);
+ return 1;
+
+ Error:
+ if (res)
+ PQclear(res);
+ return EOF;
+ }
+
+ /*
* parseInput subroutine to read a 'D' (row data) message.
* We add another tuple to the existing PGresult structure.
* Returns: 0 if completed message, EOF if error or not enough data yet.
Index: src/interfaces/libpq/libpq-fe.h
===================================================================
RCS file: /projects/cvsroot/pgsql/src/interfaces/libpq/libpq-fe.h,v
retrieving revision 1.129
diff -c -r1.129 libpq-fe.h
*** src/interfaces/libpq/libpq-fe.h 23 May 2006 22:13:19 -0000 1.129
--- src/interfaces/libpq/libpq-fe.h 24 Jun 2006 11:31:14 -0000
***************
*** 407,412 ****
--- 407,416 ----
extern int PQgetlength(const PGresult *res, int tup_num, int field_num);
extern int PQgetisnull(const PGresult *res, int tup_num, int field_num);
+ /* Describe prepared statements and portals. */
+ extern int PQdescribePrepared(PGconn *conn, const char *stmt);
+ extern int PQdescribePortal(PGconn *conn, const char *portal);
+
/* Delete a PGresult */
extern void PQclear(PGresult *res);
Index: src/interfaces/libpq/libpq-int.h
===================================================================
RCS file: /projects/cvsroot/pgsql/src/interfaces/libpq/libpq-int.h,v
retrieving revision 1.113
diff -c -r1.113 libpq-int.h
*** src/interfaces/libpq/libpq-int.h 21 May 2006 20:19:23 -0000 1.113
--- src/interfaces/libpq/libpq-int.h 24 Jun 2006 11:31:14 -0000
***************
*** 193,199 ****
{
PGQUERY_SIMPLE, /* simple Query protocol (PQexec) */
PGQUERY_EXTENDED, /* full Extended protocol (PQexecParams) */
! PGQUERY_PREPARE /* Parse only (PQprepare) */
} PGQueryClass;
/* PGSetenvStatusType defines the state of the PQSetenv state machine */
--- 193,200 ----
{
PGQUERY_SIMPLE, /* simple Query protocol (PQexec) */
PGQUERY_EXTENDED, /* full Extended protocol (PQexecParams) */
! PGQUERY_PREPARE, /* Parse only (PQprepare) */
! PGQUERY_DESCRIBE /* Parse only (pqDescribe) */
} PGQueryClass;
/* PGSetenvStatusType defines the state of the PQSetenv state machine */
On Sat, Jun 24, 2006 at 02:45:33PM +0300, Volkan YAZICI wrote:
I totally agree with the followed ugly style. But IMHO the recursive
parsing (that is followed in pqParseInputN()) of received data is the main
problem behind this. I think, it will even get harder everytime somebody
try to to add another type of message parsing capability to that loop.
For instance, isn't pollution of PGQueryClass with state variables (like
PGQUERY_PREPARE or PGQUERY_DESCRIBE) one of the proofs of this.
What's the alternative? pqParseInputN() work using state machines, but
they're not recursive. You're trying to parse messages where you don't
know beforehand if you have enough data. Moreover, each message could
be quite large, you don't want to have to store all of them without
parsing what you can. You're also not allowed to wait for more data to
appear.
However, it seems to me you could simplify quite a bit of coding by
adding a pqHaveNBytes function that returns true if there are that many
bytes available. Then right after you know the number of attributes,
you can do a pqHaveNBytes(4*nattr) and skip the checking within the
loop.
Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
Show quoted text
From each according to his ability. To each according to his ability to litigate.
Volkan YAZICI <yazicivo@ttnet.net.tr> writes:
[ patch to add PQdescribePrepared and PQdescribePortal ]
After looking this over, I don't see the point of PQdescribePortal,
at least not without adding other functionality to libpq. There is
no functionality currently exposed by libpq that allows creating a
portal (that is, sending a Bind message) without also executing the
portal. And the execution always returns the portal description.
So I don't see why you'd use it.
PQdescribePrepared is useful though, as it plugs the feature omission
mentioned in the description of PQprepare, namely, you can't find out
what datatype was inferred for a parameter that you didn't specify a
type for.
My inclination is to add PQdescribePrepared, but leave out
PQdescribePortal until such time as we decide to add functions
to libpq that support separate Bind and Execute operations.
(That might be never, seeing that no one's gotten around to it
since 7.4...)
The patch is missing an asynchronous version of PQdescribePrepared.
I'm not real sure what to call it --- the naming conventions we've
used in libpq are not as orthogonal as one could wish.
PQsendDescribePrepared is the best I can manage; anyone have a better
idea?
Also, we could take a completely different tack, which is to not invent
new functions but instead fold this functionality into PQprepare and
PQsendPrepare. What Volkan's done with this patch is to define the
successful result of PQdescribePrepared as being a PGresult in which
only the number of columns and their datatypes (PQnfields and PQftype)
are meaningful. We could perfectly well use that convention in the
PGresults returned by PQprepare/PQsendPrepare. The upside of this
method is that it wouldn't require an extra network round trip to get
the information (because we'd just include the Describe Statement
request in the original Prepare packet). The downside is that we'd
always spend the time to perform Describe Statement, even if the
application doesn't need it. However I'd expect that time to be
pretty minimal in comparison to the other costs of a Prepare.
I'm leaning slightly to the fold-it-into-PQprepare way, but am by
no means set on that. Comments anyone?
regards, tom lane
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I'm leaning slightly to the fold-it-into-PQprepare way, but am by
no means set on that. Comments anyone?
As a heavy user of libpq via DBD::Pg, +1 to folding in.
- --
Greg Sabino Mullane greg@turnstep.com greg@endpoint.com
End Point Corporation
PGP Key: 0x14964AC8 200608101212
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----
iD8DBQFE21r4vJuQZxSWSsgRAh9VAJ9YBooLrf27LfOXo5JYheASXb1ytwCfbRKv
sUhZ6HAsL7Stbatoxhdp4GY=
=ttAV
-----END PGP SIGNATURE-----
"Greg Sabino Mullane" <greg@turnstep.com> writes:
I'm leaning slightly to the fold-it-into-PQprepare way, but am by
no means set on that. Comments anyone?
As a heavy user of libpq via DBD::Pg, +1 to folding in.
Another thought: I looked into the protocol description and was reminded
that Describe Statement actually returns both ParameterDescription and
RowDescription, ie, both the list of parameter datatypes and the list
of column names and types that will be returned by the eventual
execution of the statement. So another theory about how this ought to
work is that PQprepare's result PGresult ought to carry the column
name/type info where PQfname and PQftype can get them, and then we'd
have to have two new PGresult-inspection functions to pull out the
separately stored parameter-datatype info. This seems much cleaner than
overloading the meaning of PQftype, but OTOH it's yet a few more cycles
added to the execution cost of PQprepare. Anyone have a need to get the
result type info during PQprepare?
regards, tom lane
On Thu, Aug 10, 2006 at 12:31:52PM -0400, Tom Lane wrote:
"Greg Sabino Mullane" <greg@turnstep.com> writes:
I'm leaning slightly to the fold-it-into-PQprepare way, but am by
no means set on that. Comments anyone?As a heavy user of libpq via DBD::Pg, +1 to folding in.
Another thought: I looked into the protocol description and was
reminded that Describe Statement actually returns both
ParameterDescription and RowDescription, ie, both the list of
parameter datatypes and the list of column names and types that will
be returned by the eventual execution of the statement. So another
theory about how this ought to work is that PQprepare's result
PGresult ought to carry the column name/type info where PQfname and
PQftype can get them, and then we'd have to have two new
PGresult-inspection functions to pull out the separately stored
parameter-datatype info. This seems much cleaner than overloading
the meaning of PQftype, but OTOH it's yet a few more cycles added to
the execution cost of PQprepare. Anyone have a need to get the
result type info during PQprepare?
It could be handy. Perhaps a different version (or different options
to) PQprepare for those who do?
Cheers,
D
--
David Fetter <david@fetter.org> http://fetter.org/
phone: +1 415 235 3778 AIM: dfetter666
Skype: davidfetter
Remember to vote!
On Aug 10 11:35, Tom Lane wrote:
Volkan YAZICI <yazicivo@ttnet.net.tr> writes:
[ patch to add PQdescribePrepared and PQdescribePortal ]
After looking this over, I don't see the point of PQdescribePortal,
at least not without adding other functionality to libpq. There is
no functionality currently exposed by libpq that allows creating a
portal (that is, sending a Bind message) without also executing the
portal. And the execution always returns the portal description.
So I don't see why you'd use it.PQdescribePrepared is useful though, as it plugs the feature omission
mentioned in the description of PQprepare, namely, you can't find out
what datatype was inferred for a parameter that you didn't specify a
type for.My inclination is to add PQdescribePrepared, but leave out
PQdescribePortal until such time as we decide to add functions
to libpq that support separate Bind and Execute operations.
(That might be never, seeing that no one's gotten around to it
since 7.4...)
My intention while implementing PQdescribePortal() was to gather
information about a portal created by an explicit DECLARE ... CURSOR
query. In case of connections are persistenly established with some pool
mechanism, it can be handy to be able to learn will be returned row
descriptions from an existing portal.
The patch is missing an asynchronous version of PQdescribePrepared.
I'm not real sure what to call it --- the naming conventions we've
used in libpq are not as orthogonal as one could wish.
PQsendDescribePrepared is the best I can manage; anyone have a better
idea?Also, we could take a completely different tack, which is to not invent
new functions but instead fold this functionality into PQprepare and
PQsendPrepare. What Volkan's done with this patch is to define the
successful result of PQdescribePrepared as being a PGresult in which
only the number of columns and their datatypes (PQnfields and PQftype)
are meaningful. We could perfectly well use that convention in the
PGresults returned by PQprepare/PQsendPrepare. The upside of this
method is that it wouldn't require an extra network round trip to get
the information (because we'd just include the Describe Statement
request in the original Prepare packet). The downside is that we'd
always spend the time to perform Describe Statement, even if the
application doesn't need it. However I'd expect that time to be
pretty minimal in comparison to the other costs of a Prepare.I'm leaning slightly to the fold-it-into-PQprepare way, but am by
no means set on that. Comments anyone?
IMHO, it isn't the only use case of Description messages for prepared
queries to learn the infered types just after a PQprepare() call. I
think it would be quite handy to be able to gather information about a
prepared stmt in later phases of an application. For instance one might
need to get the parameter and row types of a prepared query that he/she
isn't created. If we'd place Describe message facility into PQprepare(),
then we'll just lose that functionality of the feature.
OTOH, moving Describe data processing into the PQprepare() is fairly
conventional for introducing a new functionality at the same time
keeping the API consistent without raising any compatibility problems.
But AFAICS, that's not possible without giving over one of the features
of Describe messages for prepared statements: parameter types
information or row types information. Because, if we consider placing
Describe facility into PQprepare(), client would have to issue two
distinct PQgetResult() calls; one for parameter types and another one
for row types.
On Aug 10 12:31, Tom Lane wrote:
So another theory about how this ought to work is that PQprepare's
result PGresult ought to carry the column name/type info where PQfname
and PQftype can get them, and then we'd have to have two new
PGresult-inspection functions to pull out the separately stored
parameter-datatype info.
Yes, that's another feasible approach to the solution. But this one too
has its own PITAs as the one mentioned above.
This seems much cleaner than overloading the meaning of PQftype, but
OTOH it's yet a few more cycles added to the execution cost of
PQprepare.
I think, placing Describe facility into PQprepare() will just obfuscate
the problem. In every approach we tried to place Describe into
PQprepare(), we needed to introduce new functions or broke compatibility
with the exisiting versions. ISTM, Describe features having their own
functions is the only fair solution I could come up with.
Anyone have a need to get the result type info during PQprepare?
I don't think so. And if one would ever need such an information, can
reach it quite easily via PQdescribePrepared().
Regards.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
think it would be quite handy to be able to gather information
about a prepared stmt in later phases of an application. For
instance one might need to get the parameter and row types of
a prepared query that he/she isn't created.
Prepared statements are not visible nor survivable outside of your
session, so this doesn't really make sense. If your application needs
the information, it can get it at prepare time.
Anyone have a need to get the result type info during PQprepare?
I don't think so. And if one would ever need such an information, can
reach it quite easily via PQdescribePrepared().
That's a good point, however, along with your other arguments. :) I
could live with either way.
- --
Greg Sabino Mullane greg@turnstep.com
End Point Corporation
PGP Key: 0x14964AC8 200608110849
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----
iD8DBQFE3Hz8vJuQZxSWSsgRAuJuAJ4z/LmnoLOXIoZcdSh0VFYCdBDMlwCfd3HW
8YOwN30Jb8jHGx/OOjWzPaQ=
=9tRi
-----END PGP SIGNATURE-----
On Aug 11 12:51, Greg Sabino Mullane wrote:
think it would be quite handy to be able to gather information about
a prepared stmt in later phases of an application. For instance one
might need to get the parameter and row types of a prepared query
that he/she isn't created.Prepared statements are not visible nor survivable outside of your
session, so this doesn't really make sense. If your application needs
the information, it can get it at prepare time.
What about persistent connections? Actually, I can give lots of corner
cases to support my idea but they're not that often used. I think, as
long as we'll break compatibility, placing Describe facility in the
PQprepare() is not the way to go.
Anyone have a need to get the result type info during PQprepare?
I don't think so. And if one would ever need such an information, can
reach it quite easily via PQdescribePrepared().That's a good point, however, along with your other arguments. :) I
could live with either way.
I'm just declined to break current PQprepare() or to introduce new
PGresult-processor functions for a feature (IMHO) that needs its own
function. But the general use case is the main fact that'll say the last
word.
Regards.
Volkan YAZICI <yazicivo@ttnet.net.tr> writes:
On Aug 11 12:51, Greg Sabino Mullane wrote:
Prepared statements are not visible nor survivable outside of your
session, so this doesn't really make sense. If your application needs
the information, it can get it at prepare time.
What about persistent connections? Actually, I can give lots of corner
cases to support my idea but they're not that often used. I think, as
long as we'll break compatibility, placing Describe facility in the
PQprepare() is not the way to go.
I think this viewpoint has pretty much carried the day, so the
PQdescribe functions should remain separate. However, it still seems
to me that it'd be a shame if PQdescribePrepared couldn't return the
statement's output column types, seeing that the backend is going to
pass that info to us anyway. So I propose storing the parameter type
info in a new section of a PGresult struct, and adding new PGresult
accessor functions PQnparams, PQparamtype (or maybe PQptype to follow
the existing PQftype precedent more closely) to fetch the parameter type
info. The existing functions PQnfields etc will fetch output-column
info. Aside from being more functional, this definition maintains the
principle of least surprise, in that the interpretation of a PGresult
from Describe isn't fundamentally different from a PGresult from a
regular query.
We also need async versions PQsendDescribePrepared and
PQsendDescribePortal, as I mentioned before.
Anyone have different suggestions for the names of these functions?
regards, tom lane
On Aug 16 11:37, Tom Lane wrote:
Volkan YAZICI <yazicivo@ttnet.net.tr> writes:
On Aug 11 12:51, Greg Sabino Mullane wrote:
Prepared statements are not visible nor survivable outside of your
session, so this doesn't really make sense. If your application needs
the information, it can get it at prepare time.What about persistent connections? Actually, I can give lots of corner
cases to support my idea but they're not that often used. I think, as
long as we'll break compatibility, placing Describe facility in the
PQprepare() is not the way to go.I think this viewpoint has pretty much carried the day, so the
PQdescribe functions should remain separate. However, it still seems
to me that it'd be a shame if PQdescribePrepared couldn't return the
statement's output column types, seeing that the backend is going to
pass that info to us anyway.
I think you have a misunderstanding about the patch I previously sent.
When you issue a PQdescribePrepared() call, in the first PQgetResult()
call returned PGresult will have the input parameter types of the
prepared statement. And in the second PQgetResult() call, returned
PGresult will hold statement's output column types.
So I propose storing the parameter type
info in a new section of a PGresult struct, and adding new PGresult
accessor functions PQnparams, PQparamtype (or maybe PQptype to follow
the existing PQftype precedent more closely) to fetch the parameter type
info. The existing functions PQnfields etc will fetch output-column
info. Aside from being more functional, this definition maintains the
principle of least surprise, in that the interpretation of a PGresult
from Describe isn't fundamentally different from a PGresult from a
regular query.
Another possibility can be like this:
PGresult *PQdescribePrepared(PGconn *conn,
const char *stmt,
Oid **argtypes);
A PQdescribePrepared() call will immediatly return a PGresult
(previosly, we were just returning a boolean value that shows the result
of the command send status) result that holds statement's output column
types and argtypes will get altered to point to an Oid array that has
input parameter type information. (By assigning NULL value to argtypes,
user will decide to receive or not receive input parameter types.)
We also need async versions PQsendDescribePrepared and
PQsendDescribePortal, as I mentioned before.
If you decided on the method to use I'm volunteered to modify existing
patch. Waiting for your comments.
Regards.
Volkan YAZICI <yazicivo@ttnet.net.tr> writes:
On Aug 16 11:37, Tom Lane wrote:
I think this viewpoint has pretty much carried the day, so the
PQdescribe functions should remain separate. However, it still seems
to me that it'd be a shame if PQdescribePrepared couldn't return the
statement's output column types, seeing that the backend is going to
pass that info to us anyway.
I think you have a misunderstanding about the patch I previously sent.
When you issue a PQdescribePrepared() call, in the first PQgetResult()
call returned PGresult will have the input parameter types of the
prepared statement. And in the second PQgetResult() call, returned
PGresult will hold statement's output column types.
[ raised eyebrow... ] You're right, I didn't understand that, and now
that I do I find it completely unacceptable. We need exactly one
PGresult per operation, or things just get too weird for clients to
manage, particularly when considering async behavior. What you suggest
is a *huge* violation of the principle of least surprise. Adding a
couple more PGresult accessor functions seems far saner.
Another possibility can be like this:
PGresult *PQdescribePrepared(PGconn *conn,
const char *stmt,
Oid **argtypes);
No, because that doesn't work at all for the async case.
regards, tom lane
Volkan YAZICI <yazicivo@ttnet.net.tr> writes:
I've prepared a new patch that adds below commands to the libpq:
/* Accessor functions for PGresParamDesc field of PGresult. */
int PQnparams(const PGresult *res)
int PQparamType(const PGresult *res, int param_num)
/* Async functions. */
int PQsendDescribePrepared(PGconn *conn, const char *stmt)
int PQsendDescribePortal(PGconn *conn, const char *portal)
/* Synchronous ones. */
PGresult *PQdescribePrepared(PGconn *conn, const char *stmt)
PGresult *PQdescribePortal(PGconn *conn, const char *portal)
Applied with some small revisions to make it fit in better with the
existing libpq code (I don't think it desirable to have
copied-and-pasted versions of PQsendQueryStart, for instance).
I added some documentation also.
regards, tom lane
Import Notes
Reply to msg id not found: 20060818102305.GA1403@alamut.tdm.local