Proposal - libpq Type System beta-0.8a (was PGparam)

Hi,

Andrew Chernow wrote:

It might be something with the
attachment, who knows.

Most probably that was the case, yes. The -hackers list is limited,
please use -patches to send patches. ;-)

Regards

Markus

Markus Schiltknecht wrote:

Hi,

Andrew Chernow wrote:

It might be something with the attachment, who knows.

Most probably that was the case, yes. The -hackers list is limited,
please use -patches to send patches. ;-)

Regards

Markus

Noted.

In our case, its a little ambiguos whether -hackers or -patches is the
correct place. We are really posting a proposal that happens to have a
working implementation. We are looking for feedback and/or a discussion.

andrew

Andrew Chernow wrote:

Markus Schiltknecht wrote:

Hi,

Andrew Chernow wrote:

It might be something with the attachment, who knows.

Most probably that was the case, yes. The -hackers list is limited,
please use -patches to send patches. ;-)

Regards

Markus

Noted.

In our case, its a little ambiguos whether -hackers or -patches is the
correct place. We are really posting a proposal that happens to have
a working implementation. We are looking for feedback and/or a
discussion.

You should split it and send the proposal to -hackers. Ideally, you
would have had a proposal discussed before you wrote a line of code.

cheers

andrew

Andrew Dunstan wrote:

Andrew Chernow wrote:

Markus Schiltknecht wrote:

Hi,

Andrew Chernow wrote:

It might be something with the attachment, who knows.

Most probably that was the case, yes. The -hackers list is limited,
please use -patches to send patches. ;-)

Regards

Markus

Noted.

In our case, its a little ambiguos whether -hackers or -patches is the
correct place. We are really posting a proposal that happens to have
a working implementation. We are looking for feedback and/or a
discussion.

You should split it and send the proposal to -hackers. Ideally, you
would have had a proposal discussed before you wrote a line of code.

cheers

andrew

proposal discussed before you wrote a line of code

Yeah, we realize that. In our situation, we use this code internally
which is why it exists. Back in Aug 2007, we packaged it up and
proposed it because we thought it would be useful to others. Since
then, we have submitted several versions. Feedback was minimal.

In fact, only Tom has made suggestions, which we have taken into
consideration and adjusted the spec accordingly.

We are interested in having a discussion about the beta-0.8a proposal
and concept, not the implementation or submitting procedures. We
provided the code in case someone wants to take a test drive.

Andrew

On Jan 8, 2008 12:57 PM, Andrew Dunstan <andrew@dunslane.net> wrote:

You should split it and send the proposal to -hackers. Ideally, you
would have had a proposal discussed before you wrote a line of code.

This is the latest in a long series of submissions...check the
archives. We are (and have been) sensitive to the other pressures
resulting from the 8.3 release.

That said, we have been a little dismayed in the lack of comment. We
attributed this to either lack of interest or just general business
(we perhaps optimistically guessed the latter). We needed our
extensions for our own projects and are willing to maintain them
outside of the project if we have to...that is for the community to
decide. For the record, we are extremely excited about the libpq
changes and think others well be as well.

Tom noted the lack of documentation as well as a number of technical
issues. We addressed those issues (within the scope of what we wanted
to accomplish).

merlin

Merlin,

That said, we have been a little dismayed in the lack of comment.

I think most people can't really follow what functionality this would allow
users & driver authors to access, and what the spec for that functionality
would. I know I'm not clear on it.

A high-level proposal would arouse more general interest. Otherwise,
you'll just get a critique and eventually it'll either get applied or
rejected without much comment.

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

"Merlin Moncure" <mmoncure@gmail.com> writes:

That said, we have been a little dismayed in the lack of comment.

Personally, I'm ignoring this thread, as well as the one on
partitioning, because I'm up to my arse in 8.3-release alligators.
I'm going to try hard not to think about any 8.4 development issues
until 8.3 is actually out the door and we branch for 8.4 development.

I can't say how many other people are equally pressed for time ...
but seeing the minimal attention that seems to be getting paid to
open 8.3 issues, it doesn't look to me like the community as a whole
has a lot of spare cycles right now.

regards, tom lane

On Jan 8, 2008 4:31 PM, Josh Berkus <josh@agliodbs.com> wrote:

Merlin,

That said, we have been a little dismayed in the lack of comment.

I think most people can't really follow what functionality this would allow
users & driver authors to access, and what the spec for that functionality
would. I know I'm not clear on it.

A high-level proposal would arouse more general interest. Otherwise,
you'll just get a critique and eventually it'll either get applied or
rejected without much comment.

There are over 1500 lines of documentation attached in the patch.
That's a good place to start learning (or the attached regression
test, if you want to get right to it). There's a lot of material to
cover. Here's a very high level overview of the functionality:

Goal:
The ultimate goal is to be able to put parameters for command
execution and get results in a consistent way. The wire format
(either text or binary) is completely abstracted. We added many
functions to facilitate this, but the core functionality comes from
two varargs style functions, PQgetf and PQputf, and the PGparam, which
complements PGresult.

Features:
*) Binary transfer of all built in types is supported in both
directions. Basically, we marshal backend wire format to/from C types
(some native, some were introduced). We also support text results so
you can pull data in a consistent interface.
*) For user types (with custom send/recv functions), functions can be
registered to marshal them through a type registration interface.
*) Arrays and composites are supported automatically (composites have
to be 'registered').
*) Client side handlers can be aliased (domains) or subclassed...type
handlers can chained together for special handling and/or conversion
to exotic application types.

Here is a short example which demonstrates some of the major features.
There are many other examples and discussions of minutia in the
documentation.

int resfmt = 1; /* binary format */

/* Put an int4 and a text */
PGparam *param = PQparamCreate(conn);
PQputf(param, "%int4 %text", 2000, "foobar");
PQparamExec(conn, param, resfmt,
"insert into foo(id, user) values ($1, $2)");

/* The above as a one liner, internally 'puts' for you */
res = PQexecParamsf(conn, resfmt,
"insert into foo(id, user) values (%int4, %text)",
2000, "foobar");

int i4;
char *text;
PGresult *res = PQparamExec(conn, NULL, resfmt, "select * from foo limit 1");

/* From tuple 0, get an int4 at field 0 and a text
* from the user field
* '%' denotes by field num, '#' by field name
*/
PQgetf(res, 0, "%int4 #text*", 0, &i4, "user", &text);

note the above line is not wired to binary, text results would be fine as well.

/* let's get an array */

PGresult *res = PQparamExec(conn, NULL, resfmt, "select current_schemas(true)");

/* pop an array object out of the result. it creates a new result
* with one field and one 'tuple' for each array element.
*
* arrays of composites return one field for each attribute of the composite.
*/
PGarray array;
PQgetf(res, 0, "%name[]", 0, &array);
PQclear(res);

for (i = 0; i < PQntuples(array.res); i++)
{
char *name;
PQgetf(array.res, i, "%name*", 0, &name);
printf("%s\n", name);
}
PQclear(array.res);

/* return data from composite type which we create and register */
CREATE TYPE merlin as (a int, t text); -- on server
PGresult *merlin;
PQregisterTypeHandler(conn, "merlin", NULL, NULL);
res = PQparamExec(conn, NULL, resfmt, "select (1, 'abc')::merlin");
PQgetf(res, 0, "%merlin", 0, &merlin);
PQclear(res);
PQgetf(merlin, 0, "%int4 #text*", 0, &i4, "t", &text);
PQclear(merlin);

merlin

Tom Lane wrote:

"Merlin Moncure" <mmoncure@gmail.com> writes:

That said, we have been a little dismayed in the lack of comment.

Personally, I'm ignoring this thread, as well as the one on
partitioning, because I'm up to my arse in 8.3-release alligators.
I'm going to try hard not to think about any 8.4 development issues
until 8.3 is actually out the door and we branch for 8.4 development.

I can't say how many other people are equally pressed for time ...
but seeing the minimal attention that seems to be getting paid to
open 8.3 issues, it doesn't look to me like the community as a whole
has a lot of spare cycles right now.

regards, tom lane

Thanks for the heads up. We had a feeling the 8.3 crunch was a factor.
We don't want to slow that down because we were are waiting for 8.3 to
perform yet another mysql to postgresql migration :)

andrew

On Tue, Jan 08, 2008 at 05:33:51PM -0500, Merlin Moncure wrote:

Here is a short example which demonstrates some of the major features.
There are many other examples and discussions of minutia in the
documentation.

I havn't looked at the source but FWIW I think it's an awesome idea.

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/