PHP and PostgreSQL

Bruce Momjian wrote:

I have been asked by the major PHP developer Rasmus Lerdorf to see if
the PostgreSQL/PHP interface needs any improvements.

Is the current PostgreSQL interface module in PHP adequate? Does it
support all the current libpq features?

If not, would someone submit some patches to the PHP folks. They want
us to work well with PHP. They are basically encouraging us to improve
it in any way we can.

In a current project, I am using Postgres, Oracle, and PHP. I think
PHP's API is very complete and usable, I can't think of anything we've
run into that was a result of PHP lacking a feature.

The only thing I'd like to see would be an Oracle construct moved to
Postgres. The idea that we can "prepare" a statement then execute it
multiple times without going through the planner more than once. This
would also require the ability to create and set arbitrary variables in
a Postgres session.

Since these are programmatic queries, only a handful of different SQL
queries are ever executed. The ability to cache the planned query tree
(is this the right terminology?) in a shared block, and simply reuse it,
in the multiple back-ends, with a new variable value, would speed up
some of the more complex queries, I guess.

--
http://www.mohawksoft.com

I have been asked by the major PHP developer Rasmus Lerdorf to see
if
the PostgreSQL/PHP interface needs any improvements.

Is the current PostgreSQL interface module in PHP adequate? Does it
support all the current libpq features?

The only problem we have run into (and I have heard of others having this
problem also) is with persistent connections. I have seen discussion on
persistent connection problems but I'm not sure the problem was ever
resolved. The problem we have seen is that when using persistent
connections the web server doesn't seen to reuse the connections or somthing
to that effect. The result being that we eventually use up our postgres
limit of 48 connections and nothing can connect to postgre anymore. It is
possible that this is a configuration problem that we haven't sufficiently
investigated, but I meniton it because I have heard other talk of this.
Anyone have more information?

Matthew

Yeah, this is not a "bug", but a feature. :)

In a nutshell, the persistent connection number is not how many persistent
connections there are available to the webserver. It is the number
available per webserver PROCESS.

So, if you have 5 persistent connections, but 10 webserver processes open,
you have 50 persistent connections open.

Adam Lang
Systems Engineer
Rutgers Casualty Insurance Company
http://www.rutgersinsurance.com
----- Original Message -----
From: "Matthew" <matt@ctlno.com>
To: "'Bruce Momjian'" <pgman@candle.pha.pa.us>; "PostgreSQL-development"
<pgsql-hackers@postgresql.org>; "PostgreSQL-interfaces"
<pgsql-interfaces@postgresql.org>
Sent: Wednesday, December 27, 2000 8:54 AM
Subject: [INTERFACES] RE: [HACKERS] PHP and PostgreSQL

I have been asked by the major PHP developer Rasmus Lerdorf to see
if
the PostgreSQL/PHP interface needs any improvements.

Is the current PostgreSQL interface module in PHP adequate? Does it
support all the current libpq features?

The only problem we have run into (and I have heard of others having this
problem also) is with persistent connections. I have seen discussion on
persistent connection problems but I'm not sure the problem was ever
resolved. The problem we have seen is that when using persistent
connections the web server doesn't seen to reuse the connections or

somthing

Bruce Momjian wrote:

I have been asked by the major PHP developer Rasmus Lerdorf to see if
the PostgreSQL/PHP interface needs any improvements.

Is the current PostgreSQL interface module in PHP adequate? Does it
support all the current libpq features?

If not, would someone submit some patches to the PHP folks. They want
us to work well with PHP. They are basically encouraging us to improve
it in any way we can.

If I can put my 0.02 Euros, I think there are two issues that deserve to
be fixed, both are relative to persistent connections.

The first is that the transaction state is (obviously) kept between
connections. This is not a problem in itself, but there are some
situations where the transaction block is not closed with COMMIT or
ABORT with the obvious consequencies when the connection is reused (and
with the locks not released). As a workaround I registered a shutdown
funcion that sends an ABORT command every time a script finishes his
processing but this results in a "Not in trasaction block" error most of
the time (harmless but pretty annoying).
IMHO there should be some sort of lightweight reset that puts the
connection in its pristine state without the need to fork a backend
againg and reopen the TCP connection.

The second is that if PostgreSQL is restarted, PHP doesn't detect that
and an error message is seen for every persistent connection. IMHO PHP
should detect a closed connection and clear the connections pool.

Bye!

--
Daniele

-------------------------------------------------------------------------------
Daniele Orlandi - Utility Line Italia - http://www.orlandi.com
Via Mezzera 29/A - 20030 - Seveso (MI) - Italy
-------------------------------------------------------------------------------

The only problem we have run into (and I have heard of others having this
problem also) is with persistent connections. I have seen discussion on
persistent connection problems but I'm not sure the problem was ever
resolved. The problem we have seen is that when using persistent
connections the web server doesn't seen to reuse the connections or

somthing

to that effect. The result being that we eventually use up our postgres
limit of 48 connections and nothing can connect to postgre anymore. It is
possible that this is a configuration problem that we haven't sufficiently
investigated, but I meniton it because I have heard other talk of this.
Anyone have more information?

The *real* problem with persistent connections is:

Script1: BEGIN;
Script1: UPDATE table set row = 'things';
Script2: Insert into table (id) values ('bad data');
Script1: COMMIT;

Since script2 managed to do a BAD insert in the middle of script1's
transaction, the transaction in script1 fails. Obvious solution? Don't do
connection sharing when a transaction is enabled. The whole persistent
connection thing is only valid for mysql as it's the only thing that doesn't
really support transactions (and even thats partially changed).

They need to look for stuff going through (keywords like BEGIN) and 'lock'
that connection to the single entity that opened it.

It's much easier to write your own. I wrote a few functions like:

get_connection('DB PARMS');
begin_transaction();

commit_transaction();
close_connection();

All of this is done in a class which has knowledge of all connections that a
script is currently using. Beginning a transaction locks down the
connection from use by any other handler, they're all bumped to another one.
Problem? It requires atleast 1 connection per page, but since they're
actually dropped at close it's not so bad.

Matthew wrote:

I have been asked by the major PHP developer Rasmus Lerdorf to see
if
the PostgreSQL/PHP interface needs any improvements.

Is the current PostgreSQL interface module in PHP adequate? Does it
support all the current libpq features?

The only problem we have run into (and I have heard of others having this
problem also) is with persistent connections. I have seen discussion on
persistent connection problems but I'm not sure the problem was ever
resolved. The problem we have seen is that when using persistent
connections the web server doesn't seen to reuse the connections or somthing
to that effect. The result being that we eventually use up our postgres
limit of 48 connections and nothing can connect to postgre anymore. It is
possible that this is a configuration problem that we haven't sufficiently
investigated, but I meniton it because I have heard other talk of this.
Anyone have more information?

I have not seen this, though now that you mention it, I will be on the
lookout.

We have a load balenced system with [n] apache servers, each with a
local Postgres database that receives updates from a master.

On the local web servers, we have done a good deal of load testing. Our
apache instances are pretty persistent, i.e. we do not have a time out
or maximum number of transactions. I never see more more postgres
instances than I have apache instances. We are using pg_pConnect to
connect to the postgres system.

The behavior may differ if the database is on a different system.

--
http://www.mohawksoft.com

I'd say the most important thing would be to get it upto speed with 7.1.
Make sure PHP supports large objects and the TOAST properly.

Adam Lang
Systems Engineer
Rutgers Casualty Insurance Company
http://www.rutgersinsurance.com
----- Original Message -----
From: "Bruce Momjian" <pgman@candle.pha.pa.us>
To: "PostgreSQL-development" <pgsql-hackers@postgresql.org>;
"PostgreSQL-interfaces" <pgsql-interfaces@postgresql.org>
Sent: Wednesday, December 27, 2000 12:56 AM
Subject: [INTERFACES] PHP and PostgreSQL

Matthew wrote:

I have been asked by the major PHP developer Rasmus Lerdorf to see
if
the PostgreSQL/PHP interface needs any improvements.

Is the current PostgreSQL interface module in PHP adequate? Does it
support all the current libpq features?

The only problem we have run into (and I have heard of others having this
problem also) is with persistent connections. I have seen discussion on
persistent connection problems but I'm not sure the problem was ever
resolved. The problem we have seen is that when using persistent
connections the web server doesn't seen to reuse the connections or somthing
to that effect. The result being that we eventually use up our postgres
limit of 48 connections and nothing can connect to postgre anymore. It is
possible that this is a configuration problem that we haven't sufficiently
investigated, but I meniton it because I have heard other talk of this.
Anyone have more information?

Persistent connections behave exactly as advertised. Each apache process
sets up and maintains persistent connections as needed. The problem is
that for a typical web server, there are so many subprocesses that
persistent connections are probably consume more resources than they
save, unless they are combined with connection pooling across ALL the
apache processes.

Implementation of connection pooling is by far the most serious
shortcoming of the current implementation, IMHO.

I would dearly love to see this addressed as our postgresql database
sees connections from about 300 servers for 6 databases. Since our
postgresql server cannot support 1800 simultaneous active backends,
persistent connections are useless without pooling. So instead we
initiate 10 or more backends every second for generally very simple
queries. Most of the queries are pretty simple, so I would not be at all
surprised if we sent more system resources opening connections than we
do actually answering queries

--
Karl DeBisschop kdebisschop@alert.infoplease.com
Learning Network/Information Please http://www.infoplease.com
Netsaint Plugin Developer kdebisschop@users.sourceforge.net

Adam Lang wrote:

I'd say the most important thing would be to get it upto speed with 7.1.
Make sure PHP supports large objects and the TOAST properly.

What would be the problem there? As I understand TOAST, the application shouldn't take any
notice of it's inner workings whatsoever . . . also, I've used TOAST with PHP (i.e. I've
managed to insert more than 8 k into a row with the rowsize set to 8 k), without any
trouble. I don't know about large objects but as I heard, the interface didn't change?!

- Frank

I have been searching without luck on a howto on how to create locales for
postgres. Could somebody please point me in the general direction? Thanks.

:wq
Tim Uckun
Due Diligence Inc. http://www.diligence.com/ Americas Background
Investigation Expert.
If your company isn't doing background checks, maybe you haven't considered
the risks of a bad hire.

I thought I saw mention on the interfaces list that the ODBC driver needed
to be modified to properly use the large objects.

Adam Lang
Systems Engineer
Rutgers Casualty Insurance Company
http://www.rutgersinsurance.com
----- Original Message -----
From: "Frank Joerdens" <frank@joerdens.de>
To: "Adam Lang" <aalang@rutgersinsurance.com>
Cc: "Bruce Momjian" <pgman@candle.pha.pa.us>; "PostgreSQL-development"
<pgsql-hackers@postgresql.org>; "PostgreSQL-interfaces"
<pgsql-interfaces@postgresql.org>
Sent: Wednesday, December 27, 2000 12:03 PM
Subject: Re: [INTERFACES] PHP and PostgreSQL

Adam Lang wrote:

I'd say the most important thing would be to get it upto speed with 7.1.
Make sure PHP supports large objects and the TOAST properly.

What would be the problem there? As I understand TOAST, the application

shouldn't take any

notice of it's inner workings whatsoever . . . also, I've used TOAST with

PHP (i.e. I've

managed to insert more than 8 k into a row with the rowsize set to 8 k),

without any

trouble. I don't know about large objects but as I heard, the interface

didn't change?!

I agree, but I figured that if the 7.1 upgrade breaks some of the large
object interface in the ODBC code, it may possibly break some of the
interface in the php-psql code. That's why I said it should probably be
checked. ;)

Adam Lang
Systems Engineer
Rutgers Casualty Insurance Company
http://www.rutgersinsurance.com
----- Original Message -----
From: "Frank Joerdens" <frank@joerdens.de>
To: "Adam Lang" <aalang@rutgersinsurance.com>
Cc: "PostgreSQL-development" <pgsql-hackers@postgresql.org>;
"PostgreSQL-interfaces" <pgsql-interfaces@postgresql.org>
Sent: Wednesday, December 27, 2000 5:22 PM
Subject: Re: [INTERFACES] PHP and PostgreSQL

Adam Lang wrote:

I thought I saw mention on the interfaces list that the ODBC driver

needed

Adam Lang wrote:

I thought I saw mention on the interfaces list that the ODBC driver needed
to be modified to properly use the large objects.

You normally wouldn't use ODBC to access Postgres from PHP but rather
the Postgres-specific interface. You _can_ use ODBC as well but normally
that doesn't provide any advantage. If your concern is database
abstraction, there are several libraries for PHP that provide it in
varying degrees, ODBC would be just one, possibly not optimal, solution
for the problem. Generally, from following the PHP lists and looking at
columns on, for instance, phpbuilder.com, PHP folks don't seem to
concerned with the abstraction issue, as far as I can make out anyway
(just my impression). Rasmus' question, IMHO, was about the PHP/Postgres
interface, not about ODBC.

- Frank

Adam Lang wrote:

I thought I saw mention on the interfaces list that the ODBC driver needed
to be modified to properly use the large objects.

You normally wouldn't use ODBC to access Postgres from PHP but rather
the Postgres-specific interface. You _can_ use ODBC as well but normally
that doesn't provide any advantage. If your concern is database
abstraction, there are several libraries for PHP that provide it in
varying degrees, ODBC would be just one, possibly not optimal, solution
for the problem. Generally, from following the PHP lists and looking at
columns on, for instance, phpbuilder.com, PHP folks don't seem to
concerned with the abstraction issue, as far as I can make out anyway
(just my impression).

- Frank

On Wed, Dec 27, 2000 at 12:56:26AM -0500, Bruce Momjian wrote:

I have been asked by the major PHP developer Rasmus Lerdorf to see if
the PostgreSQL/PHP interface needs any improvements.

Is the current PostgreSQL interface module in PHP adequate? Does it
support all the current libpq features?

If not, would someone submit some patches to the PHP folks. They want
us to work well with PHP. They are basically encouraging us to improve
it in any way we can.

I use PHP and Postgres together quite a bit, and find the APIs
complete enough for most things. However, just last week I implemented

pg_lolseek($loid, $offset $whence)
and
pg_lotell($loid)

For some stuff that we are working on. They are pretty straightforward,
and I can package them up and submit them if someone wants.

--
Adam Haberlach |A cat spends her life conflicted between a
adam@newsnipple.com |deep, passionate, and profound desire for
http://www.newsnipple.com |fish and an equally deep, passionate, and
'88 EX500 |profound desire to avoid getting wet.

You do not need create locale specially for postgres. Postgres uses
system locale. You could look at /usr/share/locale

regards,
Oleg
On Wed, 27 Dec 2000, Tim Uckun wrote:

Date: Wed, 27 Dec 2000 10:18:01 -0700
From: Tim Uckun <tim@diligence.com>
To: pgsql-hackers@postgresql.org
Subject: [HACKERS] Creating locales.

I have been searching without luck on a howto on how to create locales for
postgres. Could somebody please point me in the general direction? Thanks.

:wq
Tim Uckun
Due Diligence Inc. http://www.diligence.com/ Americas Background
Investigation Expert.
If your company isn't doing background checks, maybe you haven't considered
the risks of a bad hire.

_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

Tim Uckun wrote:

I have been searching without luck on a howto on how to create locales for
postgres. Could somebody please point me in the general direction? Thanks.

PostgreSQL uses system locales (even to the extent that I was unable to
get
it use any other locale by setting LC_* variables in
/etc/init.d/rc.d/postgres)

I did have to modify my locale (et_EE) to _not_ group \xff (&yumlaut;)
with
other y-s but to sort it after all other "standard" chars to work
properly
with LIKE optimisations. Perhaps the right approach would be to leave
\xff
alone and to choose some character in the 128-159 range for
bigger-than-any-other-char but as \xff is not used in estonian i was
able
to do away with just moving it.

OTOH, it may be a good idea to start a repository for locales that are
specifilally modified to work well with postgres.

We may want to move the locale stuff inside the backend someday as
mandated
by SQL standards and then some work would be already done.

--------
Hannu

Yes, please send them over to the PHP folks. Thanks.

On Wed, Dec 27, 2000 at 12:56:26AM -0500, Bruce Momjian wrote:

I have been asked by the major PHP developer Rasmus Lerdorf to see if
the PostgreSQL/PHP interface needs any improvements.

Is the current PostgreSQL interface module in PHP adequate? Does it
support all the current libpq features?

If not, would someone submit some patches to the PHP folks. They want
us to work well with PHP. They are basically encouraging us to improve
it in any way we can.

I use PHP and Postgres together quite a bit, and find the APIs
complete enough for most things. However, just last week I implemented

pg_lolseek($loid, $offset $whence)
and
pg_lotell($loid)

For some stuff that we are working on. They are pretty straightforward,
and I can package them up and submit them if someone wants.

--
Adam Haberlach |A cat spends her life conflicted between a
adam@newsnipple.com |deep, passionate, and profound desire for
http://www.newsnipple.com |fish and an equally deep, passionate, and
'88 EX500 |profound desire to avoid getting wet.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

This is interesting. I always wondered how the persistent connection
stuff handled this, and not I see that it doesn't.

[ Charset ISO-8859-1 unsupported, converting... ]

The only problem we have run into (and I have heard of others having this
problem also) is with persistent connections. I have seen discussion on
persistent connection problems but I'm not sure the problem was ever
resolved. The problem we have seen is that when using persistent
connections the web server doesn't seen to reuse the connections or

somthing

to that effect. The result being that we eventually use up our postgres
limit of 48 connections and nothing can connect to postgre anymore. It is
possible that this is a configuration problem that we haven't sufficiently
investigated, but I meniton it because I have heard other talk of this.
Anyone have more information?

The *real* problem with persistent connections is:

Script1: BEGIN;
Script1: UPDATE table set row = 'things';
Script2: Insert into table (id) values ('bad data');
Script1: COMMIT;

Since script2 managed to do a BAD insert in the middle of script1's
transaction, the transaction in script1 fails. Obvious solution? Don't do
connection sharing when a transaction is enabled. The whole persistent
connection thing is only valid for mysql as it's the only thing that doesn't
really support transactions (and even thats partially changed).

They need to look for stuff going through (keywords like BEGIN) and 'lock'
that connection to the single entity that opened it.

It's much easier to write your own. I wrote a few functions like:

get_connection('DB PARMS');
begin_transaction();

commit_transaction();
close_connection();

All of this is done in a class which has knowledge of all connections that a
script is currently using. Beginning a transaction locks down the
connection from use by any other handler, they're all bumped to another one.
Problem? It requires atleast 1 connection per page, but since they're
actually dropped at close it's not so bad.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026