Notify enhancement
Josh Berkus asked me recently what cool things I would be working on
now. I have just received the go ahead to work on this TODO item for the
next release:
Add optional textual message to NOTIFY
This would allow an informational message to be added to the notify
message, perhaps indicating the row modified or other custom information.
cheers
andrew
Andrew Dunstan wrote:
Josh Berkus asked me recently what cool things I would be working on
now. I have just received the go ahead to work on this TODO item for the
next release:Add optional textual message to NOTIFY
This would allow an informational message to be added to the notify
message, perhaps indicating the row modified or other custom information.
Great!
Are you also planning to do the removal of the pg_listener catalog?
There were some ideas on how to do this, but nothing thoroughly
designed.
Neil Conway talked about using SLRU as a mechanism for storing listeners
and messages. Maybe you can find that discussion in the archives. Part
of it was via IRC though, and I don't remember how much of it was. So
details on the archives may be thin.
--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.
Alvaro Herrera wrote:
Andrew Dunstan wrote:
Josh Berkus asked me recently what cool things I would be working on
now. I have just received the go ahead to work on this TODO item for the
next release:Add optional textual message to NOTIFY
This would allow an informational message to be added to the notify
message, perhaps indicating the row modified or other custom information.Great!
Are you also planning to do the removal of the pg_listener catalog?
There were some ideas on how to do this, but nothing thoroughly
designed.Neil Conway talked about using SLRU as a mechanism for storing listeners
and messages. Maybe you can find that discussion in the archives. Part
of it was via IRC though, and I don't remember how much of it was. So
details on the archives may be thin.
I have no fixed plans. The only thing I know is a) we need a message and
b) we need to change the semantics slightly so that events are no longer
collapsed.
If anyone has bright ideas for this, now is the time to air them. I
expect to begin work on it in a couple of weeks.
cheers
andrew
Andrew Dunstan wrote:
I have no fixed plans. The only thing I know is a) we need a message and
b) we need to change the semantics slightly so that events are no longer
collapsed.If anyone has bright ideas for this, now is the time to air them. I
expect to begin work on it in a couple of weeks.
I havo no bright ideas on this topic, but I'll willingly try update the
PHP API once things have settled down.
Best regards
--
Matteo Beccati
http://phpadsnew.com
http://phppgads.com
Alvaro Herrera a ecrit le 01/12/2006 00:19:
Andrew Dunstan wrote:
Josh Berkus asked me recently what cool things I would be working on
now. I have just received the go ahead to work on this TODO item for the
next release:Add optional textual message to NOTIFY
This would allow an informational message to be added to the notify
message, perhaps indicating the row modified or other custom information.Great!
+1
Are you also planning to do the removal of the pg_listener catalog?
There were some ideas on how to do this, but nothing thoroughly
designed.
I would love to see this coming (the removal of the pg_listener
catalog). It would help me to put this future release in my clients'
servers.
Neil Conway talked about using SLRU as a mechanism for storing listeners
and messages. Maybe you can find that discussion in the archives. Part
of it was via IRC though, and I don't remember how much of it was. So
details on the archives may be thin.
--
Guillaume.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I have no fixed plans. The only thing I know is a) we need a message and
b) we need to change the semantics slightly so that events are no longer
collapsed.
As long as things are backwards compatible, I'm fine with that, although
the "no longer collapsed" makes me a little nervous.
If anyone has bright ideas for this, now is the time to air them. I
expect to begin work on it in a couple of weeks.
Not necessarily related to your work, per se, but it would be nice if we
could finally liberate notify strings from the relation constraint and
make them a real string.
- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 200612031756
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----
iD8DBQFFc1YcvJuQZxSWSsgRAlrVAJ4hJiff3T/bGxg+FLAIH0X4e1GAJQCg0T5f
/TK3B/tMsRzoGYerK3W+fvg=
=i0qd
-----END PGP SIGNATURE-----
"Greg Sabino Mullane" <greg@turnstep.com> writes:
Not necessarily related to your work, per se, but it would be nice if we
could finally liberate notify strings from the relation constraint and
make them a real string.
What "relation constraint"? They can be any identifier you want.
regards, tom lane
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Tom Lane asked:
What "relation constraint"? They can be any identifier you want.
To clarify, it would be nice to use any arbitrary string (perhaps up
to NAMEDATALEN), rather than bumping into the rules for valid identifiers:
LISTEN notify.name.like.this;
LISTEN table.mutation;
It would also make things like this act more intuitively:
LISTEN nosuchschema.abc;
NOTIFY foo.bar.abc;
- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 200612031856
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----
iD8DBQFFc2WSvJuQZxSWSsgRAkQOAJ93c1ZiTx9h9Bho0YLpFmNwrRHVzACgp/tl
hi1sQRbEcFRpMNNbQQ7xnhM=
=LAI2
-----END PGP SIGNATURE-----
Greg Sabino Mullane wrote:
Tom Lane asked:
What "relation constraint"? They can be any identifier you want.
To clarify, it would be nice to use any arbitrary string (perhaps up
to NAMEDATALEN), rather than bumping into the rules for valid identifiers:LISTEN notify.name.like.this;
LISTEN table.mutation;
It would also make things like this act more intuitively:
LISTEN nosuchschema.abc;
NOTIFY foo.bar.abc;
If it's going to be an arbitrary string it will have to be single quoted,
like other string literals, and unlike identifiers. But I honestly can't
see why we would introduce such an unnecessary backwards incompatibility.
You can use $ as a delimiter if you want some sort of hierarchy of events,
e.g.
LISTEN foo$bar;
Incidentally, I think we will also need to limit the length of the message
string if we're going to store this in shared memory. I'm currently
thinking of NAMEDATALEN per message, but I am open to argument.
cheers
andrew
"Greg Sabino Mullane" <greg@turnstep.com> writes:
To clarify, it would be nice to use any arbitrary string (perhaps up
to NAMEDATALEN), rather than bumping into the rules for valid identifiers:
LISTEN notify.name.like.this;
So use double quotes:
LISTEN "notify.name.like.this";
I can't imagine why we'd change the syntax from "identifier" to "string
literal" --- it would break existing programs for essentially zero gain.
regards, tom lane
"Andrew Dunstan" <andrew@dunslane.net> writes:
Incidentally, I think we will also need to limit the length of the message
string if we're going to store this in shared memory. I'm currently
thinking of NAMEDATALEN per message, but I am open to argument.
I suppose you're envisioning a ring of fixed-size message buffers
similar to the sinval implementation. With two NAMEDATALEN items
per message this would be 128 bytes each, a lot of which would go unused
in typical applications ... but on the other hand I can foresee some
apps wishing they could send payload strings longer than NAMEDATALEN.
Seems like it would not be that much harder to allow variable-length
messages, remove the padding and avoid any hard limit on message size.
You'd have to track the "fill" and "empty" pointers at the level of
bytes not message numbers, but so what?
regards, tom lane
Tom Lane wrote:
"Andrew Dunstan" <andrew@dunslane.net> writes:
Incidentally, I think we will also need to limit the length of the message
string if we're going to store this in shared memory. I'm currently
thinking of NAMEDATALEN per message, but I am open to argument.I suppose you're envisioning a ring of fixed-size message buffers
similar to the sinval implementation. With two NAMEDATALEN items
per message this would be 128 bytes each, a lot of which would go unused
in typical applications ... but on the other hand I can foresee some
apps wishing they could send payload strings longer than NAMEDATALEN.Seems like it would not be that much harder to allow variable-length
messages, remove the padding and avoid any hard limit on message size.
You'd have to track the "fill" and "empty" pointers at the level of
bytes not message numbers, but so what?
Ok. But I think the buffer size as a whole needs to be fixed, no? And if
so, we probably need some limit on message size to prevent "NOTIFY
some_event 'a really long string'; " from filling up the buffer in one hit.
I'm also trying to figure out what a reasonable default buffer size will
be. Thinking of the needs for which I will be providing (one listener,
small names/payloads), 256Kb or 512Kb would be ample, possibly even
excessive. But other users might have bigger needs.
cheers
andrew
Andrew Dunstan wrote:
Ok. But I think the buffer size as a whole needs to be fixed, no? And if
so, we probably need some limit on message size to prevent "NOTIFY
some_event 'a really long string'; " from filling up the buffer in one hit.I'm also trying to figure out what a reasonable default buffer size will
be. Thinking of the needs for which I will be providing (one listener,
small names/payloads), 256Kb or 512Kb would be ample, possibly even
excessive. But other users might have bigger needs.
Make it configurable via GUC.
What will happen when the ring is full? NOTIFY blocks?
--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Alvaro Herrera wrote:
Andrew Dunstan wrote:
Ok. But I think the buffer size as a whole needs to be fixed, no? And if
so, we probably need some limit on message size to prevent "NOTIFY
some_event 'a really long string'; " from filling up the buffer in one hit.I'm also trying to figure out what a reasonable default buffer size will
be. Thinking of the needs for which I will be providing (one listener,
small names/payloads), 256Kb or 512Kb would be ample, possibly even
excessive. But other users might have bigger needs.Make it configurable via GUC.
That was my intention. The question was what the default should be.
What will happen when the ring is full? NOTIFY blocks?
An error, I think, don't you? Blocking seems totally unacceptable.
cheers
andrew
Andrew Dunstan wrote:
Alvaro Herrera wrote:
What will happen when the ring is full? NOTIFY blocks?
An error, I think, don't you? Blocking seems totally unacceptable.
Actually, to me the thing should be spilling to disk :-)
--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.
Andrew Dunstan <andrew@dunslane.net> writes:
Alvaro Herrera wrote:
What will happen when the ring is full? NOTIFY blocks?
An error, I think, don't you? Blocking seems totally unacceptable.
That seems like a curious answer. Imagine the documentation:
"NOTIFY may fail for no reason whatsoever." I'd vote for blocking.
Otherwise, apps have to treat NOTIFY as an unreliable communication
mechanism, which reduces its use-case to approximately nil.
regards, tom lane
Tom Lane wrote:
Andrew Dunstan <andrew@dunslane.net> writes:
Alvaro Herrera wrote:
What will happen when the ring is full? NOTIFY blocks?
An error, I think, don't you? Blocking seems totally unacceptable.
That seems like a curious answer. Imagine the documentation:
"NOTIFY may fail for no reason whatsoever." I'd vote for blocking.
Otherwise, apps have to treat NOTIFY as an unreliable communication
mechanism, which reduces its use-case to approximately nil.
OK, good point. I guess I'm a bit concerned about providing a possible
DOS vector, but it could be either way.
How about Alvaro's suggestion of spilling to disk?
cheers
andrew
"Tom Lane" <tgl@sss.pgh.pa.us> writes:
Andrew Dunstan <andrew@dunslane.net> writes:
Alvaro Herrera wrote:
What will happen when the ring is full? NOTIFY blocks?
An error, I think, don't you? Blocking seems totally unacceptable.
That seems like a curious answer. Imagine the documentation:
"NOTIFY may fail for no reason whatsoever." I'd vote for blocking.
Otherwise, apps have to treat NOTIFY as an unreliable communication
mechanism, which reduces its use-case to approximately nil.
This will run into the same issue that bidirectional pipes run into with
deadlocks. The usual approach in Unix for dealing with this is having the
application always -- even while blocked trying to write -- read any pending
input and buffer it in user-space until it has enough to proceed. This may be
hard to arrange in SQL? I think you would need a way for a PL/pgsql to escape
a blocking write and read any pending notifications.
--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Gregory Stark <stark@enterprisedb.com> writes:
This will run into the same issue that bidirectional pipes run into with
deadlocks. The usual approach in Unix for dealing with this is having the
application always -- even while blocked trying to write -- read any pending
input and buffer it in user-space until it has enough to proceed. This may be
hard to arrange in SQL? I think you would need a way for a PL/pgsql to escape
a blocking write and read any pending notifications.
None of that is any different from the situation with sinval
messaging.
Note BTW that writes to the buffer will happen only in a very
circumscribed place (at COMMIT) so your worries about plpgsql
seem misplaced.
regards, tom lane
Andrew Dunstan <andrew@dunslane.net> writes:
How about Alvaro's suggestion of spilling to disk?
Won't that reintroduce most of the problems we're hoping to get rid of
by removing the pg_listener table? Certainly I'd not recommend putting
that into the first iteration.
regards, tom lane
