Avahi support for Postgresql

This has been saved for the 8.4 release:

http://momjian.postgresql.org/cgi-bin/pgpatches_hold

---------------------------------------------------------------------------

Mathias Hasselmann wrote:

Postmaster already has code to announce its services via DNS-SD
(ZeroConf) by using Apple's Bonjour API. This series of patches
implements that capability on top of the Avahi library[1] which
is free software, available for a wider variety of platforms.

I've separated the change set into smaller pieces for easy review. Also
I don't know, if the 4th patch adding subtypes describing Postgresql's
capabilities is wanted by the Postgresql community. Being a desktop guy,
using Postgresql as embedded database, I'd really like to see all of the
patches merged, but I could imagine some opposition against the verbose
nature of the subtypes patch.

Each of the patches as a short description of its purpose.

I couldn't find any ChangeLog files in the repository, and also I didn't
invest much time in investigating how the HISTORY file of distribution
tarballs is assembled. So if the repository contains file I should have
modified for documenting my changes, I apologize for not updating them.

[1] http://www.avahi.org/
--
Mathias Hasselmann <mathias@openismus.com>
www.openismus.com

[ application/mbox is not supported, skipping... ]

[ application/mbox is not supported, skipping... ]

[ application/mbox is not supported, skipping... ]

[ application/mbox is not supported, skipping... ]

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://postgres.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Hello,

Congratulations to the 8.3 release. No, that pre-release stress is over
I'd really like to put attention on my Avahi patches again.

Avahi/Bonjour/DNS-SD support[1]http://www.dns-sd.org/ is very important, for integrating
Postgresql with modern desktop environments like OSX, GNOME, KDE: It's
very convenient to choose active DBMS servers in your local network from
a list, instead of memorizing "cryptic" connection parameters.

In Glom[2]http://www.glom.org/ we use Postgresql as DBMS of choice. This application would
greatly benefit from the possibility to choose central Postgresql
servers from an automatically updated list.

For OSX builds Postgresql already has DNS-SD support via Apple's Bonjour
library. For Linux Avahi[3]http://www.avahi.org/ is the library of choice. Modern distribu-
tions ship it as main package, since both GNOME and KDE make use of it.
Since DNS-SD is a published IETF standard [4]http://www.ietf.org/rfc/rfc2782.txt client applications
choose whatever DNS-SD implementation they like.

Applying this patche series would bring Postgresql's Linux port on par
with the OSX port in terms of desktop integration.

People not wanting DNS-SD support for their server can easily control
that feature via the "--with-avahi" configure scripts.

Thank your for attention,
Mathias

[1]: http://www.dns-sd.org/
[2]: http://www.glom.org/
[3]: http://www.avahi.org/
[4]: http://www.ietf.org/rfc/rfc2782.txt

Am Dienstag, den 27.11.2007, 11:19 +0100 schrieb Mathias Hasselmann:

Postmaster already has code to announce its services via DNS-SD
(ZeroConf) by using Apple's Bonjour API. This series of patches
implements that capability on top of the Avahi library[1] which
is free software, available for a wider variety of platforms.

I've separated the change set into smaller pieces for easy review. Also
I don't know, if the 4th patch adding subtypes describing Postgresql's
capabilities is wanted by the Postgresql community. Being a desktop guy,
using Postgresql as embedded database, I'd really like to see all of the
patches merged, but I could imagine some opposition against the verbose
nature of the subtypes patch.

Each of the patches as a short description of its purpose.

I couldn't find any ChangeLog files in the repository, and also I didn't
invest much time in investigating how the HISTORY file of distribution
tarballs is assembled. So if the repository contains file I should have
modified for documenting my changes, I apologize for not updating them.

[1] http://www.avahi.org/
---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

--
Mathias Hasselmann <mathias@openismus.com>
http://www.openismus.com/ - We can get it done.

Mathias Hasselmann wrote:

Congratulations to the 8.3 release. No, that pre-release stress is over
I'd really like to put attention on my Avahi patches again.

I've added your item to the patch list
(http://developer.postgresql.org/index.php/Todo:PatchStatus). It would help
if you could include or point to your actual patch.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Am Samstag, den 23.02.2008, 21:08 +0100 schrieb Peter Eisentraut:

Mathias Hasselmann wrote:

Congratulations to the 8.3 release. Now, that pre-release stress is over
I'd really like to put attention on my Avahi patches again.

I've added your item to the patch list
(http://developer.postgresql.org/index.php/Todo:PatchStatus). It would help
if you could include or point to your actual patch.

Thanks a lot.

The patches were in my initial mail, but now I've also uploaded them to
my personal site for convenience:

http://taschenorakel.de/files/pgsql-avahi-support/

I just failed to update the Wiki page:

This page has been locked to prevent editing.
You can view and copy the source of this page:

Ciao,
Mathias
--
Mathias Hasselmann <mathias@openismus.com>
http://www.openismus.com/ - We can get it done.

Am Dienstag, 27. November 2007 schrieb Mathias Hasselmann:

Postmaster already has code to announce its services via DNS-SD
(ZeroConf) by using Apple's Bonjour API. This series of patches
implements that capability on top of the Avahi library[1] which
is free software, available for a wider variety of platforms.

Is there a reason we couldn't use the Bonjour compatibility layer offered by
Avahi to keep the code differences to a minimum?

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Am Dienstag, 27. November 2007 schrieb Mathias Hasselmann:

Postmaster already has code to announce its services via DNS-SD
(ZeroConf) by using Apple's Bonjour API. This series of patches
implements that capability on top of the Avahi library[1] which
is free software, available for a wider variety of platforms.

Note to hackers: This set of patches renames the GUC parameter bonjour_name
to zeroconf_name to make the more general meaning clear. We have generally
not hesitated to rename GUC parameters between major releases, but if this
bothers you, please speak up.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Am Dienstag, 27. November 2007 schrieb Mathias Hasselmann:

I don't know, if the 4th patch adding subtypes describing Postgresql's
capabilities is wanted by the Postgresql community. Being a desktop guy,
using Postgresql as embedded database, I'd really like to see all of the
patches merged, but I could imagine some opposition against the verbose
nature of the subtypes patch.

I am not convinced about the usefulness of the subtypes patch. The examples
you cite -- FTP, bittorrent, distcc -- are cases where a client could pick
the most appropriate server from a set of equivalent instances. I don't see
how this can be useful in a PostgreSQL setting, at least not based on the
parameters server version and protocol version that you implemented.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Mathias Hasselmann wrote:

The patches were in my initial mail, but now I've also uploaded them to
my personal site for convenience:

http://taschenorakel.de/files/pgsql-avahi-support/

Hmm, a quick look at the third patch reveals that it is using the
"threaded" Avahi client. That's a showstopper. Please consider using
some other approach -- writing our own handlers for AvahiPoll would seem
apropos.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Am Dienstag, 27. November 2007 schrieb Mathias Hasselmann:

Postmaster already has code to announce its services via DNS-SD
(ZeroConf) by using Apple's Bonjour API. This series of patches
implements that capability on top of the Avahi library[1] which
is free software, available for a wider variety of platforms.

One thing I found curious: The Bonjour code uses service
type "_postgresql._tcp." whereas your code uses "_postgresql._tcp" (note the
dot). Is that an expected difference?

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Am Montag, 25. Februar 2008 schrieb Alvaro Herrera:

Hmm, a quick look at the third patch reveals that it is using the
"threaded" Avahi client. �That's a showstopper.

Could you elaborate why that is?

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:

Am Montag, 25. Februar 2008 schrieb Alvaro Herrera:

Hmm, a quick look at the third patch reveals that it is using the
"threaded" Avahi client. �That's a showstopper.

Could you elaborate why that is?

Because it creates a new thread under the Postmaster to handle Avahi
events, if I'm reading the Avahi docs right. This is verboten.

We have an event loop in the postmaster -- see ServerLoop. Is there a
reason the Avahi events could not be hooked in there?

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Am Samstag, 23. Februar 2008 schrieb Mathias Hasselmann:

The patches were in my initial mail, but now I've also uploaded them to
my personal site for convenience:

http://taschenorakel.de/files/pgsql-avahi-support/

I have reworked the first three of your patches (mostly formatting and some
refactoring) and published them here:

http://git.postgresql.org/?p=~petere/postgresql.git;a=shortlog;h=avahi-support

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Am Montag, den 25.02.2008, 15:45 +0100 schrieb Peter Eisentraut:

Am Dienstag, 27. November 2007 schrieb Mathias Hasselmann:

Postmaster already has code to announce its services via DNS-SD
(ZeroConf) by using Apple's Bonjour API. This series of patches
implements that capability on top of the Avahi library[1] which
is free software, available for a wider variety of platforms.

One thing I found curious: The Bonjour code uses service
type "_postgresql._tcp." whereas your code uses "_postgresql._tcp" (note the
dot). Is that an expected difference?

Oh, that difference is really interesting. I didn't even see it.

DNS-SD uses the convention "_<protoname>._<type>" to describe services,
and that's the convention Avahi follows. I don't know why the Bonjour
API uses that trailing dot, but it seems wrong:

"a trailing dot indicates the root domain. so bonjour is clearly
wrong, since there's no ._tcp top-level domain."

-- James Andrewartha on #avahi today

Avahi's maintainer Sjoerd Simons agreed, that the trailing dot seems to
be wrong.

Ciao,
Mathias
--
Mathias Hasselmann <mathias@openismus.com>
http://www.openismus.com/ - We can get it done.

Am Montag, den 25.02.2008, 15:05 +0100 schrieb Peter Eisentraut:

Am Dienstag, 27. November 2007 schrieb Mathias Hasselmann:

Postmaster already has code to announce its services via DNS-SD
(ZeroConf) by using Apple's Bonjour API. This series of patches
implements that capability on top of the Avahi library[1] which
is free software, available for a wider variety of platforms.

Is there a reason we couldn't use the Bonjour compatibility layer offered by
Avahi to keep the code differences to a minimum?

1) The Avahi's compatibility layer doesn't implement the now deprecated
function DNSServiceRegistrationCreate().

2) Unless DNSServiceRegistrationCreate() installs a lot of black magic,
the Bonjour code in postmaster.c has very poor error handling: There
seem to be no attempts made to handle name collisions (unless Bonjour
does this automatically, of course).

Ciao,
Mathias
--
Mathias Hasselmann <mathias@openismus.com>
http://www.openismus.com/ - We can get it done.

Am Montag, den 25.02.2008, 15:27 +0100 schrieb Peter Eisentraut:

Am Dienstag, 27. November 2007 schrieb Mathias Hasselmann:

I don't know, if the 4th patch adding subtypes describing Postgresql's
capabilities is wanted by the Postgresql community. Being a desktop guy,
using Postgresql as embedded database, I'd really like to see all of the
patches merged, but I could imagine some opposition against the verbose
nature of the subtypes patch.

I am not convinced about the usefulness of the subtypes patch. The examples
you cite -- FTP, bittorrent, distcc -- are cases where a client could pick
the most appropriate server from a set of equivalent instances. I don't see
how this can be useful in a PostgreSQL setting, at least not based on the
parameters server version and protocol version that you implemented.

Well, I could imagine certain cases, where applications only want to
list Postgresql servers newer than release x.y - since they depend on
certain new features of that release.

But you could be right, that the approach I've implemented only allows
filtering for "revision == x.y", but not the intended "revision >= x.y".

So maybe the release number, or a feature list (similar the capability
list of SMTP) should be put into the announcement's TXT record.

Well, but I do not consider that sub-type feature that important.

Ciao,
Mathias
--
Mathias Hasselmann <mathias@openismus.com>
http://www.openismus.com/ - We can get it done.

Mathias Hasselmann <mathias@openismus.com> writes:

Oh, that difference is really interesting. I didn't even see it.

DNS-SD uses the convention "_<protoname>._<type>" to describe services,
and that's the convention Avahi follows. I don't know why the Bonjour
API uses that trailing dot, but it seems wrong:

Is removing that dot going to create compatibility problems for us?
If it means that an 8.3 client wouldn't find an 8.4 server, for
instance, I don't think that's going to be acceptable. Do we need
to "listen" on both names (whatever the heck that means in zeroconf)?

regards, tom lane

Am Montag, den 25.02.2008, 19:12 +0100 schrieb Peter Eisentraut:

Am Samstag, 23. Februar 2008 schrieb Mathias Hasselmann:

The patches were in my initial mail, but now I've also uploaded them to
my personal site for convenience:

http://taschenorakel.de/files/pgsql-avahi-support/

I have reworked the first three of your patches (mostly formatting and some
refactoring) and published them here:

http://git.postgresql.org/?p=~petere/postgresql.git;a=shortlog;h=avahi-support

Thanks alot.

Didn't know about that git repository, btw.

Is git://git.postgresql.org/git/postgresql.git assumed to reliably
mirror the CVS repository?

Ciao,
Mathias
--
Mathias Hasselmann <mathias@openismus.com>
http://www.openismus.com/ - We can get it done.

Mathias Hasselmann <mathias@openismus.com> writes:

Am Montag, den 25.02.2008, 15:05 +0100 schrieb Peter Eisentraut:

Is there a reason we couldn't use the Bonjour compatibility layer offered by
Avahi to keep the code differences to a minimum?

1) The Avahi's compatibility layer doesn't implement the now deprecated
function DNSServiceRegistrationCreate().

Note that Apple themselves have been deprecating
DNSServiceRegistrationCreate for some time:

postmaster.c: In function 'PostmasterMain':
postmaster.c:856: warning: 'DNSServiceRegistrationCreate' is deprecated (declared at /usr/include/DNSServiceDiscovery/DNSServiceDiscovery.h:139)

It's a fairly good bet that the function will disappear entirely from OS
X at some point, so we're going to have to change this code soon anyway.

What I'd like to know is whether the Avahi API that this patch is using
is compatible with whatever Apple is pushing as the not-deprecated API.
It would be annoying to tell Mac users that they have to install Avahi
to get at functionality that their platform provides natively.

regards, tom lane

Am Montag, den 25.02.2008, 14:32 -0300 schrieb Alvaro Herrera:

Peter Eisentraut wrote:

Am Montag, 25. Februar 2008 schrieb Alvaro Herrera:

Hmm, a quick look at the third patch reveals that it is using the
"threaded" Avahi client. That's a showstopper.

Could you elaborate why that is?

Because it creates a new thread under the Postmaster to handle Avahi
events, if I'm reading the Avahi docs right. This is verboten.

Just to be sure we talk about the same topic: I assume the prohibition
you talk about is something like "no use of threads in Postmaster"?

If that's the case: Are there some docs, mails, ... explaining the
rationale behind this restriction? I could imagine your do not want
random locking in the postmaster code?

See, interaction points with the main thread are very small:

1) Lock-free creation of the threaded Avahi client in PostmasterMain()
2) Locked shutdown of the Avahi client in ExitAvahiClient(), which only
is called from ExitPostmaster().

So IMHO usage of the threaded poll API has much smaller impact on the
behavior of the postmaster process, than any attempt to integrate Avahi
with postmaster's main loop.

We have an event loop in the postmaster -- see ServerLoop. Is there a
reason the Avahi events could not be hooked in there?

Currently there are four ___well tested___ implementations of Avahi's
poll API: AvahiSimplePoll, which really just works for simple command
line tools and demonstration purposes. The single threaded APIs that
integrate with the main loops of glib and Qt, and the threaded poll API.

Avahi's requirements for a poll API aren't exactly trivial: You don't
only have to care about file descriptors, you also have to implement
some kind of timeout scheduling. So in favor of reinventing the wheel
and delivering an untested custom poll API, I've chosen the threaded
poll API: It's the only well-tested poll API that fits into Postgresql,
and its interaction points with the Postmaster process are minimal.

From looking at ServerLoop() I do not see any facilities for

registering timeout callbacks. Select timeouts are static. So for
implementing Avahi's poll API in ServerLoop() some radical code changes
would be needed. I don't believe such changes would be justified,
unless other portions of postmaster also need timeout callbacks.

Ciao,
Mathias
--
Mathias Hasselmann <mathias@openismus.com>
http://www.openismus.com/ - We can get it done.