idea for 'module' support

On Sun, 20 Jun 1999, Mark Hollomon wrote:

For instance, a user may type:

LOAD PACKAGE 'plperl';

This would not only load the shared library "plperl.so", but call
the function "package_init" which could the use the SPI facilities
to properly setup the system tables for the new language.

New types could be packaged the same way.

This would give a very modular way to add 'packages' of functionality.

What do you think?

Lots of people do this, but you might want to take a look at how we do
this in Dents, just because it's moderately well documented. I did the
initial version inspired by how GTK+ loads new themes, but the basic
idea is the same everywhere: dlopen() an object, resolve symbols in it,
use those symbols to populate a struct filled with function pointers
and whatnot.

You can find docs on how to write Dents modules at:
http://www.dents.org/modules/modules.html

For how we actually do it, look in the source code under "src/*module*".

I personally think that this is a neat idea in general and might be very
nifty for Postgres.

--
Todd Graham Lewis Postmaster, MindSpring Enterprises
tlewis@mindspring.net (800) 719-4664, x22804

"There is no spoon."

Hi Mark,

This is truly a very nice idea, since it would from an application
developers perspective simplify the process of getting users to for
instance update the inner works of a postgresql driven application.

It would be great to also in a package be able to include SQL scripts,
so one could do "patches" for database applications that are easily
installed by end-users/administrators.

For instance, updating stored procedures, classes and external .so
libs could be done with a simple LOAD PACKAGE 'package-name' facility.

I'm uncertain if this really fits in the pgsql db itself, it might be
better to have it outside of the main package, as a utility (suite)
like pg_package-install/update/list.

Some more thoughts on what would fit nicely in the package-idea:
- Scriptability. A package should be a PostgreSQL "SQL" script,
runable from psql.
- Digitally signed (encrypted?) packages
- Version control
- External transport (ftp, httpd from update server)

I think the XEmacs 21.x package handling might be a good model, which
according to me works very well even for relatively newbie users.

Just some thoughts, might need refining and discussion, but the main
idea is to have an increased modularity, adding/updating/managing
separate modules easily.

Be well,
/Daniel

Mark Hollomon writes:

I would like to add the following command:

LOAD PACKAGE 'package-name';

...

LOAD PACKAGE 'plperl';

This would not only load the shared library "plperl.so", but call
the function "package_init" which could the use the SPI facilities
to properly setup the system tables for the new language.
New types could be packaged the same way.
This would give a very modular way to add 'packages' of functionality.

--
_______________________________________________________________ /\__
Daniel Lundin - MediaCenter, UNIX and BeOS Developer \/
http://www.umc.se/~daniel/

Mark Hollomon wrote:

As I've been playing with the PL/Perl implementation, it has dawned on me a
fairly simple, but nice feature could be added.

I would like to add the following command:

LOAD PACKAGE 'package-name';

Like the current 'LOAD' it would treat 'package-name'
as shared library. But it would also call an intialization function
in the library (package_init maybe?).

For instance, a user may type:

LOAD PACKAGE 'plperl';

This would not only load the shared library "plperl.so", but call
the function "package_init" which could the use the SPI facilities
to properly setup the system tables for the new language.

New types could be packaged the same way.

This would give a very modular way to add 'packages' of functionality.

1. All the functionality required to install such a package
is only needed once per database (or if thrown into
template1 once per installation). But the entire shared
object has to be linked into each time a backend needs
one single function from it. I'm not sure if it's such a
good idea to waste more and more address space for
administrative stuff that isn't required at runtime.

2. Most of such functionality requires PostgreSQL superuser
rights (like installing C language functions). Thus it is
useless for a regular user.

3. Some of the features might be customizable. Procedural
languages for example can be installed as trusted ones or
not. Trusted languages can be used by any regular user to
CREATE FUNCTION, untrusted ones can't. Placing the
installation procedure inside the module itself doesn't
make things easier here.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

Jan Wieck wrote:

1. All the functionality required to install such a package
is only needed once per database (or if thrown into
template1 once per installation). But the entire shared
object has to be linked into each time a backend needs
one single function from it. I'm not sure if it's such a
good idea to waste more and more address space for
administrative stuff that isn't required at runtime.

*shrug*. I don't see this as very important, but then, I'm not trying
to run a server that is servicing 100's or 1000's of requests per
hour either. A valid point in the general case.

Hmmmm. Of course, nothing says that the package_init stuff has be in
the same file as the runtine stuff. But then what would be the
difference from a \i of a SQL script?

2. Most of such functionality requires PostgreSQL superuser
rights (like installing C language functions). Thus it is
useless for a regular user.

Well, no more useless than C language functions right now. I had
in the back of my mind that there would be an 'approved' module
location. If a module resides there, then loading would occur as
if done by the superuser. It would be the admin's responibility to
make sure that that location was secure.

Of course, depending on the way the security model in PostgreSQL is
implemented, this may not be possible. I haven't looked.

3. Some of the features might be customizable. Procedural
languages for example can be installed as trusted ones or
not. Trusted languages can be used by any regular user to
CREATE FUNCTION, untrusted ones can't. Placing the
installation procedure inside the module itself doesn't
make things easier here.

Passing parameters couldn't be that hard :

LOAD PACKAGE 'package_name' [WITH [attr=value]+];

Thank you for the feedback. I think the concept is worth while.
But apparently a good bit of work is needed on the mechanics.

--

Mark Hollomon
mhh@nortelnetworks.com