More PostgreSQL+CORBA
CORBA in fifty words or less:
CORBA is an architecture which specifies how to:
- define the method signatures of an object type
- obtain a reference to a object instance
- invoke a method, with parameters, on an object instance
- receive the results
- all interoperably between different programming languages, object
implementations, and platforms.
Why CORBA?
If you don't do object-oriented programming and system design, the
rationale for CORBA will be hard to understand. If you don't understand
why PostgreSQL is called an "object-relational" database (and why every
row has an OID), the rationale for PostgreSQL+CORBA will be hard to
understand.
The short version goes like this:
Think of a database table as a "typedef" of a data structure, with each
row representing a malloc'ed instance of that structure type. The
database provides for persistant storage, and concurrent data access,
but with a considerable access overhead: sending an SQL query string down
a socket, parsing the query string into an execution plan, executing
the plan, coverting the returned result set into text strings, sending
the strings down a socket, retrieving the strings from the socket, and,
finally, converting the text strings back into usable data values.
With CORBA, though, you could keep a reference (OID, pointer, etc.) to
each data structure of interest, and just call a function to read or
write data to fields in that structure. Another way to think of it
is cursors without queries. The database (PostgreSQL in our case)
continues to maintain persistence and concurrent access, and the data
is also always available for relational queries.
Which ORB?
GNOME started with Mico. Mico, apparently, makes use of C++ templates,
which caused the compiler they were using to generate bloated, wallowing
code.
GNOME then adopted ORBit, which has two wins: it's in C, and (this is
the biggy) it has provisions to shortcut parameter marshalling,
transmission, authentication, reception, and demarshalling--if the client
stub and server skeleton are in the same address space, and both stub
and skeleton permit this.
This means that, with ORBit, CORBA method calls can be almost as
efficient as normal function calls.
How to use CORBA with PostgreSQL?
There are three ways I can see this working:
1. As a simple alternative for the current FE<->BE communication protocol.
The SQL query engine continues to intermediate all transactions. This
has some benefits, but is really boring to me.
2. As an alternative to both the FE<->BE communication protocol and the
SQL query engine. In this case, programs could have efficient direct
row access, but all data transfers would still be shoved through a
socket (via the Internet Inter-Orb Protocol). This could be useful,
and mildly interesting.
3. As an alternative API to libpq that would allow, for example,
embedding a Python interpreter in the backend, with PostgreSQL tables
exposed through CORBA as native Python classes, and with high
performance via ORBit method shortcutting. This, in my opinion,
would be the most useful and interesting.
-Michael Robinson
On Sat, 14 Nov 1998, Michael Robinson wrote:
< a very well written description removed >
Which ORB?
GNOME started with Mico. Mico, apparently, makes use of C++ templates,
which caused the compiler they were using to generate bloated, wallowing
code.
Is that still accurate today?
GNOME then adopted ORBit, which has two wins: it's in C, and (this is
the biggy) it has provisions to shortcut parameter marshalling,
So...implement an OO 'environment' with a non-OO language? :)
My experience is that for pretty much every pro, there is a
con...what are we losing with ORBit that we'd have with mico? Short
and/or long term? mico is reputed to be Corba 2.2 compliant..orbit?
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
Michael Robinson <robinson@public.bta.net.cn> writes:
[ pithy summary of CORBA snipped --- thanks, Michael! ]
There are three ways I can see this working:
1. As a simple alternative for the current FE<->BE communication protocol.
The SQL query engine continues to intermediate all transactions. This
has some benefits, but is really boring to me.
I agree, that's not too exciting ... although replacing our current ad-hoc
FE/BE protocol with a standards-based protocol (I assume CORBA has a
recognized standard for the wire-level protocol?) might well be worth doing.
2. As an alternative to both the FE<->BE communication protocol and the
SQL query engine. In this case, programs could have efficient direct
row access, but all data transfers would still be shoved through a
socket (via the Internet Inter-Orb Protocol). This could be useful,
and mildly interesting.
Actually, I find that one Extremely Useful and indeed Fascinating ;-).
In the applications I'm currently using, a large fraction of the
update queries act on single rows and have the form
UPDATE table SET field(s) WHERE oid = 123456;
The overhead of doing this is horrendous, of course. Being able to
access individual rows as if they were CORBA objects would be a lovely
performance improvement, I suspect.
3. As an alternative API to libpq that would allow, for example,
embedding a Python interpreter in the backend, with PostgreSQL tables
exposed through CORBA as native Python classes, and with high
performance via ORBit method shortcutting. This, in my opinion,
would be the most useful and interesting.
I'm leery of this, not only because of the implementation problems other
people have mentioned (bringing the backend to a state where it is
thread-safe would be a large effort), but because it subverts all the
protection and security reasons for having the Postgres frontend/backend
architecture in the first place. The *last* thing I'd want is client
code executing in the same process as the database server.
However, if I understand things correctly, the CORBA interface will hide
whether client code is in the same process as the backend server or not;
so we could each assemble the parts in the way we prefer. At least, I
could build my separate-processes setup right away, and we could work
towards making the backend thread-safe so you could build your
installation your way.
Does CORBA have any provision for saying "this object is not thread
safe, don't send it concurrent operations"? If there's something along
that line, then maybe we don't have to fix the backend before it can
live in a common address space with the client...
regards, tom lane
Import Notes
Reply to msg id not found: YourmessageofSat14Nov1998011949+0800199811131719.BAA26341@public.bta.net.cn | Resolved by subject fallback
1. simple alternative for the current FE<->BE communication
protocol.... although replacing our current ad-hoc
FE/BE protocol with a standards-based protocol (I assume CORBA has a
recognized standard for the wire-level protocol?) might well be worth
doing.
Corba does specify a wire-level protocol, and also several layers above
that, so we would get endian issues resolved as well as
security/encryption, authentication, etc. Also, it may (at least
partially) address the need for different client libraries for different
languages, since there are Corba bindings available for a bunch of
languages. Some of that is up to the package one chooses to use...
- Tom
Tom Lane <tgl@sss.pgh.pa.us> writes:
(I assume CORBA has a recognized standard for the wire-level protocol?)
It's called the General Inter-ORB Protocol (GIOP). It's stream based.
When the streams are TCP/IP sockets, it becomes the Internet Inter-ORB
Protocol (GIOP, plus a specification for addresses, etc.).
I'm leery of this, not only because of the implementation problems other
people have mentioned (bringing the backend to a state where it is
thread-safe would be a large effort),
It may be possible just to put a master lock on the whole backend, so that
only one one thread was active in there at a time.
but because it subverts all the
protection and security reasons for having the Postgres frontend/backend
architecture in the first place. The *last* thing I'd want is client
code executing in the same process as the database server.
In the general case, I'd agree. However, for specific applications where
you want tight integration with a reasonably well-disciplined system,
such as a language interpreter, it could be a huge performance win. This
is sort of what Oracle did with their Java integration. They wrote a
common-denominator VM, and implemented both PL/SQL, and Java on top of it,
so that both languages could get "raw" access to the database tables.
However, if I understand things correctly, the CORBA interface will hide
whether client code is in the same process as the backend server or not;
so we could each assemble the parts in the way we prefer.
That is exactly correct.
Does CORBA have any provision for saying "this object is not thread
safe, don't send it concurrent operations"? If there's something along
that line, then maybe we don't have to fix the backend before it can
live in a common address space with the client...
That sort of question is resolved at the "Object Adapter" layer. The
ORBit "Portable Object Adapter" supports two thread policies, "ORB_CTRL_MODEL"
and "SINGLE_THREAD_MODEL" (I don't know the specifics; I just pulled that
out of the Interface Definition for the POA). The Postgres Object Adapter
could, presumably, do whatever it wanted in this regard.
In any case, the full text of the CORBA 2.x spec is here in all it's HTML
glory:
http://www.infosys.tuwien.ac.at/Research/Corba/OMG/corb2prf.htm
I'm really impressed by the quality of the architecture design.
-Michael Robinson
Import Notes
Resolved by subject fallback
The Hermit Hacker <scrappy@hub.org> writes:
GNOME then adopted ORBit, which has two wins: it's in C, and (this is
the biggy) it has provisions to shortcut parameter marshalling,My experience is that for pretty much every pro, there is a
con...what are we losing with ORBit that we'd have with mico? Short
and/or long term? mico is reputed to be Corba 2.2 compliant..orbit?
The big con for ORBit, short term:
% cd /usr/src/gnome/ORBit-0.3.0
% find . -name "*.c" -exec grep "Not yet implemented" {} \; | wc
857 2613 30302
That's 857 items specified in the IDL for ORBit that are currently empty
functions.
I think long-term, with the financial backing of RedHat, and the key role
it plays in the GNOME, I think ORBit will be the premier open-source ORB.
-Michael
Import Notes
Resolved by subject fallback
On Sat, 14 Nov 1998, Michael Robinson wrote:
The Hermit Hacker <scrappy@hub.org> writes:
GNOME then adopted ORBit, which has two wins: it's in C, and (this is
the biggy) it has provisions to shortcut parameter marshalling,My experience is that for pretty much every pro, there is a
con...what are we losing with ORBit that we'd have with mico? Short
and/or long term? mico is reputed to be Corba 2.2 compliant..orbit?The big con for ORBit, short term:
% cd /usr/src/gnome/ORBit-0.3.0
% find . -name "*.c" -exec grep "Not yet implemented" {} \; | wc
857 2613 30302That's 857 items specified in the IDL for ORBit that are currently empty
functions.I think long-term, with the financial backing of RedHat, and the key role
it plays in the GNOME, I think ORBit will be the premier open-source ORB.
Can we do the first implemntation using MICO and convert over
later, if the need arises? Or are those 857 items not important?
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
Michael Robinson <robinson@public.bta.net.cn> writes:
[ pithy summary of CORBA snipped --- thanks, Michael! ]
There are three ways I can see this working:
1. As a simple alternative for the current FE<->BE communication protocol.
The SQL query engine continues to intermediate all transactions. This
has some benefits, but is really boring to me.I agree, that's not too exciting ... although replacing our current ad-hoc
FE/BE protocol with a standards-based protocol (I assume CORBA has a
recognized standard for the wire-level protocol?) might well be worth doing.
Current FE/BE protocol seems pretty optimized to me, but you should know
the best. Seems like a waste to try and get it to match some standard,
expecially if we can just create a module to do it on top of the
existing protocol.
--
Bruce Momjian | http://www.op.net/~candle
maillist@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026On Fri, 13 Nov 1998, Bruce Momjian wrote:
Michael Robinson <robinson@public.bta.net.cn> writes:
[ pithy summary of CORBA snipped --- thanks, Michael! ]
There are three ways I can see this working:
1. As a simple alternative for the current FE<->BE communication protocol.
The SQL query engine continues to intermediate all transactions. This
has some benefits, but is really boring to me.I agree, that's not too exciting ... although replacing our current ad-hoc
FE/BE protocol with a standards-based protocol (I assume CORBA has a
recognized standard for the wire-level protocol?) might well be worth doing.Current FE/BE protocol seems pretty optimized to me, but you should know
the best. Seems like a waste to try and get it to match some standard,
expecially if we can just create a module to do it on top of the
existing protocol.
Except...if I'm understanding even half of this correctly...by
implementing CORBA at the FE/BE level, this effectively eliminates the
need for *us* to maintain a seperate interface for each language we want
to support, since that is what one of CORBA's design goals is...
In fact, again, if I'm understanding this correctly, this could
potentially open us up to languages we currently don't support...?
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
Bruce Momjian <maillist@candle.pha.pa.us> writes:
Current FE/BE protocol seems pretty optimized to me, but you should know
the best. Seems like a waste to try and get it to match some standard,
Well, I suppose you could make the same argument about SQL-92 compliance.
Standards compliance is useful because it plugs you into the positive-
feedback cycle of product development. Your product works with more
products, more people use your product, more products are developed which
work with your product, etc.
-Michael Robinson
Import Notes
Resolved by subject fallback
Except...if I'm understanding even half of this correctly...by
implementing CORBA at the FE/BE level, this effectively eliminates the
need for *us* to maintain a seperate interface for each language we want
to support, since that is what one of CORBA's design goals is...In fact, again, if I'm understanding this correctly, this could
potentially open us up to languages we currently don't support...?
Yea, that would be neat. But considering no one really totally supports
CORBA yet, and we already have tons of working interfaces, perhaps we
can consider it in the future, or were you thinking in the next 6-9
months?
--
Bruce Momjian | http://www.op.net/~candle
maillist@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026Bruce Momjian <maillist@candle.pha.pa.us> writes:
Current FE/BE protocol seems pretty optimized to me, but you should know
the best. Seems like a waste to try and get it to match some standard,Well, I suppose you could make the same argument about SQL-92 compliance.
Standards compliance is useful because it plugs you into the positive-
feedback cycle of product development. Your product works with more
products, more people use your product, more products are developed which
work with your product, etc.
Not really the same. There are end-user visible changes, and
developer-visible changes.
--
Bruce Momjian | http://www.op.net/~candle
maillist@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026
On Fri, Nov 13, 1998 at 01:39:17PM -0400, The Hermit Hacker wrote:
GNOME started with Mico. Mico, apparently, makes use of C++ templates,
which caused the compiler they were using to generate bloated, wallowing
code.Is that still accurate today?
I think so, yes.
GNOME then adopted ORBit, which has two wins: it's in C, and (this is
the biggy) it has provisions to shortcut parameter marshalling,So...implement an OO 'environment' with a non-OO language? :)
My experience is that for pretty much every pro, there is a
con...what are we losing with ORBit that we'd have with mico? Short
and/or long term? mico is reputed to be Corba 2.2 compliant..orbit?
In the short term we lose lots of functions with ORBit. In the long run
however we win performance.
Michael
--
Dr. Michael Meskes, Manager of the Western Branch Office, Datenrevision GmbH
work: Cuxhavener Str. 36, D-21149 Hamburg
home: Th.-Heuss-Str. 61, D-41812 Erkelenz, Michael.Meskes@usa.net
Go SF49ers! Go Rhein Fire! Use Debian GNU/Linux!
On Sat, Nov 14, 1998 at 11:08:52AM +0800, Michael Robinson wrote:
I think long-term, with the financial backing of RedHat, and the key role
it plays in the GNOME, I think ORBit will be the premier open-source ORB.
Couldn't agree more.
Michael
--
Dr. Michael Meskes, Manager of the Western Branch Office, Datenrevision GmbH
work: Cuxhavener Str. 36, D-21149 Hamburg
home: Th.-Heuss-Str. 61, D-41812 Erkelenz, Michael.Meskes@usa.net
Go SF49ers! Go Rhein Fire! Use Debian GNU/Linux!
On Fri, Nov 13, 1998 at 11:29:11PM -0400, The Hermit Hacker wrote:
Can we do the first implemntation using MICO and convert over
later, if the need arises? Or are those 857 items not important?
I think so. After all that's what GNOME did too.
Michael
--
Dr. Michael Meskes, Manager of the Western Branch Office, Datenrevision GmbH
work: Cuxhavener Str. 36, D-21149 Hamburg
home: Th.-Heuss-Str. 61, D-41812 Erkelenz, Michael.Meskes@usa.net
Go SF49ers! Go Rhein Fire! Use Debian GNU/Linux!
On Sat, 14 Nov 1998, Bruce Momjian wrote:
Except...if I'm understanding even half of this correctly...by
implementing CORBA at the FE/BE level, this effectively eliminates the
need for *us* to maintain a seperate interface for each language we want
to support, since that is what one of CORBA's design goals is...In fact, again, if I'm understanding this correctly, this could
potentially open us up to languages we currently don't support...?Yea, that would be neat. But considering no one really totally supports
CORBA yet, and we already have tons of working interfaces, perhaps we
can consider it in the future, or were you thinking in the next 6-9
months?
Guess that's the next question (vs statement)...who actually
supports Corba at this time? two, off the top of my head, are Gnome and
Koffice...anyone know of a list of others?
As for 6-9 months...I think this is more in Michael court then
anything...I don't see why work can't start on it now, even if its nothing
more then Michael submitting patches that have the relevant sections
#ifdef's so that they are only enabled for those working on it. I don't
imagine this is going to be a "now it isn't, now it is" sort of thing...it
might take 6-9 months to implement...
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
On Sat, 14 Nov 1998, Bruce Momjian wrote:
Bruce Momjian <maillist@candle.pha.pa.us> writes:
Current FE/BE protocol seems pretty optimized to me, but you should know
the best. Seems like a waste to try and get it to match some standard,Well, I suppose you could make the same argument about SQL-92 compliance.
Standards compliance is useful because it plugs you into the positive-
feedback cycle of product development. Your product works with more
products, more people use your product, more products are developed which
work with your product, etc.Not really the same. There are end-user visible changes, and
developer-visible changes.
From how things sound, I think that implementing Corba will be the
trigger for 7.0...:)
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
Except...if I'm understanding even half of this correctly...by
implementing CORBA at the FE/BE level, this effectively eliminates the
need for *us* to maintain a seperate interface for each language we want
to support, since that is what one of CORBA's design goals is...In fact, again, if I'm understanding this correctly, this could
potentially open us up to languages we currently don't support...?Yea, that would be neat. But considering no one really totally supports
CORBA yet, and we already have tons of working interfaces, perhaps we
can consider it in the future, or were you thinking in the next 6-9
months?
I think I get it now. Currently, all the non-C interfaces use libpq to
go over the wire to the backend. If we made the FE/BE protocol CORBA, we
could modify libpq, and all the current interfaces would still work.
Then if someone came up with a Smalltalk-to-CORBA interface, they could
use it for PostgreSQL. Also, if someone came up with a better
Perl-to-CORBA interface, we could throw ours away, and just use that
one.
Would nice. Hope there is no performance penalty.
--
Bruce Momjian | http://www.op.net/~candle
maillist@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026On Sat, 14 Nov 1998, Bruce Momjian wrote:
Bruce Momjian <maillist@candle.pha.pa.us> writes:
Current FE/BE protocol seems pretty optimized to me, but you should know
the best. Seems like a waste to try and get it to match some standard,Well, I suppose you could make the same argument about SQL-92 compliance.
Standards compliance is useful because it plugs you into the positive-
feedback cycle of product development. Your product works with more
products, more people use your product, more products are developed which
work with your product, etc.Not really the same. There are end-user visible changes, and
developer-visible changes.From how things sound, I think that implementing Corba will be the
trigger for 7.0...:)
Yep.
--
Bruce Momjian | http://www.op.net/~candle
maillist@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026Bruce Momjian <maillist@candle.pha.pa.us> writes:
Yea, that would be neat. But considering no one really totally supports
CORBA yet, and we already have tons of working interfaces, perhaps we
can consider it in the future, or were you thinking in the next 6-9
months?
I'm not sure what Michael was thinking, but I was seeing this as a
long-term kind of project. Maybe for the release after 6.5, or even
the one after that. (Do we have any definite plans for future release
frequency?)
Even if the open-source ORBs are adequately up-to-speed today (which
sounded iffy), we need to learn about CORBA, or import some expertise
from somewhere, before we can do much. This is unknown territory for
me, and evidently also for most of the pgsql crew. I'd be inclined to
just play around for a few months and try to write a paper design.
It does sound like an idea worth pursuing, though.
regards, tom lane
Import Notes
Reply to msg id not found: YourmessageofSat14Nov1998023404-0500199811140734.CAA21920@candle.pha.pa.us | Resolved by subject fallback
