Is there going to be a port to Solaris 9 x86 in the near future???

In an attempt to throw the authorities off his trail, csmiga@n2bb.com (Christoper Smiga) transmitted:

Does anyone know if there is going to be a port to Solaris 9 x86 in
the near future. What is the decission to develop on this platform
since Sun is pushing Solaris x86 harder than ever.

If you're running Solaris on x86, then you're free to try PostgreSQL
out there. It works quite well on SPARC; it is not evident that/why
it _wouldn't_ work on the x86 version.

On the other hand, the impression that I got was that the "pushing"
taking place with Solaris x86 was more of the "into the dumpster" sort
than "pushing hard to customers." I thought their new strategy
involved Linux on x86...
--
(reverse (concatenate 'string "gro.gultn" "@" "enworbbc"))
http://cbbrowne.com/info/spreadsheets.html
Rules of the Evil Overlord #220. "Whatever my one vulnerability is, I
will fake a different one. For example, ordering all mirrors removed
from the palace, screaming and flinching whenever someone accidentally
holds up a mirror, etc. In the climax when the hero whips out a mirror
and thrusts it at my face, my reaction will be ``Hmm...I think I need
a shave.''" <http://www.eviloverlord.com/&gt;

On Tue, 18 Nov 2003, Christoper Smiga wrote:

All:

Does anyone know if there is going to be a port to Solaris 9 x86 in the
near future. What is the decission to develop on this platform since Sun
is pushing Solaris x86 harder than ever.

Doesn't it work? I've run on Solaris 8 x86 extensively in the past, and
played a bit with Solaris 9, but know (or haven't heard of) any issues
with Solaris 9 + 7.4 ...

I think they are actually trying to pull it out of the dumpster,
whether from desperation of marketing acumen no one knows. I think
they've gone back to the 'if we can get them hooked on a dual opteron
box, we can sell them some massive E10000' or whatever.

On Nov 18, 2003, at 11:32 AM, Christopher Browne wrote:

In an attempt to throw the authorities off his trail, csmiga@n2bb.com
(Christoper Smiga) transmitted:

Does anyone know if there is going to be a port to Solaris 9 x86 in
the near future. What is the decission to develop on this platform
since Sun is pushing Solaris x86 harder than ever.

If you're running Solaris on x86, then you're free to try PostgreSQL
out there. It works quite well on SPARC; it is not evident that/why
it _wouldn't_ work on the x86 version.

On the other hand, the impression that I got was that the "pushing"
taking place with Solaris x86 was more of the "into the dumpster" sort
than "pushing hard to customers." I thought their new strategy
involved Linux on x86...
--
(reverse (concatenate 'string "gro.gultn" "@" "enworbbc"))
http://cbbrowne.com/info/spreadsheets.html
Rules of the Evil Overlord #220. "Whatever my one vulnerability is, I
will fake a different one. For example, ordering all mirrors removed
from the palace, screaming and flinching whenever someone accidentally
holds up a mirror, etc. In the climax when the hero whips out a mirror
and thrusts it at my face, my reaction will be ``Hmm...I think I need
a shave.''" <http://www.eviloverlord.com/&gt;

---------------------------(end of
broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if
your
joining column's datatypes do not match

--------------------

Andrew Rawnsley
President
The Ravensfield Digital Resource Group, Ltd.
(740) 587-0114
www.ravensfield.com

PostgreSQL most definitely works great on Solaris x86 !

At UC Berkeley, we have our undergraduate students hack on the
internals of PostgreSQL in the upper-division "Introduction to
Database Systems" class ..

http://www-inst.eecs.berkeley.edu/~cs186/

The "official" platform is Solaris x86 - that's where the students get
accounts and they have to get their code working on that platform as
the TAs will only test and grade their submissions on Solaris x86.

(Besides, I also got TelegraphCQ running on Solaris x86 .. just for
kicks though .. and TelegraphCQ is based off of pgsql-7.3.2)

--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh

http://www-inst.eecs.berkeley.edu/~cs186/hwk0/index.html

Are these screenshots of PgAccess on Mac OSX?

Robert Treat

On Tue, 2003-11-18 at 13:07, Sailesh Krishnamurthy wrote:

PostgreSQL most definitely works great on Solaris x86 !

At UC Berkeley, we have our undergraduate students hack on the
internals of PostgreSQL in the upper-division "Introduction to
Database Systems" class ..

http://www-inst.eecs.berkeley.edu/~cs186/

The "official" platform is Solaris x86 - that's where the students get
accounts and they have to get their code working on that platform as
the TAs will only test and grade their submissions on Solaris x86.

(Besides, I also got TelegraphCQ running on Solaris x86 .. just for
kicks though .. and TelegraphCQ is based off of pgsql-7.3.2)

--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Robert Treat wrote:

http://www-inst.eecs.berkeley.edu/~cs186/hwk0/index.html

Are these screenshots of PgAccess on Mac OSX?

It's pretty sad that "Mike Stonebraker" only has a salary of $15,000. ;-)

I also thought this SIGMOD article was a nice read:

http://www.acm.org/sigmod/record/issues/0309/4.JHdbcourseS03.pdf

How about extra credit for PITR?

Mike Mascari
mascarm@mascari.com

"Mike" == Mike Mascari <mascarm@mascari.com> writes:

Mike> Robert Treat wrote:

http://www-inst.eecs.berkeley.edu/~cs186/hwk0/index.html

Are these screenshots of PgAccess on Mac OSX?

Yup .. that's from Joe Hellerstein, who was the instructor in the
Spring when I was a TA.

Mike> It's pretty sad that "Mike Stonebraker" only has a salary of
Mike> $15,000. ;-)

We've got to do something now that he's a competitor :-)

Mike> I also thought this SIGMOD article was a nice read:

Mike> http://www.acm.org/sigmod/record/issues/0309/4.JHdbcourseS03.pdf

That's right .. it was a fun experience and I like to think that the
students enjoyed it. The best part was that we got to pick up 3 great
undergraduates for our research - and pgsql hacking experience is
invaluable in hacking TelegraphCQ.

Mike> How about extra credit for PITR?

One step at a time :-)

Actually a big problem is figuring out new pieces for the
projects. Most of the items in the TODO list are way too much for a
class project - we gave 'em 3 weeks to make the Hash GroupedAgg work
for large numbers of unique values (by using a form of hybrid hashing).

Another thing I toyed with was having an implementation of a
Tid-List-Fetch .. sorting a TID-list from an index and fetching the
records of the relation off the sorted list for better IO
performance. AFAICT something like this isn't present yet .. can pgsql
do this already ?

--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh

PostgreSQL most definitely works great on Solaris x86 !

At UC Berkeley, we have our undergraduate students hack on the
internals of PostgreSQL in the upper-division "Introduction to
Database Systems" class ..

http://www-inst.eecs.berkeley.edu/~cs186/

Hi Sailesh,

You know what would be kind of cool? If you could write a "Guide to
PostgreSQL to Teach Databases".

eg. You could cover how to set up the server securely (eg. schemas for
each person), etc.

How to manage it all, handle upgrades, etc. Mention what things are
good to get students to hack on in the internals, etc.

Could be a good techdocs.postgresql.org article.

Just a thought :)

Chris

"Chris" == Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:

PostgreSQL most definitely works great on Solaris x86 ! At UC
Berkeley, we have our undergraduate students hack on the
internals of PostgreSQL in the upper-division "Introduction to
Database Systems" class ..
http://www-inst.eecs.berkeley.edu/~cs186/

Chris> Hi Sailesh,

Chris> You know what would be kind of cool? If you could write a
Chris> "Guide to PostgreSQL to Teach Databases".

Chris> eg. You could cover how to set up the server securely
Chris> (eg. schemas for each person), etc.

Chris> How to manage it all, handle upgrades, etc. Mention what
Chris> things are good to get students to hack on in the
Chris> internals, etc.

Chris> Could be a good techdocs.postgresql.org article.

Hmm .. this is probably a good idea. I ought to do it for the benefit
of future TAs here anyway - but I wasn't thinking on the lines of a
full-fledged article .. just burnt out with too much writing :-)

I'm just underwater until the end of the semester. If I don't come
through by the end of december do ping me.

--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh

On Tue, Nov 18, 2003 at 02:31:03PM -0800, Sailesh Krishnamurthy wrote:

Another thing I toyed with was having an implementation of a
Tid-List-Fetch .. sorting a TID-list from an index and fetching the
records of the relation off the sorted list for better IO
performance. AFAICT something like this isn't present yet .. can pgsql
do this already ?

Nope. In fact it's on the TODO list. I think people call it "bitmap
indexes", the idea being that the TID list is represented as a "sparse
bitmap" (go ask some JPEG hacker what that means).

Extra points if multiple bitmaps can be ANDed and ORed, to allow using
multiple indexes simultaneously ...

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
Oh, oh, las chicas galacianas, lo har�n por las perlas,
�Y las de Arrakis por el agua! Pero si buscas damas
Que se consuman como llamas, �Prueba una hija de Caladan! (Gurney Halleck)

Alvaro Herrera <alvherre@dcc.uchile.cl> writes:

On Tue, Nov 18, 2003 at 02:31:03PM -0800, Sailesh Krishnamurthy wrote:

Another thing I toyed with was having an implementation of a
Tid-List-Fetch .. sorting a TID-list from an index and fetching the
records of the relation off the sorted list for better IO
performance. AFAICT something like this isn't present yet .. can pgsql
do this already ?

I think you're talking about situations like
"where x = ? or y = ?"
or
"where x = ? and y = ?"

When both `x' and `y' are indexed. It's possible to do the index lookup,
gather a list of tid pointers in some efficient format like a bit vector, and
apply the and/or and any other clauses.

Oracle can do this, and it's useful in some cases when you have DSS-style
where all the indexes have poor selectivity but using enough of them together
gets you a reasonable number of records.

In my experience it was never really very useful. I suspect there are specific
situations where it makes a big difference though.

Nope. In fact it's on the TODO list. I think people call it "bitmap
indexes", the idea being that the TID list is represented as a "sparse
bitmap" (go ask some JPEG hacker what that means).

Extra points if multiple bitmaps can be ANDed and ORed, to allow using
multiple indexes simultaneously ...

I think this is different from what he meant, but yes, bitmap indexes might be
an interesting project. Like hash indexes they have limited uses, but when
they're useful they're VERY useful. In Oracle they're mainly useful for
low-cardinality attributes like flags on rarely-updated tables. (Actually I
wonder whether locking in postgres would be simpler than locking Oracle
because of the no-update style of MVCC, hmm.)

--
greg

"Greg" == Greg Stark <gsstark@mit.edu> writes:

Greg> I think you're talking about situations like "where x = ? or
Greg> y = ?" or "where x = ? and y = ?"

Greg> When both `x' and `y' are indexed. It's possible to do the
Greg> index lookup, gather a list of tid pointers in some
Greg> efficient format like a bit vector, and apply the and/or and
Greg> any other clauses.

Yes, Index ANDing/ORing are useful whether or not the list of tids are
in an efficient format. Especially ORing for performing disjunctions.

Greg> Oracle can do this, and it's useful in some cases when you
Greg> have DSS-style where all the indexes have poor selectivity
Greg> but using enough of them together gets you a reasonable
Greg> number of records.

I guess this is the piece where "variant indexes" is useful -
essentially when you have a large number of matches for a given key.

I'm not sure how useful it is in practice - I've only read the
original research paper.

Greg> I think this is different from what he meant, but yes,
Greg> bitmap indexes might be an interesting project. Like hash

You're right .. a sorted TLF is really something quite simple that can
make quite a difference in accessing a non-clustered index. I believe
this is something all the commercial guys do. Sorting the Tids before
fetching 'em buys you buffer cache locality. When there are large
numbers of hits, it also buys you sequential scans where the file
system prefetcher can help. The additional overhead you pay is the
sorting cost.

--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh

On Tue, 2003-11-18 at 17:31, Sailesh Krishnamurthy wrote:

"Mike" == Mike Mascari <mascarm@mascari.com> writes:

Mike> How about extra credit for PITR?

One step at a time :-)

Actually a big problem is figuring out new pieces for the
projects. Most of the items in the TODO list are way too much for a
class project - we gave 'em 3 weeks to make the Hash GroupedAgg work
for large numbers of unique values (by using a form of hybrid hashing).

Something like PITR could be interesting, as there is already a patch
that starts the work, the extra credit would be to take the existing
patch and actually make it work.

Another thing I toyed with was having an implementation of a
Tid-List-Fetch .. sorting a TID-list from an index and fetching the
records of the relation off the sorted list for better IO
performance. AFAICT something like this isn't present yet .. can pgsql
do this already ?

While some form of bitmapped indexing would be cool, other ideas might
be to implement different buffer manager strategies. I was impressed by
how quickly Jan was able to implement ARC over LRU, but there are a host
of other strategies that could also be implemented.

I think there are other good projects in there, like allowing indexes
for searching nulls, or adding concurrency to GIST, or allowing non
btree indexes to handle unique's

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Robert Treat wrote:

On Tue, 2003-11-18 at 17:31, Sailesh Krishnamurthy wrote:

One step at a time :-)

Actually a big problem is figuring out new pieces for the
projects. Most of the items in the TODO list are way too much for a
class project - we gave 'em 3 weeks to make the Hash GroupedAgg work
for large numbers of unique values (by using a form of hybrid hashing).

Another thing I toyed with was having an implementation of a
Tid-List-Fetch .. sorting a TID-list from an index and fetching the
records of the relation off the sorted list for better IO
performance. AFAICT something like this isn't present yet .. can pgsql
do this already ?

While some form of bitmapped indexing would be cool, other ideas might
be to implement different buffer manager strategies. I was impressed by
how quickly Jan was able to implement ARC over LRU, but there are a host
of other strategies that could also be implemented.

Remember that interview with Jim Gray:

http://www.acmqueue.org/modules.php?name=Content&amp;pa=showpage&amp;pid=43

"Certainly we have to convert from random disk access to sequential
access patterns. Disks will give you 200 accesses per second, so if
you read a few kilobytes in each access, you're in the
megabyte-per-second realm, and it will take a year to read a
20-terabyte disk.

If you go to sequential access of larger chunks of the disk, you will
get 500 times more bandwidth�you can read or write the disk in a day.
So programmers have to start thinking of the disk as a sequential
device rather than a random access device."

Isn't a TID-List-Fetch implementation a crucial first step in the
right direction?

Mike Mascari
mascarm@mascari.com

Hi Christopher,

Personally, I haven't tested it on Solaris 9 INTEL yet (Sun took some
decent time in getting the Sol9 x86 kit to me, and I'm busy developing
OpenOffice training for now).

I'd be very surprised if PostgreSQL doesn't work on Solaris x86, as it
*used* to work on Solaris x86 from versions 2.6 -> 8.

This is the guide I wrote for compiling it on Solaris 8:

http://techdocs.postgresql.org/v2/Guides/Installation%20Guides/Compiling%20PostgreSQL%207.3.2%20on%20Solaris%208%209/view

That may be useful for you.

:-)

Regards and best wishes,

Justin Clift

Christoper Smiga wrote:

"Mike" == Mike Mascari <mascarm@mascari.com> writes:

Mike> Robert Treat wrote:

While some form of bitmapped indexing would be cool, other ideas might
be to implement different buffer manager strategies. I was impressed by
how quickly Jan was able to implement ARC over LRU, but there are a host
of other strategies that could also be implemented.

We already do that !

We have a first "warm-up" assignment for which they get 2 weeks and
have to change the strategy to MRU from LRU (in an earlier semester
they were assigned 2Q). The idea here more to just get used to the
code and the debugger.

Sadly the undergraduate OS class uses Java (horrors) as an
implementation language and many of our juniors and seniors are not as
uncomfortable with C programming (and pointers) as I'd like. The good
news is that they all pretty much got into the groove fast.

Re PITR, maybe that's an option - the thing is we are looking less at
a full semester long project and more at a 3/4 week assignment where
students get to hack something, learn about the practical side to
what's in lecture, and learn to do some performance comparisons.

Mike> If you go to sequential access of larger chunks of the disk, you will
Mike> get 500 times more bandwidth�you can read or write the disk in a day.
Mike> So programmers have to start thinking of the disk as a sequential
Mike> device rather than a random access device."

Mike> Isn't a TID-List-Fetch implementation a crucial first step in the
Mike> right direction?

I believe so .. I think it's a clear win. I believe there are some
concurrency issues although I'm not sure .. what if there is a vaccuum
that comes in between building the Tid list and then doing a fetch ?

--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh

Mucho Thanks!

Justin Clift wrote:

"Robert" == Robert Treat <xzilla@users.sourceforge.net> writes:

Robert> allowing indexes for searching nulls, or adding
Robert> concurrency to GIST, or allowing non btree indexes to

Oh this has come up before on -hackers and I've been meaning to chime
in.

Marcel Kornacker did implement concurrency for GiST - I confirmed as
much with Joe Hellerstein (his advisor). I know there's a paper he
wrote with C.Mohan on it. I don't know which version his
implementation was for.

--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh

On Wed, 2003-11-19 at 12:16, Sailesh Krishnamurthy wrote:

"Robert" == Robert Treat <xzilla@users.sourceforge.net> writes:

Robert> allowing indexes for searching nulls, or adding
Robert> concurrency to GIST, or allowing non btree indexes to

Oh this has come up before on -hackers and I've been meaning to chime
in.

Marcel Kornacker did implement concurrency for GiST - I confirmed as
much with Joe Hellerstein (his advisor). I know there's a paper he
wrote with C.Mohan on it. I don't know which version his
implementation was for.

I did a bit of googleing and came up with the following papers:
http://www.eecs.berkeley.edu/IPRO/Summary/97abstracts/marcel.1.html

but the only references I saw to implementation were in something called
amdb. If he did code it for postgresql, even an old version, having that
code/info available (if even as a link from the TODO) might be enough to
inspire someone to implement it in the current sources.

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

I have used Postgres on Solaris x86 7, 8 and 9. Haven't tested Solaris
10 but it probably would work. In fact, I've tarballed /usr/local
directories from Sol7, moved then to Sol8/9 machines and the software
worked without recompilation.

On Wed, 2003-11-19 at 10:47, Sailesh Krishnamurthy wrote:

"Mike" == Mike Mascari <mascarm@mascari.com> writes:

Mike> Robert Treat wrote:

While some form of bitmapped indexing would be cool, other ideas might
be to implement different buffer manager strategies. I was impressed by
how quickly Jan was able to implement ARC over LRU, but there are a host
of other strategies that could also be implemented.

We already do that !

:-)

We have a first "warm-up" assignment for which they get 2 weeks and
have to change the strategy to MRU from LRU (in an earlier semester
they were assigned 2Q). The idea here more to just get used to the
code and the debugger.

It would be cool if some of these were posted to -patches... *in theory*
it would give folks a chance to do real stress testing on different
implementations. if nothing else we could bug Jan some more about making
the buffer management code configurable ;-)

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Marcel Kornacker did implement concurrency for GiST - I confirmed as
much with Joe Hellerstein (his advisor). I know there's a paper he
wrote with C.Mohan on it. I don't know which version his
implementation was for.

The 7.4 GiST docs have a link to Kornacker's thesis that details how to
implement concurrent GiST and unique GiST:

http://citeseer.nj.nec.com/448594.html

I have been reading it, but I think my skills aren't really sufficient
to implement it :P

Chris