Solaris

On Wednesday 23 April 2003 20:33, Ian Harding wrote:

I am running my PostgreSQL database on a Dell PowerEdge 2400. What,
advantages/disadvantages are there to learning Solaris and migrating my
stuff over to the coffee table machine? Does the Dell fall under the
heading of "cheap" (crummy) hardware that Tom alluded to causing corruption
issues?

Dell server might be good enough for most tasks but if the sun machine is 64
bit, then you have a significant advantage there. Besides they should be bit
more robust(Sorry, this is absolute wild shot. Have no idea about specs of
any of these two machines.)

Just one real advice, from what I have heard on lists. If the sun machine
supports, install linux rather than solaris. Apparently linux on sparc is far
faster than solaris, at least at lower to middle end. Besides postgresql and
solaris have had performance issues in past.

Shridhar

On Wed, 23 Apr 2003, Shridhar Daithankar wrote:

On Wednesday 23 April 2003 20:33, Ian Harding wrote:

I am running my PostgreSQL database on a Dell PowerEdge 2400. What,
advantages/disadvantages are there to learning Solaris and migrating my
stuff over to the coffee table machine? Does the Dell fall under the
heading of "cheap" (crummy) hardware that Tom alluded to causing corruption
issues?

Dell server might be good enough for most tasks but if the sun machine is 64
bit, then you have a significant advantage there. Besides they should be bit
more robust(Sorry, this is absolute wild shot. Have no idea about specs of
any of these two machines.)

Just one real advice, from what I have heard on lists. If the sun machine
supports, install linux rather than solaris. Apparently linux on sparc is far
faster than solaris, at least at lower to middle end. Besides postgresql and
solaris have had performance issues in past.

Even better, throw in an extra drive and configure it to dual boot. Then
you can test the two against each other be sure which is faster for what
you're doing.

On Wed, 23 Apr 2003 20:44:02 +0530 Shridhar Daithankar <shridhar_daithankar@persistent.co.in> wrote:

Just one real advice, from what I have heard on lists. If the sun
machine
supports, install linux rather than solaris. Apparently linux on sparc
is far
faster than solaris, at least at lower to middle end. Besides postgresql
and
solaris have had performance issues in past.

investigate OpenBSD and NetBSD as well (although if the Sun has multiple
processors, which is likely with an E, OpenBSD is no longer a candidate due
to lack of SMP support.) the *BSD systems are generally very stable and
reliable, and have a couple of advantages over Linux in a number of
situations.

richard
--
Richard Welty rwelty@averillpark.net
Averill Park Networking 518-573-7592
Unix, Linux, IP Network Engineering, Security

You should at least have ECC RAM in the Solaris box, but then again, you
should have ECC RAM in any "server" to avoid in memory corruption.

If the disk "Array" blew it's toys, then it sounds like you may have
yourself a RAID controller, you could easily buy a few new hard disks,
and get yourself back up and running. RAID 5 needs a minimum of three
disks, RAID 1 Needs two. Failing the RAID controller, the solaris will
have a SCSI subsystem .... mmmmm, 15,000rpm, 16MB Cache disks....

Hadley.

On Thu, 2003-04-24 at 03:34, scott.marlowe wrote:

On Wed, 23 Apr 2003, Shridhar Daithankar wrote:

On Wednesday 23 April 2003 20:33, Ian Harding wrote:

I am running my PostgreSQL database on a Dell PowerEdge 2400. What,
advantages/disadvantages are there to learning Solaris and migrating my
stuff over to the coffee table machine? Does the Dell fall under the
heading of "cheap" (crummy) hardware that Tom alluded to causing corruption
issues?

Dell server might be good enough for most tasks but if the sun machine is 64
bit, then you have a significant advantage there. Besides they should be bit
more robust(Sorry, this is absolute wild shot. Have no idea about specs of
any of these two machines.)

Just one real advice, from what I have heard on lists. If the sun machine
supports, install linux rather than solaris. Apparently linux on sparc is far
faster than solaris, at least at lower to middle end. Besides postgresql and
solaris have had performance issues in past.

Even better, throw in an extra drive and configure it to dual boot. Then
you can test the two against each other be sure which is faster for what
you're doing.

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

--
Hadley Willan > Systems Development > Deeper Design Limited. +64(7)377-3328
hadley.willan@deeperdesign.co.nz > www.deeperdesign.com > +64(21)-28-41-463
Level 1, 4 Tamamutu St, PO Box 90, TAUPO 2730, New Zealand.

Shridhar Daithankar wrote:

Just one real advice, from what I have heard on lists. If the sun machine
supports, install linux rather than solaris. Apparently linux on sparc is far
faster than solaris, at least at lower to middle end.

Any idea how much faster (and in what areas)?

Besides postgresql and
solaris have had performance issues in past.

I believe this is all *sorted* now :-) The issue was the qsort library
bundled with Solaris - Postgresql now supplies its own. So Solaris is ok
for Postgresql (I think quite a few folk actually use this combination).

best wishes

Mark

On Fri, Apr 25, 2003 at 02:03:24PM +1200, Mark Kirkwood wrote:

I believe this is all *sorted* now :-) The issue was the qsort library
bundled with Solaris - Postgresql now supplies its own. So Solaris is ok
for Postgresql (I think quite a few folk actually use this combination).

It's still not all that fast. Better, but not what you'd expect.
There's something fishy with the SYSV shared memory management, but
I'm darned if I can figure out what it is.

A

-- 
----
Andrew Sullivan                         204-4141 Yonge Street
Liberty RMS                           Toronto, Ontario Canada
<andrew@libertyrms.info>                              M2P 2A8
                                         +1 416 646 3304 x110

Andrew Sullivan <andrew@libertyrms.info> writes:

There's something fishy with the SYSV shared memory management, but
I'm darned if I can figure out what it is.

Hmm, you're still running 7.2.*, right? There's some code added in 7.3
to enable "intimate shared memory" on Solaris:

#if defined(solaris) && defined(__sparc__)
/* use intimate shared memory on SPARC Solaris */
memAddress = shmat(shmid, 0, SHM_SHARE_MMU);
#else
memAddress = shmat(shmid, 0, 0);
#endif

I disremember the details but we were told this would improve
performance. It'd be an easy enough patch in 7.2
(src/backend/storage/ipc/ipc.c about line 638) if you care to try it.

regards, tom lane

On Thu, Apr 24, 2003 at 11:08:11PM -0400, Tom Lane wrote:

Hmm, you're still running 7.2.*, right? There's some code added in 7.3
to enable "intimate shared memory" on Solaris:

Tried it. I was unable to show that it helped, and it seemed to make
things worse in some cases. As usual, the time I actually got to
spend doing this was (it felt) inversely proportionate to its value,
which means that any numbers I managed to produce are not real
reliable. But it sure didn't have the effect I was hoping for. The
best I can say is that the numbers were inconclusive for our
application. I have some more tests planned, if I can find the time.

A

-- 
----
Andrew Sullivan                         204-4141 Yonge Street
Liberty RMS                           Toronto, Ontario Canada
<andrew@libertyrms.info>                              M2P 2A8
                                         +1 416 646 3304 x110

Andrew Sullivan wrote:

It's still not all that fast. Better, but not what you'd expect.
There's something fishy with the SYSV shared memory management, but
I'm darned if I can figure out what it is.

Just out of curiousity, where are you seeing the effect of this the most?
(Running through my mind is the thought that VM management in Linux is
not so hot either....)

cheers
Mark

On Fri, 25 Apr 2003, Mark Kirkwood wrote:

Shridhar Daithankar wrote:

Just one real advice, from what I have heard on lists. If the sun machine
supports, install linux rather than solaris. Apparently linux on sparc is far
faster than solaris, at least at lower to middle end.

Any idea how much faster (and in what areas)?

My general experience from a year ago was that Linux was about 2 to 3
times faster on older 32 bit sparc hardware than Solaris, and just under 2
times as fast on 64 bit hardware. This is with one CPU. I'd expect a 64
CPU E10k to run postgresql faster running Solaris rather than Linux, but
until I can come up with a spare $24k to buy a used one on Ebay I won't
know. :-)

Besides postgresql and
solaris have had performance issues in past.

I believe this is all *sorted* now :-) The issue was the qsort library
bundled with Solaris - Postgresql now supplies its own. So Solaris is ok
for Postgresql (I think quite a few folk actually use this combination).

Mostly. The issue in the past was mainly that Solaris had a brain damaged
sort() call that was very slow when it had a lot of duplicate keys. That
has since been updated by Sun.

However, Solaris' heavy process / light thread design is probably sub
optimal for Postgresql in an environment where you are forking the server
all the time. If you have pooled connections, then Solaris should do
fine. I'd love to get ahold of an older SMP Sparc box and test the two
against each other.

scott.marlowe wrote:

My general experience from a year ago was that Linux was about 2 to 3
times faster on older 32 bit sparc hardware than Solaris, and just under 2
times as fast on 64 bit hardware.

Wow - 2 times faster is significant ! It would be interesting to try a
midrange SMP box ( e.g E280/E480 ).

The issue in the past was mainly that Solaris had a brain damaged
sort() call that was very slow when it had a lot of duplicate keys. That
has since been updated by Sun.

Hmm - I ran into that situation with qsort() and many equal keys (see
Hackers thread "Solaris Performance"). Sadly Sun have not amended that
situation at all (in Solaris 8 anyway).

best wishes

Mark

On Fri, Apr 25, 2003 at 11:19:47PM +1200, Mark Kirkwood wrote:

Just out of curiousity, where are you seeing the effect of this the most?
(Running through my mind is the thought that VM management in Linux is
not so hot either....)

The problems seem to be somewhat related to load. With no load, of
course, everything is real fast. But with any sort of load, we get
occasional spikes of unexpectedly slow performance, even on things
like INSERT. Some of this I have been able to attribute to
maintenance issues, but not all of it. We see from truss, however,
that a significant amount of timee is spent in the SYSV system
calls, so it appears that there is some kind of inefficiency there.
Sorry I'm not able to provide more detail; as I say, I'm not real
happy with the extent of the tests I've been able to conduct, and in
any case, the problems aren't reproducable at will.

A

-- 
----
Andrew Sullivan                         204-4141 Yonge Street
Liberty RMS                           Toronto, Ontario Canada
<andrew@libertyrms.info>                              M2P 2A8
                                         +1 416 646 3304 x110

On Sat, 26 Apr 2003, Mark Kirkwood wrote:

scott.marlowe wrote:

My general experience from a year ago was that Linux was about 2 to 3
times faster on older 32 bit sparc hardware than Solaris, and just under 2
times as fast on 64 bit hardware.

Wow - 2 times faster is significant ! It would be interesting to try a
midrange SMP box ( e.g E280/E480 ).

Keep in mind, this is running Postgresql that measures out at about twice
as fast. I was testing with things like the 10k row database in the
regression tests, a few of our own 1M row tables, and pgbench with
parallel access up to a good amount (I think about 64 or so) The boxes
were an Ultra 1, and a Sparc 20. The Sparc 20 / Linux was running Solaris
and Linux in dual boot, and the linux performance on it was astounding,
Solaris was painfully slow at everything.

On the Ultra 1, the race was much closer, i.e. Linux was about 80% faster
than solaris. I think we were running whatever the latest production
quality versions of RedHat (6.2, the last Sparc version) and solaris at
the time.

note that we're talking about a year and a half or so ago, so things may
have changed under both OSes.

The issue in the past was mainly that Solaris had a brain damaged
sort() call that was very slow when it had a lot of duplicate keys. That
has since been updated by Sun.

Hmm - I ran into that situation with qsort() and many equal keys (see
Hackers thread "Solaris Performance"). Sadly Sun have not amended that
situation at all (in Solaris 8 anyway).

Actually, I thought there was patch out there somewhere. Anyone else
know?

scott.marlowe wrote:

note that we're talking about a year and a half or so ago, so things may
have changed under both OSes.

The issue in the past was mainly that Solaris had a brain damaged
sort() call that was very slow when it had a lot of duplicate keys. That
has since been updated by Sun.

Hmm - I ran into that situation with qsort() and many equal keys (see
Hackers thread "Solaris Performance"). Sadly Sun have not amended that
situation at all (in Solaris 8 anyway).

Actually, I thought there was patch out there somewhere. Anyone else
know?

qsort() patch is in PostgreSQL 7.3. We used the FreeBSD one for Solaris.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

scott.marlowe wrote:

Actually, I thought there was patch out there somewhere. Anyone else
know?

It looks like Solaris 9 has this included in a libc patch:
http://sunsolve.sun.com/pub-cgi/retrieve.pl?doc=fpatches/112874

However I think the solution chosen by the Postgresql developers -
using qsort.c from BSD is the best one, as it *sorts* all the Solaris
releases (its in your Pg source : src/port/qsort.c).

In fact I wonder if using the BSD qsort for *all* platforms might be a
good idea - it would provide cross platform consistency and possibly
better performance (e.g. It was several times quicker than glibc qsort,
when I checked this on Linux last year.... )

best wishes
Mark

Andrew Sullivan wrote:

We see from truss, however,
that a significant amount of timee is spent in the SYSV system
calls, so it appears that there is some kind of inefficiency there.

It would be interesting to run a profiling enabled postgres binary -
altho that's not really nice on a production system :-).

in any case, the problems aren't reproducable at will.

The best ones never seem to be - means you can't try the profiling in a
safe environment....:-(

regards

Mark

Mark Kirkwood wrote:

scott.marlowe wrote:

Actually, I thought there was patch out there somewhere. Anyone else
know?

It looks like Solaris 9 has this included in a libc patch:
http://sunsolve.sun.com/pub-cgi/retrieve.pl?doc=fpatches/112874

However I think the solution chosen by the Postgresql developers -
using qsort.c from BSD is the best one, as it *sorts* all the Solaris
releases (its in your Pg source : src/port/qsort.c).

In fact I wonder if using the BSD qsort for *all* platforms might be a
good idea - it would provide cross platform consistency and possibly
better performance (e.g. It was several times quicker than glibc qsort,
when I checked this on Linux last year.... )

Replacing qsort() on all platforms has a "we know better than the OS"
feel to it that we try to avoid.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Mark Kirkwood wrote:

In fact I wonder if using the BSD qsort for *all* platforms might be a
good idea - it would provide cross platform consistency and possibly
better performance (e.g. It was several times quicker than glibc qsort,
when I checked this on Linux last year.... )

Replacing qsort() on all platforms has a "we know better than the OS"
feel to it that we try to avoid.

I agree on that --- but when it's provable that we do know better than a
*particular* OS, dropping in the BSD qsort seems like an easy win. Can
anyone back up Mark's finding that the BSD qsort is quicker than glibc's?

regards, tom lane

In article <26779.1051629468@sss.pgh.pa.us>,
Tom Lane <tgl@sss.pgh.pa.us> wrote:

I agree on that --- but when it's provable that we do know better than a
*particular* OS, dropping in the BSD qsort seems like an easy win. Can
anyone back up Mark's finding that the BSD qsort is quicker than glibc's?

Better yet: Anyway of running performance tests from configure?

Would a simple counter in the compare function be sufficient to determine
the speed?

mrc

--
Mike Castle dalgoda@ix.netcom.com www.netcom.com/~dalgoda/
We are all of us living in the shadow of Manhattan. -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc