Recomended FS

Ben-Nes Michael wrote:

Hi

I'm upgrading the DB sever hardware and also the Linux OS.

My Questions are:

1. What is the preferred FS to go with ? EXT3, Reiseref, JFS, XFS ? ( speed,
efficiency )

Thats a flamebait. People never agree due to their experiences. Besides that
depends upon what kind of database you are dealing with.

Best bet is benchmark for your own app. Reiser/XFS/JFS are all good. Ext3
requires selection of proper mode. Its almost equally good. You decide what
works best for you..

2. What is the most importent part in the Hardware ? fast HD, alot of mem,
or maybe strong cpu ?

A fast HD with a good RAID controller. Subject to budget, SCSI are beter buy
than IDE. So does hardware SCSI RAID.

Shridhar

On Mon, 20 Oct 2003, Shridhar Daithankar wrote:

A fast HD with a good RAID controller. Subject to budget, SCSI are beter buy
than IDE. So does hardware SCSI RAID.

I hate asking this again. But WHY?

What SCSI gain in spinning at 15000RPM and larger buffers. They
lose in Space, and a slower bus. I would like to see some profe. Sorry.

IDE Hard Disk 40Gb 7200RPM = 133Mbs = 50UKP
SCSI Hard Disk 36Gb 10000RPM = 160Mbs = 110UKP

Is that extra 27Mbs worth another IDE Disk. and while you can get
bigger faster SCSI disks prices go through the roof. Its no longer RAID
but RAED (Redundant Array of Expensive Disks)

My advise not that I've got any proof is that the money is better
spent on a good disk controller and many disks than on each disk.

In short if you have money to burn then by all means get SCSI but
most people are better of spending

$200 Disk Controller $200 Disk Controller
$100 40Gb Disks Than $200 40Gb Disk

Prices only approx.

Peter Childs

Peter Childs wrote:

On Mon, 20 Oct 2003, Shridhar Daithankar wrote:

A fast HD with a good RAID controller. Subject to budget, SCSI are beter buy
than IDE. So does hardware SCSI RAID.

I hate asking this again. But WHY?

OK.. There are only few SCSI disks that I have handled so take it with grain of
salt.

1. SCSI bus can share bandwidth much better than IDE disks. Put two IDE disks on
same channel and two SCSI disks. See which combo performs better.
2. <Unconfirmed> SCSI disks are idividually tested and IDEs are sampled. Makes a
big difference in reliability. I know for some people IDE disks do not crash at
all but majority think SCSI are more reliable than IDEs.
3. SCSI disks have Tag commands and things alike, that makes them better at
handling load.

Technically, if you don't know the load, SCSI would make a better choice. If
you know your load very well and it is predictive, IDE might be a choice.

I would personally prefer IDE disk array with hardware RAID controller because I
can put it in my home machine, unlike SCSI. But every developer I have asked
around here, says that IDE performance starts dropping once you hit real world load.

Shridhar

Peter Childs wrote:

On Mon, 20 Oct 2003, Shridhar Daithankar wrote:

A fast HD with a good RAID controller. Subject to budget, SCSI are beter buy
than IDE. So does hardware SCSI RAID.

I hate asking this again. But WHY?

The duty cycle of SCSI drives is 100%. The duty cycle of IDE drives is
around 30-40%. Therefore one uses SCSI drives in mail and news servers
where disk access is more-or-less permanent. IDE drives usually degrade
or fail faster under such load.

From experience I have noticed that IDE drives that initially perform
at 30Mbyte/sec dropped to around 10Mbyte/sec after a year or so.

What SCSI gain in spinning at 15000RPM and larger buffers. They
lose in Space, and a slower bus. I would like to see some profe. Sorry.

IDE Hard Disk 40Gb 7200RPM = 133Mbs = 50UKP
SCSI Hard Disk 36Gb 10000RPM = 160Mbs = 110UKP

On new servers doing a software RAID1 sync between two disks, I find the
following sustained transfer rates:

SuperMicro 6013P-i ATA 133 80Gb IDE 7200rpm: 39000kbytes/sec.
SuperMicro 6013P-8 SCSI 320 72Gb SCSI 10000rpm: 65000kbytes/sec.

The IDE drives are on seperate busses. The SCSI drives are on the same bus.

I think that the 320Mhz SCSI busses are a bit faster than the 133Mhz ATA
busses.

Is that extra 27Mbs worth another IDE Disk. and while you can get
bigger faster SCSI disks prices go through the roof. Its no longer RAID
but RAED (Redundant Array of Expensive Disks)

My advise not that I've got any proof is that the money is better
spent on a good disk controller and many disks than on each disk.

In short if you have money to burn then by all means get SCSI but
most people are better of spending

I suppose that's your choice. Another way of looking that things is to
consider the worth the server has to your business and factor that into
how much you should consider spending on equipment.

e.g. if the server can be attributed to �10,000/year, then perhaps a
cheap PC will do. If �1 million of your business relies on the server,
then perhaps you should look into investing more into it.

Regards,

Nick.

--
Nick Burrett
Network Engineer, Designer Servers Ltd. http://www.dsvr.co.uk

On Mon, 20 Oct 2003, Nick Burrett wrote:

Peter Childs wrote:

On Mon, 20 Oct 2003, Shridhar Daithankar wrote:

A fast HD with a good RAID controller. Subject to budget, SCSI are beter buy
than IDE. So does hardware SCSI RAID.

I hate asking this again. But WHY?

The duty cycle of SCSI drives is 100%. The duty cycle of IDE drives is
around 30-40%. Therefore one uses SCSI drives in mail and news servers
where disk access is more-or-less permanent. IDE drives usually degrade
or fail faster under such load.

From experience I have noticed that IDE drives that initially perform
at 30Mbyte/sec dropped to around 10Mbyte/sec after a year or so.

What SCSI gain in spinning at 15000RPM and larger buffers. They
lose in Space, and a slower bus. I would like to see some profe. Sorry.

IDE Hard Disk 40Gb 7200RPM = 133Mbs = 50UKP
SCSI Hard Disk 36Gb 10000RPM = 160Mbs = 110UKP

On new servers doing a software RAID1 sync between two disks, I find the
following sustained transfer rates:

SuperMicro 6013P-i ATA 133 80Gb IDE 7200rpm: 39000kbytes/sec.
SuperMicro 6013P-8 SCSI 320 72Gb SCSI 10000rpm: 65000kbytes/sec.

The IDE drives are on seperate busses. The SCSI drives are on the same bus.

I think that the 320Mhz SCSI busses are a bit faster than the 133Mhz ATA
busses.

Is that extra 27Mbs worth another IDE Disk. and while you can get
bigger faster SCSI disks prices go through the roof. Its no longer RAID
but RAED (Redundant Array of Expensive Disks)

My advise not that I've got any proof is that the money is better
spent on a good disk controller and many disks than on each disk.

In short if you have money to burn then by all means get SCSI but
most people are better of spending

I suppose that's your choice. Another way of looking that things is to
consider the worth the server has to your business and factor that into
how much you should consider spending on equipment.

e.g. if the server can be attributed to £10,000/year, then perhaps a
cheap PC will do. If £1 million of your business relies on the server,
then perhaps you should look into investing more into it.

At last somone who has the real answers that I thought ought to be
true all the time. Its a shame nobody can give some hard and fast numbers
that I can get the budget people to understand!

Peter Childs

I'm not a HD specialist but I know scsi can handle load much better the IDE.

I read a benchmark lately ( don't really remember where ) checking SATA
against U160, the result show that SATA give better performance at start.

but later on the SCSI take it while HD cpu load is 30% and the SATA is 100%
load for the same task.

So I see its kinda obvious for me, if its a server serve lots of files and
the HD will work against lots of users ill go for the SCSI.
For a workstation or backup server ill go for IDE.

But still the greatest question is what FS to put on ?

I heard Reiesref can handle small files very quickly.
--------------------------
Canaan Surfing Ltd.
Internet Service Providers
Ben-Nes Michael - Manager
Tel: 972-4-6991122
Fax: 972-4-6990098
http://www.canaan.net.il
--------------------------
----- Original Message -----
From: "Peter Childs" <blue.dragon@blueyonder.co.uk>
To: "Shridhar Daithankar" <shridhar_daithankar@persistent.co.in>
Cc: "Ben-Nes Michael" <miki@canaan.co.il>; "postgresql"
<pgsql-general@postgresql.org>
Sent: Monday, October 20, 2003 11:51 AM
Subject: Re: [GENERAL] Recomended FS

On Mon, 20 Oct 2003, Shridhar Daithankar wrote:

A fast HD with a good RAID controller. Subject to budget, SCSI are beter

buy

Quoth miki@canaan.co.il ("Ben-Nes Michael"):

I'm not a HD specialist but I know scsi can handle load much better the IDE.

I read a benchmark lately ( don't really remember where ) checking SATA
against U160, the result show that SATA give better performance at start.
but later on the SCSI take it while HD cpu load is 30% and the SATA is 100%
load for the same task.

So I see its kinda obvious for me, if its a server serve lots of files and
the HD will work against lots of users ill go for the SCSI.
For a workstation or backup server ill go for IDE.

But still the greatest question is what FS to put on ?

I heard Reiesref can handle small files very quickly.

ReiserFS was designed to cope with having huge hordes of tiny files.
PostgreSQL doesn't create files in that pattern; it only creates
fairly large files, and that tends to be the pathological case where
ReiserFS works somewhat badly.

When I ran some transaction-heavy benchmarks between ext3, XFS, and
JFS, I found JFS to be pretty consistently faster. I didn't bother
trying reiserfs because:
a) It has a history of being slower for big files;
b) I have had some cases of losing data to it, diminishing my trust
of it.
--
output = ("cbbrowne" "@" "ntlug.org")
http://www.ntlug.org/~cbbrowne/unix.html
"sic transit discus mundi"
-- From the System Administrator's Guide, by Lars Wirzenius

Ben-Nes Michael wrote:

But still the greatest question is what FS to put on ?

I heard Reiesref can handle small files very quickly.

Switching from ext3 to reiserfs for our name servers reduced the time
taken to load 110,000 zones from 45 minutes to 5 minutes.

However for a database, I don't think you can really factor this type of
stuff into the equation. The performance benefits you get from
different filesystem types are going to be small compared to the
modifications that you can make to your database structure, queries and
applications. The actual algorithms used in processing the data will be
much slower than the time taken to fetch the data off disk.

--
Nick Burrett
Network Engineer, Designer Servers Ltd. http://www.dsvr.co.uk

On Mon, 20 Oct 2003 11:07:20 +0100
Nick Burrett <nick@dsvr.net> wrote:

From experience I have noticed that IDE drives that initially perform

at 30Mbyte/sec dropped to around 10Mbyte/sec after a year or so.

Yes. This is very true - a good test I like to show of IDE falling apart
is to start up one client and show it go very fast. Then start up 20
and see what happens :)

Also - you can easily have many, many more scsi devices (and external
scsi devices) than IDE. More platters / disks == faster IO.

IDE Hard Disk 40Gb 7200RPM = 133Mbs = 50UKP
SCSI Hard Disk 36Gb 10000RPM = 160Mbs = 110UKP

If you don't mind refurb disks that still have a warranty, check out
ebay. Friday I won a lot of 10 18GB disks for $96 + $27
insured shipping. But yeah, new scsi is quite expensive, but it can be
worth it... IMHO scsi is to be used in a raid, not alone. No one disk
can saturate the bw offered. (both ide and scsi).

--
Jeff Trout <jeff@jefftrout.com>
http://www.jefftrout.com/
http://www.stuarthamm.net/

----- Original Message -----
From: "Nick Burrett" <nick@dsvr.net>
To: "Ben-Nes Michael" <miki@canaan.co.il>
Cc: "Peter Childs" <blue.dragon@blueyonder.co.uk>; "Shridhar Daithankar"
<shridhar_daithankar@persistent.co.in>; "postgresql"
<pgsql-general@postgresql.org>
Sent: Monday, October 20, 2003 2:08 PM
Subject: Re: [GENERAL] Recomended FS

Ben-Nes Michael wrote:

But still the greatest question is what FS to put on ?

I heard Reiesref can handle small files very quickly.

Switching from ext3 to reiserfs for our name servers reduced the time
taken to load 110,000 zones from 45 minutes to 5 minutes.

However for a database, I don't think you can really factor this type of
stuff into the equation. The performance benefits you get from
different filesystem types are going to be small compared to the
modifications that you can make to your database structure, queries and
applications. The actual algorithms used in processing the data will be
much slower than the time taken to fetch the data off disk.

So you say the FS has no real speed impact on the SB ?

In my pg data folder i have 2367 files, some big some small.

Ben-Nes Michael wrote:

----- Original Message -----
From: "Nick Burrett" <nick@dsvr.net>

Ben-Nes Michael wrote:

But still the greatest question is what FS to put on ?

I heard Reiesref can handle small files very quickly.

Switching from ext3 to reiserfs for our name servers reduced the time
taken to load 110,000 zones from 45 minutes to 5 minutes.

However for a database, I don't think you can really factor this type of
stuff into the equation. The performance benefits you get from
different filesystem types are going to be small compared to the
modifications that you can make to your database structure, queries and
applications. The actual algorithms used in processing the data will be
much slower than the time taken to fetch the data off disk.

So you say the FS has no real speed impact on the SB ?

In my pg data folder i have 2367 files, some big some small.

I'm saying: don't expect your DB performance to come on leaps and bounds
just because you changed to a different filesystem format. If you've
got speed problems then it might help to look elsewhere first.

--
Nick Burrett
Network Engineer, Designer Servers Ltd. http://www.dsvr.co.uk

----- Original Message -----
From: "Nick Burrett" <nick@dsvr.net>
To: "Ben-Nes Michael" <miki@canaan.co.il>
Cc: "postgresql" <pgsql-general@postgresql.org>
Sent: Monday, October 20, 2003 2:54 PM
Subject: Re: [GENERAL] Recomended FS

But still the greatest question is what FS to put on ?

I heard Reiesref can handle small files very quickly.

Switching from ext3 to reiserfs for our name servers reduced the time
taken to load 110,000 zones from 45 minutes to 5 minutes.

However for a database, I don't think you can really factor this type of
stuff into the equation. The performance benefits you get from
different filesystem types are going to be small compared to the
modifications that you can make to your database structure, queries and
applications. The actual algorithms used in processing the data will be
much slower than the time taken to fetch the data off disk.

So you say the FS has no real speed impact on the SB ?

In my pg data folder i have 2367 files, some big some small.

I'm saying: don't expect your DB performance to come on leaps and bounds
just because you changed to a different filesystem format. If you've
got speed problems then it might help to look elsewhere first.

I dont expect miracles :)
but still i have to choose one,so why shouldnt i choose the one which best
fit ?

Ben-Nes Michael wrote:

I'm saying: don't expect your DB performance to come on leaps and bounds
just because you changed to a different filesystem format. If you've
got speed problems then it might help to look elsewhere first.

I dont expect miracles :)
but still i have to choose one,so why shouldnt i choose the one which best
fit ?

All things being equal, you should optimise your application design and database
tuning before you choose file system.

If a thing works well for you, with a better file system it will just work
better. That's the point.

Shridhar

----- Original Message -----
From: "Shridhar Daithankar" <shridhar_daithankar@persistent.co.in>
To: "Ben-Nes Michael" <miki@canaan.co.il>
Cc: "Nick Burrett" <nick@dsvr.net>; "postgresql"
<pgsql-general@postgresql.org>
Sent: Monday, October 20, 2003 3:06 PM
Subject: Re: [GENERAL] Recomended FS

Ben-Nes Michael wrote:

I'm saying: don't expect your DB performance to come on leaps and bounds
just because you changed to a different filesystem format. If you've
got speed problems then it might help to look elsewhere first.

I dont expect miracles :)
but still i have to choose one,so why shouldnt i choose the one which

best

fit ?

All things being equal, you should optimise your application design and

database

tuning before you choose file system.

If a thing works well for you, with a better file system it will just work
better. That's the point.

I agree, but still ill have to choose an FS, does the list have any opinion
on what FS to choose ?

Peter Childs wrote:

On Mon, 20 Oct 2003, Shridhar Daithankar wrote:

A fast HD with a good RAID controller. Subject to budget, SCSI are
beter buy than IDE. So does hardware SCSI RAID.

I hate asking this again. But WHY?

What SCSI gain in spinning at 15000RPM and larger
buffers. They lose in Space, and a slower bus. I would like
to see some profe. Sorry.

They win it, easily, on random disk accesses and mixed reads and writes.
And the bus is, much, faster not slower.

IDE Hard Disk 40Gb 7200RPM = 133Mbs = 50UKP
SCSI Hard Disk 36Gb 10000RPM = 160Mbs = 110UKP

Is that extra 27Mbs worth another IDE Disk. and while
you can get bigger faster SCSI disks prices go through the
roof. Its no longer RAID but RAED (Redundant Array of Expensive Disks)

You're looking at the BUS speed, not the actual speed the disk achieves.
My guess is that that SCSI disk is, on some fields, twice as fast as the
IDE and on average 10-30% faster.

My advise not that I've got any proof is that the money
is better spent on a good disk controller and many disks than
on each disk.

This havily depends on your setup and tasks.

- SCSI has a (supposedly) better lifetime, due to (much) better disk
components.
- SCSI disks are designed for servertasks (many random accesses) and
have their queue-management (better) tuned for that. This also applies
to mixed reads and writes.
- SCSI disks have, often, smaller and thicker platters which can spin
more stable and at higher RPMs.
- The SCSI bus allows all the disks to operate at maximum speed (as far
as the PCI-bus can handle it of course), while the IDE bus is shared
among both disks.
- SCSI allows more disks and longer cables on the same controller.

Anyway, you don't need all those advantages all the time, since the
major disadvantage is of course the pricetag.
For simple backup solutions (many storage for with reasonable
performance and an acceptable price), IDE is quite good in RAID5 orso.
For a high performing Database, you really want to look into a RAID
setup with scsi (or at least WD Raptor IDE disks or something like
that).

In short if you have money to burn then by all means
get SCSI but most people are better of spending

Also if you don't have money to burn, but simply need the higher
performance (which is really there) for, for instance, the random disk
accesses.

Best regards,

Arjen

On Mon, 20 Oct 2003, Peter Childs wrote:

On Mon, 20 Oct 2003, Shridhar Daithankar wrote:

A fast HD with a good RAID controller. Subject to budget, SCSI are beter buy
than IDE. So does hardware SCSI RAID.

I hate asking this again. But WHY?

What SCSI gain in spinning at 15000RPM and larger buffers. They
lose in Space, and a slower bus. I would like to see some profe. Sorry.

SCSI beats IDE hands down for databases, and for one reason above all the
rest. They don't generally lie about fsync.

With SCSI, you can initiate 'pgbench -c 100 -t 1000000' and pull the plug
on your server, and voila, the whole thing will come back up (assuming a
journaling file system, and solid hardware.)

Do that with IDE with write cache enabled and you WILL have a scrambled
database that needs to be re-initdbed and restored.

Now, turn off the write cache on the IDE drive, which will make it solid
and reliable like the SCSI drive, and compare speed, it's not even close.

Until the IDE drive manufacturers start making IDE drives that actually
report fsync properly, they're a toy that should not be used for your
database unless you know the dangers they present.

You'd be well served if you could benchmark several filesystems and see
which one gives the best "performance" (talk about a loaded term) for your
application. Having said that, however, I'd recommend XFS for its
combination of performance and userspace tools (particularly xfsdump and
xfs_freeze).

Cheers,
Murthy

Ben-Nes Michael writes:

1. What is the preferred FS to go with ? EXT3, Reiseref, JFS, XFS ? ( speed,
efficiency )

PostgreSQL might work better on "simple" file systems, so you avoid making
the head run all over the place for writing its own log and the PostgreSQL
log. Some have even suggested FAT for the data files. Good bets for
improving performance are putting the WAL logs and the indexes not on the
same spindle as the table files. Of course, certain RAID configurations
achieve a similar effect.

2. What is the most importent part in the Hardware ? fast HD, alot of mem,
or maybe strong cpu ?

Lots of memory, so you can cache a large fraction of the data in memory.
A good hard disk, if you do a lot of updates and/or your memory is not big
enough to cache most of the data. CPU is not as important.

--
Peter Eisentraut peter_e@gmx.net

Hi Ben,

You asked so here's my take on the subject, but I've gotta say that you
can't go far wrong with reading Bruce Momjian's paper at:

http://www.ca.postgresql.org/docs/momjian/hw_performance/

But with that aside.

1. Unless your doing major league DB stuff, the FS should make more than
marginal difference, if it's Journaled then it's good. You can take all
the time benchmarking that you want, just be sure your ROI is worth the
time you invest. My favourite fs is Reiser, but in the cold light of
day, ext3 is supported in more places. My first choice is Reiser, since
I used it even when it was "unstable" on production servers and it never
let me down. I often use one or the other.

2. Bruce's article really is good for this question, but in a nutshell
you need to get as much of the DB as close to the CPU as possible. As
with any serious application, you can't beat a good L1/L2 cache, then
plenty of RAM/Memory ... DBs yum RAM, the more the merrier. Lastly fast
and wide disc access, remember disk access will be the slowest part of
the system, and in an ideal world you'd fit nearly all of your DB in RAM
if it was practical and safe.

You'd probably gain more from taking the time to really ensure that your
DB is designed flawlessly, and all your indexes are where they're
needed. All of the basics come into play, but a well built RDBMS system
is greater than the sum of its parts.

For further reading check out:

http://www.argudo.org/postgresql/soft-tuning.html

It all adds up!!.

Good Luck

Tony.

Ben-Nes Michael wrote: