software or hardware RAID?

On 17/03/19, Rory Campbell-Lange (rory@campbell-lange.net) wrote:

We aren't sure whether to use software MDRaid or a MegaRAID card.

We're buying some new Postgres servers with

2 x 240GB Intel SSD S4610 (RAID1 : system)
4 x 960GB Intel SSD S4610 (RAID10 : db)

We'll be using Postgres 11 on Debian.

The MegaRAID 9271-8i with flash cache protection is available from our
provider. I think they may also have the 9361-8i which is 12Gb/s.

Our current servers which use the LSI 9261 with SSDs and we don't see
any IO significant load as we are in RAM most of the time and the RAID
card seems to flatten out any IO spikes.

We use MDRaid elsewhere but we've never used it for our databases
before.

Apologies for re-heating this email from last week. I could really do with the
advice.

Has anyone got any general comments on whether software RAID or an LSI card
is preferable?

We will be replicating load on an existing server, which has an LSI 9261 card.
Below is some stats from sar showing a "heavy" period of load on vdisk sda

00:00:01 DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util
14:15:01 sda 112.82 643.09 14986.24 138.53 2.09 18.50 0.25 2.86
14:25:01 sda 108.52 270.17 15682.94 147.01 1.87 17.22 0.25 2.73
14:35:01 sda 107.96 178.25 14868.52 139.37 1.70 15.73 0.23 2.53
14:45:01 sda 150.97 748.94 16919.69 117.03 1.83 12.11 0.22 3.28

Thanks for any advice.

Rory

On 3/23/19 7:09 AM, Rory Campbell-Lange wrote:

On 17/03/19, Rory Campbell-Lange (rory@campbell-lange.net) wrote:

We aren't sure whether to use software MDRaid or a MegaRAID card.

We're buying some new Postgres servers with

2 x 240GB Intel SSD S4610 (RAID1 : system)
4 x 960GB Intel SSD S4610 (RAID10 : db)

We'll be using Postgres 11 on Debian.

The MegaRAID 9271-8i with flash cache protection is available from our
provider. I think they may also have the 9361-8i which is 12Gb/s.

Our current servers which use the LSI 9261 with SSDs and we don't see
any IO significant load as we are in RAM most of the time and the RAID
card seems to flatten out any IO spikes.

We use MDRaid elsewhere but we've never used it for our databases
before.

Apologies for re-heating this email from last week. I could really do with the
advice.

Has anyone got any general comments on whether software RAID or an LSI card
is preferable?

We will be replicating load on an existing server, which has an LSI 9261 card.
Below is some stats from sar showing a "heavy" period of load on vdisk sda

00:00:01 DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util
14:15:01 sda 112.82 643.09 14986.24 138.53 2.09 18.50 0.25 2.86
14:25:01 sda 108.52 270.17 15682.94 147.01 1.87 17.22 0.25 2.73
14:35:01 sda 107.96 178.25 14868.52 139.37 1.70 15.73 0.23 2.53
14:45:01 sda 150.97 748.94 16919.69 117.03 1.83 12.11 0.22 3.28

Thanks for any advice.

Rory

I have run both software and hardware (though different than the card you listed), and had good success with both. In cases where I had little money, just drop 6 drives into a md raid 10, and run happy for years and years. I run production PG 11 on software raid 10 as we speek.

I personally prefer software raid, for a few reasons:
1) you'll probably be running on a batter backup anyway, so missing raid card battery isn't that much
2) 100% compatible with any other hardware you wanna run. Sucky thing about hardware card is your on that one forever.
3) tooling is much better and simpler. I really hate the crappy bios raid screen. I never know if adding an HD to an exiting raid will wipe it or maintain it.
4) I setup smartctl to watch and report on drives. Even a 50% chance it detects before failure is a net benefit. You cant always to that through hardware raid

You can always start with software raid, see how it runs for a while, then buy hardware raid if its not working out.

-Andy

On Sat, Mar 23, 2019 at 9:40 AM Andy Colson <andy@squeakycode.net> wrote:

On 3/23/19 7:09 AM, Rory Campbell-Lange wrote:

On 17/03/19, Rory Campbell-Lange (rory@campbell-lange.net) wrote:

We aren't sure whether to use software MDRaid or a MegaRAID card.

We're buying some new Postgres servers with

2 x 240GB Intel SSD S4610 (RAID1 : system)
4 x 960GB Intel SSD S4610 (RAID10 : db)

We'll be using Postgres 11 on Debian.

The MegaRAID 9271-8i with flash cache protection is available from our
provider. I think they may also have the 9361-8i which is 12Gb/s.

Our current servers which use the LSI 9261 with SSDs and we don't see
any IO significant load as we are in RAM most of the time and the RAID
card seems to flatten out any IO spikes.

We use MDRaid elsewhere but we've never used it for our databases
before.

Apologies for re-heating this email from last week. I could really do

with the

advice.

Has anyone got any general comments on whether software RAID or an LSI

card

is preferable?

We will be replicating load on an existing server, which has an LSI 9261

card.

Below is some stats from sar showing a "heavy" period of load on vdisk

sda

00:00:01 DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz

await svctm %util

14:15:01 sda 112.82 643.09 14986.24 138.53 2.09

18.50 0.25 2.86

14:25:01 sda 108.52 270.17 15682.94 147.01 1.87

17.22 0.25 2.73

14:35:01 sda 107.96 178.25 14868.52 139.37 1.70

15.73 0.23 2.53

14:45:01 sda 150.97 748.94 16919.69 117.03 1.83

12.11 0.22 3.28

Thanks for any advice.

Rory

I have run both software and hardware (though different than the card you
listed), and had good success with both. In cases where I had little
money, just drop 6 drives into a md raid 10, and run happy for years and
years. I run production PG 11 on software raid 10 as we speek.

I personally prefer software raid, for a few reasons:
1) you'll probably be running on a batter backup anyway, so missing raid
card battery isn't that much
2) 100% compatible with any other hardware you wanna run. Sucky thing
about hardware card is your on that one forever.

This is the main reason I ditched the rsi raid card. But I went with zfs.
It’s worth a look.

3) tooling is much better and simpler. I really hate the crappy bios raid
screen. I never know if adding an HD to an exiting raid will wipe it or
maintain it.
4) I setup smartctl to watch and report on drives. Even a 50% chance it
detects before failure is a net benefit. You cant always to that through
hardware raid

You can always start with software raid, see how it runs for a while, then
buy hardware raid if its not working out.

-Andy

--

Charles L. Martin
Martin Jones & Piemonte
BUSINESS email: service@mjpdisability.com
Personal email: clmartin@mjpdisability.com
Decatur Office:
123 N. McDonough St.
Decatur, GA 30030
404-373-3116
Fax 404-373-4110

Charlotte Office:
4601 Charlotte Park Drive, Suite 390
Charlotte, NC 28217
704-399-8890
Fax 888-490-1315

On 23/03/19, Andy Colson (andy@squeakycode.net) wrote:

On 3/23/19 7:09 AM, Rory Campbell-Lange wrote:

On 17/03/19, Rory Campbell-Lange (rory@campbell-lange.net) wrote:

...

We're buying some new Postgres servers with

2 x 240GB Intel SSD S4610 (RAID1 : system)
4 x 960GB Intel SSD S4610 (RAID10 : db)

We'll be using Postgres 11 on Debian.

The MegaRAID 9271-8i with flash cache protection is available from our
provider. I think they may also have the 9361-8i which is 12Gb/s.

Our current servers which use the LSI 9261 with SSDs and we don't see
any IO significant load as we are in RAM most of the time and the RAID
card seems to flatten out any IO spikes.

We use MDRaid elsewhere but we've never used it for our databases
before.

Has anyone got any general comments on whether software RAID or an LSI card
is preferable?

We will be replicating load on an existing server, which has an LSI 9261 card.
Below is some stats from sar showing a "heavy" period of load on vdisk sda

00:00:01 DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util
14:15:01 sda 112.82 643.09 14986.24 138.53 2.09 18.50 0.25 2.86
14:25:01 sda 108.52 270.17 15682.94 147.01 1.87 17.22 0.25 2.73
14:35:01 sda 107.96 178.25 14868.52 139.37 1.70 15.73 0.23 2.53
14:45:01 sda 150.97 748.94 16919.69 117.03 1.83 12.11 0.22 3.28

I have run both software and hardware (though different than the card you listed), and had good success with both. In cases where I had little money, just drop 6 drives into a md raid 10, and run happy for years and years. I run production PG 11 on software raid 10 as we speek.

I personally prefer software raid, for a few reasons:
1) you'll probably be running on a batter backup anyway, so missing raid card battery isn't that much
2) 100% compatible with any other hardware you wanna run. Sucky thing about hardware card is your on that one forever.
3) tooling is much better and simpler. I really hate the crappy bios raid screen. I never know if adding an HD to an exiting raid will wipe it or maintain it.
4) I setup smartctl to watch and report on drives. Even a 50% chance it detects before failure is a net benefit. You cant always to that through hardware raid

You can always start with software raid, see how it runs for a while, then buy hardware raid if its not working out.

Thanks very much for the comments, Andy.

If money was no object, would you choose a fancy hardware RAID card?

You are right that the SSDs we are purchasing have enough cache + power to not need a BBU, and I agree that the management tools for software raid are much more convenient.

Rory

On 3/23/19 11:51 AM, Rory Campbell-Lange wrote:

On 23/03/19, Andy Colson (andy@squeakycode.net) wrote:

On 3/23/19 7:09 AM, Rory Campbell-Lange wrote:

On 17/03/19, Rory Campbell-Lange (rory@campbell-lange.net) wrote:

...

We're buying some new Postgres servers with

2 x 240GB Intel SSD S4610 (RAID1 : system)
4 x 960GB Intel SSD S4610 (RAID10 : db)

We'll be using Postgres 11 on Debian.

The MegaRAID 9271-8i with flash cache protection is available from our
provider. I think they may also have the 9361-8i which is 12Gb/s.

Our current servers which use the LSI 9261 with SSDs and we don't see
any IO significant load as we are in RAM most of the time and the RAID
card seems to flatten out any IO spikes.

We use MDRaid elsewhere but we've never used it for our databases
before.

Has anyone got any general comments on whether software RAID or an LSI card
is preferable?

We will be replicating load on an existing server, which has an LSI 9261 card.
Below is some stats from sar showing a "heavy" period of load on vdisk sda

00:00:01 DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util
14:15:01 sda 112.82 643.09 14986.24 138.53 2.09 18.50 0.25 2.86
14:25:01 sda 108.52 270.17 15682.94 147.01 1.87 17.22 0.25 2.73
14:35:01 sda 107.96 178.25 14868.52 139.37 1.70 15.73 0.23 2.53
14:45:01 sda 150.97 748.94 16919.69 117.03 1.83 12.11 0.22 3.28

I have run both software and hardware (though different than the card you listed), and had good success with both. In cases where I had little money, just drop 6 drives into a md raid 10, and run happy for years and years. I run production PG 11 on software raid 10 as we speek.

I personally prefer software raid, for a few reasons:
1) you'll probably be running on a batter backup anyway, so missing raid card battery isn't that much
2) 100% compatible with any other hardware you wanna run. Sucky thing about hardware card is your on that one forever.
3) tooling is much better and simpler. I really hate the crappy bios raid screen. I never know if adding an HD to an exiting raid will wipe it or maintain it.
4) I setup smartctl to watch and report on drives. Even a 50% chance it detects before failure is a net benefit. You cant always to that through hardware raid

You can always start with software raid, see how it runs for a while, then buy hardware raid if its not working out.

Thanks very much for the comments, Andy.

If money was no object, would you choose a fancy hardware RAID card?

You are right that the SSDs we are purchasing have enough cache + power to not need a BBU, and I agree that the management tools for software raid are much more convenient.

Rory

That's a tough question. I don't even have SSD's yet. I can't say what performance is like on HW Raid SSD vs SW Raid SSD. If money where no object, and I wanted performance over all else, I'd benchmark both.

If usability were >= performance, I'd go with extra ram, and SW Raid SSD. And a BMW. :-)

-Andy

On Sun, Mar 17, 2019 at 11:54 PM Rory Campbell-Lange <
rory@campbell-lange.net> wrote:

We aren't sure whether to use software MDRaid or a MegaRAID card.

Never go with hardRaid. I have had a breakdown on a hardware RAID and as
it was special and not off-the-shelf, I could not move the disk to another
controller. I think it was a capacitor, maybe capasitor plaegue.
Only thing I had to do was to restore to the day before and the customer
lost one days work.

From that on, I only use softRAID.

On 23/03/19, Hans Schou (hans.schou@gmail.com) wrote:

On Sun, Mar 17, 2019 at 11:54 PM Rory Campbell-Lange <
rory@campbell-lange.net> wrote:

We aren't sure whether to use software MDRaid or a MegaRAID card.

Never go with hardRaid. I have had a breakdown on a hardware RAID and as
it was special and not off-the-shelf, I could not move the disk to another
controller. I think it was a capacitor, maybe capasitor plaegue.
Only thing I had to do was to restore to the day before and the customer
lost one days work.

From that on, I only use softRAID.

Thanks for that advice, Hans.

On Sat, Mar 23, 2019 at 12:09:11PM +0000, Rory Campbell-Lange wrote:

On 17/03/19, Rory Campbell-Lange (rory@campbell-lange.net) wrote:

We aren't sure whether to use software MDRaid or a MegaRAID card.

We're buying some new Postgres servers with

2 x 240GB Intel SSD S4610 (RAID1 : system)
4 x 960GB Intel SSD S4610 (RAID10 : db)

We'll be using Postgres 11 on Debian.

The MegaRAID 9271-8i with flash cache protection is available from our
provider. I think they may also have the 9361-8i which is 12Gb/s.

Our current servers which use the LSI 9261 with SSDs and we don't see
any IO significant load as we are in RAM most of the time and the RAID
card seems to flatten out any IO spikes.

We use MDRaid elsewhere but we've never used it for our databases
before.

Apologies for re-heating this email from last week. I could really do with the
advice.

Has anyone got any general comments on whether software RAID or an LSI card
is preferable?

We will be replicating load on an existing server, which has an LSI 9261 card.
Below is some stats from sar showing a "heavy" period of load on vdisk sda

00:00:01 DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util
14:15:01 sda 112.82 643.09 14986.24 138.53 2.09 18.50 0.25 2.86
14:25:01 sda 108.52 270.17 15682.94 147.01 1.87 17.22 0.25 2.73
14:35:01 sda 107.96 178.25 14868.52 139.37 1.70 15.73 0.23 2.53
14:45:01 sda 150.97 748.94 16919.69 117.03 1.83 12.11 0.22 3.28

Thanks for any advice.
Rory

Hi Rory,

The main reason, in my opinion, to use a HW RAID card is for the NVRAM
battery backed cache to support writing to traditional spinning disks.
Since your SSDs have power-loss support, you do not need that and the HW
RAID controller. For database use, you would almost certainly be using
RAID 10 and software RAID 10 is extremely performant. I am in the middle
of setting up a new system with NVMe SSD drives and HW RAID would be a
terrible bottle-neck and software RAID is really the only realistice
option.

Regards,
Ken

Hi All

Just would like to know conclusion here ,

What is best RAID method (Software Or Hardware) for Postgres DB and what
level ?

Thanks,
Raj

On Sat, Mar 23, 2019 at 3:12 PM Kenneth Marshall <ktm@rice.edu> wrote:

On 28/03/19, Perumal Raj (perucinci@gmail.com) wrote:

On Sat, Mar 23, 2019 at 3:12 PM Kenneth Marshall <ktm@rice.edu> wrote:

On Sat, Mar 23, 2019 at 12:09:11PM +0000, Rory Campbell-Lange wrote:

On 17/03/19, Rory Campbell-Lange (rory@campbell-lange.net) wrote:

We aren't sure whether to use software MDRaid or a MegaRAID card.

...

We use MDRaid elsewhere but we've never used it for our databases
before.

Apologies for re-heating this email from last week. I could really
do with the advice.

Has anyone got any general comments on whether software RAID or an
LSI card is preferable?

...

The main reason, in my opinion, to use a HW RAID card is for the NVRAM
battery backed cache to support writing to traditional spinning disks.
Since your SSDs have power-loss support, you do not need that and the HW
RAID controller. For database use, you would almost certainly be using
RAID 10 and software RAID 10 is extremely performant. I am in the middle
of setting up a new system with NVMe SSD drives and HW RAID would be a
terrible bottle-neck and software RAID is really the only realistice
option.

Just would like to know conclusion here ,

What is best RAID method (Software Or Hardware) for Postgres DB and what
level ?

Hi Perumal

Ken's comments summarise the general tenor of the advice I've been
given.

Rory