Re: [GENERAL] Do we need more emphasis on backup?

Bruce Momjian <pgman@candle.pha.pa.us> wrote:

pgman wrote:

Jim Seymour wrote:

[snip]

My position is that your data is only as reliable as your hardware,
period.

[snip]

There is a basic misconception that all PC hardware is created equal ---
that hard drives, mother boards, and RAM are all the same because they
are all PC-compatible. Compatible != Similar Quality.

[snip]

Should I add an FAQ discussing hardware selection and the importance of
reliable hardware?

Scary to think that people who don't know enough about running a proper
server, and thus have to be *told* this, are admin'ing databases.

Start with: Gotta have ECC, or at least parity-checking, RAM.

Mention the (some kinds of?) IDE drives issue.

Mention that hardware RAID systems must have battery-backed write
cache. (I actually had a RAID vendor, whose products no longer had
batter-backed write cache, tell me "Just use a UPS.")

Jim

On Tue, Jul 06, 2004 at 08:32:15AM -0400, Jim Seymour wrote:

Bruce Momjian <pgman@candle.pha.pa.us> wrote:

Should I add an FAQ discussing hardware selection and the importance of
reliable hardware?

Scary to think that people who don't know enough about running a proper
server, and thus have to be *told* this, are admin'ing databases.

Maybe it's scary for people in the US or the western/wealthier Europe.
But people also live in other parts of the world (I could even mention
that real people do live in places like Iraq and they even have
computers there).

For us rest-of-the-worlders, it's not always possible to get top of the
line hardware. I am somewhat lucky because I live in a country where
people can afford an UPS. Some people are not.

Of course you can talk about the value of the data and how much good
hardware can help you not lose that data -- but truth is, people need to
make an economical analysis on the worthiness of such protection, and
while some of them will be able to afford redundant battery backed SCSI
RAID controllers, some others won't be able to buy an UPS.

So I think it's a good idea to mention that better hardware can be
helpful. But don't make that sound like if you don't have it, it's the
end of the world or the admin is an uneducated moron.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"Hoy es el primer d�a del resto de mi vida"

Alvaro Herrera wrote:

On Tue, Jul 06, 2004 at 08:32:15AM -0400, Jim Seymour wrote:

Bruce Momjian <pgman@candle.pha.pa.us> wrote:

Should I add an FAQ discussing hardware selection and the importance of
reliable hardware?

Scary to think that people who don't know enough about running a proper
server, and thus have to be *told* this, are admin'ing databases.

Maybe it's scary for people in the US or the western/wealthier Europe.
But people also live in other parts of the world (I could even mention
that real people do live in places like Iraq and they even have
computers there).

For us rest-of-the-worlders, it's not always possible to get top of the
line hardware. I am somewhat lucky because I live in a country where
people can afford an UPS. Some people are not.

Of course you can talk about the value of the data and how much good
hardware can help you not lose that data -- but truth is, people need to
make an economical analysis on the worthiness of such protection, and
while some of them will be able to afford redundant battery backed SCSI
RAID controllers, some others won't be able to buy an UPS.

So I think it's a good idea to mention that better hardware can be
helpful. But don't make that sound like if you don't have it, it's the
end of the world or the admin is an uneducated moron.

Good point. I will write up some text for the FAQ and post it. I will
emphasize that there is great variation in the quality of PC-compatible
hardware, and mention a few specific examples.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Alvaro Herrera <alvherre@dcc.uchile.cl> wrote:

On Tue, Jul 06, 2004 at 08:32:15AM -0400, Jim Seymour wrote:

Bruce Momjian <pgman@candle.pha.pa.us> wrote:

Should I add an FAQ discussing hardware selection and the importance of
reliable hardware?

Scary to think that people who don't know enough about running a proper
server, and thus have to be *told* this, are admin'ing databases.

[snip]

For us rest-of-the-worlders, it's not always possible to get top of the
line hardware.

[snip]

There's a difference between "not knowing your should" and "not being
able to do it." I addressed cluelessness, not economic ability.

[snip]

So I think it's a good idea to mention that better hardware can be
helpful.

"Helpful?" Hmmm... You know, recently there was a "consumer alert"
item during one of the major U.S. TV networks' news shows. It seems
that all kinds of counterfeit products are making it into the stores
here. Even after-market automotive brake pads made of nothing but
compressed and painted/dyed vegetable matter. Needless to say, using
brake bads made of proper brake pad material is "helpful." ;)

Somebody's *ability* to employ proper hardware does not affect whether
or not it's advisable to do so. This isn't a "rich" vs. "poor" issue.
If you cannot do it, you cannot do it, and that's that. But that does
not lessen the importance of knowing you should.

But don't make that sound like if you don't have it, it's the
end of the world or the admin is an uneducated moron.

If your db destructs due to an undetected RAM problem (undetected
because your computer has not even parity error detection) and the data
in that db is "life or death" critical to your business, it might well
be the end of the world as far as your business is concerned.

Jim

Scary if the person has a DBA background. But there are lots of people,
like myself, who find themselves doing this sort of work and do not have the
formal training (small business, small db, but still an important part of
the business IS). Now, I know my limits and I would not take on the task of
administering a high-volume, mission-critical DB. Nonetheless, there are
many tips, tricks, and gotcha's that I have yet to learn.

I welcome an FAQ on this subject and genuinely appreciate the efforts of
those who create it. It will make me a better "wanna-be"!

-----Original Message-----
From: Jim Seymour [mailto:jseymour@linxnet.com]
Sent: July 6, 2004 6:32 AM
To: pgsql-general@postgresql.org
Subject: Re: Do we need more emphasis on backup?

Bruce Momjian <pgman@candle.pha.pa.us> wrote:

pgman wrote:

Jim Seymour wrote:

[snip]

My position is that your data is only as reliable as your hardware,
period.

[snip]

There is a basic misconception that all PC hardware is created equal ---
that hard drives, mother boards, and RAM are all the same because they
are all PC-compatible. Compatible != Similar Quality.

[snip]

Should I add an FAQ discussing hardware selection and the importance of
reliable hardware?

Scary to think that people who don't know enough about running a proper
server, and thus have to be *told* this, are admin'ing databases.

Start with: Gotta have ECC, or at least parity-checking, RAM.

Mention the (some kinds of?) IDE drives issue.

Mention that hardware RAID systems must have battery-backed write
cache. (I actually had a RAID vendor, whose products no longer had
batter-backed write cache, tell me "Just use a UPS.")

Jim

On Tue, 06 Jul 2004 11:25:07 -0600, Anony Mous <a.mous@shaw.ca> wrote:

Scary if the person has a DBA background. But there are lots of people,
like myself, who find themselves doing this sort of work and do not have the
formal training (small business, small db, but still an important part of
the business IS). Now, I know my limits and I would not take on the task of
administering a high-volume, mission-critical DB. Nonetheless, there are
many tips, tricks, and gotcha's that I have yet to learn.

I welcome an FAQ on this subject and genuinely appreciate the efforts of
those who create it. It will make me a better "wanna-be"!

One thing this strikes me as is an opportunity to help shape the
perception of DBAs and "real" databases versus "toy" databases.
Various products have gotten a lot of people out there running
databases with lots of mission criticle data in them, but they're
trusting a lot of slip-shod software and not putting into place any
proceedures to deal with backups and other things.

For example, I recently dealt with a customer who was using another
database, and had been running backups religiously for the past year.
The problem? The backups were of a live database, and it turned out
when they tried to restore, it was unable to get the tables back into
a position where they were usable without dropping a huge amount of
data from the tables, and then rebuilding indexes. A pain. All because
they'd not taken the right steps.

If the PostgreSQL community can help put together a "best practices"
guide, that will help all database people, but it will also help show
the PostgreSQL community as being more "serious." I'm more than happy
to contribute practices, and things, as well as help edit other
people's work, but I can't "lead" such an effort right now.

Chris
--
| Christopher Petrilli
| petrilli@gmail.com

Bruce Momjian wrote:

Should I add an FAQ discussing hardware selection and the importance
of reliable hardware?

I'm not a fan of having N "FAQs" for N topics. Just put it into the
documentation.

Peter Eisentraut wrote:

Bruce Momjian wrote:

Should I add an FAQ discussing hardware selection and the importance
of reliable hardware?

I'm not a fan of having N "FAQs" for N topics. Just put it into the
documentation.

No, I was thinking of adding an FAQ _item_ about it, not a new FAQ.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Christopher Petrilli <petrilli@gmail.com> wrote:

[snip]

If the PostgreSQL community can help put together a "best practices"
guide, that will help all database people, but it will also help show
the PostgreSQL community as being more "serious."

[snip]

*That* would be most welcome!

Jim

I'm not a fan of having N "FAQs" for N topics. Just put it into the
documentation.

No, I was thinking of adding an FAQ _item_ about it, not a new FAQ.

I think it would actually make a nice subchapter of the official
documentation. Although not strictly PostgreSQL specific but a "best
practices" section under Administration and with a reference from
Introduction would be good.

Kaarel

New FAQ added:

<H4><A name="3.11">3.11</A>) What computer hardware should I use?</H4>

<P>Because PC hardware is mostly compatible, people tend to believe that
all PC hardware is of equal quality. It is not. ECC RAM, SCSI, and
quality motherboards are more reliable and have better performance than
less expensive hardware. PostgreSQL will run on almost any hardware,
but if you are building a server where reliability and performance are
concerns, it is wise to research your hardware options thoroughly. Our
email lists can be used to discuss hardware options and tradeoffs.</P>

Adjustments?

---------------------------------------------------------------------------

Jim Seymour wrote:

Bruce Momjian <pgman@candle.pha.pa.us> wrote:

pgman wrote:

Jim Seymour wrote:

[snip]

My position is that your data is only as reliable as your hardware,
period.

[snip]

There is a basic misconception that all PC hardware is created equal ---
that hard drives, mother boards, and RAM are all the same because they
are all PC-compatible. Compatible != Similar Quality.

[snip]

Should I add an FAQ discussing hardware selection and the importance of
reliable hardware?

Scary to think that people who don't know enough about running a proper
server, and thus have to be *told* this, are admin'ing databases.

Start with: Gotta have ECC, or at least parity-checking, RAM.

Mention the (some kinds of?) IDE drives issue.

Mention that hardware RAID systems must have battery-backed write
cache. (I actually had a RAID vendor, whose products no longer had
batter-backed write cache, tell me "Just use a UPS.")

Jim

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> wrote:

New FAQ added:

<H4><A name="3.11">3.11</A>) What computer hardware should I use?</H4>

<P>Because PC hardware is mostly compatible, people tend to believe that
all PC hardware is of equal quality. It is not. ECC RAM, SCSI, and
quality motherboards are more reliable and have better performance than
less expensive hardware. PostgreSQL will run on almost any hardware,
but if you are building a server where reliability and performance are
concerns, it is wise to research your hardware options thoroughly. Our
email lists can be used to discuss hardware options and tradeoffs.</P>

Adjustments?

Looks good to me. Succinct and non-judgemental. Well done!

Oh, if you're accepting punctuation nits ;), in most cases, the comma
should come after "but," not before it. So your sentence should read
"PostgreSQL will run on almost any hardware but, if you are..."

If you wanted to mention other issues to consider, you might include:

---------------------------------------------------------------------------

Jim Seymour wrote:

[snip]

Mention that hardware RAID systems must have battery-backed write
cache. (I actually had a RAID vendor, whose products no longer had
batter-backed write cache, tell me "Just use a UPS.")

Also desirable are a quality UPS, with monitoring on the server for
graceful shutdown of the server on battery exhaustion, and redundant
power supplies.

Jim

OK, comma moved. I didn't add those other items because I didn't want
the list to be anything near a complete list.

---------------------------------------------------------------------------

Jim Seymour wrote:

Bruce Momjian <pgman@candle.pha.pa.us> wrote:

New FAQ added:

<H4><A name="3.11">3.11</A>) What computer hardware should I use?</H4>

<P>Because PC hardware is mostly compatible, people tend to believe that
all PC hardware is of equal quality. It is not. ECC RAM, SCSI, and
quality motherboards are more reliable and have better performance than
less expensive hardware. PostgreSQL will run on almost any hardware,
but if you are building a server where reliability and performance are
concerns, it is wise to research your hardware options thoroughly. Our
email lists can be used to discuss hardware options and tradeoffs.</P>

Adjustments?

Looks good to me. Succinct and non-judgemental. Well done!

Oh, if you're accepting punctuation nits ;), in most cases, the comma
should come after "but," not before it. So your sentence should read
"PostgreSQL will run on almost any hardware but, if you are..."

If you wanted to mention other issues to consider, you might include:

---------------------------------------------------------------------------

Jim Seymour wrote:

[snip]

Mention that hardware RAID systems must have battery-backed write
cache. (I actually had a RAID vendor, whose products no longer had
batter-backed write cache, tell me "Just use a UPS.")

Also desirable are a quality UPS, with monitoring on the server for
graceful shutdown of the server on battery exhaustion, and redundant
power supplies.

Jim

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

jseymour@linxnet.com (Jim Seymour) writes:

Oh, if you're accepting punctuation nits ;), in most cases, the comma
should come after "but," not before it. So your sentence should read
"PostgreSQL will run on almost any hardware but, if you are..."

Wrong. :)

You are sentenced to go read Strunk and White again.

-Doug

Doug McNaught wrote:

jseymour@linxnet.com (Jim Seymour) writes:

Oh, if you're accepting punctuation nits ;), in most cases, the comma
should come after "but," not before it. So your sentence should read
"PostgreSQL will run on almost any hardware but, if you are..."

Wrong. :)

You are sentenced to go read Strunk and White again.

Text adjusted:

<P>Because PC hardware is mostly compatible, people tend to believe that
all PC hardware is of equal quality. It is not. ECC RAM, SCSI, and
quality motherboards are more reliable and have better performance than
less expensive hardware. PostgreSQL will run on almost any hardware,
but if you are building a server where you are concerned about reliability
and performance it is wise to research your hardware options thoroughly. Our
email lists can be used to discuss hardware options and tradeoffs.</P>

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

On Sun, 2004-07-11 at 19:37, Bruce Momjian wrote:

Doug McNaught wrote:

jseymour@linxnet.com (Jim Seymour) writes:

Oh, if you're accepting punctuation nits ;), in most cases, the comma
should come after "but," not before it. So your sentence should read
"PostgreSQL will run on almost any hardware but, if you are..."

Wrong. :)

You are sentenced to go read Strunk and White again.

Text adjusted:

<P>Because PC hardware is mostly compatible, people tend to believe that
all PC hardware is of equal quality. It is not. ECC RAM, SCSI, and
quality motherboards are more reliable and have better performance than
less expensive hardware. PostgreSQL will run on almost any hardware,
but if you are building a server where you are concerned about reliability
and performance it is wise to research your hardware options thoroughly. Our
email lists can be used to discuss hardware options and tradeoffs.</P>

Further, a system should be burned in and have all the components
thoroughly tested. Brand new computers can, and often do, have broken
or misbehaving hardware. While relatively simple tasks like installing
an operating system or browsing the web only use a small portion of a
computer's resources, running a database relies on all of it working
properly. Basically, it all boils down to to this, if you're not sure
of your hardware, you shouldn't be building a database server with it,
and you can't be sure of your hardware without testing it.

... PostgreSQL will run on almost any hardware,
but if you are building a server where you are concerned about reliability
and performance it is wise to research your hardware options thoroughly. ...

That's okay, of course. But I'll suggest the following as food for
thought

PostgreSQL will run on almost any hardware, but,
if you are building a server where you are concerned
about reliability and performance, it is wise to
research your hardware options thoroughly.

or

PostgreSQL will run on almost any hardware, but,
especially where reliability and performance are
required, it is wise to research your
hardware options thoroughly.

(where/when)

--
Joel <rees@ddcom.co.jp>

Doug McNaught <doug@mcnaught.org> wrote:

jseymour@linxnet.com (Jim Seymour) writes:

Oh, if you're accepting punctuation nits ;), in most cases, the comma
should come after "but," not before it. So your sentence should read
"PostgreSQL will run on almost any hardware but, if you are..."

Wrong. :)

You are sentenced to go read Strunk and White again.

I stand corrected. I always used to put the comma before "but," but
changed after reading somewhere, or thinking I had, that was incorrect
for certain sentence structures.

Neither of my style guides supports what I said earlier, so I cannot
imagine whence I got that idea.

Jim

Joel wrote:

... PostgreSQL will run on almost any hardware,
but if you are building a server where you are concerned about reliability
and performance it is wise to research your hardware options thoroughly. ...

That's okay, of course. But I'll suggest the following as food for
thought

PostgreSQL will run on almost any hardware, but,
if you are building a server where you are concerned
about reliability and performance, it is wise to
research your hardware options thoroughly.

or

PostgreSQL will run on almost any hardware, but,
especially where reliability and performance are
required, it is wise to research your
hardware options thoroughly.

New text:

<P>Because PC hardware is mostly compatible, people tend to believe that
all PC hardware is of equal quality. It is not. ECC RAM, SCSI, and
quality motherboards are more reliable and have better performance than
less expensive hardware. PostgreSQL will run on almost any hardware,
but if reliability and performance are important it is wise to
research your hardware options thoroughly. Our email lists can be used
to discuss hardware options and tradeoffs.</P>

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073