Growing a live database

Started by Rick Genteralmost 15 years ago3 messagesgeneral
Jump to latest
#1Rick Genter
rick.genter@gmail.com

I have a PostgreSQL 8.4.4 database that is running 24/7. The drive that the
database is on is becoming full and I need to expand it. We are currently
doing log-shipping of the WAL files to a slave system to run in a hot
standby mode. I have two servers: S1 (currently running as master) and S2
(currently running as slave)

My current plan is to do the following:

- fail S1 over to S2 by doing pg_ctl stop on S1, then ending the recovery
process on S2 and let it come up as the master
- add a new larger drive to S1
- swap roles; backup S2 and restore it on S1, then start log shipping from
S2 to S1
- let the S1 "catch up" on the log files from S2
- fail S2 back over to S1 by doing pg_ctl stop on S2, then ending the
recovery process on S1 and let it come up as the master
- add a new larger drive to S2
- backup S1 and restore it on S2, then start log shipping from S1 to S2

I believe that this accomplishes the goal (increasing available drive space)
with a minimum amount of down time. Am I thinking correctly, or have I
missed something?

--
Rick Genter
rick.genter@gmail.com

#2Vick Khera
vivek@khera.org
In reply to: Rick Genter (#1)
Re: Growing a live database

On Fri, May 6, 2011 at 9:27 AM, Rick Genter <rick.genter@gmail.com> wrote:

I believe that this accomplishes the goal (increasing available drive space)
with a minimum amount of down time. Am I thinking correctly, or have I
missed something?

That is the general procedure you would follow to perform maintenance
on a busy DB server. Just practice it on a spare set of machines (or
virtual machines, even) and write down *every* step and command you
need to do so you are not thinking of it on the fly.

#3Scott Marlowe
scott.marlowe@gmail.com
In reply to: Rick Genter (#1)
Re: Growing a live database

On Fri, May 6, 2011 at 7:27 AM, Rick Genter <rick.genter@gmail.com> wrote:

I have a PostgreSQL 8.4.4 database that is running 24/7. The drive that the
database is on is becoming full and I need to expand it. We are currently
doing log-shipping of the WAL files to a slave system to run in a hot
standby mode. I have two servers: S1 (currently running as master) and S2
(currently running as slave)
My current plan is to do the following:
- fail S1 over to S2 by doing pg_ctl stop on S1, then ending the recovery
process on S2 and let it come up as the master
- add a new larger drive to S1
- swap roles; backup S2 and restore it on S1, then start log shipping from
S2 to S1
- let the S1 "catch up" on the log files from S2
- fail S2 back over to S1 by doing pg_ctl stop on S2, then ending the
recovery process on S1 and let it come up as the master
- add a new larger drive to S2
- backup S1 and restore it on S2, then start log shipping from S1 to S2
I believe that this accomplishes the goal (increasing available drive space)
with a minimum amount of down time. Am I thinking correctly, or have I
missed something?

That should work, but you can replace one failover with rebuilding s2
first, failover to it, rebuid s1, and leaving s2 as the master. That
assumes that they're identical machines otherwise.