(one more time) Patches with vacuum fixes available.

Started by Alfred Perlsteinabout 25 years ago7 messages
#1Alfred Perlstein
bright@wintelcom.net

I know you guys are pretty busy with the upcoming release but I
was hoping for more interest in this work.

With this (which needs forward porting) we're able to cut
vacuum time down from ~10minutes to under 30 seconds.

The code is a nop unless you compile with special options(MMNB)
specify the special vacuum flag (VLAZY) and doesn't look like it
messes with anything otherwise.

I was hoping to see it go into 7.0.x because of the non-intrusiveness
of it and also because Vadim did it so he should understand it so
that it won't cause any problems (and on the slight chance that it
does, he should be able to fix it).

Basically Vadim left it up to me to campaign for acceptance of this
work and he said he wouldn't have a problem bringing it in as long
as it was ok with the rest of the development team.

So can we get a go-ahead on this? :)

thanks,
-Alfred

----- Forwarded message from Alfred Perlstein <bright@wintelcom.net> -----

From: Alfred Perlstein <bright@wintelcom.net>
To: pgsql-hackers@postgresql.org
Subject: [HACKERS] Patches with vacuum fixes available for 7.0.x
Date: Thu, 7 Dec 2000 14:57:32 -0800
Message-ID: <20001207145732.X16205@fw.wintelcom.net>
User-Agent: Mutt/1.2.5i
Sender: pgsql-hackers-owner@postgresql.org

We recently had a very satisfactory contract completed by
Vadim.

Basically Vadim has been able to reduce the amount of time
taken by a vacuum from 10-15 minutes down to under 10 seconds.

We've been running with these patches under heavy load for
about a week now without any problems except one:
don't 'lazy' (new option for vacuum) a table which has just
had an index created on it, or at least don't expect it to
take any less time than a normal vacuum would.

There's three patchsets and they are available at:

http://people.freebsd.org/~alfred/vacfix/

complete diff:
http://people.freebsd.org/~alfred/vacfix/v.diff

only lazy vacuum option to speed up index vacuums:
http://people.freebsd.org/~alfred/vacfix/vlazy.tgz

only lazy vacuum option to only scan from start of modified
data:
http://people.freebsd.org/~alfred/vacfix/mnmb.tgz

Although the patches are for 7.0.x I'm hoping that they
can be forward ported (if Vadim hasn't done it already)
to 7.1.

enjoy!

--
-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
"I have the heart of a child; I keep it in a jar on my desk."

----- End forwarded message -----

--
-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
"I have the heart of a child; I keep it in a jar on my desk."

#2Bruce Momjian
pgman@candle.pha.pa.us
In reply to: Alfred Perlstein (#1)
Re: (one more time) Patches with vacuum fixes available.

Sounds good to me. [Of course, I never met a patch I didn't like, or so
they say.]

I know you guys are pretty busy with the upcoming release but I
was hoping for more interest in this work.

With this (which needs forward porting) we're able to cut
vacuum time down from ~10minutes to under 30 seconds.

The code is a nop unless you compile with special options(MMNB)
specify the special vacuum flag (VLAZY) and doesn't look like it
messes with anything otherwise.

I was hoping to see it go into 7.0.x because of the non-intrusiveness
of it and also because Vadim did it so he should understand it so
that it won't cause any problems (and on the slight chance that it
does, he should be able to fix it).

Basically Vadim left it up to me to campaign for acceptance of this
work and he said he wouldn't have a problem bringing it in as long
as it was ok with the rest of the development team.

So can we get a go-ahead on this? :)

thanks,
-Alfred

----- Forwarded message from Alfred Perlstein <bright@wintelcom.net> -----

From: Alfred Perlstein <bright@wintelcom.net>
To: pgsql-hackers@postgresql.org
Subject: [HACKERS] Patches with vacuum fixes available for 7.0.x
Date: Thu, 7 Dec 2000 14:57:32 -0800
Message-ID: <20001207145732.X16205@fw.wintelcom.net>
User-Agent: Mutt/1.2.5i
Sender: pgsql-hackers-owner@postgresql.org

We recently had a very satisfactory contract completed by
Vadim.

Basically Vadim has been able to reduce the amount of time
taken by a vacuum from 10-15 minutes down to under 10 seconds.

We've been running with these patches under heavy load for
about a week now without any problems except one:
don't 'lazy' (new option for vacuum) a table which has just
had an index created on it, or at least don't expect it to
take any less time than a normal vacuum would.

There's three patchsets and they are available at:

http://people.freebsd.org/~alfred/vacfix/

complete diff:
http://people.freebsd.org/~alfred/vacfix/v.diff

only lazy vacuum option to speed up index vacuums:
http://people.freebsd.org/~alfred/vacfix/vlazy.tgz

only lazy vacuum option to only scan from start of modified
data:
http://people.freebsd.org/~alfred/vacfix/mnmb.tgz

Although the patches are for 7.0.x I'm hoping that they
can be forward ported (if Vadim hasn't done it already)
to 7.1.

enjoy!

--
-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
"I have the heart of a child; I keep it in a jar on my desk."

----- End forwarded message -----

--
-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
"I have the heart of a child; I keep it in a jar on my desk."

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
#3Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alfred Perlstein (#1)
Re: (one more time) Patches with vacuum fixes available.

Alfred Perlstein <bright@wintelcom.net> writes:

Basically Vadim left it up to me to campaign for acceptance of this
work and he said he wouldn't have a problem bringing it in as long
as it was ok with the rest of the development team.
So can we get a go-ahead on this? :)

If Vadim isn't sufficiently confident of it to commit it on his own
authority, I'm inclined to leave it out of 7.1. My concern is mostly
schedule. We are well into beta cycle now and this seems like way too
critical (not to say high-risk) a feature to be adding after start of
beta.

regards, tom lane

#4Bruce Momjian
pgman@candle.pha.pa.us
In reply to: Tom Lane (#3)
Re: (one more time) Patches with vacuum fixes available.

Alfred Perlstein <bright@wintelcom.net> writes:

Basically Vadim left it up to me to campaign for acceptance of this
work and he said he wouldn't have a problem bringing it in as long
as it was ok with the rest of the development team.
So can we get a go-ahead on this? :)

If Vadim isn't sufficiently confident of it to commit it on his own
authority, I'm inclined to leave it out of 7.1. My concern is mostly
schedule. We are well into beta cycle now and this seems like way too
critical (not to say high-risk) a feature to be adding after start of
beta.

I was wondering if Vadim was hesitant because he had done this under
contract. Vadim, are you concerned about reliability or are there other
issues?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
#5The Hermit Hacker
scrappy@hub.org
In reply to: Tom Lane (#3)
Re: (one more time) Patches with vacuum fixes available.

On Mon, 11 Dec 2000, Tom Lane wrote:

Alfred Perlstein <bright@wintelcom.net> writes:

Basically Vadim left it up to me to campaign for acceptance of this
work and he said he wouldn't have a problem bringing it in as long
as it was ok with the rest of the development team.
So can we get a go-ahead on this? :)

If Vadim isn't sufficiently confident of it to commit it on his own
authority, I'm inclined to leave it out of 7.1. My concern is mostly
schedule. We are well into beta cycle now and this seems like way too
critical (not to say high-risk) a feature to be adding after start of
beta.

As this is post-beta release, and this isn't a bug fix, I concur with this
... its something we can easily put up on the web site as a patch to v7.1,
but would prefer this not going mainstream until *after* release ...

#6The Hermit Hacker
scrappy@hub.org
In reply to: Bruce Momjian (#4)
Re: (one more time) Patches with vacuum fixes available.

On Mon, 11 Dec 2000, Bruce Momjian wrote:

Alfred Perlstein <bright@wintelcom.net> writes:

Basically Vadim left it up to me to campaign for acceptance of this
work and he said he wouldn't have a problem bringing it in as long
as it was ok with the rest of the development team.
So can we get a go-ahead on this? :)

If Vadim isn't sufficiently confident of it to commit it on his own
authority, I'm inclined to leave it out of 7.1. My concern is mostly
schedule. We are well into beta cycle now and this seems like way too
critical (not to say high-risk) a feature to be adding after start of
beta.

I was wondering if Vadim was hesitant because he had done this under
contract. Vadim, are you concerned about reliability or are there other
issues?

Irrelevant .. we are post-beta release, and this doesn't fix a bug, so it
doesn't go in ...

#7Alfred Perlstein
bright@wintelcom.net
In reply to: The Hermit Hacker (#6)
Re: (one more time) Patches with vacuum fixes available.

* The Hermit Hacker <scrappy@hub.org> [001211 14:27] wrote:

On Mon, 11 Dec 2000, Bruce Momjian wrote:

Alfred Perlstein <bright@wintelcom.net> writes:

Basically Vadim left it up to me to campaign for acceptance of this
work and he said he wouldn't have a problem bringing it in as long
as it was ok with the rest of the development team.
So can we get a go-ahead on this? :)

If Vadim isn't sufficiently confident of it to commit it on his own
authority, I'm inclined to leave it out of 7.1. My concern is mostly
schedule. We are well into beta cycle now and this seems like way too
critical (not to say high-risk) a feature to be adding after start of
beta.

I was wondering if Vadim was hesitant because he had done this under
contract. Vadim, are you concerned about reliability or are there other
issues?

Irrelevant .. we are post-beta release, and this doesn't fix a bug, so it
doesn't go in ...

I'm hoping this just means it won't be investigated until the release
is made?

--
-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
"I have the heart of a child; I keep it in a jar on my desk."