Moving pg_autovacuum from contrib to src/bin

On Sat, 2004-05-29 at 01:31, Matthew T. O'Connor wrote:

Having no response from my email to hackers yesterday, I will assume
(hope) that no news is good news and proceed onwards.

As outlined in my email, step 1 is to move pg_autovacuum from contrib to
src/bin. Attached is a patch that modifies src/bin/Makefile and the
pg_autovacuum Makefile which will be in src/bin/pg_autovacuum/Makefile
once the pg_autovacuum directory is moved.

If there is anything else I can to to facilitate this move, let me know
and I'll do it.

Actually attaching the patch might be something else I could do :-)

Anyway, here is the patch for real this time....

Matthew

"Matthew T. O'Connor" <matthew@zeut.net> writes:

As outlined in my email, step 1 is to move pg_autovacuum from contrib to
src/bin. Attached is a patch that modifies src/bin/Makefile and the
pg_autovacuum Makefile which will be in src/bin/pg_autovacuum/Makefile
once the pg_autovacuum directory is moved.

Um, you didn't actually attach the patch ... but in any case this is not
the direction I would have expected. An integrated autovacuum daemon
would certainly not live in src/bin. It would be a module within the
postgres executable, most likely living within src/backend/postmaster
since that seems to be where we're putting integrated postmaster children.

(To clarify: that's where the pgstat daemon is now, and that's where I'm
about to put the bgwriter daemon, so if anyone thinks this is clearly
the wrong place then now would be a good time to mention it...)

regards, tom lane

On Sat, 2004-05-29 at 02:15, Tom Lane wrote:

"Matthew T. O'Connor" <matthew@zeut.net> writes:

As outlined in my email, step 1 is to move pg_autovacuum from contrib to
src/bin. Attached is a patch that modifies src/bin/Makefile and the
pg_autovacuum Makefile which will be in src/bin/pg_autovacuum/Makefile
once the pg_autovacuum directory is moved.

Um, you didn't actually attach the patch ... but in any case this is not
the direction I would have expected. An integrated autovacuum daemon
would certainly not live in src/bin. It would be a module within the
postgres executable, most likely living within src/backend/postmaster
since that seems to be where we're putting integrated postmaster children.

(To clarify: that's where the pgstat daemon is now, and that's where I'm
about to put the bgwriter daemon, so if anyone thinks this is clearly
the wrong place then now would be a good time to mention it...)

The only reason I put it in src/bin is because I thought people wanted
pg_autovacuum to still be a runable stand alone app. But, I guess there
is nothing preventing it from being a runable stand alone app located in
src/backend/postmaster.

Ok, I will move it (again) and send another patch.

Thanks for the feedback.

Matthew

Matthew T. O'Connor wrote:

The only reason I put it in src/bin is because I thought people
wanted pg_autovacuum to still be a runable stand alone app.

I see no reason for that. If it's integrated as a postmaster child,
then it is run in a daemon-like mode automatically.

But, I
guess there is nothing preventing it from being a runable stand alone
app located in src/backend/postmaster.

Yes, there are a lot of things. You didn't understand what Tom was
saying. We were expecting the autovacuum daemon to be integrated as a
postmaster child, not a separate program.

On Sat, 2004-05-29 at 02:45, Peter Eisentraut wrote:

Matthew T. O'Connor wrote:

The only reason I put it in src/bin is because I thought people
wanted pg_autovacuum to still be a runable stand alone app.

I see no reason for that. If it's integrated as a postmaster child,
then it is run in a daemon-like mode automatically.

I don't really see a strong reason to keep it runable as a standalone
except that I thought that was what was decided in previous discussion
on hackers.

Unfortunately, I can't find the thread anywhere on
archives.postgresql.org, when I search for "pg_autovacuum" it doesn't
return any hits more recent than Jan 2004, and I know there have been a
few pg_autovacuum related discussions since then so it seems like
something is broken.

But, I
guess there is nothing preventing it from being a runable stand alone
app located in src/backend/postmaster.

Yes, there are a lot of things. You didn't understand what Tom was
saying. We were expecting the autovacuum daemon to be integrated as a
postmaster child, not a separate program.

I did understand Tom, but based on the hacker discussion I think the
"postmaster integration" will consist of the postmaster launching and
killing the pg_autovacuum standalone executable as required. In that
sense, I don't think it matters if pg_autovacuum is located in src/bin
or src/backend/postmaster.

Is this form of integration something people would object to? If so, I
can fold it into the backend such that pg_autovacuum can no longer be
run from the command line.

Comments anyone?

Matthew T. O'Connor wrote:

I did understand Tom, but based on the hacker discussion I think the
"postmaster integration" will consist of the postmaster launching and
killing the pg_autovacuum standalone executable as required. In that
sense, I don't think it matters if pg_autovacuum is located in
src/bin or src/backend/postmaster.

It is supposed to be linked into the postmaster and forked from there.

-----Original Message-----
From: Peter Eisentraut [mailto:peter_e@gmx.net]
Sent: 29 May 2004 09:44
To: Matthew T. O'Connor
Cc: Tom Lane; pgsql-patches
Subject: Re: [PATCHES] Moving pg_autovacuum from contrib to src/bin

It is supposed to be linked into the postmaster and forked from there.

That was my understanding - if it remains a standalone program it may as
well remain in /contrib (and have my win32 service patch applied - which
oddly I'd rather didn't have to happen :-) )

Regards, Dave

Peter Eisentraut <peter_e@gmx.net> writes:

Matthew T. O'Connor wrote:

I did understand Tom, but based on the hacker discussion I think the
"postmaster integration" will consist of the postmaster launching and
killing the pg_autovacuum standalone executable as required. In that
sense, I don't think it matters if pg_autovacuum is located in
src/bin or src/backend/postmaster.

It is supposed to be linked into the postmaster and forked from there.

In the current state of pg_autovacuum it wouldn't matter a lot, but
I am assuming that we will soon migrate it to depend on being part of
the postmaster environment. For instance, it ought to be configured
from GUC, which will mean it has to receive SIGHUP from the postmaster.
In an only slightly longer timeframe, it will probably want access to
shared memory so it can look at stats maintained in the FSM. These
attributes would make it quite inappropriate for autovacuum to live in
src/bin.

BTW, Matthew, I am currently working on promoting the bgwriter into a
more full-fledged postmaster child. If you can wait a day or so you
should have a decent model to work from. I'll try to commit as soon
as a working skeleton is in place.

regards, tom lane

On Sat, 2004-05-29 at 11:04, Tom Lane wrote:

Peter Eisentraut <peter_e@gmx.net> writes:

It is supposed to be linked into the postmaster and forked from there.

In the current state of pg_autovacuum it wouldn't matter a lot, but
I am assuming that we will soon migrate it to depend on being part of
the postmaster environment. For instance, it ought to be configured
from GUC, which will mean it has to receive SIGHUP from the postmaster.
In an only slightly longer timeframe, it will probably want access to
shared memory so it can look at stats maintained in the FSM. These
attributes would make it quite inappropriate for autovacuum to live in
src/bin.

Ok.

BTW, Matthew, I am currently working on promoting the bgwriter into a
more full-fledged postmaster child. If you can wait a day or so you
should have a decent model to work from. I'll try to commit as soon
as a working skeleton is in place.

I can wait, but I am really trying not to miss the feature freeze which
AFAIK, is still happening in a few days. Is that changing? Will I have
time if I wait a few days? Especially given that my backend hacking
skill leave much to be desired.

My main priority is that I want pg_autovacuum to move ahead and I don't
want to miss 7.5.

Thanks,

Matthew

On Sat, 29 May 2004, Matthew T. O'Connor wrote:

On Sat, 2004-05-29 at 11:04, Tom Lane wrote:

Peter Eisentraut <peter_e@gmx.net> writes:

It is supposed to be linked into the postmaster and forked from there.

In the current state of pg_autovacuum it wouldn't matter a lot, but
I am assuming that we will soon migrate it to depend on being part of
the postmaster environment. For instance, it ought to be configured
from GUC, which will mean it has to receive SIGHUP from the postmaster.
In an only slightly longer timeframe, it will probably want access to
shared memory so it can look at stats maintained in the FSM. These
attributes would make it quite inappropriate for autovacuum to live in
src/bin.

Ok.

BTW, Matthew, I am currently working on promoting the bgwriter into a
more full-fledged postmaster child. If you can wait a day or so you
should have a decent model to work from. I'll try to commit as soon
as a working skeleton is in place.

I can wait, but I am really trying not to miss the feature freeze which
AFAIK, is still happening in a few days. Is that changing? Will I have
time if I wait a few days? Especially given that my backend hacking
skill leave much to be desired.

We're discussing it in core right now, but Tom's feel is that we can have
PITR if we wait the extra month, which will also give a bit more time to
hammer out any bugs in Win32 ... so figure on having the extra 4 weeks to
work with ...

----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664

On Sat, 2004-05-29 at 15:02, Marc G. Fournier wrote:

On Sat, 29 May 2004, Matthew T. O'Connor wrote:

I can wait, but I am really trying not to miss the feature freeze which
AFAIK, is still happening in a few days. Is that changing? Will I have
time if I wait a few days? Especially given that my backend hacking
skill leave much to be desired.

We're discussing it in core right now, but Tom's feel is that we can have
PITR if we wait the extra month, which will also give a bit more time to
hammer out any bugs in Win32 ... so figure on having the extra 4 weeks to
work with ...

That will help me a lot with pg_autovacuum integration also. So, I will
wait until Tom finishes work on the bgwriter and use that as a template
for autovacuum integration.

Please let me know when the official decision is made so that I can plan
appropriately.

Thanks much,

Matthew