pg_migrator

On Mon, Apr 26, 2010 at 9:26 PM, Bruce Momjian <bruce@momjian.us> wrote:

There was talk of including pg_migrator in Postgres 9.0 in /contrib.  Do
we still want to do that?

I think you articulated some pretty good reasons previously for
keeping it separate and, at any rate, I'm not eager to do it at the
11th hour without due consideration and adequate engineering time. So
I vote for holding off for this release and possibly revisiting at
some point down the road.

...Robert

Robert Haas wrote:

On Mon, Apr 26, 2010 at 9:26 PM, Bruce Momjian <bruce@momjian.us> wrote:

There was talk of including pg_migrator in Postgres 9.0 in /contrib. ?Do
we still want to do that?

I think you articulated some pretty good reasons previously for
keeping it separate and, at any rate, I'm not eager to do it at the
11th hour without due consideration and adequate engineering time. So
I vote for holding off for this release and possibly revisiting at
some point down the road.

You might also remember I was outvoted. It will not be hard to put it
in /contrib as that is already a valid build option for pg_migrator.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

On Mon, Apr 26, 2010 at 9:46 PM, Bruce Momjian <bruce@momjian.us> wrote:

Robert Haas wrote:

On Mon, Apr 26, 2010 at 9:26 PM, Bruce Momjian <bruce@momjian.us> wrote:

There was talk of including pg_migrator in Postgres 9.0 in /contrib. ?Do
we still want to do that?

I think you articulated some pretty good reasons previously for
keeping it separate and, at any rate, I'm not eager to do it at the
11th hour without due consideration and adequate engineering time.  So
I vote for holding off for this release and possibly revisiting at
some point down the road.

You might also remember I was outvoted.  It will not be hard to put it
in /contrib as that is already a valid build option for pg_migrator.

[shrug...]

If that's the consensus I'll go along with it, but I'm not excited
about adding more things to our to-do list at this point, even
apparently simple ones.

...Robert

Robert Haas <robertmhaas@gmail.com> writes:

On Mon, Apr 26, 2010 at 9:26 PM, Bruce Momjian <bruce@momjian.us> wrote:

There was talk of including pg_migrator in Postgres 9.0 in /contrib. �Do
we still want to do that?

I think you articulated some pretty good reasons previously for
keeping it separate and, at any rate, I'm not eager to do it at the
11th hour without due consideration and adequate engineering time.

I concur; it's about a month too late to propose this.

regards, tom lane

Tom Lane wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Mon, Apr 26, 2010 at 9:26 PM, Bruce Momjian <bruce@momjian.us> wrote:

There was talk of including pg_migrator in Postgres 9.0 in /contrib. ���Do
we still want to do that?

I think you articulated some pretty good reasons previously for
keeping it separate and, at any rate, I'm not eager to do it at the
11th hour without due consideration and adequate engineering time.

I concur; it's about a month too late to propose this.

I am confused why it is late. We add to /contrib even during beta, and
I didn't bring it up earlier because I didn't want to be pushing my own
software. Was someone else supposed to suggest it a month ago?

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

Bruce Momjian wrote:

I concur; it's about a month too late to propose this.

I am confused why it is late. We add to /contrib even during beta, and
I didn't bring it up earlier because I didn't want to be pushing my own
software. Was someone else supposed to suggest it a month ago?

Bruce,

you're not usually such a shrinking violet. If you don't push your
project it's less likely others will, IMNSHO.

cheers

andrew

Andrew Dunstan wrote:

Bruce Momjian wrote:

I concur; it's about a month too late to propose this.

I am confused why it is late. We add to /contrib even during beta, and
I didn't bring it up earlier because I didn't want to be pushing my own
software. Was someone else supposed to suggest it a month ago?

Bruce,

you're not usually such a shrinking violet. If you don't push your
project it's less likely others will, IMNSHO.

Well, I am sensitive to be pushing my external project into /contrib when
I am someone who puts other stuff into contrib, and I have been working
on pg_migrator for only one year.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

Tom Lane wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Mon, Apr 26, 2010 at 9:26 PM, Bruce Momjian <bruce@momjian.us> wrote:

There was talk of including pg_migrator in Postgres 9.0 in /contrib. ���Do
we still want to do that?

I think you articulated some pretty good reasons previously for
keeping it separate and, at any rate, I'm not eager to do it at the
11th hour without due consideration and adequate engineering time.

I concur; it's about a month too late to propose this.

I talked to a few people personally about this, and it seems there was a
misunderstanding. I was not asking if pg_migrator should be in 9.0
beta1. I was asking if we should think about putting it into a later
9.0 beta, like 9.0 beta3. It would be another major 9.0 feature.

Because pg_migrator is an external project, it seemed best to do it
after beta1, while we are just waiting for bug reports, and not during
our main push to beta1.

FYI, here was the last discussion about having pg_migrator in /contrib
in December 2009:

http://archives.postgresql.org/pgsql-hackers/2009-12/msg01787.php

and most of the limitations of pg_migrator are gone when migrating to
9.0, even from Postgres 8.3. This could help beta testers move their
data to 9.0 as well.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

Bruce Momjian wrote:

and most of the limitations of pg_migrator are gone when migrating to
9.0, even from Postgres 8.3. This could help beta testers move their
data to 9.0 as well.

Wouldn't this help even for beta1?

Cheers

Mark

Mark Kirkwood <mark.kirkwood@catalyst.net.nz> writes:

Bruce Momjian wrote:

and most of the limitations of pg_migrator are gone when migrating to
9.0, even from Postgres 8.3. This could help beta testers move their
data to 9.0 as well.

Wouldn't this help even for beta1?

It's too late for beta1. It probably should have been proposed when
there was still time ... but it wasn't.

I'm not necessarily averse to shoving it in as of beta2 or beta3 or
so; we've always been laxer about contrib than the core server.

regards, tom lane

Mark Kirkwood wrote:

Bruce Momjian wrote:

and most of the limitations of pg_migrator are gone when migrating to
9.0, even from Postgres 8.3. This could help beta testers move their
data to 9.0 as well.

Wouldn't this help even for beta1?

It would, but there is so much work going into getting other features
done that there just isn't enough time. We don't want pg_migrator
delaying beta1.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

On tor, 2010-04-29 at 17:34 -0400, Bruce Momjian wrote:

I talked to a few people personally about this, and it seems there was a
misunderstanding. I was not asking if pg_migrator should be in 9.0
beta1. I was asking if we should think about putting it into a later
9.0 beta, like 9.0 beta3. It would be another major 9.0 feature.

If it's supposed to be a major feature, then it should be in src/bin,
and fully integrated in the install, the documentation, etc.

If you want to put it in contrib because the standards are more lax
there, then by definition it's not really a major feature, it's just a
random feature that was snuck in. You can't have it both ways.

My personal feeling is that pg_migrator should be fully integrated, but
it's too late for that, obviously. Let's do it for 9.1.

I also think that the standards for contrib should not be so lax that a
completely new module can be added after beta. (This is mostly informed
by the feeling that contrib should go away entirely.)

Peter Eisentraut <peter_e@gmx.net> writes:

My personal feeling is that pg_migrator should be fully integrated, but
it's too late for that, obviously. Let's do it for 9.1.

+1

I also think that the standards for contrib should not be so lax that a
completely new module can be added after beta. (This is mostly informed
by the feeling that contrib should go away entirely.)

+1

For the record, the contrib replacement would look like proper Extension
handling in dump&restore, PGXS support for windows, and PGAN for source
level archive distribution. We'd still rely on distributions support for
binaries.

Those are the technical layers we need, then we'd have a PGAN entry for
replacing contrib, and a host of other ones. The contrib Archive Network
would contain -core reviewed (and maintained?) extensions, the other
ones are on their own. Maybe the main other one would be (could be?) a
new component of pgfoundry.

Regards,
--
dim

Dimitri Fontaine <dfontaine@hi-media.com> writes:

Peter Eisentraut <peter_e@gmx.net> writes:

I also think that the standards for contrib should not be so lax that a
completely new module can be added after beta. (This is mostly informed
by the feeling that contrib should go away entirely.)

+1

For the record, the contrib replacement would look like proper Extension
handling in dump&restore, PGXS support for windows, and PGAN for source
level archive distribution. We'd still rely on distributions support for
binaries.

Both of you are living in some fantasy land. The reason contrib is held
to a lower standard than core is that nobody is willing to put the same
level of effort into contrib. There are modules in there (most of them,
in fact) that haven't been touched for years, other than as part of
system-wide search-and-replace patches. Extension support is not going
to magically fix that and cause maintenance effort to appear from
nowhere.

In the end, the main useful function that contrib serves is to provide
examples of how to write Postgres extensions. Because of that, removing
it as Peter suggests doesn't seem like a good idea to me.

regards, tom lane

Tom Lane wrote:

Dimitri Fontaine <dfontaine@hi-media.com> writes:

Peter Eisentraut <peter_e@gmx.net> writes:

I also think that the standards for contrib should not be so lax that a
completely new module can be added after beta. (This is mostly informed
by the feeling that contrib should go away entirely.)

+1

For the record, the contrib replacement would look like proper Extension
handling in dump&restore, PGXS support for windows, and PGAN for source
level archive distribution. We'd still rely on distributions support for
binaries.

Both of you are living in some fantasy land. The reason contrib is held
to a lower standard than core is that nobody is willing to put the same
level of effort into contrib. There are modules in there (most of them,
in fact) that haven't been touched for years, other than as part of
system-wide search-and-replace patches. Extension support is not going
to magically fix that and cause maintenance effort to appear from
nowhere.

In the end, the main useful function that contrib serves is to provide
examples of how to write Postgres extensions. Because of that, removing
it as Peter suggests doesn't seem like a good idea to me.

Quite so. Getting a better extensions mechanism doesn't mean we should
abandon what we currently have, IMNSHO.

cheers

andrew

On fre, 2010-04-30 at 10:45 -0400, Tom Lane wrote:

In the end, the main useful function that contrib serves is to provide
examples of how to write Postgres extensions.

Maybe, but pg_migrator surely doesn't fit that. And neither does about
a third of the other contrib modules, IMO.

Because of that, removing
it as Peter suggests doesn't seem like a good idea to me.

contrib means many things to many people, and that's exactly the problem
in my mind: It doesn't mean anything in particular. If we were to
separate it into

- examples

- production-quality add-ons with small user base

- production-quality add-ons that everyone wants, but we keep them as
plugins because plugins are cool

- experimental code that we wanted to ship anyway

- (historically) differently licensed code

then these discussions would be much simpler.

Andrew Dunstan <andrew@dunslane.net> writes:

Quite so. Getting a better extensions mechanism doesn't mean we should
abandon what we currently have, IMNSHO.

Yeah, agreed. Exactly what I proposed. The only change is the
distribution mean. Either we keep things as they are now exactly, or we
use the new Archive Network facilities, in the spirit of being sure they
get exercised, requiring that commiters gets to use them and maybe use a
separate code repository for contribs. Or simply an adjusted Makefile.

Summary: the only proposed change is how to do it, not what we do.

Regards,
--
dim

Tom Lane wrote:

Dimitri Fontaine <dfontaine@hi-media.com> writes:

Peter Eisentraut <peter_e@gmx.net> writes:

I also think that the standards for contrib should not be so lax that a
completely new module can be added after beta. (This is mostly informed
by the feeling that contrib should go away entirely.)

+1

For the record, the contrib replacement would look like proper Extension
handling in dump&restore, PGXS support for windows, and PGAN for source
level archive distribution. We'd still rely on distributions support for
binaries.

Both of you are living in some fantasy land. The reason contrib is held
to a lower standard than core is that nobody is willing to put the same
level of effort into contrib. There are modules in there (most of them,
in fact) that haven't been touched for years, other than as part of
system-wide search-and-replace patches. Extension support is not going
to magically fix that and cause maintenance effort to appear from
nowhere.

In the end, the main useful function that contrib serves is to provide
examples of how to write Postgres extensions. Because of that, removing
it as Peter suggests doesn't seem like a good idea to me.

So what do people want to do with pg_migrator? I don't think calling
pg_migrator a major features requires it to be in /bin. We added full
text search to /contrib years ago and that was a major feature.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

On Sat, May 1, 2010 at 02:23, Bruce Momjian <bruce@momjian.us> wrote:

Tom Lane wrote:

Dimitri Fontaine <dfontaine@hi-media.com> writes:

Peter Eisentraut <peter_e@gmx.net> writes:

I also think that the standards for contrib should not be so lax that a
completely new module can be added after beta.  (This is mostly informed
by the feeling that contrib should go away entirely.)

+1

For the record, the contrib replacement would look like proper Extension
handling in dump&restore, PGXS support for windows, and PGAN for source
level archive distribution. We'd still rely on distributions support for
binaries.

Both of you are living in some fantasy land.  The reason contrib is held
to a lower standard than core is that nobody is willing to put the same
level of effort into contrib.  There are modules in there (most of them,
in fact) that haven't been touched for years, other than as part of
system-wide search-and-replace patches.  Extension support is not going
to magically fix that and cause maintenance effort to appear from
nowhere.

In the end, the main useful function that contrib serves is to provide
examples of how to write Postgres extensions.  Because of that, removing
it as Peter suggests doesn't seem like a good idea to me.

So what do people want to do with pg_migrator?  I don't think calling
pg_migrator a major features requires it to be in /bin.  We added full
text search to /contrib years ago and that was a major feature.

There is a reason people said that "8.3 comes with full text search" -
that's when it really became a major feature to the outside world.
Before that, it wasn't included in most comparisons. It was a PITA to
install (well, not install, but upgrading when you had it, etc). (once
you knew the insides, it was a major feature yes, but people didn't
know about that)

A lot of people are not willing to put stuff labeled "contrib" on
their production boxes.

And as Tom says, even we *ourselves* acknowledge that things in
/contrib are held to a lower standard. If we put that label on
pg_migrator, it doesn't exactly signal people that this is something
they should use on their critical database.

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/