pg_upgrade versus MSVC build scripts

Excerpts from Tom Lane's message of mié may 12 14:07:13 -0400 2010:

We could try to make this a supported build arrangement, but I'm
inclined to think that a cleaner solution is to split out the loadable
module as a separate contrib subdirectory. Thoughts?

Do you mean contrib/pg_upgrade/somelib? If so, +1. If you instead mean
putting the library in contrib/somelib, I'm not so sure that it's
cleaner than trying to support executable+shared lib in a single contrib
module.
--

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

Excerpts from Tom Lane's message of mié may 12 14:07:13 -0400 2010:

We could try to make this a supported build arrangement, but I'm
inclined to think that a cleaner solution is to split out the loadable
module as a separate contrib subdirectory. Thoughts?

Do you mean contrib/pg_upgrade/somelib? If so, +1.

Hmm. I had been thinking the other way, but I'll see if that can be
made to work.

regards, tom lane

Tom Lane wrote:

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

Excerpts from Tom Lane's message of mié may 12 14:07:13 -0400 2010:

We could try to make this a supported build arrangement, but I'm
inclined to think that a cleaner solution is to split out the loadable
module as a separate contrib subdirectory. Thoughts?

Do you mean contrib/pg_upgrade/somelib? If so, +1.

Hmm. I had been thinking the other way, but I'll see if that can be
made to work.

Not sure this will work on its own with the MSVC build system - I don't
think it's set up for sub-modules. But we can finessee that if
necessary, just as we make special provision for pgcrypto.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

Tom Lane wrote:

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

Do you mean contrib/pg_upgrade/somelib? If so, +1.

Hmm. I had been thinking the other way, but I'll see if that can be
made to work.

Not sure this will work on its own with the MSVC build system - I don't
think it's set up for sub-modules.

Oh, right. Since the entire point here is to *not* require new
buildsystem infrastructure for pg_upgrade, I'm back to thinking that
a separate contrib module is the way to go.

regards, tom lane

Tom Lane wrote:

A look at the MSVC buildfarm members shows that they are not building
most of the files added to contrib/pg_upgrade. The reason seems to be
that that module tries to build both an executable program *and* a
shared library, which it does by dint of setting both PROGRAM and
MODULES in its Makefile. Now that is a dirty hack that is nowhere
documented to work, and in fact the pgxs documentation explicitly says
not to do that. It accidentally fails to fail, at the moment, because
of the way pgxs.mk is set up, and because the OBJS variable is only
needed for one of these targets. But the MSVC build scripts aren't
expecting this and evidently disregard PROGRAM after they see MODULES.

Yeah, I was stumped by the problem of creating an executable and shared
object and didn't find any usage of this case. Leave it me to find a
loop-hole. ;-)

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

Tom Lane wrote:

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

Do you mean contrib/pg_upgrade/somelib? If so, +1.

Hmm. I had been thinking the other way, but I'll see if that can be
made to work.

Not sure this will work on its own with the MSVC build system - I don't
think it's set up for sub-modules.

Oh, right. Since the entire point here is to *not* require new
buildsystem infrastructure for pg_upgrade, I'm back to thinking that
a separate contrib module is the way to go.

Uh, if you do 'make install' in the pg_upgrade directory, would it also
install the shared lib contrib? If not, it seems kind of complicated
from a user perspective. Can't we pass a 'make' down into a
subdirectory and have a separate Makefile just run? pg_migrator had
this rule:

all install installdirs uninstall distprep clean distclean maintainer-clean:
$(MAKE) -C src $@
$(MAKE) -C func $@

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

Bruce Momjian wrote:

Can't we pass a 'make' down into a
subdirectory and have a separate Makefile just run? pg_migrator had
this rule:

all install installdirs uninstall distprep clean distclean maintainer-clean:
$(MAKE) -C src $@
$(MAKE) -C func $@

Yes, you can on Unix, of course. But I'm pretty sure it won't work with
the MSVC build system.

cheers

andrew

Bruce Momjian <bruce@momjian.us> writes:

Uh, if you do 'make install' in the pg_upgrade directory, would it also
install the shared lib contrib? If not, it seems kind of complicated
from a user perspective. Can't we pass a 'make' down into a
subdirectory and have a separate Makefile just run?

No. You're still failing to consider the MSVC build case.

I think that anyone who can cope with building pg_upgrade from source
can deal with building pg_upgrade_sysoids in addition, especially if
the documentation tells him to. In practice, 99% of users just build
(or install) all of contrib/ at once, I think, so it's unlikely to
affect them much anyway.

I understand your desire to save one step in the build process, but
I don't think it's worth contorting our build system for --- especially
since pg_migrator isn't likely to stay in contrib indefinitely.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <bruce@momjian.us> writes:

Uh, if you do 'make install' in the pg_upgrade directory, would it also
install the shared lib contrib? If not, it seems kind of complicated
from a user perspective. Can't we pass a 'make' down into a
subdirectory and have a separate Makefile just run?

No. You're still failing to consider the MSVC build case.

I think that anyone who can cope with building pg_upgrade from source
can deal with building pg_upgrade_sysoids in addition, especially if
the documentation tells him to. In practice, 99% of users just build
(or install) all of contrib/ at once, I think, so it's unlikely to
affect them much anyway.

If we make it /contrib/pg_upgrade_shlibs, will it need a documentation
page? Can I built multiple shared libs in there if needed? If we put
it under /contrib/pg_upgrade, can it still be a separate build step?
Would that work?

I understand your desire to save one step in the build process, but
I don't think it's worth contorting our build system for --- especially
since pg_migrator isn't likely to stay in contrib indefinitely.

OK.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

Bruce Momjian <bruce@momjian.us> writes:

If we make it /contrib/pg_upgrade_shlibs, will it need a documentation
page?

I don't see a need for that. Also, why would you make the directory
name different from the name of the shlib it's building --- or are
you having second thoughts about the present name?

Can I built multiple shared libs in there if needed?

No, but why would you need more than one? What you might need
(and can't have with the present hack) is more than one .o file
getting built into the shared library.

If we put
it under /contrib/pg_upgrade, can it still be a separate build step?
Would that work?

Isn't that the same idea you just proposed?

regards, tom lane

Tom Lane wrote:

Bruce Momjian <bruce@momjian.us> writes:

If we make it /contrib/pg_upgrade_shlibs, will it need a documentation
page?

I don't see a need for that. Also, why would you make the directory
name different from the name of the shlib it's building --- or are
you having second thoughts about the present name?

Well, previously I needed separate shared objects because I didn't know
what _new_ backend version I would be linking to and the symbols could
be different. I actually dynamically linked in different SO's for
different major versions.

Now that it only targets the packaged version, I can do with a single
shared object, but maybe it needs to be more generic, like
pg_upgrade_tools.so or something like that.

Can I built multiple shared libs in there if needed?

No, but why would you need more than one? What you might need
(and can't have with the present hack) is more than one .o file
getting built into the shared library.

Again, I used to need this, but I don't now.

If we put
it under /contrib/pg_upgrade, can it still be a separate build step?
Would that work?

Isn't that the same idea you just proposed?

I realize we need a separate pgxs makefile for the executable and shared
libraries. My question was whether the shared library directory should
be under /contrib or under /contrib/pg_upgrade.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

Bruce Momjian <bruce@momjian.us> writes:

Now that it only targets the packaged version, I can do with a single
shared object, but maybe it needs to be more generic, like
pg_upgrade_tools.so or something like that.

+1 for pg_upgrade_tools or pg_upgrade_support or some such name.

I realize we need a separate pgxs makefile for the executable and shared
libraries. My question was whether the shared library directory should
be under /contrib or under /contrib/pg_upgrade.

It has to be directly under /contrib, because the MSVC build scripts
only look there for contrib modules to build.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <bruce@momjian.us> writes:

Now that it only targets the packaged version, I can do with a single
shared object, but maybe it needs to be more generic, like
pg_upgrade_tools.so or something like that.

+1 for pg_upgrade_tools or pg_upgrade_support or some such name.

I like 'pg_upgrade_support'. I could also do 'pg_upgrade_funcs'.

I realize we need a separate pgxs makefile for the executable and shared
libraries. My question was whether the shared library directory should
be under /contrib or under /contrib/pg_upgrade.

It has to be directly under /contrib, because the MSVC build scripts
only look there for contrib modules to build.

OK. Should I get started?

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

On May 12, 2010, at 4:07 PM, Bruce Momjian wrote:

I like 'pg_upgrade_support'. I could also do 'pg_upgrade_funcs'.

I misread the second one at a glance, so I recommend the first.

Best,

David

Bruce Momjian wrote:

Tom Lane wrote:

Bruce Momjian <bruce@momjian.us> writes:

If we make it /contrib/pg_upgrade_shlibs, will it need a documentation
page?

I don't see a need for that. Also, why would you make the directory
name different from the name of the shlib it's building --- or are
you having second thoughts about the present name?

Well, previously I needed separate shared objects because I didn't know
what _new_ backend version I would be linking to and the symbols could
be different. I actually dynamically linked in different SO's for
different major versions.

Now that it only targets the packaged version, I can do with a single
shared object, but maybe it needs to be more generic, like
pg_upgrade_tools.so or something like that.

FYI, to be more explicit, with the pgFoundry code, I didn't know what
target major PG version I would be linking to, so I needed different
shared object files because there were some symbols that would only
resolve to specific backend versions. I would probe the target major
version and link in the matching shared object file. This was
particularly common for Win32 binaries.

That is no longer an issue with /contrib.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com