Trimming the Fat, Part Deux ...

Started by The Hermit Hackerover 23 years ago27 messageshackers
Jump to latest
#1The Hermit Hacker
scrappy@hub.org

Okay ... since this is pretty much going to be 'one camp for, one camp
against' without anything to really back up either camps perspectives /
arguments, I did some research on CVS in order to find a nice, effective
middle ground ... and it actually works quite sweet ...

Basically, CVS let's you "merge" modules into one mega-module, which is
what I've just done ...

I pulled out two sub-directories from the pgsql CVS repository ...
contrib/earthdistance and src/interfaces/libpqxx ... if you do:

cvs checkout -P libpqxx

you'll see *just* the libpqxx code ... same with doing a checkout of
'earthdistance' ...

If you checkout pgsql, it will get the source tree *minus* libpqxx and
earthdistance ...

And, finally, if you want it all:

cvs checkout -P pgsql-all

Now, for those that are going to yell out about already checked out code
... don't, I've already tested that too ...

From within your checked out code, remove the above two directories, then
from the top level (ie. if you have your src in src/pgsql, go to the src
directory, *not* the root of the source tree itself) and type:

cvs checkout -P contrib interfaces

and you will be 'in sync' ...

Next thing I'm going to do is modify the scripts that build the .tar.gz
packages so that they now build a libpqxx.tar.gz and earthdistance.tar.gz
distribution, so that they can be downloaded seperately, as well as
pulling out the README files so that ppl know what they are useful for in
the first place ...

For those that like to work on the 'mega source tree', nothing has changed
... you check out pgsql-all, and commits will commit to the right places
in the repository ...

If ppl want to download everything, including the kitchen sink, then they
will have that option ... but if ppl want to just download what they need,
that option is now open to them as well ...

Now comes the 'tricky part, which I'm hoping someone like Peter might know
how to do ... I know there is a way of writing configure to 'run
configure' in sub projects .. for instance, with libpqxx ... at the top
level of pgsql, one could type:

./configure --enable-libpqxx

and it would run the appropriate configure in the libpqxx subdirectory and
add libpqxx to the 'targets' in src/interfaces/Makefile ...

If we can get *that* done, removing something like libpq++ would be a
simple matter of removing the option from configure.in at the top level,
and removing its inclusion in the pgsql-all meta on the CVS repository ...

nice and clean ...

I've only done libpqxx and earthdistance right now ... none of the history
is lost, but I want to get the packaging issue worked out using those two
first, before I dive into the others ... also, want to make sure that I
haven't overlooked anything in my testing ...

*Eventually*, a simple checkout of 'pgsql' should result in a "server
only" distribution that we can pull bits and pieces into transparently ...

the really cool thing is that using PHP, I could probably come up with a
nice simple interface that ppl could create a custom download bundle:

"I want the base server + jdbc + the earth distance module"

and it would package that up and present it to them to download ... then
everyone can package whatever they want and download it ...

#2Christopher Kings-Lynne
chriskl@familyhealth.com.au
In reply to: The Hermit Hacker (#1)
Re: Trimming the Fat, Part Deux ...

Okay ... since this is pretty much going to be 'one camp for, one camp
against' without anything to really back up either camps perspectives /
arguments, I did some research on CVS in order to find a nice, effective
middle ground ... and it actually works quite sweet ...

Personally, I'd like to be able to have a "client-only" distro, but other
than that I'm not fussed. I like the convenience of having all the contribs
in contrib, however some sort of CPgAN thing would be sweet as.

Currently, the guy who maintains the FreeBSD postgres port has hacked up a
client only version that does all the building, but only installs the client
stuff.

Chris

#3The Hermit Hacker
scrappy@hub.org
In reply to: Christopher Kings-Lynne (#2)
Re: Trimming the Fat, Part Deux ...

On Thu, 1 Aug 2002, Christopher Kings-Lynne wrote:

Okay ... since this is pretty much going to be 'one camp for, one camp
against' without anything to really back up either camps perspectives /
arguments, I did some research on CVS in order to find a nice, effective
middle ground ... and it actually works quite sweet ...

Personally, I'd like to be able to have a "client-only" distro, but other
than that I'm not fussed. I like the convenience of having all the contribs
in contrib, however some sort of CPgAN thing would be sweet as.

Currently, the guy who maintains the FreeBSD postgres port has hacked up a
client only version that does all the building, but only installs the client
stuff.

Ya, which is what the MySQL port does also ... but you still have to
download the complete distro ... the other thing to consider is that we
have how many mirrors right now? That are paying for the bandwidth to
mirror the software and provide it to the community? If 100 ppl that just
need, for instance, the libpq libraries could download *just* the
libpq.tar.gz file and install it, how much bandwidth does that save
overall? Bruce figured that the libpq.tar.gz stuff he did for me saved
~1/10 the space of the whole distro ... so if the distro is ~8Meg, that
means that the libpq is ~800k ... so we're talking about a saving of,
what, ~720Meg of bandwidth? It doesn't sound like alot when you aren't
paying for it, but it definitely adds up quickly to those that are ...

#4Thomas Lockhart
lockhart@fourpalms.org
In reply to: The Hermit Hacker (#1)
Re: Trimming the Fat, Part Deux ...

...

*Eventually*, a simple checkout of 'pgsql' should result in a "server
only" distribution that we can pull bits and pieces into transparently ...

I'm still not quite sure where this is headed or why, but if nothing
else pgsql could and should be the whole thing, and pgsql-server could
be the server subset. Server-only makes sense for only some users, and
nothing apparently makes sense for everyone. But we may as well have the
old names do the old things...

- Thomas

#5The Hermit Hacker
scrappy@hub.org
In reply to: Thomas Lockhart (#4)
Re: Trimming the Fat, Part Deux ...

On Wed, 31 Jul 2002, Thomas Lockhart wrote:

...

*Eventually*, a simple checkout of 'pgsql' should result in a "server
only" distribution that we can pull bits and pieces into transparently ...

I'm still not quite sure where this is headed or why, but if nothing
else pgsql could and should be the whole thing, and pgsql-server could
be the server subset. Server-only makes sense for only some users, and
nothing apparently makes sense for everyone. But we may as well have the
old names do the old things...

will make those changes in the morning ...

#6Lamar Owen
lamar.owen@wgcr.org
In reply to: The Hermit Hacker (#1)
Re: Trimming the Fat, Part Deux ...

On Wednesday 31 July 2002 09:38 pm, Marc G. Fournier wrote:

Okay ... since this is pretty much going to be 'one camp for, one camp
against' without anything to really back up either camps perspectives /
arguments, I did some research on CVS in order to find a nice, effective
middle ground ... and it actually works quite sweet ...

MAn, did I ever pick a bad day to be offline for a whole day. :-)

If anyone cares to look, or for that matter cares at all, something similar is
already being done in the binary RPMs. I have split, sliced, and diced this
distribution for over three years. So, let me share some of the experiences
learned from that exercise.

Now comes the 'tricky part, which I'm hoping someone like Peter might know
how to do ... I know there is a way of writing configure to 'run
configure' in sub projects .. for instance, with libpqxx ... at the top
level of pgsql, one could type:

./configure --enable-libpqxx

I like this idea, up to an extent. I guess it boils down to this:
1.) What is the minimum build environment necessary to build anything in the
source distribution;
2.) What degree of granularity is desired;
3.) We must not assume the presence of a full source tree to build _anything_,
only the minimum build system, which can then handle everything else as a
module.

Then I could much more easily package a 'postgresql-devel' package that would
allow, for instance, PostGIS to be built as a server module without having to
have the entire source distribution tree in place (which is currently
required). As PostGIS is a separately developed and distributed module, it
seems reasonable that it should be buildable without the full source tree in
place.

As to getting rid of bloat, I'm all for splitting out contrib modules, client
libraries, and the like into separate projects. If the build system is a
unit, and is not dependent upon the server or libpq source code, then it is
easier and more consistent to just require it to be in place to build ANY
contrib or client module.

We're too big and too spread out. It was said (by Andrew) that we're hard to
install -- why is that? Is it possible that it is _because_ of the number of
pieces we include?

And the sooner our very old perl client goes away, the better I like it. It
is a good client, don't get me wrong: but DBD:Pg is the standard now.

But, if you are an RPM user, you can already just download the pieces for a
minimal client-side system. And you have been able to do so for right at
three years, give or take.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

#7Jeff MacDonald
jeff@tsunamicreek.com
In reply to: Lamar Owen (#6)
Re: Trimming the Fat, Part Deux ...

And the sooner our very old perl client goes away, the better I
like it. It
is a good client, don't get me wrong: but DBD:Pg is the standard now.

This may sound like a dumb question, but DBD::Pg == DBI right ? not pg.pm

I code perl about 25 hours a week, and DBI has never failed me yet.

Jeff.

#8Lamar Owen
lamar.owen@wgcr.org
In reply to: Jeff MacDonald (#7)
Re: Trimming the Fat, Part Deux ...

On Thursday 01 August 2002 12:05 pm, Jeff MacDonald wrote:

And the sooner our very old perl client goes away, the better I
like it. It
is a good client, don't get me wrong: but DBD:Pg is the standard now.

This may sound like a dumb question, but DBD::Pg == DBI right ? not pg.pm

Right.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

#9Bruce Momjian
bruce@momjian.us
In reply to: Lamar Owen (#6)
Re: Trimming the Fat, Part Deux ...

Lamar Owen wrote:

And the sooner our very old perl client goes away, the better I like it. It
is a good client, don't get me wrong: but DBD:Pg is the standard now.

I have been in contact with Edmund about moving DBD into our CVS and
updating CPAN ourselves. Should have a final answer soon.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
#10Bruce Momjian
bruce@momjian.us
In reply to: Bruce Momjian (#9)
Re: Trimming the Fat, Part Deux ...

Bruce Momjian wrote:

Lamar Owen wrote:

And the sooner our very old perl client goes away, the better I like it. It
is a good client, don't get me wrong: but DBD:Pg is the standard now.

I have been in contact with Edmund about moving DBD into our CVS and
updating CPAN ourselves. Should have a final answer soon.

OK, I got the go-ahead from Edmund. We will have DBD:pg in the 7.3
tarball. I will add it to CVS today or tomorrow.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
#11Oleg Bartunov
oleg@sai.msu.su
In reply to: Bruce Momjian (#10)
Re: Trimming the Fat, Part Deux ...

On Thu, 1 Aug 2002, Bruce Momjian wrote:

Bruce Momjian wrote:

Lamar Owen wrote:

And the sooner our very old perl client goes away, the better I like it. It
is a good client, don't get me wrong: but DBD:Pg is the standard now.

I have been in contact with Edmund about moving DBD into our CVS and
updating CPAN ourselves. Should have a final answer soon.

OK, I got the go-ahead from Edmund. We will have DBD:pg in the 7.3
tarball. I will add it to CVS today or tomorrow.

Hmm,

according README from DBD-Pg ( 1.13 ), it's maintained now by
Jeffrey W. Baker (jwbaker@acm.org)

DBD::Pg -- a PostgreSQL interface for Perl 5.

$Id: README,v 1.3 2002/04/10 02:01:38 jwb Exp $

Copyright (c) 1997,1998,1999,2000 Edmund Mergl
Copyright (c) 2002 Jeffrey W. Baker
Portions Copyright (c) 1994,1995,1996,1997 Tim Bunce

I'm a little bit aware about DBD::Pg because it needs many changes and
testing to be 7.3 compliant.

Jeffrey, wake up :-)

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

#12Bruce Momjian
bruce@momjian.us
In reply to: Oleg Bartunov (#11)
Re: Trimming the Fat, Part Deux ...

Oleg Bartunov wrote:

On Thu, 1 Aug 2002, Bruce Momjian wrote:

Bruce Momjian wrote:

Lamar Owen wrote:

And the sooner our very old perl client goes away, the better I like it. It
is a good client, don't get me wrong: but DBD:Pg is the standard now.

I have been in contact with Edmund about moving DBD into our CVS and
updating CPAN ourselves. Should have a final answer soon.

OK, I got the go-ahead from Edmund. We will have DBD:pg in the 7.3
tarball. I will add it to CVS today or tomorrow.

Hmm,

according README from DBD-Pg ( 1.13 ), it's maintained now by
Jeffrey W. Baker (jwbaker@acm.org)

DBD::Pg -- a PostgreSQL interface for Perl 5.

$Id: README,v 1.3 2002/04/10 02:01:38 jwb Exp $

Copyright (c) 1997,1998,1999,2000 Edmund Mergl
Copyright (c) 2002 Jeffrey W. Baker
Portions Copyright (c) 1994,1995,1996,1997 Tim Bunce

I'm a little bit aware about DBD::Pg because it needs many changes and
testing to be 7.3 compliant.

Jeffrey, wake up :-)

Oh, that's strange. I wonder why Edmund didn't mention that.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
#13The Hermit Hacker
scrappy@hub.org
In reply to: Bruce Momjian (#10)
Re: Trimming the Fat, Part Deux ...

On Thu, 1 Aug 2002, Bruce Momjian wrote:

Bruce Momjian wrote:

Lamar Owen wrote:

And the sooner our very old perl client goes away, the better I like it. It
is a good client, don't get me wrong: but DBD:Pg is the standard now.

I have been in contact with Edmund about moving DBD into our CVS and
updating CPAN ourselves. Should have a final answer soon.

OK, I got the go-ahead from Edmund. We will have DBD:pg in the 7.3
tarball. I will add it to CVS today or tomorrow.

If you know how to, please import it as a seperate module, do not add it
to the pgsql tree itself ... if you don't know how, please just forward it
onto me and I'll get it into the repository ...

#14The Hermit Hacker
scrappy@hub.org
In reply to: Oleg Bartunov (#11)
Re: Trimming the Fat, Part Deux ...

Ummm ... stupid question, but can we even bring this into the 'core'?

You may distribute under the terms of either the GNU General Public
License or the Artistic License, as specified in the Perl README file.

On Thu, 1 Aug 2002, Oleg Bartunov wrote:

Show quoted text

On Thu, 1 Aug 2002, Bruce Momjian wrote:

Bruce Momjian wrote:

Lamar Owen wrote:

And the sooner our very old perl client goes away, the better I like it. It
is a good client, don't get me wrong: but DBD:Pg is the standard now.

I have been in contact with Edmund about moving DBD into our CVS and
updating CPAN ourselves. Should have a final answer soon.

OK, I got the go-ahead from Edmund. We will have DBD:pg in the 7.3
tarball. I will add it to CVS today or tomorrow.

Hmm,

according README from DBD-Pg ( 1.13 ), it's maintained now by
Jeffrey W. Baker (jwbaker@acm.org)

DBD::Pg -- a PostgreSQL interface for Perl 5.

$Id: README,v 1.3 2002/04/10 02:01:38 jwb Exp $

Copyright (c) 1997,1998,1999,2000 Edmund Mergl
Copyright (c) 2002 Jeffrey W. Baker
Portions Copyright (c) 1994,1995,1996,1997 Tim Bunce

I'm a little bit aware about DBD::Pg because it needs many changes and
testing to be 7.3 compliant.

Jeffrey, wake up :-)

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

#15Bruce Momjian
bruce@momjian.us
In reply to: The Hermit Hacker (#14)
Re: Trimming the Fat, Part Deux ...

Marc G. Fournier wrote:

Ummm ... stupid question, but can we even bring this into the 'core'?

You may distribute under the terms of either the GNU General Public
License or the Artistic License, as specified in the Perl README file.

Artistic License is fine, I think.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
#16Lamar Owen
lamar.owen@wgcr.org
In reply to: Bruce Momjian (#10)
Re: Trimming the Fat, Part Deux ...

On Thursday 01 August 2002 02:21 pm, Bruce Momjian wrote:

Bruce Momjian wrote:

Lamar Owen wrote:

And the sooner our very old perl client goes away, the better I like
it. It is a good client, don't get me wrong: but DBD:Pg is the
standard now.

I have been in contact with Edmund about moving DBD into our CVS and
updating CPAN ourselves. Should have a final answer soon.

OK, I got the go-ahead from Edmund. We will have DBD:pg in the 7.3
tarball. I will add it to CVS today or tomorrow.

Um, is putting it into our tarball necessary, or even desireable? It is
separately maintained as part of CPAN. Is there some fundamental reason that
we _must_ ship a perl client (the same goes for tcl/tk/python/c++ as well) if
it is adequately maintained in the standard location? Perlers know to go to
CPAN. Likewise Pythonistas have their own place.

And that's the crux of Marc's message, I believe -- why can't we minimize
here?
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

#17Peter Eisentraut
peter_e@gmx.net
In reply to: Bruce Momjian (#15)
Re: Trimming the Fat, Part Deux ...

Bruce Momjian writes:

Artistic License is fine, I think.

The Artistic License doesn't even qualify as Free Software as far as the
FSF is concerned.

More generally, it is a different license, and that is a problem.

--
Peter Eisentraut peter_e@gmx.net

#18Peter Eisentraut
peter_e@gmx.net
In reply to: Bruce Momjian (#10)
Re: Trimming the Fat, Part Deux ...

Bruce Momjian writes:

OK, I got the go-ahead from Edmund. We will have DBD:pg in the 7.3
tarball. I will add it to CVS today or tomorrow.

Pleeeeease, no more Perl modules in our CVS! The ones we have are already
messy enough to build.

I thought we were talking about trimming the source tree, not adding more.
Why not put it on gborg or somewhere else?

--
Peter Eisentraut peter_e@gmx.net

#19Lamar Owen
lamar.owen@wgcr.org
In reply to: Peter Eisentraut (#18)
Re: Trimming the Fat, Part Deux ...

On Thursday 01 August 2002 04:37 pm, Peter Eisentraut wrote:

I thought we were talking about trimming the source tree, not adding more.
Why not put it on gborg or somewhere else?

It's already in CPAN. A link to CPAN should suffice, IMHO.

I also thought we were discussing trimming the tree; and that was a good
feeling.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

#20The Hermit Hacker
scrappy@hub.org
In reply to: Lamar Owen (#19)
Re: Trimming the Fat, Part Deux ...

On Thu, 1 Aug 2002, Lamar Owen wrote:

On Thursday 01 August 2002 04:37 pm, Peter Eisentraut wrote:

I thought we were talking about trimming the source tree, not adding more.
Why not put it on gborg or somewhere else?

It's already in CPAN. A link to CPAN should suffice, IMHO.

I also thought we were discussing trimming the tree; and that was a good
feeling.

Oh thank you to both you and Peter .... I was feeling sooooo alone out
here ...

#21Bruce Momjian
bruce@momjian.us
In reply to: Peter Eisentraut (#17)
#22Bruce Momjian
bruce@momjian.us
In reply to: Lamar Owen (#19)
#23Lamar Owen
lamar.owen@wgcr.org
In reply to: Bruce Momjian (#22)
#24Nigel J. Andrews
nandrews@investsystems.co.uk
In reply to: Lamar Owen (#16)
#25Tom Lane
tgl@sss.pgh.pa.us
In reply to: Bruce Momjian (#15)
#26The Hermit Hacker
scrappy@hub.org
In reply to: Tom Lane (#25)
#27Michael Alan Dorman
mdorman@debian.org
In reply to: Peter Eisentraut (#17)