Packaging 7.1.1

Please,

apply a little patch:

--- src/test/locale/test-ctype.c        Tue Sep  1 08:40:33 1998
+++ /u/megera/app/locale/test/test-ctype.c      Fri Sep 15 19:12:06 2000
@@ -39,7 +39,7 @@
 void
 describe_char(int c)
 {
-       char            cp = c,
+       unsigned char         cp = c,
                              up = toupper(c),
                              lo = tolower(c);

Regards,

Oleg

On Thu, 3 May 2001, Bruce Momjian wrote:

I am starting to package 7.1.1, and I see I did not brand 7.1 properly.
I forgot the date in the HISTORY file, and didn't update register.txt.
I will do all those now for 7.1.1.

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

OK, Oleg, I am applying this on your word only. I don't understand its
purpose, but you sent it with a 7.1.1 subject so I assume you want it in
there. This is not a critical area of our code.

Please,

apply a little patch:

--- src/test/locale/test-ctype.c        Tue Sep  1 08:40:33 1998
+++ /u/megera/app/locale/test/test-ctype.c      Fri Sep 15 19:12:06 2000
@@ -39,7 +39,7 @@
void
describe_char(int c)
{
-       char            cp = c,
+       unsigned char         cp = c,
up = toupper(c),
lo = tolower(c);

Regards,

Oleg

On Thu, 3 May 2001, Bruce Momjian wrote:

I am starting to package 7.1.1, and I see I did not brand 7.1 properly.
I forgot the date in the HISTORY file, and didn't update register.txt.
I will do all those now for 7.1.1.

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

oh btw, i completely forgot to mention the minor fixes to the linux init
scripts i mentioned earlier (about 2 weeks ago) for things that perhaps
should be in the 7.1.1 release. (someone sent out a mail that they were
branching 7.1.1)

Also i never got a response on who actually packages those linux init
scripts that appear in the RPM but not on the pgsql cvs tree. (i am also
curious on why it is different, and how the RPM is built).

-rchit

-----Original Message-----
From: Bruce Momjian [mailto:pgman@candle.pha.pa.us]
Sent: Thursday, May 03, 2001 9:16 AM
To: PostgreSQL-development
Subject: [HACKERS] Packaging 7.1.1

I am starting to package 7.1.1, and I see I did not brand 7.1 properly.
I forgot the date in the HISTORY file, and didn't update register.txt.
I will do all those now for 7.1.1.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Not sure on their status. Are they listed on the outstanding patches
page at the bottom of the developers page? Probably too late for 7.1.1
now.

[ Charset ISO-8859-1 unsupported, converting... ]

oh btw, i completely forgot to mention the minor fixes to the linux init
scripts i mentioned earlier (about 2 weeks ago) for things that perhaps
should be in the 7.1.1 release. (someone sent out a mail that they were
branching 7.1.1)

Also i never got a response on who actually packages those linux init
scripts that appear in the RPM but not on the pgsql cvs tree. (i am also
curious on why it is different, and how the RPM is built).

-rchit

-----Original Message-----
From: Bruce Momjian [mailto:pgman@candle.pha.pa.us]
Sent: Thursday, May 03, 2001 9:16 AM
To: PostgreSQL-development
Subject: [HACKERS] Packaging 7.1.1

I am starting to package 7.1.1, and I see I did not brand 7.1 properly.
I forgot the date in the HISTORY file, and didn't update register.txt.
I will do all those now for 7.1.1.

-- 
Bruce Momjian                        |  http://candle.pha.pa.us
pgman@candle.pha.pa.us               |  (610) 853-3000
+  If your life is a hard drive,     |  830 Blythe Avenue
+  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Rachit Siamwalla <rachit@ensim.com> writes:

Also i never got a response on who actually packages those linux init
scripts that appear in the RPM but not on the pgsql cvs tree. (i am also
curious on why it is different, and how the RPM is built).

Lamar Owen and I.

--
Trond Eivind Glomsr�d
Red Hat, Inc.

Rachit Siamwalla wrote:

oh btw, i completely forgot to mention the minor fixes to the linux init
scripts i mentioned earlier (about 2 weeks ago) for things that perhaps
should be in the 7.1.1 release. (someone sent out a mail that they were
branching 7.1.1)

Also i never got a response on who actually packages those linux init
scripts that appear in the RPM but not on the pgsql cvs tree. (i am also
curious on why it is different, and how the RPM is built).

That would be me. Before building and releasing 7.1.1 RPMs I will be
reviewing the various bugs and changes planned for the 7.1.1 RPM.

As to why the RPM init script is different from the one packaged in the
main source tree -- I can make assumptions in the RPM set that the
version in the source tree cannot.

As to how the RPMs are built -- to answer that question sanely requires
me to know how much experience you have with the whole RPM paradigm.
'How is the RPM built?' is a multifaceted question. The obvious simple
answer is that I maintain:
1.) A set of patches to make certain portions of the source
tree 'behave' in the different environment of the RPMset;
2.) The initscript;
3.) Any other ancilliary scripts and files;
4.) A README.rpm-dist document that tries to adequately document
both the differences between the RPM build and the WHY of the
differences, as well as useful RPM environment operations
(like, using syslog, upgrading, getting postmaster to
start at OS boot, etc);
5.) The spec file that throws it all together. This is not a
trivial undertaking in a package of this size.

I then download and build on as many different canonical distributions
as I can -- currently I am able to build on Red Hat 6.2, 7.0, and 7.1 on
my personal hardware. Occasionally I receive opportunity from certain
commercial enterprises such as Great Bridge and PostgreSQL Inc to build
on other distributions.

I test the build by installing the resulting packages and running the
regression tests. Once the build passes these tests, I upload to the
postgresql.org ftp server and make a release announcement. I am also
responsible for maintaining the RPM download area on the ftp site.

You'll notice I said 'canonical' distributions above. That simply means
that the machine is as stock 'out of the box' as practical -- that is,
everything (except select few programs) on these boxen are installed by
RPM; only official Red Hat released RPMs are used (except in unusual
circumstances involving software that will not alter the build -- for
example, installing a newer non-RedHat version of the Dia diagramming
package is OK -- installing Python 2.1 on the box that has Python 1.5.2
installed is not, as that alters the PostgreSQL build). The RPM as
uploaded is built to as close to out-of-the-box pristine as is
possible. Only the standard released 'official to that release'
compiler is used -- and only the standard official kernel is used as
well.

For a time I built on Mandrake for RedHat consumption -- no more.
Nonstandard RPM building systems are worse than useless. Which is not
to say that Mandrake is useless! By no means is Mandrake useless --
unless you are building Red Hat RPMs -- and Red Hat is useless if you're
trying to build Mandrake or SuSE RPMs, for that matter. But I would be
foolish to use 'Lamar Owen's Super Special RPM Blend Distro 0.1.2' to
build for public consumption! :-)

I _do_ attempt to make the _source_ RPM compatible with as many
distributions as possible -- however, since I have limited resources (as
a volunteer RPM maintainer) I am limited as to the amount of testing
said build will get on other distributions, architectures, or systems.

And, while I understand people's desire to immediately upgrade to the
newest version, realize that I do this as a side interest -- I have a
regular, full-time job as a broadcast
engineer/webmaster/sysadmin/Technical Director which occasionally
prevents me from making timely RPM releases. This happened during the
early part of the 7.1 beta cycle -- but I believe I was pretty much on
the ball for the Release Candidates and the final release.

I am working towards a more open RPM distribution -- I would dearly love
to more fully document the process and put everything into CVS -- once I
figure out how I want to represent things such as the spec file in a CVS
form. It makes no sense to maintain a changelog, for instance, in the
spec file in CVS when CVS does a better job of changelogs -- I will need
to write a tool to generate a real spec file from a CVS spec-source file
that would add version numbers, changelog entries, etc to the result
before building the RPM. IOW, I need to rethink the process -- and then
go through the motions of putting my long RPM history into CVS one
version at a time so that version history information isn't lost.

As to why all these files aren't part of the source tree, well, unless
there was a large cry for it to happen, I don't believe it should.
PostgreSQL is very platform-agnostic -- and I like that. Including the
RPM stuff as part of the Official Tarball (TM) would, IMHO, slant that
agnostic stance in a negative way. But maybe I'm too sensitive to
that. I'm not opposed to doing that if that is the consensus of the
core group -- and that would be a sneaky way to get the stuff into CVS
:-). But if the core group isn't thrilled with the idea (and my
instinct says they're not likely to be), I am opposed to the idea -- not
to keep the stuff to myself, but to not hinder the platform-neutral
stance. IMHO, of course.

Of course, there are many projects that DO include all the files
necessary to build RPMs from their Official Tarball (TM).

Bruce, should portions of that answer be part of the linux FAQ? I don't
want to have to write that too many times :-).
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Trond Eivind Glomsr�d wrote:

Rachit Siamwalla <rachit@ensim.com> writes:

Also i never got a response on who actually packages those linux init
scripts that appear in the RPM but not on the pgsql cvs tree. (i am also
curious on why it is different, and how the RPM is built).

Lamar Owen and I.

Egads! I forgot to mention Trond! My apologies! (I'm being serious...)

Trond, of Red Hat; Reinhard Max, of SuSE; and Thomas Lockhart, of
PostgreSQL Inc (:-)) have all been major contributors to the RPM
distribution. Karl DeBisschop, Mike Mascari, and many others have
provided fixes and ideas as well.

Sorry guys -- I got caught up in the process and forgot the people! :-(
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Thanks to input Bruce M., figured out my performance problems - had to do
with a few QUERIES FROM HELL! After running EXPLAIN a few times I fine
tuned some of the worst ones, mostly over use of sub queries. Still
combing through my query log.

Getting there...

-r

Ryan Mahoney wrote:

Any input would be helpful! If you need additional info let me know.

BTW, Thanks to Lamar for some great tips today!

You're more than welcome.

I forgot a basic tip, which leads to a question:
How often are you running VACUUM ANALYZE?

If this were PostgreSQL 7.0.3, we could ask Alfred about his lazy vacuum
patches, as they work as well for Red Hat 7 as they do for FreeBSD.

Personally, I look forward to the following note being placed into the
docs:
VACUUM: deprecated. And the feature that makes that note possible.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Thanks a lot for your total and complete description of the process. (i
should have checked out the sprm first before asking). I empathize with
what you said about packaging not being a simple task, i have been through
the agony.

About putting your stuff into the postgres tree, i believe it would be a
"good" thing other than bad to include it in pgsq. It can be put into the
contrib directory (because it isn't part of the "core" portable stuff). This
solution was done for the portable openssh cvs tree. not only redhat
packaging stuff was included, but the solaris pkg mechanism was also in
there (and i also believe there were some others). It usually isn't a lot of
files (ie. the spec file and maybe the initscript). Of course its up to the
gods of the pgsql tree what they want to do with it, so i'm just going to
raise this suggestion and shut up.

anyways, getting back to the what brought me to ask about this, can you add
the fixes to these two small problems in your initscripts?

1. `pidof` should be `pidof -s` (2 instances)
2. restart) should be stop; sleep x; start
ideally, stop should actually wait till postgres fully stops. The sleep is
just a temporary fix.

I have a more thorough email i sent earlier, i can resend it to you if you
want.

-rchit

Lamar Owen <lamar.owen@wgcr.org> writes:

Ryan Mahoney wrote:

Any input would be helpful! If you need additional info let me know.

BTW, Thanks to Lamar for some great tips today!

You're more than welcome.

I forgot a basic tip, which leads to a question:
How often are you running VACUUM ANALYZE?

If this were PostgreSQL 7.0.3, we could ask Alfred about his lazy vacuum
patches, as they work as well for Red Hat 7 as they do for FreeBSD.

Postgresql 7.0.3 from Red Hat Linux 7.1 should work just fine on Red
Hat Linux 7.

--
Trond Eivind Glomsr�d
Red Hat, Inc.

Rachit Siamwalla wrote:

Thanks a lot for your total and complete description of the process. (i
should have checked out the sprm first before asking). I empathize with
what you said about packaging not being a simple task, i have been through
the agony.

Empathize is appropriate if you've been there. But, it's better than
going six months to a year for a newer RPM -- the release lag was one
ofthe two triggers that caused me to go do this -- the other was the
upgrading issue. I won't say any more about that right now --too tired.

About putting your stuff into the postgres tree, i believe it would be a
"good" thing other than bad to include it in pgsq. It can be put into the
contrib directory (because it isn't part of the "core" portable stuff). This

We'll see what transpires.

I have a more thorough email i sent earlier, i can resend it to you if you
want.

Hmmm.. lessee... I have Bruce's reply, which includes your message in
its entirety, I think. But, just to be safe, resend directly to me, and
add the [HACKERS] part to the subject (so it will go to the correct mail
folder, otherwise I might miss it). I have a list of messages in an
'RPMS for 7.1' subfolder of my mail folder 'Postgres' that I work
through for each release.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Trond Eivind Glomsr�d wrote:

Rachit Siamwalla <rachit@ensim.com> writes:

Also i never got a response on who actually packages those linux init
scripts that appear in the RPM but not on the pgsql cvs tree. (i am also
curious on why it is different, and how the RPM is built).

Lamar Owen and I.

Is the current snapshot available? I have submitted fixes twice now for what I am fairly sure is a bug in the init script. At least one of the posts was the shortly after lamar posted the RC3 RPM. Yet the bug remained.

This is not a complaint -- you guys have put alot of effort into the RPMs and they are very solid IMHO. But I would like the chance to look at the RPM as it stands sometime before 7.1, as I have to customize the RPM yet again to distribute a working init script to our servers.

Have you thought about a CVS store some place for the RPM files?

--
Karl

Lamar Owen <lamar.owen@wgcr.org> writes:

As to why all these files aren't part of the source tree, well, unless
there was a large cry for it to happen, I don't believe it should.
PostgreSQL is very platform-agnostic -- and I like that. Including the
RPM stuff as part of the Official Tarball (TM) would, IMHO, slant that
agnostic stance in a negative way.

Seems like that stuff should be in CVS somewhere ... if only so someone
else can pick up the ball if you get run over by a truck :-(.

If it's just a small amount of code, I don't see what the harm would be
in including it in the regular distro, though we should talk about just
where it should go. If it's a large amount of code then perhaps a
separate CVS project would be better, so that people who have no use for
it don't end up pulling/downloading it.

regards, tom lane

On Thu, 3 May 2001, Rachit Siamwalla wrote:

1. `pidof` should be `pidof -s` (2 instances)
2. restart) should be stop; sleep x; start
ideally, stop should actually wait till postgres fully stops. The sleep is
just a temporary fix.

Perhaps a naive question, but why not use the pg_ctl for starting and
stopping?

It has a -w option to have it wait for the stop/start/restart to complete.

-rocco

Tom Lane wrote:

Seems like that stuff should be in CVS somewhere ... if only so someone
else can pick up the ball if you get run over by a truck :-(.

My wife appreciates the sentiment :-). As it stands now, better
documentation distributed in the source RPM would help greatly.
Everything necessary to do the build and maintain the package is in the
source RPM as it stands now -- evidenced by the Linux distributors being
able to take our source RPM, massage it to fit their particular system,
and run with it. And I have a scad of history available in specfile
form....

If it's just a small amount of code, I don't see what the harm would be
in including it in the regular distro, though we should talk about just
where it should go. If it's a large amount of code then perhaps a
separate CVS project would be better, so that people who have no use for
it don't end up pulling/downloading it.

Not counting the JDBC jars, it's a hundred K or so uncompressed. The
spec file is around 30k -- a small amount of code.

contrib/rpm-dist?
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Lamar Owen <lamar.owen@wgcr.org> writes:

contrib/rpm-dist?

Contrib was my first thought also --- but on second thought, the RPM
packaging support is hardly contrib-grade material. For a large
proportion of our users it's a critical part of the distribution.
So, if we are going to have it in the CVS tree at all, I'd vote for
putting it in the main tree.

Perhaps src/rpm-tools/ or some such name.

regards, tom lane

Tom Lane wrote:

Contrib was my first thought also --- but on second thought, the RPM
packaging support is hardly contrib-grade material. For a large
proportion of our users it's a critical part of the distribution.
So, if we are going to have it in the CVS tree at all, I'd vote for
putting it in the main tree.

Perhaps src/rpm-tools/ or some such name.

Let's see where the rest of core and hackers weighs in....
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Lamar Owen wrote:

Tom Lane wrote:

Seems like that stuff should be in CVS somewhere ... if only so someone
else can pick up the ball if you get run over by a truck :-(.

My wife appreciates the sentiment :-). As it stands now, better
documentation distributed in the source RPM would help greatly.
Everything necessary to do the build and maintain the package is in the
source RPM as it stands now -- evidenced by the Linux distributors being
able to take our source RPM, massage it to fit their particular system,
and run with it. And I have a scad of history available in specfile
form....

If it's just a small amount of code, I don't see what the harm would be
in including it in the regular distro, though we should talk about just
where it should go. If it's a large amount of code then perhaps a
separate CVS project would be better, so that people who have no use for
it don't end up pulling/downloading it.

Not counting the JDBC jars, it's a hundred K or so uncompressed. The
spec file is around 30k -- a small amount of code.

contrib/rpm-dist?

Seems to work. But I would prefer to look at how ither packaging schemes
work and come up with something that might be consistent and useful
across the board.

For starters, I'd make contrib/package/

Then make an rpm subdirectory. Also a pkg directory for system that use
pkgmk/pkginfo/pkgadd/pkgrm. If there's a way to may debain packages paly
the game, put them in as well. Then, if someaone is packages for a
variety of systems, there is alt least the possibility of some small
amount of consistency.

Extending things, you could have contrib/package/rpm/redhat for
redhat-specific stuff. contrib/package/rpm/mandrake for mandrafke stuff.
You get the idea.

At that point, I could even imagine contrib/mkpackage script that di som
OS detection, and built wahtever you wanted. That may be a little far
off, but I think there is an important nuggent in here. Tarballs are
great for developers, but they are not that great for system
administrators with large installed bases. PostgreSQL builds are great
for the portability. The next logical step might in fact be to extend
some of that consistency to the package creation arena.

--
Karl