Buildfarm vs. Linux Distro classification

Andrew Dunstan wrote:

Lately there have been some buildfarm registrations for "Debian
testing/unstable" or similarly described machines. I have kicked back
against these, as the description seems to me to be far too open ended.
Likewise, I also have difficulty with Gentoo because a version there
seems to describe a build system but not any reasonably bounded set of
software. The whole point of classifying buildfarm machines is so that
we can pin down problems to some class of machines, so I'm not really
sure what I should do in these cases that essentially represent highly
mobile targets.

Thoughts?

Well, the one downside to kicking these machines is that we won't get
representation from the varying glibc/gcc combinations. It *almost*
seems that we care more about:

Archicteture
32/64bit (I could be running 32bit on my Athlon64)
GCC version
glibc version

?

Sincerely,

Joshua D. Drake

cheers

andrew

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Andrew Dunstan wrote:

Lately there have been some buildfarm registrations for "Debian
testing/unstable" or similarly described machines. I have kicked back
against these, as the description seems to me to be far too open
ended.

Then again, it would be useful to actually test on Debian
testing/unstable (or pre-release branches of other OS), because that
would (a) expose problems with new toolchains and such earlier than in
released versions, and (b) provide advance testing for when testing
becomes the release. Consider, the number of users that will run 8.2
on Debian stable is probably going to be less than the number of users
who will run 8.2 on what today is testing.

I agree that the lack of a fixed version designation is unsatisfactory.
I'm not sure whether that is actually necessary, though. If PostgreSQL
doesn't work on some machine, then that's a problem anyway.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:

Andrew Dunstan wrote:

Lately there have been some buildfarm registrations for "Debian
testing/unstable" or similarly described machines. I have kicked back
against these, as the description seems to me to be far too open
ended.

Then again, it would be useful to actually test on Debian
testing/unstable (or pre-release branches of other OS), because that
would (a) expose problems with new toolchains and such earlier than in
released versions, and (b) provide advance testing for when testing
becomes the release. Consider, the number of users that will run 8.2
on Debian stable is probably going to be less than the number of users
who will run 8.2 on what today is testing.

I agree that the lack of a fixed version designation is unsatisfactory.
I'm not sure whether that is actually necessary, though. If PostgreSQL
doesn't work on some machine, then that's a problem anyway.

well I think Andrew is more scared of having multiple boxes on the
buildfarm all stating to be "Debian testing" or "Debian unstable" but
without much information on how regulary those boxes are actually synced
to those moving/changing branches and causing discussions on "why is it
suddenly failung on that box but not on the other".
There might be quite a difference between a 2 month old testing and a
recent one for example.
However - we already have some precedence on the buildfarm for that
(like the various -current BSD-members)

Stefan

Stefan Kaltenbrunner wrote:

well I think Andrew is more scared of having multiple boxes on the
buildfarm all stating to be "Debian testing" or "Debian unstable" but
without much information on how regulary those boxes are actually synced
to those moving/changing branches and causing discussions on "why is it
suddenly failung on that box but not on the other".

It might make sense to attach the version information (in this case,
gcc, glibc, kernel versions) to each particular test run, rather than to
each particular farm member.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Alvaro Herrera wrote:

Stefan Kaltenbrunner wrote:

well I think Andrew is more scared of having multiple boxes on the
buildfarm all stating to be "Debian testing" or "Debian unstable" but
without much information on how regulary those boxes are actually synced
to those moving/changing branches and causing discussions on "why is it
suddenly failung on that box but not on the other".

It might make sense to attach the version information (in this case,
gcc, glibc, kernel versions) to each particular test run, rather than to
each particular farm member.

that is a good idea generally but that information might not be
available in a portable way on all platforms (and even than we might
want to have version information on things/libs like readline,zlib,ldap,
too) ...

Stefan

Peter Eisentraut <peter_e@gmx.net> writes:

I agree that the lack of a fixed version designation is unsatisfactory.
I'm not sure whether that is actually necessary, though. If PostgreSQL
doesn't work on some machine, then that's a problem anyway.

The buildfarm script already seems to record various info such as
"uname" output on-the-fly. If we could get it to record compiler
version ("gcc -v" is easy, but equivalent incantations for vendor
compilers might be harder to find) and a few other facts on-the-fly,
I think the instability of the platforms might not be that big a deal.

In practice, it seems that only Linux-based distros have bought into
this idea that bleeding-edge tools are a good thing, so solutions that
work only on Linux may be sufficient.

regards, tom lane

Tom Lane wrote:

The buildfarm script already seems to record various info such as
"uname" output on-the-fly. If we could get it to record compiler
version ("gcc -v" is easy, but equivalent incantations for vendor
compilers might be harder to find) and a few other facts on-the-fly,
I think the instability of the platforms might not be that big a
deal.

The configure script actually tries to get the compiler to put out a
version number, and you can see the result of that in config.log.

For other components, however, it seems impossible to get the version
numbers across platforms.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Quoth peter_e@gmx.net (Peter Eisentraut):

Andrew Dunstan wrote:

Lately there have been some buildfarm registrations for "Debian
testing/unstable" or similarly described machines. I have kicked back
against these, as the description seems to me to be far too open
ended.

Then again, it would be useful to actually test on Debian
testing/unstable (or pre-release branches of other OS), because that
would (a) expose problems with new toolchains and such earlier than
in released versions, and (b) provide advance testing for when
testing becomes the release. Consider, the number of users that
will run 8.2 on Debian stable is probably going to be less than the
number of users who will run 8.2 on what today is testing.

I suppose I'm partly to blame for bringing this one up; I proposed
doing a build on a box at home and at work (one IA-32, one AMD-64),
both running something of a mix of Debian unstable/testing.

I agree that the lack of a fixed version designation is unsatisfactory.
I'm not sure whether that is actually necessary, though. If PostgreSQL
doesn't work on some machine, then that's a problem anyway.

^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Yeah. There are a somewhat limited number of sorts of problems that
could cause this:

- I could be running an icky, out of date set of libraries or such.

In which case there's not much value in trying to fix things on the
PostgreSQL side.

- A change perhaps forthcoming in some future Debian release has
busted something.

Knowing that sooner is better than knowing that later...

- Such a problem may be ephermal; it may be something I need to report
to someone _else_ upstream.

For instance, a bleeding-edge readline may have just "drawn blood,"
and I need to let them know...

It seems to me that there is some value in putting together a script
that tries to identify some of the interesting bits of the toolchain.

Here's a "Mark 0" version:

================================================
#!/bin/sh
FINDGCCVERSION=`$CC --version`
KERNELVERSION=`uname -a`
ARCH=`arch`

# Uncomment one of the following...

#DEBIAN
#LIBC=`dpkg -l libc6`

#REDHAT/SuSE/Fedora
#LIBC=`rpm -q libc6`

#Slackware, etc
#LIBC=`ls -l /lib/libc.so.*`
================================================

I'll bet one could get more information pretty cheaply, including
release names, and such, by recognizing the existence of
distribution-specific files in /etc, such as /etc/debian_release,
/etc/SuSE-release, /etc/fedora-release, /etc/redhat-release....

It's not necessarily vital to recognize every combination of anything
that's plausible; if someone has an oddball system, reporting the
details are their problem...
--
output = ("cbbrowne" "@" "linuxfinances.info")
http://linuxdatabases.info/info/x.html
"Natives who beat drums to drive off evil spirits are objects of scorn
to smart Americans who blow horns to break up traffic jams."
-- Unknown

Christopher Browne wrote:

It seems to me that there is some value in putting together a script
that tries to identify some of the interesting bits of the toolchain.

Yeah; but why not just a bunch of commands, some of which are expected
to work on any particular machine, and save the whole output as a single
string? It's not very clean, but should get the important details. To
support a "new machine," just add more commands to the script.

A simple version of this, based on your Mark 0, could be:

uname -a
$CC --version
$CC -V
$CC -v
ls -l /lib/libc.so*

No need to comment/uncomment anything.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Alvaro Herrera <alvherre@commandprompt.com> writes:

Christopher Browne wrote:

It seems to me that there is some value in putting together a script
that tries to identify some of the interesting bits of the toolchain.

Yeah; but why not just a bunch of commands, some of which are expected
to work on any particular machine, and save the whole output as a single
string? It's not very clean, but should get the important details. To
support a "new machine," just add more commands to the script.

A simple version of this, based on your Mark 0, could be:

uname -a
$CC --version
$CC -V
$CC -v
ls -l /lib/libc.so*

No need to comment/uncomment anything.

I would have said "ldd postgres" would work on any ELF system and show you all
the library so versions it depends on. I guess that only helps if it actually
builds and then fails the regression tests -- not if the build fails.

On Debian it would be useful to do something like below. Though note that a)
this depends on having a postgres package installed which the build machines
may not have and b) it shows the libraries that package depends on not the
versions of the *-dev packages installed.

stark@stark:~$ reportbug --offline --template postgresql-8.1 2>/dev/null | sed '1,/^-- System Information/d'
Debian Release: testing/unstable
APT prefers unstable
APT policy: (500, 'unstable')
Architecture: i386 (i686)
Shell: /bin/sh linked to /bin/bash
Kernel: Linux 2.6.17.7-swsusp2
Locale: LANG=en_GB, LC_CTYPE=en_GB (charmap=ISO-8859-1)

Versions of packages postgresql-8.1 depends on:
ii libc6 2.3.6.ds1-4 GNU C Library: Shared libraries
ii libcomerr2 1.39-1 common error description library
ii libkrb53 1.4.3-8 MIT Kerberos runtime libraries
ii libpam0g 0.79-3.1 Pluggable Authentication Modules l
ii libpq4 8.1.4-5 PostgreSQL C client library
ii libssl0.9.8 0.9.8b-2 SSL shared libraries
ii postgresql-client-8.1 8.1.4-4 front-end programs for PostgreSQL
ii postgresql-common 57 manager for PostgreSQL database cl

postgresql-8.1 recommends no packages.

-- no debconf information

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com