show precise repos version for dev builds?

Started by Fabien COELHOover 8 years ago15 messages
#1Fabien COELHO
coelho@cri.ensmp.fr

Hello,

I use postgres "11devel" packages kindly distributed on
"apt.postgresql.org" and recompiled every few hours.

I wanted to know which version it was, and "11devel" is kind of imprecise.

Ok, there is a hint in the deb package name:

11~~devel~20170930.2214-1~87.git2632bcc.pgdg16.04+1

But this information does not seem to be available from the executables
themselves:

sh> psql --version
psql (PostgreSQL) 11devel

sh> pgbench --version
pgbench (PostgreSQL) 9.6.5

Some projects provide a repository or build date information:

sh> clang --version
clang version 6.0.0 (trunk 314585)

sh> gcc --version
gcc (GCC) 8.0.0 20170930 (experimental)

With some svn project I use "svnversion" which shows a summary of the
status, eg "5432M" which tells that the sources are locally modified
from version 5432.

I would find it useful to have something like that in pg as well, and I
have not found it available.

ISTM that extending the version name with the commit id and or date in
some version output, eg "11devel [2632bcc 2017-09-30 ...]", would do it.

Thoughts?

--
Fabien.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Fabien COELHO (#1)
Re: show precise repos version for dev builds?

Fabien COELHO <coelho@cri.ensmp.fr> writes:

I wanted to know which version it was, and "11devel" is kind of imprecise.
...
ISTM that extending the version name with the commit id and or date in
some version output, eg "11devel [2632bcc 2017-09-30 ...]", would do it.

configure --with-extra-version=whateveryouwant

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Tom Lane (#2)
Re: show precise repos version for dev builds?

Hello,

Fabien COELHO <coelho@cri.ensmp.fr> writes:

I wanted to know which version it was, and "11devel" is kind of imprecise.
...
ISTM that extending the version name with the commit id and or date in
some version output, eg "11devel [2632bcc 2017-09-30 ...]", would do it.

configure --with-extra-version=whateveryouwant

Thanks for the pointer!

So now I have to convince the apt.postgresql.org people to build the devel
version with this trick.

--
Fabien.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4Jeremy Schneider
schneider@ardentperf.com
In reply to: Tom Lane (#2)
Re: show precise repos version for dev builds?

On Sun, Oct 1, 2017 at 8:10 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

configure --with-extra-version=whateveryouwant

I see that this build option has been around since 9.4; is anyone
using it to mark patched production builds? EnterpriseDB or
2ndQuadrant? How about the cloud providers?

-Jeremy

--
http://about.me/jeremy_schneider

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#5Craig Ringer
craig@2ndquadrant.com
In reply to: Jeremy Schneider (#4)
Re: show precise repos version for dev builds?

On 11 October 2017 at 11:44, Jeremy Schneider <schneider@ardentperf.com> wrote:

On Sun, Oct 1, 2017 at 8:10 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

configure --with-extra-version=whateveryouwant

I see that this build option has been around since 9.4; is anyone
using it to mark patched production builds? EnterpriseDB or
2ndQuadrant? How about the cloud providers?

We started using it for BDR, but unfortunately too much software
explodes spectacularly when you use it, due to simplistic/buggy
version parsing.

This led me to push for wider adoption of server_version_num - in
particular, exposing it as an initial connection parameter via
GUC_REPORT, and exposing it in pg_config. After a few attempts I've
given up on that as a lost cause now, though I still disagree with the
rationale.

Right now, if you use it, various 3rd party software will fail in a
variety of exciting ways, some more subtle than others.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6Jeremy Schneider
schneider@ardentperf.com
In reply to: Craig Ringer (#5)
Re: show precise repos version for dev builds?

On Wed, Oct 11, 2017 at 1:19 AM, Craig Ringer <craig@2ndquadrant.com> wrote:

We started using it for BDR, but unfortunately too much software
explodes spectacularly when you use it, due to simplistic/buggy
version parsing.

Since 10.0 will break most of that software anyway, maybe this is a
good time to revisit the question? :)

-J

--
http://about.me/jeremy_schneider

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#7Peter Eisentraut
peter.eisentraut@2ndquadrant.com
In reply to: Craig Ringer (#5)
Re: show precise repos version for dev builds?

On 10/11/17 04:19, Craig Ringer wrote:

On 11 October 2017 at 11:44, Jeremy Schneider <schneider@ardentperf.com> wrote:

On Sun, Oct 1, 2017 at 8:10 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

configure --with-extra-version=whateveryouwant

I see that this build option has been around since 9.4; is anyone
using it to mark patched production builds? EnterpriseDB or
2ndQuadrant? How about the cloud providers?

We started using it for BDR, but unfortunately too much software
explodes spectacularly when you use it, due to simplistic/buggy
version parsing.

I've been using

--with-extra-version=+git`date +%Y%m%d`"~"`git rev-parse --short HEAD`

for my local builds for some time, and I've not experienced any such
problems.

However, using the various numeric reporting options is clearly better
if you want to do version comparisons. The "extra version" stuff should
be mainly for labeling.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#8Craig Ringer
craig@2ndquadrant.com
In reply to: Peter Eisentraut (#7)
Re: show precise repos version for dev builds?

On 12 October 2017 at 06:46, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:

I've been using

--with-extra-version=+git`date +%Y%m%d`"~"`git rev-parse --short HEAD`

for my local builds for some time, and I've not experienced any such
problems.

Interesting.

I've seen issues with a number of tools. The one I can remember most
clearly is check_postgres.pl . Nobody's going to argue that this is
pretty code, but last time I tested (9.4-era, admittedly) it exploded
messily with extra-version.

However, using the various numeric reporting options is clearly better
if you want to do version comparisons. The "extra version" stuff should
be mainly for labeling.

The trouble there is that we lack numeric version in some (IMO
crucial) places where we only have the string version:

- In the startup packet. We send server_version, but not
server_version_num, as GUC_REPORT. So if a client wants
server_version_num it has to do another round trip to query for it.

- In pg_config, where we don't expose any --version-num only --version.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#9Tom Lane
tgl@sss.pgh.pa.us
In reply to: Craig Ringer (#8)
Re: show precise repos version for dev builds?

Craig Ringer <craig@2ndquadrant.com> writes:

On 12 October 2017 at 06:46, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:

I've been using
--with-extra-version=+git`date +%Y%m%d`"~"`git rev-parse --short HEAD`
for my local builds for some time, and I've not experienced any such
problems.

I've seen issues with a number of tools. The one I can remember most
clearly is check_postgres.pl . Nobody's going to argue that this is
pretty code, but last time I tested (9.4-era, admittedly) it exploded
messily with extra-version.

FWIW, Salesforce tried to do something similar to Peter's example
while I was there. It did not work as well as we'd hoped :-( because
what got baked into the built executables was the latest git commit hash
as of the time you'd last run configure, not what was current as of the
latest "make". Not to mention that you might have built from an
uncommitted state. We tried to find a fix for the former problem that
didn't create lots of overhead, without much success. No idea what
to do about the latter.

These are not big problems for package-building since you basically
never distribute executables that don't get built from a clean state.
But they put a crimp in the usefulness of the idea for developers.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#10Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Tom Lane (#9)
Re: show precise repos version for dev builds?

Hello Tom,

I've seen issues with a number of tools. The one I can remember most
clearly is check_postgres.pl . Nobody's going to argue that this is
pretty code, but last time I tested (9.4-era, admittedly) it exploded
messily with extra-version.

FWIW, Salesforce tried to do something similar to Peter's example
while I was there. It did not work as well as we'd hoped :-( because
what got baked into the built executables was the latest git commit hash
as of the time you'd last run configure, not what was current as of the
latest "make". Not to mention that you might have built from an
uncommitted state. We tried to find a fix for the former problem that
didn't create lots of overhead, without much success.

My 0.02ᅵ:

For a research project we regenerate a header file with a string
containing the working copy status information,

// file version.h
#define REV "<version-output>"

and there is a very small C file which is recompiled with a constant
string based on the version:

// version.c
#include "version.h"
const char * version = REV;

The make dependencies ensure that the header file is regenerated on each
build with a phony target, and the C file is thus recompiled and linked
into the executables on each build. It means that all executables are
linked on each rebuild, even if not necessary, though.

No idea what to do about the latter.

"svnversion" adds a "M" for modified on the status. There is an option
with "git describe" to get something similar:

git describe --long --always --all --dirty

Also there is a need of a fall back if this fails, to get "<unknown
version>" instead.

--
Fabien.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#11Andres Freund
andres@anarazel.de
In reply to: Fabien COELHO (#10)
Re: show precise repos version for dev builds?

Hi,

On 2017-10-12 22:50:47 +0200, Fabien COELHO wrote:

The make dependencies ensure that the header file is regenerated on each
build with a phony target, and the C file is thus recompiled and linked into
the executables on each build. It means that all executables are linked on
each rebuild, even if not necessary, though.

That'd be quite painful, consider e.g. people using LTO. If done right
however, that should be avoidable to some degree. When creating the
version file, only replace its contents if the contents differ - that's
just a few lines of scripting.

Greetings,

Andres Freund

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#12Fabien COELHO
fabien.coelho@mines-paristech.fr
In reply to: Andres Freund (#11)
Re: show precise repos version for dev builds?

The make dependencies ensure that the header file is regenerated on each
build with a phony target, and the C file is thus recompiled and linked into
the executables on each build. It means that all executables are linked on
each rebuild, even if not necessary, though.

That'd be quite painful, consider e.g. people using LTO. If done right
however, that should be avoidable to some degree. When creating the
version file, only replace its contents if the contents differ - that's
just a few lines of scripting.

Indeed.

A potential issue is with dynamic linking, potentially someone could
recompile/reinstall just one shared object or dll, and the executable
using the lib would change its behavior, and run with libs from
heterogeneous version. What is the actual version? Hard to say.

In dev mode we often use static linking so that we can copy the executable
for a previous version and it would not change depending on updated libs,
and so that we always know (or should know) what actual version is
running.

--
Fabien.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#13Robert Haas
robertmhaas@gmail.com
In reply to: Fabien COELHO (#10)
Re: show precise repos version for dev builds?

On Thu, Oct 12, 2017 at 4:50 PM, Fabien COELHO <coelho@cri.ensmp.fr> wrote:

"svnversion" adds a "M" for modified on the status. There is an option with
"git describe" to get something similar:

git describe --long --always --all --dirty

I tried this out a bit:

[rhaas pgsql]$ git describe --long --always --all --dirty
heads/master-0-gd133982d59
[rhaas pgsql]$ git checkout head~
...blah blah...
[rhaas pgsql]$ git describe --long --always --all --dirty
heads/x-218-g6393613b6a

Eh, what? Hmm, maybe it's because I have a local branch called 'x'
lying around from something or other.

[rhaas pgsql]$ git branch -D x
Deleted branch x (was 77b6b5e9ce).
[rhaas pgsql]$ git describe --long --always --all --dirty
tags/REL_10_BETA3-458-g6393613b6a

Mmph. I understand the desire to identify the exact commit used for a
build somehow, but something whose output depends on whether or not I
left a branch lying around locally doesn't seem that great.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#14Michael Paquier
michael.paquier@gmail.com
In reply to: Robert Haas (#13)
Re: show precise repos version for dev builds?

On Sat, Oct 14, 2017 at 4:47 AM, Robert Haas <robertmhaas@gmail.com> wrote:

Mmph. I understand the desire to identify the exact commit used for a
build somehow, but something whose output depends on whether or not I
left a branch lying around locally doesn't seem that great.

Similarly to Peter, I prefer a minimum amount of information so I tend
to just use `git rev-parse --short HEAD` with --extra-version for my
own builds. Looking at the timestamp of the files installed is enough
to know when you worked on them, and when testing a patch and
committing it on a local branch before compiling you can know easily
where you left things off. git branch --contains is also useful to get
from which branch is commit from.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#15Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Robert Haas (#13)
Re: show precise repos version for dev builds?

Hello Robert,

Mmph. I understand the desire to identify the exact commit used for a
build somehow, but something whose output depends on whether or not I
left a branch lying around locally doesn't seem that great.

Indeed, the branch/tag search might have a little strange behavior.

Probably you would be more happy with just the commit (--always) & knowing
that it was changed (--dirty).

sh> git describe --always --dirty
b81eba6

sh> vi README
# edit

sh> git describe --always --dirty
b81eba6-dirty

--
Fabien.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers