Upgrading our minimum required flex version for 8.5
I'd like to return to the project I suggested here:
http://archives.postgresql.org/message-id/18653.1239741426@sss.pgh.pa.us
of getting rid of plpgsql's private lexer and having it use the core
lexer instead. This will require making the core lexer re-entrant,
which is not possible with our oldest supported version of flex
(2.5.4a). A look at the flex change history indicates that %reentrant
was added in 2.5.6 but nasty bugs were being fixed up to 2.5.30;
so the new minimum supported version would probably be 2.5.33
(2.5.31 was kinda broken for other reasons, and there was no 2.5.32).
Since 2.5.33 is now over three years old, this does not seem like an
onerous requirement, but I thought I'd better ask if anyone has an
objection?
regards, tom lane
Tom Lane wrote:
I'd like to return to the project I suggested here:
http://archives.postgresql.org/message-id/18653.1239741426@sss.pgh.pa.us
of getting rid of plpgsql's private lexer and having it use the core
lexer instead. This will require making the core lexer re-entrant,
which is not possible with our oldest supported version of flex
(2.5.4a). A look at the flex change history indicates that %reentrant
was added in 2.5.6 but nasty bugs were being fixed up to 2.5.30;
so the new minimum supported version would probably be 2.5.33
(2.5.31 was kinda broken for other reasons, and there was no 2.5.32).Since 2.5.33 is now over three years old, this does not seem like an
onerous requirement, but I thought I'd better ask if anyone has an
objection?
I think it would need to be benchmarked. My faint recollection is that
the re-entrant lexers are slower.
cheers
andrew
Tom Lane <tgl@sss.pgh.pa.us> wrote:
so the new minimum supported version would probably be 2.5.33
(2.5.31 was kinda broken for other reasons, and there was no
2.5.32).Since 2.5.33 is now over three years old, this does not seem like an
onerous requirement, but I thought I'd better ask if anyone has an
objection?
You'd be causing problems for SuSE Enterprise users, like us:
kgrittn@CCDEV-DB:/etc/init.d> cat /etc/SuSE-release
SUSE Linux Enterprise Server 10 (x86_64)
VERSION = 10
PATCHLEVEL = 2
kgrittn@CCDEV-DB:/etc/init.d> flex --version
flex 2.5.31
Just how broken is the version we're using?
-Kevin
"Kevin Grittner" <Kevin.Grittner@wicourts.gov> wrote:
Tom Lane <tgl@sss.pgh.pa.us> wrote:
so the new minimum supported version would probably be 2.5.33
We still have five SuSE Enterprise 9 boxes in use as database servers;
while we've got someone tasked with replacing them, I wonder if that
version is so dead that we want to abandon support for it.
kgrittn@INHOUSEAPPS:~> cat /etc/SuSE-release
SUSE LINUX Enterprise Server 9 (i586)
VERSION = 9
PATCHLEVEL = 3
kgrittn@INHOUSEAPPS:~> flex --version
flex version 2.5.4
Of course, one could always download source for a more recent version
and build and install flex that way. (We do that for PostgreSQL
itself and libpoppler.) If someone's willing to build PostgreSQL,
perhaps it's reasonable to require them to build flex first?
-Kevin
Andrew Dunstan <andrew@dunslane.net> writes:
I think it would need to be benchmarked. My faint recollection is that
the re-entrant lexers are slower.
The flex documentation states in so many words:
The option `--reentrant' does not affect the performance of the scanner.
Do you feel a need to verify their claim?
regards, tom lane
Tom Lane wrote:
Andrew Dunstan <andrew@dunslane.net> writes:
I think it would need to be benchmarked. My faint recollection is that
the re-entrant lexers are slower.The flex documentation states in so many words:
The option `--reentrant' does not affect the performance of the scanner.
Do you feel a need to verify their claim?
No, I'll take their word for it. I must have been thinking of something
else.
cheers
andrew
Kevin Grittner wrote:
Tom Lane <tgl@sss.pgh.pa.us> wrote:
so the new minimum supported version would probably be 2.5.33
(2.5.31 was kinda broken for other reasons, and there was no
2.5.32).Since 2.5.33 is now over three years old, this does not seem like an
onerous requirement, but I thought I'd better ask if anyone has an
objection?You'd be causing problems for SuSE Enterprise users, like us:
kgrittn@CCDEV-DB:/etc/init.d> cat /etc/SuSE-release
SUSE Linux Enterprise Server 10 (x86_64)
VERSION = 10
PATCHLEVEL = 2
kgrittn@CCDEV-DB:/etc/init.d> flex --version
flex 2.5.31
Keep in mind that if you build from a tarball, the lexer has already
been run elsewhere and you don't need flex. Since you do some
experimental playing from CVS I guess you could install flex from source
in your environment only ...
--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.
"Kevin Grittner" <Kevin.Grittner@wicourts.gov> writes:
kgrittn@CCDEV-DB:/etc/init.d> flex --version
flex 2.5.31
Just how broken is the version we're using?
You might want to take that up with SuSE. The flex NEWS file just cites
"numerous bug and security fixes" from .31 to .33.
Our CVS logs show half a dozen changes we made some years ago to make
our code work with 2.5.31, but I can't tell from the log entries whether
those were workarounds for bugs in flex or fixes for obsolete coding on
our side.
We could try setting the min version at 2.5.31, though, until we find
some specific reason it doesn't work for us. 2.5.31 is significantly
older (release date 2003-04-01, vs 2006-02-20), so there's definitely
more chance of people having it.
regards, tom lane
Alvaro Herrera <alvherre@commandprompt.com> writes:
Kevin Grittner wrote:
Tom Lane <tgl@sss.pgh.pa.us> wrote:
Since 2.5.33 is now over three years old, this does not seem like an
onerous requirement, but I thought I'd better ask if anyone has an
objection?You'd be causing problems for SuSE Enterprise users, like us:
Keep in mind that if you build from a tarball, the lexer has already
been run elsewhere and you don't need flex.
Right, this will only affect people doing development or otherwise
building from a CVS pull.
regards, tom lane
Tom Lane wrote:
Alvaro Herrera <alvherre@commandprompt.com> writes:
Kevin Grittner wrote:
Tom Lane <tgl@sss.pgh.pa.us> wrote:
Since 2.5.33 is now over three years old, this does not seem like an
onerous requirement, but I thought I'd better ask if anyone has an
objection?You'd be causing problems for SuSE Enterprise users, like us:
Keep in mind that if you build from a tarball, the lexer has already
been run elsewhere and you don't need flex.Right, this will only affect people doing development or otherwise
building from a CVS pull.
Of course, that includes the whole buildfarm. We might need to ask some
people to upgrade there.
cheers
andrew
Alvaro Herrera <alvherre@commandprompt.com> wrote:
Keep in mind that if you build from a tarball, the lexer has already
been run elsewhere and you don't need flex.
Does that hold for the daily snapshots? If so, I should be good. My
workstation is OK on flex version. (I run kubuntu on my desktop.)
-Kevin
Andrew Dunstan <andrew@dunslane.net> writes:
Tom Lane wrote:
Right, this will only affect people doing development or otherwise
building from a CVS pull.
Of course, that includes the whole buildfarm. We might need to ask some
people to upgrade there.
Yes. What I was thinking of doing was committing a configure change to
reject flex < 2.5.31, and waiting to see how much of the buildfarm goes
red.
One point here is that if buildfarm owners update to more recent flex,
that will also affect their testing of back branches. I think this
should be okay --- AFAICT from the CVS logs, all of the stuff we did to
be compatible with 2.5.31 was pre-7.4 and so all supported versions
should still work. But we'd find out for sure ;-)
regards, tom lane
PS: speaking of buildfarm, I don't see any REL8_4_STABLE entries yet.
Need to ping the owners anyway, apparently.
Kevin Grittner wrote:
Alvaro Herrera <alvherre@commandprompt.com> wrote:
Keep in mind that if you build from a tarball, the lexer has already
been run elsewhere and you don't need flex.Does that hold for the daily snapshots? If so, I should be good.
Yes, the snapshots have the derived files too.
--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Tom Lane wrote:
Right, this will only affect people doing development or otherwise
building from a CVS pull.
Someone doing a CVS pull already needs a specific recent version of
autoconf anyways. How old is this version of flex compared to the
version of autoconf we require?
On Thu, Jul 2, 2009 at 11:39 AM, Greg Stark<gsstark@mit.edu> wrote:
Tom Lane wrote:
Right, this will only affect people doing development or otherwise
building from a CVS pull.Someone doing a CVS pull already needs a specific recent version of
autoconf anyways. How old is this version of flex compared to the
version of autoconf we require?
Really? configure is in CVS, so I don't think you need autoconf
unless you want to modify configure.in. Which people may want to do,
but certainly not for every patch.
...Robert
Robert Haas <robertmhaas@gmail.com> writes:
On Thu, Jul 2, 2009 at 11:39 AM, Greg Stark<gsstark@mit.edu> wrote:
Someone doing a CVS pull already needs a specific recent version of
autoconf anyways. How old is this version of flex compared to the
version of autoconf we require?
Really? configure is in CVS, so I don't think you need autoconf
unless you want to modify configure.in.
Right. That's been debated on occasion in the past, but right now
only those who commit changes to configure.in (and related files)
need to have autoconf at all.
But to answer Greg's question: 8.4 and HEAD require autoconf 2.61
which was released 2006-11-17, or rather later than flex 2.5.33.
regards, tom lane
Alvaro Herrera <alvherre@commandprompt.com> wrote:
Yes, the snapshots have the derived files too.
Then I take it back -- the new flex versions would have little or no
impact on me. Worst case, I might need to download a snapshot to
apply my patch for testing on the "big" machines. If I understood
what make options I could use on my machine to create derived files
for other machines, even that would go away.
-Kevin
"Kevin Grittner" <Kevin.Grittner@wicourts.gov> writes:
Then I take it back -- the new flex versions would have little or no
impact on me. Worst case, I might need to download a snapshot to
apply my patch for testing on the "big" machines. If I understood
what make options I could use on my machine to create derived files
for other machines, even that would go away.
Peter or Marc could clue you in a bit better, but I think it's as simple
as saying "make dist" at the top level of a modified source tree. This
gets you a source tarball the same way the release tarballs are made.
regards, tom lane
Tom Lane <tgl@sss.pgh.pa.us> wrote:
Peter or Marc could clue you in a bit better, but I think it's as
simple as saying "make dist" at the top level of a modified source
tree. This gets you a source tarball the same way the release
tarballs are made.
I seem to be able to "sneak up on it from behind" by doing a regular
make and then "make distclean" and copying the results. Perhaps
someone knows off-hand what I'm missing that prevents "make dist" from
working. The attempt ends with:
make[3]: Entering directory
`/home/kgrittn/pg/pgsql/postgresql-8.5devel/contrib/xml2'
make[3]: Nothing to be done for `distprep'.
make[3]: Leaving directory
`/home/kgrittn/pg/pgsql/postgresql-8.5devel/contrib/xml2'
make[2]: Leaving directory
`/home/kgrittn/pg/pgsql/postgresql-8.5devel/contrib'
make[1]: Leaving directory
`/home/kgrittn/pg/pgsql/postgresql-8.5devel'
make -C postgresql-8.5devel/doc/src/sgml/ HISTORY INSTALL
regress_README
make[1]: Entering directory
`/home/kgrittn/pg/pgsql/postgresql-8.5devel/doc/src/sgml'
"/usr/bin/perl" generate_history.pl "." release.sgml
tempfile_HISTORY.sgml
jade -D . -d stylesheet.dsl -i output-text -t sgml -V nochunks
tempfile_HISTORY.sgml >HISTORY.html
/bin/sh: jade: not found
make[1]: *** [HISTORY.html] Error 127
make[1]: *** Deleting file `HISTORY.html'
make[1]: Leaving directory
`/home/kgrittn/pg/pgsql/postgresql-8.5devel/doc/src/sgml'
make: *** [distdir] Error 2
-Kevin
I wrote:
Yes. What I was thinking of doing was committing a configure change to
reject flex < 2.5.31, and waiting to see how much of the buildfarm goes
red.
Actually, most of the buildfarm members show which flex version they are
running in the configure output. A quick look shows that of the 45
members that have reported on HEAD in the past 2 days, 22 are running
2.5.4, which is a lot higher than I was expecting. Most of these are
the Solaris boxen, which I imagine can be updated fairly painlessly
since there are some of them that are already running something newer.
However I'm a bit worried about the situation for Windows --- does
anyone know whether a newer flex is readily available for Windows?
In any case it seems like it'd be prudent to prod the buildfarm owners
to update their flex *before* we pull the trigger...
regards, tom lane
