meson vs windows perl

On 2024-04-02 Tu 09:34, Andrew Dunstan wrote:

meson.build has this code

    ldopts = run_command(perl, '-MExtUtils::Embed', '-e',
'ldopts', check: true).stdout().strip()     undesired =
run_command(perl_conf_cmd, 'ccdlflags', check:
true).stdout().split()     undesired += run_command(perl_conf_cmd,
'ldflags', check: true).stdout().split()     perl_ldopts = []    
foreach ldopt : ldopts.split(' ')       if ldopt == '' or ldopt in
undesired         continue       endif       perl_ldopts +=
ldopt.strip('"')     endforeach     message('LDFLAGS recommended
by perl: "@0@"'.format(ldopts))     message('LDFLAGS for embedding
perl: "@0@"'.format(' '.join(perl_ldopts)))

This code is seriously broken if perl reports items including spaces,
when a) removing the quotes is quite wrong, and b) splitting on spaces
is also wrong.

Here's an example from one of my colleagues:

C:\Program Files\Microsoft Visual Studio\2022\Professional>perl.EXE -MExtUtils::Embed -e ldopts
-nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"C:\edb\languagepack\v4\Perl-5.38\lib\CORE"
-machine:AMD64 -subsystem:console,"5.02" "C:\edb\languagepack\v4\Perl-5.38\lib\CORE\perl538.lib"
"C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.38.33130\lib\x64\oldnames.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\kernel32.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\user32.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\gdi32.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\winspool.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\comdlg32.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\advapi32.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\shell32.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\ole32.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\oleaut32.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\netapi32.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\uuid.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\ws2_32.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\mpr.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\winmm.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\version.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\odbc32.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\odbccp32.lib"
"C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64\comctl32.lib"
"C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.38.33130\lib\x64\msvcrt.lib"
"C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.38.33130\lib\x64\vcruntime.lib"
"C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64\ucrt.lib"

And with that we get errors like

cl : Command line warning D9024 : unrecognized source file type 'C:\Program', object file assumed
cl : Command line warning D9024 : unrecognized source file type 'Files\Microsoft', object file assumed
cl : Command line warning D9024 : unrecognized source file type 'Visual', object file assumed
cl : Command line warning D9024 : unrecognized source file type 'C:\Program', object file assumed
cl : Command line warning D9024 : unrecognized source file type 'Files', object file assumed
cl : Command line warning D9024 : unrecognized source file type '(x86)\Windows', object file assumed

It looks like we need to get smarter about how we process the ldopts and strip out the ccdlflags and ldflags

Here is an attempt to fix all that. It's ugly, but I think it's more
principled.

First, instead of getting the ldopts and then trying to filter out the
ldflags and ccdlflags, it tells perl not to include those in the first
place, by overriding a couple of routines in ExtUtils::Embed. And
second, it's smarter about splitting what's left, so that it doesn't
split on a space that's in a quoted item. The perl that's used to do
that second bit is not pretty, but it has been tested on the system
where the problem arose and apparently cures the problem. (No doubt some
perl guru could improve it.) It also works on my Ubuntu system, so I
don't think we'll be breaking anything (famous last words).

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

On 2024-04-05 Fr 08:25, Andrew Dunstan wrote:

Here is an attempt to fix all that. It's ugly, but I think it's more
principled.

First, instead of getting the ldopts and then trying to filter out the
ldflags and ccdlflags, it tells perl not to include those in the first
place, by overriding a couple of routines in ExtUtils::Embed. And
second, it's smarter about splitting what's left, so that it doesn't
split on a space that's in a quoted item. The perl that's used to do
that second bit is not pretty, but it has been tested on the system
where the problem arose and apparently cures the problem. (No doubt
some perl guru could improve it.) It also works on my Ubuntu system,
so I don't think we'll be breaking anything (famous last words).

Apparently I spoke too soon. Please ignore the above for now.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

On 2024-04-05 Fr 10:12, Andrew Dunstan wrote:

On 2024-04-05 Fr 08:25, Andrew Dunstan wrote:

Here is an attempt to fix all that. It's ugly, but I think it's more
principled.

First, instead of getting the ldopts and then trying to filter out
the ldflags and ccdlflags, it tells perl not to include those in the
first place, by overriding a couple of routines in ExtUtils::Embed.
And second, it's smarter about splitting what's left, so that it
doesn't split on a space that's in a quoted item. The perl that's
used to do that second bit is not pretty, but it has been tested on
the system where the problem arose and apparently cures the problem.
(No doubt some perl guru could improve it.) It also works on my
Ubuntu system, so I don't think we'll be breaking anything (famous
last words).

Apparently I spoke too soon. Please ignore the above for now.

OK, this has been fixed and checked. The attached is what I propose.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Hi,

On 2024-04-05 16:12:12 -0400, Andrew Dunstan wrote:

OK, this has been fixed and checked. The attached is what I propose.

The perl command is pretty hard to read. What about using python's shlex
module instead? Rough draft attached. Still not very pretty, but seems easier
to read?

It'd be even better if we could just get perl to print out the flags in an
easier to parse way, but I couldn't immediately see a way.

Greetings,

Andres Freund

On 2024-05-28 Tu 6:13 PM, Andres Freund wrote:

Hi,

On 2024-04-05 16:12:12 -0400, Andrew Dunstan wrote:

OK, this has been fixed and checked. The attached is what I propose.

The perl command is pretty hard to read. What about using python's shlex
module instead? Rough draft attached. Still not very pretty, but seems easier
to read?

It'd be even better if we could just get perl to print out the flags in an
easier to parse way, but I couldn't immediately see a way.

Thanks for looking.

The attached should be easier to read. The perl package similar to shlex
is Text::ParseWords, which is already a requirement.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

On 2024-07-20 Sa 9:41 AM, Andrew Dunstan wrote:

On 2024-05-28 Tu 6:13 PM, Andres Freund wrote:

Hi,

On 2024-04-05 16:12:12 -0400, Andrew Dunstan wrote:

OK, this has been fixed and checked. The attached is what I propose.

The perl command is pretty hard to read. What about using python's shlex
module instead? Rough draft attached.  Still not very pretty, but
seems easier
to read?

It'd be even better if we could just get perl to print out the flags
in an
easier to parse way, but I couldn't immediately see a way.

Thanks for looking.

The attached should be easier to read. The perl package similar to
shlex is Text::ParseWords, which is already a requirement.

It turns out that shellwords eats backslashes, so we would need
something like this version, which I have tested on Windows. I will
probably commit this in the next few days unless there's an objection.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

On 2024-07-30 Tu 3:47 PM, Andrew Dunstan wrote:

On 2024-07-20 Sa 9:41 AM, Andrew Dunstan wrote:

On 2024-05-28 Tu 6:13 PM, Andres Freund wrote:

Hi,

On 2024-04-05 16:12:12 -0400, Andrew Dunstan wrote:

OK, this has been fixed and checked. The attached is what I propose.

The perl command is pretty hard to read. What about using python's
shlex
module instead? Rough draft attached.  Still not very pretty, but
seems easier
to read?

It'd be even better if we could just get perl to print out the flags
in an
easier to parse way, but I couldn't immediately see a way.

Thanks for looking.

The attached should be easier to read. The perl package similar to
shlex is Text::ParseWords, which is already a requirement.

It turns out that shellwords eats backslashes, so we would need
something like this version, which I have tested on Windows. I will
probably commit this in the next few days unless there's an objection.

pushed

cheers

andrew

--

Andrew Dunstan
EDB: https://www.enterprisedb.com