OS X El Capitan and DYLD_LIBRARY_PATH

Started by Peter Eisentrautabout 10 years ago10 messages

peter_e@gmx.net

about 10 years ago

The new OS X release 10.11 "El Capitan" has a "security" feature that
prevents passing DYLD_LIBRARY_PATH to child processes. Somehow, that
variable is stripped from the environment.

Consequently, the current in-tree "make check" test setup will no longer
work, because programs such as initdb and psql that are called several
layers down will no longer be pointed to the right libraries. If you
run "make install" before "make check", it will work; otherwise
something will either fail because it can't find the libraries, or it
might pick up libraries found somewhere else, with poor outcomes, in my
experience.

The exact specifications of this new behavior aren't clear to me yet.
For example, this C program works just fine:

```
#include <stdio.h>
#include <stdlib.h>

int
main()
{
printf("DYLD_LIBRARY_PATH = %s\n", getenv("DYLD_LIBRARY_PATH"));
return 0;
}
```

but this shell script does not:

```
echo "DYLD_LIBRARY_PATH = $DYLD_LIBRARY_PATH"
```

There is breakage all over the Internet if you search for this, but the
full details don't appear to be very clear.

One workaround is to disable System Integrity Protection, effectively
restoring the behavior of the previous OS release.

Or don't upgrade quite yet if you don't want to deal with this at the
moment.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andres Freund

andres@anarazel.de

about 10 years ago

In reply to: Peter Eisentraut (#1)

Re: OS X El Capitan and DYLD_LIBRARY_PATH

Hi,

On 2015-10-14 11:24:27 -0400, Peter Eisentraut wrote:

The new OS X release 10.11 "El Capitan" has a "security" feature that
prevents passing DYLD_LIBRARY_PATH to child processes. Somehow, that
variable is stripped from the environment.

Two colleagues of mine at Citus just hit that.

The exact specifications of this new behavior aren't clear to me yet.
For example, this C program works just fine:

```
#include <stdio.h>
#include <stdlib.h>

int
main()
{
printf("DYLD_LIBRARY_PATH = %s\n", getenv("DYLD_LIBRARY_PATH"));
return 0;
}
```

but this shell script does not:

```
echo "DYLD_LIBRARY_PATH = $DYLD_LIBRARY_PATH"
```

There is breakage all over the Internet if you search for this, but the
full details don't appear to be very clear.

One workaround is to disable System Integrity Protection, effectively
restoring the behavior of the previous OS release.

Or don't upgrade quite yet if you don't want to deal with this at the
moment.

Apparently the behaviour is that DYLD_LIBRARY_PATH is prevented from
being inherited into any system binaries. E.g. a shell. But specifying
it inside a shell script will allow it to be inherited to children of
that shell, unless the child is a protected process in turn.

I wonder if we could fix this by using install_name_tool during the
tempinstall to add an appropriate rpath.

Alternatively we could, apparently, specify a relative path to libraries
as explained here:
http://qin.laya.com/tech_coding_help/dylib_linking.html
% install_name_tool -change libbz2.1.0.2.dylib @executable_path/../Frameworks/libbz2.1.0.2.dylib MyFunBinary

which ought to work independently from the tempinstall and normal
installation path.

Since I'm not a mac user I won't be able to check this out myself.

Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Peter Eisentraut

peter_e@gmx.net

about 10 years ago

In reply to: Andres Freund (#2)

Re: OS X El Capitan and DYLD_LIBRARY_PATH

On 11/3/15 6:36 AM, Andres Freund wrote:

I wonder if we could fix this by using install_name_tool during the
tempinstall to add an appropriate rpath.

Alternatively we could, apparently, specify a relative path to libraries
as explained here:
http://qin.laya.com/tech_coding_help/dylib_linking.html
% install_name_tool -change libbz2.1.0.2.dylib @executable_path/../Frameworks/libbz2.1.0.2.dylib MyFunBinary

which ought to work independently from the tempinstall and normal
installation path.

That might be worth a try. I ended up disabling system integrity
protection, which also fixed a few other strange behaviors (mysterious
regression test failures in ecpg, for instance, if anyone stumbles
across that).

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Michael Paquier

michael.paquier@gmail.com

about 10 years ago

In reply to: Peter Eisentraut (#3)

Re: OS X El Capitan and DYLD_LIBRARY_PATH

On Thu, Nov 5, 2015 at 12:17 PM, Peter Eisentraut <peter_e@gmx.net> wrote:

On 11/3/15 6:36 AM, Andres Freund wrote:

I wonder if we could fix this by using install_name_tool during the
tempinstall to add an appropriate rpath.

Alternatively we could, apparently, specify a relative path to libraries
as explained here:
http://qin.laya.com/tech_coding_help/dylib_linking.html
% install_name_tool -change libbz2.1.0.2.dylib @executable_path/../Frameworks/libbz2.1.0.2.dylib MyFunBinary

which ought to work independently from the tempinstall and normal
installation path.

That might be worth a try. I ended up disabling system integrity
protection, which also fixed a few other strange behaviors (mysterious
regression test failures in ecpg, for instance, if anyone stumbles
across that).

Yeah, that's wiser IMO. I would expect at the end people doing some
serious development work to disable SIP at the end, this is known as
well to cause issues with homebrew for example.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Tom Lane

tgl@sss.pgh.pa.us

about 10 years ago

In reply to: Michael Paquier (#4)

Re: OS X El Capitan and DYLD_LIBRARY_PATH

Michael Paquier <michael.paquier@gmail.com> writes:

On Thu, Nov 5, 2015 at 12:17 PM, Peter Eisentraut <peter_e@gmx.net> wrote:

That might be worth a try. I ended up disabling system integrity
protection, which also fixed a few other strange behaviors (mysterious
regression test failures in ecpg, for instance, if anyone stumbles
across that).

Yeah, that's wiser IMO. I would expect at the end people doing some
serious development work to disable SIP at the end, this is known as
well to cause issues with homebrew for example.

I think we should all file bugs telling Apple it's insane to suppress
DYLD_LIBRARY_PATH across a shell invocation. I can see the point of
it for some other system binaries, but not sh.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Michael Paquier

michael.paquier@gmail.com

about 10 years ago

In reply to: Tom Lane (#5)

Re: OS X El Capitan and DYLD_LIBRARY_PATH

On Thu, Nov 5, 2015 at 1:16 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Michael Paquier <michael.paquier@gmail.com> writes:

On Thu, Nov 5, 2015 at 12:17 PM, Peter Eisentraut <peter_e@gmx.net> wrote:

That might be worth a try. I ended up disabling system integrity
protection, which also fixed a few other strange behaviors (mysterious
regression test failures in ecpg, for instance, if anyone stumbles
across that).

Yeah, that's wiser IMO. I would expect at the end people doing some
serious development work to disable SIP at the end, this is known as
well to cause issues with homebrew for example.

I think we should all file bugs telling Apple it's insane to suppress
DYLD_LIBRARY_PATH across a shell invocation. I can see the point of
it for some other system binaries, but not sh.

There is:
http://openradar.appspot.com/22807197

And it seems that this is intended policy because python, perl, bash,
etc processes are running as protected processes even if there script
is not, and all the dldy variables are getting purged:
https://forums.developer.apple.com/thread/13161
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andres Freund

andres@anarazel.de

about 10 years ago

In reply to: Michael Paquier (#4)

Re: OS X El Capitan and DYLD_LIBRARY_PATH

On 2015-11-05 12:51:48 +0900, Michael Paquier wrote:

On Thu, Nov 5, 2015 at 12:17 PM, Peter Eisentraut <peter_e@gmx.net> wrote:

On 11/3/15 6:36 AM, Andres Freund wrote:

I wonder if we could fix this by using install_name_tool during the
tempinstall to add an appropriate rpath.

Alternatively we could, apparently, specify a relative path to libraries
as explained here:
http://qin.laya.com/tech_coding_help/dylib_linking.html
% install_name_tool -change libbz2.1.0.2.dylib @executable_path/../Frameworks/libbz2.1.0.2.dylib MyFunBinary

which ought to work independently from the tempinstall and normal
installation path.

That might be worth a try.

It'll probably not me though, given that I scream bloody murder
everytime I've to use OSX...

I ended up disabling system integrity
protection, which also fixed a few other strange behaviors (mysterious
regression test failures in ecpg, for instance, if anyone stumbles
across that).

Yeah, that's wiser IMO.

That's probably true in the short term. I doubt it'll be viable in the
long run. Both because apple obviously doesn't care one iota about the
developer experience around this, and thus probably has a limited
interest keeping related procedures working the same, and because once
more people use El Captan it'll have to be done by more and more
people.

Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

David E. Wheeler

david@justatheory.com

about 10 years ago

In reply to: Michael Paquier (#6)

1 attachment(s)

Re: OS X El Capitan and DYLD_LIBRARY_PATH

On Nov 4, 2015, at 8:37 PM, Michael Paquier <michael.paquier@gmail.com> wrote:

There is:
http://openradar.appspot.com/22807197

Yep, I filed that because I was unable to build the DBD::Oracle Perl module, since I can’t tell it where to find the SQL*Plus libraries. Big PITA.

Apple says that the more people file bugs, the more likely the issue is to get attention. So by all means, please file radars about this. You can reference 21732670 as the bug you’re duping (they marked mine as a dupe for that one).

https://http://bugreport.apple.com/

Best,

David

Attachments:

smime.p7sapplication/pkcs7-signature; name=smime.p7sDownload

0�	*�H��
��0�10	+0�	*�H��
��i0�-0��Q�0
	*�H��
0��10	UIL10U

StartCom Ltd.1+0)U"Secure Digital Certificate Signing1806U/StartCom Class 1 Primary Intermediate Client CA0
150909050437Z
160909073817Z0F10Udavid@justatheory.com1$0"	*�H��
	david@justatheory.com0�"0
	*�H��
�0�
��zg���P����������:�BV��h/�a�UcTD��L������V�9
��&��2��O*@@�g����E�����!�Ta��G�����R�B�����RI�\g��K��.7��������8��,dr��B��}���F��)�y��n��38�]$D) ��g�'
Hz�"�������`EVl�{<���q�h��A�-���=��:�m���nlxsN~�����jo���s�FV������1IB8p�1���0��0	U00U�0U%0++0U���D�!���<�rV6
���0U#0�Sr������\|~�5N���Q�0 U0�david@justatheory.com0�LU �C0�?0�;+��70�*0.+"http://www.startssl.com/policy.pdf0��+0��0' StartCom Certification Authority0��This certificate was issued according to the Class 1 Validation requirements of the StartCom CA policy, reliance only for the intended purpose in compliance of the relying party obligations.06U/0-0+�)�'�%http://crl.startssl.com/crtu1-crl.crl0��+��009+0�-http://ocsp.startssl.com/sub/class1/client/ca0B+0�6http://aia.startssl.com/certs/sub.class1.client.ca.crt0#U0�http://www.startssl.com/0
	*�H��
�r��������.��;�-a�����������L��O��4���}��F�,^��Y�,�'�d���7:�LR|��s=�9v|\K	��C�@�Y����VW�h Ev����M6��I�oIJ!y����l_����������3������V	[�\�nsD����|�~u����P��Tn�	C��_V!�{A�>��;g���^��,�a�d�X�T��vlO7��L��S>���N*i]m���R�tn����/k0�40��0
	*�H��
0}10	UIL10U

StartCom Ltd.1+0)U"Secure Digital Certificate Signing1)0'U StartCom Certification Authority0
071024210155Z
171024210155Z0��10	UIL10U

StartCom Ltd.1+0)U"Secure Digital Certificate Signing1806U/StartCom Class 1 Primary Intermediate Client CA0�"0
	*�H��
�0�
��	���-��)�.����2����A�UG��o���#G�
��B|N�D�����Rp�M-��B��=o���-�we�5��J�Qpa>O��.�#������.���_�<���V��
[~�*��*�p�z��~�3�W�G�.�������Ml�r[�<C�e�6���f����q������O���"��u��xf�WN�#�u����i���c�gk��v$����Lb�%�������y��`�����_�{`���xK'G�N������0��0U�0�0U�0USr������\|~�5N���Q�0U#0�N��@[�i�0�4hC�A��0f+Z0X0'+0�http://ocsp.startssl.com/ca0-+0�!http://www.startssl.com/sfsca.crt0[UT0R0'�%�#�!http://www.startssl.com/sfsca.crl0'�%�#�!http://crl.startssl.com/sfsca.crl0��U y0w0u+��70f0.+"http://www.startssl.com/policy.pdf04+(http://www.startssl.com/intermediate.pdf0
	*�H��
�
�}x�,\�c�^��#wM�q�}��>UK/��^y��X��y	�����f�rMI���B6�1ymQ���������Z���0���&��;�@��#13q����&	����������o�	6�r��_��;�GO>*I�(	7�4����XS1r3��)!����y��6Ko����t��#
_�w�S�r����
�;�B
A�Dp�(f��s����������6%�����.W0J3�:b�C�<�8t X����1�<��C��n�=�����t==�wS���T������~���\�wkB�f�|1���5���zU��P)��(���I��j��VB��!����OfI=b��b�\4�-*em��/��SJm�7���N�����[�]'��@����D9�Kr>���R��7/����|�o���^I@���'��Pa$ z��9�a'L�)��(�
�I��}v��c�H]����D����*��W�}
m�>Q����|�C.�(,�l��Q�1�o0�k0��0��10	UIL10U

StartCom Ltd.1+0)U"Secure Digital Certificate Signing1806U/StartCom Class 1 Primary Intermediate Client CAQ�0	+���0	*�H��
	1	*�H��
0	*�H��
	1
151107220326Z0#	*�H��
	1��H[]���]�,P@B�s��0��	+�71��0��0��10	UIL10U

StartCom Ltd.1+0)U"Secure Digital Certificate Signing1806U/StartCom Class 1 Primary Intermediate Client CAQ�0��*�H��
	1�����0��10	UIL10U

StartCom Ltd.1+0)U"Secure Digital Certificate Signing1806U/StartCom Class 1 Primary Intermediate Client CAQ�0
	*�H��
��������Ky�br��������D��������JN�d�<��

������]
1��2|04�������B�����r#�q#�(�~rDQAe����dj���L��`��K�����\W��l��h��������.=��pRH"_�}��:Fr\^"^��W�����UW@���<���lK3�>J���2��?�c�~�%��"|�e4��`�\�����[P.���6��8��Fe��P�mnvL�B*����Le

Tom Lane

tgl@sss.pgh.pa.us

about 10 years ago

In reply to: David E. Wheeler (#8)

Re: OS X El Capitan and DYLD_LIBRARY_PATH

"David E. Wheeler" <david@justatheory.com> writes:

Apple says that the more people file bugs, the more likely the issue is to get attention. So by all means, please file radars about this. You can reference 21732670 as the bug you’re duping (they marked mine as a dupe for that one).

Done here.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#10

Michael Paquier

michael.paquier@gmail.com

about 10 years ago

In reply to: Tom Lane (#9)

Re: OS X El Capitan and DYLD_LIBRARY_PATH

On Sun, Nov 8, 2015 at 7:26 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

"David E. Wheeler" <david@justatheory.com> writes:

Apple says that the more people file bugs, the more likely the issue is to get attention. So by all means, please file radars about this. You can reference 21732670 as the bug you’re duping (they marked mine as a dupe for that one).

Done here.

Let's be noisy.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers