Build versionless .so for Android

Matthias Kuhn <matthias@opengis.ch> writes:

In order to ship libpq on Android, it needs to be built without a version
suffix on the .so file.
The attached patch will make sure that no version is added when built for
Android.

This ... seems incredibly brain-dead. Does Android really not cope
with version-to-version ABI changes?

regards, tom lane

On Sun, Dec 31, 2023 at 06:08:04AM +0100, Matthias Kuhn wrote:

In order to ship libpq on Android, it needs to be built without a version
suffix on the .so file.
The attached patch will make sure that no version is added when built for
Android.

FWIW, I have mixed feelings about patching the code to treat
android-linux as an exception while changing nothing in the
documentation to explain why this is required. A custom patch may
serve you better here, and note that you did not even touch the meson
paths. See libpq_c_args in src/interfaces/libpq/meson.build as one
example. That's just my opinion, of course, still there are no
buildfarm members that would cover your patch.

I was wondering if there are a) any comments on the approach and if I
should be handed in for a commitfest (currently waiting for the cooldown
period after account activation, I am not sure how long that is)

Such discussions are adapted in a commit fest entry. No idea if there
is a cooldown period after account creation before being able to touch
the CF app contents, though.
--
Michael

On 12/31/23 01:24, Michael Paquier wrote:

On Sun, Dec 31, 2023 at 06:08:04AM +0100, Matthias Kuhn wrote:

I was wondering if there are a) any comments on the approach and if I
should be handed in for a commitfest (currently waiting for the cooldown
period after account activation, I am not sure how long that is)

Such discussions are adapted in a commit fest entry. No idea if there
is a cooldown period after account creation before being able to touch
the CF app contents, though.

FWIW I have expedited the cooldown period, so Matthias can log in right
away.

--
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Thanks for all the comments and help.

I have added the patch to the January CF.

It looks like meson does not currently support building for android, the
following output is what I get (but I have actually no experience with
meson):

meson.build:320:2: ERROR: Problem encountered: unknown host system:
android

Please find an attached patch which also includes a documentation section.
I am happy to adjust if needed.

Kind regards
Matthias

On Sun, Dec 31, 2023 at 1:24 AM Michael Paquier <michael@paquier.xyz> wrote:

FWIW, I have mixed feelings about patching the code to treat
android-linux as an exception while changing nothing in the
documentation to explain why this is required. A custom patch may
serve you better here, and note that you did not even touch the meson
paths. See libpq_c_args in src/interfaces/libpq/meson.build as one
example. That's just my opinion, of course, still there are no
buildfarm members that would cover your patch.

It's reasonable to want good comments -- the one in the patch (1)
doesn't explain why this is required and (2) suggests that it is only
needed when cross-compiling which seems surprising and (3) has a typo.
But if it's true that this is needed, I tentatively think we might do
better to take the patch than force people to carry it out of tree.

--
Robert Haas
EDB: http://www.enterprisedb.com

Hi,

Attached a patch with a (hopefully) better wording of the comment.

I have unsuccessfully tried to find an official source for this policy.
So for reference some discussions about the topic:

-
https://stackoverflow.com/questions/11491065/linking-with-versioned-shared-library-in-android-ndk
-
https://stackoverflow.com/questions/18681401/how-can-i-remove-all-versioning-information-from-shared-object-files

Please let me know if I can help in any way

Matthias

On Tue, Jan 2, 2024 at 6:43 PM Robert Haas <robertmhaas@gmail.com> wrote:

On Thu, Jan 4, 2024 at 7:00 PM Matthias Kuhn <matthias@opengis.ch> wrote:

Attached a patch with a (hopefully) better wording of the comment.

I have unsuccessfully tried to find an official source for this policy.
So for reference some discussions about the topic:

- https://stackoverflow.com/questions/11491065/linking-with-versioned-shared-library-in-android-ndk
- https://stackoverflow.com/questions/18681401/how-can-i-remove-all-versioning-information-from-shared-object-files

This is interesting, but these links are also very old.

I like the new wording of the comment better, but I think we need more
confirmation that this is correct before committing it. Anyone else
here familiar with building on Android?

--
Robert Haas
EDB: http://www.enterprisedb.com

On 05.01.24 01:00, Matthias Kuhn wrote:

Attached a patch with a (hopefully) better wording of the comment.

I have unsuccessfully tried to find an official source for this policy.
So for reference some discussions about the topic:

-
https://stackoverflow.com/questions/11491065/linking-with-versioned-shared-library-in-android-ndk <https://stackoverflow.com/questions/11491065/linking-with-versioned-shared-library-in-android-ndk&gt;
-
https://stackoverflow.com/questions/18681401/how-can-i-remove-all-versioning-information-from-shared-object-files <https://stackoverflow.com/questions/18681401/how-can-i-remove-all-versioning-information-from-shared-object-files&gt;

What I would like to see is a specific thing that you are trying to do
that doesn't work. Probably, you are writing a program that is meant to
run on Android, and you are linking it (provide command line), and then
what happens? The linking fails? It fails to run? What is the error?
Can you provide a minimal example? And so on.

On 01.01.24 06:25, Matthias Kuhn wrote:

It looks like meson does not currently support building for android, the
following output is what I get (but I have actually no experience with
meson):

    meson.build:320:2: ERROR: Problem encountered: unknown host system:
android

FWIW, the meson source code contains numerous mentions of an 'android'
platform, so it seems like this is expected to work.

Hi,

On 2024-01-05 16:00:01 +0100, Peter Eisentraut wrote:

On 01.01.24 06:25, Matthias Kuhn wrote:

It looks like meson does not currently support building for android, the
following output is what I get (but I have actually no experience with
meson):

� � meson.build:320:2: ERROR: Problem encountered: unknown host system:
android

FWIW, the meson source code contains numerous mentions of an 'android'
platform, so it seems like this is expected to work.

This error is from our code, not meson's - the simplest fix would be to just
map android to linux, similar to how dragonflybsd is mapped to netbsd. Blind
attempt attached.

It looks like that might be all that's required, as it looks like meson knows
that android doesn't properly deal with soversion.

Greetings,

Andres Freund

Hi,

On 2024-01-05 15:57:23 +0100, Peter Eisentraut wrote:

On 05.01.24 01:00, Matthias Kuhn wrote:

Attached a patch with a (hopefully) better wording of the comment.

I have unsuccessfully tried to find an official source for this policy.
So for reference some discussions about the topic:

- https://stackoverflow.com/questions/11491065/linking-with-versioned-shared-library-in-android-ndk <https://stackoverflow.com/questions/11491065/linking-with-versioned-shared-library-in-android-ndk&gt;
- https://stackoverflow.com/questions/18681401/how-can-i-remove-all-versioning-information-from-shared-object-files <https://stackoverflow.com/questions/18681401/how-can-i-remove-all-versioning-information-from-shared-object-files&gt;

What I would like to see is a specific thing that you are trying to do that
doesn't work. Probably, you are writing a program that is meant to run on
Android, and you are linking it (provide command line), and then what
happens? The linking fails? It fails to run? What is the error? Can you
provide a minimal example? And so on.

I looked around for a bit, and couldn't really find good documentation around
this. The best I found is
https://developer.android.com/ndk/guides/abis#native-code-in-app-packages
which does strongly imply that the library names aren't versioned.

Greetings,

Andres Freund

What I try to do is packaging an app with androiddeployqt which fails with
an error:

The bundled library lib/libpq.so.5 doesn't end with .so. Android only
supports versionless libraries ending with the .so suffix.

This error was introduced in response to this issue which contains hints
about the underlying problem:

https://bugreports.qt.io/plugins/servlet/mobile#issue/QTBUG-101346

I hope this sheds some light
Matthias

On Fri, Jan 5, 2024, 21:57 Peter Eisentraut <peter@eisentraut.org> wrote:

On 14.01.24 12:37, Matthias Kuhn wrote:

What I try to do is packaging an app with androiddeployqt which fails
with an error:

The bundled library lib/libpq.so.5 doesn't end with .so. Android only
supports versionless libraries ending with the .so suffix.

This error was introduced in response to this issue which contains hints
about the underlying problem:

https://bugreports.qt.io/plugins/servlet/mobile#issue/QTBUG-101346
<https://bugreports.qt.io/plugins/servlet/mobile#issue/QTBUG-101346&gt;

I think the scenario there is using dlopen(), possibly via Qt, possibly
via Java, so it's a very specific scenario, not necessarily indicative
of a general requirement on Android, and apparently not using build-time
linking. It's quite conceivable that this issue would also exist with
Qt on other platforms.

As I mentioned before, Meson has Android support. Some people there
clearly thought about it. So I suggest you try building PostgreSQL for
your Android environment using Meson and see what it produces.(*) If
the output files are the same as the autoconf/make build, then I would
think your scenario is nonstandard. If the output files are different,
then we should check that and consider changes to get them to match.

It's of course possible that Meson is wrong, too, but then we need to
have a broader analysis, because the implicit goal is to keep the two
build systems for PostgreSQL consistent.

(*) - This means specifically that the installation trees produced by
"make install-world-bin" and "meson install" should produce exactly the
same set of files (same names in the same directories, not necessarily
the same contents).

Hi

On Thu, Jan 18, 2024 at 9:27 AM Peter Eisentraut <peter@eisentraut.org>
wrote:

On 14.01.24 12:37, Matthias Kuhn wrote:

What I try to do is packaging an app with androiddeployqt which fails
with an error:

The bundled library lib/libpq.so.5 doesn't end with .so. Android only
supports versionless libraries ending with the .so suffix.

This error was introduced in response to this issue which contains hints
about the underlying problem:

https://bugreports.qt.io/plugins/servlet/mobile#issue/QTBUG-101346
<https://bugreports.qt.io/plugins/servlet/mobile#issue/QTBUG-101346&gt;

I think the scenario there is using dlopen(), possibly via Qt, possibly
via Java, so it's a very specific scenario, not necessarily indicative
of a general requirement on Android, and apparently not using build-time
linking. It's quite conceivable that this issue would also exist with
Qt on other platforms.

I haven't experienced this issue with Qt on any other platform (and I build
and run the same stack on windows, linux, macos and ios).
Also the links posted earlier in this thread hint to the same limitation of
Android, unrelated to Qt.

As I mentioned before, Meson has Android support. Some people there
clearly thought about it. So I suggest you try building PostgreSQL for
your Android environment using Meson and see what it produces.(*) If
the output files are the same as the autoconf/make build, then I would
think your scenario is nonstandard. If the output files are different,
then we should check that and consider changes to get them to match.

It's of course possible that Meson is wrong, too, but then we need to
have a broader analysis, because the implicit goal is to keep the two
build systems for PostgreSQL consistent.

When trying to build with meson, including the patch which was provided by
Andres Freud (thanks),
I am currently stuck with the following error:

Configuring pg_config_ext.h using configuration

../src/tgresql-16-685bc9fc97.clean/src/include/meson.build:12:0: ERROR:
File port/android.h does not exist.

Kind regards
Matthias

On 19.01.24 11:08, Matthias Kuhn wrote:

When trying to build with meson, including the patch which was provided
by Andres Freud (thanks),
I am currently stuck with the following error:

Configuring pg_config_ext.h using configuration

../src/tgresql-16-685bc9fc97.clean/src/include/meson.build:12:0: ERROR:
File port/android.h does not exist.

I think there is actually small bug in the meson setup.

In the top-level meson.build, line 166 has

portname = host_system

but then later around line 190, host_system is changed for some cases
(including the android case that was added in the proposed patch). So
that won't work, and probably also won't work for the existing cases
there. The portname assignment needs to be moved later in the file.
Maybe you can try that on your own.

Thanks,

This patch brought it further and indeed, the resulting libpq.so file is
unversioned when built for Android while it's versioned when built for
linux.

Matthias

On Fri, Jan 19, 2024 at 3:00 PM Peter Eisentraut <peter@eisentraut.org>
wrote:

On 19.01.24 18:12, Matthias Kuhn wrote:

Thanks,

This patch brought it further and indeed, the resulting libpq.so file is
unversioned when built for Android while it's versioned when built for
linux.

Ok, I have committed all of this now:

- Fix for correct order of host_system and portname assignment in
meson.build.

- Patch from Andres to map android to linux in meson.build.

- Makefile.shlib support for Android (based on your patch, but I
reworked it a bit).

So this should all work for you now.