Compiling on Termux
Folks,
I'm trying to compile master (c952eae52a33069e2e92d34f217b43d0eca3d7de)
on Termux, using the supplied settings, as follows.
pg_config --configure | xargs ./configure > configure.out 2>configure.err
make -j 4 > make.out 2> make.err
There appears to be some confusion somewhere about sync_file_range,
namely that it's found by ./configure and then not found in make.
What should I be poking at to make this work?
Best,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778
Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate
On Sat, Dec 22, 2018 at 5:56 AM David Fetter <david@fetter.org> wrote:
Folks,
I'm trying to compile master (c952eae52a33069e2e92d34f217b43d0eca3d7de)
on Termux, using the supplied settings, as follows.pg_config --configure | xargs ./configure > configure.out 2>configure.err
make -j 4 > make.out 2> make.errThere appears to be some confusion somewhere about sync_file_range,
namely that it's found by ./configure and then not found in make.What should I be poking at to make this work?
Apparently your libc (or something else) defines the function so the
configure test passes, but your <fcntl.h> doesn't declare it so we
can't use it. I guess Termux supplies the headers but your Android
supplies the libraries, so there may be sync issues. I'd try hunting
around for declarations with something like find /usr/include -name
'*.h' | xargs grep sync_file_range. Here's an interesting similar
case:
https://github.com/termux/termux-packages/issues/899
That talks about using -D__ANDROID_API__=23 (or presumably higher) to
make sure that sigtimedwait is exposed by signal.h. Something similar
may be afoot here.
--
Thomas Munro
http://www.enterprisedb.com
On Sat, Dec 22, 2018 at 07:03:32AM +1100, Thomas Munro wrote:
On Sat, Dec 22, 2018 at 5:56 AM David Fetter <david@fetter.org> wrote:
Folks,
I'm trying to compile master (c952eae52a33069e2e92d34f217b43d0eca3d7de)
on Termux, using the supplied settings, as follows.pg_config --configure | xargs ./configure > configure.out 2>configure.err
make -j 4 > make.out 2> make.errThere appears to be some confusion somewhere about sync_file_range,
namely that it's found by ./configure and then not found in make.What should I be poking at to make this work?
Apparently your libc (or something else) defines the function so the
configure test passes, but your <fcntl.h> doesn't declare it so we
can't use it. I guess Termux supplies the headers but your Android
supplies the libraries, so there may be sync issues. I'd try hunting
around for declarations with something like find /usr/include -name
'*.h' | xargs grep sync_file_range. Here's an interesting similar
case:https://github.com/termux/termux-packages/issues/899
That talks about using -D__ANDROID_API__=23 (or presumably higher) to
make sure that sigtimedwait is exposed by signal.h. Something similar
may be afoot here.
That worked perfectly...to expose the next issue, namely that at least
part of the System-V IPC mechanisms have been left out. Do we support
other systems where this is true?
Best,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778
Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate
On Sat, Dec 22, 2018 at 7:32 AM David Fetter <david@fetter.org> wrote:
On Sat, Dec 22, 2018 at 07:03:32AM +1100, Thomas Munro wrote:
That talks about using -D__ANDROID_API__=23 (or presumably higher) to
make sure that sigtimedwait is exposed by signal.h. Something similar
may be afoot here.That worked perfectly...to expose the next issue, namely that at least
part of the System-V IPC mechanisms have been left out. Do we support
other systems where this is true?
Interesting, they left that stuff out deliberately out basically
because it all sucks:
PostgreSQL currently needs SysV shm just for the small segment that's
used for preventing multiple copies running in the same pgdata. You'd
need to find a new way to do that. We don't use SysV semaphores
(though we can) or message queues.
--
Thomas Munro
http://www.enterprisedb.com
On 12/21/18 4:04 PM, Thomas Munro wrote:
On Sat, Dec 22, 2018 at 7:32 AM David Fetter <david@fetter.org> wrote:
On Sat, Dec 22, 2018 at 07:03:32AM +1100, Thomas Munro wrote:
That talks about using -D__ANDROID_API__=23 (or presumably higher) to
make sure that sigtimedwait is exposed by signal.h. Something similar
may be afoot here.That worked perfectly...to expose the next issue, namely that at least
part of the System-V IPC mechanisms have been left out. Do we support
other systems where this is true?Interesting, they left that stuff out deliberately out basically
because it all sucks:PostgreSQL currently needs SysV shm just for the small segment that's
used for preventing multiple copies running in the same pgdata. You'd
need to find a new way to do that. We don't use SysV semaphores
(though we can) or message queues.
That doc says:
Killing processes automatically to make room for new ones is an
important part of Android's application lifecycle implementation.
How's that going to play with Postgres? Sounds like the OOM killer on
steroids.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Fri, Dec 21, 2018 at 04:24:20PM -0500, Andrew Dunstan wrote:
On 12/21/18 4:04 PM, Thomas Munro wrote:
On Sat, Dec 22, 2018 at 7:32 AM David Fetter <david@fetter.org> wrote:
On Sat, Dec 22, 2018 at 07:03:32AM +1100, Thomas Munro wrote:
That talks about using -D__ANDROID_API__=23 (or presumably higher) to
make sure that sigtimedwait is exposed by signal.h. Something similar
may be afoot here.That worked perfectly...to expose the next issue, namely that at least
part of the System-V IPC mechanisms have been left out. Do we support
other systems where this is true?Interesting, they left that stuff out deliberately out basically
because it all sucks:PostgreSQL currently needs SysV shm just for the small segment that's
used for preventing multiple copies running in the same pgdata. You'd
need to find a new way to do that. We don't use SysV semaphores
(though we can) or message queues.That doc says:
Killing processes automatically to make room for new ones is an
important part of Android's application lifecycle implementation.How's that going to play with Postgres? Sounds like the OOM killer on
steroids.
I don't know precisely how it's going to play with Postgres, but
Termux does supply a Postgres in its native packages. That package
appears to work, at least in the single-connection case, so they're
doing something somehow to get it up and running.
It could be that Android won't be a platform we recommend for high
workloads, at least as long as these things remain true.
Best,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778
Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate
On Sat, Dec 22, 2018 at 9:19 AM David Fetter <david@fetter.org> wrote:
On Fri, Dec 21, 2018 at 04:24:20PM -0500, Andrew Dunstan wrote:
On 12/21/18 4:04 PM, Thomas Munro wrote:
On Sat, Dec 22, 2018 at 7:32 AM David Fetter <david@fetter.org> wrote:
On Sat, Dec 22, 2018 at 07:03:32AM +1100, Thomas Munro wrote:
That talks about using -D__ANDROID_API__=23 (or presumably higher) to
make sure that sigtimedwait is exposed by signal.h. Something similar
may be afoot here.That worked perfectly...to expose the next issue, namely that at least
part of the System-V IPC mechanisms have been left out. Do we support
other systems where this is true?Interesting, they left that stuff out deliberately out basically
because it all sucks:PostgreSQL currently needs SysV shm just for the small segment that's
used for preventing multiple copies running in the same pgdata. You'd
need to find a new way to do that. We don't use SysV semaphores
(though we can) or message queues.That doc says:
Killing processes automatically to make room for new ones is an
important part of Android's application lifecycle implementation.How's that going to play with Postgres? Sounds like the OOM killer on
steroids.I don't know precisely how it's going to play with Postgres, but
Termux does supply a Postgres in its native packages. That package
appears to work, at least in the single-connection case, so they're
doing something somehow to get it up and running.It could be that Android won't be a platform we recommend for high
workloads, at least as long as these things remain true.
They use libandroid-shmem which emulates SysV shmem.
https://github.com/termux/termux-packages/blob/master/packages/postgresql/src-backend-Makefile.patch
https://github.com/termux/libandroid-shmem
--
Thomas Munro
http://www.enterprisedb.com
Thomas Munro <thomas.munro@enterprisedb.com> writes:
On Sat, Dec 22, 2018 at 9:19 AM David Fetter <david@fetter.org> wrote:
I don't know precisely how it's going to play with Postgres, but
Termux does supply a Postgres in its native packages. That package
appears to work, at least in the single-connection case, so they're
doing something somehow to get it up and running.
They use libandroid-shmem which emulates SysV shmem.
Interesting. I wonder how well it emulates the aspect we actually
care about, ie counting the number of attached processes correctly ...
regards, tom lane
On Sat, Dec 22, 2018 at 11:51 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Thomas Munro <thomas.munro@enterprisedb.com> writes:
On Sat, Dec 22, 2018 at 9:19 AM David Fetter <david@fetter.org> wrote:
I don't know precisely how it's going to play with Postgres, but
Termux does supply a Postgres in its native packages. That package
appears to work, at least in the single-connection case, so they're
doing something somehow to get it up and running.They use libandroid-shmem which emulates SysV shmem.
Interesting. I wonder how well it emulates the aspect we actually
care about, ie counting the number of attached processes correctly ...
I hadn't paid much attention to Android before, but it's quite
impressive that it can run PostgreSQL. One interesting thing (to me
at least) is that although it's some kind of Linux kernel, it has a
new libc called Bionic that is a mixture of cleanroom bits and *BSD
bits, probably because they liked the licence better[1]https://en.wikipedia.org/wiki/Bionic_(software). The
allocator is jemalloc so I wonder if it eats 40k for every 32k hash
join chunk.
I wonder if we could mmap a small file created by initdb under pgdata
and do a new kind of interlocking in there, so you don't need SysV shm
at all. That would work for Android (which is obviously just a
curiosity for now, but kinda neat), and maybe also some other
interesting new platforms like CloudABI[2]https://archive.fosdem.org/2017/schedule/event/cloudabi/ that get rid of global
stuff like the System V facilities, so that you can trap software
entirely inside a root directory identified by an inherited file
descriptor (not that I have seriously studied what else would be
required to run in that environment).
[1]: https://en.wikipedia.org/wiki/Bionic_(software)
[2]: https://archive.fosdem.org/2017/schedule/event/cloudabi/
--
Thomas Munro
http://www.enterprisedb.com