removing old ports and architectures

On Sat, Oct 12, 2013 at 5:46 PM, Andres Freund <andres@2ndquadrant.com> wrote:

I think we should remove support the following ports:
- IRIX
- UnixWare
- Tru64

Neither of those are relevant.

Seems reasonable.

I think we should remove support for the following architectures:
- VAX

Agreed.

- univel (s_lock support remaining)
- sinix (s_lock support remaining)
- sun3 (I think it's just s_lock support remaining)
- natsemi 32k

I don't know enough about these, which doesn't bode well for them.

- superH

SuperH isn't dead, but it is only used for very small embedded
systems, I think (mostly microcontrollers). So maybe.

- ALPHA (big pain in the ass to get right, nobody uses it anymore)

Yes, for many years now ALPHA has only been useful as a way of
illustrating how bad it's possible for CPU memory operation reordering
considerations to get. So I quite agree.

- m86k (doesn't have a useable CAS on later iterations like coldfire)

It does seem like Motorola 68k is vanishingly close to dead.

- M32R (no userspace CAS afaics)
- mips for anything but gcc > 4.4, using gcc's atomics support
- s390 for anything but gcc > 4.4, using gcc's atomics support
- 32bit/<v9 sparc (doesn't have proper atomics, old)

Not so sure about these.

Possibly:
- all mips
- PA-RISC. I think Tom was the remaining user there? Maybe just !gcc.

I think we should think hard about removing support for MIPS. A lot of
Chinese chip manufacturers have licensed MIPS technology in just the
last couple of years, so there is plenty of it out there; I'd be
slightly concerned that the proposed restrictions on MIPS would be
onerous. Much of this is the kind of hardware that a person might
plausibly want to run Postgres on.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andres Freund <andres@2ndquadrant.com> writes:

I think we should remove support for the following architectures:
- superH

This one was contributed just a year or two ago, if memory serves,
which suggests that somebody out there cares about it. OTOH, if
they still care, we could insist they provide whatever atomic ops
we want to depend on.

- PA-RISC. I think Tom was the remaining user there? Maybe just !gcc.

Until pretty recently, there was a PA-RISC machine (not mine) in the
buildfarm. I don't see it in the list today though. In any case,
HP's compiler has always been a PITA, so no objection to requiring gcc
for this platform.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2013-10-12 18:35:00 -0700, Peter Geoghegan wrote:

Not so sure about these.

- M32R (no userspace CAS afaics)

I don't think M32R will hurt us/anybody much.

- 32bit/<v9 sparc (doesn't have proper atomics, old)

Sparc v9 is from 1995, so I think not supporting it anymore is
fair. It's afaics not supported by sun studio's intrics either.

- mips for anything but gcc > 4.4, using gcc's atomics support

The reason I'd like to de-support mips for older GCCs is that writing
assembler for them isn't trivial enough to do it blindly and I've had -
for other stuff - difficulties getting my hand on them. GCC provides all
the atomics for mips since 4.2, so we can just rely on that.

- s390 for anything but gcc > 4.4, using gcc's atomics support

Easier to write assembler, but still untestable and even harder to get
access on.
I think 4.2 should be fine as well.

I think we should think hard about removing support for MIPS. A lot of
Chinese chip manufacturers have licensed MIPS technology in just the
last couple of years, so there is plenty of it out there; I'd be
slightly concerned that the proposed restrictions on MIPS would be
onerous. Much of this is the kind of hardware that a person might
plausibly want to run Postgres on.

That's a fair point. But all of them will use gcc, right? I've
previously thought we'd need 4.4 because there's an incompatibility
between 4.3 and 4.4 but I think it won't touch us, so 4.2 which added
atomics for mips seems fine. Given there's no buildfarm animal and
there's lots of variety out there that seems like a fair amount of
support.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2013-10-13 11:34:42 +0200, Tom Lane wrote:

Andres Freund <andres@2ndquadrant.com> writes:

I think we should remove support for the following architectures:
- superH

This one was contributed just a year or two ago, if memory serves,
which suggests that somebody out there cares about it. OTOH, if
they still care, we could insist they provide whatever atomic ops
we want to depend on.

It was 2009 - aac3c301b5e8178841e5749b3657c1a639ba06c1 . I haven't yet
verified if gcc's atomics support is acceptable for the platform
(checkout is running for the last 3h...).
If it's supported, falling back to that seems easy enough.

- PA-RISC. I think Tom was the remaining user there? Maybe just !gcc.

Until pretty recently, there was a PA-RISC machine (not mine) in the
buildfarm. I don't see it in the list today though. In any case,
HP's compiler has always been a PITA, so no objection to requiring gcc
for this platform.

The reason I'd like to generally get rid of PA-RISC is that it's the
only platform that doesn't seem to have any form of compare and
swap. GCC should provide fallbacks - with some warnings - using
spinlocks instead but I am afraid people will start doing things like
atomic operations in signal handlers that won't be noticed and will be a
PITA to debug.

Having read a fair amount of assembler looking at this I have to say,
anybody thinking LL/SC architectures are neat...

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2013-10-13 14:08:59 +0200, Andres Freund wrote:

On 2013-10-13 11:34:42 +0200, Tom Lane wrote:

Andres Freund <andres@2ndquadrant.com> writes:

I think we should remove support for the following architectures:
- superH

This one was contributed just a year or two ago, if memory serves,
which suggests that somebody out there cares about it. OTOH, if
they still care, we could insist they provide whatever atomic ops
we want to depend on.

It was 2009 - aac3c301b5e8178841e5749b3657c1a639ba06c1 . I haven't yet
verified if gcc's atomics support is acceptable for the platform
(checkout is running for the last 3h...).
If it's supported, falling back to that seems easy enough.

Ok, so the git checkout has finally
finished. http://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/config/sh/sync.md;hb=HEAD
makes for an interesting read...

In short: On newer sh variants (SH4) true hardware support is used, on
older ones the linux provides helpers. The variant has to be selected by
gcc parameters unfortunately, but I'd guess that's ok (e.g. SH4
-matomic-model=hard-llcs ).

- PA-RISC. I think Tom was the remaining user there? Maybe just !gcc.

Until pretty recently, there was a PA-RISC machine (not mine) in the
buildfarm. I don't see it in the list today though. In any case,
HP's compiler has always been a PITA, so no objection to requiring gcc
for this platform.

The reason I'd like to generally get rid of PA-RISC is that it's the
only platform that doesn't seem to have any form of compare and
swap. GCC should provide fallbacks - with some warnings - using
spinlocks instead

Hm. I had misunderstood the documentation around this a bit - source
code still is the only reasonable documentation :(.
PA-RISC is only supported on linux using kernel helpers (which switch
off interrupts for the duration of the operation...).

but I am afraid people will start doing things like
atomic operations in signal handlers that won't be noticed and will be a
PITA to debug.

That's already a possibility with our memory barriers :(.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andres Freund <andres@2ndquadrant.com> writes:

That's a fair point. But all of them will use gcc, right? I've
previously thought we'd need 4.4 because there's an incompatibility
between 4.3 and 4.4 but I think it won't touch us, so 4.2 which added
atomics for mips seems fine. Given there's no buildfarm animal and
there's lots of variety out there that seems like a fair amount of
support.

FWIW, I wouldn't have the slightest bit of difficulty with setting a
project policy that if you want some platform to be supported, you
must provide a buildfarm animal running on it.

More to the point for this specific case, it seems like our process
ought to be
(1) select a preferably-small set of gcc atomic intrinsics that we
want to use.
(2) provide macro wrappers for these so that substituting other
implementations isn't too tough.
(3) for any platform where people don't want to use recent gcc,
insist they provide substitute implementations of the macros.

I don't think the core project has to be responsible for implementing
(3), except maybe on very-mainstream arches such as x86. We can
adopt a self-help attitude for everything else.

But ... having said all that, it would be nice to see some proof of
significant performance benefits that we're going to get from kicking
those non-mainstream arches to the curb. I'm not enamored of removing
real functionality now for vague promises of benefits later.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2013-10-13 16:56:12 +0200, Tom Lane wrote:

Andres Freund <andres@2ndquadrant.com> writes:

That's a fair point. But all of them will use gcc, right? I've
previously thought we'd need 4.4 because there's an incompatibility
between 4.3 and 4.4 but I think it won't touch us, so 4.2 which added
atomics for mips seems fine. Given there's no buildfarm animal and
there's lots of variety out there that seems like a fair amount of
support.

FWIW, I wouldn't have the slightest bit of difficulty with setting a
project policy that if you want some platform to be supported, you
must provide a buildfarm animal running on it.

More to the point for this specific case, it seems like our process
ought to be
(1) select a preferably-small set of gcc atomic intrinsics that we
want to use.
(2) provide macro wrappers for these so that substituting other
implementations isn't too tough.
(3) for any platform where people don't want to use recent gcc,
insist they provide substitute implementations of the macros.

I don't think the core project has to be responsible for implementing
(3), except maybe on very-mainstream arches such as x86. We can
adopt a self-help attitude for everything else.

Sounds fair to me.

The question about platforms that simply cannot provide such atomics
like PA-RISC, which afaics is the only one, remains tho. I am not sure
we really want to provide codepaths that are only going to be tested
there.

I do plan to propose a set of macros for this once we know what cases we
need to support.

But ... having said all that, it would be nice to see some proof of
significant performance benefits that we're going to get from kicking
those non-mainstream arches to the curb. I'm not enamored of removing
real functionality now for vague promises of benefits later.

The reason I am bringing this up is that I'd like lift the patch in
http://archives.postgresql.org/message-id/20130926225545.GB26663%40awork2.anarazel.de
from a purely POC state to something committable. The numbers in it seem
to justify causing some pain.
Once the infrastructure is there we can easily get some really nice
further benefits by changing buffer pinning - that's one of the most
expensive things wrt. scalability after the above patch.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andres Freund <andres@2ndquadrant.com> writes:

The question about platforms that simply cannot provide such atomics
like PA-RISC, which afaics is the only one, remains tho. I am not sure
we really want to provide codepaths that are only going to be tested
there.

PA-RISC is a dead architecture. According to wikipedia, HP hasn't sold
any such machines since 2008, and won't support them beyond 2013. If
that really is the only case we're worried about supporting, it's an
easy decision.

What worries me more is that you mentioned several cases where the gcc
atomics exist but need kernel support. I have to think that a trap
to the kernel would make the operation so expensive as to be a serious
performance loss, not gain. So it seems to me that platforms like that
are essentially being kicked to the curb if we make this change, even
if they theoretically could still work. Are there any that we really
care about?

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2013-10-13 20:39:21 +0200, Tom Lane wrote:

Andres Freund <andres@2ndquadrant.com> writes:

The question about platforms that simply cannot provide such atomics
like PA-RISC, which afaics is the only one, remains tho. I am not sure
we really want to provide codepaths that are only going to be tested
there.

PA-RISC is a dead architecture. According to wikipedia, HP hasn't sold
any such machines since 2008, and won't support them beyond 2013. If
that really is the only case we're worried about supporting, it's an
easy decision.

Great.

What worries me more is that you mentioned several cases where the gcc
atomics exist but need kernel support. I have to think that a trap
to the kernel would make the operation so expensive as to be a serious
performance loss, not gain. So it seems to me that platforms like that
are essentially being kicked to the curb if we make this change, even
if they theoretically could still work.

I think it's not that bad on most - they all seem to use some more
lightweight trap mechanisms that basically just change protection and
stops interrupts while doing the math. No chance of scheduling and such.

Are there any that we really care about?

I don't think so. At least not if we're restricting ourselves to 32bit
cmpxchg/xadd which I think will be enough for the first rounds of
improvement.

It's:
- PA-RISC
- sparc before ultrasparcs (1995)
- Multi-CPU/core SuperH before SH4 (uni SH2 has some cute interrupt
handling tricks, that do not require a trap)
- arm before v6

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Sun, Oct 13, 2013 at 5:34 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

- PA-RISC. I think Tom was the remaining user there? Maybe just !gcc.

Until pretty recently, there was a PA-RISC machine (not mine) in the
buildfarm. I don't see it in the list today though. In any case,
HP's compiler has always been a PITA, so no objection to requiring gcc
for this platform.

I object to requiring gcc on that platform; we've had recent interest
in aCC-compiled builds on that platform.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2013-10-14 09:40:10 -0400, Robert Haas wrote:

On Sun, Oct 13, 2013 at 5:34 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

- PA-RISC. I think Tom was the remaining user there? Maybe just !gcc.

Until pretty recently, there was a PA-RISC machine (not mine) in the
buildfarm. I don't see it in the list today though. In any case,
HP's compiler has always been a PITA, so no objection to requiring gcc
for this platform.

I object to requiring gcc on that platform; we've had recent interest
in aCC-compiled builds on that platform.

On PA-RISC or Itanic?

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Mon, Oct 14, 2013 at 9:41 AM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2013-10-14 09:40:10 -0400, Robert Haas wrote:

On Sun, Oct 13, 2013 at 5:34 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

- PA-RISC. I think Tom was the remaining user there? Maybe just !gcc.

Until pretty recently, there was a PA-RISC machine (not mine) in the
buildfarm. I don't see it in the list today though. In any case,
HP's compiler has always been a PITA, so no objection to requiring gcc
for this platform.

I object to requiring gcc on that platform; we've had recent interest
in aCC-compiled builds on that platform.

On PA-RISC or Itanic?

Oh, sorry. I think it was Itanium.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2013-10-14 09:42:46 -0400, Robert Haas wrote:

On Mon, Oct 14, 2013 at 9:41 AM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2013-10-14 09:40:10 -0400, Robert Haas wrote:

On Sun, Oct 13, 2013 at 5:34 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

- PA-RISC. I think Tom was the remaining user there? Maybe just !gcc.

Until pretty recently, there was a PA-RISC machine (not mine) in the
buildfarm. I don't see it in the list today though. In any case,
HP's compiler has always been a PITA, so no objection to requiring gcc
for this platform.

I object to requiring gcc on that platform; we've had recent interest
in aCC-compiled builds on that platform.

On PA-RISC or Itanic?

Oh, sorry. I think it was Itanium.

Good. Not that Itanium is nice, but it certainly supports all required
atomic ops.

I think somebody with access to that will have to figure out what
intrinsics are provided then - I haven't found reference documentation
for aCC's ia64/sys/inline.h.
I think Tom's stance that people who want a platform to be supported
need to help is reasonable. It's really hard to develop not entirely
trivial code for platforms you don't have access to and that don't even
have a buildfarm member.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2013-10-13 16:56:12 +0200, Tom Lane wrote:

More to the point for this specific case, it seems like our process
ought to be
(1) select a preferably-small set of gcc atomic intrinsics that we
want to use.

I suggest:
* pg_atomic_load_u32(uint32 *)
* uint32 pg_atomic_store_u32(uint32 *)
* uint32 pg_atomic_exchange_u32(uint32 *ptr, uint32 val)
* bool pg_atomic_compare_exchange_u32(uint32 *ptr, uint32 *expected, uint32 newval)
* uint32 pg_atomic_fetch_add_u32(uint32 *ptr, uint32 add)
* uint32 pg_atomic_fetch_sub_u32(uint32 *ptr, uint32 add)
* uint32 pg_atomic_fetch_and_u32(uint32 *ptr, uint32 add)
* uint32 pg_atomic_fetch_or_u32(uint32 *ptr, uint32 add)
* u64 variants of the above
* bool pg_atomic_test_set(void *ptr)
* void pg_atomic_clear(void *ptr)

Ontop of that we can generically implement:
* pg_atomic_add_until_u32(uint32 *ptr, uint32 val, uint32 limit)
* pg_atomic_(add|sub|and|or)_fetch_u32()
* u64 variants of the above

We might also want to provide a generic implementation of the math
operations based on pg_atomic_compare_exchange() to make it easier to
bring up a new architecture.

I think we should leave 64bit support optional for now.

Opinions?

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Tue, Oct 15, 2013 at 8:33 AM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2013-10-13 16:56:12 +0200, Tom Lane wrote:

More to the point for this specific case, it seems like our process
ought to be
(1) select a preferably-small set of gcc atomic intrinsics that we
want to use.

I suggest:
* pg_atomic_load_u32(uint32 *)
* uint32 pg_atomic_store_u32(uint32 *)

We currently assume simple assignment suffices for this.

* uint32 pg_atomic_exchange_u32(uint32 *ptr, uint32 val)
* bool pg_atomic_compare_exchange_u32(uint32 *ptr, uint32 *expected, uint32 newval)
* uint32 pg_atomic_fetch_add_u32(uint32 *ptr, uint32 add)
* uint32 pg_atomic_fetch_sub_u32(uint32 *ptr, uint32 add)
* uint32 pg_atomic_fetch_and_u32(uint32 *ptr, uint32 add)
* uint32 pg_atomic_fetch_or_u32(uint32 *ptr, uint32 add)

Do we really need all of those? Compare-and-swap is clearly needed,
and fetch-and-add is also very useful. I'm not sure about the rest.
Not that I necessarily object to having them, but it will be a lot
more work.

* u64 variants of the above
* bool pg_atomic_test_set(void *ptr)
* void pg_atomic_clear(void *ptr)

What are the intended semantics here?

Ontop of that we can generically implement:
* pg_atomic_add_until_u32(uint32 *ptr, uint32 val, uint32 limit)
* pg_atomic_(add|sub|and|or)_fetch_u32()
* u64 variants of the above

Are these really generally needed?

We might also want to provide a generic implementation of the math
operations based on pg_atomic_compare_exchange() to make it easier to
bring up a new architecture.

+1.

I have a related problem, which is that some code I'm currently
working on vis-a-vis parallelism can run lock-free on platforms with
atomic 8 bit assignment but needs a spinlock or two elsewhere. So I'd
want to use pg_atomic_store_u64(), but I'd also need a clean way to
test, at compile time, whether it's available.

More general question: how do we move the ball down the field in this
area? I'm willing to do some of the legwork, but I doubt I can do all
of it, and I really think we need to make some progress here soon, as
it seems that you and I and Ants are all running into the same
problems in slightly different ways. What's our strategy for getting
something done here?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2013-10-16 12:26:28 -0400, Robert Haas wrote:

On Tue, Oct 15, 2013 at 8:33 AM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2013-10-13 16:56:12 +0200, Tom Lane wrote:

More to the point for this specific case, it seems like our process
ought to be
(1) select a preferably-small set of gcc atomic intrinsics that we
want to use.

I suggest:
* pg_atomic_load_u32(uint32 *)
* uint32 pg_atomic_store_u32(uint32 *)

We currently assume simple assignment suffices for this.

Partially that only works because we sprinkle 'volatile's on lots of
places. That can actually hurt performance... it'd usually be something
like:
#define pg_atomic_load(uint32) (*(volatile uint32 *)(atomic))

Even if not needed in some places because a variable is already
volatile, it's very helpful in pointing out what happens and where you
need to be careful.

* uint32 pg_atomic_exchange_u32(uint32 *ptr, uint32 val)
* bool pg_atomic_compare_exchange_u32(uint32 *ptr, uint32 *expected, uint32 newval)
* uint32 pg_atomic_fetch_add_u32(uint32 *ptr, uint32 add)
* uint32 pg_atomic_fetch_sub_u32(uint32 *ptr, uint32 add)
* uint32 pg_atomic_fetch_and_u32(uint32 *ptr, uint32 add)
* uint32 pg_atomic_fetch_or_u32(uint32 *ptr, uint32 add)

Do we really need all of those? Compare-and-swap is clearly needed,
and fetch-and-add is also very useful. I'm not sure about the rest.
Not that I necessarily object to having them, but it will be a lot
more work.

Well, _sub can clearly be implemented with _add generically. I find code
using _sub much easier to read than _add(-whatever), but that's maybe
just me.

But _and, _or are really useful because they can be used to atomically
clear and set flag bits.

* u64 variants of the above
* bool pg_atomic_test_set(void *ptr)
* void pg_atomic_clear(void *ptr)

What are the intended semantics here?

It's basically TAS() without defining whether setting a value that needs
to be set is a 1 or a 0. Gcc provides this and I think we should make
our spinlock implementation use it if there is no special cased
implementation available.

Ontop of that we can generically implement:
* pg_atomic_add_until_u32(uint32 *ptr, uint32 val, uint32 limit)
* pg_atomic_(add|sub|and|or)_fetch_u32()
* u64 variants of the above

Are these really generally needed?

_add_until() is very useful for implementing thinks like usagecount
where you don't want to increase values too high.

The lwlock scaling thing needs the add_fetch variant because we need to
know what the lockcount is *after* we've registered. I think lots of
lockless algorithm have similar requirements.

Since those are either wrappers around fetch_op or compare_swap and thus
can be implemented generically I don't really see a problem with
providing them.

I have a related problem, which is that some code I'm currently
working on vis-a-vis parallelism can run lock-free on platforms with
atomic 8 bit assignment but needs a spinlock or two elsewhere. So I'd
want to use pg_atomic_store_u64(), but I'd also need a clean way to
test, at compile time, whether it's available.

Yes, definitely. There should be a couple of #defines that declare
whether non-prerequisite options are supported. I'd guess we want at least:
* 8byte math
* 16byte compare_and_swap

More general question: how do we move the ball down the field in this
area? I'm willing to do some of the legwork, but I doubt I can do all
of it, and I really think we need to make some progress here soon, as
it seems that you and I and Ants are all running into the same
problems in slightly different ways. What's our strategy for getting
something done here?

That's a pretty good question.

I am willing to write the gcc implementation, plus the generic wrappers
and provide the infrastructure to override it per-platform. I won't be
able to do anything about non-linux, non-gcc platforms in that timeframe
though.

I was thinking of something like:
include/storage/atomic.h
include/storage/atomic-gcc.h
include/storage/atomic-msvc.h
include/storage/atomic-acc-ia64.h
where atomic.h first has a list of #ifdefs including the specialized
files and then lots of #ifndef providing generic variants if not
already provided by the platorm specific file.

We could provide not only per-compiler files but also compiler
independent files for some arches so we could e.g. define the
restrictions for arm once. I think whether that's useful will be visible
when writing the stuff.

Makes sense?

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Sat, Oct 12, 2013 at 8:46 PM, Andres Freund <andres@2ndquadrant.com> wrote:

I think we should remove support the following ports:
- IRIX
- UnixWare
- Tru64

According to http://en.wikipedia.org/wiki/IRIX, IRIX has been
officially retired. The last release of IRIX was in 2006 and support
will end in December of 2013. Therefore, it will be unsupported by
the time PostgreSQL 9.4 is released.

According to http://en.wikipedia.org/wiki/UnixWare, UnixWare is not
dead, although there have been no new releases in 5 years.

According to http://en.wikipedia.org/wiki/Tru64_UNIX, Tru64 has been
officially retired. Support ended in December, 2012. This seems safe
to remove.

So I vote for removing IRIX and Tru64 immediately, but I'm a little
more hesitant about shooting UnixWare, since it's technically still
supported.

Neither of those are relevant.

I think we should remove support for the following architectures:
- VAX

According to http://en.wikipedia.org/wiki/VAX#History, all
manufacturing of VAX computers ceased in 2005, but according to
http://en.wikipedia.org/wiki/OpenVMS#Major_release_timeline, OpenVMS
is still releasing new versions. I'm not sure what to make of that.

- univel (s_lock support remaining)

We removed the univel port in 9.2 and didn't get any complaints, but
the s_lock support is still used by the SCO and UnixWare ports, except
under GCC.

- sinix (s_lock support remaining)

This seems to be quite thoroughly dead. The best information I can
find indicates that development ended in 2002 and support in 2008. I
think we can remove this.

- sun3 (I think it's just s_lock support remaining)
- natsemi 32k

Both of these are so old I can hardly find any information on them.
Seems clear to nuke these.

- superH

Support for spinlocks on SuperH was only recently added, in 9.0. I
don't think we can assume that no one cares any more.

- ALPHA (big pain in the ass to get right, nobody uses it anymore)

It seems somehow a shame to let this one go, but I agree it's a big
pain in the ass to get it right.

- m86k (doesn't have a useable CAS on later iterations like coldfire)

I don't think we can desupport it just because it doesn't have CAS.
CAS is very useful and I think we should start using it, but I think
we should anticipate a --disable-cas or --disable-atomics option and
regularly test that our code works without it. IOW, we can rely on it
as an optimization, but not for correctness. Eventually, we can
probably desupport all platforms that don't have CAS, but I see that
as something that's probably 5 or 10 years away, not something we can
do tomorrow.

I'm also not sure that this is dead enough to kill.

- M32R (no userspace CAS afaics)

Support was added for this in 8.3 and it doesn't seem to be particularly dead.

- mips for anything but gcc > 4.4, using gcc's atomics support
- s390 for anything but gcc > 4.4, using gcc's atomics support

I'm not clearly how broadly this would sweep, but MIPS doesn't appear dead.

- 32bit/<v9 sparc (doesn't have proper atomics, old)

Seems fine to remove this.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 10/16/2013 07:04 PM, Robert Haas wrote:

On Sat, Oct 12, 2013 at 8:46 PM, Andres Freund <andres@2ndquadrant.com> wrote:

I think we should remove support the following ports:
- IRIX
- UnixWare
- Tru64

According to http://en.wikipedia.org/wiki/IRIX, IRIX has been
officially retired. The last release of IRIX was in 2006 and support
will end in December of 2013. Therefore, it will be unsupported by
the time PostgreSQL 9.4 is released.

According to http://en.wikipedia.org/wiki/UnixWare, UnixWare is not
dead, although there have been no new releases in 5 years.

According to http://en.wikipedia.org/wiki/Tru64_UNIX, Tru64 has been
officially retired. Support ended in December, 2012. This seems safe
to remove.

So I vote for removing IRIX and Tru64 immediately, but I'm a little
more hesitant about shooting UnixWare, since it's technically still
supported.

Neither of those are relevant.

agreed

I think we should remove support for the following architectures:
- VAX

According to http://en.wikipedia.org/wiki/VAX#History, all
manufacturing of VAX computers ceased in 2005, but according to
http://en.wikipedia.org/wiki/OpenVMS#Major_release_timeline, OpenVMS
is still releasing new versions. I'm not sure what to make of that.

VAX is also an officially supported OpenBSD port (see
http://www.openbsd.org/vax.html)

Stefan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2013-10-16 13:04:23 -0400, Robert Haas wrote:

So I vote for removing IRIX and Tru64 immediately, but I'm a little
more hesitant about shooting UnixWare, since it's technically still
supported.

I think if somebody wants to have it supported they need to provide a
buildfarm member and probably a bit of help. Doing any change in this
area for a platform that nobody has access to in any way seems
pointless because it will be broken anyway.

I think we should remove support for the following architectures:
- VAX

According to http://en.wikipedia.org/wiki/VAX#History, all
manufacturing of VAX computers ceased in 2005, but according to
http://en.wikipedia.org/wiki/OpenVMS#Major_release_timeline, OpenVMS
is still releasing new versions. I'm not sure what to make of that.

And the last revisions are from 2000 even. I don't think anybody will
actually run a new postgres installation on it.
I also doubt our current version actually compiles there.

- superH

Support for spinlocks on SuperH was only recently added, in 9.0. I
don't think we can assume that no one cares any more.

Yes, I since revised my opinion somewhere downthread. It's pretty much
linux and gcc only, so it's really not that problematic.

Note that our current implementation is broken on many older SuperH (<
SH4) CPUs since their tas isn't safe...

- m86k (doesn't have a useable CAS on later iterations like coldfire)

I don't think we can desupport it just because it doesn't have CAS.
CAS is very useful and I think we should start using it, but I think
we should anticipate a --disable-cas or --disable-atomics option and
regularly test that our code works without it. IOW, we can rely on it
as an optimization, but not for correctness. Eventually, we can
probably desupport all platforms that don't have CAS, but I see that
as something that's probably 5 or 10 years away, not something we can
do tomorrow.

I think that will result in loads of barely tested duplicative code. If
there were any even remotely popular platforms requiring this, ok. But
unless I miss something there really isn't.

We're talking about CPUs with mostly less than 100MHZ here, mostly with
directly soldered RAM in the one digit MB range. I really don't think
there's a usecase for running PG on them. And I doubt it still works on
many of the architectures we advocate.

I'm also not sure that this is dead enough to kill.

- M32R (no userspace CAS afaics)

Support was added for this in 8.3 and it doesn't seem to be particularly dead.

It's not? All I've read seems to point into a different direction.

The newest supported CPU seems to be
http://www.renesas.com/products/mpumcu/m32r/m32r_ecu/32196/index.jsp
sporting a whopping 1024Kb of programmable RAM and only single precision
FPU.

- mips for anything but gcc > 4.4, using gcc's atomics support
- s390 for anything but gcc > 4.4, using gcc's atomics support

I'm not clearly how broadly this would sweep, but MIPS doesn't appear dead.

Downthread I noticed it's gcc 4.2 not 4.4. There's some API change in
gcc's atomics support in 4.4 which is why I thought of 4.4 but it
shouldn't affect us after looking in more detail.
Mips seems to only be used with gcc these days, so I think that's ok.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers