Proposal: Document ABI Compatibility

Hi,

On 2024-06-03 14:43:17 -0400, David E. Wheeler wrote:

At the PGConf Unconference session on improving extension support in core,
we talked quite a bit about the recent anxiety among extension developers
about a lack of an ABI compatibility guarantee in Postgres.

Are there notes for the session?

Yurii Rashkovskii did a little header file spelunking and talked[1] about a
few changes in minor version releases, including to apparent field order in
structs.

It'd be nice if the slides for the talk could be uploaded...

You must be referring to my commit 714780dc. The new field is stored within alignment padding (though only on back branches). Has this been tied to a known problem?

At the Unconference, Tom Lane said that this approach is pretty well drilled
into the heads of every committer, and new ones pick it up through
experience. The goal, IIUC, is to never introduce binary incompatibilities
into the C APIs in minor releases. This standard would be good to document,
to let extension developers know exactly what the guarantees are.

I don't think we can really make this a hard guarantee. Yes, we try hard to
avoid ABI breaks, but there IIRC have been a few cases over the years where
that wasn't practical for some reason. If we have to decide between a bad bug
and causing an ABI issue that's unlikely to affect anybody, we'll probably
choose the ABI issue.

* The ABI is guaranteed to change only in backward compatible ways in minor
releases. If for some reason it doesn’t it’s a bug that will need to be
fixed.

Thus I am not really on board with this statement as-is.

Extensions in general can do lots of stuff, guaranteeing that bug fixes don't
cause any problems is just not feasible.

It'd be interesting to see a few examples of actual minor-version-upgrade
extension breakages, so we can judge what caused them.

But if I understand correctly, the use of techniques like adding a new field
in padding does not mean that extensions compiled on later minor releases
will work on earlier minor releases of the same major version.

I don't think it's common for such new-fields-in-padding to cause problems
when using an earlier minor PG version. For that the extension would need to
actually rely on the presence of the new field, but typically that'd not be
the case when we introduce a new field in a minor version.

Unless, that is, we can provide a complete list of things not to do (like
make use of padding) to avoid it. Is that feasible?

You can't really rely on the contents of padding, in general. So I don't think
this is really something that needs to be called out.

Greetings,

Andres Freund

On Jun 3, 2024, at 14:58, Andres Freund <andres@anarazel.de> wrote:

Hi,

Hello Andres.

Are there notes for the session?

Yes, but not posted yet. Here’s what Andreas 'ads' Scherbaum sent me for that bit of the conversation:

* Core is focused on core ABI stability
* David: No "statement of stability" in Core
* David/Jeremy/Tom: coding guidelines, style guidelines
* useful to have docs in core about what's stable and what's not, what you should compile against or not, and ABI guarantees
* Abigale: there are hooks, but no overall concept for extensions
* Tom: Peter Eisentraut is working on tests for extensions stability
* Jeremy: nothing is preventing people from installing incompatible versions

It'd be nice if the slides for the talk could be uploaded...

He also talked about it at Mini-Summit Five[1]https://justatheory.com/2024/05/mini-summit-five/, for which I posted slides[2]https://justatheory.com/shared/extension-ecosystem-summit/omni-universally-buildable-extensions.pdf (slides 11-14) and video[3]https://youtu.be/R5ijx8IJyaM (starting at 29:08).

I don't think we can really make this a hard guarantee. Yes, we try hard to
avoid ABI breaks, but there IIRC have been a few cases over the years where
that wasn't practical for some reason. If we have to decide between a bad bug
and causing an ABI issue that's unlikely to affect anybody, we'll probably
choose the ABI issue.

We can document that too, and perhaps a policy for letting people know. I thought I recalled something like this in the past, but Rob Treat did some spelunking through change logs and found only CVEs that needed repairs by manually running some SQL. So some sense of its rarity would also be useful.

Thus I am not really on board with this statement as-is.

That’s fine, we should get it to where there’s consensus on the ordering and agreement on what, exactly, it should be.

Extensions in general can do lots of stuff, guaranteeing that bug fixes don't
cause any problems is just not feasible.

It'd be interesting to see a few examples of actual minor-version-upgrade
extension breakages, so we can judge what caused them.

In the community Slack[4]https://pgtreats.info/slack-invite, Matthias van de Meent writes[5]https://postgresteam.slack.com/archives/C056ZA93H1A/p1716502630690559?thread_ts=1716500801.036709&amp;cid=C056ZA93H1A:

Citus’ pg_version_compat.h[7] where it re-implements a low-level function that was newly introduced in PG14.7. If you build against PG14.7+ headers, you may get random crashes when running on 14.6.

I suppose it would work fine on 14.7 if compiled on 14.6 though. I suspect there aren’t many examples, though, mostly just a lot of anxiety, and some have decided that extensions must be recompiled for every minor release in order to avoid the issue. StackGres[7]https://stackgres.io/doc/latest/intro/extensions/ is one example, but I suspect Omni (Yurii’s company) may follow.

I don't think it's common for such new-fields-in-padding to cause problems
when using an earlier minor PG version. For that the extension would need to
actually rely on the presence of the new field, but typically that'd not be
the case when we introduce a new field in a minor version.

That’s what I was thinking, so “compatibility assured only if you don’t use padding yourself” would be super helpful.

Unless, that is, we can provide a complete list of things not to do (like
make use of padding) to avoid it. Is that feasible?

You can't really rely on the contents of padding, in general. So I don't think
this is really something that needs to be called out.

Sure, probably not a problem, but if that’s the sole qualifier for making binary changes, I think it’s worth saying, as opposed to “we don’t make any”. Something like “Only changes to padding, which you never used anyway, right?” :-)

D

[1]: https://justatheory.com/2024/05/mini-summit-five/
[2]: https://justatheory.com/shared/extension-ecosystem-summit/omni-universally-buildable-extensions.pdf
[3]: https://youtu.be/R5ijx8IJyaM
[4]: https://pgtreats.info/slack-invite
[5]: https://postgresteam.slack.com/archives/C056ZA93H1A/p1716502630690559?thread_ts=1716500801.036709&amp;cid=C056ZA93H1A
[6]: https://github.com/citusdata/citus/blob/main/src/include/pg_version_compat.h#L236-L248
[7]: https://stackgres.io/doc/latest/intro/extensions/

Andres Freund <andres@anarazel.de> writes:

On 2024-06-03 14:43:17 -0400, David E. Wheeler wrote:

* The ABI is guaranteed to change only in backward compatible ways in minor
releases. If for some reason it doesn’t it’s a bug that will need to be
fixed.

Thus I am not really on board with this statement as-is.

Me either. There are degrees of ABI compatibility, and we'll choose
the least invasive way, but it's seldom the case that no conceivable
extension will be broken. For example, if we can't squeeze a new
field into padding space, we'll typically put it at the end of the
struct in existing branches. That's okay unless some extension has
a dependency on sizeof(the struct), for instance because it's
allocating such structs itself. Also, for either the padding-space
or add-at-the-end approaches, we're probably in trouble if some
extension is creating its own instances of such structs, because
it will not know how to fill the new field properly. We try not to
change structs that we think extensions are likely to create ...
but that's a guess not a guarantee.

It'd be interesting to see a few examples of actual minor-version-upgrade
extension breakages, so we can judge what caused them.

Yes, that could be a fruitful discussion.

You can't really rely on the contents of padding, in general. So I don't think
this is really something that needs to be called out.

For node structs, padding will generally be zero because we memset
them to zeroes. So to the extent that zero is an okay value for
such a new field, that can help --- but if zero were always okay
then we'd likely not need a new field in the first place.

regards, tom lane

I don't think it's common for such new-fields-in-padding to cause problems
when using an earlier minor PG version. For that the extension would need to
actually rely on the presence of the new field, but typically that'd not be
the case when we introduce a new field in a minor version.

That’s what I was thinking, so “compatibility assured only if you don’t use padding yourself” would be super helpful.

I was under the impression that the (a?) concern was related to
compiling the newer version and using that against an older version,
so if you always compiled the extension against the latest point
release, it wouldn't necessarily be backwards-compatible with older
installations. (While probably allocated padding is zero'd out in the
old version, I'd not guarantee that that's the case.)

Unless, that is, we can provide a complete list of things not to do (like
make use of padding) to avoid it. Is that feasible?

You can't really rely on the contents of padding, in general. So I don't think
this is really something that needs to be called out.

Sure, probably not a problem, but if that’s the sole qualifier for making binary changes, I think it’s worth saying, as opposed to “we don’t make any”. Something like “Only changes to padding, which you never used anyway, right?” :-)

Wonder if what might be more useful is some sort of distribution list
for extensions authors to be notified of specific compatibility
changes. Could be powered by parsing commit messages for conventions
about backwards-incompatible changes; padding usage could be one,
maybe there are others we could code for. (For padding, imagine a
tool that is able to look at struct offsets between git revisions and
note any offset differences/changes here that extensions authors could
be aware of; even diffs of `pahole` output between revisions could be
helpful.)

ABI guarantees for extensions are hard because all of core *is*
potentially the ABI (or at least if you are well-behaved, public
functions and structs), so some sort of "these interfaces won't break
in minor releases but use other internals and you're on your own"
might be useful distinction. (We discussed trying to classify APIs
with varying levels of support, but that also comes with its own set
of issues/overhead/maintenance for code authors/committers.)

Hi,

On 2024-06-03 15:21:04 -0400, David E. Wheeler wrote:

Extensions in general can do lots of stuff, guaranteeing that bug fixes don't
cause any problems is just not feasible.

It'd be interesting to see a few examples of actual minor-version-upgrade
extension breakages, so we can judge what caused them.

In the community Slack[4], Matthias van de Meent writes[5]:

Citus’ pg_version_compat.h[7] where it re-implements a low-level function that was newly introduced in PG14.7. If you build against PG14.7+ headers, you may get random crashes when running on 14.6.

I don't see how this would trigger random crashes.

Unfortunately [4] doesn't seem to take me to a relevant message (pruned chat
history?), so I can't infer more from that context.

I suppose it would work fine on 14.7 if compiled on 14.6 though. I suspect
there aren’t many examples, though, mostly just a lot of anxiety, and some
have decided that extensions must be recompiled for every minor release in
order to avoid the issue. StackGres[7] is one example, but I suspect Omni
(Yurii’s company) may follow.

Regardless of ABI issues, it's probably a good idea to continually run tests
against in-development minor versions, just to prevent something breaking from
creeping in. IIRC there were a handful of cases where we accidentally broke
some extension, because they relied on some implementation details.

Unless, that is, we can provide a complete list of things not to do (like
make use of padding) to avoid it. Is that feasible?

You can't really rely on the contents of padding, in general. So I don't think
this is really something that needs to be called out.

Sure, probably not a problem, but if that’s the sole qualifier for making
binary changes, I think it’s worth saying, as opposed to “we don’t make
any”. Something like “Only changes to padding, which you never used anyway,
right?” :-)

IDK, to me something like this seems to promise more than we actually can.

Greetings,

Andres Freund

On Jun 3, 2024, at 5:56 PM, Andres Freund <andres@anarazel.de> wrote:

I don't see how this would trigger random crashes.

Unfortunately [4] doesn't seem to take me to a relevant message (pruned chat
history?), so I can't infer more from that context.

You can use [4]https://pgtreats.info/slack-invite to join the Slack (if you haven’t already) and [5]https://postgresteam.slack.com/archives/C056ZA93H1A/p1716502630690559?thread_ts=1716500801.036709&amp;cid=C056ZA93H1A for the relevant post.

Regardless of ABI issues, it's probably a good idea to continually run tests
against in-development minor versions, just to prevent something breaking from
creeping in. IIRC there were a handful of cases where we accidentally broke
some extension, because they relied on some implementation details.

Oh yeah, I run regular tests against the latest minor release of all supported Postgres version for my extensions, using pgxn-tools[6]https://github.com/pgxn/docker-pgxn-tools, which looks like this[7]https://github.com/pgxn/docker-pgxn-tools/actions/runs/9351752462. Which I consider absolutely essential. But it doesn’t mean that something compiled against .4 will work with .3 and vice versa. That’s what we could use the guidance/guarantees on.

Sure, probably not a problem, but if that’s the sole qualifier for making
binary changes, I think it’s worth saying, as opposed to “we don’t make
any”. Something like “Only changes to padding, which you never used anyway,
right?” :-)

IDK, to me something like this seems to promise more than we actually can.

What I’d like to do is figure out exactly what we *can* promise and perhaps some guidelines, and start with that.

Best,

David

[4]: https://pgtreats.info/slack-invite
[5]: https://postgresteam.slack.com/archives/C056ZA93H1A/p1716502630690559?thread_ts=1716500801.036709&amp;cid=C056ZA93H1A
[6]: https://github.com/pgxn/docker-pgxn-tools
[7]: https://github.com/pgxn/docker-pgxn-tools/actions/runs/9351752462

On Mon, 2024-06-03 at 15:38 -0400, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

On 2024-06-03 14:43:17 -0400, David E. Wheeler wrote:

* The ABI is guaranteed to change only in backward compatible ways in minor
releases. If for some reason it doesn’t it’s a bug that will need to be
fixed.

Thus I am not really on board with this statement as-is.

Me either.  There are degrees of ABI compatibility, and we'll choose
the least invasive way, but it's seldom the case that no conceivable
extension will be broken.

oracle_fdw has been broken by minor releases several times in the past.
This may well be because of weird things that I am doing; still, my
experience is that minor releases are not always binary compatible.

It'd be interesting to see a few examples of actual minor-version-upgrade
extension breakages, so we can judge what caused them.

Yes, that could be a fruitful discussion.

Digging through my commits brought up 6214e2b2280462cbc3aa1986e350e167651b3905,
for one.

Yours,
Laurenz Albe

On 04.06.24 02:11, Laurenz Albe wrote:

On Mon, 2024-06-03 at 15:38 -0400, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

On 2024-06-03 14:43:17 -0400, David E. Wheeler wrote:

* The ABI is guaranteed to change only in backward compatible ways in minor
releases. If for some reason it doesn’t it’s a bug that will need to be
fixed.

Thus I am not really on board with this statement as-is.

Me either.  There are degrees of ABI compatibility, and we'll choose
the least invasive way, but it's seldom the case that no conceivable
extension will be broken.

oracle_fdw has been broken by minor releases several times in the past.
This may well be because of weird things that I am doing; still, my
experience is that minor releases are not always binary compatible.

It'd be interesting to see a few examples of actual minor-version-upgrade
extension breakages, so we can judge what caused them.

Yes, that could be a fruitful discussion.

Digging through my commits brought up 6214e2b2280462cbc3aa1986e350e167651b3905,
for one.

I'm not sure I can see how that would have broken oracle_fdw, but in any
case it's an interesting example. This patch did not change any structs
incompatibly, but it changed the semantics of a function without
changing the name:

  extern void InitResultRelInfo(ResultRelInfo *resultRelInfo,
                               Relation resultRelationDesc,
                               Index resultRelationIndex,
-                             Relation partition_root,
+                             ResultRelInfo *partition_root_rri,
                               int instrument_options);

If an extension calls this function, something would possibly crash if
it's on the wrong side of the update fence.

This could possibly be avoided by renaming the symbol in backbranches.
Maybe something like

#define InitResultRelInfo InitResultRelInfo2

Then you'd get a specific error message when loading the module, rather
than a crash.

This might be something to consider:

no ABI break is better than an explicit ABI break is better than a
silent ABI break

(Although this is actually an API break, isn't it?)

On Jun 4, 2024, at 03:18, Peter Eisentraut <peter@eisentraut.org> wrote:

This could possibly be avoided by renaming the symbol in backbranches. Maybe something like

#define InitResultRelInfo InitResultRelInfo2

Then you'd get a specific error message when loading the module, rather than a crash.

That sounds more useful, yes. Is that a practice the project would consider adopting?

There’s also oracle_fdw@d137d15[1]https://github.com/laurenz/oracle_fdw/commit/d137d15edca8c67df1e5cccca01f417f4833b028, which says:

An API break in PostgreSQL 10.4 and 9.6.9 makes it impossible
to use these versions: the "extract_actual_join_clauses" function
gained an additional parameter.

The 10.4 commit is 68fab04, and it does indeed add a new function:

``` patch
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -36,6 +36,7 @@ extern List *get_actual_clauses(List *restrictinfo_list);
 extern List *extract_actual_clauses(List *restrictinfo_list,
                       bool pseudoconstant);
 extern void extract_actual_join_clauses(List *restrictinfo_list,
+                           Relids joinrelids,
                            List **joinquals,
                            List **otherquals);
 extern bool join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel);
```

I wonder if that sort of change could be avoided in backpatches, maybe by adding and using a `extract_actual_join_clauses_compat` function and using that internally instead?

Or, to David C’s point, perhaps it would be better to say there are some categories of APIs that are not subject to any guarantees in minor releases?

Best,

David

[1]: https://github.com/laurenz/oracle_fdw/commit/d137d15edca8c67df1e5cccca01f417f4833b028
[2]: https://github.com/postgres/postgres/commit/68fab04f7c2a07c5308e3d2957198ccd7a80ebc5#diff-bb6fa74cb115e19684092f0938131cd5d99b26fa2d49480f7ea7f28e937a7fb4

Hi,

On 2024-06-10 15:05:32 -0400, David E. Wheeler wrote:

An API break in PostgreSQL 10.4 and 9.6.9 makes it impossible
to use these versions: the "extract_actual_join_clauses" function
gained an additional parameter.

The 10.4 commit is 68fab04, and it does indeed add a new function:

That's 6 years ago, not sure we can really learn that much from that.

And it's not like it's actually impossible, #ifdefs aren't great, but they are
better than nothing.

``` patch
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -36,6 +36,7 @@ extern List *get_actual_clauses(List *restrictinfo_list);
extern List *extract_actual_clauses(List *restrictinfo_list,
bool pseudoconstant);
extern void extract_actual_join_clauses(List *restrictinfo_list,
+                           Relids joinrelids,
List **joinquals,
List **otherquals);
extern bool join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel);
```

I wonder if that sort of change could be avoided in backpatches, maybe by adding and using a `extract_actual_join_clauses_compat` function and using that internally instead?

Or, to David C’s point, perhaps it would be better to say there are some categories of APIs that are not subject to any guarantees in minor releases?

I'm honestly very dubious that this is a good point to introduce a bunch of
formalism. It's a already a lot of work to maintain them, if we make it even
harder we'll end up more fixes not being backported, because it's not worth
the pain.

To be blunt, the number of examples raised here doesn't seem to indicate that
this is an area where we need to invest additional resources. We are already
severely constrained as a project by committer bandwidth, there are plenty
other things that seem more important to focus on.

Greetings,

Andres Freund

On Jun 10, 2024, at 15:39, Andres Freund <andres@anarazel.de> wrote:

That's 6 years ago, not sure we can really learn that much from that.

And it's not like it's actually impossible, #ifdefs aren't great, but they are
better than nothing.

Right, it’s just that extension authors could use some notification that such a change is coming so they can update their code, if necessary.

Or, to David C’s point, perhaps it would be better to say there are some categories of APIs that are not subject to any guarantees in minor releases?

I'm honestly very dubious that this is a good point to introduce a bunch of
formalism. It's a already a lot of work to maintain them, if we make it even
harder we'll end up more fixes not being backported, because it's not worth
the pain.

Well it’s a matter of distributing the work. I don’t want to increase anyone’s workload unnecessarily, but as it is stuff like this can be surprising to extension maintainers with some expectation of minor release stability who had no warning of the change. That kind of thing can dissuade some people from deciding to write or maintain extensions, and lead others to recompile and distribute binaries for every single minor release.

To be blunt, the number of examples raised here doesn't seem to indicate that
this is an area where we need to invest additional resources. We are already
severely constrained as a project by committer bandwidth, there are plenty
other things that seem more important to focus on.

So my question is, what’s the least onerous thing for committers to commit to doing that we can write down to properly set expectations? That’s where I want to start: can we publish a policy that reflects what committers already adhere to? And is there some way to let people know that an incompatible change is being released? Even if it just starts out in the release notes?

Based on this thread, I’ve drafted the sort of policy I have in mind. Please don’t assume I’m advocating for exactly the wording here! Let’s workshop this until it’s something the committers and core team can agree to. (At that point I’ll turn it into a doc patch) Have a look and let me know what you think.

``` md

ABI Policy
==========

The PostgreSQL core team maintains two application binary interface (ABI) guarantees: one for major releases and one for minor releases.

Major Releases
--------------

Applications that use the PostgreSQL APIs must be compiled for each major release supported by the application. The inclusion of `PG_MODULE_MAGIC` ensures that code compiled for one major version will rejected by other major versions.

Furthermore, new releases may make API changes that require code changes. Use the `PG_VERSION_NUM` constant to adjust code in a backwards compatible way:

``` c
#if PG_VERSION_NUM >= 160000
#include "varatt.h"
#endif
```

PostgreSQL avoids unnecessary API changes in major releases, but usually ships a few necessary API changes, including deprecation, renaming, and argument variation. In such cases the incompatible changes will be listed in the Release Notes.

Minor Releases
--------------

PostgreSQL makes every effort to avoid both API and ABI breaks in minor releases. In general, an application compiled against any minor release will work with any other minor release, past or future.

When a change *is* required, PostgreSQL will choose the least invasive way possible, for example by squeezing a new field into padding space or appending it to the end of a struct. This sort of change should not impact dependent applications unless they use `sizeof(the struct)` or create their own instances of such structs --- patterns best avoided.

In rare cases, however, even such non-invasive changes may be impractical or impossible. In such an event, the change will be documented in the Release Notes, and details on the issue will also be posted to [TBD; mail list? Blog post? News item?].

The project strongly recommends that developers adopt continuous integration testing at least for the latest minor release all major versions of Postgres they support.
```

Best,

David

On 11.06.24 16:55, David E. Wheeler wrote:

On Jun 10, 2024, at 15:39, Andres Freund <andres@anarazel.de> wrote:

That's 6 years ago, not sure we can really learn that much from that.

And it's not like it's actually impossible, #ifdefs aren't great, but they are
better than nothing.

Right, it’s just that extension authors could use some notification that such a change is coming so they can update their code, if necessary.

I think since around 6 years ago we have been much more vigilant about
avoiding ABI breaks. So if there aren't any more recent examples of
breakage, then maybe that was ultimately successful, and the upshot is,
continue to be vigilant at about the same level?

On Wed, 12 Jun 2024 at 14:44, Peter Eisentraut <peter@eisentraut.org> wrote:

I think since around 6 years ago we have been much more vigilant about
avoiding ABI breaks. So if there aren't any more recent examples of
breakage, then maybe that was ultimately successful, and the upshot is,
continue to be vigilant at about the same level?

While not strictly an ABI break I guess, the backport of 32d5a4974c81
broke building Citus against 13.10 and 14.7[1]https://github.com/citusdata/citus/pull/6711.

[1]: https://github.com/citusdata/citus/pull/6711

On Jun 12, 2024, at 8:43 AM, Peter Eisentraut <peter@eisentraut.org> wrote:

Right, it’s just that extension authors could use some notification that such a change is coming so they can update their code, if necessary.

I think since around 6 years ago we have been much more vigilant about avoiding ABI breaks. So if there aren't any more recent examples of breakage, then maybe that was ultimately successful, and the upshot is, continue to be vigilant at about the same level?

That sounds great to me. I’d like to get it documented, though, so that extension and other third party developers are aware of it, and not just making wild guesses and scaring each other over (perhaps) misconceptions.

D

On Jun 12, 2024, at 8:58 AM, Jelte Fennema-Nio <postgres@jeltef.nl> wrote:

While not strictly an ABI break I guess, the backport of 32d5a4974c81
broke building Citus against 13.10 and 14.7[1].

[1]: https://github.com/citusdata/citus/pull/6711

Interesting one. We might want to advise projects to use deferent names if they copy code from the core, use an extension-specific prefix perhaps. That way if it gets backported by the core, as in this example, it won’t break anything, and the extension can choose to switch.

D

On Mon, Jun 3, 2024 at 3:39 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Me either. There are degrees of ABI compatibility

Exactly this!

What I think would be useful to document is our usual practices e.g.
adding new struct members at the end of structs, trying to avoid
changing public function signatures. If we document promises to
extension authors, I don't know how much difference that will make:
we'll probably end up needing to violate them at some point for one
reason or another. But if we document what committers should do, then
we might do better than we're now, because committers will be more
likely to do it right, and extension authors can also read those
instructions to understand what our practices are.

--
Robert Haas
EDB: http://www.enterprisedb.com

On Jun 12, 2024, at 10:47, Robert Haas <robertmhaas@gmail.com> wrote:

What I think would be useful to document is our usual practices e.g.
adding new struct members at the end of structs, trying to avoid
changing public function signatures. If we document promises to
extension authors, I don't know how much difference that will make:
we'll probably end up needing to violate them at some point for one
reason or another.

I think that’s fine if there is some sort of notification process. The policy I drafted upthread starts with making sure the such a break is mentioned in the release notes.

But if we document what committers should do, then
we might do better than we're now, because committers will be more
likely to do it right, and extension authors can also read those
instructions to understand what our practices are.

Yes, this, thank you!

D

On 2024-06-11 10:55:38 -0400, David E. Wheeler wrote:

ABI Policy
==========

The PostgreSQL core team maintains two application binary interface (ABI) guarantees: one for major releases and one for minor releases.

I.e. for major versions it's "there is none"?

Major Releases
--------------

Applications that use the PostgreSQL APIs must be compiled for each major release supported by the application. The inclusion of `PG_MODULE_MAGIC` ensures that code compiled for one major version will rejected by other major versions.

Furthermore, new releases may make API changes that require code changes. Use the `PG_VERSION_NUM` constant to adjust code in a backwards compatible way:

``` c
#if PG_VERSION_NUM >= 160000
#include "varatt.h"
#endif
```

PostgreSQL avoids unnecessary API changes in major releases, but usually
ships a few necessary API changes, including deprecation, renaming, and
argument variation.

In such cases the incompatible changes will be listed in the Release Notes.

I don't think we actually exhaustively list all of them.

Minor Releases
--------------

PostgreSQL makes every effort to avoid both API and ABI breaks in minor releases. In general, an application compiled against any minor release will work with any other minor release, past or future.

s/every/a reasonable/ or just s/every/an/

When a change *is* required, PostgreSQL will choose the least invasive way
possible, for example by squeezing a new field into padding space or
appending it to the end of a struct. This sort of change should not impact
dependent applications unless they use `sizeof(the struct)` or create their
own instances of such structs --- patterns best avoided.

The padding case doesn't affect sizeof() fwiw.

I think there's too often not an alternative to using sizeof(), potentially
indirectly (via makeNode() or such. So this sounds a bit too general.

Greetings,

Andres Freund

Hi,

On 2024-06-12 14:58:04 +0200, Jelte Fennema-Nio wrote:

On Wed, 12 Jun 2024 at 14:44, Peter Eisentraut <peter@eisentraut.org> wrote:

I think since around 6 years ago we have been much more vigilant about
avoiding ABI breaks. So if there aren't any more recent examples of
breakage, then maybe that was ultimately successful, and the upshot is,
continue to be vigilant at about the same level?

While not strictly an ABI break I guess, the backport of 32d5a4974c81
broke building Citus against 13.10 and 14.7[1].

I think that kind of thing is not something we (PG devs) really can do
anything about. It's also
a) fairly easy thing to fix
b) fails during compilation
c) doesn't break ABI afaict

Greetings,

Andres Freund