PG 18 release notes draft committed

Hi Bruce,

On Fri, May 2, 2025 at 11:44 AM Bruce Momjian <bruce@momjian.us> wrote:

I have committd the first draft of the PG 18 release notes.

I will continue improving it until beta 1, and until the final release.
I will probably add markup in 1-3 weeks. Let the feedback begin. ;-)

Thanks as always for working on this.

I noticed the release notes currently combine several
partition-related commits under one item:

+<!--
+Author: Amit Langote <amitlan@postgresql.org>
+2025-01-30 [bb3ec16e1] Move PartitionPruneInfo out of plan nodes into PlannedSt
+Author: Amit Langote <amitlan@postgresql.org>
+2025-01-31 [d47cbf474] Perform runtime initial pruning outside ExecInitNode()
+Author: Amit Langote <amitlan@postgresql.org>
+2025-02-07 [cbc127917] Track unpruned relids to avoid processing pruned relatio
+Author: Amit Langote <amitlan@postgresql.org>
+2025-02-20 [525392d57] Don't lock partitions pruned by initial pruning
+Author: Amit Langote <amitlan@postgresql.org>
+2025-04-04 [88f55bc97] Make derived clause lookup in EquivalenceClass more effi
+Author: David Rowley <drowley@postgresql.org>
+2025-04-08 [d69d45a5a] Speedup child EquivalenceMember lookup in planner
+-->
+<listitem>
+<para>
+Allow partitions to be pruned earlier and quicker, and skipped in
more places (Amit Langote, Ashutosh Bapat, Yuya Watari, David Rowley)
+<ulink url="&commit_baseurl;bb3ec16e1">&sect;</ulink>
+<ulink url="&commit_baseurl;d47cbf474">&sect;</ulink>
+<ulink url="&commit_baseurl;cbc127917">&sect;</ulink>
+<ulink url="&commit_baseurl;525392d57">&sect;</ulink>
+<ulink url="&commit_baseurl;88f55bc97">&sect;</ulink>
+<ulink url="&commit_baseurl;d69d45a5a">&sect;</ulink>
+</para>
+</listitem>

But I think these really fall into three separate improvements:

1. Speed up execution of cached plans by deferring locks on partitions
subject to pruning (Amit Langote)
(bb3ec16e1, d47cbf474, cbc127917, 525392d57)

2. Speed up child EquivalenceMember lookup in planner (Yuya Watari,
David Rowley)
(d69d45a5a)

3. Speed up derived clause lookup in EquivalenceClass (Ashutosh Bapat)
(88f55bc97)

Alternatively, 2 and 3 can be combined as:

2. Speed up partition planning by improving EquivalenceClass lookups
(Yuya Watari, David Rowley, Ashutosh Bapat)

I think 1 should go under Partitioning, which I see is currently missing.

Any thoughts, David?

Can work on a patch if you'd like.

--
Thanks, Amit Langote

On Thu, May 1, 2025 at 7:44 PM Bruce Momjian <bruce@momjian.us> wrote:

I will probably add markup in 1-3 weeks. Let the feedback begin. ;-)

Thanks!

<para>
Version 18 contains a number of changes that may affect compatibility
with previous releases. Observe the following incompatibilities:
</para>
[...]
Rename server variable ssl_ecdh_curve to ssl_groups and allow multiple colon-separated ECDH curves to be specified (Erica Zhang, Daniel Gustafsson)

The previous setting name should continue to function correctly, since
it's mapped as an alias, so this can probably be moved into the
"standard" config features rather than a compatibility change.

--Jacob

On Fri, May 2, 2025 at 08:24:42AM -0700, Jacob Champion wrote:

On Thu, May 1, 2025 at 7:44 PM Bruce Momjian <bruce@momjian.us> wrote:

I will probably add markup in 1-3 weeks. Let the feedback begin. ;-)

Thanks!

<para>
Version 18 contains a number of changes that may affect compatibility
with previous releases. Observe the following incompatibilities:
</para>
[...]
Rename server variable ssl_ecdh_curve to ssl_groups and allow multiple colon-separated ECDH curves to be specified (Erica Zhang, Daniel Gustafsson)

The previous setting name should continue to function correctly, since
it's mapped as an alias, so this can probably be moved into the
"standard" config features rather than a compatibility change.

Thanks, done. The commit message didn't indicate the old name would
still work, and I didn't review the patch for that.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Do not let urgent matters crowd out time for investment in the future.

On Fri, May 2, 2025 at 01:00:57PM +0900, Amit Langote wrote:

1. Speed up execution of cached plans by deferring locks on partitions
subject to pruning (Amit Langote)
(bb3ec16e1, d47cbf474, cbc127917, 525392d57)

2. Speed up child EquivalenceMember lookup in planner (Yuya Watari,
David Rowley)
(d69d45a5a)

3. Speed up derived clause lookup in EquivalenceClass (Ashutosh Bapat)
(88f55bc97)

Alternatively, 2 and 3 can be combined as:

2. Speed up partition planning by improving EquivalenceClass lookups
(Yuya Watari, David Rowley, Ashutosh Bapat)

I think 1 should go under Partitioning, which I see is currently missing.

Any thoughts, David?

Can work on a patch if you'd like.

So, a few things. First, these set of commits was in a group of 10 that
I added since there have been complaints in the past that optimizer
improvements were not listed and therefore patch authors were not given
sufficient credit. That means the 209 item count for PG 18 is 10 higher
than my normal filtering would produce.

Second, looking at the items, these are a case of "X is faster", which
we don't normally mention in the release notes. We normally mention
"faster" when it is so much faster that use cases which were not
possible before might be possible now, so it is recommended to retest.
That is what I saw this grouped item as, whereas I don't think the
individual items meet that criteria.

Also, I didn't see enough partition items to warrant a separate
partition section, and we didn't have one in PG 17 either. We could
pull all the partition items from the sections they are already in, but
they seem more natural in the sections they are in.

I don't think most people would know what EquivalenceMember is, and even
if they did, would they be able to connect it to an SQL query?

Finally, I see the big increases in this release as being the optimizer,
monitoring, and constraints.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Do not let urgent matters crowd out time for investment in the future.

On Fri, May 2, 2025 at 12:18:06PM -0400, Bruce Momjian wrote:

Finally, I see the big increases in this release as being the optimizer,
monitoring, and constraints.

Also, and I am loving the chapter markers linking to gitweb.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Do not let urgent matters crowd out time for investment in the future.

On Fri, 2 May 2025 at 04:45, Bruce Momjian <bruce@momjian.us> wrote:

I have committd the first draft of the PG 18 release notes. The item
count looks strong:

Thanks for all the work. Some notes:

1. There's currently no mention that protocol version 3.2 was
introduced in this release. I'm not sure where/how this should be
mentioned, but I definitely think it should be somewhere. It's a
pretty major change. One option is to replace/amend the "Make cancel
request keys 256 bits" item. Maybe replace that with something like:
"Postgres 18 introduces protocol version 3.2. This is the first new
protocol version since 3.0, which was introduced in Postgres 7.4. This
new protocol version 3.2 allows a server to use longer cancel request
keys. When the client advertises support for protocol version 3.2 (or
higher) Postgres 18 will use a cancel key size of 256 bits."
2. Obviously biased since it's my contribution, but I think d38bab5
might deserve a mention.
3. The "Add PQtrace() output..." commitlist should also contain 7adec2d5fc

On Sat, May 3, 2025 at 01:46:29AM +0200, Jelte Fennema-Nio wrote:

On Fri, 2 May 2025 at 04:45, Bruce Momjian <bruce@momjian.us> wrote:

I have committd the first draft of the PG 18 release notes. The item
count looks strong:

Thanks for all the work. Some notes:

1. There's currently no mention that protocol version 3.2 was
introduced in this release. I'm not sure where/how this should be
mentioned, but I definitely think it should be somewhere. It's a
pretty major change. One option is to replace/amend the "Make cancel
request keys 256 bits" item. Maybe replace that with something like:
"Postgres 18 introduces protocol version 3.2. This is the first new
protocol version since 3.0, which was introduced in Postgres 7.4. This
new protocol version 3.2 allows a server to use longer cancel request
keys. When the client advertises support for protocol version 3.2 (or
higher) Postgres 18 will use a cancel key size of 256 bits."

Okay, I added a mention next to the libpq version function entries.

2. Obviously biased since it's my contribution, but I think d38bab5
might deserve a mention.

I disagree. pgbench limits like this are not something we give much
detail around error avoidance to in the release notes.

3. The "Add PQtrace() output..." commitlist should also contain 7adec2d5fc

Added. Patch attached.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Do not let urgent matters crowd out time for investment in the future.

On Sat, 3 May 2025 at 02:06, Bruce Momjian <bruce@momjian.us> wrote:

On Sat, May 3, 2025 at 01:46:29AM +0200, Jelte Fennema-Nio wrote:

On Fri, 2 May 2025 at 04:45, Bruce Momjian <bruce@momjian.us> wrote:

I have committd the first draft of the PG 18 release notes. The item
count looks strong:

Thanks for all the work. Some notes:

1. There's currently no mention that protocol version 3.2 was
introduced in this release. I'm not sure where/how this should be
mentioned, but I definitely think it should be somewhere. It's a
pretty major change. One option is to replace/amend the "Make cancel
request keys 256 bits" item. Maybe replace that with something like:
"Postgres 18 introduces protocol version 3.2. This is the first new
protocol version since 3.0, which was introduced in Postgres 7.4. This
new protocol version 3.2 allows a server to use longer cancel request
keys. When the client advertises support for protocol version 3.2 (or
higher) Postgres 18 will use a cancel key size of 256 bits."

Okay, I added a mention next to the libpq version function entries.

I think it should be mentioned in the server section. How about we
replace: "This is enabled when the libpq client and server are
Postgres 18 or later. ", with the following:

This is only enabled when the client supports the wire protocol
version 3.2 (or up), Wire protocol version 3.2 is introduced in this
release. For libpq based clients that means that libpq needs to be of
version 18 or higher.

That also makes it clear that for non-libpq based clients people
should check if their client supports it.

On Fri, May 2, 2025 at 10:44 AM Bruce Momjian <bruce@momjian.us> wrote:

I have committd the first draft of the PG 18 release notes. The item
count looks strong:
release-17: 182
release-18: 209

I will continue improving it until beta 1, and until the final release.
I will probably add markup in 1-3 weeks. Let the feedback begin. ;-)

You can see the most current HTML-built version here:

https://momjian.us/pgsql_docs/release-18.html

seems you missed this ([1]https://git.postgresql.org/cgit/postgresql.git/commit/?id=bd10ec529796a13670645e6acd640c6f290df020):

Detect redundant GROUP BY columns using UNIQUE indexes

d4c3a156c added support that when the GROUP BY contained all of the columns
belonging to a relation's PRIMARY KEY, all other columns belonging to that
relation would be removed from the GROUP BY clause. That's possible because all
other columns are functionally dependent on the PRIMARY KEY and those columns
alone ensure the groups are distinct.

[1]: https://git.postgresql.org/cgit/postgresql.git/commit/?id=bd10ec529796a13670645e6acd640c6f290df020

On Sat, May 3, 2025 at 01:16:24PM +0200, Jelte Fennema-Nio wrote:

On Sat, 3 May 2025 at 02:06, Bruce Momjian <bruce@momjian.us> wrote:

On Sat, May 3, 2025 at 01:46:29AM +0200, Jelte Fennema-Nio wrote:

On Fri, 2 May 2025 at 04:45, Bruce Momjian <bruce@momjian.us> wrote:

I have committd the first draft of the PG 18 release notes. The item
count looks strong:

Thanks for all the work. Some notes:

1. There's currently no mention that protocol version 3.2 was
introduced in this release. I'm not sure where/how this should be
mentioned, but I definitely think it should be somewhere. It's a
pretty major change. One option is to replace/amend the "Make cancel
request keys 256 bits" item. Maybe replace that with something like:
"Postgres 18 introduces protocol version 3.2. This is the first new
protocol version since 3.0, which was introduced in Postgres 7.4. This
new protocol version 3.2 allows a server to use longer cancel request
keys. When the client advertises support for protocol version 3.2 (or
higher) Postgres 18 will use a cancel key size of 256 bits."

Okay, I added a mention next to the libpq version function entries.

I think it should be mentioned in the server section. How about we
replace: "This is enabled when the libpq client and server are
Postgres 18 or later. ", with the following:

This is only enabled when the client supports the wire protocol
version 3.2 (or up), Wire protocol version 3.2 is introduced in this
release. For libpq based clients that means that libpq needs to be of
version 18 or higher.

That also makes it clear that for non-libpq based clients people
should check if their client supports it.

I moved the item and added some text, patch attached.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Do not let urgent matters crowd out time for investment in the future.

On Sat, May 3, 2025 at 11:04:45PM +0800, jian he wrote:

On Fri, May 2, 2025 at 10:44 AM Bruce Momjian <bruce@momjian.us> wrote:

I have committd the first draft of the PG 18 release notes. The item
count looks strong:
release-17: 182
release-18: 209

I will continue improving it until beta 1, and until the final release.
I will probably add markup in 1-3 weeks. Let the feedback begin. ;-)

You can see the most current HTML-built version here:

https://momjian.us/pgsql_docs/release-18.html

seems you missed this ([1]):

Detect redundant GROUP BY columns using UNIQUE indexes

d4c3a156c added support that when the GROUP BY contained all of the columns
belonging to a relation's PRIMARY KEY, all other columns belonging to that
relation would be removed from the GROUP BY clause. That's possible because all
other columns are functionally dependent on the PRIMARY KEY and those columns
alone ensure the groups are distinct.

[1] https://git.postgresql.org/cgit/postgresql.git/commit/?id=bd10ec529796a13670645e6acd640c6f290df020

We added this item to the PG 9.6 release notes:

<!--
2016-02-11 [d4c3a156c] Remove GROUP BY columns that are functionally dependent
-->
<para>
Ignore <literal>GROUP BY</> columns that are
functionally dependent on other columns (David Rowley)
</para>

<para>
If a <literal>GROUP BY</> clause includes all columns of a
non-deferred primary key, as well as other columns of the same
table, those other columns are redundant and can be dropped
from the grouping. This saves computation in many common cases.
</para>
</listitem>

Interestingly, the first paragraph suggests this optimization already
works for unique indexes, but the text below it states it only works for
primary keys.

As a nod to PG 9.6, which was released in 2016, I duplicated that item
and reworded it for this commit. :-) Patch attached.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Do not let urgent matters crowd out time for investment in the future.

On Sat, 3 May 2025 at 18:19, Bruce Momjian <bruce@momjian.us> wrote:

I moved the item and added some text, patch attached.

LGTM, apart from the typo in the word "client' (it's spelled as
"cliient" in the diff).

Noticed a few other small things when rereading:

1. "Add libpq functions and environment..." should be "Add libpq
connection parameters and environment
2. "Allow the specification of non-overlapping PRIMARY KEY and UNIQUE
constraints" has no commit attached to it. I noticed this because it
wasn't fully clear to me what this feature entailed, so I wanted to
look at it in more detail.
3. "Report search_path changes to the client." is currently in the
libpq section, but this is a server-only change. So that seems a bit
strange. Not sure where to put it though.

Hi hackers,

I will continue improving it until beta 1, and until the final release.
I will probably add markup in 1-3 weeks. Let the feedback begin. ;-)

You can see the most current HTML-built version here:

https://momjian.us/pgsql_docs/release-18.html

I'm not sure, but I'll ask.

I have significantly improved the handling of Unicode Case in
PostgreSQL.
The improvements affect important functions such as lower(), upper(),
casefold().
Specifically, the patch has significantly reduced the size of Unicode
Case tables (and consequently the size of the object file).
We got a significant speed gain:
ASCII by ≈10%
Cyrillic by ≈80%
Unicode in general by ≈30%

But, unfortunately, I didn't see any mention of this improvement in the
release notes.
Hence the question to the community - are such improvements worth
mentioning?

I'm just new to the community, and want to understand.

Commit:
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=27bdec06841d1bb004ca7627eac97808b08a7ac7

I am now actively working on a major improvement to Unicode
Normalization Forms.

Thanks!

--
Regards,
Alexander Borisov

On Sun, May 4, 2025 at 01:31:51AM +0300, Alexander Borisov wrote:

Hi hackers,

I will continue improving it until beta 1, and until the final release.
I will probably add markup in 1-3 weeks. Let the feedback begin. ;-)

You can see the most current HTML-built version here:

https://momjian.us/pgsql_docs/release-18.html

I'm not sure, but I'll ask.

I have significantly improved the handling of Unicode Case in
PostgreSQL.
The improvements affect important functions such as lower(), upper(),
casefold().
Specifically, the patch has significantly reduced the size of Unicode
Case tables (and consequently the size of the object file).
We got a significant speed gain:
ASCII by ≈10%
Cyrillic by ≈80%
Unicode in general by ≈30%

But, unfortunately, I didn't see any mention of this improvement in the
release notes.
Hence the question to the community - are such improvements worth
mentioning?

I'm just new to the community, and want to understand.

Commit: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=27bdec06841d1bb004ca7627eac97808b08a7ac7

I am now actively working on a major improvement to Unicode
Normalization Forms.

Given the performance numbers above, which were not in the commit, maybe
I should add it to the case folding item, and add his name as a
co-author.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Do not let urgent matters crowd out time for investment in the future.

On Sat, May 3, 2025 at 09:40:47PM +0200, Jelte Fennema-Nio wrote:

On Sat, 3 May 2025 at 18:19, Bruce Momjian <bruce@momjian.us> wrote:

I moved the item and added some text, patch attached.

LGTM, apart from the typo in the word "client' (it's spelled as
"cliient" in the diff).

Thanks, fixed.

Noticed a few other small things when rereading:

1. "Add libpq functions and environment..." should be "Add libpq
connection parameters and environment

Fixed.

2. "Allow the specification of non-overlapping PRIMARY KEY and UNIQUE
constraints" has no commit attached to it. I noticed this because it
wasn't fully clear to me what this feature entailed, so I wanted to
look at it in more detail.

Fixed.

3. "Report search_path changes to the client." is currently in the
libpq section, but this is a server-only change. So that seems a bit
strange. Not sure where to put it though.

Not sure. The only idea I had was server configuration, which doesn't
match well.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Do not let urgent matters crowd out time for investment in the future.

04.05.2025 01:47, Bruce Momjian wrote:

[...]

Given the performance numbers above, which were not in the commit, maybe
I should add it to the case folding item, and add his name as a
co-author.

I'm not a co-author, I'm the author of my own algorithm that
significantly improves PostgreSQL code.
The author of casefold() is Jeff Davis (and in general, as I understand,
Jeff is responsible for Unicode in Postgres).
I only suggested how to significantly improve the storage of Unicode
Case data and provide fast access to this data.
In other words, I improved, accelerated the algorithms.

Because of which the functions lower(), upper(), casefold() got a
significant boost.

--
Regards,
Alexander Borisov

On Sun, May 4, 2025 at 02:24:16AM +0300, Alexander Borisov wrote:

04.05.2025 01:47, Bruce Momjian wrote:

[...]

Given the performance numbers above, which were not in the commit, maybe
I should add it to the case folding item, and add his name as a
co-author.

I'm not a co-author, I'm the author of my own algorithm that
significantly improves PostgreSQL code.
The author of casefold() is Jeff Davis (and in general, as I understand,
Jeff is responsible for Unicode in Postgres).
I only suggested how to significantly improve the storage of Unicode
Case data and provide fast access to this data.
In other words, I improved, accelerated the algorithms.

Because of which the functions lower(), upper(), casefold() got a
significant boost.

It doesn't warrant its own item because it is not user-facing work. The
best we can do is add the commit to an existing item and add you as a
co-author on an existing item. You will see several items that are that
way already.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Do not let urgent matters crowd out time for investment in the future.

04.05.2025 02:28, Bruce Momjian wrote:

It doesn't warrant its own item because it is not user-facing work. The
best we can do is add the commit to an existing item and add you as a
co-author on an existing item. You will see several items that are that
way already.

Thank you for clarifying!
Users are not interested in performance gains.
Then it's not worth considering. Sorry to interrupt.

--
Regards,
Alexander Borisov

On Sun, May 4, 2025 at 02:48:31AM +0300, Alexander Borisov wrote:

04.05.2025 02:28, Bruce Momjian wrote:

It doesn't warrant its own item because it is not user-facing work. The
best we can do is add the commit to an existing item and add you as a
co-author on an existing item. You will see several items that are that
way already.

Thank you for clarifying!
Users are not interested in performance gains.
Then it's not worth considering. Sorry to interrupt.

So the logic is something I posted to this thread already:

So, a few things. First, these set of commits was in a group of 10 that
I added since there have been complaints in the past that optimizer
improvements were not listed and therefore patch authors were not given
sufficient credit. That means the 209 item count for PG 18 is 10 higher
than my normal filtering would produce.

Second, looking at the items, these are a case of "X is faster", which
we don't normally mention in the release notes. We normally mention
"faster" when it is so much faster that use cases which were not
possible before might be possible now, so it is recommended to retest.
That is what I saw this grouped item as, whereas I don't think the
individual items meet that criteria.

So, users are interested in performance in the sense it makes use cases
possible, and if your commit is making the case folding useful, we
should mention it in the release notes. I don't think making it
separate would fit though.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Do not let urgent matters crowd out time for investment in the future.