Interest in Faster Unicode Normalization
Hi, pgsql-hackers!
I've been working on a project for some time now that implements
Unicode normalization (NFD, NFC, case fold, etc.) at high speed using
SIMD instructions. The results are promising, and you can check out
the benchmarks over ICU: https://github.com/dzfrias/xxUTF#benchmarks.
I also saw a push for faster normalization in this mailing list,
/messages/by-id/844d3dd7-2955-4794-95d1-7f4c13cb89fc@gmail.com.
I was wondering if Postgres was interested in this work. I know that
my project is an early-stage library, but I think it could be of use
to Postgres in some way or another. Is there any way I could apply my
work to help Postgres? I’ve been working on this project for a while
and honestly just want to see if any larger open source projects are
interested.
Best,
Diego Frias
On Wed, Jun 03, 2026 at 05:06:09PM -0700, Diego Frias wrote:
I was wondering if Postgres was interested in this work. I know that
my project is an early-stage library, but I think it could be of use
to Postgres in some way or another. Is there any way I could apply my
work to help Postgres? I’ve been working on this project for a while
and honestly just want to see if any larger open source projects are
interested.
I would definitely encourage you to look at the other thread and
to participate in its discussion. One of the bottlenecks of the
proposed patch is that we were still discussing how the normalization
tables should be shaped to make the operations faster. Perhaps what
you have in mind could be applied on top of what the other approaches
are implementing, but it's hard to tell without a patch. Duplicating
discussions for the same matter feels useless to me, because the goal
is the same: we want to make the normalizations faster, let's decide
one design that fits best.
--
Michael