Adding init-po and update-po targets to the meson build system
Hi,
I am working on adding init-po and update-po targets to the meson
build system and have a a couple of questions. First let me explain my
current thinking.
I think the biggest problem is the nls.mk files. autoconf read these
files to generate *.pot files but meson can't read these nls.mk files
natively. So, my solution is switching to using nls.json files instead
of nls.mk files and having a Perl script so that the autoconf system
can parse these nls.json files and read them as nls.mk files as
before. So the big picture is:
- We will switch nls.mk files with nls.json files.
- autoconf will read these nls.json files with the help of perl script.
- meson will read nls.json files and will use a helper Python script
to generate *.pot files.
- meson will have init-po and update-po targets and also singular
targes like psql-init-po.
Does that sound good?
----------------------------------------
Some additional questions:
1. Can init-po and update-po meson targets regenerate *.pot files on
every run or should these files be tracked and regenerated only when
inputs change?
-- I am asking this because implementing the prior one is simpler.
Latter one requires more complicated changes.
2. How closely must the output from meson and autoconf match?
-- Should meson preserve autoconf’s source-reference paths and
ordering exactly, or are differences acceptable? In particular,
generated sources can produce build-tree relative references through
their #line directives. Should we normalize them?
Do you have anything else in mind?
I CCed Álvaro and Peter since I remember they use / work on these areas.
--
Regards,
Nazir Bilal Yavuz
Microsoft
On 2026-Oct-07, Nazir Bilal Yavuz wrote:
So, my solution is switching to using nls.json files instead
of nls.mk files and having a Perl script so that the autoconf system
can parse these nls.json files and read them as nls.mk files as
before. So the big picture is:- We will switch nls.mk files with nls.json files.
- autoconf will read these nls.json files with the help of perl script.
- meson will read nls.json files and will use a helper Python script
to generate *.pot files.
- meson will have init-po and update-po targets and also singular
targes like psql-init-po.
Hmm, so the source files would be .json and we generate .mk files as
derivative artifacts? I don't know, that sounds potentially
nightmarish. Isn't it more convenient to generate a JSON file from the
nls.mk list instead?
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Once again, thank you and all of the developers for your hard work on
PostgreSQL. This is by far the most pleasant management experience of
any database I've worked on." (Dan Harris)
http://archives.postgresql.org/pgsql-performance/2006-04/msg00247.php
Hi,
On Wed, 7 Oct 2026 at 19:24, Álvaro Herrera <alvherre@kurilemu.de> wrote:
On 2026-Oct-07, Nazir Bilal Yavuz wrote:
So, my solution is switching to using nls.json files instead
of nls.mk files and having a Perl script so that the autoconf system
can parse these nls.json files and read them as nls.mk files as
before. So the big picture is:- We will switch nls.mk files with nls.json files.
- autoconf will read these nls.json files with the help of perl script.
- meson will read nls.json files and will use a helper Python script
to generate *.pot files.
- meson will have init-po and update-po targets and also singular
targes like psql-init-po.Hmm, so the source files would be .json and we generate .mk files as
derivative artifacts? I don't know, that sounds potentially
nightmarish. Isn't it more convenient to generate a JSON file from the
nls.mk list instead?
Yes, I agree that it is kind of nightmarish. I considered both options
and chose to use JSON files because, AFAIK, we want to remove the
autoconf system and use meson in the future. So, using JSON files
makes more sense to me from a maintainability perspective for this
reason.
Perhaps it is better to generate JSON files from the nls.mk files for
now and switch later if needed.
--
Regards,
Nazir Bilal Yavuz
Microsoft
Import Notes
Reply to msg id not found: asZxSXw6qRN2X_c9@alvherre.pgsql
On Wed, Oct 7, 2026 at 9:40 AM Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
Yes, I agree that it is kind of nightmarish. I considered both options
and chose to use JSON files because, AFAIK, we want to remove the
autoconf system and use meson in the future. So, using JSON files
makes more sense to me from a maintainability perspective for this
reason.
A third path might be to disentangle the NLS makefiles from
Makefile.global, so that you don't need Autoconf in order to make use
of their targets. Then the Meson targets could just defer to that
implementation.
(sslfiles.mk took a similar approach, and I've been happy with it so
far. The init-po use case seems somewhat close, at least, since a
developer doesn't need to run it very often. Not sure about
update-po.)
--Jacob
Hi,
On Wed, 7 Oct 2026 at 19:58, Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
On Wed, Oct 7, 2026 at 9:40 AM Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
Yes, I agree that it is kind of nightmarish. I considered both options
and chose to use JSON files because, AFAIK, we want to remove the
autoconf system and use meson in the future. So, using JSON files
makes more sense to me from a maintainability perspective for this
reason.A third path might be to disentangle the NLS makefiles from
Makefile.global, so that you don't need Autoconf in order to make use
of their targets. Then the Meson targets could just defer to that
implementation.
Thanks for the suggestion! I think this is worth trying because using
existing nls.mk files tremendously reduces the #lines_changed and also
removes the need of helper scripts.
(sslfiles.mk took a similar approach, and I've been happy with it so
far. The init-po use case seems somewhat close, at least, since a
developer doesn't need to run it very often. Not sure about
update-po.)
Could you elaborate more on update-po? I didn't understand why running
it often would be a problem (I am also unsure if that is the case).
--
Regards,
Nazir Bilal Yavuz
Microsoft
Hi,
On Thu, 8 Oct 2026 at 12:54, Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
On Wed, 7 Oct 2026 at 19:58, Jacob Champion
<jacob.champion@enterprisedb.com> wrote:On Wed, Oct 7, 2026 at 9:40 AM Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
Yes, I agree that it is kind of nightmarish. I considered both options
and chose to use JSON files because, AFAIK, we want to remove the
autoconf system and use meson in the future. So, using JSON files
makes more sense to me from a maintainability perspective for this
reason.A third path might be to disentangle the NLS makefiles from
Makefile.global, so that you don't need Autoconf in order to make use
of their targets. Then the Meson targets could just defer to that
implementation.Thanks for the suggestion! I think this is worth trying because using
existing nls.mk files tremendously reduces the #lines_changed and also
removes the need of helper scripts.
After thinking more about that, I am not sure if that is an acceptable
solution. In the Postgres docs [1] we mention that 'GNU make' is not
needed when Postgres is built with meson and I think we don't want to
break this promise. Perhaps this is acceptable because the current
case is specific and not widely used.
(sslfiles.mk took a similar approach, and I've been happy with it so
far.
Also, I realized that there is no way to regenerate SSL files like
sslfiles.mk does :(.
--
Regards,
Nazir Bilal Yavuz
Microsoft
Hi,
On Thu, 8 Oct 2026 at 14:00, Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
(sslfiles.mk took a similar approach, and I've been happy with it so
far.Also, I realized that there is no way to regenerate SSL files like
sslfiles.mk does :(.
I wanted to clarify this sentence. I meant 'Currently, there is no
meson target for regenerating SSL files like sslfiles.mk does'. Sorry
for the noise.
--
Regards,
Nazir Bilal Yavuz
Microsoft
On Thu, Oct 8, 2026 at 4:01 AM Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
After thinking more about that, I am not sure if that is an acceptable
solution. In the Postgres docs [1] we mention that 'GNU make' is not
needed when Postgres is built with meson and I think we don't want to
break this promise.
I don't think it should be required for someone to run the build
itself. But realistically, saying "hey to perform this one-off
generation task as a contributor, you need GNU make as well" is not a
step backwards.
Perhaps this is acceptable because the current
case is specific and not widely used.
Right. That's what I was alluding to when I said I wasn't sure about
update-po -- I couldn't remember whether it was tangled into any of
the other build logic. I don't think it is, though.
Also, I realized that there is no way to regenerate SSL files like
sslfiles.mk does :(.
I don't think anyone has asked yet. It'd be a SMOP to have Meson
invoke the Makefile for you... but running Make manually when a new
certificate is sporadically added is a pretty small cost already.
Just my two cents.
Thanks,
--Jacob
On 07.10.26 17:35, Nazir Bilal Yavuz wrote:
I think the biggest problem is the nls.mk files. autoconf read these
files to generate *.pot files but meson can't read these nls.mk files
natively. So, my solution is switching to using nls.json files instead
of nls.mk files and having a Perl script so that the autoconf system
can parse these nls.json files and read them as nls.mk files as
before.
I would also be content if we ported init-po and update-po idiomatically
to meson and removed them from the makefiles. These targets are not
needed by most developers or package builders, so we could restrict how
they are available.
1. Can init-po and update-po meson targets regenerate *.pot files on
every run or should these files be tracked and regenerated only when
inputs change?
-- I am asking this because implementing the prior one is simpler.
Latter one requires more complicated changes.
It might not be super important, but it would be intellectually more
satisfying if they worked like normal file-updating targets. But I'm
suspicious why this is a question. I mean, if we didn't need them to be
timestamp-tracking makefile targets, we could have implemented them as
shell scripts and side-stepped this whole problem?
2. How closely must the output from meson and autoconf match?
-- Should meson preserve autoconf’s source-reference paths and
ordering exactly, or are differences acceptable? In particular,
generated sources can produce build-tree relative references through
their #line directives. Should we normalize them?
Yes, the build-tree relative references are already a problem with
autoconf vpath builds, and I would like to get that fixed.
Do you have anything else in mind?
There is also a the problem that whenever a source file is added, it
needs to be added to nls.mk manually. It would be nice to get rid of
that requirement, if we're going to make significant changes to this.
(This would also reduce the scope of nls.mk/nls.json/whatever.) There
have been one or two attempts to address this in the past, which didn't
work for some edge case reasons. I don't have them handy, but we should
look that up. The simple and obvious solution might not be sufficient.
On 2026-Oct-08, Peter Eisentraut wrote:
On 07.10.26 17:35, Nazir Bilal Yavuz wrote:
I think the biggest problem is the nls.mk files. autoconf read these
files to generate *.pot files but meson can't read these nls.mk files
natively. So, my solution is switching to using nls.json files instead
of nls.mk files and having a Perl script so that the autoconf system
can parse these nls.json files and read them as nls.mk files as
before.I would also be content if we ported init-po and update-po idiomatically to
meson and removed them from the makefiles. These targets are not needed by
most developers or package builders, so we could restrict how they are
available.
OK, that works for me. Most translators just download the .po files
from the babel.postgresql.org server, so they don't need the build
targets. Babel itself, of course, does need some way to run these, but
it can use meson just fine. (Except for branches 14 and 15, of course.)
1. Can init-po and update-po meson targets regenerate *.pot files on
every run or should these files be tracked and regenerated only when
inputs change?
-- I am asking this because implementing the prior one is simpler.
Latter one requires more complicated changes.It might not be super important, but it would be intellectually more
satisfying if they worked like normal file-updating targets. But I'm
suspicious why this is a question. I mean, if we didn't need them to be
timestamp-tracking makefile targets, we could have implemented them as shell
scripts and side-stepped this whole problem?
In practice we do generate these files on every run, because their
respective source files (essentially every single .c in the tree) change
with every commit; the only reason what we have doesn't change the files
continuously, is because those files are not source-tracked anywhere.
Now, that applies to the src/backend/po/*.pot files, which is where most
of the lively git activity takes place. It's not true for most other
tools in src/bin. But I suspect they are small enough that regenerating
whole files each time is not a problem either.
2. How closely must the output from meson and autoconf match?
-- Should meson preserve autoconf’s source-reference paths and
ordering exactly, or are differences acceptable? In particular,
generated sources can produce build-tree relative references through
their #line directives. Should we normalize them?Yes, the build-tree relative references are already a problem with autoconf
vpath builds, and I would like to get that fixed.
+1
Do you have anything else in mind?
There is also a the problem that whenever a source file is added, it needs
to be added to nls.mk manually. It would be nice to get rid of that
requirement, if we're going to make significant changes to this. (This would
also reduce the scope of nls.mk/nls.json/whatever.) There have been one or
two attempts to address this in the past, which didn't work for some edge
case reasons. I don't have them handy, but we should look that up. The
simple and obvious solution might not be sufficient.
I think the problem is that we have generated files whose strings we
want (say, generated by Perl code), and other generated files whose
strings we do not want (e.g., those generated from .l and .y files.)
Also, files like xlogreader.c have to be included in several different
catalogs. We did discuss having a way to generate catalogs by
catenating pieces from several source files though (src/common in
particular), to avoid duplicate translations.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"Escucha y olvidarás; ve y recordarás; haz y entenderás" (Confucio)
Hi,
On Thu, 8 Oct 2026 at 18:41, Peter Eisentraut <peter@eisentraut.org> wrote:
On 07.10.26 17:35, Nazir Bilal Yavuz wrote:
I think the biggest problem is the nls.mk files. autoconf read these
files to generate *.pot files but meson can't read these nls.mk files
natively. So, my solution is switching to using nls.json files instead
of nls.mk files and having a Perl script so that the autoconf system
can parse these nls.json files and read them as nls.mk files as
before.I would also be content if we ported init-po and update-po idiomatically
to meson and removed them from the makefiles. These targets are not
needed by most developers or package builders, so we could restrict how
they are available.
Do you mean removing init-po and update-po but preserving the
install-po in the makefiles? Or can we remove them altogether?
1. Can init-po and update-po meson targets regenerate *.pot files on
every run or should these files be tracked and regenerated only when
inputs change?
-- I am asking this because implementing the prior one is simpler.
Latter one requires more complicated changes.It might not be super important, but it would be intellectually more
satisfying if they worked like normal file-updating targets. But I'm
suspicious why this is a question. I mean, if we didn't need them to be
timestamp-tracking makefile targets, we could have implemented them as
shell scripts and side-stepped this whole problem?
I meant preserving that behavior in autoconf but not having it in
meson. You are right, I didn't consider using a shell script because I
was too focused on transferring the behavior.
I will work on ensuring it doesn't regenerate when inputs don't
change. If we dislike the result, we can consider another solution,
like you suggested.
2. How closely must the output from meson and autoconf match?
-- Should meson preserve autoconf’s source-reference paths and
ordering exactly, or are differences acceptable? In particular,
generated sources can produce build-tree relative references through
their #line directives. Should we normalize them?Yes, the build-tree relative references are already a problem with
autoconf vpath builds, and I would like to get that fixed.
Got it.
Do you have anything else in mind?
There is also a the problem that whenever a source file is added, it
needs to be added to nls.mk manually. It would be nice to get rid of
that requirement, if we're going to make significant changes to this.
(This would also reduce the scope of nls.mk/nls.json/whatever.) There
have been one or two attempts to address this in the past, which didn't
work for some edge case reasons. I don't have them handy, but we should
look that up. The simple and obvious solution might not be sufficient.
I found this thread 'make update-po@master stops at pg_upgrade' [1]/messages/by-id/flat/20220713.160853.453362706160476128.horikyota.ntt@gmail.com. I
will reply to that thread.
[1]: /messages/by-id/flat/20220713.160853.453362706160476128.horikyota.ntt@gmail.com
--
Regards,
Nazir Bilal Yavuz
Microsoft
Hi,
On Thu, 8 Oct 2026 at 19:10, Álvaro Herrera <alvherre@kurilemu.de> wrote:
On 2026-Oct-08, Peter Eisentraut wrote:
On 07.10.26 17:35, Nazir Bilal Yavuz wrote:
1. Can init-po and update-po meson targets regenerate *.pot files on
every run or should these files be tracked and regenerated only when
inputs change?
-- I am asking this because implementing the prior one is simpler.
Latter one requires more complicated changes.It might not be super important, but it would be intellectually more
satisfying if they worked like normal file-updating targets. But I'm
suspicious why this is a question. I mean, if we didn't need them to be
timestamp-tracking makefile targets, we could have implemented them as shell
scripts and side-stepped this whole problem?In practice we do generate these files on every run, because their
respective source files (essentially every single .c in the tree) change
with every commit; the only reason what we have doesn't change the files
continuously, is because those files are not source-tracked anywhere.Now, that applies to the src/backend/po/*.pot files, which is where most
of the lively git activity takes place. It's not true for most other
tools in src/bin. But I suspect they are small enough that regenerating
whole files each time is not a problem either.
I will first work on ensuring it doesn't regenerate when inputs don't
change. Then we can decide if the solution looks good or if we need
another one.
Do you have anything else in mind?
There is also a the problem that whenever a source file is added, it needs
to be added to nls.mk manually. It would be nice to get rid of that
requirement, if we're going to make significant changes to this. (This would
also reduce the scope of nls.mk/nls.json/whatever.) There have been one or
two attempts to address this in the past, which didn't work for some edge
case reasons. I don't have them handy, but we should look that up. The
simple and obvious solution might not be sufficient.I think the problem is that we have generated files whose strings we
want (say, generated by Perl code), and other generated files whose
strings we do not want (e.g., those generated from .l and .y files.)
I couldn't think of any solution that avoids manual maintenance. I
found a thread [1]/messages/by-id/flat/20220713.160853.453362706160476128.horikyota.ntt@gmail.com that uses wildcard to include each source file but
since there could be files that we explicitly don't want, there should
be still manual maintenance; right? Perhaps including all of the
source files and excluding some is easier than the reverse?
[1]: /messages/by-id/flat/20220713.160853.453362706160476128.horikyota.ntt@gmail.com
--
Regards,
Nazir Bilal Yavuz
Microsoft
On 2026-Oct-09, Nazir Bilal Yavuz wrote:
On Thu, 8 Oct 2026 at 19:10, Álvaro Herrera <alvherre@kurilemu.de> wrote:
On 2026-Oct-08, Peter Eisentraut wrote:
There is also a the problem that whenever a source file is added, it needs
to be added to nls.mk manually. It would be nice to get rid of that
requirement, if we're going to make significant changes to this.I think the problem is that we have generated files whose strings we
want (say, generated by Perl code), and other generated files whose
strings we do not want (e.g., those generated from .l and .y files.)I couldn't think of any solution that avoids manual maintenance. I
found a thread [1] that uses wildcard to include each source file but
since there could be files that we explicitly don't want, there should
be still manual maintenance; right? Perhaps including all of the
source files and excluding some is easier than the reverse?
I think a list of the files to ignore might work well, especially if
there's a base rule like ignoring .c files generated from .y/.l, so we
only need to list files to ignore in addition to those. (I suspect such
a list would be empty in most cases.)
By the way, here's a thread where we tried to do something related
previously:
/messages/by-id/20220713.160853.453362706160476128.horikyota.ntt@gmail.com
I think a takeaway from that thread is that searching for files in the
filesystem (or git ls-files) is not good enough. But maybe if you get
Meson to give you a list of all the source files which are compiled to
produce each program, that might work.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"La conclusión que podemos sacar de esos estudios es que
no podemos sacar ninguna conclusión de ellos" (Tanenbaum)