pg_utility ?

Started by Álvaro Herrera2 months ago12 messages
#1Álvaro Herrera
alvherre@kurilemu.de

Hello,

One of the things that came up during the pgconf.eu talk about REPACK,
proposed by Christoph Berg, is that adding another utility pg_repackdb
to run it from the command line adds more noise to an already noisy tool
neighbourhood. He asked, how about we instead add a generic tool to run
utility commands? So the user would be able to do things such as

pg_utility vacuum -t tab1 -t tab2 # what vacuumdb does
pg_utility analyze -t tab1 -t tab2 # what vacuumdb -Z does
pg_utility vacuum analyze -t tab1 -t tab2 # what vacuumdb -z does
pg_utility cluster -t tab1 -t tab2 # what clusterdb does
pg_utility reindex -t tab1 -t tab2 # what reindexdb does

Each such tool would retain more or less its current behavior, i.e., its
ability to run things in parallel, to discover tables to operate on
based on circumstances, to silently ignore objects depending on the user
lacking specific privilege bits, and so on.

This way, instead of an entire pg_repackdb tool, we would add just a new
mode to pg_utility:

pg_utility repack -t tab1 -t tab2 # what pg_repackdb would do

Christoph, can you please confirm that this is approximately what you
had in mind? Other names are of course possible.

I didn't immediately love this idea, but I'm not totally opposed to it
either, and perhaps it makes things better rather than adding yet
another very narrowly-focused tool. Also, pg_ctl already kinda has a
somewhat similar facet with its "pg_ctl init" mode.

Thanks

--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"Once again, thank you and all of the developers for your hard work on
PostgreSQL. This is by far the most pleasant management experience of
any database I've worked on." (Dan Harris)
http://archives.postgresql.org/pgsql-performance/2006-04/msg00247.php

#2Dmitry Dolgov
9erthalion6@gmail.com
In reply to: Álvaro Herrera (#1)
Re: pg_utility ?

On Tue, Nov 04, 2025 at 06:52:19PM +0100, �lvaro Herrera wrote:
One of the things that came up during the pgconf.eu talk about REPACK,
proposed by Christoph Berg, is that adding another utility pg_repackdb
to run it from the command line adds more noise to an already noisy tool
neighbourhood. He asked, how about we instead add a generic tool to run
utility commands? So the user would be able to do things such as

pg_utility vacuum -t tab1 -t tab2 # what vacuumdb does
pg_utility analyze -t tab1 -t tab2 # what vacuumdb -Z does
pg_utility vacuum analyze -t tab1 -t tab2 # what vacuumdb -z does
pg_utility cluster -t tab1 -t tab2 # what clusterdb does
pg_utility reindex -t tab1 -t tab2 # what reindexdb does

Each such tool would retain more or less its current behavior, i.e., its
ability to run things in parallel, to discover tables to operate on
based on circumstances, to silently ignore objects depending on the user
lacking specific privilege bits, and so on.

This way, instead of an entire pg_repackdb tool, we would add just a new
mode to pg_utility:

pg_utility repack -t tab1 -t tab2 # what pg_repackdb would do

FWIW I find the idea interesting, it would help structure the tooling
landscape. Looking around, looks like it's common to have some sort of
manager or a toolbox for similar purposes.

Would it only be allowed to run anything involving CMD_UTILITY, or are
"utility commands" meant here in more broader sense?

#3Sami Imseih
samimseih@gmail.com
In reply to: Álvaro Herrera (#1)
Re: pg_utility ?

pg_utility vacuum -t tab1 -t tab2 # what vacuumdb does
pg_utility analyze -t tab1 -t tab2 # what vacuumdb -Z does
pg_utility vacuum analyze -t tab1 -t tab2 # what vacuumdb -z does
pg_utility cluster -t tab1 -t tab2 # what clusterdb does
pg_utility reindex -t tab1 -t tab2 # what reindexdb does

Is the idea to get rid of most of bin/scripts and replace them with a single
pg_utility? or will it just be a wrapper for the existing utilities? meaning
they will still work stand-alone.

--
Sami Imseih
Amazon Web Services (AWS)

In reply to: Álvaro Herrera (#1)
Re: pg_utility ?

On Wed, Nov 5, 2025 at 4:29 PM Álvaro Herrera <alvherre@kurilemu.de> wrote:

I didn't immediately love this idea, but I'm not totally opposed to it
either, and perhaps it makes things better rather than adding yet
another very narrowly-focused tool. Also, pg_ctl already kinda has a
somewhat similar facet with its "pg_ctl init" mode.

I support this idea, moving these scattered commands under pg_ctl will
definitely help and bring maintenance commands in line with other
ecosystems where it's normal to follow the template of "[ecosystem]
[action] [arguments]".
Examples:
- docker run postgres
- pip install h3
- apt install postgres
- systemctl status postgresql
- pg_ctl start

In this pattern stuff like vacuumdb, createuser, createlang, createdb, ...
looks very off.

The same migration recently happened in imagemagick/graphicsmagick, where
instead of the old "convert" binary there is now "gm convert" that better
controls expectations (you won't attempt to convert to mp3 with it).

#5Christoph Berg
myon@debian.org
In reply to: Álvaro Herrera (#1)
Re: pg_utility ?

Re: �lvaro Herrera

Christoph, can you please confirm that this is approximately what you
had in mind? Other names are of course possible.

Sorry for the late reply, the mail got lost in my inbox.

Yes that's what I had in mind.

I didn't immediately love this idea, but I'm not totally opposed to it
either, and perhaps it makes things better rather than adding yet
another very narrowly-focused tool. Also, pg_ctl already kinda has a
somewhat similar facet with its "pg_ctl init" mode.

I would keep the server and client bits separate, though, so not merge
these into pg_ctl.

I don't have an idea for the ideal name, perhaps only that it should
be short, and distinct from pg_ctl so people don't get confused. (So
not pg_cmd or pg_cli.)

Perhaps pg_util? ("pg" is taken by that classic pager thingy.)

Christoph

#6Andreas Karlsson
andreas@proxel.se
In reply to: Christoph Berg (#5)
Re: pg_utility ?

On 11/19/25 12:52 PM, Christoph Berg wrote:

Re: Álvaro Herrera

I didn't immediately love this idea, but I'm not totally opposed to it
either, and perhaps it makes things better rather than adding yet
another very narrowly-focused tool. Also, pg_ctl already kinda has a
somewhat similar facet with its "pg_ctl init" mode.

I would keep the server and client bits separate, though, so not merge
these into pg_ctl.

I don't have an idea for the ideal name, perhaps only that it should
be short, and distinct from pg_ctl so people don't get confused. (So
not pg_cmd or pg_cli.)

Perhaps pg_util? ("pg" is taken by that classic pager thingy.)

I like the name pg_util. In the MySQL world it is called mysqladmin,
which is a does of pg_ctl and tools like createdb.

https://dev.mysql.com/doc/refman/9.5/en/mysqladmin.html

Maybe pg_util should only be for tools which connect to PostgreSQL over
the TCP (or a unix socket) while the all other tools, which need access
to the data directory, should have their own executables? Because in my
opinion we really have two kinds of frontend tools: those which need to
run on the same machine and with the same user as PostgreSQL and those
which connect to PostgreSQL, possibly from another machine, and run some
commands.

--
Andreas
Percona https://www.percona.com/

#7Christoph Berg
myon@debian.org
In reply to: Andreas Karlsson (#6)
Re: pg_utility ?

Re: Andreas Karlsson

Perhaps pg_util? ("pg" is taken by that classic pager thingy.)

I like the name pg_util. In the MySQL world it is called mysqladmin, which
is a does of pg_ctl and tools like createdb.

"pg_adm" would also be a candidate. (Or is that too close to pgadmin?)

Maybe pg_util should only be for tools which connect to PostgreSQL over the
TCP (or a unix socket) while the all other tools, which need access to the
data directory, should have their own executables?

That would be the idea, yes.

Christoph

#8Aleksander Alekseev
aleksander@tigerdata.com
In reply to: Álvaro Herrera (#1)
Re: pg_utility ?

Hi Álvaro,

One of the things that came up during the pgconf.eu talk about REPACK,
proposed by Christoph Berg, is that adding another utility pg_repackdb
to run it from the command line adds more noise to an already noisy tool
neighbourhood. He asked, how about we instead add a generic tool to run
utility commands? So the user would be able to do things such as

pg_utility vacuum -t tab1 -t tab2 # what vacuumdb does
pg_utility analyze -t tab1 -t tab2 # what vacuumdb -Z does
pg_utility vacuum analyze -t tab1 -t tab2 # what vacuumdb -z does
pg_utility cluster -t tab1 -t tab2 # what clusterdb does
pg_utility reindex -t tab1 -t tab2 # what reindexdb does

Each such tool would retain more or less its current behavior, i.e., its
ability to run things in parallel, to discover tables to operate on
based on circumstances, to silently ignore objects depending on the user
lacking specific privilege bits, and so on.

This way, instead of an entire pg_repackdb tool, we would add just a new
mode to pg_utility:

pg_utility repack -t tab1 -t tab2 # what pg_repackdb would do

Christoph, can you please confirm that this is approximately what you
had in mind? Other names are of course possible.

I didn't immediately love this idea, but I'm not totally opposed to it
either, and perhaps it makes things better rather than adding yet
another very narrowly-focused tool. Also, pg_ctl already kinda has a
somewhat similar facet with its "pg_ctl init" mode.

I'm not necessarily opposed to the idea in general, but I'm a bit
sceptical about the particular implementation. I fail to understand
the exact value of having "pg_utility (something)" instead of
"pg_(something)". To me it seems like we either end up supporting an
all-in-one utility, which will increase code complexity, or having an
additional utility which is going to be a wrapper for the existing
ones, which arguably has little value.

I get a feeling that the actual idea might be to refactor our CLI
tools to make the overall interface more consistent. Right now we have
several pg_* tools, and also tools like vacuumdb and createdb. Perhaps
what we might do instead is renaming/splitting the later ones into
pg_analyze, pg_vacuum, etc. Which of course is going to bring a
question about backward compatibility. I believe it can be provided by
symlinks or wrappers for several major releases.

--
Best regards,
Aleksander Alekseev

#9Andreas Karlsson
andreas@proxel.se
In reply to: Christoph Berg (#7)
Re: pg_utility ?

On 11/20/25 10:31 AM, Christoph Berg wrote:

Re: Andreas Karlsson

Perhaps pg_util? ("pg" is taken by that classic pager thingy.)

I like the name pg_util. In the MySQL world it is called mysqladmin, which
is a does of pg_ctl and tools like createdb.

"pg_adm" would also be a candidate. (Or is that too close to pgadmin?)

Yeah, that would be way too confusing.

Maybe pg_util should only be for tools which connect to PostgreSQL over the
TCP (or a unix socket) while the all other tools, which need access to the
data directory, should have their own executables?

That would be the idea, yes.

Then I like it!

Andreas

In reply to: Andreas Karlsson (#9)
Re: pg_utility ?

On Fri, Nov 21, 2025 at 4:42 AM Andreas Karlsson <andreas@proxel.se> wrote:

On 11/20/25 10:31 AM, Christoph Berg wrote:

Re: Andreas Karlsson

Perhaps pg_util? ("pg" is taken by that classic pager thingy.)

I like the name pg_util. In the MySQL world it is called mysqladmin,

which

is a does of pg_ctl and tools like createdb.

"pg_adm" would also be a candidate. (Or is that too close to pgadmin?)

Yeah, that would be way too confusing.

For the sake of simpler dictation over voice media when debugging over
phone can we please have no underscore in the command, it adds an extra
word to say.

Maybe consider:
- pgdo (pgdo vacuum table ...., as in sudo)
- pgops (pgops repack table ...)

Thanks,
Darafei.

#11Christoph Berg
myon@debian.org
In reply to: Darafei "Komяpa" Praliaskouski (#10)
Re: pg_utility ?

Re: Darafei "Komяpa" Praliaskouski

For the sake of simpler dictation over voice media when debugging over
phone can we please have no underscore in the command, it adds an extra
word to say.

Simpler as in "oh no extra underscore after pg in this one, all the
other commands like pg_ctl, pg_basebackup and pg_dump have one, but
this one does not, please remove it"? :)

Christoph

In reply to: Christoph Berg (#11)
Re: pg_utility ?

On Sun, Nov 23, 2025 at 1:51 PM Christoph Berg <myon@debian.org> wrote:

Re: Darafei "Komяpa" Praliaskouski

For the sake of simpler dictation over voice media when debugging over
phone can we please have no underscore in the command, it adds an extra
word to say.

Simpler as in "oh no extra underscore after pg in this one, all the
other commands like pg_ctl, pg_basebackup and pg_dump have one, but
this one does not, please remove it"? :)

This is also true.

Probably a perfect (but invasive) solution will be to reclaim `postgres`
binary name as user command line entry point. So things can become
`postgres server restart`, `postgres vacuum table`, `postgres cluster
something`. If done in a pluggable manner others will be able to get under
the same umbrella.

Thanks,
Darafei.