pgsql: Allow using syncfs() in frontend utilities.

On Wed, Sep 6, 2023 at 7:28 PM Nathan Bossart <nathan@postgresql.org> wrote:

Allow using syncfs() in frontend utilities.

This commit allows specifying a --sync-method in several frontend
utilities that must synchronize many files to disk (initdb,
pg_basebackup, pg_checksums, pg_dump, pg_rewind, and pg_upgrade).
On Linux, users can specify "syncfs" to synchronize the relevant
file systems instead of calling fsync() for every single file. In
many cases, using syncfs() is much faster.

As with recovery_init_sync_method, this new option comes with some
caveats. The descriptions of these caveats have been moved to a
new appendix section in the documentation.

Hi,

I'd like to complain about this commit's addition of a new appendix. I
do understand the temptation to document caveats like this centrally
instead of in multiple places, but as I've been complaining about over
in the "documentation structure" thread, our top-level documentation
index is too big, and I feel strongly that we need to de-clutter it
rather than cluttering it further. This added a new chapter which is
just 5 sentences long. I understand that this was done because the
same issue applies to a bunch of different utilities and we didn't
want to duplicate this text in all of those places, but I feel like
this approach just doesn't scale. If we did this in every place where
we have this much text that we want to avoid duplicating, we'd soon
have hundreds of appendixes.

What I would suggest we do instead is pick one of the places where
this comes up and document it there, perhaps the
recovery_init_sync_method GUC. And then make the documentation for the
other say something like, you know those issues we documented for
recovery_init_sync_method? Well they also apply to this.

--
Robert Haas
EDB: http://www.enterprisedb.com

On 22.03.24 17:52, Robert Haas wrote:

On Wed, Sep 6, 2023 at 7:28 PM Nathan Bossart <nathan@postgresql.org> wrote:

Allow using syncfs() in frontend utilities.

This commit allows specifying a --sync-method in several frontend
utilities that must synchronize many files to disk (initdb,
pg_basebackup, pg_checksums, pg_dump, pg_rewind, and pg_upgrade).
On Linux, users can specify "syncfs" to synchronize the relevant
file systems instead of calling fsync() for every single file. In
many cases, using syncfs() is much faster.

As with recovery_init_sync_method, this new option comes with some
caveats. The descriptions of these caveats have been moved to a
new appendix section in the documentation.

Hi,

I'd like to complain about this commit's addition of a new appendix.

I already complained about that at
</messages/by-id/42804669-7063-1320-ed37-3226d5f1067d@eisentraut.org&gt;
and some follow-up was announced but didn't happen. It was on my list
to look into cleaning up during beta.

On Fri, Mar 22, 2024 at 12:52:15PM -0400, Robert Haas wrote:

I'd like to complain about this commit's addition of a new appendix. I
do understand the temptation to document caveats like this centrally
instead of in multiple places, but as I've been complaining about over
in the "documentation structure" thread, our top-level documentation
index is too big, and I feel strongly that we need to de-clutter it
rather than cluttering it further. This added a new chapter which is
just 5 sentences long. I understand that this was done because the
same issue applies to a bunch of different utilities and we didn't
want to duplicate this text in all of those places, but I feel like
this approach just doesn't scale. If we did this in every place where
we have this much text that we want to avoid duplicating, we'd soon
have hundreds of appendixes.

Sorry I missed this. I explored a couple of options last year but the
discussion trailed off [0]/messages/by-id/20231009204823.GA659480@nathanxps13.

What I would suggest we do instead is pick one of the places where
this comes up and document it there, perhaps the
recovery_init_sync_method GUC. And then make the documentation for the
other say something like, you know those issues we documented for
recovery_init_sync_method? Well they also apply to this.

WFM. I'll put together a patch.

[0]: /messages/by-id/20231009204823.GA659480@nathanxps13

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

On Tue, Mar 26, 2024 at 11:18:57AM +0100, Peter Eisentraut wrote:

On 22.03.24 17:52, Robert Haas wrote:

I'd like to complain about this commit's addition of a new appendix.

I already complained about that at </messages/by-id/42804669-7063-1320-ed37-3226d5f1067d@eisentraut.org&gt;
and some follow-up was announced but didn't happen. It was on my list to
look into cleaning up during beta.

Sorry about this, I lost track of it.

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

On Tue, Mar 26, 2024 at 10:11:31AM -0500, Nathan Bossart wrote:

On Tue, Mar 26, 2024 at 11:18:57AM +0100, Peter Eisentraut wrote:

On 22.03.24 17:52, Robert Haas wrote:

I'd like to complain about this commit's addition of a new appendix.

I already complained about that at </messages/by-id/42804669-7063-1320-ed37-3226d5f1067d@eisentraut.org&gt;
and some follow-up was announced but didn't happen. It was on my list to
look into cleaning up during beta.

Sorry about this, I lost track of it.

Here's a first attempt at a patch based on Robert's suggestion from
upthread.

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

On Tue, Mar 26, 2024 at 12:34 PM Nathan Bossart
<nathandbossart@gmail.com> wrote:

Here's a first attempt at a patch based on Robert's suggestion from
upthread.

WFM.

--
Robert Haas
EDB: http://www.enterprisedb.com

On Wed, Mar 27, 2024 at 10:41:42AM -0400, Robert Haas wrote:

On Tue, Mar 26, 2024 at 12:34 PM Nathan Bossart
<nathandbossart@gmail.com> wrote:

Here's a first attempt at a patch based on Robert's suggestion from
upthread.

WFM.

Committed. Again, I apologize this took so long.

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

On Wed, Mar 27, 2024 at 11:25 AM Nathan Bossart
<nathandbossart@gmail.com> wrote:

Committed. Again, I apologize this took so long.

No worries from my side; I only noticed recently. I guess Peter's been
waiting a while, though. Thanks for committing.

--
Robert Haas
EDB: http://www.enterprisedb.com