From 7d4425744e2bc96418d6ef85c9408bed9e91aa9b Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Wed, 25 Jan 2023 15:39:23 +0900 Subject: [PATCH v11 1/3] doc: Refactor the chapter related to archive modules to be WAL modules Archive modules are now a subsection of this more generic section, and recovery modules would be a second subsection of it. --- doc/src/sgml/archive-modules.sgml | 136 -------------------------- doc/src/sgml/backup.sgml | 2 +- doc/src/sgml/basic-wal-module.sgml | 2 +- doc/src/sgml/config.sgml | 2 +- doc/src/sgml/filelist.sgml | 2 +- doc/src/sgml/postgres.sgml | 2 +- doc/src/sgml/system-views.sgml | 2 +- doc/src/sgml/wal-modules.sgml | 149 +++++++++++++++++++++++++++++ 8 files changed, 155 insertions(+), 142 deletions(-) delete mode 100644 doc/src/sgml/archive-modules.sgml create mode 100644 doc/src/sgml/wal-modules.sgml diff --git a/doc/src/sgml/archive-modules.sgml b/doc/src/sgml/archive-modules.sgml deleted file mode 100644 index 1a32006e2c..0000000000 --- a/doc/src/sgml/archive-modules.sgml +++ /dev/null @@ -1,136 +0,0 @@ - - - - Archive Modules - - Archive Modules - - - - PostgreSQL provides infrastructure to create custom modules for continuous - archiving (see ). While archiving via - a shell command (i.e., ) is much - simpler, a custom archive module will often be considerably more robust and - performant. - - - - When a custom is configured, PostgreSQL - will submit completed WAL files to the module, and the server will avoid - recycling or removing these WAL files until the module indicates that the files - were successfully archived. It is ultimately up to the module to decide what - to do with each WAL file, but many recommendations are listed at - . - - - - Archiving modules must at least consist of an initialization function (see - ) and the required callbacks (see - ). However, archive modules are - also permitted to do much more (e.g., declare GUCs and register background - workers). - - - - The contrib/basic_wal_module module contains a working - example, which demonstrates some useful techniques. - - - - Initialization Functions - - _PG_archive_module_init - - - An archive library is loaded by dynamically loading a shared library with the - 's name as the library base name. The - normal library search path is used to locate the library. To provide the - required archive module callbacks and to indicate that the library is - actually an archive module, it needs to provide a function named - _PG_archive_module_init. This function is passed a - struct that needs to be filled with the callback function pointers for - individual actions. - - -typedef struct ArchiveModuleCallbacks -{ - ArchiveCheckConfiguredCB check_configured_cb; - ArchiveFileCB archive_file_cb; - ArchiveShutdownCB shutdown_cb; -} ArchiveModuleCallbacks; -typedef void (*ArchiveModuleInit) (struct ArchiveModuleCallbacks *cb); - - - Only the archive_file_cb callback is required. The - others are optional. - - - - - Archive Module Callbacks - - The archive callbacks define the actual archiving behavior of the module. - The server will call them as required to process each individual WAL file. - - - - Check Callback - - The check_configured_cb callback is called to determine - whether the module is fully configured and ready to accept WAL files (e.g., - its configuration parameters are set to valid values). If no - check_configured_cb is defined, the server always - assumes the module is configured. - - -typedef bool (*ArchiveCheckConfiguredCB) (void); - - - If true is returned, the server will proceed with - archiving the file by calling the archive_file_cb - callback. If false is returned, archiving will not - proceed, and the archiver will emit the following message to the server log: - -WARNING: archive_mode enabled, yet archiving is not configured - - In the latter case, the server will periodically call this function, and - archiving will proceed only when it returns true. - - - - - Archive Callback - - The archive_file_cb callback is called to archive a - single WAL file. - - -typedef bool (*ArchiveFileCB) (const char *file, const char *path); - - - If true is returned, the server proceeds as if the file - was successfully archived, which may include recycling or removing the - original WAL file. If false is returned, the server will - keep the original WAL file and retry archiving later. - file will contain just the file name of the WAL - file to archive, while path contains the full - path of the WAL file (including the file name). - - - - - Shutdown Callback - - The shutdown_cb callback is called when the archiver - process exits (e.g., after an error) or the value of - changes. If no - shutdown_cb is defined, no special action is taken in - these situations. - - -typedef void (*ArchiveShutdownCB) (void); - - - - - diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml index be05a33205..12d9bb3f81 100644 --- a/doc/src/sgml/backup.sgml +++ b/doc/src/sgml/backup.sgml @@ -660,7 +660,7 @@ test ! -f /mnt/server/archivedir/00000001000000A900000065 && cp pg_wal/0 than writing a shell command. However, archive modules can be more performant than archiving via shell, and they will have access to many useful server resources. For more information about archive modules, see - . + . diff --git a/doc/src/sgml/basic-wal-module.sgml b/doc/src/sgml/basic-wal-module.sgml index c418b01eb8..f972566374 100644 --- a/doc/src/sgml/basic-wal-module.sgml +++ b/doc/src/sgml/basic-wal-module.sgml @@ -12,7 +12,7 @@ This module copies completed WAL segment files to the specified directory. This may not be especially useful, but it can serve as a starting point for developing your own archive module. For more information about archive - modules, see . + modules, see . diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index f985afc009..bba24bdcb8 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -3666,7 +3666,7 @@ include_dir 'conf.d' shared library is used for archiving. The WAL archiver process is restarted by the postmaster when this parameter changes. For more information, see and - . + . This parameter can only be set in the diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml index 2d36c34ce8..31b4dc8fba 100644 --- a/doc/src/sgml/filelist.sgml +++ b/doc/src/sgml/filelist.sgml @@ -100,7 +100,7 @@ - + diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml index 2e271862fc..817e774155 100644 --- a/doc/src/sgml/postgres.sgml +++ b/doc/src/sgml/postgres.sgml @@ -233,7 +233,7 @@ break is not needed in a wider output rendering. &bgworker; &logicaldecoding; &replication-origins; - &archive-modules; + &wal-modules; diff --git a/doc/src/sgml/system-views.sgml b/doc/src/sgml/system-views.sgml index 7c8fc3f654..e93e733051 100644 --- a/doc/src/sgml/system-views.sgml +++ b/doc/src/sgml/system-views.sgml @@ -3351,7 +3351,7 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx , a call to a C function in the extension, or the LOAD command). - For example, since archive modules + For example, since archive modules are normally loaded only by the archiver process not regular sessions, this view will not display any customized options defined by such modules unless special action is taken to load them into the backend process diff --git a/doc/src/sgml/wal-modules.sgml b/doc/src/sgml/wal-modules.sgml new file mode 100644 index 0000000000..283bba782d --- /dev/null +++ b/doc/src/sgml/wal-modules.sgml @@ -0,0 +1,149 @@ + + + + Write-Ahead Log Modules + + Write-Ahead Log Modules + + + + This chapter provides general information about writing WAL modules, related + to the manipulation of WAL segments by the backend. + + + + Archive Modules + + Archive Modules + + + + PostgreSQL provides infrastructure to create custom modules for continuous + archiving (see ). While archiving via + a shell command (i.e., ) is much + simpler, a custom WAL module will often be considerably more robust and + performant. + + + + When a custom is configured, PostgreSQL + will submit completed WAL files to the module, and the server will avoid + recycling or removing these WAL files until the module indicates that the + files were successfully archived. It is ultimately up to the module to + decide what to do with each WAL file, but many recommendations are listed at + . + + + + Archiving modules must at least consist of an initialization function (see + ) and the required callbacks (see + ). However, archive modules are + also permitted to do much more (e.g., declare GUCs and register background + workers). + + + + The contrib/basic_archive module contains a working + example, which demonstrates some useful techniques. + + + + Initialization Functions + + _PG_archive_module_init + + + An archive library is loaded by dynamically loading a shared library with + the 's name as the library base name. + The normal library search path is used to locate the library. To provide + the required archive module callbacks and to indicate that the library is + actually an archive module, it needs to provide a function named + _PG_archive_module_init. This function is passed a + struct that needs to be filled with the callback function pointers for + individual actions. + + +typedef struct ArchiveModuleCallbacks +{ + ArchiveCheckConfiguredCB check_configured_cb; + ArchiveFileCB archive_file_cb; + ArchiveShutdownCB shutdown_cb; +} ArchiveModuleCallbacks; +typedef void (*ArchiveModuleInit) (struct ArchiveModuleCallbacks *cb); + + + Only the archive_file_cb callback is required. The + others are optional. + + + + + Archive Module Callbacks + + The archive callbacks define the actual archiving behavior of the module. + The server will call them as required to process each individual WAL file. + + + + Check Callback + + The check_configured_cb callback is called to + determine whether the module is fully configured and ready to accept WAL + files (e.g., its configuration parameters are set to valid values). If no + check_configured_cb is defined, the server always + assumes the module is configured. + + +typedef bool (*ArchiveCheckConfiguredCB) (void); + + + If true is returned, the server will proceed with + archiving the file by calling the archive_file_cb + callback. If false is returned, archiving will not + proceed, and the archiver will emit the following message to the server + log: + +WARNING: archive_mode enabled, yet archiving is not configured + + In the latter case, the server will periodically call this function, and + archiving will proceed only when it returns true. + + + + + Archive Callback + + The archive_file_cb callback is called to archive a + single WAL file. + + +typedef bool (*ArchiveFileCB) (const char *file, const char *path); + + + If true is returned, the server proceeds as if the file + was successfully archived, which may include recycling or removing the + original WAL file. If false is returned, the server + will keep the original WAL file and retry archiving later. + file will contain just the file name of the WAL + file to archive, while path contains the full + path of the WAL file (including the file name). + + + + + Shutdown Callback + + The shutdown_cb callback is called when the archiver + process exits (e.g., after an error) or the value of + changes. If no + shutdown_cb is defined, no special action is taken in + these situations. + + +typedef void (*ArchiveShutdownCB) (void); + + + + + + -- 2.39.0