is the base backup protocol used by out-of-core tools?

On Tue, Feb 08, 2022 at 11:26:41AM -0500, Robert Haas wrote:

Patches for the nuking are attached. If nobody writes back, I'm going
to assume that means nobody cares, and commit these some time before
feature freeze. If one or more people do write back, then my plan is
to see what they have to say and proceed accordingly.

I created this and added that for visibility and so it's not forgotten.
ISTM that could be done post-freeze, although I don't know if that's useful or
important.

https://wiki.postgresql.org/wiki/PostgreSQL_15_Open_Items

--
Justin

On Tue, Feb 8, 2022 at 1:03 PM Justin Pryzby <pryzby@telsasoft.com> wrote:

I created this and added that for visibility and so it's not forgotten.
ISTM that could be done post-freeze, although I don't know if that's useful or
important.

https://wiki.postgresql.org/wiki/PostgreSQL_15_Open_Items

Thanks. I feel like this is in the department of things that are
unlikely to be an issue for anyone, but could be. But the set of
people who could care is basically just backup tool authors, so
hopefully they'll notice this thread, or that list.

--
Robert Haas
EDB: http://www.enterprisedb.com

On 2/8/22 12:39, Robert Haas wrote:

On Tue, Feb 8, 2022 at 1:03 PM Justin Pryzby <pryzby@telsasoft.com> wrote:

I created this and added that for visibility and so it's not forgotten.
ISTM that could be done post-freeze, although I don't know if that's useful or
important.

https://wiki.postgresql.org/wiki/PostgreSQL_15_Open_Items

Thanks. I feel like this is in the department of things that are
unlikely to be an issue for anyone, but could be. But the set of
people who could care is basically just backup tool authors, so
hopefully they'll notice this thread, or that list.

I'm aware of several tools that use pg_basebackup but they are using the
command-line tool, not the server protocol directly.

Personally, I'm in favor of simplifying the code on the server side. If
anyone is using the protocol directly (which I kind of doubt) I imagine
they would want to take advantage of all the new features that have been
added.

Regards,
-David

On Tue, Feb 8, 2022 at 1:52 PM David Steele <david@pgmasters.net> wrote:

I'm aware of several tools that use pg_basebackup but they are using the
command-line tool, not the server protocol directly.

Good to know.

Personally, I'm in favor of simplifying the code on the server side. If
anyone is using the protocol directly (which I kind of doubt) I imagine
they would want to take advantage of all the new features that have been
added.

I agree, and I'm glad you do too, but I thought it was worth asking.

--
Robert Haas
EDB: http://www.enterprisedb.com

On Tue, Feb 08, 2022 at 11:26:41AM -0500, Robert Haas wrote:

From that point of view, there's no downside to removing from the
server the old syntax for BASE_BACKUP and the old protocol for taking
backups. We can't remove anything from pg_basebackup, because it is
our practice to make new versions of pg_basebackup work with old
versions of the server. But the reverse is not true: an older
pg_basebackup will categorically refuse to work with a newer server
version. Therefore keeping the code for this stuff around in the
server has no value ... unless there is out-of-core code that (a) uses
the BASE_BACKUP command and (b) wouldn't immediately adopt the new
syntax and protocol anyway. If there is, we might want to keep the
backward-compatibility code around in the server for a few releases.
If not, we should probably nuke that code to simplify things and
reduce the maintenance burden.

This line of arguments looks sensible from here, so +1 for this
cleanup in the backend as of 15~. I am not sure if we should worry
about out-of-core tools that use replication commands, either, and the
new grammar is easy to adapt to.

FWIW, one backup tool maintained by NTT is pg_rman, which does not use
the replication protocol AFAIK:
https://github.com/ossc-db/pg_rman
Perhaps Horiguchi-san or Fujita-san have an opinion on that.
--
Michael

2022年2月9日(水) 11:21 Michael Paquier <michael@paquier.xyz>:

On Tue, Feb 08, 2022 at 11:26:41AM -0500, Robert Haas wrote:

From that point of view, there's no downside to removing from the
server the old syntax for BASE_BACKUP and the old protocol for taking
backups. We can't remove anything from pg_basebackup, because it is
our practice to make new versions of pg_basebackup work with old
versions of the server. But the reverse is not true: an older
pg_basebackup will categorically refuse to work with a newer server
version. Therefore keeping the code for this stuff around in the
server has no value ... unless there is out-of-core code that (a) uses
the BASE_BACKUP command and (b) wouldn't immediately adopt the new
syntax and protocol anyway. If there is, we might want to keep the
backward-compatibility code around in the server for a few releases.
If not, we should probably nuke that code to simplify things and
reduce the maintenance burden.

This line of arguments looks sensible from here, so +1 for this
cleanup in the backend as of 15~. I am not sure if we should worry
about out-of-core tools that use replication commands, either, and the
new grammar is easy to adapt to.

FWIW the out-of-core utility I have some involvement in and which uses base
backup functionality, uses this by executing pg_basebackup, so no problem
there. But even if it used the replication protocol directly, it'd
just be a case of
adding another bit of Pg version-specific code handling; I certainly wouldn't
expect core to hold itself back for my convenience.

Regards

Ian Barwick

--
EnterpriseDB: https://www.enterprisedb.com

At Wed, 9 Feb 2022 11:21:41 +0900, Michael Paquier <michael@paquier.xyz> wrote in

On Tue, Feb 08, 2022 at 11:26:41AM -0500, Robert Haas wrote:

From that point of view, there's no downside to removing from the
server the old syntax for BASE_BACKUP and the old protocol for taking
backups. We can't remove anything from pg_basebackup, because it is
our practice to make new versions of pg_basebackup work with old
versions of the server. But the reverse is not true: an older
pg_basebackup will categorically refuse to work with a newer server
version. Therefore keeping the code for this stuff around in the
server has no value ... unless there is out-of-core code that (a) uses
the BASE_BACKUP command and (b) wouldn't immediately adopt the new
syntax and protocol anyway. If there is, we might want to keep the
backward-compatibility code around in the server for a few releases.
If not, we should probably nuke that code to simplify things and
reduce the maintenance burden.

This line of arguments looks sensible from here, so +1 for this
cleanup in the backend as of 15~. I am not sure if we should worry
about out-of-core tools that use replication commands, either, and the
new grammar is easy to adapt to.

FWIW, one backup tool maintained by NTT is pg_rman, which does not use
the replication protocol AFAIK:
https://github.com/ossoc-db/pg_rman
Perhaps Horiguchi-san or Fujita-san have an opinion on that.

# Oh, the excessive 'o' perplexed me:p

Thanks for pining. AFAICS, as you see, pg_rman doesn't talk
basebackup protocol (nor even pg_basebackup command) as it supports
inremental backup. So there's no issue about the removal of old
syntax on our side.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

On Wed, Feb 9, 2022 at 3:14 AM Kyotaro Horiguchi
<horikyota.ntt@gmail.com> wrote:

Thanks for pining. AFAICS, as you see, pg_rman doesn't talk
basebackup protocol (nor even pg_basebackup command) as it supports
inremental backup. So there's no issue about the removal of old
syntax on our side.

Cool. Since the verdict here seems pretty unanimous and we've heard
from a good number of backup tool authors, I'll give this another day
or so and then commit, unless objections emerge before then.

--
Robert Haas
EDB: http://www.enterprisedb.com

Am Mittwoch, dem 09.02.2022 um 08:01 -0500 schrieb Robert Haas:

On Wed, Feb 9, 2022 at 3:14 AM Kyotaro Horiguchi
<horikyota.ntt@gmail.com> wrote:

Thanks for pining.  AFAICS, as you see, pg_rman doesn't talk
basebackup protocol (nor even pg_basebackup command) as it supports
inremental backup.  So there's no issue about the removal of old
syntax on our side.

Cool. Since the verdict here seems pretty unanimous and we've heard
from a good number of backup tool authors, I'll give this another day
or so and then commit, unless objections emerge before then.

A while ago i worked on a tool which talks the streaming protocol
directly. In the past there were already additions to the protocol
(like MANIFEST), so to support those optional features and multiple
major versions you already have to abstract the code to deal with that.

Having said that, i don't have any objections either to remove the old
protocol support.

Thanks
Bernd

On Wed, Feb 9, 2022 at 2:02 PM Robert Haas <robertmhaas@gmail.com> wrote:

On Wed, Feb 9, 2022 at 3:14 AM Kyotaro Horiguchi
<horikyota.ntt@gmail.com> wrote:

Thanks for pining. AFAICS, as you see, pg_rman doesn't talk
basebackup protocol (nor even pg_basebackup command) as it supports
inremental backup. So there's no issue about the removal of old
syntax on our side.

Cool. Since the verdict here seems pretty unanimous and we've heard
from a good number of backup tool authors, I'll give this another day
or so and then commit, unless objections emerge before then.

Late to the game, but here's another +1.

--
Magnus Hagander
Me: https://www.hagander.net/
Work: https://www.redpill-linpro.com/

On Wed, Feb 9, 2022 at 3:05 PM Magnus Hagander <magnus@hagander.net> wrote:

Late to the game, but here's another +1.

Not that late, thanks for chiming in.

Looks pretty unanimous, so I have committed the patches.

--
Robert Haas
EDB: http://www.enterprisedb.com