Feature Request: pg_replication_master()

On Dec 19, 2012 4:43 AM, "Josh Berkus" <josh@agliodbs.com> wrote:

Hackers,

Currently we can see each master's current replicas using
pg_stat_replication. However, there is no way from a replica, that I
know of, to figure out who its master is other than to look at
recovery.conf.

We should probably have a function, like pg_replication_master(), which
gives the host address of the current master. This would help DBAs for
large replication clusters a lot. Obviously, this would only work in
streaming.

This sounds like my previous suggestion of returning the primary conninfo
value, but with just ip. That one came with a pretty bad patch, and was
later postponed until we folded recovery.conf into the main configuration
file parsing. I'm not really sure what happened to that project? (the
configuration file one)

/Magnus

On 19 December 2012 06:10, Magnus Hagander <magnus@hagander.net> wrote:

This sounds like my previous suggestion of returning the primary conninfo
value, but with just ip. That one came with a pretty bad patch, and was
later postponed until we folded recovery.conf into the main configuration
file parsing. I'm not really sure what happened to that project? (the
configuration file one)

It stalled because the patch author decided not to implement the
request to detect recovery.conf in data directory, which allows
backwards compatibility.

I proposed a solution to how to do that, so we can move forwards if
people have time.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

It stalled because the patch author decided not to implement the
request to detect recovery.conf in data directory, which allows
backwards compatibility.

Well, I don't think we had agreement on how important backwards compatibility for recovery.conf was, particularly not on the whole recovery.conf/recovery.done functionality and the wierd formatting of recovery.conf.

However, with "include_if_exists" directives in postgresql.conf, or "include_dir", that would be easy to work around. Don't we have something like that planned for SET PERSISTENT?

--Josh Berkus

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

This sounds like my previous suggestion of returning the primary
conninfo value, but with just ip. That one came with a pretty bad
patch, and was later postponed until we folded recovery.conf into
the main configuration file parsing. I'm not really sure what
happened to that project? (the configuration file one)

Hmmm, good point. Just having primary_conninfo it in pg_settings would help a lot.

--Josh

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 19 December 2012 22:19, Joshua Berkus <josh@agliodbs.com> wrote:

It stalled because the patch author decided not to implement the
request to detect recovery.conf in data directory, which allows
backwards compatibility.

Well, I don't think we had agreement on how important backwards compatibility for recovery.conf was, particularly not on the whole recovery.conf/recovery.done functionality and the wierd formatting of recovery.conf.

As ever, we spent much energy on debating backwards compatibility
rather than just solving the problem it posed, which is fairly easy to
solve.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Wed, Dec 19, 2012 at 5:34 PM, Simon Riggs <simon@2ndquadrant.com> wrote:

As ever, we spent much energy on debating backwards compatibility
rather than just solving the problem it posed, which is fairly easy to
solve.

I'm still of the opinion (as were a lot of people on the previous
thread, IIRC) that just making them GUCs and throwing backward
compatibility under the bus is acceptable in this case. Changes that
break application code are anathema to me, because people can have a
LOT of application code and updating it can be REALLY hard. The same
cannot be said about recovery.conf - you have at most one of those per
standby, and if it needs to be changed in some way, you can do it with
a very small Perl script. Yes, third-party tools will need to be
updated; that is surely a downside, but I think it might be a
tolerable one in this case.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Thursday, December 20, 2012 3:50 AM Joshua Berkus wrote:

It stalled because the patch author decided not to implement the
request to detect recovery.conf in data directory, which allows
backwards compatibility.

Well, I don't think we had agreement on how important backwards
compatibility for recovery.conf was, particularly not on the whole
recovery.conf/recovery.done functionality and the wierd formatting of
recovery.conf.

However, with "include_if_exists" directives in postgresql.conf, or
"include_dir", that would be easy to work around. Don't we have
something like that planned for SET PERSISTENT?

Yes in SET PERSISTENT patch, it uses "include_dir".

I wonder why can't we get this information from WALRcvData structure?
It has the required information.

With Regards,
Amit Kapila.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Wed, Dec 19, 2012 at 07:32:33PM -0500, Robert Haas wrote:

On Wed, Dec 19, 2012 at 5:34 PM, Simon Riggs <simon@2ndquadrant.com> wrote:

As ever, we spent much energy on debating backwards compatibility
rather than just solving the problem it posed, which is fairly easy to
solve.

I'm still of the opinion (as were a lot of people on the previous
thread, IIRC) that just making them GUCs and throwing backward
compatibility under the bus is acceptable in this case. Changes that
break application code are anathema to me, because people can have a
LOT of application code and updating it can be REALLY hard. The same
cannot be said about recovery.conf - you have at most one of those per
standby, and if it needs to be changed in some way, you can do it with
a very small Perl script. Yes, third-party tools will need to be
updated; that is surely a downside, but I think it might be a
tolerable one in this case.

Agreed. We have always has a more lax requirement of changing the
Postgres administration interface. I am not saying to ignore backward
compatibility, but future admin interface clarity overrules backward
compatibility in most cases. If people are really worried about this,
they can write a Perl script to convert from the old to new format.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Wed, Dec 19, 2012 at 10:34:14PM +0000, Simon Riggs wrote:

On 19 December 2012 22:19, Joshua Berkus <josh@agliodbs.com> wrote:

It stalled because the patch author decided not to implement the
request to detect recovery.conf in data directory, which allows
backwards compatibility.

Well, I don't think we had agreement on how important backwards compatibility for recovery.conf was, particularly not on the whole recovery.conf/recovery.done functionality and the wierd formatting of recovery.conf.

As ever, we spent much energy on debating backwards compatibility
rather than just solving the problem it posed, which is fairly easy to
solve.

Let me also add that I am tired of having recovery.conf improvement
stalled by backward compatibility concerns. At this point, let's just
trash recovery.conf backward compatibility and move on.

And I don't want to hear complaints about tool breakage either. These
are external tools, not shipped with community Postgres, and they will
just have to adjust. I will be glad to beat all complainants into the
ground for the good of the community. ;-) We just can't operate like
this, and if we allowed these things to block us in the past, Postgres
would be a royal mess today!

At this point backward compatibility has paralized us from fixing a
recovery.conf API that everyone agrees is non-optimal, and this has
gone on for multiple major releases. I don't care what we have to do,
just clean this up for 9.3!

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Thu, Dec 20, 2012 at 02:29:49PM -0500, Bruce Momjian wrote:

Let me also add that I am tired of having recovery.conf improvement
stalled by backward compatibility concerns. At this point, let's just
trash recovery.conf backward compatibility and move on.

And I don't want to hear complaints about tool breakage either. These
are external tools, not shipped with community Postgres, and they will
just have to adjust. I will be glad to beat all complainants into the
ground for the good of the community. ;-) We just can't operate like
this, and if we allowed these things to block us in the past, Postgres
would be a royal mess today!

At this point backward compatibility has paralized us from fixing a
recovery.conf API that everyone agrees is non-optimal, and this has
gone on for multiple major releases. I don't care what we have to do,
just clean this up for 9.3!

Let me add two more things. First, no matter how many people you
interact with who use Postgres, there are many more who you do not
interact with. Please don't give excessive weight to those people you
know, or who use your tools, or who are your customers, and forget the
many more Postgres users who you will never know. They are not using
your tools or scripts --- they just want Postgres to be simple to use.

Second, no matter how successful Postgres is now, there are many more
users in our future, and we have a responsibility to give them a
database that is as easy to configure as possible, without hampering
them with decisions to avoid disruption for current Postgres users.

I am not saying we should ignore current users, or our customers or our
tool users, but it is very clear to me that we have lost the proper
balance in this area.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Let me also add that I am tired of having recovery.conf improvement
stalled by backward compatibility concerns. At this point, let's just
trash recovery.conf backward compatibility and move on.

And I don't want to hear complaints about tool breakage either. These are
external tools, not shipped with community Postgres, and they will just

have

to adjust. I will be glad to beat all complainants into the
ground for the good of the community. ;-) We just can't operate like
this, and if we allowed these things to block us in the past, Postgres

would be

a royal mess today!

At this point backward compatibility has paralized us from fixing a
recovery.conf API that everyone agrees is non-optimal, and this has gone

on

for multiple major releases. I don't care what we have to do, just clean

this

up for 9.3!

+1

And the sooner we do it before release, the more time will those external
tools have to adjust.

Regards
Petr Jelinek

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

As ever, we spent much energy on debating backwards compatibility
rather than just solving the problem it posed, which is fairly easy
to
solve.

Well, IIRC, the debate was primarily of *your* making. Almost everyone else on the thread was fine with the original patch, and it was nearly done for 9.2 before you stepped in. I can't find anyone else on that thread who thought that backwards compatibility was more important than fixing the API.

--Josh

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2012-12-18 19:43:09 -0800, Josh Berkus wrote:

We should probably have a function, like pg_replication_master(), which
gives the host address of the current master. This would help DBAs for
large replication clusters a lot. Obviously, this would only work in
streaming.

Do you want the node one step up or the top-level in the chain? Because
I don't think we can do the latter without complicating the replication
protocol noticably.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andreas,

Do you want the node one step up or the top-level in the chain?
Because
I don't think we can do the latter without complicating the
replication
protocol noticably.

Well, clearly a whole chain would be nice for the user. But even just one step up would be very useful.

--Josh

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Thu, Dec 20, 2012 at 5:07 PM, Joshua Berkus <josh@agliodbs.com> wrote:

As ever, we spent much energy on debating backwards compatibility
rather than just solving the problem it posed, which is fairly easy
to
solve.

Well, IIRC, the debate was primarily of *your* making. Almost everyone else on the thread was fine with the original patch, and it was nearly done for 9.2 before you stepped in. I can't find anyone else on that thread who thought that backwards compatibility was more important than fixing the API.

+1. Let's JFDI.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 20 December 2012 19:29, Bruce Momjian <bruce@momjian.us> wrote:

On Wed, Dec 19, 2012 at 10:34:14PM +0000, Simon Riggs wrote:

On 19 December 2012 22:19, Joshua Berkus <josh@agliodbs.com> wrote:

It stalled because the patch author decided not to implement the
request to detect recovery.conf in data directory, which allows
backwards compatibility.

Well, I don't think we had agreement on how important backwards compatibility for recovery.conf was, particularly not on the whole recovery.conf/recovery.done functionality and the wierd formatting of recovery.conf.

As ever, we spent much energy on debating backwards compatibility
rather than just solving the problem it posed, which is fairly easy to
solve.

Let me also add that I am tired of having recovery.conf improvement
stalled by backward compatibility concerns. At this point, let's just
trash recovery.conf backward compatibility and move on.

No, lets not.

The only stall happening is because of a refusal to listen to another
person's reasonable request during patch review. That requirement is
not a blocker to the idea, it just needs to be programmed.

Lets just implement the reasonable request for backwards
compatibility, rather than wasting time on reopening the debate.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 20 December 2012 19:29, Bruce Momjian <bruce@momjian.us> wrote:

At this point backward compatibility has paralized us from fixing a
recovery.conf API that everyone agrees is non-optimal, and this has
gone on for multiple major releases. I don't care what we have to do,
just clean this up for 9.3!

The main stall at this point is that the developer who wrote that
patch no longer spends much time working on Postgres. AFAICS there is
nobody working on this for 9.3 mainly because its not a priority, nor
will implementing that fix the OP's request.

There is no paralysis because there never was a blocker, only a
request for backwards compatibility, which is easily possible to
implement.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Dec 21, 2012 at 9:21 AM, Simon Riggs <simon@2ndquadrant.com> wrote:

No, lets not.

The only stall happening is because of a refusal to listen to another
person's reasonable request during patch review. That requirement is
not a blocker to the idea, it just needs to be programmed.

Lets just implement the reasonable request for backwards
compatibility, rather than wasting time on reopening the debate.

I read this as "let's do it the way I proposed, instead of the way
other people proposed". I don't see how that suggestion advances the
debate. If I recall correctly, and I might not, because it's been a
year, you wanted to implicitly include recovery.conf in
postgresql.conf only when the system is recovery mode, but that gave
rise to a bunch of thorny definitional issues that were never
adequately solved. I would have been willing to tolerate that
solution if they had been, but they were not. It is not accurate to
suggest that you presented a reasonable proposal and other people
refused to listen. That is not what happened.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 21 December 2012 17:12, Robert Haas <robertmhaas@gmail.com> wrote:

On Fri, Dec 21, 2012 at 9:21 AM, Simon Riggs <simon@2ndquadrant.com> wrote:

No, lets not.

The only stall happening is because of a refusal to listen to another
person's reasonable request during patch review. That requirement is
not a blocker to the idea, it just needs to be programmed.

Lets just implement the reasonable request for backwards
compatibility, rather than wasting time on reopening the debate.

I read this as "let's do it the way I proposed, instead of the way
other people proposed". I don't see how that suggestion advances the
debate. If I recall correctly, and I might not, because it's been a
year, you wanted to implicitly include recovery.conf in
postgresql.conf only when the system is recovery mode, but that gave
rise to a bunch of thorny definitional issues that were never
adequately solved. I would have been willing to tolerate that
solution if they had been, but they were not. It is not accurate to
suggest that you presented a reasonable proposal and other people
refused to listen. That is not what happened.

Having looked into how to solve it, I think its solvable easily enough.

If the energy expended on that thread, and now this one was directed
to actually solve the problem, it would be solved.

Characterising the problem as containing a bunch of thorny
definitional issues is just a way to further avoid that,
unintentionally or not.

"Whether you think you can or think you can't, either way you are
right" - Henry Ford

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers