Spurious pgstat_drop_replslot() call

On Fri, Mar 08, 2024 at 10:19:11AM +0000, Bertrand Drouvot wrote:

Indeed, it does not seem appropriate to remove stats during slot invalidation as
one could still be interested to look at them.

Yeah, my take is that this can still be interesting to know, at least
for debugging. This would limit the stats to be dropped when the slot
is dropped, and that looks like a sound idea.

This spurious call has been introduced in be87200efd. I think that initially we
designed to drop slots when a recovery conflict occurred during logical decoding
on a standby. But we changed our mind to invalidate such a slot instead.

The spurious call is probably due to the initial design but has not been removed.

This is not a subject that has really been touched on the original
thread mentioned on the commit, so it is a bit hard to be sure. The
only comment is that a previous version of the patch did the stats
drop in the slot invalidation path at an incorrect location.

I don't think it's worth to add a test but can do if one feel the need.

Well, that would not be complicated while being cheap, no? We have a
few paths in the 035 test where we know that a slot has been
invalidated so it is just a matter of querying once
pg_stat_replication_slot and see if some data is still there.
--
Michael

Hi,

On Fri, Mar 08, 2024 at 07:55:39PM +0900, Michael Paquier wrote:

On Fri, Mar 08, 2024 at 10:19:11AM +0000, Bertrand Drouvot wrote:

Indeed, it does not seem appropriate to remove stats during slot invalidation as
one could still be interested to look at them.

Yeah, my take is that this can still be interesting to know, at least
for debugging. This would limit the stats to be dropped when the slot
is dropped, and that looks like a sound idea.

Thanks for looking at it!

This spurious call has been introduced in be87200efd. I think that initially we
designed to drop slots when a recovery conflict occurred during logical decoding
on a standby. But we changed our mind to invalidate such a slot instead.

The spurious call is probably due to the initial design but has not been removed.

This is not a subject that has really been touched on the original
thread mentioned on the commit, so it is a bit hard to be sure. The
only comment is that a previous version of the patch did the stats
drop in the slot invalidation path at an incorrect location.

The switch in the patch from "drop" to "invalidation" is in [1]/messages/by-id/26c6f320-98f0-253c-f8b5-df1e7c1f6315@amazon.com, see:

"
Given the precedent of max_slot_wal_keep_size, I think it's wrong to
just drop
the logical slots. Instead we should just mark them as
invalid,
like InvalidateObsoleteReplicationSlots().

Makes fully sense and done that way in the attached patch.
“

But yeah, hard to be sure why this call is there, at least I don't remember...

I don't think it's worth to add a test but can do if one feel the need.

Well, that would not be complicated while being cheap, no? We have a
few paths in the 035 test where we know that a slot has been
invalidated so it is just a matter of querying once
pg_stat_replication_slot and see if some data is still there.

We can not be 100% sure that the stats are up to date when the process holding
the active slot is killed.

So v2 attached adds a test where we ensure first that we have non empty stats
before triggering the invalidation.

[1]: /messages/by-id/26c6f320-98f0-253c-f8b5-df1e7c1f6315@amazon.com

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

On Fri, Mar 08, 2024 at 03:04:10PM +0000, Bertrand Drouvot wrote:

The switch in the patch from "drop" to "invalidation" is in [1], see:

"
Given the precedent of max_slot_wal_keep_size, I think it's wrong to
just drop
the logical slots. Instead we should just mark them as
invalid,
like InvalidateObsoleteReplicationSlots().

Makes fully sense and done that way in the attached patch.
“

But yeah, hard to be sure why this call is there, at least I don't remember...

Yep, noticed that on Friday.

We can not be 100% sure that the stats are up to date when the process holding
the active slot is killed.

So v2 attached adds a test where we ensure first that we have non empty stats
before triggering the invalidation.

Ah, that explains the extra poll. What you have done here makes sense
to me, and the new test fails immediately if I add back the stats drop
in the invalidation code path.

That's a slight change in behavior, unfortunately, and it cannot be
called a bug as this improves the visibility of the stats after an
invalidation, so this is not something that can be backpatched.
--
Michael

Hi,

On Mon, Mar 11, 2024 at 04:15:40PM +0900, Michael Paquier wrote:

That's a slight change in behavior, unfortunately, and it cannot be
called a bug as this improves the visibility of the stats after an
invalidation, so this is not something that can be backpatched.

Yeah, makes sense to me.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

On Mon, Mar 11, 2024 at 04:15:40PM +0900, Michael Paquier wrote:

That's a slight change in behavior, unfortunately, and it cannot be
called a bug as this improves the visibility of the stats after an
invalidation, so this is not something that can be backpatched.

I've looked again at that and that was OK on a second look. May I
suggest the attached additions with LWLockHeldByMeInMode to make sure
that the stats are dropped and created while we hold
ReplicationSlotAllocationLock?
--
Michael

Hi,

On Tue, Mar 12, 2024 at 02:36:58PM +0900, Michael Paquier wrote:

On Mon, Mar 11, 2024 at 04:15:40PM +0900, Michael Paquier wrote:

That's a slight change in behavior, unfortunately, and it cannot be
called a bug as this improves the visibility of the stats after an
invalidation, so this is not something that can be backpatched.

I've looked again at that and that was OK on a second look.

Thanks!

May I
suggest the attached additions with LWLockHeldByMeInMode to make sure
that the stats are dropped and created while we hold
ReplicationSlotAllocationLock?

Yeah, makes fully sense and looks good to me.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Hello Bertrand and Michael,

12.03.2024 09:17, Bertrand Drouvot wrote:

May I
suggest the attached additions with LWLockHeldByMeInMode to make sure
that the stats are dropped and created while we hold
ReplicationSlotAllocationLock?

Yeah, makes fully sense and looks good to me.

Sorry for a bit off-topic, but I've remembered an anomaly with a similar
assert:
/messages/by-id/17947-b9554521ad963c9c@postgresql.org

Maybe you would find it worth considering while working in this area...
(I've just run that reproducer on b36fbd9f8 and confirmed that the
assertion failure is still here.)

Best regards,
Alexander

On Tue, Mar 12, 2024 at 12:00:00PM +0300, Alexander Lakhin wrote:

Sorry for a bit off-topic, but I've remembered an anomaly with a similar
assert:
/messages/by-id/17947-b9554521ad963c9c@postgresql.org

Thanks for the reminder. The invalidation path with the stats drop is
only in 16~.

Maybe you would find it worth considering while working in this area...
(I've just run that reproducer on b36fbd9f8 and confirmed that the
assertion failure is still here.)

Indeed, something needs to happen. I am not surprised that it still
reproduces; nothing has changed with the locking of the stats entries.
:/
--
Michael