Why forcing Hot_standby_feedback to be enabled when creating a logical decoding slot on standby

On Mon, Jun 12, 2017 at 3:16 PM, sanyam jain <sanyamjain22@live.in> wrote:

I have created a logical decoding slot on a standby but i haven't enabled
Hot_standby_feedback.What are the test cases where this setup will fail?

hot_standby_feedback needs to be enabled at all times in logical
decoding so as the node does not remove rows that are still needed for
the decoding, and a VACUUM passing by with a minimal xmin too high
would cause inconsistent decoded data.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Hi,

On 2017-06-12 06:16:50 +0000, sanyam jain wrote:

I have created a logical decoding slot on a standby but i haven't enabled Hot_standby_feedback.What are the test cases where this setup will fail?

Creating logic slots on the standby is not supported... It'll simply
currently not work correctly, even if you circumvent the protections
(like not being able to create one on a standby).

- Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Isn't XLogRecord carries full information to be decoded in itself?If so a VACCUM should not be a problem in decoding?

Thanks
Sanyam Jain

________________________________
From: Michael Paquier <michael.paquier@gmail.com>
Sent: Monday, June 12, 2017 6:52:06 AM
To: sanyam jain
Cc: Pg Hackers
Subject: Re: [HACKERS] Why forcing Hot_standby_feedback to be enabled when creating a logical decoding slot on standby

On Mon, Jun 12, 2017 at 3:16 PM, sanyam jain <sanyamjain22@live.in> wrote:

I have created a logical decoding slot on a standby but i haven't enabled
Hot_standby_feedback.What are the test cases where this setup will fail?

hot_standby_feedback needs to be enabled at all times in logical
decoding so as the node does not remove rows that are still needed for
the decoding, and a VACUUM passing by with a minimal xmin too high
would cause inconsistent decoded data.
--
Michael

Hi,

On 2017-06-16 06:31:03 +0000, sanyam jain wrote:

Isn't XLogRecord carries full information to be decoded in itself?If so a VACCUM should not be a problem in decoding?

First: Please don't full quote emails.
Secondly: You've not actually explained what you want to do, nor what
your precise question itself is. Nor why Michael or my answer wasn't
sufficient.

Logicla decoding looks at the catalogs for metadata - otherwise the WAL
would need to contain a lot more information and thus be more voluminous
- so the records do *not* contain the "full information".

Please start at the beginning and explain what you're trying to do
(since you apparently did some hacking to get a slot on a standby) and
what you're trying to achieve, instead of one-sentence questions.

- Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Hi,
I am trying to create logical decoding slot on a standby.
Enabling Hot_standby_feedback on standby is one of the requirement ; which will delay vacuuming of stale data on Master server.
I am working on a hack to avoid Hot_standby_feedback so that Standby doesn't have any dependency on Master (except receiving WAL).

Hot_standby_feedback restricts Master to do early vacuuming of catalog relation which will be in decoding WAL record using "time travelling snapshot" on a Standby.

The other way to prevent early vacuuming on Standby can be by pausing recovery on Standby when a vacuum record belonging to catalog relation is encountered.
And when walsender process belonging to logical slot on Standby reaches this record it will resume the recovery by executing SetRecoveryPause(false).

To check whether VACUUM record belongs to a catalog relation i simply check if relationID < 10000.

This hack will only work for a single logical slot on standby.
Pausing recovery will increase the size of pg_xlog directory as walreceiver will continue receiving WAL.
Querying standby might result in wrong output.

Thanks,
Sanyam Jain