pg_receivexlog does not report flush position with --synchronous
Hi guys,
while adding synchronous WAL streaming to Barman, I noticed that
pg_receivexlog - unless a replication slot is specified and --synchronous
is passed - does not become a synchronous receiver (if the application_name
is in the synchronous_standby_names value). I was a bit surprised by this
behaviour.
By reading the pg_receivexlog documentation, I assumed that:
1) if I set application_name properly for pg_receivexlog (let's say
'barman_receive_wal')
2) then I set synchronous_standby_names so that barman_receive_wal is first
in the list
3) then I run pg_receivexlog with --synchronous
I would find the pg_receivexlog in 'sync' state in the
pg_stat_replication view on the master.
However, I kept receiving the 'async' state. After looking at the
documentation once more, I noticed that '--synchronous' was mentioned also
in the '--slot-name' option but the explanation - at least to me - was not
very clear.
I tried again by creating a replication slot and passing it to
pg_receivexlog and this time I could see 'sync' as streaming state.
Looking up the code in more details I see that, unless replication slot
are enabled, pg_receivexlog does not report the flush position (this is a
precaution that's been taken when '--synchronous' was probably not around).
Please find this very short patch which - in case replication slots are not
present but synchronous is - reports the flush position.
I am not sure if it is a bug or not. I any case I guess we should
probably improve the documentation - it's a bit late here so maybe I can
try better tomorrow with a fresher mind. :)
Thanks,
Gabriele
--
Gabriele Bartolini - 2ndQuadrant Italia - Director
PostgreSQL Training, Services and Support
gabriele.bartolini@2ndQuadrant.it | www.2ndQuadrant.it
Attachments:
0001-pg_receivexlog-does-not-report-flush-position-with-s.patchapplication/octet-stream; name=0001-pg_receivexlog-does-not-report-flush-position-with-s.patchDownload+5-10
On Tue, Aug 23, 2016 at 7:34 AM, Gabriele Bartolini
<gabriele.bartolini@2ndquadrant.it> wrote:
Looking up the code in more details I see that, unless replication slot
are enabled, pg_receivexlog does not report the flush position (this is a
precaution that's been taken when '--synchronous' was probably not around).
Please find this very short patch which - in case replication slots are not
present but synchronous is - reports the flush position.
Ouch.
I am not sure if it is a bug or not. I any case I guess we should
probably improve the documentation - it's a bit late here so maybe I can try
better tomorrow with a fresher mind. :)
Looking at c4f99d20, there is the following thing from the commit log:
This commit removes -F option from and adds --synchronous to pg_receivexlog.
If --synchronous is specified, like the standby's wal receiver, pg_receivexlog
flushes WAL data as soon as there is WAL data which has not been flushed yet.
Then it sends back the feedback message identifying the latest flush location
to the server. This option is useful to make pg_receivexlog behave as sync
standby by using replication slot, for example.
The docs also clearly state what --sync should do:
Also send a status packet back to the server immediately after
flushing, regardless of --status-interval.
So it means that you are right, this is a bug and we should report the
flush position back to the server so as pg_receivexlog can be thought
as a synchronous replica. Looking at your patch, I think that this is
the right shot, but I don't think that the first comment block should
be completely removed as it still applies when replication slots are
enabled. I'd suggest rewording it a bit instead, please see attached.
And of course this needs a backpatch.
--
Michael
Attachments:
receivelog-sync-fix.patchtext/plain; charset=US-ASCII; name=receivelog-sync-fix.patchDownload+9-2
On 23 August 2016 at 08:56, Michael Paquier <michael.paquier@gmail.com> wrote:
On Tue, Aug 23, 2016 at 7:34 AM, Gabriele Bartolini
<gabriele.bartolini@2ndquadrant.it> wrote:
I'd suggest rewording it a bit instead, please see attached.
And of course this needs a backpatch.
Agreed, but I'd move all the comments above the block.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
pg-receivexlog-not-report-sync-flush.v2.patchapplication/octet-stream; name=pg-receivexlog-not-report-sync-flush.v2.patchDownload+15-13
Hi Simon and Michael,
2016-08-23 10:39 GMT+02:00 Simon Riggs <simon@2ndquadrant.com>:
Agreed, but I'd move all the comments above the block.
That's fine with me.
Thanks,
Gabriele
--
Gabriele Bartolini - 2ndQuadrant Italia - Director
PostgreSQL Training, Services and Support
gabriele.bartolini@2ndQuadrant.it | www.2ndQuadrant.it
<http://www.2ndquadrant.it/>
On Tue, Aug 23, 2016 at 9:48 PM, Gabriele Bartolini
<gabriele.bartolini@2ndquadrant.it> wrote:
Hi Simon and Michael,
2016-08-23 10:39 GMT+02:00 Simon Riggs <simon@2ndquadrant.com>:
Agreed, but I'd move all the comments above the block.
That's fine with me.
+1.
--
Michael--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 23 August 2016 at 14:57, Michael Paquier <michael.paquier@gmail.com> wrote:
On Tue, Aug 23, 2016 at 9:48 PM, Gabriele Bartolini
<gabriele.bartolini@2ndquadrant.it> wrote:Hi Simon and Michael,
2016-08-23 10:39 GMT+02:00 Simon Riggs <simon@2ndquadrant.com>:
Agreed, but I'd move all the comments above the block.
That's fine with me.
+1.
Committed and backpatched to 9.5. Thank you both.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers