Standby problem after restore_command Implementation

Started by Edson F. Lidorioalmost 11 years ago13 messagesgeneral
Jump to latest
#1Edson F. Lidorio
edson@openmailbox.org

Hello,

I'm having trouble on Standby after the implementation of the
restore_command.
I performed all the settings and it worked normally and after restart
the slave server, started to generate errors in the log of the slave:

I'm using Debian 8 with PostgreSQL 9.4.1 on x86_64-unknown-linux-gnu,
compiled by gcc-4.9. real (Debian 4.9.2-10) 4.9.2, 64-bit

Slave error log:
5/3/2015 16:46:01 BRT [10210-1] @ Replicator [unknown] error: WAL
segment requested 00000001000000000000002C has been removed
5/3/2015 16:46:05 BRT [10211-1] @ Replicator [unknown] error: WAL
segment requested 00000001000000000000002C has been removed
5/3/2015 16:46:10 BRT [10214-1] @ Replicator [unknown] error: WAL
segment requested 00000001000000000000002C has been removed
5/3/2015 16:46:15 BRT [10216-1] @ Replicator [unknown] error: WAL
segment requested 00000001000000000000002C has been removed

Master error log
5/3/2015 19:13:35 BRT [3339-1] @ Replicator [unknown] error: WAL segment
requested 00000001000000000000002C has been removed
5/3/2015 19:13:40 BRT [3341-1] @ Replicator [unknown] error: WAL segment
requested 00000001000000000000002C has been removed
5/3/2015 19:13:44 BRT [3343-1] @ Replicator [unknown] error: WAL segment
requested 00000001000000000000002C has been removed

Settings files are as follows:

master

postgresql.conf

listen_addresses = '*'
wal_level = hot_standby
archive_mode = on
archive_command = 'cp "%p" /mnt/server/archivedir/"%f"'
max_wal_senders = 2
wal_keep_segments = 20

pg_hba.conf
host replication replicador 192.168.0.112/32 trust

secondary

postgresql.conf

listen_addresses = '*'
hot_standby = on

pg_hba.conf
host all all 0.0.0.0/0 md5

recover.conf em (/var/lib/postgresql/9.4/main)

standby_mode=on
primary_conninfo='host=192.168.0.100 user=replicador application_name=
jessie-stby'
trigger_file='/tmp/pgtrigger'
restore_command = 'cp /mnt/server/archivedir/%f %p'
archive_cleanup_command = 'pg_archivecleanup /mnt/server/archivedir %r'

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#2Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Edson F. Lidorio (#1)
Re: Standby problem after restore_command Implementation

On 05/03/2015 05:57 PM, Edson F. Lidorio wrote:

Hello,

I'm having trouble on Standby after the implementation of the
restore_command.
I performed all the settings and it worked normally and after restart
the slave server, started to generate errors in the log of the slave:

So is that implying that you had the standby running without the
restore_command?

I'm using Debian 8 with PostgreSQL 9.4.1 on x86_64-unknown-linux-gnu,
compiled by gcc-4.9. real (Debian 4.9.2-10) 4.9.2, 64-bit

Slave error log:
5/3/2015 16:46:01 BRT [10210-1] @ Replicator [unknown] error: WAL
segment requested 00000001000000000000002C has been removed
5/3/2015 16:46:05 BRT [10211-1] @ Replicator [unknown] error: WAL
segment requested 00000001000000000000002C has been removed
5/3/2015 16:46:10 BRT [10214-1] @ Replicator [unknown] error: WAL
segment requested 00000001000000000000002C has been removed
5/3/2015 16:46:15 BRT [10216-1] @ Replicator [unknown] error: WAL
segment requested 00000001000000000000002C has been removed

Master error log
5/3/2015 19:13:35 BRT [3339-1] @ Replicator [unknown] error: WAL segment
requested 00000001000000000000002C has been removed
5/3/2015 19:13:40 BRT [3341-1] @ Replicator [unknown] error: WAL segment
requested 00000001000000000000002C has been removed
5/3/2015 19:13:44 BRT [3343-1] @ Replicator [unknown] error: WAL segment
requested 00000001000000000000002C has been removed

Settings files are as follows:

master

postgresql.conf

listen_addresses = '*'
wal_level = hot_standby
archive_mode = on
archive_command = 'cp "%p" /mnt/server/archivedir/"%f"'
max_wal_senders = 2
wal_keep_segments = 20

pg_hba.conf
host replication replicador 192.168.0.112/32 trust

secondary

postgresql.conf

listen_addresses = '*'
hot_standby = on

pg_hba.conf
host all all 0.0.0.0/0 md5

recover.conf em (/var/lib/postgresql/9.4/main)

standby_mode=on
primary_conninfo='host=192.168.0.100 user=replicador application_name=
jessie-stby'
trigger_file='/tmp/pgtrigger'
restore_command = 'cp /mnt/server/archivedir/%f %p'
archive_cleanup_command = 'pg_archivecleanup /mnt/server/archivedir %r'

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#3Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Edson F. Lidorio (#1)
Re: Standby problem after restore_command Implementation

On 05/05/2015 07:05 AM, Edson F. Lidorio wrote:

CCing list.

Em 2015-05-05 10:45, Adrian Klaver escreveu:

On 05/03/2015 05:57 PM, Edson F. Lidorio wrote:

Hello, I'm having trouble on Standby after the implementation of the
restore_command. I performed all the settings and it worked normally
and after restart the slave server, started to generate errors in the
log of the slave:

So is that implying that you had the standby running without the restore_command?

I'm using Debian 8 with PostgreSQL 9.4.1 on x86_64-unknown-linux-gnu,
compiled by gcc-4.9. real (Debian 4.9.2-10) 4.9.2, 64-bit Slave error
log: 5/3/2015 16:46:01 BRT [10210-1] @ Replicator [unknown] error:
WAL segment requested 00000001000000000000002C has been removed
5/3/2015 16:46:05 BRT [10211-1] @ Replicator [unknown] error: WAL
segment requested 00000001000000000000002C has been removed 5/3/2015
16:46:10 BRT [10214-1] @ Replicator [unknown] error: WAL segment
requested 00000001000000000000002C has been removed 5/3/2015 16:46:15
BRT [10216-1] @ Replicator [unknown] error: WAL segment requested
00000001000000000000002C has been removed Master error log 5/3/2015
19:13:35 BRT [3339-1] @ Replicator [unknown] error: WAL segment
requested 00000001000000000000002C has been removed 5/3/2015 19:13:40
BRT [3341-1] @ Replicator [unknown] error: WAL segment requested
00000001000000000000002C has been removed 5/3/2015 19:13:44 BRT
[3343-1] @ Replicator [unknown] error: WAL segment requested
00000001000000000000002C has been removed Settings files are as
follows: master postgresql.conf listen_addresses = '*' wal_level =
hot_standby archive_mode = on archive_command = 'cp "%p"
/mnt/server/archivedir/"%f"' max_wal_senders = 2 wal_keep_segments =
20 pg_hba.conf host replication replicador 192.168.0.112/32 trust
secondary postgresql.conf listen_addresses = '*' hot_standby = on
pg_hba.conf host all all 0.0.0.0/0 md5 recover.conf em
(/var/lib/postgresql/9.4/main) standby_mode=on
primary_conninfo='host=192.168.0.100 user=replicador
application_name= jessie-stby' trigger_file='/tmp/pgtrigger'
restore_command = 'cp /mnt/server/archivedir/%f %p'
archive_cleanup_command = 'pg_archivecleanup /mnt/server/archivedir %r'

--
Adrian Klaver
adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>

Yes,
It was working.

So what steps did you take to go from streaming only to streaming and
archiving?

I suspect there was a gap in the stop/restart that allowed a WAL file to
get recycled before the archiving started.

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#4Melvin Davidson
melvin6925@gmail.com
In reply to: Adrian Klaver (#3)
Re: Standby problem after restore_command Implementation

It's possible you have wal_keep_segments set too low. What happens is that
the master will keep the wals ( in your case 20) after processing them,
before sending them off to the great black hole in the network (deleting)
and making them unavailable to the standby. Try increasing
wal_keep_segments = 100.

On Tue, May 5, 2015 at 10:09 AM, Adrian Klaver <adrian.klaver@aklaver.com>
wrote:

On 05/05/2015 07:05 AM, Edson F. Lidorio wrote:

CCing list.

Em 2015-05-05 10:45, Adrian Klaver escreveu:

On 05/03/2015 05:57 PM, Edson F. Lidorio wrote:

Hello, I'm having trouble on Standby after the implementation of the
restore_command. I performed all the settings and it worked normally
and after restart the slave server, started to generate errors in the
log of the slave:

So is that implying that you had the standby running without the
restore_command?

I'm using Debian 8 with PostgreSQL 9.4.1 on x86_64-unknown-linux-gnu,
compiled by gcc-4.9. real (Debian 4.9.2-10) 4.9.2, 64-bit Slave error
log: 5/3/2015 16:46:01 BRT [10210-1] @ Replicator [unknown] error:
WAL segment requested 00000001000000000000002C has been removed
5/3/2015 16:46:05 BRT [10211-1] @ Replicator [unknown] error: WAL
segment requested 00000001000000000000002C has been removed 5/3/2015
16:46:10 BRT [10214-1] @ Replicator [unknown] error: WAL segment
requested 00000001000000000000002C has been removed 5/3/2015 16:46:15
BRT [10216-1] @ Replicator [unknown] error: WAL segment requested
00000001000000000000002C has been removed Master error log 5/3/2015
19:13:35 BRT [3339-1] @ Replicator [unknown] error: WAL segment
requested 00000001000000000000002C has been removed 5/3/2015 19:13:40
BRT [3341-1] @ Replicator [unknown] error: WAL segment requested
00000001000000000000002C has been removed 5/3/2015 19:13:44 BRT
[3343-1] @ Replicator [unknown] error: WAL segment requested
00000001000000000000002C has been removed Settings files are as
follows: master postgresql.conf listen_addresses = '*' wal_level =
hot_standby archive_mode = on archive_command = 'cp "%p"
/mnt/server/archivedir/"%f"' max_wal_senders = 2 wal_keep_segments =
20 pg_hba.conf host replication replicador 192.168.0.112/32 trust
secondary postgresql.conf listen_addresses = '*' hot_standby = on
pg_hba.conf host all all 0.0.0.0/0 md5 recover.conf em
(/var/lib/postgresql/9.4/main) standby_mode=on
primary_conninfo='host=192.168.0.100 user=replicador
application_name= jessie-stby' trigger_file='/tmp/pgtrigger'
restore_command = 'cp /mnt/server/archivedir/%f %p'
archive_cleanup_command = 'pg_archivecleanup /mnt/server/archivedir %r'

--
Adrian Klaver
adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>

Yes,

It was working.

So what steps did you take to go from streaming only to streaming and
archiving?

I suspect there was a gap in the stop/restart that allowed a WAL file to
get recycled before the archiving started.

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--
*Melvin Davidson*
I reserve the right to fantasize. Whether or not you
wish to share my fantasy is entirely up to you.

#5Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Edson F. Lidorio (#1)
Re: Standby problem after restore_command Implementation

On 05/05/2015 07:23 AM, Edson F. Lidorio wrote:

Please keep this on the list, for two reasons:

1) More eyes looking is better.

2) I will be off line shortly.

Em 2015-05-05 11:09, Adrian Klaver escreveu:

On 05/05/2015 07:05 AM, Edson F. Lidorio wrote:

CCing list.

Em 2015-05-05 10:45, Adrian Klaver escreveu:

On 05/03/2015 05:57 PM, Edson F. Lidorio wrote:

Hello, I'm having trouble on Standby after the implementation of
the restore_command. I performed all the settings and it worked
normally and after restart the slave server, started to generate
errors in the log of the slave:

So is that implying that you had the standby running without the
restore_command?

I'm using Debian 8 with PostgreSQL 9.4.1 on
x86_64-unknown-linux-gnu, compiled by gcc-4.9. real (Debian
4.9.2-10) 4.9.2, 64-bit Slave error log: 5/3/2015 16:46:01 BRT
[10210-1] @ Replicator [unknown] error: WAL segment requested
00000001000000000000002C has been removed 5/3/2015 16:46:05 BRT
[10211-1] @ Replicator [unknown] error: WAL segment requested
00000001000000000000002C has been removed 5/3/2015 16:46:10 BRT
[10214-1] @ Replicator [unknown] error: WAL segment requested
00000001000000000000002C has been removed 5/3/2015 16:46:15 BRT
[10216-1] @ Replicator [unknown] error: WAL segment requested
00000001000000000000002C has been removed Master error log 5/3/2015
19:13:35 BRT [3339-1] @ Replicator [unknown] error: WAL segment
requested 00000001000000000000002C has been removed 5/3/2015
19:13:40 BRT [3341-1] @ Replicator [unknown] error: WAL segment
requested 00000001000000000000002C has been removed 5/3/2015
19:13:44 BRT [3343-1] @ Replicator [unknown] error: WAL segment
requested 00000001000000000000002C has been removed Settings files
are as follows: master postgresql.conf listen_addresses = '*'
wal_level = hot_standby archive_mode = on archive_command = 'cp
"%p" /mnt/server/archivedir/"%f"' max_wal_senders = 2
wal_keep_segments = 20 pg_hba.conf host replication replicador
192.168.0.112/32 trust secondary postgresql.conf listen_addresses =
'*' hot_standby = on pg_hba.conf host all all 0.0.0.0/0 md5
recover.conf em (/var/lib/postgresql/9.4/main) standby_mode=on
primary_conninfo='host=192.168.0.100 user=replicador
application_name= jessie-stby' trigger_file='/tmp/pgtrigger'
restore_command = 'cp /mnt/server/archivedir/%f %p'
archive_cleanup_command = 'pg_archivecleanup /mnt/server/archivedir %r'

-- Adrian Klaver adrian.klaver@aklaver.com
<mailto:adrian.klaver@aklaver.com> <mailto:adrian.klaver@aklaver.com
<mailto:adrian.klaver@aklaver.com>>

Yes, It was working.

So what steps did you take to go from streaming only to streaming and archiving?

I suspect there was a gap in the stop/restart that allowed a WAL file to get recycled before the archiving started.

--
Adrian Klaver
adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>

A question thatI have!
astheslave servercan seethis folderin themaster?
/mnt/server/archivedir/

That would depend on your setup.

Can you log in to the standby and manually copy files from that directory?

Sorrymy doubtsI'mbasicbeginner!

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#6Edson F. Lidorio
edson@openmailbox.org
In reply to: Melvin Davidson (#4)
Re: Standby problem after restore_command Implementation

On 05-05-2015 11:22, Melvin Davidson wrote:

It's possible you have wal_keep_segments set too low. What happens is
that the master will keep the wals ( in your case 20) after processing
them, before sending them off to the great black hole in the network
(deleting) and making them unavailable to the standby. Try increasing
wal_keep_segments = 100.

On Tue, May 5, 2015 at 10:09 AM, Adrian Klaver
<adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>> wrote:

On 05/05/2015 07:05 AM, Edson F. Lidorio wrote:

CCing list.

Em 2015-05-05 10:45, Adrian Klaver escreveu:

On 05/03/2015 05:57 PM, Edson F. Lidorio wrote:

Hello, I'm having trouble on Standby after the
implementation of the
restore_command. I performed all the settings and it
worked normally
and after restart the slave server, started to
generate errors in the
log of the slave:

So is that implying that you had the standby running
without the restore_command?

I'm using Debian 8 with PostgreSQL 9.4.1 on
x86_64-unknown-linux-gnu,
compiled by gcc-4.9. real (Debian 4.9.2-10) 4.9.2,
64-bit Slave error
log: 5/3/2015 16:46:01 BRT [10210-1] @ Replicator
[unknown] error:
WAL segment requested 00000001000000000000002C has
been removed
5/3/2015 16:46:05 BRT [10211-1] @ Replicator [unknown]
error: WAL
segment requested 00000001000000000000002C has been
removed 5/3/2015
16:46:10 BRT [10214-1] @ Replicator [unknown] error:
WAL segment
requested 00000001000000000000002C has been removed
5/3/2015 16:46:15
BRT [10216-1] @ Replicator [unknown] error: WAL
segment requested
00000001000000000000002C has been removed Master error
log 5/3/2015
19:13:35 BRT [3339-1] @ Replicator [unknown] error:
WAL segment
requested 00000001000000000000002C has been removed
5/3/2015 19:13:40
BRT [3341-1] @ Replicator [unknown] error: WAL segment
requested
00000001000000000000002C has been removed 5/3/2015
19:13:44 BRT
[3343-1] @ Replicator [unknown] error: WAL segment
requested
00000001000000000000002C has been removed Settings
files are as
follows: master postgresql.conf listen_addresses = '*'
wal_level =
hot_standby archive_mode = on archive_command = 'cp "%p"
/mnt/server/archivedir/"%f"' max_wal_senders = 2
wal_keep_segments =
20 pg_hba.conf host replication replicador
192.168.0.112/32 <http://192.168.0.112/32&gt; trust
secondary postgresql.conf listen_addresses = '*'
hot_standby = on
pg_hba.conf host all all 0.0.0.0/0 <http://0.0.0.0/0&gt;
md5 recover.conf em
(/var/lib/postgresql/9.4/main) standby_mode=on
primary_conninfo='host=192.168.0.100 user=replicador
application_name= jessie-stby'
trigger_file='/tmp/pgtrigger'
restore_command = 'cp /mnt/server/archivedir/%f %p'
archive_cleanup_command = 'pg_archivecleanup
/mnt/server/archivedir %r'

--
Adrian Klaver
adrian.klaver@aklaver.com
<mailto:adrian.klaver@aklaver.com>
<mailto:adrian.klaver@aklaver.com
<mailto:adrian.klaver@aklaver.com>>

Yes,
It was working.

So what steps did you take to go from streaming only to streaming
and archiving?

I suspect there was a gap in the stop/restart that allowed a WAL
file to get recycled before the archiving started.

--
Adrian Klaver
adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org
<mailto:pgsql-general@postgresql.org>)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--
*Melvin Davidson*
I reserve the right to fantasize. Whether or not you
wish to share my fantasy is entirely up to you.

Increased thewal_keep_segments=100andkeeps popping upthe same message:

A question thatI have!
astheslave servercan seethis folderin themaster?
/mnt/server/archivedir/

Sorrymy doubtsI'mbasicbeginner!

#7Melvin Davidson
melvin6925@gmail.com
In reply to: Edson F. Lidorio (#6)
Re: Standby problem after restore_command Implementation

Did you change the setting on the master?
Did you remember to reload the config on the master after changing?
Did you rebuild the Standby server after making the change and reloading?
IE:
psql -U postgres
SELECT pg_reload_conf();

You must do ALL of the above to have any chance of making things work.

On Tue, May 5, 2015 at 11:35 AM, Edson F. Lidorio <edson@openmailbox.org>
wrote:

On 05-05-2015 11:22, Melvin Davidson wrote:

It's possible you have wal_keep_segments set too low. What happens is that
the master will keep the wals ( in your case 20) after processing them,
before sending them off to the great black hole in the network (deleting)
and making them unavailable to the standby. Try increasing
wal_keep_segments = 100.

On Tue, May 5, 2015 at 10:09 AM, Adrian Klaver <adrian.klaver@aklaver.com>
wrote:

On 05/05/2015 07:05 AM, Edson F. Lidorio wrote:

CCing list.

Em 2015-05-05 10:45, Adrian Klaver escreveu:

On 05/03/2015 05:57 PM, Edson F. Lidorio wrote:

Hello, I'm having trouble on Standby after the implementation of the
restore_command. I performed all the settings and it worked normally
and after restart the slave server, started to generate errors in the
log of the slave:

So is that implying that you had the standby running without the
restore_command?

I'm using Debian 8 with PostgreSQL 9.4.1 on x86_64-unknown-linux-gnu,
compiled by gcc-4.9. real (Debian 4.9.2-10) 4.9.2, 64-bit Slave error
log: 5/3/2015 16:46:01 BRT [10210-1] @ Replicator [unknown] error:
WAL segment requested 00000001000000000000002C has been removed
5/3/2015 16:46:05 BRT [10211-1] @ Replicator [unknown] error: WAL
segment requested 00000001000000000000002C has been removed 5/3/2015
16:46:10 BRT [10214-1] @ Replicator [unknown] error: WAL segment
requested 00000001000000000000002C has been removed 5/3/2015 16:46:15
BRT [10216-1] @ Replicator [unknown] error: WAL segment requested
00000001000000000000002C has been removed Master error log 5/3/2015
19:13:35 BRT [3339-1] @ Replicator [unknown] error: WAL segment
requested 00000001000000000000002C has been removed 5/3/2015 19:13:40
BRT [3341-1] @ Replicator [unknown] error: WAL segment requested
00000001000000000000002C has been removed 5/3/2015 19:13:44 BRT
[3343-1] @ Replicator [unknown] error: WAL segment requested
00000001000000000000002C has been removed Settings files are as
follows: master postgresql.conf listen_addresses = '*' wal_level =
hot_standby archive_mode = on archive_command = 'cp "%p"
/mnt/server/archivedir/"%f"' max_wal_senders = 2 wal_keep_segments =
20 pg_hba.conf host replication replicador 192.168.0.112/32 trust
secondary postgresql.conf listen_addresses = '*' hot_standby = on
pg_hba.conf host all all 0.0.0.0/0 md5 recover.conf em
(/var/lib/postgresql/9.4/main) standby_mode=on
primary_conninfo='host=192.168.0.100 user=replicador
application_name= jessie-stby' trigger_file='/tmp/pgtrigger'
restore_command = 'cp /mnt/server/archivedir/%f %p'
archive_cleanup_command = 'pg_archivecleanup /mnt/server/archivedir %r'

--
Adrian Klaver
adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>

Yes,

It was working.

So what steps did you take to go from streaming only to streaming and
archiving?

I suspect there was a gap in the stop/restart that allowed a WAL file to
get recycled before the archiving started.

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--
*Melvin Davidson*
I reserve the right to fantasize. Whether or not you
wish to share my fantasy is entirely up to you.

Increased the wal_keep_segments = 100 and keeps popping up the same
message:

A question that I have!
as the slave server can see this folder in the master?
/mnt/server/archivedir/

Sorry my doubts I'm basic beginner!

--
*Melvin Davidson*
I reserve the right to fantasize. Whether or not you
wish to share my fantasy is entirely up to you.

#8Edson F. Lidorio
edson@openmailbox.org
In reply to: Melvin Davidson (#7)
Re: Standby problem after restore_command Implementation

Em 2015-05-05 13:05, Melvin Davidson escreveu:

Did you change the setting on the master? Did you remember to reload the config on the master after changing?
Did you rebuild the Standby server after making the change and reloading? IE:
psql -U postgres SELECT pg_reload_conf();

You must do ALL of the above to have any chance of making things work.

On Tue, May 5, 2015 at 11:35 AM, Edson F. Lidorio <edson@openmailbox.org> wrote:

On 05-05-2015 11:22, Melvin Davidson wrote:

It's possible you have wal_keep_segments set too low. What happens is that the master will keep the wals ( in your case 20) after processing them, before sending them off to the great black hole in the network (deleting) and making them unavailable to the standby. Try increasing wal_keep_segments = 100.

On Tue, May 5, 2015 at 10:09 AM, Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 05/05/2015 07:05 AM, Edson F. Lidorio wrote:

CCing list.

Em 2015-05-05 10:45, Adrian Klaver escreveu:

On 05/03/2015 05:57 PM, Edson F. Lidorio wrote:
Hello, I'm having trouble on Standby after the implementation of the
restore_command. I performed all the settings and it worked normally
and after restart the slave server, started to generate errors in the
log of the slave: So is that implying that you had the standby running without the restore_command?
I'm using Debian 8 with PostgreSQL 9.4.1 on x86_64-unknown-linux-gnu,
compiled by gcc-4.9. real (Debian 4.9.2-10) 4.9.2, 64-bit Slave error
log: 5/3/2015 16:46:01 BRT [10210-1] @ Replicator [unknown] error:
WAL segment requested 00000001000000000000002C has been removed
5/3/2015 16:46:05 BRT [10211-1] @ Replicator [unknown] error: WAL
segment requested 00000001000000000000002C has been removed 5/3/2015
16:46:10 BRT [10214-1] @ Replicator [unknown] error: WAL segment
requested 00000001000000000000002C has been removed 5/3/2015 16:46:15
BRT [10216-1] @ Replicator [unknown] error: WAL segment requested
00000001000000000000002C has been removed Master error log 5/3/2015
19:13:35 BRT [3339-1] @ Replicator [unknown] error: WAL segment
requested 00000001000000000000002C has been removed 5/3/2015 19:13:40
BRT [3341-1] @ Replicator [unknown] error: WAL segment requested
00000001000000000000002C has been removed 5/3/2015 19:13:44 BRT
[3343-1] @ Replicator [unknown] error: WAL segment requested
00000001000000000000002C has been removed Settings files are as
follows: master postgresql.conf listen_addresses = '*' wal_level =
hot_standby archive_mode = on archive_command = 'cp "%p"
/mnt/server/archivedir/"%f"' max_wal_senders = 2 wal_keep_segments =
20 pg_hba.conf host replication replicador 192.168.0.112/32 [1] trust
secondary postgresql.conf listen_addresses = '*' hot_standby = on
pg_hba.conf host all all 0.0.0.0/0 [2] md5 recover.conf em
(/var/lib/postgresql/9.4/main) standby_mode=on
primary_conninfo='host=192.168.0.100 user=replicador
application_name= jessie-stby' trigger_file='/tmp/pgtrigger'
restore_command = 'cp /mnt/server/archivedir/%f %p'
archive_cleanup_command = 'pg_archivecleanup /mnt/server/archivedir %r' --
Adrian Klaver adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>

Yes,
It was working.
So what steps did you take to go from streaming only to streaming and
archiving?

I suspect there was a gap in the stop/restart that allowed a WAL file
to get recycled before the archiving started.

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general [3]http://www.postgresql.org/mailpref/pgsql-general

--

MELVIN DAVIDSON
I reserve the right to fantasize. Whether or not you
wish to share my fantasy is entirely up to you.

Increased the wal_keep_segments = 100 and keeps popping up the same
message:

A question that I have!
as the slave server can see this folder in the master?
/mnt/server/archivedir/

Sorry my doubts I'm basic beginner!

--

MELVIN DAVIDSON
I reserve the right to fantasize. Whether or not you
wish to share my fantasy is entirely up to you.

After you restart the server and it didnt work I'd done reload.
I'm going to do another standby charge today, and see what happens.
Could you explain that doubt my previous message.
I have a slight feeling that the problem may also be in the access to
the files folder of the wal.

Links:
------
[1]: http://192.168.0.112/32
[2]: http://0.0.0.0/0
[3]: http://www.postgresql.org/mailpref/pgsql-general

#9Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Edson F. Lidorio (#6)
Re: Standby problem after restore_command Implementation

On 05/05/2015 08:35 AM, Edson F. Lidorio wrote:

On 05-05-2015 11:22, Melvin Davidson wrote:

It's possible you have wal_keep_segments set too low. What happens is
that the master will keep the wals ( in your case 20) after processing
them, before sending them off to the great black hole in the network
(deleting) and making them unavailable to the standby. Try increasing
wal_keep_segments = 100.

Increased thewal_keep_segments=100andkeeps popping upthe same message:

The message did not come through. Also increasing wal_keep_segments only
works for the future, it will not deal with a WAL file that has already
been recycled before the setting was changed.

A question thatI have!
astheslave servercan seethis folderin themaster?
/mnt/server/archivedir/

So have you tried logging into the standby and then accessing the
directory on the master manually. In other words copy a file from the
master to standby using cp from the standby command line.

Sorrymy doubtsI'mbasicbeginner!

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#10Edson F. Lidorio
edson@openmailbox.org
In reply to: Adrian Klaver (#9)
Re: Standby problem after restore_command Implementation

Em 2015-05-05 14:31, Adrian Klaver escreveu:

On 05/05/2015 08:35 AM, Edson F. Lidorio wrote:
On 05-05-2015 11:22, Melvin Davidson wrote: It's possible you have wal_keep_segments set too low. What happens is that the master will keep the wals ( in your case 20) after processing them, before sending them off to the great black hole in the network (deleting) and making them unavailable to the standby. Try increasing wal_keep_segments = 100.

Increased thewal_keep_segments=100andkeeps popping upthe same message:

The message did not come through. Also increasing wal_keep_segments only
works for the future, it will not deal with a WAL file that has already
been recycled before the setting was changed.

A question thatI have! astheslave servercan seethis folderin themaster? /mnt/server/archivedir/

So have you tried logging into the standby and then accessing the
directory on the master manually. In other words copy a file from the
master to standby using cp from the standby command line.

Sorrymy doubtsI'mbasicbeginner!

--
Adrian Klaver
adrian.klaver@aklaver.com

OK, I'm going to do this test too.

--
Edson F. Lidorio

Atenção: Este e-mail pode conter anexos no formato ODF (Open Document
Format)/ABNT (extensões odt, ods, odp, odb, odg). Antes de pedir os
anexos
em outro formato, você pode instalar gratuita e livremente o LibreOffice
(
http://pt-br.libreoffice.org/).

#11Melvin Davidson
melvin6925@gmail.com
In reply to: Edson F. Lidorio (#10)
Re: Standby problem after restore_command Implementation

Have you allowed access to user "replication" in the master pg_hba.conf and
done a
SELECT pg_reload_conf()?

Can you ping the master from the standby?

Did you RTFM?
http://www.postgresql.org/docs/9.2/interactive/warm-standby.html

25.2.3 Preparing the Master for Standby Servers
25.2.4 Setting Up a Standby Server

On Tue, May 5, 2015 at 2:05 PM, Edson F. Lidorio <edson@openmailbox.org>
wrote:

Em 2015-05-05 14:31, Adrian Klaver escreveu:

On 05/05/2015 08:35 AM, Edson F. Lidorio wrote:

On 05-05-2015 11:22, Melvin Davidson wrote:

It's possible you have wal_keep_segments set too low. What happens is that
the master will keep the wals ( in your case 20) after processing them,
before sending them off to the great black hole in the network (deleting)
and making them unavailable to the standby. Try increasing
wal_keep_segments = 100.

Increased thewal_keep_segments=100andkeeps popping upthe same message:

The message did not come through. Also increasing wal_keep_segments only works for the future, it will not deal with a WAL file that has already been recycled before the setting was changed.

A question thatI have! astheslave servercan seethis folderin themaster?
/mnt/server/archivedir/

So have you tried logging into the standby and then accessing the directory on the master manually. In other words copy a file from the master to standby using cp from the standby command line.

Sorrymy doubtsI'mbasicbeginner!

--
Adrian Klaveradrian.klaver@aklaver.com

OK, I'm going to do this test too.

--
Edson F. Lidorio
Atenção: Este e-mail pode conter anexos no formato ODF (Open Document
Format)/ABNT (extensões odt, ods, odp, odb, odg). Antes de pedir os anexos
em outro formato, você pode instalar gratuita e livremente o LibreOffice (http://pt-br.libreoffice.org/).

--
*Melvin Davidson*
I reserve the right to fantasize. Whether or not you
wish to share my fantasy is entirely up to you.

#12Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Melvin Davidson (#11)
Re: Standby problem after restore_command Implementation

On 05/05/2015 11:21 AM, Melvin Davidson wrote:

Have you allowed access to user "replication" in the master pg_hba.conf
and done a
SELECT pg_reload_conf()?

Can you ping the master from the standby?

Did you RTFM?
http://www.postgresql.org/docs/9.2/interactive/warm-standby.html

25.2.3 Preparing the Master for Standby Servers
25.2.4 Setting Up a Standby Server

See the OPs original post:

/messages/by-id/5546C3DC.5070808@openmailbox.org

The OP had streaming only replication working, so the basics are there.
The wheels fell off when WAL archiving was introduced. Best guess is
this is a timing issue, the relevant WALs got recycled while archiving
was being set up and the standby was down.

On Tue, May 5, 2015 at 2:05 PM, Edson F. Lidorio <edson@openmailbox.org
<mailto:edson@openmailbox.org>> wrote:

__

Em 2015-05-05 14:31, Adrian Klaver escreveu:

On 05/05/2015 08:35 AM, Edson F. Lidorio wrote:

On 05-05-2015 11:22, Melvin Davidson wrote:

It's possible you have wal_keep_segments set too low. What
happens is that the master will keep the wals ( in your case 20)
after processing them, before sending them off to the great
black hole in the network (deleting) and making them unavailable
to the standby. Try increasing wal_keep_segments = 100.

Increased thewal_keep_segments=100andkeeps popping upthe same
message:

The message did not come through. Also increasing wal_keep_segments only works for the future, it will not deal with a WAL file that has already been recycled before the setting was changed.

A question thatI have! astheslave servercan seethis folderin
themaster? /mnt/server/archivedir/

So have you tried logging into the standby and then accessing the directory on the master manually. In other words copy a file from the master to standby using cp from the standby command line.

Sorrymy doubtsI'mbasicbeginner!

--
Adrian Klaver
adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>

OK,I'm going to dothis testtoo.

--
Edson F. Lidorio

Atenção: Este e-mail pode conter anexos no formato ODF (Open Document
Format)/ABNT (extensões odt, ods, odp, odb, odg). Antes de pedir os anexos
em outro formato, você pode instalar gratuita e livremente o LibreOffice (
http://pt-br.libreoffice.org/).

--
*Melvin Davidson*
I reserve the right to fantasize. Whether or not you
wish to share my fantasy is entirely up to you.

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#13Venkata B Nagothi
nag1010@gmail.com
In reply to: Edson F. Lidorio (#1)
Re: Standby problem after restore_command Implementation

On Tue, May 5, 2015 at 7:57 AM, Edson F. Lidorio <edson@openmailbox.org>
wrote:

On 04-05-2015 00:46, Venkata Balaji N wrote:

You do not see the above WAL file in the archive directory ?
"/mnt/server/archivedir" is shared between master and slave databases ? The
file may have been removed from the archive location.

Strange !!
Yes, this specified folder file "/ mnt / server / ArchiveDir".
At this shared folder between master and slave.

I suspect that archive cleanup job is cleaning up the archives.
"archive_cleanup_command" removes any archives not needed for recovery.

Since the the archive destination is shared across master and slave.
Archives are removed and Standby is not getting the required archives.

To avoid this, before you do any tests, i suggest remove
"archive_cleanup_command" parameter and see if you are still facing the
same problem.

Regards,
Venkata Balaji N

Fujitsu Australia