Log all queries before migration ?

Started by ben.playover 9 years ago6 messagesgeneral
Jump to latest
#1ben.play
benjamin.cohen@playrion.com

Hi guys,

We have some storage problem on our dedicated server and we have to migrate.
Do you know how can I log all queries in order to have no downtime during
migration ?

I tried many options on the conf, but i never find the good one. I didn't
find out to generate .sql file with all queries.

Thank you a lot for your help !

--
View this message in context: http://postgresql.nabble.com/Log-all-queries-before-migration-tp5913922.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#2CS DBA
cs_dba@consistentstate.com
In reply to: ben.play (#1)
Re: Log all queries before migration ?

log_min_duration_statement = 0

On 07/29/2016 05:35 AM, ben.play wrote:

Hi guys,

We have some storage problem on our dedicated server and we have to migrate.
Do you know how can I log all queries in order to have no downtime during
migration ?

I tried many options on the conf, but i never find the good one. I didn't
find out to generate .sql file with all queries.

Thank you a lot for your help !

--
View this message in context: http://postgresql.nabble.com/Log-all-queries-before-migration-tp5913922.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#3David G. Johnston
david.g.johnston@gmail.com
In reply to: ben.play (#1)
Re: Log all queries before migration ?

On Fri, Jul 29, 2016 at 7:35 AM, ben.play <benjamin.cohen@playrion.com>
wrote:

Hi guys,

We have some storage problem on our dedicated server and we have to
migrate.
Do you know how can I log all queries in order to have no downtime during
migration ?

​I don't understand how logging of queries has anything to do with downtime
during migration...

David J.​

#4ben.play
benjamin.cohen@playrion.com
In reply to: David G. Johnston (#3)
Re: Log all queries before migration ?

Hi guys,

In fact, I would like to copy a table and then, when everything is finish,
execute all inserts and updates since the beginning of the copy.

The biggest problem with the log at this time it's the query.
All my queries are not runnable.

When I log, I see this kind of queries :
"duration: 0.046 ms parse <unnamed>: UPDATE user SET money = money + $1
WHERE id = $2"

But I would like to have only : "UPDATE user SET money = money + 100 WHERE
id = 1281012"

This is our postgresql.conf :
log_destination = 'csvlog'
logging_collector = on
log_min_duration_statement = 0
log_statement = mod
log_directory = '/media/hdd/log_queries'
log_filename='log_insert.log'
log_file_mode = 0600
log_error_verbosity = verbose

What do we have to change ?

Thank you a lot for your help !

--
View this message in context: http://postgresql.nabble.com/Log-all-queries-before-migration-tp5913922p5914599.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#5Charles Clavadetscher
clavadetscher@swisspug.org
In reply to: ben.play (#4)
Re: Log all queries before migration ?

Hello

-----Original Message-----
From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org] On Behalf Of ben.play
Sent: Mittwoch, 3. August 2016 15:29
To: pgsql-general@postgresql.org
Subject: Re: [GENERAL] Log all queries before migration ?

Hi guys,

In fact, I would like to copy a table and then, when everything is finish, execute all inserts and updates since the
beginning of the copy.

The biggest problem with the log at this time it's the query.
All my queries are not runnable.

This may help you?

http://laurenz.github.io/pgreplay/

Regards
Charles

When I log, I see this kind of queries :
"duration: 0.046 ms parse <unnamed>: UPDATE user SET money = money + $1 WHERE id = $2"

But I would like to have only : "UPDATE user SET money = money + 100 WHERE id = 1281012"

This is our postgresql.conf :
log_destination = 'csvlog'
logging_collector = on
log_min_duration_statement = 0
log_statement = mod
log_directory = '/media/hdd/log_queries'
log_filename='log_insert.log'
log_file_mode = 0600
log_error_verbosity = verbose

What do we have to change ?

Thank you a lot for your help !

--
View this message in context: http://postgresql.nabble.com/Log-all-queries-before-migration-tp5913922p5914599.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#6Melvin Davidson
melvin6925@gmail.com
In reply to: Charles Clavadetscher (#5)
Re: Log all queries before migration ?

On Wed, Aug 3, 2016 at 10:10 AM, Charles Clavadetscher <
clavadetscher@swisspug.org> wrote:

Hello

-----Original Message-----
From: pgsql-general-owner@postgresql.org [mailto:

pgsql-general-owner@postgresql.org] On Behalf Of ben.play

Sent: Mittwoch, 3. August 2016 15:29
To: pgsql-general@postgresql.org
Subject: Re: [GENERAL] Log all queries before migration ?

Hi guys,

In fact, I would like to copy a table and then, when everything is

finish, execute all inserts and updates since the

beginning of the copy.

The biggest problem with the log at this time it's the query.
All my queries are not runnable.

This may help you?

http://laurenz.github.io/pgreplay/

Regards
Charles

When I log, I see this kind of queries :
"duration: 0.046 ms parse <unnamed>: UPDATE user SET money = money + $1

WHERE id = $2"

But I would like to have only : "UPDATE user SET money = money + 100

WHERE id = 1281012"

This is our postgresql.conf :
log_destination = 'csvlog'
logging_collector = on
log_min_duration_statement = 0
log_statement = mod
log_directory = '/media/hdd/log_queries'
log_filename='log_insert.log'
log_file_mode = 0600
log_error_verbosity = verbose

What do we have to change ?

Thank you a lot for your help !

--
View this message in context:

http://postgresql.nabble.com/Log-all-queries-before-migration-tp5913922p5914599.html

Sent from the PostgreSQL - general mailing list archive at Nabble.com.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To

make changes to your subscription:

http://www.postgresql.org/mailpref/pgsql-general

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

T

*here is no way to record queries and shut down the server at the same
time.That's like saying I want to shut off my car engine but keep driving.I
believe you are looking at the wrong solution. The best way to migrate and
keep all transactions is to use Slony
replication.http://slony.info/adminguide/2.2/doc/adminguide/slony.pdf
<http://slony.info/adminguide/2.2/doc/adminguide/slony.pdf&gt;http://slony.info/downloads/2.2/
<http://slony.info/downloads/2.2/&gt;You can start the migration to the new
server and when it's finished, you simply failover to the slave and
redirect the url to it. *

--
*Melvin Davidson*
I reserve the right to fantasize. Whether or not you
wish to share my fantasy is entirely up to you.