pg_stat directory and pg_stat_statements
Hi,
Thanks to 187492b6c2e8cafc5b39063ca3b67846e8155d24, pgstat
files are now saved to $PGDATA/pg_stat directory at shutdown.
But pg_stat_statements file is saved under $PGDATA/global yet.
Is this intentional or just oversight? Saving that file to global is
harmless, though.
Regards,
--
Fujii Masao
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, May 28, 2014 at 7:01 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
But pg_stat_statements file is saved under $PGDATA/global yet.
Is this intentional or just oversight?
I think it's an oversight.
--
Peter Geoghegan
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, May 29, 2014 at 12:37 AM, Peter Geoghegan <pg@heroku.com> wrote:
On Wed, May 28, 2014 at 7:01 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
But pg_stat_statements file is saved under $PGDATA/global yet.
Is this intentional or just oversight?I think it's an oversight.
OK, patch attached.
I'm afraid that it's not okay to change the file layout in $PGDATA at this beta1
stage because that change basically seems to need initdb. Otherwise something
like "no such file or directory" error can happen. But in this case what we need
to change is only the location of the pg_stat_statements permanent stats file.
So, without initdb, the server will not be able to find the
pg_stat_statements stats
file, but this is not so harmful. Only the problem is that the
pg_stat_statements
stats which were collected in past would disappear. OTOH, the server can keep
running successfully from then and no critical data will not
disappear. Therefore
I think we can commit this patch even at beta1. Thought?
Regards,
--
Fujii Masao
Attachments:
0001-Save-pg_stat_statements-statistics-file-into-PGDATA-.patchtext/x-diff; charset=US-ASCII; name=0001-Save-pg_stat_statements-statistics-file-into-PGDATA-.patchDownload
From 8f3281566399bbb088b826b8fe9c3baa5027e5da Mon Sep 17 00:00:00 2001
From: MasaoFujii <masao.fujii@gmail.com>
Date: Thu, 29 May 2014 02:14:34 +0900
Subject: [PATCH] Save pg_stat_statements statistics file into $PGDATA/pg_stat
directory at shutdown.
187492b6c2e8cafc5b39063ca3b67846e8155d24 changed pgstat.c so that
the stats files were saved into $PGDATA/pg_stat directory when the server
was shutdowned. But it accidentally forgot to change the location of
pg_stat_statements permanent stats file. This commit fixes pg_stat_statements
so that its stats file is also saved into $PGDATA/pg_stat at shutdown.
---
contrib/pg_stat_statements/pg_stat_statements.c | 2 +-
src/backend/postmaster/pgstat.c | 8 --------
src/include/pgstat.h | 8 ++++++++
3 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 32d16cc..a3e8c59 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -80,7 +80,7 @@
PG_MODULE_MAGIC;
/* Location of permanent stats file (valid when database is shut down) */
-#define PGSS_DUMP_FILE "global/pg_stat_statements.stat"
+#define PGSS_DUMP_FILE PGSTAT_STAT_PERMANENT_DIRECTORY "/pg_stat_statements.stat"
/*
* Location of external query text file. We don't keep it in the core
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index f864816..3ab1428 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -68,14 +68,6 @@
/* ----------
- * Paths for the statistics files (relative to installation's $PGDATA).
- * ----------
- */
-#define PGSTAT_STAT_PERMANENT_DIRECTORY "pg_stat"
-#define PGSTAT_STAT_PERMANENT_FILENAME "pg_stat/global.stat"
-#define PGSTAT_STAT_PERMANENT_TMPFILE "pg_stat/global.tmp"
-
-/* ----------
* Timer definitions.
* ----------
*/
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index d9de09f..0892533 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -20,6 +20,14 @@
#include "utils/relcache.h"
+/* ----------
+ * Paths for the statistics files (relative to installation's $PGDATA).
+ * ----------
+ */
+#define PGSTAT_STAT_PERMANENT_DIRECTORY "pg_stat"
+#define PGSTAT_STAT_PERMANENT_FILENAME "pg_stat/global.stat"
+#define PGSTAT_STAT_PERMANENT_TMPFILE "pg_stat/global.tmp"
+
/* Default directory to store temporary statistics data in */
#define PG_STAT_TMP_DIR "pg_stat_tmp"
--
1.7.12.4 (Apple Git-37)
On 28.5.2014 19:52, Fujii Masao wrote:
On Thu, May 29, 2014 at 12:37 AM, Peter Geoghegan <pg@heroku.com> wrote:
On Wed, May 28, 2014 at 7:01 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
But pg_stat_statements file is saved under $PGDATA/global yet.
Is this intentional or just oversight?I think it's an oversight.
OK, patch attached.
I'm afraid that it's not okay to change the file layout in $PGDATA at this beta1
stage because that change basically seems to need initdb. Otherwise something
like "no such file or directory" error can happen. But in this case what we need
to change is only the location of the pg_stat_statements permanent stats file.
So, without initdb, the server will not be able to find the
pg_stat_statements stats
file, but this is not so harmful. Only the problem is that the
pg_stat_statements
stats which were collected in past would disappear. OTOH, the server can keep
running successfully from then and no critical data will not
disappear. Therefore
I think we can commit this patch even at beta1. Thought?
For HEAD, probably yes. But what about backpatching 9.3?
Tomas
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, May 29, 2014 at 4:55 AM, Tomas Vondra <tv@fuzzy.cz> wrote:
On 28.5.2014 19:52, Fujii Masao wrote:
On Thu, May 29, 2014 at 12:37 AM, Peter Geoghegan <pg@heroku.com> wrote:
On Wed, May 28, 2014 at 7:01 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
But pg_stat_statements file is saved under $PGDATA/global yet.
Is this intentional or just oversight?I think it's an oversight.
OK, patch attached.
I'm afraid that it's not okay to change the file layout in $PGDATA at this beta1
stage because that change basically seems to need initdb. Otherwise something
like "no such file or directory" error can happen. But in this case what we need
to change is only the location of the pg_stat_statements permanent stats file.
So, without initdb, the server will not be able to find the
pg_stat_statements stats
file, but this is not so harmful. Only the problem is that the
pg_stat_statements
stats which were collected in past would disappear. OTOH, the server can keep
running successfully from then and no critical data will not
disappear. Therefore
I think we can commit this patch even at beta1. Thought?For HEAD, probably yes. But what about backpatching 9.3?
I think No. So we should not backpatch this to 9.3.
Regards,
--
Fujii Masao
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, May 29, 2014 at 8:52 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
On Thu, May 29, 2014 at 4:55 AM, Tomas Vondra <tv@fuzzy.cz> wrote:
On 28.5.2014 19:52, Fujii Masao wrote:
On Thu, May 29, 2014 at 12:37 AM, Peter Geoghegan <pg@heroku.com>
wrote:
On Wed, May 28, 2014 at 7:01 AM, Fujii Masao <masao.fujii@gmail.com>
wrote:
But pg_stat_statements file is saved under $PGDATA/global yet.
Is this intentional or just oversight?I think it's an oversight.
OK, patch attached.
I'm afraid that it's not okay to change the file layout in $PGDATA at
this beta1
stage because that change basically seems to need initdb. Otherwise
something
like "no such file or directory" error can happen. But in this case
what we need
to change is only the location of the pg_stat_statements permanent
stats file.
So, without initdb, the server will not be able to find the
pg_stat_statements stats
file, but this is not so harmful. Only the problem is that the
pg_stat_statements
stats which were collected in past would disappear. OTOH, the servercan keep
running successfully from then and no critical data will not
disappear. Therefore
I think we can commit this patch even at beta1. Thought?For HEAD, probably yes. But what about backpatching 9.3?
I think No. So we should not backpatch this to 9.3.
Just curious.
Will it work in upgrade scenario?
--
Thanks & Regards,
Ashesh Vashi
Show quoted text
Regards,
--
Fujii Masao--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, May 29, 2014 at 1:01 PM, Ashesh Vashi
<ashesh.vashi@enterprisedb.com> wrote:
On Thu, May 29, 2014 at 8:52 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
On Thu, May 29, 2014 at 4:55 AM, Tomas Vondra <tv@fuzzy.cz> wrote:
On 28.5.2014 19:52, Fujii Masao wrote:
On Thu, May 29, 2014 at 12:37 AM, Peter Geoghegan <pg@heroku.com>
wrote:On Wed, May 28, 2014 at 7:01 AM, Fujii Masao <masao.fujii@gmail.com>
wrote:But pg_stat_statements file is saved under $PGDATA/global yet.
Is this intentional or just oversight?I think it's an oversight.
OK, patch attached.
I'm afraid that it's not okay to change the file layout in $PGDATA at
this beta1
stage because that change basically seems to need initdb. Otherwise
something
like "no such file or directory" error can happen. But in this case
what we need
to change is only the location of the pg_stat_statements permanent
stats file.
So, without initdb, the server will not be able to find the
pg_stat_statements stats
file, but this is not so harmful. Only the problem is that the
pg_stat_statements
stats which were collected in past would disappear. OTOH, the server
can keep
running successfully from then and no critical data will not
disappear. Therefore
I think we can commit this patch even at beta1. Thought?For HEAD, probably yes. But what about backpatching 9.3?
I think No. So we should not backpatch this to 9.3.
Just curious.
Will it work in upgrade scenario?
You're concerned about the scenario using pg_upgrade? I'm not sure the detail
of pg_upgrade. But if it doesn't work properly, we should have gotten
the trouble
when 9.3 (which pg_stat directory was introduced) was released. But I've not
heard such trouble....
Regards,
--
Fujii Masao
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, May 28, 2014 at 10:49 PM, Fujii Masao <masao.fujii@gmail.com> wrote:
You're concerned about the scenario using pg_upgrade? I'm not sure the detail
of pg_upgrade. But if it doesn't work properly, we should have gotten
the trouble
I'm not worried about pg_upgrade, because by design pg_stat_statements
will discard stats files that originated in earlier versions. However,
I don't see a need to change pg_stat_statements to serialize its
statistics to disk in the pg_stat directory before we branch off 9.4.
As you mentioned, it's harmless.
--
Peter Geoghegan
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, May 29, 2014 at 11:32 AM, Peter Geoghegan <pg@heroku.com> wrote:
On Wed, May 28, 2014 at 10:49 PM, Fujii Masao <masao.fujii@gmail.com>
wrote:You're concerned about the scenario using pg_upgrade?
Yeah - I was.
I'm not sure the detail
of pg_upgrade. But if it doesn't work properly, we should have gotten
the troubleI'm not worried about pg_upgrade, because by design pg_stat_statements
will discard stats files that originated in earlier versions. However,
I don't see a need to change pg_stat_statements to serialize its
statistics to disk in the pg_stat directory before we branch off 9.4.
As you mentioned, it's harmless.
K.
I was just curious about the scenario.
If it was discarding the stats files that originated in earlier version, It
should be ok.
--
Thanks & Regards,
Ashesh Vashi
Show quoted text
--
Peter Geoghegan
On Thu, May 29, 2014 at 3:02 PM, Peter Geoghegan <pg@heroku.com> wrote:
On Wed, May 28, 2014 at 10:49 PM, Fujii Masao <masao.fujii@gmail.com> wrote:
You're concerned about the scenario using pg_upgrade? I'm not sure the detail
of pg_upgrade. But if it doesn't work properly, we should have gotten
the troubleI'm not worried about pg_upgrade, because by design pg_stat_statements
will discard stats files that originated in earlier versions. However,
I don't see a need to change pg_stat_statements to serialize its
statistics to disk in the pg_stat directory before we branch off 9.4.
As you mentioned, it's harmless.
Yeah, that's an idea. OTOH, there is no *strong* reason to postpone
the fix to 9.5. So I just feel inclined to apply the fix now...
Regards,
--
Fujii Masao
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2014-06-02 22:59:55 +0900, Fujii Masao wrote:
On Thu, May 29, 2014 at 3:02 PM, Peter Geoghegan <pg@heroku.com> wrote:
On Wed, May 28, 2014 at 10:49 PM, Fujii Masao <masao.fujii@gmail.com> wrote:
You're concerned about the scenario using pg_upgrade? I'm not sure the detail
of pg_upgrade. But if it doesn't work properly, we should have gotten
the troubleI'm not worried about pg_upgrade, because by design pg_stat_statements
will discard stats files that originated in earlier versions. However,
I don't see a need to change pg_stat_statements to serialize its
statistics to disk in the pg_stat directory before we branch off 9.4.
As you mentioned, it's harmless.Yeah, that's an idea. OTOH, there is no *strong* reason to postpone
the fix to 9.5. So I just feel inclined to apply the fix now...
+1 for fixing it now.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Jun 2, 2014 at 11:07 PM, Andres Freund <andres@2ndquadrant.com> wrote:
On 2014-06-02 22:59:55 +0900, Fujii Masao wrote:
On Thu, May 29, 2014 at 3:02 PM, Peter Geoghegan <pg@heroku.com> wrote:
On Wed, May 28, 2014 at 10:49 PM, Fujii Masao <masao.fujii@gmail.com> wrote:
You're concerned about the scenario using pg_upgrade? I'm not sure the detail
of pg_upgrade. But if it doesn't work properly, we should have gotten
the troubleI'm not worried about pg_upgrade, because by design pg_stat_statements
will discard stats files that originated in earlier versions. However,
I don't see a need to change pg_stat_statements to serialize its
statistics to disk in the pg_stat directory before we branch off 9.4.
As you mentioned, it's harmless.Yeah, that's an idea. OTOH, there is no *strong* reason to postpone
the fix to 9.5. So I just feel inclined to apply the fix now...+1 for fixing it now.
+1. A beta is here for that as well.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Jun 2, 2014 at 4:07 PM, Andres Freund <andres@2ndquadrant.com>
wrote:
On 2014-06-02 22:59:55 +0900, Fujii Masao wrote:
On Thu, May 29, 2014 at 3:02 PM, Peter Geoghegan <pg@heroku.com> wrote:
On Wed, May 28, 2014 at 10:49 PM, Fujii Masao <masao.fujii@gmail.com>
wrote:
You're concerned about the scenario using pg_upgrade? I'm not sure
the detail
of pg_upgrade. But if it doesn't work properly, we should have gotten
the troubleI'm not worried about pg_upgrade, because by design pg_stat_statements
will discard stats files that originated in earlier versions. However,
I don't see a need to change pg_stat_statements to serialize its
statistics to disk in the pg_stat directory before we branch off 9.4.
As you mentioned, it's harmless.Yeah, that's an idea. OTOH, there is no *strong* reason to postpone
the fix to 9.5. So I just feel inclined to apply the fix now...+1 for fixing it now.
+1 for fixing it for 9.4 before the next beta, but *not* backpatching it to
9.3 - it *is* a behaviour change after all..
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On Tue, Jun 3, 2014 at 6:38 PM, Magnus Hagander <magnus@hagander.net> wrote:
On Mon, Jun 2, 2014 at 4:07 PM, Andres Freund <andres@2ndquadrant.com>
wrote:On 2014-06-02 22:59:55 +0900, Fujii Masao wrote:
On Thu, May 29, 2014 at 3:02 PM, Peter Geoghegan <pg@heroku.com> wrote:
On Wed, May 28, 2014 at 10:49 PM, Fujii Masao <masao.fujii@gmail.com>
wrote:You're concerned about the scenario using pg_upgrade? I'm not sure
the detail
of pg_upgrade. But if it doesn't work properly, we should have gotten
the troubleI'm not worried about pg_upgrade, because by design pg_stat_statements
will discard stats files that originated in earlier versions. However,
I don't see a need to change pg_stat_statements to serialize its
statistics to disk in the pg_stat directory before we branch off 9.4.
As you mentioned, it's harmless.Yeah, that's an idea. OTOH, there is no *strong* reason to postpone
the fix to 9.5. So I just feel inclined to apply the fix now...+1 for fixing it now.
+1 for fixing it for 9.4 before the next beta, but *not* backpatching it to
9.3 - it *is* a behaviour change after all..
Yep, I just applied the patch only to HEAD.
Regards,
--
Fujii Masao
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Fujii-san,
I agree not to backpatch, but I noticed that the 9.3 document about
stats collector doesn't mention $PGDATA/pg_stat.
http://www.postgresql.org/docs/current/static/monitoring-stats.html
It just says:
When the server shuts down, a permanent copy of the statistics data is stored in the global subdirectory, so that statistics can be retained across server restarts.
I'm not sure whether we should modify the 9.3 document at the moment,
but actually the description made me confused a bit.
2014-05-29 12:22 GMT+09:00 Fujii Masao <masao.fujii@gmail.com>:
On Thu, May 29, 2014 at 4:55 AM, Tomas Vondra <tv@fuzzy.cz> wrote:
On 28.5.2014 19:52, Fujii Masao wrote:
On Thu, May 29, 2014 at 12:37 AM, Peter Geoghegan <pg@heroku.com> wrote:
On Wed, May 28, 2014 at 7:01 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
But pg_stat_statements file is saved under $PGDATA/global yet.
Is this intentional or just oversight?I think it's an oversight.
OK, patch attached.
I'm afraid that it's not okay to change the file layout in $PGDATA at this beta1
stage because that change basically seems to need initdb. Otherwise something
like "no such file or directory" error can happen. But in this case what we need
to change is only the location of the pg_stat_statements permanent stats file.
So, without initdb, the server will not be able to find the
pg_stat_statements stats
file, but this is not so harmful. Only the problem is that the
pg_stat_statements
stats which were collected in past would disappear. OTOH, the server can keep
running successfully from then and no critical data will not
disappear. Therefore
I think we can commit this patch even at beta1. Thought?For HEAD, probably yes. But what about backpatching 9.3?
I think No. So we should not backpatch this to 9.3.
Regards,
--
Fujii Masao--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
--
Shigeru HANADA
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Jun 17, 2014 at 2:11 PM, Shigeru Hanada
<shigeru.hanada@gmail.com> wrote:
Fujii-san,
I agree not to backpatch, but I noticed that the 9.3 document about
stats collector doesn't mention $PGDATA/pg_stat.http://www.postgresql.org/docs/current/static/monitoring-stats.html
It just says:
When the server shuts down, a permanent copy of the statistics data is stored in the global subdirectory, so that statistics can be retained across server restarts.
I'm not sure whether we should modify the 9.3 document at the moment,
but actually the description made me confused a bit.
Could you check 8dc90b9c4c45fa30a8f59313e21d294529ef7182 in REL9_3_STABLE
branch? You can see that I added the description of pg_stat directory
into the document
in 9.3.
Regards,
--
Fujii Masao
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Fujii-san,
I found the right description in REL9_3_STABLE branch, thanks for
pointing out the commit.
Sorry for noise.
2014-06-18 12:39 GMT+09:00 Fujii Masao <masao.fujii@gmail.com>:
On Tue, Jun 17, 2014 at 2:11 PM, Shigeru Hanada
<shigeru.hanada@gmail.com> wrote:Fujii-san,
I agree not to backpatch, but I noticed that the 9.3 document about
stats collector doesn't mention $PGDATA/pg_stat.http://www.postgresql.org/docs/current/static/monitoring-stats.html
It just says:
When the server shuts down, a permanent copy of the statistics data is stored in the global subdirectory, so that statistics can be retained across server restarts.
I'm not sure whether we should modify the 9.3 document at the moment,
but actually the description made me confused a bit.Could you check 8dc90b9c4c45fa30a8f59313e21d294529ef7182 in REL9_3_STABLE
branch? You can see that I added the description of pg_stat directory
into the document
in 9.3.Regards,
--
Fujii Masao
--
Shigeru HANADA
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers