BUG #10675: alter database set tablespace and unlogged table
The following bug has been logged on the website:
Bug reference: 10675
Logged by: Maxim Boguk
Email address: maxim.boguk@gmail.com
PostgreSQL version: 9.3.4
Operating system: Linux (Ubuntu)
Description:
Hi,
Now bug report with easy/short test case.
PostgreSQL seems doesn't flush dirty buffers related to unlogged tables in
the database during alter database set tablespace ...;
Test case:
mboguk=# create database test tablespace tmp;
CREATE DATABASE
mboguk=# \c test
You are now connected to database "test" as user "mboguk".
test=# create unlogged table test (id integer);
CREATE TABLE
test=# insert into test select * from generate_series(1,10000000);
INSERT 0 10000000
test=# \c postgres
You are now connected to database "postgres" as user "mboguk".
postgres=# alter database test set tablespace pg_default;
ALTER DATABASE
postgres=# checkpoint;
ERROR: checkpoint request failed
HINT: Consult recent messages in the server log for details.
In PostgreSQL logs:
2014-06-16 23:16:41 EDT ERROR: could not open file
"pg_tblspc/16558/PG_9.3_201306121/16559/16560": No such file or directory
2014-06-16 23:16:41 EDT CONTEXT: writing block 27059 of relation
pg_tblspc/16558/PG_9.3_201306121/16559/16560
2014-06-16 23:16:41 EDT WARNING: could not write block 27059 of
pg_tblspc/16558/PG_9.3_201306121/16559/16560
2014-06-16 23:16:41 EDT DETAIL: Multiple failures --- write error might be
permanent.
Some additional info:
select oid,* from pg_tablespace where oid=16558;
oid | spcname | spcowner | spcacl | spcoptions
-------+---------+----------+--------+------------
16558 | tmp | 16397 | |
test=# select relname from pg_class where relfilenode=16560;
relname
---------
test
So the database flush dirty shared buffers related to the unlogged table
into the old file location (pre-alter tablespace).
Kind Regards,
Maksym
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs
On Tue, Jun 17, 2014 at 8:54 AM, <maxim.boguk@gmail.com> wrote:
The following bug has been logged on the website:
Bug reference: 10675
Logged by: Maxim Boguk
Email address: maxim.boguk@gmail.com
PostgreSQL version: 9.3.4
Operating system: Linux (Ubuntu)
Description:Hi,
Now bug report with easy/short test case.
PostgreSQL seems doesn't flush dirty buffers related to unlogged tables in
the database during alter database set tablespace ...;Test case:
mboguk=# create database test tablespace tmp;
CREATE DATABASE
mboguk=# \c test
You are now connected to database "test" as user "mboguk".
test=# create unlogged table test (id integer);
CREATE TABLE
test=# insert into test select * from generate_series(1,10000000);
INSERT 0 10000000
test=# \c postgres
You are now connected to database "postgres" as user "mboguk".
postgres=# alter database test set tablespace pg_default;
ALTER DATABASE
postgres=# checkpoint;
ERROR: checkpoint request failed
HINT: Consult recent messages in the server log for details.In PostgreSQL logs:
2014-06-16 23:16:41 EDT ERROR: could not open file
"pg_tblspc/16558/PG_9.3_201306121/16559/16560": No such file or directory
2014-06-16 23:16:41 EDT CONTEXT: writing block 27059 of relation
pg_tblspc/16558/PG_9.3_201306121/16559/16560
2014-06-16 23:16:41 EDT WARNING: could not write block 27059 of
pg_tblspc/16558/PG_9.3_201306121/16559/16560
2014-06-16 23:16:41 EDT DETAIL: Multiple failures --- write error might be
permanent.
Thanks for the report. I can reproduce this on the current HEAD as well
albeit not with the exact same steps. For me, it happens during the
shutdown checkpoint.
LOG: shutting down
FATAL: could not open file "pg_tblspc/24576/PG_9.5_201406121/40971/40972":
No such file or directory
CONTEXT: writing block 0 of relation pg_tblspc/24576/PG_9.5_201406121/
40971/40972
WARNING: buffer refcount leak: [193] (rel=pg_tblspc/24576/PG_9.5_201406121/
40971/40972, blockNum=0, flags=0x97, refcount=1 1)
TRAP: FailedAssertion("!(RefCountErrors == 0)", File:
"/home/pavan.deolasee/work/pgsql/postgresql/src/backend/storage/buffer/bufmgr.c",
Line: 1773)
LOG: checkpointer process (PID 2070) was terminated by signal 6: Aborted
It's clearly a bug. During a normal or a forced CHECKPOINT, we don't write
buffers of UNLOGGED tables, even if they are dirty. ALTER DATABASE SET
TABLESPACE relies on the checkpoint mechnism to ensure that all dirty
buffers are written to the disk before proceeding with moving the files to
the new tablespace. Leaving behind those dirty buffers with old tablespace
and old relfilenode causes problems later when we try to sync those dirty
buffers to the disk. AFAICS it can happen during the SHUTDOWN checkpoint or
the End-of-Recovery checkpoint, at least in the HEAD.
ISTM that the right fix is to write *all* dirty pages during a FORCE
checkpoint since system relies on FORCE checkpoints to handle such
alterations. Attached patch fixes this for the HEAD. But this needs to be
fixed all the way to 9.1 when unlogged tables were first introduced.
Thanks,
Pavan
--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee
Attachments:
pg_bug10675.patchapplication/octet-stream; name=pg_bug10675.patchDownload+3-1
On Wed, Jun 18, 2014 at 12:31 PM, Pavan Deolasee <pavan.deolasee@gmail.com>
wrote:
Attached patch fixes this for the HEAD. But this needs to be fixed all
the way to 9.1 when unlogged tables were first introduced.
I'd forgotten to update the comment in the code. Fixed in the attached
version.
Thanks,
Pavan
--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee
Attachments:
pg_bug10675_v2.patchapplication/octet-stream; name=pg_bug10675_v2.patchDownload+5-4
On 2014-06-18 15:33:58 +0530, Pavan Deolasee wrote:
On Wed, Jun 18, 2014 at 12:31 PM, Pavan Deolasee <pavan.deolasee@gmail.com>
wrote:Attached patch fixes this for the HEAD. But this needs to be fixed all
the way to 9.1 when unlogged tables were first introduced.
I'd forgotten to update the comment in the code. Fixed in the attached
version.Thanks,
Pavan--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index c070278..e0b9aff 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -1221,11 +1221,12 @@ BufferSync(int flags) ResourceOwnerEnlargeBuffers(CurrentResourceOwner);/* - * Unless this is a shutdown checkpoint, we write only permanent, dirty - * buffers. But at shutdown or end of recovery, we write all dirty - * buffers. + * If a force checkpoint or at shutdown or end of recovery, we write all + * dirty buffers, otherwise we only write permanent buffers */ - if (!((flags & CHECKPOINT_IS_SHUTDOWN) || (flags & CHECKPOINT_END_OF_RECOVERY))) + if (!((flags & CHECKPOINT_IS_SHUTDOWN) || + (flags & CHECKPOINT_END_OF_RECOVERY) || + (flags & CHECKPOINT_FORCE))) mask |= BM_PERMANENT;
Won't that significantly regress manually issued CHECKPOINT;s?
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs
On Wed, Jun 18, 2014 at 3:40 PM, Andres Freund <andres@2ndquadrant.com>
wrote:
Won't that significantly regress manually issued CHECKPOINT;s?
May be if there are large unlogged table(s) which are frequently updated
between manual checkpoints. I don't know how unlogged tables are being
currently used to make that call. We could add another flag and use that
while taking system checkpoints. But I wonder if not flushing dirty buffers
of unlogged tables at a checkpoint is a bad idea anyways. User might expect
that the unlogged tables to sustain server crash or unclean shutdown if
there had been no writes after successful manual checkpoint(s). I
understand we provide no such guarantee and its explicitly stated in the
documentation, but user may have that expectation after a successful
checkpoint or at the least would expect a way to flush unlogged tables to
disk. Currently I see no way of doing that.
Thanks,
Pavan
--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee
On 2014-06-18 16:15:47 +0530, Pavan Deolasee wrote:
On Wed, Jun 18, 2014 at 3:40 PM, Andres Freund <andres@2ndquadrant.com>
wrote:Won't that significantly regress manually issued CHECKPOINT;s?
May be if there are large unlogged table(s) which are frequently updated
between manual checkpoints. I don't know how unlogged tables are being
currently used to make that call.
I think that's actually one of the major reasons to use unlogged tables.
We could add another flag and use that while taking system checkpoints.
Don't really have a better idea. CHECKPOINT_FLUSH_ALL?
But I wonder if not flushing dirty buffers
of unlogged tables at a checkpoint is a bad idea anyways. User might expect
that the unlogged tables to sustain server crash or unclean shutdown if
there had been no writes after successful manual checkpoint(s).
They'll get reset at unlcean startup anyway. Independent of having been
touched or not.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs
Andres Freund <andres@2ndquadrant.com> writes:
On 2014-06-18 16:15:47 +0530, Pavan Deolasee wrote:
But I wonder if not flushing dirty buffers
of unlogged tables at a checkpoint is a bad idea anyways. User might expect
that the unlogged tables to sustain server crash or unclean shutdown if
there had been no writes after successful manual checkpoint(s).
They'll get reset at unlcean startup anyway. Independent of having been
touched or not.
I'm with Pavan on this one: it's *not* a good thing that manually issued
checkpoints skip unlogged tables. That's surprising, possibly dangerous,
and no case whatsoever has been made that anyone sees it as an important
performance benefit.
I trust that a shutdown checkpoint, at least, would write such pages?
If so, I'd expect that a manual checkpoint would do it too. Maybe
I'm checkpointing because I want to be sure that the shutdown will be
quick so I can do a minor release update with minimal downtime.
I think we should just change this. -1 for new flags and more
complication.
regards, tom lane
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs
On 2014-06-18 09:45:46 -0400, Tom Lane wrote:
Andres Freund <andres@2ndquadrant.com> writes:
On 2014-06-18 16:15:47 +0530, Pavan Deolasee wrote:
But I wonder if not flushing dirty buffers
of unlogged tables at a checkpoint is a bad idea anyways. User might expect
that the unlogged tables to sustain server crash or unclean shutdown if
there had been no writes after successful manual checkpoint(s).They'll get reset at unlcean startup anyway. Independent of having been
touched or not.I'm with Pavan on this one: it's *not* a good thing that manually issued
checkpoints skip unlogged tables. That's surprising, possibly dangerous,
and no case whatsoever has been made that anyone sees it as an important
performance benefit.
I don't understand what dangers it has? Any unclean shutdown resets all
unlogged tables.
/* REDO */
if (InRecovery)
{
...
/*
* We're in recovery, so unlogged relations may be trashed and must be
* reset. This should be done BEFORE allowing Hot Standby
* connections, so that read-only backends don't try to read whatever
* garbage is left over from before.
*/
ResetUnloggedRelations(UNLOGGED_RELATION_CLEANUP);
...
And I don't really see why it's so surprising given that normal
checkpoints skip those buffers?
I trust that a shutdown checkpoint, at least, would write such pages?
Yes.
If so, I'd expect that a manual checkpoint would do it too. Maybe
I'm checkpointing because I want to be sure that the shutdown will be
quick so I can do a minor release update with minimal downtime.
That's a fair argument.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs
On Wed, Jun 18, 2014 at 7:49 PM, Andres Freund <andres@2ndquadrant.com>
wrote:
On 2014-06-18 09:45:46 -0400, Tom Lane wrote:
Andres Freund <andres@2ndquadrant.com> writes:
On 2014-06-18 16:15:47 +0530, Pavan Deolasee wrote:
But I wonder if not flushing dirty buffers
of unlogged tables at a checkpoint is a bad idea anyways. User mightexpect
that the unlogged tables to sustain server crash or unclean shutdown
if
there had been no writes after successful manual checkpoint(s).
They'll get reset at unlcean startup anyway. Independent of having been
touched or not.I'm with Pavan on this one: it's *not* a good thing that manually issued
checkpoints skip unlogged tables. That's surprising, possibly dangerous,
and no case whatsoever has been made that anyone sees it as an important
performance benefit.I don't understand what dangers it has? Any unclean shutdown resets all
unlogged tables.
Looks like there is no agreement on this. I agree with Andreas that given
the current mechanism of truncating unlogged relations at the end of redo
recovery, there is no danger in not flushing the dirty buffers belonging to
unlogged relation at a normal checkpoint. Having said that, I find it
confusing that we don't do that, for one reason that Tom explained and also
because there is practically just no way to flush those dirty buffers to
disk if the user wants so.
Also, there had been discussions about altering unlogged tables to normal
tables and we may also want to improve upon the current mechanism of
truncating unlogged relations at the end of recovery even if the table was
fully synced to the disk. It looks simpler to just flush everything instead
of devising a new flag for checkpoint.
Anyone else has an opinion on this?
Thanks,
Pavan
--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee
On Fri, Jun 20, 2014 at 11:34 PM, Pavan Deolasee <pavan.deolasee@gmail.com>
wrote:
Looks like there is no agreement on this. I agree with Andreas that given
the current mechanism of truncating unlogged relations at the end of redo
recovery, there is no danger in not flushing the dirty buffers belonging to
unlogged relation at a normal checkpoint. Having said that, I find it
confusing that we don't do that, for one reason that Tom explained and also
because there is practically just no way to flush those dirty buffers to
disk if the user wants so.Also, there had been discussions about altering unlogged tables to normal
tables and we may also want to improve upon the current mechanism of
truncating unlogged relations at the end of recovery even if the table was
fully synced to the disk. It looks simpler to just flush everything instead
of devising a new flag for checkpoint.Anyone else has an opinion on this?
Since I did not hear anything on this, I created a patch that adds a new
flag to tell checkpointer to flush all pages to the disk. Tom (and even I)
have reservations about the approach, but I would nevertheless leave it to
the committer to decide. IMV we must fix this bug one way or the other.
Otherwise users face risk of failing to do clean shutdown.
Thanks,
Pavan
--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee
Attachments:
pg_bug10675_v3.patchapplication/octet-stream; name=pg_bug10675_v3.patchDownload+23-13
On 2014-06-20 23:34:36 +0530, Pavan Deolasee wrote:
On Wed, Jun 18, 2014 at 7:49 PM, Andres Freund <andres@2ndquadrant.com>
wrote:On 2014-06-18 09:45:46 -0400, Tom Lane wrote:
Andres Freund <andres@2ndquadrant.com> writes:
On 2014-06-18 16:15:47 +0530, Pavan Deolasee wrote:
I'm with Pavan on this one: it's *not* a good thing that manually issued
checkpoints skip unlogged tables. That's surprising, possibly dangerous,
and no case whatsoever has been made that anyone sees it as an important
performance benefit.I don't understand what dangers it has? Any unclean shutdown resets all
unlogged tables.Looks like there is no agreement on this. I agree with Andreas that given
the current mechanism of truncating unlogged relations at the end of redo
recovery, there is no danger in not flushing the dirty buffers belonging to
unlogged relation at a normal checkpoint. Having said that, I find it
confusing that we don't do that, for one reason that Tom explained and also
because there is practically just no way to flush those dirty buffers to
disk if the user wants so.
Which reason, except the faster shutdown, for a user to want that?
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs
On 2014-07-02 17:28:25 +0530, Pavan Deolasee wrote:
On Fri, Jun 20, 2014 at 11:34 PM, Pavan Deolasee <pavan.deolasee@gmail.com>
wrote:Looks like there is no agreement on this. I agree with Andreas that given
the current mechanism of truncating unlogged relations at the end of redo
recovery, there is no danger in not flushing the dirty buffers belonging to
unlogged relation at a normal checkpoint. Having said that, I find it
confusing that we don't do that, for one reason that Tom explained and also
because there is practically just no way to flush those dirty buffers to
disk if the user wants so.Also, there had been discussions about altering unlogged tables to normal
tables and we may also want to improve upon the current mechanism of
truncating unlogged relations at the end of recovery even if the table was
fully synced to the disk. It looks simpler to just flush everything instead
of devising a new flag for checkpoint.Anyone else has an opinion on this?
Since I did not hear anything on this, I created a patch that adds a new
flag to tell checkpointer to flush all pages to the disk. Tom (and even I)
have reservations about the approach, but I would nevertheless leave it to
the committer to decide. IMV we must fix this bug one way or the other.
Otherwise users face risk of failing to do clean shutdown.
I think one reason for the separate flag is that the checkpoint
performed by pg_start_backup/pg_basebackup shouldn't just become more
expensive because unlogged tables are needlessly flushed to disk. After
all, unlogged tables are used because normal tables have a too high
overhead in that scenario.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs
On Wed, Jul 2, 2014 at 5:36 PM, Andres Freund <andres@2ndquadrant.com>
wrote:
Which reason, except the faster shutdown, for a user to want that?
May be to clean up shared buffers so that a regular backend does not need
to write the page during eviction? Of course bgwriter would that but not
under user's control. Say user wants to force a checkpoint before some high
priority, response time sensitive tasks are started. Not sure though if
this a valid use case in real world.
Thanks,
Pavan
--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee
Andres Freund <andres@2ndquadrant.com> writes:
I think one reason for the separate flag is that the checkpoint
performed by pg_start_backup/pg_basebackup shouldn't just become more
expensive because unlogged tables are needlessly flushed to disk. After
all, unlogged tables are used because normal tables have a too high
overhead in that scenario.
AFAIK, the "overhead" that unlogged tables are trying to avoid is WAL
I/O. Nobody has argued (until this thread) that we are worried about
whether checkpoints write them.
regards, tom lane
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs
On Wed, Jul 2, 2014 at 11:38 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Andres Freund <andres@2ndquadrant.com> writes:
I think one reason for the separate flag is that the checkpoint
performed by pg_start_backup/pg_basebackup shouldn't just become more
expensive because unlogged tables are needlessly flushed to disk. After
all, unlogged tables are used because normal tables have a too high
overhead in that scenario.AFAIK, the "overhead" that unlogged tables are trying to avoid is WAL
I/O. Nobody has argued (until this thread) that we are worried about
whether checkpoints write them.
Sorry, I didn't see this thread until just now.
I was definitely worried about that issue when I wrote the unlogged
tables patch, and I added the BM_PERMANENT for precisely that reason.
I think it's a completely legitimate worry, too. The point of
avoiding WAL I/O is that writing WAL to disk is expensive; writing
data files to disk is no less expensive, and often moreso, because
it's not always sequential I/O.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs
On 2014-07-02 11:38:51 -0400, Tom Lane wrote:
Andres Freund <andres@2ndquadrant.com> writes:
I think one reason for the separate flag is that the checkpoint
performed by pg_start_backup/pg_basebackup shouldn't just become more
expensive because unlogged tables are needlessly flushed to disk. After
all, unlogged tables are used because normal tables have a too high
overhead in that scenario.AFAIK, the "overhead" that unlogged tables are trying to avoid is WAL
I/O. Nobody has argued (until this thread) that we are worried about
whether checkpoints write them.
I don't think that's true. In production scenarios checkpoint IO is one
of the two top problems I see (the other being crazy amount of WAL due
to FPIs because of too short checkpoints). And unlogged tables have
explicitly been excluded from checkpoints, so it's not like nobody has
thought about it. I seem to recall lengthy discussions even. Yep
the discussion is around
http://archives.postgresql.org/message-id/AANLkTimxC%2BG9M9_s0dXa_huoAeZpkCmoWCo5S-7DsLi%3D%40mail.gmail.com
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs
On Wed, Jul 2, 2014 at 5:28 PM, Pavan Deolasee <pavan.deolasee@gmail.com>
wrote:
On Fri, Jun 20, 2014 at 11:34 PM, Pavan Deolasee <pavan.deolasee@gmail.com
wrote:
Looks like there is no agreement on this. I agree with Andreas that given
the current mechanism of truncating unlogged relations at the end of redo
recovery, there is no danger in not flushing the dirty buffers belonging to
unlogged relation at a normal checkpoint. Having said that, I find it
confusing that we don't do that, for one reason that Tom explained and also
because there is practically just no way to flush those dirty buffers to
disk if the user wants so.Also, there had been discussions about altering unlogged tables to normal
tables and we may also want to improve upon the current mechanism of
truncating unlogged relations at the end of recovery even if the table was
fully synced to the disk. It looks simpler to just flush everything instead
of devising a new flag for checkpoint.Anyone else has an opinion on this?
Since I did not hear anything on this, I created a patch that adds a new
flag to tell checkpointer to flush all pages to the disk. Tom (and even I)
have reservations about the approach, but I would nevertheless leave it to
the committer to decide. IMV we must fix this bug one way or the other.
Otherwise users face risk of failing to do clean shutdown.
As Robert as voted in favor of keeping existing checkpoint behavior intact,
should we consider this patch before the minor releases are out next week?
Thanks,
Pavan
--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee
On Wed, Jul 16, 2014 at 1:32 PM, Pavan Deolasee
<pavan.deolasee@gmail.com> wrote:
On Wed, Jul 2, 2014 at 5:28 PM, Pavan Deolasee <pavan.deolasee@gmail.com>
wrote:On Fri, Jun 20, 2014 at 11:34 PM, Pavan Deolasee
<pavan.deolasee@gmail.com> wrote:Looks like there is no agreement on this. I agree with Andreas that given
the current mechanism of truncating unlogged relations at the end of redo
recovery, there is no danger in not flushing the dirty buffers belonging to
unlogged relation at a normal checkpoint. Having said that, I find it
confusing that we don't do that, for one reason that Tom explained and also
because there is practically just no way to flush those dirty buffers to
disk if the user wants so.Also, there had been discussions about altering unlogged tables to normal
tables and we may also want to improve upon the current mechanism of
truncating unlogged relations at the end of recovery even if the table was
fully synced to the disk. It looks simpler to just flush everything instead
of devising a new flag for checkpoint.Anyone else has an opinion on this?
Since I did not hear anything on this, I created a patch that adds a new
flag to tell checkpointer to flush all pages to the disk. Tom (and even I)
have reservations about the approach, but I would nevertheless leave it to
the committer to decide. IMV we must fix this bug one way or the other.
Otherwise users face risk of failing to do clean shutdown.As Robert as voted in favor of keeping existing checkpoint behavior intact,
should we consider this patch before the minor releases are out next week?
What's the status of this? ISTM that the patch has not been applied yet. Right?
Regards,
--
Fujii Masao
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs
On Thu, Aug 7, 2014 at 5:54 PM, Fujii Masao <masao.fujii@gmail.com> wrote:
What's the status of this?
There was no clear consensus. The thread has patches per both the
approaches. I tried to bring it up couple of times, last time just before
the minor releases were out. But did not get any attention.
ISTM that the patch has not been applied yet. Right?
Right. I've now added it to the commitfest so that we don't lose track of
it.
Thanks,
Pavan
--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee
On Fri, Aug 8, 2014 at 4:08 PM, Pavan Deolasee <pavan.deolasee@gmail.com> wrote:
On Thu, Aug 7, 2014 at 5:54 PM, Fujii Masao <masao.fujii@gmail.com> wrote:
What's the status of this?
There was no clear consensus. The thread has patches per both the
approaches. I tried to bring it up couple of times, last time just before
the minor releases were out. But did not get any attention.
I vote to make checkpoint flush unlogged tables only when the special flag
(that maybe we will introduce) is set. Unlogged tables are not flushed by
manual checkpoint. Per Robert, that's originally intentional. No dangerous
risk that may cause has been reported yet. So I don't think that we need to
change manual checkpoint so that it always flushes unlogged tables, at least
for now.
OTOH, I agree to extend manual checkpoint so that it can flush unlogged
table to shorten the time of subsequent shutdown checkpoint. For example,
we can implement something like "CHECKPOINT [ALL | PERMANENT | UNLOGGED]".
But that's a feature request. We should work on this later, not for now.
ISTM that the patch has not been applied yet. Right?
Right. I've now added it to the commitfest so that we don't lose track of
it.
Thanks!
Regards,
--
Fujii Masao
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs