possible bug: orphaned files left after immediate shutdown during DDL
Case:
BEGIN;
CREATE TABLE foo AS SELECT generate_series(1,1000);
CHECKPOINT;
SELECT relfilenode FROM pg_class WHERE relname='foo';
Let's say that returns 23456. Send the postmaster a SIGQUIT (immediate
shutdown), and then restart. The file 23456 is still in the filesystem,
but there's no record in pg_class for it. I don't see any obvious path
where it will be removed, so it looks like it will just stay there
forever.
My question is: is this a conscious decision to be paranoid during
recovery, or is this a bug? Or is there some reason that properly
determining which files should be removed at recovery time is
challenging?
Regards,
Jeff Davis
Jeff Davis <pgsql@j-davis.com> writes:
Case:
BEGIN;
CREATE TABLE foo AS SELECT generate_series(1,1000);
CHECKPOINT;
SELECT relfilenode FROM pg_class WHERE relname='foo';
Let's say that returns 23456. Send the postmaster a SIGQUIT (immediate
shutdown), and then restart. The file 23456 is still in the filesystem,
but there's no record in pg_class for it. I don't see any obvious path
where it will be removed, so it looks like it will just stay there
forever.
My question is: is this a conscious decision to be paranoid during
recovery, or is this a bug?
It's intentional ... not that other people haven't complained about it
before. Remember that what you have done is forced a crash, and
recovery from it is crash recovery. If we proactively removed such
files we would very possibly be destroying evidence of forensic value.
IMO, immediate shutdown is not a tool to be used at random, and this
isn't something we need to fix.
regards, tom lane
On Wed, 2011-02-09 at 22:58 -0500, Tom Lane wrote:
It's intentional ... not that other people haven't complained about it
before. Remember that what you have done is forced a crash, and
recovery from it is crash recovery. If we proactively removed such
files we would very possibly be destroying evidence of forensic value.
I thought that might be the case, but I wasn't able to find any previous
discussions.
It might be a good idea to issue a warning during recovery, however,
like "possible orphaned file ...". I'm not sure if it's worth the
bookkeeping effort though.
Regards,
Jeff Davis
Jeff Davis wrote:
On Wed, 2011-02-09 at 22:58 -0500, Tom Lane wrote:
It's intentional ... not that other people haven't complained about it
before. Remember that what you have done is forced a crash, and
recovery from it is crash recovery. If we proactively removed such
files we would very possibly be destroying evidence of forensic value.I thought that might be the case, but I wasn't able to find any previous
discussions.It might be a good idea to issue a warning during recovery, however,
like "possible orphaned file ...". I'm not sure if it's worth the
bookkeeping effort though.
I thought we had a TODO item about removing orphaned files, but I don't
see it now, perhaps because I thought we had fixed that.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +