General Bug Report: TEMP TABLES becomes permanent CATALOG TABLES

============================================================================
POSTGRESQL BUG REPORT TEMPLATE
============================================================================

Your name : Ricardo Coelho
Your email address : rcoelho@px.com.br

Category : runtime: back-end
Severity : serious

Summary: TEMP TABLES becomes permanent CATALOG TABLES

System Configuration
--------------------
Operating System : RedHat Linux 5.2 Intel

PostgreSQL version : 6.5

Compiler used : gcc

Hardware:
---------
Pentium II 350MHz 128M RAM

Versions of other tools:
------------------------

--------------------------------------------------------------------------

Problem Description:
--------------------

When you execute a wrong SQL command after create a TEMP TABLE,

postgres backend doesn't drop this table after connection ends. We can't
drop pg_temp.PID.N with "drop table" command because it is a system
catalog table. So, we had to dumpall the database, edit db.out file to
erase lines of pg_temp??? create table and reload it again.

--------------------------------------------------------------------------

Test Case:
----------
psql mydb
mydb=> select * into temp table TMP from anytable;
mydb=> drop table invalidTable;
mydb=> \q
psql mydb
mydb=> \dS --> You will see a new permanent system table.

I can confirm that this a bug. I am looking at it now. As a
workaround, you can delete the temp tables by starting a postgres
backend with the -O flag to allow system table modifications, and
droping the table.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

When you execute a wrong SQL command after create a TEMP TABLE,
postgres backend doesn't drop this table after connection ends. We can't
drop pg_temp.PID.N with "drop table" command because it is a system
catalog table. So, we had to dumpall the database, edit db.out file to
erase lines of pg_temp??? create table and reload it again.

--------------------------------------------------------------------------

Test Case:
----------
psql mydb
mydb=> select * into temp table TMP from anytable;
mydb=> drop table invalidTable;
mydb=> \q
psql mydb
mydb=> \dS --> You will see a new permanent system table.

OK, I have looked at the problem, and found a few things. First, if you
look in data/base/dbname, you will see the actual temp files are not
there. The only place they exist after psql exit is in pg_class.

Second, I found that if I do:

select * into temp xx from yy;
drop table badname;
select * into temp xxe from yy;

it does not leave around the temp tables, but it if I add a second drop,
it fails again, so it appears that I have to exit on a bad command to
have the entries left around. Just issuing the select without the bad
command cleans up properly, so I am left to believe that the failed
command is doing something strange. My guess is that somehow the dirty
marks on buffers is getting cleared by the bad command, and on exit, the
new pg_class blocks are not getting put on disk.

Tom Lane, you did the code that does special things when there is a
failed command, right? It was because creating a table inside a failed
transaction was leaving around the old cache entries. Could this be
causing this problem?

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

OK, I have looked at the problem, and found a few things. First, if you
look in data/base/dbname, you will see the actual temp files are not
there. The only place they exist after psql exit is in pg_class.

Second, I found that if I do:

select * into temp xx from yy;
drop table badname;
select * into temp xxe from yy;

it does not leave around the temp tables, but it if I add a second drop,
it fails again, so it appears that I have to exit on a bad command to
have the entries left around. Just issuing the select without the bad
command cleans up properly, so I am left to believe that the failed
command is doing something strange. My guess is that somehow the dirty
marks on buffers is getting cleared by the bad command, and on exit, the
new pg_class blocks are not getting put on disk.

Tom Lane, you did the code that does special things when there is a
failed command, right? It was because creating a table inside a failed
transaction was leaving around the old cache entries. Could this be
causing this problem?

I think I may know the cause. The at_exit removal of the temp tables is
taking place in a failed transaction. Let me test that idea tomorrow.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Test Case:
----------
psql mydb
mydb=> select * into temp table TMP from anytable;
mydb=> drop table invalidTable;
mydb=> \q
psql mydb
mydb=> \dS --> You will see a new permanent system table.

I can confirm that this a bug. I am looking at it now. As a
workaround, you can delete the temp tables by starting a postgres
backend with the -O flag to allow system table modifications, and
droping the table.

OK, I have fixed the problem, and the patch is attached. The fix will
appear in 6.5.1, due out in mid-July.

The problem is that the temp tables are removed on exit, but they were
not given their own transaction, and were executed in the last
transaction of the session. If that last session was aborted, the
entries were not being removed from pg_class.

If anyone thinks doing a transaction on exit is a bad idea, please let
me know.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Bruce Momjian <maillist@candle.pha.pa.us> writes:

I think I may know the cause. The at_exit removal of the temp tables is
taking place in a failed transaction. Let me test that idea tomorrow.

Take a look at the code for cleaning up the listen/notify table
(Async_UnlistenOnExit() in backend/commands/async.c). It used to
have problems with cleaning up when the last transaction executed
by the backend had failed. I fixed that by aborting any old transaction
and starting/committing a new one. I believe that any at_exit,
on_shmem_exit, etc routine that tries to perform database changes will
need to be coded similarly.

regards, tom lane

Bruce Momjian <maillist@candle.pha.pa.us> writes:

I think I may know the cause. The at_exit removal of the temp tables is
taking place in a failed transaction. Let me test that idea tomorrow.

Take a look at the code for cleaning up the listen/notify table
(Async_UnlistenOnExit() in backend/commands/async.c). It used to
have problems with cleaning up when the last transaction executed
by the backend had failed. I fixed that by aborting any old transaction
and starting/committing a new one. I believe that any at_exit,
on_shmem_exit, etc routine that tries to perform database changes will
need to be coded similarly.

Thanks. I have added AbortOutOfAnyTransaction() to the fix. I did not
have that function call.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Problem Description: -------------------- When you execute a
wrong SQL command after create a TEMP TABLE, postgres backend
doesn't drop this table after connection ends. We can't drop
pg_temp.PID.N with "drop table" command because it is a system
catalog table. So, we had to dumpall the database, edit db.out
file to erase lines of pg_temp??? create table and reload it
again.

--------------------------------------------------------------------------

Test Case: ---------- psql mydb mydb=> select * into temp table
TMP from anytable; mydb=> drop table invalidTable; mydb=> \q
psql mydb mydb=> \dS --> You will see a new permanent system
table.

This will be fixed in 6.5.1, due out next week.

--
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026