DISTINCT ON: speak now or forever hold your peace

Tom Lane wrote:

If I don't hear loud hollers very soon, I'm going to eliminate the
DISTINCT ON "feature" for 7.0. As previously discussed, this feature
is not standard SQL and has no clear semantic interpretation.

I don't feel overly strongly about this, but if I remember right you can
do some pretty cool things with this feature, provided you do define
some semantics clearly. Like I think you can find the first tuple
(given some ORDER BY clause) that fulfills some criteria. I think it is

SELECT DISTINCT ON name name, age ORDER BY age;

will get the youngest person. This might not be clearly specified now,
but
as long as it's useful, how about clearly defining it? I don't know that
there is an easy way of doing this in standard SQL. I don't see any
problems with useful extensions to SQL. If people want standards, they
don't have to use it.

From bouncefilter Mon Jan 24 21:45:41 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id VAA55291
for <pgsql-hackers@postgreSQL.org>;
Mon, 24 Jan 2000 21:45:15 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id SAA13134;
Mon, 24 Jan 2000 18:43:43 -0800 (PST)
Message-Id: <3.0.1.32.20000124184137.01069490@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Mon, 24 Jan 2000 18:41:37 -0800
To: Tom Lane <tgl@sss.pgh.pa.us>, "Hiroshi Inoue" <Inoue@tpf.co.jp>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: [HACKERS] Happy column dropping
Cc: "Peter Eisentraut" <peter_e@gmx.net>,
"PostgreSQL Development" <pgsql-hackers@postgreSQL.org>
In-Reply-To: <4891.948766352@sss.pgh.pa.us>
References: <001001bf66d7$b531ba00$2801007e@tpf.co.jp>
<001001bf66d7$b531ba00$2801007e@tpf.co.jp>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 09:12 PM 1/24/00 -0500, Tom Lane wrote:

* VACUUM with rebuild or ADD/DROP COLUMN: write new data into
files with an incremented version number.

Just a reality check for my learning of the internals. Out of curiousity
I coincidently have spent the last hour looking to see how add column's
implemented. It doesn't appear to do anything other than the new attribute
to the proper system table. heap_getattr() just returns null if you ask
for an attribute past the end of the tuple.

This would appear to be (at least one reason) why you can't add a "not null"
constraint to a column you're adding to an existing relation, or set the
new column to some non-null default value.

Correct? (again, to see if my eyeballs and brain are working in synch
tonight)

Does your comment imply that it's planned to change this, i.e. actually
add the new column to each tuple in the relation rather than use the
existing, somewhat elegant hack?

Just curious...

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Mon Jan 24 21:51:41 2000
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67] (may
be forged)) by hub.org (8.9.3/8.9.3) with ESMTP id VAA56331
for <pgsql-hackers@postgreSQL.org>;
Mon, 24 Jan 2000 21:50:46 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
VAA21476;
Mon, 24 Jan 2000 21:48:46 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001250248.VAA21476@candle.pha.pa.us>
Subject: Re: [HACKERS] Happy column dropping
In-Reply-To: <3.0.1.32.20000124155337.010834f0@mail.pacifier.com> from Don
Baccus at "Jan 24, 2000 03:53:37 pm"
To: Don Baccus <dhogaza@pacifier.com>
Date: Mon, 24 Jan 2000 21:48:46 -0500 (EST)
CC: Peter Eisentraut <peter_e@gmx.net>, Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

I was unaware that other folks had pointed to the copy/rename approach
earlier as being a possible means of implementation. I thought you'd
pulled that one out of your hat. Still, more advance discussion would've
perhaps led to other approaches to investigate, just as discussion
now is doing.

I'm going to be blunt: implementation of "drop column" by doing a copy/rename
isn't something one expects of a competitive commercial-quality RDBMS.

Perhaps it's the best we can expect of Postgres, though. If so, so be it.

Maybe that is true. Having phantom column all over the code is going to
be a mess, and hardly worth it considering how many developers there are
and how many _big_ items still have to be done.

Messing up code for one feature is rarely worth it.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Mon Jan 24 21:53:42 2000
Received: from mailo.vtcif.telstra.com.au (mailo.vtcif.telstra.com.au
[202.12.144.17]) by hub.org (8.9.3/8.9.3) with ESMTP id VAA56842
for <pgsql-hackers@postgreSQL.org>;
Mon, 24 Jan 2000 21:53:04 -0500 (EST)
(envelope-from chris@bitmead.com)
Received: (from uucp@localhost) by mailo.vtcif.telstra.com.au (8.8.2/8.6.9) id
NAA05410 for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 13:52:33 +1100 (EST)
Received: from maili.vtcif.telstra.com.au(202.12.142.17)
via SMTP by mailo.vtcif.telstra.com.au, id smtpdiUfAj_;
Tue Jan 25 13:52:18 2000
Received: (from uucp@localhost) by maili.vtcif.telstra.com.au (8.8.2/8.6.9) id
NAA12627 for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 13:52:17 +1100 (EST)
Received: from localhost(127.0.0.1), claiming to be "mail.cdn.telstra.com.au"
via SMTP by localhost, id smtpd0WWy6e; Tue Jan 25 13:51:49 2000
Received: from lunitari.nimrod.itg.telecom.com.au
(lunitari.nimrod.itg.telecom.com.au [192.53.254.48]) by
mail.cdn.telstra.com.au (8.8.2/8.6.9) with ESMTP id NAA20099
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 13:51:48 +1100 (EST)
Received: from majere.design (majere [192.53.254.45])
by lunitari.nimrod.itg.telecom.com.au (8.9.1/8.9.3) with ESMTP id
NAA20622 for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 13:51:13 +1100 (EST)
Received: from bitmead.com (localhost [127.0.0.1])
by majere.design (8.8.8+Sun/8.8.8) with ESMTP id NAA10787
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 13:50:30 +1100 (EST)
Sender: chrisb@nimrod.itg.telstra.com.au
Message-ID: <388D0F76.6C04FADA@bitmead.com>
Date: Tue, 25 Jan 2000 13:50:30 +1100
From: Chris Bitmead <chris@bitmead.com>
Organization: IBM Global Services
X-Mailer: Mozilla 4.6 [en] (X11; I; SunOS 5.6 sun4u)
X-Accept-Language: en
MIME-Version: 1.0
To: PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Happy column dropping
References: <001001bf66d7$b531ba00$2801007e@tpf.co.jp>
<4891.948766352@sss.pgh.pa.us>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Tom Lane wrote:

"Hiroshi Inoue" <Inoue@tpf.co.jp> writes:

But I'm now inclined to generate unique relation file name each time
in order to have different file names for different versions of a same
relation oid. Without chainging relation oids,we would be to do
nothing about their attributes/constraints etc.

I was thinking about adding a "version number" to the pg_class entry
for a relation, and then having its actual filename look like

Well, if you are going to re-write the files in place on update, a
version number seems like overkill. All you need is version "a" and
"b". Every time you do a change it swaps from version RELATIONOID_a to
RELATIONOID_b , or RELATIONOID_b to RELATIONOID_a. Or you could just go
for RELATIONOID and RELATIONOID_new and do a rename (although I guess
you're trying to avoid that).

Show quoted text

RELATIONOID_vVERSION.SEGMENT

Then we have:

* Table rename: doesn't change the filename at all

* VACUUM with rebuild or ADD/DROP COLUMN: write new data into
files with an incremented version number. Also heap_update
the pg_class tuple with new version number. At instant of
commit, voila: the new files are valid, the old ones aren't.
Works for indexes, too.

Anyway this must be decided after sufficient discussion.
It's not the time to do it now.

Agreed. I think we are too close to 7.0 beta to consider doing this.
We can start thinking about it for 7.1 though.

regards, tom lane

************

Tom Lane wrote:

If I don't hear loud hollers very soon, I'm going to eliminate the
DISTINCT ON "feature" for 7.0. As previously discussed, this feature
is not standard SQL and has no clear semantic interpretation.

I grepped our source code and found this query:

INSERT INTO temp_sales
SELECT DISTINCT on key supplysources.supplysource,
incharges.supply, targets.target, incharges.saledate,
incharges.supplyunit, '', incharges.quantity, incharges.company,
incharges.costcntr, 'Replenish', incharges.price, '','','', 0,
text(supplysources.supplysource)||
text(incharges.supply)||
text(targets.target)||
text(incharges.saledate) as key
FROM supplysources, incharges, targets WHERE
supplysources.warehouse = incharges.warehouse AND
(targets.site,targets.area) = (incharges.site,incharges.area);

What happens is that a large charges file which is transferred to
a mainframe ERP application is first brought into PostgreSQL.
Depending upon certain race conditions, duplicate "sales" records
can appear in the data file. We use DISTINCT ON to pick (as you
point out) an arbitrary record when duplicates appear. I suppose
we could do a DELETE ... WHERE NOT EXISTS after the import. Using
DISTINCT ON just saves a step. I don't have any arguments beyond
the grounds that we're using it in existing code as a duplicate
record filter - :-(

Just FYI,

Mike Mascari

From bouncefilter Mon Jan 24 21:54:41 2000
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67] (may
be forged)) by hub.org (8.9.3/8.9.3) with ESMTP id VAA56999
for <pgsql-hackers@postgreSQL.org>;
Mon, 24 Jan 2000 21:53:55 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
VAA21667;
Mon, 24 Jan 2000 21:51:52 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001250251.VAA21667@candle.pha.pa.us>
Subject: Re: [HACKERS] Happy column dropping
In-Reply-To: <27422.948759943@sss.pgh.pa.us> from Tom Lane at "Jan 24,
2000 07:25:43 pm"
To: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon, 24 Jan 2000 21:51:51 -0500 (EST)
CC: Hiroshi Inoue <Inoue@tpf.co.jp>, Peter Eisentraut <peter_e@gmx.net>,
The Hermit Hacker <scrappy@hub.org>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

"Hiroshi Inoue" <Inoue@tpf.co.jp> writes:

As far as I see,fixed mapping relname to relation filename is the
problem. This doesn't allow the coexistence of old and new relation
files.

Yes, and Vadim has proposed using relation OIDs as filenames for
reasons of his own, IIRC.

If we did that we could also solve the problems we have now with
rolling back table deletion/rename inside an aborted transaction.

I've been resisting this idea, but maybe it's time to bite the bullet
and accept that relation filenames can't be the same as the logical
names of the relations.

Yes, that is going to happen, clearly. New file names will be
table_name+oid. Solves many problems.

New file name will not be just oid. Too hard to administer.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

At 13:41 25/01/00 +1100, Chris Bitmead wrote:

Tom Lane wrote:

If I don't hear loud hollers very soon, I'm going to eliminate the
DISTINCT ON "feature" for 7.0. As previously discussed, this feature
is not standard SQL and has no clear semantic interpretation.

I don't feel overly strongly about this, but if I remember right you can
do some pretty cool things with this feature, provided you do define
some semantics clearly. Like I think you can find the first tuple
(given some ORDER BY clause) that fulfills some criteria. I think it is

SELECT DISTINCT ON name name, age ORDER BY age;

will get the youngest person. This might not be clearly specified now,
but
as long as it's useful, how about clearly defining it? I don't know that
there is an easy way of doing this in standard SQL.

I don't know about PGSQL, but in other systems, I use:

Select <whatever> from <wherever> order by age asc limit to 1 row;

I *think* the PGSQL syntax is:

Select <whatever> from <wherever> order by age asc limit 1;

I have no idea if the optimizer does 'fast first' optimizations, so I don't
know how quick this would be on a large table.

----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.C.N. 008 659 498) | /(@) ______---_
Tel: +61-03-5367 7422 | _________ \
Fax: +61-03-5367 7430 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/

From bouncefilter Mon Jan 24 22:04:42 2000
Received: from netrinsics.com ([202.106.6.112])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA64467
for <pgsql-hackers@hub.org>; Mon, 24 Jan 2000 22:04:06 -0500 (EST)
(envelope-from robinson@netrinsics.com)
Received: (from robinson@localhost)
by netrinsics.com (8.9.3/8.9.3) id LAA39308;
Tue, 25 Jan 2000 11:04:12 +0800 (+0800) (envelope-from robinson)
Date: Tue, 25 Jan 2000 11:04:12 +0800 (+0800)
From: Michael Robinson <robinson@netrinsics.com>
Message-Id: <200001250304.LAA39308@netrinsics.com>
To: robinson@netrinsics.com, t-ishii@sra.co.jp
Subject: Re: [HACKERS] fatal copy in/out error (6.5.3)
Cc: pgsql-hackers@hub.org, tgl@sss.pgh.pa.us
In-Reply-To: <20000125110635M.t-ishii@sra.co.jp>

Tatsuo Ishii <t-ishii@sra.co.jp> writes:

No. some EUC's (EUC_TW and EUC_JP) has three-byte or even four-byte
codes. But you said your database has been configured as EUC_CN. As
far as I know, it only uses 1 or 2 byte-code. Another thing I am
confused is that ' \217\210' is not a valid EUC_CN data at all. \217
(0x8f) specifies code set 3 which does not exist in EUC_CN. In this
case, it is assumed that the multi-byte word to be consisted of 3-byte
code in the current implementation of PostgreSQL.

It could be that one of our users had their input method set to produce
EUC_TW or Big5.

In short, the problem you have is caused by:
1) wrong data submitted into the table

Kind of hard to control that when data is submitted by random users on
the Internet.

I would recommend you delete the data since it's not correct anyway.
In the mean time I'm going to fix 2) so that it assumes data be
consisted of 2 bytes even if wrong data sequence is submitted
(needless to say, except ascii).
Do you want the backpatch for 6.5.3?

Very much so. Thank you.

-Michael

From bouncefilter Mon Jan 24 22:06:42 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA65170
for <pgsql-hackers@postgresql.org>;
Mon, 24 Jan 2000 22:06:28 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id WAA11301
for <pgsql-hackers@postgresql.org>;
Mon, 24 Jan 2000 22:06:05 -0500 (EST)
To: pgsql-hackers@postgresql.org
Subject: Sure enough, SI buffer overrun is broken
Date: Mon, 24 Jan 2000 22:06:05 -0500
Message-ID: <11298.948769565@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

I built the current sources with MAXNUMMESSAGES set to 32 in
src/include/storage/sinvaladt.h. The regular regress tests
run OK, with just a few NOTICEs about 'cache state reset'
and 'SI buffer overflow' inserted in the normal outputs
(as you'd expect, if SI overrun occurs).

However, the parallel tests crash spectacularly, with weird errors
and Assert() coredumps. Some of the unexpected messages in the
postmaster log are:

ERROR: Relation 0 does not exist
NOTICE: LockRelease: locktable lookup failed, no lock
TRAP: Failed Assertion("!(((file) > 0 && (file) < SizeVfdCache && VfdCache[file].fileName != ((void *)0))):", File: "fd.c", Line: 817)

!(((file) > 0 && (file) < SizeVfdCache && VfdCache[file].fileName != ((void *)0))) (0)
NOTICE: LockRelease: locktable lookup failed, no lock
TRAP: Failed Assertion("!(attnum <= 0 || (attnum - 1 <= tuple_type->natts - 1 && tuple_type->attrs[attnum - 1] != ((void *)0) && variable->vartype == tuple_type->attrs[attnum - 1]->atttypid)):", File: "execQual.c", Line: 283)

!(attnum <= 0 || (attnum - 1 <= tuple_type->natts - 1 && tuple_type->attrs[attnum - 1] != ((void *)0) && variable->vartype == tuple_type->attrs[attnum - 1]->atttypid)) (0) [Not a typewriter]
TRAP: Failed Assertion("!(((file) > 0 && (file) < SizeVfdCache && VfdCache[file].fileName != ((void *)0))):", File: "fd.c", Line: 817)

!(((file) > 0 && (file) < SizeVfdCache && VfdCache[file].fileName != ((void *)0))) (0) [Not a typewriter]

We have a problem.

I think Hiroshi was beating on this code recently --- Hiroshi,
do you recall anything you might have done that would affect
SI cache reset recovery?

regards, tom lane

INSERT INTO temp_sales
SELECT DISTINCT on key supplysources.supplysource,
incharges.supply, targets.target, incharges.saledate,
incharges.supplyunit, '', incharges.quantity, incharges.company,
incharges.costcntr, 'Replenish', incharges.price, '','','', 0,
text(supplysources.supplysource)||
text(incharges.supply)||
text(targets.target)||
text(incharges.saledate) as key
FROM supplysources, incharges, targets WHERE
supplysources.warehouse = incharges.warehouse AND
(targets.site,targets.area) = (incharges.site,incharges.area);

What happens is that a large charges file which is transferred to
a mainframe ERP application is first brought into PostgreSQL.
Depending upon certain race conditions, duplicate "sales" records
can appear in the data file. We use DISTINCT ON to pick (as you
point out) an arbitrary record when duplicates appear. I suppose
we could do a DELETE ... WHERE NOT EXISTS after the import. Using
DISTINCT ON just saves a step. I don't have any arguments beyond
the grounds that we're using it in existing code as a duplicate
record filter - :-(

Why not just throw a NOTICE and keep the feature.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Philip Warner wrote:

I don't feel overly strongly about this, but if I remember right you can
do some pretty cool things with this feature, provided you do define
some semantics clearly. Like I think you can find the first tuple
(given some ORDER BY clause) that fulfills some criteria. I think it is

SELECT DISTINCT ON name name, age ORDER BY age;

will get the youngest person. This might not be clearly specified now,
but
as long as it's useful, how about clearly defining it? I don't know that
there is an easy way of doing this in standard SQL.

I don't know about PGSQL, but in other systems, I use:

Select <whatever> from <wherever> order by age asc limit to 1 row;

I *think* the PGSQL syntax is:

Select <whatever> from <wherever> order by age asc limit 1;

I think what I really meant was...

SELECT DISTINCT ON firstname firstname, age ORDER BY age.

Which would find the youngest person called "fred", the youngest person
called "paul", the youngest person called "jim" etc etc. which your
limit example wouldn't do.

From bouncefilter Mon Jan 24 22:19:42 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA68575
for <pgsql-hackers@postgreSQL.org>;
Mon, 24 Jan 2000 22:18:54 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id WAA11347;
Mon, 24 Jan 2000 22:18:29 -0500 (EST)
To: Chris Bitmead <chris@bitmead.com>
cc: PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Happy column dropping
In-reply-to: <388D0F76.6C04FADA@bitmead.com>
References: <001001bf66d7$b531ba00$2801007e@tpf.co.jp>
<4891.948766352@sss.pgh.pa.us> <388D0F76.6C04FADA@bitmead.com>
Comments: In-reply-to Chris Bitmead <chris@bitmead.com>
message dated "Tue, 25 Jan 2000 13:50:30 +1100"
Date: Mon, 24 Jan 2000 22:18:29 -0500
Message-ID: <11344.948770309@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Chris Bitmead <chris@bitmead.com> writes:

Tom Lane wrote:

I was thinking about adding a "version number" to the pg_class entry
for a relation, and then having its actual filename look like

Well, if you are going to re-write the files in place on update, a
version number seems like overkill. All you need is version "a" and
"b". Every time you do a change it swaps from version RELATIONOID_a to
RELATIONOID_b ,

And what happens when I do two (or more) DROP COLUMNs within a single
transaction? Nope, you need an open-ended counter.

... Or you could just go
for RELATIONOID and RELATIONOID_new and do a rename (although I guess
you're trying to avoid that).

Precisely. If we can avoid renaming the files, then we aren't at the
mercy of the OS while moving from "uncommitted" to "committed" state;
the only thing that matters is marking the transaction committed in
pg_log, and that's as atomic as we can make it.

If there's no rename, the worst that can happen is that no-longer-needed
files get left around (if the backend dies between committing and
removing dead files, or if it dies after making the files but before
committing the transaction). We could arrange for VACUUM to remove such
files.

regards, tom lane

From bouncefilter Mon Jan 24 22:18:42 2000
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA68287
for <pgsql-hackers@postgreSQL.org>;
Mon, 24 Jan 2000 22:17:58 -0500 (EST) (envelope-from Inoue@tpf.co.jp)
Received: from cadzone ([126.0.1.40] (may be forged))
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with SMTP
id MAA06079; Tue, 25 Jan 2000 12:16:59 +0900
From: "Hiroshi Inoue" <Inoue@tpf.co.jp>
To: "Don Baccus" <dhogaza@pacifier.com>, "Tom Lane" <tgl@sss.pgh.pa.us>
Cc: "Peter Eisentraut" <peter_e@gmx.net>,
"PostgreSQL Development" <pgsql-hackers@postgreSQL.org>
Subject: RE: [HACKERS] Happy column dropping
Date: Tue, 25 Jan 2000 12:22:44 +0900
Message-ID: <001201bf66e3$7204f4c0$2801007e@tpf.co.jp>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
In-Reply-To: <3.0.1.32.20000124184137.01069490@mail.pacifier.com>
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
Importance: Normal

-----Original Message-----
From: Don Baccus [mailto:dhogaza@pacifier.com]

At 09:12 PM 1/24/00 -0500, Tom Lane wrote:

* VACUUM with rebuild or ADD/DROP COLUMN: write new data into
files with an incremented version number.

Just a reality check for my learning of the internals. Out of curiousity
I coincidently have spent the last hour looking to see how add column's
implemented. It doesn't appear to do anything other than the new
attribute
to the proper system table. heap_getattr() just returns null if you ask
for an attribute past the end of the tuple.

This would appear to be (at least one reason) why you can't add a
"not null"
constraint to a column you're adding to an existing relation, or set the
new column to some non-null default value.

Correct? (again, to see if my eyeballs and brain are working in synch
tonight)

Probably yes.
Adding NOT NULL constraints to new column needs to apply default
value to existent rows and this would need either implicit updation of
existent rows or to copy into different version of the relation file. .

Do other DBMSs allow such things ?
For example,in Oracle NOT NULL constraint could be specified for new
column only when there's no row in the table AFAIK.

I couldn't judge it's worth the work.

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp

From bouncefilter Mon Jan 24 22:36:49 2000
Received: from corvette.mascari.com (dhcp26142035.columbus.rr.com
[24.26.142.35]) by hub.org (8.9.3/8.9.3) with ESMTP id WAA76714
for <pgsql-hackers@postgreSQL.org>;
Mon, 24 Jan 2000 22:36:42 -0500 (EST)
(envelope-from mascarm@mascari.com)
Received: from mascari.com (ferrari.mascari.com [192.168.2.1])
by corvette.mascari.com (8.9.3/8.9.3) with ESMTP id WAA09477;
Mon, 24 Jan 2000 22:31:39 -0500
Message-ID: <388D1919.DC7D369F@mascari.com>
Date: Mon, 24 Jan 2000 22:31:37 -0500
From: Mike Mascari <mascarm@mascari.com>
Organization: Mascari Development Inc
X-Mailer: Mozilla 4.7 [en] (Win95; I)
X-Accept-Language: en
MIME-Version: 1.0
To: Hiroshi Inoue <Inoue@tpf.co.jp>
CC: Don Baccus <dhogaza@pacifier.com>, Tom Lane <tgl@sss.pgh.pa.us>,
Peter Eisentraut <peter_e@gmx.net>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Happy column dropping
References: <001201bf66e3$7204f4c0$2801007e@tpf.co.jp>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hiroshi Inoue wrote:

Correct? (again, to see if my eyeballs and brain are working in synch
tonight)

Probably yes.
Adding NOT NULL constraints to new column needs to apply default
value to existent rows and this would need either implicit updation of
existent rows or to copy into different version of the relation file. .

Do other DBMSs allow such things ?
For example,in Oracle NOT NULL constraint could be specified for new
column only when there's no row in the table AFAIK.

I couldn't judge it's worth the work.

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp

But the times when I've found it would be nice to use ALTER TABLE
ADD COLUMN with NOT NULL constraints is in development mode when
the schema is changing rapidly and there actually isn't any data
yet in the tables. Otherwise, to add a new NOT NULL column during
a development cycle, one has to drop and recreate the table, all
triggers, comments, etc. Its just a real pain. :-(

Mike Mascari

Chris Bitmead <chris@bitmead.com> writes:

Tom Lane wrote:

If I don't hear loud hollers very soon, I'm going to eliminate the
DISTINCT ON "feature" for 7.0. As previously discussed, this feature
is not standard SQL and has no clear semantic interpretation.

I don't feel overly strongly about this, but if I remember right you can
do some pretty cool things with this feature, provided you do define
some semantics clearly.

We did talk about that, but I didn't hear any strong support for doing
it, as opposed to pulling the feature completely... in particular,
I didn't hear anyone volunteering to do the work...

as long as it's useful, how about clearly defining it? I don't know that
there is an easy way of doing this in standard SQL. I don't see any
problems with useful extensions to SQL.

The only reason it came to my notice in the first place was people
posting questions asking why they weren't getting the results they
expected from it (whatever the heck those were; they weren't what you
actually get from the current implementation, anyway). The problem
with a poorly-specified nonstandard feature is support costs: you
have to document it, answer questions about it, keep it working, etc.
In this case we'd also have to define how it should work and alter
the existing code to produce reasonable and predictable results. The
existing code is not merely unpredictable, it is definitely broken.
For example:

regression=# select q1,q2 from int8_tbl;
q1 | q2
------------------+-------------------
123 | 456
123 | 4567890123456789
4567890123456789 | 123
4567890123456789 | 4567890123456789
4567890123456789 | -4567890123456789
(5 rows)

regression=# select distinct on q1 q1,q2 from int8_tbl;
q1 | q2
------------------+-----
123 | 456
4567890123456789 | 123
(2 rows)

-- OK so far, but:

regression=# select distinct on q1 q1,q2 from int8_tbl order by q2;
q1 | q2
------------------+-------------------
4567890123456789 | -4567890123456789
123 | 456
4567890123456789 | 4567890123456789
(3 rows)

-- which is not "distinct on q1" by my notions...

In short, it's not clear to me that supporting DISTINCT ON is a good use
of our limited resources. I'm willing to pull it out, but not to fix it.
Does someone else want to take responsibility for it?

regards, tom lane

In short, it's not clear to me that supporting DISTINCT ON is a good use
of our limited resources. I'm willing to pull it out, but not to fix it.
Does someone else want to take responsibility for it?

OK, we can disable it and put it on the TODO list. It is already there.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Mon Jan 24 22:58:43 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA80619
for <pgsql-hackers@postgreSQL.org>;
Mon, 24 Jan 2000 22:58:33 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id WAA11576;
Mon, 24 Jan 2000 22:57:12 -0500 (EST)
To: Don Baccus <dhogaza@pacifier.com>
cc: "Hiroshi Inoue" <Inoue@tpf.co.jp>, "Peter Eisentraut" <peter_e@gmx.net>,
"PostgreSQL Development" <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Happy column dropping
In-reply-to: <3.0.1.32.20000124184137.01069490@mail.pacifier.com>
References: <001001bf66d7$b531ba00$2801007e@tpf.co.jp>
<001001bf66d7$b531ba00$2801007e@tpf.co.jp>
<3.0.1.32.20000124184137.01069490@mail.pacifier.com>
Comments: In-reply-to Don Baccus <dhogaza@pacifier.com>
message dated "Mon, 24 Jan 2000 18:41:37 -0800"
Date: Mon, 24 Jan 2000 22:57:12 -0500
Message-ID: <11573.948772632@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Don Baccus <dhogaza@pacifier.com> writes:

Just a reality check for my learning of the internals. Out of curiousity
I coincidently have spent the last hour looking to see how add column's
implemented. It doesn't appear to do anything other than the new attribute
to the proper system table. heap_getattr() just returns null if you ask
for an attribute past the end of the tuple.

This would appear to be (at least one reason) why you can't add a "not null"
constraint to a column you're adding to an existing relation, or set the
new column to some non-null default value.

Correct? (again, to see if my eyeballs and brain are working in synch
tonight)

Yup, that's about the size of it. ADD COLUMN doesn't actually touch the
table itself, so it can only add a column that's initially all NULLs.
And even this depends on some uncomfortable assumptions about the
robustness of heap_getattr(). I have always wondered whether it works
if you ADD COLUMN a 33'rd column (or anything that is just past the
next padding boundary for the null-values bitmap).

Another problem with it is seen when you do a recursive ADD COLUMN in
an inheritance tree.  The added column has the first free column number
in each table, which generally means that it has different numbers in
the children than in the parent.  There are some kluges to make this
sort-of-work for simple cases, but a lot of stuff fails unpleasantly
--- Chris Bitmead can show you some scars from that, IIRC.

Does your comment imply that it's planned to change this, i.e. actually
add the new column to each tuple in the relation rather than use the
existing, somewhat elegant hack?

That's what I would like to see: all the children should have the
same column numbers for all columns that they inherit from the parent.

(Now, this would mean not only physically altering the tuples of
the children, but also renumbering their added columns, which has
implications on stored rules and triggers and so forth. It'd be
painful, no doubt about it. Still, I'd rather pay the price in the
seldom-used ADD COLUMN case than try to deal with out-of-sync column
numbers in many other, more commonly exercised, code paths.)

regards, tom lane

From bouncefilter Mon Jan 24 23:02:57 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA81116
for <pgsql-hackers@postgreSQL.org>;
Mon, 24 Jan 2000 23:00:18 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id XAA11599;
Mon, 24 Jan 2000 23:00:04 -0500 (EST)
To: Bruce Momjian <pgman@candle.pha.pa.us>
cc: Hiroshi Inoue <Inoue@tpf.co.jp>, Peter Eisentraut <peter_e@gmx.net>,
The Hermit Hacker <scrappy@hub.org>,
PostgreSQL Development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Happy column dropping
In-reply-to: <200001250251.VAA21667@candle.pha.pa.us>
References: <200001250251.VAA21667@candle.pha.pa.us>
Comments: In-reply-to Bruce Momjian <pgman@candle.pha.pa.us>
message dated "Mon, 24 Jan 2000 21:51:51 -0500"
Date: Mon, 24 Jan 2000 23:00:04 -0500
Message-ID: <11596.948772804@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Bruce Momjian <pgman@candle.pha.pa.us> writes:

New file name will not be just oid. Too hard to administer.

But if it contains the table name, you still have the problem we'd
like to avoid: renaming the file(s) is a critical part of committing
a table-renaming transaction.

The debugging and administrative convenience of using table names as
filenames is exactly why we've resisted this for so long. But I am
starting to think that we have to give that up. It's nice, but
having robust DDL commands is nicer.

regards, tom lane

Hi,

This morning when I was inserting data to a table, after 15 minutes
running,
I received this

ERROR: cannot flush block 9782 of tbs_billing_record to stable store.

What does it mean?

Regards,
Chai

From bouncefilter Mon Jan 24 23:06:43 2000
Received: from acheron.rime.com.au (root@albatr.lnk.telstra.net
[139.130.54.222]) by hub.org (8.9.3/8.9.3) with ESMTP id XAA85919
for <pgsql-hackers@postgreSQL.org>;
Mon, 24 Jan 2000 23:05:48 -0500 (EST)
(envelope-from pjw@rhyme.com.au)
Received: from oberon (Oberon.rime.com.au [203.8.195.100])
by acheron.rime.com.au (8.9.3/8.9.3) with SMTP id PAA06255;
Tue, 25 Jan 2000 15:04:33 +1100
Message-Id: <3.0.5.32.20000125150557.036121c0@mail.rhyme.com.au>
X-Sender: pjw@mail.rhyme.com.au
X-Mailer: QUALCOMM Windows Eudora Pro Version 3.0.5 (32)
Date: Tue, 25 Jan 2000 15:05:57 +1100
To: Chris Bitmead <chris@bitmead.com>, pgsql-hackers@postgreSQL.org
From: Philip Warner <pjw@rhyme.com.au>
Subject: Re: [HACKERS] DISTINCT ON: speak now or forever hold your peace
In-Reply-To: <388D13FE.4E75CA05@bitmead.com>
References: <2295.948765015@sss.pgh.pa.us>
<3.0.5.32.20000125140349.035768d0@mail.rhyme.com.au>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 14:09 25/01/00 +1100, Chris Bitmead wrote:

Philip Warner wrote:

I don't feel overly strongly about this, but if I remember right you can
do some pretty cool things with this feature, provided you do define
some semantics clearly. Like I think you can find the first tuple
(given some ORDER BY clause) that fulfills some criteria. I think it is

SELECT DISTINCT ON name name, age ORDER BY age;

will get the youngest person. This might not be clearly specified now,
but
as long as it's useful, how about clearly defining it? I don't know that
there is an easy way of doing this in standard SQL.

I don't know about PGSQL, but in other systems, I use:

Select <whatever> from <wherever> order by age asc limit to 1 row;

I *think* the PGSQL syntax is:

Select <whatever> from <wherever> order by age asc limit 1;

I think what I really meant was...

SELECT DISTINCT ON firstname firstname, age ORDER BY age.

Which would find the youngest person called "fred", the youngest person
called "paul", the youngest person called "jim" etc etc. which your
limit example wouldn't do.

Just a thought, but would I be right in saying that this could be easily
done with the addition of a new aggregate function 'FIRST', which simply
returns the first value sent to it? Since the aggregates operate a row at a
time, you are guaranteed a consistent set of values, I think.

eg.

SELECT firstname, FIRST(age) ORDER BY age.

Just an idea, but it seems like a cute solution for the more general problem.

----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.C.N. 008 659 498) | /(@) ______---_
Tel: +61-03-5367 7422 | _________ \
Fax: +61-03-5367 7430 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/

From bouncefilter Mon Jan 24 23:37:44 2000
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67] (may
be forged)) by hub.org (8.9.3/8.9.3) with ESMTP id XAA96612
for <pgsql-hackers@postgreSQL.org>;
Mon, 24 Jan 2000 23:37:38 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
XAA25125;
Mon, 24 Jan 2000 23:35:31 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001250435.XAA25125@candle.pha.pa.us>
Subject: Re: [HACKERS] Happy column dropping
In-Reply-To: <11596.948772804@sss.pgh.pa.us> from Tom Lane at "Jan 24,
2000 11:00:04 pm"
To: Tom Lane <tgl@sss.pgh.pa.us>
Date: Mon, 24 Jan 2000 23:35:31 -0500 (EST)
CC: Hiroshi Inoue <Inoue@tpf.co.jp>, Peter Eisentraut <peter_e@gmx.net>,
The Hermit Hacker <scrappy@hub.org>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

The debugging and administrative convenience of using table names as
filenames is exactly why we've resisted this for so long. But I am
starting to think that we have to give that up. It's nice, but
having robust DDL commands is nicer.

Because you want to have multiple versions of a table with the same name
and table oid. We don't have to have the oid equal the oid of the
table. Just use any oid to append to the table name for versioning.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Tue Jan 25 00:06:43 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA10616
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 00:05:49 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id VAA03109;
Mon, 24 Jan 2000 21:05:11 -0800 (PST)
Message-Id: <3.0.1.32.20000124204431.007b3940@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Mon, 24 Jan 2000 20:44:31 -0800
To: Bruce Momjian <pgman@candle.pha.pa.us>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: [HACKERS] Happy column dropping
Cc: Peter Eisentraut <peter_e@gmx.net>, Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
In-Reply-To: <200001250248.VAA21476@candle.pha.pa.us>
References: <3.0.1.32.20000124155337.010834f0@mail.pacifier.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 09:48 PM 1/24/00 -0500, Bruce Momjian wrote:

Maybe that is true. Having phantom column all over the code is going to
be a mess, and hardly worth it considering how many developers there are
and how many _big_ items still have to be done.

Works for Oracle...I guess Postgres is just an obviously more robust,
faster, more reliable, and altogether more brilliant RDBMS than this
loser commercial DB? It's really hard to understand why Postgres has
had such a poor reputation over the years when faced with such facts,
isn't it?

Messing up code for one feature is rarely worth it.

Dropping constraints on a table just because you drop a column is
just butt-ugly.

Sorry if you disagree.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Mon Jan 24 23:46:43 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA03296
for <pgsql-hackers@postgreSQL.org>;
Mon, 24 Jan 2000 23:46:41 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id XAA11846;
Mon, 24 Jan 2000 23:46:29 -0500 (EST)
To: Bruce Momjian <pgman@candle.pha.pa.us>
cc: Hiroshi Inoue <Inoue@tpf.co.jp>, Peter Eisentraut <peter_e@gmx.net>,
The Hermit Hacker <scrappy@hub.org>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Happy column dropping
In-reply-to: <200001250435.XAA25125@candle.pha.pa.us>
References: <200001250435.XAA25125@candle.pha.pa.us>
Comments: In-reply-to Bruce Momjian <pgman@candle.pha.pa.us>
message dated "Mon, 24 Jan 2000 23:35:31 -0500"
Date: Mon, 24 Jan 2000 23:46:28 -0500
Message-ID: <11843.948775588@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Because you want to have multiple versions of a table with the same name
and table oid. We don't have to have the oid equal the oid of the
table. Just use any oid to append to the table name for versioning.

That'd do for versioning, but what about a plain table-renaming
operation? We aren't making a new file in that case, and we still
want to avoid rename().

regards, tom lane

From bouncefilter Tue Jan 25 00:06:44 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA10631
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 00:05:56 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id VAA03112;
Mon, 24 Jan 2000 21:05:12 -0800 (PST)
Message-Id: <3.0.1.32.20000124205003.00f6fac0@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Mon, 24 Jan 2000 20:50:03 -0800
To: "Hiroshi Inoue" <Inoue@tpf.co.jp>, "Tom Lane" <tgl@sss.pgh.pa.us>
From: Don Baccus <dhogaza@pacifier.com>
Subject: RE: [HACKERS] Happy column dropping
Cc: "Peter Eisentraut" <peter_e@gmx.net>,
"PostgreSQL Development" <pgsql-hackers@postgreSQL.org>
In-Reply-To: <001201bf66e3$7204f4c0$2801007e@tpf.co.jp>
References: <3.0.1.32.20000124184137.01069490@mail.pacifier.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 12:22 PM 1/25/00 +0900, Hiroshi Inoue wrote:

-----Original Message-----
From: Don Baccus [mailto:dhogaza@pacifier.com]

This would appear to be (at least one reason) why you can't add a
"not null"
constraint to a column you're adding to an existing relation, or set the
new column to some non-null default value.

Correct? (again, to see if my eyeballs and brain are working in synch
tonight)

Probably yes.
Adding NOT NULL constraints to new column needs to apply default
value to existent rows and this would need either implicit updation of
existent rows or to copy into different version of the relation file. .

Right...and my reading says neither happens.

(and I'm not arguing that either should, I posed my question as a reality
check as to whether or not a pedantic, lame-brain like myself is actually
capable of reading and understanding the code)

Do other DBMSs allow such things ?

Don't know, haven't researched it. I think it's a reasonable restriction,
though.

For example,in Oracle NOT NULL constraint could be specified for new
column only when there's no row in the table AFAIK.

I think that's right.

I couldn't judge it's worth the work.

Gosh, and I hope you didn't think I was suggesting it was. I was simply
responding to Tom's saying that "ADD/DROP column" would require a new
file for the updated relation, and asking if the current situation was
slated for change (because the current situation doesn't require that
the relation be copied to a new file after a "alter table add column",
if I read the code right).

That's all...I just wasn't able to relate Tom's point to how things
appear to work, and wondered if I was missing something crucial in my
reading of the code.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Mon Jan 24 23:59:44 2000
Received: from sraigw.sra.co.jp (sraigw.sra.co.jp [202.32.10.2])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA05606
for <pgsql-hackers@hub.org>; Mon, 24 Jan 2000 23:58:58 -0500 (EST)
(envelope-from t-ishii@sra.co.jp)
Received: from sranhk.sra.co.jp (sranhk [133.137.36.134])
by sraigw.sra.co.jp (8.8.7/3.7W-sraigw) with ESMTP id NAA05885;
Tue, 25 Jan 2000 13:57:39 +0900 (JST)
Received: from localhost (IDENT:t-ishii@srapc968-yotsuya [133.137.36.133])
by sranhk.sra.co.jp (8.9.3/3.7W-srambox) with ESMTP id NAA17832;
Tue, 25 Jan 2000 13:57:38 +0900
To: robinson@netrinsics.com
Cc: t-ishii@sra.co.jp, pgsql-hackers@hub.org, tgl@sss.pgh.pa.us
Subject: Re: [HACKERS] fatal copy in/out error (6.5.3)
In-Reply-To: <200001250304.LAA39308@netrinsics.com>
References: <20000125110635M.t-ishii@sra.co.jp>
<200001250304.LAA39308@netrinsics.com>
X-Mailer: Mew version 1.94 on Emacs 19.34 / Mule 2.3 (SUETSUMUHANA)
Mime-Version: 1.0
Content-Type: Text/Plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-Id: <20000125140004W.t-ishii@sra.co.jp>
Date: Tue, 25 Jan 2000 14:00:04 +0900
From: Tatsuo Ishii <t-ishii@sra.co.jp>
X-Dispatcher: imput version 990905(IM130)
Lines: 49

It could be that one of our users had their input method set to produce
EUC_TW or Big5.

In short, the problem you have is caused by:
1) wrong data submitted into the table

Kind of hard to control that when data is submitted by random users on
the Internet.

Yes, it's not a PostgreSQL's business but is a really big problem in
the real world. Maybe some HTML gurus might have good suggestions on
these issues (something like using a language tag?)

I would recommend you delete the data since it's not correct anyway.
In the mean time I'm going to fix 2) so that it assumes data be
consisted of 2 bytes even if wrong data sequence is submitted
(needless to say, except ascii).
Do you want the backpatch for 6.5.3?

Very much so. Thank you.

Here it is. With this patch, copy out should be happy even with the
wrong data. I'm not sure if it could be displayed correctly, though.
--
Tatsuo Ishii
------------------------------- cut here ------------------------------------
*** postgresql-6.5.3/src/backend/utils/mb/wchar.c.orig	Tue Jan 25 13:37:21 2000
--- postgresql-6.5.3/src/backend/utils/mb/wchar.c	Tue Jan 25 13:37:33 2000
***************
*** 166,176 ****
  {
  	int			len;

! 	if (*s == SS2)
! 		len = 3;
! 	else if (*s == SS3)
! 		len = 3;
! 	else if (*s & 0x80)
  		len = 2;
  	else
  		len = 1;
--- 166,172 ----
  {
  	int			len;

! if (*s & 0x80)
len = 2;
else
len = 1;

From bouncefilter Tue Jan 25 00:02:44 2000
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67] (may
be forged)) by hub.org (8.9.3/8.9.3) with ESMTP id AAA07829
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 00:01:56 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
AAA26167;
Tue, 25 Jan 2000 00:00:57 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001250500.AAA26167@candle.pha.pa.us>
Subject: Re: [HACKERS] Happy column dropping
In-Reply-To: <11843.948775588@sss.pgh.pa.us> from Tom Lane at "Jan 24,
2000 11:46:28 pm"
To: Tom Lane <tgl@sss.pgh.pa.us>
Date: Tue, 25 Jan 2000 00:00:56 -0500 (EST)
CC: Hiroshi Inoue <Inoue@tpf.co.jp>, Peter Eisentraut <peter_e@gmx.net>,
The Hermit Hacker <scrappy@hub.org>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Because you want to have multiple versions of a table with the same name
and table oid. We don't have to have the oid equal the oid of the
table. Just use any oid to append to the table name for versioning.

That'd do for versioning, but what about a plain table-renaming
operation? We aren't making a new file in that case, and we still
want to avoid rename().

I see. Hard to see how the advantage of fixing that would be worth
losing table names as files.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Tue Jan 25 00:02:44 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA07708
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 00:01:48 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id AAA11882;
Tue, 25 Jan 2000 00:01:10 -0500 (EST)
To: Philip Warner <pjw@rhyme.com.au>
cc: Chris Bitmead <chris@bitmead.com>, pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] DISTINCT ON: speak now or forever hold your peace
In-reply-to: <3.0.5.32.20000125150557.036121c0@mail.rhyme.com.au>
References: <2295.948765015@sss.pgh.pa.us>
<3.0.5.32.20000125140349.035768d0@mail.rhyme.com.au>
<3.0.5.32.20000125150557.036121c0@mail.rhyme.com.au>
Comments: In-reply-to Philip Warner <pjw@rhyme.com.au>
message dated "Tue, 25 Jan 2000 15:05:57 +1100"
Date: Tue, 25 Jan 2000 00:01:10 -0500
Message-ID: <11879.948776470@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Philip Warner <pjw@rhyme.com.au> writes:

Just a thought, but would I be right in saying that this could be easily
done with the addition of a new aggregate function 'FIRST', which simply
returns the first value sent to it? Since the aggregates operate a row at
a time, you are guaranteed a consistent set of values, I think.

No, because the system doesn't guarantee to deliver tuples to the
aggregate in any particular order. The physical order of tuples
on the disk will depend on the order in which they were last updated
(plus random perturbations introduced by VACUUM); and the order in
which they are processed by a query will depend on the query plan
chosen by the planner (sequential scan vs. indexscan, etc).
SQL in general doesn't believe that tuple ordering has any semantic
significance --- you can ask for ORDER BY, but that's only honored
at the end stage of SELECT result delivery, not necessarily anywhere
in the bowels of a query.

Also, if you are going to use an aggregate, ISTM you might as well
use one like MIN() or MAX() and get a fully predictable result.
I suppose a FIRST() aggregate would save a few cycles by not running
comparisons, but I bet it'd be tough to notice any difference.

regards, tom lane

From bouncefilter Tue Jan 25 00:06:50 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA10718
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 00:06:21 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id VAA03119;
Mon, 24 Jan 2000 21:05:13 -0800 (PST)
Message-Id: <3.0.1.32.20000124210143.0106b780@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Mon, 24 Jan 2000 21:01:43 -0800
To: Tom Lane <tgl@sss.pgh.pa.us>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: [HACKERS] Happy column dropping
Cc: "Hiroshi Inoue" <Inoue@tpf.co.jp>, "Peter Eisentraut" <peter_e@gmx.net>,
"PostgreSQL Development" <pgsql-hackers@postgreSQL.org>
In-Reply-To: <11573.948772632@sss.pgh.pa.us>
References: <3.0.1.32.20000124184137.01069490@mail.pacifier.com>
<001001bf66d7$b531ba00$2801007e@tpf.co.jp>
<001001bf66d7$b531ba00$2801007e@tpf.co.jp>
<3.0.1.32.20000124184137.01069490@mail.pacifier.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 10:57 PM 1/24/00 -0500, Tom Lane wrote:

Don Baccus <dhogaza@pacifier.com> writes:

Correct? (again, to see if my eyeballs and brain are working in synch
tonight)

Yup, that's about the size of it.

Phew! I'm sane!

ADD COLUMN doesn't actually touch the
table itself, so it can only add a column that's initially all NULLs.
And even this depends on some uncomfortable assumptions about the
robustness of heap_getattr().

Yeah...that was my thought...you did notice I included the word
"kludge", yeah?

I have always wondered whether it works
if you ADD COLUMN a 33'rd column (or anything that is just past the
next padding boundary for the null-values bitmap).

Way beyond my knowledge of things today, but if you REALLY CARE, I
could probably figure it out :)

Another problem with it is seen when you do a recursive ADD COLUMN in
an inheritance tree.  The added column has the first free column number
in each table, which generally means that it has different numbers in
the children than in the parent.  There are some kluges to make this
sort-of-work for simple cases, but a lot of stuff fails unpleasantly
--- Chris Bitmead can show you some scars from that, IIRC.

I think this probably explains some of the editorial comments about the
code. There seem to be some added by "XXX" - is that person part of the
current clan of developers? Comments like "This code is a crock because..."

Does your comment imply that it's planned to change this, i.e. actually
add the new column to each tuple in the relation rather than use the
existing, somewhat elegant hack?

That's what I would like to see: all the children should have the
same column numbers for all columns that they inherit from the parent.

(Now, this would mean not only physically altering the tuples of
the children, but also renumbering their added columns, which has
implications on stored rules and triggers and so forth. It'd be
painful, no doubt about it. Still, I'd rather pay the price in the
seldom-used ADD COLUMN case than try to deal with out-of-sync column
numbers in many other, more commonly exercised, code paths.)

Yeah...though I don't know the code well enough to comment on the pain,
in the wider, more general sense I totally agree. I don't think
efficiency of ADD COLUMN (or DROP for that matter) should be of paramount
concern, even though Oracle gives a separate quick DROP option similar to
what's being discussed as an alternative to COPY/RENAME.

My discomfort with the latter has more to do with the dropping of
constraints (and I've read enough of the code to see that rebuilding
them might be a worthy subject for a Hitchcock film)

Cool...please be patient with me as I work my way through this stuff
in my spare time.

And, Peter, if you're reading this, please be assured that my criticisms
are meant only in the context of wanting to see this open source database
become an Oracle (or other) slayer in at least a segment of the marketplace.
On its merits, not the price differential. If I'm harsh on things that I
feel don't quite meet the mark, please don't take it personally.

And if I'm proven wrong, I'm always more than willing to admit it far
and wide, and to apologize in public.

I just want to see debate over the "howtos" based on merit, that's it.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Tue Jan 25 00:09:44 2000
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67] (may
be forged)) by hub.org (8.9.3/8.9.3) with ESMTP id AAA11354
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 00:08:48 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
AAA26401;
Tue, 25 Jan 2000 00:08:01 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001250508.AAA26401@candle.pha.pa.us>
Subject: Re: [HACKERS] Happy column dropping
In-Reply-To: <3.0.1.32.20000124204431.007b3940@mail.pacifier.com> from Don
Baccus at "Jan 24, 2000 08:44:31 pm"
To: Don Baccus <dhogaza@pacifier.com>
Date: Tue, 25 Jan 2000 00:08:01 -0500 (EST)
CC: Peter Eisentraut <peter_e@gmx.net>, Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

At 09:48 PM 1/24/00 -0500, Bruce Momjian wrote:

Maybe that is true. Having phantom column all over the code is going to
be a mess, and hardly worth it considering how many developers there are
and how many _big_ items still have to be done.

Works for Oracle...I guess Postgres is just an obviously more robust,
faster, more reliable, and altogether more brilliant RDBMS than this
loser commercial DB? It's really hard to understand why Postgres has
had such a poor reputation over the years when faced with such facts,
isn't it?

Messing up code for one feature is rarely worth it.

Dropping constraints on a table just because you drop a column is
just butt-ugly.

We aren't charging 100k either. We do what we can, and spend time where
it is most valuable.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Tue Jan 25 00:20:43 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA14356
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 00:20:14 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id VAA07950;
Mon, 24 Jan 2000 21:19:39 -0800 (PST)
Message-Id: <3.0.1.32.20000124211728.00f728b0@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Mon, 24 Jan 2000 21:17:28 -0800
To: Bruce Momjian <pgman@candle.pha.pa.us>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: [HACKERS] Happy column dropping
Cc: Peter Eisentraut <peter_e@gmx.net>, Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
In-Reply-To: <200001250508.AAA26401@candle.pha.pa.us>
References: <3.0.1.32.20000124204431.007b3940@mail.pacifier.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 12:08 AM 1/25/00 -0500, Bruce Momjian wrote:

Dropping constraints on a table just because you drop a column is
just butt-ugly.

We aren't charging 100k either. We do what we can, and spend time where
it is most valuable.

Perhaps I've misunderstood, then...I'd thought part of the goal was to
compete with the 100K model without charging 100K...

If not, OK.

(BTW, Oracle for development is free and you can actually cruise with
that for some time. A five-user license for non-internet use costs
$1450 for Linux)

And further BTW...some folks recently went through a little handwringing
upon hearing the Interbase will be released in free, open source form.

And there's a recent, though not current, Sybase version available for
Linux users for free, too...

If the point's not to be competitive with other free or low cost options,
is there any point at all?

Is it time for me to apologize for having high standards?

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Tue Jan 25 00:23:43 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA15205
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 00:23:05 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id VAA08890;
Mon, 24 Jan 2000 21:22:28 -0800 (PST)
Message-Id: <3.0.1.32.20000124212022.00f70720@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Mon, 24 Jan 2000 21:20:22 -0800
To: Tom Lane <tgl@sss.pgh.pa.us>, Philip Warner <pjw@rhyme.com.au>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: [HACKERS] DISTINCT ON: speak now or forever hold your
peace
Cc: Chris Bitmead <chris@bitmead.com>, pgsql-hackers@postgreSQL.org
In-Reply-To: <11879.948776470@sss.pgh.pa.us>
References: <3.0.5.32.20000125150557.036121c0@mail.rhyme.com.au>
<2295.948765015@sss.pgh.pa.us>
<3.0.5.32.20000125140349.035768d0@mail.rhyme.com.au>
<3.0.5.32.20000125150557.036121c0@mail.rhyme.com.au>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 12:01 AM 1/25/00 -0500, Tom Lane wrote:

Philip Warner <pjw@rhyme.com.au> writes:

Just a thought, but would I be right in saying that this could be easily
done with the addition of a new aggregate function 'FIRST', which simply
returns the first value sent to it? Since the aggregates operate a row at
a time, you are guaranteed a consistent set of values, I think.

No, because the system doesn't guarantee to deliver tuples to the
aggregate in any particular order.

In fact, this is a KEY notion behind RDBMS systems...queries return an
unordered set, pure and simple, unless you supply an "order" clause
(though "group by" appears to be implemented by ordering in various
RDBMS systems, that's an efficiency hack not a given AFAIK).

SQL in general doesn't believe that tuple ordering has any semantic
significance --- you can ask for ORDER BY, but that's only honored
at the end stage of SELECT result delivery, not necessarily anywhere
in the bowels of a query.

Yep...

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Tue Jan 25 00:30:43 2000
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67] (may
be forged)) by hub.org (8.9.3/8.9.3) with ESMTP id AAA16844
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 00:30:11 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
AAA28416;
Tue, 25 Jan 2000 00:29:20 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001250529.AAA28416@candle.pha.pa.us>
Subject: Re: [HACKERS] Happy column dropping
In-Reply-To: <3.0.1.32.20000124211728.00f728b0@mail.pacifier.com> from Don
Baccus at "Jan 24, 2000 09:17:28 pm"
To: Don Baccus <dhogaza@pacifier.com>
Date: Tue, 25 Jan 2000 00:29:20 -0500 (EST)
CC: Peter Eisentraut <peter_e@gmx.net>, Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

If the point's not to be competitive with other free or low cost options,
is there any point at all?

Is it time for me to apologize for having high standards?

We have to be realistic. It is no good to have one command that has 100
fancy capabilities while 100 other commands are broken or are more
important.

Doesn't mean we don't do a great job, but that sometimes it is not worth
it considering the other things we can be doing.

If you are suggesting we don't follow that plan, I have to disagree.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Tue Jan 25 00:30:44 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA16652
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 00:29:51 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id AAA12018;
Tue, 25 Jan 2000 00:29:26 -0500 (EST)
To: Don Baccus <dhogaza@pacifier.com>
cc: "Hiroshi Inoue" <Inoue@tpf.co.jp>, "Peter Eisentraut" <peter_e@gmx.net>,
"PostgreSQL Development" <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Happy column dropping
In-reply-to: <3.0.1.32.20000124210143.0106b780@mail.pacifier.com>
References: <3.0.1.32.20000124184137.01069490@mail.pacifier.com>
<001001bf66d7$b531ba00$2801007e@tpf.co.jp>
<001001bf66d7$b531ba00$2801007e@tpf.co.jp>
<3.0.1.32.20000124184137.01069490@mail.pacifier.com>
<3.0.1.32.20000124210143.0106b780@mail.pacifier.com>
Comments: In-reply-to Don Baccus <dhogaza@pacifier.com>
message dated "Mon, 24 Jan 2000 21:01:43 -0800"
Date: Tue, 25 Jan 2000 00:29:26 -0500
Message-ID: <12015.948778166@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Don Baccus <dhogaza@pacifier.com> writes:

And, Peter, if you're reading this, please be assured that my criticisms
are meant only in the context of wanting to see this open source database
become an Oracle (or other) slayer in at least a segment of the marketplace.
On its merits, not the price differential. If I'm harsh on things that I
feel don't quite meet the mark, please don't take it personally.

Not pretending to speak for Peter, but --- I don't think there's anybody
here who hasn't got the highest standards in mind as an end goal. Yet
we have many miles to go, and we seldom agree on whether a particular
problem is the most critical next thing to work on. Each of us is doing
what comes to hand to be done. Sometimes we can fix a problem properly,
and sometimes we can only put a finger in the dike, or apply triage and
say "that's going to have to go unfixed a while longer".

Yup, it's a little disorganized, but that's both the curse and the
blessing of open-source development. Anybody who's *really* annoyed
by a particular problem is welcome to come and work on it.

regards, tom lane

From bouncefilter Tue Jan 25 00:39:44 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA21974
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 00:39:24 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id AAA12049;
Tue, 25 Jan 2000 00:39:14 -0500 (EST)
To: Don Baccus <dhogaza@pacifier.com>
cc: "PostgreSQL Development" <pgsql-hackers@postgreSQL.org>
Subject: Oh btw, about XXX
In-reply-to: <3.0.1.32.20000124210143.0106b780@mail.pacifier.com>
References: <3.0.1.32.20000124184137.01069490@mail.pacifier.com>
<001001bf66d7$b531ba00$2801007e@tpf.co.jp>
<001001bf66d7$b531ba00$2801007e@tpf.co.jp>
<3.0.1.32.20000124184137.01069490@mail.pacifier.com>
<3.0.1.32.20000124210143.0106b780@mail.pacifier.com>
Comments: In-reply-to Don Baccus <dhogaza@pacifier.com>
message dated "Mon, 24 Jan 2000 21:01:43 -0800"
Date: Tue, 25 Jan 2000 00:39:14 -0500
Message-ID: <12046.948778754@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Don Baccus <dhogaza@pacifier.com> writes:

I think this probably explains some of the editorial comments about the
code. There seem to be some added by "XXX" - is that person part of the
current clan of developers? Comments like "This code is a crock because..."

XXX isn't a signature, it's a conventional marker for a "Hey! This is
broken! FIX ME!" kind of comment. I think the original idea was you
could do "grep XXX *.c" if you were idly looking for problems to work
on. Some projects use "FIXME" in the same way.

The only signatures I've seen in the Postgres code are initials at the
ends of comments. XXX usually goes at the front of a gripe.

regards, tom lane

From bouncefilter Tue Jan 25 01:00:44 2000
Received: from acheron.rime.com.au (root@albatr.lnk.telstra.net
[139.130.54.222]) by hub.org (8.9.3/8.9.3) with ESMTP id BAA25996
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 01:00:32 -0500 (EST)
(envelope-from pjw@rhyme.com.au)
Received: from oberon (Oberon.rime.com.au [203.8.195.100])
by acheron.rime.com.au (8.9.3/8.9.3) with SMTP id QAA07476;
Tue, 25 Jan 2000 16:59:02 +1100
Message-Id: <3.0.5.32.20000125170026.0357d8f0@mail.rhyme.com.au>
X-Sender: pjw@mail.rhyme.com.au
X-Mailer: QUALCOMM Windows Eudora Pro Version 3.0.5 (32)
Date: Tue, 25 Jan 2000 17:00:26 +1100
To: Don Baccus <dhogaza@pacifier.com>, Tom Lane <tgl@sss.pgh.pa.us>
From: Philip Warner <pjw@rhyme.com.au>
Subject: Re: [HACKERS] DISTINCT ON: speak now or forever hold your
peace
Cc: Chris Bitmead <chris@bitmead.com>, pgsql-hackers@postgreSQL.org
In-Reply-To: <3.0.1.32.20000124212022.00f70720@mail.pacifier.com>
References: <11879.948776470@sss.pgh.pa.us>
<3.0.5.32.20000125150557.036121c0@mail.rhyme.com.au>
<2295.948765015@sss.pgh.pa.us>
<3.0.5.32.20000125140349.035768d0@mail.rhyme.com.au>
<3.0.5.32.20000125150557.036121c0@mail.rhyme.com.au>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 21:20 24/01/00 -0800, Don Baccus wrote:

At 12:01 AM 1/25/00 -0500, Tom Lane wrote:

Philip Warner <pjw@rhyme.com.au> writes:

Just a thought, but would I be right in saying that this could be easily
done with the addition of a new aggregate function 'FIRST', which simply
returns the first value sent to it? Since the aggregates operate a row at
a time, you are guaranteed a consistent set of values, I think.

No, because the system doesn't guarantee to deliver tuples to the
aggregate in any particular order.

In fact, this is a KEY notion behind RDBMS systems...queries return an
unordered set, pure and simple, unless you supply an "order" clause
(though "group by" appears to be implemented by ordering in various
RDBMS systems, that's an efficiency hack not a given AFAIK).

That's why there was an 'order by' in the query (there should have also
been a 'group by', to make it valid SQL). I had (naievely) assumed that the
aggregates would be passed the rows in sorted order.

SQL in general doesn't believe that tuple ordering has any semantic
significance --- you can ask for ORDER BY, but that's only honored
at the end stage of SELECT result delivery, not necessarily anywhere
in the bowels of a query.

Yep...

Out of curiosity, does the SQL spec give any rules or guidelines about when
aggregates should be applied to resultant rows? Or is it one of the
implementation-dependant things?

----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.C.N. 008 659 498) | /(@) ______---_
Tel: +61-03-5367 7422 | _________ \
Fax: +61-03-5367 7430 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/

From bouncefilter Tue Jan 25 01:10:44 2000
Received: from clio.trends.ca (root@clio.trends.ca [209.47.148.2])
by hub.org (8.9.3/8.9.3) with ESMTP id BAA33540
for <pgsql-hackers@hub.org>; Tue, 25 Jan 2000 01:10:23 -0500 (EST)
(envelope-from robinson@netrinsics.com)
Received: from netrinsics.com ([202.106.6.112])
by clio.trends.ca (8.9.3+Sun/8.9.3) with ESMTP id BAA25741
for <pgsql-hackers@hub.org>; Tue, 25 Jan 2000 01:04:58 -0500 (EST)
Received: (from robinson@localhost)
by netrinsics.com (8.9.3/8.9.3) id OAA39630;
Tue, 25 Jan 2000 14:05:36 +0800 (+0800) (envelope-from robinson)
Date: Tue, 25 Jan 2000 14:05:36 +0800 (+0800)
From: Michael Robinson <robinson@netrinsics.com>
Message-Id: <200001250605.OAA39630@netrinsics.com>
To: robinson@netrinsics.com, t-ishii@sra.co.jp
Subject: Re: [HACKERS] fatal copy in/out error (6.5.3)
Cc: pgsql-hackers@hub.org, tgl@sss.pgh.pa.us
In-Reply-To: <20000125140004W.t-ishii@sra.co.jp>

Tatsuo Ishii <t-ishii@sra.co.jp> writes:

Yes, it's not a PostgreSQL's business but is a really big problem in
the real world. Maybe some HTML gurus might have good suggestions on
these issues (something like using a language tag?)

The only solution is defensive programming. Even if there were a standard
that everyone followed, if malicious people could break things by not
following the standard, then you can be certain that somebody would do so.

Here it is. With this patch, copy out should be happy even with the
wrong data. I'm not sure if it could be displayed correctly, though.

Thank you very much. However, I think even this is too optimistic:

! if (*s & 0x80)

Shouldn't it be something like:

if ((*s & 0x80) && (*(s+1) & 0x80))

Even though "\242\242\242\0" is an invalid EUC sequence, it still shouldn't be
allowed to break the software.

-Michael Robinson

From bouncefilter Tue Jan 25 01:10:44 2000
Received: from mailo.vtcif.telstra.com.au (mailo.vtcif.telstra.com.au
[202.12.144.17]) by hub.org (8.9.3/8.9.3) with ESMTP id BAA33505
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 01:09:56 -0500 (EST)
(envelope-from chris@bitmead.com)
Received: (from uucp@localhost) by mailo.vtcif.telstra.com.au (8.8.2/8.6.9) id
RAA28771 for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 17:09:23 +1100 (EST)
Received: from maili.vtcif.telstra.com.au(202.12.142.17)
via SMTP by mailo.vtcif.telstra.com.au, id smtpdASezS_;
Tue Jan 25 17:09:02 2000
Received: (from uucp@localhost) by maili.vtcif.telstra.com.au (8.8.2/8.6.9) id
RAA15765 for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 17:09:01 +1100 (EST)
Received: from localhost(127.0.0.1), claiming to be "mail.cdn.telstra.com.au"
via SMTP by localhost, id smtpd09eiv_; Tue Jan 25 17:08:23 2000
Received: from lunitari.nimrod.itg.telecom.com.au
(lunitari.nimrod.itg.telecom.com.au [192.53.254.48]) by
mail.cdn.telstra.com.au (8.8.2/8.6.9) with ESMTP id RAA24753
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 17:08:22 +1100 (EST)
Received: from majere.design (majere [192.53.254.45])
by lunitari.nimrod.itg.telecom.com.au (8.9.1/8.9.3) with ESMTP id
RAA27662 for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 17:07:46 +1100 (EST)
Received: from bitmead.com (localhost [127.0.0.1])
by majere.design (8.8.8+Sun/8.8.8) with ESMTP id RAA11236
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 17:07:03 +1100 (EST)
Sender: chrisb@nimrod.itg.telstra.com.au
Message-ID: <388D3D87.C1BCE49B@bitmead.com>
Date: Tue, 25 Jan 2000 17:07:03 +1100
From: Chris Bitmead <chris@bitmead.com>
Organization: IBM Global Services
X-Mailer: Mozilla 4.6 [en] (X11; I; SunOS 5.6 sun4u)
X-Accept-Language: en
MIME-Version: 1.0
To: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] DISTINCT ON: speak now or forever hold your peace
References: <11879.948776470@sss.pgh.pa.us>
<3.0.5.32.20000125150557.036121c0@mail.rhyme.com.au>
<2295.948765015@sss.pgh.pa.us>
<3.0.5.32.20000125140349.035768d0@mail.rhyme.com.au>
<3.0.5.32.20000125150557.036121c0@mail.rhyme.com.au>
<3.0.5.32.20000125170026.0357d8f0@mail.rhyme.com.au>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Philip Warner wrote:

In fact, this is a KEY notion behind RDBMS systems...queries return an
unordered set, pure and simple, unless you supply an "order" clause
(though "group by" appears to be implemented by ordering in various
RDBMS systems, that's an efficiency hack not a given AFAIK).

On second thoughts, maybe I agree DISTINCT ON should be dumped. In my
example, if I wanted the youngest people with a given name sorted oldest
first things would be confusing. It's an inconsistant command, I can see
that now.

From bouncefilter Tue Jan 25 01:13:45 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id BAA33999
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 01:13:08 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id BAA12272;
Tue, 25 Jan 2000 01:12:46 -0500 (EST)
To: Philip Warner <pjw@rhyme.com.au>
cc: Don Baccus <dhogaza@pacifier.com>, Chris Bitmead <chris@bitmead.com>,
pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] DISTINCT ON: speak now or forever hold your peace
In-reply-to: <3.0.5.32.20000125170026.0357d8f0@mail.rhyme.com.au>
References: <11879.948776470@sss.pgh.pa.us>
<3.0.5.32.20000125150557.036121c0@mail.rhyme.com.au>
<2295.948765015@sss.pgh.pa.us>
<3.0.5.32.20000125140349.035768d0@mail.rhyme.com.au>
<3.0.5.32.20000125150557.036121c0@mail.rhyme.com.au>
<3.0.5.32.20000125170026.0357d8f0@mail.rhyme.com.au>
Comments: In-reply-to Philip Warner <pjw@rhyme.com.au>
message dated "Tue, 25 Jan 2000 17:00:26 +1100"
Date: Tue, 25 Jan 2000 01:12:46 -0500
Message-ID: <12269.948780766@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Philip Warner <pjw@rhyme.com.au> writes:

SQL in general doesn't believe that tuple ordering has any semantic
significance --- you can ask for ORDER BY, but that's only honored
at the end stage of SELECT result delivery, not necessarily anywhere
in the bowels of a query.

Out of curiosity, does the SQL spec give any rules or guidelines about when
aggregates should be applied to resultant rows? Or is it one of the
implementation-dependant things?

Well, SQL's conceptual model is perfectly clear about the processing
work flow of a SELECT: after you've constructed the tuple set (which
might involve joining multiple relations), you apply the WHERE condition
to filter out uninteresting tuples. Then you apply GROUP BY (if given)
to divvy the tuples into groups. Then you apply HAVING to eliminate
uninteresting groups. Then you apply aggregate functions (if any) to
individual groups, or to the whole filtered result set if no groups.
Finally you apply ORDER BY to whatever's left.

Note that at no point except the final output is there any notion
of the tuples being generated or processed in a particular order.
Given appropriate hardware, much of this could be done in parallel.
Parallel or not, an implementation is free to choose the processing
order for its convenience.

regards, tom lane

From bouncefilter Tue Jan 25 01:41:45 2000
Received: from thelab.hub.org (nat199.7.mpoweredpc.net [142.177.199.7])
by hub.org (8.9.3/8.9.3) with ESMTP id BAA41242
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 01:41:33 -0500 (EST) (envelope-from scrappy@hub.org)
Received: from localhost (scrappy@localhost)
by thelab.hub.org (8.9.3/8.9.1) with ESMTP id CAA35372;
Tue, 25 Jan 2000 02:39:53 -0400 (AST) (envelope-from scrappy@hub.org)
X-Authentication-Warning: thelab.hub.org: scrappy owned process doing -bs
Date: Tue, 25 Jan 2000 02:39:53 -0400 (AST)
From: The Hermit Hacker <scrappy@hub.org>
To: Don Baccus <dhogaza@pacifier.com>
cc: Bruce Momjian <pgman@candle.pha.pa.us>,
Peter Eisentraut <peter_e@gmx.net>,
Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Happy column dropping
In-Reply-To: <3.0.1.32.20000124204431.007b3940@mail.pacifier.com>
Message-ID: <Pine.BSF.4.21.0001250233330.94842-100000@thelab.hub.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Mon, 24 Jan 2000, Don Baccus wrote:

At 09:48 PM 1/24/00 -0500, Bruce Momjian wrote:

Maybe that is true. Having phantom column all over the code is going to
be a mess, and hardly worth it considering how many developers there are
and how many _big_ items still have to be done.

Works for Oracle...I guess Postgres is just an obviously more robust,
faster, more reliable, and altogether more brilliant RDBMS than this
loser commercial DB? It's really hard to understand why Postgres has
had such a poor reputation over the years when faced with such facts,
isn't it?

Woah ... pull back here ... I haven't got a *clue* where this response
came from, but, from what I can tell, it was *totally* uncalled
for. Oracle makes how many *millions* of dollars a year to sink into
programmers dedicated to it? We have how many developers in comparison,
who don't get paid and who work on things they feel is important ... and,
there are alot bigger items on the TODO list ...

If we had unlimited (or near so) monetary resourses, fine, but we don't,
we have alot of volunteers who spend alot of *personal* time advancing
this project ... we are *not* trying to compete directly with Oracle, we
are trying to create a product that ppl *trust* and can *rely* on
... Bruce's comment above about "going to be a mess", to me, interpretes
as "there has gotta be a cleaner way of doing it that we haven't come up
with yet" ...

Messing up code for one feature is rarely worth it.

Dropping constraints on a table just because you drop a column is
just butt-ugly.

Sorry if you disagree.

And yes, I do agree here ... I like the solution that one person presented
that had it so that you couldn't drop the column unless as constraints and
requirements were removed first ...

Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

From bouncefilter Tue Jan 25 01:42:44 2000
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
by hub.org (8.9.3/8.9.3) with ESMTP id BAA41360
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 01:42:18 -0500 (EST) (envelope-from Inoue@tpf.co.jp)
Received: from cadzone ([126.0.1.40] (may be forged))
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with SMTP
id PAA06230; Tue, 25 Jan 2000 15:40:58 +0900
From: "Hiroshi Inoue" <Inoue@tpf.co.jp>
To: "Tom Lane" <tgl@sss.pgh.pa.us>
Cc: "Peter Eisentraut" <peter_e@gmx.net>,
"PostgreSQL Development" <pgsql-hackers@postgresql.org>,
"Don Baccus" <dhogaza@pacifier.com>
Subject: RE: [HACKERS] Happy column dropping
Date: Tue, 25 Jan 2000 15:46:43 +0900
Message-ID: <001301bf66ff$f0d8d8e0$2801007e@tpf.co.jp>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
In-Reply-To: <11573.948772632@sss.pgh.pa.us>
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
Importance: Normal

-----Original Message-----
From: owner-pgsql-hackers@postgresql.org
[mailto:owner-pgsql-hackers@postgresql.org]On Behalf Of Tom Lane

robustness of heap_getattr(). I have always wondered whether it works
if you ADD COLUMN a 33'rd column (or anything that is just past the
next padding boundary for the null-values bitmap).

If so,it's very serious.
ADD COLUMN feature is much more important than other ALTER TABLE
options and it's also important to implement it without touching the table
as possible.
It seems to me that each tuple has t_natts and t_hoff for the change of
table definition.

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp

From bouncefilter Tue Jan 25 01:51:44 2000
Received: from thelab.hub.org (nat199.7.mpoweredpc.net [142.177.199.7])
by hub.org (8.9.3/8.9.3) with ESMTP id BAA42561
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 01:51:17 -0500 (EST) (envelope-from scrappy@hub.org)
Received: from localhost (scrappy@localhost)
by thelab.hub.org (8.9.3/8.9.1) with ESMTP id CAA45119;
Tue, 25 Jan 2000 02:50:57 -0400 (AST) (envelope-from scrappy@hub.org)
X-Authentication-Warning: thelab.hub.org: scrappy owned process doing -bs
Date: Tue, 25 Jan 2000 02:50:57 -0400 (AST)
From: The Hermit Hacker <scrappy@hub.org>
To: Don Baccus <dhogaza@pacifier.com>
cc: Bruce Momjian <pgman@candle.pha.pa.us>,
Peter Eisentraut <peter_e@gmx.net>,
Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Happy column dropping
In-Reply-To: <3.0.1.32.20000124211728.00f728b0@mail.pacifier.com>
Message-ID: <Pine.BSF.4.21.0001250241150.94842-100000@thelab.hub.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Mon, 24 Jan 2000, Don Baccus wrote:

At 12:08 AM 1/25/00 -0500, Bruce Momjian wrote:

Dropping constraints on a table just because you drop a column is
just butt-ugly.

We aren't charging 100k either. We do what we can, and spend time where
it is most valuable.

Perhaps I've misunderstood, then...I'd thought part of the goal was to
compete with the 100K model without charging 100K...

If not, OK.

(BTW, Oracle for development is free and you can actually cruise with
that for some time. A five-user license for non-internet use costs
$1450 for Linux)

And further BTW...some folks recently went through a little handwringing
upon hearing the Interbase will be released in free, open source form.

And there's a recent, though not current, Sybase version available for
Linux users for free, too...

If the point's not to be competitive with other free or low cost options,
is there any point at all?

Is it time for me to apologize for having high standards?

Not necessarily ... only for expecting more from volunteers then they are
willing, or have time, to put forth ...

Where does Sybase make its money? Oracle? Inbase? In Oracle's case, it
is what we work with *heavily* at my *real* job, and I know where they
make their money ... support contracts that we never use ...

PostgreSQL, Inc was created 6+ months ago to provide a means of clients
getting commerical support, and it does enough to keep things afloat, but
rich we are not getting ... my *hope* when forming this was that we could
generate enough revenues to go out and hire someone to deal with features
that are so time-consuming that they are difficult to implement ... its a
slow growth ... maybe in 6mos we'll be able to do this ... we keep coming
up with new ways of increasing revenues and we dedicate a certain
percentage of each to a 'kitty' to be used for PostgreSQL related projects
... that kitty is small, but growing ...

We try with those resources we have ...

Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

From bouncefilter Tue Jan 25 05:12:46 2000
Received: from mail.ngi.de ([195.243.0.227])
by hub.org (8.9.3/8.9.3) with ESMTP id FAA97342
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 05:11:59 -0500 (EST)
(envelope-from michael@fam-meskes.de)
Received: from p3E9B988E.dip0.t-ipconnect.de
(root@p3E9B988E.dip0.t-ipconnect.de [62.155.152.142])
by mail.ngi.de (8.9.3/8.9.3) with ESMTP id LAA78883
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 11:12:52 +0100 (CET)
Received: (from michael@localhost)
by tanja.fam-meskes.de (8.9.3/8.9.3/Debian 8.9.3-6) id IAA01782
for pgsql-hackers@postgresql.org; Tue, 25 Jan 2000 08:12:36 +0100
Date: Tue, 25 Jan 2000 08:12:36 +0100
From: Michael Meskes <meskes@postgresql.org>
To: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] DISTINCT ON: speak now or forever hold your peace
Message-ID: <20000125081236.A1774@fam-meskes.de>
Mail-Followup-To: pgsql-hackers@postgresql.org
References: <11547.948771949@sss.pgh.pa.us>
<200001250355.WAA23883@candle.pha.pa.us>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
User-Agent: Mutt/1.0i
In-Reply-To: <200001250355.WAA23883@candle.pha.pa.us>;
from pgman@candle.pha.pa.us on Mon, Jan 24, 2000 at 10:55:02PM
-0500

On Mon, Jan 24, 2000 at 10:55:02PM -0500, Bruce Momjian wrote:

In short, it's not clear to me that supporting DISTINCT ON is a good use
of our limited resources. I'm willing to pull it out, but not to fix it.
Does someone else want to take responsibility for it?

OK, we can disable it and put it on the TODO list. It is already there.

Or, as you suggested earlier, throw a NOTICE instead of disabling it. But
then I doubt Tom talked about physically removing the code anyway. Disabling
certainly is enough.

Michael
--
Michael Meskes | Go SF 49ers!
Th.-Heuss-Str. 61, D-41812 Erkelenz | Go Rhein Fire!
Tel.: (+49) 2431/72651 | Use Debian GNU/Linux!
Email: Michael@Fam-Meskes.De | Use PostgreSQL!

Tom Lane wrote:

Chris Bitmead <chris@bitmead.com> writes:

Tom Lane wrote:

If I don't hear loud hollers very soon, I'm going to eliminate the
DISTINCT ON "feature" for 7.0. As previously discussed, this feature
is not standard SQL and has no clear semantic interpretation.

I don't feel overly strongly about this, but if I remember right you can
do some pretty cool things with this feature, provided you do define
some semantics clearly.

We did talk about that, but I didn't hear any strong support for doing
it, as opposed to pulling the feature completely... in particular,
I didn't hear anyone volunteering to do the work...

I've been using DISTINCT to fix things when vacuum gets into a mess (yes, it
happened again last night,
FATAL 1: VACUUM (vc_rpfheap): BlowawayRelationBuffers returned -2
). I then have loads of duplicates in a table and by doing a SELECT DISTINCT
... INTO ... I can get rid of them. A few table renames later I'm back in
business. So how do I do this without DISTINCT?

Better would be of course to get rid of the BlowawayRelationBuffers problem.
I cannot reproduce it yet, but I think I can make it happen by running vacuum
and updating tables that are linked by triggers. Does this sound plausible?
And yes, I'm running on Alpha, i.e. 64 bits.

Adriaan

From bouncefilter Tue Jan 25 02:13:45 2000
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
by hub.org (8.9.3/8.9.3) with ESMTP id CAA48792
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 02:13:03 -0500 (EST) (envelope-from Inoue@tpf.co.jp)
Received: from cadzone ([126.0.1.40] (may be forged))
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with SMTP
id QAA06242; Tue, 25 Jan 2000 16:12:38 +0900
From: "Hiroshi Inoue" <Inoue@tpf.co.jp>
To: "Mike Mascari" <mascarm@mascari.com>
Cc: "Don Baccus" <dhogaza@pacifier.com>, "Tom Lane" <tgl@sss.pgh.pa.us>,
"Peter Eisentraut" <peter_e@gmx.net>,
"PostgreSQL Development" <pgsql-hackers@postgreSQL.org>
Subject: RE: [HACKERS] Happy column dropping
Date: Tue, 25 Jan 2000 16:18:22 +0900
Message-ID: <001401bf6704$5ca7e3a0$2801007e@tpf.co.jp>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
In-Reply-To: <388D1919.DC7D369F@mascari.com>
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
Importance: Normal

-----Original Message-----
From: Mike Mascari [mailto:mascarm@mascari.com]

Do other DBMSs allow such things ?
For example,in Oracle NOT NULL constraint could be specified for new
column only when there's no row in the table AFAIK.

I couldn't judge it's worth the work.

But the times when I've found it would be nice to use ALTER TABLE
ADD COLUMN with NOT NULL constraints is in development mode when
the schema is changing rapidly and there actually isn't any data
yet in the tables. Otherwise, to add a new NOT NULL column during
a development cycle, one has to drop and recreate the table, all
triggers, comments, etc. Its just a real pain. :-(

I don't know details about constraints.
Probably you could implement it better than me.

Even default is not allowed in ADD COLUMN now.
There may be other reasons why they aren't allowed.

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp

From bouncefilter Tue Jan 25 02:30:45 2000
Received: from corvette.mascari.com (dhcp26142035.columbus.rr.com
[24.26.142.35]) by hub.org (8.9.3/8.9.3) with ESMTP id CAA55242
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 02:30:32 -0500 (EST)
(envelope-from mascarm@mascari.com)
Received: from mascari.com (ferrari.mascari.com [192.168.2.1])
by corvette.mascari.com (8.9.3/8.9.3) with ESMTP id CAA11286;
Tue, 25 Jan 2000 02:24:11 -0500
Message-ID: <388D4F9A.AB09648C@mascari.com>
Date: Tue, 25 Jan 2000 02:24:10 -0500
From: Mike Mascari <mascarm@mascari.com>
Organization: Mascari Development Inc
X-Mailer: Mozilla 4.7 [en] (Win95; I)
X-Accept-Language: en
MIME-Version: 1.0
To: The Hermit Hacker <scrappy@hub.org>
CC: Don Baccus <dhogaza@pacifier.com>, Bruce Momjian <pgman@candle.pha.pa.us>,
Peter Eisentraut <peter_e@gmx.net>, Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Happy column dropping
References: <Pine.BSF.4.21.0001250233330.94842-100000@thelab.hub.org>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

The Hermit Hacker wrote:

On Mon, 24 Jan 2000, Don Baccus wrote:

At 09:48 PM 1/24/00 -0500, Bruce Momjian wrote:

Maybe that is true. Having phantom column all over the code is going to
be a mess, and hardly worth it considering how many developers there are
and how many _big_ items still have to be done.

Works for Oracle...I guess Postgres is just an obviously more robust,
faster, more reliable, and altogether more brilliant RDBMS than this
loser commercial DB? It's really hard to understand why Postgres has
had such a poor reputation over the years when faced with such facts,
isn't it?

Woah ... pull back here ... I haven't got a *clue* where this response
came from, but, from what I can tell, it was *totally* uncalled
for. Oracle makes how many *millions* of dollars a year to sink into

^^^^^^^^
Billions. Oracle had 8.82 billion in sales in FY 1999, 1.2
billion in earnings (profit).

programmers dedicated to it?

Oracle has 43,800 employees.

We have how many developers in comparison,
who don't get paid and who work on things they feel is important ... and,
there are alot bigger items on the TODO list ...

The market capitalization of Oracle before the split a week ago
was $147 billion dollars. Larry Ellison owns 24% of the company
and is thus worth $35 billion dollars alone. The least Don could
do is buy a T-shirt or a key ring or something... ;-)

Mike Mascari

Adriaan Joubert <a.joubert@albourne.com> writes:

I've been using DISTINCT to fix things when vacuum gets into a mess (yes, it
happened again last night,
FATAL 1: VACUUM (vc_rpfheap): BlowawayRelationBuffers returned -2
). I then have loads of duplicates in a table and by doing a SELECT DISTINCT
... INTO ... I can get rid of them. A few table renames later I'm back in
business. So how do I do this without DISTINCT?

I didn't say anything about eliminating SELECT DISTINCT; I was talking
about SELECT DISTINCT ON, which is a whole 'nother thing.

Better would be of course to get rid of the BlowawayRelationBuffers problem.

Quite so. I think this is fixed for 7.0 ... that error message
indicates a buffer refcount leak, and I fixed several problems
along that line a couple months ago. We shall see if I got 'em
all or not ...

regards, tom lane

From bouncefilter Tue Jan 25 02:55:45 2000
Received: from acheron.rime.com.au (root@albatr.lnk.telstra.net
[139.130.54.222]) by hub.org (8.9.3/8.9.3) with ESMTP id CAA61254
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 02:54:58 -0500 (EST)
(envelope-from pjw@rhyme.com.au)
Received: from oberon (Oberon.rime.com.au [203.8.195.100])
by acheron.rime.com.au (8.9.3/8.9.3) with SMTP id SAA08550;
Tue, 25 Jan 2000 18:53:34 +1100
Message-Id: <3.0.5.32.20000125185459.00cf7450@mail.rhyme.com.au>
X-Sender: pjw@mail.rhyme.com.au
X-Mailer: QUALCOMM Windows Eudora Pro Version 3.0.5 (32)
Date: Tue, 25 Jan 2000 18:54:59 +1100
To: Tom Lane <tgl@sss.pgh.pa.us>
From: Philip Warner <pjw@rhyme.com.au>
Subject: Re: [HACKERS] DISTINCT ON: speak now or forever hold your
peace
Cc: Don Baccus <dhogaza@pacifier.com>, Chris Bitmead <chris@bitmead.com>,
pgsql-hackers@postgreSQL.org
In-Reply-To: <12269.948780766@sss.pgh.pa.us>
References: <3.0.5.32.20000125170026.0357d8f0@mail.rhyme.com.au>
<11879.948776470@sss.pgh.pa.us>
<3.0.5.32.20000125150557.036121c0@mail.rhyme.com.au>
<2295.948765015@sss.pgh.pa.us>
<3.0.5.32.20000125140349.035768d0@mail.rhyme.com.au>
<3.0.5.32.20000125150557.036121c0@mail.rhyme.com.au>
<3.0.5.32.20000125170026.0357d8f0@mail.rhyme.com.au>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 01:12 25/01/00 -0500, Tom Lane wrote:

Philip Warner <pjw@rhyme.com.au> writes:

SQL in general doesn't believe that tuple ordering has any semantic
significance --- you can ask for ORDER BY, but that's only honored
at the end stage of SELECT result delivery, not necessarily anywhere
in the bowels of a query.

Out of curiosity, does the SQL spec give any rules or guidelines about when
aggregates should be applied to resultant rows? Or is it one of the
implementation-dependant things?

Well, SQL's conceptual model is perfectly clear about the processing
work flow of a SELECT: after you've constructed the tuple set (which
might involve joining multiple relations), you apply the WHERE condition
to filter out uninteresting tuples. Then you apply GROUP BY (if given)
to divvy the tuples into groups. Then you apply HAVING to eliminate
uninteresting groups. Then you apply aggregate functions (if any) to
individual groups, or to the whole filtered result set if no groups.
Finally you apply ORDER BY to whatever's left.

Thanks, but now I'm confused. I would have thought that aggregates went
*before* the having clause, since at least one DB I know allows:

select job_type,avg(age) from <wherever> where <stuff> group by job_type
having avg(age) > 70;

ie. the use of aggregate results in the 'having' clause.

----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.C.N. 008 659 498) | /(@) ______---_
Tel: +61-03-5367 7422 | _________ \
Fax: +61-03-5367 7430 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/

From bouncefilter Tue Jan 25 03:08:45 2000
Received: from mail.enterprise.net (mail.enterprise.net [194.72.192.18])
by hub.org (8.9.3/8.9.3) with ESMTP id DAA70049
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 03:07:48 -0500 (EST) (envelope-from olly@lfix.co.uk)
Received: from linda.lfix.co.uk (root@max04-069.enterprise.net
[194.72.196.189])
by mail.enterprise.net (8.8.5/8.8.5) with ESMTP id IAA08173;
Tue, 25 Jan 2000 08:07:44 GMT
Received: from lfix.co.uk (olly@localhost [127.0.0.1])
by linda.lfix.co.uk (8.9.3/8.9.3/Debian 8.9.3-6) with ESMTP id IAA03764;
Tue, 25 Jan 2000 08:06:34 GMT
Message-Id: <200001250806.IAA03764@linda.lfix.co.uk>
X-Authentication-Warning: linda.lfix.co.uk: Host olly@localhost [127.0.0.1]
claimed to be lfix.co.uk
X-Mailer: exmh version 2.1.1 10/15/1999 (debian)
To: Chris Bitmead <chris@bitmead.com>
cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Inheritance, referential integrity and other
constraints
In-Reply-To: Message from Chris Bitmead <chris@bitmead.com>
of "Tue, 25 Jan 2000 10:33:07 +1100." <388CE133.EE3ADAC5@bitmead.com>
References: <200001242152.VAA25637@linda.lfix.co.uk>
<388CE133.EE3ADAC5@bitmead.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Date: Tue, 25 Jan 2000 08:06:34 +0000
From: "Oliver Elphick" <olly@lfix.co.uk>

Chris Bitmead wrote:

As long as you're working on this area you could fix the problem where
if you do ALTER table* ADD COLUMN ... pg_dump no longer works because
the column orders have changed in different inherited tables.

It seems that this might be quite a problem; I would not like to have
to do a physical insert into every row in a huge table. Would it be
feasible to add a column order attribute to pg_attribute for tables
altered in this way? A null entry in that would indicate the table was unaltered from its creation.

Perhaps this could be combined with the idea of column hiding: a zero
column number would mean it was hidden.

--
Oliver Elphick Oliver.Elphick@lfix.co.uk
Isle of Wight http://www.lfix.co.uk/oliver
PGP key from public servers; key ID 32B8FAA1
========================================
"My little children, let us not love in word, neither
in tongue; but in deed and in truth."
I John 3:18

From bouncefilter Tue Jan 25 03:22:45 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id DAA71471
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 03:22:33 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id DAA12874;
Tue, 25 Jan 2000 03:20:20 -0500 (EST)
To: Philip Warner <pjw@rhyme.com.au>
cc: Don Baccus <dhogaza@pacifier.com>, Chris Bitmead <chris@bitmead.com>,
pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] DISTINCT ON: speak now or forever hold your peace
In-reply-to: <3.0.5.32.20000125185459.00cf7450@mail.rhyme.com.au>
References: <3.0.5.32.20000125170026.0357d8f0@mail.rhyme.com.au>
<11879.948776470@sss.pgh.pa.us>
<3.0.5.32.20000125150557.036121c0@mail.rhyme.com.au>
<2295.948765015@sss.pgh.pa.us>
<3.0.5.32.20000125140349.035768d0@mail.rhyme.com.au>
<3.0.5.32.20000125150557.036121c0@mail.rhyme.com.au>
<3.0.5.32.20000125170026.0357d8f0@mail.rhyme.com.au>
<3.0.5.32.20000125185459.00cf7450@mail.rhyme.com.au>
Comments: In-reply-to Philip Warner <pjw@rhyme.com.au>
message dated "Tue, 25 Jan 2000 18:54:59 +1100"
Date: Tue, 25 Jan 2000 03:20:20 -0500
Message-ID: <12871.948788420@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Philip Warner <pjw@rhyme.com.au> writes:

Thanks, but now I'm confused. I would have thought that aggregates went
*before* the having clause,

Arrgh. You are right of course. Shouldn't pontificate late at night...

regards, tom lane

Tom Lane wrote:

If I don't hear loud hollers very soon, I'm going to eliminate the
DISTINCT ON "feature" for 7.0. As previously discussed, this feature
is not standard SQL and has no clear semantic interpretation.

I use 'distinct on' to get the most recent reports for each of a group of
locations. E.g.:

create table reports (
location varchar(16),
report_time datetime,
report_text text);

select distinct on location * from reports where location ~~ 'Lond%' order by
location, reporttime desc;

to get the tuples that offer the most recent reports for each of London,
Londonderry, Londy etc.

Is there an alternative?

Julian Scarfe

From bouncefilter Tue Jan 25 03:18:45 2000
Received: from fw.wintelcom.net (bright@ns1.wintelcom.net [209.1.153.20])
by hub.org (8.9.3/8.9.3) with ESMTP id DAA71052
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 03:17:47 -0500 (EST)
(envelope-from bright@fw.wintelcom.net)
Received: (from bright@localhost)
by fw.wintelcom.net (8.9.3/8.9.3) id AAA25236;
Tue, 25 Jan 2000 00:42:01 -0800 (PST)
Date: Tue, 25 Jan 2000 00:42:00 -0800
From: Alfred Perlstein <bright@wintelcom.net>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Don Baccus <dhogaza@pacifier.com>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Oh btw, about XXX
Message-ID: <20000125004200.U26520@fw.wintelcom.net>
References: <3.0.1.32.20000124184137.01069490@mail.pacifier.com>
<001001bf66d7$b531ba00$2801007e@tpf.co.jp>
<001001bf66d7$b531ba00$2801007e@tpf.co.jp>
<3.0.1.32.20000124184137.01069490@mail.pacifier.com>
<3.0.1.32.20000124210143.0106b780@mail.pacifier.com>
<12046.948778754@sss.pgh.pa.us>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Mailer: Mutt 1.0.1i
In-Reply-To: <12046.948778754@sss.pgh.pa.us>;
from tgl@sss.pgh.pa.us on Tue, Jan 25, 2000 at 12:39:14AM -0500

* Tom Lane <tgl@sss.pgh.pa.us> [000124 22:10] wrote:

Don Baccus <dhogaza@pacifier.com> writes:

I think this probably explains some of the editorial comments about the
code. There seem to be some added by "XXX" - is that person part of the
current clan of developers? Comments like "This code is a crock because..."

XXX isn't a signature, it's a conventional marker for a "Hey! This is
broken! FIX ME!" kind of comment. I think the original idea was you
could do "grep XXX *.c" if you were idly looking for problems to work
on. Some projects use "FIXME" in the same way.

It can also mean 'blech, this should be cleaner' such as the use
of 'curproc' in the FreeBSD kernel do figure out which process is
using the current codepath, sometimes you need it and the API
doesn't pass it down to you, hence
struct proc *p = curproc; /* XXX */
which is strewn about the code.

The only signatures I've seen in the Postgres code are initials at the
ends of comments. XXX usually goes at the front of a gripe.

gvim even highlights them in bright inverse yellow. :)

--
-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]

From bouncefilter Tue Jan 25 04:53:47 2000
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
by hub.org (8.9.3/8.9.3) with ESMTP id EAA90792
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 04:53:15 -0500 (EST) (envelope-from Inoue@tpf.co.jp)
Received: from tpf.co.jp ([126.0.1.56] (may be forged))
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with ESMTP
id SAA06368; Tue, 25 Jan 2000 18:53:04 +0900
Message-ID: <388D72B5.88F36E5F@tpf.co.jp>
Date: Tue, 25 Jan 2000 18:53:57 +0900
From: Hiroshi Inoue <Inoue@tpf.co.jp>
X-Mailer: Mozilla 4.51 [ja] (Win98; I)
X-Accept-Language: ja
MIME-Version: 1.0
To: Bruce Momjian <pgman@candle.pha.pa.us>
CC: Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Re: vacuum timings
References: <200001221733.MAA21558@candle.pha.pa.us>
Content-Type: text/plain; charset=iso-2022-jp
Content-Transfer-Encoding: 7bit

Bruce Momjian wrote:

Quite a few people have reported finding the opposite in practice.
You should probably try vacuuming after deleting or updating some
fraction of the rows, rather than just the all or none cases.

Vacuum after delelting all rows isn't a worst case.
There's no moving in that case and vacuum doesn't need to call
index_insert() corresponding to the moving of heap tuples.

Vacuum after deleting half of rows may be one of the worst case.
In this case,index_delete() is called as many times as 'delete all'
case and expensive index_insert() is called for moved_in tuples.

I will test that.

I tried my test case in less scale than Bruce.

CREATE TABLE t (id int4, dt int4);
for (i=0; i < 2500000; i++)
insert into t values ( i, (i * 1009) % 2500000);
delete from t where id < 1250000;

1) vacuum after create index on t(id) 405sec
2) vacuum after create index on t(dt) > 3600sec
I gave up to continue execution.
3) vacuum and create index on t(id) and t(dt)
90sec + 114sec + 143sec = 347sec.

Seems random index insert is painful for vacuum.

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp

On Mon, 24 Jan 2000, Bruce Momjian wrote:

In short, it's not clear to me that supporting DISTINCT ON is a good use
of our limited resources. I'm willing to pull it out, but not to fix it.
Does someone else want to take responsibility for it?

OK, we can disable it and put it on the TODO list. It is already there.

Am I wrong or can DISTINCT ON not be rewritten in terms of GROUP BY in all
cases?

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Tue Jan 25 05:38:47 2000
Received: from meryl.it.uu.se (root@meryl.it.uu.se [130.238.12.42])
by hub.org (8.9.3/8.9.3) with ESMTP id FAA08785
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 05:38:39 -0500 (EST)
(envelope-from e99re41@DoCS.UU.SE)
Received: from Val.DoCS.UU.SE (e99re41@Val.DoCS.UU.SE [130.238.9.94])
by meryl.it.uu.se (8.8.5/8.8.5) with ESMTP id LAA24329;
Tue, 25 Jan 2000 11:38:28 +0100 (MET)
Received: from localhost (e99re41@localhost) by Val.DoCS.UU.SE (8.6.12/8.6.12)
with SMTP id LAA11967; Tue, 25 Jan 2000 11:38:27 +0100
X-Authentication-Warning: Val.DoCS.UU.SE: e99re41 owned process doing -bs
Date: Tue, 25 Jan 2000 11:38:26 +0100 (MET)
From: Peter Eisentraut <e99re41@DoCS.UU.SE>
Reply-To: Peter Eisentraut <peter_e@gmx.net>
To: Bruce Momjian <pgman@candle.pha.pa.us>
cc: Tom Lane <tgl@sss.pgh.pa.us>, Hiroshi Inoue <Inoue@tpf.co.jp>,
The Hermit Hacker <scrappy@hub.org>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Happy column dropping
In-Reply-To: <200001250251.VAA21667@candle.pha.pa.us>
Message-ID: <Pine.GSO.4.02A.10001251135570.11899-100000@Val.DoCS.UU.SE>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Mon, 24 Jan 2000, Bruce Momjian wrote:

New file name will not be just oid. Too hard to administer.

How exactly does this play into administration? You don't actually do a cp
/usr/local/pgsql/data/base/testdb/mytable /some/where to back up? ;)

What did the Unix kernel programmers think when they first introduced
inodes? The wrote the readdir() library call. We have that too, it's
called pg_class.

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Tue Jan 25 05:42:47 2000
Received: from meryl.it.uu.se (root@meryl.it.uu.se [130.238.12.42])
by hub.org (8.9.3/8.9.3) with ESMTP id FAA09182
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 05:42:02 -0500 (EST)
(envelope-from e99re41@DoCS.UU.SE)
Received: from Val.DoCS.UU.SE (e99re41@Val.DoCS.UU.SE [130.238.9.94])
by meryl.it.uu.se (8.8.5/8.8.5) with ESMTP id LAA24592;
Tue, 25 Jan 2000 11:41:57 +0100 (MET)
Received: from localhost (e99re41@localhost) by Val.DoCS.UU.SE (8.6.12/8.6.12)
with SMTP id LAA11977; Tue, 25 Jan 2000 11:41:56 +0100
X-Authentication-Warning: Val.DoCS.UU.SE: e99re41 owned process doing -bs
Date: Tue, 25 Jan 2000 11:41:55 +0100 (MET)
From: Peter Eisentraut <e99re41@DoCS.UU.SE>
Reply-To: Peter Eisentraut <peter_e@gmx.net>
To: Don Baccus <dhogaza@pacifier.com>
cc: Bruce Momjian <pgman@candle.pha.pa.us>, Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Happy column dropping
In-Reply-To: <3.0.1.32.20000124204431.007b3940@mail.pacifier.com>
Message-ID: <Pine.GSO.4.02A.10001251139220.11899-100000@Val.DoCS.UU.SE>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Mon, 24 Jan 2000, Don Baccus wrote:

Dropping constraints on a table just because you drop a column is
just butt-ugly.

Hey, nobody said that this was the final version. Keeping the constraints
was a trivial step from what there was. I guess the lesson I learned was
that around here you Release Late, Release Rarely, and that's fine, but I
didn't know that. Sorry. -> next time ;)

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Tue Jan 25 05:55:47 2000
Received: from meryl.it.uu.se (root@meryl.it.uu.se [130.238.12.42])
by hub.org (8.9.3/8.9.3) with ESMTP id FAA10880
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 05:55:32 -0500 (EST)
(envelope-from e99re41@DoCS.UU.SE)
Received: from Val.DoCS.UU.SE (e99re41@Val.DoCS.UU.SE [130.238.9.94])
by meryl.it.uu.se (8.8.5/8.8.5) with ESMTP id LAA25589;
Tue, 25 Jan 2000 11:55:26 +0100 (MET)
Received: from localhost (e99re41@localhost) by Val.DoCS.UU.SE (8.6.12/8.6.12)
with SMTP id LAA11989; Tue, 25 Jan 2000 11:55:24 +0100
X-Authentication-Warning: Val.DoCS.UU.SE: e99re41 owned process doing -bs
Date: Tue, 25 Jan 2000 11:55:23 +0100 (MET)
From: Peter Eisentraut <e99re41@DoCS.UU.SE>
Reply-To: Peter Eisentraut <peter_e@gmx.net>
To: Hiroshi Inoue <Inoue@tpf.co.jp>
cc: PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Happy column adding (was RE: [HACKERS] Happy column dropping)
In-Reply-To: <001401bf6704$5ca7e3a0$2801007e@tpf.co.jp>
Message-ID: <Pine.GSO.4.02A.10001251152160.11899-100000@Val.DoCS.UU.SE>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Tue, 25 Jan 2000, Hiroshi Inoue wrote:

Even default is not allowed in ADD COLUMN now.
There may be other reasons why they aren't allowed.

It's not a matter of *allowed*, it's a parsing deficiency. The fact that
there was a default declared gets silently ignored. If y'all allow ;) I
would like to fix that (have already started a bit) by perusing the code
in parse_func.c:transformCreateStmt and do the same for the alter table
add column part. Maybe and add/drop constraint will come out in the end as
well.

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

On Tue, 25 Jan 2000, Julian Scarfe wrote:

Tom Lane wrote:

If I don't hear loud hollers very soon, I'm going to eliminate the
DISTINCT ON "feature" for 7.0. As previously discussed, this feature
is not standard SQL and has no clear semantic interpretation.

I use 'distinct on' to get the most recent reports for each of a group of
locations. E.g.:

create table reports (
location varchar(16),
report_time datetime,
report_text text);

select distinct on location * from reports where location ~~ 'Lond%' order by
location, reporttime desc;

to get the tuples that offer the most recent reports for each of London,
Londonderry, Londy etc.

Is there an alternative?

select location, max(report_time) from reports group by location

This also has the negligible advantage that it gives you determinate
results.

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Tue Jan 25 06:01:48 2000
Received: from meryl.it.uu.se (root@meryl.it.uu.se [130.238.12.42])
by hub.org (8.9.3/8.9.3) with ESMTP id GAA14336
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 06:01:31 -0500 (EST)
(envelope-from e99re41@DoCS.UU.SE)
Received: from Val.DoCS.UU.SE (e99re41@Val.DoCS.UU.SE [130.238.9.94])
by meryl.it.uu.se (8.8.5/8.8.5) with ESMTP id MAA25974;
Tue, 25 Jan 2000 12:01:24 +0100 (MET)
Received: from localhost (e99re41@localhost) by Val.DoCS.UU.SE (8.6.12/8.6.12)
with SMTP id MAA11999; Tue, 25 Jan 2000 12:01:22 +0100
X-Authentication-Warning: Val.DoCS.UU.SE: e99re41 owned process doing -bs
Date: Tue, 25 Jan 2000 12:01:22 +0100 (MET)
From: Peter Eisentraut <e99re41@DoCS.UU.SE>
Reply-To: Peter Eisentraut <peter_e@gmx.net>
To: The Hermit Hacker <scrappy@hub.org>
cc: Don Baccus <dhogaza@pacifier.com>, Tom Lane <tgl@sss.pgh.pa.us>,
Hiroshi Inoue <Inoue@tpf.co.jp>, pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Well, then you keep your darn columns
In-Reply-To: <Pine.BSF.4.21.0001241622250.79710-100000@thelab.hub.org>
Message-ID: <Pine.GSO.4.02A.10001251200520.11899-100000@Val.DoCS.UU.SE>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Mon, 24 Jan 2000, The Hermit Hacker wrote:

On Mon, 24 Jan 2000, Don Baccus wrote:

At 03:44 PM 1/24/00 -0400, The Hermit Hacker wrote:

Oh, there is a second drawback to it though ...

DROP COLUMN name
ADD COLUMN name <of a different type>

Then what? :(

I don't understand...the idea is to make the old column name
invisible, and therefore "add column" won't see it either.

so the pg_* file that maintains the 'fields' in a table would have two
fields of the same name, one enabled, one disabled?

The entries in pg_attribute have oids as well ...

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

on 25/1/00 10:59, Peter Eisentraut at e99re41@DoCS.UU.SE wrote:

On Tue, 25 Jan 2000, Julian Scarfe wrote:

I use 'distinct on' to get the most recent reports for each of a group of
locations. E.g.:

create table reports (
location varchar(16),
report_time datetime,
report_text text);

select distinct on location * from reports where location ~~ 'Lond%' order by
location, reporttime desc;<pgsql-hackers@postgreSQL.org>

to get the tuples that offer the most recent reports for each of London,
Londonderry, Londy etc.

Is there an alternative?

select location, max(report_time) from reports group by location

This also has the negligible advantage that it gives you determinate
results.

But I want report_text *too* -- that's the important bit, and it's not an
aggregate or common column in the group. So:

Am I wrong or can DISTINCT ON not be rewritten in terms of GROUP BY in all
cases?

I don't think so.

It has to be something like:

select * from reports as r1 where r1.report_time = (select max(report_time)
from reports as r2 where r2.location = r1.location) and r1.location ~~
'Lond%';

However, to my surprise, that's as quick as doing the 'distinct on'.

host=> explain select * from reports as r1 where r1.report_time = (select
max(report_time) from reports as r2 where r2.location = r1.location) and
r1.location ~~ 'Lond%';
NOTICE: QUERY PLAN:

Index Scan using reports_by_location_issuetime on reports r1 (cost=186.24
size=2 width=334)
SubPlan
-> Aggregate (cost=3.62 size=0 width=0)
-> Index Scan using reports_by_location_report_time on reports r2
(cost=3.62 size=13 width=8)

EXPLAIN

host=> explain select distinct on location * from reports where location ~~
'Lond%' order by location, report_time desc;
NOTICE: QUERY PLAN:

Unique (cost=186.24 size=0 width=0)
-> Sort (cost=186.24 size=0 width=0)
-> Index Scan using reports_by_location_issuetime on reports
(cost=186.24 size=2 width=334)

I'm surprise that the "reports_by_location_issuetime" index got used, since
it involves the issuetime field which is not the same as the report_time
field. I edited it and numerous other fields from the quoted 'create table'
for simplicity. There's also a "reports_by_location_report_time" which
indexes on, obviously, (location, report_time). Comments?

Julian Scarfe, learning...

From bouncefilter Tue Jan 25 09:33:50 2000
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67] (may
be forged)) by hub.org (8.9.3/8.9.3) with ESMTP id JAA66285
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 09:33:20 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
JAA14452;
Tue, 25 Jan 2000 09:14:29 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001251414.JAA14452@candle.pha.pa.us>
Subject: Re: [HACKERS] Happy column dropping
In-Reply-To: <Pine.GSO.4.02A.10001251135570.11899-100000@Val.DoCS.UU.SE> from
Peter Eisentraut at "Jan 25, 2000 11:38:26 am"
To: Peter Eisentraut <peter_e@gmx.net>
Date: Tue, 25 Jan 2000 09:14:29 -0500 (EST)
CC: Tom Lane <tgl@sss.pgh.pa.us>, Hiroshi Inoue <Inoue@tpf.co.jp>,
The Hermit Hacker <scrappy@hub.org>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Mon, 24 Jan 2000, Bruce Momjian wrote:

New file name will not be just oid. Too hard to administer.

How exactly does this play into administration? You don't actually do a cp
/usr/local/pgsql/data/base/testdb/mytable /some/where to back up? ;)

What did the Unix kernel programmers think when they first introduced
inodes? The wrote the readdir() library call. We have that too, it's
called pg_class.

Ingres has table names as numbered files. It is a pain to figure out
which files match which tables. If you need to restore a table from
tape, the pg_class entry is gone and you have no way to figure out the
right table. When analyzing disk space, figuring who is using the space
is a pain.

Every Ingre admin I ever talked to agreed file numbers are a pain.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Tue Jan 25 09:33:50 2000
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67] (may
be forged)) by hub.org (8.9.3/8.9.3) with ESMTP id JAA65659
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 09:32:52 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
JAA14465;
Tue, 25 Jan 2000 09:15:04 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001251415.JAA14465@candle.pha.pa.us>
Subject: Re: [HACKERS] Happy column dropping
In-Reply-To: <Pine.GSO.4.02A.10001251139220.11899-100000@Val.DoCS.UU.SE> from
Peter Eisentraut at "Jan 25, 2000 11:41:55 am"
To: Peter Eisentraut <peter_e@gmx.net>
Date: Tue, 25 Jan 2000 09:15:04 -0500 (EST)
CC: Don Baccus <dhogaza@pacifier.com>, Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Mon, 24 Jan 2000, Don Baccus wrote:

Dropping constraints on a table just because you drop a column is
just butt-ugly.

Hey, nobody said that this was the final version. Keeping the constraints
was a trivial step from what there was. I guess the lesson I learned was
that around here you Release Late, Release Rarely, and that's fine, but I
didn't know that. Sorry. -> next time ;)

I probably lead you astray on that one.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Tue Jan 25 09:53:50 2000
Received: from smtp1.andrew.cmu.edu (SMTP1.ANDREW.CMU.EDU [128.2.10.81])
by hub.org (8.9.3/8.9.3) with ESMTP id JAA69505
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 09:53:38 -0500 (EST) (envelope-from geek+@cmu.edu)
Received: from export.andrew.cmu.edu (EXPORT.ANDREW.CMU.EDU [128.2.23.2])
by smtp1.andrew.cmu.edu (8.9.3/8.9.3) with ESMTP id JAA11789
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 09:53:34 -0500 (EST)
Date: Tue, 25 Jan 2000 09:53:34 -0500 (EST)
Message-Id: <200001251453.JAA11789@smtp1.andrew.cmu.edu>
From: Brian E Gallew <geek+@cmu.edu>
X-Mailer: BatIMail version 3.2
To: PostgreSQL Development <pgsql-hackers@postgreSQL.org>
In-reply-to: <200001251414.JAA14452@candle.pha.pa.us>
Subject: Re: [HACKERS] Happy column dropping
References: <200001251414.JAA14452@candle.pha.pa.us>
Mime-Version: 1.0 (generated by tm-edit 7.106)
Content-Type: multipart/signed; protocol="application/pgp-signature";
boundary="pgp-sign-Multipart_Tue_Jan_25_09:53:29_2000-1"; micalg=pgp-md5
Content-Transfer-Encoding: 7bit

--pgp-sign-Multipart_Tue_Jan_25_09:53:29_2000-1
Content-Type: text/plain; charset=US-ASCII

Then <pgman@candle.pha.pa.us> spoke up and said:

Ingres has table names as numbered files. It is a pain to figure out
which files match which tables. If you need to restore a table from
tape, the pg_class entry is gone and you have no way to figure out the
right table. When analyzing disk space, figuring who is using the space
is a pain.

Every Ingre admin I ever talked to agreed file numbers are a pain.

I, too, found the Ingres naming scheme to be a pain, especially since
the numbers change when you drop/recreate a table (but not for
truncation). In my case, I wrote a fairly trivial script that runs as
part of the backup job that lists all tables and filenames. This give
me an accurate map of whats out there. It has the added advantage
that I can use this list to remove tables/database owned by users who
no longer exist (although, in truth, I hesitate to do that
automatically).

On the other hand, I *hate* the Oracle tablespace scheme. At least
with Ingres, when I update a table, I *don't* have to backup all of
the other tables in that database (barring Oracle's hot-backup mode,
of course).

--
=====================================================================
| JAVA must have been developed in the wilds of West Virginia. |
| After all, why else would it support only single inheritance?? |
=====================================================================
| Finger geek@cmu.edu for my public key. |
=====================================================================

--pgp-sign-Multipart_Tue_Jan_25_09:53:29_2000-1
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

-----BEGIN PGP MESSAGE-----
Version: 2.6.2
Comment: Processed by Mailcrypt 3.3, an Emacs/PGP interface

iQBVAwUBOI247YdzVnzma+gdAQE2igIArX/tm8u2L8qxI7wLodTPE7I0tt2kR9dk
TOf2hErveGvC7idE6bQcs2yppL5ujZdjLnS8SlpfWb7+tpDwove2dA==
=ko5e
-----END PGP MESSAGE-----

--pgp-sign-Multipart_Tue_Jan_25_09:53:29_2000-1--

From bouncefilter Tue Jan 25 10:34:02 2000
Received: from bologna.nettuno.it (bologna.nettuno.it [193.43.2.1])
by hub.org (8.9.3/8.9.3) with ESMTP id KAA85668
for <pgsql-interfaces@postgresql.org>;
Tue, 25 Jan 2000 10:33:41 -0500 (EST)
(envelope-from jose@sferacarta.com)
Received: from proxy.sferacarta.com (sfcabop1.nettuno.it [193.207.10.213])
by bologna.nettuno.it (8.9.3/8.9.3/NETTuno 4.1) with ESMTP id QAA24620;
Tue, 25 Jan 2000 16:33:33 +0100 (MET)
Received: from rosso.sferacarta.com (sferacarta.com) [10.20.30.5]
by proxy.sferacarta.com with esmtp (Exim 2.05 #1 (Debian))
id 12D8ci-00040v-00; Tue, 25 Jan 2000 16:15:20 +0000
Message-ID: <388DBEC0.1232C65E@sferacarta.com>
Date: Tue, 25 Jan 2000 16:18:25 +0100
From: Jose Soares <jose@sferacarta.com>
X-Mailer: Mozilla 4.6 [en] (WinNT; I)
X-Accept-Language: en
MIME-Version: 1.0
To: webmaster@advl.org
CC: pgsql-interfaces@postgresql.org
Subject: Re: [INTERFACES] Re: ODBC drive strange behavior
References: <38881FC3.D34B1BEC@sferacarta.com> <00012415205702.01357@sultan>
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit

Compte utilisateur Sultan-advl wrote:

Le ven, 21 jan 2000, vous avez ���crit :

%_Sorry I forgot to send the attachement :)

Jose Soares wrote:

Data base server: PostgreSQL v6.5.2
OS server: Linux 2.0.37 (Debian)
Win Client: M$_Access95
PsqlODBC v6.40.0006
log file attached.

Any help would be very apreciated.

Jos���

In my case using M$_Access97 without problem.
a)Try to upgrade to PsqlODBC v6.40.0007

Is there a new release? I can't find it at:
http://www.insightdist.com/psqlodbc/
Is there another site for psqlODBC ?

b)Try to tune your odbc DSN with the help of PsqlODBC FAQ

c)Try to upgrade MDAC (odbc32 ==> About) in control panel
Regards.
Emmanuel

Jos���

From bouncefilter Tue Jan 25 10:46:50 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id KAA90313
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 10:46:48 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id HAA13882;
Tue, 25 Jan 2000 07:43:48 -0800 (PST)
Message-Id: <3.0.1.32.20000125072432.00f7c1e0@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Tue, 25 Jan 2000 07:24:32 -0800
To: Tom Lane <tgl@sss.pgh.pa.us>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: [HACKERS] Happy column dropping
Cc: "Hiroshi Inoue" <Inoue@tpf.co.jp>, "Peter Eisentraut" <peter_e@gmx.net>,
"PostgreSQL Development" <pgsql-hackers@postgreSQL.org>
In-Reply-To: <12015.948778166@sss.pgh.pa.us>
References: <3.0.1.32.20000124210143.0106b780@mail.pacifier.com>
<3.0.1.32.20000124184137.01069490@mail.pacifier.com>
<001001bf66d7$b531ba00$2801007e@tpf.co.jp>
<001001bf66d7$b531ba00$2801007e@tpf.co.jp>
<3.0.1.32.20000124184137.01069490@mail.pacifier.com>
<3.0.1.32.20000124210143.0106b780@mail.pacifier.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 12:29 AM 1/25/00 -0500, Tom Lane wrote:

Not pretending to speak for Peter, but --- I don't think there's anybody
here who hasn't got the highest standards in mind as an end goal. Yet
we have many miles to go, and we seldom agree on whether a particular
problem is the most critical next thing to work on. Each of us is doing
what comes to hand to be done. Sometimes we can fix a problem properly,
and sometimes we can only put a finger in the dike, or apply triage and
say "that's going to have to go unfixed a while longer".

Yes, I know, and I'm probably being a bit over-obnoxious.

Yup, it's a little disorganized, but that's both the curse and the
blessing of open-source development. Anybody who's *really* annoyed
by a particular problem is welcome to come and work on it.

Well...I'm trying to spend about four hours a week reading code, maybe
someday I'll get there :) Once we're done porting the arsDigita Community
System from Oracle to Posgres and once I'm done with my current contract,
I'll have more time to play.

I realize that at some point I need to do more than just whine about
things.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Tue Jan 25 10:47:50 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id KAA90418
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 10:47:11 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id HAA13901;
Tue, 25 Jan 2000 07:43:54 -0800 (PST)
Message-Id: <3.0.1.32.20000125073552.00f7a9b0@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Tue, 25 Jan 2000 07:35:52 -0800
To: Bruce Momjian <pgman@candle.pha.pa.us>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: [HACKERS] Happy column dropping
Cc: Peter Eisentraut <peter_e@gmx.net>, Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
In-Reply-To: <200001250529.AAA28416@candle.pha.pa.us>
References: <3.0.1.32.20000124211728.00f728b0@mail.pacifier.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 12:29 AM 1/25/00 -0500, Bruce Momjian wrote:

We have to be realistic. It is no good to have one command that has 100
fancy capabilities while 100 other commands are broken or are more
important.

Doesn't mean we don't do a great job, but that sometimes it is not worth
it considering the other things we can be doing.

If you are suggesting we don't follow that plan, I have to disagree.

In general, I don't disagree with this. But a drop column command that
kills all constraints on a table won't be terribly useful to folks. I'm
thinking of the fact that this same version will have referential
integrity constraints, which will be used by many. These will be dropped,
too, if I understand things correctly.

I think my emotional reaction is mostly to the fact that it was
put into sources that I presumed were to be released in beta form
just a few days afterwards. With no prior discussion.

From Peter's notes, it is clear that his perception of a beta version

might differ somewhat from that which has been traditional with the
postgres group. And that makes me very nervous, since I'm planning
to try to base further porting work on that beta. Obviously, I
don't need to use "drop column", but if the release model drifts more
towards the "break often, break early" model then then I'll have to
rethink my usage of Postgres betas.

(no, we won't release our port on the beta, we're just hoping that
the beta will be solid enough that we can work with it, and release
our beta on top of the resulting version of PG).

Now, couple this with all the problems associated with some apparently
below-par changes to libpq - which I absolutely depend on - and I start
worrying that the excellent stability of the 6.5 beta and subsequent
full releases might become an abberration rather than the norm.

I know folks don't want that...

Anyway, this perhaps is mostly a communication problem, as Peter
apparently thought that the Postgres group follows the "release early,
release often" model.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Tue Jan 25 10:39:50 2000
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
by hub.org (8.9.3/8.9.3) with ESMTP id KAA88998
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 10:39:12 -0500 (EST) (envelope-from Inoue@tpf.co.jp)
Received: from mcadnote1 (ppm118.noc.fukui.nsk.ne.jp [210.161.188.37])
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with SMTP
id AAA06489; Wed, 26 Jan 2000 00:39:03 +0900
From: "Hiroshi Inoue" <Inoue@tpf.co.jp>
To: "Tom Lane" <tgl@sss.pgh.pa.us>
Cc: <pgsql-hackers@postgreSQL.org>
Subject: RE: [HACKERS] Sure enough, SI buffer overrun is broken
Date: Wed, 26 Jan 2000 00:40:00 +0900
Message-ID: <NDBBIJLOILGIKBGDINDFEEFDCCAA.Inoue@tpf.co.jp>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-2022-jp"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
Importance: Normal
In-Reply-To: <11298.948769565@sss.pgh.pa.us>
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300

-----Original Message-----
From: owner-pgsql-hackers@postgreSQL.org
[mailto:owner-pgsql-hackers@postgreSQL.org]On Behalf Of Tom Lane

I built the current sources with MAXNUMMESSAGES set to 32 in
src/include/storage/sinvaladt.h. The regular regress tests
run OK, with just a few NOTICEs about 'cache state reset'
and 'SI buffer overflow' inserted in the normal outputs
(as you'd expect, if SI overrun occurs).

However, the parallel tests crash spectacularly, with weird errors
and Assert() coredumps. Some of the unexpected messages in the
postmaster log are:

ERROR: Relation 0 does not exist
NOTICE: LockRelease: locktable lookup failed, no lock
TRAP: Failed Assertion("!(((file) > 0 && (file) < SizeVfdCache &&
VfdCache[file].fileName != ((void *)0))):", File: "fd.c", Line: 817)

!(((file) > 0 && (file) < SizeVfdCache && VfdCache[file].fileName
!= ((void *)0))) (0)
NOTICE: LockRelease: locktable lookup failed, no lock
TRAP: Failed Assertion("!(attnum <= 0 || (attnum - 1 <=
tuple_type->natts - 1 && tuple_type->attrs[attnum - 1] != ((void
*)0) && variable->vartype == tuple_type->attrs[attnum -
1]->atttypid)):", File: "execQual.c", Line: 283)

!(attnum <= 0 || (attnum - 1 <= tuple_type->natts - 1 &&
tuple_type->attrs[attnum - 1] != ((void *)0) && variable->vartype
== tuple_type->attrs[attnum - 1]->atttypid)) (0) [Not a typewriter]
TRAP: Failed Assertion("!(((file) > 0 && (file) < SizeVfdCache &&
VfdCache[file].fileName != ((void *)0))):", File: "fd.c", Line: 817)

!(((file) > 0 && (file) < SizeVfdCache && VfdCache[file].fileName
!= ((void *)0))) (0) [Not a typewriter]

We have a problem.

I think Hiroshi was beating on this code recently --- Hiroshi,
do you recall anything you might have done that would affect
SI cache reset recovery?

Certainly crash occurs.
But I couldn't see such Assert messages.

OK,I will examine tomorrow.

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp

From bouncefilter Tue Jan 25 10:44:51 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id KAA89867
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 10:44:42 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id HAA13917;
Tue, 25 Jan 2000 07:43:56 -0800 (PST)
Message-Id: <3.0.1.32.20000125074138.00f752c0@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Tue, 25 Jan 2000 07:41:38 -0800
To: Philip Warner <pjw@rhyme.com.au>, Tom Lane <tgl@sss.pgh.pa.us>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: [HACKERS] DISTINCT ON: speak now or forever hold your
peace
Cc: Chris Bitmead <chris@bitmead.com>, pgsql-hackers@postgreSQL.org
In-Reply-To: <3.0.5.32.20000125170026.0357d8f0@mail.rhyme.com.au>
References: <3.0.1.32.20000124212022.00f70720@mail.pacifier.com>
<11879.948776470@sss.pgh.pa.us>
<3.0.5.32.20000125150557.036121c0@mail.rhyme.com.au>
<2295.948765015@sss.pgh.pa.us>
<3.0.5.32.20000125140349.035768d0@mail.rhyme.com.au>
<3.0.5.32.20000125150557.036121c0@mail.rhyme.com.au>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 05:00 PM 1/25/00 +1100, Philip Warner wrote:

Out of curiosity, does the SQL spec give any rules or guidelines about when
aggregates should be applied to resultant rows? Or is it one of the
implementation-dependant things?

Well...my copy of Date's now actually in the mail, rather than in
Boston, but at the moment that's not much help!

"order by" happens after everything, AFAIK. For instance...

select ...
union
select ...
order by

orders the result of the union, which pretty much implies that
aggregates will happen first.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Tue Jan 25 11:12:51 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA98967
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 11:11:54 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id IAA21529;
Tue, 25 Jan 2000 08:10:51 -0800 (PST)
Message-Id: <3.0.1.32.20000125075353.00f7e720@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Tue, 25 Jan 2000 07:53:53 -0800
To: Peter Eisentraut <peter_e@gmx.net>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: [HACKERS] Happy column dropping
Cc: Bruce Momjian <pgman@candle.pha.pa.us>, Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
In-Reply-To: <Pine.GSO.4.02A.10001251139220.11899-100000@Val.DoCS.UU.SE>
References: <3.0.1.32.20000124204431.007b3940@mail.pacifier.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 11:41 AM 1/25/00 +0100, Peter Eisentraut wrote:

On Mon, 24 Jan 2000, Don Baccus wrote:

Dropping constraints on a table just because you drop a column is
just butt-ugly.

Hey, nobody said that this was the final version. Keeping the constraints
was a trivial step from what there was. I guess the lesson I learned was
that around here you Release Late, Release Rarely, and that's fine, but I
didn't know that. Sorry. -> next time ;)

Yes, I've thought about this and as I mentioned in another note a few
minutes ago, realize now that you believed that to be the release model.

I now understand why you were willing to dump a change like this into
the sources a few days before a planned beta release. Perfectly
reasonable under the more agressive release early, release often
model.

I'll back off now.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Tue Jan 25 10:54:52 2000
Received: from orion3.ori-ind.com (IDENT:root@[206.81.197.190])
by hub.org (8.9.3/8.9.3) with ESMTP id KAA91964
for <pgsql-interfaces@postgreSQL.org>;
Tue, 25 Jan 2000 10:54:46 -0500 (EST) (envelope-from ken@ori-ind.com)
Received: from ken (stimpy.ori-ind.com [192.168.61.96])
by orion3.ori-ind.com (8.8.7/8.8.7) with SMTP id IAA13297;
Tue, 25 Jan 2000 08:03:11 -0800
Message-Id: <3.0.1.32.20000125075505.00970910@ori-ind.com>
X-Sender: ken@ori-ind.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Tue, 25 Jan 2000 07:55:05 -0800
To: Jose Soares <jose@sferacarta.com>
From: "Ken J. Wright" <ken@ori-ind.com>
Subject: Re: [INTERFACES] Re: ODBC drive strange behavior
Cc: pgsql-interfaces@postgreSQL.org
In-Reply-To: <388DBEC0.1232C65E@sferacarta.com>
References: <38881FC3.D34B1BEC@sferacarta.com>
<00012415205702.01357@sultan>
Mime-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by hub.org id KAA91974

At 16:18 01/25/2000 +0100, Jose Soares wrote:

Compte utilisateur Sultan-advl wrote:

Le ven, 21 jan 2000, vous avez ���crit :

%_Sorry I forgot to send the attachement :)

Jose Soares wrote:

Data base server: PostgreSQL v6.5.2
OS server: Linux 2.0.37 (Debian)
Win Client: M$_Access95
PsqlODBC v6.40.0006
log file attached.

Any help would be very apreciated.

Jos���

In my case using M$_Access97 without problem.
a)Try to upgrade to PsqlODBC v6.40.0007

Is there a new release? I can't find it at:
http://www.insightdist.com/psqlodbc/
Is there another site for psqlODBC ?

An old message from Byron:

Hi,

I've posted a new odbc driver at the following locations:

ftp://ftp.postgresql.org/pub/odbc/

AND

http://members.home.net/byron.nikolaidis (my personal homepage). Note
I have disabled downloading the full install version because of
bandwidth limitations on my ISP, but the source and dll and everything
else is there.

I'm not sure if there is a link from the postgres web site to the driver
yet.

If there are any problems, with the driver or the downloading or html
stuff, please let me know.

Byron
version.txt date: 09/02/99 version: 06.40.0007
----------------------------------------------------------------------
This file describes changes and or fixes since the last update.

NEW FEATURES:

none

BUG_FIXES:

1. Fix all info functions to check whether the table is a view.

2. Fix for Access 2000 (unconfirmed)

3. Change most statement errors to General Error S1000 instead of 08S01
(communication link failure).

From bouncefilter Tue Jan 25 11:11:51 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA98796
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 11:11:21 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id IAA21536;
Tue, 25 Jan 2000 08:10:52 -0800 (PST)
Message-Id: <3.0.1.32.20000125080125.00f7f160@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Tue, 25 Jan 2000 08:01:25 -0800
To: Peter Eisentraut <peter_e@gmx.net>, Hiroshi Inoue <Inoue@tpf.co.jp>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column
dropping)
Cc: PostgreSQL Development <pgsql-hackers@postgreSQL.org>
In-Reply-To: <Pine.GSO.4.02A.10001251152160.11899-100000@Val.DoCS.UU.SE>
References: <001401bf6704$5ca7e3a0$2801007e@tpf.co.jp>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 11:55 AM 1/25/00 +0100, Peter Eisentraut wrote:

On Tue, 25 Jan 2000, Hiroshi Inoue wrote:

Even default is not allowed in ADD COLUMN now.
There may be other reasons why they aren't allowed.

It's not a matter of *allowed*, it's a parsing deficiency. The fact that
there was a default declared gets silently ignored. If y'all allow ;) I
would like to fix that (have already started a bit) by perusing the code
in parse_func.c:transformCreateStmt and do the same for the alter table
add column part. Maybe and add/drop constraint will come out in the end as
well.

However, heap_getattr still won't see the default since it simply
checks to see of the attribute number falls off the end of the
tuple and then returns null.

There's no provision for then pulling out the default value and
returning it instead.

I think this is why Tom was implying that add column should really
alter the table?

A fully-featured "add column" feature would be very nice, indeed.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Tue Jan 25 11:05:51 2000
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA97666
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 11:05:37 -0500 (EST) (envelope-from Inoue@tpf.co.jp)
Received: from mcadnote1 (ppm151.noc.fukui.nsk.ne.jp [210.161.188.70])
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with SMTP
id BAA06502; Wed, 26 Jan 2000 01:05:25 +0900
From: "Hiroshi Inoue" <Inoue@tpf.co.jp>
To: "Peter Eisentraut" <peter_e@gmx.net>
Cc: "PostgreSQL Development" <pgsql-hackers@postgreSQL.org>
Subject: RE: Happy column adding (was RE: [HACKERS] Happy column dropping)
Date: Wed, 26 Jan 2000 01:06:22 +0900
Message-ID: <NDBBIJLOILGIKBGDINDFCEFHCCAA.Inoue@tpf.co.jp>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
Importance: Normal
In-Reply-To: <Pine.GSO.4.02A.10001251152160.11899-100000@Val.DoCS.UU.SE>
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300

-----Original Message-----
From: Peter Eisentraut [mailto:e99re41@DoCS.UU.SE]

On Tue, 25 Jan 2000, Hiroshi Inoue wrote:

Even default is not allowed in ADD COLUMN now.
There may be other reasons why they aren't allowed.

It's not a matter of *allowed*, it's a parsing deficiency. The fact that
there was a default declared gets silently ignored. If y'all allow ;) I
would like to fix that (have already started a bit) by perusing the code
in parse_func.c:transformCreateStmt and do the same for the alter table
add column part. Maybe and add/drop constraint will come out in the end as
well.

IIRC,there were some reason that default for new column had been rejected.
It may be possible now.
Probably Tom knows it.

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp

From bouncefilter Tue Jan 25 11:59:52 2000
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67] (may
be forged)) by hub.org (8.9.3/8.9.3) with ESMTP id LAA10594
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 11:59:17 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
LAA17646;
Tue, 25 Jan 2000 11:35:20 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001251635.LAA17646@candle.pha.pa.us>
Subject: Re: [HACKERS] Happy column dropping
In-Reply-To: <3.0.1.32.20000125075353.00f7e720@mail.pacifier.com> from Don
Baccus at "Jan 25, 2000 07:53:53 am"
To: Don Baccus <dhogaza@pacifier.com>
Date: Tue, 25 Jan 2000 11:35:20 -0500 (EST)
CC: Peter Eisentraut <peter_e@gmx.net>, Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

At 11:41 AM 1/25/00 +0100, Peter Eisentraut wrote:

On Mon, 24 Jan 2000, Don Baccus wrote:

Dropping constraints on a table just because you drop a column is
just butt-ugly.

Hey, nobody said that this was the final version. Keeping the constraints
was a trivial step from what there was. I guess the lesson I learned was
that around here you Release Late, Release Rarely, and that's fine, but I
didn't know that. Sorry. -> next time ;)

Yes, I've thought about this and as I mentioned in another note a few
minutes ago, realize now that you believed that to be the release model.

I now understand why you were willing to dump a change like this into
the sources a few days before a planned beta release. Perfectly
reasonable under the more agressive release early, release often
model.

Maybe I have to get that into the developers FAQ.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Tue Jan 25 11:59:51 2000
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67] (may
be forged)) by hub.org (8.9.3/8.9.3) with ESMTP id LAA10577
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 11:59:06 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
LAA17666;
Tue, 25 Jan 2000 11:36:09 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001251636.LAA17666@candle.pha.pa.us>
Subject: Re: [HACKERS] Happy column dropping
In-Reply-To: <3.0.1.32.20000125073552.00f7a9b0@mail.pacifier.com> from Don
Baccus at "Jan 25, 2000 07:35:52 am"
To: Don Baccus <dhogaza@pacifier.com>
Date: Tue, 25 Jan 2000 11:36:09 -0500 (EST)
CC: Peter Eisentraut <peter_e@gmx.net>, Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Now, couple this with all the problems associated with some apparently
below-par changes to libpq - which I absolutely depend on - and I start
worrying that the excellent stability of the 6.5 beta and subsequent
full releases might become an abberration rather than the norm.

I know folks don't want that...

Anyway, this perhaps is mostly a communication problem, as Peter
apparently thought that the Postgres group follows the "release early,
release often" model.

There are no plans to change the way we release -- release rarely,
release stable.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Tue Jan 25 11:37:52 2000
Received: from wallace.ece.rice.edu (root@wallace.ece.rice.edu
[128.42.12.154])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA07568
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 11:37:23 -0500 (EST)
(envelope-from reedstrm@wallace.ece.rice.edu)
Received: by wallace.ece.rice.edu via sendmail from stdin
id <m12D8xr-000LEMC@wallace.ece.rice.edu> (Debian Smail3.2.0.102)
for pgsql-hackers@postgresql.org; Tue, 25 Jan 2000 10:37:11 -0600 (CST)
Date: Tue, 25 Jan 2000 10:37:11 -0600
From: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
To: PostgreSQL Development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Happy column dropping
Message-ID: <20000125103711.B423@rice.edu>
References: <Pine.GSO.4.02A.10001251135570.11899-100000@Val.DoCS.UU.SE>
<200001251414.JAA14452@candle.pha.pa.us>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
User-Agent: Mutt/1.0i
In-Reply-To: <200001251414.JAA14452@candle.pha.pa.us>;
from pgman@candle.pha.pa.us on Tue, Jan 25, 2000 at 09:14:29AM
-0500

On Tue, Jan 25, 2000 at 09:14:29AM -0500, Bruce Momjian wrote:

On Mon, 24 Jan 2000, Bruce Momjian wrote:

New file name will not be just oid. Too hard to administer.

Ingres has table names as numbered files. It is a pain to figure out
which files match which tables. If you need to restore a table from
tape, the pg_class entry is gone and you have no way to figure out the
right table. When analyzing disk space, figuring who is using the space
is a pain.

Hmm, how about a map file, of conventional name, kept in the pgsql/data
dir, that contains filename -> db/tablename mappings? It would be
essentially a pretty printed dump of pg_class. That way, the admin
has access to the mapping even when the postmaster is down. Even to
restore from tape: grab pg_class_map from the tape (you did dump it
to the beginning, right?) and then grab the file you need. I don't see
this as being a particularly large file, in any case, and DDL isn't a
speed critical path, so rewriting the pg_class_map file wouldn't hurt.
And, since it's just for human/admin consumption, so major problem if
it gets out of sync: just regenerate it from pg_class. Could even be
used to sanity check the DBMS on start up: generate a new pg_class_map,
compare it to the old: if they don't match, fire a warning/die?

I'd suggest a format that's easily machine by awk/sed/grep (perl, etc.)
for those db admin tasks Bruce is talking about (space, etc.) Heck,
it'd be easy to whip up a 'generate a bunch of symlinks' script
to get the tablenames back, if you really need them.

Ross
--
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu> Tel. 713-348-6166
NSBRI Research Scientist/Programmer Fax 713-348-6182
Computer and Information Technology Institute
Rice University, 6100 S. Main St., Houston, TX 77005

Julian Scarfe <jscarfe@callnetuk.com> writes:

I use 'distinct on' to get the most recent reports for each of a group of
locations. E.g.:

select distinct on location * from reports where location ~~ 'Lond%'
order by location, reporttime desc;

to get the tuples that offer the most recent reports for each of London,
Londonderry, Londy etc.

Is there an alternative?

select location, max(report_time) from reports group by location

But I want report_text *too* -- that's the important bit, and it's not an
aggregate or common column in the group.

Right. That is the motivation that was given for keeping DISTINCT ON
and trying to clean it up: if you ORDER BY the DISTINCT field plus
enough other fields to give a unique ordering, and then rely on the
(undocumented) implementation property that you'll get the first tuple
out of each group with the same DISTINCT field, then you can get useful
effects like this one.

It's a kluge, and it's not readily extensible to cases such as needing to
apply the DISTINCT filter across two fields, but it does solve problems
that you can't straightforwardly rewrite into GROUP BY.

Tell you what: there are two things that are really bugging me about
DISTINCT ON. If I could get consensus on patching these things,
I'd be willing to do that rather than yank it.

One gripe is the lame-brain syntax: because there is no separator
between the DISTINCT ON column name and the first select targetlist
item, the DISTINCT ON target cannot be anything more complex than an
unadorned column name. For example "SELECT DISTINCT ON A+B+C, ..."
must be interpreted as DISTINCT ON A with a first target of +B+C;
if we tried to allow more complex expressions then it becomes ambiguous
whether that was meant or DISTINCT ON A+B with a first target of +C.
To fix this we need some kind of syntactic separator. The cleanest
idea that comes to my mind is to require parentheses around the ON
target:
SELECT DISTINCT ON (expression) target1, ...
One immediate advantage of allowing an expression is that you can do
distinct-on-two-fields in a rather klugy way, eg
SELECT DISTINCT ON (field1 || ' ' || field2) ...
We might someday extend it to allow multiple DISTINCT fields, eg,
SELECT DISTINCT ON (expr1 [, expr2 ...]) target1, ...
but I'm not promising to do that now (unless it's really easy ;-)).

The other gripe is the bad interaction with ORDER BY, as was illustrated
in my example of last night: if you ORDER BY something that's not the
DISTINCT field, it doesn't work. We could get something that kind of
works by applying two sorting passes: first sort by the DISTINCT field,
then run the duplicate-elimination filter, then re-sort by the ORDER BY
field(s). The trouble with that is that it means the user no longer has
control of the sort order within the same-DISTINCT-field-value groups
during the first sort phase, so there's no way for him to control which
tuple gets kept from each group. What I'd prefer to do is put in an
error check that says "if you use both DISTINCT ON and ORDER BY, then
the DISTINCT ON expression must be the first ORDER BY item". That way
the ORDER BY ordering can be the same one used for the
duplicate-elimination pass, and we don't have the semantic problem.
Note that Julian's example meets this constraint.

(BTW, ordinary SELECT DISTINCT has this same sort of problem if you try
to ORDER BY an expression that doesn't appear in the target list.
SQL92 avoids the issue by not allowing you to ORDER BY expressions that
aren't in the target list, period. We do allow that --- but not when
you use DISTINCT. Essentially, I want to enforce that same restriction
for DISTINCT ON.)

The other piece of the puzzle would be to document that DISTINCT ON
keeps the first tuple out of each set with the same DISTINCT ON value.

Does that sound like a plan?

regards, tom lane

From bouncefilter Tue Jan 25 12:24:52 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id MAA18094
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 12:24:18 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id MAA14036;
Tue, 25 Jan 2000 12:23:15 -0500 (EST)
To: "Hiroshi Inoue" <Inoue@tpf.co.jp>
cc: "Peter Eisentraut" <peter_e@gmx.net>,
"PostgreSQL Development" <pgsql-hackers@postgreSQL.org>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column dropping)
In-reply-to: <NDBBIJLOILGIKBGDINDFCEFHCCAA.Inoue@tpf.co.jp>
References: <NDBBIJLOILGIKBGDINDFCEFHCCAA.Inoue@tpf.co.jp>
Comments: In-reply-to "Hiroshi Inoue" <Inoue@tpf.co.jp>
message dated "Wed, 26 Jan 2000 01:06:22 +0900"
Date: Tue, 25 Jan 2000 12:23:15 -0500
Message-ID: <14033.948820995@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

"Hiroshi Inoue" <Inoue@tpf.co.jp> writes:

Even default is not allowed in ADD COLUMN now.

It's not a matter of *allowed*, it's a parsing deficiency. The fact that
there was a default declared gets silently ignored.

IIRC,there were some reason that default for new column had been rejected.

Well, yeah: wouldn't you expect that "ADD COLUMN x DEFAULT 42" would
cause every row currently existing in the table to acquire x = 42,
rather than x = NULL? In fact that would *have* to happen to allow
constraints to be added; consider ADD COLUMN x DEFAULT 42 NOT NULL.

The only way to make that happen is for ADD COLUMN to switch over to
an implementation that rewrites all the tuples. Which I think is the
right way to go ... but per this discussion, it's not a trivial fix.

regards, tom lane

From bouncefilter Tue Jan 25 12:39:52 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id MAA24122
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 12:39:42 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id MAA14181;
Tue, 25 Jan 2000 12:39:25 -0500 (EST)
To: Don Baccus <dhogaza@pacifier.com>
cc: Peter Eisentraut <peter_e@gmx.net>, Hiroshi Inoue <Inoue@tpf.co.jp>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column dropping)
In-reply-to: <3.0.1.32.20000125080125.00f7f160@mail.pacifier.com>
References: <001401bf6704$5ca7e3a0$2801007e@tpf.co.jp>
<3.0.1.32.20000125080125.00f7f160@mail.pacifier.com>
Comments: In-reply-to Don Baccus <dhogaza@pacifier.com>
message dated "Tue, 25 Jan 2000 08:01:25 -0800"
Date: Tue, 25 Jan 2000 12:39:25 -0500
Message-ID: <14178.948821965@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Don Baccus <dhogaza@pacifier.com> writes:

However, heap_getattr still won't see the default since it simply
checks to see of the attribute number falls off the end of the
tuple and then returns null.

Right.

There's no provision for then pulling out the default value and
returning it instead.

Would it even be possible to do that? I'm not sure that heap_getattr
has any way to find the default. It only has a TupleDesc, not a
reference to the relation itself...

regards, tom lane

From bouncefilter Tue Jan 25 12:45:54 2000
Received: from wallace.ece.rice.edu (root@wallace.ece.rice.edu
[128.42.12.154])
by hub.org (8.9.3/8.9.3) with ESMTP id MAA24839
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 12:44:54 -0500 (EST)
(envelope-from reedstrm@wallace.ece.rice.edu)
Received: by wallace.ece.rice.edu via sendmail from stdin
id <m12DA1N-000LEMC@wallace.ece.rice.edu> (Debian Smail3.2.0.102)
for pgsql-hackers@postgresql.org; Tue, 25 Jan 2000 11:44:53 -0600 (CST)
Date: Tue, 25 Jan 2000 11:44:53 -0600
From: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
To: PostgreSQL Development <pgsql-hackers@postgresql.org>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column dropping)
Message-ID: <20000125114453.E423@rice.edu>
References: <001401bf6704$5ca7e3a0$2801007e@tpf.co.jp>
<Pine.GSO.4.02A.10001251152160.11899-100000@Val.DoCS.UU.SE>
<3.0.1.32.20000125080125.00f7f160@mail.pacifier.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
User-Agent: Mutt/1.0i
In-Reply-To: <3.0.1.32.20000125080125.00f7f160@mail.pacifier.com>;
from dhogaza@pacifier.com on Tue, Jan 25, 2000 at 08:01:25AM -0800

On Tue, Jan 25, 2000 at 08:01:25AM -0800, Don Baccus wrote:

At 11:55 AM 1/25/00 +0100, Peter Eisentraut wrote:

On Tue, 25 Jan 2000, Hiroshi Inoue wrote:

Even default is not allowed in ADD COLUMN now.
There may be other reasons why they aren't allowed.

It's not a matter of *allowed*, it's a parsing deficiency. The fact that
there was a default declared gets silently ignored. If y'all allow ;) I
would like to fix that (have already started a bit) by perusing the code
in parse_func.c:transformCreateStmt and do the same for the alter table
add column part. Maybe and add/drop constraint will come out in the end as
well.

However, heap_getattr still won't see the default since it simply
checks to see of the attribute number falls off the end of the
tuple and then returns null.

Right, but that just means that existing tuples would not get the
default, but all new tuples would, right? So, while it does leave the
data inconsistent with the schema definition, it is fixable, and in a
controlled manner. A simple

UPDATE my_table SET new_field='default' WHERE new_field IS NULL;

should do it, right?

In fact, that's something I liked about the 'make invisible' strategy
for the ALTER DROP COLUMN case: it allows the DBA to control the
backends activity. If the DBA needs to drop a column from a large table,
but doesn't have space for 2X that table, what does she do? With the
invisible column, she could ALTER DROP, then do a series of updates,
similar to what Tom suggested:

UPDATE my_table SET otherfield=otherfield where table_id>0 and table_id<=100;
VACUUM mytable;
UPDATE my_table SET otherfield=otherfield where table_id>100 and table_id<=200;
VACUUM mytable;

etc.

Similarly, the aftermath of the ADD DEFAULT case could be handled in a
controlled manner, without forcing a 2X table size disk usage. I _like_
implementations that give the user (in this case, the DBA) control over
what happens, and when. I think this may answer Marc Fournier's desire
for a 'rewrite in place' version of these, since it would allow the
DBA, at their option, to update one tuple at a time (well, it'd be a royal
pain, but could be done...)

There's no provision for then pulling out the default value and
returning it instead.

Right, we don't want to special case it. What's wrong with returning a
NULL, for any tuple that hasn't been updated yet?

I think this is why Tom was implying that add column should really
alter the table?

A fully-featured "add column" feature would be very nice, indeed.

I agree, where full-featured means adding constraints. I disagree that
rewriting the entire table is a good idea. This isn't even only an edge
case for admins with little disk space. I could very easily imagine
a schema chamge on existing data, where one has a 'flatfile' sort of
table in the tens of gigabytes range, and you want to remove a column,
in order to normalize the table (get rid of address2, zip2, state2,
..., for example). Requiring transient diskspace of 2X the table,
and a complete rewrite, _as each column is dropped_ whould be really
annoying. Yes, I know, dump/restore, but what about 24/7 systems? Hey,
it's dangerous to do surgery on a live system, but sometimes, you have
little choice.

I realize I just slipped over from the ADD COLUMN to the DROP COLUMN
case, but I think fundamentally, they're very similar, and will probably
get implemented with the same mechanism.

So Bruce, would the inviso-columns really be a mess in the source? We've
already got the 'virtual column' case, with the ADD COLUMN aftermath.

Perhaps Oliver's suggestion of removing the link between logical and
physical field ordering might help. That could lead to some interesting
storage optimizations, as well. Collecting all variable types to the
end of a tuple, for example, might be a win.

Ross
--
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu>
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St., Houston, TX 77005

From bouncefilter Tue Jan 25 13:29:52 2000
Received: from wallace.ece.rice.edu (root@wallace.ece.rice.edu
[128.42.12.154])
by hub.org (8.9.3/8.9.3) with ESMTP id NAA36893
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 13:29:08 -0500 (EST)
(envelope-from reedstrm@wallace.ece.rice.edu)
Received: by wallace.ece.rice.edu via sendmail from stdin
id <m12DAiB-000LEMC@wallace.ece.rice.edu> (Debian Smail3.2.0.102)
for pgsql-hackers@postgresql.org; Tue, 25 Jan 2000 12:29:07 -0600 (CST)
Date: Tue, 25 Jan 2000 12:29:07 -0600
From: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
To: PostgreSQL Development <pgsql-hackers@postgresql.org>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column dropping)
Message-ID: <20000125122907.F423@rice.edu>
References: <NDBBIJLOILGIKBGDINDFCEFHCCAA.Inoue@tpf.co.jp>
<14033.948820995@sss.pgh.pa.us>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
User-Agent: Mutt/1.0i
In-Reply-To: <14033.948820995@sss.pgh.pa.us>;
from tgl@sss.pgh.pa.us on Tue, Jan 25, 2000 at 12:23:15PM -0500

On Tue, Jan 25, 2000 at 12:23:15PM -0500, Tom Lane wrote:

"Hiroshi Inoue" <Inoue@tpf.co.jp> writes:

Even default is not allowed in ADD COLUMN now.

It's not a matter of *allowed*, it's a parsing deficiency. The fact that
there was a default declared gets silently ignored.

IIRC,there were some reason that default for new column had been rejected.

Well, yeah: wouldn't you expect that "ADD COLUMN x DEFAULT 42" would
cause every row currently existing in the table to acquire x = 42,
rather than x = NULL? In fact that would *have* to happen to allow
constraints to be added; consider ADD COLUMN x DEFAULT 42 NOT NULL.

Actually, no I wouldn't expect it. That's mixing DDL and DML in one
statement. I expect the ALTER command to be pure DDL, and the UPDATE
to be pure DML.

For a detailed brain dump as I thought about this, see my reply to Don Baccus,
one message up. ;-)

Yes, I know the standard isn't that pure: SELECT INTO, for example. Hmm,
is SELECT INTO standard? <FX: Ross grovels through the draft standard text...>

Ouch, reading standards always makes my brain hurt. Especially how you
have to read them upside down. Turns out SELECT INTO is in the standard,
but not the way we implement it.

13.5 <select statement: single row>

Function

Retrieve values from a specified row of a table.

Format

<select statement: single row> ::=
SELECT [ <set quantifier> ] <select list>
INTO <select target list>
<table expression>

<select target list> ::=
<target specification> [ { <comma> <target specification> }... ]

and in section 6.2:

<target specification> ::=
<parameter specification>
| <variable specification>

<simple target specification> ::=
<parameter name>
| <embedded variable name>

<parameter specification> ::=
<parameter name> [ <indicator parameter> ]

and in section 5.4:

<parameter name> ::= <colon> <identifier>

So, it looks like SELECT INTO is a single row query that fills variables,
either parameters or embedded, not a combined 'create this table and
fill it' command. Oops!

Ross

--
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu>
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St., Houston, TX 77005

From bouncefilter Tue Jan 25 13:49:53 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id NAA43531
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 13:49:26 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id NAA14334;
Tue, 25 Jan 2000 13:49:01 -0500 (EST)
To: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
cc: PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column dropping)
In-reply-to: <20000125114453.E423@rice.edu>
References: <001401bf6704$5ca7e3a0$2801007e@tpf.co.jp>
<Pine.GSO.4.02A.10001251152160.11899-100000@Val.DoCS.UU.SE>
<3.0.1.32.20000125080125.00f7f160@mail.pacifier.com>
<20000125114453.E423@rice.edu>
Comments: In-reply-to "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
message dated "Tue, 25 Jan 2000 11:44:53 -0600"
Date: Tue, 25 Jan 2000 13:49:01 -0500
Message-ID: <14330.948826141@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

"Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu> writes:

Perhaps Oliver's suggestion of removing the link between logical and
physical field ordering might help. That could lead to some interesting
storage optimizations, as well. Collecting all variable types to the
end of a tuple, for example, might be a win.

I recall seeing comments in the source to the effect that this was
considered (but never implemented) long ago. It does seem like a
very clean solution to some aspects of the add/drop column problem;
but implementation would be a pain in the neck. We'd have to go through
all of the source code and decide whether each use of an attribute
number should be using logical or physical number. It'd be a long slog.

It occurs to me that in at least some of the places where attribute
numbers are currently used, we ought to use *neither* logical nor
physical column position, but rather a permanent unique ID --- the
attribute tuple's OID would work, if it's assigned soon enough to be
used for constraints given in CREATE TABLE. (Otherwise we could assign
"column serial numbers" that count from 1 for each relation, but are
never changed or recycled within the relation.)

In particular, if parsetrees for stored rules and constraints worked
that way, renumbering attributes that follow the added/dropped column
would become a lot less painful.

regards, tom lane

From bouncefilter Tue Jan 25 14:23:53 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id OAA52775
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 14:23:13 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id LAA04677;
Tue, 25 Jan 2000 11:22:10 -0800 (PST)
Message-Id: <3.0.1.32.20000125105542.00f850c0@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Tue, 25 Jan 2000 10:55:42 -0800
To: Bruce Momjian <pgman@candle.pha.pa.us>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: [HACKERS] Happy column dropping
Cc: Peter Eisentraut <peter_e@gmx.net>, Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
In-Reply-To: <200001251636.LAA17666@candle.pha.pa.us>
References: <3.0.1.32.20000125073552.00f7a9b0@mail.pacifier.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 11:36 AM 1/25/00 -0500, Bruce Momjian wrote:

There are no plans to change the way we release -- release rarely,
release stable.

Yes, I understand that now, etc etc.

I'm ready to put this thread to bed, myself. Peter, I didn't mean
my criticisms to be personal in any way. My apologies if they
came off that way.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Tue Jan 25 14:23:54 2000
Received: from clio.trends.ca (root@clio.trends.ca [209.47.148.2])
by hub.org (8.9.3/8.9.3) with ESMTP id OAA52730
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 14:23:04 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by clio.trends.ca (8.9.3+Sun/8.9.3) with ESMTP id OAA26904
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 14:22:41 -0500 (EST)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id LAA04697;
Tue, 25 Jan 2000 11:22:12 -0800 (PST)
Message-Id: <3.0.1.32.20000125110315.00f864f0@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Tue, 25 Jan 2000 11:03:15 -0800
To: Tom Lane <tgl@sss.pgh.pa.us>, "Hiroshi Inoue" <Inoue@tpf.co.jp>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column
dropping)
Cc: "Peter Eisentraut" <peter_e@gmx.net>,
"PostgreSQL Development" <pgsql-hackers@postgreSQL.org>
In-Reply-To: <14033.948820995@sss.pgh.pa.us>
References: <NDBBIJLOILGIKBGDINDFCEFHCCAA.Inoue@tpf.co.jp>
<NDBBIJLOILGIKBGDINDFCEFHCCAA.Inoue@tpf.co.jp>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 12:23 PM 1/25/00 -0500, Tom Lane wrote:

The only way to make that happen is for ADD COLUMN to switch over to
an implementation that rewrites all the tuples. Which I think is the
right way to go ... but per this discussion, it's not a trivial fix.

Might it be possible to keep the current (ummm) hack for simple
add column name type and only rewrite for fancy cases? The current
implementation is awfully fast and there's the advantage of not
needing sufficient disk storage to duplicate the table.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Tue Jan 25 14:22:54 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id OAA52673
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 14:22:41 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id LAA04707;
Tue, 25 Jan 2000 11:22:12 -0800 (PST)
Message-Id: <3.0.1.32.20000125110543.00f82ec0@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Tue, 25 Jan 2000 11:05:43 -0800
To: Tom Lane <tgl@sss.pgh.pa.us>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column
dropping)
Cc: Peter Eisentraut <peter_e@gmx.net>, Hiroshi Inoue <Inoue@tpf.co.jp>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
In-Reply-To: <14178.948821965@sss.pgh.pa.us>
References: <3.0.1.32.20000125080125.00f7f160@mail.pacifier.com>
<001401bf6704$5ca7e3a0$2801007e@tpf.co.jp>
<3.0.1.32.20000125080125.00f7f160@mail.pacifier.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 12:39 PM 1/25/00 -0500, Tom Lane wrote:

There's no provision for then pulling out the default value and
returning it instead.

Would it even be possible to do that? I'm not sure that heap_getattr
has any way to find the default. It only has a TupleDesc, not a
reference to the relation itself...

In that case, then presumably not. You've just pushed me beyond
the limits of my understanding of that code, I'm afraid!

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Tue Jan 25 14:23:01 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id OAA52671
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 14:22:41 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id LAA04722;
Tue, 25 Jan 2000 11:22:14 -0800 (PST)
Message-Id: <3.0.1.32.20000125112001.0106bec0@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Tue, 25 Jan 2000 11:20:01 -0800
To: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>,
PostgreSQL Development <pgsql-hackers@postgresql.org>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column
dropping)
In-Reply-To: <20000125122907.F423@rice.edu>
References: <14033.948820995@sss.pgh.pa.us>
<NDBBIJLOILGIKBGDINDFCEFHCCAA.Inoue@tpf.co.jp>
<14033.948820995@sss.pgh.pa.us>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 12:29 PM 1/25/00 -0600, Ross J. Reedstrom wrote:

On Tue, Jan 25, 2000 at 12:23:15PM -0500, Tom Lane wrote:

Well, yeah: wouldn't you expect that "ADD COLUMN x DEFAULT 42" would
cause every row currently existing in the table to acquire x = 42,
rather than x = NULL? In fact that would *have* to happen to allow
constraints to be added; consider ADD COLUMN x DEFAULT 42 NOT NULL.

Actually, no I wouldn't expect it. That's mixing DDL and DML in one
statement. I expect the ALTER command to be pure DDL, and the UPDATE
to be pure DML.

Hmmm...interesting...is alter table in the standard? Again, my copy
of Date's SQL 92 primer is somewhere 'wteen Boston, MA and Portland, OR,
so I can't look myself. Since you've got the standard available you
can answer perhaps?

Ouch, reading standards always makes my brain hurt. Especially how you
have to read them upside down. Turns out SELECT INTO is in the standard,
but not the way we implement it.

Scary!!! :) :)

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Tue Jan 25 14:35:53 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id OAA57974
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 14:34:49 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id LAA09053;
Tue, 25 Jan 2000 11:34:08 -0800 (PST)
Message-Id: <3.0.1.32.20000125113001.00f8acb0@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Tue, 25 Jan 2000 11:30:01 -0800
To: Tom Lane <tgl@sss.pgh.pa.us>,
"Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column
dropping)
Cc: PostgreSQL Development <pgsql-hackers@postgreSQL.org>
In-Reply-To: <14330.948826141@sss.pgh.pa.us>
References: <20000125114453.E423@rice.edu>
<001401bf6704$5ca7e3a0$2801007e@tpf.co.jp>
<Pine.GSO.4.02A.10001251152160.11899-100000@Val.DoCS.UU.SE>
<3.0.1.32.20000125080125.00f7f160@mail.pacifier.com>
<20000125114453.E423@rice.edu>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 01:49 PM 1/25/00 -0500, Tom Lane wrote:

It occurs to me that in at least some of the places where attribute
numbers are currently used, we ought to use *neither* logical nor
physical column position, but rather a permanent unique ID --- the
attribute tuple's OID would work, if it's assigned soon enough to be
used for constraints given in CREATE TABLE. (Otherwise we could assign
"column serial numbers" that count from 1 for each relation, but are
never changed or recycled within the relation.)

In particular, if parsetrees for stored rules and constraints worked
that way, renumbering attributes that follow the added/dropped column
would become a lot less painful.

Yes...I see what you're driving at. Very interesting idea. The stored
rules and constraints would in this case would still refer to the remaining
columns after a drop, right?

As far as moving from phsyical to logical attribute numbering, and
the long slog involved, it might be possible to work towards this
goal in background mode by providing two sets of access macros/functions
with the same semantics, and as folks have time to slowly work through the
code. This wouldn't break anything. Nor would it substitute for
an intense push to finish the job before switching to such a scheme,
and of course there would be many errors.

It might slowly lower the barrier towards switching, though. The cost
would be a confusing and incomplete separation during the transition
period.

Of course, moving along this path would be something to consider only
if it were decided that eventually such a switch will be made. And I'm
not at all sure I like it.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Tue Jan 25 14:53:54 2000
Received: from news.tht.net (news.hub.org [216.126.91.242])
by hub.org (8.9.3/8.9.3) with ESMTP id OAA61552
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 14:53:50 -0500 (EST)
(envelope-from news@news.tht.net)
Received: (from news@localhost) by news.tht.net (8.9.3/8.9.3) id OAA44449
for pgsql-hackers@postgresql.org; Tue, 25 Jan 2000 14:50:46 -0500 (EST)
(envelope-from news)
X-Authentication-Warning: news.tht.net: news set sender to <news> using -f
From: <daniels@1stconnect.com>
X-Newsgroups: comp.databases.postgresql.hackers
References: <pRPd4.14389$G55.169885@news1.rdc1.ab.home.com>
Subject: Re: Postgresql Perl Problem
Lines: 17
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 5.00.2314.1300
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
Message-ID: <e8nj4.355$yH.38413@tw12.nn.bcandid.com>
X-Trace: tw12.nn.bcandid.com 948829834 209.118.5.126 (Tue,
25 Jan 2000 12:50:34 MST)
Organization: bCandid - Powering the world's discussions - http://bCandid.com
Date: Tue, 25 Jan 2000 19:50:34 GMT
To: pgsql-hackers@postgresql.org

I believe the PG modules come with the distribution. However, if you
followed the instructions and installed postgres as and unpriveleged user
(postgres) then the install of pg.pm would fail since your Perl directory is
probably only writable by root. Go back into your distribution directory
and cd into "src/utilities" I don't have the structure in front of me but I
think there is a "Perl" directory there. cd into that, su to root, and run
"make install". That ought to do it. If you don't have root privleges then
you can either add the following line to your perl script:
use lib "/path/to/pg.m_directory";

Or you can call perl with the "-I" option followed by the above path. There
are other ways I'm sure. . .

Vince Daniels

From bouncefilter Tue Jan 25 15:33:54 2000
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67] (may
be forged)) by hub.org (8.9.3/8.9.3) with ESMTP id PAA74852
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 15:33:32 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
PAA20667;
Tue, 25 Jan 2000 15:08:15 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001252008.PAA20667@candle.pha.pa.us>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column dropping)
In-Reply-To: <3.0.1.32.20000125080125.00f7f160@mail.pacifier.com> from Don
Baccus at "Jan 25, 2000 08:01:25 am"
To: Don Baccus <dhogaza@pacifier.com>
Date: Tue, 25 Jan 2000 15:08:15 -0500 (EST)
CC: Peter Eisentraut <peter_e@gmx.net>, Hiroshi Inoue <Inoue@tpf.co.jp>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

At 11:55 AM 1/25/00 +0100, Peter Eisentraut wrote:

On Tue, 25 Jan 2000, Hiroshi Inoue wrote:

Even default is not allowed in ADD COLUMN now.
There may be other reasons why they aren't allowed.

It's not a matter of *allowed*, it's a parsing deficiency. The fact that
there was a default declared gets silently ignored. If y'all allow ;) I
would like to fix that (have already started a bit) by perusing the code
in parse_func.c:transformCreateStmt and do the same for the alter table
add column part. Maybe and add/drop constraint will come out in the end as
well.

However, heap_getattr still won't see the default since it simply
checks to see of the attribute number falls off the end of the
tuple and then returns null.

There's no provision for then pulling out the default value and
returning it instead.

I think this is why Tom was implying that add column should really
alter the table?

A fully-featured "add column" feature would be very nice, indeed.

Oh, so that is why ALTER TABLE can't have a NOT NULLL default. Makes
total sense.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Tue Jan 25 15:15:54 2000
Received: from mail.rdc1.md.home.com (imail@ha2.rdc1.md.home.com [24.2.2.67])
by hub.org (8.9.3/8.9.3) with ESMTP id PAA68134
for <pgsql-interfaces@postgreSQL.org>;
Tue, 25 Jan 2000 15:14:56 -0500 (EST)
(envelope-from byron.nikolaidis@home.com)
Received: from home.com ([24.4.141.52]) by mail.rdc1.md.home.com
(InterMail v4.01.01.00 201-229-111) with ESMTP
id <20000125201453.LHLZ29486.mail.rdc1.md.home.com@home.com>;
Tue, 25 Jan 2000 12:14:53 -0800
Message-ID: <388E0439.969DCD05@home.com>
Date: Tue, 25 Jan 2000 15:14:49 -0500
From: Byron Nikolaidis <byron.nikolaidis@home.com>
Organization: @Home Network
X-Mailer: Mozilla 4.5 [en]C-AtHome0405 (Win95; U)
X-Accept-Language: en
MIME-Version: 1.0
To: Jose Soares <jose@sferacarta.com>
CC: webmaster@advl.org, pgsql-interfaces@postgreSQL.org
Subject: Re: [INTERFACES] Re: ODBC drive strange behavior
References: <38881FC3.D34B1BEC@sferacarta.com> <00012415205702.01357@sultan>
<388DBEC0.1232C65E@sferacarta.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Jose Soares wrote:

Is there a new release? I can't find it at:
http://www.insightdist.com/psqlodbc/
Is there another site for psqlODBC ?

Yes, I have version 6.40.0007 on my own website. I'm not sure if it is
available elsewhere. The address is
http://members.home.net/byron.nikolaidis

Unfortunately, it does not include some recent patches that other people
have applied. I know of at least one with large objects.

I could put these in I suppose. Anybody know where these patches are?
Perhaps the author?

Byron

From bouncefilter Tue Jan 25 15:22:57 2000
Received: from wallace.ece.rice.edu (root@wallace.ece.rice.edu
[128.42.12.154])
by hub.org (8.9.3/8.9.3) with ESMTP id PAA69434
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 15:22:22 -0500 (EST)
(envelope-from reedstrm@wallace.ece.rice.edu)
Received: by wallace.ece.rice.edu via sendmail from stdin
id <m12DCTj-000LEMC@wallace.ece.rice.edu> (Debian Smail3.2.0.102)
for pgsql-hackers@postgresql.org; Tue, 25 Jan 2000 14:22:19 -0600 (CST)
Date: Tue, 25 Jan 2000 14:22:19 -0600
From: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
To: Don Baccus <dhogaza@pacifier.com>
Cc: PostgreSQL Development <pgsql-hackers@postgresql.org>
Subject: Re: Happy column adding and dropping
Message-ID: <20000125142219.A1169@rice.edu>
References: <14033.948820995@sss.pgh.pa.us>
<NDBBIJLOILGIKBGDINDFCEFHCCAA.Inoue@tpf.co.jp>
<14033.948820995@sss.pgh.pa.us> <20000125122907.F423@rice.edu>
<3.0.1.32.20000125112001.0106bec0@mail.pacifier.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
User-Agent: Mutt/1.0i
In-Reply-To: <3.0.1.32.20000125112001.0106bec0@mail.pacifier.com>;
from dhogaza@pacifier.com on Tue, Jan 25, 2000 at 11:20:01AM -0800

On Tue, Jan 25, 2000 at 11:20:01AM -0800, Don Baccus wrote:

At 12:29 PM 1/25/00 -0600, Ross J. Reedstrom wrote:

On Tue, Jan 25, 2000 at 12:23:15PM -0500, Tom Lane wrote:

Well, yeah: wouldn't you expect that "ADD COLUMN x DEFAULT 42" would
cause every row currently existing in the table to acquire x = 42,
rather than x = NULL? In fact that would *have* to happen to allow
constraints to be added; consider ADD COLUMN x DEFAULT 42 NOT NULL.

Actually, no I wouldn't expect it. That's mixing DDL and DML in one
statement. I expect the ALTER command to be pure DDL, and the UPDATE
to be pure DML.

Hmmm...interesting...is alter table in the standard? Again, my copy
of Date's SQL 92 primer is somewhere 'wteen Boston, MA and Portland, OR,
so I can't look myself. Since you've got the standard available you
can answer perhaps?

Gee, Don, that's a good idea, I should have thought of it myself! BTW,
what I have is labelled "Second Informal Review Draft" and is dated July
30, 1992. I don't know how it differs from the final standard, if at all.
I orginally got it from someone on this list, I forget who. Anyone
want a copy, I'll happily email it to you, or throw it on a website:
it's 1.6M of ASCII, ready for the line printer ;-) (273K gzipped.)

That said, ALTER is in fact described, and the ADD COLUMN case agrees
with Tom's expectations, rather than mine, see General Rule 2, below:

11.11 <add column definition>

Function

Add a column to a table.

Format

<add column definition> ::=
ADD [ COLUMN ] <column definition>

Syntax Rules

None.

Access Rules

None.

General Rules

1) The column defined by the <column definition> is added to T.

2) Let C be the column added to T. Every value in C is the default
value for C.

Note: The default value of a column is defined in Subclause 11.5,
"<default clause>".

Note: The addition of a column to a table has no effect on any
existing <query expression> included in a view descriptor or
<search condition> included in constraint descriptor because
any implicit <column reference>s in these clauses are replaced
by explicit <column reference>s when the clause is originally
evaluated. See the Syntax Rules of Subclause 7.10, "<query ex-
pression>".

For what it's worth, here's what it says about DROP COLUMN. Note that
the question of what to do about references to columns: standard says,
throw and error, unless the DBA really means it, with the CASCADE option,
except for VIEWs, which get dropped, unless the DBA is careful to say
RESTRICT.

11.15 <drop column definition>

Function

Destroy a column.

Format

<drop column definition> ::=
DROP [ COLUMN ] <column name> <drop behavior>

<drop behavior> ::= CASCADE | RESTRICT

Syntax Rules

1) Let T be the table identified by the <table name> in the con-
taining <alter table statement> and let TN be the name of T.

2) Let C be the column identified by the <column name> CN.

3) C shall be a column of T and C shall not be the only column of
T.

4) If RESTRICT is specified, then C shall not be referenced in
the <query expression> of any view descriptor or in the <search
condition> of any constraint descriptor other than a table con-
straint descriptor that contains references to no other column
and that is included in the table descriptor of T.

Note: A <drop column definition> that does not specify CASCADE
will fail if there are any references to that column resulting
from the use of CORRESPONDING, NATURAL, SELECT * (except where
contained in an exists predicate>), or REFERENCES without a
<reference column list> in its <referenced table and columns>.

Note: If CASCADE is specified, then any such dependent object
will be dropped by the execution of the <revoke statement> spec-
ified in the General Rules of this Subclause.

Access Rules

None.

General Rules

1) Let A be the current <authorization identifier>. The following
<revoke statement> is effectively executed with a current <au-
thorization identifier> of "_SYSTEM" and without further Access
Rule checking:

REVOKE INSERT(CN), UPDATE(CN), REFERENCES(CN) ON TABLE TN
FROM A CASCADE

2) Let VN be the name of any view that contains a reference to
column C of table T. The following <drop view statement> is
effectively executed with a current <authorization identifier>
of "_SYSTEM" and without further Access Rule checking:

DROP VIEW VN CASCADE

3) If the column is not based on a domain, then its data type de-
scriptor is destroyed.

4) The data associated with C is destroyed and the descriptor of C
is removed from the descriptor of T.

5) The identified column and its descriptor are destroyed.

6) The degree of T is reduced by 1. The ordinal position of all
columns having an ordinal position greater than the ordinal
position of C is reduced by 1.

From bouncefilter Tue Jan 25 15:46:55 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id PAA76819
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 15:45:33 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id MAA05814;
Tue, 25 Jan 2000 12:45:03 -0800 (PST)
Message-Id: <3.0.1.32.20000125124254.00f79ec0@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Tue, 25 Jan 2000 12:42:54 -0800
To: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: Happy column adding and dropping
Cc: PostgreSQL Development <pgsql-hackers@postgreSQL.org>
In-Reply-To: <20000125142219.A1169@rice.edu>
References: <3.0.1.32.20000125112001.0106bec0@mail.pacifier.com>
<14033.948820995@sss.pgh.pa.us>
<NDBBIJLOILGIKBGDINDFCEFHCCAA.Inoue@tpf.co.jp>
<14033.948820995@sss.pgh.pa.us> <20000125122907.F423@rice.edu>
<3.0.1.32.20000125112001.0106bec0@mail.pacifier.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 02:22 PM 1/25/00 -0600, Ross J. Reedstrom wrote:

Gee, Don, that's a good idea, I should have thought of it myself! BTW,
what I have is labelled "Second Informal Review Draft" and is dated July
30, 1992. I don't know how it differs from the final standard, if at all.

From the date, it should be quite close. From my experience, by this

point in the process changes are mostly editorial, not substantial
in the sense of adding or dropping columns...I mean features. :)

I orginally got it from someone on this list, I forget who. Anyone
want a copy, I'll happily email it to you, or throw it on a website:
it's 1.6M of ASCII, ready for the line printer ;-) (273K gzipped.)

I'd love a copy and have a DSL line, so the size doesn't bother me.

As far as putting it on a web site, Ansi drafts are generally available
for public comment so it might be OK to do so. Is there a copyright
statement attached?

That said, ALTER is in fact described, and the ADD COLUMN case agrees
with Tom's expectations, rather than mine, see General Rule 2, below:

So it goes :)

For what it's worth, here's what it says about DROP COLUMN. Note that
the question of what to do about references to columns: standard says,
throw and error, unless the DBA really means it, with the CASCADE option,
except for VIEWs, which get dropped, unless the DBA is careful to say
RESTRICT.

Note: A <drop column definition> that does not specify CASCADE
will fail if there are any references to that column resulting
from the use of CORRESPONDING, NATURAL, SELECT * (except where
contained in an exists predicate>), or REFERENCES without a
<reference column list> in its <referenced table and columns>.

This is interesting, because it implies that select * can or should
be expanded when a view is created, rather than when a view is queried.
Except for "exists" where the "*" target list can be thought of as being
a dummy as sorts.

Also, the "natural" refers to a "natural join". A "natural join", you'll
recall, is one where you supply two table names and the tables are joined
by those columns they share in common. "drop column" is supposed to
catch such cases.

Ditto the form "references tablename" where the foreign key is determined
by inspecting "tablename".

Cool! They don't expect much, do they? :)

All the above with the caveat "if I'm reading this correctly".

Note: If CASCADE is specified, then any such dependent object
will be dropped by the execution of the <revoke statement> spec-
ified in the General Rules of this Subclause.

...

1) Let A be the current <authorization identifier>. The following
<revoke statement> is effectively executed with a current <au-
thorization identifier> of "_SYSTEM" and without further Access
Rule checking:

REVOKE INSERT(CN), UPDATE(CN), REFERENCES(CN) ON TABLE TN
FROM A CASCADE

I don't know exactly what this sez offhand...

2) Let VN be the name of any view that contains a reference to
column C of table T. The following <drop view statement> is
effectively executed with a current <authorization identifier>
of "_SYSTEM" and without further Access Rule checking:

DROP VIEW VN CASCADE

This is clear enough.

...

6) The degree of T is reduced by 1. The ordinal position of all
columns having an ordinal position greater than the ordinal
position of C is reduced by 1.

Does the standard give a way to access the ordinal position of
columns?

My guess is that any preliminary implementation of drop column won't
be implementing everything spelled out in the standard!

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Tue Jan 25 16:48:55 2000
Received: from wallace.ece.rice.edu (root@wallace.ece.rice.edu
[128.42.12.154])
by hub.org (8.9.3/8.9.3) with ESMTP id QAA92010
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 16:48:42 -0500 (EST)
(envelope-from reedstrm@wallace.ece.rice.edu)
Received: by wallace.ece.rice.edu via sendmail from stdin
id <m12DDpH-000LEMC@wallace.ece.rice.edu> (Debian Smail3.2.0.102)
for pgsql-hackers@postgresql.org; Tue, 25 Jan 2000 15:48:39 -0600 (CST)
Date: Tue, 25 Jan 2000 15:48:39 -0600
From: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
To: Don Baccus <dhogaza@pacifier.com>
Cc: PostgreSQL Development <pgsql-hackers@postgresql.org>
Subject: Re: Happy column adding and dropping
Message-ID: <20000125154839.B1169@rice.edu>
References: <3.0.1.32.20000125112001.0106bec0@mail.pacifier.com>
<14033.948820995@sss.pgh.pa.us>
<NDBBIJLOILGIKBGDINDFCEFHCCAA.Inoue@tpf.co.jp>
<14033.948820995@sss.pgh.pa.us> <20000125122907.F423@rice.edu>
<3.0.1.32.20000125112001.0106bec0@mail.pacifier.com>
<20000125142219.A1169@rice.edu>
<3.0.1.32.20000125124254.00f79ec0@mail.pacifier.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
User-Agent: Mutt/1.0i
In-Reply-To: <3.0.1.32.20000125124254.00f79ec0@mail.pacifier.com>;
from dhogaza@pacifier.com on Tue, Jan 25, 2000 at 12:42:54PM -0800

On Tue, Jan 25, 2000 at 12:42:54PM -0800, Don Baccus wrote:

At 02:22 PM 1/25/00 -0600, Ross J. Reedstrom wrote:

I'd love a copy and have a DSL line, so the size doesn't bother me.

As far as putting it on a web site, Ansi drafts are generally available
for public comment so it might be OK to do so. Is there a copyright
statement attached?

Hmm, the string 'copyright' (cas insensitive search) does not appear in
the text at all. This draft was done at DEC, according to the title page.

So, grab it from

http://wallace.ece.rice.edu/sql1992.txt
or
http://wallace.ece.rice.edu/sql1992.txt.gz

(hmm, Netscape just decompressed that, and showed me the text!)

Perhaps this could go on postgresql.org, in the developers info section,
if, as I believe, we may act in the good-faith belief that the drafts of
the ANSI standards are actually in the public domain. Hmm, I just realized,
this is actually a draft of the ISO standard. s/ANSI/ISO/

Note: If CASCADE is specified, then any such dependent object
will be dropped by the execution of the <revoke statement> spec-
ified in the General Rules of this Subclause.

...

1) Let A be the current <authorization identifier>. The following
<revoke statement> is effectively executed with a current <au-
thorization identifier> of "_SYSTEM" and without further Access
Rule checking:

REVOKE INSERT(CN), UPDATE(CN), REFERENCES(CN) ON TABLE TN
FROM A CASCADE

I don't know exactly what this sez offhand...

I think it says: make these items disappear, for the current user, by
revoking all privileges for this column (CN :== column name) on this table.

Err, are privileges supposed to be granular to the _column_? The syntax for
GRANT and REVOKE don't imply it, but some of the Rules in each do.

6) The degree of T is reduced by 1. The ordinal position of all
columns having an ordinal position greater than the ordinal
position of C is reduced by 1.

Does the standard give a way to access the ordinal position of
columns?

You mean go from <column name> to <ordinal>? I don't think so, but I
haven't grovelled through the standard enough... O.K., now I have. It's
in the defined system tables, in particular the COLUMNS table, which I
won't duplicate here, since it runs to more than a printed page! It's
in clause 21.3.10, page 580, and defines a table that includes fields

[...]
TABLE_NAME INFORMATION_SCHEMA.SQL_IDENTIFIER,
COLUMN_NAME INFORMATION_SCHEMA.SQL_IDENTIFIER,
ORDINAL_POSITION INFORMATION_SCHEMA.CARDINAL_NUMBER
CONSTRAINT COLUMN_POSITION_NOT_NULL NOT NULL,
[...]

So, yes, there is a standard way to get from <table name> + <ordinal> to
<column name>, if that's what you meant (since getting at the _contents_
is trivial: SELECT <ordinal> from <table name> Gack, look what you've
done, I'm starting to type in <standard complient markup> ;-)

My guess is that any preliminary implementation of drop column won't
be implementing everything spelled out in the standard!

I wouldn't think so! But it might help to plan for future implementation,
or at least be sure not to implement something that makes it harder to
implement the rest in the future.

Ross
--
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu>
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St., Houston, TX 77005

From bouncefilter Tue Jan 25 17:00:10 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id QAA93310
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 16:59:47 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id NAA01468;
Tue, 25 Jan 2000 13:59:21 -0800 (PST)
Message-Id: <3.0.1.32.20000125135715.00f97720@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Tue, 25 Jan 2000 13:57:15 -0800
To: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: Happy column adding and dropping
Cc: PostgreSQL Development <pgsql-hackers@postgreSQL.org>
In-Reply-To: <20000125154839.B1169@rice.edu>
References: <3.0.1.32.20000125124254.00f79ec0@mail.pacifier.com>
<3.0.1.32.20000125112001.0106bec0@mail.pacifier.com>
<14033.948820995@sss.pgh.pa.us>
<NDBBIJLOILGIKBGDINDFCEFHCCAA.Inoue@tpf.co.jp>
<14033.948820995@sss.pgh.pa.us> <20000125122907.F423@rice.edu>
<3.0.1.32.20000125112001.0106bec0@mail.pacifier.com>
<20000125142219.A1169@rice.edu>
<3.0.1.32.20000125124254.00f79ec0@mail.pacifier.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 03:48 PM 1/25/00 -0600, Ross J. Reedstrom wrote:

http://wallace.ece.rice.edu/sql1992.txt.gz

Thanks! Now I have my own copy...
...

Does the standard give a way to access the ordinal position of
columns?

You mean go from <column name> to <ordinal>? I don't think so, but I
haven't grovelled through the standard enough... O.K., now I have. It's
in the defined system tables, in particular the COLUMNS table, which I
won't duplicate here, since it runs to more than a printed page! It's
in clause 21.3.10, page 580, and defines a table that includes fields

[...]
TABLE_NAME INFORMATION_SCHEMA.SQL_IDENTIFIER,
COLUMN_NAME INFORMATION_SCHEMA.SQL_IDENTIFIER,
ORDINAL_POSITION INFORMATION_SCHEMA.CARDINAL_NUMBER
CONSTRAINT COLUMN_POSITION_NOT_NULL NOT NULL,
[...]

So, yes, there is a standard way to get from <table name> + <ordinal> to
<column name>, if that's what you meant (since getting at the _contents_
is trivial: SELECT <ordinal> from <table name> Gack, look what you've
done, I'm starting to type in <standard complient markup> ;-)

OK, I asked because someone earlier said that Postgres should probably
eventually provide SQL92 standard ways to get at table information.

Interesting...

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

(BTW, ordinary SELECT DISTINCT has this same sort of problem if you try
to ORDER BY an expression that doesn't appear in the target list.
SQL92 avoids the issue by not allowing you to ORDER BY expressions that
aren't in the target list, period. We do allow that --- but not when
you use DISTINCT. Essentially, I want to enforce that same restriction
for DISTINCT ON.)

The other piece of the puzzle would be to document that DISTINCT ON
keeps the first tuple out of each set with the same DISTINCT ON value.

Does that sound like a plan?

Yes, very clear. Good.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Tue Jan 25 17:42:55 2000
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67] (may
be forged)) by hub.org (8.9.3/8.9.3) with ESMTP id RAA06877
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 17:42:51 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
RAA25731;
Tue, 25 Jan 2000 17:38:22 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001252238.RAA25731@candle.pha.pa.us>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column dropping)
In-Reply-To: <14330.948826141@sss.pgh.pa.us> from Tom Lane at "Jan 25,
2000 01:49:01 pm"
To: Tom Lane <tgl@sss.pgh.pa.us>
Date: Tue, 25 Jan 2000 17:38:22 -0500 (EST)
CC: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

It occurs to me that in at least some of the places where attribute
numbers are currently used, we ought to use *neither* logical nor
physical column position, but rather a permanent unique ID --- the
attribute tuple's OID would work, if it's assigned soon enough to be
used for constraints given in CREATE TABLE. (Otherwise we could assign
"column serial numbers" that count from 1 for each relation, but are
never changed or recycled within the relation.)

In particular, if parsetrees for stored rules and constraints worked
that way, renumbering attributes that follow the added/dropped column
would become a lot less painful.

I am going to object to any use of invisible columns just to get a nice
ALTER DROP COLUMN capability. It doesn't seeem with the added code
complexity. Our code is complex enough. Why add more to it just for
one feature.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Tue Jan 25 18:02:57 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA10197
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 18:01:55 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id SAA15406;
Tue, 25 Jan 2000 18:01:36 -0500 (EST)
To: Don Baccus <dhogaza@pacifier.com>
cc: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column dropping)
In-reply-to: <3.0.1.32.20000125113001.00f8acb0@mail.pacifier.com>
References: <20000125114453.E423@rice.edu>
<001401bf6704$5ca7e3a0$2801007e@tpf.co.jp>
<Pine.GSO.4.02A.10001251152160.11899-100000@Val.DoCS.UU.SE>
<3.0.1.32.20000125080125.00f7f160@mail.pacifier.com>
<20000125114453.E423@rice.edu>
<3.0.1.32.20000125113001.00f8acb0@mail.pacifier.com>
Comments: In-reply-to Don Baccus <dhogaza@pacifier.com>
message dated "Tue, 25 Jan 2000 11:30:01 -0800"
Date: Tue, 25 Jan 2000 18:01:36 -0500
Message-ID: <15403.948841296@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Don Baccus <dhogaza@pacifier.com> writes:

In particular, if parsetrees for stored rules and constraints worked
that way, renumbering attributes that follow the added/dropped column
would become a lot less painful.

Yes...I see what you're driving at. Very interesting idea. The stored
rules and constraints would in this case would still refer to the remaining
columns after a drop, right?

Right. You'd still need to scan through all the rules/constraints to
look for references to a column-to-be-dropped (and then either drop that
rule/constraint or kick out an error, as appropriate). But you wouldn't
have to *change* any surviving rules/constraints, because they'd still
be referring to the same permanent IDs of the remaining columns.

Also, inherited ADD COLUMN would become far easier, because it wouldn't
change the rules/constraints of child tables at all --- even though the
new column would change the logical numbering of child-table columns,
it wouldn't change their permanent IDs and thus we wouldn't have to
update rules/constraints.

If we were willing to hardwire the assumption that DROP COLUMN never
physically drops a column, but only hides it and adjusts logical column
numbers, then the physical column numbers could serve as permanent IDs;
so we'd only need two numbers not three. This might be good, or not.

regards, tom lane

From bouncefilter Tue Jan 25 18:13:56 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA14434
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 18:13:05 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id PAA28817;
Tue, 25 Jan 2000 15:12:30 -0800 (PST)
Message-Id: <3.0.1.32.20000125151022.00f8c4c0@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Tue, 25 Jan 2000 15:10:22 -0800
To: Tom Lane <tgl@sss.pgh.pa.us>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column
dropping)
Cc: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
In-Reply-To: <15403.948841296@sss.pgh.pa.us>
References: <3.0.1.32.20000125113001.00f8acb0@mail.pacifier.com>
<20000125114453.E423@rice.edu>
<001401bf6704$5ca7e3a0$2801007e@tpf.co.jp>
<Pine.GSO.4.02A.10001251152160.11899-100000@Val.DoCS.UU.SE>
<3.0.1.32.20000125080125.00f7f160@mail.pacifier.com>
<20000125114453.E423@rice.edu>
<3.0.1.32.20000125113001.00f8acb0@mail.pacifier.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 06:01 PM 1/25/00 -0500, Tom Lane wrote:

Don Baccus <dhogaza@pacifier.com> writes:

In particular, if parsetrees for stored rules and constraints worked
that way, renumbering attributes that follow the added/dropped column
would become a lot less painful.

Yes...I see what you're driving at. Very interesting idea. The stored
rules and constraints would in this case would still refer to the remaining
columns after a drop, right?

Right. You'd still need to scan through all the rules/constraints to
look for references to a column-to-be-dropped (and then either drop that
rule/constraint or kick out an error, as appropriate). But you wouldn't
have to *change* any surviving rules/constraints, because they'd still
be referring to the same permanent IDs of the remaining columns.

Good, I understand then.

Also, inherited ADD COLUMN would become far easier, because it wouldn't
change the rules/constraints of child tables at all --- even though the
new column would change the logical numbering of child-table columns,
it wouldn't change their permanent IDs and thus we wouldn't have to
update rules/constraints.

Ahhh...yes. I haven't looked at the inheritance code, yet, but I see
what you're saying. I think. Do child-table columns follow parent-table
columns by some chance (in today's absolute column number scheme)?

If we were willing to hardwire the assumption that DROP COLUMN never
physically drops a column, but only hides it and adjusts logical column
numbers, then the physical column numbers could serve as permanent IDs;
so we'd only need two numbers not three. This might be good, or not.

Yes. But if I'm right about how child-table columns are numbered,
wouldn't add column still cause a problem, i.e. you'd still have to
change their physical column number?

I'm probably misunderstanding here because I've not looked at the
inheritance mechanism at all. Maybe I'll do that for a little evening
entertainment.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Tue Jan 25 18:08:56 2000
Received: from merganser.its.uu.se (merganser.its.uu.se [130.238.6.236])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA13671
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 18:08:42 -0500 (EST)
(envelope-from peter@localhost.its.uu.se)
Received: from regulus.its.uu.se ([130.238.7.19]:62591 "EHLO
regulus.its.uu.se")
by merganser.its.uu.se with ESMTP id <S239649AbQAYXH5>;
Wed, 26 Jan 2000 00:07:57 +0100
Received: from peter (helo=localhost)
by regulus.its.uu.se with local-esmtp (Exim 3.02 #2)
id 12DFBP-0004pa-00
for pgsql-hackers@postgresql.org; Wed, 26 Jan 2000 00:15:35 +0100
Date: Wed, 26 Jan 2000 00:15:35 +0100 (CET)
From: Peter Eisentraut <peter_e@gmx.net>
To: PostgreSQL Development <pgsql-hackers@postgresql.org>
Subject: Column ADDing issues
Message-ID: <Pine.LNX.4.21.0001252139410.9535-100000@localhost.localdomain>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=ISO-8859-1
Content-Transfer-Encoding: 8BIT
Sender: Peter Eisentraut <peter@hub.org>

I took up the issue of

* Alter TABLE ADD COLUMN does not honor DEFAULT, add CONSTRAINT

and chances are excellent that this will get all done in the next day or
three.

First a syntactical issue. We currently allow the following statement,
although it is not legal SQL:

create table test (
a int4,
b int4 check (a>b)
);

It's not legal because the column constraint for "b" may only reference
column "b". Instead you could legally write

create table test (
a int4,
b int4,
check (a>b)
);

because the check constraint is now a table constraint. Big deal. Now the
problem is that because I reuse the same syntactical elements, the
following will work:

create table test (a int4);
alter table test add column b int4 check (a>b);

No harm done, but how about:

create table test (a int4, b int4);
alter table test add column c text check (a>b);

I guess this would be sort of equivalent to saying

alter table test add column c text;
alter table test add constraint check (a>b);

So, I guess what I'm saying is whether you want to allow the mentioned
weirdness or not.

Secondly, an internal question. If I use SearchSysCacheTuple() on a
condition with several potential matches, what is the defined behaviour?
Can I call it again to get the next tuple?

Thirdly, about TODO item

* ALTER TABLE ADD COLUMN to inherited table put column in wrong place

Actually, according to what I would expect out of the blue, it puts it
into the *right* place. Even good ol' SQL, although they naturally do not
know about inheritance, seems to agree:

"... the degree of [the table] is increased by 1 and the ordinal position
of that [new] column is equal to the new degree of [the table] ..."
(11.11)

What that says to me is that if you add a column to a table (during create
or alter) then the new column gets placed after all the others. Thus,
we're in compliance without even knowing it.

Or maybe look at it this way:
create table test1 (a int4);
create table test2 (b int4) inherits (test1);
^ col #1 ^ col #2
alter table test1* add column c int4;
^ col #3

Everything has its order and it's not like the inheritance as such is
broken.

Surely, trying to stick the column in between is going to be three times
as much work as dropping columns would be, whichever way you do it. (And
making attributes invisible ain't gonna help you. ;)

What do you all say?

--
Peter Eisentraut Sernanders v���g 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Tue Jan 25 18:08:56 2000
Received: from merganser.its.uu.se (merganser.its.uu.se [130.238.6.236])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA13684
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 18:08:51 -0500 (EST)
(envelope-from peter@localhost.its.uu.se)
Received: from regulus.its.uu.se ([130.238.7.19]:62718 "EHLO
regulus.its.uu.se")
by merganser.its.uu.se with ESMTP id <S299046AbQAYXII>;
Wed, 26 Jan 2000 00:08:08 +0100
Received: from peter (helo=localhost)
by regulus.its.uu.se with local-esmtp (Exim 3.02 #2)
id 12DFBb-0004qJ-00
for pgsql-hackers@postgresql.org; Wed, 26 Jan 2000 00:15:47 +0100
Date: Wed, 26 Jan 2000 00:15:47 +0100 (CET)
From: Peter Eisentraut <peter_e@gmx.net>
To: PostgreSQL Development <pgsql-hackers@postgresql.org>
Subject: --enable-debug
Message-ID: <Pine.LNX.4.21.0001251304260.345-100000@localhost.localdomain>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=ISO-8859-1
Content-Transfer-Encoding: 8BIT
Sender: Peter Eisentraut <peter@hub.org>

Those that are aware of the --enable-debug switch might have found out
that it is only marginally useful. (Perhaps you have found it to be very
useful, in which case you are free to disagree with me.)

It seems that for a debugging environment you should not only put a -g on
the command line, but also a -Wall, and you most likely do not want to
have any optimization flags there. If no one objects I would like to make
this adjustment. (Perhaps the debug and cassert options should even be
merged?)

--
Peter Eisentraut Sernanders v���g 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Tue Jan 25 18:09:56 2000
Received: from merganser.its.uu.se (merganser.its.uu.se [130.238.6.236])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA13756
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 18:09:11 -0500 (EST)
(envelope-from peter@localhost.its.uu.se)
Received: from regulus.its.uu.se ([130.238.7.19]:63033 "EHLO
regulus.its.uu.se")
by merganser.its.uu.se with ESMTP id <S243749AbQAYXI2>;
Wed, 26 Jan 2000 00:08:28 +0100
Received: from peter (helo=localhost)
by regulus.its.uu.se with local-esmtp (Exim 3.02 #2)
id 12DFBp-0004r2-00; Wed, 26 Jan 2000 00:16:01 +0100
Date: Wed, 26 Jan 2000 00:16:01 +0100 (CET)
From: Peter Eisentraut <peter_e@gmx.net>
To: Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>
cc: "'Tom Lane'" <tgl@sss.pgh.pa.us>,
"'pgsql-hackers@postgreSQL.org'" <pgsql-hackers@postgresql.org>
Subject: Re: AW: [HACKERS] Some notes on optimizer cost estimates
In-Reply-To:
<219F68D65015D011A8E000006F8590C603FDC21C@sdexcsrv1.f000.d0188.sd.spardat.at>
Message-ID: <Pine.LNX.4.21.0001250956070.345-100000@localhost.localdomain>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=ISO-8859-1
Content-Transfer-Encoding: 8BIT
Sender: Peter Eisentraut <peter@hub.org>

On 2000-01-24, Zeugswetter Andreas SB mentioned:

Couldn't we test some of these parameters inside configure and set
them there?

If we could figure out a reasonably cheap way of estimating these
numbers, it'd be worth setting up custom values at installation time.

Imho this whole idea is not so good. (Sorry)

My points are:
1. even if it is good for an optimizer to be smart,
it is even more important, that it is predictable

ISTM that by the nature of things the most important capability of an
optimizer is to yield optimal results. This, however, does not have to be
mutually exclusive with predictability. If you estimate some CPU and disk
parameters and write them into a config file, then you can always give
this config file to a bug fixer. It will still work on his machine, just
less than optimally.

2. I compile on test machine, production is completely different
(more processors, faster disks and controllers)

You're completely right. This has no place in configure. It will have to
be a separate tool which you can run after building and installing.

--
Peter Eisentraut Sernanders v���g 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Tue Jan 25 18:09:56 2000
Received: from merganser.its.uu.se (merganser.its.uu.se [130.238.6.236])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA13765
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 18:09:21 -0500 (EST)
(envelope-from peter@localhost.its.uu.se)
Received: from regulus.its.uu.se ([130.238.7.19]:63255 "EHLO
regulus.its.uu.se")
by merganser.its.uu.se with ESMTP id <S307226AbQAYXIo>;
Wed, 26 Jan 2000 00:08:44 +0100
Received: from peter (helo=localhost)
by regulus.its.uu.se with local-esmtp (Exim 3.02 #2)
id 12DFC3-0004rX-00; Wed, 26 Jan 2000 00:16:15 +0100
Date: Wed, 26 Jan 2000 00:16:15 +0100 (CET)
From: Peter Eisentraut <peter_e@gmx.net>
To: "Henry B. Hotz" <hotz@jpl.nasa.gov>
cc: Tom Lane <tgl@sss.pgh.pa.us>, pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Some notes on optimizer cost estimates
In-Reply-To: <v04020a4cb4b2354516da@[137.78.84.130]>
Message-ID: <Pine.LNX.4.21.0001250051290.525-100000@localhost.localdomain>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=ISO-8859-1
Content-Transfer-Encoding: 8BIT
Sender: Peter Eisentraut <peter@hub.org>

On 2000-01-24, Henry B. Hotz mentioned:

to be useful for cost estimating. We could make it a parameter like
we do for CPU_PAGE_WEIGHT ... but you know and I know that no one
ever bothers to adjust those numbers in the field ...

Couldn't we test some of these parameters inside configure and set them there?

Nope. configure is for testing the build environment, not the runtime
environment. But the same thing could be achieved with some
runtime-analyzing program that writes its results into some sort of
configuration file.

--
Peter Eisentraut Sernanders v���g 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Tue Jan 25 18:47:56 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA21808
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 18:43:45 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id PAA10312;
Tue, 25 Jan 2000 15:42:34 -0800 (PST)
Message-Id: <3.0.1.32.20000125153804.00f98e30@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Tue, 25 Jan 2000 15:38:04 -0800
To: Peter Eisentraut <peter_e@gmx.net>,
Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: AW: [HACKERS] Some notes on optimizer cost estimates
Cc: "'Tom Lane'" <tgl@sss.pgh.pa.us>,
"'pgsql-hackers@postgreSQL.org'" <pgsql-hackers@postgresql.org>
In-Reply-To: <Pine.LNX.4.21.0001250956070.345-100000@localhost.localdoma
in>
References:
<219F68D65015D011A8E000006F8590C603FDC21C@sdexcsrv1.f000.d0188.sd.spardat.at>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 12:16 AM 1/26/00 +0100, Peter Eisentraut wrote:

On 2000-01-24, Zeugswetter Andreas SB mentioned:

My points are:
1. even if it is good for an optimizer to be smart,
it is even more important, that it is predictable

ISTM that by the nature of things the most important capability of an
optimizer is to yield optimal results.

One problem, though, is that the optimization problem's even more
intractable for database systems than it is for compilers. Large and
growing database installations go through an evolutionary process of
adding memory, spindles, partioning of data, etc. Changing from the
"-B" default to "-B 2000", as I've done on my web site, causes the
database to live in shared memory even when other activity on the
system would tend to cause some of it to be flushed from the filesystem
cache. This changes how long, on average, it takes to read a block.

And the first time a table's referenced is always going to take longer
than subsequent references if there's caching involved...so in
theory the optimizer should take that into account if it makes presumptions
about cache hit ratios.

So forth and so forth.

I'm not disagreeing in the least with the spirit of your comment. There
are all sorts of practical barriers...

Thankfully, for the particular problem of choosing between and index
vs. sequential scan, the optimizer only has two choices to make. Thus
a rough-and-ready heuristic based on certain assumptions might work well,
which is essentially what the optimizer does today. In fact, it does
work quite well, come to think of it!

Fiddling the numbers underlying the assumptions may be good enough for
this task.

This, however, does not have to be
mutually exclusive with predictability. If you estimate some CPU and disk
parameters and write them into a config file, then you can always give
this config file to a bug fixer. It will still work on his machine, just
less than optimally.

2. I compile on test machine, production is completely different
(more processors, faster disks and controllers)

You're completely right. This has no place in configure. It will have to
be a separate tool which you can run after building and installing.

Based perhaps on statistics gathered while the system is running...

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Tue Jan 25 18:56:57 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA23792
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 18:56:21 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id SAA15514;
Tue, 25 Jan 2000 18:55:59 -0500 (EST)
To: Bruce Momjian <pgman@candle.pha.pa.us>
cc: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column dropping)
In-reply-to: <200001252238.RAA25731@candle.pha.pa.us>
References: <200001252238.RAA25731@candle.pha.pa.us>
Comments: In-reply-to Bruce Momjian <pgman@candle.pha.pa.us>
message dated "Tue, 25 Jan 2000 17:38:22 -0500"
Date: Tue, 25 Jan 2000 18:55:59 -0500
Message-ID: <15511.948844559@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Bruce Momjian <pgman@candle.pha.pa.us> writes:

I am going to object to any use of invisible columns just to get a nice
ALTER DROP COLUMN capability. It doesn't seeem with the added code
complexity. Our code is complex enough. Why add more to it just for
one feature.

I'm not convinced about it either --- but at this point the discussion
is just a gedanken-experiment to see what problems would be solved
or created if we did something like this. It could be that having
invisible columns (or more likely, separate logical and physical
column numbers) would solve enough nagging problems with ADD COLUMN
and DROP COLUMN and inherited tables that it'd be worth doing.
Or not. But let's not shut off the discussion before we see where
it leads.

Following on to my comment about logical column numbers, physical
column positions, and permanent column IDs being conceptually
distinct: one thing that needs careful consideration is just how
we identify inherited columns in child tables as being the "same
column" as the original column in the parent. Right now, because
ADD COLUMN doesn't guarantee to assign the same column number in
each child, the parser/planner handles this by looking for the
same column name in each child table, which it does on-the-fly
while setting up a "SELECT table*" operation. That's pretty much
of a kluge in my opinion. My guess is that we ought to use either
logical column number or permanent ID as the key that tells us which
child column is the inherited one. To use permanent ID, we'd have
to ensure that permanent IDs are inheritable, which would complicate
assignment of permanent IDs considerably --- they'd probably have to
become OIDs, but not the same OIDs as are assigned to the pg_attribute
rows themselves. Logical column number might work OK for this purpose
though. It seems a little shaky to me intuitively, but I can't actually
think of a reason that it wouldn't work.

regards, tom lane

From bouncefilter Tue Jan 25 19:11:57 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id TAA29887
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 19:10:57 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id TAA15553;
Tue, 25 Jan 2000 19:10:04 -0500 (EST)
To: Peter Eisentraut <peter_e@gmx.net>
cc: PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Column ADDing issues
In-reply-to: <Pine.LNX.4.21.0001252139410.9535-100000@localhost.localdomain>
References: <Pine.LNX.4.21.0001252139410.9535-100000@localhost.localdomain>
Comments: In-reply-to Peter Eisentraut <peter_e@gmx.net>
message dated "Wed, 26 Jan 2000 00:15:35 +0100"
Date: Tue, 25 Jan 2000 19:10:04 -0500
Message-ID: <15550.948845404@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Peter Eisentraut <peter_e@gmx.net> writes:

Or maybe look at it this way:
create table test1 (a int4);
create table test2 (b int4) inherits (test1);
^ col #1 ^ col #2
alter table test1* add column c int4;
^ col #3

Everything has its order and it's not like the inheritance as such is
broken.

Yes, a whole bunch of stuff is broken after this happens. Go back and
consult the archives --- or maybe Chris Bitmead will fill you in; he's
got plenty of scars to show for this set of problems. (All I recall
offhand is that pg_dump and reload can fail to generate a working
database.) The bottom line is that it would be a lot nicer if column c
had the same column position in both the parent table and the child
table(s).

I suggest you be very cautious about messing with ALTER TABLE until you
understand why inheritance makes it such a headache ;-)

regards, tom lane

From bouncefilter Tue Jan 25 19:14:56 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id TAA30466
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 19:14:33 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id TAA15575;
Tue, 25 Jan 2000 19:14:29 -0500 (EST)
To: Peter Eisentraut <peter_e@gmx.net>
cc: PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] --enable-debug
In-reply-to: <Pine.LNX.4.21.0001251304260.345-100000@localhost.localdomain>
References: <Pine.LNX.4.21.0001251304260.345-100000@localhost.localdomain>
Comments: In-reply-to Peter Eisentraut <peter_e@gmx.net>
message dated "Wed, 26 Jan 2000 00:15:47 +0100"
Date: Tue, 25 Jan 2000 19:14:29 -0500
Message-ID: <15572.948845669@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Peter Eisentraut <peter_e@gmx.net> writes:

It seems that for a debugging environment you should not only put a -g on
the command line, but also a -Wall, and you most likely do not want to
have any optimization flags there. If no one objects I would like to make
this adjustment. (Perhaps the debug and cassert options should even be
merged?)

If you want --debug to imply --cassert, OK with me, but I will get upset
if you rearrange things in such a way that I can't have -O and -g and
--cassert. In other words --cassert should not imply turning off -O.

Currently, I believe -Wall is turned on automatically if your compiler
is gcc --- at least it seems to work that way on my platform (maybe
it's in the hpux_gcc template?). Anyway I get a pretty good set of -W
options without doing anything special. It is real obvious that some
other developers do not run with the same options :-(, so I'd certainly
be glad to see more -W options turned on whenever the compiler is gcc,
--debug or no.

regards, tom lane

From bouncefilter Tue Jan 25 19:19:57 2000
Received: from hotzsun.jpl.nasa.gov (root@hotzsun.jpl.nasa.gov
[137.78.84.131])
by hub.org (8.9.3/8.9.3) with ESMTP id TAA31203
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 19:19:15 -0500 (EST)
(envelope-from hotz@jpl.nasa.gov)
Received: from [137.78.84.130] (hotzmac [137.78.84.130])
by hotzsun.jpl.nasa.gov (8.9.3/8.9.3) with ESMTP id QAA28071;
Tue, 25 Jan 2000 16:19:07 -0800 (PST)
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
X-Sender: hotzmail@hotzsun.jpl.nasa.gov
Message-Id: <v04020a04b4b3ec6066d8@[137.78.84.130]>
In-Reply-To: <Pine.LNX.4.21.0001250051290.525-100000@localhost.localdomain>
References: <v04020a4cb4b2354516da@[137.78.84.130]>
Date: Tue, 25 Jan 2000 16:18:17 -0800
To: Peter Eisentraut <peter_e@gmx.net>
From: "Henry B. Hotz" <hotz@jpl.nasa.gov>
Subject: Re: [HACKERS] Some notes on optimizer cost estimates
Cc: Tom Lane <tgl@sss.pgh.pa.us>, pgsql-hackers@postgreSQL.org

At 3:16 PM -0800 1/25/00, Peter Eisentraut wrote:

On 2000-01-24, Henry B. Hotz mentioned:

Couldn't we test some of these parameters inside configure and set them
there?

Nope. configure is for testing the build environment, not the runtime
environment. But the same thing could be achieved with some
runtime-analyzing program that writes its results into some sort of
configuration file.

And then we could make configure run the program for us, and we could
automatically put the results into some header file. . . In other words
I'm not that much of a purist.

As Tom Lane said this is not an important point. What's important is how
much can we determine how easily. It may actually make more sense to do
this tuning after installation. It only affects optimizer choices in
situations that aren't obvious.

Signature failed Preliminary Design Review.
Feasibility of a new signature is currently being evaluated.
h.b.hotz@jpl.nasa.gov, or hbhotz@oxy.edu

From bouncefilter Tue Jan 25 19:21:56 2000
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67] (may
be forged)) by hub.org (8.9.3/8.9.3) with ESMTP id TAA31606
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 19:21:18 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
TAA04129;
Tue, 25 Jan 2000 19:20:36 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001260020.TAA04129@candle.pha.pa.us>
Subject: Re: [HACKERS] --enable-debug
In-Reply-To: <Pine.LNX.4.21.0001251304260.345-100000@localhost.localdomain>
from Peter Eisentraut at "Jan 26, 2000 00:15:47 am"
To: Peter Eisentraut <peter_e@gmx.net>
Date: Tue, 25 Jan 2000 19:20:35 -0500 (EST)
CC: PostgreSQL Development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

[Charset ISO-8859-1 unsupported, filtering to ASCII...]

Those that are aware of the --enable-debug switch might have found out
that it is only marginally useful. (Perhaps you have found it to be very
useful, in which case you are free to disagree with me.)

It seems that for a debugging environment you should not only put a -g on
the command line, but also a -Wall, and you most likely do not want to
have any optimization flags there. If no one objects I would like to make
this adjustment. (Perhaps the debug and cassert options should even be
merged?)

Good idea.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Tue Jan 25 19:19:57 2000
Received: from hu.tm.ee (ppp809.tele2.ee [212.107.37.109])
by hub.org (8.9.3/8.9.3) with ESMTP id TAA31158
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 19:19:04 -0500 (EST) (envelope-from hannu@tm.ee)
Received: from tm.ee (localhost [127.0.0.1]) by hu.tm.ee (Postfix) with ESMTP
id 46B6213469; Wed, 26 Jan 2000 02:25:13 +0200 (EET)
Sender: hannu@hu.tm.ee
Message-ID: <388E3EE9.46880647@tm.ee>
Date: Wed, 26 Jan 2000 02:25:13 +0200
From: Hannu Krosing <hannu@tm.ee>
Organization: Trust-O-Matic =?iso-8859-1?Q?O=DC?=
X-Mailer: Mozilla 4.7 [en] (X11; I; Linux 2.2.13-7mdk i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Don Baccus <dhogaza@pacifier.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>,
"Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy columndropping)
References: <3.0.1.32.20000125113001.00f8acb0@mail.pacifier.com>
<20000125114453.E423@rice.edu>
<001401bf6704$5ca7e3a0$2801007e@tpf.co.jp>
<Pine.GSO.4.02A.10001251152160.11899-100000@Val.DoCS.UU.SE>
<3.0.1.32.20000125080125.00f7f160@mail.pacifier.com>
<20000125114453.E423@rice.edu>
<3.0.1.32.20000125113001.00f8acb0@mail.pacifier.com>
<3.0.1.32.20000125151022.00f8c4c0@mail.pacifier.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Don Baccus wrote:

Ahhh...yes. I haven't looked at the inheritance code, yet, but I see
what you're saying. I think. Do child-table columns follow parent-table
columns by some chance (in today's absolute column number scheme)?

If we were willing to hardwire the assumption that DROP COLUMN never
physically drops a column, but only hides it and adjusts logical column
numbers, then the physical column numbers could serve as permanent IDs;
so we'd only need two numbers not three. This might be good, or not.

Yes. But if I'm right about how child-table columns are numbered,
wouldn't add column still cause a problem, i.e. you'd still have to
change their physical column number?

If we allow deleted column as a basic feature of postgres,
it could be like that

base: COL1 | COL2 | COL3
child: COL1 | COL2 | COL3 | COL4

after add column 5 to base table

base: COL1 | COL2 | COL3 | del4 | COL5
child: COL1 | COL2 | COL3 | COL4 | COL5

after add column 6 to child

base: COL1 | COL2 | COL3 | del4 | COL5
child: COL1 | COL2 | COL3 | COL4 | COL5 | COL6

after drop column 2 from base table

base: COL1 | del2 | COL3 | del4 | COL5
child: COL1 | del2 | COL3 | COL4 | COL5 | COL6

dropping column from child table that is not a deleted column in
parent is not allowed.

The delN columns are always NULLed on reading tuple and are written as proper
null columns (taking up space only in NULL bitmask)

multiple inheritance is tricky and _requires_ unique column ids maybe oids
from pg_attribute to be doable.

-----------------
Hannu

From bouncefilter Tue Jan 25 19:25:57 2000
Received: from hu.tm.ee (ppp809.tele2.ee [212.107.37.109])
by hub.org (8.9.3/8.9.3) with ESMTP id TAA32307
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 19:25:19 -0500 (EST) (envelope-from hannu@tm.ee)
Received: from tm.ee (localhost [127.0.0.1]) by hu.tm.ee (Postfix) with ESMTP
id 6754D1346A; Wed, 26 Jan 2000 02:31:34 +0200 (EET)
Sender: hannu@hu.tm.ee
Message-ID: <388E4066.3CDEE504@tm.ee>
Date: Wed, 26 Jan 2000 02:31:34 +0200
From: Hannu Krosing <hannu@tm.ee>
Organization: Trust-O-Matic =?iso-8859-1?Q?O=DC?=
X-Mailer: Mozilla 4.7 [en] (X11; I; Linux 2.2.13-7mdk i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Don Baccus <dhogaza@pacifier.com>, Peter Eisentraut <peter_e@gmx.net>,
Hiroshi Inoue <Inoue@tpf.co.jp>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column dropping)
References: <001401bf6704$5ca7e3a0$2801007e@tpf.co.jp>
<3.0.1.32.20000125080125.00f7f160@mail.pacifier.com>
<14178.948821965@sss.pgh.pa.us>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Tom Lane wrote:

Don Baccus <dhogaza@pacifier.com> writes:

However, heap_getattr still won't see the default since it simply
checks to see of the attribute number falls off the end of the
tuple and then returns null.

Right.

There's no provision for then pulling out the default value and
returning it instead.

Would it even be possible to do that? I'm not sure that heap_getattr
has any way to find the default. It only has a TupleDesc, not a
reference to the relation itself...

but could it not return a special value, say COLUMN_DEFAULT, that upper
levels know to replace with the real default. Of course it get's funny if
if the default is nextval('myseq') or CURRENT_TIME.

another way to approach the whole problem is to have something like
generation_id in each tuple that tells which version of TupleDesc applies
to this tuple, but it seems to complicate things even further.

-----------------
Hannu

From bouncefilter Tue Jan 25 19:41:58 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id TAA38612
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 19:41:18 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id QAA02782;
Tue, 25 Jan 2000 16:40:11 -0800 (PST)
Message-Id: <3.0.1.32.20000125163804.00fa3bd0@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Tue, 25 Jan 2000 16:38:04 -0800
To: Hannu Krosing <hannu@tm.ee>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy
columndropping)
Cc: Tom Lane <tgl@sss.pgh.pa.us>,
"Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
In-Reply-To: <388E3EE9.46880647@tm.ee>
References: <3.0.1.32.20000125113001.00f8acb0@mail.pacifier.com>
<20000125114453.E423@rice.edu>
<001401bf6704$5ca7e3a0$2801007e@tpf.co.jp>
<Pine.GSO.4.02A.10001251152160.11899-100000@Val.DoCS.UU.SE>
<3.0.1.32.20000125080125.00f7f160@mail.pacifier.com>
<20000125114453.E423@rice.edu>
<3.0.1.32.20000125113001.00f8acb0@mail.pacifier.com>
<3.0.1.32.20000125151022.00f8c4c0@mail.pacifier.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 02:25 AM 1/26/00 +0200, Hannu Krosing wrote:

multiple inheritance is tricky and _requires_ unique column ids maybe oids
from pg_attribute to be doable.

Having worked on a C++ compiler (one of the first, actually) I
suspect that this might be one of the lesser problems associated
with implementing a multiple inheritance scheme :)

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Tue Jan 25 19:45:57 2000
Received: from mail.enterprise.net (mail.enterprise.net [194.72.192.18])
by hub.org (8.9.3/8.9.3) with ESMTP id TAA39379
for <pgsql-hackers@postgresql.org>;
Tue, 25 Jan 2000 19:45:39 -0500 (EST) (envelope-from olly@lfix.co.uk)
Received: from linda.lfix.co.uk (root@max01-080.enterprise.net
[194.72.195.80])
by mail.enterprise.net (8.8.5/8.8.5) with ESMTP id AAA21884;
Wed, 26 Jan 2000 00:45:32 GMT
Received: from lfix.co.uk (olly@localhost [127.0.0.1])
by linda.lfix.co.uk (8.9.3/8.9.3/Debian 8.9.3-6) with ESMTP id AAA04115;
Wed, 26 Jan 2000 00:45:20 GMT
Message-Id: <200001260045.AAA04115@linda.lfix.co.uk>
X-Authentication-Warning: linda.lfix.co.uk: Host olly@localhost [127.0.0.1]
claimed to be lfix.co.uk
X-Mailer: exmh version 2.1.1 10/15/1999 (debian)
To: Peter Eisentraut <peter_e@gmx.net>
cc: PostgreSQL Development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Column ADDing issues
In-Reply-To: Message from Peter Eisentraut <peter_e@gmx.net> of "Wed,
26 Jan 2000 00:15:35 +0100."
<Pine.LNX.4.21.0001252139410.9535-100000@localhost.localdomain>
References: <Pine.LNX.4.21.0001252139410.9535-100000@localhost.localdomain>
Mime-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0
Date: Wed, 26 Jan 2000 00:45:20 +0000
From: "Oliver Elphick" <olly@lfix.co.uk>
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by hub.org id TAA39409

Peter Eisentraut wrote:

Thirdly, about TODO item

* ALTER TABLE ADD COLUMN to inherited table put column in wrong place

Actually, according to what I would expect out of the blue, it puts it
into the *right* place. Even good ol' SQL, although they naturally do not
know about inheritance, seems to agree:

"... the degree of [the table] is increased by 1 and the ordinal position
of that [new] column is equal to the new degree of [the table] ..."
(11.11)

What that says to me is that if you add a column to a table (during create
or alter) then the new column gets placed after all the others. Thus,
we're in compliance without even knowing it.

Or maybe look at it this way:
create table test1 (a int4);
create table test2 (b int4) inherits (test1);
^ col #1 ^ col #2
alter table test1* add column c int4;
^ col #3

Everything has its order and it's not like the inheritance as such is
broken.

Surely, trying to stick the column in between is going to be three times
as much work as dropping columns would be, whichever way you do it. (And
making attributes invisible ain't gonna help you. ;)

It is:
create table test1 (a int4);
create table test2 (b int4) inherits (test1);
^ col #2 ^ col #1
alter table test1* add column c int4;
^ col #3 but needs to be #2, since it _is_
#2 of test1

As far as inheritance goes, all the descendants are treated as one table,
including those created on a different branch from test2. All of them
have to return the right columns for a single query; the two options for
dealing with this seem to be logical column numbering, or rewriting the
descendant tables. (But I haven't spent enough time in the code to be
sure of that.)

--
Oliver Elphick Oliver.Elphick@lfix.co.uk
Isle of Wight http://www.lfix.co.uk/oliver
PGP key from public servers; key ID 32B8FAA1
========================================
"My little children, let us not love in word, neither
in tongue; but in deed and in truth."
I John 3:18

From bouncefilter Tue Jan 25 19:50:57 2000
Received: from hu.tm.ee (ppp809.tele2.ee [212.107.37.109])
by hub.org (8.9.3/8.9.3) with ESMTP id TAA40176
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 19:50:01 -0500 (EST) (envelope-from hannu@tm.ee)
Received: from tm.ee (localhost [127.0.0.1]) by hu.tm.ee (Postfix) with ESMTP
id B7AB313469; Wed, 26 Jan 2000 02:56:12 +0200 (EET)
Sender: hannu@hu.tm.ee
Message-ID: <388E462C.D7DCAC4D@tm.ee>
Date: Wed, 26 Jan 2000 02:56:12 +0200
From: Hannu Krosing <hannu@tm.ee>
Organization: Trust-O-Matic =?iso-8859-1?Q?O=DC?=
X-Mailer: Mozilla 4.7 [en] (X11; I; Linux 2.2.13-7mdk i686)
X-Accept-Language: en
MIME-Version: 1.0
To: The Hermit Hacker <scrappy@hub.org>
Cc: Bruce Momjian <pgman@candle.pha.pa.us>, Tom Lane <tgl@sss.pgh.pa.us>,
Hiroshi Inoue <Inoue@tpf.co.jp>, Peter Eisentraut <peter_e@gmx.net>,
pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Well, then you keep your darn columns
References: <Pine.BSF.4.21.0001242133440.362-100000@thelab.hub.org>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

The Hermit Hacker wrote:

numbered. How does a missing attribute get handled for new rows?
My guess is that we have to keep this thing around forever. Can you
imagine having all those user apps tha query pg_attribute supress that
column. Sound like too much work to me.

I *still* think the best is the "re-write the table in place" method
... we already have most of the logic, I would think, from VACUUM ...

It is much safer with vacuum, as there is no way for vacuum to crash and have
two different tuple formats left behind (i hope ;-p)

-------------
Hannu

From bouncefilter Tue Jan 25 20:07:57 2000
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67] (may
be forged)) by hub.org (8.9.3/8.9.3) with ESMTP id UAA46904
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 20:07:15 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
UAA05685;
Tue, 25 Jan 2000 20:05:11 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001260105.UAA05685@candle.pha.pa.us>
Subject: Re: [HACKERS] --enable-debug
In-Reply-To: <15572.948845669@sss.pgh.pa.us> from Tom Lane at "Jan 25,
2000 07:14:29 pm"
To: Tom Lane <tgl@sss.pgh.pa.us>
Date: Tue, 25 Jan 2000 20:05:11 -0500 (EST)
CC: Peter Eisentraut <peter_e@gmx.net>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Peter Eisentraut <peter_e@gmx.net> writes:

It seems that for a debugging environment you should not only put a -g on
the command line, but also a -Wall, and you most likely do not want to
have any optimization flags there. If no one objects I would like to make
this adjustment. (Perhaps the debug and cassert options should even be
merged?)

If you want --debug to imply --cassert, OK with me, but I will get upset
if you rearrange things in such a way that I can't have -O and -g and
--cassert. In other words --cassert should not imply turning off -O.

Currently, I believe -Wall is turned on automatically if your compiler
is gcc --- at least it seems to work that way on my platform (maybe
it's in the hpux_gcc template?). Anyway I get a pretty good set of -W
options without doing anything special. It is real obvious that some
other developers do not run with the same options :-(, so I'd certainly
be glad to see more -W options turned on whenever the compiler is gcc,
--debug or no.

I remember when we removed every warning gcc -Wall complained about. It
fixed quite a number of problems, and pointed out coding errors in new
patches.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Tue Jan 25 20:02:57 2000
Received: from hu.tm.ee (ppp809.tele2.ee [212.107.37.109])
by hub.org (8.9.3/8.9.3) with ESMTP id UAA43205
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 20:02:12 -0500 (EST) (envelope-from hannu@tm.ee)
Received: from tm.ee (localhost [127.0.0.1]) by hu.tm.ee (Postfix) with ESMTP
id 33FF21346A; Wed, 26 Jan 2000 03:08:27 +0200 (EET)
Sender: hannu@hu.tm.ee
Message-ID: <388E490B.AD66A06D@tm.ee>
Date: Wed, 26 Jan 2000 03:08:27 +0200
From: Hannu Krosing <hannu@tm.ee>
Organization: Trust-O-Matic =?iso-8859-1?Q?O=DC?=
X-Mailer: Mozilla 4.7 [en] (X11; I; Linux 2.2.13-7mdk i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Don Baccus <dhogaza@pacifier.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>,
"Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: Happy column adding (was RE: [HACKERS] Happycolumndropping)
References: <3.0.1.32.20000125113001.00f8acb0@mail.pacifier.com>
<20000125114453.E423@rice.edu>
<001401bf6704$5ca7e3a0$2801007e@tpf.co.jp>
<Pine.GSO.4.02A.10001251152160.11899-100000@Val.DoCS.UU.SE>
<3.0.1.32.20000125080125.00f7f160@mail.pacifier.com>
<20000125114453.E423@rice.edu>
<3.0.1.32.20000125113001.00f8acb0@mail.pacifier.com>
<3.0.1.32.20000125151022.00f8c4c0@mail.pacifier.com>
<3.0.1.32.20000125163804.00fa3bd0@mail.pacifier.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Don Baccus wrote:

At 02:25 AM 1/26/00 +0200, Hannu Krosing wrote:

multiple inheritance is tricky and _requires_ unique column ids maybe oids
from pg_attribute to be doable.

Having worked on a C++ compiler (one of the first, actually) I
suspect that this might be one of the lesser problems associated
with implementing a multiple inheritance scheme :)

I was aiming at a more loose way of doing multiple inheritance, something like
is used in python - you don't check too many things at compile time and
dynamically
lookup them when needed.

for this the most_basic_column_oid scheme might work.

having unique indexes that span multiple tables would of course be tricky too,
as would triggers and rules and ...

table a (oidA1, oidA2)
table b (oidB1, oidB2)
table c (oidC1) inherits(a,b)
alter table a drop column oidA2
alter table b add column oidB3
-->
table a - (oidA1,delA2)
table b - (oidB1,oidB2,oidB3)
table c - (oidA1,delA2,oidB1,oidB2,oidC1,oidB3)

seems the only sensible way for multiple inheritance to work with ADD/DROP
COLUMN
is going with column oids for storage.

Hidden columns only would work for single inheritance.

------------
Hannu

From bouncefilter Tue Jan 25 20:22:57 2000
Received: from sraigw.sra.co.jp (sraigw.sra.co.jp [202.32.10.2])
by hub.org (8.9.3/8.9.3) with ESMTP id UAA49241
for <pgsql-hackers@hub.org>; Tue, 25 Jan 2000 20:22:42 -0500 (EST)
(envelope-from t-ishii@sra.co.jp)
Received: from sranhk.sra.co.jp (sranhk [133.137.36.134])
by sraigw.sra.co.jp (8.8.7/3.7W-sraigw) with ESMTP id KAA06240;
Wed, 26 Jan 2000 10:21:19 +0900 (JST)
Received: from localhost (IDENT:t-ishii@srapc968-yotsuya [133.137.36.133])
by sranhk.sra.co.jp (8.9.3/3.7W-srambox) with ESMTP id KAA27518;
Wed, 26 Jan 2000 10:21:18 +0900
To: robinson@netrinsics.com
Cc: pgsql-hackers@hub.org, tgl@sss.pgh.pa.us
Subject: Re: [HACKERS] fatal copy in/out error (6.5.3)
In-Reply-To: <200001250605.OAA39630@netrinsics.com>
References: <20000125140004W.t-ishii@sra.co.jp>
<200001250605.OAA39630@netrinsics.com>
X-Mailer: Mew version 1.94 on Emacs 19.34 / Mule 2.3 (SUETSUMUHANA)
Mime-Version: 1.0
Content-Type: Text/Plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-Id: <20000126102351B.t-ishii@sra.co.jp>
Date: Wed, 26 Jan 2000 10:23:51 +0900
From: Tatsuo Ishii <t-ishii@sra.co.jp>
X-Dispatcher: imput version 990905(IM130)
Lines: 54

Tatsuo Ishii <t-ishii@sra.co.jp> writes:

Yes, it's not a PostgreSQL's business but is a really big problem in
the real world. Maybe some HTML gurus might have good suggestions on
these issues (something like using a language tag?)

The only solution is defensive programming. Even if there were a standard
that everyone followed, if malicious people could break things by not
following the standard, then you can be certain that somebody would do so.

Defensive programming saves the system but does not user. Once
corrupted data is stored in the system, it's totally useless for the
user anyway. What about validating data *before* inserting it into a
table? You expect EUC_CN data, and it should be possible to determine
if the data is valid or not by doing some simple checking in most
cases. Maybe I could provide a new libpq function something like:

bool pg_validate_euc_cn(const unsigned char *euc_str);

If it returns false, then euc_str is not a valid EUC_CN.
So you show a message:
"Sorry, but we only accepts EUC_CN data. Please try another input
method..." or jump to other pages for EUC_TW or Big5 or whatever...

Of course the function does not guarantee the string is 100% correct
EUC_CN (on the other hand it can tell that the string is not
valid) because:

1) there are chances that, for example, a EUC_CN string and a EUC_JP
string has same bit patterns accidently.

2) I do not have enough information to implement it perfectly. At this
point I could only perform minimal checking. However, it can be good
a start point for someone who has more knowledge (on the other hand, I
could implement pg_validate_euc_jp in much better way, since I have
precise info for EUC_JP).

Here it is. With this patch, copy out should be happy even with the
wrong data. I'm not sure if it could be displayed correctly, though.

Thank you very much. However, I think even this is too optimistic:

! if (*s & 0x80)

Shouldn't it be something like:

if ((*s & 0x80) && (*(s+1) & 0x80))

Even though "\242\242\242\0" is an invalid EUC sequence, it still shouldn't be
allowed to break the software.

Thanks for the suggestion. More robust code is always good.
--
Tatsuo Ishii

From bouncefilter Tue Jan 25 21:54:58 2000
Received: from localhost (IDENT:root@hectic-1.jpl.nasa.gov [128.149.68.203])
by hub.org (8.9.3/8.9.3) with ESMTP id VAA68936
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 21:54:55 -0500 (EST)
(envelope-from lockhart@alumni.caltech.edu)
Received: from alumni.caltech.edu (lockhart@localhost [127.0.0.1])
by localhost (8.8.7/8.8.7) with ESMTP id DAA08146;
Wed, 26 Jan 2000 03:03:18 GMT
Sender: lockhart@hub.org
Message-ID: <388E63F5.216E42B0@alumni.caltech.edu>
Date: Wed, 26 Jan 2000 03:03:17 +0000
From: Thomas Lockhart <lockhart@alumni.caltech.edu>
X-Mailer: Mozilla 4.6 [en] (X11; I; Linux 2.0.36 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
CC: Don Baccus <dhogaza@pacifier.com>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Re: Happy column adding and dropping
References: <3.0.1.32.20000125112001.0106bec0@mail.pacifier.com>
<14033.948820995@sss.pgh.pa.us>
<NDBBIJLOILGIKBGDINDFCEFHCCAA.Inoue@tpf.co.jp>
<14033.948820995@sss.pgh.pa.us> <20000125122907.F423@rice.edu>
<3.0.1.32.20000125112001.0106bec0@mail.pacifier.com>
<20000125142219.A1169@rice.edu>
<3.0.1.32.20000125124254.00f79ec0@mail.pacifier.com>
<20000125154839.B1169@rice.edu>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

As far as putting it on a web site, Ansi drafts are generally available
for public comment so it might be OK to do so. Is there a copyright
statement attached?

Hmm, the string 'copyright' (cas insensitive search) does not appear in
the text at all. This draft was done at DEC, according to the title page.

Right. I picked it up from a late, great DEC ftp site somewhere, and
might have been the one to forward it to you.

afaik, the drafts are in the public domain, or have a copyright which
allows distribution. They make their money from the final version...

- Thomas

--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California

From bouncefilter Tue Jan 25 22:05:59 2000
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67] (may
be forged)) by hub.org (8.9.3/8.9.3) with ESMTP id WAA73453
for <pgsql-hackers@postgreSQL.org>;
Tue, 25 Jan 2000 22:05:34 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
WAA10726;
Tue, 25 Jan 2000 22:04:17 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001260304.WAA10726@candle.pha.pa.us>
Subject: Re: [HACKERS] Re: Happy column adding and dropping
In-Reply-To: <388E63F5.216E42B0@alumni.caltech.edu> from Thomas Lockhart at
"Jan 26, 2000 03:03:17 am"
To: Thomas Lockhart <lockhart@alumni.caltech.edu>
Date: Tue, 25 Jan 2000 22:04:17 -0500 (EST)
CC: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>,
Don Baccus <dhogaza@pacifier.com>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

As far as putting it on a web site, Ansi drafts are generally available
for public comment so it might be OK to do so. Is there a copyright
statement attached?

Hmm, the string 'copyright' (cas insensitive search) does not appear in
the text at all. This draft was done at DEC, according to the title page.

Right. I picked it up from a late, great DEC ftp site somewhere, and
might have been the one to forward it to you.

afaik, the drafts are in the public domain, or have a copyright which
allows distribution. They make their money from the final version...

I have to admit, I was hoping for a nice PDF version with bookmarks.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Wed Jan 26 00:17:00 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA00961
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 00:13:56 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id VAA08798;
Tue, 25 Jan 2000 21:12:59 -0800 (PST)
Message-Id: <3.0.1.32.20000125203934.0106e610@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Tue, 25 Jan 2000 20:39:34 -0800
To: Bruce Momjian <pgman@candle.pha.pa.us>,
Thomas Lockhart <lockhart@alumni.caltech.edu>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: [HACKERS] Re: Happy column adding and dropping
Cc: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
In-Reply-To: <200001260304.WAA10726@candle.pha.pa.us>
References: <388E63F5.216E42B0@alumni.caltech.edu>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 10:04 PM 1/25/00 -0500, Bruce Momjian wrote:

I have to admit, I was hoping for a nice PDF version with bookmarks.

1992? Not likely :) I mean, not in its original form.

We live in a fast-changing world, don't we?

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Wed Jan 26 00:29:00 2000
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67] (may
be forged)) by hub.org (8.9.3/8.9.3) with ESMTP id AAA03248
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 00:28:32 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
AAA15565;
Wed, 26 Jan 2000 00:27:39 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001260527.AAA15565@candle.pha.pa.us>
Subject: Re: [HACKERS] Re: Happy column adding and dropping
In-Reply-To: <3.0.1.32.20000125203934.0106e610@mail.pacifier.com> from Don
Baccus at "Jan 25, 2000 08:39:34 pm"
To: Don Baccus <dhogaza@pacifier.com>
Date: Wed, 26 Jan 2000 00:27:39 -0500 (EST)
CC: Thomas Lockhart <lockhart@alumni.caltech.edu>,
"Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

At 10:04 PM 1/25/00 -0500, Bruce Momjian wrote:

I have to admit, I was hoping for a nice PDF version with bookmarks.

1992? Not likely :) I mean, not in its original form.

We live in a fast-changing world, don't we?

I was usually a paper manuals guy, but with PDF and bookmarks so you can
jump to the section you want, I like PDF better.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Wed Jan 26 00:27:00 2000
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA02951
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 00:26:41 -0500 (EST) (envelope-from Inoue@tpf.co.jp)
Received: from cadzone ([126.0.1.40] (may be forged))
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with SMTP
id OAA06828; Wed, 26 Jan 2000 14:26:30 +0900
From: "Hiroshi Inoue" <Inoue@tpf.co.jp>
To: "Don Baccus" <dhogaza@pacifier.com>
Cc: "PostgreSQL Development" <pgsql-hackers@postgreSQL.org>,
"Peter Eisentraut" <peter_e@gmx.net>
Subject: RE: Happy column adding (was RE: [HACKERS] Happy column dropping)
Date: Wed, 26 Jan 2000 14:32:16 +0900
Message-ID: <000801bf67be$b490d120$2801007e@tpf.co.jp>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
In-Reply-To: <3.0.1.32.20000125080125.00f7f160@mail.pacifier.com>
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
Importance: Normal

-----Original Message-----
From: owner-pgsql-hackers@postgreSQL.org
[mailto:owner-pgsql-hackers@postgreSQL.org]On Behalf Of Don Baccus

At 11:55 AM 1/25/00 +0100, Peter Eisentraut wrote:

On Tue, 25 Jan 2000, Hiroshi Inoue wrote:

Even default is not allowed in ADD COLUMN now.
There may be other reasons why they aren't allowed.

It's not a matter of *allowed*, it's a parsing deficiency. The fact that
there was a default declared gets silently ignored. If y'all allow ;) I
would like to fix that (have already started a bit) by perusing the code
in parse_func.c:transformCreateStmt and do the same for the alter table
add column part. Maybe and add/drop constraint will come out in

the end as

well.

However, heap_getattr still won't see the default since it simply
checks to see of the attribute number falls off the end of the
tuple and then returns null.

Sorry,the following question might be already answered but too
many postings for me.

Do we have to refer default value for already inserted rows ?
Doesn't 'default' have its meaning only when rows are about to be
inserted ?

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp

From bouncefilter Wed Jan 26 00:48:01 2000
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67] (may
be forged)) by hub.org (8.9.3/8.9.3) with ESMTP id AAA09505
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 00:47:43 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id AAA22702
for pgsql-hackers@postgreSQL.org; Wed, 26 Jan 2000 00:46:43 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001260546.AAA22702@candle.pha.pa.us>
Subject: Many files modified tonight
To: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Date: Wed, 26 Jan 2000 00:46:43 -0500 (EST)
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=ELM948865603-14234-0_
Content-Transfer-Encoding: 7bit

--ELM948865603-14234-0_
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

I had to modify many C files tonight. This will cause cvs updates to
take some time for users. Sorry.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

--ELM948865603-14234-0_
Content-Type: message/rfc822
Content-Disposition: inline
Content-Description: Forwarded message from The Hermit Hacker
Content-Transfer-Encoding: 7bit

Received: from thelab.hub.org (nat194.150.mpoweredpc.net [142.177.194.150]
(may be forged))
by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id AAA22579
for <pgman@candle.pha.pa.us>; Wed, 26 Jan 2000 00:43:10 -0500 (EST)
Received: from localhost (scrappy@localhost)
by thelab.hub.org (8.9.3/8.9.1) with ESMTP id BAA06663;
Wed, 26 Jan 2000 01:40:17 -0400 (AST) (envelope-from scrappy@hub.org)
X-Authentication-Warning: thelab.hub.org: scrappy owned process doing -bs
Date: Wed, 26 Jan 2000 01:40:17 -0400 (AST)
From: The Hermit Hacker <scrappy@hub.org>
To: Vadim Mikheev <vadim@krs.ru>
cc: Bruce Momjian <pgman@candle.pha.pa.us>, tgl@hub.org,
pgsql-core@postgreSQL.org
Subject: Re: [CORE] introduction (fwd)
In-Reply-To: <388E86F1.5C72B546@krs.ru>
Message-ID: <Pine.BSF.4.21.0001260139120.652-100000@thelab.hub.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Wed, 26 Jan 2000, Vadim Mikheev wrote:

Bruce Momjian wrote:

I'm curious now ...

Tom, did you receive one of these too? I'm wondering if he sent to all
developers, or just those of us listed as 'steering'....

Not to me -:)

Now that is *really* weird ... then again, I wonder how much it costs to
fly to Russia :) But, you would have thought they would have at least
made contact ...

Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

--ELM948865603-14234-0_--

From bouncefilter Wed Jan 26 01:02:00 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id BAA14579
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 01:01:51 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id BAA16270;
Wed, 26 Jan 2000 01:01:40 -0500 (EST)
To: Bruce Momjian <pgman@candle.pha.pa.us>
cc: Peter Eisentraut <peter_e@gmx.net>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] --enable-debug
In-reply-to: <200001260105.UAA05685@candle.pha.pa.us>
References: <200001260105.UAA05685@candle.pha.pa.us>
Comments: In-reply-to Bruce Momjian <pgman@candle.pha.pa.us>
message dated "Tue, 25 Jan 2000 20:05:11 -0500"
Date: Wed, 26 Jan 2000 01:01:40 -0500
Message-ID: <16267.948866500@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Currently, I believe -Wall is turned on automatically if your compiler
is gcc --- at least it seems to work that way on my platform (maybe
it's in the hpux_gcc template?). Anyway I get a pretty good set of -W
options without doing anything special.

I remember when we removed every warning gcc -Wall complained about.

Um, did we stop?

In my experience, gcc is an excellent detector of portability problems.
I believe that it is good practice to crank gcc's warning level out
to the max, and to fix everything that it complains about.

Some gcc warnings are not really portability problems. For example,
it may gripe about "variable X may be used without being initialized"
when a human can easily see from the code logic that X will not actually
get read in any execution path that it wasn't previously set in.
Still, I'm willing to shut gcc up by adding a redundant initialization
of X, rather than turning down its warning level or making a habit of
ignoring what it says. If you go down one of those other paths you
are likely to miss warnings that could have saved you from genuine bugs.

In short: I like to see Postgres compile with no warnings at all
even with a very anal-retentive set of gcc settings.

Just to give you an idea of what I consider anal-retentive ;-),
here is what I customarily compile the IJG JPEG sources with:

CFLAGS= -g -O2 -ansi -Wall -Wtraditional -Wshadow -Wwrite-strings \
-Wpointer-arith -Wbad-function-cast -Wcast-qual -Wid-clash-15 \
-Wstrict-prototypes -Wmissing-prototypes
# -Wconversion would be nice, but it complains about some stdio macros on HPUX

Before any release I also eliminate any warnings shown by several
other Unix and PC compilers cranked out to their max warning levels.
I believe that this practice has helped to make libjpeg one of the
most portable libraries on the planet.

Compiler warnings can be your friend. Use 'em.

regards, tom lane
organizer, Independent JPEG Group

From bouncefilter Wed Jan 26 01:10:01 2000
Received: from clio.trends.ca (root@clio.trends.ca [209.47.148.2])
by hub.org (8.9.3/8.9.3) with ESMTP id BAA20813
for <pgsql-hackers@postgresql.org>;
Wed, 26 Jan 2000 01:09:48 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: from candle.pha.pa.us (s5-03.ppp.op.net [209.152.195.67])
by clio.trends.ca (8.9.3+Sun/8.9.3) with ESMTP id BAA24952
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 01:04:57 -0500 (EST)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
BAA23774;
Wed, 26 Jan 2000 01:04:12 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <200001260604.BAA23774@candle.pha.pa.us>
Subject: Re: [HACKERS] --enable-debug
In-Reply-To: <16267.948866500@sss.pgh.pa.us> from Tom Lane at "Jan 26,
2000 01:01:40 am"
To: Tom Lane <tgl@sss.pgh.pa.us>
Date: Wed, 26 Jan 2000 01:04:12 -0500 (EST)
CC: Peter Eisentraut <peter_e@gmx.net>,
PostgreSQL Development <pgsql-hackers@postgresql.org>
X-Mailer: ELM [version 2.4ME+ PL66 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Currently, I believe -Wall is turned on automatically if your compiler
is gcc --- at least it seems to work that way on my platform (maybe
it's in the hpux_gcc template?). Anyway I get a pretty good set of -W
options without doing anything special.

I remember when we removed every warning gcc -Wall complained about.

Um, did we stop?

In my experience, gcc is an excellent detector of portability problems.
I believe that it is good practice to crank gcc's warning level out
to the max, and to fix everything that it complains about.

No, we fixed all the code gcc warned about. Our code is 100% clean.
The only problems are fork() handling, and patches people have submitted
that were not 100% clean and will be fixed before final.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Wed Jan 26 03:40:03 2000
Received: from hu.tm.ee (ppp732.tele2.ee [212.107.37.32])
by hub.org (8.9.3/8.9.3) with ESMTP id DAA58123
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 03:39:26 -0500 (EST) (envelope-from hannu@tm.ee)
Received: from tm.ee (localhost [127.0.0.1]) by hu.tm.ee (Postfix) with ESMTP
id 52495134C0; Wed, 26 Jan 2000 10:45:42 +0200 (EET)
Sender: hannu@hu.tm.ee
Message-ID: <388EB436.8AE7334F@tm.ee>
Date: Wed, 26 Jan 2000 10:45:42 +0200
From: Hannu Krosing <hannu@tm.ee>
Organization: Trust-O-Matic =?iso-8859-1?Q?O=DC?=
X-Mailer: Mozilla 4.7 [en] (X11; I; Linux 2.2.13-7mdk i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Hiroshi Inoue <Inoue@tpf.co.jp>
Cc: Don Baccus <dhogaza@pacifier.com>,
PostgreSQL Development <pgsql-hackers@postgreSQL.org>,
Peter Eisentraut <peter_e@gmx.net>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column dropping)
References: <000801bf67be$b490d120$2801007e@tpf.co.jp>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hiroshi Inoue wrote:

However, heap_getattr still won't see the default since it simply
checks to see of the attribute number falls off the end of the
tuple and then returns null.

Sorry,the following question might be already answered but too
many postings for me.

Do we have to refer default value for already inserted rows ?
Doesn't 'default' have its meaning only when rows are about to be
inserted ?

I think the case was about adding a NOT NULL column and setting current NULL
columns to DEFAULT seemed like a natural thing to do.

But, considering the fact that DEFAULT can be something reaaly complex, like
function that does a lot of things, it may be better to have the constraints
checked at the end of transaction, like

BEGIN;
ALTER TABLE T1 ADD COLUMN C1 TEXT NOT NULL;
UPDATE T1 SET C1='MYDEFAULTVALUE';
COMMIT;

only IIRC we have quite poor support for DDL statements inside transactions.

---------------
Hannu

From bouncefilter Wed Jan 26 03:54:03 2000
Received: from gandalf.telecom.at (gandalf.telecom.at [194.118.26.84])
by hub.org (8.9.3/8.9.3) with ESMTP id DAA59787
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 03:53:17 -0500 (EST)
(envelope-from ZeugswetterA@wien.spardat.at)
Received: from sdexcgtw01.f000.d0188.sd.spardat.at
(sdexcgtw01.f000.d0188.sd.spardat.at [172.18.1.16])
by gandalf.telecom.at (xxx/xxx) with ESMTP id JAA128048;
Wed, 26 Jan 2000 09:51:41 +0100
Received: by sdexcgtw01.f000.d0188.sd.spardat.at with Internet Mail Service
(5.5.2448.0) id <CCAL3VZ2>; Wed, 26 Jan 2000 09:51:40 +0100
Message-ID:
<219F68D65015D011A8E000006F8590C603FDC21E@sdexcsrv1.f000.d0188.sd.spardat.at>
From: Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>
To: "'Peter Eisentraut'" <peter_e@gmx.net>
Cc: "'Tom Lane'" <tgl@sss.pgh.pa.us>, "'pgsql-hackers@postgreSQL.org'"
<pgsql-hackers@postgreSQL.org>
Subject: AW: AW: [HACKERS] Some notes on optimizer cost estimates
Date: Wed, 26 Jan 2000 09:51:40 +0100
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2448.0)
Content-Type: text/plain;
charset="iso-8859-1"

Couldn't we test some of these parameters inside

configure and set

them there?

If we could figure out a reasonably cheap way of estimating these
numbers, it'd be worth setting up custom values at

installation time.

Imho this whole idea is not so good. (Sorry)

My points are:
1. even if it is good for an optimizer to be smart,
it is even more important, that it is predictable

ISTM that by the nature of things the most important capability of an
optimizer is to yield optimal results.

Yes, but beleive me it cannot do that.
There are various reasons for that e.g.:
1. bad/old statistics
2. not considered concurrency issues
3. iterference from other applications
It will only lead a high percentage of optimal plans.
The rest ranges from not so good to far off. It is the responsibility of the
SQL programmer to avoid these, but he can only do that if the
optimizer is predictable.

This, however, does
not have to be
mutually exclusive with predictability. If you estimate some
CPU and disk
parameters and write them into a config file, then you can always give
this config file to a bug fixer. It will still work on his machine, just
less than optimally.

That is an idea, yes. But I doubt that it is really worth it.
Imho it would be more important to consider concurrency
issues first, and that would be very hard to do.

e.g.

2 sessions doing seq scan on huge table x.
they start their query with a time offset, that
does not allow session 2 to use pages from session 1
buffer cache (not enough memory).
It would be optimal if session 1 would wait so long that
session 2 can read from cache.
They would both benefit from this strategy.

Andreas

From bouncefilter Wed Jan 26 03:59:03 2000
Received: from gandalf.telecom.at (gandalf.telecom.at [194.118.26.84])
by hub.org (8.9.3/8.9.3) with ESMTP id DAA60379
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 03:58:11 -0500 (EST)
(envelope-from ZeugswetterA@wien.spardat.at)
Received: from sdexcgtw01.f000.d0188.sd.spardat.at
(sdexcgtw01.f000.d0188.sd.spardat.at [172.18.1.16])
by gandalf.telecom.at (xxx/xxx) with ESMTP id JAA205372
for <pgsql-hackers@postgreSQL.org>; Wed, 26 Jan 2000 09:58:09 +0100
Received: by sdexcgtw01.f000.d0188.sd.spardat.at with Internet Mail Service
(5.5.2448.0) id <CCAL3V5R>; Wed, 26 Jan 2000 09:58:08 +0100
Message-ID:
<219F68D65015D011A8E000006F8590C603FDC21F@sdexcsrv1.f000.d0188.sd.spardat.at>
From: Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>
To: "'pgsql-hackers@postgreSQL.org'" <pgsql-hackers@postgreSQL.org>
Subject: AW: [HACKERS] Well, then you keep your darn columns
Date: Wed, 26 Jan 2000 09:58:08 +0100
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2448.0)
Content-Type: text/plain;
charset="iso-8859-1"

"Hiroshi Inoue" <Inoue@tpf.co.jp> writes:

My idea is as follows.

1)add a visibile/invisible flag to pg_attribute

might use a xid as flag, so rows before xid have the column, rows after
xid don't have it.

Andreas

From bouncefilter Wed Jan 26 04:10:03 2000
Received: from gandalf.telecom.at (gandalf.telecom.at [194.118.26.84])
by hub.org (8.9.3/8.9.3) with ESMTP id EAA65258
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 04:09:49 -0500 (EST)
(envelope-from ZeugswetterA@wien.spardat.at)
Received: from sdexcgtw01.f000.d0188.sd.spardat.at
(sdexcgtw01.f000.d0188.sd.spardat.at [172.18.1.16])
by gandalf.telecom.at (xxx/xxx) with ESMTP id KAA191862
for <pgsql-hackers@postgreSQL.org>; Wed, 26 Jan 2000 10:09:47 +0100
Received: by sdexcgtw01.f000.d0188.sd.spardat.at with Internet Mail Service
(5.5.2448.0) id <CCAL3V7A>; Wed, 26 Jan 2000 10:05:27 +0100
Message-ID:
<219F68D65015D011A8E000006F8590C603FDC220@sdexcsrv1.f000.d0188.sd.spardat.at>
From: Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>
To: "'pgsql-hackers@postgreSQL.org'" <pgsql-hackers@postgreSQL.org>
Subject: AW: [HACKERS] Well, then you keep your darn columns
Date: Wed, 26 Jan 2000 10:05:26 +0100
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2448.0)
Content-Type: text/plain;
charset="iso-8859-1"

Oh, there is a second drawback to it though ...

DROP COLUMN name

the dropped column looses its original name, and gets a
system generated name, like " dropped_col_5"

ADD COLUMN name <of a different type>

Then what? :(

Andreas

From bouncefilter Wed Jan 26 04:21:03 2000
Received: from acheron.rime.com.au (root@albatr.lnk.telstra.net
[139.130.54.222]) by hub.org (8.9.3/8.9.3) with ESMTP id EAA66481
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 04:21:00 -0500 (EST)
(envelope-from pjw@rhyme.com.au)
Received: from oberon (Oberon.rime.com.au [203.8.195.100])
by acheron.rime.com.au (8.9.3/8.9.3) with SMTP id UAA23436;
Wed, 26 Jan 2000 20:05:49 +1100
Message-Id: <3.0.5.32.20000126200717.03472bf0@mail.rhyme.com.au>
X-Sender: pjw@mail.rhyme.com.au
X-Mailer: QUALCOMM Windows Eudora Pro Version 3.0.5 (32)
Date: Wed, 26 Jan 2000 20:07:17 +1100
To: Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>,
"'Peter Eisentraut'" <peter_e@gmx.net>
From: Philip Warner <pjw@rhyme.com.au>
Subject: Re: AW: AW: [HACKERS] Some notes on optimizer cost estimates
Cc: "'Tom Lane'" <tgl@sss.pgh.pa.us>,
"'pgsql-hackers@postgreSQL.org'" <pgsql-hackers@postgreSQL.org>
In-Reply-To: <219F68D65015D011A8E000006F8590C603FDC21E@sdexcsrv1.f000.d0
188.sd.spardat.at>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 09:51 26/01/00 +0100, Zeugswetter Andreas SB wrote:

Couldn't we test some of these parameters inside

configure and set

them there?

If we could figure out a reasonably cheap way of estimating these
numbers, it'd be worth setting up custom values at

installation time.

Imho this whole idea is not so good. (Sorry)

My points are:
1. even if it is good for an optimizer to be smart,
it is even more important, that it is predictable

ISTM that by the nature of things the most important capability of an
optimizer is to yield optimal results.

Yes, but beleive me it cannot do that.
There are various reasons for that e.g.:
1. bad/old statistics
2. not considered concurrency issues
3. iterference from other applications
It will only lead a high percentage of optimal plans.
The rest ranges from not so good to far off. It is the responsibility of the
SQL programmer to avoid these, but he can only do that if the
optimizer is predictable.

Based on experience with optimizer improvements across releases of DB
products (not PostgreSQL, I hastily add), I would be inclined to say (from
bitter experience) that no optimizer is ever truly predicatable. The SQL
programmer has to be given the tools to ensure that a 'bad' query can be
forced to run the same way with each release, and release notes should
indicate what extra strategies are now available, in case the 'bad' query
can be made better.

It gets my goat (a bit) when commercial DB manufacturers believe that they
can solve intractable optimization problems - it would be a pity for PGSQL
to go the same way. I'd love to have the opportunity to prove my point with
PGSQL, but since I can't affect the optimizers choices in any way, I am
left with rhetoric, and examples from commercial DBs, which aren't, really,
relevant.

Sorry about the chestnut.

This, however, does
not have to be
mutually exclusive with predictability. If you estimate some
CPU and disk
parameters and write them into a config file, then you can always give
this config file to a bug fixer. It will still work on his machine, just
less than optimally.

That is an idea, yes. But I doubt that it is really worth it.
Imho it would be more important to consider concurrency
issues first, and that would be very hard to do.

e.g.

2 sessions doing seq scan on huge table x.
they start their query with a time offset, that
does not allow session 2 to use pages from session 1
buffer cache (not enough memory).
It would be optimal if session 1 would wait so long that
session 2 can read from cache.
They would both benefit from this strategy.

Andreas

************

----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.C.N. 008 659 498) | /(@) ______---_
Tel: +61-03-5367 7422 | _________ \
Fax: +61-03-5367 7430 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/

From bouncefilter Wed Jan 26 04:16:03 2000
Received: from gandalf.telecom.at (gandalf.telecom.at [194.118.26.84])
by hub.org (8.9.3/8.9.3) with ESMTP id EAA65799
for <pgsql-hackers@postgresql.org>;
Wed, 26 Jan 2000 04:15:33 -0500 (EST)
(envelope-from ZeugswetterA@wien.spardat.at)
Received: from sdexcgtw01.f000.d0188.sd.spardat.at
(sdexcgtw01.f000.d0188.sd.spardat.at [172.18.1.16])
by gandalf.telecom.at (xxx/xxx) with ESMTP id KAA13610
for <pgsql-hackers@postgresql.org>; Wed, 26 Jan 2000 10:15:30 +0100
Received: by sdexcgtw01.f000.d0188.sd.spardat.at with Internet Mail Service
(5.5.2448.0) id <CCAL3V8P>; Wed, 26 Jan 2000 10:15:30 +0100
Message-ID:
<219F68D65015D011A8E000006F8590C603FDC221@sdexcsrv1.f000.d0188.sd.spardat.at>
From: Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>
To: "'pgsql-hackers@postgresql.org'" <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Well, then you keep your darn columns (oid)
Date: Wed, 26 Jan 2000 10:15:29 +0100
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2448.0)
Content-Type: text/plain;
charset="iso-8859-1"

If it is now a general opinion that OIDs will remain, it
should be stated somewhere,
as I suspect that much of pg community operates under the
impression that they are
going away in future as have some other nice but not fully
developed features like time travel.

I think the concesus was, that the feature would stay, but that
you would have to specify on a per table basis that oid's
are wanted.

create table foo (bar char(16)) with oids;

Andreas

From bouncefilter Wed Jan 26 04:24:03 2000
Received: from gandalf.telecom.at (gandalf.telecom.at [194.118.26.84])
by hub.org (8.9.3/8.9.3) with ESMTP id EAA66834
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 04:23:49 -0500 (EST)
(envelope-from ZeugswetterA@wien.spardat.at)
Received: from sdexcgtw01.f000.d0188.sd.spardat.at
(sdexcgtw01.f000.d0188.sd.spardat.at [172.18.1.16])
by gandalf.telecom.at (xxx/xxx) with ESMTP id KAA194682
for <pgsql-hackers@postgreSQL.org>; Wed, 26 Jan 2000 10:23:43 +0100
Received: by sdexcgtw01.f000.d0188.sd.spardat.at with Internet Mail Service
(5.5.2448.0) id <CCAL3V0G>; Wed, 26 Jan 2000 10:23:43 +0100
Message-ID:
<219F68D65015D011A8E000006F8590C603FDC222@sdexcsrv1.f000.d0188.sd.spardat.at>
From: Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>
To: "'pgsql-hackers@postgreSQL.org'" <pgsql-hackers@postgreSQL.org>
Subject: AW: [HACKERS] Well, then you keep your darn columns
Date: Wed, 26 Jan 2000 10:23:43 +0100
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2448.0)
Content-Type: text/plain;
charset="iso-8859-1"

But the decision was (from Vadim IIRC) to drop them, at

least in non system

tables.
The cited reasons were:
* crappy implementation that taxed performance (probably

fixed by now)

* nobody else seemed to have them and the push then was to

the direction of

mainstream bean-counting DB with main objective of getting

that base

functionality right.

Regarding this last, Oracle has an equivalent - rowid. In the web
toolkit I'm helping port, it's used somewhat often, and having oid
available has been a convenience.

rowid's are something completely different.
They do exist in Postgresql, and are called xtid.
(I suggested naming them rowid on the SQL level)
They have the advantage, that they don't need an index,
because they represent the physical location of the row.

Andreas

From bouncefilter Wed Jan 26 04:42:03 2000
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
by hub.org (8.9.3/8.9.3) with ESMTP id EAA72455
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 04:41:50 -0500 (EST) (envelope-from Inoue@tpf.co.jp)
Received: from cadzone ([126.0.1.40] (may be forged))
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with SMTP
id SAA06974; Wed, 26 Jan 2000 18:41:27 +0900
From: "Hiroshi Inoue" <Inoue@tpf.co.jp>
To: "Zeugswetter Andreas SB" <ZeugswetterA@wien.spardat.at>
Cc: <pgsql-hackers@postgreSQL.org>
Subject: RE: [HACKERS] Well, then you keep your darn columns
Date: Wed, 26 Jan 2000 18:47:11 +0900
Message-ID: <000f01bf67e2$51ac9d40$2801007e@tpf.co.jp>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
In-Reply-To:
<219F68D65015D011A8E000006F8590C603FDC21F@sdexcsrv1.f000.d0188.sd.spardat.at>
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
Importance: Normal

-----Original Message-----
From: owner-pgsql-hackers@postgreSQL.org
[mailto:owner-pgsql-hackers@postgreSQL.org]On Behalf Of Zeugswetter
Andreas SB

"Hiroshi Inoue" <Inoue@tpf.co.jp> writes:

My idea is as follows.

1)add a visibile/invisible flag to pg_attribute

might use a xid as flag, so rows before xid have the column, rows after
xid don't have it.

From where do we get the xid ?

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp

From bouncefilter Wed Jan 26 05:44:04 2000
Received: from gandalf.telecom.at (gandalf.telecom.at [194.118.26.84])
by hub.org (8.9.3/8.9.3) with ESMTP id FAA87073
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 05:43:14 -0500 (EST)
(envelope-from ZeugswetterA@wien.spardat.at)
Received: from sdexcgtw01.f000.d0188.sd.spardat.at
(sdexcgtw01.f000.d0188.sd.spardat.at [172.18.1.16])
by gandalf.telecom.at (xxx/xxx) with ESMTP id LAA201022;
Wed, 26 Jan 2000 11:42:56 +0100
Received: by sdexcgtw01.f000.d0188.sd.spardat.at with Internet Mail Service
(5.5.2448.0) id <CCAL3WP5>; Wed, 26 Jan 2000 11:42:56 +0100
Message-ID:
<219F68D65015D011A8E000006F8590C603FDC223@sdexcsrv1.f000.d0188.sd.spardat.at>
From: Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>
To: "'Hiroshi Inoue'" <Inoue@tpf.co.jp>
Cc: pgsql-hackers@postgreSQL.org
Subject: AW: [HACKERS] Well, then you keep your darn columns
Date: Wed, 26 Jan 2000 11:42:55 +0100
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2448.0)
Content-Type: text/plain;
charset="iso-8859-1"

My idea is as follows.

1)add a visibile/invisible flag to pg_attribute

might use a xid as flag, so rows before xid have the

column, rows after

xid don't have it.

From where do we get the xid ?

Transaction id of alter table command ?

Andreas

From bouncefilter Wed Jan 26 05:57:04 2000
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
by hub.org (8.9.3/8.9.3) with ESMTP id FAA88944
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 05:56:57 -0500 (EST) (envelope-from Inoue@tpf.co.jp)
Received: from cadzone ([126.0.1.40] (may be forged))
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with SMTP
id TAA07003; Wed, 26 Jan 2000 19:56:50 +0900
From: "Hiroshi Inoue" <Inoue@tpf.co.jp>
To: "Zeugswetter Andreas SB" <ZeugswetterA@wien.spardat.at>
Cc: <pgsql-hackers@postgreSQL.org>
Subject: RE: [HACKERS] Well, then you keep your darn columns
Date: Wed, 26 Jan 2000 20:02:36 +0900
Message-ID: <001001bf67ec$da596e20$2801007e@tpf.co.jp>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
In-Reply-To:
<219F68D65015D011A8E000006F8590C603FDC223@sdexcsrv1.f000.d0188.sd.spardat.at>
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
Importance: Normal

-----Original Message-----
From: owner-pgsql-hackers@postgreSQL.org
[mailto:owner-pgsql-hackers@postgreSQL.org]On Behalf Of Zeugswetter
Andreas SB

My idea is as follows.

1)add a visibile/invisible flag to pg_attribute

might use a xid as flag, so rows before xid have the

column, rows after

xid don't have it.

From where do we get the xid ?

Transaction id of alter table command ?

Does it work well after one more column is added ?

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp

From bouncefilter Wed Jan 26 06:00:05 2000
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
by hub.org (8.9.3/8.9.3) with ESMTP id FAA89294
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 05:59:34 -0500 (EST) (envelope-from Inoue@tpf.co.jp)
Received: from cadzone ([126.0.1.40] (may be forged))
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with SMTP
id TAA07004; Wed, 26 Jan 2000 19:59:15 +0900
From: "Hiroshi Inoue" <Inoue@tpf.co.jp>
To: "Hannu Krosing" <hannu@tm.ee>
Cc: "Don Baccus" <dhogaza@pacifier.com>,
"PostgreSQL Development" <pgsql-hackers@postgreSQL.org>,
"Peter Eisentraut" <peter_e@gmx.net>
Subject: RE: Happy column adding (was RE: [HACKERS] Happy column dropping)
Date: Wed, 26 Jan 2000 20:05:00 +0900
Message-ID: <001101bf67ed$304372e0$2801007e@tpf.co.jp>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
In-Reply-To: <388EB436.8AE7334F@tm.ee>
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
Importance: Normal

-----Original Message-----
From: hannu@hu.tm.ee [mailto:hannu@hu.tm.ee]On Behalf Of Hannu Krosing

Hiroshi Inoue wrote:

However, heap_getattr still won't see the default since it simply
checks to see of the attribute number falls off the end of the
tuple and then returns null.

Sorry,the following question might be already answered but too
many postings for me.

Do we have to refer default value for already inserted rows ?
Doesn't 'default' have its meaning only when rows are about to be
inserted ?

Oh,I found it from Ross's posting.

General Rules

1) The column defined by the <column definition> is added to T.

2) Let C be the column added to T. Every value in C is the default
value for C.

Note: The default value of a column is defined in Subclause 11.5,
"<default clause>"

Is this reasonable ?
.

I think the case was about adding a NOT NULL column and setting
current NULL
columns to DEFAULT seemed like a natural thing to do.

But, considering the fact that DEFAULT can be something reaaly
complex, like
function that does a lot of things, it may be better to have the
constraints
checked at the end of transaction, like

BEGIN;
ALTER TABLE T1 ADD COLUMN C1 TEXT NOT NULL;

Isn't 'iNITIALLY DEFERRED' needed ?
ALTER TABLE T1 ADD COLUMN C1 TEXT NOT NULL INITIALLY DEFERRED;

UPDATE T1 SET C1='MYDEFAULTVALUE';
COMMIT;

It seems more reasonable than standard.
But is it worth breaking SQL standard ?

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp

From bouncefilter Wed Jan 26 06:14:04 2000
Received: from s-nath-exch1.nath-gpbry.co.za (mail.newaftech.com
[209.212.104.161]) by hub.org (8.9.3/8.9.3) with ESMTP id GAA07377
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 06:13:27 -0500 (EST)
(envelope-from Michael.Ansley@intec.co.za)
Received: by S-NATH-EXCH1 with Internet Mail Service (5.5.2448.0)
id <D3KATPVX>; Wed, 26 Jan 2000 13:09:38 +0200
Message-ID: <1BF7C7482189D211B03F00805F8527F748C494@S-NATH-EXCH2>
From: "Ansley, Michael" <Michael.Ansley@intec.co.za>
To: Peter Eisentraut <peter_e@gmx.net>
Cc: PostgreSQL Development <pgsql-hackers@postgreSQL.org>, "'Bruce Momjian'"
<pgman@candle.pha.pa.us>, Tom Lane <tgl@sss.pgh.pa.us>
Subject: RE: [HACKERS] --enable-debug
Date: Wed, 26 Jan 2000 13:10:50 +0200
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2448.0)
Content-Type: text/plain;
charset="iso-8859-1"

When I initially mentioned this, I had tried to debug, only to find that
there were no symbols. So I tried the standard autoconf option, which is
--enable-debug, to get some symbols. I'm not particularly concerned about
--cassert, but --enable-debug is the standard autoconf option, and most
people (developers) are aware of it. If it's just a synonym for --cassert,
well, I don't really care. I just want some symbols to debug with. I
landed up fixing it by setting
CC=gcc -g -Wall

or something, and recompiling.

MikeA

-----Original Message-----
From: Bruce Momjian [mailto:pgman@candle.pha.pa.us]
Sent: Wednesday, January 26, 2000 8:04 AM
To: Tom Lane
Cc: Peter Eisentraut; PostgreSQL Development
Subject: Re: [HACKERS] --enable-debug

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Currently, I believe -Wall is turned on automatically

if your compiler

is gcc --- at least it seems to work that way on my

platform (maybe

it's in the hpux_gcc template?). Anyway I get a pretty

good set of -W

options without doing anything special.

I remember when we removed every warning gcc -Wall

complained about.

Um, did we stop?

In my experience, gcc is an excellent detector of

portability problems.

I believe that it is good practice to crank gcc's warning level out
to the max, and to fix everything that it complains about.

No, we fixed all the code gcc warned about. Our code is 100% clean.
The only problems are fork() handling, and patches people
have submitted
that were not 100% clean and will be fixed before final.

-- 
Bruce Momjian                        |  http://www.op.net/~candle
pgman@candle.pha.pa.us               |  (610) 853-3000
+  If your life is a hard drive,     |  830 Blythe Avenue
+  Christ can be your backup.        |  Drexel Hill, 
Pennsylvania 19026

************

From bouncefilter Wed Jan 26 07:26:05 2000
Received: from gandalf.telecom.at (gandalf.telecom.at [194.118.26.84])
by hub.org (8.9.3/8.9.3) with ESMTP id HAA22074
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 07:25:59 -0500 (EST)
(envelope-from ZeugswetterA@wien.spardat.at)
Received: from sdexcgtw01.f000.d0188.sd.spardat.at
(sdexcgtw01.f000.d0188.sd.spardat.at [172.18.1.16])
by gandalf.telecom.at (xxx/xxx) with ESMTP id NAA191822
for <pgsql-hackers@postgreSQL.org>; Wed, 26 Jan 2000 13:25:57 +0100
Received: by sdexcgtw01.f000.d0188.sd.spardat.at with Internet Mail Service
(5.5.2448.0) id <CCAL3X10>; Wed, 26 Jan 2000 13:25:56 +0100
Message-ID:
<219F68D65015D011A8E000006F8590C603FDC224@sdexcsrv1.f000.d0188.sd.spardat.at>
From: Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>
To: "'pgsql-hackers@postgreSQL.org'" <pgsql-hackers@postgreSQL.org>
Subject: AW: AW: AW: [HACKERS] Some notes on optimizer cost estimates
Date: Wed, 26 Jan 2000 13:25:55 +0100
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2448.0)
Content-Type: text/plain;
charset="iso-8859-1"

Based on experience with optimizer improvements across releases of DB
products (not PostgreSQL, I hastily add), I would be inclined
to say (from
bitter experience) that no optimizer is ever truly
predicatable. The SQL
programmer has to be given the tools to ensure that a 'bad'
query can be
forced to run the same way with each release, and release notes should
indicate what extra strategies are now available, in case the
'bad' query
can be made better.

Yes, I think syntax to force or disallow a particular index,
choose a join method or order, force/disallow seq scans ...
is sometimes useful.
Even Informix, who always refused to supply such a feature
now has it.

Andreas

From bouncefilter Wed Jan 26 08:09:05 2000
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
by hub.org (8.9.3/8.9.3) with ESMTP id IAA34481
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 08:08:06 -0500 (EST) (envelope-from Inoue@tpf.co.jp)
Received: from mcadnote1 (ppm225.noc.fukui.nsk.ne.jp [210.161.188.100])
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with SMTP
id WAA07049; Wed, 26 Jan 2000 22:07:46 +0900
From: "Hiroshi Inoue" <Inoue@tpf.co.jp>
To: "Hannu Krosing" <hannu@tm.ee>
Cc: "Don Baccus" <dhogaza@pacifier.com>,
"PostgreSQL Development" <pgsql-hackers@postgreSQL.org>,
"Peter Eisentraut" <peter_e@gmx.net>
Subject: RE: Happy column adding (was RE: [HACKERS] Happy column dropping)
Date: Wed, 26 Jan 2000 22:08:44 +0900
Message-ID: <NDBBIJLOILGIKBGDINDFIEFOCCAA.Inoue@tpf.co.jp>
MIME-Version: 1.0
Content-Type: text/plain;
charset="Windows-1252"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
Importance: Normal
In-Reply-To:
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300

-----Original Message-----
From: Hiroshi Inoue [mailto:Inoue@tpf.co.jp]

But, considering the fact that DEFAULT can be something reaaly
complex, like
function that does a lot of things, it may be better to have the
constraints
checked at the end of transaction, like

BEGIN;
ALTER TABLE T1 ADD COLUMN C1 TEXT NOT NULL;

Isn't 'iNITIALLY DEFERRED' needed ?
ALTER TABLE T1 ADD COLUMN C1 TEXT NOT NULL INITIALLY DEFERRED;

UPDATE T1 SET C1='MYDEFAULTVALUE';
COMMIT;

It seems more reasonable than standard.
But is it worth breaking SQL standard ?

Seems I was wrong. It's not so good.
Current spec to reject NOT NULL/DEFAULT for new column
isn't so bad and ADD CONSTRAINT feature is much more
important. How about the following ?

BEGIN;
ALTER TABLE T1 ADD COLUMN C1 TEXT;
UPDATE T1 SET C1='MYDEFAULTVALUE';
ALTER TABLE T1 ADD CONSTRAINT CHECK C1 NOT NULL;
COMMIT;

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp

From bouncefilter Wed Jan 26 08:23:05 2000
Received: from bologna.nettuno.it (bologna.nettuno.it [193.43.2.1])
by hub.org (8.9.3/8.9.3) with ESMTP id IAA36193
for <pgsql-hackers@postgresql.org>;
Wed, 26 Jan 2000 08:22:25 -0500 (EST)
(envelope-from jose@sferacarta.com)
Received: from proxy.sferacarta.com (sfcabop1.nettuno.it [193.207.10.213])
by bologna.nettuno.it (8.9.3/8.9.3/NETTuno 4.1) with ESMTP id OAA23958;
Wed, 26 Jan 2000 14:21:54 +0100 (MET)
Received: from rosso.sferacarta.com (sferacarta.com) [10.20.30.5]
by proxy.sferacarta.com with esmtp (Exim 2.05 #1 (Debian))
id 12DTJk-0004fr-00; Wed, 26 Jan 2000 14:21:08 +0000
Message-ID: <388EF57E.C0FBF2EC@sferacarta.com>
Date: Wed, 26 Jan 2000 14:24:14 +0100
From: Jose Soares <jose@sferacarta.com>
X-Mailer: Mozilla 4.6 [en] (WinNT; I)
X-Accept-Language: en
MIME-Version: 1.0
To: Hiroshi Inoue <Inoue@tpf.co.jp>
CC: Don Baccus <dhogaza@pacifier.com>,
PostgreSQL Development <pgsql-hackers@postgresql.org>,
Peter Eisentraut <peter_e@gmx.net>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column dropping)
References: <000801bf67be$b490d120$2801007e@tpf.co.jp>
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit

Hiroshi Inoue wrote:

-----Original Message-----
From: owner-pgsql-hackers@postgreSQL.org
[mailto:owner-pgsql-hackers@postgreSQL.org]On Behalf Of Don Baccus

At 11:55 AM 1/25/00 +0100, Peter Eisentraut wrote:

On Tue, 25 Jan 2000, Hiroshi Inoue wrote:

Even default is not allowed in ADD COLUMN now.
There may be other reasons why they aren't allowed.

It's not a matter of *allowed*, it's a parsing deficiency. The fact that
there was a default declared gets silently ignored. If y'all allow ;) I
would like to fix that (have already started a bit) by perusing the code
in parse_func.c:transformCreateStmt and do the same for the alter table
add column part. Maybe and add/drop constraint will come out in

the end as

well.

However, heap_getattr still won't see the default since it simply
checks to see of the attribute number falls off the end of the
tuple and then returns null.

Sorry,the following question might be already answered but too
many postings for me.

Do we have to refer default value for already inserted rows ?
Doesn't 'default' have its meaning only when rows are about to be
inserted ?

Of course yes.
from "A guide to SQL Standard" page 106:

"ALTER TABLE S ADD COLUMN DISCOUNT SMALLINT DEFAULT -1

this statement adds a new fifth column called DISCOUNT to base table S. All
existing S rows are extended to include a value for new column; that value is
-1 in every such row...."

The problem is when we define a constraint for the column like:

ALTER TABLE S ADD COLUMN DISCOUNT SMALLINT NOT NULL

In such case IMO this should be refused because it violates data integrity,
an less you define also a default value for the column as in:

ALTER TABLE S ADD COLUMN DISCOUNT SMALLINT NOT NULL DEFAULT -1

Jos���

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp

************

From bouncefilter Wed Jan 26 08:37:06 2000
Received: from bologna.nettuno.it (bologna.nettuno.it [193.43.2.1])
by hub.org (8.9.3/8.9.3) with ESMTP id IAA41542
for <pgsql-hackers@postgresql.org>;
Wed, 26 Jan 2000 08:37:04 -0500 (EST)
(envelope-from jose@sferacarta.com)
Received: from proxy.sferacarta.com (sfcabop1.nettuno.it [193.207.10.213])
by bologna.nettuno.it (8.9.3/8.9.3/NETTuno 4.1) with ESMTP id OAA00759;
Wed, 26 Jan 2000 14:36:56 +0100 (MET)
Received: from rosso.sferacarta.com (sferacarta.com) [10.20.30.5]
by proxy.sferacarta.com with esmtp (Exim 2.05 #1 (Debian))
id 12DTYK-0004g0-00; Wed, 26 Jan 2000 14:36:12 +0000
Message-ID: <388EF907.65BAD9A4@sferacarta.com>
Date: Wed, 26 Jan 2000 14:39:19 +0100
From: Jose Soares <jose@sferacarta.com>
X-Mailer: Mozilla 4.6 [en] (WinNT; I)
X-Accept-Language: en
MIME-Version: 1.0
To: Don Baccus <dhogaza@pacifier.com>
CC: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>,
PostgreSQL Development <pgsql-hackers@postgresql.org>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy columndropping)
References: <14033.948820995@sss.pgh.pa.us>
<NDBBIJLOILGIKBGDINDFCEFHCCAA.Inoue@tpf.co.jp>
<14033.948820995@sss.pgh.pa.us>
<3.0.1.32.20000125112001.0106bec0@mail.pacifier.com>
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit

Don Baccus wrote:

At 12:29 PM 1/25/00 -0600, Ross J. Reedstrom wrote:

On Tue, Jan 25, 2000 at 12:23:15PM -0500, Tom Lane wrote:

Well, yeah: wouldn't you expect that "ADD COLUMN x DEFAULT 42" would
cause every row currently existing in the table to acquire x = 42,
rather than x = NULL? In fact that would *have* to happen to allow
constraints to be added; consider ADD COLUMN x DEFAULT 42 NOT NULL.

Actually, no I wouldn't expect it. That's mixing DDL and DML in one
statement. I expect the ALTER command to be pure DDL, and the UPDATE
to be pure DML.

Hmmm...interesting...is alter table in the standard?

Yes, of course.
...
<alter table statement> ::=
ALTER TABLE <table name> <alter table action>

<alter table action> ::=
<add column definition>
| <alter column definition>
| <drop column definition>
| <add table constraint definition>
| <drop table constraint definition>
...

Again, my copy
of Date's SQL 92 primer is somewhere 'wteen Boston, MA and Portland, OR,
so I can't look myself. Since you've got the standard available you
can answer perhaps?

Ouch, reading standards always makes my brain hurt. Especially how you
have to read them upside down. Turns out SELECT INTO is in the standard,
but not the way we implement it.

Scary!!! :) :)

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

************

Jos���

From bouncefilter Wed Jan 26 11:06:08 2000
Received: from wallace.ece.rice.edu (root@wallace.ece.rice.edu
[128.42.12.154])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA82570
for <pgsql-hackers@postgresql.org>;
Wed, 26 Jan 2000 11:05:38 -0500 (EST)
(envelope-from reedstrm@wallace.ece.rice.edu)
Received: by wallace.ece.rice.edu via sendmail from stdin
id <m12DUwr-000LELC@wallace.ece.rice.edu> (Debian Smail3.2.0.102)
for pgsql-hackers@postgresql.org; Wed, 26 Jan 2000 10:05:37 -0600 (CST)
Date: Wed, 26 Jan 2000 10:05:37 -0600
From: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
To: Philip Warner <pjw@rhyme.com.au>
Cc: pgsql-hackers@postgresql.org
Subject: Re: AW: AW: [HACKERS] Some notes on optimizer cost estimates
Message-ID: <20000126100537.B4802@rice.edu>
References: <219F68D65015D011A8E000006F8590C603FDC21E@sdexcsrv1.f000.d0
188.sd.spardat.at>
<3.0.5.32.20000126200717.03472bf0@mail.rhyme.com.au>
Mime-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit
User-Agent: Mutt/1.0i
In-Reply-To: <3.0.5.32.20000126200717.03472bf0@mail.rhyme.com.au>;
from pjw@rhyme.com.au on Wed, Jan 26, 2000 at 08:07:17PM +1100

On Wed, Jan 26, 2000 at 08:07:17PM +1100, Philip Warner wrote:

Based on experience with optimizer improvements across releases of DB
products (not PostgreSQL, I hastily add), I would be inclined to say (from
bitter experience) that no optimizer is ever truly predicatable. The SQL
programmer has to be given the tools to ensure that a 'bad' query can be
forced to run the same way with each release, and release notes should
indicate what extra strategies are now available, in case the 'bad' query
can be made better.

It gets my goat (a bit) when commercial DB manufacturers believe that they
can solve intractable optimization problems - it would be a pity for PGSQL
to go the same way. I'd love to have the opportunity to prove my point with
PGSQL, but since I can't affect the optimizers choices in any way, I am
left with rhetoric, and examples from commercial DBs, which aren't, really,
relevant.

Sorry about the chestnut.

Ah, but you _can_ affect how the plans chosen, which in turn can affect
the optimizer. Not as part of a running, production system, I grant you,
but for debugging performance problems (and in particular, changes from
one release to the next) it can be useful. What I'm talking about are
the switches to the backend that tell pgsql not use particular kinds
of joins/scans in planning a query

From postgres(1):

-f Forbids the use of particular scan and join meth���
ods: s and i disable sequential and index scans
respectively, while n, m and h disable nested-loop,
merge and hash joins respectively. (Neither
sequential scans nor nested-loop joins can be dis���
abled completely; the -fs and -fn options simply
discourage the optimizer from using those plan
types if it has any other alternative.)

While not the whole ball of wax in terms of controlling the
planner/optimizer stages, it does give you one more knob to tweak,
beyond saying "This query took 2 sec. on release X, now it takes 2
min. on release X+1"

Perhaps someone (Phil?) could collect 'bad' queries, and run them against
each release, and donate that part of the release notes to Bruce.

Ross

--
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu>
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St., Houston, TX 77005

From bouncefilter Wed Jan 26 11:06:11 2000
Received: from matsu.interwiz.koganei.tokyo.jp
(matsu.interwiz.koganei.tokyo.jp [202.230.201.2])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA82642
for <pgsql-interfaces@postgreSQL.org>;
Wed, 26 Jan 2000 11:06:07 -0500 (EST)
(envelope-from kataoka@interwiz.koganei.tokyo.jp)
Received: from suzume (suzume [202.230.201.68])
by matsu.interwiz.koganei.tokyo.jp (8.9.1/3.7W) with SMTP id BAA06037
for <pgsql-interfaces@postgreSQL.org>;
Thu, 27 Jan 2000 01:05:57 +0900 (JST)
From: "Hiroki Kataoka" <kataoka@interwiz.koganei.tokyo.jp>
To: <pgsql-interfaces@postgreSQL.org>
Subject: Re: ODBC drive strange behavior
Date: Thu, 27 Jan 2000 01:06:02 +0900
Message-ID: <NDBBIKCHILNJOAAPDPKMGEEBCAAA.kataoka@interwiz.koganei.tokyo.jp>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
In-Reply-To: <388E0439.969DCD05@home.com>
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
Importance: Normal

Byron Nikolaidis wrote:

Jose Soares wrote:

Is there a new release? I can't find it at:
http://www.insightdist.com/psqlodbc/
Is there another site for psqlODBC ?

Yes, I have version 6.40.0007 on my own website. I'm not sure if it is
available elsewhere. The address is
http://members.home.net/byron.nikolaidis

Unfortunately, it does not include some recent patches that other people
have applied. I know of at least one with large objects.

I could put these in I suppose. Anybody know where these patches are?
Perhaps the author?

I posted an large object patch.
I will send this patch to you.

=====
Hiroki Kataoka

From bouncefilter Wed Jan 26 12:07:08 2000
Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id MAA10612
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 12:06:33 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
by sss2.sss.pgh.pa.us (8.9.3/8.9.3) with ESMTP id MAA18014;
Wed, 26 Jan 2000 12:06:15 -0500 (EST)
To: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
cc: Philip Warner <pjw@rhyme.com.au>, pgsql-hackers@postgreSQL.org
Subject: Re: AW: AW: [HACKERS] Some notes on optimizer cost estimates
In-reply-to: <20000126100537.B4802@rice.edu>
References: <219F68D65015D011A8E000006F8590C603FDC21E@sdexcsrv1.f000.d0
188.sd.spardat.at>
<3.0.5.32.20000126200717.03472bf0@mail.rhyme.com.au>
<20000126100537.B4802@rice.edu>
Comments: In-reply-to "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
message dated "Wed, 26 Jan 2000 10:05:37 -0600"
Date: Wed, 26 Jan 2000 12:06:15 -0500
Message-ID: <18011.948906375@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

"Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu> writes:

Ah, but you _can_ affect how the plans chosen, which in turn can affect
the optimizer. Not as part of a running, production system, I grant you,
but for debugging performance problems (and in particular, changes from
one release to the next) it can be useful. What I'm talking about are
the switches to the backend that tell pgsql not use particular kinds
of joins/scans in planning a query

BTW, I have been thinking that it'd be a lot better if these flags could
be twiddled via SET/SHOW commands, instead of having to restart psql.
Nothing done about it yet, but it's an idea...

Also, you already can twiddle the basic cost parameters (cpu_page_weight
and cpu_index_page_weight) via SET variables whose names I forget at the
moment. There will be probably be at least one more such variable
before 7.0 comes out, to control cost of random page fetch vs. sequential.

regards, tom lane

From bouncefilter Wed Jan 26 16:10:11 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id QAA90577
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 16:09:53 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id KAA17494;
Wed, 26 Jan 2000 10:34:40 -0800 (PST)
Message-Id: <3.0.1.32.20000126100750.00fa6340@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Wed, 26 Jan 2000 10:07:50 -0800
To: Hannu Krosing <hannu@tm.ee>, Hiroshi Inoue <Inoue@tpf.co.jp>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: Happy column adding (was RE: [HACKERS] Happy column
dropping)
Cc: PostgreSQL Development <pgsql-hackers@postgreSQL.org>,
Peter Eisentraut <peter_e@gmx.net>
In-Reply-To: <388EB436.8AE7334F@tm.ee>
References: <000801bf67be$b490d120$2801007e@tpf.co.jp>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 10:45 AM 1/26/00 +0200, Hannu Krosing wrote:

Do we have to refer default value for already inserted rows ?
Doesn't 'default' have its meaning only when rows are about to be
inserted ?

I think the case was about adding a NOT NULL column and setting current NULL
columns to DEFAULT seemed like a natural thing to do.

etc...

It depends on whether we want to be SQL92 compliant. The SQL92 standard
seems to make it clear that "add column ... default" is supposed to
set the column in ALL the rows in the table to that value.

I think it's actually much more useful this way. If you set a default
value, it is normal to write application code that depends on its
existence. If you're going to write your application to work with
the column set to NULL, then you probably don't need the default
anyway.

And...if you don't want to pay the penalty of having to set default
values for all the rows when you add a column with a default value,
you can always add the column without the default value and use a
trigger to set new rows to a default value. This would give the
functionality you want, no?

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Wed Jan 26 15:54:13 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id PAA84192
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 15:53:40 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id KAA17506;
Wed, 26 Jan 2000 10:34:42 -0800 (PST)
Message-Id: <3.0.1.32.20000126101331.0106c390@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Wed, 26 Jan 2000 10:13:31 -0800
To: "Ansley, Michael" <Michael.Ansley@intec.co.za>,
Peter Eisentraut <peter_e@gmx.net>
From: Don Baccus <dhogaza@pacifier.com>
Subject: RE: [HACKERS] --enable-debug
Cc: PostgreSQL Development <pgsql-hackers@postgreSQL.org>,
"'Bruce Momjian'" <pgman@candle.pha.pa.us>, Tom Lane <tgl@sss.pgh.pa.us>
In-Reply-To: <1BF7C7482189D211B03F00805F8527F748C494@S-NATH-EXCH2>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 01:10 PM 1/26/00 +0200, Ansley, Michael wrote:

When I initially mentioned this, I had tried to debug, only to find that
there were no symbols. So I tried the standard autoconf option, which is
--enable-debug, to get some symbols. I'm not particularly concerned about
--cassert, but --enable-debug is the standard autoconf option, and most
people (developers) are aware of it. If it's just a synonym for --cassert,
well, I don't really care. I just want some symbols to debug with. I
landed up fixing it by setting
CC=gcc -g -Wall

gmake COPT="-g" all (after a clean) worked for me...that leave -O2 on,
though. Gcc seems to do a reasonably good job anyway.

We are doing -Wall on Linux, BTW, in case there's any doubt (Tom mentioned
it was used on his platform but I think he's an HP-UX kinda guy). At
least, when I do a make on my RH 6.1 box with the current sources "-Wall"
is on.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Wed Jan 26 15:54:11 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id PAA84122
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 15:53:22 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id KAA17645;
Wed, 26 Jan 2000 10:35:00 -0800 (PST)
Message-Id: <3.0.1.32.20000126101427.0106e5e0@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Wed, 26 Jan 2000 10:14:27 -0800
To: Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>,
"'pgsql-hackers@postgreSQL.org'" <pgsql-hackers@postgreSQL.org>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: AW: AW: AW: [HACKERS] Some notes on optimizer cost
estimates
In-Reply-To: <219F68D65015D011A8E000006F8590C603FDC224@sdexcsrv1.f000.d0
188.sd.spardat.at>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 01:25 PM 1/26/00 +0100, Zeugswetter Andreas SB wrote:

Yes, I think syntax to force or disallow a particular index,
choose a join method or order, force/disallow seq scans ...
is sometimes useful.
Even Informix, who always refused to supply such a feature
now has it.

Can you give some sample syntax, for those of us who aren't
really database people but merely trying to maintain a facade? :)

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Wed Jan 26 16:09:12 2000
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id QAA90288
for <pgsql-hackers@postgreSQL.org>;
Wed, 26 Jan 2000 16:08:27 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id KAA17755;
Wed, 26 Jan 2000 10:35:12 -0800 (PST)
Message-Id: <3.0.1.32.20000126102346.00fb92c0@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Wed, 26 Jan 2000 10:23:46 -0800
To: Tom Lane <tgl@sss.pgh.pa.us>,
"Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: AW: AW: [HACKERS] Some notes on optimizer cost estimates
Cc: Philip Warner <pjw@rhyme.com.au>, pgsql-hackers@postgreSQL.org
In-Reply-To: <18011.948906375@sss.pgh.pa.us>
References: <20000126100537.B4802@rice.edu>
<219F68D65015D011A8E000006F8590C603FDC21E@sdexcsrv1.f000.d0
188.sd.spardat.at>
<3.0.5.32.20000126200717.03472bf0@mail.rhyme.com.au>
<20000126100537.B4802@rice.edu>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 12:06 PM 1/26/00 -0500, Tom Lane wrote:

"Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu> writes:

Ah, but you _can_ affect how the plans chosen, which in turn can affect
the optimizer. Not as part of a running, production system, I grant you,
but for debugging performance problems (and in particular, changes from
one release to the next) it can be useful. What I'm talking about are
the switches to the backend that tell pgsql not use particular kinds
of joins/scans in planning a query

BTW, I have been thinking that it'd be a lot better if these flags could
be twiddled via SET/SHOW commands, instead of having to restart psql.
Nothing done about it yet, but it's an idea...

If something like this could be done for only the current transaction
or only the current backend, you'd have a sledgehammer-quality tool
for tuning individual queries, no? Obviously not an ideal tool but
maybe helpful to some folks?

I've not looked at the code, maybe doing a SET would only affect
the current backend?

Also, you already can twiddle the basic cost parameters (cpu_page_weight
and cpu_index_page_weight) via SET variables whose names I forget at the
moment. There will be probably be at least one more such variable
before 7.0 comes out, to control cost of random page fetch vs. sequential.

These should be helpful, too...again, are they system-wide or
limited to the current backend? It would probably be nice to be
able to futz them for a particular query without impacting all other
queries going on at the same time.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Wed Jan 26 13:28:10 2000
Received: from merganser.its.uu.se (merganser.its.uu.se [130.238.6.236])
by hub.org (8.9.3/8.9.3) with ESMTP id NAA31682
for <pgsql-hackers@postgresql.org>;
Wed, 26 Jan 2000 13:27:20 -0500 (EST)
(envelope-from peter@localhost.its.uu.se)
Received: from regulus.its.uu.se ([130.238.7.19]:61089 "EHLO
regulus.its.uu.se")
by merganser.its.uu.se with ESMTP id <S61470AbQAZS0p>;
Wed, 26 Jan 2000 19:26:45 +0100
Received: from peter (helo=localhost)
by regulus.its.uu.se with local-esmtp (Exim 3.02 #2)
id 12DXGm-0000Pb-00; Wed, 26 Jan 2000 19:34:20 +0100
Date: Wed, 26 Jan 2000 19:34:20 +0100 (CET)
From: Peter Eisentraut <peter_e@gmx.net>
To: The Hermit Hacker <scrappy@hub.org>
cc: Adriaan Joubert <a.joubert@albourne.com>, pgsql-hackers@postgresql.org
Subject: OIDS (Re: [HACKERS] Well, then you keep your darn columns)
In-Reply-To: <Pine.BSF.4.21.0001241233300.79710-100000@thelab.hub.org>
Message-ID: <Pine.LNX.4.21.0001260034440.9535-100000@localhost.localdomain>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=ISO-8859-1
Content-Transfer-Encoding: 8BIT
Sender: Peter Eisentraut <peter@hub.org>

On 2000-01-24, The Hermit Hacker mentioned:

Except, as Chris Bitmead brought up, OIDs appear to be a key requirement
in ODBMSs ... so, if we want to go what I *think* is 'next generation',
OIDs have to be kept ...

Independent of everything else I would like to point out that although
oids do appear in a central role in the theory of object oriented
databases they are still not a user-level feature. The system uses them to
in essence do what some people already do with them now: use them as links
in foreign key settings. This sort of scheme is supposed to eliminate the
need for costly joins, since you already know the location of the data
(assuming that you have a scheme to map the oid to the storage location).

This past summer this sort of idea was discussed around these parts and
most of us came to the conclusion that a) OODBs are a pipe-dream at this
point in time, and b) this is not worth doing in PostgreSQL as it stands.
If we wanna become an OODBs we might as well say that now so we can start
by dropping SQL and the optimizer and the storage manager -- okay, I'm
being sarcastic (about OODBs).

However, once again, users would have no knowledge of these "oids". The
system is free to do whatever it wants in order to do its thing, in
particular it is free to *change* oids when it needs it (because when it
copies the data elsewhere it presumably needs to tag the location
differently).

Our oids are something different (though not sure what), PostgreSQL is
something different. I am by all means against breaking what oids
represent now, but incidentally I am also against them becoming (being) a
user-level feature.

--
Peter Eisentraut Sernanders v���g 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Wed Jan 26 13:29:09 2000
Received: from merganser.its.uu.se (merganser.its.uu.se [130.238.6.236])
by hub.org (8.9.3/8.9.3) with ESMTP id NAA31898
for <pgsql-hackers@postgresql.org>;
Wed, 26 Jan 2000 13:28:22 -0500 (EST)
(envelope-from peter@localhost.its.uu.se)
Received: from regulus.its.uu.se ([130.238.7.19]:61226 "EHLO
regulus.its.uu.se")
by merganser.its.uu.se with ESMTP id <S157733AbQAZS0y>;
Wed, 26 Jan 2000 19:26:54 +0100
Received: from peter (helo=localhost)
by regulus.its.uu.se with local-esmtp (Exim 3.02 #2)
id 12DXGv-0000Pv-00; Wed, 26 Jan 2000 19:34:29 +0100
Date: Wed, 26 Jan 2000 19:34:29 +0100 (CET)
From: Peter Eisentraut <peter_e@gmx.net>
To: Chris Bitmead <chris@bitmead.com>
cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Inheritance,
referential integrity and other constraints
In-Reply-To: <388CE133.EE3ADAC5@bitmead.com>
Message-ID: <Pine.LNX.4.21.0001260055100.9535-100000@localhost.localdomain>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=ISO-8859-1
Content-Transfer-Encoding: 8BIT
Sender: Peter Eisentraut <peter@hub.org>

On 2000-01-25, Chris Bitmead mentioned:

As long as you're working on this area you could fix the problem where
if you do ALTER table* ADD COLUMN ... pg_dump no longer works because
the column orders have changed in different inherited tables.

This should be fixed in pg_dump then. As I see it, ALTER table* ADD COLUMN
does exactly the right thing.

--
Peter Eisentraut Sernanders v���g 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden