uniqueness not always correct

Frank Cusack wrote:

Solaris 2.6/sparc; postgres 6.5.1

dns=> create table test (zone int4, net cidr, unique(zone, net));
NOTICE: CREATE TABLE/UNIQUE will create implicit index 'test_zone_key' for table 'test'
CREATE
dns=> insert into test (zone, net) values (1, '1.2.3/24');
INSERT 21750 1
dns=> insert into test (zone, net) values (1, '2.3.4/24');
INSERT 21751 1
dns=> insert into test (zone, net) values (1, '1.2.3/24');
INSERT 21752 1
dns=> insert into test (zone, net) values (1, '2.3.4/24');
ERROR: Cannot insert a duplicate key into a unique index

Yes, I reproduced this (Solaris 2.5/sparc).
Seems like CIDR problem(??!):

ais=> create table test (zone int4, net int4, unique(zone, net));
^^^^
NOTICE: CREATE TABLE/UNIQUE will create implicit index 'test_zone_key' for table 'test'
CREATE
ais=> insert into test (zone, net) values (1, 1);
INSERT 7712479 1
ais=> insert into test (zone, net) values (1, 2);
INSERT 7712480 1
ais=> insert into test (zone, net) values (1, 1);
ERROR: Cannot insert a duplicate key into a unique index

Vadim

From bouncefilter Thu Nov 11 10:29:44 1999
Received: from localhost (IDENT:root@hectic-4.jpl.nasa.gov [128.149.68.206])
by hub.org (8.9.3/8.9.3) with ESMTP id KAA65554;
Thu, 11 Nov 1999 10:29:29 -0500 (EST)
(envelope-from lockhart@alumni.caltech.edu)
Received: from alumni.caltech.edu (lockhart@localhost [127.0.0.1])
by localhost (8.8.7/8.8.7) with ESMTP id PAA05333;
Thu, 11 Nov 1999 15:35:58 GMT
Sender: lockhart@hub.org
Message-ID: <382AE25E.485114A@alumni.caltech.edu>
Date: Thu, 11 Nov 1999 15:35:58 +0000
From: Thomas Lockhart <lockhart@alumni.caltech.edu>
X-Mailer: Mozilla 4.6 [en] (X11; I; Linux 2.0.36 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Bruce Stephens <bruce@cenderis.demon.co.uk>
CC: pgsql-hackers@postgreSQL.org, pgsql-interfaces@postgreSQL.org
Subject: Re: [INTERFACES] Re: [HACKERS] CORBA STATUS
References: <199911101542.PAA05115@linda.lfix.co.uk>
<87emdx7wal.fsf@cenderis.demon.co.uk>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

It probably makes sense to try to get things working with one ORB, and
then see if it's worth generalising. I'd guess ORBit is a good one to
start with since it's C-based, and it's pretty small: if I have to
install an extra package on my machine just to run PostgreSQL, I'm not
going to mind ORBit, whereas TAO might annoy me (IIRC, TAO is quite
big; I may be thinking of another ORB, though). ORBit does IIOP, I
believe, so that covers GNOME and KDE people.

Yeah, TAO might annoy you; it takes 1.3GB of disk space to build,
though much less once built ;)

- Thomas

--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California

From bouncefilter Thu Nov 11 10:38:45 1999
Received: from Radha.DoCS.UU.SE (root@Radha.DoCS.UU.SE [130.238.9.99])
by hub.org (8.9.3/8.9.3) with SMTP id KAA66939
for <hackers@postgreSQL.org>; Thu, 11 Nov 1999 10:38:04 -0500 (EST)
(envelope-from e99re41@DoCS.UU.SE)
Received: from Lodjur.DoCS.UU.SE (e99re41@Lodjur.DoCS.UU.SE [130.238.9.178])
by Radha.DoCS.UU.SE (8.6.12/8.6.12) with ESMTP id QAA17316;
Thu, 11 Nov 1999 16:37:49 +0100
Received: from localhost (e99re41@localhost) by Lodjur.DoCS.UU.SE
(8.6.12/8.6.12) with SMTP id QAA22491;
Thu, 11 Nov 1999 16:37:47 +0100
X-Authentication-Warning: Lodjur.DoCS.UU.SE: e99re41 owned process doing -bs
Date: Thu, 11 Nov 1999 16:37:47 +0100 (MET)
From: Peter Eisentraut <e99re41@DoCS.UU.SE>
Reply-To: Peter Eisentraut <peter_e@gmx.net>
To: Bruce Momjian <maillist@candle.pha.pa.us>
cc: Tom Lane <tgl@sss.pgh.pa.us>, Bernard Frankpitt <frankpit@pop.dn.net>,
hackers@postgreSQL.org
Subject: Re: [HACKERS] Indent
In-Reply-To: <199911110230.VAA08219@candle.pha.pa.us>
Message-ID: <Pine.GSO.4.02A.9911111631090.22368-100000@Lodjur.DoCS.UU.SE>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Wed, 10 Nov 1999, Bruce Momjian wrote:

ObFlameBait: Personally I think we should switch over to standard
8-column tabs, but Bruce is apparently still using some medieval editor
in which physical tab widths dictate logical indent levels :-(

Vadim is also in agreement on this. Not many editors can handle
cases where tab size is different from indent size. Emacs obviously
can, and Tom has enjoyed pointing out. :-)

I am willing to re-open the discussion if we people would prefer
something else.

Trying to redefine a tab to be 4 spaces is asking for trouble. How about
making pgindent replace 8 spaces with a tab and 4 spaces with, well, 4
spaces. This is how emacs handles bsd indent style if you don't change the
tab sizes. And all other editors should be fine with this.

Personally, I'm always in favour of using no tabs at all, because of this
very problem, and just because they annoy me. But I tend to be alone with
that position.

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Thu Nov 11 10:46:52 1999
Received: from fep132.fep.ru (mail@fep132.fep.ru [195.230.89.88])
by hub.org (8.9.3/8.9.3) with ESMTP id KAA68243
for <pgsql-hackers@postgresql.org>;
Thu, 11 Nov 1999 10:45:51 -0500 (EST) (envelope-from phd@phd.russ.ru)
Received: from localhost [127.0.0.1] (phd)
by fep132.fep.ru with esmtp (Exim 2.05 #1 (Debian))
id 11lwPu-00041q-00; Thu, 11 Nov 1999 18:45:42 +0300
Date: Thu, 11 Nov 1999 15:45:42 +0000 (GMT)
From: Oleg Broytmann <phd@phd.russ.ru>
X-Sender: phd@fep132.fep.ru
Reply-To: phd2@earthling.net
To: PostgreSQL-development <pgsql-hackers@postgresql.org>
cc: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
Subject: PostgreSQL and Zope
Message-ID: <Pine.LNX.4.20.9911111539010.15448-100000@fep132.fep.ru>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

Hello!

Ross J. Reedstrom, current maintainer of PostgreSQL Database Adapter for
Zope (based, of course, on PyGreSQL), said very positive words about
PostgreSQL and Postgres team:

"My experience with the PostgreSQL developers and mailing lists are very
similar to the Zope lists: lots of clueful, helpful people (some of them
are the same people ;-) Bugs exist (what software is perfect?), but get
diagnosed and fixed very rapidly. Features are added (and limitations
removed) on a weekly basis. I get the feeling that the pgsql project has
turned a corner in the last year or two: the code base has been cleaned
up, making it easier to understand, and contribute code. New developers
are being attracted, and quickly becoming contributers. It feels like
it's reaching critical mass. The pgsql-hackers list reminds me of the
linux-kernel list from about '92 or so, in that respect.

If you haven't looked at PostgreSQL since it's Postgres95 days, it's
definitely time to look again."

http://www.egroups.com/list/zope/md185412286.html

Yes, I was one of those who sent him "works for me" message. :)

Oleg.
----
Oleg Broytmann http://members.xoom.com/phd2/ phd2@earthling.net
Programmers don't die, they just GOSUB without RETURN.

From bouncefilter Thu Nov 11 10:49:47 1999
Received: from localhost (IDENT:root@hectic-4.jpl.nasa.gov [128.149.68.206])
by hub.org (8.9.3/8.9.3) with ESMTP id KAA68774
for <hackers@postgreSQL.org>; Thu, 11 Nov 1999 10:49:14 -0500 (EST)
(envelope-from lockhart@alumni.caltech.edu)
Received: from alumni.caltech.edu (lockhart@localhost [127.0.0.1])
by localhost (8.8.7/8.8.7) with ESMTP id PAA06283;
Thu, 11 Nov 1999 15:56:18 GMT
Sender: lockhart@hub.org
Message-ID: <382AE722.F1CB966D@alumni.caltech.edu>
Date: Thu, 11 Nov 1999 15:56:18 +0000
From: Thomas Lockhart <lockhart@alumni.caltech.edu>
X-Mailer: Mozilla 4.6 [en] (X11; I; Linux 2.0.36 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Bernard Frankpitt <frankpit@pop.dn.net>
CC: Peter Eisentraut <peter_e@gmx.net>, hackers@postgreSQL.org
Subject: Re: [HACKERS] Indent
References: <Pine.LNX.4.20.9911082316520.4161-100000@peter-e.yi.org>
<3829C81A.F195D285@pop.dn.net>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

;;; Set tab-width in c-mode to 4 spaces
(add-hook 'c-mode-hook
(function (lambda () (set tab-width 4))))

Typo:

set -> setq (from Tom Lane's code)

I'm now a happy emacs camper :)

- Thomas

--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California

Vadim Mikheev <vadim@krs.ru> writes:

Yes, I reproduced this (Solaris 2.5/sparc).
Seems like CIDR problem(??!):

Yes. Looks like the low-order bits of a CIDR address are garbage,
but network_cmp() compares them as though all bits are significant.
So, indeed, it may think two different instances of '1.2.3/24'
are not equal.

The regular inet comparison functions at least *try* to mask out
garbage bits, but I think they get it wrong too --- they should be
taking the smaller of ip_bits(a1) and ip_bits(a2) as the number of
bits to compare. They don't. Thus, for example,

regression=> select '1.2.5/16'::cidr < '1.2.3/24'::cidr;
?column?
--------
f
(1 row)

which looks wrong to me.

In short, it's a bug in the inet data types, not a generic problem
with unique indexes.

regards, tom lane

From bouncefilter Thu Nov 11 13:22:47 1999
Received: from candle.pha.pa.us (maillist@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id NAA90993
for <pgsql-hackers@postgreSQL.org>;
Thu, 11 Nov 1999 13:22:28 -0500 (EST)
(envelope-from maillist@candle.pha.pa.us)
Received: (from maillist@localhost)
by candle.pha.pa.us (8.9.0/8.9.0) id NAA05463;
Thu, 11 Nov 1999 13:21:34 -0500 (EST)
From: Bruce Momjian <maillist@candle.pha.pa.us>
Message-Id: <199911111821.NAA05463@candle.pha.pa.us>
Subject: psql and \p\g
In-Reply-To: <Pine.GSO.4.02A.9911091006100.10112-103000@Iller.DoCS.UU.SE> from
Peter Eisentraut at "Nov 9, 1999 10:12:50 am"
To: Peter Eisentraut <peter_e@gmx.net>
Date: Thu, 11 Nov 1999 13:21:34 -0500 (EST)
CC: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

I have found that typing:

test=> select * from pg_class\p\g

no longer works. I honors the \p, but ignores the \g.

Any ideas Peter?

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Thu Nov 11 13:28:47 1999
Received: from candle.pha.pa.us (maillist@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id NAA91734
for <pgsql-hackers@postgreSQL.org>;
Thu, 11 Nov 1999 13:27:53 -0500 (EST)
(envelope-from maillist@candle.pha.pa.us)
Received: (from maillist@localhost)
by candle.pha.pa.us (8.9.0/8.9.0) id NAA05614;
Thu, 11 Nov 1999 13:27:04 -0500 (EST)
From: Bruce Momjian <maillist@candle.pha.pa.us>
Message-Id: <199911111827.NAA05614@candle.pha.pa.us>
Subject: failure of \e in psql
In-Reply-To: <Pine.GSO.4.02A.9911091006100.10112-103000@Iller.DoCS.UU.SE> from
Peter Eisentraut at "Nov 9, 1999 10:12:50 am"
To: Peter Eisentraut <peter_e@gmx.net>
Date: Thu, 11 Nov 1999 13:27:04 -0500 (EST)
CC: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

I see that \e no longer works as expected:

test=> select * from pg_class;
...
test=> \e

and in the editor, the query is not appearing. The 'select' query
should appear in the editor because I have not entered a non-backslash
command to clear the query buffer.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Thu Nov 11 14:09:47 1999
Received: from ara.zf.jcu.cz (zakkr@ara.zf.jcu.cz [160.217.161.4])
by hub.org (8.9.3/8.9.3) with ESMTP id OAA97250
for <pgsql-hackers@postgreSQL.org>;
Thu, 11 Nov 1999 14:09:12 -0500 (EST) (envelope-from zakkr@zf.jcu.cz)
Received: from localhost (zakkr@localhost)
by ara.zf.jcu.cz (8.9.3/8.9.3/Debian/GNU) with SMTP id TAA04368
for <pgsql-hackers@postgreSQL.org>; Thu, 11 Nov 1999 19:55:56 +0100
Date: Thu, 11 Nov 1999 19:55:56 +0100 (CET)
From: Karel Zak - Zakkr <zakkr@zf.jcu.cz>
To: pgsql-hackers <pgsql-hackers@postgreSQL.org>
Subject: compression in LO and other fields
Message-ID: <Pine.LNX.3.96.991111194051.4055A-100000@ara.zf.jcu.cz>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

Hi,

in TODO is the item "Allow compression of large fields or a
compressed field type". It is good idea, but it prabably needs
binary field firstly (or not?).

I see the inv_api and other LO routines, and my idea is add support
for bzip2 stream to the inv_api and allow in current LO routines used
compression. It si good idea?

Karel Zak

------------------------------------------------------------------------------
Karel Zak <zakkr@zf.jcu.cz> http://home.zf.jcu.cz/~zakkr/

Docs: http://docs.linux.cz (big docs archive)
Kim Project: http://home.zf.jcu.cz/~zakkr/kim/ (process manager)
FTP: ftp://ftp2.zf.jcu.cz/users/zakkr/ (C/ncurses/PgSQL)
------------------------------------------------------------------------------

From bouncefilter Thu Nov 11 14:40:48 1999
Received: from flame.flame.co.za (flame.flame.co.za [160.124.170.1])
by hub.org (8.9.3/8.9.3) with ESMTP id OAA01265
for <pgsql-hackers@postgresql.org>;
Thu, 11 Nov 1999 14:39:44 -0500 (EST)
(envelope-from theo@flame.co.za)
Received: from flame.co.za (flame [160.124.170.1])
by flame.flame.co.za (8.8.7/8.8.7) with ESMTP id VAA22591
for <pgsql-hackers@postgresql.org>; Thu, 11 Nov 1999 21:41:09 +0200
Sender: theo@flame.flame.co.za
Message-ID: <382B1BD5.8A7BA9DB@flame.co.za>
Date: Thu, 11 Nov 1999 21:41:09 +0200
From: Theo Kramer <theo@flame.co.za>
Organization: Flame Computing Enterprises
X-Mailer: Mozilla 4.5 [en] (X11; I; Linux 2.0.35 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: pgsql-hackers@postgresql.org
Subject: Slow - grindingly slow - query
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hi

I have a single table with two views. The table effectively contains both
master and detail info (legacy stuff I'm afraid). The query in question is
used to see if any records exist in the detail that do not exist in the
master. The table and index definition is as follows

create table accounts (
domain text,
registrationtype char
/* Plus a couple of other irrelevant fields */
);

create index domain_idx on accounts (domain);
create index domain_type_idx on accounts (domain, registrationtype);

The views are

create view accountmaster as SELECT * from accounts where registrationtype =
'N';
create view accountdetail as SELECT * from accounts where registrationtype <>
'N';

The query is

select accountdetail.domain from accountdetail where
accountdetail.domain not in
(select accountmaster.domain from accountmaster);

I started the query about 5 hours ago and it is still running. I did the same
on Informix Online 7 and it took less than two minutes...

My system details are
postgres: 6.5.3
O/S: RH6.0 Kernel 2.2.5-15smp

Explain shows the following

explain select accountdetail.domain from accountdetail where
accountdetail.domain not in
(select accountmaster.domain from accountmaster) limit 10;
NOTICE: QUERY PLAN:

Seq Scan on accounts (cost=3667.89 rows=34958 width=12)
SubPlan
-> Seq Scan on accounts (cost=3667.89 rows=33373 width=12)

EXPLAIN

The number of records in the two views are

psql -c "select count(*) from accountmaster" coza;
count
-----
45527
(1 row)

psql -c "select count(*) from accountdetail" coza;
count
-----
22803

I know of exactly one record (I put it there myself) that satisfies the
selection criteria.

Any ideas would be appreciated

--------
Regards
Theo

PS We have it running live at http://co.za (commercial domains in South Africa).

From bouncefilter Thu Nov 11 14:52:48 1999
Received: from flame.flame.co.za (flame.flame.co.za [160.124.170.1])
by hub.org (8.9.3/8.9.3) with ESMTP id OAA03270
for <hackers@postgresql.org>; Thu, 11 Nov 1999 14:52:09 -0500 (EST)
(envelope-from theo@flame.co.za)
Received: from flame.co.za (flame [160.124.170.1])
by flame.flame.co.za (8.8.7/8.8.7) with ESMTP id VAA22618
for <hackers@postgresql.org>; Thu, 11 Nov 1999 21:53:31 +0200
Sender: theo@flame.flame.co.za
Message-ID: <382B1EBB.E58133A8@flame.co.za>
Date: Thu, 11 Nov 1999 21:53:31 +0200
From: Theo Kramer <theo@flame.co.za>
Organization: Flame Computing Enterprises
X-Mailer: Mozilla 4.5 [en] (X11; I; Linux 2.0.35 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: hackers@postgresql.org
Subject: Re: [HACKERS] Indent
References: <Pine.GSO.4.02A.9911111631090.22368-100000@Lodjur.DoCS.UU.SE>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Peter Eisentraut wrote:

Personally, I'm always in favour of using no tabs at all, because of this
very problem, and just because they annoy me. But I tend to be alone with
that position.

No you are not :). Twenty odd years of software development have taught me
to remove the tab key from my keyboard. Two spaces do the trick for me and
try to avoid at all costs lines over 80 cols. (I use emacs in vi[per] mode),
even on win98... Oh how I wish we were back in the days of TECO.
--------
Regards
Theo

From bouncefilter Thu Nov 11 15:34:48 1999
Received: from thelab.hub.org (nat198.58.mpoweredpc.net [142.177.198.58])
by hub.org (8.9.3/8.9.3) with ESMTP id PAA08515
for <pgsql-hackers@postgreSQL.org>;
Thu, 11 Nov 1999 15:34:07 -0500 (EST) (envelope-from scrappy@hub.org)
Received: from localhost (scrappy@localhost)
by thelab.hub.org (8.9.3/8.9.1) with ESMTP id QAA63237;
Thu, 11 Nov 1999 16:33:48 -0400 (AST) (envelope-from scrappy@hub.org)
X-Authentication-Warning: thelab.hub.org: scrappy owned process doing -bs
Date: Thu, 11 Nov 1999 16:33:47 -0400 (AST)
From: The Hermit Hacker <scrappy@hub.org>
To: Theo Kramer <theo@flame.co.za>
cc: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Slow - grindingly slow - query
In-Reply-To: <382B1BD5.8A7BA9DB@flame.co.za>
Message-ID: <Pine.BSF.4.10.9911111623200.2296-100000@thelab.hub.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

What does:

explain select domain from accountdetail
where domain not in (
select domain from accountmaster);

show?

Also, did you do a 'vacuum analyze' on the tables?

Also, how about if you get rid of the views

SELECT domain FROM account
WHERE registrationtype <> 'N';

*shakes head* am I missing something here? I'm reading your SELECT and
'CREATE VIEW's and don't they negate each other? *scratch head*

If I'm reading your select properly, and with the amount of sleep I've had
recently, its possible I'm not...

The subselect is saying give me all domains whose registration type = 'N'.
The select itself is saying give me all domains whoe registration type <>
'N' (select accountdetail.domain from accountdetail), and narrow that
listing down further to only include those domains whose registration type
<> 'N'?

Either I'm reading this *totally* wrong, or you satisfy that condition
ujust by doing a 'SELECT domain FROM accountdetail;' ...

No?

On Thu, 11 Nov 1999, Theo Kramer wrote:

Hi

I have a single table with two views. The table effectively contains both
master and detail info (legacy stuff I'm afraid). The query in question is
used to see if any records exist in the detail that do not exist in the
master. The table and index definition is as follows

create table accounts (
domain text,
registrationtype char
/* Plus a couple of other irrelevant fields */
);

create index domain_idx on accounts (domain);
create index domain_type_idx on accounts (domain, registrationtype);

The views are

create view accountmaster as SELECT * from accounts where registrationtype =
'N';
create view accountdetail as SELECT * from accounts where registrationtype <>
'N';

The query is

select accountdetail.domain from accountdetail where
accountdetail.domain not in
(select accountmaster.domain from accountmaster);

I started the query about 5 hours ago and it is still running. I did the same
on Informix Online 7 and it took less than two minutes...

My system details are
postgres: 6.5.3
O/S: RH6.0 Kernel 2.2.5-15smp

Explain shows the following

explain select accountdetail.domain from accountdetail where
accountdetail.domain not in
(select accountmaster.domain from accountmaster) limit 10;
NOTICE: QUERY PLAN:

Seq Scan on accounts (cost=3667.89 rows=34958 width=12)
SubPlan
-> Seq Scan on accounts (cost=3667.89 rows=33373 width=12)

EXPLAIN

The number of records in the two views are

psql -c "select count(*) from accountmaster" coza;
count
-----
45527
(1 row)

psql -c "select count(*) from accountdetail" coza;
count
-----
22803

I know of exactly one record (I put it there myself) that satisfies the
selection criteria.

Any ideas would be appreciated

--------
Regards
Theo

PS We have it running live at http://co.za (commercial domains in South Africa).

************

Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

From bouncefilter Thu Nov 11 15:42:48 1999
Received: from hu.tm.ee (isdn-54.uninet.ee [194.204.0.118])
by hub.org (8.9.3/8.9.3) with ESMTP id PAA09627
for <pgsql-hackers@postgreSQL.org>;
Thu, 11 Nov 1999 15:41:50 -0500 (EST) (envelope-from hannu@tm.ee)
Received: from tm.ee (localhost [127.0.0.1]) by hu.tm.ee (Postfix) with ESMTP
id 4C5075442; Thu, 11 Nov 1999 22:41:31 +0200 (EET)
Sender: hannu@hu.tm.ee
Message-ID: <382B29FB.9FFBF61C@tm.ee>
Date: Thu, 11 Nov 1999 22:41:31 +0200
From: Hannu Krosing <hannu@tm.ee>
Organization: Trust-O-Matic =?iso-8859-1?Q?O=DC?=
X-Mailer: Mozilla 4.7 [en] (X11; I; Linux 2.2.13-7mdk i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Theo Kramer <theo@flame.co.za>
Cc: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Slow - grindingly slow - query
References: <382B1BD5.8A7BA9DB@flame.co.za>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Theo Kramer wrote:

Hi

I have a single table with two views. The table effectively contains both
master and detail info (legacy stuff I'm afraid). The query in question is
used to see if any records exist in the detail that do not exist in the
master. The table and index definition is as follows

create table accounts (
domain text,
registrationtype char
/* Plus a couple of other irrelevant fields */
);

create index domain_idx on accounts (domain);
create index domain_type_idx on accounts (domain, registrationtype);

try using
create index registrationtype_index on accounts (registrationtype);

------
Hannu

From bouncefilter Thu Nov 11 15:49:48 1999
Received: from flame.flame.co.za (flame.flame.co.za [160.124.170.1])
by hub.org (8.9.3/8.9.3) with ESMTP id PAA10559
for <pgsql-hackers@postgreSQL.org>;
Thu, 11 Nov 1999 15:48:53 -0500 (EST)
(envelope-from theo@flame.co.za)
Received: from flame.co.za (flame [160.124.170.1])
by flame.flame.co.za (8.8.7/8.8.7) with ESMTP id WAA22758
for <pgsql-hackers@postgreSQL.org>; Thu, 11 Nov 1999 22:50:14 +0200
Sender: theo@flame.flame.co.za
Message-ID: <382B2C06.356F9234@flame.co.za>
Date: Thu, 11 Nov 1999 22:50:14 +0200
From: Theo Kramer <theo@flame.co.za>
Organization: Flame Computing Enterprises
X-Mailer: Mozilla 4.5 [en] (X11; I; Linux 2.0.35 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Slow - grindingly slow - query
References: <Pine.BSF.4.10.9911111623200.2296-100000@thelab.hub.org>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

The Hermit Hacker wrote:

What does:

explain select domain from accountdetail
where domain not in (
select domain from accountmaster);

show?

NOTICE: QUERY PLAN:

Seq Scan on accounts (cost=3667.89 rows=34958 width=12)
SubPlan
-> Seq Scan on accounts (cost=3667.89 rows=33373 width=12)

EXPLAIN

Also, did you do a 'vacuum analyze' on the tables?

Yes - should have mentioned that.

Also, how about if you get rid of the views

SELECT domain FROM account
WHERE registrationtype <> 'N';

*shakes head* am I missing something here? I'm reading your SELECT and
'CREATE VIEW's and don't they negate each other? *scratch head*

No - a domain can both be new (registrationtype 'N') and updated
(registrationtype 'U') ie. one or more rows with the same domain with one row
containing a domain with registrationtype 'N' and zero or more rows containing
the same domain with registrationtype not 'N'. The reason for the <> 'N' and
not just = 'U' is that we have a couple of rows with registrationtype set to
something else.

The subselect is saying give me all domains whose registration type = 'N'.
The select itself is saying give me all domains whoe registration type <>
'N' (select accountdetail.domain from accountdetail), and narrow that
listing down further to only include those domains whose registration type
<> 'N'?

Either I'm reading this *totally* wrong, or you satisfy that condition
ujust by doing a 'SELECT domain FROM accountdetail;' ...

No?

No :). See above

--------
Regards
Theo

I'm not sure that a '<' comparison is really meaningful for inet/cidr?
At least not the '<' comparison you are doing. For networks (cf hosts),
the only really meanininful operators are '<<' (contained within), etc.

A nice easy fix might be to make sure that the unmasked portion of the
data is set to all 0's when storing the data.

~f
ps. I'm not subscribed to the lists so this will probably bounce. Please
repost for me.

On Thu, 11 Nov 1999, "Tom" == Tom Lane wrote:

Tom> Vadim Mikheev <vadim@krs.ru> writes:

  +> Yes, I reproduced this (Solaris 2.5/sparc).  Seems like CIDR
  +> problem(??!):

Tom> Yes. Looks like the low-order bits of a CIDR address are garbage,
Tom> but network_cmp() compares them as though all bits are significant.
Tom> So, indeed, it may think two different instances of '1.2.3/24' are
Tom> not equal.

Tom> The regular inet comparison functions at least *try* to mask out
Tom> garbage bits, but I think they get it wrong too --- they should be
Tom> taking the smaller of ip_bits(a1) and ip_bits(a2) as the number of
Tom> bits to compare. They don't. Thus, for example,

Tom> regression=> select '1.2.5/16'::cidr < '1.2.3/24'::cidr;
Tom> ?column?
Tom> --------
Tom> f
Tom> (1 row)

Tom> which looks wrong to me.

Tom> In short, it's a bug in the inet data types, not a generic problem
Tom> with unique indexes.

Tom> regards, tom lane

On Thu, 11 Nov 1999,
"Tom" == Tom Lane wrote:

Tom> Vadim Mikheev <vadim@krs.ru> writes:

  +> Yes, I reproduced this (Solaris 2.5/sparc).
  +> Seems like CIDR problem(??!):

Tom> Yes. Looks like the low-order bits of a CIDR address are garbage,
Tom> but network_cmp() compares them as though all bits are significant.
Tom> So, indeed, it may think two different instances of '1.2.3/24'
Tom> are not equal.

Tom> The regular inet comparison functions at least *try* to mask out
Tom> garbage bits, but I think they get it wrong too --- they should be
Tom> taking the smaller of ip_bits(a1) and ip_bits(a2) as the number of
Tom> bits to compare. They don't. Thus, for example,

Tom> regression=> select '1.2.5/16'::cidr < '1.2.3/24'::cidr;
Tom> ?column?
Tom> --------
Tom> f
Tom> (1 row)

Tom> which looks wrong to me.

Tom> In short, it's a bug in the inet data types, not a generic problem
Tom> with unique indexes.

Tom> regards, tom lane

From bouncefilter Thu Nov 11 16:05:49 1999
Received: from ns1.foothill.net (root@ns1.foothill.net [206.170.175.1])
by hub.org (8.9.3/8.9.3) with ESMTP id QAA12795
for <pgsql-interfaces@postgresql.org>;
Thu, 11 Nov 1999 16:04:55 -0500 (EST)
(envelope-from stephen@sealteam.demon.co.uk)
Received: from stephens (redwood-Q5.foothill.net [204.131.12.5])
by ns1.foothill.net (8.9.3/8.9.3) with SMTP id MAA05171
for <pgsql-interfaces@postgresql.org>;
Thu, 11 Nov 1999 12:50:54 -0800 (PST)
From: "Stephen Martin" <stephen@sealteam.demon.co.uk>
To: <pgsql-interfaces@postgresql.org>
Subject: Error on db recovery..
Date: Thu, 11 Nov 1999 13:00:57 -0800
Message-ID: <NDBBKKNKKLBACABPPCCNAEPPCKAA.stephen@sealteam.demon.co.uk>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2615.200
In-Reply-To: <382AE25E.485114A@alumni.caltech.edu>
Importance: Normal

Hello,
I am attempting to recover a database previously secured with pg_dump,
however on attempting to restore using

pgsql < db_security_file

I get the following error(s)

ERROR: tbl_breeders relation already exists

I have removed all data tables and user from the database,
what am I over looking?

Stephen

From bouncefilter Thu Nov 11 16:09:50 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id QAA13431
for <pgsql-hackers@postgreSQL.org>;
Thu, 11 Nov 1999 16:09:23 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id QAA16204;
Thu, 11 Nov 1999 16:08:10 -0500 (EST)
To: Karel Zak - Zakkr <zakkr@zf.jcu.cz>
cc: pgsql-hackers <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] compression in LO and other fields
In-reply-to: Your message of Thu, 11 Nov 1999 19:55:56 +0100 (CET)
<Pine.LNX.3.96.991111194051.4055A-100000@ara.zf.jcu.cz>
Date: Thu, 11 Nov 1999 16:08:10 -0500
Message-ID: <16202.942354490@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Karel Zak - Zakkr <zakkr@zf.jcu.cz> writes:

I see the inv_api and other LO routines, and my idea is add support
for bzip2 stream to the inv_api and allow in current LO routines used
compression. It si good idea?

LO is a dead end. What we really want to do is eliminate tuple-size
restrictions and then have large ordinary fields (probably of type
bytea) in regular tuples. I'd suggest working on compression in that
context, say as a new data type called "bytez" or something like that.
bytez would act just like bytea except the on-disk representation would
be compressed. A compressed variant of type "text" would also be useful.
In the long run this will be much more useful and easier to work with
than adding another frammish to large objects.

regards, tom lane

From bouncefilter Thu Nov 11 16:09:49 1999
Received: from flame.flame.co.za (flame.flame.co.za [160.124.170.1])
by hub.org (8.9.3/8.9.3) with ESMTP id QAA13406
for <pgsql-hackers@postgreSQL.org>;
Thu, 11 Nov 1999 16:08:53 -0500 (EST)
(envelope-from theo@flame.co.za)
Received: from flame.co.za (flame [160.124.170.1])
by flame.flame.co.za (8.8.7/8.8.7) with ESMTP id XAA22816
for <pgsql-hackers@postgreSQL.org>; Thu, 11 Nov 1999 23:10:20 +0200
Sender: theo@flame.flame.co.za
Message-ID: <382B30BC.E31B0C20@flame.co.za>
Date: Thu, 11 Nov 1999 23:10:20 +0200
From: Theo Kramer <theo@flame.co.za>
Organization: Flame Computing Enterprises
X-Mailer: Mozilla 4.5 [en] (X11; I; Linux 2.0.35 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Slow - grindingly slow - query
References: <382B1BD5.8A7BA9DB@flame.co.za> <382B29FB.9FFBF61C@tm.ee>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hannu Krosing wrote:

try using
create index registrationtype_index on accounts (registrationtype);

OK did that, and am rerunning the query.

The explain now shows
explain select accountdetail.domain from accountdetail where
accountdetail.domain not in
(select accountmaster.domain from accountmaster);
NOTICE: QUERY PLAN:

Seq Scan on accounts (cost=3667.89 rows=34958 width=12)
SubPlan
-> Index Scan using registrationtype_idx on accounts (cost=2444.62
rows=33373 width=12)

EXPLAIN

Will let you all know when it completes.
--------
Regards
Theo

From bouncefilter Thu Nov 11 16:18:48 1999
Received: (from news@localhost) by hub.org (8.9.3/8.9.3) id QAA14864
for pgsql-hackers@postgresql.org; Thu, 11 Nov 1999 16:18:01 -0500 (EST)
(envelope-from news)
X-Authentication-Warning: hub.org: news set sender to <news> using -f
Message-ID: <382B348A.B9B3F57B@cpsgroup.com>
Date: Thu, 11 Nov 1999 15:26:35 -0600
From: Bryan Ingram <bingram@cpsgroup.com>
Reply-To: bingram@cpsgroup.com
X-Mailer: Mozilla 4.6 [en] (WinNT; I)
X-Accept-Language: en
MIME-Version: 1.0
X-Newsgroups: comp.databases.postgresql.hackers
Subject: syncing, replicating & crash recovery Q's
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-Trace: 11 Nov 1999 15:12:56 -0600, 216.207.61.67
Lines: 49
X-Authenticated-User: boneyjames
X-Report: Report abuse to abuse@newsfeeds.com
X-Abuse-Info: Please be sure to forward a copy of ALL headers,
INCLUDING the body
X-Abuse-Info2: ALL Spam complaints are acted upon within 24 hours!
Organization: Newsfeeds.com http://www.newsfeeds.com 60,
000+ UNCENSORED Newsgroups.
To: pgsql-hackers@postgresql.org

Hi,

I was hoping one of the contributors here on the list could point me in
the right direction with some of the issues I am currently facing with
postgres 6.5.0

I apologize if these topics have been thoroughly discussed before, but I
wasn't able to find anything in the archives.

For these questions, assume that the slave and master are separate
machines each running postmaster, but the postgres data files are
mounted on the same physical (but on different partitions) raid-5
server.

1) real-time syncing
Is it possible to have a "slave" postgres server (a separate machine)
doing nothing but syncing to a master? If the master were to go down,
the slave would be up-to-date and ready to handle connection requests
almost instantly.

2) Replicating
If #1 isn't possible, what would be the best way to periodically sync
the two databases? Would it be possible to exchange deltas (the slave's
only job is to duplicate the master) or would a complete backup and
restore frome one machine to another be necessary?

3) Crash-Recovery!!
The documentation does not have this section filled in. So I'm not even
sure whether or not postgres maintains transaction logging or some other
way to recover from a serious crash. Can someone please give me the
scoop on the current state of this, or what should be done to recover
from a crash?

Are there any other concerns or technologies I'm not aware of that might
be of use?

All comments appreciated!

Thanks,
Bryan Ingram

-----------== Posted via Newsfeeds.Com, Uncensored Usenet News ==----------
http://www.newsfeeds.com The Largest Usenet Servers in the World!
------== Over 73,000 Newsgroups - Including Dedicated Binaries Servers ==-----

From bouncefilter Thu Nov 11 16:29:49 1999
Received: from merganser.its.uu.se (merganser.its.uu.se [130.238.6.236])
by hub.org (8.9.3/8.9.3) with ESMTP id QAA17061
for <hackers@postgresql.org>; Thu, 11 Nov 1999 16:29:06 -0500 (EST)
(envelope-from peter@peter-e.yi.org)
Received: from regulus.its.uu.se ([130.238.7.19]:64453 "EHLO peter-e.yi.org")
by merganser.its.uu.se with ESMTP id <S.s8nIQ203139>;
Thu, 11 Nov 1999 22:29:00 +0100
Received: from peter (helo=localhost)
by peter-e.yi.org with local-esmtp (Exim 3.02 #2)
id 11m1q8-0000ET-00; Thu, 11 Nov 1999 22:33:08 +0100
Date: Thu, 11 Nov 1999 22:33:08 +0100 (CET)
From: Peter Eisentraut <peter_e@gmx.net>
To: Tom Lane <tgl@sss.pgh.pa.us>
cc: Zeugswetter Andreas SEV <ZeugswetterA@wien.spardat.at>,
"'hackers@postgresql.org'" <hackers@postgresql.org>
Subject: Re: AW: [HACKERS] Re: [GENERAL] users in Postgresql
In-Reply-To: <13363.942288057@sss.pgh.pa.us>
Message-ID: <Pine.LNX.4.20.9911112230211.442-100000@peter-e.yi.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: Peter Eisentraut <peter@peter-e.yi.org>

On 1999-11-10, Tom Lane mentioned:

AFAIK it's not *essential* to make Postgres and Unix UIDs the same
... but I think it is convenient to do so from an admin standpoint.
(One less set of numbers to keep track of, and one fewer way to get
confused about who is who.) I would not like to see you remove a
feature that makes it easy to do that.

Of course there's no value in it if you are running a setup in which
not all the Postgres users have Unix-system accounts. But that doesn't
mean there is no value in it for installations where there is such a
correspondence.

Excuse my ignorance once again, but

1) Why bother about those sysids at all? To the end user/administrator they
have about the same informational value as the oid of the float4 type. As
long as you always write "float4" or "username" you don't have to bother.

2) The mere fact of mentioning or even prompting for these ids confuses users.

3) If you really "keep track" of user ids (Unix or PostgreSQL) you really don't
have enough users or a really superior brain.

4) The purpose of the wrapper scripts was to provide "wrappers" around the
various SQL commands. If you do something in the scripts that you can't do in
"SQL" then we'll never stop having these confused users that at one point
almost caused us to remove the scripts altogether.

5) How exactly are you supposed to set the uid? The behaviour of
INSERT INTO pg_shadow VALUES (...);
and
UPDATE pg_shadow SET usesysid = ...;
is non-deterministic at best, unfortunately. The proper fix (ignoring the first
4 points above) would be to provide an extension to CREATE/ALTER USER, and
*then* we can extend the scripts that way.

I seems to me that the scripts were written before there even was a CREATE USER
command and then the functionality was just carried over without much
contemplation.

Well, okay, everyone that wants to set their PostgreSQL user id
explicitly, send me a note and I'll put it back in, which ever way.

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Thu Nov 11 16:32:49 1999
Received: from merganser.its.uu.se (merganser.its.uu.se [130.238.6.236])
by hub.org (8.9.3/8.9.3) with ESMTP id QAA17784
for <pgsql-hackers@postgresql.org>;
Thu, 11 Nov 1999 16:32:27 -0500 (EST)
(envelope-from peter@peter-e.yi.org)
Received: from regulus.its.uu.se ([130.238.7.19]:64856 "EHLO peter-e.yi.org")
by merganser.its.uu.se with ESMTP id <S.s8nLU211319>;
Thu, 11 Nov 1999 22:32:16 +0100
Received: from peter (helo=localhost)
by peter-e.yi.org with local-esmtp (Exim 3.02 #2)
id 11m1tK-0000Ea-00; Thu, 11 Nov 1999 22:36:26 +0100
Date: Thu, 11 Nov 1999 22:36:26 +0100 (CET)
From: Peter Eisentraut <peter_e@gmx.net>
To: Bruce Momjian <maillist@candle.pha.pa.us>
cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: psql and \p\g
In-Reply-To: <199911111821.NAA05463@candle.pha.pa.us>
Message-ID: <Pine.LNX.4.20.9911112233310.442-100000@peter-e.yi.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: Peter Eisentraut <peter@peter-e.yi.org>

On 1999-11-11, Bruce Momjian mentioned:

I have found that typing:

test=> select * from pg_class\p\g

no longer works. I honors the \p, but ignores the \g.

Any ideas Peter?

select * from foo \p \g

This was done to normalize the grammar a little bit (haha, very
funny). In particular it allows this sort of stuff:
=> select * from foo \p \o out.txt \g \\ select * from foo 2 \x \g
etc.

Is it *really* necessary to be able to omit the space?

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Thu Nov 11 16:37:49 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id QAA18632
for <hackers@postgreSQL.org>; Thu, 11 Nov 1999 16:37:24 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id QAA16353;
Thu, 11 Nov 1999 16:36:39 -0500 (EST)
To: Peter Eisentraut <peter_e@gmx.net>
cc: Zeugswetter Andreas SEV <ZeugswetterA@wien.spardat.at>,
"'hackers@postgresql.org'" <hackers@postgreSQL.org>
Subject: Re: AW: [HACKERS] Re: [GENERAL] users in Postgresql
In-reply-to: Your message of Thu, 11 Nov 1999 22:33:08 +0100 (CET)
<Pine.LNX.4.20.9911112230211.442-100000@peter-e.yi.org>
Date: Thu, 11 Nov 1999 16:36:39 -0500
Message-ID: <16351.942356199@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Peter Eisentraut <peter_e@gmx.net> writes:

5) How exactly are you supposed to set the uid? The behaviour of
INSERT INTO pg_shadow VALUES (...);
and
UPDATE pg_shadow SET usesysid = ...;
is non-deterministic at best, unfortunately.

INSERT seems to have worked fine in the old version of createuser...
but I agree CREATE/ALTER USER ought to have the same functionality.

regards, tom lane

From bouncefilter Thu Nov 11 16:33:49 1999
Received: from merganser.its.uu.se (merganser.its.uu.se [130.238.6.236])
by hub.org (8.9.3/8.9.3) with ESMTP id QAA17912
for <pgsql-hackers@postgresql.org>;
Thu, 11 Nov 1999 16:33:47 -0500 (EST)
(envelope-from peter@peter-e.yi.org)
Received: from regulus.its.uu.se ([130.238.7.19]:62349 "EHLO peter-e.yi.org")
by merganser.its.uu.se with ESMTP id <S.s8nMn199043>;
Thu, 11 Nov 1999 22:33:39 +0100
Received: from peter (helo=localhost)
by peter-e.yi.org with local-esmtp (Exim 3.02 #2)
id 11m1uf-0000Ec-00; Thu, 11 Nov 1999 22:37:49 +0100
Date: Thu, 11 Nov 1999 22:37:49 +0100 (CET)
From: Peter Eisentraut <peter_e@gmx.net>
To: Bruce Momjian <maillist@candle.pha.pa.us>
cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: failure of \e in psql
In-Reply-To: <199911111827.NAA05614@candle.pha.pa.us>
Message-ID: <Pine.LNX.4.20.9911112236460.442-100000@peter-e.yi.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: Peter Eisentraut <peter@peter-e.yi.org>

On 1999-11-11, Bruce Momjian mentioned:

I see that \e no longer works as expected:

test=> select * from pg_class;
...
test=> \e

and in the editor, the query is not appearing. The 'select' query
should appear in the editor because I have not entered a non-backslash
command to clear the query buffer.

Well, once you send it, it's sent and gone. You have to edit it before you
send it. I guess I'm not following you. Of course you should somehow be
able to re-edit the previous query. Hmm.

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Thu Nov 11 16:52:50 1999
Received: from candle.pha.pa.us (maillist@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id QAA21378
for <pgsql-hackers@postgreSQL.org>;
Thu, 11 Nov 1999 16:52:07 -0500 (EST)
(envelope-from maillist@candle.pha.pa.us)
Received: (from maillist@localhost)
by candle.pha.pa.us (8.9.0/8.9.0) id QAA16006;
Thu, 11 Nov 1999 16:50:19 -0500 (EST)
From: Bruce Momjian <maillist@candle.pha.pa.us>
Message-Id: <199911112150.QAA16006@candle.pha.pa.us>
Subject: Re: psql and \p\g
In-Reply-To: <Pine.LNX.4.20.9911112233310.442-100000@peter-e.yi.org> from
Peter
Eisentraut at "Nov 11, 1999 10:36:26 pm"
To: Peter Eisentraut <peter_e@gmx.net>
Date: Thu, 11 Nov 1999 16:50:18 -0500 (EST)
CC: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On 1999-11-11, Bruce Momjian mentioned:

I have found that typing:

test=> select * from pg_class\p\g

no longer works. I honors the \p, but ignores the \g.

Any ideas Peter?

select * from foo \p \g

This was done to normalize the grammar a little bit (haha, very
funny). In particular it allows this sort of stuff:
=> select * from foo \p \o out.txt \g \\ select * from foo 2 \x \g
etc.

Is it *really* necessary to be able to omit the space?

Yes, I believe it is, in the sense that many people are used to doing
them together. Can a backslash trigger some separation of commands, or
at least \p\g be recognized correctly. I don't think there are other
meaningful combinations.

Can you add a unix-style timestamp for \T?

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Thu Nov 11 16:52:49 1999
Received: from candle.pha.pa.us (maillist@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id QAA21422
for <pgsql-hackers@postgreSQL.org>;
Thu, 11 Nov 1999 16:52:31 -0500 (EST)
(envelope-from maillist@candle.pha.pa.us)
Received: (from maillist@localhost)
by candle.pha.pa.us (8.9.0/8.9.0) id QAA16058;
Thu, 11 Nov 1999 16:51:14 -0500 (EST)
From: Bruce Momjian <maillist@candle.pha.pa.us>
Message-Id: <199911112151.QAA16058@candle.pha.pa.us>
Subject: Re: failure of \e in psql
In-Reply-To: <Pine.LNX.4.20.9911112236460.442-100000@peter-e.yi.org> from
Peter
Eisentraut at "Nov 11, 1999 10:37:49 pm"
To: Peter Eisentraut <peter_e@gmx.net>
Date: Thu, 11 Nov 1999 16:51:14 -0500 (EST)
CC: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On 1999-11-11, Bruce Momjian mentioned:

I see that \e no longer works as expected:

test=> select * from pg_class;
...
test=> \e

and in the editor, the query is not appearing. The 'select' query
should appear in the editor because I have not entered a non-backslash
command to clear the query buffer.

Well, once you send it, it's sent and gone. You have to edit it before you
send it. I guess I'm not following you. Of course you should somehow be
able to re-edit the previous query. Hmm.

That is how it used to work. You run the query, get an error, and \e
pulls it into the editor for fixing.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Thu Nov 11 16:57:49 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id QAA22207
for <pgsql-hackers@postgreSQL.org>;
Thu, 11 Nov 1999 16:57:45 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id QAA16516;
Thu, 11 Nov 1999 16:56:36 -0500 (EST)
To: Theo Kramer <theo@flame.co.za>
cc: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Slow - grindingly slow - query
In-reply-to: Your message of Thu, 11 Nov 1999 21:41:09 +0200
<382B1BD5.8A7BA9DB@flame.co.za>
Date: Thu, 11 Nov 1999 16:56:36 -0500
Message-ID: <16514.942357396@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Theo Kramer <theo@flame.co.za> writes:

The query is

select accountdetail.domain from accountdetail where
accountdetail.domain not in
(select accountmaster.domain from accountmaster);

Try something like

select accountdetail.domain from accountdetail where
not exists (select accountmaster.domain from accountmaster where
accountmaster.domain = accountdetail.domain);

I believe this is in the FAQ...

regards, tom lane

From bouncefilter Thu Nov 11 17:04:49 1999
Received: from perio.unlp.edu.ar (IDENT:root@perio.unlp.edu.ar [163.10.16.2])
by hub.org (8.9.3/8.9.3) with ESMTP id RAA23332
for <pgsql-interfaces@postgresql.org>;
Thu, 11 Nov 1999 17:04:36 -0500 (EST)
(envelope-from ser@perio.unlp.edu.ar)
Received: from speedy ([163.10.16.11])
by perio.unlp.edu.ar (8.9.3/8.8.7) with SMTP id TAA25062;
Thu, 11 Nov 1999 19:04:45 GMT
From: "Sergio A. Kessler" <ser@perio.unlp.edu.ar>
Message-Id: <SAK.1999.11.11.ifhlcnef@speedy>
In-Reply-To: <NDBBKKNKKLBACABPPCCNAEPPCKAA.stephen@sealteam.demon.co.uk>
Date: Thu, 11 Nov 1999 19:04:20 -0300
X-Priority: 3
X-Mailer: Correo F���cil
To: pgsql-interfaces@postgresql.org, stephen@sealteam.demon.co.uk
MIME-Version: 1.0
Subject: Re: [INTERFACES] Error on db recovery..
Content-Type: Text/Plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8Bit

"Stephen Martin" <stephen@sealteam.demon.co.uk> el d���a Thu, 11 Nov 1999
13:00:57 -0800, escribi���:

Hello,
I am attempting to recover a database previously secured with pg_dump,
however on attempting to restore using

pgsql < db_security_file

bad, bad ...

go to /usr/doc/postgres and read ...

Sergio

From bouncefilter Thu Nov 11 17:17:49 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id RAA25448
for <pgsql-hackers@postgreSQL.org>;
Thu, 11 Nov 1999 17:16:57 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id RAA16638;
Thu, 11 Nov 1999 17:16:12 -0500 (EST)
To: Bruce Momjian <maillist@candle.pha.pa.us>
cc: Peter Eisentraut <peter_e@gmx.net>,
PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Re: psql and \p\g
In-reply-to: Your message of Thu, 11 Nov 1999 16:50:18 -0500 (EST)
<199911112150.QAA16006@candle.pha.pa.us>
Date: Thu, 11 Nov 1999 17:16:12 -0500
Message-ID: <16636.942358572@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Bruce Momjian <maillist@candle.pha.pa.us> writes:

This was done to normalize the grammar a little bit (haha, very
funny). In particular it allows this sort of stuff:
=> select * from foo \p \o out.txt \g \\ select * from foo 2 \x \g
etc.

Is it *really* necessary to be able to omit the space?

Yes, I believe it is, in the sense that many people are used to doing
them together. Can a backslash trigger some separation of commands, or
at least \p\g be recognized correctly. I don't think there are other
meaningful combinations.

It'd probably be sufficient if backslash-commands that never take
parameters can be adjacent to a following backslash command.

regards, tom lane

From bouncefilter Thu Nov 11 18:21:50 1999
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA34311
for <hackers@postgreSQL.org>; Thu, 11 Nov 1999 18:20:52 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id PAA16254;
Thu, 11 Nov 1999 15:20:21 -0800 (PST)
Message-Id: <3.0.1.32.19991111151615.00fd5130@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Thu, 11 Nov 1999 15:16:15 -0800
To: Theo Kramer <theo@flame.co.za>, hackers@postgreSQL.org
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: [HACKERS] Indent
In-Reply-To: <382B1EBB.E58133A8@flame.co.za>
References: <Pine.GSO.4.02A.9911111631090.22368-100000@Lodjur.DoCS.UU.SE>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 09:53 PM 11/11/99 +0200, Theo Kramer wrote:

Peter Eisentraut wrote:

Personally, I'm always in favour of using no tabs at all, because of this
very problem, and just because they annoy me. But I tend to be alone with
that position.

No you are not :). Twenty odd years of software development have taught me
to remove the tab key from my keyboard. Two spaces do the trick for me and
try to avoid at all costs lines over 80 cols. (I use emacs in vi[per] mode),
even on win98... Oh how I wish we were back in the days of TECO.

TECO! If you were to dig up a copy of the original manual for
Dec's OS/8 TECO, you'd see a reference to me on the first page. Not
by name, but by organization, for having done the first version.

Sheesh, those where old days.

Anyway, I too avoid tabs. Life's too short to figure out how to
make all the tools I use tab the way I want them to.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Thu Nov 11 18:45:50 1999
Received: from atbib.be (IDENT:root@a01-148.antw.online.be [62.112.2.148])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA37220
for <pgsql-hackers@postgresql.org>;
Thu, 11 Nov 1999 18:45:46 -0500 (EST) (envelope-from fve@atbib.be)
Received: from frans.atbib.be (fransnieuw [193.75.233.10])
by atbib.be (8.9.3/8.9.3) with SMTP id BAA08204
for <pgsql-hackers@postgresql.org>; Fri, 12 Nov 1999 01:45:20 +0100
Message-Id: <3.0.6.32.19991112005011.0086ce30@193.75.233.1>
X-Sender: fve@193.75.233.1
X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32)
Date: Fri, 12 Nov 1999 00:50:11 +0100
To: pgsql-hackers@postgresql.org
From: Frans Van Elsacker <fve@atbib.be>
Subject: new version 6.5.3
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

Probably it is of interest for jou, or i did something very stupid:
I had a little piece of code who works fine up till know, just after
installing
the version 6.5.3 i got the message 'parse error near union'

create table work as
select * from opdracht
union
select * from opdrachtproost

I know i can solve the problem with
insert into work in stead of the union keyword, but it is enoying that all
the programs had to be recompiled

many thanks, its a good product

Frans

From bouncefilter Thu Nov 11 18:50:50 1999
Received: from atbib.be (IDENT:root@a01-148.antw.online.be [62.112.2.148])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA37748
for <pgsql-hackers@postgreSQL.org>;
Thu, 11 Nov 1999 18:50:17 -0500 (EST) (envelope-from fve@atbib.be)
Received: from frans.atbib.be (fransnieuw [193.75.233.10])
by atbib.be (8.9.3/8.9.3) with SMTP id BAA08262
for <pgsql-hackers@postgreSQL.org>; Fri, 12 Nov 1999 01:49:57 +0100
Message-Id: <3.0.6.32.19991112005448.00876670@193.75.233.1>
X-Sender: fve@193.75.233.1
X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32)
Date: Fri, 12 Nov 1999 00:54:48 +0100
To: pgsql-hackers@postgreSQL.org
From: Frans Van Elsacker <fve@atbib.be>
Subject:
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

subscribe

From bouncefilter Thu Nov 11 18:53:50 1999
Received: from atbib.be (IDENT:root@a01-148.antw.online.be [62.112.2.148])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA37869
for <pgsql-hackers@postgreSQL.org>;
Thu, 11 Nov 1999 18:53:40 -0500 (EST) (envelope-from fve@atbib.be)
Received: from frans.atbib.be (fransnieuw [193.75.233.10])
by atbib.be (8.9.3/8.9.3) with SMTP id BAA08295
for <pgsql-hackers@postgreSQL.org>; Fri, 12 Nov 1999 01:53:19 +0100
Message-Id: <3.0.6.32.19991112005811.00877450@193.75.233.1>
X-Sender: fve@193.75.233.1
X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32)
Date: Fri, 12 Nov 1999 00:58:11 +0100
To: pgsql-hackers@postgreSQL.org
From: Frans Van Elsacker <fve@atbib.be>
Subject: subscribe
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

subscribe

From bouncefilter Thu Nov 11 19:05:50 1999
Received: from atbib.be (IDENT:root@a02-063.antw.online.be [62.112.3.63])
by hub.org (8.9.3/8.9.3) with ESMTP id TAA39360
for <pgsql-hackers@postgresql.org>;
Thu, 11 Nov 1999 19:05:35 -0500 (EST) (envelope-from fve@atbib.be)
Received: from frans.atbib.be (fransnieuw [193.75.233.10])
by atbib.be (8.9.3/8.9.3) with SMTP id CAA08404
for <pgsql-hackers@postgresql.org>; Fri, 12 Nov 1999 02:05:14 +0100
Message-Id: <3.0.6.32.19991112011006.00876c00@193.75.233.1>
X-Sender: fve@193.75.233.1
X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32)
Date: Fri, 12 Nov 1999 01:10:06 +0100
To: pgsql-hackers@postgresql.org
From: Frans Van Elsacker <fve@atbib.be>
Subject: union problem version 6.5.3
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

Probably it is of interest for jou, or i did something very stupid:
I had a little piece of code who works fine up till know, just after
installing
the version 6.5.3 i got the message 'parse error near union'

create table work as
select * from opdracht
union
select * from opdrachtproost

I know i can solve the problem with
insert into work in stead of the union keyword, but it is enoying that all
the programs had to be recompiled

many thanks, its a good product

Frans

From bouncefilter Thu Nov 11 21:01:51 1999
Received: from sraigw.sra.co.jp (sraigw.sra.co.jp [202.32.10.2])
by hub.org (8.9.3/8.9.3) with ESMTP id VAA56809
for <pgsql-hackers@postgreSQL.org>;
Thu, 11 Nov 1999 21:01:44 -0500 (EST)
(envelope-from t-ishii@srapc451.sra.co.jp)
Received: from srapc451.sra.co.jp (srapc451 [133.137.44.37])
by sraigw.sra.co.jp (8.8.7/3.7W-sraigw) with ESMTP id LAA11506;
Fri, 12 Nov 1999 11:00:08 +0900 (JST)
Received: from srapc451.sra.co.jp (localhost [127.0.0.1]) by
srapc451.sra.co.jp (8.8.8/3.5Wpl7) with ESMTP id LAA19911;
Fri, 12 Nov 1999 11:00:08 +0900 (JST)
Message-Id: <199911120200.LAA19911@srapc451.sra.co.jp>
To: Tom Lane <tgl@sss.pgh.pa.us>
cc: Karel Zak - Zakkr <zakkr@zf.jcu.cz>,
pgsql-hackers <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] compression in LO and other fields
From: Tatsuo Ishii <t-ishii@sra.co.jp>
Reply-To: t-ishii@sra.co.jp
In-reply-to: Your message of Thu, 11 Nov 1999 16:08:10 EST.
<16202.942354490@sss.pgh.pa.us>
Date: Fri, 12 Nov 1999 11:00:08 +0900
Sender: t-ishii@srapc451.sra.co.jp

LO is a dead end. What we really want to do is eliminate tuple-size
restrictions and then have large ordinary fields (probably of type
bytea) in regular tuples. I'd suggest working on compression in that
context, say as a new data type called "bytez" or something like that.

It sounds ideal but I remember that Vadim said inserting a 2GB record
is not good idea since it will be written into the log too. If it's a
necessary limitation from the point of view of WAL, we have to accept
it, I think.

BTW, I still don't have enough time to run the huge sort tests on
6.5.x. Probably I would have chance next week to do that...
---
Tatsuo Ishii

From bouncefilter Thu Nov 11 22:41:53 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id WAA68559
for <pgsql-hackers@postgreSQL.org>;
Thu, 11 Nov 1999 22:41:38 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgreSQL.org
id m11m7SM-0003kLC; Fri, 12 Nov 99 04:32 MET
Message-Id: <m11m7SM-0003kLC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] compression in LO and other fields
To: t-ishii@sra.co.jp
Date: Fri, 12 Nov 1999 04:32:58 +0100 (MET)
Cc: tgl@sss.pgh.pa.us, zakkr@zf.jcu.cz, pgsql-hackers@postgreSQL.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <199911120200.LAA19911@srapc451.sra.co.jp> from "Tatsuo Ishii" at
Nov 12, 99 11:00:08 am
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Tatsuo Ishii wrote:

LO is a dead end. What we really want to do is eliminate tuple-size
restrictions and then have large ordinary fields (probably of type
bytea) in regular tuples. I'd suggest working on compression in that
context, say as a new data type called "bytez" or something like that.

It sounds ideal but I remember that Vadim said inserting a 2GB record
is not good idea since it will be written into the log too. If it's a
necessary limitation from the point of view of WAL, we have to accept
it, I think.

Just in case someone want to implement a complete compressed
data type (including comarision functions, operators and
indexing default operator class).

I already made some tests with a type I called 'lztext'
locally. Only the input-/output-functions exist so far and
as the name might suggest, it would be an alternative for
'text'. It uses a simple but fast, byte oriented LZ backward
pointing method. No Huffman coding or variable offset/size
tagging. First byte of a chunk tells bitwise if the next
following 8 items are raw bytes to copy or 12 bit offset, 4
bit size copy information. That is max back offset 4096 and
max match size 17 bytes.

What made it my preferred method was the fact, that
decompression is done entirely using the already decompressed
portion of the data, so it does not need any code tables or
the like at that time.

It is really FASTEST on decompression, which I assume would
be the mostly often used operation on huge data types. With
some care, comparision could be done on the fly while
decompressing two values, so that the entire comparision can
be aborted at the occurence of the first difference.

The compression rates aren't that giantic. I've got 30-50%
for rule plan strings (size limit on views!!!). And the
method used only allows for buffer back references of 4K
offsets at most, so the rate will not grow for larger data
chunks. That's a heavy tradeoff between compression rate and
no memory leakage for sure and speed, I know, but I prefer
not to force it, instead I usually use a bigger hammer (the
tuple size limit is still our original problem - and another
IBM 72GB disk doing 22-37 MB/s will make any compressing data
type obsolete then).

Sorry for the compression specific slang here. Well, anyone
interested in the code?

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Thu Nov 11 22:55:53 1999
Received: from candle.pha.pa.us (maillist@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA69821
for <pgsql-hackers@postgreSQL.org>;
Thu, 11 Nov 1999 22:55:00 -0500 (EST)
(envelope-from maillist@candle.pha.pa.us)
Received: (from maillist@localhost)
by candle.pha.pa.us (8.9.0/8.9.0) id WAA26738;
Thu, 11 Nov 1999 22:50:16 -0500 (EST)
From: Bruce Momjian <maillist@candle.pha.pa.us>
Message-Id: <199911120350.WAA26738@candle.pha.pa.us>
Subject: Re: [HACKERS] compression in LO and other fields
In-Reply-To: <m11m7SM-0003kLC@orion.SAPserv.Hamburg.dsh.de> from Jan Wieck at
"Nov 12, 1999 04:32:58 am"
To: Jan Wieck <wieck@debis.com>
Date: Thu, 11 Nov 1999 22:50:16 -0500 (EST)
CC: t-ishii@sra.co.jp, tgl@sss.pgh.pa.us, zakkr@zf.jcu.cz,
pgsql-hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

The compression rates aren't that giantic. I've got 30-50%
for rule plan strings (size limit on views!!!). And the
method used only allows for buffer back references of 4K
offsets at most, so the rate will not grow for larger data
chunks. That's a heavy tradeoff between compression rate and
no memory leakage for sure and speed, I know, but I prefer
not to force it, instead I usually use a bigger hammer (the
tuple size limit is still our original problem - and another
IBM 72GB disk doing 22-37 MB/s will make any compressing data
type obsolete then).

Sorry for the compression specific slang here. Well, anyone
interested in the code?

In contrib?

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Fri Nov 12 00:07:54 1999
Received: from flame.flame.co.za (flame.flame.co.za [160.124.170.1])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA81540
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 00:07:38 -0500 (EST)
(envelope-from theo@flame.co.za)
Received: from flame.co.za (flame [160.124.170.1])
by flame.flame.co.za (8.8.7/8.8.7) with ESMTP id HAA23704
for <pgsql-hackers@postgreSQL.org>; Fri, 12 Nov 1999 07:09:15 +0200
Sender: theo@flame.flame.co.za
Message-ID: <382BA0FB.82D96E3C@flame.co.za>
Date: Fri, 12 Nov 1999 07:09:15 +0200
From: Theo Kramer <theo@flame.co.za>
Organization: Flame Computing Enterprises
X-Mailer: Mozilla 4.5 [en] (X11; I; Linux 2.0.35 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Slow - grindingly slow - query
References: <16514.942357396@sss.pgh.pa.us>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Tom Lane wrote:

Theo Kramer <theo@flame.co.za> writes:

The query is

select accountdetail.domain from accountdetail where
accountdetail.domain not in
(select accountmaster.domain from accountmaster);

This takes more than 5 hours and 30 minutes.

Try something like

select accountdetail.domain from accountdetail where
not exists (select accountmaster.domain from accountmaster where
accountmaster.domain = accountdetail.domain);

This takes 5 seconds - wow!

I believe this is in the FAQ...

Will check out the FAQs. Many thanks.
--------
Regards
Theo

From bouncefilter Fri Nov 12 00:52:55 1999
Received: from sunpine.krs.ru (SunPine.krs.ru [195.161.16.37])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA87549
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 00:52:40 -0500 (EST) (envelope-from vadim@krs.ru)
Received: from krs.ru (dune.krs.ru [195.161.16.38])
by sunpine.krs.ru (8.8.8/8.8.8) with ESMTP id MAA10217;
Fri, 12 Nov 1999 12:52:30 +0700 (KRS)
Sender: root@sunpine.krs.ru
Message-ID: <382BAB1D.4AF25BFE@krs.ru>
Date: Fri, 12 Nov 1999 12:52:29 +0700
From: Vadim Mikheev <vadim@krs.ru>
Organization: OJSC Rostelecom (Krasnoyarsk)
X-Mailer: Mozilla 4.5 [en] (X11; I; FreeBSD 3.0-RELEASE i386)
X-Accept-Language: ru, en
MIME-Version: 1.0
To: Theo Kramer <theo@flame.co.za>
CC: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Slow - grindingly slow - query
References: <16514.942357396@sss.pgh.pa.us> <382BA0FB.82D96E3C@flame.co.za>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Theo Kramer wrote:

Try something like

select accountdetail.domain from accountdetail where
not exists (select accountmaster.domain from accountmaster where
accountmaster.domain = accountdetail.domain);

This takes 5 seconds - wow!

I did the same on Informix Online 7 and it took less than two minutes...

^^^^^^^^^^^
Could you run the query above in Informix?
How long would it take to complete?

Vadim

From bouncefilter Fri Nov 12 01:08:55 1999
Received: from candle.pha.pa.us (maillist@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id BAA89650
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 01:08:41 -0500 (EST)
(envelope-from maillist@candle.pha.pa.us)
Received: (from maillist@localhost)
by candle.pha.pa.us (8.9.0/8.9.0) id BAA01516;
Fri, 12 Nov 1999 01:07:46 -0500 (EST)
From: Bruce Momjian <maillist@candle.pha.pa.us>
Message-Id: <199911120607.BAA01516@candle.pha.pa.us>
Subject: Re: failure of \e in psql
In-Reply-To: <Pine.LNX.4.20.9911112236460.442-100000@peter-e.yi.org> from
Peter
Eisentraut at "Nov 11, 1999 10:37:49 pm"
To: Peter Eisentraut <peter_e@gmx.net>
Date: Fri, 12 Nov 1999 01:07:46 -0500 (EST)
CC: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On 1999-11-11, Bruce Momjian mentioned:

I see that \e no longer works as expected:

test=> select * from pg_class;
...
test=> \e

and in the editor, the query is not appearing. The 'select' query
should appear in the editor because I have not entered a non-backslash
command to clear the query buffer.

Well, once you send it, it's sent and gone. You have to edit it before you
send it. I guess I'm not following you. Of course you should somehow be
able to re-edit the previous query. Hmm.

Peter, before I go hunting around, can you tell me any other things psql
used to do that it doesn't do anymore?

We had hand-tuned psql over the years, and it would be good to know what
features no longer exist so we can decide if they are needed.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Fri Nov 12 01:15:55 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id BAA90635
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 01:15:53 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id BAA25686;
Fri, 12 Nov 1999 01:14:44 -0500 (EST)
To: t-ishii@sra.co.jp
cc: Karel Zak - Zakkr <zakkr@zf.jcu.cz>,
pgsql-hackers <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] compression in LO and other fields
In-reply-to: Your message of Fri, 12 Nov 1999 11:00:08 +0900
<199911120200.LAA19911@srapc451.sra.co.jp>
Date: Fri, 12 Nov 1999 01:14:43 -0500
Message-ID: <25684.942387283@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Tatsuo Ishii <t-ishii@sra.co.jp> writes:

LO is a dead end. What we really want to do is eliminate tuple-size
restrictions and then have large ordinary fields (probably of type
bytea) in regular tuples. I'd suggest working on compression in that
context, say as a new data type called "bytez" or something like that.

It sounds ideal but I remember that Vadim said inserting a 2GB record
is not good idea since it will be written into the log too. If it's a
necessary limitation from the point of view of WAL, we have to accept
it, I think.

LO won't make that any better: the data still goes into a table.
You'd have 2GB worth of WAL entries either way.

The only thing LO would do for you is divide the data into block-sized
tuples, so there would be a bunch of little WAL entries instead of one
big one. But that'd probably be easy to duplicate too. If we implement
big tuples by chaining together disk-block-sized segments, which seems
like the most likely approach, couldn't WAL log each segment as a
separate log entry? If so, there's almost no difference between LO and
inline field for logging purposes.

regards, tom lane

From bouncefilter Fri Nov 12 01:47:59 1999
Received: from localhost (IDENT:root@hectic-2.jpl.nasa.gov [128.149.68.204])
by hub.org (8.9.3/8.9.3) with ESMTP id BAA93966
for <hackers@postgresql.org>; Fri, 12 Nov 1999 01:47:11 -0500 (EST)
(envelope-from lockhart@alumni.caltech.edu)
Received: from alumni.caltech.edu (lockhart@localhost [127.0.0.1])
by localhost (8.8.7/8.8.7) with ESMTP id GAA08182;
Fri, 12 Nov 1999 06:54:20 GMT
Sender: lockhart@hub.org
Message-ID: <382BB99C.7B8B94E0@alumni.caltech.edu>
Date: Fri, 12 Nov 1999 06:54:20 +0000
From: Thomas Lockhart <lockhart@alumni.caltech.edu>
X-Mailer: Mozilla 4.6 [en] (X11; I; Linux 2.0.36 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: applixware-list@applix.com
CC: Postgres Hackers List <hackers@postgresql.org>
Subject: Re: AWL: Re: tm1
References: <199911112148.QAA25209@saltmine.radix.net>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

People may have problems with the NULL statements with some versions
of PostgreSQL. I have information about editing the applix macro
on that creates the tables my web site:
http://www.radix.net/~cobrien/applix/applix.txt

Just in case someone cares ;)

The "NULL" constraint for a column definition is not defined in SQL92,
and is not necessary and could be dropped from Applix's definition of
the table. The default behavior of any column defined in SQL is to
allow NULL values.

Postgres does not implement this redundant syntax extension because
yacc-style parsers such as the one used in Postgres find the use of
the bare NULL an ambiguous context. Presumably that is why SQL92 does
not define it.

However, I see that in a limited context, such as a bare NULL with no
other qualifiers, yacc can handle its use. I'll add it to Postgres'
next release...

- Thomas

--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California

From bouncefilter Fri Nov 12 02:39:59 1999
Received: from localhost (IDENT:root@hectic-2.jpl.nasa.gov [128.149.68.204])
by hub.org (8.9.3/8.9.3) with ESMTP id CAA98659
for <hackers@postgreSQL.org>; Fri, 12 Nov 1999 02:39:13 -0500 (EST)
(envelope-from lockhart@alumni.caltech.edu)
Received: from alumni.caltech.edu (lockhart@localhost [127.0.0.1])
by localhost (8.8.7/8.8.7) with ESMTP id HAA08267;
Fri, 12 Nov 1999 07:39:55 GMT
Sender: lockhart@hub.org
Message-ID: <382BC44B.2BE4AA39@alumni.caltech.edu>
Date: Fri, 12 Nov 1999 07:39:55 +0000
From: Thomas Lockhart <lockhart@alumni.caltech.edu>
X-Mailer: Mozilla 4.6 [en] (X11; I; Linux 2.0.36 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Peter Eisentraut <peter_e@gmx.net>
CC: Tom Lane <tgl@sss.pgh.pa.us>,
Zeugswetter Andreas SEV <ZeugswetterA@wien.spardat.at>,
"'hackers@postgresql.org'" <hackers@postgreSQL.org>
Subject: Re: AW: [HACKERS] Re: [GENERAL] users in Postgresql
References: <Pine.LNX.4.20.9911112230211.442-100000@peter-e.yi.org>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Well, okay, everyone that wants to set their PostgreSQL user id
explicitly, send me a note and I'll put it back in, which ever way.

I thought that Tom Lane was representing me just fine, so was keeping
quiet ;)

An aside on procedures: on a change like this, I might have expected a
discussion on functionality *before* the patch was developed, since it
changes a seemingly fundamental feature. Though I haven't thought of a
strong, or even weak, argument for why id matching is necessary, it is
a topic about which there has been no discussion in the past, so I
didn't realize I needed an opinion until now.

Another aside: I'd like to think that most good ideas which stand the
test of an extended discussion will get a consensus to form. So if you
really think this is a step forward then keep talking about it; don't
give up too soon...

Back on topic: If there is currently no apparent need for a link
between Postgres user ids and external system ids, it is the case that
this is an obvious mechanism to make that link. So if someday a user
or a system feature needs it, it is already there and has been so from
day 1. afaik other DBs have a similar attribute for users.

- Thomas

--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California

From bouncefilter Fri Nov 12 03:03:59 1999
Received: from flame.flame.co.za (flame.flame.co.za [160.124.170.1])
by hub.org (8.9.3/8.9.3) with ESMTP id DAA04732
for <pgsql-hackers@postgresql.org>;
Fri, 12 Nov 1999 03:03:44 -0500 (EST)
(envelope-from theo@flame.flame.co.za)
Received: (from theo@localhost) by flame.flame.co.za (8.8.7/8.8.7) id
KAA24099;
Fri, 12 Nov 1999 10:04:58 +0200
From: Theo Kramer <theo@flame.flame.co.za>
Message-Id: <199911120804.KAA24099@flame.flame.co.za>
Subject: Re: [HACKERS] Slow - grindingly slow - query
To: vadim@krs.ru (Vadim Mikheev)
Date: Fri, 12 Nov 1999 10:04:58 +0200 (SAST)
Cc: pgsql-hackers@postgresql.org
In-Reply-To: <382BAB1D.4AF25BFE@krs.ru> from "Vadim Mikheev" at Nov 12,
99 12:52:29 pm
Content-Type: text

Vadim wrote:

I did the same on Informix Online 7 and it took less than two minutes...

Could you run the query above in Informix?
How long would it take to complete?

I include both explain and timing for the queries for both postgres and
Informix.

Explain from postgres for the two queries.
------------------------------------------

explain select accountdetail.domain from accountdetail where
accountdetail.domain not in
(select accountmaster.domain from accountmaster);
NOTICE: QUERY PLAN:

Seq Scan on accounts (cost=3667.89 rows=34958 width=12)
SubPlan
-> Index Scan using registrationtype_idx on accounts (cost=2444.62 rows=33373 width=12)

EXPLAIN

explain select accountdetail.domain from accountdetail
where not exists (
select accountmaster.domain from accountmaster where
accountmaster.domain = accountdetail.domain);
NOTICE: QUERY PLAN:

Seq Scan on accounts (cost=3667.89 rows=34958 width=12)
SubPlan
-> Index Scan using domain_type_idx on accounts (cost=2.04 rows=1 width=12)

EXPLAIN

Explain from informix online 7 for the two queries
--------------------------------------------------

QUERY:
------
select accountdetail.domain from accountdetail where
accountdetail.domain not in (select accountmaster.domain from accountmaster)

Estimated Cost: 8995
Estimated # of Rows Returned: 47652

1) informix.accounts: SEQUENTIAL SCAN

Filters: (informix.accounts.domain != ALL <subquery> AND informix.accounts.registrationtype != 'N' )

Subquery:
---------
Estimated Cost: 4497
Estimated # of Rows Returned: 5883

1) informix.accounts: SEQUENTIAL SCAN

Filters: informix.accounts.registrationtype = 'N'

QUERY:
------
select accountdetail.domain from accountdetail where
accountdetail.domain not in (select accountmaster.domain from accountmaster)

Estimated Cost: 4510
Estimated # of Rows Returned: 58810

1) informix.accounts: SEQUENTIAL SCAN

Filters: (informix.accounts.domain != ALL <subquery> AND informix.accounts.registrationtype != 'N' )

Subquery:
---------
Estimated Cost: 12
Estimated # of Rows Returned: 10

1) informix.accounts: INDEX PATH

(1) Index Keys: registrationtype
Lower Index Filter: informix.accounts.registrationtype = 'N'

Timing from postgres 6.5.3 for the two queries
----------------------------------------------
explain select accountdetail.domain from accountdetail where
accountdetail.domain not in
(select accountmaster.domain from accountmaster);

Greater than 5 hours and 30 minutes

explain select accountdetail.domain from accountdetail
where not exists (
select accountmaster.domain from accountmaster where
accountmaster.domain = accountdetail.domain);

0.00user 0.01system 0:04.75elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k

Timing from Informix Online 7 for the two queries
----------------------------------------------
explain select accountdetail.domain from accountdetail where
accountdetail.domain not in
(select accountmaster.domain from accountmaster);

0.03user 0.01system 0:10.35elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k

explain select accountdetail.domain from accountdetail
where not exists (
select accountmaster.domain from accountmaster where
accountmaster.domain = accountdetail.domain);

0.03user 0.00system 0:03.56elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k

The machine is a Pentium II 400 MHz with Fast Wide SCSI and is the same
for both Informix and Postgres. Informix uses Linux I/O ie. it does not
use a raw partition. The datasets are the same.

Regards
Theo

From bouncefilter Fri Nov 12 03:12:56 1999
Received: from flame.flame.co.za (flame.flame.co.za [160.124.170.1])
by hub.org (8.9.3/8.9.3) with ESMTP id DAA05802
for <pgsql-hackers@postgresql.org>;
Fri, 12 Nov 1999 03:12:43 -0500 (EST)
(envelope-from theo@flame.flame.co.za)
Received: (from theo@localhost) by flame.flame.co.za (8.8.7/8.8.7) id KAA24132
for pgsql-hackers@postgresql.org; Fri, 12 Nov 1999 10:14:25 +0200
From: Theo Kramer <theo@flame.flame.co.za>
Message-Id: <199911120814.KAA24132@flame.flame.co.za>
Subject: Re: [HACKERS] Slow - grindingly slow - query
To: pgsql-hackers@postgresql.org
Date: Fri, 12 Nov 1999 10:14:25 +0200 (SAST)
Reply-To: theo@flame.co.za
Content-Type: text

Vadim wrote:

I did the same on Informix Online 7 and it took less than two minutes...

Could you run the query above in Informix?
How long would it take to complete?

I include both explain and timing for the queries for both postgres and
Informix.

Explain from postgres for the two queries.
------------------------------------------

explain select accountdetail.domain from accountdetail where
accountdetail.domain not in
(select accountmaster.domain from accountmaster);
NOTICE: QUERY PLAN:

Seq Scan on accounts (cost=3667.89 rows=34958 width=12)
SubPlan
-> Index Scan using registrationtype_idx on accounts (cost=2444.62 rows=33373 width=12)

EXPLAIN

explain select accountdetail.domain from accountdetail
where not exists (
select accountmaster.domain from accountmaster where
accountmaster.domain = accountdetail.domain);
NOTICE: QUERY PLAN:

Seq Scan on accounts (cost=3667.89 rows=34958 width=12)
SubPlan
-> Index Scan using domain_type_idx on accounts (cost=2.04 rows=1 width=12)

EXPLAIN

Explain from informix online 7 for the two queries
--------------------------------------------------

QUERY:
------
select accountdetail.domain from accountdetail where
accountdetail.domain not in (select accountmaster.domain from accountmaster)

Estimated Cost: 8995
Estimated # of Rows Returned: 47652

1) informix.accounts: SEQUENTIAL SCAN

Filters: (informix.accounts.domain != ALL <subquery> AND informix.accounts.registrationtype != 'N' )

Subquery:
---------
Estimated Cost: 4497
Estimated # of Rows Returned: 5883

1) informix.accounts: SEQUENTIAL SCAN

Filters: informix.accounts.registrationtype = 'N'

QUERY:
------
select accountdetail.domain from accountdetail where
accountdetail.domain not in (select accountmaster.domain from accountmaster)

Estimated Cost: 4510
Estimated # of Rows Returned: 58810

1) informix.accounts: SEQUENTIAL SCAN

Filters: (informix.accounts.domain != ALL <subquery> AND informix.accounts.registrationtype != 'N' )

Subquery:
---------
Estimated Cost: 12
Estimated # of Rows Returned: 10

1) informix.accounts: INDEX PATH

(1) Index Keys: registrationtype
Lower Index Filter: informix.accounts.registrationtype = 'N'

Timing from postgres 6.5.3 for the two queries
----------------------------------------------
explain select accountdetail.domain from accountdetail where
accountdetail.domain not in
(select accountmaster.domain from accountmaster);

Greater than 5 hours and 30 minutes

explain select accountdetail.domain from accountdetail
where not exists (
select accountmaster.domain from accountmaster where
accountmaster.domain = accountdetail.domain);

0.00user 0.01system 0:04.75elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k

Timing from Informix Online 7 for the two queries
----------------------------------------------
explain select accountdetail.domain from accountdetail where
accountdetail.domain not in
(select accountmaster.domain from accountmaster);

0.03user 0.01system 0:10.35elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k

explain select accountdetail.domain from accountdetail
where not exists (
select accountmaster.domain from accountmaster where
accountmaster.domain = accountdetail.domain);

0.03user 0.00system 0:03.56elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k

The machine is a Pentium II 400 MHz with Fast Wide SCSI and is the same
for both Informix and Postgres. Informix uses Linux I/O ie. it does not
use a raw partition. The datasets are the same.

Regards
Theo

From bouncefilter Fri Nov 12 04:33:57 1999
Received: from ara.zf.jcu.cz (zakkr@ara.zf.jcu.cz [160.217.161.4])
by hub.org (8.9.3/8.9.3) with ESMTP id EAA14416
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 04:33:23 -0500 (EST) (envelope-from zakkr@zf.jcu.cz)
Received: from localhost (zakkr@localhost)
by ara.zf.jcu.cz (8.9.3/8.9.3/Debian/GNU) with SMTP id KAA16562;
Fri, 12 Nov 1999 10:16:21 +0100
Date: Fri, 12 Nov 1999 10:16:21 +0100 (CET)
From: Karel Zak - Zakkr <zakkr@zf.jcu.cz>
To: Tom Lane <tgl@sss.pgh.pa.us>
cc: t-ishii@sra.co.jp, pgsql-hackers <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] compression in LO and other fields
In-Reply-To: <25684.942387283@sss.pgh.pa.us>
Message-ID: <Pine.LNX.3.96.991112094645.14930A-100000@ara.zf.jcu.cz>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Fri, 12 Nov 1999, Tom Lane wrote:

Tatsuo Ishii <t-ishii@sra.co.jp> writes:

LO is a dead end. What we really want to do is eliminate tuple-size
restrictions and then have large ordinary fields (probably of type
bytea) in regular tuples. I'd suggest working on compression in that
context, say as a new data type called "bytez" or something like that.

--- cut ---

The only thing LO would do for you is divide the data into block-sized
tuples, so there would be a bunch of little WAL entries instead of one
big one. But that'd probably be easy to duplicate too. If we implement
big tuples by chaining together disk-block-sized segments, which seems
like the most likely approach, couldn't WAL log each segment as a
separate log entry? If so, there's almost no difference between LO and
inline field for logging purposes.

I'am not sure, that LO is a dead end for every users. Big (blob) fields
going during SQL engine (?), but why - if I needn't use this data as
typically SQL data (I not need index, search .. in (example) gif files).
Will pity if LO devel. will go down. I still thing that LO compression is
not bad idea :-)

Other eventual compression questions:

* some aplication allow use over slow networks between client<->server
a compressed stream, and PostgreSQL?

* MySQL dump allow make compressed dump file, it is good, and PostgreSQL?

Karel

From bouncefilter Fri Nov 12 04:52:57 1999
Received: from ara.zf.jcu.cz (zakkr@ara.zf.jcu.cz [160.217.161.4])
by hub.org (8.9.3/8.9.3) with ESMTP id EAA16432
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 04:52:14 -0500 (EST) (envelope-from zakkr@zf.jcu.cz)
Received: from localhost (zakkr@localhost)
by ara.zf.jcu.cz (8.9.3/8.9.3/Debian/GNU) with SMTP id KAA17726;
Fri, 12 Nov 1999 10:38:55 +0100
Date: Fri, 12 Nov 1999 10:38:55 +0100 (CET)
From: Karel Zak - Zakkr <zakkr@zf.jcu.cz>
To: Jan Wieck <wieck@debis.com>
cc: t-ishii@sra.co.jp, tgl@sss.pgh.pa.us, pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] compression in LO and other fields
In-Reply-To: <m11m7SM-0003kLC@orion.SAPserv.Hamburg.dsh.de>
Message-ID: <Pine.LNX.3.96.991112101708.14930B-100000@ara.zf.jcu.cz>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Fri, 12 Nov 1999, Jan Wieck wrote:

Just in case someone want to implement a complete compressed
data type (including comarision functions, operators and
indexing default operator class).

I already made some tests with a type I called 'lztext'
locally. Only the input-/output-functions exist so far and
as the name might suggest, it would be an alternative for
'text'. It uses a simple but fast, byte oriented LZ backward
pointing method. No Huffman coding or variable offset/size
tagging. First byte of a chunk tells bitwise if the next
following 8 items are raw bytes to copy or 12 bit offset, 4
bit size copy information. That is max back offset 4096 and
max match size 17 bytes.

I is your original implementation or you use any current compression
code? I try bzip2, but output from this algorithm is total binary,
I don't know how this use in PgSQL if in backend are all routines
(in/out) use *char (yes, I'am newbie for PgSQL hacking:-).

What made it my preferred method was the fact, that
decompression is done entirely using the already decompressed
portion of the data, so it does not need any code tables or
the like at that time.

It is really FASTEST on decompression, which I assume would
be the mostly often used operation on huge data types. With
some care, comparision could be done on the fly while
decompressing two values, so that the entire comparision can
be aborted at the occurence of the first difference.

The compression rates aren't that giantic. I've got 30-50%

Not is problem, that your implementation compress all data at once?
Typically compression use a stream, and compress only small a buffer
in any cycle.

for rule plan strings (size limit on views!!!). And the
method used only allows for buffer back references of 4K
offsets at most, so the rate will not grow for larger data
chunks. That's a heavy tradeoff between compression rate and
no memory leakage for sure and speed, I know, but I prefer
not to force it, instead I usually use a bigger hammer (the
tuple size limit is still our original problem - and another
IBM 72GB disk doing 22-37 MB/s will make any compressing data
type obsolete then).

Sorry for the compression specific slang here. Well, anyone
interested in the code?

Yes, for me - I finish to_char()/to_data() ora compatible routines
(Thomas, you still quiet?) and this is new appeal for me :-)

Karel

From bouncefilter Fri Nov 12 04:49:57 1999
Received: from loopy.berkhirt.com (loopy.berkhirt.com [209.220.85.54])
by hub.org (8.9.3/8.9.3) with ESMTP id EAA16038
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 04:49:31 -0500 (EST)
(envelope-from bhirt@loopy.berkhirt.com)
Received: (from bhirt@localhost)
by loopy.berkhirt.com (8.9.3/8.8.7) id DAA21601;
Fri, 12 Nov 1999 03:49:01 -0600
Date: Fri, 12 Nov 1999 03:49:01 -0600
From: Brian Hirt <bhirt@mobygames.com>
To: pgsql-hackers@postgreSQL.org
Cc: bhirt@loopy.berkhirt.com
Subject: Re: [HACKERS] Slow - grindingly slow - query
Message-ID: <19991112034901.B21136@loopy.berkhirt.com>
Reply-To: bhirt@mobygames.com
References: <16514.942357396@sss.pgh.pa.us> <382BA0FB.82D96E3C@flame.co.za>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Mailer: Mutt 0.95.4us
In-Reply-To: <382BA0FB.82D96E3C@flame.co.za>;
from Theo Kramer on Fri, Nov 12, 1999 at 07:09:15AM +0200
X-PC-Gaming: http://www.mobygames.com/

select accountdetail.domain from accountdetail where
accountdetail.domain not in
(select accountmaster.domain from accountmaster);

This takes more than 5 hours and 30 minutes.

select accountdetail.domain from accountdetail where
not exists (select accountmaster.domain from accountmaster where
accountmaster.domain = accountdetail.domain);

This takes 5 seconds - wow!

I have a general comment/question here. Why do in/not in clauses seem
to perform so slowly? I've noticed this type of behavior with with my
system also. I think the above queries will always return the exact
same results regardless of the data. From looking at the query plan
with explain, it's clear the second query makes better use of the
indexes. Can't the rewrite engine recognize a simple case like the
one above and rewrite it to use exists and not exists with the proper
joins? Or possibly the optimizer can generate a better plan? Sometimes
it's not so easy to just change a query in the code. Sometimes you can't
change the code because you only have executables and sometimes you are
using a tool that automatically generates SQL using in clauses.
Additionally, since intersect and union get rewritten as in clauses they
suffer the same performance problems.

-brian

--
The world's most ambitious and comprehensive PC game database project.

http://www.mobygames.com

From bouncefilter Fri Nov 12 05:04:57 1999
Received: from Radha.DoCS.UU.SE (root@Radha.DoCS.UU.SE [130.238.9.99])
by hub.org (8.9.3/8.9.3) with SMTP id FAA19065
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 05:04:08 -0500 (EST)
(envelope-from e99re41@DoCS.UU.SE)
Received: from Gepard.DoCS.UU.SE (e99re41@Gepard.DoCS.UU.SE [130.238.9.100])
by Radha.DoCS.UU.SE (8.6.12/8.6.12) with ESMTP id LAA19610;
Fri, 12 Nov 1999 11:04:05 +0100
Received: from localhost (e99re41@localhost) by Gepard.DoCS.UU.SE
(8.6.12/8.6.12) with SMTP id LAA00806;
Fri, 12 Nov 1999 11:04:04 +0100
X-Authentication-Warning: Gepard.DoCS.UU.SE: e99re41 owned process doing -bs
Date: Fri, 12 Nov 1999 11:04:03 +0100 (MET)
From: Peter Eisentraut <e99re41@DoCS.UU.SE>
Reply-To: Peter Eisentraut <peter_e@gmx.net>
To: Bruce Momjian <maillist@candle.pha.pa.us>
cc: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Subject: Re: psql and \p\g
In-Reply-To: <199911112150.QAA16006@candle.pha.pa.us>
Message-ID: <Pine.GSO.4.02A.9911121103020.791-100000@Gepard.DoCS.UU.SE>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Thu, 11 Nov 1999, Bruce Momjian wrote:

Can you add a unix-style timestamp for \T?

Do you mean \echo `date` ?

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Fri Nov 12 05:11:59 1999
Received: from Radha.DoCS.UU.SE (root@Radha.DoCS.UU.SE [130.238.9.99])
by hub.org (8.9.3/8.9.3) with SMTP id FAA20748
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 05:11:21 -0500 (EST)
(envelope-from e99re41@DoCS.UU.SE)
Received: from Gepard.DoCS.UU.SE (e99re41@Gepard.DoCS.UU.SE [130.238.9.100])
by Radha.DoCS.UU.SE (8.6.12/8.6.12) with ESMTP id LAA20063;
Fri, 12 Nov 1999 11:11:19 +0100
Received: from localhost (e99re41@localhost) by Gepard.DoCS.UU.SE
(8.6.12/8.6.12) with SMTP id LAA00810;
Fri, 12 Nov 1999 11:11:18 +0100
X-Authentication-Warning: Gepard.DoCS.UU.SE: e99re41 owned process doing -bs
Date: Fri, 12 Nov 1999 11:11:17 +0100 (MET)
From: Peter Eisentraut <e99re41@DoCS.UU.SE>
Reply-To: Peter Eisentraut <peter_e@gmx.net>
To: Bruce Momjian <maillist@candle.pha.pa.us>
cc: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Subject: Re: failure of \e in psql
In-Reply-To: <199911120607.BAA01516@candle.pha.pa.us>
Message-ID: <Pine.GSO.4.02A.9911121104290.791-100000@Gepard.DoCS.UU.SE>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Fri, 12 Nov 1999, Bruce Momjian wrote:

Peter, before I go hunting around, can you tell me any other things psql
used to do that it doesn't do anymore?

Well, let's put it this way: Everythings that used to work, that people
found useful, and that doesn't work anymore is a bug. That's what it's all
about after all.

However: About the \e thing I simply didn't know. The \p\g was removed for
consistency. You might also be interested to know that \E no longer
exists, because I couldn't make sense of it. Also \d* is slated for
implementation but no one wanted to respond to my request to explain what
this is actually supposed to do. That's all I can come up with right now.

We had hand-tuned psql over the years, and it would be good to know what
features no longer exist so we can decide if they are needed.

Well, I really comes down to what Tom said, doesn't it: If the docs don't
match the code, the code it wrong. And it will get fixed. A lot of those
"tunings" seemed to be of the nature "If I put \o after \x I want it to do
<foo> instead".

That doesn't mean that they were bad of course, but the purpose of all of
this was to put a consistent face on things.

Having said that, if I mess it up I'll fix it of course.

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Fri Nov 12 06:17:00 1999
Received: from Radha.DoCS.UU.SE (root@Radha.DoCS.UU.SE [130.238.9.99])
by hub.org (8.9.3/8.9.3) with SMTP id GAA29615
for <hackers@postgreSQL.org>; Fri, 12 Nov 1999 06:16:28 -0500 (EST)
(envelope-from e99re41@DoCS.UU.SE)
Received: from Ko.DoCS.UU.SE (e99re41@Ko.DoCS.UU.SE [130.238.9.93]) by
Radha.DoCS.UU.SE (8.6.12/8.6.12) with ESMTP id MAA24083;
Fri, 12 Nov 1999 12:16:26 +0100
Received: from localhost (e99re41@localhost) by Ko.DoCS.UU.SE (8.6.12/8.6.12)
with SMTP id MAA10800; Fri, 12 Nov 1999 12:16:24 +0100
X-Authentication-Warning: Ko.DoCS.UU.SE: e99re41 owned process doing -bs
Date: Fri, 12 Nov 1999 12:16:23 +0100 (MET)
From: Peter Eisentraut <e99re41@DoCS.UU.SE>
Reply-To: Peter Eisentraut <peter_e@gmx.net>
To: Thomas Lockhart <lockhart@alumni.caltech.edu>
cc: Tom Lane <tgl@sss.pgh.pa.us>,
"'hackers@postgresql.org'" <hackers@postgreSQL.org>
Subject: Re: AW: [HACKERS] Re: [GENERAL] users in Postgresql
In-Reply-To: <382BC44B.2BE4AA39@alumni.caltech.edu>
Message-ID: <Pine.GSO.4.02A.9911121204060.10772-100000@Ko.DoCS.UU.SE>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Fri, 12 Nov 1999, Thomas Lockhart wrote:

Well, okay, everyone that wants to set their PostgreSQL user id
explicitly, send me a note and I'll put it back in, which ever way.

I thought that Tom Lane was representing me just fine, so was keeping
quiet ;)

Okay, vote noted.

An aside on procedures: on a change like this, I might have expected a
discussion on functionality *before* the patch was developed, since it
changes a seemingly fundamental feature. Though I haven't thought of a
strong, or even weak, argument for why id matching is necessary, it is
a topic about which there has been no discussion in the past, so I
didn't realize I needed an opinion until now.

It seems to me that especially in the code I'm digging around now, there
are a lot of way old things lying around (think Postgres95). When I ask if
someone actually uses them I usually get responses like "No, we can't yank
it, someone might be using it", which doesn't answer my question at all.

Thus I found it more effective to threaten removal first and then see if
someone speaks up.

Another aside: I'd like to think that most good ideas which stand the
test of an extended discussion will get a consensus to form. So if you
really think this is a step forward then keep talking about it; don't
give up too soon...

I just remember the heart-breaking discussion about the pg_ prefix ;)

Well, I outlined my points: 1) It confuses users, 2) It doesn't match the
SQL, 3) user IDs are internal representations that you should not be able
to mess with with user-level tools, 4) If you can pick it, you should also
be able to change it later. But you can't, really.

Back on topic: If there is currently no apparent need for a link
between Postgres user ids and external system ids, it is the case that
this is an obvious mechanism to make that link. So if someday a user
or a system feature needs it, it is already there and has been so from
day 1. afaik other DBs have a similar attribute for users.

This is based on the premise that it would somehow be useful to link Unix
and PostgreSQL users. In that case this would certainly be needed.

However, this would be a significant step backwards, since database users
are in general not equal to system users, most importantly since clients
might run on completely different systems than the server.

-Peter

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Fri Nov 12 07:01:58 1999
Received: from gandalf.telecom.at (gandalf.telecom.at [194.118.26.84])
by hub.org (8.9.3/8.9.3) with ESMTP id HAA35770
for <hackers@postgreSQL.org>; Fri, 12 Nov 1999 07:01:15 -0500 (EST)
(envelope-from ZeugswetterA@wien.spardat.at)
Received: from sdexcgtw01.f000.d0188.sd.spardat.at
(sdexcgtw01.f000.d0188.sd.spardat.at [172.18.1.16])
by gandalf.telecom.at (xxx/xxx) with ESMTP id NAA51944;
Fri, 12 Nov 1999 13:01:04 +0100
Received: by sdexcgtw01.f000.d0188.sd.spardat.at with Internet Mail Service
(5.5.2448.0) id <45YXP2X5>; Fri, 12 Nov 1999 13:01:04 +0100
Message-ID:
<219F68D65015D011A8E000006F8590C603FDC168@sdexcsrv1.f000.d0188.sd.spardat.at>
From: Zeugswetter Andreas SEV <ZeugswetterA@wien.spardat.at>
To: "'Peter Eisentraut'" <peter_e@gmx.net>
Cc: "'hackers@postgresql.org'" <hackers@postgreSQL.org>
Subject: AW: AW: [HACKERS] Re: [GENERAL] users in Postgresql
Date: Fri, 12 Nov 1999 13:01:03 +0100
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2448.0)
Content-Type: text/plain;
charset="iso-8859-1"

This is based on the premise that it would somehow be useful to link Unix
and PostgreSQL users.

This is useful, for a system like postgres, since due to the user types,
some user functions will eventually be executed with a setuid to a specific
unix user.
This may be the function owner, (dba procedure) or the user who is
connected.

In that case this would certainly be needed.

Yes imho.

However, this would be a significant step backwards, since
database users
are in general not equal to system users, most importantly
since clients
might run on completely different systems than the server.

Well , since I need all users as unix users, I do not want
postgres users at all. At the very least I do not want to keep
separate passwords for db users in the db.

Andreas

From bouncefilter Fri Nov 12 08:45:00 1999
Received: from ext04.sra.co.jp (IDENT:root@ykh28DS03.kng.mesh.ad.jp
[133.205.214.3]) by hub.org (8.9.3/8.9.3) with ESMTP id IAA46711
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 08:44:35 -0500 (EST)
(envelope-from t-ishii@ext04.sra.co.jp)
Received: from ext04.sra.co.jp (t-ishii@localhost [127.0.0.1])
by ext04.sra.co.jp (8.8.8/8.8.8) with ESMTP id WAA01906;
Fri, 12 Nov 1999 22:42:22 +0900
Message-Id: <199911121342.WAA01906@ext04.sra.co.jp>
To: Tom Lane <tgl@sss.pgh.pa.us>
cc: t-ishii@sra.co.jp, Karel Zak - Zakkr <zakkr@zf.jcu.cz>,
pgsql-hackers <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] compression in LO and other fields
From: Tatsuo Ishii <t-ishii@sra.co.jp>
Reply-To: t-ishii@sra.co.jp
In-reply-to: Your message of Fri, 12 Nov 1999 01:14:43 EST.
<25684.942387283@sss.pgh.pa.us>
Date: Fri, 12 Nov 1999 22:42:22 +0900
Sender: t-ishii@ext04.sra.co.jp

It sounds ideal but I remember that Vadim said inserting a 2GB record
is not good idea since it will be written into the log too. If it's a
necessary limitation from the point of view of WAL, we have to accept
it, I think.

LO won't make that any better: the data still goes into a table.
You'd have 2GB worth of WAL entries either way.

What in my mind was LO that is not under the transaction control. I
would not say this is a good thing, but I'm afraid we might need this
kind of beast in WAL.

The only thing LO would do for you is divide the data into block-sized
tuples, so there would be a bunch of little WAL entries instead of one
big one. But that'd probably be easy to duplicate too. If we implement
big tuples by chaining together disk-block-sized segments, which seems
like the most likely approach, couldn't WAL log each segment as a
separate log entry? If so, there's almost no difference between LO and
inline field for logging purposes.

Right.

BTW, does anybody know how BLOBs are handled by WAL in commercial
DBMSs?
---
Tatsuo Ishii

From bouncefilter Fri Nov 12 09:08:00 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id JAA49633
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 09:07:19 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgreSQL.org
id m11mHDk-0003kLC; Fri, 12 Nov 99 14:58 MET
Message-Id: <m11mHDk-0003kLC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] compression in LO and other fields
To: zakkr@zf.jcu.cz (Karel Zak - Zakkr)
Date: Fri, 12 Nov 1999 14:58:32 +0100 (MET)
Cc: wieck@debis.com, t-ishii@sra.co.jp, tgl@sss.pgh.pa.us,
pgsql-hackers@postgreSQL.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <Pine.LNX.3.96.991112101708.14930B-100000@ara.zf.jcu.cz> from
"Karel Zak - Zakkr" at Nov 12, 99 10:38:55 am
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Karel Zak - Zakkr wrote:

On Fri, 12 Nov 1999, Jan Wieck wrote:

I already made some tests with a type I called 'lztext'
locally. Only the input-/output-functions exist so far and

I is your original implementation or you use any current compression
code? I try bzip2, but output from this algorithm is total binary,
I don't know how this use in PgSQL if in backend are all routines
(in/out) use *char (yes, I'am newbie for PgSQL hacking:-).

The internal storage format is based on an article I found
at:

http://www.neutralzone.org/home/faqsys/docs/slz_art.txt

Simple Compression using an LZ buffer
Part 3 Revision 1.d:
An introduction to compression on the Amiga by Adisak Pochanayon

Freely Distributable as long as reproduced completely.
Copyright 1993 Adisak Pochanayon

I've written the code from scratch.

The internal representation is binary, for sure. It's a
PostgreSQL variable length data format as usual.

I don't know if there's a compression library available that
fit's our need. First and most important it must have a
license that permits us to include it in the distribution
under our existing license. Second it's implementation must
not cause any problems in the backend like memory leakage or
the like.

The compression rates aren't that giantic. I've got 30-50%

Not is problem, that your implementation compress all data at once?
Typically compression use a stream, and compress only small a buffer
in any cycle.

No, that's no problem. On type input, the original value is
completely in memory given as a char*, and the internal
representation is returned as a palloc()'d Datum. For output
it's vice versa.

O.K. some details on the compression rate. I've used 112
.html files with a total size of 1188346 bytes this time.
The smallest one was 131 bytes, the largest one 114549 bytes
and most of the files are somewhere between 3-12K.

Compression results on the binary level are:

gzip -9 outputs 398180 bytes (66.5% rate)

gzip -1 outputs 447597 bytes (62.3% rate)

my code outputs 529420 bytes (55.4% rate)

Html input might be somewhat optimal for Adisak's storage
format, but taking into account that my source implementing
the type input and output functions is smaller than 600
lines, I think 11% difference to a gzip -9 is a good result
anyway.

Sorry for the compression specific slang here. Well, anyone
interested in the code?

Yes, for me - I finish to_char()/to_data() ora compatible routines
(Thomas, you still quiet?) and this is new appeal for me :-)

Bruce suggested the contrib area, but I'm not sure if that's
the right place. If it goes into the distribution at all, I'd
like to use this data type for rule plan strings and function
source text in the system catalogs. I don't expect we'll have
a general solution for tuples split across multiple blocks
for v7.0. And using lztext for rules and function sources
would lower some FRP's. But using it in the catalogs requires
to be builtin.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Fri Nov 12 09:46:00 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id JAA54587
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 09:45:16 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id JAA26514;
Fri, 12 Nov 1999 09:44:03 -0500 (EST)
To: wieck@debis.com (Jan Wieck)
cc: zakkr@zf.jcu.cz (Karel Zak - Zakkr), t-ishii@sra.co.jp,
pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] compression in LO and other fields
In-reply-to: Your message of Fri, 12 Nov 1999 14:58:32 +0100 (MET)
<m11mHDk-0003kLC@orion.SAPserv.Hamburg.dsh.de>
Date: Fri, 12 Nov 1999 09:44:03 -0500
Message-ID: <26512.942417843@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

wieck@debis.com (Jan Wieck) writes:

Html input might be somewhat optimal for Adisak's storage
format, but taking into account that my source implementing
the type input and output functions is smaller than 600
lines, I think 11% difference to a gzip -9 is a good result
anyway.

These strike me as very good results. I'm not at all sure that using
gzip or bzip would give much better results in practice in Postgres,
because those compressors are optimized for relatively large files,
whereas a compressed-field datatype would likely be getting relatively
small field values to work on. (So your test data set is probably a
good one for our purposes --- do the numbers change if you exclude
all the files over, say, 10K?)

Bruce suggested the contrib area, but I'm not sure if that's
the right place. If it goes into the distribution at all, I'd
like to use this data type for rule plan strings and function
source text in the system catalogs.

Right, if we are going to bother with it at all, we should put it
into the core so that we can use it for rule plans.

I don't expect we'll have
a general solution for tuples split across multiple blocks
for v7.0.

I haven't given up hope of that yet --- but even if we do, compressing
the data is an attractive choice to reduce the frequency with which
tuples must be split across blocks.

It occurred to me last night that applying compression to individual
fields might not be the best approach. Certainly a "bytez" data type
is the easiest thing to fit into the existing system, but it's leaving
some space savings on the table. What about compressing the *whole*
data contents of a tuple on-disk, as a single entity? That should save
more space than field-by-field compression. It could be triggered in
the tuple storage routines whenever the uncompressed size exceeds some
threshold. (We'd need a flag in the tuple header to indicate compressed
data, but I think there are bits to spare.) When we get around to
having split tuples, the code would still be useful because it'd be
applied as a first resort before splitting a large tuple; it'd reduce
the frequency of splits and the number of sections big tuples get split
into. All automatic and transparent, too --- the user doesn't have to
change data declarations at all.

Also, if we do it that way, then it would *automatically* apply to
both regular tuples and LO, because the current LO implementation is
just tuples. (Tatsuo's idea of a non-transaction-controlled LO would
need extra work, of course, if we decide that's a good idea...)

regards, tom lane

From bouncefilter Fri Nov 12 09:51:00 1999
Received: from thelab.hub.org (nat198.58.mpoweredpc.net [142.177.198.58])
by hub.org (8.9.3/8.9.3) with ESMTP id JAA55114
for <pgsql-hackers@postgresql.org>;
Fri, 12 Nov 1999 09:50:02 -0500 (EST) (envelope-from scrappy@hub.org)
Received: from localhost (scrappy@localhost)
by thelab.hub.org (8.9.3/8.9.1) with ESMTP id KAA69715;
Fri, 12 Nov 1999 10:49:32 -0400 (AST) (envelope-from scrappy@hub.org)
X-Authentication-Warning: thelab.hub.org: scrappy owned process doing -bs
Date: Fri, 12 Nov 1999 10:49:31 -0400 (AST)
From: The Hermit Hacker <scrappy@hub.org>
To: Jan Wieck <wieck@debis.com>
cc: Karel Zak - Zakkr <zakkr@zf.jcu.cz>, t-ishii@sra.co.jp, tgl@sss.pgh.pa.us,
pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] compression in LO and other fields
In-Reply-To: <m11mHDk-0003kLC@orion.SAPserv.Hamburg.dsh.de>
Message-ID: <Pine.BSF.4.10.9911121047360.2296-100000@thelab.hub.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Fri, 12 Nov 1999, Jan Wieck wrote:

I don't know if there's a compression library available that
fit's our need. First and most important it must have a
license that permits us to include it in the distribution
under our existing license. Second it's implementation must
not cause any problems in the backend like memory leakage or
the like.

Is this something that could be a configure option? Put the stubs in
place, and if someone wants to enable that feature, they can install the
compression library first and run with it?

Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

From bouncefilter Fri Nov 12 09:59:00 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id JAA56512
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 09:58:38 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgreSQL.org
id m11mI1a-0003kLC; Fri, 12 Nov 99 15:50 MET
Message-Id: <m11mI1a-0003kLC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] compression in LO and other fields
To: tgl@sss.pgh.pa.us (Tom Lane)
Date: Fri, 12 Nov 1999 15:50:02 +0100 (MET)
Cc: wieck@debis.com, zakkr@zf.jcu.cz, t-ishii@sra.co.jp,
pgsql-hackers@postgreSQL.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <26512.942417843@sss.pgh.pa.us> from "Tom Lane" at Nov 12,
99 09:44:03 am
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Tom Lane wrote:

wieck@debis.com (Jan Wieck) writes:

Html input might be somewhat optimal for Adisak's storage
format, but taking into account that my source implementing
the type input and output functions is smaller than 600
lines, I think 11% difference to a gzip -9 is a good result
anyway.

These strike me as very good results. I'm not at all sure that using
gzip or bzip would give much better results in practice in Postgres,
because those compressors are optimized for relatively large files,
whereas a compressed-field datatype would likely be getting relatively
small field values to work on. (So your test data set is probably a
good one for our purposes --- do the numbers change if you exclude
all the files over, say, 10K?)

Will give it a try.

It occurred to me last night that applying compression to individual
fields might not be the best approach. Certainly a "bytez" data type
is the easiest thing to fit into the existing system, but it's leaving
some space savings on the table. What about compressing the *whole*
data contents of a tuple on-disk, as a single entity? That should save
more space than field-by-field compression.

But it requires decompression of every tuple into palloc()'d
memory during heap access. AFAIK, the heap access routines
currently return a pointer to the tuple inside the shm
buffer. Don't know what it's performance impact would be.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Fri Nov 12 09:46:02 1999
Received: from localhost (IDENT:root@hectic-3.jpl.nasa.gov [128.149.68.205])
by hub.org (8.9.3/8.9.3) with ESMTP id JAA54608
for <hackers@postgresql.org>; Fri, 12 Nov 1999 09:45:52 -0500 (EST)
(envelope-from lockhart@alumni.caltech.edu)
Received: from alumni.caltech.edu (lockhart@localhost [127.0.0.1])
by localhost (8.8.7/8.8.7) with ESMTP id OAA08796;
Fri, 12 Nov 1999 14:52:58 GMT
Sender: lockhart@hub.org
Message-ID: <382C29CA.8EE3466F@alumni.caltech.edu>
Date: Fri, 12 Nov 1999 14:52:58 +0000
From: Thomas Lockhart <lockhart@alumni.caltech.edu>
X-Mailer: Mozilla 4.6 [en] (X11; I; Linux 2.0.36 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Karel Zak - Zakkr <zakkr@zf.jcu.cz>
CC: Postgres Hackers List <hackers@postgresql.org>
Subject: Re: internationalizing and etc..
References: <Pine.LNX.3.96.991110162143.7055A-100000@ara.zf.jcu.cz>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

I send you my last letter again, because I'am not sure if you obtain it
(you had problem with connection (?)).

Actually yes! I was off the air for a week or so, and didn't remember
that I had a mail to answer. Sorry about that...

For normal strings, we can implement NATIONAL CHARACTER and CHARACTER
SET features from SQL92 to handle locales and alternative languages.
We'll do this by defining new types for each language or locale. For
date/time, a clear path is to standardize on ISO-8601 formats (already
available as an option) which use numeric fields only. SQL92 offers no
suggestions on how to do date/time types with alphabetic fields.

I'm going to switch the default date format to ISO-8601 for the next
release, which should help. We could also think about
internationalizing the date/time support as you suggest, with external
language-specific catalogs, but a catalog lookup to do date/time i/o
would seem to be very slow, for a catalog within Postgres or for an
external flat file.

Yes, external catalogs is problem (speed, needs glibc..), better resolution is
probably (cached) system catalogs. If I good underatand you, your idea is
add langs and locales to pg_type (..example), well. After this si not problem
make internal catalogs for locales with months, days names...etc. And join
this locales table (pg_locale?) with pg_type via oid and we can implement
any translator between langs (for datetime strings).

(This is better, because in the glibc's catalogs has features which we
needn't in PgSQL.)

Will feature which allow make possible to set LOCALE type during transaction,
SET LOCALE/NATIONAL command ?

If you have any exactly ideas I can help you with it (if you want).
I have a little time now and I want spend it with PgSQL :-)

My thought for the *first* implementation of locales and character
sets is to do it all through pg_type, pg_proc, and pg_operator, with a
little help from built-in features in the parser to recognize the
SQL92 conventions for character sets.

For the moment then, we focus on the character set features, not on
how to tie this into the date/time features. For SQL92, you can't
change character sets on the fly, though you can specify that they be
converted on the fly. And the same would hold true of the date/time
types. You can't do a "SET LOCALE" and magically find that both
character types and date/time types change locale too. Or I should say
I haven't seen how to do that yet.

I complete to_char/to_date, and I testing it now. Where/Who I must send this
routines for moving to contrib (to You or to other major developer)?

Send it to the "patches" list, or directly to the "hackers" list if it
is under 40kbytes. After we look at the code, we will decide if it
goes into contrib or directly into the backend, and will put it into
the tree.

- Thomas

--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California

From bouncefilter Fri Nov 12 09:48:00 1999
Received: from localhost (IDENT:root@hectic-3.jpl.nasa.gov [128.149.68.205])
by hub.org (8.9.3/8.9.3) with ESMTP id JAA54723
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 09:47:11 -0500 (EST)
(envelope-from lockhart@alumni.caltech.edu)
Received: from alumni.caltech.edu (lockhart@localhost [127.0.0.1])
by localhost (8.8.7/8.8.7) with ESMTP id OAA08803;
Fri, 12 Nov 1999 14:54:04 GMT
Sender: lockhart@hub.org
Message-ID: <382C2A0C.39D59678@alumni.caltech.edu>
Date: Fri, 12 Nov 1999 14:54:04 +0000
From: Thomas Lockhart <lockhart@alumni.caltech.edu>
X-Mailer: Mozilla 4.6 [en] (X11; I; Linux 2.0.36 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Karel Zak - Zakkr <zakkr@zf.jcu.cz>
CC: Jan Wieck <wieck@debis.com>, t-ishii@sra.co.jp, tgl@sss.pgh.pa.us,
pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] compression in LO and other fields
References: <Pine.LNX.3.96.991112101708.14930B-100000@ara.zf.jcu.cz>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Yes, for me - I finish to_char()/to_data() ora compatible routines
(Thomas, you still quiet?) and this is new appeal for me :-)

Ack! When I saw this I rolled up my primary Netscape window and found
you're almost-completed reply underneath. I've sent it along now.

- Thomas

--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California

From bouncefilter Fri Nov 12 10:06:06 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id KAA58257
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 10:05:43 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgreSQL.org
id m11mI8O-0003kLC; Fri, 12 Nov 99 15:57 MET
Message-Id: <m11mI8O-0003kLC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] compression in LO and other fields
To: scrappy@hub.org (The Hermit Hacker)
Date: Fri, 12 Nov 1999 15:57:04 +0100 (MET)
Cc: wieck@debis.com, zakkr@zf.jcu.cz, t-ishii@sra.co.jp, tgl@sss.pgh.pa.us,
pgsql-hackers@postgreSQL.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <Pine.BSF.4.10.9911121047360.2296-100000@thelab.hub.org> from
"The Hermit Hacker" at Nov 12, 99 10:49:31 am
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Marc G. Fournier wrote:

On Fri, 12 Nov 1999, Jan Wieck wrote:

I don't know if there's a compression library available that
fit's our need. First and most important it must have a
license that permits us to include it in the distribution
under our existing license. Second it's implementation must
not cause any problems in the backend like memory leakage or
the like.

Is this something that could be a configure option? Put the stubs in
place, and if someone wants to enable that feature, they can install the
compression library first and run with it?

If using the new type in system catalogs, the option could
only be what kind of compression to use. And we need our own
default compression code shipped anyway.

Of course, it could depend on the config what types are used
in the syscat. But making the catalog headers things that are
shipped as a .in isn't really that good IMHO.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Fri Nov 12 10:00:00 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id JAA56607
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 09:59:19 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id JAA26651;
Fri, 12 Nov 1999 09:58:14 -0500 (EST)
To: bhirt@mobygames.com
cc: pgsql-hackers@postgreSQL.org, bhirt@loopy.berkhirt.com
Subject: Re: [HACKERS] Slow - grindingly slow - query
In-reply-to: Your message of Fri, 12 Nov 1999 03:49:01 -0600
<19991112034901.B21136@loopy.berkhirt.com>
Date: Fri, 12 Nov 1999 09:58:14 -0500
Message-ID: <26649.942418694@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Brian Hirt <bhirt@mobygames.com> writes:

Can't the rewrite engine recognize a simple case like the
one above and rewrite it to use exists and not exists with the proper
joins? Or possibly the optimizer can generate a better plan?

This is on the TODO list, and will get done someday. IMHO it's not as
urgent as a lot of the planner/optimizer's other shortcomings, because
it can usually be worked around by revising the query.

If it's bugging you enough to go fix it now, contributions are always
welcome ;-)

regards, tom lane

From bouncefilter Fri Nov 12 19:48:08 1999
Received: from merganser.its.uu.se (merganser.its.uu.se [130.238.6.236])
by hub.org (8.9.3/8.9.3) with ESMTP id TAA49183
for <hackers@postgresql.org>; Fri, 12 Nov 1999 19:47:31 -0500 (EST)
(envelope-from peter@peter-e.yi.org)
Received: from regulus.its.uu.se ([130.238.7.19]:61852 "EHLO peter-e.yi.org")
by merganser.its.uu.se with ESMTP id <S.s99I.168312>;
Sat, 13 Nov 1999 01:46:56 +0100
Received: from peter (helo=localhost)
by peter-e.yi.org with local-esmtp (Exim 3.02 #2) id 11mIDt-0000py-00
for hackers@postgresql.org; Fri, 12 Nov 1999 16:02:45 +0100
Date: Fri, 12 Nov 1999 16:02:45 +0100 (CET)
From: Peter Eisentraut <peter_e@gmx.net>
To: hackers@postgresql.org
Subject: RFC: create/alter user extension
Message-ID: <Pine.LNX.4.20.9911121555290.1261-100000@peter-e.yi.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: Peter Eisentraut <peter@peter-e.yi.org>

Is everyone okay with the following syntax:

CREATE USER username
[ WITH ID digits ]
^^^^^^^^^^^^^^^^^^
[ WITH PASSWORD password ]
[ CREATEDB | NOCREATEDB ]
[ CREATEUSER | NOCREATEUSER ]
[ IN GROUP groupname [, ...] ]
[ VALID UNTIL 'abstime' ]

ALTER USER username
[ WITH ID digits ]
^^^^^^^^^^^^^^^^^^
[ WITH PASSWORD password ]
[ CREATEDB | NOCREATEDB ]
[ CREATEUSER | NOCREATEUSER ]
[ IN GROUP groupname [, ...] ]
[ VALID UNTIL 'abstime' ]

The catch is that ID would have to be a new keyword and we'd have to live
with that for a long time. Other choices include:
* UID
* SYSID
* USESYSID
etc.

What do the standards and pseudo-standards say?

I think I'll take a stab at this and settle the createuser script issue
the proper way.

-Peter

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Fri Nov 12 10:16:05 1999
Received: from thelab.hub.org (nat198.58.mpoweredpc.net [142.177.198.58])
by hub.org (8.9.3/8.9.3) with ESMTP id KAA60391
for <pgsql-hackers@postgresql.org>;
Fri, 12 Nov 1999 10:15:29 -0500 (EST) (envelope-from scrappy@hub.org)
Received: from localhost (scrappy@localhost)
by thelab.hub.org (8.9.3/8.9.1) with ESMTP id LAA69953
for <pgsql-hackers@postgresql.org>;
Fri, 12 Nov 1999 11:15:27 -0400 (AST) (envelope-from scrappy@hub.org)
X-Authentication-Warning: thelab.hub.org: scrappy owned process doing -bs
X-Received: from localhost (localhost.hub.org [127.0.0.1])
by thelab.hub.org (8.9.3/8.9.1) with ESMTP id LAA69918
for <scrappy@localhost>; Fri, 12 Nov 1999 11:11:35 -0400 (AST)
(envelope-from pjw@rhyme.com.au)
X-Received: from mail.hub.org by localhost with IMAP (fetchmail-5.0.5)
for scrappy@localhost (single-drop);
Fri, 12 Nov 1999 11:11:37 -0400 (AST)
X-Received: (from majordom@localhost) by hub.org (8.9.3/8.9.3) id KAA58731
for pgsql-hackers-team; Fri, 12 Nov 1999 10:08:01 -0500 (EST)
(envelope-from owner-majordomo)
X-Received: from acheron.rime.com.au (root@albatr.lnk.telstra.net
[139.130.54.222]) by hub.org (8.9.3/8.9.3) with ESMTP id KAA58637
for <owner-pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 10:07:26 -0500 (EST)
(envelope-from pjw@rhyme.com.au)
X-Received: from oberon (Oberon.rime.com.au [203.8.195.100])
by acheron.rime.com.au (8.9.3/8.9.3) with SMTP id CAA30915;
Sat, 13 Nov 1999 02:06:51 +1100
Message-Id: <3.0.5.32.19991113020723.00c7f9a0@mail.rhyme.com.au>
X-Sender: pjw@mail.rhyme.com.au
X-Mailer: QUALCOMM Windows Eudora Pro Version 3.0.5 (32)
Date: Sat, 13 Nov 1999 02:07:23 +1100
To: t-ishii@sra.co.jp
From: Philip Warner <pjw@rhyme.com.au>
Subject: Re: [HACKERS] compression in LO and other fields
Cc: owner-pgsql-hackers@postgresql.org
In-Reply-To: <199911121342.WAA01906@ext04.sra.co.jp>
References: <Your message of Fri,
12 Nov 1999 01:14:43 EST. <25684.942387283@sss.pgh.pa.us>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
ReSent-Date: Fri, 12 Nov 1999 11:15:22 -0400 (AST)
ReSent-From: The Hermit Hacker <scrappy@hub.org>
ReSent-To: pgsql-hackers@postgresql.org
ReSent-Subject: Re: [HACKERS] compression in LO and other fields
ReSent-Message-ID: <Pine.BSF.4.10.9911121115220.2296@thelab.hub.org>

BTW, does anybody know how BLOBs are handled by WAL in commercial
DBMSs?

Dec/RDB stores them in it's equivalent of the WAL; full rollback etc is
supported. If you load lots of large blobs, you need to make sure you have
enough disk space for the journal copies.

----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.C.N. 008 659 498) | /(@) ______---_
Tel: +61-03-5367 7422 | _________ \
Fax: +61-03-5367 7430 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/

From bouncefilter Fri Nov 12 10:14:01 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id KAA59770
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 10:13:27 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id KAA26762;
Fri, 12 Nov 1999 10:12:16 -0500 (EST)
To: wieck@debis.com (Jan Wieck)
cc: zakkr@zf.jcu.cz, t-ishii@sra.co.jp, pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] compression in LO and other fields
In-reply-to: Your message of Fri, 12 Nov 1999 15:50:02 +0100 (MET)
<m11mI1a-0003kLC@orion.SAPserv.Hamburg.dsh.de>
Date: Fri, 12 Nov 1999 10:12:15 -0500
Message-ID: <26760.942419535@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

wieck@debis.com (Jan Wieck) writes:

Tom Lane wrote:

It occurred to me last night that applying compression to individual
fields might not be the best approach. Certainly a "bytez" data type
is the easiest thing to fit into the existing system, but it's leaving
some space savings on the table. What about compressing the *whole*
data contents of a tuple on-disk, as a single entity? That should save
more space than field-by-field compression.

But it requires decompression of every tuple into palloc()'d
memory during heap access. AFAIK, the heap access routines
currently return a pointer to the tuple inside the shm
buffer. Don't know what it's performance impact would be.

Good point, but the same will be needed when a tuple is split across
multiple blocks. I would expect that (given a reasonably fast
decompressor) there will be a net performance *gain* due to having
less disk I/O to do. Also, this won't be happening for "every" tuple,
just those exceeding a size threshold --- we'd be able to tune the
threshold value to trade off speed and space.

One thing that does occur to me is that we need to store the
uncompressed as well as the compressed data size, so that the
working space can be palloc'd before starting the decompression.

Also, in case it wasn't clear, I was envisioning leaving the tuple
header uncompressed, so that time quals etc can be checked before
decompressing the tuple data.

regards, tom lane

From bouncefilter Fri Nov 12 10:32:01 1999
Received: from ara.zf.jcu.cz (zakkr@ara.zf.jcu.cz [160.217.161.4])
by hub.org (8.9.3/8.9.3) with ESMTP id KAA64056
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 10:31:35 -0500 (EST) (envelope-from zakkr@zf.jcu.cz)
Received: from localhost (zakkr@localhost)
by ara.zf.jcu.cz (8.9.3/8.9.3/Debian/GNU) with SMTP id QAA30515;
Fri, 12 Nov 1999 16:18:30 +0100
Date: Fri, 12 Nov 1999 16:18:30 +0100 (CET)
From: Karel Zak - Zakkr <zakkr@zf.jcu.cz>
To: Jan Wieck <wieck@debis.com>
cc: pgsql-hackers <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] compression in LO and other fields
In-Reply-To: <m11mHDk-0003kLC@orion.SAPserv.Hamburg.dsh.de>
Message-ID: <Pine.LNX.3.96.991112160056.30102A-100000@ara.zf.jcu.cz>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Fri, 12 Nov 1999, Jan Wieck wrote:

I don't know if there's a compression library available that
fit's our need. First and most important it must have a
license that permits us to include it in the distribution

IMHO bzip2 compression algorith is free and open source - README
from bzip2 source:

"bzip2-0.9.5 is distributed under a BSD-style license. For details,
see the file LICENSE"

Bruce suggested the contrib area, but I'm not sure if that's
the right place. If it goes into the distribution at all, I'd

Is any space (on postgresql ftp?) for this unstable code? Good project
has incoming of ftp for devel. versions...

If you this code not move to contrib, send me it (patch?), please.

Karel

From bouncefilter Fri Nov 12 10:36:01 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id KAA64880
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 10:35:49 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id KAA26966;
Fri, 12 Nov 1999 10:34:46 -0500 (EST)
To: Frans Van Elsacker <fve@atbib.be>
cc: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] union problem version 6.5.3
In-reply-to: Your message of Fri, 12 Nov 1999 01:10:06 +0100
<3.0.6.32.19991112011006.00876c00@193.75.233.1>
Date: Fri, 12 Nov 1999 10:34:45 -0500
Message-ID: <26964.942420885@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Frans Van Elsacker <fve@atbib.be> writes:

I had a little piece of code who works fine up till know, just after
installing
the version 6.5.3 i got the message 'parse error near union'

create table work as
select * from opdracht
union
select * from opdrachtproost

Hmm, the grammar has

CreateAsStmt: CREATE OptTemp TABLE relation_name OptCreateAs AS SubSelect

and SubSelect doesn't allow unions. This is overly restrictive,
I agree, but I thought it had been that way for a good while.
What version were you using before?

regards, tom lane

From bouncefilter Fri Nov 12 10:50:01 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id KAA67355
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 10:49:46 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgreSQL.org
id m11mIp4-0003kLC; Fri, 12 Nov 99 16:41 MET
Message-Id: <m11mIp4-0003kLC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] compression in LO and other fields
To: tgl@sss.pgh.pa.us (Tom Lane)
Date: Fri, 12 Nov 1999 16:41:10 +0100 (MET)
Cc: wieck@debis.com, zakkr@zf.jcu.cz, t-ishii@sra.co.jp,
pgsql-hackers@postgreSQL.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <26760.942419535@sss.pgh.pa.us> from "Tom Lane" at Nov 12,
99 10:12:15 am
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Tom Lane wrote:

wieck@debis.com (Jan Wieck) writes:

But it requires decompression of every tuple into palloc()'d
memory during heap access. AFAIK, the heap access routines
currently return a pointer to the tuple inside the shm
buffer. Don't know what it's performance impact would be.

Good point, but the same will be needed when a tuple is split across
multiple blocks. I would expect that (given a reasonably fast
decompressor) there will be a net performance *gain* due to having
less disk I/O to do. Also, this won't be happening for "every" tuple,
just those exceeding a size threshold --- we'd be able to tune the
threshold value to trade off speed and space.

Right, this time it's your good point. All of the problems
will be there on tuple split implementation.

The major problem I see is that a palloc()'d tuple should be
pfree()'d after the fetcher is done with it. Since they are
in buffer actually, the fetcher doesn't have to care.

One thing that does occur to me is that we need to store the
uncompressed as well as the compressed data size, so that the
working space can be palloc'd before starting the decompression.

Yepp - and I'm doing so. Only during compression the result
size isn't known. But there is a well known maximum, that is
the header overhead plus the data size by 1.125 plus 2 bytes
(totally worst case on uncompressable data). And a general
mechanism working on the tuple level would fallback to store
uncompressed data in the case the compressed size is bigger.

Also, in case it wasn't clear, I was envisioning leaving the tuple
header uncompressed, so that time quals etc can be checked before
decompressing the tuple data.

Of course.

Well, you asked for the rates on the smaller html files only.
78 files, 131 bytes min, 10000 bytes max, 4582 bytes avg,
357383 bytes total.

gzip -9 outputs 145659 bytes (59.2%)
gzip -1 outputs 155113 bytes (56.6%)
my code outputs 184109 bytes (48.5%)

67 files, 2000 bytes min, 10000 bytes max, 5239 bytes avg,
351006 bytes total.

gzip -9 outputs 141772 bytes (59.6%)
gzip -1 outputs 151150 bytes (56.9%)
my code outputs 179428 bytes (48.9%)

The threshold will surely be a tuning parameter of interest.
Another tuning option must be to allow/deny compression per
table at all. Then we could have both options, using a
compressing field type to define which portion of a tuple to
compress, or allow to compress the entire tuples.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Fri Nov 12 11:01:03 1999
Received: from wallace.ece.rice.edu (root@wallace.ece.rice.edu
[128.42.12.154])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA70602
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 11:00:06 -0500 (EST)
(envelope-from reedstrm@wallace.ece.rice.edu)
Received: by wallace.ece.rice.edu via sendmail from stdin
id <m11mJ7D-000LECC@wallace.ece.rice.edu> (Debian Smail3.2.0.102)
for pgsql-hackers@postgreSQL.org; Fri, 12 Nov 1999 09:59:55 -0600 (CST)
Date: Fri, 12 Nov 1999 09:59:55 -0600
From: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
To: Karel Zak - Zakkr <zakkr@zf.jcu.cz>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, t-ishii@sra.co.jp,
pgsql-hackers <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] compression in LO and other fields
Message-ID: <19991112095955.D9755@wallace.ece.rice.edu>
References: <25684.942387283@sss.pgh.pa.us>
<Pine.LNX.3.96.991112094645.14930A-100000@ara.zf.jcu.cz>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Mailer: Mutt 0.95.3i
In-Reply-To: <Pine.LNX.3.96.991112094645.14930A-100000@ara.zf.jcu.cz>;
from Karel Zak - Zakkr on Fri, Nov 12, 1999 at 10:16:21AM +0100

On Fri, Nov 12, 1999 at 10:16:21AM +0100, Karel Zak - Zakkr wrote:

Other eventual compression questions:

* MySQL dump allow make compressed dump file, it is good, and PostgreSQL?

pg_dump database | gzip >db.out.gz

Ross
--
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu>
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St., Houston, TX 77005

From bouncefilter Fri Nov 12 11:32:02 1999
Received: from ara.zf.jcu.cz (zakkr@ara.zf.jcu.cz [160.217.161.4])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA76392
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 11:31:50 -0500 (EST) (envelope-from zakkr@zf.jcu.cz)
Received: from localhost (zakkr@localhost)
by ara.zf.jcu.cz (8.9.3/8.9.3/Debian/GNU) with SMTP id RAA31638;
Fri, 12 Nov 1999 17:15:06 +0100
Date: Fri, 12 Nov 1999 17:15:05 +0100 (CET)
From: Karel Zak - Zakkr <zakkr@zf.jcu.cz>
To: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
cc: Tom Lane <tgl@sss.pgh.pa.us>, t-ishii@sra.co.jp,
pgsql-hackers <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] compression in LO and other fields
In-Reply-To: <19991112095955.D9755@wallace.ece.rice.edu>
Message-ID: <Pine.LNX.3.96.991112171242.31595A-100000@ara.zf.jcu.cz>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Fri, 12 Nov 1999, Ross J. Reedstrom wrote:

On Fri, Nov 12, 1999 at 10:16:21AM +0100, Karel Zak - Zakkr wrote:

Other eventual compression questions:

* MySQL dump allow make compressed dump file, it is good, and PostgreSQL?

pg_dump database | gzip >db.out.gz

Thank... :-))

But mysqldump --compress is very nice.

Karel

From bouncefilter Fri Nov 12 11:22:02 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA74392
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 11:21:12 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id LAA27124;
Fri, 12 Nov 1999 11:19:58 -0500 (EST)
To: wieck@debis.com (Jan Wieck)
cc: zakkr@zf.jcu.cz, t-ishii@sra.co.jp, pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] compression in LO and other fields
In-reply-to: Your message of Fri, 12 Nov 1999 16:41:10 +0100 (MET)
<m11mIp4-0003kLC@orion.SAPserv.Hamburg.dsh.de>
Date: Fri, 12 Nov 1999 11:19:57 -0500
Message-ID: <27122.942423597@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

wieck@debis.com (Jan Wieck) writes:

The major problem I see is that a palloc()'d tuple should be
pfree()'d after the fetcher is done with it. Since they are
in buffer actually, the fetcher doesn't have to care.

I think this may not be as big a problem as it looks. Most places
in the executor keep tuples in TupleTableSlots, which are responsible
for pfree'ing the tuple if (and only if) necessary; all that code is
ready for this change already. There are probably some routines in
heapam/indexam that assume they only work with tuples that never need
to be freed, but I don't think the fixes will be pervasive. And we're
going to have to do that work in any case to support big tuples
(assuming we do it by splitting tuples into segments that fit in disk
pages).

And a general
mechanism working on the tuple level would fallback to store
uncompressed data in the case the compressed size is bigger.

Right. Another possible place for speed-vs-space tuning would be to
store the uncompressed representation unless the compressed version is
at least X percent smaller, not just at-least-one-byte smaller.

regards, tom lane

From bouncefilter Fri Nov 12 11:52:03 1999
Received: from candle.pha.pa.us (maillist@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA80011
for <hackers@postgreSQL.org>; Fri, 12 Nov 1999 11:51:37 -0500 (EST)
(envelope-from maillist@candle.pha.pa.us)
Received: (from maillist@localhost)
by candle.pha.pa.us (8.9.0/8.9.0) id LAA15737;
Fri, 12 Nov 1999 11:50:06 -0500 (EST)
From: Bruce Momjian <maillist@candle.pha.pa.us>
Message-Id: <199911121650.LAA15737@candle.pha.pa.us>
Subject: Re: [HACKERS] Re: AWL: Re: tm1
In-Reply-To: <382BB99C.7B8B94E0@alumni.caltech.edu> from Thomas Lockhart at
"Nov 12, 1999 06:54:20 am"
To: Thomas Lockhart <lockhart@alumni.caltech.edu>
Date: Fri, 12 Nov 1999 11:50:06 -0500 (EST)
CC: applixware-list@applix.com, Postgres Hackers List <hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

People may have problems with the NULL statements with some versions
of PostgreSQL. I have information about editing the applix macro
on that creates the tables my web site:
http://www.radix.net/~cobrien/applix/applix.txt

Just in case someone cares ;)

The "NULL" constraint for a column definition is not defined in SQL92,
and is not necessary and could be dropped from Applix's definition of
the table. The default behavior of any column defined in SQL is to
allow NULL values.

Postgres does not implement this redundant syntax extension because
yacc-style parsers such as the one used in Postgres find the use of
the bare NULL an ambiguous context. Presumably that is why SQL92 does
not define it.

However, I see that in a limited context, such as a bare NULL with no
other qualifiers, yacc can handle its use. I'll add it to Postgres'
next release...

Yes, we are hearing people use it. Seems like we could just ignore the
NULL if possible.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Fri Nov 12 11:57:02 1999
Received: from candle.pha.pa.us (maillist@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA80703
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 11:55:53 -0500 (EST)
(envelope-from maillist@candle.pha.pa.us)
Received: (from maillist@localhost)
by candle.pha.pa.us (8.9.0/8.9.0) id LAA15841;
Fri, 12 Nov 1999 11:54:44 -0500 (EST)
From: Bruce Momjian <maillist@candle.pha.pa.us>
Message-Id: <199911121654.LAA15841@candle.pha.pa.us>
Subject: Re: psql and \p\g
In-Reply-To: <Pine.GSO.4.02A.9911121103020.791-100000@Gepard.DoCS.UU.SE> from
Peter Eisentraut at "Nov 12, 1999 11:04:03 am"
To: Peter Eisentraut <peter_e@gmx.net>
Date: Fri, 12 Nov 1999 11:54:44 -0500 (EST)
CC: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Thu, 11 Nov 1999, Bruce Momjian wrote:

Can you add a unix-style timestamp for \T?

Do you mean \echo `date` ?

Oh, very nifty. Never mind. I didn't see that.

Seems you have added more powerful flags to take over some of the old
flag usage. If you want to remove some of the older psql flags and
require them to use your newer syntax that allows more functionality,
you can do it.

If you want, just print an error message for the old flag showing them
the new syntax to use and we can remove the messages after a few
releases.

However, some of the more popular flags should probably be left in.
Maybe it should just be left alone. Not sure.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Fri Nov 12 12:02:06 1999
Received: from candle.pha.pa.us (maillist@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id MAA81679
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 12:01:22 -0500 (EST)
(envelope-from maillist@candle.pha.pa.us)
Received: (from maillist@localhost)
by candle.pha.pa.us (8.9.0/8.9.0) id MAA16016;
Fri, 12 Nov 1999 12:00:27 -0500 (EST)
From: Bruce Momjian <maillist@candle.pha.pa.us>
Message-Id: <199911121700.MAA16016@candle.pha.pa.us>
Subject: Re: failure of \e in psql
In-Reply-To: <Pine.GSO.4.02A.9911121104290.791-100000@Gepard.DoCS.UU.SE> from
Peter Eisentraut at "Nov 12, 1999 11:11:17 am"
To: Peter Eisentraut <peter_e@gmx.net>
Date: Fri, 12 Nov 1999 12:00:27 -0500 (EST)
CC: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Fri, 12 Nov 1999, Bruce Momjian wrote:

Peter, before I go hunting around, can you tell me any other things psql
used to do that it doesn't do anymore?

Well, let's put it this way: Everythings that used to work, that people
found useful, and that doesn't work anymore is a bug. That's what it's all
about after all.

However: About the \e thing I simply didn't know. The \p\g was removed for
consistency. You might also be interested to know that \E no longer
exists, because I couldn't make sense of it. Also \d* is slated for
implementation but no one wanted to respond to my request to explain what
this is actually supposed to do. That's all I can come up with right now.

First, let me say I am very glad you overhauled psql. It was very
needed, and I like the new functionality. Already learned \echo `date`.
Quite handy and very flexible.

I was just curious if there was any stuff you found confusing and
skipped so we could comment on it all at once.

We have fixed the pager off by default, and looks like \p\g and \e need
work, but that is small compared to how much functionality does work
perfectly. I personally found the \e and \p\g stuff very tricky to
implement.

[Of course, with the new output, I am going to have to re-do every one
of my SQL query outputs for the book. :-) ]

No idea what \d* does, nor \E.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Fri Nov 12 12:15:03 1999
Received: from candle.pha.pa.us (maillist@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id MAA83639
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 12:14:08 -0500 (EST)
(envelope-from maillist@candle.pha.pa.us)
Received: (from maillist@localhost)
by candle.pha.pa.us (8.9.0/8.9.0) id MAA16458;
Fri, 12 Nov 1999 12:10:35 -0500 (EST)
From: Bruce Momjian <maillist@candle.pha.pa.us>
Message-Id: <199911121710.MAA16458@candle.pha.pa.us>
Subject: Re: [HACKERS] compression in LO and other fields
In-Reply-To: <26512.942417843@sss.pgh.pa.us> from Tom Lane at "Nov 12,
1999 09:44:03 am"
To: Tom Lane <tgl@sss.pgh.pa.us>
Date: Fri, 12 Nov 1999 12:10:35 -0500 (EST)
CC: Jan Wieck <wieck@debis.com>, Karel Zak - Zakkr <zakkr@zf.jcu.cz>,
t-ishii@sra.co.jp, pgsql-hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Bruce suggested the contrib area, but I'm not sure if that's
the right place. If it goes into the distribution at all, I'd
like to use this data type for rule plan strings and function
source text in the system catalogs.

Right, if we are going to bother with it at all, we should put it
into the core so that we can use it for rule plans.

Agreed. I suggested contrib so the code doesn't just disappear.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Fri Nov 12 12:35:04 1999
Received: (from news@localhost) by hub.org (8.9.3/8.9.3) id MAA86879
for pgsql-hackers@postgresql.org; Fri, 12 Nov 1999 12:34:44 -0500 (EST)
(envelope-from news)
X-Authentication-Warning: hub.org: news set sender to <news> using -f
From: "gov-boi" <gov-boi@hack.co.za>
X-Newsgroups: alt.marketplace.crack-corner, alt.master.hackers,
alt.music.cracker-cvb, alt.nl.binaries.hack, alt.phreaking,
alt.recovery.codependency, alt.source-code.mac,
borland.public.codecentral, chinese.text.unicode,
comp.databases.postgresql.hackers, comp.databas
Subject: www.hack.co.za - exploit archives updates - 0day
Date: Fri, 12 Nov 1999 19:32:13 +0200
Lines: 52
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 5.00.2615.200
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2615.200
Message-ID: <382c4fad.0@news1.mweb.co.za>
X-Trace: 12 Nov 1999 19:34:37 +0200, cpt-dial-196-2-56-33.mweb.co.za
To: pgsql-hackers@postgresql.org

[[-Fri 12 November-]]
Added su.c Unixware 7.0 local exploit by K2, posted by demi.(0-day)
Added xlock.c Unixware 7.0 local exploit by K2, posted by demi.(0-day)
Added Xsco.c Unixware 7.0 local exploit by K2, posted by demi.(0-day)

[[-Thur 11 November-]]
Added seyon.sh FreeBSD 3.3 local exploit by Brock Tellier.
Added nfsd2.c RPC + Debian 2.1 + Redhat 5.2 remote exploit by tmoggie.
Added ypbreak.c RPC/nis exploit by anonymous. (old)
Added ypsnarf.c RPC/nis exploit by anonymous. (old)
Added hijaak.sh Sendmail 8.8.8 local exploit by Michal Zalewski
Added Win95/98 and WinNT exploit section.(requested numerous times)
Added wftpdexp.tgz Win95/98/NT remote exploit by Alberto Solino.
Added msadc2.pl WinNT 4.0 remote exploit by rfp.
Added ex_cmail.c Win98 remote exploit by UNYUN.
Added ex_fuse.c Win98 remote exploit by UNYUN.
Added ex_netsrv.c Win98 remote exploit by UNYUN.
Added ex_servu.c Win98 remote exploit by UNYUN.
Added ex_tinyftpd.c Win98 remote exploit by UNYUN.
Added ex_zommail.c Win98 remote exploit by UNYUN.
Added ex_almail.c Win98 local exploit by UNYUN.
Added ex_midiplug.c Win98 local exploit by UNYUN.
Added ex_ssmail.c WinNT 4.0 remote exploit by UNYUN.
Added sendexp.c WinNT 4.0 remote exploit by UNYUN.

[[-Tues 9 November-]]
Added dtappgather.sh Unixware 7.0 local exploit by K2.
Added dopewarez.c Linux/misc remote exploit by nuuB.
Added hylafax.c FreeBSD 3.3 local exploit by Brock Tellier.

[[-Wed 3 November-]]
Added amanda.c FreeBSD 3.3 local exploit by Brock Tellier.
Added canuum.c TurboLinux 3.5 local exploit by UNYUN.
Added sendmail-8.9.3.tar.gz by icesk. (0-day)
Added sperl4.036.c FreeBSD 2.2.8 exploit by OVX. (old)
Added tcpdump.c Linux/misc exploit BLADI. (old)

(exploits/code to post? submit@hack.co.za)
(queries/questions? wwwboard is running)

[[-Disclaimer-]]
The members at www.hack.co.za can not and will not be held
responsible for anyone's actions, nothing on this site is meant
to be used to malicious intent. We will not give out server logs
of people connecting, or will not supply any information that
envolves the "tracking down" of people. We believe in freedom
of speech, any code posted to this page will not be removed
under any circumstances, regardless of copyright or anything
similar. If you do not agree with any of the above, leave now.

From bouncefilter Fri Nov 12 13:03:03 1999
Received: from loopy.berkhirt.com (loopy.berkhirt.com [209.220.85.54])
by hub.org (8.9.3/8.9.3) with ESMTP id NAA91068
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 13:02:42 -0500 (EST)
(envelope-from bhirt@loopy.berkhirt.com)
Received: (from bhirt@localhost)
by loopy.berkhirt.com (8.9.3/8.8.7) id MAA28394;
Fri, 12 Nov 1999 12:02:07 -0600
Date: Fri, 12 Nov 1999 12:02:07 -0600
From: Brian Hirt <bhirt@mobygames.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: bhirt@mobygames.com, pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Slow - grindingly slow - query
Message-ID: <19991112120207.A27778@loopy.berkhirt.com>
Reply-To: bhirt@mobygames.com
References: <19991112034901.B21136@loopy.berkhirt.com>
<26649.942418694@sss.pgh.pa.us>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Mailer: Mutt 0.95.4us
In-Reply-To: <26649.942418694@sss.pgh.pa.us>;
from Tom Lane on Fri, Nov 12, 1999 at 09:58:14AM -0500
X-PC-Gaming: http://www.mobygames.com/

On Fri, Nov 12, 1999 at 09:58:14AM -0500, Tom Lane wrote:

Brian Hirt <bhirt@mobygames.com> writes:

Can't the rewrite engine recognize a simple case like the
one above and rewrite it to use exists and not exists with the proper
joins? Or possibly the optimizer can generate a better plan?

This is on the TODO list, and will get done someday. IMHO it's not as
urgent as a lot of the planner/optimizer's other shortcomings, because
it can usually be worked around by revising the query.

If it's bugging you enough to go fix it now, contributions are always
welcome ;-)

Okay, what would be the correct approach to solving the problem,
and where would be a good place to start? I'v only been on this list
for a few weeks, so I'm missed discussion on the approach to solving
this problem. Should this change be localized to just the planner?
Should the rewrite system be creating a different query tree? Will both
need to be changed? If a lot of work is being done to this part of
the system, is now a bad time to try this work?

I'm willing to jump in to this, but I may take a while to figure it out
and ask a lot of questions that are obvious to the hardened postgres
programmer. I'm not famaliar with the postgres code, yet.

-brian

--
The world's most ambitious and comprehensive PC game database project.

http://www.mobygames.com

From bouncefilter Fri Nov 12 13:45:04 1999
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id NAA96941
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 13:44:48 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id KAA28936;
Fri, 12 Nov 1999 10:43:17 -0800 (PST)
Message-Id: <3.0.1.32.19991112100957.010f0300@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Fri, 12 Nov 1999 10:09:57 -0800
To: Karel Zak - Zakkr <zakkr@zf.jcu.cz>,
"Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: [HACKERS] compression in LO and other fields
Cc: Tom Lane <tgl@sss.pgh.pa.us>, t-ishii@sra.co.jp,
pgsql-hackers <pgsql-hackers@postgreSQL.org>
In-Reply-To: <Pine.LNX.3.96.991112171242.31595A-100000@ara.zf.jcu.cz>
References: <19991112095955.D9755@wallace.ece.rice.edu>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 05:15 PM 11/12/99 +0100, Karel Zak - Zakkr wrote:

pg_dump database | gzip >db.out.gz

Thank... :-))

But mysqldump --compress is very nice.

Why add functionality for something that can be done so
easily by piping the output of pg_dump?

This is exactly the kind of coupling of tools that pipes
were invented for.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Fri Nov 12 14:33:04 1999
Received: from gandalf.telecom.at (gandalf.telecom.at [194.118.26.84])
by hub.org (8.9.3/8.9.3) with ESMTP id OAA04955
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 14:32:38 -0500 (EST)
(envelope-from ZeugswetterA@wien.spardat.at)
Received: from sdexcgtw01.f000.d0188.sd.spardat.at
(sdexcgtw01.f000.d0188.sd.spardat.at [172.18.1.16])
by gandalf.telecom.at (xxx/xxx) with ESMTP id UAA262682;
Fri, 12 Nov 1999 20:32:34 +0100
Received: by sdexcgtw01.f000.d0188.sd.spardat.at with Internet Mail Service
(5.5.2448.0) id <45YXPK4F>; Fri, 12 Nov 1999 20:32:34 +0100
Message-ID:
<219F68D65015D011A8E000006F8590C603FDC16D@sdexcsrv1.f000.d0188.sd.spardat.at>
From: Zeugswetter Andreas SEV <ZeugswetterA@wien.spardat.at>
To: "'Jan Wieck'" <wieck@debis.com>
Cc: "'pgsql-hackers@postgreSQL.org'" <pgsql-hackers@postgreSQL.org>
Subject: AW: [HACKERS] compression in LO and other fields
Date: Fri, 12 Nov 1999 20:32:34 +0100
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2448.0)
Content-Type: text/plain;
charset="iso-8859-1"

I don't know if there's a compression library available that
fit's our need. First and most important it must have a
license that permits us to include it in the distribution
under our existing license. Second it's implementation must
not cause any problems in the backend like memory leakage or
the like.

LZO (Lempel-Ziv-Oberhuemer) sounds like a candidate.

http://wildsau.idv.uni-linz.ac.at/mfx/lzo.html

It is a realtime compressor, designed for real time
compression/decompression of data.
It is really fast, like >20 Mb/s decompressions are easily possible and has
a pretty good compression ratio,
almost as good as gzip, since it favours speed over ratio.

It seems to have a pretty safe and compatible license.

It comes with a gzip cmdline compatible program called lzop.

I really love it.
Andreas

From bouncefilter Fri Nov 12 16:20:05 1999
Received: from lycanthrope.crosslink.net (lycanthrope.crosslink.net
[206.246.124.36]) by hub.org (8.9.3/8.9.3) with ESMTP id QAA18241
for <pgsql-hackers@postgresql.org>;
Fri, 12 Nov 1999 16:19:53 -0500 (EST)
(envelope-from bayardk@crosslink.net)
Received: from crosslink.net ([209.147.70.109]) by lycanthrope.crosslink.net
(8.9.3/) with ESMTP id QAA04635 for
<pgsql-hackers@postgresql.org>; Fri, 12 Nov 1999 16:19:51 -0500
X-Really-To: <pgsql-hackers@postgresql.org>
Message-ID: <382C821C.36863C16@crosslink.net>
Date: Fri, 12 Nov 1999 16:09:48 -0500
From: bayard kohlhepp <bayardk@crosslink.net>
X-Mailer: Mozilla 4.7 [en]C-CCK-MCD NSCPCD47 (Win98; I)
X-Accept-Language: en
MIME-Version: 1.0
To: pgsql-hackers@postgresql.org
Subject: how to handle struct within EXEC SQL DECLARE SECTION?
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

i tried posting this to other groups, as well as searching the archives,
and came up with nothing.

i am using 6.5.2/Mandrake(RedHat)linux 6.0/pentium. i am trying to port
an existing informix 7 web application to postgresql. all of the CGI
programs use embedded SQL. they all use constructs of the form:
...
EXEC SQL BEGIN DECLARE SECTION;
struct user { /* 40 or 50 or 80 fields... */
int x;
...
};
EXEC SQL END DECLARE SECTION;

func1(struct user *x,...);
func2(struct user *X, ...);

main()
{
EXEC SQL BEGIN DECLARE SECTION;
struct user user_rec;
EXEC SQL END DECLARE SECTION;

...
}

ecpg doesn't recognize (ie, "parse error") struct declarations that
occurred outside of the current BEGIN/END section. that forces me to
redeclare the entire struct definition with every variable declaration.
i have gone through and exploded all the declarations to do that, but
now ecpg dies with a segmentation fault, does not report a line number,
and erases the .c in the process so i can't tell how far it got (i
experimented with 2 and 3 field structs, and it worked okay, but some of
these structs have 140+ fields).

what is the proper solution for defining a database record structure,
then declaring variables that use that definition within EXEC SQL
sections?

From bouncefilter Fri Nov 12 16:58:06 1999
Received: from lycanthrope.crosslink.net (lycanthrope.crosslink.net
[206.246.124.36]) by hub.org (8.9.3/8.9.3) with ESMTP id QAA22374
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 16:57:19 -0500 (EST)
(envelope-from bayardk@crosslink.net)
Received: from crosslink.net ([209.147.70.106]) by lycanthrope.crosslink.net
(8.9.3/) with ESMTP id QAA14108 for
<pgsql-hackers@postgreSQL.org>; Fri, 12 Nov 1999 16:57:14 -0500
X-Really-To: <pgsql-hackers@postgreSQL.org>
Message-ID: <382C8AF5.D4EEFBD6@crosslink.net>
Date: Fri, 12 Nov 1999 16:47:33 -0500
From: bayard kohlhepp <bayardk@crosslink.net>
X-Mailer: Mozilla 4.7 [en]C-CCK-MCD NSCPCD47 (Win98; I)
X-Accept-Language: en
MIME-Version: 1.0
To: pgsql-hackers@postgreSQL.org
Subject: how should you define a struct within EXEC SQL section?
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

i tried posting this to other groups, as well as searching the archives,

and came up with nothing.

i am using 6.5.2/Mandrake(RedHat)linux 6.0/pentium. i am trying to port
an existing informix 7 web application to postgresql. all of the CGI
programs use embedded SQL. they all use constructs of the form:
...
EXEC SQL BEGIN DECLARE SECTION;
struct user { /* 40 or 50 or 80 fields... */
int x;
...
};
EXEC SQL END DECLARE SECTION;

func1(struct user *x,...);
func2(struct user *X, ...);

main()
{
EXEC SQL BEGIN DECLARE SECTION;
struct user user_rec;
EXEC SQL END DECLARE SECTION;

...
}

ecpg doesn't recognize (ie, "parse error") struct declarations that
occurred outside of the current BEGIN/END section. that forces me to
redeclare the entire struct definition with every variable declaration.
i have gone through and exploded all the declarations to do that, but
now ecpg dies with a segmentation fault, does not report a line number,
and erases the .c in the process so i can't tell how far it got (i
experimented with 2 and 3 field structs, and it worked okay, but some of

these structs have 140+ fields).

what is the proper solution for defining a database record structure,
then declaring variables that use that definition within EXEC SQL
sections?

From bouncefilter Fri Nov 12 17:04:06 1999
Received: from hu.tm.ee (isdn-54.uninet.ee [194.204.0.118])
by hub.org (8.9.3/8.9.3) with ESMTP id RAA23329
for <pgsql-hackers@postgresql.org>;
Fri, 12 Nov 1999 17:03:07 -0500 (EST) (envelope-from hannu@tm.ee)
Received: from tm.ee (localhost [127.0.0.1]) by hu.tm.ee (Postfix) with ESMTP
id 6A22A5554; Sat, 13 Nov 1999 00:02:54 +0200 (EET)
Sender: hannu@hu.tm.ee
Message-ID: <382C8E8E.999C67A1@tm.ee>
Date: Sat, 13 Nov 1999 00:02:54 +0200
From: Hannu Krosing <hannu@tm.ee>
Organization: Trust-O-Matic =?iso-8859-1?Q?O=DC?=
X-Mailer: Mozilla 4.7 [en] (X11; I; Linux 2.2.13-7mdk i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Jan Wieck <wieck@debis.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, zakkr@zf.jcu.cz, t-ishii@sra.co.jp,
pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] compression in LO and other fields
References: <m11mIp4-0003kLC@orion.SAPserv.Hamburg.dsh.de>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Jan Wieck wrote:

Of course.

Well, you asked for the rates on the smaller html files only.
78 files, 131 bytes min, 10000 bytes max, 4582 bytes avg,
357383 bytes total.

gzip -9 outputs 145659 bytes (59.2%)
gzip -1 outputs 155113 bytes (56.6%)
my code outputs 184109 bytes (48.5%)

67 files, 2000 bytes min, 10000 bytes max, 5239 bytes avg,
351006 bytes total.

gzip -9 outputs 141772 bytes (59.6%)
gzip -1 outputs 151150 bytes (56.9%)
my code outputs 179428 bytes (48.9%)

The threshold will surely be a tuning parameter of interest.
Another tuning option must be to allow/deny compression per
table at all. Then we could have both options, using a
compressing field type to define which portion of a tuple to
compress, or allow to compress the entire tuples.

The next step would be tweaking the costs for sequential scans vs.
index scans.

I guess that the indexes would stay uncompressed ?

------
Hannu

From bouncefilter Fri Nov 12 17:12:06 1999
Received: from hu.tm.ee (isdn-54.uninet.ee [194.204.0.118])
by hub.org (8.9.3/8.9.3) with ESMTP id RAA24778
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 17:11:25 -0500 (EST) (envelope-from hannu@tm.ee)
Received: from tm.ee (localhost [127.0.0.1]) by hu.tm.ee (Postfix) with ESMTP
id 492A25554; Sat, 13 Nov 1999 00:11:13 +0200 (EET)
Sender: hannu@hu.tm.ee
Message-ID: <382C9080.103696C9@tm.ee>
Date: Sat, 13 Nov 1999 00:11:12 +0200
From: Hannu Krosing <hannu@tm.ee>
Organization: Trust-O-Matic =?iso-8859-1?Q?O=DC?=
X-Mailer: Mozilla 4.7 [en] (X11; I; Linux 2.2.13-7mdk i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Don Baccus <dhogaza@pacifier.com>
Cc: Karel Zak - Zakkr <zakkr@zf.jcu.cz>,
"Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>,
Tom Lane <tgl@sss.pgh.pa.us>, t-ishii@sra.co.jp,
pgsql-hackers <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] compression in LO and other fields
References: <19991112095955.D9755@wallace.ece.rice.edu>
<3.0.1.32.19991112100957.010f0300@mail.pacifier.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Don Baccus wrote:

At 05:15 PM 11/12/99 +0100, Karel Zak - Zakkr wrote:

pg_dump database | gzip >db.out.gz

Thank... :-))

But mysqldump --compress is very nice.

Why add functionality for something that can be done so
easily by piping the output of pg_dump?

This is exactly the kind of coupling of tools that pipes
were invented for.

Exactly !

Another version of the same is used when dumping databases
bigger than 2GB (or whatever the file system size limit is)

just do:

pg_dump database | gzip | split -b 1000000000 db.out.gz.

and restore it using

cat db.out.gz* | gunzip > psql

you could also do other fancy things with your dumps - send
them directly to tape storage in Japan or whatever ;)

If you need that functionality often enough then write a shell
script.

---------------
Hannu

From bouncefilter Fri Nov 12 18:28:07 1999
Received: from atbib.be (IDENT:root@a00-212.antw.online.be [62.112.1.212])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA34067
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 18:27:08 -0500 (EST) (envelope-from fve@atbib.be)
Received: from frans.atbib.be (fransnieuw [193.75.233.10])
by atbib.be (8.9.3/8.9.3) with SMTP id BAA10111;
Sat, 13 Nov 1999 01:26:38 +0100
Message-Id: <3.0.6.32.19991113003130.0087d230@193.75.233.1>
X-Sender: fve@193.75.233.1
X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32)
Date: Sat, 13 Nov 1999 00:31:30 +0100
To: Tom Lane <tgl@sss.pgh.pa.us>
From: Frans Van Elsacker <fve@atbib.be>
Subject: Re: [HACKERS] union problem version 6.5.3
Cc: pgsql-hackers@postgreSQL.org
In-Reply-To: <26964.942420885@sss.pgh.pa.us>
References: <Your message of Fri, 12 Nov 1999 01:10:06 +0100
<3.0.6.32.19991112011006.00876c00@193.75.233.1>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

thanks for your quick answer,

Our earlier version was postgresql 6.4.2

Regards, Frans

At 10:34 12/11/99 -0500, Tom Lane wrote:

Frans Van Elsacker <fve@atbib.be> writes:

I had a little piece of code who works fine up till know, just after
installing
the version 6.5.3 i got the message 'parse error near union'

create table work as
select * from opdracht
union
select * from opdrachtproost

Hmm, the grammar has

CreateAsStmt: CREATE OptTemp TABLE relation_name OptCreateAs AS SubSelect

and SubSelect doesn't allow unions. This is overly restrictive,
I agree, but I thought it had been that way for a good while.
What version were you using before?

regards, tom lane

************

From bouncefilter Fri Nov 12 19:00:07 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id TAA39309
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 19:00:03 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id SAA28158;
Fri, 12 Nov 1999 18:58:59 -0500 (EST)
To: Frans Van Elsacker <fve@atbib.be>
cc: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] union problem version 6.5.3
In-reply-to: Your message of Sat, 13 Nov 1999 00:31:30 +0100
<3.0.6.32.19991113003130.0087d230@193.75.233.1>
Date: Fri, 12 Nov 1999 18:58:59 -0500
Message-ID: <28156.942451139@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Frans Van Elsacker <fve@atbib.be> writes:

Our earlier version was postgresql 6.4.2

OK, I thought the change was older than that. It probably got into 6.5
as a side-effect of incorporating the INTERSECT/EXCEPT feature. Anyway,
we ought to try to restore the old functionality.

regards, tom lane

Hmm, the grammar has

CreateAsStmt: CREATE OptTemp TABLE relation_name OptCreateAs AS SubSelect

and SubSelect doesn't allow unions. This is overly restrictive,
I agree, but I thought it had been that way for a good while.
What version were you using before?

From bouncefilter Fri Nov 12 20:53:08 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id UAA57089
for <pgsql-hackers@postgresql.org>;
Fri, 12 Nov 1999 20:52:21 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgresql.org
id m11mSEF-0003kLC; Sat, 13 Nov 99 02:43 MET
Message-Id: <m11mSEF-0003kLC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] compression in LO and other fields
To: hannu@tm.ee (Hannu Krosing)
Date: Sat, 13 Nov 1999 02:43:47 +0100 (MET)
Cc: wieck@debis.com, tgl@sss.pgh.pa.us, zakkr@zf.jcu.cz, t-ishii@sra.co.jp,
pgsql-hackers@postgresql.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <382C8E8E.999C67A1@tm.ee> from "Hannu Krosing" at Nov 13,
99 00:02:54 am
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Jan Wieck wrote:

Of course.

Well, you asked for the rates on the smaller html files only.
78 files, 131 bytes min, 10000 bytes max, 4582 bytes avg,
357383 bytes total.

gzip -9 outputs 145659 bytes (59.2%)
gzip -1 outputs 155113 bytes (56.6%)
my code outputs 184109 bytes (48.5%)

67 files, 2000 bytes min, 10000 bytes max, 5239 bytes avg,
351006 bytes total.

gzip -9 outputs 141772 bytes (59.6%)
gzip -1 outputs 151150 bytes (56.9%)
my code outputs 179428 bytes (48.9%)

The threshold will surely be a tuning parameter of interest.
Another tuning option must be to allow/deny compression per
table at all. Then we could have both options, using a
compressing field type to define which portion of a tuple to
compress, or allow to compress the entire tuples.

The next step would be tweaking the costs for sequential scans vs.
index scans.

I guess that the indexes would stay uncompressed ?

------
Hannu

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Fri Nov 12 21:02:08 1999
Received: from thelab.hub.org (nat198.58.mpoweredpc.net [142.177.198.58])
by hub.org (8.9.3/8.9.3) with ESMTP id VAA58176
for <pgsql-hackers@postgresql.org>;
Fri, 12 Nov 1999 21:02:03 -0500 (EST) (envelope-from scrappy@hub.org)
Received: from localhost (scrappy@localhost)
by thelab.hub.org (8.9.3/8.9.1) with ESMTP id WAA74835;
Fri, 12 Nov 1999 22:02:10 -0400 (AST) (envelope-from scrappy@hub.org)
X-Authentication-Warning: thelab.hub.org: scrappy owned process doing -bs
Date: Fri, 12 Nov 1999 22:02:10 -0400 (AST)
From: The Hermit Hacker <scrappy@hub.org>
To: Tom Lane <tgl@sss.pgh.pa.us>
cc: Jan Wieck <wieck@debis.com>, zakkr@zf.jcu.cz, t-ishii@sra.co.jp,
pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] compression in LO and other fields
In-Reply-To: <26760.942419535@sss.pgh.pa.us>
Message-ID: <Pine.BSF.4.10.9911122156360.2296-100000@thelab.hub.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Fri, 12 Nov 1999, Tom Lane wrote:

wieck@debis.com (Jan Wieck) writes:

Tom Lane wrote:

It occurred to me last night that applying compression to individual
fields might not be the best approach. Certainly a "bytez" data type
is the easiest thing to fit into the existing system, but it's leaving
some space savings on the table. What about compressing the *whole*
data contents of a tuple on-disk, as a single entity? That should save
more space than field-by-field compression.

But it requires decompression of every tuple into palloc()'d
memory during heap access. AFAIK, the heap access routines
currently return a pointer to the tuple inside the shm
buffer. Don't know what it's performance impact would be.

Good point, but the same will be needed when a tuple is split across
multiple blocks. I would expect that (given a reasonably fast
decompressor) there will be a net performance *gain* due to having
less disk I/O to do.

Right now, we're dealing theory...my concern is what Jan points
out "what it's performance impact would be"...would much harder would it
be to extent our "CREATE TABLE" syntax to do something like:

CREATE TABLE classname ( .. ) compressed;

Or something similar? Something that leaves the ability to do
this in the core, but makes the use of this the choice of the admin?

*Assuming* that I'm also reading this thread correctly, it should
almost be extended into "ALTER TABLE classname SET COMPRESSED on;", or
something like that. Where all new records are *written* compressed (or
uncompressed), but any reads checks if compressed size == uncompressed
size, and decompresses accordingly...

Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

From bouncefilter Fri Nov 12 21:34:09 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id VAA62253
for <pgsql-hackers@postgresql.org>;
Fri, 12 Nov 1999 21:33:35 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgresql.org
id m11mSsA-0003kzC; Sat, 13 Nov 99 03:25 MET
Message-Id: <m11mSsA-0003kzC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] compression in LO and other fields
To: hannu@tm.ee (Hannu Krosing)
Date: Sat, 13 Nov 1999 03:25:02 +0100 (MET)
Cc: wieck@debis.com, tgl@sss.pgh.pa.us, zakkr@zf.jcu.cz, t-ishii@sra.co.jp,
pgsql-hackers@postgresql.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <382C8E8E.999C67A1@tm.ee> from "Hannu Krosing" at Nov 13,
99 00:02:54 am
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Ech - wrong key :-)

Hannu Krosing wrote:

Jan Wieck wrote:

The next step would be tweaking the costs for sequential scans vs.
index scans.

I guess that the indexes would stay uncompressed ?

I'm sure about this. On a database of significant size,
anyone indexing a field with a possible size over 100 bytes
is doing something wrong (and only idiots go above 500
bytes). They are IMPLEMENTING a not well thought out database
DESIGN. A database engine should support indices on bigger
fields, but it's still a bad schema and thus idiotic.

Currently, we don't check the size of indexed fields. And the
only problems I've seen with it where some reports that huge
PL functions could not be created because there was an unused
(idiotic) index on the prosrc attribute and they exceeded the
4K limit for index tuples. I've removed this index already in
the v7.0 tree. The ?bug? in the btree code, failing to split
a page if the key values exceed 4K, is still there. But I
don't think anyone really cares for it.

Thus, I assume there aren't many idiots out there. And I
don't expect that anyone would ever create an index on a
compressed data type.

?bug? -> The difference between a bug and a feature is
DOCUMENTATION. Thomas, would you please add this limit on
index tuples to the doc's so we have a new FEATURE to tell in
the v7.0 announcement?

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Fri Nov 12 22:01:09 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id WAA66773
for <pgsql-hackers@postgresql.org>;
Fri, 12 Nov 1999 22:00:23 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgresql.org
id m11mTI4-0003l0C; Sat, 13 Nov 99 03:51 MET
Message-Id: <m11mTI4-0003l0C@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] compression in LO and other fields
To: scrappy@hub.org (The Hermit Hacker)
Date: Sat, 13 Nov 1999 03:51:48 +0100 (MET)
Cc: tgl@sss.pgh.pa.us, wieck@debis.com, zakkr@zf.jcu.cz, t-ishii@sra.co.jp,
pgsql-hackers@postgresql.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <Pine.BSF.4.10.9911122156360.2296-100000@thelab.hub.org> from
"The Hermit Hacker" at Nov 12, 99 10:02:10 pm
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Marc G. Fournier wrote:

On Fri, 12 Nov 1999, Tom Lane wrote:

wieck@debis.com (Jan Wieck) writes:

Tom Lane wrote:

Right now, we're dealing theory...my concern is what Jan points
out "what it's performance impact would be"...would much harder would it
be to extent our "CREATE TABLE" syntax to do something like:

CREATE TABLE classname ( .. ) compressed;

Or something similar? Something that leaves the ability to do
this in the core, but makes the use of this the choice of the admin?

Yepp, exactly that's what I meant with making tuple
compression a per table option. Obviously, ALTER TABLE ...
must be supported too - that's simply a parser -> utility ->
flip flag in pg_class thing (90% cut&paste).

I think the part on deciding what to compress is easy,
because the flag telling if heap access should try to
compress a tuple on append (i.e. INSERT or UPDATE) has to be
in pg_class. And the content of a relations pg_class entry is
somewhere below the Relation struct (thus already known after
heap_open).

The idea was to use another bit in the tuple header to tell
if an existing heap tuple's data is compressed or not. So the
heap fetching allways looks at the bit in the tuple header,
and the heap appending looks at the flag in the relation
pointer. That's exactly what you want, no?

The major part is to make all callers of heap_fetch() and
sisters treat in memory decompressed (or from block split
reconstructed) tuples the right way.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Fri Nov 12 22:48:09 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA85359
for <pgsql-hackers@postgresql.org>;
Fri, 12 Nov 1999 22:47:43 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id WAA28579;
Fri, 12 Nov 1999 22:45:57 -0500 (EST)
To: wieck@debis.com (Jan Wieck)
cc: scrappy@hub.org (The Hermit Hacker), zakkr@zf.jcu.cz, t-ishii@sra.co.jp,
pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] compression in LO and other fields
In-reply-to: Your message of Sat, 13 Nov 1999 03:51:48 +0100 (MET)
<m11mTI4-0003l0C@orion.SAPserv.Hamburg.dsh.de>
Date: Fri, 12 Nov 1999 22:45:57 -0500
Message-ID: <28577.942464757@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

wieck@debis.com (Jan Wieck) writes:

The idea was to use another bit in the tuple header to tell
if an existing heap tuple's data is compressed or not. So the
heap fetching allways looks at the bit in the tuple header,
and the heap appending looks at the flag in the relation
pointer. That's exactly what you want, no?

Right. Compressed tuples must be unambiguously marked as such on-disk.
Whether to compress a tuple when writing it out is a decision that
can be made on-the-fly, using strategies that could change from time
to time, without invalidating the data that's already out there or
affecting the tuple-reading code.

If we choose to provide also a way of compressing individual fields
rather than whole tuples, it would be good to provide the same
flexibility at the field level. Some tuples might contain the field
in compressed form, some in uncompressed form. The reading logic
should not need to be aware of the way that the writing logic chooses
which to do.

regards, tom lane

From bouncefilter Fri Nov 12 23:11:12 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA89676
for <hackers@postgreSQL.org>; Fri, 12 Nov 1999 23:10:42 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id XAA28744;
Fri, 12 Nov 1999 23:10:02 -0500 (EST)
To: Peter Eisentraut <peter_e@gmx.net>
cc: hackers@postgreSQL.org
Subject: Re: [HACKERS] RFC: create/alter user extension
In-reply-to: Your message of Fri, 12 Nov 1999 16:02:45 +0100 (CET)
<Pine.LNX.4.20.9911121555290.1261-100000@peter-e.yi.org>
Date: Fri, 12 Nov 1999 23:10:02 -0500
Message-ID: <28742.942466202@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Peter Eisentraut <peter_e@gmx.net> writes:

Is everyone okay with the following syntax:
CREATE USER username
[ WITH ID digits ]
^^^^^^^^^^^^^^^^^^

The catch is that ID would have to be a new keyword and we'd have to live
with that for a long time. Other choices include:
* UID
* SYSID
* USESYSID
etc.

I'd be inclined to go with UID or SYSID. In any case, since the new
keyword is used in such a limited context, we could almost certainly
still allow it as a ColId and thus not create any real compatibility
problem.

regards, tom lane

From bouncefilter Fri Nov 12 23:33:10 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA92483
for <pgsql-hackers@postgreSQL.org>;
Fri, 12 Nov 1999 23:32:17 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id XAA28765;
Fri, 12 Nov 1999 23:30:55 -0500 (EST)
To: bhirt@mobygames.com
cc: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Slow - grindingly slow - query
In-reply-to: Your message of Fri, 12 Nov 1999 12:02:07 -0600
<19991112120207.A27778@loopy.berkhirt.com>
Date: Fri, 12 Nov 1999 23:30:55 -0500
Message-ID: <28763.942467455@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Brian Hirt <bhirt@mobygames.com> writes:

On Fri, Nov 12, 1999 at 09:58:14AM -0500, Tom Lane wrote:

If it's bugging you enough to go fix it now, contributions are always
welcome ;-)

Okay, what would be the correct approach to solving the problem,
and where would be a good place to start? I'v only been on this list
for a few weeks, so I'm missed discussion on the approach to solving
this problem. Should this change be localized to just the planner?
Should the rewrite system be creating a different query tree? Will both
need to be changed? If a lot of work is being done to this part of
the system, is now a bad time to try this work?

Well, actually, figuring out how & where to do it is the trickiest part
of the work. Might not be the best project for a newbie backend-hacker
to start with :-(.

After a few moments' thought, it seems to me that this issue might be
closely intertwined with the OUTER JOIN stuff that Thomas is working on
and the querytree representation redesign that Jan and I have been
muttering about (but not yet actually doing anything about). We want
to handle SELECT ... WHERE expr IN (SELECT ...) like a join, but the
semantics aren't exactly the same as a conventional join, so it might
be that the thing needs to be rewritten as a special join type. In
that case it'd fit right in with OUTER JOIN, I suspect.

The Informix EXPLAIN results that Theo Kramer posted (a few messages
back in this thread) are pretty interesting too. If I'm reading that
printout right, Informix is not any smarter than we are about choosing
the scan types for the outer and inner queries; and yet they have a much
faster runtime for the WHERE IN query. I speculate that they are doing
the physical matching of outer and inner tuples in a smarter way than we
are --- perhaps they are doing one scan of the inner query and entering
all the values into a hashtable that's then probed for each outer tuple.
(As opposed to rescanning the inner query for each outer tuple, as we
currently do.) If that's the answer, then it could probably be
implemented as a localized change: rewrite the SubPlan node executor to
look more like the HashJoin node executor. This isn't perfect --- it
wouldn't pick up the possibility of a merge-style join --- but it would
be better than what we have for a lot less work than the "full" solution.

This is all shooting from the hip; I haven't spent time looking into it.
Has anyone else got insights to offer?

regards, tom lane

From bouncefilter Sat Nov 13 00:17:11 1999
Received: from mailhub.southeast.net (root@mailhub.southeast.net
[207.98.192.83]) by hub.org (8.9.3/8.9.3) with ESMTP id AAA99914
for <pgsql-hackers@postgresql.org>;
Sat, 13 Nov 1999 00:16:58 -0500 (EST)
(envelope-from mtsinc@southeast.net)
Received: from southeast.net (mail.mousetech.com [216.199.14.19])
by mailhub.southeast.net (8.8.5/8.8.5) with ESMTP id AAA16724
for <pgsql-hackers@postgresql.org>;
Sat, 13 Nov 1999 00:16:55 -0500 (EST)
Message-ID: <382CF140.CBC119C9@southeast.net>
Date: Sat, 13 Nov 1999 00:04:00 -0500
From: Tim Holloway <mtsinc@southeast.net>
X-Mailer: Mozilla 4.7 [en] (Win98; I)
X-Accept-Language: en
MIME-Version: 1.0
To: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Thread-safe queueing?
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

I need to create a cross-process producer/consumer data queue (e.g. singly-linked list).

That is - Processes A, B, and C add nodes to a controlled list and process D removes them.
Not sure if the creation of the nodes would be best done by the producers or consumers,
but destruction would have to be done by the consumer, as the producers don't wait for
processing. For optimal results, the consumer process should sleep until item(s) are added
to its queue.

Query: within the existing backend framework, what's the best way to accomplish this?

Thanks,

Tim Holloway

From bouncefilter Sat Nov 13 02:54:12 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id CAA17107
for <hackers@postgreSQL.org>; Sat, 13 Nov 1999 02:53:44 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id CAA29204;
Sat, 13 Nov 1999 02:53:09 -0500 (EST)
To: Peter Eisentraut <peter_e@gmx.net>
cc: hackers@postgreSQL.org
Subject: Re: [HACKERS] Backend build fails in current
In-reply-to: Your message of Mon, 8 Nov 1999 15:37:02 +0100 (MET)
<Pine.GSO.4.02A.9911081528230.2099-100000@Krokodil.DoCS.UU.SE>
Date: Sat, 13 Nov 1999 02:53:09 -0500
Message-ID: <29202.942479589@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Peter Eisentraut <e99re41@DoCS.UU.SE> writes:

...
make -C common parser/parse.h
make[4]: Entering directory `/home/fenix0/eh99/e99re41/pgsql/src/backend/access/common'
make[4]: *** No rule to make target `parser/parse.h'. Stop.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Are you still seeing this? I didn't see it with a pull from CVS
yesterday. If you are, what version of make are you using?

regards, tom lane

From bouncefilter Sat Nov 13 04:54:14 1999
Received: from flame.flame.co.za (flame.flame.co.za [160.124.170.1])
by hub.org (8.9.3/8.9.3) with ESMTP id EAA36562
for <pgsql-hackers@postgreSQL.org>;
Sat, 13 Nov 1999 04:53:27 -0500 (EST)
(envelope-from theo@flame.co.za)
Received: from flame.co.za (flame.flame.co.za [160.124.170.1])
by flame.flame.co.za (8.8.7/8.8.7) with ESMTP id LAA26775
for <pgsql-hackers@postgreSQL.org>; Sat, 13 Nov 1999 11:55:38 +0200
Sender: theo@flame.flame.co.za
Message-ID: <382D359A.DB093E91@flame.co.za>
Date: Sat, 13 Nov 1999 11:55:38 +0200
From: Theo Kramer <theo@flame.co.za>
Organization: Flame Computing Enterprises
X-Mailer: Mozilla 4.5 [en] (X11; I; Linux 2.0.35 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Slow - grindingly slow - query
References: <28763.942467455@sss.pgh.pa.us>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Tom Lane wrote:

The Informix EXPLAIN results that Theo Kramer posted (a few messages
back in this thread) are pretty interesting too. If I'm reading that
printout right, Informix is not any smarter than we are about choosing
the scan types for the outer and inner queries; and yet they have a much
faster runtime for the WHERE IN query.

The informix EXPLAIN for the 'not in' query was when I did not have an
index on registrationtype (the explain appends to file sqexplain.out so I
missed it :(). Anyway here is the Informix EXPLAIN with the index on
registrationtype.

QUERY:
------
select accountdetail.domain from accountdetail where
accountdetail.domain not in (select accountmaster.domain from accountmaster)

Estimated Cost: 4510
Estimated # of Rows Returned: 58810

1) informix.accounts: SEQUENTIAL SCAN

Filters: (informix.accounts.domain != ALL <subquery> AND
informix.accounts.registrationtype != 'N' )

Subquery:
---------
Estimated Cost: 12
Estimated # of Rows Returned: 10

1) informix.accounts: INDEX PATH

(1) Index Keys: registrationtype
Lower Index Filter: informix.accounts.registrationtype = 'N'

The speed difference with or without the subquery index is neglible for
Informix.
--------
Regards
Theo

From bouncefilter Sat Nov 13 08:37:16 1999
Received: from Radha.DoCS.UU.SE (root@Radha.DoCS.UU.SE [130.238.9.99])
by hub.org (8.9.3/8.9.3) with SMTP id IAA58009
for <hackers@postgreSQL.org>; Sat, 13 Nov 1999 08:36:44 -0500 (EST)
(envelope-from e99re41@DoCS.UU.SE)
Received: from Val.DoCS.UU.SE (e99re41@Val.DoCS.UU.SE [130.238.9.94]) by
Radha.DoCS.UU.SE (8.6.12/8.6.12) with ESMTP id OAA22716;
Sat, 13 Nov 1999 14:36:35 +0100
Received: from localhost (e99re41@localhost) by Val.DoCS.UU.SE (8.6.12/8.6.12)
with SMTP id OAA01327; Sat, 13 Nov 1999 14:36:32 +0100
X-Authentication-Warning: Val.DoCS.UU.SE: e99re41 owned process doing -bs
Date: Sat, 13 Nov 1999 14:36:31 +0100 (MET)
From: Peter Eisentraut <e99re41@DoCS.UU.SE>
Reply-To: Peter Eisentraut <peter_e@gmx.net>
To: Tom Lane <tgl@sss.pgh.pa.us>
cc: hackers@postgreSQL.org
Subject: Re: [HACKERS] Backend build fails in current
In-Reply-To: <29202.942479589@sss.pgh.pa.us>
Message-ID: <Pine.GSO.4.02A.9911131431230.606-100000@Val.DoCS.UU.SE>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Sat, 13 Nov 1999, Tom Lane wrote:

Peter Eisentraut <e99re41@DoCS.UU.SE> writes:

...
make -C common parser/parse.h
make[4]: Entering directory `/home/fenix0/eh99/e99re41/pgsql/src/backend/access/common'
make[4]: *** No rule to make target `parser/parse.h'. Stop.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Are you still seeing this? I didn't see it with a pull from CVS
yesterday. If you are, what version of make are you using?

Affirmative. Same problem.

GNU Make version 3.74, by Richard Stallman and Roland McGrath.

That's a little old it seems, but I don't have any power to upgrade it on
this particular machine. It should certainly be possible to fix the make
files, since requiring GNU make is already a hassle for some, but
requiring the latest version might be too much to ask for?

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Sat Nov 13 08:38:16 1999
Received: from Radha.DoCS.UU.SE (root@Radha.DoCS.UU.SE [130.238.9.99])
by hub.org (8.9.3/8.9.3) with SMTP id IAA58046
for <hackers@postgreSQL.org>; Sat, 13 Nov 1999 08:38:06 -0500 (EST)
(envelope-from e99re41@DoCS.UU.SE)
Received: from Val.DoCS.UU.SE (e99re41@Val.DoCS.UU.SE [130.238.9.94]) by
Radha.DoCS.UU.SE (8.6.12/8.6.12) with ESMTP id OAA22800;
Sat, 13 Nov 1999 14:38:05 +0100
Received: from localhost (e99re41@localhost) by Val.DoCS.UU.SE (8.6.12/8.6.12)
with SMTP id OAA01331; Sat, 13 Nov 1999 14:38:03 +0100
X-Authentication-Warning: Val.DoCS.UU.SE: e99re41 owned process doing -bs
Date: Sat, 13 Nov 1999 14:38:03 +0100 (MET)
From: Peter Eisentraut <e99re41@DoCS.UU.SE>
Reply-To: Peter Eisentraut <peter_e@gmx.net>
To: Tom Lane <tgl@sss.pgh.pa.us>
cc: hackers@postgreSQL.org
Subject: Re: [HACKERS] RFC: create/alter user extension
In-Reply-To: <28742.942466202@sss.pgh.pa.us>
Message-ID: <Pine.GSO.4.02A.9911131436450.606-100000@Val.DoCS.UU.SE>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Fri, 12 Nov 1999, Tom Lane wrote:

Peter Eisentraut <peter_e@gmx.net> writes:

Is everyone okay with the following syntax:
CREATE USER username
[ WITH ID digits ]
^^^^^^^^^^^^^^^^^^

The catch is that ID would have to be a new keyword and we'd have to live
with that for a long time. Other choices include:
* UID
* SYSID
* USESYSID
etc.

I'd be inclined to go with UID or SYSID. In any case, since the new
keyword is used in such a limited context, we could almost certainly
still allow it as a ColId and thus not create any real compatibility
problem.

I'm not sure about this distinction. Where would that be reflected in the
(parser) code?

-Peter

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Sat Nov 13 09:51:17 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id JAA65303
for <hackers@postgreSQL.org>; Sat, 13 Nov 1999 09:50:41 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id JAA00281;
Sat, 13 Nov 1999 09:50:08 -0500 (EST)
To: Peter Eisentraut <peter_e@gmx.net>
cc: hackers@postgreSQL.org
Subject: Re: [HACKERS] Backend build fails in current
In-reply-to: Your message of Sat, 13 Nov 1999 14:36:31 +0100 (MET)
<Pine.GSO.4.02A.9911131431230.606-100000@Val.DoCS.UU.SE>
Date: Sat, 13 Nov 1999 09:50:08 -0500
Message-ID: <279.942504608@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Peter Eisentraut <e99re41@DoCS.UU.SE> writes:

Are you still seeing this? I didn't see it with a pull from CVS
yesterday. If you are, what version of make are you using?

Affirmative. Same problem.

GNU Make version 3.74, by Richard Stallman and Roland McGrath.

That's a little old it seems,

It is. I'd suggest leaning on your sysadmin to get it updated to
something current (3.78.1 is current I think).

In the meantime, please try the attached patch. If it seems to
straighten out the behavior on your make, I'll commit it.

regards, tom lane

*** src/backend/Makefile.orig	Sun Mar  7 18:05:56 1999
--- src/backend/Makefile	Sat Nov 13 09:43:17 1999
***************
*** 116,127 ****
  # make files in our subdirectories.

parse.h: parser/parse.h
- $(MAKE) -C parser parse.h
cp parser/parse.h .

! fmgr.h:
! $(MAKE) -C utils fmgr.h
cp utils/fmgr.h .

  #############################################################################
  clean:
--- 116,131 ----
  # make files in our subdirectories.

parse.h: parser/parse.h
cp parser/parse.h .

! parser/parse.h:
! $(MAKE) -C parser parse.h
!
! fmgr.h: utils/fmgr.h
cp utils/fmgr.h .
+
+ utils/fmgr.h:
+ $(MAKE) -C utils fmgr.h

#############################################################################
clean:

From bouncefilter Sat Nov 13 09:58:17 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id JAA65998
for <hackers@postgreSQL.org>; Sat, 13 Nov 1999 09:57:40 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id JAA00320;
Sat, 13 Nov 1999 09:57:08 -0500 (EST)
To: Peter Eisentraut <peter_e@gmx.net>
cc: hackers@postgreSQL.org
Subject: Re: [HACKERS] RFC: create/alter user extension
In-reply-to: Your message of Sat, 13 Nov 1999 14:38:03 +0100 (MET)
<Pine.GSO.4.02A.9911131436450.606-100000@Val.DoCS.UU.SE>
Date: Sat, 13 Nov 1999 09:57:08 -0500
Message-ID: <318.942505028@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Peter Eisentraut <e99re41@DoCS.UU.SE> writes:

I'd be inclined to go with UID or SYSID. In any case, since the new
keyword is used in such a limited context, we could almost certainly
still allow it as a ColId and thus not create any real compatibility
problem.

I'm not sure about this distinction. Where would that be reflected in the
(parser) code?

You should try to add this (or any other) new keyword to the list in the
ColId: production in gram.y. If that doesn't provoke any complaints
from yacc (shift/reduce conflicts etc), then you're home free: the
parser won't get confused if the keyword is used as a column name.

If it does cause a shift/reduce conflict, which is fairly likely for
anything that can appear inside an expression, you might still be
able to add the new keyword to the ColLabel: list. That allows it
to be used as an identifier in a more restricted set of contexts.

Only if neither of these will work does the keyword need to be a
truly "reserved" word.

regards, tom lane

From bouncefilter Sat Nov 13 10:16:17 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id KAA67927
for <pgsql-hackers@postgreSQL.org>;
Sat, 13 Nov 1999 10:15:28 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id KAA00433;
Sat, 13 Nov 1999 10:14:21 -0500 (EST)
To: Tim Holloway <mtsinc@southeast.net>
cc: pgsql-hackers <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Thread-safe queueing?
In-reply-to: Your message of Sat, 13 Nov 1999 00:04:00 -0500
<382CF140.CBC119C9@southeast.net>
Date: Sat, 13 Nov 1999 10:14:21 -0500
Message-ID: <431.942506061@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Tim Holloway <mtsinc@southeast.net> writes:

I need to create a cross-process producer/consumer data queue
(e.g. singly-linked list). That is - Processes A, B, and C add nodes
to a controlled list and process D removes them. Not sure if the
creation of the nodes would be best done by the producers or
consumers, but destruction would have to be done by the consumer, as
the producers don't wait for processing. For optimal results, the
consumer process should sleep until item(s) are added to its queue.

Query: within the existing backend framework, what's the best way to
accomplish this?

More context, please. What are you trying to accomplish? Is this
really a communication path between backends (and if so, what backend
code needs it?), or are you trying to set up a queue between SQL
clients? How much data might need to be in the queue at one time?

regards, tom lane

From bouncefilter Sat Nov 13 10:42:18 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id KAA70642
for <pgsql-hackers@postgreSQL.org>;
Sat, 13 Nov 1999 10:41:35 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id KAA00704;
Sat, 13 Nov 1999 10:41:02 -0500 (EST)
To: Peter Eisentraut <peter_e@gmx.net>
cc: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Backend build fails in current
In-reply-to: Your message of Sat, 13 Nov 1999 16:19:38 +0100 (MET)
<Pine.GSO.4.02A.9911131606450.606-100000@Val.DoCS.UU.SE>
Date: Sat, 13 Nov 1999 10:41:02 -0500
Message-ID: <702.942507662@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Okay, the build went fine, but the following came up:
make -C parser all
make[1]: Entering directory `/home/fenix0/eh99/e99re41/postgresql-cur/src/backend/parser'
Makefile:39: warning: overriding commands for target `parse.h'
Makefile:34: warning: ignoring old commands for target `parse.h'

With that fixed you could commit it.

I haven't the foggiest how to work around that --- but since my make
(3.76.1, no spring chicken itself) doesn't generate any such complaint,
I'd say it's another bug in that old version you have.

My inclination is to apply the patch anyway, since it's cleaner coding.

If make 3.74 does the right things despite the warning, then you could
live with it. Otherwise, time to upgrade.

It seems we ought to add a minimum GNU make version number to the list
of prerequisites for Postgres. Data points so far are that 3.74 has
problems and 3.76.1 is OK --- can anyone fill in more observations?
Anyone using 3.75, for instance?

regards, tom lane

From bouncefilter Sat Nov 13 11:34:19 1999
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA75554
for <pgsql-hackers@postgreSQL.org>;
Sat, 13 Nov 1999 11:34:09 -0500 (EST)
(envelope-from maillist@candle.pha.pa.us)
Received: (from maillist@localhost)
by candle.pha.pa.us (8.9.0/8.9.0) id LAA24305;
Sat, 13 Nov 1999 11:19:42 -0500 (EST)
From: Bruce Momjian <maillist@candle.pha.pa.us>
Message-Id: <199911131619.LAA24305@candle.pha.pa.us>
Subject: Re: [HACKERS] Backend build fails in current
In-Reply-To: <702.942507662@sss.pgh.pa.us> from Tom Lane at "Nov 13,
1999 10:41:02 am"
To: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sat, 13 Nov 1999 11:19:42 -0500 (EST)
CC: Peter Eisentraut <peter_e@gmx.net>, pgsql-hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

It seems we ought to add a minimum GNU make version number to the list
of prerequisites for Postgres. Data points so far are that 3.74 has
problems and 3.76.1 is OK --- can anyone fill in more observations?
Anyone using 3.75, for instance?

3.75 ships with BSD/OS and is fine.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Sat Nov 13 11:37:19 1999
Received: from candle.pha.pa.us (maillist@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA75992
for <pgsql-hackers@postgreSQL.org>;
Sat, 13 Nov 1999 11:36:53 -0500 (EST)
(envelope-from maillist@candle.pha.pa.us)
Received: (from maillist@localhost)
by candle.pha.pa.us (8.9.0/8.9.0) id LAA24800;
Sat, 13 Nov 1999 11:35:47 -0500 (EST)
From: Bruce Momjian <maillist@candle.pha.pa.us>
Message-Id: <199911131635.LAA24800@candle.pha.pa.us>
Subject: Failure to show descriptions in \df command
In-Reply-To: <Pine.LNX.4.20.9911112236460.442-100000@peter-e.yi.org> from
Peter
Eisentraut at "Nov 11, 1999 10:37:49 pm"
To: Peter Eisentraut <peter_e@gmx.net>
Date: Sat, 13 Nov 1999 11:35:47 -0500 (EST)
CC: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Peter, I no longer see the pg_description descriptions when using the
\do, \df, and \dT commands.

The commands are much less useful without the descriptions. Seems \dd
with a string is much smarter, and pulls descriptions based on string
matching.

Interesting that \dd shows descriptions of everything.

Not sure how to recommend you change this. The new \df and \do
displays are much clearer without the descriptions. It seems \df and
\do show additional information about argument types and return values,
while \dd shows comments.

Maybe just add descriptions to \dT, and suggest people use \dd to get
info about specific operators of functions? But that kind of messes the
clarity of using \dd for descriptions.

I am stumped. Maybe it can't be improved.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Sat Nov 13 12:16:19 1999
Received: from finch-post-11.mail.demon.net (finch-post-11.mail.demon.net
[194.217.242.39]) by hub.org (8.9.3/8.9.3) with ESMTP id MAA80940
for <pgsql-hackers@postgresql.org>;
Sat, 13 Nov 1999 12:15:43 -0500 (EST)
(envelope-from emkxp01@mtcc.demon.co.uk)
Received: from mtcc.demon.co.uk ([158.152.183.103])
by finch-post-11.mail.demon.net with esmtp (Exim 2.12 #1)
id 11mgm3-000P0j-0B; Sat, 13 Nov 1999 17:15:39 +0000
Received: from mtcc by mtcc.demon.co.uk (8.9.1b+Sun/SMI-SVR4)
id RAA16056; Sat, 13 Nov 1999 17:15:28 GMT
Message-Id: <199911131715.RAA16056@mtcc.demon.co.uk>
Date: Sat, 13 Nov 1999 17:15:28 +0000 (GMT)
From: Keith Parks <emkxp01@mtcc.demon.co.uk>
Reply-To: Keith Parks <emkxp01@mtcc.demon.co.uk>
Subject: Re: [HACKERS] translate function (BUG?)
To: pgsql-hackers@postgresql.org, ramirez@doc.mssm.edu
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: yG7jW03obvAJZUF/dUwfdQ==
X-Mailer: dtmail 1.3.0 CDE Version 1.3 SunOS 5.7 sun4m sparc

What happened with this?

I think I saw a response from Bruce but it wasn't fixed and
doesn't appear in the current CVS of V7.

Keith.

From: Edwin Ramirez <ramirez@doc.mssm.edu>
Hello all,

I was looking at the translate function and I think that it does not
behave quite right. I modified the translate function in
oracle_compat.c (included below) to make work more like its Oracle
counterpart. It seems to work but it returns the following message:
NOTICE: PortalHeapMemoryFree: 0x8241fcc not in alloc set!

Below are the Oracle and Postgres session transcripts.

select translate('edwin', 'wi', 'af') from dual;
ORACLE:
TRANS
-----
edafn
1 row selected.

POSTGRES
translate
---------
edain
(1 row)

<snip>

From bouncefilter Sat Nov 13 12:16:19 1999
Received: from Radha.DoCS.UU.SE (root@Radha.DoCS.UU.SE [130.238.9.99])
by hub.org (8.9.3/8.9.3) with SMTP id MAA80971
for <pgsql-hackers@postgreSQL.org>;
Sat, 13 Nov 1999 12:15:57 -0500 (EST)
(envelope-from e99re41@DoCS.UU.SE)
Received: from Val.DoCS.UU.SE (e99re41@Val.DoCS.UU.SE [130.238.9.94]) by
Radha.DoCS.UU.SE (8.6.12/8.6.12) with ESMTP id SAA05318;
Sat, 13 Nov 1999 18:15:55 +0100
Received: from localhost (e99re41@localhost) by Val.DoCS.UU.SE (8.6.12/8.6.12)
with SMTP id SAA03140; Sat, 13 Nov 1999 18:15:53 +0100
X-Authentication-Warning: Val.DoCS.UU.SE: e99re41 owned process doing -bs
Date: Sat, 13 Nov 1999 18:15:53 +0100 (MET)
From: Peter Eisentraut <e99re41@DoCS.UU.SE>
Reply-To: Peter Eisentraut <peter_e@gmx.net>
To: Bruce Momjian <maillist@candle.pha.pa.us>
cc: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Subject: Re: Failure to show descriptions in \df command
In-Reply-To: <199911131635.LAA24800@candle.pha.pa.us>
Message-ID: <Pine.GSO.4.02A.9911131804490.606-100000@Val.DoCS.UU.SE>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

You can turn the descriptions on by typing \set description on (or \set
description foo, or whatever, as long as it's something), for example, in
your ~/.psqlrc (or in your .psqlrc-7.0.0 if you don't want to interfere
with the current version).

The reason for having descriptions off by default was that in a number of
views (I recall functions and operators), they don't fit on the screen
very nicely. On the other hand, the \dd command always shows descriptions,
because it's sort of the built-in manual, but it doesn't show anything
else (argument types, etc.).

Read the fine (SGML) manual ;)

-Peter

On Sat, 13 Nov 1999, Bruce Momjian wrote:

Peter, I no longer see the pg_description descriptions when using the
\do, \df, and \dT commands.

The commands are much less useful without the descriptions. Seems \dd
with a string is much smarter, and pulls descriptions based on string
matching.

Interesting that \dd shows descriptions of everything.

Not sure how to recommend you change this. The new \df and \do
displays are much clearer without the descriptions. It seems \df and
\do show additional information about argument types and return values,
while \dd shows comments.

Maybe just add descriptions to \dT, and suggest people use \dd to get
info about specific operators of functions? But that kind of messes the
clarity of using \dd for descriptions.

I am stumped. Maybe it can't be improved.

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Sat Nov 13 13:21:20 1999
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id NAA88720
for <pgsql-hackers@postgreSQL.org>;
Sat, 13 Nov 1999 13:21:04 -0500 (EST)
(envelope-from maillist@candle.pha.pa.us)
Received: (from maillist@localhost)
by candle.pha.pa.us (8.9.0/8.9.0) id NAA27752;
Sat, 13 Nov 1999 13:19:19 -0500 (EST)
From: Bruce Momjian <maillist@candle.pha.pa.us>
Message-Id: <199911131819.NAA27752@candle.pha.pa.us>
Subject: Re: [HACKERS] Re: Failure to show descriptions in \df command
In-Reply-To: <Pine.GSO.4.02A.9911131804490.606-100000@Val.DoCS.UU.SE> from
Peter Eisentraut at "Nov 13, 1999 06:15:53 pm"
To: Peter Eisentraut <peter_e@gmx.net>
Date: Sat, 13 Nov 1999 13:19:19 -0500 (EST)
CC: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

You can turn the descriptions on by typing \set description on (or \set
description foo, or whatever, as long as it's something), for example, in
your ~/.psqlrc (or in your .psqlrc-7.0.0 if you don't want to interfere
with the current version).

OK. I noticed that the existance of an .psqlrc file causes an extra
newline to be printed on startup before the first prompt. Is that
intentional?

The reason for having descriptions off by default was that in a number of
views (I recall functions and operators), they don't fit on the screen
very nicely. On the other hand, the \dd command always shows descriptions,
because it's sort of the built-in manual, but it doesn't show anything
else (argument types, etc.).

Got it. Yes, much clearer for \df and \do. I noticed that using \set
description on and then using \dT generates an error of:

test=> \set description on
test=> \dT
ERROR: Relation 'p' does not exist

Also, the \set commands don't seem to complain about bad commands:

test=> \set figgle
test=>

Is that intentional?

Read the fine (SGML) manual ;)

That was part of my problem. I hadn't figured out how to generate html
from the sgml ref stuff. I just spent some time and figured out I have
to issue the 'make' command from the upper sgml directory because there
is no Makefile in the sgml/ref directory. I can view them fine now.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Sat Nov 13 16:15:22 1999
Received: from flame.flame.co.za (flame.flame.co.za [160.124.170.1])
by hub.org (8.9.3/8.9.3) with ESMTP id QAA06162
for <pgsql-hackers@postgresql.org>;
Sat, 13 Nov 1999 16:14:51 -0500 (EST)
(envelope-from theo@flame.co.za)
Received: from flame.co.za (flame.flame.co.za [160.124.170.1])
by flame.flame.co.za (8.8.7/8.8.7) with ESMTP id XAA27921
for <pgsql-hackers@postgresql.org>; Sat, 13 Nov 1999 23:17:21 +0200
Sender: theo@flame.flame.co.za
Message-ID: <382DD561.4A6E05E7@flame.co.za>
Date: Sat, 13 Nov 1999 23:17:21 +0200
From: Theo Kramer <theo@flame.co.za>
Organization: Flame Computing Enterprises
X-Mailer: Mozilla 4.5 [en] (X11; I; Linux 2.0.35 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: pgsql-hackers@postgresql.org
Subject: My bits moved right off the end of the world...
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

This message appeared from the backend when running vacuum
analyze on our live data. I remember seeing a posting
regarding this sometime back but no longer have this message.

The version of postgres is 6.5.1 on RH6. There is more than
sufficient disk space left, and postmaster does not seem
to be running out of memory.

Any help would be much appreciated.

--------
Regards
Theo

From bouncefilter Sat Nov 13 16:26:22 1999
Received: from Radha.DoCS.UU.SE (root@Radha.DoCS.UU.SE [130.238.9.99])
by hub.org (8.9.3/8.9.3) with SMTP id QAA09093
for <pgsql-hackers@postgreSQL.org>;
Sat, 13 Nov 1999 16:26:01 -0500 (EST)
(envelope-from e99re41@DoCS.UU.SE)
Received: from Val.DoCS.UU.SE (e99re41@Val.DoCS.UU.SE [130.238.9.94]) by
Radha.DoCS.UU.SE (8.6.12/8.6.12) with ESMTP id WAA19120;
Sat, 13 Nov 1999 22:25:57 +0100
Received: from localhost (e99re41@localhost) by Val.DoCS.UU.SE (8.6.12/8.6.12)
with SMTP id WAA03486; Sat, 13 Nov 1999 22:25:56 +0100
X-Authentication-Warning: Val.DoCS.UU.SE: e99re41 owned process doing -bs
Date: Sat, 13 Nov 1999 22:25:55 +0100 (MET)
From: Peter Eisentraut <e99re41@DoCS.UU.SE>
Reply-To: Peter Eisentraut <peter_e@gmx.net>
To: Theo Kramer <theo@flame.co.za>
cc: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] My bits moved right off the end of the world...
In-Reply-To: <382DD561.4A6E05E7@flame.co.za>
Message-ID: <Pine.GSO.4.02A.9911132223590.606-100000@Val.DoCS.UU.SE>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

I recall that this was due to a corrupted B-Tree index. Try dropping and
rebuilding that, if you have one.

I also recall that Bruce, this being his "favourite error message", wanted
to make it "show it more often", so perhaps it now comes up in different
situations as well. ;)

-Peter

On Sat, 13 Nov 1999, Theo Kramer wrote:

This message appeared from the backend when running vacuum
analyze on our live data. I remember seeing a posting
regarding this sometime back but no longer have this message.

The version of postgres is 6.5.1 on RH6. There is more than
sufficient disk space left, and postmaster does not seem
to be running out of memory.

Any help would be much appreciated.

--------
Regards
Theo

************

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Sat Nov 13 17:21:22 1999
Received: from candle.pha.pa.us (maillist@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id RAA15062
for <pgsql-hackers@postgreSQL.org>;
Sat, 13 Nov 1999 17:20:38 -0500 (EST)
(envelope-from maillist@candle.pha.pa.us)
Received: (from maillist@localhost)
by candle.pha.pa.us (8.9.0/8.9.0) id RAA04771;
Sat, 13 Nov 1999 17:19:06 -0500 (EST)
From: Bruce Momjian <maillist@candle.pha.pa.us>
Message-Id: <199911132219.RAA04771@candle.pha.pa.us>
Subject: Re: [HACKERS] My bits moved right off the end of the world...
In-Reply-To: <Pine.GSO.4.02A.9911132223590.606-100000@Val.DoCS.UU.SE> from
Peter Eisentraut at "Nov 13, 1999 10:25:55 pm"
To: Peter Eisentraut <peter_e@gmx.net>
Date: Sat, 13 Nov 1999 17:19:06 -0500 (EST)
CC: Theo Kramer <theo@flame.co.za>, pgsql-hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

I recall that this was due to a corrupted B-Tree index. Try dropping and
rebuilding that, if you have one.

I also recall that Bruce, this being his "favourite error message", wanted
to make it "show it more often", so perhaps it now comes up in different
situations as well. ;)

That's still on my TODO list. :-)

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Sat Nov 13 17:22:22 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id RAA15114
for <pgsql-hackers@postgreSQL.org>;
Sat, 13 Nov 1999 17:21:38 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id RAA01877
for <pgsql-hackers@postgreSQL.org>;
Sat, 13 Nov 1999 17:21:07 -0500 (EST)
To: pgsql-hackers@postgreSQL.org
Subject: Status of sql_help.h
Date: Sat, 13 Nov 1999 17:21:07 -0500
Message-ID: <1874.942531667@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

I see that src/bin/psql/sql_help.h is now generated automatically
from the SGML documentation. This is a Good Thing. But: since
sql_help.h is now a derived file, shouldn't it be removed from the
CVS repository, for the same reasons that we don't keep gram.c
and other derived files in CVS? If we leave it there, it'll generate
a lot of extra update traffic.

The only reason I can see for leaving it in CVS is that if we remove it,
people who pull sources from CVS would need Perl in order to build psql.
(People who download tarballs would *not*, since release_prep updates
sql_help.h along with the other derived files.) That's annoying, but
I think it may not be a fatal objection. Most hackers are probably
more likely to already have Perl than to already have bison or flex...

I thought about suggesting that create_help.pl be rewritten in some
"more portable" fashion such as an awk script. But really, if you
consider non-Unix platforms, Perl is more portable than awk or any
other likely alternative. (It might be worthwhile to remove the one
or two unnecessary Perl-5-isms in the script, so that it will run on
Perl 4 if that's what's available.)

Comments? Anyone feel that we really can't expect users of the CVS
repository to have Perl?

regards, tom lane

PS: "make distclean" should probably not remove sql_help.h, for the
same reasons that we don't remove gram.c --- it *is* a distributed
file, and a particular user might not have the tools to rebuild it.
This is true whether or not we leave it in CVS.

From bouncefilter Sat Nov 13 17:24:23 1999
Received: from mail1.radix.net (mail1.radix.net [207.192.128.31])
by hub.org (8.9.3/8.9.3) with ESMTP id RAA15250
for <pgsql-hackers@hub.org>; Sat, 13 Nov 1999 17:23:29 -0500 (EST)
(envelope-from cobrien@saltmine.radix.net)
Received: from saltmine.radix.net (saltmine.radix.net [207.192.128.40])
by mail1.radix.net (8.9.3/8.9.3) with ESMTP id RAA00597
for <pgsql-hackers@hub.org>; Sat, 13 Nov 1999 17:23:18 -0500 (EST)
Received: (from cobrien@localhost)
by saltmine.radix.net (8.8.7/8.8.7) id RAA05987
for pgsql-hackers@hub.org; Sat, 13 Nov 1999 17:23:15 -0500 (EST)
From: "Cary O'Brien" <cobrien@Radix.Net>
Message-Id: <199911132223.RAA05987@saltmine.radix.net>
Subject: Compression in LO and other fields
To: pgsql-hackers@hub.org
Date: Sat, 13 Nov 1999 17:23:15 -0500 (EST)
X-Mailer: ELM [version 2.4ME+ PL48 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Some observations about compression (sorry if this is obvious):

Compression won't work unless there is redundancy in the information
being compressed. In a normal [1]Whatever the heck that is. database, there may be a little
redundancy in each row. The real redundancy will be when you group
rows together. I.E. in it is quite likely that many rows will have
identical or similar values for some columns. It may be worth
considering compressing each storage block[2]My lack of understanding of PostgreSQL storage manager internals is showing., in the hopes that each
block will contain several rows, and that the rows will have some
similar (redundant) information. Then compression will work. Just
compressing one column of one row may work in some situations, but
in general, it won't.

Once a while ago we implemented a storage system where we grouped
records (about 1k in size) into blocks (about 16k in size) and
compressed each block individually. To get at a record you had to
decompress the block it was in, and skip to the desired offset. Even
though the system had to throw away a lot [3]Half a block on average. of data for each
retrieval, the system turned out to be be both economical of disk[4]Optical disk in this particular app.
storage and fast at retrieval. For this app we found that compression
rates increased as we got up to about 16 records, and then flattend
off. I.E. compressing 16 records used less space than compressing 4
groups of 4, but compressing 256 records together used as much space
as compressing 16 groups of 16.

The point?

I wish I understood the PostgreSQL backend storage algorithms better,
but it seems that the combination of a Tuneable block size [5]This is an option now, right? But is it compile time, or run-time? and is it postmaster-wide, per database? I *try* to read the pgsql-hackers-digest but there is just too much going on! and
an option to compress individual storage blocks [6]Can this work with the existing data file structure? might be worth
looking at.

-- cary

[1]: Whatever the heck that is.

[2]: My lack of understanding of PostgreSQL storage manager internals is showing.
internals is showing.

[3]: Half a block on average.

[4]: Optical disk in this particular app.

[5]: This is an option now, right? But is it compile time, or run-time? and is it postmaster-wide, per database? I *try* to read the pgsql-hackers-digest but there is just too much going on!
and is it postmaster-wide, per database? I *try* to read the pgsql-hackers-digest
but there is just too much going on!

[6]: Can this work with the existing data file structure?

From bouncefilter Sat Nov 13 20:18:24 1999
Received: from mailhub.southeast.net (root@mailhub.southeast.net
[207.98.192.83]) by hub.org (8.9.3/8.9.3) with ESMTP id UAA36341
for <pgsql-hackers@postgreSQL.org>;
Sat, 13 Nov 1999 20:17:54 -0500 (EST)
(envelope-from mtsinc@southeast.net)
Received: from southeast.net (mail.mousetech.com [216.199.14.19])
by mailhub.southeast.net (8.8.5/8.8.5) with ESMTP id UAA10484;
Sat, 13 Nov 1999 20:17:45 -0500 (EST)
Message-ID: <382E0926.2C3F536E@southeast.net>
Date: Sat, 13 Nov 1999 19:58:14 -0500
From: Tim Holloway <mtsinc@southeast.net>
X-Mailer: Mozilla 4.7 [en] (Win98; I)
X-Accept-Language: en
MIME-Version: 1.0
To: Tom Lane <tgl@sss.pgh.pa.us>
CC: pgsql-hackers <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Thread-safe queueing?
References: <431.942506061@sss.pgh.pa.us>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Tom Lane wrote:

Tim Holloway <mtsinc@southeast.net> writes:

I need to create a cross-process producer/consumer data queue
(e.g. singly-linked list). That is - Processes A, B, and C add nodes
to a controlled list and process D removes them. Not sure if the
creation of the nodes would be best done by the producers or
consumers, but destruction would have to be done by the consumer, as
the producers don't wait for processing. For optimal results, the
consumer process should sleep until item(s) are added to its queue.

Query: within the existing backend framework, what's the best way to
accomplish this?

More context, please. What are you trying to accomplish? Is this
really a communication path between backends (and if so, what backend
code needs it?), or are you trying to set up a queue between SQL
clients? How much data might need to be in the queue at one time?

regards, tom lane

This is for the logging subsystem I'm developing. The backends call pg_log(),
which is like elog(), except that the message is a resource ID + any parameters
in order to support locales and custom message formatting. These ID+parameter
packets are then pipelined down to the logging channels via the log engine to
be formatted and output according to rules in the configuration file.

I *think* that the log engine should be a distinct process. I'm not sure I can
trust the output not to come out sliced and diced if each backend can run the engine
directly -- and for that matter, I see problems if the engine is reconfigured on the
fly owing to the need for each backend to replicate the configuration process (among
other things). The basic singly-linked list component is all I need to handle the
FIFO, but obviously I need guards to preserve its integrity. As to the amount of data
involved, I sincerely hope the queue would stay pretty shallow!

I have the configuration parser and logging engine operational, so the last
significant hurdle is making sure that A) the data to be logged is
accessable/addressable by the engine, and B) that the process runs in the
proper sequence. A description of what it all will look like is now online at http://postgres.mousetech.com/index.html
(with apologies for the ugly formatting).

Thanks,

TIm Holloway

From bouncefilter Sat Nov 13 21:30:25 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id VAA43683
for <pgsql-hackers@postgreSQL.org>;
Sat, 13 Nov 1999 21:29:57 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id VAA02229;
Sat, 13 Nov 1999 21:28:55 -0500 (EST)
To: Tim Holloway <mtsinc@southeast.net>
cc: pgsql-hackers <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Thread-safe queueing?
In-reply-to: Your message of Sat, 13 Nov 1999 19:58:14 -0500
<382E0926.2C3F536E@southeast.net>
Date: Sat, 13 Nov 1999 21:28:54 -0500
Message-ID: <2227.942546534@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Tim Holloway <mtsinc@southeast.net> writes:

Tom Lane wrote:

More context, please.

This is for the logging subsystem I'm developing. The backends call
pg_log(), which is like elog(), except that the message is a resource
ID + any parameters in order to support locales and custom message
formatting. These ID+parameter packets are then pipelined down to the
logging channels via the log engine to be formatted and output
according to rules in the configuration file.

OK. You probably want something roughly comparable to the shared-inval
message queue --- see include/storage/sinvaladt.h and
backend/storage/ipc/sinvaladt.c. That's more complex than your problem
in one way (the sinval queue must be read by all N backends, not just
one process) but simpler in another (we can shoehorn all SI messages
into a fixed-size structure; is that practical for log data?). Anyway,
a queue structure in shared memory protected by spinlocks is what you
want, and sinval is about the closest thing we have to that at the
moment.

I *think* that the log engine should be a distinct process.

Probably so, if you use a shared-memory queue. Shared memory is a
finite resource; AFAIK it's not practical to enlarge it on-the-fly.
So you will have to set a maximum queue size --- either a compile-time
constant, or at best a value chosen at postmaster start time. This
implies that there will be scenarios where backends are waiting for room
to be made in the log queue. If the queue emptier is a separate process
then those waits can't turn into deadlocks. (sinval works around the
memory-overflow problem with a special "reset" mechanism, but that
doesn't seem appropriate for logging.)

Alternatively, you could forget about a queue per se, and just allow
each backend to execute the sending of its own log messages, using
a spinlock in shared memory to prevent concurrent issuance of log
messages on channels where that's a problem. That might be the
simplest and most robust approach.

regards, tom lane

From bouncefilter Sat Nov 13 22:13:25 1999
Received: from candle.pha.pa.us (maillist@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA48061
for <pgsql-hackers@postgreSQL.org>;
Sat, 13 Nov 1999 22:12:38 -0500 (EST)
(envelope-from maillist@candle.pha.pa.us)
Received: (from maillist@localhost)
by candle.pha.pa.us (8.9.0/8.9.0) id WAA10351;
Sat, 13 Nov 1999 22:11:36 -0500 (EST)
From: Bruce Momjian <maillist@candle.pha.pa.us>
Message-Id: <199911140311.WAA10351@candle.pha.pa.us>
Subject: Re: [HACKERS] Thread-safe queueing?
In-Reply-To: <2227.942546534@sss.pgh.pa.us> from Tom Lane at "Nov 13,
1999 09:28:54 pm"
To: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sat, 13 Nov 1999 22:11:36 -0500 (EST)
CC: Tim Holloway <mtsinc@southeast.net>,
pgsql-hackers <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Alternatively, you could forget about a queue per se, and just allow
each backend to execute the sending of its own log messages, using
a spinlock in shared memory to prevent concurrent issuance of log
messages on channels where that's a problem. That might be the
simplest and most robust approach.

Hold on. Unix guarantees all write() calls are atomic, so no one gets
in between that write. Why not just collect the output into one buffer
in the backend, and blast the entire buffer in one write() to the log
file.

I don't think there is any way another backend could mess that up.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Sat Nov 13 22:50:26 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA52408
for <pgsql-hackers@postgreSQL.org>;
Sat, 13 Nov 1999 22:49:49 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id WAA02326;
Sat, 13 Nov 1999 22:48:13 -0500 (EST)
To: Bruce Momjian <maillist@candle.pha.pa.us>
cc: Tim Holloway <mtsinc@southeast.net>,
pgsql-hackers <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Thread-safe queueing?
In-reply-to: Your message of Sat, 13 Nov 1999 22:11:36 -0500 (EST)
<199911140311.WAA10351@candle.pha.pa.us>
Date: Sat, 13 Nov 1999 22:48:13 -0500
Message-ID: <2324.942551293@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Bruce Momjian <maillist@candle.pha.pa.us> writes:

Alternatively, you could forget about a queue per se, and just allow
each backend to execute the sending of its own log messages, using
a spinlock in shared memory to prevent concurrent issuance of log
messages on channels where that's a problem. That might be the
simplest and most robust approach.

Hold on. Unix guarantees all write() calls are atomic, so no one gets
in between that write.

Actually, I didn't say that I *believed* there were any channel types
where such an interlock is essential ;-). I just said that spinlocking
is a solution if the problem comes up.

Tim mentioned on-the-fly reconfiguration of logging as an area that
might need interlocking, and I'm more prepared to believe that.
Still, we seem to be getting on just fine with each backend responding
independently to reconfiguration of the pg_options values. So I'd be
inclined to build it that way, and wait for evidence of a performance
problem before spending effort to make it smarter.

Which I guess is Bruce's point also...

regards, tom lane

From bouncefilter Sun Nov 14 01:17:28 1999
Received: from flame.flame.co.za (flame.flame.co.za [160.124.170.1])
by hub.org (8.9.3/8.9.3) with ESMTP id BAA74297
for <pgsql-hackers@postgreSQL.org>;
Sun, 14 Nov 1999 01:16:41 -0500 (EST)
(envelope-from theo@flame.co.za)
Received: from flame.co.za (flame.flame.co.za [160.124.170.1])
by flame.flame.co.za (8.8.7/8.8.7) with ESMTP id IAA31095
for <pgsql-hackers@postgreSQL.org>; Sun, 14 Nov 1999 08:19:22 +0200
Sender: theo@flame.flame.co.za
Message-ID: <382E546A.558777C9@flame.co.za>
Date: Sun, 14 Nov 1999 08:19:22 +0200
From: Theo Kramer <theo@flame.co.za>
Organization: Flame Computing Enterprises
X-Mailer: Mozilla 4.5 [en] (X11; I; Linux 2.0.35 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] My bits moved right off the end of the world...
References: <Pine.GSO.4.02A.9911132223590.606-100000@Val.DoCS.UU.SE>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Peter Eisentraut wrote:

I recall that this was due to a corrupted B-Tree index. Try dropping and
rebuilding that, if you have one.

Thanks Peter, that did the trick.

Any chance of adding the following to the FAQ

4.24) What is the meaning of 'my bits moved right off the end of the world'

This message may appear in the backend log and may be due to a possibly
corrupt index.

It may be preceded (by several minutes) with a notice to the client such as

NOTICE: Index my_idx: NUMBER OF INDEX' TUPLES (78933) IS NOT THE SAME AS HEAP
(78931)

The message may occur when running a 'vacuum analyze' with the backend
terminating abnormally.

[Other explanations...]

The postmaster must be started without the '-S' option for this
notice to appear.

--------
Regards
Theo

From bouncefilter Sun Nov 14 03:30:29 1999
Received: from loopy.berkhirt.com (loopy.berkhirt.com [209.220.85.54])
by hub.org (8.9.3/8.9.3) with ESMTP id DAA90110
for <pgsql-hackers@postgreSQL.org>;
Sun, 14 Nov 1999 03:29:37 -0500 (EST)
(envelope-from bhirt@loopy.berkhirt.com)
Received: (from bhirt@localhost)
by loopy.berkhirt.com (8.9.3/8.8.7) id CAA29789;
Sun, 14 Nov 1999 02:29:07 -0600
Date: Sun, 14 Nov 1999 02:29:06 -0600
From: Brian Hirt <bhirt@mobygames.com>
To: pgsql-hackers@postgreSQL.org
Cc: bhirt@loopy.berkhirt.com
Subject: pg_dump - rebuilding indexes
Message-ID: <19991114022906.A29291@loopy.berkhirt.com>
Reply-To: bhirt@mobygames.com
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Mailer: Mutt 0.95.4us
X-PC-Gaming: http://www.mobygames.com/

Hi,

A few time in the past, my indexes have become corrupted somehow
and a vacuum analyze will cause the backend to dump core. I've
seen other people post similiar problems on hackers and admin. All
of the suggestions seem to be dumping the database and reloading it
to have the indexes rebuilt or vacuuming the tables one by one until
the system crashes and then drop the indexes for that table and
recreate them from your DDL scripts. Well, this happened to me again,
so I searched the list looking for any way to automatically rebuild the
indexes but didn't manage to find any. It got me thinking, pg_dump
already dumps index creates, all we need to do is modify it to dump
index drops and prevent all of the other schema and data from being
dumped.

I got into the code and quickly learned about the -c option (creates
drops statements) that wasn't talked about on my outdated manpage.
Doh! So now the problem is even easier -- I only have to suppress the
dumping of everying except indexes and turn on the -c flag. I added
an option called -r (rebuild indexes). It turns on the dropSchema,
schemaOnly and dataOnly flags which in essence causes pg_dump to
do nothing. An extra snippet of code checks to see if indexes should
be rebuilt and dumps the index creates. It's a real hack, but it
suites my needs.

Now, whenever I want to rebuild my indexes I can just type:

pg_dump -r mydatabase | psql mydatabase

If something else already exists -- oops maybe we could add it to the
faq. I actually would have liked to implement the code differently, but
the current code isn't very condusive to a more elegant solution and I din't
want to put much time into something that might be rejected by the source
code maintainers. If this is rejected, I only wasted 10 minutes. Ideally,
what I would like is a flag that allows you to specify the types to be
dumped. This way, it would be flexible enough to allow you to dump any
combination of types. If you only wanted to dump trigger schema you could.
If you wanted sequences and indexes, no problem. Something like:

pg_dump --dump-types "type trigger aggregate"

Anyway,

diff -u ./pg_dump.c ../../../../postgresql-6.5.3/src/bin/pg_dump/pg_dump.c
--- ./pg_dump.c	Sun Nov 14 01:41:05 1999
+++ ../../../../postgresql-6.5.3/src/bin/pg_dump/pg_dump.c	Thu Sep 23 14:13:49 1999
@@ -112,7 +112,6 @@
 PGconn	   *g_conn;				/* the database connection */

 bool		force_quotes;		/* User wants to suppress double-quotes */
-bool		rebuildIndexes;		/* dump DDL for index rebuilds   */
 bool		dumpData;			/* dump data using proper insert strings */
 bool		attrNames;			/* put attr names into insert strings */
 bool		schemaOnly;
@@ -543,7 +542,6 @@

g_verbose = false;
force_quotes = true;
- rebuildIndexes = false;
dropSchema = false;

strcpy(g_comment_start, "-- ");
@@ -554,9 +552,7 @@

progname = *argv;

-        /* Get the arguments for the command line via getopts cycle through
-           each option, setting the appropriate flags as necessary */
-	while ((c = getopt(argc, argv, "acdDf:h:nNop:rst:uvxz")) != EOF)
+	while ((c = getopt(argc, argv, "acdDf:h:nNop:st:uvxz")) != EOF)
 	{
 		switch (c)
 		{
@@ -594,17 +590,6 @@
 			case 'p':			/* server port */
 				pgport = optarg;
 				break;
-			case 'r':			/* rebuild indexes */
-				rebuildIndexes = true;  /* forces only indexes to be dumped        */
-				dropSchema = true;      /* causes drop statements to be created    */
-
-				/* Setting data only to true, causes the dumpSchema() to dump 
-				   schema to a NULL file handle, and setting schemaOnly to true
-				   prevents dumpClasses() from dumping the data -- it's a HACK */
-				schemaOnly = true;      
-				dataOnly = true;      
-
-				break;
 			case 's':			/* dump schema only */
 				schemaOnly = true;
 				break;
@@ -765,13 +750,8 @@
 	if (!schemaOnly)
 		dumpClasses(tblinfo, numTables, g_fout, tablename, oids);

-
-	/* dump indexes and triggers at the end for performance */
-	if (rebuildIndexes)
-	{
-		dumpSchemaIdx(g_fout, tablename, tblinfo, numTables);
-	}
-	else if (!dataOnly)				
+	if (!dataOnly)				/* dump indexes and triggers at the end
+								 * for performance */
 	{
 		dumpSchemaIdx(g_fout, tablename, tblinfo, numTables);
 		dumpTriggers(g_fout, tablename, tblinfo, numTables);

--
The world's most ambitious and comprehensive PC game database project.

http://www.mobygames.com

From bouncefilter Sun Nov 14 11:02:35 1999
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA41863
for <pgsql-hackers@postgreSQL.org>;
Sun, 14 Nov 1999 11:01:59 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
KAA28601;
Sun, 14 Nov 1999 10:52:33 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911141552.KAA28601@candle.pha.pa.us>
Subject: Re: [HACKERS] My bits moved right off the end of the world...
In-Reply-To: <382E546A.558777C9@flame.co.za> from Theo Kramer at "Nov 14, 1999
08:19:22 am"
To: Theo Kramer <theo@flame.co.za>
Date: Sun, 14 Nov 1999 10:52:33 -0500 (EST)
CC: pgsql-hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Peter Eisentraut wrote:

I recall that this was due to a corrupted B-Tree index. Try dropping and
rebuilding that, if you have one.

Thanks Peter, that did the trick.

Any chance of adding the following to the FAQ

4.24) What is the meaning of 'my bits moved right off the end of the world'

With specific messages like this one, it is best to put the fix right in
the error message.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Sun Nov 14 11:20:35 1999
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA44117
for <pgsql-hackers@postgreSQL.org>;
Sun, 14 Nov 1999 11:20:20 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
LAA29490;
Sun, 14 Nov 1999 11:19:02 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911141619.LAA29490@candle.pha.pa.us>
Subject: Re: [HACKERS] My bits moved right off the end of the world...
In-Reply-To: <199911141552.KAA28601@candle.pha.pa.us> from Bruce Momjian at
"Nov 14, 1999 10:52:33 am"
To: Bruce Momjian <pgman@candle.pha.pa.us>
Date: Sun, 14 Nov 1999 11:19:02 -0500 (EST)
CC: Theo Kramer <theo@flame.co.za>, pgsql-hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Peter Eisentraut wrote:

I recall that this was due to a corrupted B-Tree index. Try dropping and
rebuilding that, if you have one.

Thanks Peter, that did the trick.

Any chance of adding the following to the FAQ

4.24) What is the meaning of 'my bits moved right off the end of the world'

With specific messages like this one, it is best to put the fix right in
the error message.

With great pain, I have added an additional sentence to the error
message stating to try and recreate index.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Sun Nov 14 12:26:36 1999
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id MAA50491
for <pgsql-hackers@postgreSQL.org>;
Sun, 14 Nov 1999 12:26:21 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
MAA00558;
Sun, 14 Nov 1999 12:24:19 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911141724.MAA00558@candle.pha.pa.us>
Subject: Re: [HACKERS] My bits moved right off the end of the world...
In-Reply-To: <382E546A.558777C9@flame.co.za> from Theo Kramer at "Nov 14, 1999
08:19:22 am"
To: Theo Kramer <theo@flame.co.za>
Date: Sun, 14 Nov 1999 12:24:18 -0500 (EST)
CC: pgsql-hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Peter Eisentraut wrote:

I recall that this was due to a corrupted B-Tree index. Try dropping and
rebuilding that, if you have one.

Thanks Peter, that did the trick.

Any chance of adding the following to the FAQ

4.24) What is the meaning of 'my bits moved right off the end of the world'

This message may appear in the backend log and may be due to a possibly
corrupt index.

It may be preceded (by several minutes) with a notice to the client such as

NOTICE: Index my_idx: NUMBER OF INDEX' TUPLES (78933) IS NOT THE SAME AS HEAP
(78931)

I have added a mention when this message appears to try recreating the
index.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Sun Nov 14 12:16:36 1999
Received: (from news@localhost) by hub.org (8.9.3/8.9.3) id MAA49533
for pgsql-hackers@postgresql.org; Sun, 14 Nov 1999 12:16:07 -0500 (EST)
(envelope-from news)
X-Authentication-Warning: hub.org: news set sender to <news> using -f
From: "link2" <admin@link2casino.com>
X-Newsgroups: comp.databases.postgresql.hackers
Subject: Autocount
Date: Sun, 14 Nov 1999 13:17:31 -0500
Organization: Interpacket Group Inc.
Lines: 59
Message-ID: <80mqnq$2mhp$1@news.interpacket.net>
Mime-Version: 1.0
Content-Type: multipart/alternative;
boundary="----=_NextPart_000_0083_01BF2EA2.9B1CF890"
X-Trace: news.interpacket.net 942599738 88633 207.42.132.90 (14 Nov 1999
17:15:38 GMT)
X-Complaints-To: usenet@news.interpacket.net
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 5.00.2314.1300
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
To: pgsql-hackers@postgresql.org

This is a multi-part message in MIME format.

------=_NextPart_000_0083_01BF2EA2.9B1CF890
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Hi all
I'm very new to SQL server I can do allot of things in access database =
but I'm moving to SQL. isen that grate?

What i m looking for is witch one of the datatype in table design view I =
can used for an autocount like in access?

Please reply to=20
kenneth@internetcasino.ag

Webmaster
webmaster@link2casino.com
http://www.link2casino.com
Auto

------=_NextPart_000_0083_01BF2EA2.9B1CF890
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content=3D"text/html; charset=3Diso-8859-1" =
http-equiv=3DContent-Type>
<META content=3D"MSHTML 5.00.2314.1000" name=3DGENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY bgColor=3D#ffffff=20
onunload=3D"hotpick =3D =
window.open('http://www.link2casino.com/index.htm&#39;, 'ossponsor', =
'toolbar=3Dyes,location=3Dyes,directories=3Dyes,status=3Dyes,scrollbars=3D=
yes,menubar=3Dno,resizable=3Dyes')">
<DIV><FONT size=3D2>Hi all</FONT></DIV>
<DIV><FONT size=3D2>I'm very new to SQL server I can do allot of things =
in access=20
database but I'm moving to SQL. isen that grate?</FONT></DIV>
<DIV>&nbsp;</DIV>
<DIV><FONT size=3D2>What i m looking for is witch one of the datatype in =
table=20
design view I can used for an autocount like in access?</FONT></DIV>
<DIV><FONT size=3D2><BR>Please reply to </FONT></DIV>
<DIV><FONT size=3D2><A=20
href=3D"mailto:kennth@internetcasino.ag">kenneth@internetcasino.ag</A></F=
ONT></DIV>
<DIV><FONT size=3D2><BR>Webmaster<BR><A=20
href=3D"mailto:webmaster@link2casino.com">webmaster@link2casino.com</A><B=
R><A=20
href=3D"http://www.link2casino.com&quot;&gt;http://www.link2casino.com&lt;/A&gt;&lt;/FONT&gt;=
</DIV>Auto</BODY></HTML>

------=_NextPart_000_0083_01BF2EA2.9B1CF890--

From bouncefilter Mon Nov 15 17:56:56 1999
Received: from merganser.its.uu.se (merganser.its.uu.se [130.238.6.236])
by hub.org (8.9.3/8.9.3) with ESMTP id RAA82354
for <pgsql-hackers@postgresql.org>;
Mon, 15 Nov 1999 17:55:58 -0500 (EST)
(envelope-from peter@peter-e.yi.org)
Received: from regulus.its.uu.se ([130.238.7.19]:62579 "EHLO peter-e.yi.org")
by merganser.its.uu.se with ESMTP id <S.sA6xt221581>;
Mon, 15 Nov 1999 23:55:53 +0100
Received: from peter (helo=localhost)
by peter-e.yi.org with local-esmtp (Exim 3.02 #2)
id 11n4Tq-0000Fg-00; Sun, 14 Nov 1999 19:34:26 +0100
Date: Sun, 14 Nov 1999 19:34:26 +0100 (CET)
From: Peter Eisentraut <peter_e@gmx.net>
To: Tom Lane <tgl@sss.pgh.pa.us>
cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Status of sql_help.h
In-Reply-To: <1874.942531667@sss.pgh.pa.us>
Message-ID: <Pine.LNX.4.20.9911141850340.797-100000@peter-e.yi.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: Peter Eisentraut <peter@peter-e.yi.org>

On 1999-11-13, Tom Lane mentioned:

sql_help.h is now a derived file, shouldn't it be removed from the
CVS repository, for the same reasons that we don't keep gram.c

You're the CVS guys. Whatever works.

I thought about suggesting that create_help.pl be rewritten in some
"more portable" fashion such as an awk script. But really, if you

If I'm supposed to maintain this (or do *anything* with it), it can't be
awk. No reason to make a step backwards to accomodate an undocumented
portability problem. I would argue that a lot more people are familiar
with Perl and can read that script than an awk alternative. We don't write
strict (-pedantic) ANSI C code either for the sake of portability.

other likely alternative. (It might be worthwhile to remove the one
or two unnecessary Perl-5-isms in the script, so that it will run on
Perl 4 if that's what's available.)

On a quick look I couldn't find a useful listing of things new in Perl 5
or some way to test for Perl 4 compatibility. Shortly, I don't know what a
Perl-5-ism is and I really don't feel like finding out either. However, if
someone is inclined to fix those things if it doesn't make it all ugly, be
my guest.

Comments? Anyone feel that we really can't expect users of the CVS
repository to have Perl?

If you don't have Perl, the question is really: Do you have CVS? Do you
have rlogin? Do you have networking support in your kernel? Do you have a
computer?

Seriously, I'd suggest that we wait for a documented problem before taking
unnecessary steps.

Hmm, interesting. From the GNU Makefile standards:

"The `configure' script and the Makefile rules for building and
installation should not use any utilities directly except these:

cat cmp cp diff echo egrep expr false grep install-info
ln ls mkdir mv pwd rm rmdir sed sleep sort tar test touch true"

No awk there either.

PS: "make distclean" should probably not remove sql_help.h, for the
same reasons that we don't remove gram.c --- it *is* a distributed
file, and a particular user might not have the tools to rebuild it.

That was my bad. For some reason I had the idea that "distclean" stood for
"distinctly clean" (really clean). :-\ I'll fix that. Perhaps we ought to
decide on some standard targets. "maintainer-clean" would be the proper
one to use (in GNU, again). It also contains the note:

"... Since these files are normally included in the distribution, we don't
take care to make them easy to reconstruct. If you find you need to
unpack the full distribution again, don't blame us."

Well said.

-Peter

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Sun Nov 14 13:48:37 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id NAA58345
for <pgsql-hackers@postgreSQL.org>;
Sun, 14 Nov 1999 13:48:22 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id NAA07221;
Sun, 14 Nov 1999 13:47:13 -0500 (EST)
To: Frans Van Elsacker <fve@atbib.be>
cc: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] union problem version 6.5.3
In-reply-to: Your message of Fri, 12 Nov 1999 18:58:59 -0500
<28156.942451139@sss.pgh.pa.us>
Date: Sun, 14 Nov 1999 13:47:13 -0500
Message-ID: <7219.942605233@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Hmm, the grammar has

CreateAsStmt: CREATE OptTemp TABLE relation_name OptCreateAs AS SubSelect

and SubSelect doesn't allow unions. This is overly restrictive,

As far as I can tell, it should work to just change the above line in
src/backend/parser/gram.y to

CreateAsStmt: CREATE OptTemp TABLE relation_name OptCreateAs AS SelectStmt

I am doing this in current sources right now. I have not tried it in
REL6_5, but if the problem is getting in your way then give it a try...

regards, tom lane

From bouncefilter Sun Nov 14 14:14:37 1999
Received: from sapphire.albourne.com (root@sapphire.albourne.com
[195.212.241.227]) by hub.org (8.9.3/8.9.3) with ESMTP id OAA61926
for <pgsql-hackers@postgresql.org>;
Sun, 14 Nov 1999 14:14:06 -0500 (EST)
(envelope-from a.joubert@albourne.com)
Received: from albourne.com (akamas.albourne.com [195.212.241.254])
by sapphire.albourne.com (8.9.3/8.9.3/Albourne/CYS/1.6X) with ESMTP id
VAA23310 for <pgsql-hackers@postgresql.org>;
Sun, 14 Nov 1999 21:17:45 +0200 (EET)
Sender: a.joubert@albourne.com
Message-ID: <382F0ACD.81BE2A56@albourne.com>
Date: Sun, 14 Nov 1999 21:17:33 +0200
From: Adriaan Joubert <a.joubert@albourne.com>
X-Mailer: Mozilla 4.61 [en] (X11; U; Linux 2.0.38 i586)
X-Accept-Language: en
MIME-Version: 1.0
To: pgsql-hackers@postgresql.org
Subject: Help with SPI and oids
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hi,

I need to keep track of changes to a set of tables in a generic way.
To do this, I want to keep track of oid's. I'm writing the code using
the SPI interface, but I've hit the following problem: after I have
inserted a tuple into a table from the C routine, I cannot figure out
how to get hold of its oid. I had hope that there may be something in
SPI_tuptable, but there isn't. SPI_processed is 1, and the row is
inserted successfully. Surely there must be some way to get the oid of
the row that was just inserted.

I'd really appreciate any help on this one -- even digging aruond in the
backend code hasn't helped up to now.

Cheers!

Adriaan

From bouncefilter Sun Nov 14 14:24:37 1999
Received: from web2106.mail.yahoo.com (web2106.mail.yahoo.com [128.11.68.250])
by hub.org (8.9.3/8.9.3) with SMTP id OAA63594
for <pgsql-hackers@postgresql.org>;
Sun, 14 Nov 1999 14:24:21 -0500 (EST)
(envelope-from mascarim@yahoo.com)
Message-ID: <19991114192420.15160.rocketmail@web2106.mail.yahoo.com>
Received: from [206.246.185.100] by web2106.mail.yahoo.com;
Sun, 14 Nov 1999 11:24:20 PST
Date: Sun, 14 Nov 1999 11:24:20 -0800 (PST)
From: Mike Mascari <mascarim@yahoo.com>
Subject: Postmaster dies with FATAL 1: ReleaseLruFile: No opened files - no
one can be closed
To: tgl@sss.pgh.pa.us
Cc: pgsql-hackers@postgresql.org
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Hello Tom,

I was hoping you might have some insight on a problem we've encountered
with PostgreSQL 6.5.0 (RedHat 5.2) this morning, since you are the
"file descriptor" king, as it were :-) . The database is backing a website
used by a network of hospitals for materials management and this morning,
the postmaster died with the following appearing in the system log:

Nov 14 11:50:14 emptoris logger:
FATAL 1: ReleaseLruFile: No opened files - no one can be closed

This is the first time this has ever happened. I've had such good luck
with PostgreSQL that I didn't have the postmaster started by inittab.
The number of backends should have been very light today (Sunday) --
only a few ODBC users and an occassional HTTP user, so after the
postmaster exited, the log (I assume these are forked backend complaints)
shows:

Nov 14 11:55:03 emptoris logger:
pq_recvbuf: unexpected EOF on client connection
Nov 14 11:55:03 emptoris logger:
pq_recvbuf: unexpected EOF on client connection
Nov 14 11:55:04 emptoris logger:
pq_flush: send() failed: Broken pipe
Nov 14 11:55:04 emptoris logger:
FATAL: pq_endmessage failed: errno=32

From previous posts, I know you've done a cleanup with respect to

file descriptors, but all I see in the log after 6.5.0 is a 6.5.1 entry:

ACL file descriptor leak fix(Atsushi Ogawa)

Is this a rare occurence or something that might have been fixed between
6.5.0 and 6.5.3? Like I said, this is the first time this has happened and
otherwise has been very robust under much heavier loads -- so much so
that I didn't put the postmaster into inittab for respawning. Its
been working pretty much flawlessly in production for about a year.

Anyways, after starting the postmaster again, I vacuum analyzed the
database, accessed the HTTP application, etc. without problems.

Any info would be greatly appreciated,

Mike Mascari
(mascarim@yahoo.com)

=====

__________________________________________________
Do You Yahoo!?
Bid and sell for free at http://auctions.yahoo.com

From bouncefilter Sun Nov 14 15:40:38 1999
Received: from ra.sai.msu.su (ra.sai.msu.su [158.250.29.2])
by hub.org (8.9.3/8.9.3) with ESMTP id PAA73309
for <pgsql-hackers@postgresql.org>;
Sun, 14 Nov 1999 15:40:10 -0500 (EST) (envelope-from oleg@sai.msu.su)
Received: from ra (ra [158.250.29.2])
by ra.sai.msu.su (8.9.1/8.9.1) with ESMTP id XAA21513;
Sun, 14 Nov 1999 23:40:06 +0300 (MSK)
Date: Sun, 14 Nov 1999 23:40:06 +0300 (MSK)
From: Oleg Bartunov <oleg@sai.msu.su>
X-Sender: megera@ra
To: pgsql-hackers@postgresql.org
cc: t-ishii@sra.co.jp
Subject: problem with MB ? 6.5.3 under freebsd-elf
Message-ID: <Pine.GSO.3.96.SK.991114232654.3910y-100000@ra>
Organization: Sternberg Astronomical Institute (Moscow University)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=koi8-r
Content-Transfer-Encoding: 8bit

Hi,

today I found a problem with MB on freebsd-elf running 6.5.3 -
I have compiled 6.5.3 with --enable-locale --with-mb=KOI8
and in psql I tried set client_encoding to 'WIN'
but result of simple query looks like 8-bit was stripped.
The same query works as expected under Linux.
Also, when I tried
select * from t1 where a ~* '���';
I got:
ERROR: Can't find right op '~*' for type 25
I'm outside of city and has access only to b/w terminal
without cut'n paste support ( old 286 + dialup ) and I can't
provide more information. Will do this tomorrow.

Oleg

PS. The same weird thing happens even if I
set client_encoding to 'KOI8' - native encoding !

_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

From bouncefilter Sun Nov 14 18:13:39 1999
Received: from thelab.hub.org (nat198.58.mpoweredpc.net [142.177.198.58])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA92851;
Sun, 14 Nov 1999 18:13:16 -0500 (EST) (envelope-from scrappy@hub.org)
Received: from localhost (scrappy@localhost)
by thelab.hub.org (8.9.3/8.9.1) with ESMTP id TAA90257;
Sun, 14 Nov 1999 19:13:28 -0400 (AST) (envelope-from scrappy@hub.org)
X-Authentication-Warning: thelab.hub.org: scrappy owned process doing -bs
Date: Sun, 14 Nov 1999 19:13:27 -0400 (AST)
From: The Hermit Hacker <scrappy@hub.org>
To: pgsql-general@postgresql.org
cc: pgsql-hackers@postgresql.org
Subject: Looks like hell, but ...
Message-ID: <Pine.BSF.4.10.9911141908330.75865-100000@thelab.hub.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

http://www.pgsql.com/app-index ...

Its a start, at least. I have to add 'update' facilities for ppl to edit
records, and it *looks* like hell, but its a start...

I'm concentrating on apps first, and, eventually, its going to include
'Usage of' records...

All input has to be approved before it becomes live...and it will
eventually contain information similar to what FreshMeat provides
(dependancies, license, etc)...

Check it out, add content and watch it evolve...

Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

From bouncefilter Sun Nov 14 18:24:40 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA94691
for <pgsql-hackers@postgreSQL.org>;
Sun, 14 Nov 1999 18:24:25 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id SAA12270;
Sun, 14 Nov 1999 18:23:44 -0500 (EST)
To: Mike Mascari <mascarim@yahoo.com>
cc: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Postmaster dies with FATAL 1: ReleaseLruFile: No opened
files - no one can be closed
In-reply-to: Your message of Sun, 14 Nov 1999 11:24:20 -0800 (PST)
<19991114192420.15160.rocketmail@web2106.mail.yahoo.com>
Date: Sun, 14 Nov 1999 18:23:44 -0500
Message-ID: <12268.942621824@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Mike Mascari <mascarim@yahoo.com> writes:

FATAL 1: ReleaseLruFile: No opened files - no one can be closed

This is the first time this has ever happened.

I've never seen that either. Offhand I do not recall any post-6.5
changes that would affect it, so the problem (whatever it is) is
probably still there.

After eyeballing the code, it seems there are only two ways this
could happen:

1. the number of "allocated" (non-virtual) file descriptors grew to
exceed the number of files Postgres thinks it can have open;

2. something else was temporarily exhausting your kernel's file table
space, so that ENFILE was returned for many successive attempts to
open a file. (After each one, fd.c will close another file and try
again.)

#2 seems improbable on an unloaded system, and isn't real probable even
on a loaded one, since you'd have to assume that some other process
managed to suck up each filetable slot that fd.c released before fd.c
could re-acquire it. Once, yes, but several dozen times in a row?

So I'm guessing a leak of allocated file descriptors.

After grovelling through the calls to AllocateFile, I only see one
prospect for a leak: it looks to me like verify_password() neglects
to close the password file if an invalid user name is given. Do you
use a plain (non-encrypted) password file? If so, I'll bet you can
reproduce the crash by trying repeatedly to connect with a username
that's not in the password file. If that pans out, it's a simple fix:
add "FreeFile(pw_file);" near the bottom of verify_password() in
src/backend/libpq/password.c. Let me know if this guess is right...

regards, tom lane

From bouncefilter Sun Nov 14 18:38:40 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA96948
for <pgsql-hackers@postgreSQL.org>;
Sun, 14 Nov 1999 18:37:57 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id SAA12294
for <pgsql-hackers@postgreSQL.org>;
Sun, 14 Nov 1999 18:37:22 -0500 (EST)
To: pgsql-hackers@postgreSQL.org
Subject: elog() executed by postmaster
Date: Sun, 14 Nov 1999 18:37:22 -0500
Message-ID: <12291.942622642@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

While chasing the apparent FD leakage reported by Mike Mascari,
I realized that there are probably code paths in which the postmaster
itself will invoke elog(ERROR) or elog(FATAL). This is fairly likely
anyplace that the postmaster uses routines also used by the backend.
Mascari's example shows that it *will* happen, in the current state of
the code, if the postmaster does enough failed password lookups.
That's a bug of course, but there will always be bugs. Trying to ensure
that the postmaster will never call elog() seems like a losing game;
instead we need to ensure that something acceptable will happen.

Right now, what will happen is a postmaster coredump (or worse,
undefined behavior) due to trying to longjmp through an uninitialized
jmp_buf to return to the never-yet-entered backend main loop. That
doesn't rate as acceptable in my book.

elog() should probably check for being in the postmaster and force
a postmaster shutdown if it gets elog(FATAL). Should elog(ERROR)
do the same, or do we want to try to add a return-to-main-loop
capability in the postmaster? I'm inclined to keep it simple and
just do an orderly shutdown in both cases. Any such code paths that
are out there are obviously not heavily exercised, so I doubt that
it's worth adding new code to try to recover from elog(ERROR).

Comments?

regards, tom lane

From bouncefilter Sun Nov 14 19:55:40 1999
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id TAA05601
for <pgsql-hackers@postgreSQL.org>;
Sun, 14 Nov 1999 19:55:03 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
TAA07293;
Sun, 14 Nov 1999 19:48:17 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911150048.TAA07293@candle.pha.pa.us>
Subject: Re: [HACKERS] elog() executed by postmaster
In-Reply-To: <12291.942622642@sss.pgh.pa.us> from Tom Lane at "Nov 14,
1999 06:37:22 pm"
To: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sun, 14 Nov 1999 19:48:17 -0500 (EST)
CC: pgsql-hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

elog() should probably check for being in the postmaster and force
a postmaster shutdown if it gets elog(FATAL). Should elog(ERROR)
do the same, or do we want to try to add a return-to-main-loop
capability in the postmaster? I'm inclined to keep it simple and
just do an orderly shutdown in both cases. Any such code paths that
are out there are obviously not heavily exercised, so I doubt that
it's worth adding new code to try to recover from elog(ERROR).

Agreed. Just try simple first.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Mon Nov 15 12:13:52 1999
Received: from www.wgcr.org (IDENT:root@www.wgcr.org [206.74.232.194])
by hub.org (8.9.3/8.9.3) with ESMTP id MAA27662
for <pgsql-hackers@postgresql.org>;
Mon, 15 Nov 1999 12:13:27 -0500 (EST)
(envelope-from lamar.owen@wgcr.org)
Received: from wgcr.org ([206.74.232.197])
by www.wgcr.org (8.8.7/8.8.5) with ESMTP id MAA19195;
Mon, 15 Nov 1999 12:12:54 -0500
Message-ID: <38303F0F.90E320D@wgcr.org>
Date: Mon, 15 Nov 1999 12:12:47 -0500
From: Lamar Owen <lamar.owen@wgcr.org>
Organization: WGCR Internet Radio
X-Mailer: Mozilla 4.61 [en] (Win95; I)
X-Accept-Language: en
MIME-Version: 1.0
To: The Hermit Hacker <scrappy@hub.org>
CC: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Looks like hell, but ...
References: <Pine.BSF.4.10.9911141908330.75865-100000@thelab.hub.org>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

The Hermit Hacker wrote:

Check it out, add content and watch it evolve...

A good start, Marc. I added the interesting package called onShore
Timesheet -- looks like your entry form covers most of the bases.

--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

From bouncefilter Mon Nov 15 12:51:53 1999
Received: from thelab.hub.org (nat198.58.mpoweredpc.net [142.177.198.58])
by hub.org (8.9.3/8.9.3) with ESMTP id MAA32678
for <pgsql-hackers@postgresql.org>;
Mon, 15 Nov 1999 12:50:55 -0500 (EST) (envelope-from scrappy@hub.org)
Received: from localhost (scrappy@localhost)
by thelab.hub.org (8.9.3/8.9.1) with ESMTP id NAA97214;
Mon, 15 Nov 1999 13:50:34 -0400 (AST) (envelope-from scrappy@hub.org)
X-Authentication-Warning: thelab.hub.org: scrappy owned process doing -bs
Date: Mon, 15 Nov 1999 13:50:33 -0400 (AST)
From: The Hermit Hacker <scrappy@hub.org>
To: Lamar Owen <lamar.owen@wgcr.org>
cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Looks like hell, but ...
In-Reply-To: <38303F0F.90E320D@wgcr.org>
Message-ID: <Pine.BSF.4.10.9911151349390.75865-100000@thelab.hub.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Mon, 15 Nov 1999, Lamar Owen wrote:

The Hermit Hacker wrote:

Check it out, add content and watch it evolve...

A good start, Marc. I added the interesting package called onShore
Timesheet -- looks like your entry form covers most of the bases.

Thanks...I've been populating it with what I can readily find (through
freshmeat so far), but I definitely don't know everything that is out
there :)

Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

From bouncefilter Mon Nov 15 13:16:53 1999
Received: from loopy.berkhirt.com (loopy.berkhirt.com [209.220.85.54])
by hub.org (8.9.3/8.9.3) with ESMTP id NAA36443
for <pgsql-hackers@postgreSQL.org>;
Mon, 15 Nov 1999 13:16:13 -0500 (EST)
(envelope-from bhirt@loopy.berkhirt.com)
Received: (from bhirt@localhost)
by loopy.berkhirt.com (8.9.3/8.8.7) id MAA25428;
Mon, 15 Nov 1999 12:14:26 -0600
Date: Mon, 15 Nov 1999 12:14:26 -0600
From: Brian Hirt <bhirt@mobygames.com>
To: The Hermit Hacker <scrappy@hub.org>
Cc: Lamar Owen <lamar.owen@wgcr.org>, pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Looks like hell, but ...
Message-ID: <19991115121426.A25286@loopy.berkhirt.com>
Reply-To: bhirt@mobygames.com
References: <38303F0F.90E320D@wgcr.org>
<Pine.BSF.4.10.9911151349390.75865-100000@thelab.hub.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Mailer: Mutt 0.95.4us
In-Reply-To: <Pine.BSF.4.10.9911151349390.75865-100000@thelab.hub.org>;
from The Hermit Hacker on Mon, Nov 15, 1999 at 01:50:33PM -0400
X-PC-Gaming: http://www.mobygames.com/

On Mon, Nov 15, 1999 at 01:50:33PM -0400, The Hermit Hacker wrote:

Thanks...I've been populating it with what I can readily find (through
freshmeat so far), but I definitely don't know everything that is out
there :)

I'm not sure if you worked on the user gallery also, but I like that as
well. I do have one suggestion and a comment. It would be nice if a
description of the project could be included. I'd like to browse through
some of the other project but clicking on each one and seeing a description
of the project would help decide which ones to look at.

My comment is that there appears to be no maintainer of the gallery. There
is a project name testing_shit. I counted at least 10 duplicate entries and
many that are obvously incomplete. Do you want any help with the gallery?

-brian

--
The world's most ambitious and comprehensive PC game database project.

http://www.mobygames.com

From bouncefilter Mon Nov 15 14:10:53 1999
Received: from nile.purplefrog.com (inca.incanta.com [209.144.154.2] (may be
forged)) by hub.org (8.9.3/8.9.3) with ESMTP id OAA43476
for <pgsql-hackers@postgresql.org>;
Mon, 15 Nov 1999 14:10:34 -0500 (EST)
(envelope-from thoth@nile.purplefrog.com)
Received: from nile (localhost [127.0.0.1])
by nile.purplefrog.com (8.8.7/8.8.7) with ESMTP id OAA01316
for <pgsql-hackers@postgresql.org>; Mon, 15 Nov 1999 14:10:34 -0500
Message-Id: <199911151910.OAA01316@nile.purplefrog.com>
To: pgsql-hackers@postgresql.org
Subject: 6.5.1 -vs- null values for an int8
Date: Mon, 15 Nov 1999 14:10:34 -0500
From: Robert Forsman <thoth@nile.purplefrog.com>

Any clue why Postgresql version 6.5.1 can't handle null values for int8 in
a WHERE clause?

incanta=> create table t(v int8);
CREATE
incanta=> insert into t(v) values(0);
INSERT 101737 1
incanta=> insert into t(v) values(1);
INSERT 101738 1
incanta=> insert into t(v) values(-1);
INSERT 101739 1
incanta=> select * from t where v>=0;
v
-
0
1
(2 rows)

incanta=> insert into t(v) values (null);
INSERT 101740 1
incanta=> select * from t;
v
--
0
1
-1

(4 rows)

incanta=> select * from t where v>=0;
pqReadData() -- backend closed the channel unexpectedly.
This probably means the backend terminated abnormally
before or while processing the request.
We have lost the connection to the backend, so further processing is impossible. Terminating.

From bouncefilter Mon Nov 15 14:27:57 1999
Received: from thelab.hub.org (nat198.58.mpoweredpc.net [142.177.198.58])
by hub.org (8.9.3/8.9.3) with ESMTP id OAA45692
for <pgsql-hackers@postgreSQL.org>;
Mon, 15 Nov 1999 14:27:41 -0500 (EST) (envelope-from scrappy@hub.org)
Received: from localhost (scrappy@localhost)
by thelab.hub.org (8.9.3/8.9.1) with ESMTP id PAA97842;
Mon, 15 Nov 1999 15:26:57 -0400 (AST) (envelope-from scrappy@hub.org)
X-Authentication-Warning: thelab.hub.org: scrappy owned process doing -bs
Date: Mon, 15 Nov 1999 15:26:56 -0400 (AST)
From: The Hermit Hacker <scrappy@hub.org>
To: Brian Hirt <bhirt@mobygames.com>
cc: Lamar Owen <lamar.owen@wgcr.org>, pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Looks like hell, but ...
In-Reply-To: <19991115121426.A25286@loopy.berkhirt.com>
Message-ID: <Pine.BSF.4.10.9911151524190.75865-100000@thelab.hub.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Mon, 15 Nov 1999, Brian Hirt wrote:

On Mon, Nov 15, 1999 at 01:50:33PM -0400, The Hermit Hacker wrote:

Thanks...I've been populating it with what I can readily find (through
freshmeat so far), but I definitely don't know everything that is out
there :)

I'm not sure if you worked on the user gallery also, but I like that as
well. I do have one suggestion and a comment. It would be nice if a
description of the project could be included. I'd like to browse through
some of the other project but clicking on each one and seeing a description
of the project would help decide which ones to look at.

My comment is that there appears to be no maintainer of the gallery. There
is a project name testing_shit. I counted at least 10 duplicate entries and
many that are obvously incomplete. Do you want any help with the gallery?

Now that I have what looks like a nice format (technically, not visually)
for the apps, I'm going to redo the user gallery itself also...assuming
nothing comes up later tonight, will hopefully have most of it converted
then...

Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

From bouncefilter Mon Nov 15 18:22:56 1999
Received: from web2103.mail.yahoo.com (web2103.mail.yahoo.com [128.11.68.247])
by hub.org (8.9.3/8.9.3) with SMTP id SAA85962
for <pgsql-hackers@postgresql.org>;
Mon, 15 Nov 1999 18:22:22 -0500 (EST)
(envelope-from mascarim@yahoo.com)
Message-ID: <19991115232221.5713.rocketmail@web2103.mail.yahoo.com>
Received: from [24.26.131.45] by web2103.mail.yahoo.com;
Mon, 15 Nov 1999 15:22:21 PST
Date: Mon, 15 Nov 1999 15:22:21 -0800 (PST)
From: Mike Mascari <mascarim@yahoo.com>
Subject: Re: [HACKERS] Postmaster dies with FATAL 1: ReleaseLruFile: No opened
files - no one can be closed
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-hackers@postgresql.org
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

--- Tom Lane <tgl@sss.pgh.pa.us> wrote:

Mike Mascari <mascarim@yahoo.com> writes:

FATAL 1: ReleaseLruFile: No opened files - no one can be closed

This is the first time this has ever happened.

I've never seen that either. Offhand I do not recall any post-6.5
changes that would affect it, so the problem (whatever it is) is
probably still there.

After eyeballing the code, it seems there are only two ways this
could happen:

1. the number of "allocated" (non-virtual) file descriptors grew to
exceed the number of files Postgres thinks it can have open;

2. something else was temporarily exhausting your kernel's file table
space, so that ENFILE was returned for many successive attempts to
open a file. (After each one, fd.c will close another file and try
again.)

#2 seems improbable on an unloaded system, and isn't real probable even
on a loaded one, since you'd have to assume that some other process
managed to suck up each filetable slot that fd.c released before fd.c
could re-acquire it. Once, yes, but several dozen times in a row?

Thanks for the response, Tom. When looking at the system log,
the kernel was logging messages regarding IPX network name collisions
which apprently can happen when there are autoconfigured Win95 boxes
on the same subnet. These messages were flooding the log at a rate of
one every second or two...Even though #2 seems improbable, and just
glancing at the IPX kernel code didn't point to how that may have
caused a continual consumption of file descriptors, I'm willing to
blame the kernel on this (and me for using autoprimary and autointerface
options).

Thanks again,

Mike Mascari
(mascarim@yahoo.com)

=====

__________________________________________________
Do You Yahoo!?
Bid and sell for free at http://auctions.yahoo.com

From bouncefilter Mon Nov 15 18:31:19 1999
Received: (from news@localhost) by hub.org (8.9.3/8.9.3) id SAA87371
for pgsql-hackers@postgresql.org; Mon, 15 Nov 1999 18:30:45 -0500 (EST)
(envelope-from news)
X-Authentication-Warning: hub.org: news set sender to <news> using -f
From: Speedy Fast <speedyfast26@hotmail.com>
X-Newsgroups: comp.databases.postgresql.hackers
Subject: shell accounts that offer postgresql?
Message-ID: <15cwOKV3giCHNpAfUybGmz+T+CRY@4ax.com>
X-Newsreader: Forte Agent 1.6/32.525
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Lines: 9
Date: Mon, 15 Nov 1999 23:30:38 GMT
X-Complaints-To: abuse@home.net
X-Trace: news2.rdc1.on.home.com 942708638 24.112.54.253 (Mon,
15 Nov 1999 15:30:38 PST)
Organization: @Home Network Canada
To: pgsql-hackers@postgresql.org

Are there any web hosting services that offer postgresql database
access (cgi would be nice too)?

==============================================================
Here's some for the killfiles
@aol.com crush@home.com fredsanfordrules@aol.commedian
@webtv.com

From bouncefilter Mon Nov 15 20:03:57 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id UAA02621
for <pgsql-hackers@postgreSQL.org>;
Mon, 15 Nov 1999 20:03:28 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id UAA19907;
Mon, 15 Nov 1999 20:02:55 -0500 (EST)
To: Peter Eisentraut <peter_e@gmx.net>
cc: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Status of sql_help.h
In-reply-to: Your message of Sun, 14 Nov 1999 19:34:26 +0100 (CET)
<Pine.LNX.4.20.9911141850340.797-100000@peter-e.yi.org>
Date: Mon, 15 Nov 1999 20:02:55 -0500
Message-ID: <19905.942714175@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Peter Eisentraut <peter_e@gmx.net> writes:

"The `configure' script and the Makefile rules for building and
installation should not use any utilities directly except these:
cat cmp cp diff echo egrep expr false grep install-info
ln ls mkdir mv pwd rm rmdir sed sleep sort tar test touch true"
No awk there either.

Do I need to point out that Perl isn't there either? But this GNU rule
is irrelevant, because it applies to tools needed to build a standard
*distribution* of a package. Maintainer tools can include other things.
Using perl to generate sql_help.h seems perfectly appropriate to me,
as I said before.

What I wanted to find out was whether there were a lot of people using
the CVS server who don't have Perl and would object to installing it.
That's what will determine whether we can remove sql_help.h from the CVS
archive (as opposed to distributed tarballs).

PS: "make distclean" should probably not remove sql_help.h, for the
same reasons that we don't remove gram.c --- it *is* a distributed
file, and a particular user might not have the tools to rebuild it.

That was my bad. For some reason I had the idea that "distclean" stood for
"distinctly clean" (really clean). :-\ I'll fix that. Perhaps we ought to
decide on some standard targets. "maintainer-clean" would be the proper
one to use (in GNU, again).

No, it wouldn't be. We use distclean precisely as specified in the GNU
coding standards:

`distclean'
Delete all files from the current directory that are created by
configuring or building the program. If you have unpacked the
source and built the program without creating any other files,
`make distclean' should leave only the files that were in the
distribution.

sql_help.h will now be in the distribution, therefore distclean
shouldn't remove it.

regards, tom lane

From bouncefilter Mon Nov 15 20:26:57 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id UAA05708
for <pgsql-hackers@postgreSQL.org>;
Mon, 15 Nov 1999 20:26:47 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id UAA20092;
Mon, 15 Nov 1999 20:26:10 -0500 (EST)
To: Mike Mascari <mascarim@yahoo.com>
cc: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Postmaster dies with FATAL 1: ReleaseLruFile: No opened
files - no one can be closed
In-reply-to: Your message of Mon, 15 Nov 1999 15:22:21 -0800 (PST)
<19991115232221.5713.rocketmail@web2103.mail.yahoo.com>
Date: Mon, 15 Nov 1999 20:26:10 -0500
Message-ID: <20090.942715570@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Mike Mascari <mascarim@yahoo.com> writes:

Thanks for the response, Tom. When looking at the system log,
the kernel was logging messages regarding IPX network name collisions
which apprently can happen when there are autoconfigured Win95 boxes
on the same subnet. These messages were flooding the log at a rate of
one every second or two...Even though #2 seems improbable, and just
glancing at the IPX kernel code didn't point to how that may have
caused a continual consumption of file descriptors, I'm willing to
blame the kernel on this (and me for using autoprimary and autointerface
options).

That doesn't strike me as a bulletproof explanation. fd.c has a tight
loop that close()s an FD and then tries to open() the file it wants,
repeat until success or an error other than ENFILE/EMFILE. If the
scenario really is that it got ENFILE every time until it was down to
zero FDs, there'd have to be something sucking up each freed FD within
microseconds of its being freed. Repeatedly. Forty or fifty (or more)
times in a row. I don't think a once-a-second Win95 lossage will do
that. And if you were down to zero free FDs system-wide, Postgres
wouldn't be the only thing having troubles!

I take it you don't use Postgres password authentication at all? If you
do, the other theory looks a lot more viable to me... I haven't had time
to try to reproduce a crash yet, but I'm pretty sure there's one there.

regards, tom lane

From bouncefilter Mon Nov 15 21:45:58 1999
Received: from localhost (IDENT:root@hectic-1.jpl.nasa.gov [128.149.68.203])
by hub.org (8.9.3/8.9.3) with ESMTP id VAA14363
for <hackers@postgresql.org>; Mon, 15 Nov 1999 21:45:02 -0500 (EST)
(envelope-from lockhart@alumni.caltech.edu)
Received: from alumni.caltech.edu (lockhart@localhost [127.0.0.1])
by localhost (8.8.7/8.8.7) with ESMTP id CAA14991;
Tue, 16 Nov 1999 02:52:10 GMT
Sender: lockhart@hub.org
Message-ID: <3830C6DA.5CC933CB@alumni.caltech.edu>
Date: Tue, 16 Nov 1999 02:52:10 +0000
From: Thomas Lockhart <lockhart@alumni.caltech.edu>
X-Mailer: Mozilla 4.6 [en] (X11; I; Linux 2.0.36 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Brian Waters <jbw@InnovaSystems.com>
CC: Postgres Hackers List <hackers@postgresql.org>
Subject: Re: Postgresql Docs....
References: <199911152245.RAA12751@innsys.InnovaSystems.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

I notice that the postgresql docs say that postgresql is a public domain
program, while they really carry a Berkley copyright. You might want to
correct this for the next release.
http://www.bbin.com/pd/

Ooh. I guess I'm not familiar with the fine points here. Our
Berkeley-style license allows use, modification, sale, gift, theft,
etc. of the software with only one provision: that the copyright
notice remain intact. Clearly, this copyright notice is designed to
protect UCB from rabid lawyers once the software is no longer under
UCB's control, and this copyright allows any and all of the above
uses, and any other use also.
So what about this would not be considered public domain software?

Something can not be both Copyrighted and in the public domain.

Hmm. I've taken this on-list, just in case someone else has a comment.
But in the absence of alternate information, I'll just assume that we
are not public domain software. But I sure still have the feeling that
we are getting gypped by the legaleze.

Thanks for the heads-up...

- Thomas

--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California

From bouncefilter Mon Nov 15 22:08:58 1999
Received: from paprika.michvhf.com (paprika.michvhf.com [209.57.60.12])
by hub.org (8.9.3/8.9.3) with SMTP id WAA17405
for <hackers@postgreSQL.org>; Mon, 15 Nov 1999 22:08:14 -0500 (EST)
(envelope-from vev@michvhf.com)
Received: (qmail 346 invoked by uid 1001); 16 Nov 1999 03:08:12 -0000
Message-ID: <XFMail.991115220812.vev@michvhf.com>
X-Mailer: XFMail 1.4.0 on FreeBSD
X-Priority: 3 (Normal)
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
In-Reply-To: <3830C6DA.5CC933CB@alumni.caltech.edu>
Date: Mon, 15 Nov 1999 22:08:12 -0500 (EST)
X-Face: *<Qp5V!eyV,gni`N^N%1YX'$I&uuX]ay;
oq#ZL5Hn8EQsu'.oK0j9$#JM0V?!=Q^[i.81u9
@~=ZjeI}gHY`?2_1,xy/,Gde>0^4Iw)<k8}vg!%l;
&]@PF0LjU)N*m*2"R^UO+PAQ<w}/y)5UVE==w
H$q0*b`HN{+Ekeo?5V(0$MH&NZA3~vOThJxhY(7M:"`CrqO9[VC!^W&&eih!MTq4qk=Vg'd&`{dpgp
3-nck}7do'o/|<RI,
igc#cg8t|PZUEh{Rrx4<~tm`/G8E*wE{G:x}bva@[+YVT`g(u]*^!`1iO*
Sender: vev@paprika.michvhf.com
From: Vince Vielhaber <vev@michvhf.com>
To: Thomas Lockhart <lockhart@alumni.caltech.edu>
Subject: RE: [HACKERS] Re: Postgresql Docs....
Cc: Postgres Hackers List <hackers@postgreSQL.org>,
Brian Waters <jbw@InnovaSystems.com>

On 16-Nov-99 Thomas Lockhart wrote:

I notice that the postgresql docs say that postgresql is a public domain
program, while they really carry a Berkley copyright. You might want to
correct this for the next release.
http://www.bbin.com/pd/

Ooh. I guess I'm not familiar with the fine points here. Our
Berkeley-style license allows use, modification, sale, gift, theft,
etc. of the software with only one provision: that the copyright
notice remain intact. Clearly, this copyright notice is designed to
protect UCB from rabid lawyers once the software is no longer under
UCB's control, and this copyright allows any and all of the above
uses, and any other use also.
So what about this would not be considered public domain software?

Something can not be both Copyrighted and in the public domain.

Hmm. I've taken this on-list, just in case someone else has a comment.
But in the absence of alternate information, I'll just assume that we
are not public domain software. But I sure still have the feeling that
we are getting gypped by the legaleze.

IIRC, All copyright notices must be kept intact. Software in the public
domain carries no protection whatsoever. PD software can be taken and
renamed to whatever by the person that renamed it and claimed to be their
own property. This happened to the WinVN project at least once (it's a
PD Windows newsreader). At least one commercial project came directly
from the WinVN sources - which are in the public domain.

Vince.
--
==========================================================================
Vince Vielhaber -- KA8CSH email: vev@michvhf.com flame-mail: /dev/null
# include <std/disclaimers.h> Have you seen http://www.pop4.net?
Online Campground Directory http://www.camping-usa.com
Online Giftshop Superstore http://www.cloudninegifts.com
==========================================================================

From bouncefilter Mon Nov 15 14:45:54 1999
Received: from adamastor.med.up.pt (root@mail.med.up.pt [193.136.35.3])
by hub.org (8.9.3/8.9.3) with ESMTP id OAA51941
for <pgsql-hackers@postgresql.org>;
Mon, 15 Nov 1999 14:45:43 -0500 (EST) (envelope-from const@med.up.pt)
Received: from pegasus (pegasus.med.up.pt [193.136.35.44])
by adamastor.med.up.pt (8.8.7/8.8.7) with SMTP id UAA27444
for <pgsql-hackers@postgresql.org>; Mon, 15 Nov 1999 20:06:13 GMT
Message-Id: <3.0.6.32.19991115193604.02cc9e80@mail.med.up.pt>
X-Sender: const@mail.med.up.pt
X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32)
Date: Mon, 15 Nov 1999 19:36:04 -0800
To: pgsql-hackers@postgresql.org
From: Constantino Martins <const@med.up.pt>
Subject:
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

Dear Sir

Can you help me with a problem I have:

When more than 30 process query my Postgres data base, then my postmaster
create zombies process.
Like this:
USER PID %CPU %MEM SIZE RSS TTY STAT START TIME COMMAND
postgres 4139 0.1 0.0 0 0 ? Z 18:38 0:01 (postmaster
<zombie>)
postgres 4140 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4141 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4142 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4143 0.1 0.0 0 0 ? Z 18:38 0:01 (postmaster
<zombie>)
postgres 4146 0.1 0.0 0 0 ? Z 18:38 0:02 (postmaster
<zombie>)
postgres 4150 0.1 0.0 0 0 ? Z 18:38 0:02 (postmaster
<zombie>)
postgres 4152 0.1 0.0 0 0 ? Z 18:38 0:02 (postmaster
<zombie>)
postgres 4159 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4170 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4174 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4175 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4177 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4178 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4179 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4194 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4195 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4197 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4199 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4200 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4201 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4202 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4203 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4204 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4205 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4206 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4209 0.0 0.0 0 0 ? Z 18:38 0:00 (postmaster
<zombie>)
postgres 4597 0.0 0.9 1148 608 p4 S 18:52 0:00 bash
postgres 11356 0.0 1.3 4816 824 ? S Nov 11 0:08
/usr/local/pgsql/bin/postmaster -B 256 -i -D /usr/local/pgsql/data
root 1 0.0 0.3 828 192 ? S Nov 9 0:05 init [3]Half a block on average.
root 2 0.0 0.0 0 0 ? SW Nov 9 0:00 (kflushd)
root 3 0.0 0.0 0 0 ? SW< Nov 9 0:00 (kswapd)

I have a server with 64 MB.
Can you help me ?
thanks

sorry for my English

Atentamente e com os melhores cumprimentos

Constantino Martins

From bouncefilter Mon Nov 15 23:03:44 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA24505
for <hackers@postgreSQL.org>; Mon, 15 Nov 1999 23:02:39 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id XAA20679;
Mon, 15 Nov 1999 23:00:56 -0500 (EST)
To: Thomas Lockhart <lockhart@alumni.caltech.edu>
cc: Brian Waters <jbw@InnovaSystems.com>,
Postgres Hackers List <hackers@postgresql.org>
Subject: Re: [HACKERS] Re: Postgresql Docs....
In-reply-to: Your message of Tue, 16 Nov 1999 02:52:10 +0000
<3830C6DA.5CC933CB@alumni.caltech.edu>
Date: Mon, 15 Nov 1999 23:00:55 -0500
Message-ID: <20677.942724855@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Thomas Lockhart <lockhart@alumni.caltech.edu> writes:

I notice that the postgresql docs say that postgresql is a public domain
program, while they really carry a Berkley copyright. You might want to
correct this for the next release.

So what about this would not be considered public domain software?

Something can not be both Copyrighted and in the public domain.

Hmm. I've taken this on-list, just in case someone else has a comment.
But in the absence of alternate information, I'll just assume that we
are not public domain software. But I sure still have the feeling that
we are getting gypped by the legaleze.

IANAL, but I've paid considerable attention to these issues over the
past ten years. My understanding is that "public domain" means
specifically that there is *no* copyright or any other intellectual-
property restriction on the software. In particular, anything that
has either a BSD- or GPL-style license is most certainly not public
domain.

I'd suggest replacing all uses of the phrase "public domain" with
"open source" or "freely available" or some other term that hasn't
got such a clearly-inapplicable legal meaning.

regards, tom lane

From bouncefilter Mon Nov 15 23:40:00 1999
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA30043
for <pgsql-hackers@postgreSQL.org>;
Mon, 15 Nov 1999 23:39:18 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
XAA02308;
Mon, 15 Nov 1999 23:37:56 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911160437.XAA02308@candle.pha.pa.us>
Subject: Unique indexes on system tables
In-Reply-To: <000501bf1a97$b925a860$2801007e@cadzone.tpf.co.jp> from Hiroshi
Inoue at "Oct 20, 1999 10:09:13 am"
To: Hiroshi Inoue <Inoue@tpf.co.jp>
Date: Mon, 15 Nov 1999 23:37:56 -0500 (EST)
CC: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Hiroshi, there are two things I want to do for 7.0.

First, I want to make more of the system indexes unique. Are you aware
of any reasons not to do that? I see you have done some of them
already. I talked to Tom Lane, and he thinks that it will not cause
problems because unique insertions/updates wait for transactions to
commit before doing a conflicting change to the index, right?

Second, I want to add more system indexes to match all caches. Anything
that could cause problems there?

Are you working on any of this?

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Tue Nov 16 00:23:00 1999
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA38616
for <pgsql-hackers@postgreSQL.org>;
Tue, 16 Nov 1999 00:22:00 -0500 (EST) (envelope-from Inoue@tpf.co.jp)
Received: from cadzone ([126.0.1.40] (may be forged))
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with SMTP
id OAA00059; Tue, 16 Nov 1999 14:21:08 +0900
From: "Hiroshi Inoue" <Inoue@tpf.co.jp>
To: "Bruce Momjian" <pgman@candle.pha.pa.us>
Cc: "PostgreSQL-development" <pgsql-hackers@postgreSQL.org>
Subject: RE: Unique indexes on system tables
Date: Tue, 16 Nov 1999 14:25:37 +0900
Message-ID: <000001bf2ff3$0391abe0$2801007e@cadzone.tpf.co.jp>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
In-Reply-To: <199911160437.XAA02308@candle.pha.pa.us>
Importance: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V4.72.2106.4

-----Original Message-----
From: Bruce Momjian [mailto:pgman@candle.pha.pa.us]
Sent: Tuesday, November 16, 1999 1:38 PM
To: Hiroshi Inoue
Cc: PostgreSQL-development
Subject: Unique indexes on system tables

Hiroshi, there are two things I want to do for 7.0.

First, I want to make more of the system indexes unique. Are you aware
of any reasons not to do that?

No.
It should be done to guarantee the uniqueness of system tuples.

I see you have done some of them
already. I talked to Tom Lane, and he thinks that it will not cause
problems because unique insertions/updates wait for transactions to
commit before doing a conflicting change to the index, right?

Yes.

Second, I want to add more system indexes to match all caches. Anything
that could cause problems there?

I am only afraid of index corruption.
The more we have system indexes,the more index corruption would happen.

How could we recover from the state ?
Tom suggested rebuilding indexes in vacuum.
According to Jan,there was a utility called reindexdb.
WAL by Vadim may be able to recover indexes completely in case of crash.

Are you working on any of this?

No.

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp

From bouncefilter Tue Nov 16 03:44:09 1999
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
by hub.org (8.9.3/8.9.3) with ESMTP id DAA64126
for <pgsql-hackers@postgreSQL.org>;
Tue, 16 Nov 1999 03:43:14 -0500 (EST) (envelope-from Inoue@tpf.co.jp)
Received: from tpf.co.jp ([126.0.1.56] (may be forged))
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with ESMTP
id RAA00171; Tue, 16 Nov 1999 17:41:32 +0900
Message-ID: <383118B2.AAC9E730@tpf.co.jp>
Date: Tue, 16 Nov 1999 17:41:22 +0900
From: inoue <Inoue@tpf.co.jp>
X-Mailer: Mozilla 4.51 [ja] (Win98; I)
X-Accept-Language: ja
MIME-Version: 1.0
To: Tom Lane <tgl@sss.pgh.pa.us>
CC: t-ishii@sra.co.jp, Karel Zak - Zakkr <zakkr@zf.jcu.cz>,
pgsql-hackers <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] compression in LO and other fields
References: <25684.942387283@sss.pgh.pa.us>
Content-Type: text/plain; charset=iso-2022-jp
Content-Transfer-Encoding: 7bit

Tom Lane wrote:

Tatsuo Ishii <t-ishii@sra.co.jp> writes:

LO is a dead end. What we really want to do is eliminate tuple-size
restrictions and then have large ordinary fields (probably of type
bytea) in regular tuples. I'd suggest working on compression in that
context, say as a new data type called "bytez" or something like that.

It sounds ideal but I remember that Vadim said inserting a 2GB record
is not good idea since it will be written into the log too. If it's a
necessary limitation from the point of view of WAL, we have to accept
it, I think.

LO won't make that any better: the data still goes into a table.
You'd have 2GB worth of WAL entries either way.

The only thing LO would do for you is divide the data into block-sized
tuples, so there would be a bunch of little WAL entries instead of one
big one. But that'd probably be easy to duplicate too. If we implement
big tuples by chaining together disk-block-sized segments, which seems
like the most likely approach, couldn't WAL log each segment as a
separate log entry? If so, there's almost no difference between LO and
inline field for logging purposes.

I don't know LO well.
But seems LO allows partial update.
Big tuples
If so,isn't it a significant difference ?

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp

From bouncefilter Tue Nov 16 03:48:10 1999
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
by hub.org (8.9.3/8.9.3) with ESMTP id DAA64791
for <pgsql-hackers@postgreSQL.org>;
Tue, 16 Nov 1999 03:47:35 -0500 (EST) (envelope-from Inoue@tpf.co.jp)
Received: from tpf.co.jp ([126.0.1.56] (may be forged))
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with ESMTP
id RAA00176; Tue, 16 Nov 1999 17:45:53 +0900
Message-ID: <383119B6.952A06F3@tpf.co.jp>
Date: Tue, 16 Nov 1999 17:45:43 +0900
From: inoue <Inoue@tpf.co.jp>
X-Mailer: Mozilla 4.51 [ja] (Win98; I)
X-Accept-Language: ja
MIME-Version: 1.0
To: Tom Lane <tgl@sss.pgh.pa.us>, t-ishii@sra.co.jp,
Karel Zak - Zakkr <zakkr@zf.jcu.cz>,
pgsql-hackers <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] compression in LO and other fields
References: <25684.942387283@sss.pgh.pa.us> <383118B2.AAC9E730@tpf.co.jp>
Content-Type: text/plain; charset=iso-2022-jp
Content-Transfer-Encoding: 7bit

Sorry,
I sent a mail by mistake.
Ignore my previous mail.

inoue wrote:

Tom Lane wrote:

Tatsuo Ishii <t-ishii@sra.co.jp> writes:

LO is a dead end. What we really want to do is eliminate tuple-size
restrictions and then have large ordinary fields (probably of type
bytea) in regular tuples. I'd suggest working on compression in that
context, say as a new data type called "bytez" or something like that.

It sounds ideal but I remember that Vadim said inserting a 2GB record
is not good idea since it will be written into the log too. If it's a
necessary limitation from the point of view of WAL, we have to accept
it, I think.

LO won't make that any better: the data still goes into a table.
You'd have 2GB worth of WAL entries either way.

The only thing LO would do for you is divide the data into block-sized
tuples, so there would be a bunch of little WAL entries instead of one
big one. But that'd probably be easy to duplicate too. If we implement
big tuples by chaining together disk-block-sized segments, which seems
like the most likely approach, couldn't WAL log each segment as a
separate log entry? If so, there's almost no difference between LO and
inline field for logging purposes.

I don't know LO well.
But seems LO allows partial update.
Big tuples
If so,isn't it a significant difference ?

Regards.

Hiroshi Inoue
Inoue@tpf.co.jp

From bouncefilter Tue Nov 16 11:29:15 1999
Received: from genisys.gtv.ca (h-207-228-97-106.gen.cadvision.com
[207.228.97.106]) by hub.org (8.9.3/8.9.3) with ESMTP id LAA48853
for <pgsql-hackers@postgreSQL.org>;
Tue, 16 Nov 1999 11:28:57 -0500 (EST) (envelope-from aaron@gtv.ca)
Received: from stilborne (h-207-228-97-110.gen.cadvision.com [207.228.97.110])
by genisys.gtv.ca (8.9.3/8.8.7) with SMTP id JAA25932
for <pgsql-hackers@postgreSQL.org>; Tue, 16 Nov 1999 09:29:19 -0700
From: "Aaron J. Seigo" <aaron@gtv.ca>
To: pgsql-hackers@postgreSQL.org
Subject: replication
Date: Tue, 16 Nov 1999 09:26:50 -0700
X-Mailer: KMail [version 1.0.28]
Content-Type: text/plain
References: <16514.942357396@sss.pgh.pa.us> <382BA0FB.82D96E3C@flame.co.za>
In-Reply-To: <382BA0FB.82D96E3C@flame.co.za>
MIME-Version: 1.0
Message-Id: <99111609271703.18220@stilborne>
Content-Transfer-Encoding: 8bit

hi...

is anyone working on replication services in pgsql?

--
Aaron J. Seigo
Sys Admin

From bouncefilter Tue Nov 16 18:51:20 1999
Received: (from news@localhost) by hub.org (8.9.3/8.9.3) id SAA14651
for pgsql-hackers@postgresql.org; Tue, 16 Nov 1999 18:51:10 -0500 (EST)
(envelope-from news)
X-Authentication-Warning: hub.org: news set sender to <news> using -f
Message-ID: <3831EC53.D5B285F3@iserv.net>
From: Tyson Oswald <tysono@iserv.net>
X-Mailer: Mozilla 4.7 [en] (WinNT; I)
X-Accept-Language: en
MIME-Version: 1.0
X-Newsgroups: comp.databases.postgresql.hackers
Subject: Slow access
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Lines: 63
X-Trace: typ11.nn.bcandid.com 942796267 204.157.27.218 (Tue,
16 Nov 1999 18:51:07 EST)
Organization: bCandid - Powering the world's discussions - http://bCandid.com
Date: Tue, 16 Nov 1999 18:44:19 -0500
To: pgsql-hackers@postgresql.org

Hello,
Not sure is this is the place for this but, I have two object
created a table (customers) and a view cust which joins customers and
cust runs pretty slow when select count(*) from cust. I think it may be
an index issue, can anyone help? there are about 1200 records in
customers and it takes about 30 seconds to count(*). Also how to I
create a foreign key constraint?

Tyson Oswald

Create table Customers
(
id serial,
Name varchar(25),
UID varchar(7),
Ext char(4),
fkMailcode int4,
fkContext int4,
fkServer int4,
fkAdmin int4
);

drop index idx_Customers_fkAdmin;
drop index idx_Customers_fkMailcode;
drop index idx_Customers_fkServer;
drop index idx_Customers_fkContext;

Create index idx_Customers_fkAdmin on Customers(fkadmin);
Create index idx_Customers_fkMailcode on Customers(fkMailcode);
Create index idx_Customers_fkContext on Customers(fkContext);
Create index idx_Customers_fkServer on Customers(fkServer);

Create table Codes
(
id serial,
description varchar(50),
type varchar(5),
typecode varchar(5)
);
create view cust as
select
c.name,
c.ext,
c.uid,
tmail.description as mail,
tcontext.description as context,
tserver.description as server,
admins.name as admin,
c.id
from
customers c,
codes tmail,
codes tserver,
codes tcontext,
admins
where
c.fkadmin=admins.id
and c.fkmailcode = tmail.id
and c.fkcontext = tcontext.id
and c.fkserver = tserver.id;

From bouncefilter Wed Nov 17 00:13:25 1999
Received: from wallace.ece.rice.edu (root@wallace.ece.rice.edu
[128.42.12.154])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA63364
for <pgsql-hackers@postgresql.org>;
Wed, 17 Nov 1999 00:13:17 -0500 (EST)
(envelope-from reedstrm@wallace.ece.rice.edu)
Received: by wallace.ece.rice.edu via sendmail from stdin
id <m11nxP8-000LECC@wallace.ece.rice.edu> (Debian Smail3.2.0.102)
for pgsql-hackers@postgresql.org; Tue, 16 Nov 1999 23:13:14 -0600 (CST)
Date: Tue, 16 Nov 1999 23:13:14 -0600
From: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
To: Christopher Petrilli <petrilli@digicool.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: PostgreSQL install
Message-ID: <19991116231314.A1880@wallace.ece.rice.edu>
References: <19991116223509.A1664@wallace.ece.rice.edu>
<B4579DEF.615E%petrilli@digicool.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Mailer: Mutt 0.95.3i
In-Reply-To: <B4579DEF.615E%petrilli@digicool.com>;
from Christopher Petrilli on Tue, Nov 16, 1999 at 11:49:03PM -0500

Hey Hackers -
I wouldn't normally forward an install problem from general to the
hackers list, but Chris is the database guy at Digital Creations, the
company behind Zope, the really cool web app. building tool, that Sybase
recently endorsed as their offical web frontend. Any of you FreeBSD
types recognize the problem?

Ross

On Tue, Nov 16, 1999 at 11:49:03PM -0500, Christopher Petrilli wrote:

On 11/16/99 11:35 PM, Ross J. Reedstrom at reedstrm@wallace.ece.rice.edu
wrote:

Chris -
Did I see you post to postgresql-general, looking for help with an install
on one of the BSDs? I seem to have had a snafu with my email clients, and
lost a few emails today (Mutt doesn't lock files properly...) so I can't
find the exact mail. If you haven't resolved the build/install problem,
let me know: a number of the core developers run various BSD flavors
(www.postgresql.org is FreeBSD, for example) so this should be easily
resolvable, although I run linux, myself.

Yeah, basically it looks like:

./configure
make
make install
instaldb

I seem to remember from your other post that you did get this last command
right: initdb

However, it is critical that initdb be run as the postgres user, rather
than as root. Other than that, I don't know. I'll forward your message
to the hackers list.

Ah, one last thought: how do you set up access to shared libraries
on FreeBSD? the make install will have dropped several libs in
/usr/local/pgsql/lib that the executables need access to.

Dosn't create the PG_VERSION files, nor the pg_user tables correctly... I
tried it on 3 different FreeBSD3.3 machines each downloaded seperately...
bizarre.

And as I recall, this is the pgsql-6.5.3 tar ball.

Chris
--
| Christopher Petrilli Python Powered Digital Creations, Inc.
| petrilli@digicool.com http://www.digicool.com

From bouncefilter Wed Nov 17 00:27:24 1999
Received: from gandalf.digicool.com (digitalcreations05.erols.com
[216.164.72.5]) by hub.org (8.9.3/8.9.3) with ESMTP id AAA66269
for <pgsql-hackers@postgresql.org>;
Wed, 17 Nov 1999 00:27:21 -0500 (EST)
(envelope-from petrilli@digicool.com)
Received: from [24.6.104.179] (cj479042-a.alex1.va.home.com [24.6.104.179]) by
gandalf.digicool.com with SMTP (Microsoft Exchange Internet
Mail Service Version 5.5.1960.3)
id WSNC13H4; Wed, 17 Nov 1999 00:37:20 -0500
User-Agent: Microsoft Outlook Express Macintosh Edition - 5.0 (1513)
Date: Wed, 17 Nov 1999 00:27:03 -0500
Subject: Re: PostgreSQL install
From: Christopher Petrilli <petrilli@digicool.com>
To: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
CC: <pgsql-hackers@postgresql.org>
Message-ID: <B457A6D7.616B%petrilli@digicool.com>
In-Reply-To: <19991116231314.A1880@wallace.ece.rice.edu>
Mime-version: 1.0
Content-type: text/plain; charset="US-ASCII"
Content-transfer-encoding: 7bit

On 11/17/99 12:13 AM, Ross J. Reedstrom at reedstrm@wallace.ece.rice.edu
wrote:

Hey Hackers -
I wouldn't normally forward an install problem from general to the
hackers list, but Chris is the database guy at Digital Creations, the
company behind Zope, the really cool web app. building tool, that Sybase
recently endorsed as their offical web frontend. Any of you FreeBSD
types recognize the problem?

Flattry flattery.... we're jus evaluating options for support in the
future, no commitments, of course, but we've had some people ask, and so we
feel the need to understand it better. I've only used Postgres (under
Stonebraker) and Illustra.

On Tue, Nov 16, 1999 at 11:49:03PM -0500, Christopher Petrilli wrote:

On 11/16/99 11:35 PM, Ross J. Reedstrom at reedstrm@wallace.ece.rice.edu
wrote:

Chris -
Did I see you post to postgresql-general, looking for help with an install
on one of the BSDs? I seem to have had a snafu with my email clients, and
lost a few emails today (Mutt doesn't lock files properly...) so I can't
find the exact mail. If you haven't resolved the build/install problem,
let me know: a number of the core developers run various BSD flavors
(www.postgresql.org is FreeBSD, for example) so this should be easily
resolvable, although I run linux, myself.

Yeah, basically it looks like:

./configure
make
make install
instaldb

I seem to remember from your other post that you did get this last command
right: initdb

Yeah, that part was from memory :-)

However, it is critical that initdb be run as the postgres user, rather
than as root. Other than that, I don't know. I'll forward your message
to the hackers list.

Yup, run as uid 'postgres'.

Ah, one last thought: how do you set up access to shared libraries
on FreeBSD? the make install will have dropped several libs in
/usr/local/pgsql/lib that the executables need access to.

I put LD_LIBRARY_PATH to be correct. This is in the startup for the user,
so it should always be correct.

Dosn't create the PG_VERSION files, nor the pg_user tables correctly... I
tried it on 3 different FreeBSD3.3 machines each downloaded seperately...
bizarre.

And as I recall, this is the pgsql-6.5.3 tar ball.

Yup, off the main FTP site... just ./configure and go with it...

Chris
--
| Christopher Petrilli Python Powered Digital Creations, Inc.
| petrilli@digicool.com http://www.digicool.com

From bouncefilter Wed Nov 17 00:41:24 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA68119
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 00:41:09 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id AAA24549;
Wed, 17 Nov 1999 00:39:57 -0500 (EST)
To: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
cc: Christopher Petrilli <petrilli@digicool.com>, pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Re: PostgreSQL install
In-reply-to: Your message of Tue, 16 Nov 1999 23:13:14 -0600
<19991116231314.A1880@wallace.ece.rice.edu>
Date: Wed, 17 Nov 1999 00:39:56 -0500
Message-ID: <24547.942817196@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

"Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu> writes:

Yeah, basically it looks like:
./configure
make
make install
instaldb

I seem to remember from your other post that you did get this last command
right: initdb

Two other thoughts: (1) is initdb in your path, and is the *right
version* the first one in your path (I've been burnt by that on
upgrades). (2) I think that initdb requires USER, PGLIB, PGDATA
env variables to be set properly for reliable operation; also PATH
had better find the new version of postgres, psql, etc before any
older versions.

If that doesn't strike gold, a copy of the failing initdb session's
printout would be useful.

regards, tom lane

From bouncefilter Wed Nov 17 05:16:27 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id FAA05853
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 05:15:54 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgreSQL.org
id m11o20C-0003kIC; Wed, 17 Nov 99 11:07 MET
Message-Id: <m11o20C-0003kIC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: regression tests
To: pgsql-hackers@postgreSQL.org (PostgreSQL HACKERS)
Date: Wed, 17 Nov 1999 11:07:48 +0100 (MET)
Reply-To: wieck@debis.com (Jan Wieck)
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Hi,

I know that it's the new psql output format why all the
regression tests currently fail. But I think we are in this
state for a little too long now. With the latest CVS I got
this near the end of the suite after the plpgsql test:

NOTICE: trying to delete a reldesc that does not exist.
NOTICE: trying to delete a reldesc that does not exist.
Server process (pid 12207) exited with status 139 at Wed Nov 17 10:57:36 1999
Terminating any active server processes...
Server processes were terminated at Wed Nov 17 10:57:36 1999
Reinitializing shared memory and semaphores
DEBUG: Data Base System is starting up at Wed Nov 17 10:57:36 1999

This indicates that someone made changes that really broke
something and since he wasn't able to get any useful results
from a regression run, he just didn't do it.

I see a little problem with checking if the output is still
O.K. too. It seems that psql now buffers all the query
result messages until a SELECT is done. So if the regression
input contains only INSERT/UPDATE/DELETE statements, all the
responses are at the end, not after each statement.

It's really a mess. How should someone check if a system
catalog change is O.K. in this situation? I intend to do so
soon!

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Wed Nov 17 06:07:29 1999
Received: from px.com.br ([200.253.250.2])
by hub.org (8.9.3/8.9.3) with SMTP id GAA12108
for <pgsql-hackers@postgresql.org>;
Wed, 17 Nov 1999 06:06:41 -0500 (EST)
(envelope-from rcoelho@px.com.br)
Received: from [200.253.250.3] by px.com.br (AIX 4.1/UCB 5.64/4.03)
id AA15984; Wed, 17 Nov 1999 07:03:09 -0400
Message-Id: <002401bf30e3$c1a6b260$03fafdc8@px.com.br>
From: "Ricardo Coelho" <rcoelho@px.com.br>
To: <pgsql-hackers@postgresql.org>
Subject: select MIN/MAX when no row selected
Date: Wed, 17 Nov 1999 08:08:51 -0200
Organization:
Mime-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-Msmail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.00.2314.1300
X-Mimeole: Produced By Microsoft MimeOLE V5.00.2314.1300

Hi All,

I think I had read this question with count(*) before.....

PgSql returns one row with a null field when we use select MIN or MAX on a
table, but the result should be "no rows selected".

I didn't try with C function, but I got the same result with psql.

So, I'm sending a Java example to Peter.

try {
rs=stmt.executeQuery("select min(field1) from tab where
field1>maxValueOfField1");
if (rs.next()) {
System.out.println("I found a row !!!!");
theResult=rs.getString(1);
if (theResult==null)
System.out.println("Min of field1 is NULL !!!!");
}
} catch (.............

maxValueofField1 = select max(field1) from tab;

Is it correct ?

I'm using PgSql 6.5.2, RHLinux/Intel 6.0

Thanks,

Ricardo Coelho.

From bouncefilter Wed Nov 17 06:08:29 1999
Received: from Radha.DoCS.UU.SE (root@Radha.DoCS.UU.SE [130.238.9.99])
by hub.org (8.9.3/8.9.3) with SMTP id GAA12166
for <hackers@postgreSQL.org>; Wed, 17 Nov 1999 06:07:52 -0500 (EST)
(envelope-from e99re41@DoCS.UU.SE)
Received: from Lodjur.DoCS.UU.SE (e99re41@Lodjur.DoCS.UU.SE [130.238.9.178])
by Radha.DoCS.UU.SE (8.6.12/8.6.12) with ESMTP id MAA10768;
Wed, 17 Nov 1999 12:07:47 +0100
Received: from localhost (e99re41@localhost) by Lodjur.DoCS.UU.SE
(8.6.12/8.6.12) with SMTP id MAA29954;
Wed, 17 Nov 1999 12:07:46 +0100
X-Authentication-Warning: Lodjur.DoCS.UU.SE: e99re41 owned process doing -bs
Date: Wed, 17 Nov 1999 12:07:45 +0100 (MET)
From: Peter Eisentraut <e99re41@DoCS.UU.SE>
Reply-To: Peter Eisentraut <peter_e@gmx.net>
To: Thomas Lockhart <lockhart@alumni.caltech.edu>
cc: Brian Waters <jbw@InnovaSystems.com>,
Postgres Hackers List <hackers@postgreSQL.org>
Subject: Re: [HACKERS] Re: Postgresql Docs....
In-Reply-To: <3830C6DA.5CC933CB@alumni.caltech.edu>
Message-ID: <Pine.GSO.4.02A.9911171205190.29920-100000@Lodjur.DoCS.UU.SE>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Tue, 16 Nov 1999, Thomas Lockhart wrote:

So what about this would not be considered public domain software?

Something can not be both Copyrighted and in the public domain.

Hmm. I've taken this on-list, just in case someone else has a comment.
But in the absence of alternate information, I'll just assume that we
are not public domain software. But I sure still have the feeling that
we are getting gypped by the legaleze.

How about "free software" or "freely available"? As in "free to do
whatever you want", not Free(tm) as in FSF. IMHO, "open source" sounds to
buzzword-compliant these days.

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Wed Nov 17 06:12:28 1999
Received: from Radha.DoCS.UU.SE (root@Radha.DoCS.UU.SE [130.238.9.99])
by hub.org (8.9.3/8.9.3) with SMTP id GAA12763
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 06:11:44 -0500 (EST)
(envelope-from e99re41@DoCS.UU.SE)
Received: from Lodjur.DoCS.UU.SE (e99re41@Lodjur.DoCS.UU.SE [130.238.9.178])
by Radha.DoCS.UU.SE (8.6.12/8.6.12) with ESMTP id MAA11068;
Wed, 17 Nov 1999 12:11:41 +0100
Received: from localhost (e99re41@localhost) by Lodjur.DoCS.UU.SE
(8.6.12/8.6.12) with SMTP id MAA29958;
Wed, 17 Nov 1999 12:11:39 +0100
X-Authentication-Warning: Lodjur.DoCS.UU.SE: e99re41 owned process doing -bs
Date: Wed, 17 Nov 1999 12:11:39 +0100 (MET)
From: Peter Eisentraut <e99re41@DoCS.UU.SE>
Reply-To: Peter Eisentraut <peter_e@gmx.net>
To: Hiroshi Inoue <Inoue@tpf.co.jp>
cc: Bruce Momjian <pgman@candle.pha.pa.us>,
PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] RE: Unique indexes on system tables
In-Reply-To: <000001bf2ff3$0391abe0$2801007e@cadzone.tpf.co.jp>
Message-ID: <Pine.GSO.4.02A.9911171209050.29920-100000@Lodjur.DoCS.UU.SE>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Tue, 16 Nov 1999, Hiroshi Inoue wrote:

I am only afraid of index corruption.
The more we have system indexes,the more index corruption would happen.

Just a concerned user question: Why does index corruption seem to happen
so often or is a genuine concern? Wouldn't the next thing be table
corruption? Or are indices optimized for speed rather than correctness
because they don't contain important data?

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Wed Nov 17 06:22:28 1999
Received: from Radha.DoCS.UU.SE (root@Radha.DoCS.UU.SE [130.238.9.99])
by hub.org (8.9.3/8.9.3) with SMTP id GAA14377
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 06:22:15 -0500 (EST)
(envelope-from e99re41@DoCS.UU.SE)
Received: from Lodjur.DoCS.UU.SE (e99re41@Lodjur.DoCS.UU.SE [130.238.9.178])
by Radha.DoCS.UU.SE (8.6.12/8.6.12) with ESMTP id MAA11660;
Wed, 17 Nov 1999 12:22:13 +0100
Received: from localhost (e99re41@localhost) by Lodjur.DoCS.UU.SE
(8.6.12/8.6.12) with SMTP id MAA29962;
Wed, 17 Nov 1999 12:22:12 +0100
X-Authentication-Warning: Lodjur.DoCS.UU.SE: e99re41 owned process doing -bs
Date: Wed, 17 Nov 1999 12:22:11 +0100 (MET)
From: Peter Eisentraut <e99re41@DoCS.UU.SE>
Reply-To: Peter Eisentraut <peter_e@gmx.net>
To: Jan Wieck <wieck@debis.com>
cc: PostgreSQL HACKERS <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] regression tests
In-Reply-To: <m11o20C-0003kIC@orion.SAPserv.Hamburg.dsh.de>
Message-ID: <Pine.GSO.4.02A.9911171215150.29920-100000@Lodjur.DoCS.UU.SE>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Wed, 17 Nov 1999, Jan Wieck wrote:

I know that it's the new psql output format why all the
regression tests currently fail. But I think we are in this
state for a little too long now. With the latest CVS I got

As I mentioned before: use the old psql for regression tests and/or just
run the regression tests once with the old one and once with the new one
and make those results the new reference. (Maybe I'm oversimplifying here,
though.)

Once again this thought also: How about running the regression tests on a
single user postgres backend directly? That way you don't rely on some
obscure frontend and some client library which might change soon, too.
Also you have more control over internals. Finally, you could easily run
the regression tests on an uninstalled build. Think ./configure; make;
make check; make install. Or am I way out there now?

I see a little problem with checking if the output is still
O.K. too. It seems that psql now buffers all the query
result messages until a SELECT is done. So if the regression
input contains only INSERT/UPDATE/DELETE statements, all the
responses are at the end, not after each statement.

Huh? psql doesn't buffer anything. Could you please elaborate on this
and/or give me an example? I never heard of that one and I thought Bruce
was a really thorough tester . . .

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Wed Nov 17 06:55:28 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id GAA18210
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 06:54:42 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgreSQL.org
id m11o3Xh-0003kIC; Wed, 17 Nov 99 12:46 MET
Message-Id: <m11o3Xh-0003kIC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] regression tests
To: peter_e@gmx.net
Date: Wed, 17 Nov 1999 12:46:29 +0100 (MET)
Cc: wieck@debis.com, pgsql-hackers@postgreSQL.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <Pine.GSO.4.02A.9911171215150.29920-100000@Lodjur.DoCS.UU.SE>
from "Peter Eisentraut" at Nov 17, 99 12:22:11 pm
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Peter Eisentraut wrote:

Once again this thought also: How about running the regression tests on a
single user postgres backend directly? That way you don't rely on some
obscure frontend and some client library which might change soon, too.
Also you have more control over internals. Finally, you could easily run
the regression tests on an uninstalled build. Think ./configure; make;
make check; make install. Or am I way out there now?

That should have been done BEFORE messing up anything. This
hasn't been done, so I think it's the job of those who change
output formats, to provide new expected regression results
too. This hasn't been done too, and that's bad.

I see a little problem with checking if the output is still
O.K. too. It seems that psql now buffers all the query
result messages until a SELECT is done. So if the regression
input contains only INSERT/UPDATE/DELETE statements, all the
responses are at the end, not after each statement.

Huh? psql doesn't buffer anything. Could you please elaborate on this
and/or give me an example? I never heard of that one and I thought Bruce
was a really thorough tester . . .

As I see, the result messages aren't in the (old) expected
outputs at all. But they are now. From the boolean test:

CREATE TABLE BOOLTBL1 (f1 bool);

INSERT INTO BOOLTBL1 (f1) VALUES ('t'::bool);

INSERT INTO BOOLTBL1 (f1) VALUES ('True'::bool);

INSERT INTO BOOLTBL1 (f1) VALUES ('true'::bool);

-- BOOLTBL1 should be full of true's at this point
SELECT '' AS t_3, BOOLTBL1.*;
CREATE
INSERT 18633 1
INSERT 18634 1
INSERT 18635 1
t_3 | f1
-----+----
| t
| t
| t
(3 rows)

As you can see, the CREATE and INSERT responses are printed
after the SELECT statement, just before it's own output.

Again, if someone changes things that change output, he has
to provide new expected results for the regression suite. If
the changes to psql are still a work in progress, it should
have been done on separated sources until at least the output
format is stable.

What actually happened isn't good practice (IMHO). Ask all
other developers to work around some temporary misbehaviour
that makes the entire backend development a blind flight. And
the fatal abnormal backend termination at the end of the
regression show's what this lazyness can end in. ISTM someone
has broken something and didn't notice. Thus, at least that
other one didn't do it with your mentioned workaround.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Wed Nov 17 07:48:29 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id HAA24072
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 07:47:49 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgreSQL.org
id m11o4N4-0003kIC; Wed, 17 Nov 99 13:39 MET
Message-Id: <m11o4N4-0003kIC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] RE: Unique indexes on system tables
To: peter_e@gmx.net
Date: Wed, 17 Nov 1999 13:39:34 +0100 (MET)
Cc: Inoue@tpf.co.jp, pgman@candle.pha.pa.us, pgsql-hackers@postgreSQL.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <Pine.GSO.4.02A.9911171209050.29920-100000@Lodjur.DoCS.UU.SE>
from "Peter Eisentraut" at Nov 17, 99 12:11:39 pm
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

On Tue, 16 Nov 1999, Hiroshi Inoue wrote:

I am only afraid of index corruption.
The more we have system indexes,the more index corruption would happen.

Just a concerned user question: Why does index corruption seem to happen
so often or is a genuine concern? Wouldn't the next thing be table
corruption? Or are indices optimized for speed rather than correctness
because they don't contain important data?

There are more complicated concurrency issues on indices than
for regular tables. That's where the corrupt indices but not
tables come from.

For a user index, this isn't very critical, because a
drop/create index sequence will recover to consistent data.

For system catalog indices, this is a desaster, because you
cannot drop and recreate indices on system tables. At least
we need to tackle this problem by reincarnating reindexdb.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Wed Nov 17 07:57:31 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id HAA25306
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 07:56:44 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgreSQL.org
id m11o4Vr-0003kIC; Wed, 17 Nov 99 13:48 MET
Message-Id: <m11o4Vr-0003kIC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] regression tests
To: wieck@debis.com
Date: Wed, 17 Nov 1999 13:48:39 +0100 (MET)
Cc: pgsql-hackers@postgreSQL.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <m11o20C-0003kIC@orion.SAPserv.Hamburg.dsh.de> from "Jan Wieck"
at Nov 17, 99 11:07:48 am
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

I wrote:

NOTICE: trying to delete a reldesc that does not exist.
NOTICE: trying to delete a reldesc that does not exist.
Server process (pid 12207) exited with status 139 at Wed Nov 17 10:57:36 1999
Terminating any active server processes...
Server processes were terminated at Wed Nov 17 10:57:36 1999
Reinitializing shared memory and semaphores
DEBUG: Data Base System is starting up at Wed Nov 17 10:57:36 1999

I took Peter Eisentraut's advice and did it with the old pslq
(thanks for the hint).

This problem (as expected) remains and happens in the temp
test. The two notices occur on creating the temp table and
the index on it. After that, the database connection get's
lost on the attempt to drop the temp table.

Since the postmaster is doing recovery then, the numeric test
hasn't been run. All other tests are still O.K.

The question is, who did something that could cause this
error?

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Wed Nov 17 08:34:29 1999
Received: from Radha.DoCS.UU.SE (root@Radha.DoCS.UU.SE [130.238.9.99])
by hub.org (8.9.3/8.9.3) with SMTP id IAA29431
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 08:33:40 -0500 (EST)
(envelope-from e99re41@DoCS.UU.SE)
Received: from Uggla.DoCS.UU.SE (e99re41@Uggla.DoCS.UU.SE [130.238.9.156]) by
Radha.DoCS.UU.SE (8.6.12/8.6.12) with ESMTP id OAA19306;
Wed, 17 Nov 1999 14:33:38 +0100
Received: from localhost (e99re41@localhost) by Uggla.DoCS.UU.SE
(8.6.12/8.6.12) with SMTP id OAA18789;
Wed, 17 Nov 1999 14:33:37 +0100
X-Authentication-Warning: Uggla.DoCS.UU.SE: e99re41 owned process doing -bs
Date: Wed, 17 Nov 1999 14:33:37 +0100 (MET)
From: Peter Eisentraut <e99re41@DoCS.UU.SE>
Reply-To: Peter Eisentraut <peter_e@gmx.net>
To: Jan Wieck <wieck@debis.com>
cc: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] regression tests
In-Reply-To: <m11o3Xh-0003kIC@orion.SAPserv.Hamburg.dsh.de>
Message-ID: <Pine.GSO.4.02A.9911171427070.18751-100000@Uggla.DoCS.UU.SE>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Wed, 17 Nov 1999, Jan Wieck wrote:

That should have been done BEFORE messing up anything. This
hasn't been done, so I think it's the job of those who change
output formats, to provide new expected regression results
too. This hasn't been done too, and that's bad.

I explicitly informed everyone that this would happen a long time before I
finalized psql. Nobody seemed to care a lot. Now, weeks after the fact
some people start wondering that perhaps they want to run regression tests
once in a while. I'm not the regression test maintainer, nor do I have
knowledge of how to remake them, so all I can do is inform everyone and
cooperate on anything that's necessary. But silence is implicit approval.

--
Peter Eisentraut Sernanders vaeg 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Wed Nov 17 10:08:30 1999
Received: from www.wgcr.org (IDENT:root@www.wgcr.org [206.74.232.194])
by hub.org (8.9.3/8.9.3) with ESMTP id KAA41201
for <hackers@postgreSQL.org>; Wed, 17 Nov 1999 10:07:55 -0500 (EST)
(envelope-from lamar.owen@wgcr.org)
Received: from wgcr.org ([206.74.232.197])
by www.wgcr.org (8.8.7/8.8.5) with ESMTP id KAA21672;
Wed, 17 Nov 1999 10:07:27 -0500
Message-ID: <3832C4A9.BEDCD413@wgcr.org>
Date: Wed, 17 Nov 1999 10:07:21 -0500
From: Lamar Owen <lamar.owen@wgcr.org>
Organization: WGCR Internet Radio
X-Mailer: Mozilla 4.61 [en] (Win95; I)
X-Accept-Language: en
MIME-Version: 1.0
To: Peter Eisentraut <peter_e@gmx.net>
CC: Thomas Lockhart <lockhart@alumni.caltech.edu>,
Brian Waters <jbw@InnovaSystems.com>,
Postgres Hackers List <hackers@postgreSQL.org>
Subject: Re: [HACKERS] Re: Postgresql Docs....
References: <Pine.GSO.4.02A.9911171205190.29920-100000@Lodjur.DoCS.UU.SE>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Peter Eisentraut wrote:

On Tue, 16 Nov 1999, Thomas Lockhart wrote:

So what about this would not be considered public domain software?

Something can not be both Copyrighted and in the public domain.

Hmm. I've taken this on-list, just in case someone else has a comment.
But in the absence of alternate information, I'll just assume that we
are not public domain software. But I sure still have the feeling that
we are getting gypped by the legaleze.

How about "free software" or "freely available"? As in "free to do
whatever you want", not Free(tm) as in FSF. IMHO, "open source" sounds to
buzzword-compliant these days.

How about simply "BSD licensed?"

--
Lamar Owen
WGCR Internet Radio

From bouncefilter Wed Nov 17 11:01:31 1999
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA47855
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 11:00:49 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
KAA10335;
Wed, 17 Nov 1999 10:52:40 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911171552.KAA10335@candle.pha.pa.us>
Subject: Re: [HACKERS] regression tests
In-Reply-To: <m11o4Vr-0003kIC@orion.SAPserv.Hamburg.dsh.de> from Jan Wieck at
"Nov 17, 1999 01:48:39 pm"
To: Jan Wieck <wieck@debis.com>
Date: Wed, 17 Nov 1999 10:52:40 -0500 (EST)
CC: pgsql-hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

I took Peter Eisentraut's advice and did it with the old pslq
(thanks for the hint).

This problem (as expected) remains and happens in the temp
test. The two notices occur on creating the temp table and
the index on it. After that, the database connection get's
lost on the attempt to drop the temp table.

Since the postmaster is doing recovery then, the numeric test
hasn't been run. All other tests are still O.K.

The question is, who did something that could cause this
error?

I am sure it was me changing the temp behavior. I will look at it.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Wed Nov 17 11:06:31 1999
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA48495
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 11:05:40 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
LAA11548;
Wed, 17 Nov 1999 11:04:42 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911171604.LAA11548@candle.pha.pa.us>
Subject: Re: [HACKERS] regression tests
In-Reply-To: From "(env:" "pgman)" at "Nov 17, 1999 10:52:40 am"
To: pgman@candle.pha.pa.us
Date: Wed, 17 Nov 1999 11:04:42 -0500 (EST)
CC: Jan Wieck <wieck@debis.com>, pgsql-hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

I took Peter Eisentraut's advice and did it with the old pslq
(thanks for the hint).

This problem (as expected) remains and happens in the temp
test. The two notices occur on creating the temp table and
the index on it. After that, the database connection get's
lost on the attempt to drop the temp table.

Since the postmaster is doing recovery then, the numeric test
hasn't been run. All other tests are still O.K.

The question is, who did something that could cause this
error?

I am sure it was me changing the temp behavior. I will look at it.

Jan, I can't reproduce the temp regression failure here. I did make
changes yesterday morning to this. I assume you have an updated cvs,
right?

Anyway to make the default numeric regression test run faster. It seems
to take quite a while compared to the others.

We certainly need someone to update the regression tests to match the
new format so people can continue running regression tests before
applying patches.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Wed Nov 17 11:26:32 1999
Received: from genisys.gtv.ca (h-207-228-97-106.gen.cadvision.com
[207.228.97.106]) by hub.org (8.9.3/8.9.3) with ESMTP id LAA51172
for <hackers@postgresql.org>; Wed, 17 Nov 1999 11:25:58 -0500 (EST)
(envelope-from aaron@gtv.ca)
Received: from stilborne (h-207-228-97-110.gen.cadvision.com [207.228.97.110])
by genisys.gtv.ca (8.9.3/8.8.7) with SMTP id JAA28453;
Wed, 17 Nov 1999 09:26:00 -0700
From: "Aaron J. Seigo" <aaron@gtv.ca>
To: Lamar Owen <lamar.owen@wgcr.org>
Subject: Re: [HACKERS] Re: Postgresql Docs....
Date: Wed, 17 Nov 1999 09:10:12 -0700
X-Mailer: KMail [version 1.0.28]
Content-Type: text/plain
References: <Pine.GSO.4.02A.9911171205190.29920-100000@Lodjur.DoCS.UU.SE>
<3832C4A9.BEDCD413@wgcr.org>
In-Reply-To: <3832C4A9.BEDCD413@wgcr.org>
Cc: Postgres Hackers List <hackers@postgresql.org>
MIME-Version: 1.0
Message-Id: <99111709235700.23706@stilborne>
Content-Transfer-Encoding: 8bit

hi...

Hmm. I've taken this on-list, just in case someone else has a comment.
But in the absence of alternate information, I'll just assume that we
are not public domain software. But I sure still have the feeling that
we are getting gypped by the legaleze.

How about "free software" or "freely available"? As in "free to do
whatever you want", not Free(tm) as in FSF. IMHO, "open source" sounds to
buzzword-compliant these days.

How about simply "BSD licensed?"

traditional "BSD Liscences" have that silly advertising
clause.. which postgres does as well, unfortunately... quite honestly, i find
that irritating and antiquated. *shrug* not like the regents are exactly doing
anything important w/postgres now, right? and for all the shouting of "its
TRULY free", there are string attatched...

anyways... as long as a lisence protects what needs to be protected, all is
good. instead of arguing silly semantics (BSD/XFree/Public
Domain/GPL/blahblahblah) we should be looking more importantly at which rights
we want to secure and which we don't really care about.

BSD/XFree style liscences are good when a permisiveness is desired (like apache
and how it help keep HTTP on track) and bad when you aren't trying to enforce
certain standards but endevouring to keep a software available to others...
fortunately, postgres isn't a trivial piece of software, which serves as a
protection. but it isn't so complex that it couldn't be taken on by another
entity. in fact, a compay could easily come along and swoop up the core 4
programmers with terrific job offers and that would pretty much be that =)
lets hope people's scruples and dedications are in the place We would like them
to be...

public domain would be horrid. BSD/XFree style is fine, though probably more
permissive than needed (and perhaps even desired). the GPL is
probably a little too demanding for this type of software though...

it would be interesting to see it settle somewhere in between. e.g. if you want
to extend it, GREAT! if you distribute it gratis, you have to make it available
to everyone... perhaps require source code be available for the current
release (not distributed, but available)... and if someone wants to _sell_ it
as a closed package, fine! but require they give something back to the postgres
development team.

--
Aaron J. Seigo
Sys Admin

From bouncefilter Wed Nov 17 11:54:32 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id LAA54461
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 11:53:39 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id LAA26002;
Wed, 17 Nov 1999 11:51:24 -0500 (EST)
To: Bruce Momjian <pgman@candle.pha.pa.us>
cc: Jan Wieck <wieck@debis.com>, pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] regression tests
In-reply-to: Your message of Wed, 17 Nov 1999 11:04:42 -0500 (EST)
<199911171604.LAA11548@candle.pha.pa.us>
Date: Wed, 17 Nov 1999 11:51:24 -0500
Message-ID: <26000.942857484@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Bruce Momjian <pgman@candle.pha.pa.us> writes:

We certainly need someone to update the regression tests to match the
new format so people can continue running regression tests before
applying patches.

I've been using old psql (per Peter's suggestion) to run regression
tests. I have not pulled CVS since Saturday but things seemed OK then.
If there is a breakage, it's recent.

I thought we were putting off committing a new set of regress test
expected outputs until the dust settles in the new psql. Isn't Peter
still tweaking the output format? Not much point in generating new
expected files until everyone agrees the format is frozen.

Of course there's a bit of a catch-22 situation here: since I am not
using the new psql, I'm not contributing any feedback on it. The same
is probably true of some other developers... when we do finally adopt
new psql (after regress test update) there may be a bunch of belated
requests for changes...

regards, tom lane

From bouncefilter Wed Nov 17 12:56:32 1999
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id MAA66294
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 12:56:02 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
MAA15385;
Wed, 17 Nov 1999 12:54:03 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911171754.MAA15385@candle.pha.pa.us>
Subject: Re: [HACKERS] regression tests
In-Reply-To: <26000.942857484@sss.pgh.pa.us> from Tom Lane at "Nov 17,
1999 11:51:24 am"
To: Tom Lane <tgl@sss.pgh.pa.us>
Date: Wed, 17 Nov 1999 12:54:03 -0500 (EST)
CC: Jan Wieck <wieck@debis.com>, pgsql-hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Bruce Momjian <pgman@candle.pha.pa.us> writes:

We certainly need someone to update the regression tests to match the
new format so people can continue running regression tests before
applying patches.

I've been using old psql (per Peter's suggestion) to run regression
tests. I have not pulled CVS since Saturday but things seemed OK then.
If there is a breakage, it's recent.

I thought we were putting off committing a new set of regress test
expected outputs until the dust settles in the new psql. Isn't Peter
still tweaking the output format? Not much point in generating new
expected files until everyone agrees the format is frozen.

Of course there's a bit of a catch-22 situation here: since I am not
using the new psql, I'm not contributing any feedback on it. The same
is probably true of some other developers... when we do finally adopt
new psql (after regress test update) there may be a bunch of belated
requests for changes...

Yes, I am waiting to see if anyone changes the format before updating
all the queries in my book.

Is everyone OK with the new format? Do you like it more or less than
the old one? Please someone weight in on one side or the other so we
can conclude this. People have been very quiet on this issue.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Wed Nov 17 13:03:32 1999
Received: from www2.hapenney.com ([207.22.245.171])
by hub.org (8.9.3/8.9.3) with ESMTP id NAA67145
for <hackers@postgreSQL.org>; Wed, 17 Nov 1999 13:02:39 -0500 (EST)
(envelope-from evan@4-am.com)
Received: from 4-am.com ([216.178.132.153])
by www2.hapenney.com (8.9.3/8.8.7) with ESMTP id MAA13972
for <hackers@postgreSQL.org>; Wed, 17 Nov 1999 12:02:21 -0600
Sender: evan@hapenney.com
Message-ID: <3832EDA3.92AE9AF7@4-am.com>
Date: Wed, 17 Nov 1999 12:02:12 -0600
From: Evan Simpson <evan@4-am.com>
X-Mailer: Mozilla 4.7 [en] (X11; I; Linux 2.2.13-7mdk i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Postgres Hackers List <hackers@postgreSQL.org>
Subject: Re: [HACKERS] Re: Postgresql Docs....
References: <Pine.GSO.4.02A.9911171205190.29920-100000@Lodjur.DoCS.UU.SE>
<3832C4A9.BEDCD413@wgcr.org> <99111709235700.23706@stilborne>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

One half-decent newspeak alternative I've seen to "free software" and "open
source" is "Wide-Open Source" (WOS). It seems designed to imply that your licence
is "open source, and then some", such as BSD-style without the advertising clause
and meaningless reference to Berkeley.

"Aaron J. Seigo" wrote:

How about "free software" or "freely available"? As in "free to do
whatever you want", not Free(tm) as in FSF. IMHO, "open source" sounds to
buzzword-compliant these days.

How about simply "BSD licensed?"

traditional "BSD Liscences" have that silly advertising
clause.. which postgres does as well, unfortunately... quite honestly, i find
that irritating and antiquated. *shrug* not like the regents are exactly doing
anything important w/postgres now, right? and for all the shouting of "its
TRULY free", there are string attatched...

Cheers,

Evan @ 4-am

From bouncefilter Wed Nov 17 13:51:33 1999
Received: from portal1.visa.com (portal1.visa.com [198.80.42.2])
by hub.org (8.9.3/8.9.3) with SMTP id NAA73452
for <pgsql-hackers@postgresql.org>;
Wed, 17 Nov 1999 13:51:15 -0500 (EST)
(envelope-from knarayan@visa.com)
Received: by portal1.visa.com id KAA07097
(InterLock SMTP Gateway 4.2 for pgsql-hackers@postgresql.org);
Wed, 17 Nov 1999 10:51:13 -0800
Received: by portal1.visa.com (Protected-side Proxy Mail Agent-1);
Wed, 17 Nov 1999 10:51:13 -0800
Message-Id: <E3940A45EE1CD21196100001FA444AE702B0D18E@sw720x019.visa.com>
From: "Narayanan, Kannan" <knarayan@visa.com>
To: "'pgsql-hackers@postgresql.org'" <pgsql-hackers@postgresql.org>
Subject: FW: How to select the millisecond/microsecond parts of the dateti
me column
Date: Wed, 17 Nov 1999 10:51:12 -0800
Importance: high
X-Priority: 1
Mime-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2448.0)
Content-Type: text/plain;
charset="iso-8859-1"

-----Original Message-----
From: Narayanan, Kannan
Sent: Wednesday, November 17, 1999 10:47 AM
To: 'pgsql-questions@postgresql.org'
Subject: How to select the millisecond/microsecond parts of the
datetime column
Importance: High

Hello,

Product version: Postgres 6.5.3

SELECT DATETIME('MILLISECOND', 'NOW'::DATETIME) always returns 0(ZERO) and
so does 'MICROSECOND'. However a reading of the manuals indicate that the
database supports much higher precision values. How do I retrieve the
additional precision values (beyond seconds) when I use the datetime
field? This is a requirement for a conversion project that I am working
on. Could someone help please.

Thanks
Kannan

From bouncefilter Wed Nov 17 14:07:33 1999
Received: from paprika.michvhf.com (paprika.michvhf.com [209.57.60.12])
by hub.org (8.9.3/8.9.3) with SMTP id OAA75194
for <hackers@postgresql.org>; Wed, 17 Nov 1999 14:07:18 -0500 (EST)
(envelope-from vev@michvhf.com)
Received: (qmail 5111 invoked by uid 1001); 17 Nov 1999 19:07:21 -0000
Message-ID: <XFMail.991117140721.vev@michvhf.com>
X-Mailer: XFMail 1.4.0 on FreeBSD
X-Priority: 3 (Normal)
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Resent-Date: Wed, 17 Nov 1999 09:49:20 -0800 (PST)
Resent-Message-Id: <19991117174920.10322.rocketmail@web2103.mail.yahoo.com>
Resent-From: =?iso-8859-1?q?nitin=20thakkar?= <nitinthakkar@yahoo.com>
Resent-To: webmaster@postgresql.org
Date: Wed, 17 Nov 1999 14:07:21 -0500 (EST)
X-Face: *<Qp5V!eyV,gni`N^N%1YX'$I&uuX]ay;
oq#ZL5Hn8EQsu'.oK0j9$#JM0V?!=Q^[i.81u9
@~=ZjeI}gHY`?2_1,xy/,Gde>0^4Iw)<k8}vg!%l;
&]@PF0LjU)N*m*2"R^UO+PAQ<w}/y)5UVE==w
H$q0*b`HN{+Ekeo?5V(0$MH&NZA3~vOThJxhY(7M:"`CrqO9[VC!^W&&eih!MTq4qk=Vg'd&`{dpgp
3-nck}7do'o/|<RI,
igc#cg8t|PZUEh{Rrx4<~tm`/G8E*wE{G:x}bva@[+YVT`g(u]*^!`1iO*
Sender: vev@paprika.michvhf.com
From: Vince Vielhaber <vev@michvhf.com>
To: hackers@postgresql.org
Subject: FW: Query

This was sent to webmaster@postgresql.org. Anyone have an answer
to it? I'm not familiar with oracle so I don't know what he's
talking about.

Vince.

-----FW: <19991117174920.10322.rocketmail@web2103.mail.yahoo.com>-----
From: =?iso-8859-1?q?nitin=20thakkar?= <nitinthakkar@yahoo.com>
To: webmaster@postgresql.org
Subject: Query

Hi,

I am PostGRESql user and have a query :

DOES POSTGRESQL SUPPORT DATABASE LINK AS IN ORACLE 8.0

Regards
Nitin

=====

__________________________________________________
Do You Yahoo!?
Bid and sell for free at http://auctions.yahoo.com

--------------End of forwarded message-------------------------

From bouncefilter Wed Nov 17 14:36:33 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id OAA79167
for <hackers@postgreSQL.org>; Wed, 17 Nov 1999 14:36:26 -0500 (EST)
(envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for hackers@postgreSQL.org
id m11oAkV-0003kGC; Wed, 17 Nov 99 20:28 MET
Message-Id: <m11oAkV-0003kGC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] FW: Query
To: vev@michvhf.com (Vince Vielhaber)
Date: Wed, 17 Nov 1999 20:28:11 +0100 (MET)
Cc: hackers@postgreSQL.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <XFMail.991117140721.vev@michvhf.com> from "Vince Vielhaber" at
Nov 17, 99 02:07:21 pm
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

This was sent to webmaster@postgresql.org. Anyone have an answer
to it? I'm not familiar with oracle so I don't know what he's
talking about.

Vince.

I am PostGRESql user and have a query :

DOES POSTGRESQL SUPPORT DATABASE LINK AS IN ORACLE 8.0

AFAIK it means to show up a virtual relation, that is a
relation in another database. So if you query your local
relation, your backend connects to the other database and
virtually shares this relation.

No, PostgreSQL does not support this.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Wed Nov 17 14:46:37 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id OAA80569
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 14:45:45 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgreSQL.org
id m11oAtY-0003kGC; Wed, 17 Nov 99 20:37 MET
Message-Id: <m11oAtY-0003kGC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] regression tests
To: pgman@candle.pha.pa.us (Bruce Momjian)
Date: Wed, 17 Nov 1999 20:37:32 +0100 (MET)
Cc: wieck@debis.com, pgsql-hackers@postgreSQL.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <199911171552.KAA10335@candle.pha.pa.us> from "Bruce Momjian" at
Nov 17, 99 10:52:40 am
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Bruce Momjian wrote:

The question is, who did something that could cause this
error?

I am sure it was me changing the temp behavior. I will look at it.

Running the queries in question in gdb shows:

backend> CREATE TABLE temptest(col int);
backend> CREATE INDEX i_temptest ON temptest(col);
backend> CREATE TEMP TABLE temptest(col int);
NOTICE: trying to delete a reldesc that does not exist.
NOTICE: trying to delete a reldesc that does not exist.
backend> CREATE INDEX i_temptest ON temptest(col);
NOTICE: trying to delete a reldesc that does not exist.
NOTICE: trying to delete a reldesc that does not exist.
backend> DROP INDEX i_temptest;
backend> DROP TABLE temptest;

Program received signal SIGSEGV, Segmentation fault.
0x806b47d in heap_openr (relationName=0x81c4e90 "temptest", lockmode=7)
at heapam.c:569
569 if (RelationIsValid(r) && r->rd_rel->relkind == RELKIND_INDEX)

(gdb) print *r
$2 = {rd_fd = 65536, rd_nblocks = 184, rd_refcnt = 38017,
rd_myxactonly = 16 '\020', rd_isnailed = 8 '\b', rd_isnoname = 0 '\000',
rd_unlinked = 0 '\000', rd_am = 0xb8, rd_rel = 0x2, rd_id = 2,
rd_lockInfo = {lockRelId = {relId = 403, dbId = 131072}}, rd_att = 0xb8,
rd_rules = 0x8109480, rd_istrat = 0x0, rd_support = 0xb8, trigdesc = 0x2}

The problem at this point is that r->rd_rel is 0x2, causing
the SIGSEGV. But I assume the real problem occured earlier
where the notice's came from. The relation descriptor must
have gotten messed up somehow during the CREATE TEMP TABLE.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Wed Nov 17 15:17:34 1999
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id PAA84117
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 15:16:32 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
PAA26856;
Wed, 17 Nov 1999 15:15:54 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911172015.PAA26856@candle.pha.pa.us>
Subject: Re: [HACKERS] regression tests
In-Reply-To: <m11oAtY-0003kGC@orion.SAPserv.Hamburg.dsh.de> from Jan Wieck at
"Nov 17, 1999 08:37:32 pm"
To: Jan Wieck <wieck@debis.com>
Date: Wed, 17 Nov 1999 15:15:54 -0500 (EST)
CC: pgsql-hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Bruce Momjian wrote:

The question is, who did something that could cause this
error?

I am sure it was me changing the temp behavior. I will look at it.

Running the queries in question in gdb shows:

backend> CREATE TABLE temptest(col int);
backend> CREATE INDEX i_temptest ON temptest(col);
backend> CREATE TEMP TABLE temptest(col int);
NOTICE: trying to delete a reldesc that does not exist.
NOTICE: trying to delete a reldesc that does not exist.
backend> CREATE INDEX i_temptest ON temptest(col);
NOTICE: trying to delete a reldesc that does not exist.
NOTICE: trying to delete a reldesc that does not exist.
backend> DROP INDEX i_temptest;
backend> DROP TABLE temptest;

Let me try some tests now. I have an idea.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Wed Nov 17 15:23:34 1999
Received: from www.wgcr.org (IDENT:root@www.wgcr.org [206.74.232.194])
by hub.org (8.9.3/8.9.3) with ESMTP id PAA84890
for <hackers@postgresql.org>; Wed, 17 Nov 1999 15:23:18 -0500 (EST)
(envelope-from lamar.owen@wgcr.org)
Received: from wgcr.org ([206.74.232.197])
by www.wgcr.org (8.8.7/8.8.5) with ESMTP id PAA22148;
Wed, 17 Nov 1999 15:23:03 -0500
Message-ID: <38330E9E.517B3CA1@wgcr.org>
Date: Wed, 17 Nov 1999 15:22:54 -0500
From: Lamar Owen <lamar.owen@wgcr.org>
Organization: WGCR Internet Radio
X-Mailer: Mozilla 4.61 [en] (Win95; I)
X-Accept-Language: en
MIME-Version: 1.0
To: "Aaron J. Seigo" <aaron@gtv.ca>
CC: Postgres Hackers List <hackers@postgresql.org>
Subject: Re: [HACKERS] Re: Postgresql Docs....
References: <Pine.GSO.4.02A.9911171205190.29920-100000@Lodjur.DoCS.UU.SE>
<3832C4A9.BEDCD413@wgcr.org> <99111709235700.23706@stilborne>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

"Aaron J. Seigo" wrote:

How about simply "BSD licensed?"

traditional "BSD Liscences" have that silly advertising
clause.. which postgres does as well, unfortunately... quite honestly, i find
that irritating and antiquated. *shrug* not like the regents are exactly doing
anything important w/postgres now, right? and for all the shouting of "its
TRULY free", there are string attatched...

There are always strings attached. The BSD license has the fewest
strings short of fully public domain.

Domain/GPL/blahblahblah) we should be looking more importantly at which rights
we want to secure and which we don't really care about.

That has already been done -- PostgreSQL still has Berkeley code in it,
and therefore HAS TO BE BSD licensed -- if the license terms are to be
changed (which is not likely to happen), Berkeley code will have to be
eradicated -- which is also not likely to happen.

[snip]

The point is this: the license is not changing (unless ALL contributors
past and present agree to it). I just stated the fact of what license
it is.

There is really no use in discussing what license to put PostgreSQL
under, as it is already under one. That means that there is absolutely
no obligation on anyone who uses the software to give back to the
community -- in fact, if they want to take PostgreSQL, rename it, and
sell it, they are free to do so -- and they don't have to give anything
back. In fact, the original Postgres had this very thing happen -- the
commercial database Illustra was the result, and that got swallowed by
Informix. PostgreSQL lives -- Illustra is dead. Long live PostgreSQL!

--
Lamar Owen
WGCR Internet Radio

From bouncefilter Wed Nov 17 15:57:34 1999
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id PAA90232
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 15:56:43 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
PAA01829;
Wed, 17 Nov 1999 15:56:23 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911172056.PAA01829@candle.pha.pa.us>
Subject: Re: [HACKERS] regression tests
In-Reply-To: <m11oAtY-0003kGC@orion.SAPserv.Hamburg.dsh.de> from Jan Wieck at
"Nov 17, 1999 08:37:32 pm"
To: Jan Wieck <wieck@debis.com>
Date: Wed, 17 Nov 1999 15:56:23 -0500 (EST)
CC: pgsql-hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Bruce Momjian wrote:

The question is, who did something that could cause this
error?

I am sure it was me changing the temp behavior. I will look at it.

Running the queries in question in gdb shows:

backend> CREATE TABLE temptest(col int);
backend> CREATE INDEX i_temptest ON temptest(col);
backend> CREATE TEMP TABLE temptest(col int);
NOTICE: trying to delete a reldesc that does not exist.
NOTICE: trying to delete a reldesc that does not exist.
backend> CREATE INDEX i_temptest ON temptest(col);
NOTICE: trying to delete a reldesc that does not exist.
NOTICE: trying to delete a reldesc that does not exist.
backend> DROP INDEX i_temptest;
backend> DROP TABLE temptest;

Sorry. I see it now. Let me fix it.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Wed Nov 17 17:47:35 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id RAA11951
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 17:46:38 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgreSQL.org
id m11oDim-0003kGC; Wed, 17 Nov 99 23:38 MET
Message-Id: <m11oDim-0003kGC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: LZ compressing data type
To: pgsql-hackers@postgreSQL.org (PostgreSQL HACKERS)
Date: Wed, 17 Nov 1999 23:38:36 +0100 (MET)
Reply-To: wieck@debis.com (Jan Wieck)
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Hi,

I just committed some changes that require an initdb.

New are the discussed, simple LZ compressor, placed into
/utils/adt/pg_compress.c, and a new lztext data type based on
it. You'll find a fairly detailed description of the
compression algorithm in the comments at the top of
pg_lzcompress.c.

Not very surprisingly to me it turns out, that the compressor
does a very good job on rule action strings. I used the 48
rules that can be found in pg_rewrite after the regression
test. The original string sizes range from 820 to 4615 and
the compression rates from 35-76% with an average of 60%. The
4615 size rule action has been coded into a 1126
octet_length.

For the lztext type, there are conversion functions to/from
text and the length() and octet_length() functions available.
Length() returns the same as length on text would. While
octet_length returns the compressed size without VARHDRSZ.

The type does not support MULTIBYTE or CYR_ENCODE up to now.
It shouldn't be too hard to add it and after that, we might
add another lzbpchar type too. The latter is really
interesting, because an empty char(200) (thus containing 200
spaces) could result in an octet_length of 12 instead of 204
- that's a compression rate of 94.1%! It actually wouldn't,
because the compressors default is to start only if the input
is at least 256 bytes, but there is a mechanism so a lzbpchar
type could force this behaviour.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Wed Nov 17 18:51:36 1999
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA20216
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 18:51:10 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
SAA17886;
Wed, 17 Nov 1999 18:50:44 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911172350.SAA17886@candle.pha.pa.us>
Subject: Re: [HACKERS] regression tests
In-Reply-To: <199911172056.PAA01829@candle.pha.pa.us> from Bruce Momjian at
"Nov 17, 1999 03:56:23 pm"
To: Bruce Momjian <pgman@candle.pha.pa.us>
Date: Wed, 17 Nov 1999 18:50:44 -0500 (EST)
CC: Jan Wieck <wieck@debis.com>, pgsql-hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Bruce Momjian wrote:

The question is, who did something that could cause this
error?

I am sure it was me changing the temp behavior. I will look at it.

Running the queries in question in gdb shows:

backend> CREATE TABLE temptest(col int);
backend> CREATE INDEX i_temptest ON temptest(col);
backend> CREATE TEMP TABLE temptest(col int);
NOTICE: trying to delete a reldesc that does not exist.
NOTICE: trying to delete a reldesc that does not exist.
backend> CREATE INDEX i_temptest ON temptest(col);
NOTICE: trying to delete a reldesc that does not exist.
NOTICE: trying to delete a reldesc that does not exist.
backend> DROP INDEX i_temptest;
backend> DROP TABLE temptest;

Sorry. I see it now. Let me fix it.

OK, fixed. Seems there was some confusion in the cache over whether
tables where indexed by logical or phsical names in the hash table to
clear out the cache. Fixed now. Sorry.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Wed Nov 17 18:53:36 1999
Received: from argh.demon.co.uk (IDENT:grim@argh.demon.co.uk [193.237.6.55])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA20391
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 18:53:26 -0500 (EST)
(envelope-from grim@argh.demon.co.uk)
Received: (from grim@localhost) by argh.demon.co.uk (8.9.3/8.9.3) id XAA00481;
Wed, 17 Nov 1999 23:54:07 GMT
From: Michael Simms <grim@argh.demon.co.uk>
Message-Id: <199911172354.XAA00481@argh.demon.co.uk>
Subject: Re: [HACKERS] LZ compressing data type
To: wieck@debis.com
Date: Wed, 17 Nov 1999 23:54:07 +0000 (GMT)
Cc: pgsql-hackers@postgreSQL.org (PostgreSQL HACKERS)
In-Reply-To: <m11oDim-0003kGC@orion.SAPserv.Hamburg.dsh.de> from "Jan Wieck"
at Nov 17, 1999 11:38:36 PM
X-Mailer: ELM [version 2.5 PL0pre8]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hi,

I just committed some changes that require an initdb.

New are the discussed, simple LZ compressor, placed into
/utils/adt/pg_compress.c, and a new lztext data type based on
it. You'll find a fairly detailed description of the
compression algorithm in the comments at the top of
pg_lzcompress.c.

One question.

You say this is an LZ algorythm. Is this the same as LZW, as in the
same kind that gifs use. If so, are we sure that this algorythm is not
covered by the Unisys patent? Their patent is on the algorythm for
compression not on gifs themselves.

If it is, you are liable for a $5,000 fee to use it throughout your
site, or a per-licence fee if you are distributing (thay are worked out
on a per case basis but typical licences are $5 per unit sold from what
I am told).

I came across this problem with a gif manipulation program that *I
WROTE FROM SCRATCH* and had to switch to using the libungif
compression routines.

Just thought Id mention this, in case it has been overlooked.

~Michael

From bouncefilter Wed Nov 17 19:29:36 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id TAA24536
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 19:28:50 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgreSQL.org
id m11oFJd-0003kGC; Thu, 18 Nov 99 01:20 MET
Message-Id: <m11oFJd-0003kGC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] LZ compressing data type
To: grim@argh.demon.co.uk (Michael Simms)
Date: Thu, 18 Nov 1999 01:20:45 +0100 (MET)
Cc: wieck@debis.com, pgsql-hackers@postgreSQL.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <199911172354.XAA00481@argh.demon.co.uk> from "Michael Simms" at
Nov 17, 99 11:54:07 pm
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Hi,

I just committed some changes that require an initdb.

New are the discussed, simple LZ compressor, placed into
/utils/adt/pg_compress.c, and a new lztext data type based on
it. You'll find a fairly detailed description of the
compression algorithm in the comments at the top of
pg_lzcompress.c.

One question.

You say this is an LZ algorythm. Is this the same as LZW, as in the
same kind that gifs use. If so, are we sure that this algorythm is not
covered by the Unisys patent? Their patent is on the algorythm for
compression not on gifs themselves.

If it is, you are liable for a $5,000 fee to use it throughout your
site, or a per-licence fee if you are distributing (thay are worked out
on a per case basis but typical licences are $5 per unit sold from what
I am told).

It't an SLZ algorithm, not LZW. There are FLZ and LZ77 out
too (and I don't know how many other subtypes). At least, LZ
is a family of compression algorithms where LZW is just a
member of them.

I've written the entire code from scatch, inspired by an
article from Adisak Pochanayon dated back in 1993. If they
have a license on this algorithm, they have one on code that
can be coded from scatch in 20 hours after reading
information that's claimed to be free on the internet,
congrats.

This is M$ practice, I never thought that there's more than
one company out doing this kind of business.

But thanks for the info.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Wed Nov 17 19:53:20 1999
Received: from hotzsun.jpl.nasa.gov (root@hotzsun.jpl.nasa.gov
[137.78.84.131])
by hub.org (8.9.3/8.9.3) with ESMTP id TAA27512
for <hackers@postgreSQL.org>; Wed, 17 Nov 1999 19:52:36 -0500 (EST)
(envelope-from hotz@jpl.nasa.gov)
Received: from [137.78.84.130] (hotzmac [137.78.84.130])
by hotzsun.jpl.nasa.gov (8.9.3/8.9.3) with ESMTP id QAA06093;
Wed, 17 Nov 1999 16:52:13 -0800 (PST)
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
X-Sender: hotzmail@hotzsun.jpl.nasa.gov
Message-Id: <v04020a0bb458fd4780ee@[137.78.84.130]>
In-Reply-To: <38330E9E.517B3CA1@wgcr.org>
References: <Pine.GSO.4.02A.9911171205190.29920-100000@Lodjur.DoCS.UU.SE>
<3832C4A9.BEDCD413@wgcr.org> <99111709235700.23706@stilborne>
Date: Wed, 17 Nov 1999 16:51:25 -0800
To: Lamar Owen <lamar.owen@wgcr.org>, "Aaron J. Seigo" <aaron@gtv.ca>
From: "Henry B. Hotz" <hotz@jpl.nasa.gov>
Subject: Re: [HACKERS] Re: Postgresql Docs....
Cc: Postgres Hackers List <hackers@postgreSQL.org>

At 12:22 PM -0800 11/17/99, Lamar Owen wrote:

There is really no use in discussing what license to put PostgreSQL
under, as it is already under one. That means that there is absolutely
no obligation on anyone who uses the software to give back to the
community -- in fact, if they want to take PostgreSQL, rename it, and
sell it, they are free to do so -- and they don't have to give anything
back. In fact, the original Postgres had this very thing happen -- the

Well there is *one* thing they have to give back: credit. They have to
reproduce the copyright notice.

Signature failed Preliminary Design Review.
Feasibility of a new signature is currently being evaluated.
h.b.hotz@jpl.nasa.gov, or hbhotz@oxy.edu

From bouncefilter Wed Nov 17 21:32:38 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id VAA37655
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 21:31:45 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgreSQL.org
id m11oHEb-0003kGC; Thu, 18 Nov 99 03:23 MET
Message-Id: <m11oHEb-0003kGC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] LZ compressing data type
To: wieck@debis.com
Date: Thu, 18 Nov 1999 03:23:41 +0100 (MET)
Cc: grim@argh.demon.co.uk, pgsql-hackers@postgreSQL.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <m11oFJd-0003kGC@orion.SAPserv.Hamburg.dsh.de> from "Jan Wieck"
at Nov 18, 99 01:20:45 am
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

If it is, you are liable for a $5,000 fee to use it throughout your
site, or a per-licence fee if you are distributing (thay are worked out
on a per case basis but typical licences are $5 per unit sold from what
I am told).

It't an SLZ algorithm, not LZW. There are FLZ and LZ77 out
too (and I don't know how many other subtypes). At least, LZ
is a family of compression algorithms where LZW is just a
member of them.

[...]

But thanks for the info.

From the very beginning of US patent No. 4,558,302 (the thing
behind Unisys's LZW):

The compressor searches the input stream to determine the
longest match to a stored string. Each stored string
comprises a prefix string and an extension character
where the extension character is the last character in
the string and the prefix string comprises all but the
extension character. Each string has a code signal
associated therewith and a string is stored in the string
table by, at least implicitly, storing the code signal
for the string, the code signal for the string prefix and
the extension character.

The format my code stores is different to this definition.
It only stores offset/length pairs or literal bytes, signaled
by a bit in a control unit. So there is no prefix string
and/or extension character at all.

If someone might state that storing TAG's and LITERAL chars
is still equivalent to PREFIX and EXTension character, I say
that my code possibly output's sequences of immediately
following TAG's, that aren't neccessarily PREFIX'es. One TAG
could code a sequence of characters, not previously occured
in any other TAG, without any intermediate LITERAL character
occured.

And my code forgets AUTOMATICALLY about too far behind
occurences of matching strings for speed AND compression rate
improvement.

Thus I think my implementation is not related to this patent.

When reading patent #4,558,302 I really remembered the
question, if anyone could ever have a patent on rectangle
front lights for cars. The definition is so general, that I
cannot think that gzip, bzip or any other existing
compression tool doesn't use it's METHOD. If such a GENERAL
patent is possible at all, car manufacturers all over the
world would have to pay millions of dollars license fee's
just to sell cars with rectangular lights.

I don't see any relationship between my code and what Unisys
has a patent for. If they see, I might be blind - or they see
a horizon from inside a black hole. If they need to, I'll
enlighten them.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Wed Nov 17 22:26:38 1999
Received: from acheron.rime.com.au (root@[139.130.54.222])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA42953
for <pgsql-hackers@postgreSQL.org>;
Wed, 17 Nov 1999 22:26:28 -0500 (EST)
(envelope-from pjw@rhyme.com.au)
Received: from oberon (Oberon.rime.com.au [203.8.195.100])
by acheron.rime.com.au (8.9.3/8.9.3) with SMTP id OAA29765;
Thu, 18 Nov 1999 14:25:14 +1100
Message-Id: <3.0.5.32.19991118142605.00c63a70@mail.rhyme.com.au>
X-Sender: pjw@mail.rhyme.com.au
X-Mailer: QUALCOMM Windows Eudora Pro Version 3.0.5 (32)
Date: Thu, 18 Nov 1999 14:26:05 +1100
To: wieck@debis.com (Jan Wieck), wieck@debis.com
From: Philip Warner <pjw@rhyme.com.au>
Subject: Re: [HACKERS] LZ compressing data type
Cc: grim@argh.demon.co.uk, pgsql-hackers@postgreSQL.org
In-Reply-To: <m11oHEb-0003kGC@orion.SAPserv.Hamburg.dsh.de>
References: <m11oFJd-0003kGC@orion.SAPserv.Hamburg.dsh.de>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 03:23 18/11/99 +0100, Jan Wieck wrote:

From the very beginning of US patent No. 4,558,302 (the thing
behind Unisys's LZW):

The other thing to remember is that there are a very large number of
countries in which this patent does not apply because it was not even
applied for until after the alogorithm was made public. In the first
publication of the LZW algorithm, there was no patent notice, but I think
the US patent had been applied for.

For US-based distribution sites, however, you may find the threat of legal
action from a large company is enough to make it less desirable to
distribute.

FWIW, I think Unisys patented the LZW algorithm only.

It's probably a bit late to ask, but how difficult would it be to
generalize your code to use the compressor/encryption library of choice?
zlib and pgp spring to mind. This would kill three birds with one stone.

----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.C.N. 008 659 498) | /(@) ______---_
Tel: +61-03-5367 7422 | _________ \
Fax: +61-03-5367 7430 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/

From bouncefilter Wed Nov 17 23:30:39 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA50371
for <pgsql-hackers@postgresql.org>;
Wed, 17 Nov 1999 23:29:47 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id XAA27482;
Wed, 17 Nov 1999 23:28:23 -0500 (EST)
To: Philip Warner <pjw@rhyme.com.au>
cc: wieck@debis.com (Jan Wieck), grim@argh.demon.co.uk,
pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] LZ compressing data type
In-reply-to: Your message of Thu, 18 Nov 1999 14:26:05 +1100
<3.0.5.32.19991118142605.00c63a70@mail.rhyme.com.au>
Date: Wed, 17 Nov 1999 23:28:23 -0500
Message-ID: <27480.942899303@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Philip Warner <pjw@rhyme.com.au> writes:

FWIW, I think Unisys patented the LZW algorithm only.

More specifically, Unisys patented Terry Welch's variant of the LZ78
class of algorithms. Jan's code falls in the substantially different
LZ77 class. There could be problematic patents out there, but Unisys'
is certainly not one.

(If you don't know the difference between LZ77 and LZ78, see the
comp.compression FAQ ... but don't raise alarms about compression
patents if you don't know even that much about the field, eh?)

regards, tom lane

From bouncefilter Wed Nov 17 23:50:40 1999
Received: from acheron.rime.com.au (root@albatr.lnk.telstra.net
[139.130.54.222]) by hub.org (8.9.3/8.9.3) with ESMTP id XAA53571
for <pgsql-hackers@postgresql.org>;
Wed, 17 Nov 1999 23:50:17 -0500 (EST)
(envelope-from pjw@rhyme.com.au)
Received: from oberon (Oberon.rime.com.au [203.8.195.100])
by acheron.rime.com.au (8.9.3/8.9.3) with SMTP id PAA30364;
Thu, 18 Nov 1999 15:49:19 +1100
Message-Id: <3.0.5.32.19991118155010.00cdfc30@mail.rhyme.com.au>
X-Sender: pjw@mail.rhyme.com.au
X-Mailer: QUALCOMM Windows Eudora Pro Version 3.0.5 (32)
Date: Thu, 18 Nov 1999 15:50:10 +1100
To: Tom Lane <tgl@sss.pgh.pa.us>
From: Philip Warner <pjw@rhyme.com.au>
Subject: Re: [HACKERS] LZ compressing data type
Cc: wieck@debis.com (Jan Wieck), grim@argh.demon.co.uk,
pgsql-hackers@postgresql.org
In-Reply-To: <27480.942899303@sss.pgh.pa.us>
References: <Your message of Thu, 18 Nov 1999 14:26:05 +1100
<3.0.5.32.19991118142605.00c63a70@mail.rhyme.com.au>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 23:28 17/11/99 -0500, Tom Lane wrote:

Philip Warner <pjw@rhyme.com.au> writes:

FWIW, I think Unisys patented the LZW algorithm only.

More specifically, Unisys patented Terry Welch's variant of the LZ78
class of algorithms.

Hence the 'W' in LZW, no?

Jan's code falls in the substantially different
LZ77 class. There could be problematic patents out there, but Unisys'
is certainly not one.

Is this a legal opinion, or a personal one?

(If you don't know the difference between LZ77 and LZ78, see the
comp.compression FAQ ... but don't raise alarms about compression
patents if you don't know even that much about the field, eh?)

1. I did not raise the alarm, I just responded to it.

2. Since Unisys have moved to block code that does not even use LZW
compression, but happens to be compatible with most GIF decompressors, I
think your comment is misinformed, eh? (See the GD graphics library)

The fear is not whether one wins court cases, but whether you can afford to
fight them.

----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.C.N. 008 659 498) | /(@) ______---_
Tel: +61-03-5367 7422 | _________ \
Fax: +61-03-5367 7430 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/

From bouncefilter Thu Nov 18 00:43:40 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA63912
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 00:43:25 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id AAA27644;
Thu, 18 Nov 1999 00:40:45 -0500 (EST)
To: Philip Warner <pjw@rhyme.com.au>
cc: wieck@debis.com (Jan Wieck), grim@argh.demon.co.uk,
pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] LZ compressing data type
In-reply-to: Your message of Thu, 18 Nov 1999 15:50:10 +1100
<3.0.5.32.19991118155010.00cdfc30@mail.rhyme.com.au>
Date: Thu, 18 Nov 1999 00:40:45 -0500
Message-ID: <27642.942903645@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Philip Warner <pjw@rhyme.com.au> writes:

At 23:28 17/11/99 -0500, Tom Lane wrote:

Jan's code falls in the substantially different
LZ77 class. There could be problematic patents out there, but Unisys'
is certainly not one.

Is this a legal opinion, or a personal one?

I'm not a lawyer, but I believe I qualify as an expert witness when
it comes to compression questions. It is not a matter of opinion
whether Jan's code is LZ77 or LZ78, nor is it a matter of opinion
which class Unisys has claims on a small piece of.

The fear is not whether one wins court cases, but whether you can afford to
fight them.

There are patents related to databases. Shall we therefore shut down
Postgres development and run screaming for the hills? If we let
ourselves be intimidated by irrelevant patents, the Microsofts and
Unisyses will win without a fight.

regards, tom lane

From bouncefilter Thu Nov 18 05:10:09 1999
Received: from s-nath-exch1.nath-gpbry.co.za (mail.newaftech.com
[209.212.104.161]) by hub.org (8.9.3/8.9.3) with ESMTP id FAA00627
for <hackers@postgreSQL.org>; Thu, 18 Nov 1999 05:09:51 -0500 (EST)
(envelope-from Michael.Ansley@intec.co.za)
Received: by S-NATH-EXCH1 with Internet Mail Service (5.5.2448.0)
id <W08AKVXK>; Thu, 18 Nov 1999 12:08:17 +0200
Message-ID: <1BF7C7482189D211B03F00805F8527F748C26B@S-NATH-EXCH2>
From: "Ansley, Michael" <Michael.Ansley@intec.co.za>
To: "'Vince Vielhaber'" <vev@michvhf.com>, hackers@postgreSQL.org
Subject: RE: [HACKERS] FW: Query
Date: Thu, 18 Nov 1999 11:56:25 +0200
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2448.0)
Content-Type: text/plain;
charset="iso-8859-1"

No, we don't support anything like this. This is the ability to create a
link in one database that points to a table in another database, so that it
can be accessed as if local. The other database can be a separate instance
(a concept which PG doesn't really have), or even on another machine.

MikeA

-----Original Message-----
From: Vince Vielhaber [mailto:vev@michvhf.com]
Sent: Wednesday, November 17, 1999 9:07 PM
To: hackers@postgreSQL.org
Subject: [HACKERS] FW: Query

This was sent to webmaster@postgresql.org. Anyone have an answer
to it? I'm not familiar with oracle so I don't know what he's
talking about.

Vince.

-----FW:
<19991117174920.10322.rocketmail@web2103.mail.yahoo.com>-----
From: =?iso-8859-1?q?nitin=20thakkar?= <nitinthakkar@yahoo.com>
To: webmaster@postgresql.org
Subject: Query

Hi,

I am PostGRESql user and have a query :

DOES POSTGRESQL SUPPORT DATABASE LINK AS IN ORACLE 8.0

Regards
Nitin

=====

__________________________________________________
Do You Yahoo!?
Bid and sell for free at http://auctions.yahoo.com

--------------End of forwarded message-------------------------

************

From bouncefilter Thu Nov 18 07:22:10 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id HAA15233
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 07:22:04 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgreSQL.org
id m11oQRW-0003kGC; Thu, 18 Nov 99 13:13 MET
Message-Id: <m11oQRW-0003kGC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] LZ compressing data type
To: tgl@sss.pgh.pa.us (Tom Lane)
Date: Thu, 18 Nov 1999 13:13:38 +0100 (MET)
Cc: pjw@rhyme.com.au, wieck@debis.com, grim@argh.demon.co.uk,
pgsql-hackers@postgreSQL.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <27642.942903645@sss.pgh.pa.us> from "Tom Lane" at Nov 18,
99 00:40:45 am
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Tom Lane wrote:

There are patents related to databases. Shall we therefore shut down
Postgres development and run screaming for the hills? If we let
ourselves be intimidated by irrelevant patents, the Microsofts and
Unisyses will win without a fight.

It's definitely a LZ77 coder, and this technique is used in
many other compressors too (lha, arj, zlib, gzip, ...). So I
don't think we have to fear anything.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Thu Nov 18 09:06:11 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id JAA74452
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 09:05:34 -0500 (EST) (envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for pgsql-hackers@postgreSQL.org
id m11oS48-0003kGC; Thu, 18 Nov 99 14:57 MET
Message-Id: <m11oS48-0003kGC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: rules use lztext - initdb required
To: pgsql-hackers@postgreSQL.org (PostgreSQL HACKERS)
Date: Thu, 18 Nov 1999 14:57:36 +0100 (MET)
Reply-To: wieck@debis.com (Jan Wieck)
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Hi,

I just committed a little change to pg_rewrite.h and related
sources. Needs a new clean compile and initdb. Attributes
ev_qual and ev_action are now of type lztext (formerly text).

This one is impressive:

create table t1 (
typname name ,
typowner int4 ,
typlen int2 ,
typprtlen int2 ,
typbyval bool ,
typtype char ,
.
. (totally 54 attributes of various types)
.
atthasdef bool
);
CREATE

create view v1 as select * from t1;
CREATE 148540 1

select rulename, length(ev_action), octet_length(ev_action),
(100 - octet_length(ev_action) * 100 / length(ev_action))::text || '%'
as ratio
from pg_rewrite;
rulename |length|octet_length|ratio
--------------+------+------------+-----
_RETpg_user | 2683| 794|71%
_RETpg_rules | 2562| 934|64%
_RETpg_views | 3740| 1043|73%
_RETpg_tables | 4615| 1126|76%
_RETpg_indexes| 2639| 854|68%
_RETv1 | 14121| 1910|87%
(6 rows)

That means, the rule action string of the view v1 has an
original length of 14121 bytes and is stored in pg_rewrite in
1910 bytes only.

This should give us some room for complicated views/rules.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Thu Nov 18 10:19:12 1999
Received: from px.com.br ([200.253.250.2])
by hub.org (8.9.3/8.9.3) with SMTP id KAA83325
for <pgsql-hackers@postgresql.org>;
Thu, 18 Nov 1999 10:19:04 -0500 (EST)
(envelope-from rcoelho@px.com.br)
Received: from [200.253.250.3] by px.com.br (AIX 4.1/UCB 5.64/4.03)
id AA15296; Thu, 18 Nov 1999 11:15:42 -0400
Message-Id: <006d01bf31d0$330b4b40$03fafdc8@px.com.br>
From: "Ricardo Coelho" <rcoelho@px.com.br>
To: <pgsql-hackers@postgresql.org>
Subject: Change your own password
Date: Thu, 18 Nov 1999 12:21:24 -0200
Organization:
Mime-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-Msmail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.00.2314.1300
X-Mimeole: Produced By Microsoft MimeOLE V5.00.2314.1300

Hi,

Does anybody know how a user could change your own password ? (without
pg_shadow access, of course).

I would like to transfer this task to each one.

I don't remember. Trusted languages overlaps security restrictions, isn't it
?

If I build a C function chgpwd(username,oldpassword,newpassword), it will
work ?

Thanks,

Ricardo Coelho.

From bouncefilter Thu Nov 18 10:21:12 1999
Received: from biology.nmsu.edu (biology.NMSU.Edu [128.123.5.72])
by hub.org (8.9.3/8.9.3) with ESMTP id KAA83716
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 10:20:25 -0500 (EST)
(envelope-from brook@biology.nmsu.edu)
Received: (from brook@localhost) by biology.nmsu.edu (8.8.8/8.8.8) id
IAA19229;
Thu, 18 Nov 1999 08:20:19 -0700 (MST)
Date: Thu, 18 Nov 1999 08:20:19 -0700 (MST)
Message-Id: <199911181520.IAA19229@biology.nmsu.edu>
X-Authentication-Warning: biology.nmsu.edu: brook set sender to
brook@biology.nmsu.edu using -f
From: Brook Milligan <brook@biology.nmsu.edu>
To: wieck@debis.com
CC: pgsql-hackers@postgreSQL.org
In-reply-to: <m11oS48-0003kGC@orion.SAPserv.Hamburg.dsh.de> (wieck@debis.com)
Subject: Re: [HACKERS] rules use lztext - initdb required
References: <m11oS48-0003kGC@orion.SAPserv.Hamburg.dsh.de>

That means, the rule action string of the view v1 has an
original length of 14121 bytes and is stored in pg_rewrite in
1910 bytes only.

This should give us some room for complicated views/rules.

This is great! One of my major frustrations has been designing a
great set of tables/views and then running out of space for the
rules.

Thanks for putting this together.

Cheers,
Brook

From bouncefilter Thu Nov 18 16:21:16 1999
Received: from mail.enterprise.net (mail.enterprise.net [194.72.192.18])
by hub.org (8.9.3/8.9.3) with ESMTP id QAA36489
for <pgsql-hackers@postgresql.org>;
Thu, 18 Nov 1999 16:20:48 -0500 (EST) (envelope-from olly@lfix.co.uk)
Received: from linda.lfix.co.uk (root@max01-068.enterprise.net
[194.72.195.68])
by mail.enterprise.net (8.8.5/8.8.5) with ESMTP id VAA05407
for <pgsql-hackers@postgresql.org>; Thu, 18 Nov 1999 21:20:43 GMT
Received: from lfix.co.uk (olly@localhost [127.0.0.1])
by linda.lfix.co.uk (8.9.3/8.9.3/Debian 8.9.3-6) with ESMTP id VAA22284
for <pgsql-hackers@postgresql.org>; Thu, 18 Nov 1999 21:19:33 GMT
Message-Id: <199911182119.VAA22284@linda.lfix.co.uk>
X-Authentication-Warning: linda.lfix.co.uk: Host olly@localhost [127.0.0.1]
claimed to be lfix.co.uk
X-Mailer: exmh version 2.1.1 10/15/1999 (debian)
To: pgsql-hackers@postgresql.org
Subject: Createdb problem report
Mime-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0
Date: Thu, 18 Nov 1999 21:19:33 +0000
From: "Oliver Elphick" <olly@lfix.co.uk>
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by hub.org id QAA36525

The attached report is still correct as of 6.5.3; but will having a
database name that has to be quoted cause problems anywhere that SQL
queries are constructed to do internal operations?

The SQL command `CREATE DATABASE "www-data"' works correctly.

------- Forwarded Message

Date: Thu, 18 Nov 1999 20:39:51 +0100
From: Eric Gentilini <eric.gentilini@eleve.emn.fr>
To: olly@lfix.co.uk
Subject: [POSTGRESQL] Does createdb belong to postgresql ?

hi !

I fixed a _very_ little bug in createdb and destroydb that prevents the
creation of postgresql users whose name contains special characters, and
especially '-', useful when accessing a database through a CGI script, whose
user is 'www-data' by default.

But I don't know if this prog is debian specific or if it belongs to
postrgesql.

Can you help me ?
thx !

============================================================================
POSTGRESQL BUG REPORT TEMPLATE
============================================================================

Your name : Eric Gentilini
Your email address : eric.gentilini@eleve.emn.fr

System Configuration
- ---------------------
Architecture (example: Intel Pentium) : Intel Pentium&PentiumII

Operating System (example: Linux 2.0.26 ELF) : Linux 2.2.13 ELF

PostgreSQL version (example: PostgreSQL-6.5.2): PostgreSQL-6.5.2

Compiler used (example: gcc 2.8.0) : not compiled by me

Please enter a FULL description of your problem:
- ------------------------------------------------
I don't know if it is really a bug or if it is intentionnal, but
I found out that createdb and destroydb prevented the
creation/destruction of postgresql users whose
name contained special characters, and
especially '-', useful when accessing a database through a CGI script, whose
user is 'www-data' by default.
(In this case, createdb is executed by createuser)
The query aborts with the message "ERROR: parser: parse error at or near "-""

Please describe a way to repeat the problem. Please try to provide a
concise reproducible example, if at all possible:
- ----------------------------------------------------------------------
for instance : createdb www-data

If you know how this problem might be fixed, list the solution below:
- ---------------------------------------------------------------------
The simplest solution I found is to modify the scripts createdb and destroydb.
createdb : replace line 114 with :
psql $PASSWDOPT -tq $AUTHOPT $PGHOSTOPT $PGPORTOPT -c "create
database \"$dbname\" $location $encoding" template1
^^^ ^^^
destroydb : replace line 78 with :
psql -tq $AUTHOPT $PGHOSTOPT $PGPORTOPT -c "drop database \"$dbname\"" template
1
^^^ ^^^

Eric (Yam) Gentilini

Linux ��� Nantes sur http://www.linux-nantes.fr.eu.org

------- End of Forwarded Message

--
Vote against SPAM: http://www.politik-digital.de/spam/
========================================
Oliver Elphick Oliver.Elphick@lfix.co.uk
Isle of Wight http://www.lfix.co.uk/oliver
PGP key from public servers; key ID 32B8FAA1
========================================
"A Song for the sabbath day. It is a good thing to
give thanks unto the LORD, and to sing praises unto
thy name, O most High." Psalms 92:1

From bouncefilter Thu Nov 18 16:48:18 1999
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id QAA40568
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 16:47:03 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
QAA12254;
Thu, 18 Nov 1999 16:46:30 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911182146.QAA12254@candle.pha.pa.us>
Subject: Re: [HACKERS] Createdb problem report
In-Reply-To: <199911182119.VAA22284@linda.lfix.co.uk> from Oliver Elphick at
"Nov 18, 1999 09:19:33 pm"
To: Oliver Elphick <olly@lfix.co.uk>
Date: Thu, 18 Nov 1999 16:46:30 -0500 (EST)
CC: pgsql-hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Applied. Will appear in 7.0

[Charset iso-8859-1 unsupported, filtering to ASCII...]

The attached report is still correct as of 6.5.3; but will having a
database name that has to be quoted cause problems anywhere that SQL
queries are constructed to do internal operations?

The SQL command `CREATE DATABASE "www-data"' works correctly.

------- Forwarded Message

Date: Thu, 18 Nov 1999 20:39:51 +0100
From: Eric Gentilini <eric.gentilini@eleve.emn.fr>
To: olly@lfix.co.uk
Subject: [POSTGRESQL] Does createdb belong to postgresql ?

hi !

I fixed a _very_ little bug in createdb and destroydb that prevents the
creation of postgresql users whose name contains special characters, and
especially '-', useful when accessing a database through a CGI script, whose
user is 'www-data' by default.

But I don't know if this prog is debian specific or if it belongs to
postrgesql.

Can you help me ?
thx !

============================================================================
POSTGRESQL BUG REPORT TEMPLATE
============================================================================

Your name : Eric Gentilini
Your email address : eric.gentilini@eleve.emn.fr

System Configuration
- ---------------------
Architecture (example: Intel Pentium) : Intel Pentium&PentiumII

Operating System (example: Linux 2.0.26 ELF) : Linux 2.2.13 ELF

PostgreSQL version (example: PostgreSQL-6.5.2): PostgreSQL-6.5.2

Compiler used (example: gcc 2.8.0) : not compiled by me

Please enter a FULL description of your problem:
- ------------------------------------------------
I don't know if it is really a bug or if it is intentionnal, but
I found out that createdb and destroydb prevented the
creation/destruction of postgresql users whose
name contained special characters, and
especially '-', useful when accessing a database through a CGI script, whose
user is 'www-data' by default.
(In this case, createdb is executed by createuser)
The query aborts with the message "ERROR: parser: parse error at or near "-""

Please describe a way to repeat the problem. Please try to provide a
concise reproducible example, if at all possible:
- ----------------------------------------------------------------------
for instance : createdb www-data

If you know how this problem might be fixed, list the solution below:
- ---------------------------------------------------------------------
The simplest solution I found is to modify the scripts createdb and destroydb.
createdb : replace line 114 with :
psql $PASSWDOPT -tq $AUTHOPT $PGHOSTOPT $PGPORTOPT -c "create
database \"$dbname\" $location $encoding" template1
^^^ ^^^
destroydb : replace line 78 with :
psql -tq $AUTHOPT $PGHOSTOPT $PGPORTOPT -c "drop database \"$dbname\"" template
1
^^^ ^^^

Eric (Yam) Gentilini

Linux _ Nantes sur http://www.linux-nantes.fr.eu.org

------- End of Forwarded Message

--
Vote against SPAM: http://www.politik-digital.de/spam/
========================================
Oliver Elphick Oliver.Elphick@lfix.co.uk
Isle of Wight http://www.lfix.co.uk/oliver
PGP key from public servers; key ID 32B8FAA1
========================================
"A Song for the sabbath day. It is a good thing to
give thanks unto the LORD, and to sing praises unto
thy name, O most High." Psalms 92:1

************

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Thu Nov 18 17:26:17 1999
Received: from merganser.its.uu.se (merganser.its.uu.se [130.238.6.236])
by hub.org (8.9.3/8.9.3) with ESMTP id RAA49551
for <hackers@postgresql.org>; Thu, 18 Nov 1999 17:25:43 -0500 (EST)
(envelope-from peter@localhost.its.uu.se)
Received: from regulus.its.uu.se ([130.238.7.19]:61690 "EHLO
regulus.its.uu.se") by merganser.its.uu.se with ESMTP id
<S.sB5nQ174483>; Thu, 18 Nov 1999 23:25:32 +0100
Received: from peter (helo=localhost)
by regulus.its.uu.se with local-esmtp (Exim 3.02 #2)
id 11oa3y-0000Cs-00
for hackers@postgresql.org; Thu, 18 Nov 1999 23:29:58 +0100
Date: Thu, 18 Nov 1999 23:29:58 +0100 (CET)
From: Peter Eisentraut <peter_e@gmx.net>
To: hackers@postgresql.org
Subject: psql & regress tests
Message-ID: <Pine.LNX.4.20.9911182327450.714-100000@localhost.localdomain>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=ISO-8859-1
Content-Transfer-Encoding: 8BIT
Sender: Peter Eisentraut <peter@hub.org>

Okay, here is my semiofficial take on the situation:

* Use the psql from the 6.5.* distro to do regression tests until further
notice.

* Although the output format in the current psql is not under intensive
development anymore, I am not sure if I can guarantee a "freeze" soon.
Once in a while I find some sort of flaw in really strange query results;
the aim of the output is to be visually pleasing, not to provide an exact
match so something. Having said that, I do not expect any major changes to
take place anymore though.

* The other problem is *what* is actually printed, as opposed to the
peculiarities of the table format. This is still somewhat confusing even
to me and I am still fixing things so they make sense at the end.

Example:
***OLD
QUERY: CREATE TABLE BOOLTBL1 (f1 bool);
QUERY: INSERT INTO BOOLTBL1 (f1) VALUES ('t'::bool);
QUERY: INSERT INTO BOOLTBL1 (f1) VALUES ('True'::bool);
QUERY: INSERT INTO BOOLTBL1 (f1) VALUES ('true'::bool);
QUERY: SELECT '' AS t_3, BOOLTBL1.*;
t_3|f1
---+--
|t
|t
|t
(3 rows)

***CURRENT
CREATE TABLE BOOLTBL1 (f1 bool);

INSERT INTO BOOLTBL1 (f1) VALUES ('t'::bool);

INSERT INTO BOOLTBL1 (f1) VALUES ('True'::bool);

INSERT INTO BOOLTBL1 (f1) VALUES ('true'::bool);

-- BOOLTBL1 should be full of true's at this point
SELECT '' AS t_3, BOOLTBL1.*;
t_3 | f1
-----+----
| t
| t
| t
(3 rows)

(In fact, it's so current, it's not even in CVS yet, thanks to some
problems pointed out by Jan.)

Yes, there actually is a reasoning behind all of this, I'm just not sure
right now what it was ;). If someone is interested, I can bore you with
the details.

* Since no one has picked up on my idea to run the tests directly on the
backend, I will keep reiterating this idea until someone shuts me up
:*) The idea would be to have a target "check" in the top level makefile
like this (conceptually):

check: all
mkdir ./regress
initdb -l . -d ./regress
for i in test1 test2 test3 ...; do
postgres -D ./regress -E template1 \
< $(srcdir)/test/regress/sql/$i.sql \

& output-$i.out

done
for i in test1 test2 test3 ...; do
cmp output-$i.out expected-$i.out
if [ $? == 1]; then
echo "Test $i failed."
else
echo "Test $i passed."
rm -f output-$i.out
fi
done
rm -rf ./regress

Then you can do
./configure
make
make check
make install

Or am I missing something here? Of course this change would require some
work, but I'm just getting at the concept here.

Finally, I'd like to apologize for the extra trouble some must have had. I
can only offer to cooperate on anything that needs to be done.

-Peter

--
Peter Eisentraut Sernanders v���g 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Thu Nov 18 18:01:17 1999
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA56758
for <hackers@postgresql.org>; Thu, 18 Nov 1999 18:00:51 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
RAA16165;
Thu, 18 Nov 1999 17:41:36 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911182241.RAA16165@candle.pha.pa.us>
Subject: Re: [HACKERS] psql & regress tests
In-Reply-To: <Pine.LNX.4.20.9911182327450.714-100000@localhost.localdomain>
from Peter Eisentraut at "Nov 18, 1999 11:29:58 pm"
To: Peter Eisentraut <peter_e@gmx.net>
Date: Thu, 18 Nov 1999 17:41:36 -0500 (EST)
CC: hackers@postgresql.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

* Since no one has picked up on my idea to run the tests directly on the
backend, I will keep reiterating this idea until someone shuts me up
:*) The idea would be to have a target "check" in the top level makefile
like this (conceptually):

Running the backend standalone does not use locking with other backends,
so it is dangerous.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Fri Nov 19 20:33:38 1999
Received: from merganser.its.uu.se (merganser.its.uu.se [130.238.6.236])
by hub.org (8.9.3/8.9.3) with ESMTP id UAA05102
for <pgsql-hackers@postgresql.org>;
Fri, 19 Nov 1999 20:33:24 -0500 (EST)
(envelope-from peter@localhost.its.uu.se)
Received: from regulus.its.uu.se ([130.238.7.19]:64148 "EHLO
regulus.its.uu.se") by merganser.its.uu.se with ESMTP id
<S.sBTdB168336>; Sat, 20 Nov 1999 02:33:01 +0100
Received: from peter (helo=localhost)
by regulus.its.uu.se with local-esmtp (Exim 3.02 #2)
id 11oaer-0000D7-00; Fri, 19 Nov 1999 00:08:05 +0100
Date: Fri, 19 Nov 1999 00:08:05 +0100 (CET)
From: Peter Eisentraut <peter_e@gmx.net>
To: Bruce Momjian <pgman@candle.pha.pa.us>
cc: Oliver Elphick <olly@lfix.co.uk>, pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Createdb problem report
In-Reply-To: <199911182146.QAA12254@candle.pha.pa.us>
Message-ID: <Pine.LNX.4.20.9911190004060.714-100000@localhost.localdomain>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=ISO-8859-1
Content-Transfer-Encoding: 8BIT
Sender: Peter Eisentraut <peter@hub.org>

The cleanup on the scripts I recently did contained that, too. I think
it's on hold now because I'm going to fix up the create user SQL statement
to allow picking your user id first. Or was there any other reason?
Anyway, just letting you know that this problem has been recognized.

-Peter

On 1999-11-18, Bruce Momjian mentioned:

Applied. Will appear in 7.0

I fixed a _very_ little bug in createdb and destroydb that prevents the
creation of postgresql users whose name contains special characters, and
especially '-', useful when accessing a database through a CGI script, whose
user is 'www-data' by default.

--
Peter Eisentraut Sernanders v���g 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

From bouncefilter Thu Nov 18 18:14:19 1999
Received: from wallace.ece.rice.edu (root@wallace.ece.rice.edu
[128.42.12.154])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA63079
for <hackers@postgresql.org>; Thu, 18 Nov 1999 18:13:34 -0500 (EST)
(envelope-from reedstrm@wallace.ece.rice.edu)
Received: by wallace.ece.rice.edu via sendmail from stdin
id <m11oak9-000LECC@wallace.ece.rice.edu> (Debian Smail3.2.0.102)
for hackers@postgresql.org; Thu, 18 Nov 1999 17:13:33 -0600 (CST)
Date: Thu, 18 Nov 1999 17:13:33 -0600
From: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
To: hackers@postgresql.org
Subject: Re: [HACKERS] psql & regress tests
Message-ID: <19991118171333.A18543@wallace.ece.rice.edu>
References: <Pine.LNX.4.20.9911182327450.714-100000@localhost.localdomain>
<199911182241.RAA16165@candle.pha.pa.us>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Mailer: Mutt 0.95.3i
In-Reply-To: <199911182241.RAA16165@candle.pha.pa.us>;
from Bruce Momjian on Thu, Nov 18, 1999 at 05:41:36PM -0500

On Thu, Nov 18, 1999 at 05:41:36PM -0500, Bruce Momjian wrote:

* Since no one has picked up on my idea to run the tests directly on the
backend, I will keep reiterating this idea until someone shuts me up
:*) The idea would be to have a target "check" in the top level makefile
like this (conceptually):

Running the backend standalone does not use locking with other backends,
so it is dangerous.

Bruce, how dos this apply to Peter's suggestion? We're talking about
_regression_ tests. Things to do after changing the code. Do you often
recompile, and run regression tests against a db with a (now out of date)
postmaster running against it? Do other developers? I'd have thought a
complete shutdown/restart is part of the cycle anyway. has to be if an
initdb is in there of course. Checking to make sure a postmaster isn't
running could be added to Peter's script, just to be sure.

Ross
--
Ross J. Reedstrom, Ph.D., <reedstrm@rice.edu>
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St., Houston, TX 77005

From bouncefilter Thu Nov 18 18:37:18 1999
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id SAA81894
for <hackers@postgreSQL.org>; Thu, 18 Nov 1999 18:36:34 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
SAA18215;
Thu, 18 Nov 1999 18:35:15 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911182335.SAA18215@candle.pha.pa.us>
Subject: Re: [HACKERS] psql & regress tests
In-Reply-To: <19991118171333.A18543@wallace.ece.rice.edu> from "Ross J.
Reedstrom" at "Nov 18, 1999 05:13:33 pm"
To: "Ross J. Reedstrom" <reedstrm@wallace.ece.rice.edu>
Date: Thu, 18 Nov 1999 18:35:15 -0500 (EST)
CC: hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Thu, Nov 18, 1999 at 05:41:36PM -0500, Bruce Momjian wrote:

* Since no one has picked up on my idea to run the tests directly on the
backend, I will keep reiterating this idea until someone shuts me up
:*) The idea would be to have a target "check" in the top level makefile
like this (conceptually):

Running the backend standalone does not use locking with other backends,
so it is dangerous.

Bruce, how dos this apply to Peter's suggestion? We're talking about
_regression_ tests. Things to do after changing the code. Do you often
recompile, and run regression tests against a db with a (now out of date)
postmaster running against it? Do other developers? I'd have thought a
complete shutdown/restart is part of the cycle anyway. has to be if an
initdb is in there of course. Checking to make sure a postmaster isn't
running could be added to Peter's script, just to be sure.

Regression tests are often run by end-users as part of the install. I
can imagine someone seeing a problem and running the regression tests
while having running backends to see if everything is still working.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Thu Nov 18 18:58:18 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id SAA84595
for <hackers@postgreSQL.org>; Thu, 18 Nov 1999 18:57:47 -0500 (EST)
(envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for hackers@postgreSQL.org
id m11obJH-0003kGC; Fri, 19 Nov 99 00:49 MET
Message-Id: <m11obJH-0003kGC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] psql & regress tests
To: reedstrm@wallace.ece.rice.edu (Ross J. Reedstrom)
Date: Fri, 19 Nov 1999 00:49:51 +0100 (MET)
Cc: hackers@postgreSQL.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <19991118171333.A18543@wallace.ece.rice.edu> from "Ross J.
Reedstrom" at Nov 18, 99 05:13:33 pm
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

On Thu, Nov 18, 1999 at 05:41:36PM -0500, Bruce Momjian wrote:

* Since no one has picked up on my idea to run the tests directly on the
backend, I will keep reiterating this idea until someone shuts me up
:*) The idea would be to have a target "check" in the top level makefile
like this (conceptually):

Running the backend standalone does not use locking with other backends,
so it is dangerous.

Bruce, how dos this apply to Peter's suggestion? We're talking about
_regression_ tests. Things to do after changing the code. Do you often
recompile, and run regression tests against a db with a (now out of date)
postmaster running against it? Do other developers? I'd have thought a
complete shutdown/restart is part of the cycle anyway. has to be if an
initdb is in there of course. Checking to make sure a postmaster isn't
running could be added to Peter's script, just to be sure.

I'm actually doing some tests if the 'make check' would be
possible. I.e. doing a complete install with POSTGRESDIR
below the regress dir, running initdb with the libdir and
datadir pointing into there too, then starting new postmaster
on a different port in background etc., etc., etc..

That would make it possible to do a complete check without
doing a regular install at all.

Will give some results soon.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Thu Nov 18 19:19:18 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id TAA87051
for <hackers@postgreSQL.org>; Thu, 18 Nov 1999 19:18:18 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id TAA00320;
Thu, 18 Nov 1999 19:17:23 -0500 (EST)
To: Bruce Momjian <pgman@candle.pha.pa.us>
cc: Peter Eisentraut <peter_e@gmx.net>, hackers@postgreSQL.org
Subject: Re: [HACKERS] psql & regress tests
In-reply-to: Your message of Thu, 18 Nov 1999 17:41:36 -0500 (EST)
<199911182241.RAA16165@candle.pha.pa.us>
Date: Thu, 18 Nov 1999 19:17:23 -0500
Message-ID: <318.942970643@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Bruce Momjian <pgman@candle.pha.pa.us> writes:

* Since no one has picked up on my idea to run the tests directly on the
backend, I will keep reiterating this idea until someone shuts me up

Running the backend standalone does not use locking with other backends,
so it is dangerous.

It wouldn't be particularly "dangerous" if we assume that no one else is
accessing the regression database. What it *would* be is less useful at
catching problems. Standalone mode wouldn't test the cross-backend
interlocking code at all.

Admittedly, running a bunch of tests serially isn't a strong stress test
of cross-backend behavior, but it's not as content-free a check as you
might think. On my machine, at least, the old backend is still around
doing shutdown for the first half-second or so while the next one is
running.

What I'd really like to see is some deliberate parallelism in some of
the regress tests...

regards, tom lane

From bouncefilter Thu Nov 18 19:31:19 1999
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id TAA89183
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 19:30:46 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id TAA19161
for pgsql-hackers@postgreSQL.org; Thu, 18 Nov 1999 19:22:48 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911190022.TAA19161@candle.pha.pa.us>
Subject: pg version date file
To: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Date: Thu, 18 Nov 1999 19:22:48 -0500 (EST)
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Where is that file that makes an initdb required? We are supposed to
change that file when an initdb is needed.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Thu Nov 18 20:06:19 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id UAA93892
for <hackers@postgreSQL.org>; Thu, 18 Nov 1999 20:05:42 -0500 (EST)
(envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for hackers@postgreSQL.org
id m11ocMk-0003kGC; Fri, 19 Nov 99 01:57 MET
Message-Id: <m11ocMk-0003kGC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] psql & regress tests
To: tgl@sss.pgh.pa.us (Tom Lane)
Date: Fri, 19 Nov 1999 01:57:30 +0100 (MET)
Cc: pgman@candle.pha.pa.us, peter_e@gmx.net, hackers@postgreSQL.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <318.942970643@sss.pgh.pa.us> from "Tom Lane" at Nov 18,
99 07:17:23 pm
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Tom Lane wrote:

Admittedly, running a bunch of tests serially isn't a strong stress test
of cross-backend behavior, but it's not as content-free a check as you
might think. On my machine, at least, the old backend is still around
doing shutdown for the first half-second or so while the next one is
running.

What I'd really like to see is some deliberate parallelism in some of
the regress tests...

It's amusing how often we two have the same wishes or ideas.

The run_check.sh script, I'm actually hacking on, would be a
replacement for the regress.sh, started off from the 'make
check'. And from the first try I added syntax to the
sql/tests file to run either groups of tests parallel
intermixed with single tests sequentially.

The new syntax will look like this:

parallel group1
test boolean
test char
test name
endparallel

test varchar
test text
test strings

parallel group2
test int2
test int4
test int8
endparallel

.
.
.

The above will run boolean, char and name parallel. After all
three terminated, it will check their output and continue to
run varchar, text and strings sequentially, followed by the
next parallel group.

To test real concurrency we might need to split up some or
create new tests, where the same tables are accessed
concurrently. But that wouldn't be hard to do I think.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Thu Nov 18 20:04:19 1999
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id UAA93505
for <hackers@postgreSQL.org>; Thu, 18 Nov 1999 20:03:33 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
TAA19713;
Thu, 18 Nov 1999 19:58:40 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911190058.TAA19713@candle.pha.pa.us>
Subject: Re: [HACKERS] psql & regress tests
In-Reply-To: <318.942970643@sss.pgh.pa.us> from Tom Lane at "Nov 18,
1999 07:17:23 pm"
To: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu, 18 Nov 1999 19:58:40 -0500 (EST)
CC: Peter Eisentraut <peter_e@gmx.net>, hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Bruce Momjian <pgman@candle.pha.pa.us> writes:

* Since no one has picked up on my idea to run the tests directly on the
backend, I will keep reiterating this idea until someone shuts me up

Running the backend standalone does not use locking with other backends,
so it is dangerous.

It wouldn't be particularly "dangerous" if we assume that no one else is
accessing the regression database. What it *would* be is less useful at
catching problems. Standalone mode wouldn't test the cross-backend
interlocking code at all.

Any modifications to shared pg_ tables would be a problem. Also, pg_log
and pg_variable locking is not happening in there either, is it?

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Thu Nov 18 20:13:19 1999
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id UAA94730
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 20:13:07 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id UAA20026
for pgsql-hackers@postgreSQL.org; Thu, 18 Nov 1999 20:12:18 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911190112.UAA20026@candle.pha.pa.us>
Subject: Getting OID in psql of recent insert
To: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Date: Thu, 18 Nov 1999 20:12:18 -0500 (EST)
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

In writing the book, I see the serious limitation that there is no way
in psql to access the most recently inserted oid. Without it, there
seems to be no way to use the oid value as a foreign key in another
table.

Should I add a function to return the most recently assigned oid to the
backend, or is there a better way?

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Thu Nov 18 20:11:19 1999
Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
by hub.org (8.9.3/8.9.3) with ESMTP id UAA94485
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 20:10:27 -0500 (EST) (envelope-from Inoue@tpf.co.jp)
Received: from cadzone ([126.0.1.40] (may be forged))
by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with SMTP
id KAA01196 for <pgsql-hackers@postgreSQL.org>;
Fri, 19 Nov 1999 10:10:25 +0900
From: "Hiroshi Inoue" <Inoue@tpf.co.jp>
To: "pgsql-hackers" <pgsql-hackers@postgreSQL.org>
Subject: Hash Join is very slooow in some cases
Date: Fri, 19 Nov 1999 10:14:56 +0900
Message-ID: <000201bf322b$7df2a620$2801007e@cadzone.tpf.co.jp>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
X-MimeOLE: Produced By Microsoft MimeOLE V4.72.2106.4
Importance: Normal

Hi all,

We are converting Oracle system to PostgreSQL.
But some queries takes very looong time.

I have tried variously and made a typical example
in PostgreSQL 6.5.2 .
It's already known ?

select count(*) from a,b where a.id1=b.id1;

returns immeidaitely and EXPLAIN shows

Aggregate (cost=3380.24 rows=24929 width=8)
-> Hash Join (cost=3380.24 rows=24929 width=8)
-> Seq Scan on a (cost=1551.41 rows=27558 width=4)
-> Hash (cost=604.21 rows=8885 width=4)
-> Seq Scan on b (cost=604.21 rows=8885 width=4)

But

select count(*) from a,b where a.id1=b.id1 and a.id2=b.id2;

takes very looong time.
EXPLAIN shows

Aggregate (cost=3382.24 rows=8195 width=12)
-> Hash Join (cost=3382.24 rows=8195 width=12)
-> Seq Scan on a (cost=1551.41 rows=27558 width=6)
-> Hash (cost=604.21 rows=8885 width=6)
-> Seq Scan on b (cost=604.21 rows=8885 width=6)

Query plans are almost same.
Why is there a such difference ?

I examined an output by EXPLAIN VERBOSE and found that
the 1st query uses id1 as its hashkey and 2nd query uses id2
as its hashkey.

So I tried the following query.

select count(*) from a,b where a.id2=b.id2 and a.id1=b.id1;

This returns immediately as I expected.

id1-s are type int4 and their disbursions are 0.00010181/
0.000203409.
id2-s are type int2 and their disbursions are 0.523526/
0.328712 .
Is id2 suitable for a hashkey ?

I can't try it in current now,sorry.
But seems the following code in createplan.c is unchanged.

/* Now the righthand op of the sole hashclause is the inner hash key. */
innerhashkey = get_rightop(lfirst(hashclauses));

Comments ?

Hiroshi Inoue
Inoue@tpf.co.jp

From bouncefilter Thu Nov 18 20:59:19 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id UAA00364
for <hackers@postgreSQL.org>; Thu, 18 Nov 1999 20:58:34 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id UAA00544;
Thu, 18 Nov 1999 20:57:50 -0500 (EST)
To: Bruce Momjian <pgman@candle.pha.pa.us>
cc: Peter Eisentraut <peter_e@gmx.net>, hackers@postgreSQL.org
Subject: Re: [HACKERS] psql & regress tests
In-reply-to: Your message of Thu, 18 Nov 1999 19:58:40 -0500 (EST)
<199911190058.TAA19713@candle.pha.pa.us>
Date: Thu, 18 Nov 1999 20:57:50 -0500
Message-ID: <542.942976670@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Bruce Momjian <pgman@candle.pha.pa.us> writes:

It wouldn't be particularly "dangerous" if we assume that no one else is
accessing the regression database. What it *would* be is less useful at
catching problems. Standalone mode wouldn't test the cross-backend
interlocking code at all.

Any modifications to shared pg_ tables would be a problem. Also, pg_log
and pg_variable locking is not happening in there either, is it?

Good point --- it wouldn't be just that database, but the whole
installation (data directory) that would have to be unused. You really
wouldn't dare even have a postmaster running, at least not in the same
data directory. But that could be made safe by using a nonstandard
location for the data directory for regression.

However, this is all beside the main point: we want the regress tests
to run in an environment as close as possible to the way Postgres is
normally used. The more we hack up a special regress-test environment,
the less the tests reflect reality.

Aside from the cross-backend synchronization issue, I forgot to point
out a really obvious benefit: doing it the current way allows the regress
tests to help check the backend's frontend communication code, and
libpq, and psql itself. We'd need some other way of testing all that
code if we switched to a standalone-backend regression test set.

What I *would* like to see is more support for running regress tests on
a not-yet-installed build, so people can test a fresh build before they
blow away their working installation. This requires doing an initdb
into a temporary directory, starting a postmaster therein (using a
nonstandard port number), and running the tests there. This is doable
by hand, of course, but it's tedious and error-prone even for an expert;
I think it's out of the question for a novice installer. We ought to
offer a canned script to do it that way.

regards, tom lane

From bouncefilter Thu Nov 18 21:33:20 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id VAA04417
for <hackers@postgreSQL.org>; Thu, 18 Nov 1999 21:32:44 -0500 (EST)
(envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for hackers@postgreSQL.org
id m11odj9-0003kGC; Fri, 19 Nov 99 03:24 MET
Message-Id: <m11odj9-0003kGC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] psql & regress tests
To: tgl@sss.pgh.pa.us (Tom Lane)
Date: Fri, 19 Nov 1999 03:24:43 +0100 (MET)
Cc: pgman@candle.pha.pa.us, peter_e@gmx.net, hackers@postgreSQL.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <542.942976670@sss.pgh.pa.us> from "Tom Lane" at Nov 18,
99 08:57:50 pm
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Any modifications to shared pg_ tables would be a problem. Also, pg_log
and pg_variable locking is not happening in there either, is it?

Good point --- it wouldn't be just that database, but the whole
installation (data directory) that would have to be unused. You really
wouldn't dare even have a postmaster running, at least not in the same
data directory. But that could be made safe by using a nonstandard
location for the data directory for regression.

However, this is all beside the main point: we want the regress tests
to run in an environment as close as possible to the way Postgres is
normally used. The more we hack up a special regress-test environment,
the less the tests reflect reality.

My new script actually does a

make POSTGRESDIR=somewhere_else install
PATH="somewhere_else/bin:$PATH"

Then it initializes a database below there and starts a
postmaster with the resulting data directory, listening on
port 65432.

So I think it's very close to a real live setup, while
another "production" running installation isn't affected at
all.

Aside from the cross-backend synchronization issue, I forgot to point
out a really obvious benefit: doing it the current way allows the regress
tests to help check the backend's frontend communication code, and
libpq, and psql itself. We'd need some other way of testing all that
code if we switched to a standalone-backend regression test set.

What I *would* like to see is more support for running regress tests on
a not-yet-installed build, so people can test a fresh build before they
blow away their working installation. This requires doing an initdb
into a temporary directory, starting a postmaster therein (using a
nonstandard port number), and running the tests there. This is doable
by hand, of course, but it's tedious and error-prone even for an expert;
I think it's out of the question for a novice installer. We ought to
offer a canned script to do it that way.

Right, right, right - I'm on it.

The ugly detail, I'm currently running into, is that there
already seems to be a concurrency problem I discovered with
my testing. Occationally I get this into the postmaster log
for parallel executing tests:

ERROR: Bad boolean external representation 'XXX'
FATAL 1: SearchSysCache: recursive use of cache 10
FATAL 2: elog: error in postmaster or backend startup, giving up!
pq_flush: send() failed: Broken pipe
Server process (pid 9791) exited with status 512 at Fri Nov 19 03:17:09 1999
Terminating any active server processes...

It happens during the first parallel group of 11 tests. Not
allways, so it's timing critical. Outch.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Thu Nov 18 22:04:20 1999
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA08006
for <hackers@postgreSQL.org>; Thu, 18 Nov 1999 22:03:25 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
VAA21649;
Thu, 18 Nov 1999 21:42:17 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911190242.VAA21649@candle.pha.pa.us>
Subject: Re: [HACKERS] psql & regress tests
In-Reply-To: <m11odj9-0003kGC@orion.SAPserv.Hamburg.dsh.de> from Jan Wieck at
"Nov 19, 1999 03:24:43 am"
To: Jan Wieck <wieck@debis.com>
Date: Thu, 18 Nov 1999 21:42:17 -0500 (EST)
CC: Tom Lane <tgl@sss.pgh.pa.us>, peter_e@gmx.net, hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

ERROR: Bad boolean external representation 'XXX'
FATAL 1: SearchSysCache: recursive use of cache 10
FATAL 2: elog: error in postmaster or backend startup, giving up!
pq_flush: send() failed: Broken pipe
Server process (pid 9791) exited with status 512 at Fri Nov 19 03:17:09 1999
Terminating any active server processes...

It happens during the first parallel group of 11 tests. Not
allways, so it's timing critical. Outch.

Now that we know numeric is working, can we make the test run faster in
the default mode?

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Thu Nov 18 22:26:20 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA10672
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 22:25:42 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id WAA00716;
Thu, 18 Nov 1999 22:24:52 -0500 (EST)
To: Bruce Momjian <pgman@candle.pha.pa.us>
cc: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] pg version date file
In-reply-to: Your message of Thu, 18 Nov 1999 19:22:48 -0500 (EST)
<199911190022.TAA19161@candle.pha.pa.us>
Date: Thu, 18 Nov 1999 22:24:52 -0500
Message-ID: <714.942981892@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Where is that file that makes an initdb required? We are supposed to
change that file when an initdb is needed.

src/include/catalog/catversion.h.

I put it there on the theory that include/catalog/*.h changes would
be the most common reason for wanting to bump the serial number.
But maybe it belongs somewhere else...

regards, tom lane

From bouncefilter Thu Nov 18 22:27:20 1999
Received: from candle.pha.pa.us (root@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA10725
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 22:26:54 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id WAA22622
for pgsql-hackers@postgreSQL.org; Thu, 18 Nov 1999 22:26:01 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911190326.WAA22622@candle.pha.pa.us>
Subject: Primary key requires SERIAL
To: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Date: Thu, 18 Nov 1999 22:26:01 -0500 (EST)
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

We currently only allow the words PRIMARY KEY on a SERIAL column. Is
there a reason we don't allow PRIMARY KEY on an integer field? Seems it
should be allowed.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Thu Nov 18 22:29:20 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA10877
for <hackers@postgreSQL.org>; Thu, 18 Nov 1999 22:28:38 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id WAA00739;
Thu, 18 Nov 1999 22:28:06 -0500 (EST)
To: wieck@debis.com (Jan Wieck)
cc: hackers@postgreSQL.org
Subject: Re: [HACKERS] psql & regress tests
In-reply-to: Your message of Fri, 19 Nov 1999 00:49:51 +0100 (MET)
<m11obJH-0003kGC@orion.SAPserv.Hamburg.dsh.de>
Date: Thu, 18 Nov 1999 22:28:05 -0500
Message-ID: <737.942982085@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

wieck@debis.com (Jan Wieck) writes:

I'm actually doing some tests if the 'make check' would be
possible. I.e. doing a complete install with POSTGRESDIR
below the regress dir, running initdb with the libdir and
datadir pointing into there too, then starting new postmaster
on a different port in background etc., etc., etc..
That would make it possible to do a complete check without
doing a regular install at all.

Sounds great! I was just griping elsewhere in this thread that we
needed that.

It's amusing how often we two have the same wishes or ideas.

Well, they're so obviously the Right Thing ;-)

regards, tom lane

From bouncefilter Thu Nov 18 22:34:20 1999
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA11629
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 22:33:55 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
WAA22967;
Thu, 18 Nov 1999 22:31:15 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911190331.WAA22967@candle.pha.pa.us>
Subject: Re: [HACKERS] pg version date file
In-Reply-To: <714.942981892@sss.pgh.pa.us> from Tom Lane at "Nov 18,
1999 10:24:52 pm"
To: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu, 18 Nov 1999 22:31:15 -0500 (EST)
CC: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Where is that file that makes an initdb required? We are supposed to
change that file when an initdb is needed.

src/include/catalog/catversion.h.

I put it there on the theory that include/catalog/*.h changes would
be the most common reason for wanting to bump the serial number.
But maybe it belongs somewhere else...

I just couldn't find it. I looked in include only because version.h.in
was there.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Thu Nov 18 22:44:20 1999
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA13290
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 22:43:24 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id WAA23334
for pgsql-hackers@postgreSQL.org; Thu, 18 Nov 1999 22:43:15 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911190343.WAA23334@candle.pha.pa.us>
Subject: 7.0 status request
To: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Date: Thu, 18 Nov 1999 22:43:15 -0500 (EST)
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Here are the major open issues for 7.0 that I remember:

Foreign Keys - Jan
WAL - Vadim
Function args - Tom
System indcxes - Bruce

Outer joins and new multi-query parse tree are questionable items for
7.0.

Is this currect?

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Thu Nov 18 22:47:20 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA13736
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 22:46:20 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id WAA00764;
Thu, 18 Nov 1999 22:45:15 -0500 (EST)
To: Bruce Momjian <pgman@candle.pha.pa.us>
cc: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Getting OID in psql of recent insert
In-reply-to: Your message of Thu, 18 Nov 1999 20:12:18 -0500 (EST)
<199911190112.UAA20026@candle.pha.pa.us>
Date: Thu, 18 Nov 1999 22:45:14 -0500
Message-ID: <762.942983114@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Bruce Momjian <pgman@candle.pha.pa.us> writes:

In writing the book, I see the serious limitation that there is no way
in psql to access the most recently inserted oid. Without it, there
seems to be no way to use the oid value as a foreign key in another
table.

Should I add a function to return the most recently assigned oid to the
backend, or is there a better way?

I'm not sure why, but a backend-side function seems like the wrong way
to approach it. I guess I'm worried that the state would be too
volatile on the backend side. (Example: if you use the hypothetical
lastoid() function in an SQL query that causes triggers or rules to
be invoked behind your back, those triggers/rules could do new inserts.
Will lastoid() still return the "right" value by the time it gets
executed?)

It'd certainly be easy enough for psql to save off the OID anytime it
gets an "INSERT nnn" command response. The missing link is to invent
a way for a psql script to access that value and insert it into
subsequent SQL commands.

If you want to attack this, I'd suggest thinking a little larger than
just the last-OID problem. I'd like to be able to save off both
insertion OIDs and values extracted by SELECTs into named variables
of some sort, and then insert those values into as many later commands
as I want. Right now there's no way to do any such thing in a psql
script; you have to move up a level of difficulty into ecpg or pgtcl
or even C code if your application needs this. Plain psql scripts
would become substantially more powerful if psql had a capability
like this.

OTOH: we shouldn't ask psql to do everything under the sun. I'd
certainly think that it'd be unreasonable to try to do conditional
evaluation or looping in psql scripts, for instance. Maybe the right
answer is to teach people a little bit about using honest-to-goodness
scripting languages when their applications reach this level of
complexity. How much daylight is there between needing script
variables and needing control flow, do you think?

regards, tom lane

PS: not relevant to your main point, but to your example: I think it's
a real bad idea to teach people to use OIDs as foreign keys. That'll
create all kinds of trouble when it comes time to dump/reload their
database. Better to tell them to use SERIAL columns as keys. Not so
incidentally, we have currval() already...

From bouncefilter Thu Nov 18 22:52:20 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id WAA14489
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 22:51:56 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id WAA00791;
Thu, 18 Nov 1999 22:51:20 -0500 (EST)
To: "Hiroshi Inoue" <Inoue@tpf.co.jp>
cc: "pgsql-hackers" <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Hash Join is very slooow in some cases
In-reply-to: Your message of Fri, 19 Nov 1999 10:14:56 +0900
<000201bf322b$7df2a620$2801007e@cadzone.tpf.co.jp>
Date: Thu, 18 Nov 1999 22:51:20 -0500
Message-ID: <789.942983480@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

"Hiroshi Inoue" <Inoue@tpf.co.jp> writes:

select count(*) from a,b where a.id1=b.id1;
returns immeidaitely ...
But
select count(*) from a,b where a.id1=b.id1 and a.id2=b.id2;
takes very looong time.
I examined an output by EXPLAIN VERBOSE and found that
the 1st query uses id1 as its hashkey and 2nd query uses id2
as its hashkey.

Yes, and since id2 has terrible disbursion, most of the hashtable
entries end up in a small number of hash buckets, resulting in
an unexpectedly large number of comparisons done for each outer
tuple. I've seen this effect before.

I have a TODO item to make the optimizer pay attention to disbursion
when estimating the cost of a hashjoin. That would cause it to make
the right choice of key in this example. Not done yet though :-(.
Feel free to jump in if you need it today...

regards, tom lane

From bouncefilter Thu Nov 18 23:08:21 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA17209
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 23:08:04 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id XAA00842;
Thu, 18 Nov 1999 23:06:58 -0500 (EST)
To: Bruce Momjian <pgman@candle.pha.pa.us>
cc: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] pg version date file
In-reply-to: Your message of Thu, 18 Nov 1999 22:31:15 -0500 (EST)
<199911190331.WAA22967@candle.pha.pa.us>
Date: Thu, 18 Nov 1999 23:06:58 -0500
Message-ID: <840.942984418@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Bruce Momjian <pgman@candle.pha.pa.us> writes:

src/include/catalog/catversion.h.

I put it there on the theory that include/catalog/*.h changes would
be the most common reason for wanting to bump the serial number.
But maybe it belongs somewhere else...

I just couldn't find it. I looked in include only because version.h.in
was there.

You could certainly make a case for moving it to src/include. If
you want to do that, I won't object.

regards, tom lane

From bouncefilter Thu Nov 18 23:16:21 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA18504
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 23:15:38 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id XAA00900;
Thu, 18 Nov 1999 23:14:35 -0500 (EST)
To: Bruce Momjian <pgman@candle.pha.pa.us>
cc: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Primary key requires SERIAL
In-reply-to: Your message of Thu, 18 Nov 1999 22:26:01 -0500 (EST)
<199911190326.WAA22622@candle.pha.pa.us>
Date: Thu, 18 Nov 1999 23:14:35 -0500
Message-ID: <898.942984875@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Bruce Momjian <pgman@candle.pha.pa.us> writes:

We currently only allow the words PRIMARY KEY on a SERIAL column.

Say what? There are ColConstraintElem and ConstraintElem productions
for PRIMARY KEY ... are they broken?

regards, tom lane

From bouncefilter Thu Nov 18 23:17:21 1999
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA18610
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 23:16:27 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
XAA24427;
Thu, 18 Nov 1999 23:16:13 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911190416.XAA24427@candle.pha.pa.us>
Subject: Re: [HACKERS] Getting OID in psql of recent insert
In-Reply-To: <762.942983114@sss.pgh.pa.us> from Tom Lane at "Nov 18,
1999 10:45:14 pm"
To: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu, 18 Nov 1999 23:16:13 -0500 (EST)
CC: "PostgreSQL-development"@candle.pha.pa.us, pgsql-hackers@postgreSQL.org,
peter_e@gmx.net
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

If you want to attack this, I'd suggest thinking a little larger than
just the last-OID problem. I'd like to be able to save off both
insertion OIDs and values extracted by SELECTs into named variables
of some sort, and then insert those values into as many later commands
as I want. Right now there's no way to do any such thing in a psql
script; you have to move up a level of difficulty into ecpg or pgtcl
or even C code if your application needs this. Plain psql scripts
would become substantially more powerful if psql had a capability
like this.

Yes, I understand. The new psql has the ability to have variables, so
this seems like a natural use for this:

testdb=> \set foo bar

Maybe we could have:

testdb=> \set foo lastoid

testdb=> \echo "foo is now ${foo}."

Seems those variables are not available in queries, though.

OTOH: we shouldn't ask psql to do everything under the sun. I'd
certainly think that it'd be unreasonable to try to do conditional
evaluation or looping in psql scripts, for instance. Maybe the right
answer is to teach people a little bit about using honest-to-goodness
scripting languages when their applications reach this level of
complexity. How much daylight is there between needing script
variables and needing control flow, do you think?

I think I agree, but a powerful psql interface is very important for any
database.

PS: not relevant to your main point, but to your example: I think it's
a real bad idea to teach people to use OIDs as foreign keys. That'll
create all kinds of trouble when it comes time to dump/reload their
database. Better to tell them to use SERIAL columns as keys. Not so
incidentally, we have currval() already...

OK, I am dealing with this in the book. What are oids good for then?

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Thu Nov 18 23:17:21 1999
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA18625
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 23:16:43 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
XAA24436;
Thu, 18 Nov 1999 23:16:30 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911190416.XAA24436@candle.pha.pa.us>
Subject: Re: [HACKERS] pg version date file
In-Reply-To: <840.942984418@sss.pgh.pa.us> from Tom Lane at "Nov 18,
1999 11:06:58 pm"
To: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu, 18 Nov 1999 23:16:30 -0500 (EST)
CC: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Bruce Momjian <pgman@candle.pha.pa.us> writes:

src/include/catalog/catversion.h.

I put it there on the theory that include/catalog/*.h changes would
be the most common reason for wanting to bump the serial number.
But maybe it belongs somewhere else...

I just couldn't find it. I looked in include only because version.h.in
was there.

You could certainly make a case for moving it to src/include. If
you want to do that, I won't object.

No, seems fine there.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Thu Nov 18 23:20:22 1999
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA18948
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 23:19:36 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
XAA24590;
Thu, 18 Nov 1999 23:19:23 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911190419.XAA24590@candle.pha.pa.us>
Subject: Re: [HACKERS] Primary key requires SERIAL
In-Reply-To: <898.942984875@sss.pgh.pa.us> from Tom Lane at "Nov 18,
1999 11:14:35 pm"
To: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu, 18 Nov 1999 23:19:23 -0500 (EST)
CC: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Bruce Momjian <pgman@candle.pha.pa.us> writes:

We currently only allow the words PRIMARY KEY on a SERIAL column.

Say what? There are ColConstraintElem and ConstraintElem productions
for PRIMARY KEY ... are they broken?

regards, tom lane

Oh, I see it now. The grammer seems to only support it in SERIAL, but I
see it works now. I guess i am surprised SERIAL PRIMARY creates the
index and sequence, while INTEGER PRIMARY only creates the index.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Thu Nov 18 23:35:22 1999
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de
[53.2.131.8]) by hub.org (8.9.3/8.9.3) with SMTP id XAA22006
for <hackers@postgreSQL.org>; Thu, 18 Nov 1999 23:35:13 -0500 (EST)
(envelope-from wieck@debis.com)
Received: by orion.SAPserv.Hamburg.dsh.de for hackers@postgreSQL.org
id m11ofdb-0003kGC; Fri, 19 Nov 99 05:27 MET
Message-Id: <m11ofdb-0003kGC@orion.SAPserv.Hamburg.dsh.de>
From: wieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] psql & regress tests
To: pgman@candle.pha.pa.us (Bruce Momjian)
Date: Fri, 19 Nov 1999 05:27:07 +0100 (MET)
Cc: wieck@debis.com, tgl@sss.pgh.pa.us, peter_e@gmx.net,
hackers@postgreSQL.org
Reply-To: wieck@debis.com (Jan Wieck)
In-Reply-To: <199911190242.VAA21649@candle.pha.pa.us> from "Bruce Momjian" at
Nov 18, 99 09:42:17 pm
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text

Bruce Momjian wrote:

ERROR: Bad boolean external representation 'XXX'
FATAL 1: SearchSysCache: recursive use of cache 10
FATAL 2: elog: error in postmaster or backend startup, giving up!
pq_flush: send() failed: Broken pipe
Server process (pid 9791) exited with status 512 at Fri Nov 19 03:17:09 1999
Terminating any active server processes...

It happens during the first parallel group of 11 tests. Not
allways, so it's timing critical. Outch.

Hmmm,

the first FATAL is emitted from catcache.c in line 988. I
think that the cache->busy lives in shared memory and isn't
protected against concurrent usage, as it should be. Cache
#10 is RELNAME. That really makes sense, because most of the
tests I'm running parallel now issue CREATE TABLE commands
first.

Now that we know numeric is working, can we make the test run faster in
the default mode?

It is already down to 100 digits after the decimal point. I
don't want to lower it too much, but maybe 30 or 50 are
enough too - no?

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck@debis.com (Jan Wieck) #

From bouncefilter Thu Nov 18 23:47:22 1999
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA23596
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 23:46:32 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id UAA14632;
Thu, 18 Nov 1999 20:46:05 -0800 (PST)
Message-Id: <3.0.1.32.19991118203420.00f1a9a0@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Thu, 18 Nov 1999 20:34:20 -0800
To: Bruce Momjian <pgman@candle.pha.pa.us>,
PostgreSQL-development <pgsql-hackers@postgreSQL.org>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: [HACKERS] Primary key requires SERIAL
In-Reply-To: <199911190326.WAA22622@candle.pha.pa.us>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 10:26 PM 11/18/99 -0500, Bruce Momjian wrote:

We currently only allow the words PRIMARY KEY on a SERIAL column. Is
there a reason we don't allow PRIMARY KEY on an integer field? Seems it
should be allowed.

Presumably the only reason to disallow this is to make life difficult
for those of us who want to port Oracle-based applications.

Given that Oracle represents a huge slice of the established market,
and given that in the past Postgres has been an "Oracle-friendly" db in
terms of dialectical support (nextval and currval on sequences being
germane to the subject at hand) one can only presume that the Postgres
development group wants to make porting of Oracle-ish systems difficult.

Why?

"Currently" must mean the 7.0-in-work because 6.5.1 supports primary
key on integer just fine.

Why support "serial" and not support "primary key on integer" when
Oracle rules the roost, not Sybase?

If your statement's true, this is a horrible shift in direction for
the PostgreSQL project.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Thu Nov 18 23:37:22 1999
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA22174
for <hackers@postgreSQL.org>; Thu, 18 Nov 1999 23:37:17 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
XAA25069;
Thu, 18 Nov 1999 23:36:58 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911190436.XAA25069@candle.pha.pa.us>
Subject: Re: [HACKERS] psql & regress tests
In-Reply-To: <m11ofdb-0003kGC@orion.SAPserv.Hamburg.dsh.de> from Jan Wieck at
"Nov 19, 1999 05:27:07 am"
To: Jan Wieck <wieck@debis.com>
Date: Thu, 18 Nov 1999 23:36:58 -0500 (EST)
CC: tgl@sss.pgh.pa.us, peter_e@gmx.net, hackers@postgreSQL.org
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Now that we know numeric is working, can we make the test run faster in
the default mode?

It is already down to 100 digits after the decimal point. I
don't want to lower it too much, but maybe 30 or 50 are
enough too - no?

It is just taking longer than most other tests. Seems 30 would be fine.
They can always run the bigtest.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Thu Nov 18 23:47:22 1999
Received: from eclipse.pacifier.com (eclipse.pacifier.com [199.2.117.78])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA23591
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 23:46:31 -0500 (EST)
(envelope-from dhogaza@pacifier.com)
Received: from desktop (dsl-dhogaza.pacifier.net [216.65.147.68])
by eclipse.pacifier.com (8.9.3/8.9.3pop) with SMTP id UAA14646;
Thu, 18 Nov 1999 20:46:07 -0800 (PST)
Message-Id: <3.0.1.32.19991118204353.00f1e7a0@mail.pacifier.com>
X-Sender: dhogaza@mail.pacifier.com
X-Mailer: Windows Eudora Pro Version 3.0.1 (32)
Date: Thu, 18 Nov 1999 20:43:53 -0800
To: Bruce Momjian <pgman@candle.pha.pa.us>, Tom Lane <tgl@sss.pgh.pa.us>
From: Don Baccus <dhogaza@pacifier.com>
Subject: Re: [HACKERS] Primary key requires SERIAL
Cc: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
In-Reply-To: <199911190419.XAA24590@candle.pha.pa.us>
References: <898.942984875@sss.pgh.pa.us>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

At 11:19 PM 11/18/99 -0500, Bruce Momjian wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

We currently only allow the words PRIMARY KEY on a SERIAL column.

Say what? There are ColConstraintElem and ConstraintElem productions
for PRIMARY KEY ... are they broken?

regards, tom lane

Oh, I see it now. The grammer seems to only support it in SERIAL, but I
see it works now. I guess i am surprised SERIAL PRIMARY creates the
index and sequence, while INTEGER PRIMARY only creates the index.

Oops, I guess I blew it by responding to a post by Bruce assuming he
was right.

Postgres supports a quasi-serial type by creating an index and
sequence (while Sybase supports it more transparently)

Postgres REALLY supports sequences much like Oracle (and others?
I don't know, my DB knowledge is very sketchy). In Oracle, if
you define a primary key of type integer and want to sequence
it, you define a sequence and use "sequence_name.nextval" and
"sequence_name.currval". This is very much like "nextval" and
"currval" in Postgres, and I presume no accident.

And in Oracle you create the sequence by hand - just like you do
in Postgres.

Personally, I think maintaining an "Oracle-ish" framework is wise,
for the simple selfish reason that I'm interested in porting
Oracle-dependent SQL to Postgres.

If being "Oracle-ish" is still a goal (it was once a goal of at
least some of the implementors, it's obvious) then generating
the sequence just makes porting more difficult.

Actually, I think the inclusion of "serial" as a more integrated
type and leaving primary key stuff alone for existing types is
what makes sense. You could provide a higher level of Sybase
portability without messing up us Oracle-derived folk.

- Don Baccus, Portland OR <dhogaza@pacifier.com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.

From bouncefilter Thu Nov 18 23:49:22 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA23794
for <hackers@postgreSQL.org>; Thu, 18 Nov 1999 23:48:33 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id XAA01016;
Thu, 18 Nov 1999 23:48:00 -0500 (EST)
To: wieck@debis.com (Jan Wieck)
cc: hackers@postgreSQL.org
Subject: Re: [HACKERS] psql & regress tests
In-reply-to: Your message of Fri, 19 Nov 1999 03:24:43 +0100 (MET)
<m11odj9-0003kGC@orion.SAPserv.Hamburg.dsh.de>
Date: Thu, 18 Nov 1999 23:48:00 -0500
Message-ID: <1014.942986880@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

wieck@debis.com (Jan Wieck) writes:

The ugly detail, I'm currently running into, is that there
already seems to be a concurrency problem I discovered with
my testing. Occationally I get this into the postmaster log
for parallel executing tests:

ERROR: Bad boolean external representation 'XXX'
FATAL 1: SearchSysCache: recursive use of cache 10
FATAL 2: elog: error in postmaster or backend startup, giving up!
pq_flush: send() failed: Broken pipe
Server process (pid 9791) exited with status 512 at Fri Nov 19 03:17:09 1999
Terminating any active server processes...

It happens during the first parallel group of 11 tests. Not
allways, so it's timing critical. Outch.

In other words, you've already exposed a bug! Right on!

Commit the thing, so more eyes can look for the problem. I expect
you have found a pre-existing backend bug, not a problem in your
new regress test scaffold.

regards, tom lane

From bouncefilter Fri Nov 19 00:16:30 1999
Received: from trends.net (clio.trends.ca [209.47.148.2])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA30890
for <pgsql-hackers@postgreSQL.org>;
Fri, 19 Nov 1999 00:15:52 -0500 (EST) (envelope-from aaron@gtv.ca)
Received: from genisys.gtv.ca (h-207-228-97-106.gen.cadvision.com
[207.228.97.106]) by trends.net (8.8.8/8.8.8) with ESMTP id AAA25328
for <pgsql-hackers@postgreSQL.org>;
Fri, 19 Nov 1999 00:01:16 -0500 (EST)
Received: from stilborne (24.67.89.147.ab.wave.home.com [24.67.89.147])
by genisys.gtv.ca (8.9.3/8.8.7) with SMTP id WAA00401
for <pgsql-hackers@postgreSQL.org>; Thu, 18 Nov 1999 22:01:20 -0700
From: "Aaron J. Seigo" <aaron@gtv.ca>
To: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Getting OID in psql of recent insert
Date: Thu, 18 Nov 1999 21:48:54 -0700
X-Mailer: KMail [version 1.0.28]
Content-Type: text/plain
References: <199911190416.XAA24427@candle.pha.pa.us>
In-Reply-To: <199911190416.XAA24427@candle.pha.pa.us>
MIME-Version: 1.0
Message-Id: <99111821550700.01629@stilborne>
Content-Transfer-Encoding: 8bit

On Thu, 18 Nov 1999, Bruce Momjian wrote:

If you want to attack this, I'd suggest thinking a little larger than
just the last-OID problem. I'd like to be able to save off both
insertion OIDs and values extracted by SELECTs into named variables
of some sort, and then insert those values into as many later commands
as I want. Right now there's no way to do any such thing in a psql
script; you have to move up a level of difficulty into ecpg or pgtcl
or even C code if your application needs this. Plain psql scripts
would become substantially more powerful if psql had a capability
like this.

we talked about this a few weeks ago as users... even those of us using C or
higher level scripting languages agreed it would be nice to be able to have
arbitrary values that are the result of an insert/update/delete able to be
returned, without a subsequent select. if this made it into postgres, i think
you'd have many happy users =)

OK, I am dealing with this in the book. What are oids good for then?

i can tell you what i use them for as someone who works with postgres daily...
i'm not sure if this was what they were intended for.. but =)

once inserted, a row keeps its oid. so, when performing complex selects, i'll
often grab the oid too... do some tests on the returned values, and if an action
is appropriate on that row, i reference it by its oid. the only chance of this
failing is if the database is dumped then restored between the select and the
update (not gonna happen, as the program requires the database available for
execution)... using the oid this way, its often simpler and faster to update a
known row, especially when the initial select involved many fields.

--
Aaron J. Seigo
Sys Admin

From bouncefilter Thu Nov 18 23:59:22 1999
Received: from candle.pha.pa.us (pgman@s5-03.ppp.op.net [209.152.195.67])
by hub.org (8.9.3/8.9.3) with ESMTP id XAA25357
for <pgsql-hackers@postgreSQL.org>;
Thu, 18 Nov 1999 23:58:50 -0500 (EST)
(envelope-from pgman@candle.pha.pa.us)
Received: (from pgman@localhost) by candle.pha.pa.us (8.9.0/8.9.0) id
XAA25578;
Thu, 18 Nov 1999 23:58:19 -0500 (EST)
From: Bruce Momjian <pgman@candle.pha.pa.us>
Message-Id: <199911190458.XAA25578@candle.pha.pa.us>
Subject: Re: [HACKERS] Primary key requires SERIAL
In-Reply-To: <3.0.1.32.19991118204353.00f1e7a0@mail.pacifier.com> from Don
Baccus at "Nov 18, 1999 08:43:53 pm"
To: Don Baccus <dhogaza@pacifier.com>
Date: Thu, 18 Nov 1999 23:58:19 -0500 (EST)
CC: Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL-development <pgsql-hackers@postgreSQL.org>
X-Mailer: ELM [version 2.4ME+ PL65 (25)]
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

At 11:19 PM 11/18/99 -0500, Bruce Momjian wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

We currently only allow the words PRIMARY KEY on a SERIAL column.

Say what? There are ColConstraintElem and ConstraintElem productions
for PRIMARY KEY ... are they broken?

regards, tom lane

Oh, I see it now. The grammer seems to only support it in SERIAL, but I
see it works now. I guess i am surprised SERIAL PRIMARY creates the
index and sequence, while INTEGER PRIMARY only creates the index.

Oops, I guess I blew it by responding to a post by Bruce assuming he
was right.

Seems I can't realy on understanding what we support by just looking at
gram.y.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

From bouncefilter Fri Nov 19 00:41:25 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA34123
for <pgsql-hackers@postgreSQL.org>;
Fri, 19 Nov 1999 00:41:12 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id AAA01197;
Fri, 19 Nov 1999 00:40:02 -0500 (EST)
To: "Aaron J. Seigo" <aaron@gtv.ca>
cc: pgsql-hackers@postgreSQL.org
Subject: Re: [HACKERS] Getting OID in psql of recent insert
In-reply-to: Your message of Thu, 18 Nov 1999 21:48:54 -0700
<99111821550700.01629@stilborne>
Date: Fri, 19 Nov 1999 00:40:02 -0500
Message-ID: <1195.942990002@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

"Aaron J. Seigo" <aaron@gtv.ca> writes:

On Thu, 18 Nov 1999, Bruce Momjian wrote:

OK, I am dealing with this in the book. What are oids good for then?

once inserted, a row keeps its oid. so, when performing complex
selects, i'll often grab the oid too... do some tests on the returned
values, and if an action is appropriate on that row, i reference it by
its oid. the only chance of this failing is if the database is dumped
then restored between the select and the update (not gonna happen, as
the program requires the database available for execution)... using
the oid this way, its often simpler and faster to update a known row,
especially when the initial select involved many fields.

Yes, I use 'em the same way. I think an OID is kind of like a pointer
in a C program: good for fast, unique access to an object within the
context of the execution of a particular application (and maybe not
even that long). You don't write pointers into files to be used again
by other programs, though, and in the same way an OID isn't a good
candidate for a long-lasting reference from one table to another.

regards, tom lane

From bouncefilter Fri Nov 19 00:51:25 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id AAA35290
for <hackers@postgreSQL.org>; Fri, 19 Nov 1999 00:51:18 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id AAA01264;
Fri, 19 Nov 1999 00:50:32 -0500 (EST)
To: wieck@debis.com (Jan Wieck)
cc: pgman@candle.pha.pa.us (Bruce Momjian), peter_e@gmx.net,
hackers@postgreSQL.org
Subject: Re: [HACKERS] psql & regress tests
In-reply-to: Your message of Fri, 19 Nov 1999 05:27:07 +0100 (MET)
<m11ofdb-0003kGC@orion.SAPserv.Hamburg.dsh.de>
Date: Fri, 19 Nov 1999 00:50:32 -0500
Message-ID: <1262.942990632@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

wieck@debis.com (Jan Wieck) writes:

Bruce Momjian wrote:

Now that we know numeric is working, can we make the test run faster in
the default mode?

It is already down to 100 digits after the decimal point. I
don't want to lower it too much, but maybe 30 or 50 are
enough too - no?

Since multiply and so on are presumably O(N^2), cutting the precision
to 30 might cut the runtime by almost a factor of 10.

Jan probably has a better idea than the rest of us whether a test of
100, or 30, or 10 digits is likely to expose bugs that would not be
exposed by a test with less precision --- that depends on whether the
code has any internal behavioral dependence on the length of numeric
values. The numeric test certainly is a lot slower than the others, so
I think it would be a good idea to trim the precision as much as we can.
Anyone who's actually touching the numeric code could and should run
the "bigtest", but the rest of us just want to know whether we've got
porting problems. Seems like it shouldn't take 100-digit tests to
expose porting problems.

regards, tom lane

From bouncefilter Fri Nov 19 01:34:26 1999
Received: from localhost (IDENT:root@hectic-1.jpl.nasa.gov [128.149.68.203])
by hub.org (8.9.3/8.9.3) with ESMTP id BAA39532
for <pgsql-hackers@postgreSQL.org>;
Fri, 19 Nov 1999 01:33:50 -0500 (EST)
(envelope-from lockhart@alumni.caltech.edu)
Received: from alumni.caltech.edu (lockhart@localhost [127.0.0.1])
by localhost (8.8.7/8.8.7) with ESMTP id GAA21831;
Fri, 19 Nov 1999 06:32:14 GMT
Sender: lockhart@hub.org
Message-ID: <3834EEEE.DEE4EBD1@alumni.caltech.edu>
Date: Fri, 19 Nov 1999 06:32:14 +0000
From: Thomas Lockhart <lockhart@alumni.caltech.edu>
X-Mailer: Mozilla 4.6 [en] (X11; I; Linux 2.0.36 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Bruce Momjian <pgman@candle.pha.pa.us>
CC: Don Baccus <dhogaza@pacifier.com>, Tom Lane <tgl@sss.pgh.pa.us>,
PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Primary key requires SERIAL
References: <199911190458.XAA25578@candle.pha.pa.us>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

We currently only allow the words PRIMARY KEY on a SERIAL column.

Oops, I guess I blew it by responding to a post by Bruce assuming he
was right.

Seems I can't realy on understanding what we support by just looking at
gram.y.

You can, if you read carefully :)

The grammar allows *only* PRIMARY KEY on the SERIAL column
declaration, since the other keywords or clauses are either redundant
or nonsensical in the context of a serial column. As others have
pointed out, PRIMARY KEY is also allowed elsewhere.

- Thomas

--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California

From bouncefilter Fri Nov 19 01:41:26 1999
Received: from localhost (IDENT:root@hectic-1.jpl.nasa.gov [128.149.68.203])
by hub.org (8.9.3/8.9.3) with ESMTP id BAA40447
for <pgsql-hackers@postgreSQL.org>;
Fri, 19 Nov 1999 01:40:54 -0500 (EST)
(envelope-from lockhart@alumni.caltech.edu)
Received: from alumni.caltech.edu (lockhart@localhost [127.0.0.1])
by localhost (8.8.7/8.8.7) with ESMTP id GAA21841;
Fri, 19 Nov 1999 06:35:59 GMT
Sender: lockhart@hub.org
Message-ID: <3834EFCF.48875CF1@alumni.caltech.edu>
Date: Fri, 19 Nov 1999 06:35:59 +0000
From: Thomas Lockhart <lockhart@alumni.caltech.edu>
X-Mailer: Mozilla 4.6 [en] (X11; I; Linux 2.0.36 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Bruce Momjian <pgman@candle.pha.pa.us>
CC: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] 7.0 status request
References: <199911190343.WAA23334@candle.pha.pa.us>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Here are the major open issues for 7.0 that I remember:
Foreign Keys - Jan
WAL - Vadim
Function args - Tom
System indcxes - Bruce
Outer joins and new multi-query parse tree are questionable items for
7.0.

You might include "join syntax", which will be ready even if outer
joins are not.

Also, didn't some folks express concern that indices on system tables
would make the backend more fragile? Did we resolve that issue?

- Thomas

--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California

From bouncefilter Fri Nov 19 01:50:26 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id BAA41260
for <pgsql-hackers@postgreSQL.org>;
Fri, 19 Nov 1999 01:49:32 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id BAA01504;
Fri, 19 Nov 1999 01:48:03 -0500 (EST)
To: Thomas Lockhart <lockhart@alumni.caltech.edu>
cc: Bruce Momjian <pgman@candle.pha.pa.us>, Don Baccus <dhogaza@pacifier.com>,
PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] Primary key requires SERIAL
In-reply-to: Your message of Fri, 19 Nov 1999 06:32:14 +0000
<3834EEEE.DEE4EBD1@alumni.caltech.edu>
Date: Fri, 19 Nov 1999 01:48:03 -0500
Message-ID: <1502.942994083@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Thomas Lockhart <lockhart@alumni.caltech.edu> writes:

The grammar allows *only* PRIMARY KEY on the SERIAL column
declaration, since the other keywords or clauses are either redundant
or nonsensical in the context of a serial column.

Just to put another item on your todo list ;-) ...

I think it's poor practice to try to enforce such a restriction via
the grammar, because that way you cannot generate an error more
specific than "parse error near FOO". It'd be better to allow the
same ColQualifier for SERIAL as for any other column type, and then to
put sanity checks in analyze.c that would complain about conflicting
specifications. We have, or should have, most of those checks in
place already to catch conflicting ColQualifier entries for a plain
column type (eg, "foo int4 NULL NOT NULL"). Also, I do not like
generating hard errors for specifications that are merely redundant
("foo SERIAL NOT NULL"); is there any basis in the SQL spec for
refusing such constructs?

regards, tom lane

From bouncefilter Fri Nov 19 02:04:27 1999
Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
by hub.org (8.9.3/8.9.3) with ESMTP id CAA43004
for <pgsql-hackers@postgreSQL.org>;
Fri, 19 Nov 1999 02:03:49 -0500 (EST)
(envelope-from tgl@sss.pgh.pa.us)
Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id CAA01553;
Fri, 19 Nov 1999 02:02:45 -0500 (EST)
To: Bruce Momjian <pgman@candle.pha.pa.us>
cc: PostgreSQL-development <pgsql-hackers@postgreSQL.org>
Subject: Re: [HACKERS] 7.0 status request
In-reply-to: Your message of Thu, 18 Nov 1999 22:43:15 -0500 (EST)
<199911190343.WAA23334@candle.pha.pa.us>
Date: Fri, 19 Nov 1999 02:02:45 -0500
Message-ID: <1551.942994965@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Here are the major open issues for 7.0 that I remember:
Foreign Keys - Jan
WAL - Vadim
Function args - Tom
System indcxes - Bruce
Outer joins and new multi-query parse tree are questionable items for
7.0.

I have a bunch of optimizer tweaking that I'd like to finish before 7.0,
but perhaps that doesn't qualify as a major open issue. There's no
single item there that I would rank as "must fix or don't ship"; yet
I feel we need to make some progress in that area.

I think there are also a lot of unresolved questions about interlocking
and updating of the catalog caches and relcache. These might be
must-fix items. IIRC, Hiroshi is pretty concerned about that area...

regards, tom lane