I've got it, now should I commit it?

After reading a couple more complaints of hashtable-overflow error
messages, I went ahead and rewrote the hash join modules so that they
don't use fixed-size hash buckets and a fixed-size overflow area.
Instead, each bucket is just a linked list of tuples (thus no wasted
space for underused buckets) and everything is put into a private portal
so that reclaiming the space is easy/quick. The code is noticeably
shorter and more readable than before.

The limited amount of testing I've been able to do here shows no
problems.

Now: do I commit it, or wait till after 6.5? I promised Marc the latter
a couple weeks ago, but I am mighty tempted to just go for it...

Shhh. He will never know. Did you promise Marc, or did you answer him
evasively, like I suggested?

Basically, with the new optimizer, this may be a bug fix because of the
more frequent hashjoins. That has always been my smokescreen to add the
feature.

:-)

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

On Tue, 18 May 1999, Bruce Momjian wrote:

After reading a couple more complaints of hashtable-overflow error
messages, I went ahead and rewrote the hash join modules so that they
don't use fixed-size hash buckets and a fixed-size overflow area.
Instead, each bucket is just a linked list of tuples (thus no wasted
space for underused buckets) and everything is put into a private portal
so that reclaiming the space is easy/quick. The code is noticeably
shorter and more readable than before.

The limited amount of testing I've been able to do here shows no
problems.

Now: do I commit it, or wait till after 6.5? I promised Marc the latter
a couple weeks ago, but I am mighty tempted to just go for it...

Shhh. He will never know. Did you promise Marc, or did you answer him
evasively, like I suggested?

Basically, with the new optimizer, this may be a bug fix because of the
more frequent hashjoins. That has always been my smokescreen to add the
feature.

Tom...make you a deal. If you are confident enough with the code that
when v6.5 goes out in ~13days, it won't generate more bug reports then its
fixing...go for it. :)

Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

Basically, with the new optimizer, this may be a bug fix because of the
more frequent hashjoins. That has always been my smokescreen to add the
feature.

Tom...make you a deal. If you are confident enough with the code that
when v6.5 goes out in ~13days, it won't generate more bug reports then its
fixing...go for it. :)

Ah, man, that was way too easy. :-)

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

The Hermit Hacker <scrappy@hub.org> writes:

Basically, with the new optimizer, this may be a bug fix because of the
more frequent hashjoins. That has always been my smokescreen to add the
feature.

Tom...make you a deal. If you are confident enough with the code that
when v6.5 goes out in ~13days, it won't generate more bug reports then its
fixing...go for it. :)

OK, you're on --- I feel pretty good about this code, although I'm never
prepared to guarantee zero bugs ;-). If there are any, we can hope
they'll show up before the end of beta.

A note for anyone testing the new code: the hashtable size (which is now
a target estimate, not a hard limit) is now driven by the postmaster's
-S switch, not the -B switch. -S seems more reasonable since the table
is private memory in a backend, not shared memory.

regards, tom lane

The Hermit Hacker <scrappy@hub.org> writes:

Basically, with the new optimizer, this may be a bug fix because of the
more frequent hashjoins. That has always been my smokescreen to add the
feature.

Tom...make you a deal. If you are confident enough with the code that
when v6.5 goes out in ~13days, it won't generate more bug reports then its
fixing...go for it. :)

OK, you're on --- I feel pretty good about this code, although I'm never
prepared to guarantee zero bugs ;-). If there are any, we can hope
they'll show up before the end of beta.

A note for anyone testing the new code: the hashtable size (which is now
a target estimate, not a hard limit) is now driven by the postmaster's
-S switch, not the -B switch. -S seems more reasonable since the table
is private memory in a backend, not shared memory.

I see no documenation that -B was ever used for hash size. I see -B for
shared buffers for both postmaster and postgres manual pages.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Bruce Momjian <maillist@candle.pha.pa.us> writes:

A note for anyone testing the new code: the hashtable size (which is now
a target estimate, not a hard limit) is now driven by the postmaster's
-S switch, not the -B switch.

I see no documenation that -B was ever used for hash size.

Er, did I say anything about documentation?

The code *was* using NBuffers to size the hashtable, whether or not
that was ever documented anywhere except in the "hash table out of
memory. Use -B parameter to increase buffers" message. Now it uses
the SortMem variable.

I do have it on my to-do list to update the relevant documentation.
(Yo, Thomas: what's the deadline for 6.5 doco changes? I've got a bunch
of doc to-dos that I suspect I'd better get moving on...)

regards, tom lane

(Yo, Thomas: what's the deadline for 6.5 doco changes? I've got a bunch
of doc to-dos that I suspect I'd better get moving on...)

Yup. Nominally, I should have frozen about May 15, but I still have
some writing I want to do. I can freeze docs in stages; which docs are
you planning on touching?

Bruce, can we get the sgml version of release notes started?

Vadim, you had mentioned some docs for MVCC; where would that show up?
If nothing else, we should update ref/lock.sgml and ref/set.sgml to
cover the grammar changes. And it would be great to have some words
for the User's Guide.

- Thomas

--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California

Thomas Lockhart <lockhart@alumni.caltech.edu> writes:

I can freeze docs in stages; which docs are
you planning on touching?

Several, but for most of them I have only small changes. I will try to
do those tonight so that as much as possible can be frozen, and then let
you know what I still have to work on.

Is there any equivalent in the SGML docs to the postmaster.1 and
postgres.1 man pages (specifically, doco for the postmaster/backend
command line switches)? I couldn't find it but maybe it's there...

regards, tom lane

Several, but for most of them I have only small changes. I will try to
do those tonight so that as much as possible can be frozen, and then let
you know what I still have to work on.

Great.

Is there any equivalent in the SGML docs to the postmaster.1 and
postgres.1 man pages (specifically, doco for the postmaster/backend
command line switches)? I couldn't find it but maybe it's there...

No, but I want to add them by converting the man pages to User's Guide
reference pages. Will wait a day or two for you to update the man
pages, unless you would prefer to have me convert and then make your
changes directly in sgml.

I'm doing a small reorg on the Admin Guide. Most chapters stay the
same, but things are streamlined and flow better.

- Thomas

--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California

Thomas Lockhart <lockhart@alumni.caltech.edu> writes:

Is there any equivalent in the SGML docs to the postmaster.1 and
postgres.1 man pages (specifically, doco for the postmaster/backend
command line switches)? I couldn't find it but maybe it's there...

No, but I want to add them by converting the man pages to User's Guide
reference pages. Will wait a day or two for you to update the man
pages,

Already committed my changes to the .1 files; you may fire when ready.
(Unless anyone else has updates to make there?)

regards, tom lane