cvs problem

Hello

What is the rule size for views?

This seems to have changed in this beta; I have a view that worked in a
previous beta but fails to create.
"ERROR: DefineQueryRewrite: rule plan string too big."

I have about 33 column headings with a join of 5 tables.

I have the latest cvsup tree as of a few minutes ago.
Linux 2.0.30
pgcc compiler

Michael

* Michael J. Rogan, Network Administrator, 905-624-3020 *
* Mark IV Industries, F-P Electronics & I.V.H.S. Divisions *
* mrogan@fpelectronics.com mrogan@ivhs.com *

darrenk

So when I do it as a select statement the plan can grow larger than 8K
because I can run the SQL command itself.

Has the plan generation changed in the last week? I was able to create
another view in an earlier beta but cannot now.

Is there a way to find out the size of the plan?

Michael

What is the rule size for views?

This seems to have changed in this beta; I have a view that worked in a
previous beta but fails to create.
"ERROR: DefineQueryRewrite: rule plan string too big."

I have about 33 column headings with a join of 5 tables.

I have the latest cvsup tree as of a few minutes ago.
Linux 2.0.30
pgcc compiler

The rule plan is stored as a tuple, hence for this release that limit
is a little under 8k.

For 6.4, you will be able to use a bigger block size, therefore have
bigger tuples and this should then work again.

Sounds like the plan internally grew (possible w/subselect stuff?!?) and
has gone over that 8k limit.

I know it's little consolation, but as soon as I can get the variable
block size working (actually only breaks in the create_triggers regression
test when sorting to disk), I could post it as a patch for 6.3 as well as
put it into the 6.4 tree.

darrenk

* Michael J. Rogan, Network Administrator, 905-624-3020 *
* Mark IV Industries, F-P Electronics & I.V.H.S. Divisions *
* mrogan@fpelectronics.com mrogan@ivhs.com *

Fixed, sorry for the delay...

On Sat, 28 Feb 1998, Bruce Momjian wrote:

Still getting these errors:

cvs checkout: cannot open
/usr/local/cvsroot/pgsql/src/test/regress/regression.Solaris-Sparc,v:
Permission denied
cvs checkout: cannot open
/usr/local/cvsroot/pgsql/src/test/regress/regression.Solaris-Sparc,v:
Permission denied
cvs checkout: nothing known about
pgsql/src/test/regress/regression.Solaris-Sparc
$
-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
+  If your life is a hard drive,     |  (610) 353-9879(w)
+  Christ can be your backup.        |  (610) 853-3000(h)

Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

What is the rule size for views?

This seems to have changed in this beta; I have a view that worked in a
previous beta but fails to create.
"ERROR: DefineQueryRewrite: rule plan string too big."

I have about 33 column headings with a join of 5 tables.

I have the latest cvsup tree as of a few minutes ago.
Linux 2.0.30
pgcc compiler

The rule plan is stored as a tuple, hence for this release that limit
is a little under 8k.

For 6.4, you will be able to use a bigger block size, therefore have
bigger tuples and this should then work again.

Sounds like the plan internally grew (possible w/subselect stuff?!?) and
has gone over that 8k limit.

I know it's little consolation, but as soon as I can get the variable
block size working (actually only breaks in the create_triggers regression
test when sorting to disk), I could post it as a patch for 6.3 as well as
put it into the 6.4 tree.

darrenk

Hello

What is the rule size for views?

This seems to have changed in this beta; I have a view that worked in a
previous beta but fails to create.
"ERROR: DefineQueryRewrite: rule plan string too big."

I have about 33 column headings with a join of 5 tables.

I have the latest cvsup tree as of a few minutes ago.
Linux 2.0.30
pgcc compiler

Yes. More of the structure elements are dumped into the view than it
used to. We hoped to allow larger block sizes in 6.3, but no luck.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Sounds like the plan internally grew (possible w/subselect stuff?!?) and
has gone over that 8k limit.

I know it's little consolation, but as soon as I can get the variable
block size working (actually only breaks in the create_triggers regression
test when sorting to disk), I could post it as a patch for 6.3 as well as
put it into the 6.4 tree.

Some structure members were skipped in the dump, but now they are
included.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

darrenk

Just 'Darren' for short. :)

So when I do it as a select statement the plan can grow larger than 8K
because I can run the SQL command itself.

The plan is the internal plan generated by the parser/optimizer. A very
short sql statement could generate a very big plan. No real easy way to
tell without actually running it.

Has the plan generation changed in the last week? I was able to create
another view in an earlier beta but cannot now.

This I don't know. Any little change to the nodes or the parser/optimizer
could have bumped you over the limit.

Is there a way to find out the size of the plan?

Could add some debugging statements in the rewrite code to show it. Snoop
around the src/backend/rewrite directory a little.

Darren

What is the rule size for views?

This seems to have changed in this beta; I have a view that worked in a
previous beta but fails to create.
"ERROR: DefineQueryRewrite: rule plan string too big."

I have about 33 column headings with a join of 5 tables.

I have the latest cvsup tree as of a few minutes ago.
Linux 2.0.30
pgcc compiler

The rule plan is stored as a tuple, hence for this release that limit
is a little under 8k.

For 6.4, you will be able to use a bigger block size, therefore have
bigger tuples and this should then work again.

Sounds like the plan internally grew (possible w/subselect stuff?!?) and
has gone over that 8k limit.

I think it must be due to the cleanups in the node-print and
read funcs. They now output/read ALL fields in the nodes.

I know it's little consolation, but as soon as I can get the variable
block size working (actually only breaks in the create_triggers regression
test when sorting to disk), I could post it as a patch for 6.3 as well as
put it into the 6.4 tree.

darrenk

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#======================================== jwieck@debis.com (Jan Wieck) #

Sounds like the plan internally grew (possible w/subselect stuff?!?) and
has gone over that 8k limit.

I think it must be due to the cleanups in the node-print and
read funcs. They now output/read ALL fields in the nodes.

Oh, I didn't realize that the print functions were actually used for something
other than printing and debugging. I had started to add a few new nodes when I
was trying to debug the "primary key" code.

Should we go through and bracket some of those with #ifdef QUERYDEBUG or
something like that? Where are they actually used? Should we try to keep these
at a minimum for production compiles of the system??

- Tom

Sounds like the plan internally grew (possible w/subselect stuff?!?) and
has gone over that 8k limit.

I think it must be due to the cleanups in the node-print and
read funcs. They now output/read ALL fields in the nodes.

Oh, I didn't realize that the print functions were actually used for something
other than printing and debugging. I had started to add a few new nodes when I
was trying to debug the "primary key" code.

Should we go through and bracket some of those with #ifdef QUERYDEBUG or
something like that? Where are they actually used? Should we try to keep these
at a minimum for production compiles of the system??

Did you add stuff to dump that wasn't dumped before, or were they fields
of existing structure that used to be skipped?

They are used for the rewrite system and for views.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Sounds like the plan internally grew (possible w/subselect stuff?!?) and
has gone over that 8k limit.

I think it must be due to the cleanups in the node-print and
read funcs. They now output/read ALL fields in the nodes.

Oh, I didn't realize that the print functions were actually used for something
other than printing and debugging. I had started to add a few new nodes when I
was trying to debug the "primary key" code.

Should we go through and bracket some of those with #ifdef QUERYDEBUG or
something like that? Where are they actually used? Should we try to keep these
at a minimum for production compiles of the system??

Did you add stuff to dump that wasn't dumped before, or were they fields
of existing structure that used to be skipped?

They are used for the rewrite system and for views.

I added a few nodes (maybe two?), _and_ added some call-outs to existing nodes to
follow their children down. This stuff can be bracketed with debugging #ifdef's; it
was very helpful for me when debugging but it isn't good if they are adding
unnecessary limitations on sizes. The additional nodes I added are a "don't care";
it's the additional printing of child nodes (fields of existing structures) which
is loading things down.

I am not absolutely certain that we are talking about the same thing, so should
confirm that it is the same routines which are causing trouble; this is the same
printing/formatting code used for "explain"??

I'd be happy to help clean them up, but I think you should help too; as I recall we
were both mucking around there about the same time ;)

- Tom

I added a few nodes (maybe two?), _and_ added some call-outs to existing nodes to
follow their children down. This stuff can be bracketed with debugging #ifdef's; it
was very helpful for me when debugging but it isn't good if they are adding
unnecessary limitations on sizes. The additional nodes I added are a "don't care";
it's the additional printing of child nodes (fields of existing structures) which
is loading things down.

The stuff is in nodes/outfuncs.c, and is used in EXPLAIN VERBOSE. I
question whether your structures would actually be output as part of a
rule.

I hesitate to remove any of the outfuncs stuff. It is very useful, and
if it is missing, things are harder to debug. Adding the fields I did
helped solve several problems I had when testing subselects, and I know
Vadim uses that output too. Shame it goes into the rule, but hard to
imagine why the rule would not need it, except for fields that are only
used by the parser, but I think we need to be complete. A better
solution would be to allow rewrite rules to span multiple blocks, or a
least allow them to take the space of two blocks.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Hi,

Sounds like the plan internally grew (possible w/subselect stuff?!?) and
has gone over that 8k limit.

I think it must be due to the cleanups in the node-print and
read funcs. They now output/read ALL fields in the nodes.

Oh, I didn't realize that the print functions were actually used for something
other than printing and debugging. I had started to add a few new nodes when I
was trying to debug the "primary key" code.

Should we go through and bracket some of those with #ifdef QUERYDEBUG or
something like that? Where are they actually used? Should we try to keep these
at a minimum for production compiles of the system??

Did you add stuff to dump that wasn't dumped before, or were they fields
of existing structure that used to be skipped?

They are used for the rewrite system and for views.

I added a few nodes (maybe two?), _and_ added some call-outs to existing nodes to
follow their children down. This stuff can be bracketed with debugging #ifdef's; it
was very helpful for me when debugging but it isn't good if they are adding
unnecessary limitations on sizes. The additional nodes I added are a "don't care";
it's the additional printing of child nodes (fields of existing structures) which
is loading things down.

I am not absolutely certain that we are talking about the same thing, so should
confirm that it is the same routines which are causing trouble; this is the same
printing/formatting code used for "explain"??

I'd be happy to help clean them up, but I think you should help too; as I recall we
were both mucking around there about the same time ;)

I think we talk about the same group of functions. But I'm
not sure if the nodes you're talking about are parsenodes.
Only parsenodes are printed during rule creation to build the
querytree string stored in the pg_rewrite catalog (views are
implemented by rules and do an internal CREATE RULE on CREATE
VIEW).

Anyway - If your nodes are parsenodes and you didn't created
the corresponding node readfuncs, I would expect trouble
during the rewrite handling when the printed rule action is
parsed back to the backends internal querytree structures.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#======================================== jwieck@debis.com (Jan Wieck) #

Bruce Momjian wrote:

I added a few nodes (maybe two?), _and_ added some call-outs to existing nodes to
follow their children down. This stuff can be bracketed with debugging #ifdef's; it
was very helpful for me when debugging but it isn't good if they are adding
unnecessary limitations on sizes. The additional nodes I added are a "don't care";
it's the additional printing of child nodes (fields of existing structures) which
is loading things down.

The stuff is in nodes/outfuncs.c, and is used in EXPLAIN VERBOSE. I
question whether your structures would actually be output as part of a
rule.

I hesitate to remove any of the outfuncs stuff. It is very useful, and
if it is missing, things are harder to debug. Adding the fields I did
helped solve several problems I had when testing subselects, and I know
Vadim uses that output too. Shame it goes into the rule, but hard to
imagine why the rule would not need it, except for fields that are only
used by the parser, but I think we need to be complete. A better
solution would be to allow rewrite rules to span multiple blocks, or a
least allow them to take the space of two blocks.

Or use LO.

Vadim

I hesitate to remove any of the outfuncs stuff. It is very useful, and
if it is missing, things are harder to debug. Adding the fields I did
helped solve several problems I had when testing subselects, and I know
Vadim uses that output too. Shame it goes into the rule, but hard to
imagine why the rule would not need it, except for fields that are only
used by the parser, but I think we need to be complete. A better
solution would be to allow rewrite rules to span multiple blocks, or a
least allow them to take the space of two blocks.

Or use LO.

Yea, that makes a lot of sense.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Jan Wieck wrote:

Hi,

Sounds like the plan internally grew (possible w/subselect stuff?!?) and
has gone over that 8k limit.

I think it must be due to the cleanups in the node-print and
read funcs. They now output/read ALL fields in the nodes.

Oh, I didn't realize that the print functions were actually used for something
other than printing and debugging. I had started to add a few new nodes when I
was trying to debug the "primary key" code.

Should we go through and bracket some of those with #ifdef QUERYDEBUG or
something like that? Where are they actually used? Should we try to keep these
at a minimum for production compiles of the system??

Did you add stuff to dump that wasn't dumped before, or were they fields
of existing structure that used to be skipped?

They are used for the rewrite system and for views.

I added a few nodes (maybe two?), _and_ added some call-outs to existing nodes to
follow their children down. This stuff can be bracketed with debugging #ifdef's; it
was very helpful for me when debugging but it isn't good if they are adding
unnecessary limitations on sizes. The additional nodes I added are a "don't care";
it's the additional printing of child nodes (fields of existing structures) which
is loading things down.

I am not absolutely certain that we are talking about the same thing, so should
confirm that it is the same routines which are causing trouble; this is the same
printing/formatting code used for "explain"??

I'd be happy to help clean them up, but I think you should help too; as I recall we
were both mucking around there about the same time ;)

I think we talk about the same group of functions. But I'm
not sure if the nodes you're talking about are parsenodes.
Only parsenodes are printed during rule creation to build the
querytree string stored in the pg_rewrite catalog (views are
implemented by rules and do an internal CREATE RULE on CREATE
VIEW).

Anyway - If your nodes are parsenodes and you didn't created
the corresponding node readfuncs, I would expect trouble
during the rewrite handling when the printed rule action is
parsed back to the backends internal querytree structures.

Well, how to we come to a conclusion here? Here are apparently new calls (since v6.2.1)
in outfuncs.c which could print out something. There are a few, and some may be printed
as children of parsetree nodes. Anyone have an idea if any of these are in parsetree
nodes and could be affecting the view and rewrite storage?

btw, I don't recall doing anything in the readfuncs, so there aren't likely any new
routines there.

- Tom

I don't see any of these affecting the rewrite system. They should
already have been in the code. Were they not?

_outCreateStmt(StringInfo str, CreateStmt *node)
_outNode(str, node->tableElts);
_outNode(str, node->inhRelnames);
_outNode(str, node->constraints);
_outIndexStmt(StringInfo str, IndexStmt *node)
_outNode(str, node->indexParams);
_outNode(str, node->withClause);
_outNode(str, node->whereClause);
_outNode(str, node->rangetable);
_outColumnDef(StringInfo str, ColumnDef *node)
_outNode(str, node->typename);
_outNode(str, node->constraints);
_outTypeName(StringInfo str, TypeName *node)
_outNode(str, node->arrayBounds);
_outIndexElem(StringInfo str, IndexElem *node)
_outNode(str, node->args);
_outNode(str, node->tname);
case T_CreateStmt:
_outNode(str, node->utilityStmt);
case T_IndexStmt:
_outNode(str, node->utilityStmt);
case T_NotifyStmt:
_outNode(str, node->sortClause);
_outNode(str, node->groupClause);
_outNode(str, node->havingQual);
_outNode(str, node->unionClause);
_outSortClause(StringInfo str, SortClause *node)
_outNode(str, node->resdom);
_outGroupClause(StringInfo str, GroupClause *node)
_outNode(str, node->entry);
sprintf(buf, " :size %d ", node->plan_size);
sprintf(buf, " :width %d ", node->plan_width);
_outIntList(str, node->extParam);
_outIntList(str, node->locParam);
_outNode(str, node->initPlan);
_outNode(str, node->unionrts);
_outSubPlan(StringInfo str, SubPlan *node)
_outNode(str, node->plan);
sprintf(buf, " :planid %u ", node->plan_id);
_outNode(str, node->rtable);
_outIntList(str, node->setParam);
_outIntList(str, node->parParam);
_outNode(str, node->sublink);
_outNode(str, node->aggs);
appendStringInfo(str, node->fj_initialized ? "true" : "false");
sprintf(buf, " :nNodes %d ", node->fj_nNodes);
appendStringInfo(str, (node->fj_alwaysDone[i]) ? "true" : "false");
case SUBPLAN_EXPR:
_outNode(str, node->target);
_outSubLink(StringInfo str, SubLink *node)
_outNode(str, node->lefthand);
_outNode(str, node->oper);
_outNode(str, node->subselect);
_outNode(str, node->expr);
_outAExpr(StringInfo str, A_Expr *node)
_outNode(str, node->lexpr);
_outNode(str, node->rexpr);
_outIdent(StringInfo str, Ident *node)
_outAConst(StringInfo str, A_Const *node)
_outValue(str, &(node->val));
case T_CreateStmt:
_outCreateStmt(str, obj);
case T_IndexStmt:
_outIndexStmt(str, obj);
case T_ColumnDef:
_outColumnDef(str, obj);
case T_TypeName:
_outTypeName(str, obj);
case T_IndexElem:
_outIndexElem(str, obj);
case T_SortClause:
_outSortClause(str, obj);
case T_GroupClause:
_outGroupClause(str, obj);
case T_SubPlan:
_outSubPlan(str, obj);
case T_SubLink:
_outSubLink(str, obj);
case T_A_Expr:
_outAExpr(str, obj);
case T_Ident:
_outIdent(str, obj);
case T_A_Const:
_outAConst(str, obj);

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

I don't see any of these affecting the rewrite system. They should
already have been in the code. Were they not?

I recall adding code to print the nodes of a create index statement, and the nodes of a
table create statement. I don't quite remember which other ones, but everything listed
below is new since v6.2.1.

I don't see any of these affecting the rewrite system. They should
already have been in the code. Were they not?

I recall adding code to print the nodes of a create index statement, and the nodes of a
table create statement. I don't quite remember which other ones, but everything listed
below is new since v6.2.1.

I think I added many of these to fixed rule problems.

_outCreateStmt(StringInfo str, CreateStmt *node)
_outNode(str, node->tableElts);
_outNode(str, node->inhRelnames);
_outNode(str, node->constraints);
_outIndexStmt(StringInfo str, IndexStmt *node)
_outNode(str, node->indexParams);
_outNode(str, node->withClause);
_outNode(str, node->whereClause);
_outNode(str, node->rangetable);

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)