FETCH FIRST clause PERCENT option
FETCH FIRST with PERCENT option is SQL standard that state limit count to
be specified in a percentage in addition to specify it in exact count and
listed as one of Major features simply not implemented yet in recent wiki
page [1].https://wiki.postgresql.org/wiki/PostgreSQL_vs_SQL_Standard.
I implemented it by executing the query plan twice. One without limit
filter to find the total number of row that will be feed to limit node so
the exact limit count can be calculated and the query plan execute again
with limit filter with newly calculated exact count .
[1]: .https://wiki.postgresql.org/wiki/PostgreSQL_vs_SQL_Standard
Regards
Surafel
Attachments:
fetch-wth-percent-v1.patchtext/x-patch; charset=US-ASCII; name=fetch-wth-percent-v1.patchDownload
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 4db8142afa..8491b7831a 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1397,7 +1397,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1407,7 +1407,8 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
+ with <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage.<literal>ROW</literal>
and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index bb28cf7d1d..8f1e937906 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -60,6 +60,24 @@ ExecLimit(PlanState *pstate)
{
case LIMIT_INITIAL:
+ if (node->limitOption == PERCENTAGE)
+ {
+
+ /*
+ * Find the total number of row the plan will return.
+ */
+ for (;;)
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ break;
+ }
+ node->total++;
+ }
+ ExecReScan(outerPlan);
+ }
+
/*
* First call for this node, so compute limit/offset. (We can't do
* this any earlier, because parameters from upper nodes will not
@@ -283,11 +301,16 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
+ if (node->limitOption == PERCENTAGE)
+ node->count = (DatumGetInt64(val) * node->total) / 100;
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+ }
node->noCount = false;
}
}
@@ -374,6 +397,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
+ limitstate->total = 0;
/*
* Initialize result slot and type. (XXX not actually used, but upper
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 7c8220cf65..ba3e086e24 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1139,6 +1139,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3024,6 +3025,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3108,6 +3110,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 378f2facb8..ba74fe9ffc 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -973,6 +973,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1047,6 +1048,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 6269f474d2..a3572c73d4 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -986,6 +986,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2197,6 +2198,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2785,6 +2787,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2995,6 +2998,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 3254524223..a10f989042 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -266,6 +266,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2286,6 +2287,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index ae41c9efa0..e9df3ccda0 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2139,7 +2139,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2444,7 +2445,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6446,7 +6448,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6458,6 +6460,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 96bf0601a8..7322dfc90f 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2166,6 +2166,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index c5aaaf5c22..0315e434b6 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3412,6 +3412,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3433,6 +3434,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index c601b6d40d..7c682c5c11 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1301,6 +1301,7 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_OFFSET, "OFFSET");
qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1547,6 +1548,7 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_OFFSET, "OFFSET");
qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1782,6 +1784,7 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_OFFSET, "OFFSET");
qry->limitCount = transformLimitClause(pstate, limitCount,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 87f5e95827..424fcfd418 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -164,6 +164,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -387,7 +388,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
+ reloption_list group_clause TriggerFuncArgs select_limit limit_clause
opt_select_limit opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
@@ -449,7 +450,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -662,7 +663,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11218,7 +11219,7 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
@@ -11226,6 +11227,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
list_nth($4, 0), list_nth($4, 1),
+ (list_nth($4, 2)),
NULL,
yyscanner);
$$ = $1;
@@ -11234,6 +11236,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
list_nth($3, 0), list_nth($3, 1),
+ list_nth($3, 2),
NULL,
yyscanner);
$$ = $1;
@@ -11242,7 +11245,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11250,7 +11253,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11258,6 +11261,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
list_nth($5, 0), list_nth($5, 1),
+ list_nth($5, 2),
$1,
yyscanner);
$$ = $2;
@@ -11266,6 +11270,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
list_nth($4, 0), list_nth($4, 1),
+ list_nth($4, 2),
$1,
yyscanner);
$$ = $2;
@@ -11552,20 +11557,20 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause { $$ = list_make3($2, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause limit_clause { $$ = list_make3($1, list_nth($2, 0), list_nth($2, 1)); }
+ | limit_clause { $$ = list_make3(NULL, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause { $$ = list_make3($1, NULL, NULL); }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */ { $$ = list_make3(NULL, NULL, NULL); }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ { $$ = list_make2($2, NULL); }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11583,9 +11588,11 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ { $$ = list_make2($3, makeString("EXACT_NUMBER")); }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ { $$ = list_make2($3, makeString("PERCENTAGE")); }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ { $$ = list_make2(makeIntConst(1, -1), NULL); }
;
offset_clause:
@@ -15453,6 +15460,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCES
@@ -15836,6 +15844,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15874,6 +15883,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 41fa2052a2..4350da1907 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2219,8 +2219,10 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ int64 total; /* total number of row outer node return */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 697d3d7a5f..3a4ca4b27e 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -811,4 +811,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+} LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 07ab1a3dde..58702f8afe 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1560,6 +1561,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 7c2abbd03a..e2cc85789b 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -952,6 +952,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
index 41caf873fb..cb69a68777 100644
--- a/src/include/nodes/relation.h
+++ b/src/include/nodes/relation.h
@@ -1735,6 +1735,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 7c5ff22650..17b62c7b8c 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -248,6 +248,7 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index c8ab0280d2..4e1e3cebe2 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -64,7 +64,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 23db40147b..150d51df8f 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -300,6 +300,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, RESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 8927b21ba2..442b7622ea 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -79,7 +79,7 @@ CREATE TABLE student (
gpa float8
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "name"
NOTICE: merging multiple inherited definitions of column "age"
diff --git a/src/test/regress/sql/create_table.sql b/src/test/regress/sql/create_table.sql
index 81fa7658b0..81ec6e368d 100644
--- a/src/test/regress/sql/create_table.sql
+++ b/src/test/regress/sql/create_table.sql
@@ -92,7 +92,7 @@ CREATE TABLE student (
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
Hi,
On 2018-08-16 17:27:45 +0300, Surafel Temesgen wrote:
FETCH FIRST with PERCENT option is SQL standard that state limit count to
be specified in a percentage in addition to specify it in exact count and
listed as one of Major features simply not implemented yet in recent wiki
page [1].I implemented it by executing the query plan twice. One without limit
filter to find the total number of row that will be feed to limit node so
the exact limit count can be calculated and the query plan execute again
with limit filter with newly calculated exact count .
Won't that have rather massive issues with multiple evaluations of
clauses in the query? Besides being really expensive?
I think you'd have to instead spill the query results into a tuplestore
(like we do for FOR HOLD queries at end of xact), and then do the
computations based on that.
- Andres
"Andres" == Andres Freund <andres@anarazel.de> writes:
Andres> I think you'd have to instead spill the query results into a
Andres> tuplestore (like we do for FOR HOLD queries at end of xact),
Andres> and then do the computations based on that.
The approach I've suggested to some people who wanted to look into this
was to inject a window function call into the query, and teach the Limit
node to be able to stop on a boolean condition (which would be based on
that function result). This method should also work for WITH TIES by
using a different window function.
--
Andrew (irc:RhodiumToad)
On Thu, Aug 16, 2018 at 5:34 PM, Andres Freund <andres@anarazel.de> wrote:
Won't that have rather massive issues with multiple evaluations of
clauses in the query? Besides being really expensive?
The plan re-scane after first execution I can’t see issue for multiple
execution of a clause in this case
I think you'd have to instead spill the query results into a tuplestore
The downside of it is all the result have to be stored even if needed
tuple is a fraction of it and also store it for longer so the choice became
memory or cpu utilization
- Andres
Regards
Surafel
On 2018-08-21 15:47:07 +0300, Surafel Temesgen wrote:
On Thu, Aug 16, 2018 at 5:34 PM, Andres Freund <andres@anarazel.de> wrote:
Won't that have rather massive issues with multiple evaluations of
clauses in the query? Besides being really expensive?The plan re-scane after first execution I can’t see issue for multiple
execution of a clause in this case
Imagine volatile functions being used. That can be problematic because
of repeated side-effects, but also will lead to plainly wrong
results. Consider what happens with your approach where somebody does
something like WHERE value < random().
Greetings,
Andres Freund
On Tue, Aug 21, 2018 at 3:50 PM Andres Freund <andres@anarazel.de> wrote:
Imagine volatile functions being used. That can be problematic because
of repeated side-effects, but also will lead to plainly wrong
results. Consider what happens with your approach where somebody does
something like WHERE value < random().
Thanks for explaining . The attach patch use tuplestore instead
Regards
Surafel
Attachments:
fetch-wth-percent-v2.patchtext/x-patch; charset=US-ASCII; name=fetch-wth-percent-v2.patchDownload
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 4db8142afa..8491b7831a 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1397,7 +1397,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1407,7 +1407,8 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
+ with <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage.<literal>ROW</literal>
and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index bb28cf7d1d..0dd514fafb 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -52,6 +52,7 @@ ExecLimit(PlanState *pstate)
*/
direction = node->ps.state->es_direction;
outerPlan = outerPlanState(node);
+ slot = node->ps.ps_ResultTupleSlot;
/*
* The main logic is a simple state machine.
@@ -60,6 +61,23 @@ ExecLimit(PlanState *pstate)
{
case LIMIT_INITIAL:
+ if (node->limitOption == PERCENTAGE)
+ {
+
+ /*
+ * Find all rows the plan will return.
+ */
+ for (;;)
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ break;
+ }
+ tuplestore_puttupleslot(node->totalTuple, slot);
+ }
+ }
+
/*
* First call for this node, so compute limit/offset. (We can't do
* this any earlier, because parameters from upper nodes will not
@@ -87,24 +105,45 @@ ExecLimit(PlanState *pstate)
return NULL;
}
- /*
- * Fetch rows from subplan until we reach position > offset.
- */
- for (;;)
+ if (node->limitOption == PERCENTAGE)
{
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
+ for (;;)
{
- /*
- * The subplan returns too few tuples for us to produce
- * any output at all.
- */
- node->lstate = LIMIT_EMPTY;
- return NULL;
+ if (!tuplestore_gettupleslot(node->totalTuple, true, false, slot))
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ else
+ {
+ node->subSlot = slot;
+ if (++node->position > node->offset)
+ break;
+ }
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Fetch rows from subplan until we reach position > offset.
+ */
+ for (;;)
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ /*
+ * The subplan returns too few tuples for us to produce
+ * any output at all.
+ */
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ node->subSlot = slot;
+ if (++node->position > node->offset)
+ break;
}
- node->subSlot = slot;
- if (++node->position > node->offset)
- break;
}
/*
@@ -144,18 +183,34 @@ ExecLimit(PlanState *pstate)
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
- /*
- * Get next tuple from subplan, if any.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- {
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ node->subSlot = slot;
+ node->position++;
}
- node->subSlot = slot;
- node->position++;
}
else
{
@@ -168,15 +223,29 @@ ExecLimit(PlanState *pstate)
node->lstate = LIMIT_WINDOWSTART;
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
- /*
- * Get previous tuple from subplan; there should be one!
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Get previous tuple from subplan; there should be one!
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->position--;
+ }
}
break;
@@ -184,15 +253,29 @@ ExecLimit(PlanState *pstate)
if (ScanDirectionIsForward(direction))
return NULL;
- /*
- * Backing up from subplan EOF, so re-fetch previous tuple; there
- * should be one! Note previous tuple must be in window.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->lstate = LIMIT_INWINDOW;
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Backing up from subplan EOF, so re-fetch previous tuple; there
+ * should be one! Note previous tuple must be in window.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
/* position does not change 'cause we didn't advance it before */
break;
@@ -283,11 +366,16 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
+ if (node->limitOption == PERCENTAGE)
+ node->count = (DatumGetInt64(val) * tuplestore_tuple_count(node->totalTuple)) / 100;
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+ }
node->noCount = false;
}
}
@@ -374,6 +462,10 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
+
+ if (node->limitOption == PERCENTAGE)
+ limitstate->totalTuple= tuplestore_begin_heap(true, false, work_mem);
/*
* Initialize result slot and type. (XXX not actually used, but upper
@@ -402,6 +494,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->totalTuple!= NULL)
+ tuplestore_end(node->totalTuple);
}
@@ -421,4 +515,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->totalTuple!= NULL)
+ tuplestore_rescan(node->totalTuple);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 7c8220cf65..ba3e086e24 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1139,6 +1139,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3024,6 +3025,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3108,6 +3110,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 378f2facb8..ba74fe9ffc 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -973,6 +973,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1047,6 +1048,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 6269f474d2..a3572c73d4 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -986,6 +986,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2197,6 +2198,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2785,6 +2787,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2995,6 +2998,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 3254524223..a10f989042 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -266,6 +266,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2286,6 +2287,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index ae41c9efa0..e9df3ccda0 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2139,7 +2139,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2444,7 +2445,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6446,7 +6448,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6458,6 +6460,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 96bf0601a8..7322dfc90f 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2166,6 +2166,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index c5aaaf5c22..0315e434b6 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3412,6 +3412,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3433,6 +3434,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index c601b6d40d..7c682c5c11 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1301,6 +1301,7 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_OFFSET, "OFFSET");
qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1547,6 +1548,7 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_OFFSET, "OFFSET");
qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1782,6 +1784,7 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_OFFSET, "OFFSET");
qry->limitCount = transformLimitClause(pstate, limitCount,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 4bd2223f26..a97ee30924 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -164,6 +164,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -387,7 +388,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
+ reloption_list group_clause TriggerFuncArgs select_limit limit_clause
opt_select_limit opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
@@ -449,7 +450,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -662,7 +663,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11223,7 +11224,7 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
@@ -11231,6 +11232,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
list_nth($4, 0), list_nth($4, 1),
+ (list_nth($4, 2)),
NULL,
yyscanner);
$$ = $1;
@@ -11239,6 +11241,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
list_nth($3, 0), list_nth($3, 1),
+ list_nth($3, 2),
NULL,
yyscanner);
$$ = $1;
@@ -11247,7 +11250,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11255,7 +11258,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11263,6 +11266,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
list_nth($5, 0), list_nth($5, 1),
+ list_nth($5, 2),
$1,
yyscanner);
$$ = $2;
@@ -11271,6 +11275,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
list_nth($4, 0), list_nth($4, 1),
+ list_nth($4, 2),
$1,
yyscanner);
$$ = $2;
@@ -11557,20 +11562,20 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause { $$ = list_make3($2, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause limit_clause { $$ = list_make3($1, list_nth($2, 0), list_nth($2, 1)); }
+ | limit_clause { $$ = list_make3(NULL, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause { $$ = list_make3($1, NULL, NULL); }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */ { $$ = list_make3(NULL, NULL, NULL); }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ { $$ = list_make2($2, NULL); }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11588,9 +11593,11 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ { $$ = list_make2($3, makeString("EXACT_NUMBER")); }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ { $$ = list_make2($3, makeString("PERCENTAGE")); }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ { $$ = list_make2(makeIntConst(1, -1), NULL); }
;
offset_clause:
@@ -15458,6 +15465,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCES
@@ -15841,6 +15849,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15879,6 +15888,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 41fa2052a2..dca6466fcb 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2219,8 +2219,10 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ Tuplestorestate *totalTuple; /* total number of row outer node return */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 697d3d7a5f..3a4ca4b27e 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -811,4 +811,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+} LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 07ab1a3dde..58702f8afe 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1560,6 +1561,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 7c2abbd03a..e2cc85789b 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -952,6 +952,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
index 41caf873fb..cb69a68777 100644
--- a/src/include/nodes/relation.h
+++ b/src/include/nodes/relation.h
@@ -1735,6 +1735,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 7c5ff22650..17b62c7b8c 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -248,6 +248,7 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index c8ab0280d2..4e1e3cebe2 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -64,7 +64,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 23db40147b..150d51df8f 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -300,6 +300,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, RESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 8927b21ba2..442b7622ea 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -79,7 +79,7 @@ CREATE TABLE student (
gpa float8
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "name"
NOTICE: merging multiple inherited definitions of column "age"
diff --git a/src/test/regress/sql/create_table.sql b/src/test/regress/sql/create_table.sql
index 81fa7658b0..81ec6e368d 100644
--- a/src/test/regress/sql/create_table.sql
+++ b/src/test/regress/sql/create_table.sql
@@ -92,7 +92,7 @@ CREATE TABLE student (
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
On 2018-08-28 14:14, Surafel Temesgen wrote:
On Tue, Aug 21, 2018 at 3:50 PM Andres Freund <andres@anarazel.de>
wrote:Imagine volatile functions being used. That can be problematic because
of repeated side-effects, but also will lead to plainly wrong
results. Consider what happens with your approach where somebody does
something like WHERE value < random().Thanks for explaining . The attach patch use tuplestore instead
[fetch-wth-percent-v2.patch]
I had a quick look at this. Apply, compile, make check are ok.
In straightforward usage seems to work ok.
But I also ran across this crash:
create table if not exists t as
select i from generate_series(1, 100) as f(i);
select * from t
offset 60 rows
fetch next 3 percent rows only
FOR UPDATE
;
TRAP: FailedAssertion("!(slot != ((void *)0))", File: "execTuples.c",
Line: 428)
Erik Rijkers
On Tue, Aug 28, 2018 at 7:33 PM Erik Rijkers <er@xs4all.nl> wrote:
;
TRAP: FailedAssertion("!(slot != ((void *)0))", File: "execTuples.c",
Line: 42
The attache patch include a fix for the crash .can you check it again?
Regards
Surafel
Attachments:
fetch-wth-percent-v3.patchtext/x-patch; charset=US-ASCII; name=fetch-wth-percent-v3.patchDownload
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 4db8142afa..8491b7831a 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1397,7 +1397,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1407,7 +1407,8 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
+ with <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage.<literal>ROW</literal>
and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index bb28cf7d1d..e842e17542 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -43,6 +43,7 @@ ExecLimit(PlanState *pstate)
LimitState *node = castNode(LimitState, pstate);
ScanDirection direction;
TupleTableSlot *slot;
+ TupleDesc tupleDescriptor;
PlanState *outerPlan;
CHECK_FOR_INTERRUPTS();
@@ -52,6 +53,8 @@ ExecLimit(PlanState *pstate)
*/
direction = node->ps.state->es_direction;
outerPlan = outerPlanState(node);
+ slot = node->ps.ps_ResultTupleSlot;
+ tupleDescriptor = node->ps.ps_ResultTupleSlot->tts_tupleDescriptor;
/*
* The main logic is a simple state machine.
@@ -60,6 +63,23 @@ ExecLimit(PlanState *pstate)
{
case LIMIT_INITIAL:
+ if (node->limitOption == PERCENTAGE)
+ {
+
+ /*
+ * Find all rows the plan will return.
+ */
+ for (;;)
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ break;
+ }
+ tuplestore_puttupleslot(node->totalTuple, slot);
+ }
+ }
+
/*
* First call for this node, so compute limit/offset. (We can't do
* this any earlier, because parameters from upper nodes will not
@@ -87,24 +107,46 @@ ExecLimit(PlanState *pstate)
return NULL;
}
- /*
- * Fetch rows from subplan until we reach position > offset.
- */
- for (;;)
+ if (node->limitOption == PERCENTAGE)
{
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
+ for (;;)
{
- /*
- * The subplan returns too few tuples for us to produce
- * any output at all.
- */
- node->lstate = LIMIT_EMPTY;
- return NULL;
+ slot = MakeSingleTupleTableSlot(tupleDescriptor);
+ if (!tuplestore_gettupleslot(node->totalTuple, true, true, slot))
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ else
+ {
+ node->subSlot = slot;
+ if (++node->position > node->offset)
+ break;
+ }
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Fetch rows from subplan until we reach position > offset.
+ */
+ for (;;)
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ /*
+ * The subplan returns too few tuples for us to produce
+ * any output at all.
+ */
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ node->subSlot = slot;
+ if (++node->position > node->offset)
+ break;
}
- node->subSlot = slot;
- if (++node->position > node->offset)
- break;
}
/*
@@ -144,18 +186,34 @@ ExecLimit(PlanState *pstate)
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
- /*
- * Get next tuple from subplan, if any.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- {
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ node->subSlot = slot;
+ node->position++;
}
- node->subSlot = slot;
- node->position++;
}
else
{
@@ -168,15 +226,29 @@ ExecLimit(PlanState *pstate)
node->lstate = LIMIT_WINDOWSTART;
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
- /*
- * Get previous tuple from subplan; there should be one!
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Get previous tuple from subplan; there should be one!
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->position--;
+ }
}
break;
@@ -184,15 +256,29 @@ ExecLimit(PlanState *pstate)
if (ScanDirectionIsForward(direction))
return NULL;
- /*
- * Backing up from subplan EOF, so re-fetch previous tuple; there
- * should be one! Note previous tuple must be in window.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->lstate = LIMIT_INWINDOW;
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Backing up from subplan EOF, so re-fetch previous tuple; there
+ * should be one! Note previous tuple must be in window.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
/* position does not change 'cause we didn't advance it before */
break;
@@ -283,11 +369,16 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
+ if (node->limitOption == PERCENTAGE)
+ node->count = (DatumGetInt64(val) * tuplestore_tuple_count(node->totalTuple)) / 100;
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+ }
node->noCount = false;
}
}
@@ -374,6 +465,10 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
+
+ if (node->limitOption == PERCENTAGE)
+ limitstate->totalTuple= tuplestore_begin_heap(true, false, work_mem);
/*
* Initialize result slot and type. (XXX not actually used, but upper
@@ -402,6 +497,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->totalTuple!= NULL)
+ tuplestore_end(node->totalTuple);
}
@@ -421,4 +518,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->totalTuple!= NULL)
+ tuplestore_rescan(node->totalTuple);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 7c8220cf65..ba3e086e24 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1139,6 +1139,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3024,6 +3025,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3108,6 +3110,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 378f2facb8..ba74fe9ffc 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -973,6 +973,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1047,6 +1048,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 6269f474d2..a3572c73d4 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -986,6 +986,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2197,6 +2198,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2785,6 +2787,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2995,6 +2998,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 3254524223..a10f989042 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -266,6 +266,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2286,6 +2287,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index ae41c9efa0..e9df3ccda0 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2139,7 +2139,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2444,7 +2445,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6446,7 +6448,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6458,6 +6460,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 96bf0601a8..7322dfc90f 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2166,6 +2166,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index c5aaaf5c22..0315e434b6 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3412,6 +3412,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3433,6 +3434,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index c601b6d40d..7c682c5c11 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1301,6 +1301,7 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_OFFSET, "OFFSET");
qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1547,6 +1548,7 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_OFFSET, "OFFSET");
qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1782,6 +1784,7 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_OFFSET, "OFFSET");
qry->limitCount = transformLimitClause(pstate, limitCount,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 4bd2223f26..a97ee30924 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -164,6 +164,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -387,7 +388,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
+ reloption_list group_clause TriggerFuncArgs select_limit limit_clause
opt_select_limit opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
@@ -449,7 +450,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -662,7 +663,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11223,7 +11224,7 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
@@ -11231,6 +11232,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
list_nth($4, 0), list_nth($4, 1),
+ (list_nth($4, 2)),
NULL,
yyscanner);
$$ = $1;
@@ -11239,6 +11241,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
list_nth($3, 0), list_nth($3, 1),
+ list_nth($3, 2),
NULL,
yyscanner);
$$ = $1;
@@ -11247,7 +11250,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11255,7 +11258,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11263,6 +11266,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
list_nth($5, 0), list_nth($5, 1),
+ list_nth($5, 2),
$1,
yyscanner);
$$ = $2;
@@ -11271,6 +11275,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
list_nth($4, 0), list_nth($4, 1),
+ list_nth($4, 2),
$1,
yyscanner);
$$ = $2;
@@ -11557,20 +11562,20 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause { $$ = list_make3($2, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause limit_clause { $$ = list_make3($1, list_nth($2, 0), list_nth($2, 1)); }
+ | limit_clause { $$ = list_make3(NULL, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause { $$ = list_make3($1, NULL, NULL); }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */ { $$ = list_make3(NULL, NULL, NULL); }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ { $$ = list_make2($2, NULL); }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11588,9 +11593,11 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ { $$ = list_make2($3, makeString("EXACT_NUMBER")); }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ { $$ = list_make2($3, makeString("PERCENTAGE")); }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ { $$ = list_make2(makeIntConst(1, -1), NULL); }
;
offset_clause:
@@ -15458,6 +15465,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCES
@@ -15841,6 +15849,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15879,6 +15888,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 41fa2052a2..dca6466fcb 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2219,8 +2219,10 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ Tuplestorestate *totalTuple; /* total number of row outer node return */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 697d3d7a5f..3a4ca4b27e 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -811,4 +811,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+} LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 07ab1a3dde..58702f8afe 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1560,6 +1561,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 7c2abbd03a..e2cc85789b 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -952,6 +952,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
index 41caf873fb..cb69a68777 100644
--- a/src/include/nodes/relation.h
+++ b/src/include/nodes/relation.h
@@ -1735,6 +1735,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 7c5ff22650..17b62c7b8c 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -248,6 +248,7 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index c8ab0280d2..4e1e3cebe2 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -64,7 +64,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 23db40147b..150d51df8f 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -300,6 +300,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, RESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 8927b21ba2..442b7622ea 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -79,7 +79,7 @@ CREATE TABLE student (
gpa float8
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "name"
NOTICE: merging multiple inherited definitions of column "age"
diff --git a/src/test/regress/sql/create_table.sql b/src/test/regress/sql/create_table.sql
index 81fa7658b0..81ec6e368d 100644
--- a/src/test/regress/sql/create_table.sql
+++ b/src/test/regress/sql/create_table.sql
@@ -92,7 +92,7 @@ CREATE TABLE student (
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
On Fri, Aug 31, 2018 at 11:42 PM Surafel Temesgen <surafel3000@gmail.com> wrote:
On Tue, Aug 28, 2018 at 7:33 PM Erik Rijkers <er@xs4all.nl> wrote:
;
TRAP: FailedAssertion("!(slot != ((void *)0))", File: "execTuples.c",
Line: 42The attache patch include a fix for the crash .can you check it again?
FYI this fails[1]https://travis-ci.org/postgresql-cfbot/postgresql/builds/430777123 in src/test/modules/test_ddl_deparse's create_table
test because of the keyword:
CREATE TABLE stud_emp (
percent int4
) INHERITS (emp, student);
! ERROR: syntax error at or near "percent"
[1]: https://travis-ci.org/postgresql-cfbot/postgresql/builds/430777123
--
Thomas Munro
http://www.enterprisedb.com
On Aug 16, 2018, at 7:34 AM, Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2018-08-16 17:27:45 +0300, Surafel Temesgen wrote:
FETCH FIRST with PERCENT option is SQL standard that state limit count to
be specified in a percentage in addition to specify it in exact count and
listed as one of Major features simply not implemented yet in recent wiki
page [1].I implemented it by executing the query plan twice. One without limit
filter to find the total number of row that will be feed to limit node so
the exact limit count can be calculated and the query plan execute again
with limit filter with newly calculated exact count .
Surafel, there are no regression tests that I can see in your patch. It
would help if you added some, as then I could precisely what behavior you
are expecting. As it is, I'm just guessing here, but here goes....
Won't that have rather massive issues with multiple evaluations of
clauses in the query? Besides being really expensive?I think you'd have to instead spill the query results into a tuplestore
(like we do for FOR HOLD queries at end of xact), and then do the
computations based on that.
I should think that spilling anything to a tuplestore would only be needed
if the query contains an ORDER BY expression. If you query
FETCH FIRST 50 PERCENT * FROM foo;
you should just return every other row, discarding the rest, right? It's
only when an explicit ordering is given that the need to store the results
arises. Even with
FETCH FIRST 50 PERCENT name FROM foo ORDER BY name;
you can return one row for every two rows that you get back from the
sort node, reducing the maximum number you need to store at any time to
no more than 25% of all rows.
Or am I missing something?
mark
Hi,
On 2018-09-20 17:06:36 -0700, Mark Dilger wrote:
I should think that spilling anything to a tuplestore would only be needed
if the query contains an ORDER BY expression. If you queryFETCH FIRST 50 PERCENT * FROM foo;
you should just return every other row, discarding the rest, right? It's
only when an explicit ordering is given that the need to store the results
arises. Even withFETCH FIRST 50 PERCENT name FROM foo ORDER BY name;
you can return one row for every two rows that you get back from the
sort node, reducing the maximum number you need to store at any time to
no more than 25% of all rows.
I'm doubtful about the validity of these optimizations, particularly
around being surprising. But I think more importantly, we should focus
on the basic implementation that's needed anyway.
Greetings,
Andres Freund
On Sep 20, 2018, at 5:29 PM, Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2018-09-20 17:06:36 -0700, Mark Dilger wrote:
I should think that spilling anything to a tuplestore would only be needed
if the query contains an ORDER BY expression. If you queryFETCH FIRST 50 PERCENT * FROM foo;
you should just return every other row, discarding the rest, right? It's
only when an explicit ordering is given that the need to store the results
arises. Even withFETCH FIRST 50 PERCENT name FROM foo ORDER BY name;
you can return one row for every two rows that you get back from the
sort node, reducing the maximum number you need to store at any time to
no more than 25% of all rows.I'm doubtful about the validity of these optimizations, particularly
around being surprising. But I think more importantly, we should focus
on the basic implementation that's needed anyway.
You may be right that getting the basic implementation finished first
is better than optimizing at this stage. So the rest of what I'm going
to say is just in defense of the optimization, and not an argument for
needing to optimize right away.
As for reducing the surprise factor, I think that it would be surprising
if I ask for a smallish percentage of rows and it takes significantly longer
and significantly more memory or disk than asking for all the rows takes.
If I'm including an explicit ORDER BY, then that explains it, but otherwise,
I'd be surprised. Note that I'm not saying I'd be surprised by it taking
roughly the same length of time / memory / disk. I'd only be surprised if
it took a lot more.
There are plenty of SQL generation engines that people put in their software.
I'd expect something like
sprintf("FETCH FIRST %d PERCENT %s FROM %s", percentage, columns, tablename)
to show up in such engines, and percentage to sometimes be 100. At least
in that case you should just return all rows rather than dumping them into
a tuplestore. Likewise, if the percentage is 0, you'll want to finish quickly.
Actually, I don't know if the SQL spec would require side effects to still
happen, in which case you'd still have to generate all rows for their side
effects to happen, and then just not return them. But still, no tuplestore.
So the implementation of FETCH FIRST would at least need to think about what
percentage is being requested, rather than just mindlessly adding a node to
the tree for storing everything, then computing the LIMIT based on the number
of rows stored, and then returning that number of rows.
mark
hey
On 9/21/18, Mark Dilger <hornschnorter@gmail.com> wrote:
Surafel, there are no regression tests that I can see in your patch. It
would help if you added some, as then I could precisely what behavior you
are expecting.
thank you for looking at it .the attach patch add regression tests
Attachments:
fetch-wth-percent-v4.patchtext/x-patch; charset=US-ASCII; name=fetch-wth-percent-v4.patchDownload
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 4db8142afa..8491b7831a 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1397,7 +1397,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1407,7 +1407,8 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
+ with <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage.<literal>ROW</literal>
and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index bb28cf7d1d..1d5b48bbe9 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -43,6 +43,7 @@ ExecLimit(PlanState *pstate)
LimitState *node = castNode(LimitState, pstate);
ScanDirection direction;
TupleTableSlot *slot;
+ TupleDesc tupleDescriptor;
PlanState *outerPlan;
CHECK_FOR_INTERRUPTS();
@@ -52,6 +53,8 @@ ExecLimit(PlanState *pstate)
*/
direction = node->ps.state->es_direction;
outerPlan = outerPlanState(node);
+ slot = node->ps.ps_ResultTupleSlot;
+ tupleDescriptor = node->ps.ps_ResultTupleSlot->tts_tupleDescriptor;
/*
* The main logic is a simple state machine.
@@ -60,6 +63,23 @@ ExecLimit(PlanState *pstate)
{
case LIMIT_INITIAL:
+ if (node->limitOption == PERCENTAGE)
+ {
+
+ /*
+ * Find all rows the plan will return.
+ */
+ for (;;)
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ break;
+ }
+ tuplestore_puttupleslot(node->totalTuple, slot);
+ }
+ }
+
/*
* First call for this node, so compute limit/offset. (We can't do
* this any earlier, because parameters from upper nodes will not
@@ -87,24 +107,46 @@ ExecLimit(PlanState *pstate)
return NULL;
}
- /*
- * Fetch rows from subplan until we reach position > offset.
- */
- for (;;)
+ if (node->limitOption == PERCENTAGE)
{
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
+ for (;;)
{
- /*
- * The subplan returns too few tuples for us to produce
- * any output at all.
- */
- node->lstate = LIMIT_EMPTY;
- return NULL;
+ slot = MakeSingleTupleTableSlot(tupleDescriptor);
+ if (!tuplestore_gettupleslot(node->totalTuple, true, true, slot))
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ else
+ {
+ node->subSlot = slot;
+ if (++node->position > node->offset)
+ break;
+ }
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Fetch rows from subplan until we reach position > offset.
+ */
+ for (;;)
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ /*
+ * The subplan returns too few tuples for us to produce
+ * any output at all.
+ */
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ node->subSlot = slot;
+ if (++node->position > node->offset)
+ break;
}
- node->subSlot = slot;
- if (++node->position > node->offset)
- break;
}
/*
@@ -144,18 +186,34 @@ ExecLimit(PlanState *pstate)
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
- /*
- * Get next tuple from subplan, if any.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- {
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ node->subSlot = slot;
+ node->position++;
}
- node->subSlot = slot;
- node->position++;
}
else
{
@@ -168,15 +226,29 @@ ExecLimit(PlanState *pstate)
node->lstate = LIMIT_WINDOWSTART;
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, false, false, slot))
+ {
+ node->subSlot = slot;
+ node->position--;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
- /*
- * Get previous tuple from subplan; there should be one!
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Get previous tuple from subplan; there should be one!
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->position--;
+ }
}
break;
@@ -184,15 +256,29 @@ ExecLimit(PlanState *pstate)
if (ScanDirectionIsForward(direction))
return NULL;
- /*
- * Backing up from subplan EOF, so re-fetch previous tuple; there
- * should be one! Note previous tuple must be in window.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->lstate = LIMIT_INWINDOW;
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Backing up from subplan EOF, so re-fetch previous tuple; there
+ * should be one! Note previous tuple must be in window.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
/* position does not change 'cause we didn't advance it before */
break;
@@ -283,11 +369,16 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
+ if (node->limitOption == PERCENTAGE)
+ node->count = DatumGetInt64((val * tuplestore_tuple_count(node->totalTuple)) / 100);
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+ }
node->noCount = false;
}
}
@@ -374,6 +465,10 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
+
+ if (node->limitOption == PERCENTAGE)
+ limitstate->totalTuple= tuplestore_begin_heap(true, false, work_mem);
/*
* Initialize result slot and type. (XXX not actually used, but upper
@@ -402,6 +497,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->totalTuple!= NULL)
+ tuplestore_end(node->totalTuple);
}
@@ -421,4 +518,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->totalTuple!= NULL)
+ tuplestore_rescan(node->totalTuple);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 7c8220cf65..ba3e086e24 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1139,6 +1139,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3024,6 +3025,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3108,6 +3110,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 378f2facb8..ba74fe9ffc 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -973,6 +973,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1047,6 +1048,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 6269f474d2..a3572c73d4 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -986,6 +986,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2197,6 +2198,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2785,6 +2787,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2995,6 +2998,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 3254524223..a10f989042 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -266,6 +266,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2286,6 +2287,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index ae41c9efa0..e9df3ccda0 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2139,7 +2139,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2444,7 +2445,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6446,7 +6448,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6458,6 +6460,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 96bf0601a8..7322dfc90f 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2166,6 +2166,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index c5aaaf5c22..0315e434b6 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3412,6 +3412,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3433,6 +3434,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index c601b6d40d..7c682c5c11 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1301,6 +1301,7 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_OFFSET, "OFFSET");
qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1547,6 +1548,7 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_OFFSET, "OFFSET");
qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1782,6 +1784,7 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_OFFSET, "OFFSET");
qry->limitCount = transformLimitClause(pstate, limitCount,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 4bd2223f26..a97ee30924 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -164,6 +164,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -387,7 +388,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
+ reloption_list group_clause TriggerFuncArgs select_limit limit_clause
opt_select_limit opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
@@ -449,7 +450,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -662,7 +663,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11223,7 +11224,7 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
@@ -11231,6 +11232,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
list_nth($4, 0), list_nth($4, 1),
+ (list_nth($4, 2)),
NULL,
yyscanner);
$$ = $1;
@@ -11239,6 +11241,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
list_nth($3, 0), list_nth($3, 1),
+ list_nth($3, 2),
NULL,
yyscanner);
$$ = $1;
@@ -11247,7 +11250,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11255,7 +11258,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11263,6 +11266,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
list_nth($5, 0), list_nth($5, 1),
+ list_nth($5, 2),
$1,
yyscanner);
$$ = $2;
@@ -11271,6 +11275,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
list_nth($4, 0), list_nth($4, 1),
+ list_nth($4, 2),
$1,
yyscanner);
$$ = $2;
@@ -11557,20 +11562,20 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause { $$ = list_make3($2, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause limit_clause { $$ = list_make3($1, list_nth($2, 0), list_nth($2, 1)); }
+ | limit_clause { $$ = list_make3(NULL, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause { $$ = list_make3($1, NULL, NULL); }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */ { $$ = list_make3(NULL, NULL, NULL); }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ { $$ = list_make2($2, NULL); }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11588,9 +11593,11 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ { $$ = list_make2($3, makeString("EXACT_NUMBER")); }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ { $$ = list_make2($3, makeString("PERCENTAGE")); }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ { $$ = list_make2(makeIntConst(1, -1), NULL); }
;
offset_clause:
@@ -15458,6 +15465,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCES
@@ -15841,6 +15849,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15879,6 +15888,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 41fa2052a2..dca6466fcb 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2219,8 +2219,10 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ Tuplestorestate *totalTuple; /* total number of row outer node return */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 697d3d7a5f..3a4ca4b27e 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -811,4 +811,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+} LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 07ab1a3dde..58702f8afe 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1560,6 +1561,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 7c2abbd03a..e2cc85789b 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -952,6 +952,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
index 41caf873fb..cb69a68777 100644
--- a/src/include/nodes/relation.h
+++ b/src/include/nodes/relation.h
@@ -1735,6 +1735,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 7c5ff22650..17b62c7b8c 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -248,6 +248,7 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index c8ab0280d2..4e1e3cebe2 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -64,7 +64,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 23db40147b..150d51df8f 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -300,6 +300,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, RESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/test/modules/test_ddl_deparse/sql/create_table.sql b/src/test/modules/test_ddl_deparse/sql/create_table.sql
index 5e78452729..4325de2d04 100644
--- a/src/test/modules/test_ddl_deparse/sql/create_table.sql
+++ b/src/test/modules/test_ddl_deparse/sql/create_table.sql
@@ -94,7 +94,7 @@ CREATE TABLE student (
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 8927b21ba2..442b7622ea 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -79,7 +79,7 @@ CREATE TABLE student (
gpa float8
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "name"
NOTICE: merging multiple inherited definitions of column "age"
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index c18f547cbd..fcadc92f5d 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -108,6 +108,61 @@ SELECT ''::text AS five, unique1, unique2, stringu1
| 904 | 793 | UIAAAA
(5 rows)
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 51 | 76 | ZBAAAA
+ | 52 | 985 | ACAAAA
+ | 53 | 196 | BCAAAA
+ | 54 | 356 | CCAAAA
+ | 55 | 627 | DCAAAA
+ | 56 | 54 | ECAAAA
+ | 57 | 942 | FCAAAA
+ | 58 | 114 | GCAAAA
+ | 59 | 593 | HCAAAA
+(9 rows)
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 61 | 560 | JCAAAA
+(1 row)
+
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+ three | unique1 | unique2 | stringu1
+-------+---------+---------+----------
+ | 121 | 700 | REAAAA
+ | 122 | 519 | SEAAAA
+ | 123 | 777 | TEAAAA
+ | 124 | 503 | UEAAAA
+ | 125 | 849 | VEAAAA
+ | 126 | 330 | WEAAAA
+ | 127 | 511 | XEAAAA
+ | 128 | 721 | YEAAAA
+(8 rows)
+
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+ eleven | unique1 | unique2 | stringu1
+--------+---------+---------+----------
+ | 10 | 520 | KAAAAA
+ | 9 | 49 | JAAAAA
+ | 8 | 653 | IAAAAA
+ | 7 | 647 | HAAAAA
+ | 6 | 978 | GAAAAA
+(5 rows)
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -286,6 +341,43 @@ fetch all in c4;
----+----
(0 rows)
+declare c6 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c6;
+ q1 | q2
+-----+------------------
+ 123 | 456
+ 123 | 4567890123456789
+(2 rows)
+
+fetch 1 in c6;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c6;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+(1 row)
+
+fetch backward all in c6;
+ q1 | q2
+-----+-----
+ 123 | 456
+(1 row)
+
+fetch backward 1 in c6;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c6;
+ q1 | q2
+-----+------------------
+ 123 | 456
+ 123 | 4567890123456789
+(2 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
@@ -503,3 +595,19 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
45020 | 45020
(3 rows)
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
+ s1 | s2
+-------+-------
+ 45000 | 45000
+ 45010 | 45010
+ 45020 | 45020
+ 45030 | 45030
+ 45040 | 45040
+ 45050 | 45050
+ 45060 | 45060
+ 45070 | 45070
+ 45080 | 45080
+ 45090 | 45090
+(10 rows)
+
diff --git a/src/test/regress/sql/create_table.sql b/src/test/regress/sql/create_table.sql
index 81fa7658b0..81ec6e368d 100644
--- a/src/test/regress/sql/create_table.sql
+++ b/src/test/regress/sql/create_table.sql
@@ -92,7 +92,7 @@ CREATE TABLE student (
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index 2a313d80ca..6609f8eb21 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -30,6 +30,24 @@ SELECT ''::text AS five, unique1, unique2, stringu1
SELECT ''::text AS five, unique1, unique2, stringu1
FROM onek
ORDER BY unique1 LIMIT 5 OFFSET 900;
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
@@ -38,7 +56,6 @@ select * from int8_tbl offset (case when random() < 0.5 then null::bigint end);
-- Test assorted cases involving backwards fetch from a LIMIT plan node
begin;
-
declare c1 cursor for select * from int8_tbl limit 10;
fetch all in c1;
fetch 1 in c1;
@@ -71,6 +88,15 @@ fetch backward all in c4;
fetch backward 1 in c4;
fetch all in c4;
+
+declare c6 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c6;
+fetch 1 in c6;
+fetch backward 1 in c6;
+fetch backward all in c6;
+fetch backward 1 in c6;
+fetch all in c6;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
@@ -141,3 +167,6 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
from tenk1 group by thousand order by thousand limit 3;
+
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
On Sep 25, 2018, at 5:07 AM, Surafel Temesgen <surafel3000@gmail.com> wrote:
hey
On 9/21/18, Mark Dilger <hornschnorter@gmail.com> wrote:
Surafel, there are no regression tests that I can see in your patch. It
would help if you added some, as then I could precisely what behavior you
are expecting.thank you for looking at it .the attach patch add regression tests
<fetch-wth-percent-v4.patch>
Surafel,
I messed around with your changes to the grammar and it seems you don't
need to add PERCENT as a reserved keyword. Moving this to the unreserved
keyword section does not cause any shift/reduce errors, and the regression
tests still pass. Relative to your patch v4, these changes help:
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index a97ee30924..9872cf7257 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -15186,6 +15186,7 @@ unreserved_keyword:
| PARTITION
| PASSING
| PASSWORD
+ | PERCENT
| PLANS
| POLICY
| PRECEDING
@@ -15465,7 +15466,6 @@ reserved_keyword:
| ONLY
| OR
| ORDER
- | PERCENT
| PLACING
| PRIMARY
| REFERENCES
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 150d51df8f..f23fee0653 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -300,7 +300,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
-PG_KEYWORD("percent", PERCENT, RESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, UNRESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
On Sep 25, 2018, at 8:08 AM, Mark Dilger <hornschnorter@gmail.com> wrote:
On Sep 25, 2018, at 5:07 AM, Surafel Temesgen <surafel3000@gmail.com> wrote:
hey
On 9/21/18, Mark Dilger <hornschnorter@gmail.com> wrote:
Surafel, there are no regression tests that I can see in your patch. It
would help if you added some, as then I could precisely what behavior you
are expecting.thank you for looking at it .the attach patch add regression tests
<fetch-wth-percent-v4.patch>Surafel,
I messed around with your changes to the grammar and it seems you don't
need to add PERCENT as a reserved keyword. Moving this to the unreserved
keyword section does not cause any shift/reduce errors, and the regression
tests still pass. Relative to your patch v4, these changes help:
I spoke too soon. The main regression tests pass, but your change to
src/test/modules/test_ddl_deparse/sql/create_table.sql per Thomas's
suggestion is no longer needed, since PERCENT no longer needs to be
quoted.
I recommend you also apply the following to your v4 patch, which just
rolls back that one change you made, and at least for me, is enough
to get `make check-world` to pass:
diff --git a/src/test/modules/test_ddl_deparse/sql/create_table.sql b/src/test/modules/test_ddl_deparse/sql/create_table.sql
index 4325de2d04..5e78452729 100644
--- a/src/test/modules/test_ddl_deparse/sql/create_table.sql
+++ b/src/test/modules/test_ddl_deparse/sql/create_table.sql
@@ -94,7 +94,7 @@ CREATE TABLE student (
) INHERITS (person);
CREATE TABLE stud_emp (
- "percent" int4
+ percent int4
) INHERITS (emp, student);
I messed around with your changes to the grammar and it seems you don't
need to add PERCENT as a reserved keyword. Moving this to the unreserved
keyword section does not cause any shift/reduce errors, and the regression
tests still pass. Relative to your patch v4, these changes help:
In sql standard PERCENT list as reserved world so i don't think it is a
thing that can change by me.
I think the main reason create_table fail in test_ddl_deparse is because
i don't surround PERCENT key word in
/src/test/regress/expected/create_table.out too
regards
Surafel
Attachments:
fetch-wth-percent-v5.patchtext/x-patch; charset=US-ASCII; name=fetch-wth-percent-v5.patchDownload
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 4db8142afa..8491b7831a 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1397,7 +1397,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1407,7 +1407,8 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
+ with <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage.<literal>ROW</literal>
and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 6792f9e86c..762024be61 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -43,6 +43,7 @@ ExecLimit(PlanState *pstate)
LimitState *node = castNode(LimitState, pstate);
ScanDirection direction;
TupleTableSlot *slot;
+ TupleDesc tupleDescriptor;
PlanState *outerPlan;
CHECK_FOR_INTERRUPTS();
@@ -52,6 +53,8 @@ ExecLimit(PlanState *pstate)
*/
direction = node->ps.state->es_direction;
outerPlan = outerPlanState(node);
+ slot = node->subSlot;
+ tupleDescriptor = node->ps.ps_ResultTupleDesc;
/*
* The main logic is a simple state machine.
@@ -60,6 +63,23 @@ ExecLimit(PlanState *pstate)
{
case LIMIT_INITIAL:
+ if (node->limitOption == PERCENTAGE)
+ {
+
+ /*
+ * Find all rows the plan will return.
+ */
+ for (;;)
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ break;
+ }
+ tuplestore_puttupleslot(node->totalTuple, slot);
+ }
+ }
+
/*
* First call for this node, so compute limit/offset. (We can't do
* this any earlier, because parameters from upper nodes will not
@@ -87,24 +107,46 @@ ExecLimit(PlanState *pstate)
return NULL;
}
- /*
- * Fetch rows from subplan until we reach position > offset.
- */
- for (;;)
+ if (node->limitOption == PERCENTAGE)
{
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
+ for (;;)
{
- /*
- * The subplan returns too few tuples for us to produce
- * any output at all.
- */
- node->lstate = LIMIT_EMPTY;
- return NULL;
+ slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if (!tuplestore_gettupleslot(node->totalTuple, true, true, slot))
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ else
+ {
+ node->subSlot = slot;
+ if (++node->position > node->offset)
+ break;
+ }
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Fetch rows from subplan until we reach position > offset.
+ */
+ for (;;)
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ /*
+ * The subplan returns too few tuples for us to produce
+ * any output at all.
+ */
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ node->subSlot = slot;
+ if (++node->position > node->offset)
+ break;
}
- node->subSlot = slot;
- if (++node->position > node->offset)
- break;
}
/*
@@ -144,18 +186,34 @@ ExecLimit(PlanState *pstate)
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
- /*
- * Get next tuple from subplan, if any.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- {
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ node->subSlot = slot;
+ node->position++;
}
- node->subSlot = slot;
- node->position++;
}
else
{
@@ -168,15 +226,29 @@ ExecLimit(PlanState *pstate)
node->lstate = LIMIT_WINDOWSTART;
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, false, false, slot))
+ {
+ node->subSlot = slot;
+ node->position--;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
- /*
- * Get previous tuple from subplan; there should be one!
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Get previous tuple from subplan; there should be one!
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->position--;
+ }
}
break;
@@ -184,15 +256,29 @@ ExecLimit(PlanState *pstate)
if (ScanDirectionIsForward(direction))
return NULL;
- /*
- * Backing up from subplan EOF, so re-fetch previous tuple; there
- * should be one! Note previous tuple must be in window.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->lstate = LIMIT_INWINDOW;
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Backing up from subplan EOF, so re-fetch previous tuple; there
+ * should be one! Note previous tuple must be in window.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
/* position does not change 'cause we didn't advance it before */
break;
@@ -283,11 +369,16 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
+ if (node->limitOption == PERCENTAGE)
+ node->count = DatumGetInt64((val * tuplestore_tuple_count(node->totalTuple)) / 100);
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+ }
node->noCount = false;
}
}
@@ -374,6 +465,10 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
+
+ if (node->limitOption == PERCENTAGE)
+ limitstate->totalTuple= tuplestore_begin_heap(true, false, work_mem);
/*
* Initialize result type.
@@ -405,6 +500,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->totalTuple!= NULL)
+ tuplestore_end(node->totalTuple);
}
@@ -424,4 +521,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->totalTuple!= NULL)
+ tuplestore_rescan(node->totalTuple);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index db49968409..f9e5a27a20 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1136,6 +1136,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3021,6 +3022,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3105,6 +3107,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 3a084b4d1f..4b1e845739 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -973,6 +973,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1047,6 +1048,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f0c396530d..08c173ff52 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -987,6 +987,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2198,6 +2199,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2786,6 +2788,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2995,6 +2998,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index e117867de5..42bacdc7d0 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -277,6 +277,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2319,6 +2320,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index da7a92081a..a72f6abcbb 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2134,7 +2134,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2439,7 +2440,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6418,7 +6420,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6430,6 +6432,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index c729a99f8b..367772a8d6 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2140,6 +2140,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index d50d86b252..5fc3010538 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3410,6 +3410,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3431,6 +3432,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 226927b7ab..ac885659f4 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1302,6 +1302,7 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_OFFSET, "OFFSET");
qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1548,6 +1549,7 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_OFFSET, "OFFSET");
qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1783,6 +1785,7 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_OFFSET, "OFFSET");
qry->limitCount = transformLimitClause(pstate, limitCount,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 2c2208ffb7..56920c3456 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -164,6 +164,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -387,7 +388,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
+ reloption_list group_clause TriggerFuncArgs select_limit limit_clause
opt_select_limit opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
@@ -449,7 +450,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -662,7 +663,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11185,7 +11186,7 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
@@ -11193,6 +11194,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
list_nth($4, 0), list_nth($4, 1),
+ (list_nth($4, 2)),
NULL,
yyscanner);
$$ = $1;
@@ -11201,6 +11203,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
list_nth($3, 0), list_nth($3, 1),
+ list_nth($3, 2),
NULL,
yyscanner);
$$ = $1;
@@ -11209,7 +11212,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11217,7 +11220,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11225,6 +11228,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
list_nth($5, 0), list_nth($5, 1),
+ list_nth($5, 2),
$1,
yyscanner);
$$ = $2;
@@ -11233,6 +11237,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
list_nth($4, 0), list_nth($4, 1),
+ list_nth($4, 2),
$1,
yyscanner);
$$ = $2;
@@ -11519,20 +11524,20 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause { $$ = list_make3($2, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause limit_clause { $$ = list_make3($1, list_nth($2, 0), list_nth($2, 1)); }
+ | limit_clause { $$ = list_make3(NULL, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause { $$ = list_make3($1, NULL, NULL); }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */ { $$ = list_make3(NULL, NULL, NULL); }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ { $$ = list_make2($2, NULL); }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11550,9 +11555,11 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ { $$ = list_make2($3, makeString("EXACT_NUMBER")); }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ { $$ = list_make2($3, makeString("PERCENTAGE")); }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ { $$ = list_make2(makeIntConst(1, -1), NULL); }
;
offset_clause:
@@ -15419,6 +15426,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCES
@@ -15802,6 +15810,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15840,6 +15849,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 5ed0f40f69..e9e19cc385 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2288,8 +2288,10 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ Tuplestorestate *totalTuple; /* total number of row outer node return */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index cac6ff0eda..49aeb4482a 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -814,4 +814,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+} LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index e5bdc1cec5..a574f07a11 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1572,6 +1573,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index f116bc23ff..d0d5afc0f9 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -945,6 +945,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
index 6fd24203dd..96f7abaea0 100644
--- a/src/include/nodes/relation.h
+++ b/src/include/nodes/relation.h
@@ -1737,6 +1737,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 81abcf53a8..4fb67dcf96 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -248,6 +248,7 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index c8ab0280d2..4e1e3cebe2 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -64,7 +64,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 23db40147b..150d51df8f 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -300,6 +300,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, RESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/test/modules/test_ddl_deparse/expected/create_table.out b/src/test/modules/test_ddl_deparse/expected/create_table.out
index 2d7dfd533e..eb04e36aac 100644
--- a/src/test/modules/test_ddl_deparse/expected/create_table.out
+++ b/src/test/modules/test_ddl_deparse/expected/create_table.out
@@ -105,7 +105,7 @@ CREATE TABLE student (
) INHERITS (person);
NOTICE: DDL test: type simple, tag CREATE TABLE
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "id"
NOTICE: merging multiple inherited definitions of column "name"
diff --git a/src/test/modules/test_ddl_deparse/sql/create_table.sql b/src/test/modules/test_ddl_deparse/sql/create_table.sql
index dd3a908638..f158dd4296 100644
--- a/src/test/modules/test_ddl_deparse/sql/create_table.sql
+++ b/src/test/modules/test_ddl_deparse/sql/create_table.sql
@@ -94,7 +94,7 @@ CREATE TABLE student (
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index e92748c1ea..cf05eb49b5 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -79,7 +79,7 @@ CREATE TABLE student (
gpa float8
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "name"
NOTICE: merging multiple inherited definitions of column "age"
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index c18f547cbd..fcadc92f5d 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -108,6 +108,61 @@ SELECT ''::text AS five, unique1, unique2, stringu1
| 904 | 793 | UIAAAA
(5 rows)
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 51 | 76 | ZBAAAA
+ | 52 | 985 | ACAAAA
+ | 53 | 196 | BCAAAA
+ | 54 | 356 | CCAAAA
+ | 55 | 627 | DCAAAA
+ | 56 | 54 | ECAAAA
+ | 57 | 942 | FCAAAA
+ | 58 | 114 | GCAAAA
+ | 59 | 593 | HCAAAA
+(9 rows)
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 61 | 560 | JCAAAA
+(1 row)
+
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+ three | unique1 | unique2 | stringu1
+-------+---------+---------+----------
+ | 121 | 700 | REAAAA
+ | 122 | 519 | SEAAAA
+ | 123 | 777 | TEAAAA
+ | 124 | 503 | UEAAAA
+ | 125 | 849 | VEAAAA
+ | 126 | 330 | WEAAAA
+ | 127 | 511 | XEAAAA
+ | 128 | 721 | YEAAAA
+(8 rows)
+
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+ eleven | unique1 | unique2 | stringu1
+--------+---------+---------+----------
+ | 10 | 520 | KAAAAA
+ | 9 | 49 | JAAAAA
+ | 8 | 653 | IAAAAA
+ | 7 | 647 | HAAAAA
+ | 6 | 978 | GAAAAA
+(5 rows)
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -286,6 +341,43 @@ fetch all in c4;
----+----
(0 rows)
+declare c6 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c6;
+ q1 | q2
+-----+------------------
+ 123 | 456
+ 123 | 4567890123456789
+(2 rows)
+
+fetch 1 in c6;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c6;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+(1 row)
+
+fetch backward all in c6;
+ q1 | q2
+-----+-----
+ 123 | 456
+(1 row)
+
+fetch backward 1 in c6;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c6;
+ q1 | q2
+-----+------------------
+ 123 | 456
+ 123 | 4567890123456789
+(2 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
@@ -503,3 +595,19 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
45020 | 45020
(3 rows)
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
+ s1 | s2
+-------+-------
+ 45000 | 45000
+ 45010 | 45010
+ 45020 | 45020
+ 45030 | 45030
+ 45040 | 45040
+ 45050 | 45050
+ 45060 | 45060
+ 45070 | 45070
+ 45080 | 45080
+ 45090 | 45090
+(10 rows)
+
diff --git a/src/test/regress/sql/create_table.sql b/src/test/regress/sql/create_table.sql
index 90cc1a578f..66917069da 100644
--- a/src/test/regress/sql/create_table.sql
+++ b/src/test/regress/sql/create_table.sql
@@ -92,7 +92,7 @@ CREATE TABLE student (
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index 2a313d80ca..6609f8eb21 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -30,6 +30,24 @@ SELECT ''::text AS five, unique1, unique2, stringu1
SELECT ''::text AS five, unique1, unique2, stringu1
FROM onek
ORDER BY unique1 LIMIT 5 OFFSET 900;
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
@@ -38,7 +56,6 @@ select * from int8_tbl offset (case when random() < 0.5 then null::bigint end);
-- Test assorted cases involving backwards fetch from a LIMIT plan node
begin;
-
declare c1 cursor for select * from int8_tbl limit 10;
fetch all in c1;
fetch 1 in c1;
@@ -71,6 +88,15 @@ fetch backward all in c4;
fetch backward 1 in c4;
fetch all in c4;
+
+declare c6 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c6;
+fetch 1 in c6;
+fetch backward 1 in c6;
+fetch backward all in c6;
+fetch backward 1 in c6;
+fetch all in c6;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
@@ -141,3 +167,6 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
from tenk1 group by thousand order by thousand limit 3;
+
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
On 25/11/2018 10:00, Surafel Temesgen wrote:
I messed around with your changes to the grammar and it seems you don't
need to add PERCENT as a reserved keyword. Moving this to the
unreserved
keyword section does not cause any shift/reduce errors, and the
regression
tests still pass. Relative to your patch v4, these changes help:In sql standard PERCENT list as reserved world so i don't think it is a
thing that can change by me.
Yes it absolutely is. In PostgreSQL we only make words as reserved as
they need to be, and PERCENT doesn't need to be reserved at all.
Also, this query returns 210 rows instead of the expected 208:
select *
from generate_series(1, 1000)
fetch first 20.8 percent rows only
As for the design, I think you should put some serious consideration
into Andrew Gierth's approach. I would rather see something like that.
--
Vik Fearing +33 6 46 75 15 36
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
On Sun, Nov 25, 2018 at 1:24 PM Vik Fearing <vik.fearing@2ndquadrant.com>
wrote:
Also, this query returns 210 rows instead of the expected 208:
select *
from generate_series(1, 1000)
fetch first 20.8 percent rows only
this is because fetch first values work with integer and it change
fractional number to nearest integer number like select * from
generate_series(1, 1000) fetch first 20.3 rows only; is not an error rather
it return 20 rows.
As for the design, I think you should put some serious consideration
into Andrew Gierth's approach. I would rather see something like that.
I didn't go in that direction because I don’t understand why this approach
is inferior to using window function and I am not fully understand on how
to implement it.
regards
Surafel
On 25/11/2018 12:49, Surafel Temesgen wrote:
On Sun, Nov 25, 2018 at 1:24 PM Vik Fearing <vik.fearing@2ndquadrant.com
<mailto:vik.fearing@2ndquadrant.com>> wrote:Also, this query returns 210 rows instead of the expected 208:
select *
from generate_series(1, 1000)
fetch first 20.8 percent rows onlythis is because fetch first values work with integer and it change
fractional number to nearest integer number like select * from
generate_series(1, 1000) fetch first 20.3 rows only; is not an error
rather it return 20 rows.
I don't see how this behavior is justified by reading the SQL standard.
Obviously only an integer number of rows is going to be returned, but
the percentage should be calculated correctly.
I assume you meant 200 rows there, but the correct number of rows to
return is 203 for that query. Please fix it.
--
Vik Fearing +33 6 46 75 15 36
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
"Surafel" == Surafel Temesgen <surafel3000@gmail.com> writes:
Surafel> this is because fetch first values work with integer and it
Surafel> change fractional number to nearest integer number like select
Surafel> * from generate_series(1, 1000) fetch first 20.3 rows only; is
Surafel> not an error rather it return 20 rows.
31) The declared type of the <simple value specification> simply
contained in <fetch first percentage> shall be numeric.
Note that unlike <fetch first row count> there is no requirement for the
type to be _exact_ numeric or to have scale 0.
later:
ii) If <fetch first percentage> is specified, then let FFP be the
<simple value specification> simply contained in <fetch first
percentage>, and let LOCT be a <literal> whose value is OCT. Let
FFRC be the value of
CEILING ( FFP * LOCT / 100.0E0 )
NOTE 333 -- The percentage is computed using the number
of rows before removing the rows specified by <offset
row count>.
ceil(20.3 * 1000 / 100.0E0) is definitely 203, not 200.
--
Andrew.
On 11/25/18 1:05 PM, Vik Fearing wrote:
On 25/11/2018 12:49, Surafel Temesgen wrote:
On Sun, Nov 25, 2018 at 1:24 PM Vik Fearing <vik.fearing@2ndquadrant.com
<mailto:vik.fearing@2ndquadrant.com>> wrote:Also, this query returns 210 rows instead of the expected 208:
select *
from generate_series(1, 1000)
fetch first 20.8 percent rows onlythis is because fetch first values work with integer and it change
fractional number to nearest integer number like select * from
generate_series(1, 1000) fetch first 20.3 rows only; is not an error
rather it return 20 rows.I don't see how this behavior is justified by reading the SQL standard.
Obviously only an integer number of rows is going to be returned, but
the percentage should be calculated correctly.
Right. My draft of SQL standard says this:
<fetch first clause> ::=
FETCH { FIRST | NEXT } [ <fetch first quantity> ] { ROW | ROWS }
{ ONLY | WITH TIES }
<fetch first quantity> ::= <fetch first row count>
| <fetch first percentage>
<fetch first percentage> ::= <simple value specification> PERCENT
and then
30) The declared type of <fetch first row count> shall be an exact
numeric with scale 0 (zero).
31) The declared type of the <simple value specification> simply
contained in <fetch first percentage> shall
be numeric.
So the standard pretty much requires treating the value as numeric.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Tomas Vondra <tomas.vondra@2ndquadrant.com> writes:
On 11/25/18 1:05 PM, Vik Fearing wrote:
I don't see how this behavior is justified by reading the SQL standard.
Obviously only an integer number of rows is going to be returned, but
the percentage should be calculated correctly.
Right. My draft of SQL standard says this:
31) The declared type of the <simple value specification> simply
contained in <fetch first percentage> shall be numeric.
So the standard pretty much requires treating the value as numeric.
Note that this should not be read as "it must be of the type that PG
calls numeric". I'd read it as requiring a value that's in the numeric
type hierarchy. float8 would be a reasonable implementation choice
if we're going to coerce to a specific type.
regards, tom lane
On Sun, Nov 25, 2018 at 3:05 PM Vik Fearing <vik.fearing@2ndquadrant.com>
wrote:
I assume you meant 200 rows there, but the correct number of rows to
return is 203 for that query. Please fix it.
Attach is a patch that include a fix for it
regards
Surafel
Attachments:
fetch-wth-percent-v6.patchtext/x-patch; charset=US-ASCII; name=fetch-wth-percent-v6.patchDownload
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 4db8142afa..8491b7831a 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1397,7 +1397,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1407,7 +1407,8 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
+ with <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage.<literal>ROW</literal>
and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 6792f9e86c..ddc11bf6ec 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -43,6 +43,7 @@ ExecLimit(PlanState *pstate)
LimitState *node = castNode(LimitState, pstate);
ScanDirection direction;
TupleTableSlot *slot;
+ TupleDesc tupleDescriptor;
PlanState *outerPlan;
CHECK_FOR_INTERRUPTS();
@@ -52,6 +53,8 @@ ExecLimit(PlanState *pstate)
*/
direction = node->ps.state->es_direction;
outerPlan = outerPlanState(node);
+ slot = node->subSlot;
+ tupleDescriptor = node->ps.ps_ResultTupleDesc;
/*
* The main logic is a simple state machine.
@@ -60,6 +63,23 @@ ExecLimit(PlanState *pstate)
{
case LIMIT_INITIAL:
+ if (node->limitOption == PERCENTAGE)
+ {
+
+ /*
+ * Find all rows the plan will return.
+ */
+ for (;;)
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ break;
+ }
+ tuplestore_puttupleslot(node->totalTuple, slot);
+ }
+ }
+
/*
* First call for this node, so compute limit/offset. (We can't do
* this any earlier, because parameters from upper nodes will not
@@ -87,24 +107,46 @@ ExecLimit(PlanState *pstate)
return NULL;
}
- /*
- * Fetch rows from subplan until we reach position > offset.
- */
- for (;;)
+ if (node->limitOption == PERCENTAGE)
{
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
+ for (;;)
{
- /*
- * The subplan returns too few tuples for us to produce
- * any output at all.
- */
- node->lstate = LIMIT_EMPTY;
- return NULL;
+ slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if (!tuplestore_gettupleslot(node->totalTuple, true, true, slot))
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ else
+ {
+ node->subSlot = slot;
+ if (++node->position > node->offset)
+ break;
+ }
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Fetch rows from subplan until we reach position > offset.
+ */
+ for (;;)
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ /*
+ * The subplan returns too few tuples for us to produce
+ * any output at all.
+ */
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ node->subSlot = slot;
+ if (++node->position > node->offset)
+ break;
}
- node->subSlot = slot;
- if (++node->position > node->offset)
- break;
}
/*
@@ -144,18 +186,34 @@ ExecLimit(PlanState *pstate)
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
- /*
- * Get next tuple from subplan, if any.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- {
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ node->subSlot = slot;
+ node->position++;
}
- node->subSlot = slot;
- node->position++;
}
else
{
@@ -168,15 +226,29 @@ ExecLimit(PlanState *pstate)
node->lstate = LIMIT_WINDOWSTART;
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, false, false, slot))
+ {
+ node->subSlot = slot;
+ node->position--;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
- /*
- * Get previous tuple from subplan; there should be one!
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Get previous tuple from subplan; there should be one!
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->position--;
+ }
}
break;
@@ -184,15 +256,29 @@ ExecLimit(PlanState *pstate)
if (ScanDirectionIsForward(direction))
return NULL;
- /*
- * Backing up from subplan EOF, so re-fetch previous tuple; there
- * should be one! Note previous tuple must be in window.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->lstate = LIMIT_INWINDOW;
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Backing up from subplan EOF, so re-fetch previous tuple; there
+ * should be one! Note previous tuple must be in window.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
/* position does not change 'cause we didn't advance it before */
break;
@@ -283,11 +369,16 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
+ if (node->limitOption == PERCENTAGE)
+ node->count = DatumGetInt64((DatumGetFloat8(val) * tuplestore_tuple_count(node->totalTuple)) / 100);
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+ }
node->noCount = false;
}
}
@@ -374,6 +465,10 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
+
+ if (node->limitOption == PERCENTAGE)
+ limitstate->totalTuple= tuplestore_begin_heap(true, false, work_mem);
/*
* Initialize result type.
@@ -405,6 +500,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->totalTuple!= NULL)
+ tuplestore_end(node->totalTuple);
}
@@ -424,4 +521,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->totalTuple!= NULL)
+ tuplestore_rescan(node->totalTuple);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index db49968409..f9e5a27a20 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1136,6 +1136,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3021,6 +3022,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3105,6 +3107,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 3a084b4d1f..4b1e845739 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -973,6 +973,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1047,6 +1048,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f0c396530d..08c173ff52 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -987,6 +987,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2198,6 +2199,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2786,6 +2788,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2995,6 +2998,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index e117867de5..42bacdc7d0 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -277,6 +277,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2319,6 +2320,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index da7a92081a..a72f6abcbb 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2134,7 +2134,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2439,7 +2440,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6418,7 +6420,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6430,6 +6432,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index c729a99f8b..367772a8d6 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2140,6 +2140,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index d50d86b252..5fc3010538 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3410,6 +3410,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3431,6 +3432,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 226927b7ab..f13eecf505 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1298,10 +1298,11 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
}
/* transform LIMIT */
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1544,10 +1545,11 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_ORDER_BY,
false /* allow SQL92 rules */ );
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1779,10 +1781,11 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
parser_errposition(pstate,
exprLocation(list_nth(qry->targetList, tllen)))));
- qry->limitOffset = transformLimitClause(pstate, limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, limitCount,
+ qry->limitCount = transformLimitClause(pstate, limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 2c2208ffb7..56920c3456 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -164,6 +164,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -387,7 +388,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
+ reloption_list group_clause TriggerFuncArgs select_limit limit_clause
opt_select_limit opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
@@ -449,7 +450,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -662,7 +663,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11185,7 +11186,7 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
@@ -11193,6 +11194,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
list_nth($4, 0), list_nth($4, 1),
+ (list_nth($4, 2)),
NULL,
yyscanner);
$$ = $1;
@@ -11201,6 +11203,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
list_nth($3, 0), list_nth($3, 1),
+ list_nth($3, 2),
NULL,
yyscanner);
$$ = $1;
@@ -11209,7 +11212,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11217,7 +11220,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11225,6 +11228,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
list_nth($5, 0), list_nth($5, 1),
+ list_nth($5, 2),
$1,
yyscanner);
$$ = $2;
@@ -11233,6 +11237,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
list_nth($4, 0), list_nth($4, 1),
+ list_nth($4, 2),
$1,
yyscanner);
$$ = $2;
@@ -11519,20 +11524,20 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause { $$ = list_make3($2, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause limit_clause { $$ = list_make3($1, list_nth($2, 0), list_nth($2, 1)); }
+ | limit_clause { $$ = list_make3(NULL, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause { $$ = list_make3($1, NULL, NULL); }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */ { $$ = list_make3(NULL, NULL, NULL); }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ { $$ = list_make2($2, NULL); }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11550,9 +11555,11 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ { $$ = list_make2($3, makeString("EXACT_NUMBER")); }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ { $$ = list_make2($3, makeString("PERCENTAGE")); }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ { $$ = list_make2(makeIntConst(1, -1), NULL); }
;
offset_clause:
@@ -15419,6 +15426,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCES
@@ -15802,6 +15810,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15840,6 +15849,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 4ba51203a6..9dc994d854 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1708,7 +1708,7 @@ transformWhereClause(ParseState *pstate, Node *clause,
* constructName does not affect the semantics, but is used in error messages
*/
Node *
-transformLimitClause(ParseState *pstate, Node *clause,
+transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName)
{
Node *qual;
@@ -1717,8 +1717,10 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
-
- qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
+ if (limitOption == PERCENTAGE)
+ qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
+ else
+ qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
/* LIMIT can't refer to any variables of the current query */
checkExprIsVarFree(pstate, qual, constructName);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 5ed0f40f69..e9e19cc385 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2288,8 +2288,10 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ Tuplestorestate *totalTuple; /* total number of row outer node return */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index cac6ff0eda..49aeb4482a 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -814,4 +814,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+} LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index e5bdc1cec5..a574f07a11 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1572,6 +1573,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index f116bc23ff..d0d5afc0f9 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -945,6 +945,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
index 6fd24203dd..96f7abaea0 100644
--- a/src/include/nodes/relation.h
+++ b/src/include/nodes/relation.h
@@ -1737,6 +1737,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 81abcf53a8..4fb67dcf96 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -248,6 +248,7 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index c8ab0280d2..4e1e3cebe2 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -64,7 +64,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 23db40147b..150d51df8f 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -300,6 +300,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, RESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h
index f37f55302f..8a969862e4 100644
--- a/src/include/parser/parse_clause.h
+++ b/src/include/parser/parse_clause.h
@@ -22,7 +22,7 @@ extern int setTargetTable(ParseState *pstate, RangeVar *relation,
extern Node *transformWhereClause(ParseState *pstate, Node *clause,
ParseExprKind exprKind, const char *constructName);
-extern Node *transformLimitClause(ParseState *pstate, Node *clause,
+extern Node *transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName);
extern List *transformGroupClause(ParseState *pstate, List *grouplist,
List **groupingSets,
diff --git a/src/test/modules/test_ddl_deparse/expected/create_table.out b/src/test/modules/test_ddl_deparse/expected/create_table.out
index 2d7dfd533e..eb04e36aac 100644
--- a/src/test/modules/test_ddl_deparse/expected/create_table.out
+++ b/src/test/modules/test_ddl_deparse/expected/create_table.out
@@ -105,7 +105,7 @@ CREATE TABLE student (
) INHERITS (person);
NOTICE: DDL test: type simple, tag CREATE TABLE
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "id"
NOTICE: merging multiple inherited definitions of column "name"
diff --git a/src/test/modules/test_ddl_deparse/sql/create_table.sql b/src/test/modules/test_ddl_deparse/sql/create_table.sql
index dd3a908638..f158dd4296 100644
--- a/src/test/modules/test_ddl_deparse/sql/create_table.sql
+++ b/src/test/modules/test_ddl_deparse/sql/create_table.sql
@@ -94,7 +94,7 @@ CREATE TABLE student (
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index e92748c1ea..cf05eb49b5 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -79,7 +79,7 @@ CREATE TABLE student (
gpa float8
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "name"
NOTICE: merging multiple inherited definitions of column "age"
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index c18f547cbd..91d32953e9 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -108,6 +108,48 @@ SELECT ''::text AS five, unique1, unique2, stringu1
| 904 | 793 | UIAAAA
(5 rows)
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 51 | 76 | ZBAAAA
+ | 52 | 985 | ACAAAA
+ | 53 | 196 | BCAAAA
+ | 54 | 356 | CCAAAA
+ | 55 | 627 | DCAAAA
+ | 56 | 54 | ECAAAA
+ | 57 | 942 | FCAAAA
+ | 58 | 114 | GCAAAA
+ | 59 | 593 | HCAAAA
+(9 rows)
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 61 | 560 | JCAAAA
+(1 row)
+
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+ three | unique1 | unique2 | stringu1
+-------+---------+---------+----------
+(0 rows)
+
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+ eleven | unique1 | unique2 | stringu1
+--------+---------+---------+----------
+(0 rows)
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -286,6 +328,43 @@ fetch all in c4;
----+----
(0 rows)
+declare c6 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c6;
+ q1 | q2
+-----+------------------
+ 123 | 456
+ 123 | 4567890123456789
+(2 rows)
+
+fetch 1 in c6;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c6;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+(1 row)
+
+fetch backward all in c6;
+ q1 | q2
+-----+-----
+ 123 | 456
+(1 row)
+
+fetch backward 1 in c6;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c6;
+ q1 | q2
+-----+------------------
+ 123 | 456
+ 123 | 4567890123456789
+(2 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
@@ -503,3 +582,19 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
45020 | 45020
(3 rows)
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
+ s1 | s2
+-------+-------
+ 45000 | 45000
+ 45010 | 45010
+ 45020 | 45020
+ 45030 | 45030
+ 45040 | 45040
+ 45050 | 45050
+ 45060 | 45060
+ 45070 | 45070
+ 45080 | 45080
+ 45090 | 45090
+(10 rows)
+
diff --git a/src/test/regress/sql/create_table.sql b/src/test/regress/sql/create_table.sql
index 90cc1a578f..66917069da 100644
--- a/src/test/regress/sql/create_table.sql
+++ b/src/test/regress/sql/create_table.sql
@@ -92,7 +92,7 @@ CREATE TABLE student (
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index 2a313d80ca..6609f8eb21 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -30,6 +30,24 @@ SELECT ''::text AS five, unique1, unique2, stringu1
SELECT ''::text AS five, unique1, unique2, stringu1
FROM onek
ORDER BY unique1 LIMIT 5 OFFSET 900;
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
@@ -38,7 +56,6 @@ select * from int8_tbl offset (case when random() < 0.5 then null::bigint end);
-- Test assorted cases involving backwards fetch from a LIMIT plan node
begin;
-
declare c1 cursor for select * from int8_tbl limit 10;
fetch all in c1;
fetch 1 in c1;
@@ -71,6 +88,15 @@ fetch backward all in c4;
fetch backward 1 in c4;
fetch all in c4;
+
+declare c6 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c6;
+fetch 1 in c6;
+fetch backward 1 in c6;
+fetch backward all in c6;
+fetch backward 1 in c6;
+fetch all in c6;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
@@ -141,3 +167,6 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
from tenk1 group by thousand order by thousand limit 3;
+
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
On Fri, Nov 30, 2018 at 10:23 AM Surafel Temesgen <surafel3000@gmail.com>
wrote:
Attach is a patch that include a fix for it
By mistake the patch coerce OFFSET clause to float8 too which is not
necessary .
The attached patch correct it
regards
Surafel
Attachments:
fetch-wth-percent-v7.patchtext/x-patch; charset=US-ASCII; name=fetch-wth-percent-v7.patchDownload
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 4db8142afa..8491b7831a 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1397,7 +1397,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1407,7 +1407,8 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
+ with <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage.<literal>ROW</literal>
and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index 6792f9e86c..ddc11bf6ec 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -43,6 +43,7 @@ ExecLimit(PlanState *pstate)
LimitState *node = castNode(LimitState, pstate);
ScanDirection direction;
TupleTableSlot *slot;
+ TupleDesc tupleDescriptor;
PlanState *outerPlan;
CHECK_FOR_INTERRUPTS();
@@ -52,6 +53,8 @@ ExecLimit(PlanState *pstate)
*/
direction = node->ps.state->es_direction;
outerPlan = outerPlanState(node);
+ slot = node->subSlot;
+ tupleDescriptor = node->ps.ps_ResultTupleDesc;
/*
* The main logic is a simple state machine.
@@ -60,6 +63,23 @@ ExecLimit(PlanState *pstate)
{
case LIMIT_INITIAL:
+ if (node->limitOption == PERCENTAGE)
+ {
+
+ /*
+ * Find all rows the plan will return.
+ */
+ for (;;)
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ break;
+ }
+ tuplestore_puttupleslot(node->totalTuple, slot);
+ }
+ }
+
/*
* First call for this node, so compute limit/offset. (We can't do
* this any earlier, because parameters from upper nodes will not
@@ -87,24 +107,46 @@ ExecLimit(PlanState *pstate)
return NULL;
}
- /*
- * Fetch rows from subplan until we reach position > offset.
- */
- for (;;)
+ if (node->limitOption == PERCENTAGE)
{
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
+ for (;;)
{
- /*
- * The subplan returns too few tuples for us to produce
- * any output at all.
- */
- node->lstate = LIMIT_EMPTY;
- return NULL;
+ slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if (!tuplestore_gettupleslot(node->totalTuple, true, true, slot))
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ else
+ {
+ node->subSlot = slot;
+ if (++node->position > node->offset)
+ break;
+ }
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Fetch rows from subplan until we reach position > offset.
+ */
+ for (;;)
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ /*
+ * The subplan returns too few tuples for us to produce
+ * any output at all.
+ */
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ node->subSlot = slot;
+ if (++node->position > node->offset)
+ break;
}
- node->subSlot = slot;
- if (++node->position > node->offset)
- break;
}
/*
@@ -144,18 +186,34 @@ ExecLimit(PlanState *pstate)
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
- /*
- * Get next tuple from subplan, if any.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- {
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ node->subSlot = slot;
+ node->position++;
}
- node->subSlot = slot;
- node->position++;
}
else
{
@@ -168,15 +226,29 @@ ExecLimit(PlanState *pstate)
node->lstate = LIMIT_WINDOWSTART;
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, false, false, slot))
+ {
+ node->subSlot = slot;
+ node->position--;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
- /*
- * Get previous tuple from subplan; there should be one!
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Get previous tuple from subplan; there should be one!
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->position--;
+ }
}
break;
@@ -184,15 +256,29 @@ ExecLimit(PlanState *pstate)
if (ScanDirectionIsForward(direction))
return NULL;
- /*
- * Backing up from subplan EOF, so re-fetch previous tuple; there
- * should be one! Note previous tuple must be in window.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->lstate = LIMIT_INWINDOW;
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Backing up from subplan EOF, so re-fetch previous tuple; there
+ * should be one! Note previous tuple must be in window.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
/* position does not change 'cause we didn't advance it before */
break;
@@ -283,11 +369,16 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
+ if (node->limitOption == PERCENTAGE)
+ node->count = DatumGetInt64((DatumGetFloat8(val) * tuplestore_tuple_count(node->totalTuple)) / 100);
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+ }
node->noCount = false;
}
}
@@ -374,6 +465,10 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
+
+ if (node->limitOption == PERCENTAGE)
+ limitstate->totalTuple= tuplestore_begin_heap(true, false, work_mem);
/*
* Initialize result type.
@@ -405,6 +500,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->totalTuple!= NULL)
+ tuplestore_end(node->totalTuple);
}
@@ -424,4 +521,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->totalTuple!= NULL)
+ tuplestore_rescan(node->totalTuple);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index db49968409..f9e5a27a20 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1136,6 +1136,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3021,6 +3022,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3105,6 +3107,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 3a084b4d1f..4b1e845739 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -973,6 +973,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1047,6 +1048,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f0c396530d..08c173ff52 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -987,6 +987,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2198,6 +2199,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2786,6 +2788,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2995,6 +2998,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index e117867de5..42bacdc7d0 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -277,6 +277,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2319,6 +2320,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index da7a92081a..a72f6abcbb 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2134,7 +2134,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2439,7 +2440,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6418,7 +6420,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6430,6 +6432,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index c729a99f8b..367772a8d6 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2140,6 +2140,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index d50d86b252..5fc3010538 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3410,6 +3410,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3431,6 +3432,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 226927b7ab..f13eecf505 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1298,10 +1298,11 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
}
/* transform LIMIT */
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1544,10 +1545,11 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_ORDER_BY,
false /* allow SQL92 rules */ );
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1779,10 +1781,11 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
parser_errposition(pstate,
exprLocation(list_nth(qry->targetList, tllen)))));
- qry->limitOffset = transformLimitClause(pstate, limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, limitCount,
+ qry->limitCount = transformLimitClause(pstate, limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 2c2208ffb7..56920c3456 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -164,6 +164,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -387,7 +388,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
+ reloption_list group_clause TriggerFuncArgs select_limit limit_clause
opt_select_limit opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
@@ -449,7 +450,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -662,7 +663,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11185,7 +11186,7 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
@@ -11193,6 +11194,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
list_nth($4, 0), list_nth($4, 1),
+ (list_nth($4, 2)),
NULL,
yyscanner);
$$ = $1;
@@ -11201,6 +11203,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
list_nth($3, 0), list_nth($3, 1),
+ list_nth($3, 2),
NULL,
yyscanner);
$$ = $1;
@@ -11209,7 +11212,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11217,7 +11220,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11225,6 +11228,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
list_nth($5, 0), list_nth($5, 1),
+ list_nth($5, 2),
$1,
yyscanner);
$$ = $2;
@@ -11233,6 +11237,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
list_nth($4, 0), list_nth($4, 1),
+ list_nth($4, 2),
$1,
yyscanner);
$$ = $2;
@@ -11519,20 +11524,20 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause { $$ = list_make3($2, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause limit_clause { $$ = list_make3($1, list_nth($2, 0), list_nth($2, 1)); }
+ | limit_clause { $$ = list_make3(NULL, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause { $$ = list_make3($1, NULL, NULL); }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */ { $$ = list_make3(NULL, NULL, NULL); }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ { $$ = list_make2($2, NULL); }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11550,9 +11555,11 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ { $$ = list_make2($3, makeString("EXACT_NUMBER")); }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ { $$ = list_make2($3, makeString("PERCENTAGE")); }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ { $$ = list_make2(makeIntConst(1, -1), NULL); }
;
offset_clause:
@@ -15419,6 +15426,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCES
@@ -15802,6 +15810,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15840,6 +15849,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 4ba51203a6..ebcb74fb64 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1708,7 +1708,7 @@ transformWhereClause(ParseState *pstate, Node *clause,
* constructName does not affect the semantics, but is used in error messages
*/
Node *
-transformLimitClause(ParseState *pstate, Node *clause,
+transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName)
{
Node *qual;
@@ -1717,8 +1717,10 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
-
- qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
+ if (limitOption == PERCENTAGE && (strcmp(constructName, "LIMIT") == 0))
+ qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
+ else
+ qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
/* LIMIT can't refer to any variables of the current query */
checkExprIsVarFree(pstate, qual, constructName);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 5ed0f40f69..e9e19cc385 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2288,8 +2288,10 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ Tuplestorestate *totalTuple; /* total number of row outer node return */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index cac6ff0eda..49aeb4482a 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -814,4 +814,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+} LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index e5bdc1cec5..a574f07a11 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1572,6 +1573,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index f116bc23ff..d0d5afc0f9 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -945,6 +945,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
index 6fd24203dd..96f7abaea0 100644
--- a/src/include/nodes/relation.h
+++ b/src/include/nodes/relation.h
@@ -1737,6 +1737,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 81abcf53a8..4fb67dcf96 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -248,6 +248,7 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index c8ab0280d2..4e1e3cebe2 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -64,7 +64,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 23db40147b..150d51df8f 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -300,6 +300,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, RESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h
index f37f55302f..8a969862e4 100644
--- a/src/include/parser/parse_clause.h
+++ b/src/include/parser/parse_clause.h
@@ -22,7 +22,7 @@ extern int setTargetTable(ParseState *pstate, RangeVar *relation,
extern Node *transformWhereClause(ParseState *pstate, Node *clause,
ParseExprKind exprKind, const char *constructName);
-extern Node *transformLimitClause(ParseState *pstate, Node *clause,
+extern Node *transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName);
extern List *transformGroupClause(ParseState *pstate, List *grouplist,
List **groupingSets,
diff --git a/src/test/modules/test_ddl_deparse/expected/create_table.out b/src/test/modules/test_ddl_deparse/expected/create_table.out
index 2d7dfd533e..eb04e36aac 100644
--- a/src/test/modules/test_ddl_deparse/expected/create_table.out
+++ b/src/test/modules/test_ddl_deparse/expected/create_table.out
@@ -105,7 +105,7 @@ CREATE TABLE student (
) INHERITS (person);
NOTICE: DDL test: type simple, tag CREATE TABLE
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "id"
NOTICE: merging multiple inherited definitions of column "name"
diff --git a/src/test/modules/test_ddl_deparse/sql/create_table.sql b/src/test/modules/test_ddl_deparse/sql/create_table.sql
index dd3a908638..f158dd4296 100644
--- a/src/test/modules/test_ddl_deparse/sql/create_table.sql
+++ b/src/test/modules/test_ddl_deparse/sql/create_table.sql
@@ -94,7 +94,7 @@ CREATE TABLE student (
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index e92748c1ea..cf05eb49b5 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -79,7 +79,7 @@ CREATE TABLE student (
gpa float8
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "name"
NOTICE: merging multiple inherited definitions of column "age"
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index c18f547cbd..fcadc92f5d 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -108,6 +108,61 @@ SELECT ''::text AS five, unique1, unique2, stringu1
| 904 | 793 | UIAAAA
(5 rows)
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 51 | 76 | ZBAAAA
+ | 52 | 985 | ACAAAA
+ | 53 | 196 | BCAAAA
+ | 54 | 356 | CCAAAA
+ | 55 | 627 | DCAAAA
+ | 56 | 54 | ECAAAA
+ | 57 | 942 | FCAAAA
+ | 58 | 114 | GCAAAA
+ | 59 | 593 | HCAAAA
+(9 rows)
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 61 | 560 | JCAAAA
+(1 row)
+
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+ three | unique1 | unique2 | stringu1
+-------+---------+---------+----------
+ | 121 | 700 | REAAAA
+ | 122 | 519 | SEAAAA
+ | 123 | 777 | TEAAAA
+ | 124 | 503 | UEAAAA
+ | 125 | 849 | VEAAAA
+ | 126 | 330 | WEAAAA
+ | 127 | 511 | XEAAAA
+ | 128 | 721 | YEAAAA
+(8 rows)
+
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+ eleven | unique1 | unique2 | stringu1
+--------+---------+---------+----------
+ | 10 | 520 | KAAAAA
+ | 9 | 49 | JAAAAA
+ | 8 | 653 | IAAAAA
+ | 7 | 647 | HAAAAA
+ | 6 | 978 | GAAAAA
+(5 rows)
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -286,6 +341,43 @@ fetch all in c4;
----+----
(0 rows)
+declare c6 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c6;
+ q1 | q2
+-----+------------------
+ 123 | 456
+ 123 | 4567890123456789
+(2 rows)
+
+fetch 1 in c6;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c6;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+(1 row)
+
+fetch backward all in c6;
+ q1 | q2
+-----+-----
+ 123 | 456
+(1 row)
+
+fetch backward 1 in c6;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c6;
+ q1 | q2
+-----+------------------
+ 123 | 456
+ 123 | 4567890123456789
+(2 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
@@ -503,3 +595,19 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
45020 | 45020
(3 rows)
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
+ s1 | s2
+-------+-------
+ 45000 | 45000
+ 45010 | 45010
+ 45020 | 45020
+ 45030 | 45030
+ 45040 | 45040
+ 45050 | 45050
+ 45060 | 45060
+ 45070 | 45070
+ 45080 | 45080
+ 45090 | 45090
+(10 rows)
+
diff --git a/src/test/regress/sql/create_table.sql b/src/test/regress/sql/create_table.sql
index 90cc1a578f..66917069da 100644
--- a/src/test/regress/sql/create_table.sql
+++ b/src/test/regress/sql/create_table.sql
@@ -92,7 +92,7 @@ CREATE TABLE student (
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index 2a313d80ca..6609f8eb21 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -30,6 +30,24 @@ SELECT ''::text AS five, unique1, unique2, stringu1
SELECT ''::text AS five, unique1, unique2, stringu1
FROM onek
ORDER BY unique1 LIMIT 5 OFFSET 900;
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
@@ -38,7 +56,6 @@ select * from int8_tbl offset (case when random() < 0.5 then null::bigint end);
-- Test assorted cases involving backwards fetch from a LIMIT plan node
begin;
-
declare c1 cursor for select * from int8_tbl limit 10;
fetch all in c1;
fetch 1 in c1;
@@ -71,6 +88,15 @@ fetch backward all in c4;
fetch backward 1 in c4;
fetch all in c4;
+
+declare c6 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c6;
+fetch 1 in c6;
+fetch backward 1 in c6;
+fetch backward all in c6;
+fetch backward 1 in c6;
+fetch all in c6;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
@@ -141,3 +167,6 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
from tenk1 group by thousand order by thousand limit 3;
+
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
Hi,
I've been looking at this patch today, and I think there's a couple of
issues with the costing and execution. Consider a simple table with 1M rows:
create table t as select i from generate_series(1,1000000) s(i);
and these two queries, that both select 1% of rows
select * from t fetch first 10000 rows only;
select * from t fetch first 1 percent rows only;
which are costed like this
test=# explain select * from t fetch first 10000 rows only;
QUERY PLAN
-----------------------------------------------------------------
Limit (cost=0.00..144.25 rows=10000 width=4)
-> Seq Scan on t (cost=0.00..14425.00 rows=1000000 width=4)
(2 rows)
test=# explain select * from t fetch first 1 percent rows only;
QUERY PLAN
-----------------------------------------------------------------
Limit (cost=0.00..14425.00 rows=1000000 width=4)
-> Seq Scan on t (cost=0.00..14425.00 rows=1000000 width=4)
(2 rows)
There are two issues with the costs in the "percent" case.
Firstly, the estimated number of rows should be about the same (10k) in
both cases, but the "percent" case incorrectly uses the total row count
(i.e. as if 100% rows were returned).
It's correct that the total cost for the "percent" case is based on 100%
of rows, of course, because the code has to fetch everything and stash
it into the tuplestore in LIMIT_INITIAL phase.
But that implies the startup cost for the "percent" case can't be 0,
because all this work has to be done before emitting the first row.
So these costing aspects need fixing, IMHO.
The execution part of the patch seems to be working correctly, but I
think there's an improvement - we don't need to execute the outer plan
to completion before emitting the first row. For example, let's say the
outer plan produces 10000 rows in total and we're supposed to return the
first 1% of those rows. We can emit the first row after fetching the
first 100 rows, we don't have to wait for fetching all 10k rows.
This may seem pointless at first because we eventually have to scan all
the rows anyway (as we don't know we're done otherwise). But it may be
nested in another early-terminating node in which case it will matter,
e.g. like this:
SELECT * FROM t1 WHERE EXISTS (
SELECT * FROM t2 FETCH FIRST 1 PERCENT ROWS ONLY
) foo;
I admit this is example is somewhat artificial, but it's pretty easy to
end with queries like this with views etc. Overall the fast startup
seems like a quite valuable feature and we should try keeping it.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Hi
On Tue, Jan 1, 2019 at 10:08 PM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:
Hi,
I've been looking at this patch today, and I think there's a couple of
issues with the costing and execution. Consider a simple table with 1M
rows:create table t as select i from generate_series(1,1000000) s(i);
and these two queries, that both select 1% of rows
select * from t fetch first 10000 rows only;
select * from t fetch first 1 percent rows only;
which are costed like this
test=# explain select * from t fetch first 10000 rows only;
QUERY PLAN
-----------------------------------------------------------------
Limit (cost=0.00..144.25 rows=10000 width=4)
-> Seq Scan on t (cost=0.00..14425.00 rows=1000000 width=4)
(2 rows)test=# explain select * from t fetch first 1 percent rows only;
QUERY PLAN
-----------------------------------------------------------------
Limit (cost=0.00..14425.00 rows=1000000 width=4)
-> Seq Scan on t (cost=0.00..14425.00 rows=1000000 width=4)
(2 rows)There are two issues with the costs in the "percent" case.
Firstly, the estimated number of rows should be about the same (10k) in
both cases, but the "percent" case incorrectly uses the total row count
(i.e. as if 100% rows were returned).Ok I will correct it
It's correct that the total cost for the "percent" case is based on 100%
of rows, of course, because the code has to fetch everything and stash
it into the tuplestore in LIMIT_INITIAL phase.But that implies the startup cost for the "percent" case can't be 0,
because all this work has to be done before emitting the first row.
Correct, startup cost must be equal to total cost of source data path and
total cost have to be slightly greater than startup cost . I am planing to
increase the total cost by limitCount * 0.1(similar to the
parallel_tuple_cost) because getting tuple from tuplestore are almost
similar operation to passing a tuple from worker to master backend.
So these costing aspects need fixing, IMHO.
The execution part of the patch seems to be working correctly, but I
think there's an improvement - we don't need to execute the outer plan
to completion before emitting the first row. For example, let's say the
outer plan produces 10000 rows in total and we're supposed to return the
first 1% of those rows. We can emit the first row after fetching the
first 100 rows, we don't have to wait for fetching all 10k rows.
but total rows count is not given how can we determine safe to return row
Regards
Surafel
On 1/3/19 1:00 PM, Surafel Temesgen wrote:
Hi
On Tue, Jan 1, 2019 at 10:08 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:...
Firstly, the estimated number of rows should be about the same (10k) in
both cases, but the "percent" case incorrectly uses the total row count
(i.e. as if 100% rows were returned).Ok I will correct it
It's correct that the total cost for the "percent" case is based on 100%
of rows, of course, because the code has to fetch everything and stash
it into the tuplestore in LIMIT_INITIAL phase.But that implies the startup cost for the "percent" case can't be 0,
because all this work has to be done before emitting the first row.Correct, startup cost must be equal to total cost of source data path
and total cost have to be slightly greater than startup cost . I am
planing to increase the total cost by limitCount * 0.1(similar to the
parallel_tuple_cost) because getting tuple from tuplestore are almost
similar operation to passing a tuple from worker to master backend.
OK, sounds good in principle.
So these costing aspects need fixing, IMHO.
The execution part of the patch seems to be working correctly, but I
think there's an improvement - we don't need to execute the outer plan
to completion before emitting the first row. For example, let's say the
outer plan produces 10000 rows in total and we're supposed to return the
first 1% of those rows. We can emit the first row after fetching the
first 100 rows, we don't have to wait for fetching all 10k rows.but total rows count is not given how can we determine safe to return row
But you know how many rows were fetched from the outer plan, and this
number only grows grows. So the number of rows returned by FETCH FIRST
... PERCENT also only grows. For example with 10% of rows, you know that
once you reach 100 rows you should emit ~10 rows, with 200 rows you know
you should emit ~20 rows, etc. So you may track how many rows we're
supposed to return / returned so far, and emit them early.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Thu, Jan 3, 2019 at 4:51 PM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:
On 1/3/19 1:00 PM, Surafel Temesgen wrote:
Hi
On Tue, Jan 1, 2019 at 10:08 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>>wrote:
The execution part of the patch seems to be working correctly, but I
think there's an improvement - we don't need to execute the outerplan
to completion before emitting the first row. For example, let's say
the
outer plan produces 10000 rows in total and we're supposed to return
the
first 1% of those rows. We can emit the first row after fetching the
first 100 rows, we don't have to wait for fetching all 10k rows.but total rows count is not given how can we determine safe to return row
But you know how many rows were fetched from the outer plan, and this
number only grows grows. So the number of rows returned by FETCH FIRST
... PERCENT also only grows. For example with 10% of rows, you know that
once you reach 100 rows you should emit ~10 rows, with 200 rows you know
you should emit ~20 rows, etc. So you may track how many rows we're
supposed to return / returned so far, and emit them early.
yes that is clear but i don't find it easy to put that in formula. may be
someone with good mathematics will help
regards
Surafel
On Tue, Jan 1, 2019 at 10:08 PM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:
The execution part of the patch seems to be working correctly, but I
think there's an improvement - we don't need to execute the outer plan
to completion before emitting the first row. For example, let's say the
outer plan produces 10000 rows in total and we're supposed to return the
first 1% of those rows. We can emit the first row after fetching the
first 100 rows, we don't have to wait for fetching all 10k rows.
my other concern is in this case we are going to determine limit and safe
row to return on the fly
which have additional computation and i fair that it will not perform
better on most of the case
regards
Surafel
On 1/4/19 7:40 AM, Surafel Temesgen wrote:
On Thu, Jan 3, 2019 at 4:51 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:On 1/3/19 1:00 PM, Surafel Temesgen wrote:
Hi
On Tue, Jan 1, 2019 at 10:08 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com<mailto:tomas.vondra@2ndquadrant.com>
<mailto:tomas.vondra@2ndquadrant.com
<mailto:tomas.vondra@2ndquadrant.com>>> wrote:The execution part of the patch seems to be working correctly,
but I
think there's an improvement - we don't need to execute the
outer plan
to completion before emitting the first row. For example,
let's say the
outer plan produces 10000 rows in total and we're supposed to
return the
first 1% of those rows. We can emit the first row after
fetching the
first 100 rows, we don't have to wait for fetching all 10k rows.
but total rows count is not given how can we determine safe to
return row
But you know how many rows were fetched from the outer plan, and this
number only grows grows. So the number of rows returned by FETCH FIRST
... PERCENT also only grows. For example with 10% of rows, you know that
once you reach 100 rows you should emit ~10 rows, with 200 rows you know
you should emit ~20 rows, etc. So you may track how many rows we're
supposed to return / returned so far, and emit them early.yes that is clear but i don't find it easy to put that in formula. may
be someone with good mathematics will help
What formula? All the math remains exactly the same, you just need to
update the number of rows to return and track how many rows are already
returned.
I haven't tried doing that, but AFAICS you'd need to tweak how/when
node->count is computed - instead of computing it only once it needs to
be updated after fetching each row from the subplan.
Furthermore, you'll need to stash the subplan rows somewhere (into a
tuplestore probably), and whenever the node->count value increments,
you'll need to grab a row from the tuplestore and return that (i.e.
tweak node->position and set node->subSlot).
I hope that makes sense. The one thing I'm not quite sure about is
whether tuplestore allows adding and getting rows at the same time.
Does that make sense?
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 1/4/19 8:08 AM, Surafel Temesgen wrote:
On Tue, Jan 1, 2019 at 10:08 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:The execution part of the patch seems to be working correctly, but I
think there's an improvement - we don't need to execute the outer plan
to completion before emitting the first row. For example, let's say the
outer plan produces 10000 rows in total and we're supposed to return the
first 1% of those rows. We can emit the first row after fetching the
first 100 rows, we don't have to wait for fetching all 10k rows.my other concern is in this case we are going to determine limit and
safe row to return on the fly
which have additional computation and i fair that it will not perform
better on most of the case
I very much doubt recomputing the number of rows to return will be
measurable at all, considering those are int64 values. It's likely
negligible compared to all the other overhead (reading rows from the
subplan, etc.).
And even if it turns out expensive, there are ways to make it less
expensive (e.g. not computing it every time, but only when it actually
can increment the value - for example with 1% rows to return, it's
enough to recompute it every 100 rows).
It also very much depends on which use cases you consider important.
Clearly, you're only thinking about the case that actually requires
fetching everything. But there's also the use case where you have
another limit node somewhere above.
So let's do a simple "cost" analysis of these two use cases:
1) case #1 (no LIMIT above, requires fetching everything)
Let's assume the "cost" of the current approach (fetching everything in
advance, computing node->count once) is the baseline here.
With the incremental approach, there likely is some extra overhead.
Let's overshoot it a bit - with 5% overhead, it's 1.05 of the baseline.
2) case #2 (LIMIT/EXISTS above, only needs to fetch the first few rows,
perhaps just a single one)
Here, we take the incremental approach as the baseline. With the "fetch
everything first" approach we need to evaluate the whole subplan and
fetch all the results, instead of just some small fraction. But the
exact ratio is pretty much arbitrary - I can easily construct queries
that need to fetch 100x or 100000x more data from the subplan.
So essentially using the "incremental fetch" approach means we may pay
5% overhead in cases where we need to fetch everything, while the "fetch
everything first" approach means we'll add arbitrary amount of overhead
to cases where we don't need to fetch everything.
Of course, you one way to deal with this would be to argue the cases
benefiting from incremental approach are very rare / non-existent.
Perhaps that's the case, although I don't see why it would be.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Fri, Jan 4, 2019 at 5:27 PM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:
What formula? All the math remains exactly the same, you just need to
update the number of rows to return and track how many rows are already
returned.I haven't tried doing that, but AFAICS you'd need to tweak how/when
node->count is computed - instead of computing it only once it needs to
be updated after fetching each row from the subplan.Furthermore, you'll need to stash the subplan rows somewhere (into a
tuplestore probably), and whenever the node->count value increments,
you'll need to grab a row from the tuplestore and return that (i.e.
tweak node->position and set node->subSlot).I hope that makes sense. The one thing I'm not quite sure about is
whether tuplestore allows adding and getting rows at the same time.Does that make sense
In current implementation in LIMIT_INITIAL state we execute outer plan to
the end , store the resulting tuples in tuplestore and calculate limitCount
in number .We are in this state only once and did not return any tuple.
Once we are at LIMIT_INWINDOW state and inner node execution asking for
tuple it return from tuple store immediately.
Inorder to do fast startup what I was thinking was dividing the work done
at LIMIT_INITIAL state in to limitCount. For example we want 0.5 percent of
the result which means in LIMIT_INWINDOW state we execute outer node 200
times ,store the result in tuplestore and return the first tuple. if we
don’t get that much tuple that means we reach end of the limit . But I
think i can’t do it in this way because percent have meaning only with
total number so LIMIT_INITIAL work can’t be avoid
regards
Surafel
On 1/6/19 12:40 PM, Surafel Temesgen wrote:
On Fri, Jan 4, 2019 at 5:27 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:What formula? All the math remains exactly the same, you just need to
update the number of rows to return and track how many rows are already
returned.I haven't tried doing that, but AFAICS you'd need to tweak how/when
node->count is computed - instead of computing it only once it needs to
be updated after fetching each row from the subplan.Furthermore, you'll need to stash the subplan rows somewhere (into a
tuplestore probably), and whenever the node->count value increments,
you'll need to grab a row from the tuplestore and return that (i.e.
tweak node->position and set node->subSlot).I hope that makes sense. The one thing I'm not quite sure about is
whether tuplestore allows adding and getting rows at the same time.Does that make sense
In current implementation in LIMIT_INITIAL state we execute outer
plan to the end , store the resulting tuples in tuplestore and
calculate limitCount in number .We are in this state only once and
did not return any tuple. Once we are at LIMIT_INWINDOW state and
inner node execution asking for tuple it return from tuple store
immediately.
Right.
Inorder to do fast startup what I was thinking was dividing the work
done at LIMIT_INITIAL state in to limitCount. For example we want
0.5 percent of the result which means in LIMIT_INWINDOW state we
execute outer node 200 times ,store the result in tuplestore and
return the first tuple. if we don’t get that much tuple that means we
reach end of the limit.
Yes, pretty much.
But I think i can’t do it in this way because percent have meaning
only with total number so LIMIT_INITIAL work can’t be avoid.
I'm not sure I understand what you mean by "percent would have meaning
only with the total number". The important thing is that you can track
how many tuples you're supposed to return (based on the current number
of rows from the outer plan) and how many you've produced so far. And
those numbers are only growing.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Sun, Jan 6, 2019 at 5:51 PM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:
On 1/6/19 12:40 PM, Surafel Temesgen wrote:
On Fri, Jan 4, 2019 at 5:27 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>>wrote:
What formula? All the math remains exactly the same, you just need to
update the number of rows to return and track how many rows arealready
returned.
I haven't tried doing that, but AFAICS you'd need to tweak how/when
node->count is computed - instead of computing it only once it needsto
be updated after fetching each row from the subplan.
Furthermore, you'll need to stash the subplan rows somewhere (into a
tuplestore probably), and whenever the node->count value increments,
you'll need to grab a row from the tuplestore and return that (i.e.
tweak node->position and set node->subSlot).I hope that makes sense. The one thing I'm not quite sure about is
whether tuplestore allows adding and getting rows at the same time.Does that make sense
In current implementation in LIMIT_INITIAL state we execute outer
plan to the end , store the resulting tuples in tuplestore and
calculate limitCount in number .We are in this state only once and
did not return any tuple. Once we are at LIMIT_INWINDOW state and
inner node execution asking for tuple it return from tuple store
immediately.Right.
Inorder to do fast startup what I was thinking was dividing the work
done at LIMIT_INITIAL state in to limitCount. For example we want
0.5 percent of the result which means in LIMIT_INWINDOW state we
execute outer node 200 times ,store the result in tuplestore and
return the first tuple. if we don’t get that much tuple that means we
reach end of the limit.Yes, pretty much.
While working on this method i found an issue that did not work will on
percent
with fractional part like 2.6 percent which means we have to emit per every
38.4615
tuple so it have to be round up to 39 or round down to 38 either way it
leads to incorrect
result .Even with integer percentage sometimes the result came out with
fractional part
and needs rounding
regards
Surafel
On 1/9/19 7:09 AM, Surafel Temesgen wrote:
On Sun, Jan 6, 2019 at 5:51 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:On 1/6/19 12:40 PM, Surafel Temesgen wrote:
On Fri, Jan 4, 2019 at 5:27 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com<mailto:tomas.vondra@2ndquadrant.com>
<mailto:tomas.vondra@2ndquadrant.com
<mailto:tomas.vondra@2ndquadrant.com>>> wrote:What formula? All the math remains exactly the same, you just
need to
update the number of rows to return and track how many rows
are already
returned.
I haven't tried doing that, but AFAICS you'd need to tweak
how/when
node->count is computed - instead of computing it only once it
needs to
be updated after fetching each row from the subplan.
Furthermore, you'll need to stash the subplan rows somewhere
(into a
tuplestore probably), and whenever the node->count value
increments,
you'll need to grab a row from the tuplestore and return that
(i.e.
tweak node->position and set node->subSlot).
I hope that makes sense. The one thing I'm not quite sure about is
whether tuplestore allows adding and getting rows at the sametime.
Does that make sense
In current implementation in LIMIT_INITIAL state we execute outer
plan to the end , store the resulting tuples in tuplestore and
calculate limitCount in number .We are in this state only once and
did not return any tuple. Once we are at LIMIT_INWINDOW state and
inner node execution asking for tuple it return from tuple store
immediately.Right.
Inorder to do fast startup what I was thinking was dividing the work
done at LIMIT_INITIAL state in to limitCount. For example we want
0.5 percent of the result which means in LIMIT_INWINDOW state we
execute outer node 200 times ,store the result in tuplestore and
return the first tuple. if we don’t get that much tuple that means we
reach end of the limit.Yes, pretty much.
While working on this method i found an issue that did not work will
on percent with fractional part like 2.6 percent which means we have
to emit per every 38.4615 tuple so it have to be round up to 39 or
round down to 38 either way it leads to incorrect result .Even with
integer percentage sometimes the result came out with fractional
part and needs rounding
It's hard to say what exactly are you doing, because you haven't shared
any code nor examples. But it sounds as if you're computing how often to
produce an additional row at the beginning, and then simply multiplying
it. I'm not sure that's quite possible, because there will be issues due
to rounding.
Firstly, I see your last patch (from 30/11) does this:
node->count = DatumGetInt64((DatumGetFloat8(val) *
tuplestore_tuple_count(node->totalTuple)) / 100);
but the SQL standard I have says this in the <query expression> section:
If <fetch first percentage> is specified, then let FFP be the
<simple value specification> simply contained in <fetch first
percentage>, and let LOCT be a <literal> whose value is OCT.
Let FFRC be the value of
CEILING ( FFP * LOCT / 100.0E0 )
That's not what you patch does, though, because it relies on automatic
float->int conversion, which does floor() and not ceil(). FWIW the final
DatumGetInt64 seems unnecessary, because the expression is float8 and
not Datum.
The patch should do something like this instead:
node->count = ceil((DatumGetFloat8(val) *
tuplestore_tuple_count(node->totalTuple)) / 100);
Now, back to the row count issue - let's assume the user requests
FETCH FIRST 3 PERCENT ROWS ONLY
This means we should produce 1 rows every 33 rows, but only very
roughly. In reality it's a bit less, because 3% is not 1/33 exactly.
Consider this query, which computes number of rows for
select sample_cnt, l - lag(l) over (order by sample_cnt) AS d from (
select sample_cnt, min(nrows) AS l from (
select
nrows,
(nrows * 3.0 / 100.0) AS raw_sample_cnt,
ceil(nrows * 3.0 / 100.0) AS sample_cnt
from generate_series(1,10000) s(nrows)
) foo group by sample_cnt order by sample_cnt
) bar;
The inner-most query simply computes how many rows we're supposed to
return based on certain row count. Then we compute points at which the
sample_cnt actually changes. And finally we compute the distance between
those points. You should get something like
sample_cnt | d
------------+----
1 |
2 | 33
3 | 33
4 | 34
5 | 33
6 | 33
7 | 34
8 | 33
9 | 33
10 | 34
11 | 33
...
This shows that the distance oscillates between 33 and 34, so no matter
what value you pick, you can't simply multiply it to get the desired
sample size. It needs to be re-computed as you go.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Wed, Jan 9, 2019 at 5:38 PM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:
It's hard to say what exactly are you doing, because you haven't shared
any code nor examples.
okay i attach in progress patch
regards
Surafel
Attachments:
fetch-with-percent-inprogress.patchtext/x-patch; charset=US-ASCII; name=fetch-with-percent-inprogress.patchDownload
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 4db8142afa..8491b7831a 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1397,7 +1397,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1407,7 +1407,8 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
+ with <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage.<literal>ROW</literal>
and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index baa669abe8..ee04992c9f 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -44,6 +44,9 @@ ExecLimit(PlanState *pstate)
ScanDirection direction;
TupleTableSlot *slot;
PlanState *outerPlan;
+ TupleDesc tupleDescriptor;
+ slot = node->subSlot;
+ tupleDescriptor = node->ps.ps_ResultTupleDesc;
CHECK_FOR_INTERRUPTS();
@@ -131,7 +134,7 @@ ExecLimit(PlanState *pstate)
* the state machine state to record having done so.
*/
if (!node->noCount &&
- node->position - node->offset >= node->count)
+ node->position - node->offset >= node->count && node->limitOption != PERCENTAGE)
{
node->lstate = LIMIT_WINDOWEND;
@@ -144,6 +147,42 @@ ExecLimit(PlanState *pstate)
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+
+ for (int i=1; node->count >= i; i++)
+ {
+
+ slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_WINDOWEND;
+
+ /*
+ * If we know we won't need to back up, we can release
+ * resources at this point.
+ */
+ if (!(node->ps.state->es_top_eflags & EXEC_FLAG_BACKWARD))
+ (void) ExecShutdownNode(outerPlan);
+
+ return NULL;
+ }
+ tuplestore_puttupleslot(node->totalTuple, slot);
+ }
+ slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if (tuplestore_gettupleslot(node->totalTuple, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ } else {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
/*
* Get next tuple from subplan, if any.
@@ -156,6 +195,7 @@ ExecLimit(PlanState *pstate)
}
node->subSlot = slot;
node->position++;
+ }
}
else
{
@@ -283,12 +323,16 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
- node->noCount = false;
+ if (node->limitOption == PERCENTAGE)
+ node->count = DatumGetInt64(100 / DatumGetFloat8(val));
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+ }
}
}
else
@@ -374,6 +418,9 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
+ if (node->limitOption == PERCENTAGE)
+ limitstate->totalTuple= tuplestore_begin_heap(true, false, work_mem);
/*
* Initialize result type.
@@ -405,6 +452,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->totalTuple!= NULL)
+ tuplestore_end(node->totalTuple);
}
@@ -424,4 +473,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->totalTuple!= NULL)
+ tuplestore_rescan(node->totalTuple);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 006a3d1772..60e955cbe1 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1136,6 +1136,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3021,6 +3022,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3105,6 +3107,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 133df1b364..5bad7cdbf8 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -974,6 +974,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1048,6 +1049,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 0fde876c77..1a53068fe2 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -900,6 +900,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2099,6 +2100,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2680,6 +2682,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2889,6 +2892,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index ec6f2569ab..1efd54f900 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -278,6 +278,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2320,6 +2321,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 066685c3c7..0a39328234 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2158,7 +2158,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2463,7 +2464,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6487,7 +6489,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6499,6 +6501,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 3e33a17a5b..26d5c287a2 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2140,6 +2140,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 5921e893c1..8a71fef4cc 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3410,6 +3410,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3431,6 +3432,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 5ff6964d51..d0f0f4df59 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1298,10 +1298,11 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
}
/* transform LIMIT */
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1544,10 +1545,11 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_ORDER_BY,
false /* allow SQL92 rules */ );
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1779,10 +1781,11 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
parser_errposition(pstate,
exprLocation(list_nth(qry->targetList, tllen)))));
- qry->limitOffset = transformLimitClause(pstate, limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, limitCount,
+ qry->limitCount = transformLimitClause(pstate, limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c086235b25..f5692f4037 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -164,6 +164,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -387,7 +388,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
+ reloption_list group_clause TriggerFuncArgs select_limit limit_clause
opt_select_limit opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
@@ -449,7 +450,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -662,7 +663,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11183,7 +11184,7 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
@@ -11191,6 +11192,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
list_nth($4, 0), list_nth($4, 1),
+ (list_nth($4, 2)),
NULL,
yyscanner);
$$ = $1;
@@ -11199,6 +11201,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
list_nth($3, 0), list_nth($3, 1),
+ list_nth($3, 2),
NULL,
yyscanner);
$$ = $1;
@@ -11207,7 +11210,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11215,7 +11218,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11223,6 +11226,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
list_nth($5, 0), list_nth($5, 1),
+ list_nth($5, 2),
$1,
yyscanner);
$$ = $2;
@@ -11231,6 +11235,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
list_nth($4, 0), list_nth($4, 1),
+ list_nth($4, 2),
$1,
yyscanner);
$$ = $2;
@@ -11517,20 +11522,20 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause { $$ = list_make3($2, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause limit_clause { $$ = list_make3($1, list_nth($2, 0), list_nth($2, 1)); }
+ | limit_clause { $$ = list_make3(NULL, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause { $$ = list_make3($1, NULL, NULL); }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */ { $$ = list_make3(NULL, NULL, NULL); }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ { $$ = list_make2($2, NULL); }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11548,9 +11553,11 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ { $$ = list_make2($3, makeString("EXACT_NUMBER")); }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ { $$ = list_make2($3, makeString("PERCENTAGE")); }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ { $$ = list_make2(makeIntConst(1, -1), NULL); }
;
offset_clause:
@@ -15417,6 +15424,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCES
@@ -15800,6 +15808,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15838,6 +15847,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 6963922b0e..5dc80ea4a9 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1708,7 +1708,7 @@ transformWhereClause(ParseState *pstate, Node *clause,
* constructName does not affect the semantics, but is used in error messages
*/
Node *
-transformLimitClause(ParseState *pstate, Node *clause,
+transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName)
{
Node *qual;
@@ -1717,8 +1717,10 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
-
- qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
+ if (limitOption == PERCENTAGE && (strcmp(constructName, "LIMIT") == 0))
+ qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
+ else
+ qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
/* LIMIT can't refer to any variables of the current query */
checkExprIsVarFree(pstate, qual, constructName);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index a93bb61bf5..24d440e428 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2288,8 +2288,10 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ Tuplestorestate *totalTuple; /* total number of row outer node return */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 10dac60cd3..7065771adc 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -814,4 +814,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+} LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 27782fed6c..9114a5ddea 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1572,6 +1573,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 6d087c268f..1cdfa706df 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -946,6 +946,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
index 3430061361..f789ee4b79 100644
--- a/src/include/nodes/relation.h
+++ b/src/include/nodes/relation.h
@@ -1737,6 +1737,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index bd905d3328..303501a14c 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -248,6 +248,7 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 5072a46ddd..ef7e71f095 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -66,7 +66,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index b8902d3403..e2ed08afe8 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -300,6 +300,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, RESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h
index 179f3ab3c3..39a74c32d2 100644
--- a/src/include/parser/parse_clause.h
+++ b/src/include/parser/parse_clause.h
@@ -22,7 +22,7 @@ extern int setTargetTable(ParseState *pstate, RangeVar *relation,
extern Node *transformWhereClause(ParseState *pstate, Node *clause,
ParseExprKind exprKind, const char *constructName);
-extern Node *transformLimitClause(ParseState *pstate, Node *clause,
+extern Node *transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName);
extern List *transformGroupClause(ParseState *pstate, List *grouplist,
List **groupingSets,
On 1/9/19 4:43 PM, Surafel Temesgen wrote:
On Wed, Jan 9, 2019 at 5:38 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:It's hard to say what exactly are you doing, because you haven't shared
any code nor examples.okay i attach in progress patch
Yeah, that's what I thought - the patch computes
node->count = DatumGetInt64(100 / DatumGetFloat8(val));
and then always fetches this number of records before emitting the next
row from the tuplestore. That's wrong, as I explained before, because
the distance does change, due to rounding.
See the attached patch, which recomputes the count regularly. I don't
claim the patch is committable or that it has no other bugs, but
hopefully it shows what I meant.
FWIW you don't need to create any slots - the two already created are
enough. You certainly don't need to create the slots within a loop.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
percent-incremental.patchtext/x-patch; name=percent-incremental.patchDownload
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 4db8142afa..8491b7831a 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1397,7 +1397,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1407,7 +1407,8 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
+ with <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage.<literal>ROW</literal>
and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index baa669abe8..87d751bdbe 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -21,6 +21,8 @@
#include "postgres.h"
+#include <math.h>
+
#include "executor/executor.h"
#include "executor/nodeLimit.h"
#include "miscadmin.h"
@@ -44,6 +46,7 @@ ExecLimit(PlanState *pstate)
ScanDirection direction;
TupleTableSlot *slot;
PlanState *outerPlan;
+ slot = node->subSlot;
CHECK_FOR_INTERRUPTS();
@@ -81,7 +84,15 @@ ExecLimit(PlanState *pstate)
/*
* Check for empty window; if so, treat like empty subplan.
*/
- if (node->count <= 0 && !node->noCount)
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (node->percent == 0.0)
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ }
+ else if (node->count <= 0 && !node->noCount)
{
node->lstate = LIMIT_EMPTY;
return NULL;
@@ -122,6 +133,7 @@ ExecLimit(PlanState *pstate)
return NULL;
case LIMIT_INWINDOW:
+
if (ScanDirectionIsForward(direction))
{
/*
@@ -130,6 +142,32 @@ ExecLimit(PlanState *pstate)
* advancing the subplan or the position variable; but change
* the state machine state to record having done so.
*/
+
+ /*
+ * When in percentage mode, we need to see if we can get any
+ * additional rows from the subplan (enough to increase the
+ * node->count value).
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ /* loop until the node->count increments */
+ while (node->position - node->offset >= node->count)
+ {
+ int64 cnt;
+
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ break;
+
+ tuplestore_puttupleslot(node->totalTuple, slot);
+
+ /* plus 1, because the first tuple is returned from LIMIT_RESCAN */
+ cnt = 1 + tuplestore_tuple_count(node->totalTuple);
+
+ node->count = ceil(node->percent * cnt / 100.0);
+ }
+ }
+
if (!node->noCount &&
node->position - node->offset >= node->count)
{
@@ -145,6 +183,20 @@ ExecLimit(PlanState *pstate)
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ while (node->position - node->offset < node->count)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
/*
* Get next tuple from subplan, if any.
*/
@@ -156,6 +208,7 @@ ExecLimit(PlanState *pstate)
}
node->subSlot = slot;
node->position++;
+ }
}
else
{
@@ -278,17 +331,29 @@ recompute_limits(LimitState *node)
/* Interpret NULL count as no count (LIMIT ALL) */
if (isNull)
{
- node->count = 0;
+ node->count = 1;
node->noCount = true;
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
- node->noCount = false;
+ if (node->limitOption == PERCENTAGE)
+ {
+ /*
+ * We expect to return at least one row (unless there
+ * are no rows in the subplan), and we'll update this
+ * count later as we go.
+ */
+ node->count = 0;
+ node->percent = DatumGetFloat8(val);
+ }
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+ }
}
}
else
@@ -374,6 +439,9 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
+ if (node->limitOption == PERCENTAGE)
+ limitstate->totalTuple= tuplestore_begin_heap(true, false, work_mem);
/*
* Initialize result type.
@@ -405,6 +473,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->totalTuple!= NULL)
+ tuplestore_end(node->totalTuple);
}
@@ -424,4 +494,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->totalTuple!= NULL)
+ tuplestore_rescan(node->totalTuple);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 006a3d1772..60e955cbe1 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1136,6 +1136,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3021,6 +3022,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3105,6 +3107,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 133df1b364..5bad7cdbf8 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -974,6 +974,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1048,6 +1049,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 0fde876c77..1a53068fe2 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -900,6 +900,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2099,6 +2100,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2680,6 +2682,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2889,6 +2892,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index ec6f2569ab..1efd54f900 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -278,6 +278,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2320,6 +2321,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 066685c3c7..0a39328234 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2158,7 +2158,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2463,7 +2464,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6487,7 +6489,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6499,6 +6501,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 3e33a17a5b..26d5c287a2 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2140,6 +2140,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 5921e893c1..8a71fef4cc 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3410,6 +3410,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3431,6 +3432,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 5ff6964d51..d0f0f4df59 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1298,10 +1298,11 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
}
/* transform LIMIT */
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1544,10 +1545,11 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_ORDER_BY,
false /* allow SQL92 rules */ );
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1779,10 +1781,11 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
parser_errposition(pstate,
exprLocation(list_nth(qry->targetList, tllen)))));
- qry->limitOffset = transformLimitClause(pstate, limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, limitCount,
+ qry->limitCount = transformLimitClause(pstate, limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c086235b25..f5692f4037 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -164,6 +164,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -387,7 +388,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
+ reloption_list group_clause TriggerFuncArgs select_limit limit_clause
opt_select_limit opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
@@ -449,7 +450,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -662,7 +663,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11183,7 +11184,7 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
@@ -11191,6 +11192,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
list_nth($4, 0), list_nth($4, 1),
+ (list_nth($4, 2)),
NULL,
yyscanner);
$$ = $1;
@@ -11199,6 +11201,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
list_nth($3, 0), list_nth($3, 1),
+ list_nth($3, 2),
NULL,
yyscanner);
$$ = $1;
@@ -11207,7 +11210,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11215,7 +11218,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11223,6 +11226,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
list_nth($5, 0), list_nth($5, 1),
+ list_nth($5, 2),
$1,
yyscanner);
$$ = $2;
@@ -11231,6 +11235,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
list_nth($4, 0), list_nth($4, 1),
+ list_nth($4, 2),
$1,
yyscanner);
$$ = $2;
@@ -11517,20 +11522,20 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause { $$ = list_make3($2, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause limit_clause { $$ = list_make3($1, list_nth($2, 0), list_nth($2, 1)); }
+ | limit_clause { $$ = list_make3(NULL, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause { $$ = list_make3($1, NULL, NULL); }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */ { $$ = list_make3(NULL, NULL, NULL); }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ { $$ = list_make2($2, NULL); }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11548,9 +11553,11 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ { $$ = list_make2($3, makeString("EXACT_NUMBER")); }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ { $$ = list_make2($3, makeString("PERCENTAGE")); }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ { $$ = list_make2(makeIntConst(1, -1), NULL); }
;
offset_clause:
@@ -15417,6 +15424,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCES
@@ -15800,6 +15808,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15838,6 +15847,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 6963922b0e..5dc80ea4a9 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1708,7 +1708,7 @@ transformWhereClause(ParseState *pstate, Node *clause,
* constructName does not affect the semantics, but is used in error messages
*/
Node *
-transformLimitClause(ParseState *pstate, Node *clause,
+transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName)
{
Node *qual;
@@ -1717,8 +1717,10 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
-
- qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
+ if (limitOption == PERCENTAGE && (strcmp(constructName, "LIMIT") == 0))
+ qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
+ else
+ qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
/* LIMIT can't refer to any variables of the current query */
checkExprIsVarFree(pstate, qual, constructName);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index a93bb61bf5..fb60379d31 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2288,8 +2288,11 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ float8 percent; /* percentage */
+ Tuplestorestate *totalTuple; /* total number of row outer node return */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 10dac60cd3..7065771adc 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -814,4 +814,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+} LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 27782fed6c..9114a5ddea 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1572,6 +1573,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 6d087c268f..1cdfa706df 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -946,6 +946,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
index 3430061361..f789ee4b79 100644
--- a/src/include/nodes/relation.h
+++ b/src/include/nodes/relation.h
@@ -1737,6 +1737,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index bd905d3328..303501a14c 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -248,6 +248,7 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 5072a46ddd..ef7e71f095 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -66,7 +66,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index b8902d3403..e2ed08afe8 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -300,6 +300,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, RESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h
index 179f3ab3c3..39a74c32d2 100644
--- a/src/include/parser/parse_clause.h
+++ b/src/include/parser/parse_clause.h
@@ -22,7 +22,7 @@ extern int setTargetTable(ParseState *pstate, RangeVar *relation,
extern Node *transformWhereClause(ParseState *pstate, Node *clause,
ParseExprKind exprKind, const char *constructName);
-extern Node *transformLimitClause(ParseState *pstate, Node *clause,
+extern Node *transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName);
extern List *transformGroupClause(ParseState *pstate, List *grouplist,
List **groupingSets,
On Wed, Jan 9, 2019 at 8:18 PM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:
See the attached patch, which recomputes the count regularly. I don't
claim the patch is committable or that it has no other bugs, but
hopefully it shows what I meant.
I got only one issue it is not work well with cursor
postgres=# START TRANSACTION;
START TRANSACTION
postgres=# create table t as select i from generate_series(1,1000) s(i);
SELECT 1000
postgres=# declare c cursor for select * from t fetch first 5 percent rows
only;
DECLARE CURSOR
postgres=# fetch all in c;
ERROR: trying to store a minimal tuple into wrong type of slot
I am looking at it .
meanwhile i fix row estimation and cost and make create_ordered_paths
creation with no LIMIT consideration in PERCENTAGE case
regards
Surafel
Attachments:
percent-incremental-v2.patchtext/x-patch; charset=US-ASCII; name=percent-incremental-v2.patchDownload
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 4db8142afa..8491b7831a 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1397,7 +1397,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1407,7 +1407,8 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
+ with <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage.<literal>ROW</literal>
and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index baa669abe8..8fc70b1b5f 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -21,6 +21,8 @@
#include "postgres.h"
+#include <math.h>
+
#include "executor/executor.h"
#include "executor/nodeLimit.h"
#include "miscadmin.h"
@@ -44,6 +46,7 @@ ExecLimit(PlanState *pstate)
ScanDirection direction;
TupleTableSlot *slot;
PlanState *outerPlan;
+ slot = node->subSlot;
CHECK_FOR_INTERRUPTS();
@@ -81,7 +84,15 @@ ExecLimit(PlanState *pstate)
/*
* Check for empty window; if so, treat like empty subplan.
*/
- if (node->count <= 0 && !node->noCount)
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (node->percent == 0.0)
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ }
+ else if (node->count <= 0 && !node->noCount)
{
node->lstate = LIMIT_EMPTY;
return NULL;
@@ -122,6 +133,7 @@ ExecLimit(PlanState *pstate)
return NULL;
case LIMIT_INWINDOW:
+
if (ScanDirectionIsForward(direction))
{
/*
@@ -130,6 +142,32 @@ ExecLimit(PlanState *pstate)
* advancing the subplan or the position variable; but change
* the state machine state to record having done so.
*/
+
+ /*
+ * When in percentage mode, we need to see if we can get any
+ * additional rows from the subplan (enough to increase the
+ * node->count value).
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ /* loop until the node->count increments */
+ while (node->position - node->offset >= node->count)
+ {
+ int64 cnt;
+
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ break;
+
+ tuplestore_puttupleslot(node->totalTuple, slot);
+
+ /* plus 1, because the first tuple is returned from LIMIT_RESCAN */
+ cnt = 1 + tuplestore_tuple_count(node->totalTuple);
+
+ node->count = ceil(node->percent * cnt / 100.0);
+ }
+ }
+
if (!node->noCount &&
node->position - node->offset >= node->count)
{
@@ -145,6 +183,20 @@ ExecLimit(PlanState *pstate)
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ while (node->position - node->offset < node->count)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
/*
* Get next tuple from subplan, if any.
*/
@@ -156,6 +208,7 @@ ExecLimit(PlanState *pstate)
}
node->subSlot = slot;
node->position++;
+ }
}
else
{
@@ -278,17 +331,29 @@ recompute_limits(LimitState *node)
/* Interpret NULL count as no count (LIMIT ALL) */
if (isNull)
{
- node->count = 0;
+ node->count = 1;
node->noCount = true;
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
- node->noCount = false;
+ if (node->limitOption == PERCENTAGE)
+ {
+ /*
+ * We expect to return at least one row (unless there
+ * are no rows in the subplan), and we'll update this
+ * count later as we go.
+ */
+ node->count = 0;
+ node->percent = DatumGetFloat8(val);
+ }
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+ }
}
}
else
@@ -311,7 +376,8 @@ recompute_limits(LimitState *node)
* must update the child node anyway, in case this is a rescan and the
* previous time we got a different result.
*/
- ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
+ if (node->limitOption == EXACT_NUMBER)
+ ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
}
/*
@@ -374,6 +440,9 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
+ if (node->limitOption == PERCENTAGE)
+ limitstate->totalTuple= tuplestore_begin_heap(true, false, work_mem);
/*
* Initialize result type.
@@ -405,6 +474,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->totalTuple!= NULL)
+ tuplestore_end(node->totalTuple);
}
@@ -424,4 +495,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->totalTuple!= NULL)
+ tuplestore_rescan(node->totalTuple);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 3eb7e95d64..16afcac059 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1136,6 +1136,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3021,6 +3022,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3105,6 +3107,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 5c4fa7d077..617fd846d6 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -974,6 +974,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1048,6 +1049,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 0fde876c77..1a53068fe2 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -900,6 +900,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2099,6 +2100,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2680,6 +2682,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2889,6 +2892,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index ec6f2569ab..1efd54f900 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -278,6 +278,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2320,6 +2321,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 97d0c28132..0e5eed5807 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2155,7 +2155,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2460,7 +2461,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6325,7 +6327,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6337,6 +6339,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 4465f002c8..146eb7ca28 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2077,12 +2077,20 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
*/
if (parse->sortClause)
{
- current_rel = create_ordered_paths(root,
- current_rel,
- final_target,
- final_target_parallel_safe,
- have_postponed_srfs ? -1.0 :
- limit_tuples);
+ if (parse->limitOption == PERCENTAGE)
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ -1.0);
+ else
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ limit_tuples);
/* Fix things up if final_target contains SRFs */
if (parse->hasTargetSRFs)
adjust_paths_for_srfs(root, current_rel,
@@ -2145,6 +2153,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index b2637d0e89..5cb2c115ed 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3411,6 +3411,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3432,6 +3433,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
@@ -3473,9 +3475,21 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
count_rows = (double) count_est;
else
count_rows = clamp_row_est(subpath->rows * 0.10);
+ if (limitOption == PERCENTAGE)
+ {
+ double per_count = DatumGetFloat8(count_est);
+ count_rows = clamp_row_est((subpath->rows * per_count) / 100);
+ if (subpath->rows > 0)
+ {
+ pathnode->path.startup_cost = (count_rows *
+ subpath->total_cost) / subpath->rows;
+ pathnode->path.total_cost = subpath->total_cost +
+ (count_rows * 0.1);
+ }
+ }
if (count_rows > pathnode->path.rows)
count_rows = pathnode->path.rows;
- if (subpath->rows > 0)
+ if (subpath->rows > 0 && limitOption == EXACT_NUMBER)
pathnode->path.total_cost = pathnode->path.startup_cost +
(subpath->total_cost - subpath->startup_cost)
* count_rows / subpath->rows;
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 8f96558b3d..0e47758eb3 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1300,10 +1300,11 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
}
/* transform LIMIT */
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1548,10 +1549,11 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_ORDER_BY,
false /* allow SQL92 rules */ );
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1783,10 +1785,11 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
parser_errposition(pstate,
exprLocation(list_nth(qry->targetList, tllen)))));
- qry->limitOffset = transformLimitClause(pstate, limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, limitCount,
+ qry->limitCount = transformLimitClause(pstate, limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index d8a3c2d4cc..1abe076fa4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -164,6 +164,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -387,7 +388,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
+ reloption_list group_clause TriggerFuncArgs select_limit limit_clause
opt_select_limit opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
@@ -449,7 +450,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -662,7 +663,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11191,7 +11192,7 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
@@ -11199,6 +11200,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
list_nth($4, 0), list_nth($4, 1),
+ (list_nth($4, 2)),
NULL,
yyscanner);
$$ = $1;
@@ -11207,6 +11209,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
list_nth($3, 0), list_nth($3, 1),
+ list_nth($3, 2),
NULL,
yyscanner);
$$ = $1;
@@ -11215,7 +11218,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11223,7 +11226,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11231,6 +11234,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
list_nth($5, 0), list_nth($5, 1),
+ list_nth($5, 2),
$1,
yyscanner);
$$ = $2;
@@ -11239,6 +11243,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
list_nth($4, 0), list_nth($4, 1),
+ list_nth($4, 2),
$1,
yyscanner);
$$ = $2;
@@ -11525,20 +11530,20 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause { $$ = list_make3($2, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause limit_clause { $$ = list_make3($1, list_nth($2, 0), list_nth($2, 1)); }
+ | limit_clause { $$ = list_make3(NULL, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause { $$ = list_make3($1, NULL, NULL); }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */ { $$ = list_make3(NULL, NULL, NULL); }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ { $$ = list_make2($2, NULL); }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11556,9 +11561,11 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ { $$ = list_make2($3, makeString("EXACT_NUMBER")); }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ { $$ = list_make2($3, makeString("PERCENTAGE")); }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ { $$ = list_make2(makeIntConst(1, -1), NULL); }
;
offset_clause:
@@ -15425,6 +15432,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCES
@@ -15808,6 +15816,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15846,6 +15855,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 8805543da7..5876d092b5 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1708,7 +1708,7 @@ transformWhereClause(ParseState *pstate, Node *clause,
* constructName does not affect the semantics, but is used in error messages
*/
Node *
-transformLimitClause(ParseState *pstate, Node *clause,
+transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName)
{
Node *qual;
@@ -1717,8 +1717,10 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
-
- qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
+ if (limitOption == PERCENTAGE && (strcmp(constructName, "LIMIT") == 0))
+ qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
+ else
+ qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
/* LIMIT can't refer to any variables of the current query */
checkExprIsVarFree(pstate, qual, constructName);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 7cae085177..911f6c4021 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2287,8 +2287,11 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ float8 percent; /* percentage */
+ Tuplestorestate *totalTuple; /* total number of row outer node return */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 10dac60cd3..7065771adc 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -814,4 +814,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+} LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index addc2c2ec7..f861bc4da4 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1572,6 +1573,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 6d087c268f..1cdfa706df 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -946,6 +946,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
index 3430061361..f789ee4b79 100644
--- a/src/include/nodes/relation.h
+++ b/src/include/nodes/relation.h
@@ -1737,6 +1737,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index bd905d3328..303501a14c 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -248,6 +248,7 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index bec0c38617..1932df1517 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -66,7 +66,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index adeb834ce8..ab816083b7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -299,6 +299,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, RESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h
index 179f3ab3c3..39a74c32d2 100644
--- a/src/include/parser/parse_clause.h
+++ b/src/include/parser/parse_clause.h
@@ -22,7 +22,7 @@ extern int setTargetTable(ParseState *pstate, RangeVar *relation,
extern Node *transformWhereClause(ParseState *pstate, Node *clause,
ParseExprKind exprKind, const char *constructName);
-extern Node *transformLimitClause(ParseState *pstate, Node *clause,
+extern Node *transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName);
extern List *transformGroupClause(ParseState *pstate, List *grouplist,
List **groupingSets,
diff --git a/src/test/modules/test_ddl_deparse/expected/create_table.out b/src/test/modules/test_ddl_deparse/expected/create_table.out
index 2d7dfd533e..eb04e36aac 100644
--- a/src/test/modules/test_ddl_deparse/expected/create_table.out
+++ b/src/test/modules/test_ddl_deparse/expected/create_table.out
@@ -105,7 +105,7 @@ CREATE TABLE student (
) INHERITS (person);
NOTICE: DDL test: type simple, tag CREATE TABLE
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "id"
NOTICE: merging multiple inherited definitions of column "name"
diff --git a/src/test/modules/test_ddl_deparse/sql/create_table.sql b/src/test/modules/test_ddl_deparse/sql/create_table.sql
index dd3a908638..f158dd4296 100644
--- a/src/test/modules/test_ddl_deparse/sql/create_table.sql
+++ b/src/test/modules/test_ddl_deparse/sql/create_table.sql
@@ -94,7 +94,7 @@ CREATE TABLE student (
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 7e52c27e3f..a30bfa919c 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -79,7 +79,7 @@ CREATE TABLE student (
gpa float8
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "name"
NOTICE: merging multiple inherited definitions of column "age"
diff --git a/src/test/regress/sql/create_table.sql b/src/test/regress/sql/create_table.sql
index a2cae9663c..ed94b78978 100644
--- a/src/test/regress/sql/create_table.sql
+++ b/src/test/regress/sql/create_table.sql
@@ -92,7 +92,7 @@ CREATE TABLE student (
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
On 1/24/19 10:57 AM, Surafel Temesgen wrote:
On Wed, Jan 9, 2019 at 8:18 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:
See the attached patch, which recomputes the count regularly. I don't
claim the patch is committable or that it has no other bugs, but
hopefully it shows what I meant.I got only one issue it is not work well with cursor
postgres=# START TRANSACTION;
START TRANSACTION
postgres=# create table t as select i from generate_series(1,1000) s(i);
SELECT 1000
postgres=# declare c cursor for select * from t fetch first 5 percent
rows only;DECLARE CURSOR
postgres=# fetch all in c;
ERROR: trying to store a minimal tuple into wrong type of slot
I am looking at it .
OK. Does that mean you agree the incremental approach is reasonable?
meanwhile i fix row estimation and cost and make create_ordered_paths
creation with no LIMIT consideration in PERCENTAGE case
I haven't reviewed the costing code yet, but linear interpolation seems
reasonable in principle. I find the create_limit_path changes somewhat
sloppy and difficult to comprehend (particularly without comments).
For example, the fact that the code computes the total cost based on
count_rows, which may be tweaked later seems suspicious. I mean, the doe
now does this:
if (limitOption = PERCENTAGE)
{
... update count_rows
... compute total_cost
}
if (count_rows > pathnode->path.rows)
count_rows = pathnode->path.rows;
... compute total_cost for the EXACT_NUMBER case
pathnode->path.rows = count_rows;
But I need to do think about this a bit more.
FWIW I see there are no regression tests for the PERCENT option.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Mon, Jan 28, 2019 at 1:28 AM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:
OK. Does that mean you agree the incremental approach is reasonable?
there are no noticeable performance difference but i love previous
approach more regarding cursor operation it fetch tuple forward and
backward from tuplestore only but in incremental approach we have to
re execute outer node in every forward and backward fetching operation
regards
Surafel
here is a rebased version of previous patch with planner
comment included
regards
Surafel
On Wed, Jan 30, 2019 at 9:07 AM Surafel Temesgen <surafel3000@gmail.com>
wrote:
Show quoted text
On Mon, Jan 28, 2019 at 1:28 AM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:OK. Does that mean you agree the incremental approach is reasonable?
there are no noticeable performance difference but i love previous
approach more regarding cursor operation it fetch tuple forward and
backward from tuplestore only but in incremental approach we have to
re execute outer node in every forward and backward fetching operationregards
Surafel
Attachments:
fetch-with-percent-v8.patchtext/x-patch; charset=US-ASCII; name=fetch-with-percent-v8.patchDownload
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 4db8142afa..8491b7831a 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1397,7 +1397,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1407,7 +1407,8 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
+ with <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage.<literal>ROW</literal>
and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index baa669abe8..5ba895362d 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -43,6 +43,7 @@ ExecLimit(PlanState *pstate)
LimitState *node = castNode(LimitState, pstate);
ScanDirection direction;
TupleTableSlot *slot;
+ TupleDesc tupleDescriptor;
PlanState *outerPlan;
CHECK_FOR_INTERRUPTS();
@@ -52,6 +53,8 @@ ExecLimit(PlanState *pstate)
*/
direction = node->ps.state->es_direction;
outerPlan = outerPlanState(node);
+ slot = node->subSlot;
+ tupleDescriptor = node->ps.ps_ResultTupleDesc;
/*
* The main logic is a simple state machine.
@@ -60,6 +63,23 @@ ExecLimit(PlanState *pstate)
{
case LIMIT_INITIAL:
+ if (node->limitOption == PERCENTAGE)
+ {
+
+ /*
+ * Find all rows the plan will return.
+ */
+ for (;;)
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ break;
+ }
+ tuplestore_puttupleslot(node->totalTuple, slot);
+ }
+ }
+
/*
* First call for this node, so compute limit/offset. (We can't do
* this any earlier, because parameters from upper nodes will not
@@ -87,24 +107,46 @@ ExecLimit(PlanState *pstate)
return NULL;
}
- /*
- * Fetch rows from subplan until we reach position > offset.
- */
- for (;;)
+ if (node->limitOption == PERCENTAGE)
{
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
+ for (;;)
{
- /*
- * The subplan returns too few tuples for us to produce
- * any output at all.
- */
- node->lstate = LIMIT_EMPTY;
- return NULL;
+ slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if (!tuplestore_gettupleslot(node->totalTuple, true, true, slot))
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ else
+ {
+ node->subSlot = slot;
+ if (++node->position > node->offset)
+ break;
+ }
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Fetch rows from subplan until we reach position > offset.
+ */
+ for (;;)
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ /*
+ * The subplan returns too few tuples for us to produce
+ * any output at all.
+ */
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ node->subSlot = slot;
+ if (++node->position > node->offset)
+ break;
}
- node->subSlot = slot;
- if (++node->position > node->offset)
- break;
}
/*
@@ -144,18 +186,35 @@ ExecLimit(PlanState *pstate)
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if (tuplestore_gettupleslot(node->totalTuple, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
- /*
- * Get next tuple from subplan, if any.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- {
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ node->subSlot = slot;
+ node->position++;
}
- node->subSlot = slot;
- node->position++;
}
else
{
@@ -168,15 +227,29 @@ ExecLimit(PlanState *pstate)
node->lstate = LIMIT_WINDOWSTART;
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, false, false, slot))
+ {
+ node->subSlot = slot;
+ node->position--;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
- /*
- * Get previous tuple from subplan; there should be one!
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Get previous tuple from subplan; there should be one!
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->position--;
+ }
}
break;
@@ -184,15 +257,29 @@ ExecLimit(PlanState *pstate)
if (ScanDirectionIsForward(direction))
return NULL;
- /*
- * Backing up from subplan EOF, so re-fetch previous tuple; there
- * should be one! Note previous tuple must be in window.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->lstate = LIMIT_INWINDOW;
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->totalTuple, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Backing up from subplan EOF, so re-fetch previous tuple; there
+ * should be one! Note previous tuple must be in window.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
/* position does not change 'cause we didn't advance it before */
break;
@@ -283,11 +370,16 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
+ if (node->limitOption == PERCENTAGE)
+ node->count = DatumGetInt64((DatumGetFloat8(val) * tuplestore_tuple_count(node->totalTuple)) / 100);
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+ }
node->noCount = false;
}
}
@@ -374,6 +466,10 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
+
+ if (node->limitOption == PERCENTAGE)
+ limitstate->totalTuple= tuplestore_begin_heap(true, false, work_mem);
/*
* Initialize result type.
@@ -405,6 +501,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->totalTuple!= NULL)
+ tuplestore_end(node->totalTuple);
}
@@ -424,4 +522,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->totalTuple!= NULL)
+ tuplestore_rescan(node->totalTuple);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 807393dfaa..d89b7435ca 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1136,6 +1136,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3021,6 +3022,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3105,6 +3107,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index a397de155e..14d09c611c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -974,6 +974,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1048,6 +1049,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 9d44e3e4c6..38a1d646cb 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -900,6 +900,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2099,6 +2100,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2679,6 +2681,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2888,6 +2891,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 43491e297b..a3eb8c0490 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -278,6 +278,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2323,6 +2324,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 1b4f7db649..5e60e78596 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2185,7 +2185,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2490,7 +2491,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6393,7 +6395,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6405,6 +6407,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index b2239728cf..cd2b9091db 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2099,12 +2099,22 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
*/
if (parse->sortClause)
{
- current_rel = create_ordered_paths(root,
- current_rel,
- final_target,
- final_target_parallel_safe,
- have_postponed_srfs ? -1.0 :
- limit_tuples);
+
+ /* In PERCENTAGE option there are no bound on the number of output tuples */
+ if (parse->limitOption == PERCENTAGE)
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ -1.0);
+ else
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ limit_tuples);
/* Fix things up if final_target contains SRFs */
if (parse->hasTargetSRFs)
adjust_paths_for_srfs(root, current_rel,
@@ -2167,6 +2177,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index b57de6b4c6..ded35bb5b0 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3441,6 +3441,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3462,6 +3463,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
@@ -3503,9 +3505,21 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
count_rows = (double) count_est;
else
count_rows = clamp_row_est(subpath->rows * 0.10);
+ if (limitOption == PERCENTAGE)
+ {
+ double per_count = DatumGetFloat8(count_est);
+ count_rows = clamp_row_est((subpath->rows * per_count) / 100);
+ if (subpath->rows > 0)
+ {
+ pathnode->path.startup_cost = (count_rows *
+ subpath->total_cost) / subpath->rows;
+ pathnode->path.total_cost = subpath->total_cost +
+ (count_rows * 0.1);
+ }
+ }
if (count_rows > pathnode->path.rows)
count_rows = pathnode->path.rows;
- if (subpath->rows > 0)
+ if (subpath->rows > 0 && limitOption == EXACT_NUMBER)
pathnode->path.total_cost = pathnode->path.startup_cost +
(subpath->total_cost - subpath->startup_cost)
* count_rows / subpath->rows;
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 125ee5d84b..f7eee8a034 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1300,10 +1300,11 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
}
/* transform LIMIT */
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1548,10 +1549,11 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_ORDER_BY,
false /* allow SQL92 rules */ );
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1783,10 +1785,11 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
parser_errposition(pstate,
exprLocation(list_nth(qry->targetList, tllen)))));
- qry->limitOffset = transformLimitClause(pstate, limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, limitCount,
+ qry->limitCount = transformLimitClause(pstate, limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c1faf4152c..68eedaca4d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -164,6 +164,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -387,7 +388,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
+ reloption_list group_clause TriggerFuncArgs select_limit limit_clause
opt_select_limit opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
@@ -449,7 +450,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -661,7 +662,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11137,7 +11138,7 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
@@ -11145,6 +11146,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
list_nth($4, 0), list_nth($4, 1),
+ (list_nth($4, 2)),
NULL,
yyscanner);
$$ = $1;
@@ -11153,6 +11155,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
list_nth($3, 0), list_nth($3, 1),
+ list_nth($3, 2),
NULL,
yyscanner);
$$ = $1;
@@ -11161,7 +11164,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11169,7 +11172,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11177,6 +11180,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
list_nth($5, 0), list_nth($5, 1),
+ list_nth($5, 2),
$1,
yyscanner);
$$ = $2;
@@ -11185,6 +11189,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
list_nth($4, 0), list_nth($4, 1),
+ list_nth($4, 2),
$1,
yyscanner);
$$ = $2;
@@ -11471,20 +11476,20 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause { $$ = list_make3($2, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause limit_clause { $$ = list_make3($1, list_nth($2, 0), list_nth($2, 1)); }
+ | limit_clause { $$ = list_make3(NULL, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause { $$ = list_make3($1, NULL, NULL); }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */ { $$ = list_make3(NULL, NULL, NULL); }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ { $$ = list_make2($2, NULL); }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11502,9 +11507,11 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ { $$ = list_make2($3, makeString("EXACT_NUMBER")); }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ { $$ = list_make2($3, makeString("PERCENTAGE")); }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ { $$ = list_make2(makeIntConst(1, -1), NULL); }
;
offset_clause:
@@ -15371,6 +15378,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCES
@@ -15754,6 +15762,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15792,6 +15801,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index c6ce1011e2..6cb5a5dbc4 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1707,7 +1707,7 @@ transformWhereClause(ParseState *pstate, Node *clause,
* constructName does not affect the semantics, but is used in error messages
*/
Node *
-transformLimitClause(ParseState *pstate, Node *clause,
+transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName)
{
Node *qual;
@@ -1716,8 +1716,10 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
-
- qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
+ if (limitOption == PERCENTAGE && (strcmp(constructName, "LIMIT") == 0))
+ qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
+ else
+ qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
/* LIMIT can't refer to any variables of the current query */
checkExprIsVarFree(pstate, qual, constructName);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 3b789ee7cf..220f5738e3 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2287,8 +2287,10 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ Tuplestorestate *totalTuple; /* total number of row outer node return */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index fbe2dc14a7..956739d9f3 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -814,4 +814,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+} LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 4ec8a83541..95e1ff9127 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1575,6 +1576,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index d3c477a542..b8b880898e 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1742,6 +1742,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 6d087c268f..1cdfa706df 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -946,6 +946,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index d0c8f99d0a..9e6a41618b 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -252,6 +252,7 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 3bbdb5e2f7..6f21cd076a 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -56,7 +56,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index adeb834ce8..ab816083b7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -299,6 +299,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, RESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h
index 179f3ab3c3..39a74c32d2 100644
--- a/src/include/parser/parse_clause.h
+++ b/src/include/parser/parse_clause.h
@@ -22,7 +22,7 @@ extern int setTargetTable(ParseState *pstate, RangeVar *relation,
extern Node *transformWhereClause(ParseState *pstate, Node *clause,
ParseExprKind exprKind, const char *constructName);
-extern Node *transformLimitClause(ParseState *pstate, Node *clause,
+extern Node *transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName);
extern List *transformGroupClause(ParseState *pstate, List *grouplist,
List **groupingSets,
diff --git a/src/test/modules/test_ddl_deparse/expected/create_table.out b/src/test/modules/test_ddl_deparse/expected/create_table.out
index 2d7dfd533e..eb04e36aac 100644
--- a/src/test/modules/test_ddl_deparse/expected/create_table.out
+++ b/src/test/modules/test_ddl_deparse/expected/create_table.out
@@ -105,7 +105,7 @@ CREATE TABLE student (
) INHERITS (person);
NOTICE: DDL test: type simple, tag CREATE TABLE
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "id"
NOTICE: merging multiple inherited definitions of column "name"
diff --git a/src/test/modules/test_ddl_deparse/sql/create_table.sql b/src/test/modules/test_ddl_deparse/sql/create_table.sql
index dd3a908638..f158dd4296 100644
--- a/src/test/modules/test_ddl_deparse/sql/create_table.sql
+++ b/src/test/modules/test_ddl_deparse/sql/create_table.sql
@@ -94,7 +94,7 @@ CREATE TABLE student (
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 100fa53ab0..5400127067 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -79,7 +79,7 @@ CREATE TABLE student (
gpa float8
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "name"
NOTICE: merging multiple inherited definitions of column "age"
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index c18f547cbd..fcadc92f5d 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -108,6 +108,61 @@ SELECT ''::text AS five, unique1, unique2, stringu1
| 904 | 793 | UIAAAA
(5 rows)
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 51 | 76 | ZBAAAA
+ | 52 | 985 | ACAAAA
+ | 53 | 196 | BCAAAA
+ | 54 | 356 | CCAAAA
+ | 55 | 627 | DCAAAA
+ | 56 | 54 | ECAAAA
+ | 57 | 942 | FCAAAA
+ | 58 | 114 | GCAAAA
+ | 59 | 593 | HCAAAA
+(9 rows)
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 61 | 560 | JCAAAA
+(1 row)
+
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+ three | unique1 | unique2 | stringu1
+-------+---------+---------+----------
+ | 121 | 700 | REAAAA
+ | 122 | 519 | SEAAAA
+ | 123 | 777 | TEAAAA
+ | 124 | 503 | UEAAAA
+ | 125 | 849 | VEAAAA
+ | 126 | 330 | WEAAAA
+ | 127 | 511 | XEAAAA
+ | 128 | 721 | YEAAAA
+(8 rows)
+
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+ eleven | unique1 | unique2 | stringu1
+--------+---------+---------+----------
+ | 10 | 520 | KAAAAA
+ | 9 | 49 | JAAAAA
+ | 8 | 653 | IAAAAA
+ | 7 | 647 | HAAAAA
+ | 6 | 978 | GAAAAA
+(5 rows)
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -286,6 +341,43 @@ fetch all in c4;
----+----
(0 rows)
+declare c6 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c6;
+ q1 | q2
+-----+------------------
+ 123 | 456
+ 123 | 4567890123456789
+(2 rows)
+
+fetch 1 in c6;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c6;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+(1 row)
+
+fetch backward all in c6;
+ q1 | q2
+-----+-----
+ 123 | 456
+(1 row)
+
+fetch backward 1 in c6;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c6;
+ q1 | q2
+-----+------------------
+ 123 | 456
+ 123 | 4567890123456789
+(2 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
@@ -503,3 +595,19 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
45020 | 45020
(3 rows)
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
+ s1 | s2
+-------+-------
+ 45000 | 45000
+ 45010 | 45010
+ 45020 | 45020
+ 45030 | 45030
+ 45040 | 45040
+ 45050 | 45050
+ 45060 | 45060
+ 45070 | 45070
+ 45080 | 45080
+ 45090 | 45090
+(10 rows)
+
diff --git a/src/test/regress/sql/create_table.sql b/src/test/regress/sql/create_table.sql
index 22a3d90101..c5b84ab796 100644
--- a/src/test/regress/sql/create_table.sql
+++ b/src/test/regress/sql/create_table.sql
@@ -92,7 +92,7 @@ CREATE TABLE student (
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index 2a313d80ca..6609f8eb21 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -30,6 +30,24 @@ SELECT ''::text AS five, unique1, unique2, stringu1
SELECT ''::text AS five, unique1, unique2, stringu1
FROM onek
ORDER BY unique1 LIMIT 5 OFFSET 900;
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
@@ -38,7 +56,6 @@ select * from int8_tbl offset (case when random() < 0.5 then null::bigint end);
-- Test assorted cases involving backwards fetch from a LIMIT plan node
begin;
-
declare c1 cursor for select * from int8_tbl limit 10;
fetch all in c1;
fetch 1 in c1;
@@ -71,6 +88,15 @@ fetch backward all in c4;
fetch backward 1 in c4;
fetch all in c4;
+
+declare c6 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c6;
+fetch 1 in c6;
+fetch backward 1 in c6;
+fetch backward all in c6;
+fetch backward 1 in c6;
+fetch all in c6;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
@@ -141,3 +167,6 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
from tenk1 group by thousand order by thousand limit 3;
+
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
On 1/30/19 7:07 AM, Surafel Temesgen wrote:
On Mon, Jan 28, 2019 at 1:28 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:OK. Does that mean you agree the incremental approach is reasonable?
there are no noticeable performance difference but i love previous
approach more regarding cursor operation it fetch tuple forward and
backward from tuplestore only but in incremental approach we have to
re execute outer node in every forward and backward fetching operation
I'm not sure I understand - are you saying every time the user does a
FETCH, we have to run the outer plan from scratch? I don't see why would
that be necessary? And if it is, how come there's no noticeable
performance difference?
Can you share a patch implementing the incremental approach, and a query
demonstrating the issue?
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 2/1/19 12:10 PM, Surafel Temesgen wrote:
here is a rebased version of previous patch with planner
comment included
It's really hard to say which comment is that ..
FWIW I find the changes in nodeLimit lacking sufficient comments. The
comments present are somewhat obvious - what we need are comments
explaining why things happen. For example the LIMIT_INITIAL now includes
this chunk of code:
case LIMIT_INITIAL:
if (node->limitOption == PERCENTAGE)
{
/*
* Find all rows the plan will return.
*/
for (;;)
{
slot = ExecProcNode(outerPlan);
if (TupIsNull(slot))
{
break;
}
tuplestore_puttupleslot(node->totalTuple, slot);
}
}
Ignoring the fact that the comment is incorrectly indented, it states a
rather obvious fact - that we fetch all rows from a plan and stash them
into a tuplestore. What is missing is comment explaining why we need to
do that.
This applies to other changes in nodeLimit too, and elsewhere.
Another detail is that we generally leave a free line before "if", i.e.
instead of
}
if (node->limitOption == PERCENTAGE)
{
it should be
}
if (node->limitOption == PERCENTAGE)
{
because it's fairly easy to misread as "else if". Even better, there
should be a comment before the "if" explaining what the branch does.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Sun, Feb 10, 2019 at 2:22 AM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:
I'm not sure I understand - are you saying every time the user does a
FETCH, we have to run the outer plan from scratch? I don't see why would
that be necessary? And if it is, how come there's no noticeable
performance difference?Can you share a patch implementing the incremental approach, and a query
demonstrating the issue?
I didn't implement it but its obvious that it doesn't work similarly with
previous approach.
We need different implementation and my plan was to use tuplestore per call
and clear
it after returning tuple but I see that the plan will not go far because
mainly the last returned
slot is not the last slot we get from outerPlan execution
regards
Surafel
On 2/23/19 8:53 AM, Surafel Temesgen wrote:
On Sun, Feb 10, 2019 at 2:22 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:
I'm not sure I understand - are you saying every time the user does a
FETCH, we have to run the outer plan from scratch? I don't see why would
that be necessary? And if it is, how come there's no noticeable
performance difference?Can you share a patch implementing the incremental approach, and a query
demonstrating the issue?I didn't implement it but its obvious that it doesn't work similarly
with previous approach.
Sure, but that's hardly a sufficient argument for the current approach.
We need different implementation and my plan was to use tuplestore per
call and clearit after returning tuple but I see that the plan will not go far because
mainly the last returnedslot is not the last slot we get from outerPlan execution
I'm sorry, I still don't understand what the supposed problem is. I
don't think it's all that different from what nodeMaterial.c does, for
example.
As I explained before, having to execute the outer plan till completion
before returning any tuples is an issue. So either it needs fixing or an
explanation why it's not an issue.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Sun, Feb 24, 2019 at 12:27 AM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:
I'm sorry, I still don't understand what the supposed problem is. I
don't think it's all that different from what nodeMaterial.c does, for
example.
sorry for the noise .Attache is complete patch for incremental approach
regards
Surafel
Attachments:
percent-incremental-v2.patchtext/x-patch; charset=US-ASCII; name=percent-incremental-v2.patchDownload
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 06d611b64c..e3ce4d7e36 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1430,7 +1430,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1440,7 +1440,8 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
+ with <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage.<literal>ROW</literal>
and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index baa669abe8..ca957901e3 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -21,6 +21,8 @@
#include "postgres.h"
+#include <math.h>
+
#include "executor/executor.h"
#include "executor/nodeLimit.h"
#include "miscadmin.h"
@@ -44,6 +46,9 @@ ExecLimit(PlanState *pstate)
ScanDirection direction;
TupleTableSlot *slot;
PlanState *outerPlan;
+ TupleDesc tupleDescriptor;
+ slot = node->subSlot;
+ tupleDescriptor = node->ps.ps_ResultTupleDesc;
CHECK_FOR_INTERRUPTS();
@@ -81,7 +86,15 @@ ExecLimit(PlanState *pstate)
/*
* Check for empty window; if so, treat like empty subplan.
*/
- if (node->count <= 0 && !node->noCount)
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (node->percent == 0.0)
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ }
+ else if (node->count <= 0 && !node->noCount)
{
node->lstate = LIMIT_EMPTY;
return NULL;
@@ -107,6 +120,15 @@ ExecLimit(PlanState *pstate)
break;
}
+ /*
+ * We may needed this tuple in backward scan so put it into tuplestore.
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ tuplestore_puttupleslot(node->tuple_store, slot);
+ tuplestore_advance(node->tuple_store, true);
+ }
+
/*
* Okay, we have the first tuple of the window.
*/
@@ -130,6 +152,72 @@ ExecLimit(PlanState *pstate)
* advancing the subplan or the position variable; but change
* the state machine state to record having done so.
*/
+
+ /*
+ * In case of coming back from backward scan the tuple is already
+ * in tuple store.
+ */
+ if (node->limitOption == PERCENTAGE && node->backwardPosition > 0)
+ {
+ slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if (tuplestore_gettupleslot(node->tuple_store, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ node->backwardPosition--;
+ return slot;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+
+ /*
+ * In PERCENTAGE case no need of executing outerPlan multiple times.
+ */
+ if (node->limitOption == PERCENTAGE && node->reachEnd)
+ {
+ node->lstate = LIMIT_WINDOWEND;
+
+ /*
+ * If we know we won't need to back up, we can release
+ * resources at this point.
+ */
+ if (!(node->ps.state->es_top_eflags & EXEC_FLAG_BACKWARD))
+ (void) ExecShutdownNode(outerPlan);
+
+ return NULL;
+ }
+
+ /*
+ * When in percentage mode, we need to see if we can get any
+ * additional rows from the subplan (enough to increase the
+ * node->count value).
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ /* loop until the node->count increments */
+ while (node->position - node->offset >= node->count)
+ {
+ int64 cnt;
+
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->reachEnd = true;
+ break;
+ }
+
+ tuplestore_puttupleslot(node->tuple_store, slot);
+
+ cnt = tuplestore_tuple_count(node->tuple_store);
+
+ node->count = ceil(node->percent * cnt / 100.0);
+ }
+ }
+
if (!node->noCount &&
node->position - node->offset >= node->count)
{
@@ -145,6 +233,21 @@ ExecLimit(PlanState *pstate)
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ while (node->position - node->offset < node->count)
+ {
+ slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if (tuplestore_gettupleslot(node->tuple_store, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
/*
* Get next tuple from subplan, if any.
*/
@@ -156,6 +259,7 @@ ExecLimit(PlanState *pstate)
}
node->subSlot = slot;
node->position++;
+ }
}
else
{
@@ -169,6 +273,21 @@ ExecLimit(PlanState *pstate)
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if (tuplestore_gettupleslot(node->tuple_store, false, false, slot))
+ {
+ node->subSlot = slot;
+ node->position--;
+ node->backwardPosition++;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
/*
* Get previous tuple from subplan; there should be one!
*/
@@ -177,6 +296,7 @@ ExecLimit(PlanState *pstate)
elog(ERROR, "LIMIT subplan failed to run backwards");
node->subSlot = slot;
node->position--;
+ }
}
break;
@@ -184,6 +304,19 @@ ExecLimit(PlanState *pstate)
if (ScanDirectionIsForward(direction))
return NULL;
+ if (node->limitOption == PERCENTAGE)
+ {
+ slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if (tuplestore_gettupleslot(node->tuple_store, false, false, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
/*
* Backing up from subplan EOF, so re-fetch previous tuple; there
* should be one! Note previous tuple must be in window.
@@ -194,6 +327,7 @@ ExecLimit(PlanState *pstate)
node->subSlot = slot;
node->lstate = LIMIT_INWINDOW;
/* position does not change 'cause we didn't advance it before */
+ }
break;
case LIMIT_WINDOWEND:
@@ -278,17 +412,29 @@ recompute_limits(LimitState *node)
/* Interpret NULL count as no count (LIMIT ALL) */
if (isNull)
{
- node->count = 0;
+ node->count = 1;
node->noCount = true;
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
- node->noCount = false;
+ if (node->limitOption == PERCENTAGE)
+ {
+ /*
+ * We expect to return at least one row (unless there
+ * are no rows in the subplan), and we'll update this
+ * count later as we go.
+ */
+ node->count = 0;
+ node->percent = DatumGetFloat8(val);
+ }
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+ }
}
}
else
@@ -299,8 +445,10 @@ recompute_limits(LimitState *node)
}
/* Reset position to start-of-scan */
- node->position = 0;
+ node->position = 0;;
node->subSlot = NULL;
+ node->reachEnd = false;
+ node->backwardPosition = 0;
/* Set state-machine state */
node->lstate = LIMIT_RESCAN;
@@ -309,9 +457,11 @@ recompute_limits(LimitState *node)
* Notify child node about limit. Note: think not to "optimize" by
* skipping ExecSetTupleBound if compute_tuples_needed returns < 0. We
* must update the child node anyway, in case this is a rescan and the
- * previous time we got a different result.
+ * previous time we got a different result.In PERCENTAGE option there are
+ * no bound on the number of output tuples */
*/
- ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
+ if (node->limitOption != PERCENTAGE)
+ ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
}
/*
@@ -374,6 +524,9 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
+ if (node->limitOption == PERCENTAGE)
+ limitstate->tuple_store= tuplestore_begin_heap(true, false, work_mem);
/*
* Initialize result type.
@@ -405,6 +558,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->tuple_store!= NULL)
+ tuplestore_end(node->tuple_store);
}
@@ -424,4 +579,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->tuple_store!= NULL)
+ tuplestore_rescan(node->tuple_store);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index e15724bb0e..dc0fe7b693 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1136,6 +1136,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3022,6 +3023,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3106,6 +3108,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 31499eb798..146984cd44 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -974,6 +974,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1048,6 +1049,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 65302fe65b..62fc11582d 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -900,6 +900,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2097,6 +2098,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2689,6 +2691,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2898,6 +2901,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 5aa42242a9..f93f543dc9 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -278,6 +278,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2324,6 +2325,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 236f506cfb..a2fc9f25f6 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2190,7 +2190,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2495,7 +2496,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6359,7 +6361,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6371,6 +6373,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index bc81535905..c5a4ae7008 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2122,12 +2122,22 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
*/
if (parse->sortClause)
{
- current_rel = create_ordered_paths(root,
- current_rel,
- final_target,
- final_target_parallel_safe,
- have_postponed_srfs ? -1.0 :
- limit_tuples);
+
+ /* In PERCENTAGE option there are no bound on the number of output tuples */
+ if (parse->limitOption == PERCENTAGE)
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ -1.0);
+ else
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ limit_tuples);
/* Fix things up if final_target contains SRFs */
if (parse->hasTargetSRFs)
adjust_paths_for_srfs(root, current_rel,
@@ -2190,6 +2200,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 169e51e792..09b9c936a2 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3526,6 +3526,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3547,6 +3548,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
@@ -3588,6 +3590,18 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
count_rows = (double) count_est;
else
count_rows = clamp_row_est(subpath->rows * 0.10);
+ if (limitOption == PERCENTAGE)
+ {
+ double per_count = DatumGetFloat8(count_est);
+ count_rows = clamp_row_est((subpath->rows * per_count) / 100);
+ if (subpath->rows > 0)
+ {
+ pathnode->path.startup_cost = (count_rows *
+ subpath->total_cost) / subpath->rows;
+ pathnode->path.total_cost = subpath->total_cost +
+ (count_rows * 0.1);
+ }
+ }
if (count_rows > pathnode->path.rows)
count_rows = pathnode->path.rows;
if (subpath->rows > 0)
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index e3544efb6f..3894fbd02b 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1301,10 +1301,11 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
}
/* transform LIMIT */
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1549,10 +1550,11 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_ORDER_BY,
false /* allow SQL92 rules */ );
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1784,10 +1786,11 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
parser_errposition(pstate,
exprLocation(list_nth(qry->targetList, tllen)))));
- qry->limitOffset = transformLimitClause(pstate, limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, limitCount,
+ qry->limitCount = transformLimitClause(pstate, limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 0279013120..80e5aafe91 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -164,6 +164,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -387,7 +388,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
+ reloption_list group_clause TriggerFuncArgs select_limit limit_clause
opt_select_limit opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
@@ -449,7 +450,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -661,7 +662,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11161,7 +11162,7 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
@@ -11169,6 +11170,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
list_nth($4, 0), list_nth($4, 1),
+ (list_nth($4, 2)),
NULL,
yyscanner);
$$ = $1;
@@ -11177,6 +11179,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
list_nth($3, 0), list_nth($3, 1),
+ list_nth($3, 2),
NULL,
yyscanner);
$$ = $1;
@@ -11185,7 +11188,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11193,7 +11196,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11201,6 +11204,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
list_nth($5, 0), list_nth($5, 1),
+ list_nth($5, 2),
$1,
yyscanner);
$$ = $2;
@@ -11209,6 +11213,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
list_nth($4, 0), list_nth($4, 1),
+ list_nth($4, 2),
$1,
yyscanner);
$$ = $2;
@@ -11502,20 +11507,20 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause { $$ = list_make3($2, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause limit_clause { $$ = list_make3($1, list_nth($2, 0), list_nth($2, 1)); }
+ | limit_clause { $$ = list_make3(NULL, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause { $$ = list_make3($1, NULL, NULL); }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */ { $$ = list_make3(NULL, NULL, NULL); }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ { $$ = list_make2($2, NULL); }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11533,9 +11538,11 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ { $$ = list_make2($3, makeString("EXACT_NUMBER")); }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ { $$ = list_make2($3, makeString("PERCENTAGE")); }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ { $$ = list_make2(makeIntConst(1, -1), NULL); }
;
offset_clause:
@@ -15403,6 +15410,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCES
@@ -15786,6 +15794,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15824,6 +15833,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index c6ce1011e2..6cb5a5dbc4 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1707,7 +1707,7 @@ transformWhereClause(ParseState *pstate, Node *clause,
* constructName does not affect the semantics, but is used in error messages
*/
Node *
-transformLimitClause(ParseState *pstate, Node *clause,
+transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName)
{
Node *qual;
@@ -1716,8 +1716,10 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
-
- qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
+ if (limitOption == PERCENTAGE && (strcmp(constructName, "LIMIT") == 0))
+ qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
+ else
+ qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
/* LIMIT can't refer to any variables of the current query */
checkExprIsVarFree(pstate, qual, constructName);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 09f8217c80..e906396d75 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2289,8 +2289,13 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ float8 percent; /* percentage */
+ int64 backwardPosition; /* the number of tuple returned in backward scan*/
+ bool reachEnd; /* if true, outerPlan execute until the end */
+ Tuplestorestate *tuple_store; /* holds the returned tuple */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index f9389257c6..094c6ee3bb 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -820,4 +820,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+} LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index a7e859dc90..6eb3fe922d 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1583,6 +1584,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index a008ae07da..fe0fc2f975 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1775,6 +1775,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 6d087c268f..1cdfa706df 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -946,6 +946,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 574bb85b50..fe593170d3 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -264,6 +264,7 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 3bbdb5e2f7..6f21cd076a 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -56,7 +56,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index f05444008c..0a80bb9b76 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -299,6 +299,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, RESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h
index 179f3ab3c3..39a74c32d2 100644
--- a/src/include/parser/parse_clause.h
+++ b/src/include/parser/parse_clause.h
@@ -22,7 +22,7 @@ extern int setTargetTable(ParseState *pstate, RangeVar *relation,
extern Node *transformWhereClause(ParseState *pstate, Node *clause,
ParseExprKind exprKind, const char *constructName);
-extern Node *transformLimitClause(ParseState *pstate, Node *clause,
+extern Node *transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName);
extern List *transformGroupClause(ParseState *pstate, List *grouplist,
List **groupingSets,
diff --git a/src/test/modules/test_ddl_deparse/expected/create_table.out b/src/test/modules/test_ddl_deparse/expected/create_table.out
index 2d7dfd533e..eb04e36aac 100644
--- a/src/test/modules/test_ddl_deparse/expected/create_table.out
+++ b/src/test/modules/test_ddl_deparse/expected/create_table.out
@@ -105,7 +105,7 @@ CREATE TABLE student (
) INHERITS (person);
NOTICE: DDL test: type simple, tag CREATE TABLE
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "id"
NOTICE: merging multiple inherited definitions of column "name"
diff --git a/src/test/modules/test_ddl_deparse/sql/create_table.sql b/src/test/modules/test_ddl_deparse/sql/create_table.sql
index dd3a908638..f158dd4296 100644
--- a/src/test/modules/test_ddl_deparse/sql/create_table.sql
+++ b/src/test/modules/test_ddl_deparse/sql/create_table.sql
@@ -94,7 +94,7 @@ CREATE TABLE student (
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index d51e547278..d2055f21cd 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -79,7 +79,7 @@ CREATE TABLE student (
gpa float8
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "name"
NOTICE: merging multiple inherited definitions of column "age"
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index c18f547cbd..358378b151 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -108,6 +108,70 @@ SELECT ''::text AS five, unique1, unique2, stringu1
| 904 | 793 | UIAAAA
(5 rows)
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 51 | 76 | ZBAAAA
+ | 52 | 985 | ACAAAA
+ | 53 | 196 | BCAAAA
+ | 54 | 356 | CCAAAA
+ | 55 | 627 | DCAAAA
+ | 56 | 54 | ECAAAA
+ | 57 | 942 | FCAAAA
+ | 58 | 114 | GCAAAA
+ | 59 | 593 | HCAAAA
+ | 60 | 483 | ICAAAA
+(10 rows)
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 61 | 560 | JCAAAA
+(1 row)
+
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+ three | unique1 | unique2 | stringu1
+-------+---------+---------+----------
+ | 121 | 700 | REAAAA
+ | 122 | 519 | SEAAAA
+ | 123 | 777 | TEAAAA
+ | 124 | 503 | UEAAAA
+ | 125 | 849 | VEAAAA
+ | 126 | 330 | WEAAAA
+ | 127 | 511 | XEAAAA
+ | 128 | 721 | YEAAAA
+ | 129 | 696 | ZEAAAA
+(9 rows)
+
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+ eleven | unique1 | unique2 | stringu1
+--------+---------+---------+----------
+ | 10 | 520 | KAAAAA
+ | 9 | 49 | JAAAAA
+(2 rows)
+
+select * from int8_tbl;
+ q1 | q2
+------------------+-------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+ 4567890123456789 | 4567890123456789
+ 4567890123456789 | -4567890123456789
+(5 rows)
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -286,6 +350,46 @@ fetch all in c4;
----+----
(0 rows)
+declare c5 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c5;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
+fetch 1 in c5;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c5;
+ q1 | q2
+------------------+-----
+ 4567890123456789 | 123
+(1 row)
+
+fetch backward all in c5;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+ 123 | 456
+(2 rows)
+
+fetch backward 1 in c5;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c5;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
@@ -503,3 +607,19 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
45020 | 45020
(3 rows)
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
+ s1 | s2
+-------+-------
+ 45000 | 45000
+ 45010 | 45010
+ 45020 | 45020
+ 45030 | 45030
+ 45040 | 45040
+ 45050 | 45050
+ 45060 | 45060
+ 45070 | 45070
+ 45080 | 45080
+ 45090 | 45090
+(10 rows)
+
diff --git a/src/test/regress/sql/create_table.sql b/src/test/regress/sql/create_table.sql
index 4091c19cf0..a759ab5717 100644
--- a/src/test/regress/sql/create_table.sql
+++ b/src/test/regress/sql/create_table.sql
@@ -92,7 +92,7 @@ CREATE TABLE student (
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index 2a313d80ca..cd68953848 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -31,6 +31,24 @@ SELECT ''::text AS five, unique1, unique2, stringu1
FROM onek
ORDER BY unique1 LIMIT 5 OFFSET 900;
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+select * from int8_tbl;
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -71,6 +89,14 @@ fetch backward all in c4;
fetch backward 1 in c4;
fetch all in c4;
+declare c5 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c5;
+fetch 1 in c5;
+fetch backward 1 in c5;
+fetch backward all in c5;
+fetch backward 1 in c5;
+fetch all in c5;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
@@ -141,3 +167,6 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
from tenk1 group by thousand order by thousand limit 3;
+
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
Hello.
At Sat, 23 Feb 2019 22:27:44 +0100, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote in <81a5c0e9-c17d-28f3-4647-8a4659cdfdb1@2ndquadrant.com>
On 2/23/19 8:53 AM, Surafel Temesgen wrote:
On Sun, Feb 10, 2019 at 2:22 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:
I'm not sure I understand - are you saying every time the user does a
FETCH, we have to run the outer plan from scratch? I don't see why would
that be necessary? And if it is, how come there's no noticeable
performance difference?Can you share a patch implementing the incremental approach, and a query
demonstrating the issue?I didn't implement it but its obvious that it doesn't work similarly
with previous approach.Sure, but that's hardly a sufficient argument for the current approach.
We need different implementation and my plan was to use tuplestore per
call and clearit after returning tuple but I see that the plan will not go far because
mainly the last returnedslot is not the last slot we get from outerPlan execution
I'm sorry, I still don't understand what the supposed problem is. I
don't think it's all that different from what nodeMaterial.c does, for
example.As I explained before, having to execute the outer plan till completion
before returning any tuples is an issue. So either it needs fixing or an
explanation why it's not an issue.
One biggest issue seems to be we don't know the total number of
outer tuples before actually reading a null tuple. I doubt of
general shortcut for that. It also seems preventing limit node
from just using materialized outer.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
At Thu, 28 Feb 2019 13:32:17 +0300, Surafel Temesgen <surafel3000@gmail.com> wrote in <CALAY4q_TrFN2z1oQcpxCKB5diy+A7m5hhG4VOV5d1BaWpf+qqA@mail.gmail.com>
On Sun, Feb 24, 2019 at 12:27 AM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:I'm sorry, I still don't understand what the supposed problem is. I
don't think it's all that different from what nodeMaterial.c does, for
example.sorry for the noise .Attache is complete patch for incremental approach
Mmm. It seems just moving
reading-all-before-return-the-first-tuple from LIMIT_INITIAL to
LIMIT_RESCAN. If I read it correctly node->conut is not correct
since it is calculated as "the percentage of the number of tuples
read *so far* excluding the offset portion.).
| while (node->position - node->offset >= node->count)
| {
| slot = ExecProcNode(outerPlan);
| (if Tupleis Null then break;)
| tuplestore_puttupleslot(node->tuple_store, slot);
| cnt = tuplestore_tuple_count(node->tuple_store);
| node->count = ceil(node->percent * cnt / 100.0);
| }
If I'm missing nothing, this seems almost same with the follows.
while ((position - offset) * percent / 100 >=
(position - offset) * percent / 100) {}
In the first place, the patch breaks nodeLimit.c and build fails.
====
- * previous time we got a different result.
+ * previous time we got a different result.In PERCENTAGE option there are
+ * no bound on the number of output tuples */
*/
====
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
On 2/28/19 12:26 PM, Kyotaro HORIGUCHI wrote:
Hello.
At Sat, 23 Feb 2019 22:27:44 +0100, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote in <81a5c0e9-c17d-28f3-4647-8a4659cdfdb1@2ndquadrant.com>
On 2/23/19 8:53 AM, Surafel Temesgen wrote:
On Sun, Feb 10, 2019 at 2:22 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:
I'm not sure I understand - are you saying every time the user does a
FETCH, we have to run the outer plan from scratch? I don't see why would
that be necessary? And if it is, how come there's no noticeable
performance difference?Can you share a patch implementing the incremental approach, and a query
demonstrating the issue?I didn't implement it but its obvious that it doesn't work similarly
with previous approach.Sure, but that's hardly a sufficient argument for the current approach.
We need different implementation and my plan was to use tuplestore per
call and clearit after returning tuple but I see that the plan will not go far because
mainly the last returnedslot is not the last slot we get from outerPlan execution
I'm sorry, I still don't understand what the supposed problem is. I
don't think it's all that different from what nodeMaterial.c does, for
example.As I explained before, having to execute the outer plan till completion
before returning any tuples is an issue. So either it needs fixing or an
explanation why it's not an issue.One biggest issue seems to be we don't know the total number of
outer tuples before actually reading a null tuple. I doubt of
general shortcut for that. It also seems preventing limit node
from just using materialized outer.
Sure, if you actually want all tuples, you'll have to execute the outer
plan till completion. But that's not what I'm talking about - what if we
only ever need to read one row from the limit?
To give you a (admittedly, somewhat contrived and artificial example):
SELECT * FROM t1 WHERE id IN (
SELECT id FROM t2 ORDER BY x FETCH FIRST 10 PERCENT ROWS ONLY
);
Maybe this example is bogus and/or does not really matter in practice. I
don't know, but I've been unable to convince myself that's the case.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Hello.
At Thu, 28 Feb 2019 21:16:25 +0100, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote in <fbd08ad3-5dd8-3169-6cba-38d610d7be7f@2ndquadrant.com>
One biggest issue seems to be we don't know the total number of
# One *of* the biggest *issues*?
outer tuples before actually reading a null tuple. I doubt of
general shortcut for that. It also seems preventing limit node
from just using materialized outer.Sure, if you actually want all tuples, you'll have to execute the outer
plan till completion. But that's not what I'm talking about - what if we
only ever need to read one row from the limit?
We have no choice than once reading all tuples just to find we
are to return just one row, since estimator is not guaranteed to
be exact as required for this purpose.
To give you a (admittedly, somewhat contrived and artificial example):
SELECT * FROM t1 WHERE id IN (
SELECT id FROM t2 ORDER BY x FETCH FIRST 10 PERCENT ROWS ONLY
);Maybe this example is bogus and/or does not really matter in practice. I
don't know, but I've been unable to convince myself that's the case.
I see such kind of idiom common. Even in the quite simple example
above, *we* cannot tell how many tuples the inner should return
unless we actually fetch all tuples in t2. This is the same
problem with count(*).
The query is equivalent to the folloing one.
SELECT * FROM t1 WHERE id IN (
SELECT id FROM t2 ORDER BY x
FETCH FIRST (SELECT ceil(count(*) * 0.1) FROM t2) ROWS ONLY
);
This scans t2 twice, but this patch does only one full scan
moving another partial scan to tuplestore. We would win if the
outer is complex enough.
Anyway, even excluding the count(*) issue, it seems that we are
not alone in that point. (That is, at least Oracle shows
different E-Rows and A-Rows for PERCENT).
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
On Fri, Mar 1, 2019 at 4:33 AM Kyotaro HORIGUCHI <
horiguchi.kyotaro@lab.ntt.co.jp> wrote:
Hello.
At Thu, 28 Feb 2019 21:16:25 +0100, Tomas Vondra <
tomas.vondra@2ndquadrant.com> wrote in <
fbd08ad3-5dd8-3169-6cba-38d610d7be7f@2ndquadrant.com>One biggest issue seems to be we don't know the total number of
# One *of* the biggest *issues*?
outer tuples before actually reading a null tuple. I doubt of
general shortcut for that. It also seems preventing limit node
from just using materialized outer.Sure, if you actually want all tuples, you'll have to execute the outer
plan till completion. But that's not what I'm talking about - what if we
only ever need to read one row from the limit?We have no choice than once reading all tuples just to find we
are to return just one row, since estimator is not guaranteed to
be exact as required for this purpose.To give you a (admittedly, somewhat contrived and artificial example):
SELECT * FROM t1 WHERE id IN (
SELECT id FROM t2 ORDER BY x FETCH FIRST 10 PERCENT ROWS ONLY
);Maybe this example is bogus and/or does not really matter in practice. I
don't know, but I've been unable to convince myself that's the case.I see such kind of idiom common. Even in the quite simple example
above, *we* cannot tell how many tuples the inner should return
unless we actually fetch all tuples in t2. This is the same
problem with count(*).The query is equivalent to the folloing one.
SELECT * FROM t1 WHERE id IN (
SELECT id FROM t2 ORDER BY x
FETCH FIRST (SELECT ceil(count(*) * 0.1) FROM t2) ROWS ONLY
);This scans t2 twice, but this patch does only one full scan
moving another partial scan to tuplestore. We would win if the
outer is complex enough.
Okay here is the previous implementation with uptread review comment
included and it also consider OFFSET clause in percentage calculation
regards
Surafel
Attachments:
fetch-with-percent-v9.patchtext/x-patch; charset=US-ASCII; name=fetch-with-percent-v9.patchDownload
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 06d611b64c..e3ce4d7e36 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1430,7 +1430,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1440,7 +1440,8 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
+ with <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage.<literal>ROW</literal>
and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index baa669abe8..2be6f655fb 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -43,6 +43,7 @@ ExecLimit(PlanState *pstate)
LimitState *node = castNode(LimitState, pstate);
ScanDirection direction;
TupleTableSlot *slot;
+ TupleDesc tupleDescriptor;
PlanState *outerPlan;
CHECK_FOR_INTERRUPTS();
@@ -52,6 +53,8 @@ ExecLimit(PlanState *pstate)
*/
direction = node->ps.state->es_direction;
outerPlan = outerPlanState(node);
+ slot = node->subSlot;
+ tupleDescriptor = node->ps.ps_ResultTupleDesc;
/*
* The main logic is a simple state machine.
@@ -60,6 +63,26 @@ ExecLimit(PlanState *pstate)
{
case LIMIT_INITIAL:
+ if (node->limitOption == PERCENTAGE)
+ {
+
+ /*
+ * In PERCENTAGE option the number of tuple to return can't
+ * be determine before receiving the last tuple. so execute
+ * outerPlan until the end and store the result tuple to avoid
+ * executing twice
+ */
+ for (;;)
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ break;
+ }
+ tuplestore_puttupleslot(node->tuple_store, slot);
+ }
+ }
+
/*
* First call for this node, so compute limit/offset. (We can't do
* this any earlier, because parameters from upper nodes will not
@@ -87,24 +110,47 @@ ExecLimit(PlanState *pstate)
return NULL;
}
- /*
- * Fetch rows from subplan until we reach position > offset.
- */
- for (;;)
+ if (node->limitOption == PERCENTAGE)
{
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
+ /* In PERCENTAGE case the result is already in tuplestore */
+ for (;;)
{
- /*
- * The subplan returns too few tuples for us to produce
- * any output at all.
- */
- node->lstate = LIMIT_EMPTY;
- return NULL;
+ slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if (!tuplestore_gettupleslot(node->tuple_store, true, true, slot))
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ else
+ {
+ node->subSlot = slot;
+ if (++node->position > node->offset)
+ break;
+ }
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Fetch rows from subplan until we reach position > offset.
+ */
+ for (;;)
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ /*
+ * The subplan returns too few tuples for us to produce
+ * any output at all.
+ */
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ node->subSlot = slot;
+ if (++node->position > node->offset)
+ break;
}
- node->subSlot = slot;
- if (++node->position > node->offset)
- break;
}
/*
@@ -145,17 +191,36 @@ ExecLimit(PlanState *pstate)
return NULL;
}
- /*
- * Get next tuple from subplan, if any.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
+ /* In PERCENTAGE case the result is already in tuplestore */
+ if (node->limitOption == PERCENTAGE)
{
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
+ slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if (tuplestore_gettupleslot(node->tuple_store, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ node->subSlot = slot;
+ node->position++;
}
- node->subSlot = slot;
- node->position++;
}
else
{
@@ -169,14 +234,30 @@ ExecLimit(PlanState *pstate)
return NULL;
}
- /*
- * Get previous tuple from subplan; there should be one!
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
+ /* In PERCENTAGE case the result is already in tuplestore */
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->tuple_store, false, false, slot))
+ {
+ node->subSlot = slot;
+ node->position--;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Get previous tuple from subplan; there should be one!
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->position--;
+ }
}
break;
@@ -184,15 +265,30 @@ ExecLimit(PlanState *pstate)
if (ScanDirectionIsForward(direction))
return NULL;
- /*
- * Backing up from subplan EOF, so re-fetch previous tuple; there
- * should be one! Note previous tuple must be in window.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->lstate = LIMIT_INWINDOW;
+ /* In PERCENTAGE case the result is already in tuplestore */
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot(node->tuple_store, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
+ /*
+ * Backing up from subplan EOF, so re-fetch previous tuple; there
+ * should be one! Note previous tuple must be in window.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
/* position does not change 'cause we didn't advance it before */
break;
@@ -283,11 +379,16 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
+ if (node->limitOption == PERCENTAGE)
+ node->count = DatumGetFloat8(val) * tuplestore_tuple_count(node->tuple_store) / 100;
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+ }
node->noCount = false;
}
}
@@ -311,7 +412,8 @@ recompute_limits(LimitState *node)
* must update the child node anyway, in case this is a rescan and the
* previous time we got a different result.
*/
- ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
+ if (node->limitOption != PERCENTAGE)
+ ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
}
/*
@@ -374,6 +476,10 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
+
+ if (node->limitOption == PERCENTAGE)
+ limitstate->tuple_store= tuplestore_begin_heap(true, false, work_mem);
/*
* Initialize result type.
@@ -405,6 +511,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->tuple_store!= NULL)
+ tuplestore_end(node->tuple_store);
}
@@ -424,4 +532,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->tuple_store!= NULL)
+ tuplestore_rescan(node->tuple_store);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index e15724bb0e..dc0fe7b693 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1136,6 +1136,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3022,6 +3023,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3106,6 +3108,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 31499eb798..146984cd44 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -974,6 +974,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1048,6 +1049,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 65302fe65b..62fc11582d 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -900,6 +900,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2097,6 +2098,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2689,6 +2691,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2898,6 +2901,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 5aa42242a9..f93f543dc9 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -278,6 +278,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2324,6 +2325,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 236f506cfb..a2fc9f25f6 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2190,7 +2190,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2495,7 +2496,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6359,7 +6361,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6371,6 +6373,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index bc81535905..c5a4ae7008 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2122,12 +2122,22 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
*/
if (parse->sortClause)
{
- current_rel = create_ordered_paths(root,
- current_rel,
- final_target,
- final_target_parallel_safe,
- have_postponed_srfs ? -1.0 :
- limit_tuples);
+
+ /* In PERCENTAGE option there are no bound on the number of output tuples */
+ if (parse->limitOption == PERCENTAGE)
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ -1.0);
+ else
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ limit_tuples);
/* Fix things up if final_target contains SRFs */
if (parse->hasTargetSRFs)
adjust_paths_for_srfs(root, current_rel,
@@ -2190,6 +2200,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 169e51e792..69b6e4f072 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3526,6 +3526,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3547,6 +3548,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
@@ -3588,9 +3590,21 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
count_rows = (double) count_est;
else
count_rows = clamp_row_est(subpath->rows * 0.10);
+ if (limitOption == PERCENTAGE)
+ {
+ double per_count = DatumGetFloat8(count_est);
+ count_rows = clamp_row_est((subpath->rows * per_count) / 100);
+ if (subpath->rows > 0)
+ {
+ pathnode->path.startup_cost = (count_rows *
+ subpath->total_cost) / subpath->rows;
+ pathnode->path.total_cost = subpath->total_cost +
+ (count_rows * 0.1);
+ }
+ }
if (count_rows > pathnode->path.rows)
count_rows = pathnode->path.rows;
- if (subpath->rows > 0)
+ if (subpath->rows > 0 && limitOption == EXACT_NUMBER)
pathnode->path.total_cost = pathnode->path.startup_cost +
(subpath->total_cost - subpath->startup_cost)
* count_rows / subpath->rows;
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index e3544efb6f..3894fbd02b 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1301,10 +1301,11 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
}
/* transform LIMIT */
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1549,10 +1550,11 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_ORDER_BY,
false /* allow SQL92 rules */ );
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1784,10 +1786,11 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
parser_errposition(pstate,
exprLocation(list_nth(qry->targetList, tllen)))));
- qry->limitOffset = transformLimitClause(pstate, limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, limitCount,
+ qry->limitCount = transformLimitClause(pstate, limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 0279013120..80e5aafe91 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -164,6 +164,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -387,7 +388,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
+ reloption_list group_clause TriggerFuncArgs select_limit limit_clause
opt_select_limit opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
@@ -449,7 +450,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -661,7 +662,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11161,7 +11162,7 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
@@ -11169,6 +11170,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
list_nth($4, 0), list_nth($4, 1),
+ (list_nth($4, 2)),
NULL,
yyscanner);
$$ = $1;
@@ -11177,6 +11179,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
list_nth($3, 0), list_nth($3, 1),
+ list_nth($3, 2),
NULL,
yyscanner);
$$ = $1;
@@ -11185,7 +11188,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11193,7 +11196,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11201,6 +11204,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
list_nth($5, 0), list_nth($5, 1),
+ list_nth($5, 2),
$1,
yyscanner);
$$ = $2;
@@ -11209,6 +11213,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
list_nth($4, 0), list_nth($4, 1),
+ list_nth($4, 2),
$1,
yyscanner);
$$ = $2;
@@ -11502,20 +11507,20 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause { $$ = list_make3($2, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause limit_clause { $$ = list_make3($1, list_nth($2, 0), list_nth($2, 1)); }
+ | limit_clause { $$ = list_make3(NULL, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause { $$ = list_make3($1, NULL, NULL); }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */ { $$ = list_make3(NULL, NULL, NULL); }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ { $$ = list_make2($2, NULL); }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11533,9 +11538,11 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ { $$ = list_make2($3, makeString("EXACT_NUMBER")); }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ { $$ = list_make2($3, makeString("PERCENTAGE")); }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ { $$ = list_make2(makeIntConst(1, -1), NULL); }
;
offset_clause:
@@ -15403,6 +15410,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCES
@@ -15786,6 +15794,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15824,6 +15833,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index c6ce1011e2..6cb5a5dbc4 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1707,7 +1707,7 @@ transformWhereClause(ParseState *pstate, Node *clause,
* constructName does not affect the semantics, but is used in error messages
*/
Node *
-transformLimitClause(ParseState *pstate, Node *clause,
+transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName)
{
Node *qual;
@@ -1716,8 +1716,10 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
-
- qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
+ if (limitOption == PERCENTAGE && (strcmp(constructName, "LIMIT") == 0))
+ qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
+ else
+ qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
/* LIMIT can't refer to any variables of the current query */
checkExprIsVarFree(pstate, qual, constructName);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 09f8217c80..e8d7a3351e 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2289,8 +2289,10 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ Tuplestorestate *tuple_store; /* holds the returned tuple */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index f9389257c6..094c6ee3bb 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -820,4 +820,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+} LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index a7e859dc90..6eb3fe922d 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1583,6 +1584,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index a008ae07da..fe0fc2f975 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1775,6 +1775,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 6d087c268f..1cdfa706df 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -946,6 +946,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 574bb85b50..fe593170d3 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -264,6 +264,7 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 3bbdb5e2f7..6f21cd076a 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -56,7 +56,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index f05444008c..0a80bb9b76 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -299,6 +299,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, RESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h
index 179f3ab3c3..39a74c32d2 100644
--- a/src/include/parser/parse_clause.h
+++ b/src/include/parser/parse_clause.h
@@ -22,7 +22,7 @@ extern int setTargetTable(ParseState *pstate, RangeVar *relation,
extern Node *transformWhereClause(ParseState *pstate, Node *clause,
ParseExprKind exprKind, const char *constructName);
-extern Node *transformLimitClause(ParseState *pstate, Node *clause,
+extern Node *transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName);
extern List *transformGroupClause(ParseState *pstate, List *grouplist,
List **groupingSets,
diff --git a/src/test/modules/test_ddl_deparse/expected/create_table.out b/src/test/modules/test_ddl_deparse/expected/create_table.out
index 2d7dfd533e..eb04e36aac 100644
--- a/src/test/modules/test_ddl_deparse/expected/create_table.out
+++ b/src/test/modules/test_ddl_deparse/expected/create_table.out
@@ -105,7 +105,7 @@ CREATE TABLE student (
) INHERITS (person);
NOTICE: DDL test: type simple, tag CREATE TABLE
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "id"
NOTICE: merging multiple inherited definitions of column "name"
diff --git a/src/test/modules/test_ddl_deparse/sql/create_table.sql b/src/test/modules/test_ddl_deparse/sql/create_table.sql
index dd3a908638..f158dd4296 100644
--- a/src/test/modules/test_ddl_deparse/sql/create_table.sql
+++ b/src/test/modules/test_ddl_deparse/sql/create_table.sql
@@ -94,7 +94,7 @@ CREATE TABLE student (
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index d51e547278..d2055f21cd 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -79,7 +79,7 @@ CREATE TABLE student (
gpa float8
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "name"
NOTICE: merging multiple inherited definitions of column "age"
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index c18f547cbd..fcadc92f5d 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -108,6 +108,61 @@ SELECT ''::text AS five, unique1, unique2, stringu1
| 904 | 793 | UIAAAA
(5 rows)
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 51 | 76 | ZBAAAA
+ | 52 | 985 | ACAAAA
+ | 53 | 196 | BCAAAA
+ | 54 | 356 | CCAAAA
+ | 55 | 627 | DCAAAA
+ | 56 | 54 | ECAAAA
+ | 57 | 942 | FCAAAA
+ | 58 | 114 | GCAAAA
+ | 59 | 593 | HCAAAA
+(9 rows)
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 61 | 560 | JCAAAA
+(1 row)
+
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+ three | unique1 | unique2 | stringu1
+-------+---------+---------+----------
+ | 121 | 700 | REAAAA
+ | 122 | 519 | SEAAAA
+ | 123 | 777 | TEAAAA
+ | 124 | 503 | UEAAAA
+ | 125 | 849 | VEAAAA
+ | 126 | 330 | WEAAAA
+ | 127 | 511 | XEAAAA
+ | 128 | 721 | YEAAAA
+(8 rows)
+
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+ eleven | unique1 | unique2 | stringu1
+--------+---------+---------+----------
+ | 10 | 520 | KAAAAA
+ | 9 | 49 | JAAAAA
+ | 8 | 653 | IAAAAA
+ | 7 | 647 | HAAAAA
+ | 6 | 978 | GAAAAA
+(5 rows)
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -286,6 +341,43 @@ fetch all in c4;
----+----
(0 rows)
+declare c6 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c6;
+ q1 | q2
+-----+------------------
+ 123 | 456
+ 123 | 4567890123456789
+(2 rows)
+
+fetch 1 in c6;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c6;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+(1 row)
+
+fetch backward all in c6;
+ q1 | q2
+-----+-----
+ 123 | 456
+(1 row)
+
+fetch backward 1 in c6;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c6;
+ q1 | q2
+-----+------------------
+ 123 | 456
+ 123 | 4567890123456789
+(2 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
@@ -503,3 +595,19 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
45020 | 45020
(3 rows)
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
+ s1 | s2
+-------+-------
+ 45000 | 45000
+ 45010 | 45010
+ 45020 | 45020
+ 45030 | 45030
+ 45040 | 45040
+ 45050 | 45050
+ 45060 | 45060
+ 45070 | 45070
+ 45080 | 45080
+ 45090 | 45090
+(10 rows)
+
diff --git a/src/test/regress/sql/create_table.sql b/src/test/regress/sql/create_table.sql
index 4091c19cf0..a759ab5717 100644
--- a/src/test/regress/sql/create_table.sql
+++ b/src/test/regress/sql/create_table.sql
@@ -92,7 +92,7 @@ CREATE TABLE student (
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index 2a313d80ca..6609f8eb21 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -30,6 +30,24 @@ SELECT ''::text AS five, unique1, unique2, stringu1
SELECT ''::text AS five, unique1, unique2, stringu1
FROM onek
ORDER BY unique1 LIMIT 5 OFFSET 900;
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
@@ -38,7 +56,6 @@ select * from int8_tbl offset (case when random() < 0.5 then null::bigint end);
-- Test assorted cases involving backwards fetch from a LIMIT plan node
begin;
-
declare c1 cursor for select * from int8_tbl limit 10;
fetch all in c1;
fetch 1 in c1;
@@ -71,6 +88,15 @@ fetch backward all in c4;
fetch backward 1 in c4;
fetch all in c4;
+
+declare c6 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c6;
+fetch 1 in c6;
+fetch backward 1 in c6;
+fetch backward all in c6;
+fetch backward 1 in c6;
+fetch all in c6;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
@@ -141,3 +167,6 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
from tenk1 group by thousand order by thousand limit 3;
+
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
On 3/1/19 2:31 AM, Kyotaro HORIGUCHI wrote:
Hello.
At Thu, 28 Feb 2019 21:16:25 +0100, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote in <fbd08ad3-5dd8-3169-6cba-38d610d7be7f@2ndquadrant.com>
One biggest issue seems to be we don't know the total number of
# One *of* the biggest *issues*?
outer tuples before actually reading a null tuple. I doubt of
general shortcut for that. It also seems preventing limit node
from just using materialized outer.Sure, if you actually want all tuples, you'll have to execute the outer
plan till completion. But that's not what I'm talking about - what if we
only ever need to read one row from the limit?We have no choice than once reading all tuples just to find we
are to return just one row, since estimator is not guaranteed to
be exact as required for this purpose.
I don't follow - I'm not suggesting to base this on row estimates at
all. I'm saying that we can read in rows, and decide how many rows we
are expected to produce.
For example, with 1% sample, we'll produce about 1 row for each 100 rows
read from the outer plan. So we read the first row, and compute
rows_to_produce = ceil(0.01 * rows_read)
and every time this increases, we emit another row from a tuplestore.
To give you a (admittedly, somewhat contrived and artificial example):
SELECT * FROM t1 WHERE id IN (
SELECT id FROM t2 ORDER BY x FETCH FIRST 10 PERCENT ROWS ONLY
);Maybe this example is bogus and/or does not really matter in practice. I
don't know, but I've been unable to convince myself that's the case.I see such kind of idiom common. Even in the quite simple example
above, *we* cannot tell how many tuples the inner should return
unless we actually fetch all tuples in t2. This is the same
problem with count(*).The query is equivalent to the folloing one.
SELECT * FROM t1 WHERE id IN (
SELECT id FROM t2 ORDER BY x
FETCH FIRST (SELECT ceil(count(*) * 0.1) FROM t2) ROWS ONLY
);This scans t2 twice, but this patch does only one full scan
moving another partial scan to tuplestore. We would win if the
outer is complex enough.
But what if all t1 rows match the very first row in the subselect? In
that case we don't need to read all rows from the subplan - we only need
to read the first chunk (until we emit the first row).
Anyway, even excluding the count(*) issue, it seems that we are
not alone in that point. (That is, at least Oracle shows
different E-Rows and A-Rows for PERCENT).
I'm not sure hat E-Rows and A-Rows are? I suppose that's estimated and
actual - but as I explained above, that's not what I propose the
incremental approach to use.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Hi Surafel,
On 3/1/19 8:18 PM, Tomas Vondra wrote:
I'm not sure hat E-Rows and A-Rows are? I suppose that's estimated and
actual - but as I explained above, that's not what I propose the
incremental approach to use.
Marked as Waiting for Author since it appears Tomas is waiting for a
response from you.
Regards,
--
-David
david@pgmasters.net
On Thu, Feb 28, 2019 at 11:16 PM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:
To give you a (admittedly, somewhat contrived and artificial example):
SELECT * FROM t1 WHERE id IN (
SELECT id FROM t2 ORDER BY x FETCH FIRST 10 PERCENT ROWS ONLY
);Maybe this example is bogus and/or does not really matter in practice. I
don't know, but I've been unable to convince myself that's the case.
does this means we abandon incremental approach? and am not sure of
calculating
percentage after OFFSET clause is acceptable or not
regards
Surafel
On Tue, Mar 26, 2019 at 03:06:52PM +0300, Surafel Temesgen wrote:
On Thu, Feb 28, 2019 at 11:16 PM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:To give you a (admittedly, somewhat contrived and artificial example):
SELECT * FROM t1 WHERE id IN (
SELECT id FROM t2 ORDER BY x FETCH FIRST 10 PERCENT ROWS ONLY
);Maybe this example is bogus and/or does not really matter in practice. I
don't know, but I've been unable to convince myself that's the case.does this means we abandon incremental approach? and am not sure of
calculating
percentage after OFFSET clause is acceptable or not
I don't follow. I don't think I've suggested to abandon the incremental
approach - I've explained why I think it's what the patch should be doing,
and illustrated that with an example. I admit the example may be too
artificial and we never (or very infrequently) generate such plans. But
no such argument was presented here.
As for the OFFSET, I don't see why that would be incompatible with PERCENT
clause. I'm not sure if it should compute the percentage from the total
number of rows (including those skipped by the OFFSET) or the rest, but I
assume SQL standard says something about that.
cheers
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Hi,
On Thu, Feb 28, 2019 at 2:50 PM Kyotaro HORIGUCHI <
horiguchi.kyotaro@lab.ntt.co.jp> wrote:
==== - * previous time we got a different result. + * previous time we got a different result.In PERCENTAGE option there are + * no bound on the number of output tuples */ */ ====
Thank you for testing .Fixed
Attachments:
percent-incremental-v3.patchtext/x-patch; charset=US-ASCII; name=percent-incremental-v3.patchDownload
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 06d611b64c..e3ce4d7e36 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1430,7 +1430,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1440,7 +1440,8 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
+ with <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage.<literal>ROW</literal>
and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index baa669abe8..7bfd7fd034 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -21,6 +21,8 @@
#include "postgres.h"
+#include <math.h>
+
#include "executor/executor.h"
#include "executor/nodeLimit.h"
#include "miscadmin.h"
@@ -44,6 +46,7 @@ ExecLimit(PlanState *pstate)
ScanDirection direction;
TupleTableSlot *slot;
PlanState *outerPlan;
+ TupleDesc tupleDescriptor;
CHECK_FOR_INTERRUPTS();
@@ -52,6 +55,8 @@ ExecLimit(PlanState *pstate)
*/
direction = node->ps.state->es_direction;
outerPlan = outerPlanState(node);
+ slot = node->subSlot;
+ tupleDescriptor = node->ps.ps_ResultTupleDesc;
/*
* The main logic is a simple state machine.
@@ -81,7 +86,15 @@ ExecLimit(PlanState *pstate)
/*
* Check for empty window; if so, treat like empty subplan.
*/
- if (node->count <= 0 && !node->noCount)
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (node->percent == 0.0)
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ }
+ else if (node->count <= 0 && !node->noCount)
{
node->lstate = LIMIT_EMPTY;
return NULL;
@@ -107,6 +120,15 @@ ExecLimit(PlanState *pstate)
break;
}
+ /*
+ * We may needed this tuple in backward scan so put it into tuplestore.
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ tuplestore_puttupleslot(node->tupleStore, slot);
+ tuplestore_advance(node->tupleStore, true);
+ }
+
/*
* Okay, we have the first tuple of the window.
*/
@@ -124,6 +146,72 @@ ExecLimit(PlanState *pstate)
case LIMIT_INWINDOW:
if (ScanDirectionIsForward(direction))
{
+
+ /*
+ * In case of coming back from backward scan the tuple is already
+ * in tuple store.
+ */
+ if (node->limitOption == PERCENTAGE && node->backwardPosition > 0)
+ {
+ slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if (tuplestore_gettupleslot(node->tupleStore, true, false, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ node->backwardPosition--;
+ return slot;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+
+ /*
+ * In PERCENTAGE case no need of executing outerPlan multiple times.
+ */
+ if (node->limitOption == PERCENTAGE && node->reachEnd)
+ {
+ node->lstate = LIMIT_WINDOWEND;
+
+ /*
+ * If we know we won't need to back up, we can release
+ * resources at this point.
+ */
+ if (!(node->ps.state->es_top_eflags & EXEC_FLAG_BACKWARD))
+ (void) ExecShutdownNode(outerPlan);
+
+ return NULL;
+ }
+
+ /*
+ * When in percentage mode, we need to see if we can get any
+ * additional rows from the subplan (enough to increase the
+ * node->count value).
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ /* loop until the node->count increments */
+ while (node->position - node->offset >= node->count)
+ {
+ int64 cnt;
+
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->reachEnd = true;
+ break;
+ }
+
+ tuplestore_puttupleslot(node->tupleStore, slot);
+
+ cnt = tuplestore_tuple_count(node->tupleStore);
+
+ node->count = ceil(node->percent * cnt / 100.0);
+ }
+ }
+
/*
* Forwards scan, so check for stepping off end of window. If
* we are at the end of the window, return NULL without
@@ -145,6 +233,21 @@ ExecLimit(PlanState *pstate)
return NULL;
}
+ if (node->limitOption == PERCENTAGE)
+ {
+ while (node->position - node->offset < node->count)
+ {
+ slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if (tuplestore_gettupleslot(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
/*
* Get next tuple from subplan, if any.
*/
@@ -156,6 +259,7 @@ ExecLimit(PlanState *pstate)
}
node->subSlot = slot;
node->position++;
+ }
}
else
{
@@ -169,6 +273,22 @@ ExecLimit(PlanState *pstate)
return NULL;
}
+ /* In PERCENTAGE case the result is already in tuplestore */
+ if (node->limitOption == PERCENTAGE)
+ {
+ slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if (tuplestore_gettupleslot(node->tupleStore, false, false, slot))
+ {
+ node->subSlot = slot;
+ node->position--;
+ node->backwardPosition++;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+
/*
* Get previous tuple from subplan; there should be one!
*/
@@ -177,6 +297,7 @@ ExecLimit(PlanState *pstate)
elog(ERROR, "LIMIT subplan failed to run backwards");
node->subSlot = slot;
node->position--;
+ }
}
break;
@@ -184,6 +305,20 @@ ExecLimit(PlanState *pstate)
if (ScanDirectionIsForward(direction))
return NULL;
+ /* In PERCENTAGE case the result is already in tuplestore */
+ if (node->limitOption == PERCENTAGE)
+ {
+ slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if (tuplestore_gettupleslot(node->tupleStore, false, false, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
/*
* Backing up from subplan EOF, so re-fetch previous tuple; there
* should be one! Note previous tuple must be in window.
@@ -194,6 +329,7 @@ ExecLimit(PlanState *pstate)
node->subSlot = slot;
node->lstate = LIMIT_INWINDOW;
/* position does not change 'cause we didn't advance it before */
+ }
break;
case LIMIT_WINDOWEND:
@@ -278,17 +414,29 @@ recompute_limits(LimitState *node)
/* Interpret NULL count as no count (LIMIT ALL) */
if (isNull)
{
- node->count = 0;
+ node->count = 1;
node->noCount = true;
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
- node->noCount = false;
+ if (node->limitOption == PERCENTAGE)
+ {
+ /*
+ * We expect to return at least one row (unless there
+ * are no rows in the subplan), and we'll update this
+ * count later as we go.
+ */
+ node->count = 0;
+ node->percent = DatumGetFloat8(val);
+ }
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+ }
}
}
else
@@ -299,8 +447,10 @@ recompute_limits(LimitState *node)
}
/* Reset position to start-of-scan */
- node->position = 0;
+ node->position = 0;;
node->subSlot = NULL;
+ node->reachEnd = false;
+ node->backwardPosition = 0;
/* Set state-machine state */
node->lstate = LIMIT_RESCAN;
@@ -309,9 +459,11 @@ recompute_limits(LimitState *node)
* Notify child node about limit. Note: think not to "optimize" by
* skipping ExecSetTupleBound if compute_tuples_needed returns < 0. We
* must update the child node anyway, in case this is a rescan and the
- * previous time we got a different result.
+ * previous time we got a different result.In PERCENTAGE option there are
+ * no bound on the number of output tuples
*/
- ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
+ if (node->limitOption != PERCENTAGE)
+ ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
}
/*
@@ -374,6 +526,7 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
/*
* Initialize result type.
@@ -390,6 +543,9 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
limitstate->ps.ps_ProjInfo = NULL;
+ if (node->limitOption == PERCENTAGE)
+ limitstate->tupleStore= tuplestore_begin_heap(true, false, work_mem);
+
return limitstate;
}
@@ -405,6 +561,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->tupleStore!= NULL)
+ tuplestore_end(node->tupleStore);
}
@@ -424,4 +582,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->tupleStore!= NULL)
+ tuplestore_rescan(node->tupleStore);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 04cc15606d..1da37da976 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1143,6 +1143,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3031,6 +3032,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3115,6 +3117,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 91c007ad5b..fb9048d7f5 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -975,6 +975,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1049,6 +1050,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 910a738c20..2bf10b917a 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -907,6 +907,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2106,6 +2107,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2699,6 +2701,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2908,6 +2911,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index eff98febf1..2cb836b376 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -278,6 +278,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2332,6 +2333,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 979c3c212f..e41d11fd0b 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2201,7 +2201,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2514,7 +2515,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6399,7 +6401,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6411,6 +6413,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index ca7a0fbbf5..8cabfd190d 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2105,12 +2105,22 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
*/
if (parse->sortClause)
{
- current_rel = create_ordered_paths(root,
- current_rel,
- final_target,
- final_target_parallel_safe,
- have_postponed_srfs ? -1.0 :
- limit_tuples);
+
+ /* In PERCENTAGE option there are no bound on the number of output tuples */
+ if (parse->limitOption == PERCENTAGE)
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ -1.0);
+ else
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ limit_tuples);
/* Fix things up if final_target contains SRFs */
if (parse->hasTargetSRFs)
adjust_paths_for_srfs(root, current_rel,
@@ -2173,6 +2183,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 56de8fc370..c11603c11b 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3554,6 +3554,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3575,6 +3576,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
@@ -3616,6 +3618,18 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
count_rows = (double) count_est;
else
count_rows = clamp_row_est(subpath->rows * 0.10);
+ if (limitOption == PERCENTAGE)
+ {
+ double per_count = DatumGetFloat8(count_est);
+ count_rows = clamp_row_est((subpath->rows * per_count) / 100);
+ if (subpath->rows > 0)
+ {
+ pathnode->path.startup_cost = (count_rows *
+ subpath->total_cost) / subpath->rows;
+ pathnode->path.total_cost = subpath->total_cost +
+ (count_rows * 0.1);
+ }
+ }
if (count_rows > pathnode->path.rows)
count_rows = pathnode->path.rows;
if (subpath->rows > 0)
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index d6cdd16607..a34ffc0ccb 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1288,10 +1288,11 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
}
/* transform LIMIT */
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1536,10 +1537,11 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_ORDER_BY,
false /* allow SQL92 rules */ );
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1770,10 +1772,11 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
parser_errposition(pstate,
exprLocation(list_nth(qry->targetList, tllen)))));
- qry->limitOffset = transformLimitClause(pstate, limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, limitCount,
+ qry->limitCount = transformLimitClause(pstate, limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 0a4822829a..c93b36bdd3 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -165,6 +165,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -392,7 +393,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
+ reloption_list group_clause TriggerFuncArgs select_limit limit_clause
opt_select_limit opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
@@ -454,7 +455,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -666,7 +667,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11188,7 +11189,7 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
@@ -11196,6 +11197,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
list_nth($4, 0), list_nth($4, 1),
+ (list_nth($4, 2)),
NULL,
yyscanner);
$$ = $1;
@@ -11204,6 +11206,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
list_nth($3, 0), list_nth($3, 1),
+ list_nth($3, 2),
NULL,
yyscanner);
$$ = $1;
@@ -11212,7 +11215,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11220,7 +11223,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11228,6 +11231,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
list_nth($5, 0), list_nth($5, 1),
+ list_nth($5, 2),
$1,
yyscanner);
$$ = $2;
@@ -11236,6 +11240,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
list_nth($4, 0), list_nth($4, 1),
+ list_nth($4, 2),
$1,
yyscanner);
$$ = $2;
@@ -11529,20 +11534,20 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause { $$ = list_make3($2, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause limit_clause { $$ = list_make3($1, list_nth($2, 0), list_nth($2, 1)); }
+ | limit_clause { $$ = list_make3(NULL, list_nth($1, 0), list_nth($1, 1)); }
+ | offset_clause { $$ = list_make3($1, NULL, NULL); }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */ { $$ = list_make3(NULL, NULL, NULL); }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ { $$ = list_make2($2, NULL); }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11560,9 +11565,11 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ { $$ = list_make2($3, makeString("EXACT_NUMBER")); }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ { $$ = list_make2($3, makeString("PERCENTAGE")); }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ { $$ = list_make2(makeIntConst(1, -1), NULL); }
;
offset_clause:
@@ -15435,6 +15442,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCES
@@ -15818,6 +15826,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15856,6 +15865,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index c6ce1011e2..6cb5a5dbc4 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1707,7 +1707,7 @@ transformWhereClause(ParseState *pstate, Node *clause,
* constructName does not affect the semantics, but is used in error messages
*/
Node *
-transformLimitClause(ParseState *pstate, Node *clause,
+transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName)
{
Node *qual;
@@ -1716,8 +1716,10 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
-
- qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
+ if (limitOption == PERCENTAGE && (strcmp(constructName, "LIMIT") == 0))
+ qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
+ else
+ qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
/* LIMIT can't refer to any variables of the current query */
checkExprIsVarFree(pstate, qual, constructName);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 869c303e15..b8c055f3c8 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2291,8 +2291,13 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ float8 percent; /* percentage */
+ int64 backwardPosition; /* the number of tuple returned in backward scan*/
+ bool reachEnd; /* if true, outerPlan execute until the end */
+ Tuplestorestate *tupleStore; /* holds the returned tuple */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index ffb4cd4bcc..9db5be69d1 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -821,4 +821,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+} LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index bdd2bd2fd9..8ee2ad76af 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1583,6 +1584,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 88c8973f3c..2443dfc599 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1785,6 +1785,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 24740c31e3..63e07cc388 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -953,6 +953,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 9e79e1cd63..33838952d0 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -264,6 +264,7 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 6d10bf3ee8..45f071f819 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -56,7 +56,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators, Oid *grpCollations,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index f05444008c..0a80bb9b76 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -299,6 +299,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, RESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h
index 179f3ab3c3..39a74c32d2 100644
--- a/src/include/parser/parse_clause.h
+++ b/src/include/parser/parse_clause.h
@@ -22,7 +22,7 @@ extern int setTargetTable(ParseState *pstate, RangeVar *relation,
extern Node *transformWhereClause(ParseState *pstate, Node *clause,
ParseExprKind exprKind, const char *constructName);
-extern Node *transformLimitClause(ParseState *pstate, Node *clause,
+extern Node *transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName);
extern List *transformGroupClause(ParseState *pstate, List *grouplist,
List **groupingSets,
diff --git a/src/test/modules/test_ddl_deparse/expected/create_table.out b/src/test/modules/test_ddl_deparse/expected/create_table.out
index 2d7dfd533e..eb04e36aac 100644
--- a/src/test/modules/test_ddl_deparse/expected/create_table.out
+++ b/src/test/modules/test_ddl_deparse/expected/create_table.out
@@ -105,7 +105,7 @@ CREATE TABLE student (
) INHERITS (person);
NOTICE: DDL test: type simple, tag CREATE TABLE
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "id"
NOTICE: merging multiple inherited definitions of column "name"
diff --git a/src/test/modules/test_ddl_deparse/sql/create_table.sql b/src/test/modules/test_ddl_deparse/sql/create_table.sql
index dd3a908638..f158dd4296 100644
--- a/src/test/modules/test_ddl_deparse/sql/create_table.sql
+++ b/src/test/modules/test_ddl_deparse/sql/create_table.sql
@@ -94,7 +94,7 @@ CREATE TABLE student (
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index ad0cb32678..1430367607 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -79,7 +79,7 @@ CREATE TABLE student (
gpa float8
) INHERITS (person);
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
NOTICE: merging multiple inherited definitions of column "name"
NOTICE: merging multiple inherited definitions of column "age"
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index c18f547cbd..21fddfc713 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -108,6 +108,60 @@ SELECT ''::text AS five, unique1, unique2, stringu1
| 904 | 793 | UIAAAA
(5 rows)
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 51 | 76 | ZBAAAA
+ | 52 | 985 | ACAAAA
+ | 53 | 196 | BCAAAA
+ | 54 | 356 | CCAAAA
+ | 55 | 627 | DCAAAA
+ | 56 | 54 | ECAAAA
+ | 57 | 942 | FCAAAA
+ | 58 | 114 | GCAAAA
+ | 59 | 593 | HCAAAA
+ | 60 | 483 | ICAAAA
+(10 rows)
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 61 | 560 | JCAAAA
+(1 row)
+
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+ three | unique1 | unique2 | stringu1
+-------+---------+---------+----------
+ | 121 | 700 | REAAAA
+ | 122 | 519 | SEAAAA
+ | 123 | 777 | TEAAAA
+ | 124 | 503 | UEAAAA
+ | 125 | 849 | VEAAAA
+ | 126 | 330 | WEAAAA
+ | 127 | 511 | XEAAAA
+ | 128 | 721 | YEAAAA
+ | 129 | 696 | ZEAAAA
+(9 rows)
+
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+ eleven | unique1 | unique2 | stringu1
+--------+---------+---------+----------
+ | 10 | 520 | KAAAAA
+ | 9 | 49 | JAAAAA
+(2 rows)
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -286,6 +340,46 @@ fetch all in c4;
----+----
(0 rows)
+declare c5 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c5;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
+fetch 1 in c5;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c5;
+ q1 | q2
+------------------+-----
+ 4567890123456789 | 123
+(1 row)
+
+fetch backward all in c5;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+ 123 | 456
+(2 rows)
+
+fetch backward 1 in c5;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c5;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
@@ -503,3 +597,19 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
45020 | 45020
(3 rows)
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
+ s1 | s2
+-------+-------
+ 45000 | 45000
+ 45010 | 45010
+ 45020 | 45020
+ 45030 | 45030
+ 45040 | 45040
+ 45050 | 45050
+ 45060 | 45060
+ 45070 | 45070
+ 45080 | 45080
+ 45090 | 45090
+(10 rows)
+
diff --git a/src/test/regress/sql/create_table.sql b/src/test/regress/sql/create_table.sql
index 751c0d39f5..08adde1c99 100644
--- a/src/test/regress/sql/create_table.sql
+++ b/src/test/regress/sql/create_table.sql
@@ -92,7 +92,7 @@ CREATE TABLE student (
CREATE TABLE stud_emp (
- percent int4
+ "percent" int4
) INHERITS (emp, student);
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index 2a313d80ca..a513e4d60d 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -31,6 +31,23 @@ SELECT ''::text AS five, unique1, unique2, stringu1
FROM onek
ORDER BY unique1 LIMIT 5 OFFSET 900;
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -71,6 +88,14 @@ fetch backward all in c4;
fetch backward 1 in c4;
fetch all in c4;
+declare c5 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c5;
+fetch 1 in c5;
+fetch backward 1 in c5;
+fetch backward all in c5;
+fetch backward 1 in c5;
+fetch all in c5;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
@@ -141,3 +166,6 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
from tenk1 group by thousand order by thousand limit 3;
+
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
Hi,
On 2019-03-29 12:04:50 +0300, Surafel Temesgen wrote:
+ if (node->limitOption == PERCENTAGE) + { + while (node->position - node->offset < node->count) + { + slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple); + if (tuplestore_gettupleslot(node->tupleStore, true, true, slot))
There's several blocks like this - how's this not going to be a resource
leak? As far as I can tell you're just going to create new slots, and
their previous contents aren't going to be cleared? I think there very
rarely are cases where an executor node's *Next function (or its
subsidiaries) creates slots. Normally you ought to create them *once* on
the *Init function.
You might not directly leak memory, because this is probably running in
a short lived context, but you'll leak tuple desc references etc. (Or if
it were a heap buffer slot, buffer pins).
+ /* In PERCENTAGE case the result is already in tuplestore */ + if (node->limitOption == PERCENTAGE) + { + slot = MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple); + if (tuplestore_gettupleslot(node->tupleStore, false, false, slot)) + { + node->subSlot = slot; + node->lstate = LIMIT_INWINDOW; + } + else + elog(ERROR, "LIMIT subplan failed to run backwards"); + } + else if (node->limitOption == EXACT_NUMBER) + { /* * Backing up from subplan EOF, so re-fetch previous tuple; there * should be one! Note previous tuple must be in window. @@ -194,6 +329,7 @@ ExecLimit(PlanState *pstate) node->subSlot = slot; node->lstate = LIMIT_INWINDOW; /* position does not change 'cause we didn't advance it before */ + }
The indentation here looks wrong...
break;
case LIMIT_WINDOWEND:
@@ -278,17 +414,29 @@ recompute_limits(LimitState *node)
/* Interpret NULL count as no count (LIMIT ALL) */
if (isNull)
{
- node->count = 0;
+ node->count = 1;
node->noCount = true;
Huh?
} else { - node->count = DatumGetInt64(val); - if (node->count < 0) - ereport(ERROR, - (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE), - errmsg("LIMIT must not be negative"))); - node->noCount = false; + if (node->limitOption == PERCENTAGE) + { + /* + * We expect to return at least one row (unless there + * are no rows in the subplan), and we'll update this + * count later as we go. + */ + node->count = 0; + node->percent = DatumGetFloat8(val); + } + else + { + node->count = DatumGetInt64(val); + if (node->count < 0) + ereport(ERROR, + (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE), + errmsg("LIMIT must not be negative"))); + }
Shouldn't we error out with a negative count, regardless of PERCENT? Or
is that prevented elsewhere?
}
}
else
@@ -299,8 +447,10 @@ recompute_limits(LimitState *node)
}/* Reset position to start-of-scan */ - node->position = 0; + node->position = 0;;
unnecessary
if (parse->sortClause) { - current_rel = create_ordered_paths(root, - current_rel, - final_target, - final_target_parallel_safe, - have_postponed_srfs ? -1.0 : - limit_tuples); + + /* In PERCENTAGE option there are no bound on the number of output tuples */ + if (parse->limitOption == PERCENTAGE) + current_rel = create_ordered_paths(root, + current_rel, + final_target, + final_target_parallel_safe, + have_postponed_srfs ? -1.0 : + -1.0); + else + current_rel = create_ordered_paths(root, + current_rel, + final_target, + final_target_parallel_safe, + have_postponed_srfs ? -1.0 : + limit_tuples);
I'd rather not duplicate those two calls, and just have the limit_tuples
computation inside an if.
offset_clause:
@@ -15435,6 +15442,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCES
Are we really ok with adding a new reserved keyword 'PERCENT' for this?
That seems awfully likely to already exist in columns etc. Is there a
chance we can use a less strong category?
Greetings,
Andres Freund
Andres Freund <andres@anarazel.de> writes:
offset_clause:
@@ -15435,6 +15442,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCES
Are we really ok with adding a new reserved keyword 'PERCENT' for this?
I'm not. It doesn't look like it ought to be necessary to reserve it,
either, given that we don't have to reduce the production right there.
(If that doesn't work right away, try getting rid of row_or_rows
in favor of spelling out those alternatives in separate productions.)
More generally: using an undocumented list as the data structure for
select_limit's result was already a pretty bad idea, I think, and
this patch certainly makes it totally unreadable. Probably ought
to refactor that to use some sort of struct.
regards, tom lane
On Tue, Mar 26, 2019 at 5:46 PM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:
On Tue, Mar 26, 2019 at 03:06:52PM +0300, Surafel Temesgen wrote:
As for the OFFSET, I don't see why that would be incompatible with PERCENT
clause. I'm not sure if it should compute the percentage from the total
number of rows (including those skipped by the OFFSET) or the rest, but I
assume SQL standard says something about that.
I check SQL standard and it is different in PERCENT cause. It state that
the percentage is computed using the number of rows before removing the
rows specified by offset row count
regards
Surafel
Thank you for looking at it
On Mon, Apr 1, 2019 at 12:34 AM Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2019-03-29 12:04:50 +0300, Surafel Temesgen wrote:
+ if (node->limitOption == PERCENTAGE) + { + while (node->position -node->offset < node->count)
+ { + slot =MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if
(tuplestore_gettupleslot(node->tupleStore, true, true, slot))
There's several blocks like this - how's this not going to be a resource
leak? As far as I can tell you're just going to create new slots, and
their previous contents aren't going to be cleared? I think there very
rarely are cases where an executor node's *Next function (or its
subsidiaries) creates slots. Normally you ought to create them *once* on
the *Init function.
create it in init stage is good idea. i make it this way because
tuplestore_gettupleslot
work with TTSOpsMinimalTuple
You might not directly leak memory, because this is probably running in
a short lived context, but you'll leak tuple desc references etc. (Or if
it were a heap buffer slot, buffer pins).+ /* In PERCENTAGE case the result is already in
tuplestore */
+ if (node->limitOption == PERCENTAGE) + { + slot =MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if
(tuplestore_gettupleslot(node->tupleStore, false, false, slot))
+ { + node->subSlot = slot; + node->lstate = LIMIT_INWINDOW; + } + else + elog(ERROR, "LIMIT subplan failedto run backwards");
+ } + else if (node->limitOption == EXACT_NUMBER) + { /* * Backing up from subplan EOF, so re-fetchprevious tuple; there
* should be one! Note previous tuple must be in
window.
@@ -194,6 +329,7 @@ ExecLimit(PlanState *pstate)
node->subSlot = slot;
node->lstate = LIMIT_INWINDOW;
/* position does not change 'cause we didn'tadvance it before */
+ }
The indentation here looks wrong...
break;
case LIMIT_WINDOWEND:
@@ -278,17 +414,29 @@ recompute_limits(LimitState *node)
/* Interpret NULL count as no count (LIMIT ALL) */
if (isNull)
{
- node->count = 0;
+ node->count = 1;
node->noCount = true;Huh?
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
-(errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be
negative")));
- node->noCount = false; + if (node->limitOption == PERCENTAGE) + { + /* + * We expect to return at least one row(unless there
+ * are no rows in the subplan), and we'll
update this
+ * count later as we go. + */ + node->count = 0; + node->percent = DatumGetFloat8(val); + } + else + { + node->count = DatumGetInt64(val); + if (node->count < 0) + ereport(ERROR, +(errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT
must not be negative")));
+ }
Shouldn't we error out with a negative count, regardless of PERCENT? Or
is that prevented elsewhere?
yes it should error out .will fix it in next patch
}
}
else
@@ -299,8 +447,10 @@ recompute_limits(LimitState *node)
}/* Reset position to start-of-scan */ - node->position = 0; + node->position = 0;;unnecessary
if (parse->sortClause)
{
- current_rel = create_ordered_paths(root,
-current_rel,
-
final_target,
-
final_target_parallel_safe,
-
have_postponed_srfs ? -1.0 :
-
limit_tuples);
+ + /* In PERCENTAGE option there are no bound on the numberof output tuples */
+ if (parse->limitOption == PERCENTAGE) + current_rel = create_ordered_paths(root, +current_rel,
+
final_target,
+
final_target_parallel_safe,
+
have_postponed_srfs ? -1.0 :
+
-1.0);
+ else + current_rel = create_ordered_paths(root, +current_rel,
+
final_target,
+
final_target_parallel_safe,
+
have_postponed_srfs ? -1.0 :
+
limit_tuples);
I'd rather not duplicate those two calls, and just have the limit_tuples
computation inside an if.
-1.0 means there are no limit. In percentage case the query execute until
the end
offset_clause:
@@ -15435,6 +15442,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCESAre we really ok with adding a new reserved keyword 'PERCENT' for this?
That seems awfully likely to already exist in columns etc. Is there a
chance we can use a less strong category?
There are already a case in regression test. I will make it unreserved
keyword in
next patch
regards
Surafel
On Tue, Mar 26, 2019 at 5:46 PM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:
On Tue, Mar 26, 2019 at 03:06:52PM +0300, Surafel Temesgen wrote:
On Thu, Feb 28, 2019 at 11:16 PM Tomas Vondra <
tomas.vondra@2ndquadrant.com>
wrote:
To give you a (admittedly, somewhat contrived and artificial example):
SELECT * FROM t1 WHERE id IN (
SELECT id FROM t2 ORDER BY x FETCH FIRST 10 PERCENT ROWS ONLY
);Maybe this example is bogus and/or does not really matter in practice. I
don't know, but I've been unable to convince myself that's the case.does this means we abandon incremental approach? and am not sure of
calculating
percentage after OFFSET clause is acceptable or notI don't follow. I don't think I've suggested to abandon the incremental
approach - I've explained why I think it's what the patch should be doing,
and illustrated that with an example.
but it is more complex than the previous approach and it will be more
complex
in starting calculating percentage before offset row count. Doesn't
simplicity prefer?
regards
Surafel
On Fri, Apr 05, 2019 at 03:14:56PM +0300, Surafel Temesgen wrote:
On Tue, Mar 26, 2019 at 5:46 PM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:On Tue, Mar 26, 2019 at 03:06:52PM +0300, Surafel Temesgen wrote:
On Thu, Feb 28, 2019 at 11:16 PM Tomas Vondra <
tomas.vondra@2ndquadrant.com>
wrote:
To give you a (admittedly, somewhat contrived and artificial example):
SELECT * FROM t1 WHERE id IN (
SELECT id FROM t2 ORDER BY x FETCH FIRST 10 PERCENT ROWS ONLY
);Maybe this example is bogus and/or does not really matter in practice. I
don't know, but I've been unable to convince myself that's the case.does this means we abandon incremental approach? and am not sure of
calculating
percentage after OFFSET clause is acceptable or notI don't follow. I don't think I've suggested to abandon the incremental
approach - I've explained why I think it's what the patch should be doing,
and illustrated that with an example.but it is more complex than the previous approach and it will be more
complex
in starting calculating percentage before offset row count. Doesn't
simplicity prefer?
Sure, simplicity has it's value. And often the simplest solution is the
best one. But in But in plenty of cases it's a tradeoff between simplicity
and efficiency, i.e. the simplest solution may perform poorly. I've tried
to explain why I think the incremental solution is the right approach
here - not having to always execute the subplan to completion. I still
haven't heard reasons why that argument is irrelevant (it may be).
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Hello,
The attached patch include the fix for all the comment given
regards
Surafel
On Mon, Apr 1, 2019 at 12:34 AM Andres Freund <andres@anarazel.de> wrote:
Show quoted text
Hi,
On 2019-03-29 12:04:50 +0300, Surafel Temesgen wrote:
+ if (node->limitOption == PERCENTAGE) + { + while (node->position -node->offset < node->count)
+ { + slot =MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if
(tuplestore_gettupleslot(node->tupleStore, true, true, slot))
There's several blocks like this - how's this not going to be a resource
leak? As far as I can tell you're just going to create new slots, and
their previous contents aren't going to be cleared? I think there very
rarely are cases where an executor node's *Next function (or its
subsidiaries) creates slots. Normally you ought to create them *once* on
the *Init function.You might not directly leak memory, because this is probably running in
a short lived context, but you'll leak tuple desc references etc. (Or if
it were a heap buffer slot, buffer pins).+ /* In PERCENTAGE case the result is already in
tuplestore */
+ if (node->limitOption == PERCENTAGE) + { + slot =MakeSingleTupleTableSlot(tupleDescriptor, &TTSOpsMinimalTuple);
+ if
(tuplestore_gettupleslot(node->tupleStore, false, false, slot))
+ { + node->subSlot = slot; + node->lstate = LIMIT_INWINDOW; + } + else + elog(ERROR, "LIMIT subplan failedto run backwards");
+ } + else if (node->limitOption == EXACT_NUMBER) + { /* * Backing up from subplan EOF, so re-fetchprevious tuple; there
* should be one! Note previous tuple must be in
window.
@@ -194,6 +329,7 @@ ExecLimit(PlanState *pstate)
node->subSlot = slot;
node->lstate = LIMIT_INWINDOW;
/* position does not change 'cause we didn'tadvance it before */
+ }
The indentation here looks wrong...
break;
case LIMIT_WINDOWEND:
@@ -278,17 +414,29 @@ recompute_limits(LimitState *node)
/* Interpret NULL count as no count (LIMIT ALL) */
if (isNull)
{
- node->count = 0;
+ node->count = 1;
node->noCount = true;Huh?
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
-(errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be
negative")));
- node->noCount = false; + if (node->limitOption == PERCENTAGE) + { + /* + * We expect to return at least one row(unless there
+ * are no rows in the subplan), and we'll
update this
+ * count later as we go. + */ + node->count = 0; + node->percent = DatumGetFloat8(val); + } + else + { + node->count = DatumGetInt64(val); + if (node->count < 0) + ereport(ERROR, +(errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT
must not be negative")));
+ }
Shouldn't we error out with a negative count, regardless of PERCENT? Or
is that prevented elsewhere?}
}
else
@@ -299,8 +447,10 @@ recompute_limits(LimitState *node)
}/* Reset position to start-of-scan */ - node->position = 0; + node->position = 0;;unnecessary
if (parse->sortClause)
{
- current_rel = create_ordered_paths(root,
-current_rel,
-
final_target,
-
final_target_parallel_safe,
-
have_postponed_srfs ? -1.0 :
-
limit_tuples);
+ + /* In PERCENTAGE option there are no bound on the numberof output tuples */
+ if (parse->limitOption == PERCENTAGE) + current_rel = create_ordered_paths(root, +current_rel,
+
final_target,
+
final_target_parallel_safe,
+
have_postponed_srfs ? -1.0 :
+
-1.0);
+ else + current_rel = create_ordered_paths(root, +current_rel,
+
final_target,
+
final_target_parallel_safe,
+
have_postponed_srfs ? -1.0 :
+
limit_tuples);
I'd rather not duplicate those two calls, and just have the limit_tuples
computation inside an if.offset_clause:
@@ -15435,6 +15442,7 @@ reserved_keyword:
| ONLY
| OR
| ORDER
+ | PERCENT
| PLACING
| PRIMARY
| REFERENCESAre we really ok with adding a new reserved keyword 'PERCENT' for this?
That seems awfully likely to already exist in columns etc. Is there a
chance we can use a less strong category?Greetings,
Andres Freund
Attachments:
percent-incremental-v4.patchtext/x-patch; charset=US-ASCII; name=percent-incremental-v4.patchDownload
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 06d611b64c..e3ce4d7e36 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1430,7 +1430,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1440,7 +1440,8 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
+ with <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage.<literal>ROW</literal>
and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index baa669abe8..cc170602be 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -21,6 +21,8 @@
#include "postgres.h"
+#include <math.h>
+
#include "executor/executor.h"
#include "executor/nodeLimit.h"
#include "miscadmin.h"
@@ -52,6 +54,7 @@ ExecLimit(PlanState *pstate)
*/
direction = node->ps.state->es_direction;
outerPlan = outerPlanState(node);
+ slot = node->subSlot;
/*
* The main logic is a simple state machine.
@@ -81,7 +84,15 @@ ExecLimit(PlanState *pstate)
/*
* Check for empty window; if so, treat like empty subplan.
*/
- if (node->count <= 0 && !node->noCount)
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (node->percent == 0.0)
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ }
+ else if (node->count <= 0 && !node->noCount)
{
node->lstate = LIMIT_EMPTY;
return NULL;
@@ -102,11 +113,32 @@ ExecLimit(PlanState *pstate)
node->lstate = LIMIT_EMPTY;
return NULL;
}
+
+ /*
+ * Count the number of rows to return in exact number so far
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ node->perExactCount = ceil(node->percent * node->position / 100.0);
+
+ if (node->perExactCount == node->perExactCount + 1)
+ node->perExactCount++;
+ }
node->subSlot = slot;
if (++node->position > node->offset)
break;
}
+ /*
+ * We may needed this tuple in backward scan so put it into
+ * tuplestore.
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ tuplestore_puttupleslot(node->tupleStore, slot);
+ tuplestore_advance(node->tupleStore, true);
+ }
+
/*
* Okay, we have the first tuple of the window.
*/
@@ -124,6 +156,106 @@ ExecLimit(PlanState *pstate)
case LIMIT_INWINDOW:
if (ScanDirectionIsForward(direction))
{
+ /*
+ * In case of coming back from backward scan the tuple is
+ * already in tuple store.
+ */
+ if (node->limitOption == PERCENTAGE && node->backwardPosition > 0)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ node->backwardPosition--;
+ return slot;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+
+ /*
+ * In PERCENTAGE case no need of executing outerPlan multiple
+ * times.
+ */
+ if (node->limitOption == PERCENTAGE && node->reachEnd)
+ {
+ node->lstate = LIMIT_WINDOWEND;
+
+ /*
+ * If we know we won't need to back up, we can release
+ * resources at this point.
+ */
+ if (!(node->ps.state->es_top_eflags & EXEC_FLAG_BACKWARD))
+ (void) ExecShutdownNode(outerPlan);
+
+ return NULL;
+ }
+
+ /*
+ * Return the tuple up to the number of exact count in OFFSET
+ * clause without percentage value consideration.
+ */
+ if (node->perExactCount > 0)
+ {
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ node->reachEnd = true;
+ return NULL;
+ }
+ tuplestore_puttupleslot(node->tupleStore, slot);
+ tuplestore_advance(node->tupleStore, true);
+ node->subSlot = slot;
+ node->position++;
+ node->perExactCount--;
+ return slot;
+ }
+
+ /*
+ * When in percentage mode, we need to see if we can get any
+ * additional rows from the subplan (enough to increase the
+ * node->count value).
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ /* loop until the node->count increments */
+ while (node->position - node->offset >= node->count)
+ {
+ int64 cnt;
+
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->reachEnd = true;
+ node->lstate = LIMIT_SUBPLANEOF;
+
+ /*
+ * The only operation from here is backward scan
+ * but there's no API to refetch the tuple at the
+ * current position. We have to move one tuple
+ * backward, and then we will scan forward for it
+ * for the first tuple and precede as usual
+ * for the rest
+ */
+ tuplestore_advance(node->tupleStore, false);
+ return NULL;
+ }
+
+ tuplestore_puttupleslot(node->tupleStore, slot);
+
+ cnt = tuplestore_tuple_count(node->tupleStore) + node->offset;
+
+ node->count = ceil(node->percent * cnt / 100.0);
+ }
+ }
+
/*
* Forwards scan, so check for stepping off end of window. If
* we are at the end of the window, return NULL without
@@ -145,17 +277,31 @@ ExecLimit(PlanState *pstate)
return NULL;
}
- /*
- * Get next tuple from subplan, if any.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
+ if (node->limitOption == PERCENTAGE)
{
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
+ while (node->position - node->offset < node->count)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ node->subSlot = slot;
+ node->position++;
}
- node->subSlot = slot;
- node->position++;
}
else
{
@@ -168,15 +314,29 @@ ExecLimit(PlanState *pstate)
node->lstate = LIMIT_WINDOWSTART;
return NULL;
}
-
- /*
- * Get previous tuple from subplan; there should be one!
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
+ /* In PERCENTAGE case the result is already in tuplestore */
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, false, true, slot))
+ {
+ node->subSlot = slot;
+ node->position--;
+ node->backwardPosition++;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+ /*
+ * Get previous tuple from subplan; there should be one!
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->position--;
+ }
}
break;
@@ -185,15 +345,32 @@ ExecLimit(PlanState *pstate)
return NULL;
/*
- * Backing up from subplan EOF, so re-fetch previous tuple; there
- * should be one! Note previous tuple must be in window.
+ * Scan forward for the first tuple
*/
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->lstate = LIMIT_INWINDOW;
- /* position does not change 'cause we didn't advance it before */
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+ /*
+ * Backing up from subplan EOF, so re-fetch previous tuple;
+ * there should be one! Note previous tuple must be in
+ * window.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ /* position does not change 'cause we didn't advance it before */
+ }
break;
case LIMIT_WINDOWEND:
@@ -283,12 +460,28 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
- node->noCount = false;
+ if (node->limitOption == PERCENTAGE)
+ {
+ /*
+ * We expect to return at least one row (unless there are no
+ * rows in the subplan), and we'll update this count later as
+ * we go.
+ */
+ node->count = 0;
+ node->percent = DatumGetFloat8(val);
+ if (node->percent < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_RESULT_OFFSET_CLAUSE),
+ errmsg("PERCENT must not be negative")));
+ }
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+ }
}
}
else
@@ -301,6 +494,9 @@ recompute_limits(LimitState *node)
/* Reset position to start-of-scan */
node->position = 0;
node->subSlot = NULL;
+ node->reachEnd = false;
+ node->backwardPosition = 0;
+ node->perExactCount = 0;
/* Set state-machine state */
node->lstate = LIMIT_RESCAN;
@@ -309,9 +505,11 @@ recompute_limits(LimitState *node)
* Notify child node about limit. Note: think not to "optimize" by
* skipping ExecSetTupleBound if compute_tuples_needed returns < 0. We
* must update the child node anyway, in case this is a rescan and the
- * previous time we got a different result.
+ * previous time we got a different result.In PERCENTAGE option there are
+ * no bound on the number of output tuples
*/
- ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
+ if (node->limitOption != PERCENTAGE)
+ ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
}
/*
@@ -374,6 +572,7 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
/*
* Initialize result type.
@@ -390,6 +589,9 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
limitstate->ps.ps_ProjInfo = NULL;
+ if (node->limitOption == PERCENTAGE)
+ limitstate->tupleStore = tuplestore_begin_heap(true, false, work_mem);
+
return limitstate;
}
@@ -405,6 +607,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->tupleStore != NULL)
+ tuplestore_end(node->tupleStore);
}
@@ -424,4 +628,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->tupleStore != NULL)
+ tuplestore_rescan(node->tupleStore);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 78deade89b..8307da7d3a 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1143,6 +1143,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3031,6 +3032,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3115,6 +3117,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 4f2ebe5118..2e14fa72b4 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -975,6 +975,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1049,6 +1050,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8400dd319e..3561bd8ae2 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -907,6 +907,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2104,6 +2105,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2698,6 +2700,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2908,6 +2911,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 6c2626ee62..85b514c9c2 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -278,6 +278,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2333,6 +2334,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 608d5adfed..6a6e733810 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2331,7 +2331,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2644,7 +2645,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6512,7 +6514,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6524,6 +6526,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index cb897cc7f4..9331dbc545 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2241,12 +2241,25 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
*/
if (parse->sortClause)
{
- current_rel = create_ordered_paths(root,
- current_rel,
- final_target,
- final_target_parallel_safe,
- have_postponed_srfs ? -1.0 :
- limit_tuples);
+
+ /*
+ * In PERCENTAGE option there are no bound on the number of output
+ * tuples
+ */
+ if (parse->limitOption == PERCENTAGE)
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ -1.0);
+ else
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ limit_tuples);
/* Fix things up if final_target contains SRFs */
if (parse->hasTargetSRFs)
adjust_paths_for_srfs(root, current_rel,
@@ -2309,6 +2322,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index d884d2bb00..3c49a716eb 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3570,6 +3570,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3591,6 +3592,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
@@ -3598,7 +3600,8 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
adjust_limit_rows_costs(&pathnode->path.rows,
&pathnode->path.startup_cost,
&pathnode->path.total_cost,
- offset_est, count_est);
+ offset_est, count_est,
+ limitOption);
return pathnode;
}
@@ -3624,7 +3627,8 @@ adjust_limit_rows_costs(double *rows, /* in/out parameter */
Cost *startup_cost, /* in/out parameter */
Cost *total_cost, /* in/out parameter */
int64 offset_est,
- int64 count_est)
+ int64 count_est,
+ LimitOption limitOption)
{
double input_rows = *rows;
Cost input_startup_cost = *startup_cost;
@@ -3657,6 +3661,19 @@ adjust_limit_rows_costs(double *rows, /* in/out parameter */
count_rows = (double) count_est;
else
count_rows = clamp_row_est(input_rows * 0.10);
+ if (limitOption == PERCENTAGE)
+ {
+ double per_count = DatumGetFloat8(count_est);
+
+ count_rows = clamp_row_est((input_rows * per_count) / 100);
+ if (rows > 0)
+ {
+ *startup_cost = count_rows *
+ input_total_cost / input_rows;
+ *total_cost = input_total_cost +
+ (count_rows * 0.1);
+ }
+ }
if (count_rows > *rows)
count_rows = *rows;
if (input_rows > 0)
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index b13c246183..2c7feff026 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1288,10 +1288,11 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
}
/* transform LIMIT */
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1536,10 +1537,11 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_ORDER_BY,
false /* allow SQL92 rules */ );
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1770,10 +1772,11 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
parser_errposition(pstate,
exprLocation(list_nth(qry->targetList, tllen)))));
- qry->limitOffset = transformLimitClause(pstate, limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, limitCount,
+ qry->limitCount = transformLimitClause(pstate, limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 8311b1dd46..91fb2e4143 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -127,6 +127,21 @@ typedef struct ImportQual
List *table_names;
} ImportQual;
+/* Private struct for the result of opt_select_limit production */
+typedef struct SelectLimit
+{
+ Node *limitOffset;
+ Node *limitCount;
+ void *limitOption;
+} SelectLimit;
+
+/* Private struct for the result of limit_clause production */
+typedef struct LimitClause
+{
+ Node *limitCount;
+ void *limitOption;
+} LimitClause;
+
/* ConstraintAttributeSpec yields an integer bitmask of these flags: */
#define CAS_NOT_DEFERRABLE 0x01
#define CAS_DEFERRABLE 0x02
@@ -165,6 +180,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -242,6 +258,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
PartitionSpec *partspec;
PartitionBoundSpec *partboundspec;
RoleSpec *rolespec;
+ struct SelectLimit *SelectLimit;
+ struct LimitClause *LimitClause;
}
%type <node> stmt schema_stmt
@@ -373,6 +391,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <ival> import_qualification_type
%type <importqual> import_qualification
%type <node> vacuum_relation
+%type <SelectLimit> opt_select_limit select_limit
+%type <LimitClause> limit_clause
%type <list> stmtblock stmtmulti
OptTableElementList TableElementList OptInherit definition
@@ -393,8 +413,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
- opt_select_limit opclass_item_list opclass_drop_list
+ reloption_list group_clause TriggerFuncArgs
+ opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
prep_type_clause
@@ -455,7 +475,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -667,7 +687,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11225,14 +11245,15 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
| select_clause opt_sort_clause for_locking_clause opt_select_limit
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
- list_nth($4, 0), list_nth($4, 1),
+ ($4)->limitOffset, ($4)->limitCount,
+ ($4)->limitOption,
NULL,
yyscanner);
$$ = $1;
@@ -11240,7 +11261,8 @@ select_no_parens:
| select_clause opt_sort_clause select_limit opt_for_locking_clause
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
- list_nth($3, 0), list_nth($3, 1),
+ ($3)->limitOffset, ($3)->limitCount,
+ ($3)->limitOption,
NULL,
yyscanner);
$$ = $1;
@@ -11249,7 +11271,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11257,14 +11279,15 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
| with_clause select_clause opt_sort_clause for_locking_clause opt_select_limit
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
- list_nth($5, 0), list_nth($5, 1),
+ ($5)->limitOffset, ($5)->limitCount,
+ ($5)->limitOption,
$1,
yyscanner);
$$ = $2;
@@ -11272,7 +11295,8 @@ select_no_parens:
| with_clause select_clause opt_sort_clause select_limit opt_for_locking_clause
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
- list_nth($4, 0), list_nth($4, 1),
+ ($4)->limitOffset, ($4)->limitCount,
+ ($4)->limitOption,
$1,
yyscanner);
$$ = $2;
@@ -11566,20 +11590,60 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $2;
+ n->limitCount = ($1)->limitCount;
+ n->limitOption = ($1)->limitOption;
+ $$ = n;
+ }
+ | offset_clause limit_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $1;
+ n->limitCount = ($2)->limitCount;
+ n->limitOption = ($2)->limitOption;
+ $$ = n;
+ }
+ | limit_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = NULL;
+ n->limitCount = ($1)->limitCount;
+ n->limitOption = ($1)->limitOption;
+ $$ = n;
+ }
+ | offset_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $1;
+ n->limitCount = NULL;
+ n->limitOption = NULL;
+ $$ = n;
+ }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = NULL;
+ n->limitCount = NULL;
+ n->limitOption = NULL;
+ $$ = n;
+ }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $2;
+ n->limitOption = NULL;
+ $$ = n;
+ }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11597,9 +11661,26 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $3;
+ n->limitOption = makeString("EXACT_NUMBER");
+ $$ = n;
+ }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $3;
+ n->limitOption = makeString("PERCENTAGE");
+ $$ = n;
+ }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = makeIntConst(1, -1);
+ n->limitOption = NULL;
+ $$ = n;
+ }
;
offset_clause:
@@ -15192,6 +15273,7 @@ unreserved_keyword:
| PARTITION
| PASSING
| PASSWORD
+ | PERCENT
| PLANS
| POLICY
| PRECEDING
@@ -15856,6 +15938,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15894,6 +15977,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 2a6b2ff153..c8b2fbdc7a 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1707,7 +1707,7 @@ transformWhereClause(ParseState *pstate, Node *clause,
* constructName does not affect the semantics, but is used in error messages
*/
Node *
-transformLimitClause(ParseState *pstate, Node *clause,
+transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName)
{
Node *qual;
@@ -1716,8 +1716,10 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
-
- qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
+ if (limitOption == PERCENTAGE && (strcmp(constructName, "LIMIT") == 0))
+ qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
+ else
+ qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
/* LIMIT can't refer to any variables of the current query */
checkExprIsVarFree(pstate, qual, constructName);
diff --git a/src/backend/utils/sort/tuplestore.c b/src/backend/utils/sort/tuplestore.c
index 3fc7f92182..a03e568f1f 100644
--- a/src/backend/utils/sort/tuplestore.c
+++ b/src/backend/utils/sort/tuplestore.c
@@ -1100,6 +1100,39 @@ tuplestore_gettupleslot(Tuplestorestate *state, bool forward,
}
}
+/*
+ * tuplestore_gettupleslot_heaptuple
+ * It is similar to tuplestore_gettupleslot except it return stored HeapTuple
+ * instead of MinimalTuple
+ */
+bool
+tuplestore_gettupleslot_heaptuple(Tuplestorestate *state, bool forward,
+ bool copy, TupleTableSlot *slot)
+{
+ MinimalTuple tuple;
+ HeapTuple htuple;
+ bool should_free;
+
+ tuple = (MinimalTuple) tuplestore_gettuple(state, forward, &should_free);
+
+ if (tuple)
+ {
+ if (copy && !should_free)
+ {
+ tuple = heap_copy_minimal_tuple(tuple);
+ should_free = true;
+ }
+ htuple = heap_tuple_from_minimal_tuple(tuple);
+ ExecForceStoreHeapTuple(htuple, slot, should_free);
+ return true;
+ }
+ else
+ {
+ ExecClearTuple(slot);
+ return false;
+ }
+}
+
/*
* tuplestore_advance - exported function to adjust position without fetching
*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 99b9fa414f..64c7581ddc 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2302,8 +2302,16 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ float8 percent; /* percentage */
+ int64 backwardPosition; /* the number of tuple returned in
+ * backward scan */
+ int64 perExactCount; /* the number of tuple in excact count to
+ * return in OFFSET cluase */
+ bool reachEnd; /* if true, outerPlan executed until the end */
+ Tuplestorestate *tupleStore; /* holds the returned tuple */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 4e2fb39105..9f1e2fbfab 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -821,4 +821,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+}LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 94ded3c135..b158e9eaeb 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1595,6 +1596,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 441e64eca9..94f121525c 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1789,6 +1789,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 70f8b8e22b..3cfcf9f63d 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -953,6 +953,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index e70d6a3f18..c03a939f98 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -264,10 +264,11 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern void adjust_limit_rows_costs(double *rows,
Cost *startup_cost, Cost *total_cost,
- int64 offset_est, int64 count_est);
+ int64 offset_est, int64 count_est, LimitOption limitOption);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
Relids required_outer,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index e7aaddd50d..3395e558c4 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -56,7 +56,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators, Oid *grpCollations,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 00ace8425e..e1a8d703ab 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -299,6 +299,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, UNRESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h
index 42adc63d1f..e66510e965 100644
--- a/src/include/parser/parse_clause.h
+++ b/src/include/parser/parse_clause.h
@@ -22,7 +22,7 @@ extern int setTargetTable(ParseState *pstate, RangeVar *relation,
extern Node *transformWhereClause(ParseState *pstate, Node *clause,
ParseExprKind exprKind, const char *constructName);
-extern Node *transformLimitClause(ParseState *pstate, Node *clause,
+extern Node *transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName);
extern List *transformGroupClause(ParseState *pstate, List *grouplist,
List **groupingSets,
diff --git a/src/include/utils/tuplestore.h b/src/include/utils/tuplestore.h
index f9b6fcec29..eeefde3da4 100644
--- a/src/include/utils/tuplestore.h
+++ b/src/include/utils/tuplestore.h
@@ -47,7 +47,6 @@ typedef struct Tuplestorestate Tuplestorestate;
extern Tuplestorestate *tuplestore_begin_heap(bool randomAccess,
bool interXact,
int maxKBytes);
-
extern void tuplestore_set_eflags(Tuplestorestate *state, int eflags);
extern void tuplestore_puttupleslot(Tuplestorestate *state,
@@ -72,9 +71,12 @@ extern bool tuplestore_in_memory(Tuplestorestate *state);
extern bool tuplestore_gettupleslot(Tuplestorestate *state, bool forward,
bool copy, TupleTableSlot *slot);
+extern bool tuplestore_gettupleslot_heaptuple(Tuplestorestate *state, bool forward,
+ bool copy, TupleTableSlot *slot);
extern bool tuplestore_advance(Tuplestorestate *state, bool forward);
+
extern bool tuplestore_skiptuples(Tuplestorestate *state,
int64 ntuples, bool forward);
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index c18f547cbd..95f9f5f3d9 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -108,6 +108,63 @@ SELECT ''::text AS five, unique1, unique2, stringu1
| 904 | 793 | UIAAAA
(5 rows)
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 51 | 76 | ZBAAAA
+ | 52 | 985 | ACAAAA
+ | 53 | 196 | BCAAAA
+ | 54 | 356 | CCAAAA
+ | 55 | 627 | DCAAAA
+ | 56 | 54 | ECAAAA
+ | 57 | 942 | FCAAAA
+ | 58 | 114 | GCAAAA
+ | 59 | 593 | HCAAAA
+ | 60 | 483 | ICAAAA
+(10 rows)
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 61 | 560 | JCAAAA
+(1 row)
+
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+ three | unique1 | unique2 | stringu1
+-------+---------+---------+----------
+ | 121 | 700 | REAAAA
+ | 122 | 519 | SEAAAA
+ | 123 | 777 | TEAAAA
+ | 124 | 503 | UEAAAA
+ | 125 | 849 | VEAAAA
+ | 126 | 330 | WEAAAA
+ | 127 | 511 | XEAAAA
+ | 128 | 721 | YEAAAA
+ | 129 | 696 | ZEAAAA
+(9 rows)
+
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+ eleven | unique1 | unique2 | stringu1
+--------+---------+---------+----------
+ | 10 | 520 | KAAAAA
+ | 9 | 49 | JAAAAA
+ | 8 | 653 | IAAAAA
+ | 7 | 647 | HAAAAA
+ | 6 | 978 | GAAAAA
+(5 rows)
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -286,6 +343,46 @@ fetch all in c4;
----+----
(0 rows)
+declare c5 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c5;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
+fetch 1 in c5;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c5;
+ q1 | q2
+------------------+-----
+ 4567890123456789 | 123
+(1 row)
+
+fetch backward all in c5;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+ 123 | 456
+(2 rows)
+
+fetch backward 1 in c5;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c5;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
@@ -503,3 +600,19 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
45020 | 45020
(3 rows)
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
+ s1 | s2
+-------+-------
+ 45000 | 45000
+ 45010 | 45010
+ 45020 | 45020
+ 45030 | 45030
+ 45040 | 45040
+ 45050 | 45050
+ 45060 | 45060
+ 45070 | 45070
+ 45080 | 45080
+ 45090 | 45090
+(10 rows)
+
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index 2a313d80ca..051b21a099 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -31,6 +31,23 @@ SELECT ''::text AS five, unique1, unique2, stringu1
FROM onek
ORDER BY unique1 LIMIT 5 OFFSET 900;
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -38,7 +55,6 @@ select * from int8_tbl offset (case when random() < 0.5 then null::bigint end);
-- Test assorted cases involving backwards fetch from a LIMIT plan node
begin;
-
declare c1 cursor for select * from int8_tbl limit 10;
fetch all in c1;
fetch 1 in c1;
@@ -71,6 +87,14 @@ fetch backward all in c4;
fetch backward 1 in c4;
fetch all in c4;
+declare c5 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c5;
+fetch 1 in c5;
+fetch backward 1 in c5;
+fetch backward all in c5;
+fetch backward 1 in c5;
+fetch all in c5;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
@@ -141,3 +165,6 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
from tenk1 group by thousand order by thousand limit 3;
+
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
On Thu, Jun 27, 2019 at 9:06 PM Surafel Temesgen <surafel3000@gmail.com> wrote:
The attached patch include the fix for all the comment given
Hi Surafel,
There's a call to adjust_limit_rows_costs() hiding under
contrib/postgres_fdw, so this fails check-world.
--
Thomas Munro
https://enterprisedb.com
The following review has been posted through the commitfest application:
make installcheck-world: not tested
Implements feature: tested, passed
Spec compliant: not tested
Documentation: not tested
The basic functionality works as I expect. In the following example I would have guessed it would return 4 rows instead of 5. I don't mind that it uses ceil here, but think that deserves a mention in the documentation.
CREATE TABLE r100 (id INT);
INSERT INTO r100 SELECT generate_series(1, 100);
SELECT * FROM r100 FETCH FIRST 4.01 PERCENT ROWS ONLY;
id
----
1
2
3
4
5
(5 rows)
There's a missing space between the period and following sentence in src\backend\executor\nodeLimit.c
"previous time we got a different result.In PERCENTAGE option there are"
There's a missing space and the beginning "w" should be capitalized in doc\src\sgml\ref\select.sgml
with <literal>PERCENT</literal> count specifies the maximum number of rows to return
in percentage.<literal>ROW</literal>
Another missing space after the period.
previous time we got a different result.In PERCENTAGE option there are"
Ryan Lambert
The new status of this patch is: Waiting on Author
Hi Thomas,
Thank you for informing me
Hi Surafel,
There's a call to adjust_limit_rows_costs() hiding under
contrib/postgres_fdw, so this fails check-world.
Fixed . I also include the review given by Ryan in attached patch
regards
Surafel
Attachments:
percent-incremental-v5.patchtext/x-patch; charset=US-ASCII; name=percent-incremental-v5.patchDownload
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 1759b9e1b6..fb9c2f5319 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -3035,7 +3035,8 @@ estimate_path_cost_size(PlannerInfo *root,
if (fpextra && fpextra->has_limit)
{
adjust_limit_rows_costs(&rows, &startup_cost, &total_cost,
- fpextra->offset_est, fpextra->count_est);
+ fpextra->offset_est, fpextra->count_est,
+ EXACT_NUMBER);
retrieved_rows = rows;
}
}
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 06d611b64c..1c240a3dab 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1430,7 +1430,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1440,7 +1440,8 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
+ With <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage round up to the nearest integer.<literal>ROW</literal>
and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index baa669abe8..c503512588 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -21,6 +21,8 @@
#include "postgres.h"
+#include <math.h>
+
#include "executor/executor.h"
#include "executor/nodeLimit.h"
#include "miscadmin.h"
@@ -52,6 +54,7 @@ ExecLimit(PlanState *pstate)
*/
direction = node->ps.state->es_direction;
outerPlan = outerPlanState(node);
+ slot = node->subSlot;
/*
* The main logic is a simple state machine.
@@ -81,7 +84,15 @@ ExecLimit(PlanState *pstate)
/*
* Check for empty window; if so, treat like empty subplan.
*/
- if (node->count <= 0 && !node->noCount)
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (node->percent == 0.0)
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ }
+ else if (node->count <= 0 && !node->noCount)
{
node->lstate = LIMIT_EMPTY;
return NULL;
@@ -102,11 +113,32 @@ ExecLimit(PlanState *pstate)
node->lstate = LIMIT_EMPTY;
return NULL;
}
+
+ /*
+ * Count the number of rows to return in exact number so far
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ node->perExactCount = ceil(node->percent * node->position / 100.0);
+
+ if (node->perExactCount == node->perExactCount + 1)
+ node->perExactCount++;
+ }
node->subSlot = slot;
if (++node->position > node->offset)
break;
}
+ /*
+ * We may needed this tuple in backward scan so put it into
+ * tuplestore.
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ tuplestore_puttupleslot(node->tupleStore, slot);
+ tuplestore_advance(node->tupleStore, true);
+ }
+
/*
* Okay, we have the first tuple of the window.
*/
@@ -124,6 +156,106 @@ ExecLimit(PlanState *pstate)
case LIMIT_INWINDOW:
if (ScanDirectionIsForward(direction))
{
+ /*
+ * In case of coming back from backward scan the tuple is
+ * already in tuple store.
+ */
+ if (node->limitOption == PERCENTAGE && node->backwardPosition > 0)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ node->backwardPosition--;
+ return slot;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+
+ /*
+ * In PERCENTAGE case no need of executing outerPlan multiple
+ * times.
+ */
+ if (node->limitOption == PERCENTAGE && node->reachEnd)
+ {
+ node->lstate = LIMIT_WINDOWEND;
+
+ /*
+ * If we know we won't need to back up, we can release
+ * resources at this point.
+ */
+ if (!(node->ps.state->es_top_eflags & EXEC_FLAG_BACKWARD))
+ (void) ExecShutdownNode(outerPlan);
+
+ return NULL;
+ }
+
+ /*
+ * Return the tuple up to the number of exact count in OFFSET
+ * clause without percentage value consideration.
+ */
+ if (node->perExactCount > 0)
+ {
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ node->reachEnd = true;
+ return NULL;
+ }
+ tuplestore_puttupleslot(node->tupleStore, slot);
+ tuplestore_advance(node->tupleStore, true);
+ node->subSlot = slot;
+ node->position++;
+ node->perExactCount--;
+ return slot;
+ }
+
+ /*
+ * When in percentage mode, we need to see if we can get any
+ * additional rows from the subplan (enough to increase the
+ * node->count value).
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ /* loop until the node->count increments */
+ while (node->position - node->offset >= node->count)
+ {
+ int64 cnt;
+
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->reachEnd = true;
+ node->lstate = LIMIT_SUBPLANEOF;
+
+ /*
+ * The only operation from here is backward scan
+ * but there's no API to refetch the tuple at the
+ * current position. We have to move one tuple
+ * backward, and then we will scan forward for it
+ * for the first tuple and precede as usual
+ * for the rest
+ */
+ tuplestore_advance(node->tupleStore, false);
+ return NULL;
+ }
+
+ tuplestore_puttupleslot(node->tupleStore, slot);
+
+ cnt = tuplestore_tuple_count(node->tupleStore) + node->offset;
+
+ node->count = ceil(node->percent * cnt / 100.0);
+ }
+ }
+
/*
* Forwards scan, so check for stepping off end of window. If
* we are at the end of the window, return NULL without
@@ -145,17 +277,31 @@ ExecLimit(PlanState *pstate)
return NULL;
}
- /*
- * Get next tuple from subplan, if any.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
+ if (node->limitOption == PERCENTAGE)
{
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
+ while (node->position - node->offset < node->count)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ node->subSlot = slot;
+ node->position++;
}
- node->subSlot = slot;
- node->position++;
}
else
{
@@ -168,15 +314,29 @@ ExecLimit(PlanState *pstate)
node->lstate = LIMIT_WINDOWSTART;
return NULL;
}
-
- /*
- * Get previous tuple from subplan; there should be one!
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
+ /* In PERCENTAGE case the result is already in tuplestore */
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, false, true, slot))
+ {
+ node->subSlot = slot;
+ node->position--;
+ node->backwardPosition++;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+ /*
+ * Get previous tuple from subplan; there should be one!
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->position--;
+ }
}
break;
@@ -185,15 +345,32 @@ ExecLimit(PlanState *pstate)
return NULL;
/*
- * Backing up from subplan EOF, so re-fetch previous tuple; there
- * should be one! Note previous tuple must be in window.
+ * Scan forward for the first tuple
*/
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->lstate = LIMIT_INWINDOW;
- /* position does not change 'cause we didn't advance it before */
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+ /*
+ * Backing up from subplan EOF, so re-fetch previous tuple;
+ * there should be one! Note previous tuple must be in
+ * window.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ /* position does not change 'cause we didn't advance it before */
+ }
break;
case LIMIT_WINDOWEND:
@@ -283,12 +460,28 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
- node->noCount = false;
+ if (node->limitOption == PERCENTAGE)
+ {
+ /*
+ * We expect to return at least one row (unless there are no
+ * rows in the subplan), and we'll update this count later as
+ * we go.
+ */
+ node->count = 0;
+ node->percent = DatumGetFloat8(val);
+ if (node->percent < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_RESULT_OFFSET_CLAUSE),
+ errmsg("PERCENT must not be negative")));
+ }
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+ }
}
}
else
@@ -301,6 +494,9 @@ recompute_limits(LimitState *node)
/* Reset position to start-of-scan */
node->position = 0;
node->subSlot = NULL;
+ node->reachEnd = false;
+ node->backwardPosition = 0;
+ node->perExactCount = 0;
/* Set state-machine state */
node->lstate = LIMIT_RESCAN;
@@ -309,9 +505,11 @@ recompute_limits(LimitState *node)
* Notify child node about limit. Note: think not to "optimize" by
* skipping ExecSetTupleBound if compute_tuples_needed returns < 0. We
* must update the child node anyway, in case this is a rescan and the
- * previous time we got a different result.
+ * previous time we got a different result. In PERCENTAGE option there are
+ * no bound on the number of output tuples
*/
- ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
+ if (node->limitOption != PERCENTAGE)
+ ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
}
/*
@@ -374,6 +572,7 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
/*
* Initialize result type.
@@ -390,6 +589,9 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
limitstate->ps.ps_ProjInfo = NULL;
+ if (node->limitOption == PERCENTAGE)
+ limitstate->tupleStore = tuplestore_begin_heap(true, false, work_mem);
+
return limitstate;
}
@@ -405,6 +607,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->tupleStore != NULL)
+ tuplestore_end(node->tupleStore);
}
@@ -424,4 +628,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->tupleStore != NULL)
+ tuplestore_rescan(node->tupleStore);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 78deade89b..8307da7d3a 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1143,6 +1143,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3031,6 +3032,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3115,6 +3117,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 4f2ebe5118..2e14fa72b4 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -975,6 +975,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1049,6 +1050,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8400dd319e..3561bd8ae2 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -907,6 +907,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2104,6 +2105,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2698,6 +2700,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2908,6 +2911,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 6c2626ee62..85b514c9c2 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -278,6 +278,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2333,6 +2334,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 608d5adfed..6a6e733810 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2331,7 +2331,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2644,7 +2645,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6512,7 +6514,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6524,6 +6526,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index cb897cc7f4..9331dbc545 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2241,12 +2241,25 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
*/
if (parse->sortClause)
{
- current_rel = create_ordered_paths(root,
- current_rel,
- final_target,
- final_target_parallel_safe,
- have_postponed_srfs ? -1.0 :
- limit_tuples);
+
+ /*
+ * In PERCENTAGE option there are no bound on the number of output
+ * tuples
+ */
+ if (parse->limitOption == PERCENTAGE)
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ -1.0);
+ else
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ limit_tuples);
/* Fix things up if final_target contains SRFs */
if (parse->hasTargetSRFs)
adjust_paths_for_srfs(root, current_rel,
@@ -2309,6 +2322,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index d884d2bb00..3c49a716eb 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3570,6 +3570,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3591,6 +3592,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
@@ -3598,7 +3600,8 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
adjust_limit_rows_costs(&pathnode->path.rows,
&pathnode->path.startup_cost,
&pathnode->path.total_cost,
- offset_est, count_est);
+ offset_est, count_est,
+ limitOption);
return pathnode;
}
@@ -3624,7 +3627,8 @@ adjust_limit_rows_costs(double *rows, /* in/out parameter */
Cost *startup_cost, /* in/out parameter */
Cost *total_cost, /* in/out parameter */
int64 offset_est,
- int64 count_est)
+ int64 count_est,
+ LimitOption limitOption)
{
double input_rows = *rows;
Cost input_startup_cost = *startup_cost;
@@ -3657,6 +3661,19 @@ adjust_limit_rows_costs(double *rows, /* in/out parameter */
count_rows = (double) count_est;
else
count_rows = clamp_row_est(input_rows * 0.10);
+ if (limitOption == PERCENTAGE)
+ {
+ double per_count = DatumGetFloat8(count_est);
+
+ count_rows = clamp_row_est((input_rows * per_count) / 100);
+ if (rows > 0)
+ {
+ *startup_cost = count_rows *
+ input_total_cost / input_rows;
+ *total_cost = input_total_cost +
+ (count_rows * 0.1);
+ }
+ }
if (count_rows > *rows)
count_rows = *rows;
if (input_rows > 0)
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index b13c246183..2c7feff026 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1288,10 +1288,11 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
}
/* transform LIMIT */
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1536,10 +1537,11 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_ORDER_BY,
false /* allow SQL92 rules */ );
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1770,10 +1772,11 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
parser_errposition(pstate,
exprLocation(list_nth(qry->targetList, tllen)))));
- qry->limitOffset = transformLimitClause(pstate, limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, limitCount,
+ qry->limitCount = transformLimitClause(pstate, limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 8311b1dd46..91fb2e4143 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -127,6 +127,21 @@ typedef struct ImportQual
List *table_names;
} ImportQual;
+/* Private struct for the result of opt_select_limit production */
+typedef struct SelectLimit
+{
+ Node *limitOffset;
+ Node *limitCount;
+ void *limitOption;
+} SelectLimit;
+
+/* Private struct for the result of limit_clause production */
+typedef struct LimitClause
+{
+ Node *limitCount;
+ void *limitOption;
+} LimitClause;
+
/* ConstraintAttributeSpec yields an integer bitmask of these flags: */
#define CAS_NOT_DEFERRABLE 0x01
#define CAS_DEFERRABLE 0x02
@@ -165,6 +180,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -242,6 +258,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
PartitionSpec *partspec;
PartitionBoundSpec *partboundspec;
RoleSpec *rolespec;
+ struct SelectLimit *SelectLimit;
+ struct LimitClause *LimitClause;
}
%type <node> stmt schema_stmt
@@ -373,6 +391,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <ival> import_qualification_type
%type <importqual> import_qualification
%type <node> vacuum_relation
+%type <SelectLimit> opt_select_limit select_limit
+%type <LimitClause> limit_clause
%type <list> stmtblock stmtmulti
OptTableElementList TableElementList OptInherit definition
@@ -393,8 +413,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
- opt_select_limit opclass_item_list opclass_drop_list
+ reloption_list group_clause TriggerFuncArgs
+ opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
prep_type_clause
@@ -455,7 +475,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -667,7 +687,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11225,14 +11245,15 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
| select_clause opt_sort_clause for_locking_clause opt_select_limit
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
- list_nth($4, 0), list_nth($4, 1),
+ ($4)->limitOffset, ($4)->limitCount,
+ ($4)->limitOption,
NULL,
yyscanner);
$$ = $1;
@@ -11240,7 +11261,8 @@ select_no_parens:
| select_clause opt_sort_clause select_limit opt_for_locking_clause
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
- list_nth($3, 0), list_nth($3, 1),
+ ($3)->limitOffset, ($3)->limitCount,
+ ($3)->limitOption,
NULL,
yyscanner);
$$ = $1;
@@ -11249,7 +11271,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11257,14 +11279,15 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
| with_clause select_clause opt_sort_clause for_locking_clause opt_select_limit
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
- list_nth($5, 0), list_nth($5, 1),
+ ($5)->limitOffset, ($5)->limitCount,
+ ($5)->limitOption,
$1,
yyscanner);
$$ = $2;
@@ -11272,7 +11295,8 @@ select_no_parens:
| with_clause select_clause opt_sort_clause select_limit opt_for_locking_clause
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
- list_nth($4, 0), list_nth($4, 1),
+ ($4)->limitOffset, ($4)->limitCount,
+ ($4)->limitOption,
$1,
yyscanner);
$$ = $2;
@@ -11566,20 +11590,60 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $2;
+ n->limitCount = ($1)->limitCount;
+ n->limitOption = ($1)->limitOption;
+ $$ = n;
+ }
+ | offset_clause limit_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $1;
+ n->limitCount = ($2)->limitCount;
+ n->limitOption = ($2)->limitOption;
+ $$ = n;
+ }
+ | limit_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = NULL;
+ n->limitCount = ($1)->limitCount;
+ n->limitOption = ($1)->limitOption;
+ $$ = n;
+ }
+ | offset_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $1;
+ n->limitCount = NULL;
+ n->limitOption = NULL;
+ $$ = n;
+ }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = NULL;
+ n->limitCount = NULL;
+ n->limitOption = NULL;
+ $$ = n;
+ }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $2;
+ n->limitOption = NULL;
+ $$ = n;
+ }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11597,9 +11661,26 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $3;
+ n->limitOption = makeString("EXACT_NUMBER");
+ $$ = n;
+ }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $3;
+ n->limitOption = makeString("PERCENTAGE");
+ $$ = n;
+ }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = makeIntConst(1, -1);
+ n->limitOption = NULL;
+ $$ = n;
+ }
;
offset_clause:
@@ -15192,6 +15273,7 @@ unreserved_keyword:
| PARTITION
| PASSING
| PASSWORD
+ | PERCENT
| PLANS
| POLICY
| PRECEDING
@@ -15856,6 +15938,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15894,6 +15977,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 2a6b2ff153..c8b2fbdc7a 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1707,7 +1707,7 @@ transformWhereClause(ParseState *pstate, Node *clause,
* constructName does not affect the semantics, but is used in error messages
*/
Node *
-transformLimitClause(ParseState *pstate, Node *clause,
+transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName)
{
Node *qual;
@@ -1716,8 +1716,10 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
-
- qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
+ if (limitOption == PERCENTAGE && (strcmp(constructName, "LIMIT") == 0))
+ qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
+ else
+ qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
/* LIMIT can't refer to any variables of the current query */
checkExprIsVarFree(pstate, qual, constructName);
diff --git a/src/backend/utils/sort/tuplestore.c b/src/backend/utils/sort/tuplestore.c
index 3fc7f92182..a03e568f1f 100644
--- a/src/backend/utils/sort/tuplestore.c
+++ b/src/backend/utils/sort/tuplestore.c
@@ -1100,6 +1100,39 @@ tuplestore_gettupleslot(Tuplestorestate *state, bool forward,
}
}
+/*
+ * tuplestore_gettupleslot_heaptuple
+ * It is similar to tuplestore_gettupleslot except it return stored HeapTuple
+ * instead of MinimalTuple
+ */
+bool
+tuplestore_gettupleslot_heaptuple(Tuplestorestate *state, bool forward,
+ bool copy, TupleTableSlot *slot)
+{
+ MinimalTuple tuple;
+ HeapTuple htuple;
+ bool should_free;
+
+ tuple = (MinimalTuple) tuplestore_gettuple(state, forward, &should_free);
+
+ if (tuple)
+ {
+ if (copy && !should_free)
+ {
+ tuple = heap_copy_minimal_tuple(tuple);
+ should_free = true;
+ }
+ htuple = heap_tuple_from_minimal_tuple(tuple);
+ ExecForceStoreHeapTuple(htuple, slot, should_free);
+ return true;
+ }
+ else
+ {
+ ExecClearTuple(slot);
+ return false;
+ }
+}
+
/*
* tuplestore_advance - exported function to adjust position without fetching
*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 99b9fa414f..64c7581ddc 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2302,8 +2302,16 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ float8 percent; /* percentage */
+ int64 backwardPosition; /* the number of tuple returned in
+ * backward scan */
+ int64 perExactCount; /* the number of tuple in excact count to
+ * return in OFFSET cluase */
+ bool reachEnd; /* if true, outerPlan executed until the end */
+ Tuplestorestate *tupleStore; /* holds the returned tuple */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 4e2fb39105..9f1e2fbfab 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -821,4 +821,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+}LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 94ded3c135..b158e9eaeb 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1595,6 +1596,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 441e64eca9..94f121525c 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1789,6 +1789,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 70f8b8e22b..3cfcf9f63d 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -953,6 +953,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index e70d6a3f18..c03a939f98 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -264,10 +264,11 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern void adjust_limit_rows_costs(double *rows,
Cost *startup_cost, Cost *total_cost,
- int64 offset_est, int64 count_est);
+ int64 offset_est, int64 count_est, LimitOption limitOption);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
Relids required_outer,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index e7aaddd50d..3395e558c4 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -56,7 +56,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators, Oid *grpCollations,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 00ace8425e..e1a8d703ab 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -299,6 +299,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, UNRESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h
index 42adc63d1f..e66510e965 100644
--- a/src/include/parser/parse_clause.h
+++ b/src/include/parser/parse_clause.h
@@ -22,7 +22,7 @@ extern int setTargetTable(ParseState *pstate, RangeVar *relation,
extern Node *transformWhereClause(ParseState *pstate, Node *clause,
ParseExprKind exprKind, const char *constructName);
-extern Node *transformLimitClause(ParseState *pstate, Node *clause,
+extern Node *transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName);
extern List *transformGroupClause(ParseState *pstate, List *grouplist,
List **groupingSets,
diff --git a/src/include/utils/tuplestore.h b/src/include/utils/tuplestore.h
index f9b6fcec29..eeefde3da4 100644
--- a/src/include/utils/tuplestore.h
+++ b/src/include/utils/tuplestore.h
@@ -47,7 +47,6 @@ typedef struct Tuplestorestate Tuplestorestate;
extern Tuplestorestate *tuplestore_begin_heap(bool randomAccess,
bool interXact,
int maxKBytes);
-
extern void tuplestore_set_eflags(Tuplestorestate *state, int eflags);
extern void tuplestore_puttupleslot(Tuplestorestate *state,
@@ -72,9 +71,12 @@ extern bool tuplestore_in_memory(Tuplestorestate *state);
extern bool tuplestore_gettupleslot(Tuplestorestate *state, bool forward,
bool copy, TupleTableSlot *slot);
+extern bool tuplestore_gettupleslot_heaptuple(Tuplestorestate *state, bool forward,
+ bool copy, TupleTableSlot *slot);
extern bool tuplestore_advance(Tuplestorestate *state, bool forward);
+
extern bool tuplestore_skiptuples(Tuplestorestate *state,
int64 ntuples, bool forward);
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index c18f547cbd..95f9f5f3d9 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -108,6 +108,63 @@ SELECT ''::text AS five, unique1, unique2, stringu1
| 904 | 793 | UIAAAA
(5 rows)
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 51 | 76 | ZBAAAA
+ | 52 | 985 | ACAAAA
+ | 53 | 196 | BCAAAA
+ | 54 | 356 | CCAAAA
+ | 55 | 627 | DCAAAA
+ | 56 | 54 | ECAAAA
+ | 57 | 942 | FCAAAA
+ | 58 | 114 | GCAAAA
+ | 59 | 593 | HCAAAA
+ | 60 | 483 | ICAAAA
+(10 rows)
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 61 | 560 | JCAAAA
+(1 row)
+
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+ three | unique1 | unique2 | stringu1
+-------+---------+---------+----------
+ | 121 | 700 | REAAAA
+ | 122 | 519 | SEAAAA
+ | 123 | 777 | TEAAAA
+ | 124 | 503 | UEAAAA
+ | 125 | 849 | VEAAAA
+ | 126 | 330 | WEAAAA
+ | 127 | 511 | XEAAAA
+ | 128 | 721 | YEAAAA
+ | 129 | 696 | ZEAAAA
+(9 rows)
+
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+ eleven | unique1 | unique2 | stringu1
+--------+---------+---------+----------
+ | 10 | 520 | KAAAAA
+ | 9 | 49 | JAAAAA
+ | 8 | 653 | IAAAAA
+ | 7 | 647 | HAAAAA
+ | 6 | 978 | GAAAAA
+(5 rows)
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -286,6 +343,46 @@ fetch all in c4;
----+----
(0 rows)
+declare c5 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c5;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
+fetch 1 in c5;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c5;
+ q1 | q2
+------------------+-----
+ 4567890123456789 | 123
+(1 row)
+
+fetch backward all in c5;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+ 123 | 456
+(2 rows)
+
+fetch backward 1 in c5;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c5;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
@@ -503,3 +600,19 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
45020 | 45020
(3 rows)
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
+ s1 | s2
+-------+-------
+ 45000 | 45000
+ 45010 | 45010
+ 45020 | 45020
+ 45030 | 45030
+ 45040 | 45040
+ 45050 | 45050
+ 45060 | 45060
+ 45070 | 45070
+ 45080 | 45080
+ 45090 | 45090
+(10 rows)
+
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index 2a313d80ca..051b21a099 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -31,6 +31,23 @@ SELECT ''::text AS five, unique1, unique2, stringu1
FROM onek
ORDER BY unique1 LIMIT 5 OFFSET 900;
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -38,7 +55,6 @@ select * from int8_tbl offset (case when random() < 0.5 then null::bigint end);
-- Test assorted cases involving backwards fetch from a LIMIT plan node
begin;
-
declare c1 cursor for select * from int8_tbl limit 10;
fetch all in c1;
fetch 1 in c1;
@@ -71,6 +87,14 @@ fetch backward all in c4;
fetch backward 1 in c4;
fetch all in c4;
+declare c5 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c5;
+fetch 1 in c5;
+fetch backward 1 in c5;
+fetch backward all in c5;
+fetch backward 1 in c5;
+fetch all in c5;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
@@ -141,3 +165,6 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
from tenk1 group by thousand order by thousand limit 3;
+
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
Hi Surafel,
The v5 patch [1]/messages/by-id/attachment/102333/percent-incremental-v5.patch applies cleanly and passes make installcheck-world.
My concern with this is its performance. As a user I would expect using
this feature to enable queries to run faster than they would simply pulling
the full table. I tested on tables ranging from 10k rows up to 10 million
with the same basic result that using FETCH FIRST N PERCENT is slower than
using the full table. At best it ran slightly slower than querying the
full table, at worst it increased execution times by 1400% when using a
large high percentage (95%).
Testing was on a DigitalOcean 2 CPU, 2GB RAM droplet. All queries were
executed multiple times (6+) with EXPLAIN (ANALYZE, COSTS) and /timing
enabled, query plans provided are from the faster end of each query.
Create the test table:
DROP TABLE IF EXISTS r10mwide;
CREATE TABLE r10mwide AS
SELECT generate_series(1, 10000000)::INT AS id,
random() AS v1,
random() AS v2,
random() AS v3,
random() AS v4,
random() AS v5,
random() AS v6,
random() AS v7,
random() AS v8,
random() AS v9,
random() AS v10
;
VACUUM ANALYZE;
Start with a baseline query from the full table, the limiting will be done
within the CTE as I would typically do in a production query. The outer
query does basic aggregates. Below I provide each full query for easy
copy/paste but the only real change is how the inner results are limited.
This runs in 2.2s off the full table.
EXPLAIN (ANALYZE, COSTS)
WITH t AS (
SELECT id, v1, v2
FROM r10mwide
) SELECT AVG(v1), MIN(v1), AVG(v1 + v2) FROM t
;
QUERY
PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------
Finalize Aggregate (cost=227192.07..227192.08 rows=1 width=24) (actual
time=2230.419..2230.419 rows=1 loops=1)
-> Gather (cost=227191.84..227192.05 rows=2 width=72) (actual
time=2228.321..2231.225 rows=3 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Partial Aggregate (cost=226191.84..226191.85 rows=1 width=72)
(actual time=2218.754..2218.754 rows=1 loops=3)
-> Parallel Seq Scan on r10mwide (cost=0.00..184524.92
rows=4166692 width=16) (actual time=0.032..934.652 rows=3333333 loops=3)
Planning Time: 0.116 ms
Execution Time: 2231.935 ms
(8 rows)
Time: 2232.459 ms (00:02.232)
It did use parallel, since the FETCH FIRST queries apparently won't I
turned that off and ran it again.
SET max_parallel_workers_per_gather = 0;
New query plan for full table, no parallel, just over 4 seconds.
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------
Aggregate (cost=342859.21..342859.22 rows=1 width=24) (actual
time=4253.861..4253.862 rows=1 loops=1)
-> Seq Scan on r10mwide (cost=0.00..242858.60 rows=10000060 width=16)
(actual time=0.062..1728.346 rows=10000000 loops=1)
Planning Time: 0.082 ms
Execution Time: 4253.918 ms
(4 rows)
The following uses the explicit row count to pull 100k rows (1%) and gives
a massive improvement in speed, this query ranged from 55- 120ms. This is
the type of speedup I would expect, and it beats the full-table query hands
down, regardless of parallel query.
EXPLAIN (ANALYZE, COSTS)
WITH t AS (
SELECT id, v1, v2
FROM r10mwide
FETCH FIRST 100000 ROWS ONLY
) SELECT AVG(v1), MIN(v1), AVG(v1 + v2) FROM t
;
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------
Aggregate (cost=4428.58..4428.59 rows=1 width=24) (actual
time=55.963..55.963 rows=1 loops=1)
-> Limit (cost=0.00..2428.57 rows=100000 width=20) (actual
time=0.041..35.725 rows=100000 loops=1)
-> Seq Scan on r10mwide (cost=0.00..242858.60 rows=10000060
width=20) (actual time=0.039..24.796 rows=100000 loops=1)
Planning Time: 0.134 ms
Execution Time: 56.032 ms
(5 rows)
Time: 57.262 ms
Using FETCH FIRST 1 PERCENT it takes over 7 seconds, 71% slower than
querying the full table.
EXPLAIN (ANALYZE, COSTS)
WITH t AS (
SELECT id, v1, v2
FROM r10mwide
FETCH FIRST 1 PERCENT ROWS ONLY
) SELECT AVG(v1), MIN(v1), AVG(v1 + v2) FROM t
;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------
Aggregate (cost=6857.22..6857.23 rows=1 width=24) (actual
time=7112.205..7112.206 rows=1 loops=1)
-> Limit (cost=2428.60..4857.19 rows=100001 width=20) (actual
time=0.036..7047.394 rows=100000 loops=1)
-> Seq Scan on r10mwide (cost=0.00..242858.60 rows=10000060
width=20) (actual time=0.033..3072.881 rows=10000000 loops=1)
Planning Time: 0.132 ms
Execution Time: 7214.302 ms
(5 rows)
Time: 7215.278 ms (00:07.215)
Cranking the percentage up to 95% took 59-75 seconds.
EXPLAIN (ANALYZE, COSTS)
WITH t AS (
SELECT id, v1, v2
FROM r10mwide
FETCH FIRST 95 PERCENT ROWS ONLY
) SELECT AVG(v1), MIN(v1), AVG(v1 + v2) FROM t
;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------
Aggregate (cost=651432.48..651432.49 rows=1 width=24) (actual
time=58981.043..58981.044 rows=1 loops=1)
-> Limit (cost=230715.67..461431.34 rows=9500057 width=20) (actual
time=0.017..55799.389 rows=9500000 loops=1)
-> Seq Scan on r10mwide (cost=0.00..242858.60 rows=10000060
width=20) (actual time=0.014..3847.146 rows=10000000 loops=1)
Planning Time: 0.117 ms
Execution Time: 59079.680 ms
(5 rows)
Even taking it way down to .001% (100 rows!) didn't run fast by any
measure, more than 6 seconds.
EXPLAIN (ANALYZE, COSTS)
WITH t AS (
SELECT id, v1, v2
FROM r10mwide
FETCH FIRST .001 PERCENT ROWS ONLY
) SELECT AVG(v1), MIN(v1), AVG(v1 + v2) FROM t
;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------
Aggregate (cost=6.86..6.87 rows=1 width=24) (actual
time=6414.406..6414.406 rows=1 loops=1)
-> Limit (cost=2.43..4.86 rows=100 width=20) (actual
time=0.021..6413.981 rows=100 loops=1)
-> Seq Scan on r10mwide (cost=0.00..242858.60 rows=10000060
width=20) (actual time=0.017..3086.504 rows=10000000 loops=1)
Planning Time: 0.168 ms
Execution Time: 6495.686 ms
[1]: /messages/by-id/attachment/102333/percent-incremental-v5.patch
/messages/by-id/attachment/102333/percent-incremental-v5.patch
*Ryan Lambert*
RustProof Labs
On Mon, Jul 8, 2019 at 1:09 AM Surafel Temesgen <surafel3000@gmail.com>
wrote:
Show quoted text
Hi Thomas,
Thank you for informing meHi Surafel,
There's a call to adjust_limit_rows_costs() hiding under
contrib/postgres_fdw, so this fails check-world.Fixed . I also include the review given by Ryan in attached patch
regards
Surafel
Hi Ryan,
On Tue, Jul 9, 2019 at 1:27 AM Ryan Lambert <ryan@rustprooflabs.com> wrote:
Hi Surafel,
The v5 patch [1] applies cleanly and passes make installcheck-world.
My concern with this is its performance. As a user I would expect using
this feature to enable queries to run faster than they would simply pulling
the full table. I tested on tables ranging from 10k rows up to 10 million
with the same basic result that using FETCH FIRST N PERCENT is slower than
using the full table. At best it ran slightly slower than querying the
full table, at worst it increased execution times by 1400% when using a
large high percentage (95%).
The cost of FITCH FIRST N PERCENT execution in current implementation is
the cost of pulling the full table plus the cost of storing and fetching
the tuple from tuplestore so it can not perform better than pulling the
full table in any case . This is because we can't determined the number of
rows to return without executing the plan until the end. We can find the
estimation of rows that will be return in planner estimation but that is
not exact.
regards
Surafel
Surafel,
The cost of FITCH FIRST N PERCENT execution in current implementation is
the cost of pulling the full table plus the cost of storing and fetching
the tuple from tuplestore so it can > not perform better than pulling the
full table in any case . This is because we can't determined the number of
rows to return without executing the plan until the end. We can find the >
estimation of rows that will be return in planner estimation but that is
not exact.
Ok, I can live with that for the normal use cases. This example from the
end of my previous message using 95% seems like a problem still, I don't
like syntax that unexpectedly kills performance like this one. If this
can't be improved in the initial release of the feature I'd suggest we at
least make a strong disclaimer in the docs, along the lines of:
"It is possible for FETCH FIRST N PERCENT to create poorly performing query
plans when the N supplied exceeds 50 percent. In these cases query
execution can take an order of magnitude longer to execute than simply
returning the full table. If performance is critical using an explicit row
count for limiting is recommended."
I'm not certain the 50 percent is the true threshold of where things start
to fall apart, I just used that as a likely guess for now. I can do some
more testing this week to identify where things start falling apart
performance wise. Thanks,
EXPLAIN (ANALYZE, COSTS)
WITH t AS (
SELECT id, v1, v2
FROM r10mwide
FETCH FIRST 95 PERCENT ROWS ONLY
) SELECT AVG(v1), MIN(v1), AVG(v1 + v2) FROM t
;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------
Aggregate (cost=651432.48..651432.49 rows=1 width=24) (actual
time=58981.043..58981.044 rows=1 loops=1)
-> Limit (cost=230715.67..461431.34 rows=9500057 width=20) (actual
time=0.017..55799.389 rows=9500000 loops=1)
-> Seq Scan on r10mwide (cost=0.00..242858.60 rows=10000060
width=20) (actual time=0.014..3847.146 rows=10000000 loops=1)
Planning Time: 0.117 ms
Execution Time: 59079.680 ms
(5 rows)
Ryan Lambert
Show quoted text
I did some more testing. I initialized a database with 1 million rows with
indexes and joins to test against and ran pgbench with a few different
settings for % to return. I started with a base query not utilizing the
new functionality. The queries used are similar to my prior examples, code
at [1]https://github.com/rustprooflabs/pgbench-tests.
createdb bench_test
psql -d bench_test -f init/reporting.sql -v scale=10
The following provided 3.21 TPS and an average latency of 623. The
"per_change_" columns in the table below use those values.
pgbench -c 2 -j 2 -T 600 -P 60 -s 10 \
-f tests/reporting1.sql bench_test
The remainder of the tests use the following, only adjusting fetch_percent
value:
pgbench -c 2 -j 2 -T 600 -P 60 -s 10 \
--define=fetch_percent=1 \
-f tests/reporting_fetch_percent.sql \
bench_test
Returning 1% it runs well. By 10% the TPS drops by 30% while the average
latency increases by 43%. When returning 95% of the table latency has
increased by 548%.
fetch_percent | tps | latency_avg_ms | per_change_tps | per_change_latency
---------------+------+----------------+----------------+--------------------
1 | 3.37 | 593 | 0.05 | -0.05
5 | 2.85 | 700 | -0.11 | 0.12
10 | 2.24 | 891 | -0.30 | 0.43
25 | 1.40 | 1423 | -0.56 | 1.28
45 | 0.93 | 2147 | -0.71 | 2.45
95 | 0.49 | 4035 | -0.85 | 5.48
I manually tested the inner select queries without the outer aggregation
thinking it might be a different story with a simple select and no CTE.
Unfortunately it showed the same overall characteristics. 1% returns in
about 550 ms, 45% took 1950, and 95% took 4050.
[1]: https://github.com/rustprooflabs/pgbench-tests
Ryan
Show quoted text
Hello.
At Tue, 9 Jul 2019 21:56:32 -0600, Ryan Lambert <ryan@rustprooflabs.com> wrote in <CAN-V+g-rwFp=xQEjOwbJuggNLegMi1qDhaJt3h1Eqm16yqwqmw@mail.gmail.com>
I did some more testing. I initialized a database with 1 million rows with
indexes and joins to test against and ran pgbench with a few different
settings for % to return. I started with a base query not utilizing the
new functionality. The queries used are similar to my prior examples, code
at [1].createdb bench_test
psql -d bench_test -f init/reporting.sql -v scale=10The following provided 3.21 TPS and an average latency of 623. The
"per_change_" columns in the table below use those values.pgbench -c 2 -j 2 -T 600 -P 60 -s 10 \
-f tests/reporting1.sql bench_testThe remainder of the tests use the following, only adjusting fetch_percent
value:pgbench -c 2 -j 2 -T 600 -P 60 -s 10 \
--define=fetch_percent=1 \
-f tests/reporting_fetch_percent.sql \
bench_testReturning 1% it runs well. By 10% the TPS drops by 30% while the average
latency increases by 43%. When returning 95% of the table latency has
increased by 548%.fetch_percent | tps | latency_avg_ms | per_change_tps | per_change_latency
---------------+------+----------------+----------------+--------------------
1 | 3.37 | 593 | 0.05 | -0.05
5 | 2.85 | 700 | -0.11 | 0.12
10 | 2.24 | 891 | -0.30 | 0.43
25 | 1.40 | 1423 | -0.56 | 1.28
45 | 0.93 | 2147 | -0.71 | 2.45
95 | 0.49 | 4035 | -0.85 | 5.48I manually tested the inner select queries without the outer aggregation
thinking it might be a different story with a simple select and no CTE.
Unfortunately it showed the same overall characteristics. 1% returns in
about 550 ms, 45% took 1950, and 95% took 4050.
It is seen by a simpler test.
create table t as select a from generate_series(0, 99999) a;
analyze t;
explain analyze select * from t order by a desc;
Execution Time: 116.613 ms
explain analyze select * from t order by a desc fetch first 1 percent rows only;
Execution Time: 158.458 ms
explain analyze select * from t order by a desc fetch first 100 percent rows only;
Execution Time: 364.442 ms
I didn't looked closer to the version. Fetching from tuplestore
and returning all tuples costs 206ms and it is exceeding the cost
of fething of the whole table and returning all tuples. I don't
believe tuplestore that isn't splling out to disk is so slower
than (cached) table access.
Other than that, we can rip the clause if it is 100%
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
Hello.
At Wed, 10 Jul 2019 15:02:57 +0900 (Tokyo Standard Time), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in <20190710.150257.260806103.horikyota.ntt@gmail.com>
It is seen by a simpler test.
create table t as select a from generate_series(0, 99999) a;
analyze t;
explain analyze select * from t order by a desc;
Execution Time: 116.613 ms
explain analyze select * from t order by a desc fetch first 1 percent rows only;
Execution Time: 158.458 ms
explain analyze select * from t order by a desc fetch first 100 percent rows only;
Execution Time: 364.442 msI didn't looked closer to the version. Fetching from tuplestore
and returning all tuples costs 206ms and it is exceeding the cost
of fething of the whole table and returning all tuples. I don't
believe tuplestore that isn't splling out to disk is so slower
than (cached) table access.Other than that, we can rip the clause if it is 100%
As a more significant point, I found that the first query in the
aboves runs faster by about 10-18% on master(unpatched).
explain analyze select * from t order by a desc;
Execution Time: 96.690 ms
But perf didn't give me useful information.
patched:
11857 11.7065 postgres qsort_ssup
9026 8.9114 postgres ApplySortComparator
6443 6.3612 [vdso] (tgid:8388 range:0x7ffed49ed000-0x7ffed49eefff) [vdso] (tgid:8388 range:0x7ffed49ed000-0x7ffed49eefff)
5826 5.7520 postgres btint4fastcmp
4699 4.6393 no-vmlinux /no-vmlinux
3451 3.4072 libc-2.17.so __memcpy_ssse3_back
3270 3.2285 postgres LogicalTapeWrite
2972 2.9343 postgres copytup_heap
2961 2.9234 postgres readtup_heap
2769 2.7338 postgres LogicalTapeRead
2457 2.4258 postgres GetMemoryChunkContext
2147 2.1197 postgres InstrStopNode
2021 1.9953 postgres heapgettup_pagemode
1583 1.5629 postgres writetup_heap
1555 1.5353 postgres tuplesort_gettuple_common
1508 1.4889 postgres AllocSetAlloc
...
master:
12932 12.0168 postgres qsort_ssup
9491 8.8193 postgres ApplySortComparator
6705 6.2305 postgres btint4fastcmp
6557 6.0930 [vdso] (tgid:6341 range:0x7ffdd0315000-0x7ffdd0316fff) [vdso] (tgid:6341 range:0x7ffdd0315000-0x7ffdd0316fff)
4874 4.5291 no-vmlinux /no-vmlinux
4059 3.7717 postgres readtup_heap
3707 3.4447 libc-2.17.so __memcpy_ssse3_back
3583 3.3294 postgres LogicalTapeWrite
3382 3.1427 postgres LogicalTapeRead
3001 2.7886 postgres copytup_heap
2522 2.3435 postgres GetMemoryChunkContext
2464 2.2896 postgres heapgettup_pagemode
2115 1.9653 postgres InstrStopNode
1847 1.7163 postgres tuplesort_gettuple_common
1652 1.5351 postgres writetup_heap
1565 1.4542 postgres AllocSetAlloc
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
On 2019-Jul-08, Surafel Temesgen wrote:
Hi Thomas,
Thank you for informing meHi Surafel,
There's a call to adjust_limit_rows_costs() hiding under
contrib/postgres_fdw, so this fails check-world.Fixed . I also include the review given by Ryan in attached patch
What's with the new tuplestore function for getting heap tuples? That
looks really odd.
--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Hi Alvaro,
On Wed, Jul 10, 2019 at 6:44 PM Alvaro Herrera <alvherre@2ndquadrant.com>
wrote:
What's with the new tuplestore function for getting heap tuples? That
looks really odd.
Previously I create new TTSOpsMinimalTuple type slots for every tuple
returned in order to fetch it from tuplestore because tuplestore store
tuple in MinimalTuple format and created slot format changes in subsequent
operation. This case resource leak as Andres mention and I create this
function to remove the need for creating new slot for every returned tuple
because of form difference
regards
Surafel
Hi Ryan,
On Tue, Jul 9, 2019 at 4:13 PM Ryan Lambert <ryan@rustprooflabs.com> wrote:
"It is possible for FETCH FIRST N PERCENT to create poorly performing
query plans when the N supplied exceeds 50 percent. In these cases query
execution can take an order of magnitude longer to execute than simply
returning the full table. If performance is critical using an explicit row
count for limiting is recommended."
I don’t understand how fetch first n percent functionality can be replaced
with explicit row count limiting. There may be a way to do it in a client
side but we can not be sure of its performance advantage
regards
Surafel
Surafel,
On Wed, Jul 17, 2019 at 3:45 AM Surafel Temesgen <surafel3000@gmail.com>
wrote:
Hi Ryan,
On Tue, Jul 9, 2019 at 4:13 PM Ryan Lambert <ryan@rustprooflabs.com>
wrote:"It is possible for FETCH FIRST N PERCENT to create poorly performing
query plans when the N supplied exceeds 50 percent. In these cases query
execution can take an order of magnitude longer to execute than simply
returning the full table. If performance is critical using an explicit row
count for limiting is recommended."I don’t understand how fetch first n percent functionality can be replaced
with explicit row count limiting. There may be a way to do it in a client
side but we can not be sure of its performance advantageregards
Surafel
I was suggesting a warning in the documentation so users aren't caught
unaware about the performance characteristics. My first version was very
rough, how about adding this in doc/src/sgml/ref/select.sgml?
"Using <literal>PERCENT</literal> is best suited to returning single-digit
percentages of the query's total row count."
The following paragraphs in that same section give suggestions and warnings
regarding LIMIT and OFFSET usage, I think this is more in line with the
wording of those existing warnings.
Other than that, we can rip the clause if it is 100%
You mean if PERCENT=100 it should short circuit and run the query
normally? I like that.
That got me thinking, I didn't check what happens with PERCENT>100, I'll
try to test that soon.
Thanks,
Ryan
Show quoted text
Hi Ryan,
sorry for not be fast to replay
I was suggesting a warning in the documentation so users aren't caught
unaware about the performance characteristics. My first version was very
rough, how about adding this in doc/src/sgml/ref/select.sgml?"Using <literal>PERCENT</literal> is best suited to returning single-digit
percentages of the query's total row count."The following paragraphs in that same section give suggestions and
warnings regarding LIMIT and OFFSET usage, I think this is more in line
with the wording of those existing warnings.
Agreed and done
Other than that, we can rip the clause if it is 100%
You mean if PERCENT=100 it should short circuit and run the query
normally? I like that.
The patch did not did it automatically. Its query writer obligation to do
that currently
regards
Surafel
Attachments:
percent-incremental-v6.patchtext/x-patch; charset=US-ASCII; name=percent-incremental-v6.patchDownload
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 1759b9e1b6..fb9c2f5319 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -3035,7 +3035,8 @@ estimate_path_cost_size(PlannerInfo *root,
if (fpextra && fpextra->has_limit)
{
adjust_limit_rows_costs(&rows, &startup_cost, &total_cost,
- fpextra->offset_est, fpextra->count_est);
+ fpextra->offset_est, fpextra->count_est,
+ EXACT_NUMBER);
retrieved_rows = rows;
}
}
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 06d611b64c..4ef42df51d 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1430,7 +1430,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1440,8 +1440,10 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
- and <literal>ROWS</literal> as well as <literal>FIRST</literal>
+ With <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage round up to the nearest integer. Using <literal>PERCENT</literal>
+ is best suited to returning single-digit percentages of the query's total row count.
+ <literal>ROW</literal> and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
According to the standard, the <literal>OFFSET</literal> clause must come
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index baa669abe8..c503512588 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -21,6 +21,8 @@
#include "postgres.h"
+#include <math.h>
+
#include "executor/executor.h"
#include "executor/nodeLimit.h"
#include "miscadmin.h"
@@ -52,6 +54,7 @@ ExecLimit(PlanState *pstate)
*/
direction = node->ps.state->es_direction;
outerPlan = outerPlanState(node);
+ slot = node->subSlot;
/*
* The main logic is a simple state machine.
@@ -81,7 +84,15 @@ ExecLimit(PlanState *pstate)
/*
* Check for empty window; if so, treat like empty subplan.
*/
- if (node->count <= 0 && !node->noCount)
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (node->percent == 0.0)
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ }
+ else if (node->count <= 0 && !node->noCount)
{
node->lstate = LIMIT_EMPTY;
return NULL;
@@ -102,11 +113,32 @@ ExecLimit(PlanState *pstate)
node->lstate = LIMIT_EMPTY;
return NULL;
}
+
+ /*
+ * Count the number of rows to return in exact number so far
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ node->perExactCount = ceil(node->percent * node->position / 100.0);
+
+ if (node->perExactCount == node->perExactCount + 1)
+ node->perExactCount++;
+ }
node->subSlot = slot;
if (++node->position > node->offset)
break;
}
+ /*
+ * We may needed this tuple in backward scan so put it into
+ * tuplestore.
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ tuplestore_puttupleslot(node->tupleStore, slot);
+ tuplestore_advance(node->tupleStore, true);
+ }
+
/*
* Okay, we have the first tuple of the window.
*/
@@ -124,6 +156,106 @@ ExecLimit(PlanState *pstate)
case LIMIT_INWINDOW:
if (ScanDirectionIsForward(direction))
{
+ /*
+ * In case of coming back from backward scan the tuple is
+ * already in tuple store.
+ */
+ if (node->limitOption == PERCENTAGE && node->backwardPosition > 0)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ node->backwardPosition--;
+ return slot;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+
+ /*
+ * In PERCENTAGE case no need of executing outerPlan multiple
+ * times.
+ */
+ if (node->limitOption == PERCENTAGE && node->reachEnd)
+ {
+ node->lstate = LIMIT_WINDOWEND;
+
+ /*
+ * If we know we won't need to back up, we can release
+ * resources at this point.
+ */
+ if (!(node->ps.state->es_top_eflags & EXEC_FLAG_BACKWARD))
+ (void) ExecShutdownNode(outerPlan);
+
+ return NULL;
+ }
+
+ /*
+ * Return the tuple up to the number of exact count in OFFSET
+ * clause without percentage value consideration.
+ */
+ if (node->perExactCount > 0)
+ {
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ node->reachEnd = true;
+ return NULL;
+ }
+ tuplestore_puttupleslot(node->tupleStore, slot);
+ tuplestore_advance(node->tupleStore, true);
+ node->subSlot = slot;
+ node->position++;
+ node->perExactCount--;
+ return slot;
+ }
+
+ /*
+ * When in percentage mode, we need to see if we can get any
+ * additional rows from the subplan (enough to increase the
+ * node->count value).
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ /* loop until the node->count increments */
+ while (node->position - node->offset >= node->count)
+ {
+ int64 cnt;
+
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->reachEnd = true;
+ node->lstate = LIMIT_SUBPLANEOF;
+
+ /*
+ * The only operation from here is backward scan
+ * but there's no API to refetch the tuple at the
+ * current position. We have to move one tuple
+ * backward, and then we will scan forward for it
+ * for the first tuple and precede as usual
+ * for the rest
+ */
+ tuplestore_advance(node->tupleStore, false);
+ return NULL;
+ }
+
+ tuplestore_puttupleslot(node->tupleStore, slot);
+
+ cnt = tuplestore_tuple_count(node->tupleStore) + node->offset;
+
+ node->count = ceil(node->percent * cnt / 100.0);
+ }
+ }
+
/*
* Forwards scan, so check for stepping off end of window. If
* we are at the end of the window, return NULL without
@@ -145,17 +277,31 @@ ExecLimit(PlanState *pstate)
return NULL;
}
- /*
- * Get next tuple from subplan, if any.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
+ if (node->limitOption == PERCENTAGE)
{
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
+ while (node->position - node->offset < node->count)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ node->subSlot = slot;
+ node->position++;
}
- node->subSlot = slot;
- node->position++;
}
else
{
@@ -168,15 +314,29 @@ ExecLimit(PlanState *pstate)
node->lstate = LIMIT_WINDOWSTART;
return NULL;
}
-
- /*
- * Get previous tuple from subplan; there should be one!
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
+ /* In PERCENTAGE case the result is already in tuplestore */
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, false, true, slot))
+ {
+ node->subSlot = slot;
+ node->position--;
+ node->backwardPosition++;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+ /*
+ * Get previous tuple from subplan; there should be one!
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->position--;
+ }
}
break;
@@ -185,15 +345,32 @@ ExecLimit(PlanState *pstate)
return NULL;
/*
- * Backing up from subplan EOF, so re-fetch previous tuple; there
- * should be one! Note previous tuple must be in window.
+ * Scan forward for the first tuple
*/
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->lstate = LIMIT_INWINDOW;
- /* position does not change 'cause we didn't advance it before */
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+ /*
+ * Backing up from subplan EOF, so re-fetch previous tuple;
+ * there should be one! Note previous tuple must be in
+ * window.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ /* position does not change 'cause we didn't advance it before */
+ }
break;
case LIMIT_WINDOWEND:
@@ -283,12 +460,28 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
- node->noCount = false;
+ if (node->limitOption == PERCENTAGE)
+ {
+ /*
+ * We expect to return at least one row (unless there are no
+ * rows in the subplan), and we'll update this count later as
+ * we go.
+ */
+ node->count = 0;
+ node->percent = DatumGetFloat8(val);
+ if (node->percent < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_RESULT_OFFSET_CLAUSE),
+ errmsg("PERCENT must not be negative")));
+ }
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+ }
}
}
else
@@ -301,6 +494,9 @@ recompute_limits(LimitState *node)
/* Reset position to start-of-scan */
node->position = 0;
node->subSlot = NULL;
+ node->reachEnd = false;
+ node->backwardPosition = 0;
+ node->perExactCount = 0;
/* Set state-machine state */
node->lstate = LIMIT_RESCAN;
@@ -309,9 +505,11 @@ recompute_limits(LimitState *node)
* Notify child node about limit. Note: think not to "optimize" by
* skipping ExecSetTupleBound if compute_tuples_needed returns < 0. We
* must update the child node anyway, in case this is a rescan and the
- * previous time we got a different result.
+ * previous time we got a different result. In PERCENTAGE option there are
+ * no bound on the number of output tuples
*/
- ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
+ if (node->limitOption != PERCENTAGE)
+ ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
}
/*
@@ -374,6 +572,7 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
/*
* Initialize result type.
@@ -390,6 +589,9 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
limitstate->ps.ps_ProjInfo = NULL;
+ if (node->limitOption == PERCENTAGE)
+ limitstate->tupleStore = tuplestore_begin_heap(true, false, work_mem);
+
return limitstate;
}
@@ -405,6 +607,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->tupleStore != NULL)
+ tuplestore_end(node->tupleStore);
}
@@ -424,4 +628,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->tupleStore != NULL)
+ tuplestore_rescan(node->tupleStore);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 78deade89b..8307da7d3a 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1143,6 +1143,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3031,6 +3032,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3115,6 +3117,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 4f2ebe5118..2e14fa72b4 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -975,6 +975,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1049,6 +1050,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8400dd319e..3561bd8ae2 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -907,6 +907,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2104,6 +2105,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2698,6 +2700,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2908,6 +2911,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 6c2626ee62..85b514c9c2 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -278,6 +278,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2333,6 +2334,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 608d5adfed..6a6e733810 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2331,7 +2331,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2644,7 +2645,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6512,7 +6514,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6524,6 +6526,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index cb897cc7f4..9331dbc545 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2241,12 +2241,25 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
*/
if (parse->sortClause)
{
- current_rel = create_ordered_paths(root,
- current_rel,
- final_target,
- final_target_parallel_safe,
- have_postponed_srfs ? -1.0 :
- limit_tuples);
+
+ /*
+ * In PERCENTAGE option there are no bound on the number of output
+ * tuples
+ */
+ if (parse->limitOption == PERCENTAGE)
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ -1.0);
+ else
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ limit_tuples);
/* Fix things up if final_target contains SRFs */
if (parse->hasTargetSRFs)
adjust_paths_for_srfs(root, current_rel,
@@ -2309,6 +2322,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index d884d2bb00..3c49a716eb 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3570,6 +3570,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3591,6 +3592,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
@@ -3598,7 +3600,8 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
adjust_limit_rows_costs(&pathnode->path.rows,
&pathnode->path.startup_cost,
&pathnode->path.total_cost,
- offset_est, count_est);
+ offset_est, count_est,
+ limitOption);
return pathnode;
}
@@ -3624,7 +3627,8 @@ adjust_limit_rows_costs(double *rows, /* in/out parameter */
Cost *startup_cost, /* in/out parameter */
Cost *total_cost, /* in/out parameter */
int64 offset_est,
- int64 count_est)
+ int64 count_est,
+ LimitOption limitOption)
{
double input_rows = *rows;
Cost input_startup_cost = *startup_cost;
@@ -3657,6 +3661,19 @@ adjust_limit_rows_costs(double *rows, /* in/out parameter */
count_rows = (double) count_est;
else
count_rows = clamp_row_est(input_rows * 0.10);
+ if (limitOption == PERCENTAGE)
+ {
+ double per_count = DatumGetFloat8(count_est);
+
+ count_rows = clamp_row_est((input_rows * per_count) / 100);
+ if (rows > 0)
+ {
+ *startup_cost = count_rows *
+ input_total_cost / input_rows;
+ *total_cost = input_total_cost +
+ (count_rows * 0.1);
+ }
+ }
if (count_rows > *rows)
count_rows = *rows;
if (input_rows > 0)
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index b13c246183..2c7feff026 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1288,10 +1288,11 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
}
/* transform LIMIT */
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1536,10 +1537,11 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_ORDER_BY,
false /* allow SQL92 rules */ );
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1770,10 +1772,11 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
parser_errposition(pstate,
exprLocation(list_nth(qry->targetList, tllen)))));
- qry->limitOffset = transformLimitClause(pstate, limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, limitCount,
+ qry->limitCount = transformLimitClause(pstate, limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 8311b1dd46..91fb2e4143 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -127,6 +127,21 @@ typedef struct ImportQual
List *table_names;
} ImportQual;
+/* Private struct for the result of opt_select_limit production */
+typedef struct SelectLimit
+{
+ Node *limitOffset;
+ Node *limitCount;
+ void *limitOption;
+} SelectLimit;
+
+/* Private struct for the result of limit_clause production */
+typedef struct LimitClause
+{
+ Node *limitCount;
+ void *limitOption;
+} LimitClause;
+
/* ConstraintAttributeSpec yields an integer bitmask of these flags: */
#define CAS_NOT_DEFERRABLE 0x01
#define CAS_DEFERRABLE 0x02
@@ -165,6 +180,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -242,6 +258,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
PartitionSpec *partspec;
PartitionBoundSpec *partboundspec;
RoleSpec *rolespec;
+ struct SelectLimit *SelectLimit;
+ struct LimitClause *LimitClause;
}
%type <node> stmt schema_stmt
@@ -373,6 +391,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <ival> import_qualification_type
%type <importqual> import_qualification
%type <node> vacuum_relation
+%type <SelectLimit> opt_select_limit select_limit
+%type <LimitClause> limit_clause
%type <list> stmtblock stmtmulti
OptTableElementList TableElementList OptInherit definition
@@ -393,8 +413,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
- opt_select_limit opclass_item_list opclass_drop_list
+ reloption_list group_clause TriggerFuncArgs
+ opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
prep_type_clause
@@ -455,7 +475,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -667,7 +687,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11225,14 +11245,15 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
| select_clause opt_sort_clause for_locking_clause opt_select_limit
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
- list_nth($4, 0), list_nth($4, 1),
+ ($4)->limitOffset, ($4)->limitCount,
+ ($4)->limitOption,
NULL,
yyscanner);
$$ = $1;
@@ -11240,7 +11261,8 @@ select_no_parens:
| select_clause opt_sort_clause select_limit opt_for_locking_clause
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
- list_nth($3, 0), list_nth($3, 1),
+ ($3)->limitOffset, ($3)->limitCount,
+ ($3)->limitOption,
NULL,
yyscanner);
$$ = $1;
@@ -11249,7 +11271,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11257,14 +11279,15 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
| with_clause select_clause opt_sort_clause for_locking_clause opt_select_limit
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
- list_nth($5, 0), list_nth($5, 1),
+ ($5)->limitOffset, ($5)->limitCount,
+ ($5)->limitOption,
$1,
yyscanner);
$$ = $2;
@@ -11272,7 +11295,8 @@ select_no_parens:
| with_clause select_clause opt_sort_clause select_limit opt_for_locking_clause
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
- list_nth($4, 0), list_nth($4, 1),
+ ($4)->limitOffset, ($4)->limitCount,
+ ($4)->limitOption,
$1,
yyscanner);
$$ = $2;
@@ -11566,20 +11590,60 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $2;
+ n->limitCount = ($1)->limitCount;
+ n->limitOption = ($1)->limitOption;
+ $$ = n;
+ }
+ | offset_clause limit_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $1;
+ n->limitCount = ($2)->limitCount;
+ n->limitOption = ($2)->limitOption;
+ $$ = n;
+ }
+ | limit_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = NULL;
+ n->limitCount = ($1)->limitCount;
+ n->limitOption = ($1)->limitOption;
+ $$ = n;
+ }
+ | offset_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $1;
+ n->limitCount = NULL;
+ n->limitOption = NULL;
+ $$ = n;
+ }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = NULL;
+ n->limitCount = NULL;
+ n->limitOption = NULL;
+ $$ = n;
+ }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $2;
+ n->limitOption = NULL;
+ $$ = n;
+ }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11597,9 +11661,26 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $3;
+ n->limitOption = makeString("EXACT_NUMBER");
+ $$ = n;
+ }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $3;
+ n->limitOption = makeString("PERCENTAGE");
+ $$ = n;
+ }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = makeIntConst(1, -1);
+ n->limitOption = NULL;
+ $$ = n;
+ }
;
offset_clause:
@@ -15192,6 +15273,7 @@ unreserved_keyword:
| PARTITION
| PASSING
| PASSWORD
+ | PERCENT
| PLANS
| POLICY
| PRECEDING
@@ -15856,6 +15938,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15894,6 +15977,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 2a6b2ff153..c8b2fbdc7a 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1707,7 +1707,7 @@ transformWhereClause(ParseState *pstate, Node *clause,
* constructName does not affect the semantics, but is used in error messages
*/
Node *
-transformLimitClause(ParseState *pstate, Node *clause,
+transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName)
{
Node *qual;
@@ -1716,8 +1716,10 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
-
- qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
+ if (limitOption == PERCENTAGE && (strcmp(constructName, "LIMIT") == 0))
+ qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
+ else
+ qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
/* LIMIT can't refer to any variables of the current query */
checkExprIsVarFree(pstate, qual, constructName);
diff --git a/src/backend/utils/sort/tuplestore.c b/src/backend/utils/sort/tuplestore.c
index 3fc7f92182..a03e568f1f 100644
--- a/src/backend/utils/sort/tuplestore.c
+++ b/src/backend/utils/sort/tuplestore.c
@@ -1100,6 +1100,39 @@ tuplestore_gettupleslot(Tuplestorestate *state, bool forward,
}
}
+/*
+ * tuplestore_gettupleslot_heaptuple
+ * It is similar to tuplestore_gettupleslot except it return stored HeapTuple
+ * instead of MinimalTuple
+ */
+bool
+tuplestore_gettupleslot_heaptuple(Tuplestorestate *state, bool forward,
+ bool copy, TupleTableSlot *slot)
+{
+ MinimalTuple tuple;
+ HeapTuple htuple;
+ bool should_free;
+
+ tuple = (MinimalTuple) tuplestore_gettuple(state, forward, &should_free);
+
+ if (tuple)
+ {
+ if (copy && !should_free)
+ {
+ tuple = heap_copy_minimal_tuple(tuple);
+ should_free = true;
+ }
+ htuple = heap_tuple_from_minimal_tuple(tuple);
+ ExecForceStoreHeapTuple(htuple, slot, should_free);
+ return true;
+ }
+ else
+ {
+ ExecClearTuple(slot);
+ return false;
+ }
+}
+
/*
* tuplestore_advance - exported function to adjust position without fetching
*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 99b9fa414f..64c7581ddc 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2302,8 +2302,16 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ float8 percent; /* percentage */
+ int64 backwardPosition; /* the number of tuple returned in
+ * backward scan */
+ int64 perExactCount; /* the number of tuple in excact count to
+ * return in OFFSET cluase */
+ bool reachEnd; /* if true, outerPlan executed until the end */
+ Tuplestorestate *tupleStore; /* holds the returned tuple */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 4e2fb39105..9f1e2fbfab 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -821,4 +821,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+}LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 94ded3c135..b158e9eaeb 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1595,6 +1596,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 441e64eca9..94f121525c 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1789,6 +1789,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 70f8b8e22b..3cfcf9f63d 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -953,6 +953,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index e70d6a3f18..c03a939f98 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -264,10 +264,11 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern void adjust_limit_rows_costs(double *rows,
Cost *startup_cost, Cost *total_cost,
- int64 offset_est, int64 count_est);
+ int64 offset_est, int64 count_est, LimitOption limitOption);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
Relids required_outer,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index e7aaddd50d..3395e558c4 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -56,7 +56,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators, Oid *grpCollations,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 00ace8425e..e1a8d703ab 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -299,6 +299,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, UNRESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h
index 42adc63d1f..e66510e965 100644
--- a/src/include/parser/parse_clause.h
+++ b/src/include/parser/parse_clause.h
@@ -22,7 +22,7 @@ extern int setTargetTable(ParseState *pstate, RangeVar *relation,
extern Node *transformWhereClause(ParseState *pstate, Node *clause,
ParseExprKind exprKind, const char *constructName);
-extern Node *transformLimitClause(ParseState *pstate, Node *clause,
+extern Node *transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName);
extern List *transformGroupClause(ParseState *pstate, List *grouplist,
List **groupingSets,
diff --git a/src/include/utils/tuplestore.h b/src/include/utils/tuplestore.h
index f9b6fcec29..eeefde3da4 100644
--- a/src/include/utils/tuplestore.h
+++ b/src/include/utils/tuplestore.h
@@ -47,7 +47,6 @@ typedef struct Tuplestorestate Tuplestorestate;
extern Tuplestorestate *tuplestore_begin_heap(bool randomAccess,
bool interXact,
int maxKBytes);
-
extern void tuplestore_set_eflags(Tuplestorestate *state, int eflags);
extern void tuplestore_puttupleslot(Tuplestorestate *state,
@@ -72,9 +71,12 @@ extern bool tuplestore_in_memory(Tuplestorestate *state);
extern bool tuplestore_gettupleslot(Tuplestorestate *state, bool forward,
bool copy, TupleTableSlot *slot);
+extern bool tuplestore_gettupleslot_heaptuple(Tuplestorestate *state, bool forward,
+ bool copy, TupleTableSlot *slot);
extern bool tuplestore_advance(Tuplestorestate *state, bool forward);
+
extern bool tuplestore_skiptuples(Tuplestorestate *state,
int64 ntuples, bool forward);
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index c18f547cbd..95f9f5f3d9 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -108,6 +108,63 @@ SELECT ''::text AS five, unique1, unique2, stringu1
| 904 | 793 | UIAAAA
(5 rows)
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 51 | 76 | ZBAAAA
+ | 52 | 985 | ACAAAA
+ | 53 | 196 | BCAAAA
+ | 54 | 356 | CCAAAA
+ | 55 | 627 | DCAAAA
+ | 56 | 54 | ECAAAA
+ | 57 | 942 | FCAAAA
+ | 58 | 114 | GCAAAA
+ | 59 | 593 | HCAAAA
+ | 60 | 483 | ICAAAA
+(10 rows)
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 61 | 560 | JCAAAA
+(1 row)
+
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+ three | unique1 | unique2 | stringu1
+-------+---------+---------+----------
+ | 121 | 700 | REAAAA
+ | 122 | 519 | SEAAAA
+ | 123 | 777 | TEAAAA
+ | 124 | 503 | UEAAAA
+ | 125 | 849 | VEAAAA
+ | 126 | 330 | WEAAAA
+ | 127 | 511 | XEAAAA
+ | 128 | 721 | YEAAAA
+ | 129 | 696 | ZEAAAA
+(9 rows)
+
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+ eleven | unique1 | unique2 | stringu1
+--------+---------+---------+----------
+ | 10 | 520 | KAAAAA
+ | 9 | 49 | JAAAAA
+ | 8 | 653 | IAAAAA
+ | 7 | 647 | HAAAAA
+ | 6 | 978 | GAAAAA
+(5 rows)
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -286,6 +343,46 @@ fetch all in c4;
----+----
(0 rows)
+declare c5 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c5;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
+fetch 1 in c5;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c5;
+ q1 | q2
+------------------+-----
+ 4567890123456789 | 123
+(1 row)
+
+fetch backward all in c5;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+ 123 | 456
+(2 rows)
+
+fetch backward 1 in c5;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c5;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
@@ -503,3 +600,19 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
45020 | 45020
(3 rows)
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
+ s1 | s2
+-------+-------
+ 45000 | 45000
+ 45010 | 45010
+ 45020 | 45020
+ 45030 | 45030
+ 45040 | 45040
+ 45050 | 45050
+ 45060 | 45060
+ 45070 | 45070
+ 45080 | 45080
+ 45090 | 45090
+(10 rows)
+
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index 2a313d80ca..051b21a099 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -31,6 +31,23 @@ SELECT ''::text AS five, unique1, unique2, stringu1
FROM onek
ORDER BY unique1 LIMIT 5 OFFSET 900;
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -38,7 +55,6 @@ select * from int8_tbl offset (case when random() < 0.5 then null::bigint end);
-- Test assorted cases involving backwards fetch from a LIMIT plan node
begin;
-
declare c1 cursor for select * from int8_tbl limit 10;
fetch all in c1;
fetch 1 in c1;
@@ -71,6 +87,14 @@ fetch backward all in c4;
fetch backward 1 in c4;
fetch all in c4;
+declare c5 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c5;
+fetch 1 in c5;
+fetch backward 1 in c5;
+fetch backward all in c5;
+fetch backward 1 in c5;
+fetch all in c5;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
@@ -141,3 +165,6 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
from tenk1 group by thousand order by thousand limit 3;
+
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
Surafel,
The patch did not did it automatically. Its query writer obligation to do
that currently
Ok.
Your latest patch [1]/messages/by-id/attachment/103028/percent-incremental-v6.patch passes make installcheck-world, I didn't test the
actual functionality this round.
plan = (Plan *) make_limit(plan,
subparse->limitOffset, - subparse->limitCount); + subparse->limitCount, + subparse->limitOption);
I assume the limit percentage number goes into subparse->limitCount? If
so, I don't see that documented. Or does this truly only store the count?
The remainder of the code seems to make sense. I attached a patch with a
few minor changes in the comments. I need to go back to my notes and see
if I covered all the testing I had thought of, I should get to that later
this week.
[1]: /messages/by-id/attachment/103028/percent-incremental-v6.patch
/messages/by-id/attachment/103028/percent-incremental-v6.patch
*Ryan Lambert*
Show quoted text
Attachments:
percent-incremental-v6-comment-cleanup.patchapplication/octet-stream; name=percent-incremental-v6-comment-cleanup.patchDownload
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index c503512588..159542b81e 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -130,7 +130,7 @@ ExecLimit(PlanState *pstate)
}
/*
- * We may needed this tuple in backward scan so put it into
+ * We may need this tuple in backward scan so put it into
* tuplestore.
*/
if (node->limitOption == PERCENTAGE)
@@ -219,7 +219,7 @@ ExecLimit(PlanState *pstate)
}
/*
- * When in percentage mode, we need to see if we can get any
+ * When in PERCENTAGE mode, we need to see if we can get any
* additional rows from the subplan (enough to increase the
* node->count value).
*/
@@ -369,7 +369,7 @@ ExecLimit(PlanState *pstate)
elog(ERROR, "LIMIT subplan failed to run backwards");
node->subSlot = slot;
node->lstate = LIMIT_INWINDOW;
- /* position does not change 'cause we didn't advance it before */
+ /* position does not change because we didn't advance it before */
}
break;
@@ -505,7 +505,7 @@ recompute_limits(LimitState *node)
* Notify child node about limit. Note: think not to "optimize" by
* skipping ExecSetTupleBound if compute_tuples_needed returns < 0. We
* must update the child node anyway, in case this is a rescan and the
- * previous time we got a different result. In PERCENTAGE option there are
+ * previous time we got a different result. In PERCENTAGE option there is
* no bound on the number of output tuples
*/
if (node->limitOption != PERCENTAGE)
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 407b61787d..702179e576 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2249,7 +2249,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
{
/*
- * In PERCENTAGE option there are no bound on the number of output
+ * In PERCENTAGE option there is no bound on the number of output
* tuples
*/
if (parse->limitOption == PERCENTAGE)
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index b158e9eaeb..b49ce10a31 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,7 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
- LimitOption limitOption; /* limit type */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1596,7 +1596,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
- LimitOption limitOption; /* limit type */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 21ff6a9c4c..c6195e4dd6 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1795,7 +1795,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
- LimitOption limitOption; /* LIMIT in percentage or exact number */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index c8c497295b..03f56ea247 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -967,7 +967,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
- LimitOption limitOption; /* LIMIT in percentage or exact number */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
Hello.
At Thu, 1 Aug 2019 15:54:11 +0300, Surafel Temesgen <surafel3000@gmail.com> wrote in <CALAY4q_icXPTcAVxnPckpLGrZgER9-FWM0VPFvvOsPiJrnK6Dg@mail.gmail.com>
Other than that, we can rip the clause if it is 100%
You mean if PERCENT=100 it should short circuit and run the query
normally? I like that.The patch did not did it automatically. Its query writer obligation to do
that currently
I have some comments.
This patch uses distinct parameters for exact number and
percentage. On the othe hand planner has a notion of
tuple_fraction covering the both. The same handling is also used
for tuple number estimation. Using the same scheme will make
things far simpler. See the comment of grouping_planner().
In executor part, addition to LimiteState.position, this patch
uses LimiteState.backwardPosition to count how many tuples we're
back from the end of the current tuplestore. But things will get
simpler by just asking the tuplestore for the number of holding
tuples.
+ slot = node->subSlot;
Why is this needed? The variable is properly set before use and
the assignment is bogus in the first place.
The new code block in LIMIT_RESCAN in ExecLimit is useless since
it is exatctly the same with existing code block. Why didn't you
use the existing if block?
if (node->limitOption == PERCENTAGE)
+ {
+ node->perExactCount = ceil(node->percent * node->position / 100.0);
+
+
node->position is the number of tuples returned to upper node so
far (not the number of tuples this node has read from the lower
node so far). I don't understand what the expression means.
+ if (node->perExactCount == node->perExactCount + 1)
+ node->perExactCount++;
What? The condition never be true. As the result, the following
if block below won't run.
/*
+ * Return the tuple up to the number of exact count in OFFSET
+ * clause without percentage value consideration.
+ */
+ if (node->perExactCount > 0)
+ {
+
+ /*
+ * We may needed this tuple in backward scan so put it into
+ * tuplestore.
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ tuplestore_puttupleslot(node->tupleStore, slot);
+ tuplestore_advance(node->tupleStore, true);
+ }
"needed"->"need" ? The comment says that this is needed for
backward scan, but it is actually required even in forward
scan. More to the point, tuplestore_advance lacks comment.
Anyway, the code in LIMIT_RESCAN is broken in some ways. For
example, if it is used as the inner scan of a NestLoop, the
tuplestore accumulates tuples by every scan. You will see that
the tuplestore stores up to 1000 tuples (10 times of the inner)
by the following query.
create table t0 as (select a from generate_series(0, 99) a);
create table t1 as (select a from generate_series(0, 9) a);
analyze;
set enable_hashjoin to false;
set enable_mergejoin to false;
set enable_material to false;
explain analyze select t.*, t1.* from (select a from t0 fetch first 10 percent rows only) as t join t1 on (t.a = t1.a);
QUERY PLAN
-------------------------------------------------------------------------------
Nested Loop (cost=0.20..7.35 rows=10 width=8) (actual ...)
-> Seq Scan on t1 (cost=0.00..1.10 rows=10 width=4) (actual ...)
-> Limit (cost=0.20..0.40 rows=10 width=4) (actual ...)
-> Seq Scan on t0 (cost=0.00..2.00 rows=100 width=4) (actual ...)
That's it for now.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
Hi
On Wed, Aug 7, 2019 at 6:11 AM Kyotaro Horiguchi <horikyota.ntt@gmail.com>
wrote:
I have some comments.
This patch uses distinct parameters for exact number and
percentage. On the othe hand planner has a notion of
tuple_fraction covering the both. The same handling is also used
for tuple number estimation. Using the same scheme will make
things far simpler. See the comment of grouping_planner().
Its because of data type difference .In planner the data type is the same
In executor part, addition to LimiteState.position, this patch
uses LimiteState.backwardPosition to count how many tuples we're
back from the end of the current tuplestore. But things will get
simpler by just asking the tuplestore for the number of holding
tuples.
backwardPosition hold how many tuple returned in backward scan
+ slot = node->subSlot;
Why is this needed? The variable is properly set before use and
the assignment is bogus in the first place.
its because Tuplestore needs initialized slot.
The new code block in LIMIT_RESCAN in ExecLimit is useless since
it is exatctly the same with existing code block. Why didn't you
use the existing if block?
But they test different scenario
if (node->limitOption == PERCENTAGE)
+ { + node->perExactCount = ceil(node->percent * node->position / 100.0); + +node->position is the number of tuples returned to upper node so
far (not the number of tuples this node has read from the lower
node so far). I don't understand what the expression means.
node->position hold the number of tuples this node has read from the lower
node so far. see LIMIT_RESCAN state
+ if (node->perExactCount == node->perExactCount + 1) + node->perExactCount++;What? The condition never be true. As the result, the following
if block below won't run.
it became true according to the number of tuple returned in from the lower
node so far
and percentage specification.
/*
+ * Return the tuple up to the number of exact count in OFFSET + * clause without percentage value consideration. + */ + if (node->perExactCount > 0) + { ++ /* + * We may needed this tuple in backward scan so put it into + * tuplestore. + */ + if (node->limitOption == PERCENTAGE) + { + tuplestore_puttupleslot(node->tupleStore, slot); + tuplestore_advance(node->tupleStore, true); + }"needed"->"need" ? The comment says that this is needed for
backward scan, but it is actually required even in forward
scan. More to the point, tuplestore_advance lacks comment.
ok
Anyway, the code in LIMIT_RESCAN is broken in some ways. For
example, if it is used as the inner scan of a NestLoop, the
tuplestore accumulates tuples by every scan. You will see that
the tuplestore stores up to 1000 tuples (10 times of the inner)
by the following query.
It this because in percentage we scan the whole table
regards
Surafel
The following review has been posted through the commitfest application:
make installcheck-world: tested, passed
Implements feature: tested, passed
Spec compliant: not tested
Documentation: tested, passed
The latest patch [1]/messages/by-id/attachment/103028/percent-incremental-v6.patch and the cleanup patch [2]/messages/by-id/attachment/103157/percent-incremental-v6-comment-cleanup.patch apply cleanly to the latest master (5f110933). I reviewed the conversation and don't see any outstanding questions or concerns. Updating status to ready for committer.
[1]: /messages/by-id/attachment/103028/percent-incremental-v6.patch
[2]: /messages/by-id/attachment/103157/percent-incremental-v6-comment-cleanup.patch
Ryan Lambert
The new status of this patch is: Ready for Committer
On 2019-08-19 01:33, Ryan Lambert wrote:
The following review has been posted through the commitfest
application:
make installcheck-world: tested, passed
Implements feature: tested, passed
Spec compliant: not tested
Documentation: tested, passedThe latest patch [1] and the cleanup patch [2] apply cleanly to the
latest master (5f110933). I reviewed the conversation and don't see
any outstanding questions or concerns. Updating status to ready for
committer.[1] >
/messages/by-id/attachment/103028/percent-incremental-v6.patch
[2] >
/messages/by-id/attachment/103157/percent-incremental-v6-comment-cleanup.patchRyan Lambert
The new status of this patch is: Ready for Committer
Hi,
(running with those two patches applied)
select * from onek where thousand < 5 order by thousand fetch first -1
percent rows only
is correctly caught (with "ERROR: PERCENT must not be negative") but:
select * from onek where thousand < 5 order by thousand fetch first
101 percent rows only
is not. It doesn't return, and cannot be Ctrl-C'ed out of, which I guess
is another bug?
thanks,
Erik Rijkers
Hi Erik
On Mon, Aug 19, 2019 at 10:47 AM Erik Rijkers <er@xs4all.nl> wrote:
Hi,
(running with those two patches applied)
select * from onek where thousand < 5 order by thousand fetch first -1
percent rows only
is correctly caught (with "ERROR: PERCENT must not be negative") but:select * from onek where thousand < 5 order by thousand fetch first
101 percent rows only
is not. It doesn't return, and cannot be Ctrl-C'ed out of, which I guess
is another bug?
Fixed
regards
Surafel
Attachments:
percent-incremental-v7.patchtext/x-patch; charset=US-ASCII; name=percent-incremental-v7.patchDownload
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 1759b9e1b6..fb9c2f5319 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -3035,7 +3035,8 @@ estimate_path_cost_size(PlannerInfo *root,
if (fpextra && fpextra->has_limit)
{
adjust_limit_rows_costs(&rows, &startup_cost, &total_cost,
- fpextra->offset_est, fpextra->count_est);
+ fpextra->offset_est, fpextra->count_est,
+ EXACT_NUMBER);
retrieved_rows = rows;
}
}
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 06d611b64c..4ef42df51d 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1430,7 +1430,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1440,8 +1440,10 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
- and <literal>ROWS</literal> as well as <literal>FIRST</literal>
+ With <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage round up to the nearest integer. Using <literal>PERCENT</literal>
+ is best suited to returning single-digit percentages of the query's total row count.
+ <literal>ROW</literal> and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
According to the standard, the <literal>OFFSET</literal> clause must come
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index baa669abe8..7a9626d7f9 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -21,6 +21,8 @@
#include "postgres.h"
+#include <math.h>
+
#include "executor/executor.h"
#include "executor/nodeLimit.h"
#include "miscadmin.h"
@@ -52,6 +54,7 @@ ExecLimit(PlanState *pstate)
*/
direction = node->ps.state->es_direction;
outerPlan = outerPlanState(node);
+ slot = node->subSlot;
/*
* The main logic is a simple state machine.
@@ -81,7 +84,15 @@ ExecLimit(PlanState *pstate)
/*
* Check for empty window; if so, treat like empty subplan.
*/
- if (node->count <= 0 && !node->noCount)
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (node->percent == 0.0)
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ }
+ else if (node->count <= 0 && !node->noCount)
{
node->lstate = LIMIT_EMPTY;
return NULL;
@@ -102,11 +113,32 @@ ExecLimit(PlanState *pstate)
node->lstate = LIMIT_EMPTY;
return NULL;
}
+
+ /*
+ * Count the number of rows to return in exact number so far
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ node->perExactCount = ceil(node->percent * node->position / 100.0);
+
+ if (node->perExactCount == node->perExactCount + 1)
+ node->perExactCount++;
+ }
node->subSlot = slot;
if (++node->position > node->offset)
break;
}
+ /*
+ * We may needed this tuple in subsequent scan so put it into
+ * tuplestore.
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ tuplestore_puttupleslot(node->tupleStore, slot);
+ tuplestore_advance(node->tupleStore, true);
+ }
+
/*
* Okay, we have the first tuple of the window.
*/
@@ -124,6 +156,106 @@ ExecLimit(PlanState *pstate)
case LIMIT_INWINDOW:
if (ScanDirectionIsForward(direction))
{
+ /*
+ * In case of coming back from backward scan the tuple is
+ * already in tuple store.
+ */
+ if (node->limitOption == PERCENTAGE && node->backwardPosition > 0)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ node->backwardPosition--;
+ return slot;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+
+ /*
+ * In PERCENTAGE case no need of executing outerPlan multiple
+ * times.
+ */
+ if (node->limitOption == PERCENTAGE && node->reachEnd)
+ {
+ node->lstate = LIMIT_WINDOWEND;
+
+ /*
+ * If we know we won't need to back up, we can release
+ * resources at this point.
+ */
+ if (!(node->ps.state->es_top_eflags & EXEC_FLAG_BACKWARD))
+ (void) ExecShutdownNode(outerPlan);
+
+ return NULL;
+ }
+
+ /*
+ * Return the tuple up to the number of exact count in OFFSET
+ * clause without percentage value consideration.
+ */
+ if (node->perExactCount > 0)
+ {
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ node->reachEnd = true;
+ return NULL;
+ }
+ tuplestore_puttupleslot(node->tupleStore, slot);
+ tuplestore_advance(node->tupleStore, true);
+ node->subSlot = slot;
+ node->position++;
+ node->perExactCount--;
+ return slot;
+ }
+
+ /*
+ * When in percentage mode, we need to see if we can get any
+ * additional rows from the subplan (enough to increase the
+ * node->count value).
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ /* loop until the node->count increments */
+ while (node->position - node->offset >= node->count)
+ {
+ int64 cnt;
+
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->reachEnd = true;
+ node->lstate = LIMIT_SUBPLANEOF;
+
+ /*
+ * The only operation from here is backward scan
+ * but there's no API to refetch the tuple at the
+ * current position. We have to move one tuple
+ * backward, and then we will scan forward for it
+ * for the first tuple and precede as usual
+ * for the rest
+ */
+ tuplestore_advance(node->tupleStore, false);
+ return NULL;
+ }
+
+ tuplestore_puttupleslot(node->tupleStore, slot);
+
+ cnt = tuplestore_tuple_count(node->tupleStore) + node->offset;
+
+ node->count = ceil(node->percent * cnt / 100.0);
+ }
+ }
+
/*
* Forwards scan, so check for stepping off end of window. If
* we are at the end of the window, return NULL without
@@ -145,17 +277,31 @@ ExecLimit(PlanState *pstate)
return NULL;
}
- /*
- * Get next tuple from subplan, if any.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
+ if (node->limitOption == PERCENTAGE)
{
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
+ while (node->position - node->offset < node->count)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ }
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ node->subSlot = slot;
+ node->position++;
}
- node->subSlot = slot;
- node->position++;
}
else
{
@@ -168,15 +314,29 @@ ExecLimit(PlanState *pstate)
node->lstate = LIMIT_WINDOWSTART;
return NULL;
}
-
- /*
- * Get previous tuple from subplan; there should be one!
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
+ /* In PERCENTAGE case the result is already in tuplestore */
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, false, true, slot))
+ {
+ node->subSlot = slot;
+ node->position--;
+ node->backwardPosition++;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+ /*
+ * Get previous tuple from subplan; there should be one!
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->position--;
+ }
}
break;
@@ -185,15 +345,32 @@ ExecLimit(PlanState *pstate)
return NULL;
/*
- * Backing up from subplan EOF, so re-fetch previous tuple; there
- * should be one! Note previous tuple must be in window.
+ * Scan forward for the first tuple
*/
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->lstate = LIMIT_INWINDOW;
- /* position does not change 'cause we didn't advance it before */
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+ /*
+ * Backing up from subplan EOF, so re-fetch previous tuple;
+ * there should be one! Note previous tuple must be in
+ * window.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ /* position does not change 'cause we didn't advance it before */
+ }
break;
case LIMIT_WINDOWEND:
@@ -283,12 +460,37 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
- node->noCount = false;
+ if (node->limitOption == PERCENTAGE)
+ {
+ /*
+ * We expect to return at least one row (unless there are no
+ * rows in the subplan), and we'll update this count later as
+ * we go.
+ */
+ node->count = 0;
+ node->percent = DatumGetFloat8(val);
+
+ if (node->percent < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_RESULT_OFFSET_CLAUSE),
+ errmsg("PERCENT must not be negative")));
+
+ if (node->percent > 100)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_RESULT_OFFSET_CLAUSE),
+ errmsg("PERCENT must not be greater than 100")));
+
+ }
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+
+
+ }
}
}
else
@@ -301,6 +503,9 @@ recompute_limits(LimitState *node)
/* Reset position to start-of-scan */
node->position = 0;
node->subSlot = NULL;
+ node->reachEnd = false;
+ node->backwardPosition = 0;
+ node->perExactCount = 0;
/* Set state-machine state */
node->lstate = LIMIT_RESCAN;
@@ -309,9 +514,11 @@ recompute_limits(LimitState *node)
* Notify child node about limit. Note: think not to "optimize" by
* skipping ExecSetTupleBound if compute_tuples_needed returns < 0. We
* must update the child node anyway, in case this is a rescan and the
- * previous time we got a different result.
+ * previous time we got a different result. In PERCENTAGE option there are
+ * no bound on the number of output tuples
*/
- ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
+ if (node->limitOption != PERCENTAGE)
+ ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
}
/*
@@ -374,6 +581,7 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
/*
* Initialize result type.
@@ -390,6 +598,9 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
limitstate->ps.ps_ProjInfo = NULL;
+ if (node->limitOption == PERCENTAGE)
+ limitstate->tupleStore = tuplestore_begin_heap(true, false, work_mem);
+
return limitstate;
}
@@ -405,6 +616,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->tupleStore != NULL)
+ tuplestore_end(node->tupleStore);
}
@@ -424,4 +637,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->tupleStore != NULL)
+ tuplestore_rescan(node->tupleStore);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 78deade89b..8307da7d3a 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1143,6 +1143,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3031,6 +3032,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3115,6 +3117,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 4f2ebe5118..2e14fa72b4 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -975,6 +975,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1049,6 +1050,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8400dd319e..3561bd8ae2 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -907,6 +907,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2104,6 +2105,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2698,6 +2700,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2908,6 +2911,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 6c2626ee62..85b514c9c2 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -278,6 +278,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2333,6 +2334,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 608d5adfed..6a6e733810 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2331,7 +2331,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2644,7 +2645,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6512,7 +6514,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6524,6 +6526,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index cb897cc7f4..9331dbc545 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2241,12 +2241,25 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
*/
if (parse->sortClause)
{
- current_rel = create_ordered_paths(root,
- current_rel,
- final_target,
- final_target_parallel_safe,
- have_postponed_srfs ? -1.0 :
- limit_tuples);
+
+ /*
+ * In PERCENTAGE option there are no bound on the number of output
+ * tuples
+ */
+ if (parse->limitOption == PERCENTAGE)
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ -1.0);
+ else
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ limit_tuples);
/* Fix things up if final_target contains SRFs */
if (parse->hasTargetSRFs)
adjust_paths_for_srfs(root, current_rel,
@@ -2309,6 +2322,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index d884d2bb00..3c49a716eb 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3570,6 +3570,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3591,6 +3592,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
@@ -3598,7 +3600,8 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
adjust_limit_rows_costs(&pathnode->path.rows,
&pathnode->path.startup_cost,
&pathnode->path.total_cost,
- offset_est, count_est);
+ offset_est, count_est,
+ limitOption);
return pathnode;
}
@@ -3624,7 +3627,8 @@ adjust_limit_rows_costs(double *rows, /* in/out parameter */
Cost *startup_cost, /* in/out parameter */
Cost *total_cost, /* in/out parameter */
int64 offset_est,
- int64 count_est)
+ int64 count_est,
+ LimitOption limitOption)
{
double input_rows = *rows;
Cost input_startup_cost = *startup_cost;
@@ -3657,6 +3661,19 @@ adjust_limit_rows_costs(double *rows, /* in/out parameter */
count_rows = (double) count_est;
else
count_rows = clamp_row_est(input_rows * 0.10);
+ if (limitOption == PERCENTAGE)
+ {
+ double per_count = DatumGetFloat8(count_est);
+
+ count_rows = clamp_row_est((input_rows * per_count) / 100);
+ if (rows > 0)
+ {
+ *startup_cost = count_rows *
+ input_total_cost / input_rows;
+ *total_cost = input_total_cost +
+ (count_rows * 0.1);
+ }
+ }
if (count_rows > *rows)
count_rows = *rows;
if (input_rows > 0)
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index b13c246183..2c7feff026 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1288,10 +1288,11 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
}
/* transform LIMIT */
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1536,10 +1537,11 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_ORDER_BY,
false /* allow SQL92 rules */ );
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1770,10 +1772,11 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
parser_errposition(pstate,
exprLocation(list_nth(qry->targetList, tllen)))));
- qry->limitOffset = transformLimitClause(pstate, limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, limitCount,
+ qry->limitCount = transformLimitClause(pstate, limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 8311b1dd46..91fb2e4143 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -127,6 +127,21 @@ typedef struct ImportQual
List *table_names;
} ImportQual;
+/* Private struct for the result of opt_select_limit production */
+typedef struct SelectLimit
+{
+ Node *limitOffset;
+ Node *limitCount;
+ void *limitOption;
+} SelectLimit;
+
+/* Private struct for the result of limit_clause production */
+typedef struct LimitClause
+{
+ Node *limitCount;
+ void *limitOption;
+} LimitClause;
+
/* ConstraintAttributeSpec yields an integer bitmask of these flags: */
#define CAS_NOT_DEFERRABLE 0x01
#define CAS_DEFERRABLE 0x02
@@ -165,6 +180,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -242,6 +258,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
PartitionSpec *partspec;
PartitionBoundSpec *partboundspec;
RoleSpec *rolespec;
+ struct SelectLimit *SelectLimit;
+ struct LimitClause *LimitClause;
}
%type <node> stmt schema_stmt
@@ -373,6 +391,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <ival> import_qualification_type
%type <importqual> import_qualification
%type <node> vacuum_relation
+%type <SelectLimit> opt_select_limit select_limit
+%type <LimitClause> limit_clause
%type <list> stmtblock stmtmulti
OptTableElementList TableElementList OptInherit definition
@@ -393,8 +413,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
- opt_select_limit opclass_item_list opclass_drop_list
+ reloption_list group_clause TriggerFuncArgs
+ opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
prep_type_clause
@@ -455,7 +475,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -667,7 +687,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11225,14 +11245,15 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
| select_clause opt_sort_clause for_locking_clause opt_select_limit
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
- list_nth($4, 0), list_nth($4, 1),
+ ($4)->limitOffset, ($4)->limitCount,
+ ($4)->limitOption,
NULL,
yyscanner);
$$ = $1;
@@ -11240,7 +11261,8 @@ select_no_parens:
| select_clause opt_sort_clause select_limit opt_for_locking_clause
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
- list_nth($3, 0), list_nth($3, 1),
+ ($3)->limitOffset, ($3)->limitCount,
+ ($3)->limitOption,
NULL,
yyscanner);
$$ = $1;
@@ -11249,7 +11271,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11257,14 +11279,15 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
| with_clause select_clause opt_sort_clause for_locking_clause opt_select_limit
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
- list_nth($5, 0), list_nth($5, 1),
+ ($5)->limitOffset, ($5)->limitCount,
+ ($5)->limitOption,
$1,
yyscanner);
$$ = $2;
@@ -11272,7 +11295,8 @@ select_no_parens:
| with_clause select_clause opt_sort_clause select_limit opt_for_locking_clause
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
- list_nth($4, 0), list_nth($4, 1),
+ ($4)->limitOffset, ($4)->limitCount,
+ ($4)->limitOption,
$1,
yyscanner);
$$ = $2;
@@ -11566,20 +11590,60 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $2;
+ n->limitCount = ($1)->limitCount;
+ n->limitOption = ($1)->limitOption;
+ $$ = n;
+ }
+ | offset_clause limit_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $1;
+ n->limitCount = ($2)->limitCount;
+ n->limitOption = ($2)->limitOption;
+ $$ = n;
+ }
+ | limit_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = NULL;
+ n->limitCount = ($1)->limitCount;
+ n->limitOption = ($1)->limitOption;
+ $$ = n;
+ }
+ | offset_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $1;
+ n->limitCount = NULL;
+ n->limitOption = NULL;
+ $$ = n;
+ }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = NULL;
+ n->limitCount = NULL;
+ n->limitOption = NULL;
+ $$ = n;
+ }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $2;
+ n->limitOption = NULL;
+ $$ = n;
+ }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11597,9 +11661,26 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $3;
+ n->limitOption = makeString("EXACT_NUMBER");
+ $$ = n;
+ }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $3;
+ n->limitOption = makeString("PERCENTAGE");
+ $$ = n;
+ }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = makeIntConst(1, -1);
+ n->limitOption = NULL;
+ $$ = n;
+ }
;
offset_clause:
@@ -15192,6 +15273,7 @@ unreserved_keyword:
| PARTITION
| PASSING
| PASSWORD
+ | PERCENT
| PLANS
| POLICY
| PRECEDING
@@ -15856,6 +15938,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15894,6 +15977,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 2a6b2ff153..c8b2fbdc7a 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1707,7 +1707,7 @@ transformWhereClause(ParseState *pstate, Node *clause,
* constructName does not affect the semantics, but is used in error messages
*/
Node *
-transformLimitClause(ParseState *pstate, Node *clause,
+transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName)
{
Node *qual;
@@ -1716,8 +1716,10 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
-
- qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
+ if (limitOption == PERCENTAGE && (strcmp(constructName, "LIMIT") == 0))
+ qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
+ else
+ qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
/* LIMIT can't refer to any variables of the current query */
checkExprIsVarFree(pstate, qual, constructName);
diff --git a/src/backend/utils/sort/tuplestore.c b/src/backend/utils/sort/tuplestore.c
index 3fc7f92182..a03e568f1f 100644
--- a/src/backend/utils/sort/tuplestore.c
+++ b/src/backend/utils/sort/tuplestore.c
@@ -1100,6 +1100,39 @@ tuplestore_gettupleslot(Tuplestorestate *state, bool forward,
}
}
+/*
+ * tuplestore_gettupleslot_heaptuple
+ * It is similar to tuplestore_gettupleslot except it return stored HeapTuple
+ * instead of MinimalTuple
+ */
+bool
+tuplestore_gettupleslot_heaptuple(Tuplestorestate *state, bool forward,
+ bool copy, TupleTableSlot *slot)
+{
+ MinimalTuple tuple;
+ HeapTuple htuple;
+ bool should_free;
+
+ tuple = (MinimalTuple) tuplestore_gettuple(state, forward, &should_free);
+
+ if (tuple)
+ {
+ if (copy && !should_free)
+ {
+ tuple = heap_copy_minimal_tuple(tuple);
+ should_free = true;
+ }
+ htuple = heap_tuple_from_minimal_tuple(tuple);
+ ExecForceStoreHeapTuple(htuple, slot, should_free);
+ return true;
+ }
+ else
+ {
+ ExecClearTuple(slot);
+ return false;
+ }
+}
+
/*
* tuplestore_advance - exported function to adjust position without fetching
*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 99b9fa414f..64c7581ddc 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2302,8 +2302,16 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ float8 percent; /* percentage */
+ int64 backwardPosition; /* the number of tuple returned in
+ * backward scan */
+ int64 perExactCount; /* the number of tuple in excact count to
+ * return in OFFSET cluase */
+ bool reachEnd; /* if true, outerPlan executed until the end */
+ Tuplestorestate *tupleStore; /* holds the returned tuple */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 4e2fb39105..9f1e2fbfab 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -821,4 +821,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+}LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 94ded3c135..b158e9eaeb 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1595,6 +1596,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 441e64eca9..94f121525c 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1789,6 +1789,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 70f8b8e22b..3cfcf9f63d 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -953,6 +953,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index e70d6a3f18..c03a939f98 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -264,10 +264,11 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern void adjust_limit_rows_costs(double *rows,
Cost *startup_cost, Cost *total_cost,
- int64 offset_est, int64 count_est);
+ int64 offset_est, int64 count_est, LimitOption limitOption);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
Relids required_outer,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index e7aaddd50d..3395e558c4 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -56,7 +56,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators, Oid *grpCollations,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 00ace8425e..e1a8d703ab 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -299,6 +299,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, UNRESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h
index 42adc63d1f..e66510e965 100644
--- a/src/include/parser/parse_clause.h
+++ b/src/include/parser/parse_clause.h
@@ -22,7 +22,7 @@ extern int setTargetTable(ParseState *pstate, RangeVar *relation,
extern Node *transformWhereClause(ParseState *pstate, Node *clause,
ParseExprKind exprKind, const char *constructName);
-extern Node *transformLimitClause(ParseState *pstate, Node *clause,
+extern Node *transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName);
extern List *transformGroupClause(ParseState *pstate, List *grouplist,
List **groupingSets,
diff --git a/src/include/utils/tuplestore.h b/src/include/utils/tuplestore.h
index f9b6fcec29..eeefde3da4 100644
--- a/src/include/utils/tuplestore.h
+++ b/src/include/utils/tuplestore.h
@@ -47,7 +47,6 @@ typedef struct Tuplestorestate Tuplestorestate;
extern Tuplestorestate *tuplestore_begin_heap(bool randomAccess,
bool interXact,
int maxKBytes);
-
extern void tuplestore_set_eflags(Tuplestorestate *state, int eflags);
extern void tuplestore_puttupleslot(Tuplestorestate *state,
@@ -72,9 +71,12 @@ extern bool tuplestore_in_memory(Tuplestorestate *state);
extern bool tuplestore_gettupleslot(Tuplestorestate *state, bool forward,
bool copy, TupleTableSlot *slot);
+extern bool tuplestore_gettupleslot_heaptuple(Tuplestorestate *state, bool forward,
+ bool copy, TupleTableSlot *slot);
extern bool tuplestore_advance(Tuplestorestate *state, bool forward);
+
extern bool tuplestore_skiptuples(Tuplestorestate *state,
int64 ntuples, bool forward);
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index c18f547cbd..95f9f5f3d9 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -108,6 +108,63 @@ SELECT ''::text AS five, unique1, unique2, stringu1
| 904 | 793 | UIAAAA
(5 rows)
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 51 | 76 | ZBAAAA
+ | 52 | 985 | ACAAAA
+ | 53 | 196 | BCAAAA
+ | 54 | 356 | CCAAAA
+ | 55 | 627 | DCAAAA
+ | 56 | 54 | ECAAAA
+ | 57 | 942 | FCAAAA
+ | 58 | 114 | GCAAAA
+ | 59 | 593 | HCAAAA
+ | 60 | 483 | ICAAAA
+(10 rows)
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 61 | 560 | JCAAAA
+(1 row)
+
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+ three | unique1 | unique2 | stringu1
+-------+---------+---------+----------
+ | 121 | 700 | REAAAA
+ | 122 | 519 | SEAAAA
+ | 123 | 777 | TEAAAA
+ | 124 | 503 | UEAAAA
+ | 125 | 849 | VEAAAA
+ | 126 | 330 | WEAAAA
+ | 127 | 511 | XEAAAA
+ | 128 | 721 | YEAAAA
+ | 129 | 696 | ZEAAAA
+(9 rows)
+
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+ eleven | unique1 | unique2 | stringu1
+--------+---------+---------+----------
+ | 10 | 520 | KAAAAA
+ | 9 | 49 | JAAAAA
+ | 8 | 653 | IAAAAA
+ | 7 | 647 | HAAAAA
+ | 6 | 978 | GAAAAA
+(5 rows)
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -286,6 +343,46 @@ fetch all in c4;
----+----
(0 rows)
+declare c5 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c5;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
+fetch 1 in c5;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c5;
+ q1 | q2
+------------------+-----
+ 4567890123456789 | 123
+(1 row)
+
+fetch backward all in c5;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+ 123 | 456
+(2 rows)
+
+fetch backward 1 in c5;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c5;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
@@ -503,3 +600,19 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
45020 | 45020
(3 rows)
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
+ s1 | s2
+-------+-------
+ 45000 | 45000
+ 45010 | 45010
+ 45020 | 45020
+ 45030 | 45030
+ 45040 | 45040
+ 45050 | 45050
+ 45060 | 45060
+ 45070 | 45070
+ 45080 | 45080
+ 45090 | 45090
+(10 rows)
+
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index 2a313d80ca..051b21a099 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -31,6 +31,23 @@ SELECT ''::text AS five, unique1, unique2, stringu1
FROM onek
ORDER BY unique1 LIMIT 5 OFFSET 900;
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -38,7 +55,6 @@ select * from int8_tbl offset (case when random() < 0.5 then null::bigint end);
-- Test assorted cases involving backwards fetch from a LIMIT plan node
begin;
-
declare c1 cursor for select * from int8_tbl limit 10;
fetch all in c1;
fetch 1 in c1;
@@ -71,6 +87,14 @@ fetch backward all in c4;
fetch backward 1 in c4;
fetch all in c4;
+declare c5 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c5;
+fetch 1 in c5;
+fetch backward 1 in c5;
+fetch backward all in c5;
+fetch backward 1 in c5;
+fetch all in c5;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
@@ -141,3 +165,6 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
from tenk1 group by thousand order by thousand limit 3;
+
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
On 2019-08-19 11:18, Surafel Temesgen wrote:
[..]
[percent-incremental-v7.patch]
Thanks.
Another little thing, not sure it's a bug:
limit interprets its argument by rounding up or down as one would
expect:
table onek limit 10.4; --> gives 10 rows
table onek limit 10.6; --> gives 11 rows
but FETCH count PERCENT does not do that; it rounds always up.
select * from (table onek limit 100) f fetch first 10.4 percent rows
only; --> gives 11 rows
select * from (table onek limit 100) f fetch first 10.6 percent rows
only; --> gives 11 rows
I see that it's documented in the .sgml to behave as it does, but it
seems wrong to me;
shouldn't that 10.4-percent-query yield 10 rows instead of 11?
thanks,
Erik Rijkers
On Mon, Aug 19, 2019 at 1:55 PM Erik Rijkers <er@xs4all.nl> wrote:
Another little thing, not sure it's a bug:
limit interprets its argument by rounding up or down as one would
expect:table onek limit 10.4; --> gives 10 rows
table onek limit 10.6; --> gives 11 rowsbut FETCH count PERCENT does not do that; it rounds always up.
select * from (table onek limit 100) f fetch first 10.4 percent rows
only; --> gives 11 rows
select * from (table onek limit 100) f fetch first 10.6 percent rows
only; --> gives 11 rowsI see that it's documented in the .sgml to behave as it does, but it
seems wrong to me;
shouldn't that 10.4-percent-query yield 10 rows instead of 11?
According to sql standard we have to use the result ceiling value
regards
Surafel
Hi,
At Wed, 7 Aug 2019 10:20:09 +0300, Surafel Temesgen <surafel3000@gmail.com> wrote in <CALAY4q98xbVHtZ4yj9DcCMG2-s1_JuRr7fYaNfW+bKmr22OiVQ@mail.gmail.com>
Hi
On Wed, Aug 7, 2019 at 6:11 AM Kyotaro Horiguchi <horikyota.ntt@gmail.com>
wrote:I have some comments.
This patch uses distinct parameters for exact number and
percentage. On the othe hand planner has a notion of
tuple_fraction covering the both. The same handling is also used
for tuple number estimation. Using the same scheme will make
things far simpler. See the comment of grouping_planner().Its because of data type difference .In planner the data type is the same
I meant that, with the usage of tuple_fraction, one variable can
represent both the absolute limit and relative limit. This
simplifies parse structs.
In executor part, addition to LimiteState.position, this patch
uses LimiteState.backwardPosition to count how many tuples we're
back from the end of the current tuplestore. But things will get
simpler by just asking the tuplestore for the number of holding
tuples.backwardPosition hold how many tuple returned in backward scan
I meant that we don't need to hold the number in estate.
+ slot = node->subSlot;
Why is this needed? The variable is properly set before use and
the assignment is bogus in the first place.its because Tuplestore needs initialized slot.
I meant that the initilized slot is overwritten before first use.
The new code block in LIMIT_RESCAN in ExecLimit is useless since
it is exatctly the same with existing code block. Why didn't you
use the existing if block?But they test different scenario
I meant that the two different scenario can share the code block.
if (node->limitOption == PERCENTAGE)
+ { + node->perExactCount = ceil(node->percent * node->position / 100.0); + +node->position is the number of tuples returned to upper node so
far (not the number of tuples this node has read from the lower
node so far). I don't understand what the expression means.node->position hold the number of tuples this node has read from the lower
node so far. see LIMIT_RESCAN state
Reallly? node->position is incremented when
tuplestore_gettupleslot_heaptuple() succeeded and reutnrs the
tuple to the caller immediately...
+ if (node->perExactCount == node->perExactCount + 1) + node->perExactCount++;What? The condition never be true. As the result, the following
if block below won't run.it became true according to the number of tuple returned in from the lower
node so far
and percentage specification.
Mmm. How do you think X can be equal to (X + 1)?
/*
+ * Return the tuple up to the number of exact count in OFFSET + * clause without percentage value consideration. + */ + if (node->perExactCount > 0) + { ++ /* + * We may needed this tuple in backward scan so put it into + * tuplestore. + */ + if (node->limitOption == PERCENTAGE) + { + tuplestore_puttupleslot(node->tupleStore, slot); + tuplestore_advance(node->tupleStore, true); + }"needed"->"need" ? The comment says that this is needed for
backward scan, but it is actually required even in forward
scan. More to the point, tuplestore_advance lacks comment.ok
Anyway, the code in LIMIT_RESCAN is broken in some ways. For
example, if it is used as the inner scan of a NestLoop, the
tuplestore accumulates tuples by every scan. You will see that
the tuplestore stores up to 1000 tuples (10 times of the inner)
by the following query.It this because in percentage we scan the whole table
It's useless and rather harmful that the tuplestore holds
indefinite number of duplicate set of the whole tuples from the
lower node. We must reuse tuples already stored in the tuplestore
or clear it before the next round.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
On Tue, Aug 20, 2019 at 9:10 AM Kyotaro Horiguchi <horikyota.ntt@gmail.com>
wrote:
Hi,
At Wed, 7 Aug 2019 10:20:09 +0300, Surafel Temesgen <surafel3000@gmail.com>
wrote in <
CALAY4q98xbVHtZ4yj9DcCMG2-s1_JuRr7fYaNfW+bKmr22OiVQ@mail.gmail.com>Hi
On Wed, Aug 7, 2019 at 6:11 AM Kyotaro Horiguchi <horikyota.ntt@gmail.com>
wrote:
I have some comments.
This patch uses distinct parameters for exact number and
percentage. On the othe hand planner has a notion of
tuple_fraction covering the both. The same handling is also used
for tuple number estimation. Using the same scheme will make
things far simpler. See the comment of grouping_planner().Its because of data type difference .In planner the data type is the same
I meant that, with the usage of tuple_fraction, one variable can
represent both the absolute limit and relative limit. This
simplifies parse structs.
In grouping_planner the patch set tuple bound to -1 in create_ordered_paths
because it iterate until the end in percentage. Did that answer your
question?
In executor part, addition to LimiteState.position, this patch
uses LimiteState.backwardPosition to count how many tuples we're
back from the end of the current tuplestore. But things will get
simpler by just asking the tuplestore for the number of holding
tuples.backwardPosition hold how many tuple returned in backward scan
I meant that we don't need to hold the number in estate.
I did it this way because I didn't find an API in tuplestore to do this
+ slot = node->subSlot;
Why is this needed? The variable is properly set before use and
the assignment is bogus in the first place.its because Tuplestore needs initialized slot.
I meant that the initilized slot is overwritten before first use.
The new code block in LIMIT_RESCAN in ExecLimit is useless since
it is exatctly the same with existing code block. Why didn't you
use the existing if block?But they test different scenario
I meant that the two different scenario can share the code block.
Sorry for not clarifying will .One have to be check before offsetting and
the other is after offsetting
if (node->limitOption == PERCENTAGE)
+ { + node->perExactCount = ceil(node->percent * node->position / 100.0); + +node->position is the number of tuples returned to upper node so
far (not the number of tuples this node has read from the lower
node so far). I don't understand what the expression means.node->position hold the number of tuples this node has read from the
lower
node so far. see LIMIT_RESCAN state
Reallly? node->position is incremented when
tuplestore_gettupleslot_heaptuple() succeeded and reutnrs the
tuple to the caller immediately...+ if (node->perExactCount == node->perExactCount +
1)
+ node->perExactCount++;
What? The condition never be true. As the result, the following
if block below won't run.it became true according to the number of tuple returned in from the
lower
node so far
and percentage specification.Mmm. How do you think X can be equal to (X + 1)?
Oops my bad .The attached patch remove the need of doing that
/*
+ * Return the tuple up to the number of exact count in OFFSET + * clause without percentage value consideration. + */ + if (node->perExactCount > 0) + { ++ /* + * We may needed this tuple in backward scan so put itinto
+ * tuplestore. + */ + if (node->limitOption == PERCENTAGE) + { + tuplestore_puttupleslot(node->tupleStore, slot); + tuplestore_advance(node->tupleStore, true); + }"needed"->"need" ? The comment says that this is needed for
backward scan, but it is actually required even in forward
scan. More to the point, tuplestore_advance lacks comment.ok
Anyway, the code in LIMIT_RESCAN is broken in some ways. For
example, if it is used as the inner scan of a NestLoop, the
tuplestore accumulates tuples by every scan. You will see that
the tuplestore stores up to 1000 tuples (10 times of the inner)
by the following query.It this because in percentage we scan the whole table
It's useless and rather harmful that the tuplestore holds
indefinite number of duplicate set of the whole tuples from the
lower node. We must reuse tuples already stored in the tuplestore
or clear it before the next round.
i agree with this optimization but it don't have to be in first version
regards
Surafel
Attachments:
percent-incremental-v8.patchtext/x-patch; charset=US-ASCII; name=percent-incremental-v8.patchDownload
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 82d8140ba2..562c5acc45 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -3035,7 +3035,8 @@ estimate_path_cost_size(PlannerInfo *root,
if (fpextra && fpextra->has_limit)
{
adjust_limit_rows_costs(&rows, &startup_cost, &total_cost,
- fpextra->offset_est, fpextra->count_est);
+ fpextra->offset_est, fpextra->count_est,
+ EXACT_NUMBER);
retrieved_rows = rows;
}
}
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 06d611b64c..4ef42df51d 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1430,7 +1430,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1440,8 +1440,10 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
- and <literal>ROWS</literal> as well as <literal>FIRST</literal>
+ With <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage round up to the nearest integer. Using <literal>PERCENT</literal>
+ is best suited to returning single-digit percentages of the query's total row count.
+ <literal>ROW</literal> and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
According to the standard, the <literal>OFFSET</literal> clause must come
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index baa669abe8..d23508cc99 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -21,6 +21,8 @@
#include "postgres.h"
+#include <math.h>
+
#include "executor/executor.h"
#include "executor/nodeLimit.h"
#include "miscadmin.h"
@@ -52,6 +54,7 @@ ExecLimit(PlanState *pstate)
*/
direction = node->ps.state->es_direction;
outerPlan = outerPlanState(node);
+ slot = node->subSlot;
/*
* The main logic is a simple state machine.
@@ -81,7 +84,15 @@ ExecLimit(PlanState *pstate)
/*
* Check for empty window; if so, treat like empty subplan.
*/
- if (node->count <= 0 && !node->noCount)
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (node->percent == 0.0)
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ }
+ else if (node->count <= 0 && !node->noCount)
{
node->lstate = LIMIT_EMPTY;
return NULL;
@@ -107,6 +118,16 @@ ExecLimit(PlanState *pstate)
break;
}
+ /*
+ * We may needed this tuple in subsequent scan so put it into
+ * tuplestore.
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ tuplestore_puttupleslot(node->tupleStore, slot);
+ tuplestore_advance(node->tupleStore, true);
+ }
+
/*
* Okay, we have the first tuple of the window.
*/
@@ -124,6 +145,82 @@ ExecLimit(PlanState *pstate)
case LIMIT_INWINDOW:
if (ScanDirectionIsForward(direction))
{
+ /*
+ * In case of coming back from backward scan the tuple is
+ * already in tuple store.
+ */
+ if (node->limitOption == PERCENTAGE && node->backwardPosition > 0)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ node->backwardPosition--;
+ return slot;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+
+ /*
+ * In PERCENTAGE case no need of executing outerPlan multiple
+ * times.
+ */
+ if (node->limitOption == PERCENTAGE && node->reachEnd)
+ {
+ node->lstate = LIMIT_WINDOWEND;
+
+ /*
+ * If we know we won't need to back up, we can release
+ * resources at this point.
+ */
+ if (!(node->ps.state->es_top_eflags & EXEC_FLAG_BACKWARD))
+ (void) ExecShutdownNode(outerPlan);
+
+ return NULL;
+ }
+
+ /*
+ * When in percentage mode, we need to see if we can get any
+ * additional rows from the subplan (enough to increase the
+ * node->count value).
+ */
+ if (node->limitOption == PERCENTAGE)
+ {
+ /* loop until the node->count became greater than the number of tuple returned so far */
+ do
+ {
+ int64 cnt;
+
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->reachEnd = true;
+ node->lstate = LIMIT_SUBPLANEOF;
+
+ /*
+ * The only operation from here is backward scan
+ * but there's no API to refetch the tuple at the
+ * current position. We have to move one tuple
+ * backward, and then we will scan forward for it
+ * for the first tuple and precede as usual
+ * for the rest
+ */
+ tuplestore_advance(node->tupleStore, false);
+ return NULL;
+ }
+
+ tuplestore_puttupleslot(node->tupleStore, slot);
+
+ cnt = tuplestore_tuple_count(node->tupleStore) + node->offset;
+
+ node->count = ceil(node->percent * cnt / 100.0);
+ }while (node->position - node->offset >= node->count);
+ }
+
/*
* Forwards scan, so check for stepping off end of window. If
* we are at the end of the window, return NULL without
@@ -145,17 +242,34 @@ ExecLimit(PlanState *pstate)
return NULL;
}
- /*
- * Get next tuple from subplan, if any.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
+ if (node->limitOption == PERCENTAGE)
{
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ node->subSlot = slot;
+ node->position++;
}
- node->subSlot = slot;
- node->position++;
}
else
{
@@ -168,15 +282,29 @@ ExecLimit(PlanState *pstate)
node->lstate = LIMIT_WINDOWSTART;
return NULL;
}
-
- /*
- * Get previous tuple from subplan; there should be one!
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
+ /* In PERCENTAGE case the result is already in tuplestore */
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, false, true, slot))
+ {
+ node->subSlot = slot;
+ node->position--;
+ node->backwardPosition++;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+ /*
+ * Get previous tuple from subplan; there should be one!
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->position--;
+ }
}
break;
@@ -185,15 +313,32 @@ ExecLimit(PlanState *pstate)
return NULL;
/*
- * Backing up from subplan EOF, so re-fetch previous tuple; there
- * should be one! Note previous tuple must be in window.
+ * Scan forward for the first tuple
*/
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->lstate = LIMIT_INWINDOW;
- /* position does not change 'cause we didn't advance it before */
+ if (node->limitOption == PERCENTAGE)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == EXACT_NUMBER)
+ {
+ /*
+ * Backing up from subplan EOF, so re-fetch previous tuple;
+ * there should be one! Note previous tuple must be in
+ * window.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ /* position does not change 'cause we didn't advance it before */
+ }
break;
case LIMIT_WINDOWEND:
@@ -283,12 +428,36 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
- node->noCount = false;
+ if (node->limitOption == PERCENTAGE)
+ {
+ /*
+ * We expect to return at least one row (unless there are no
+ * rows in the subplan), and we'll update this count later as
+ * we go.
+ */
+ node->count = 0;
+ node->percent = DatumGetFloat8(val);
+
+ if (node->percent < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_RESULT_OFFSET_CLAUSE),
+ errmsg("PERCENT must not be negative")));
+
+ if (node->percent > 100)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_RESULT_OFFSET_CLAUSE),
+ errmsg("PERCENT must not be greater than 100")));
+
+ }
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+
+ }
}
}
else
@@ -301,6 +470,8 @@ recompute_limits(LimitState *node)
/* Reset position to start-of-scan */
node->position = 0;
node->subSlot = NULL;
+ node->reachEnd = false;
+ node->backwardPosition = 0;
/* Set state-machine state */
node->lstate = LIMIT_RESCAN;
@@ -309,9 +480,11 @@ recompute_limits(LimitState *node)
* Notify child node about limit. Note: think not to "optimize" by
* skipping ExecSetTupleBound if compute_tuples_needed returns < 0. We
* must update the child node anyway, in case this is a rescan and the
- * previous time we got a different result.
+ * previous time we got a different result. In PERCENTAGE option there are
+ * no bound on the number of output tuples
*/
- ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
+ if (node->limitOption != PERCENTAGE)
+ ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
}
/*
@@ -374,6 +547,7 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
/*
* Initialize result type.
@@ -390,6 +564,9 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
limitstate->ps.ps_ProjInfo = NULL;
+ if (node->limitOption == PERCENTAGE)
+ limitstate->tupleStore = tuplestore_begin_heap(true, false, work_mem);
+
return limitstate;
}
@@ -405,6 +582,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->tupleStore != NULL)
+ tuplestore_end(node->tupleStore);
}
@@ -424,4 +603,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->tupleStore != NULL)
+ tuplestore_rescan(node->tupleStore);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index a2617c7cfd..d2c6ababb3 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1147,6 +1147,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3035,6 +3036,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3119,6 +3121,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 4f2ebe5118..2e14fa72b4 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -975,6 +975,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1049,6 +1050,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e6ce8e2110..959dc79a00 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -911,6 +911,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2108,6 +2109,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2704,6 +2706,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2914,6 +2917,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 764e3bb90c..65a59d6650 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -278,6 +278,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2337,6 +2338,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 0c036209f0..01c2590b08 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2333,7 +2333,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2646,7 +2647,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6549,7 +6551,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6561,6 +6563,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 17c5f086fb..a7c5d152d5 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2247,12 +2247,25 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
*/
if (parse->sortClause)
{
- current_rel = create_ordered_paths(root,
- current_rel,
- final_target,
- final_target_parallel_safe,
- have_postponed_srfs ? -1.0 :
- limit_tuples);
+
+ /*
+ * In PERCENTAGE option there are no bound on the number of output
+ * tuples
+ */
+ if (parse->limitOption == PERCENTAGE)
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ -1.0);
+ else
+ current_rel = create_ordered_paths(root,
+ current_rel,
+ final_target,
+ final_target_parallel_safe,
+ have_postponed_srfs ? -1.0 :
+ limit_tuples);
/* Fix things up if final_target contains SRFs */
if (parse->hasTargetSRFs)
adjust_paths_for_srfs(root, current_rel,
@@ -2315,6 +2328,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 34acb732ee..704d443ac0 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3546,6 +3546,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3567,6 +3568,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
@@ -3574,7 +3576,8 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
adjust_limit_rows_costs(&pathnode->path.rows,
&pathnode->path.startup_cost,
&pathnode->path.total_cost,
- offset_est, count_est);
+ offset_est, count_est,
+ limitOption);
return pathnode;
}
@@ -3600,7 +3603,8 @@ adjust_limit_rows_costs(double *rows, /* in/out parameter */
Cost *startup_cost, /* in/out parameter */
Cost *total_cost, /* in/out parameter */
int64 offset_est,
- int64 count_est)
+ int64 count_est,
+ LimitOption limitOption)
{
double input_rows = *rows;
Cost input_startup_cost = *startup_cost;
@@ -3633,6 +3637,19 @@ adjust_limit_rows_costs(double *rows, /* in/out parameter */
count_rows = (double) count_est;
else
count_rows = clamp_row_est(input_rows * 0.10);
+ if (limitOption == PERCENTAGE)
+ {
+ double per_count = DatumGetFloat8(count_est);
+
+ count_rows = clamp_row_est((input_rows * per_count) / 100);
+ if (rows > 0)
+ {
+ *startup_cost = count_rows *
+ input_total_cost / input_rows;
+ *total_cost = input_total_cost +
+ (count_rows * 0.1);
+ }
+ }
if (count_rows > *rows)
count_rows = *rows;
if (input_rows > 0)
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 85d7a96406..79b389009c 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1288,10 +1288,11 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
}
/* transform LIMIT */
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1536,10 +1537,11 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_ORDER_BY,
false /* allow SQL92 rules */ );
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1770,10 +1772,11 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
parser_errposition(pstate,
exprLocation(list_nth(qry->targetList, tllen)))));
- qry->limitOffset = transformLimitClause(pstate, limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, limitCount,
+ qry->limitCount = transformLimitClause(pstate, limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c97bb367f8..4c6d3ded5d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -127,6 +127,21 @@ typedef struct ImportQual
List *table_names;
} ImportQual;
+/* Private struct for the result of opt_select_limit production */
+typedef struct SelectLimit
+{
+ Node *limitOffset;
+ Node *limitCount;
+ void *limitOption;
+} SelectLimit;
+
+/* Private struct for the result of limit_clause production */
+typedef struct LimitClause
+{
+ Node *limitCount;
+ void *limitOption;
+} LimitClause;
+
/* ConstraintAttributeSpec yields an integer bitmask of these flags: */
#define CAS_NOT_DEFERRABLE 0x01
#define CAS_DEFERRABLE 0x02
@@ -165,6 +180,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -242,6 +258,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
PartitionSpec *partspec;
PartitionBoundSpec *partboundspec;
RoleSpec *rolespec;
+ struct SelectLimit *SelectLimit;
+ struct LimitClause *LimitClause;
}
%type <node> stmt schema_stmt
@@ -373,6 +391,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <ival> import_qualification_type
%type <importqual> import_qualification
%type <node> vacuum_relation
+%type <SelectLimit> opt_select_limit select_limit
+%type <LimitClause> limit_clause
%type <list> stmtblock stmtmulti
OptTableElementList TableElementList OptInherit definition
@@ -393,8 +413,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
- opt_select_limit opclass_item_list opclass_drop_list
+ reloption_list group_clause TriggerFuncArgs
+ opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
prep_type_clause
@@ -455,7 +475,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -667,7 +687,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11225,14 +11245,15 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, NULL, NULL,
yyscanner);
$$ = $1;
}
| select_clause opt_sort_clause for_locking_clause opt_select_limit
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
- list_nth($4, 0), list_nth($4, 1),
+ ($4)->limitOffset, ($4)->limitCount,
+ ($4)->limitOption,
NULL,
yyscanner);
$$ = $1;
@@ -11240,7 +11261,8 @@ select_no_parens:
| select_clause opt_sort_clause select_limit opt_for_locking_clause
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
- list_nth($3, 0), list_nth($3, 1),
+ ($3)->limitOffset, ($3)->limitCount,
+ ($3)->limitOption,
NULL,
yyscanner);
$$ = $1;
@@ -11249,7 +11271,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
@@ -11257,14 +11279,15 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ NULL, $1,
yyscanner);
$$ = $2;
}
| with_clause select_clause opt_sort_clause for_locking_clause opt_select_limit
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
- list_nth($5, 0), list_nth($5, 1),
+ ($5)->limitOffset, ($5)->limitCount,
+ ($5)->limitOption,
$1,
yyscanner);
$$ = $2;
@@ -11272,7 +11295,8 @@ select_no_parens:
| with_clause select_clause opt_sort_clause select_limit opt_for_locking_clause
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
- list_nth($4, 0), list_nth($4, 1),
+ ($4)->limitOffset, ($4)->limitCount,
+ ($4)->limitOption,
$1,
yyscanner);
$$ = $2;
@@ -11566,20 +11590,60 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $2;
+ n->limitCount = ($1)->limitCount;
+ n->limitOption = ($1)->limitOption;
+ $$ = n;
+ }
+ | offset_clause limit_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $1;
+ n->limitCount = ($2)->limitCount;
+ n->limitOption = ($2)->limitOption;
+ $$ = n;
+ }
+ | limit_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = NULL;
+ n->limitCount = ($1)->limitCount;
+ n->limitOption = ($1)->limitOption;
+ $$ = n;
+ }
+ | offset_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $1;
+ n->limitCount = NULL;
+ n->limitOption = NULL;
+ $$ = n;
+ }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = NULL;
+ n->limitCount = NULL;
+ n->limitOption = NULL;
+ $$ = n;
+ }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $2;
+ n->limitOption = NULL;
+ $$ = n;
+ }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11597,9 +11661,26 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $3;
+ n->limitOption = makeString("EXACT_NUMBER");
+ $$ = n;
+ }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $3;
+ n->limitOption = makeString("PERCENTAGE");
+ $$ = n;
+ }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = makeIntConst(1, -1);
+ n->limitOption = NULL;
+ $$ = n;
+ }
;
offset_clause:
@@ -15192,6 +15273,7 @@ unreserved_keyword:
| PARTITION
| PASSING
| PASSWORD
+ | PERCENT
| PLANS
| POLICY
| PRECEDING
@@ -15856,6 +15938,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ void *limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15894,6 +15977,17 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ if (strcmp(strVal(limitOption), "PERCENTAGE") == 0)
+ stmt->limitOption = PERCENTAGE;
+ else
+ stmt->limitOption = EXACT_NUMBER;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 260ccd4d7f..0fa1d5e01e 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1704,7 +1704,7 @@ transformWhereClause(ParseState *pstate, Node *clause,
* constructName does not affect the semantics, but is used in error messages
*/
Node *
-transformLimitClause(ParseState *pstate, Node *clause,
+transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName)
{
Node *qual;
@@ -1713,8 +1713,10 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
-
- qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
+ if (limitOption == PERCENTAGE && (strcmp(constructName, "LIMIT") == 0))
+ qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
+ else
+ qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
/* LIMIT can't refer to any variables of the current query */
checkExprIsVarFree(pstate, qual, constructName);
diff --git a/src/backend/utils/sort/tuplestore.c b/src/backend/utils/sort/tuplestore.c
index 3fc7f92182..a03e568f1f 100644
--- a/src/backend/utils/sort/tuplestore.c
+++ b/src/backend/utils/sort/tuplestore.c
@@ -1100,6 +1100,39 @@ tuplestore_gettupleslot(Tuplestorestate *state, bool forward,
}
}
+/*
+ * tuplestore_gettupleslot_heaptuple
+ * It is similar to tuplestore_gettupleslot except it return stored HeapTuple
+ * instead of MinimalTuple
+ */
+bool
+tuplestore_gettupleslot_heaptuple(Tuplestorestate *state, bool forward,
+ bool copy, TupleTableSlot *slot)
+{
+ MinimalTuple tuple;
+ HeapTuple htuple;
+ bool should_free;
+
+ tuple = (MinimalTuple) tuplestore_gettuple(state, forward, &should_free);
+
+ if (tuple)
+ {
+ if (copy && !should_free)
+ {
+ tuple = heap_copy_minimal_tuple(tuple);
+ should_free = true;
+ }
+ htuple = heap_tuple_from_minimal_tuple(tuple);
+ ExecForceStoreHeapTuple(htuple, slot, should_free);
+ return true;
+ }
+ else
+ {
+ ExecClearTuple(slot);
+ return false;
+ }
+}
+
/*
* tuplestore_advance - exported function to adjust position without fetching
*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index f42189d2bf..b41151cd50 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2299,8 +2299,14 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ float8 percent; /* percentage */
+ int64 backwardPosition; /* the number of tuple returned in
+ * backward scan */
+ bool reachEnd; /* if true, outerPlan executed until the end */
+ Tuplestorestate *tupleStore; /* holds the returned tuple */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 3cbb08df92..c85b9e2ab8 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -821,4 +821,16 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ EXACT_NUMBER, /* LIMIT in exact number of rows */
+ PERCENTAGE /* LIMIT in percentage */
+}LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 94ded3c135..b158e9eaeb 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1595,6 +1596,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 23a06d718e..b77697f45a 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1793,6 +1793,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 8e6594e355..c8c497295b 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -967,6 +967,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index a12af54971..117ed8d761 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -262,10 +262,11 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern void adjust_limit_rows_costs(double *rows,
Cost *startup_cost, Cost *total_cost,
- int64 offset_est, int64 count_est);
+ int64 offset_est, int64 count_est, LimitOption limitOption);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
Relids required_outer,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index e7aaddd50d..3395e558c4 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -56,7 +56,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators, Oid *grpCollations,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 00ace8425e..e1a8d703ab 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -299,6 +299,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, UNRESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h
index 42adc63d1f..e66510e965 100644
--- a/src/include/parser/parse_clause.h
+++ b/src/include/parser/parse_clause.h
@@ -22,7 +22,7 @@ extern int setTargetTable(ParseState *pstate, RangeVar *relation,
extern Node *transformWhereClause(ParseState *pstate, Node *clause,
ParseExprKind exprKind, const char *constructName);
-extern Node *transformLimitClause(ParseState *pstate, Node *clause,
+extern Node *transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName);
extern List *transformGroupClause(ParseState *pstate, List *grouplist,
List **groupingSets,
diff --git a/src/include/utils/tuplestore.h b/src/include/utils/tuplestore.h
index f9b6fcec29..eeefde3da4 100644
--- a/src/include/utils/tuplestore.h
+++ b/src/include/utils/tuplestore.h
@@ -47,7 +47,6 @@ typedef struct Tuplestorestate Tuplestorestate;
extern Tuplestorestate *tuplestore_begin_heap(bool randomAccess,
bool interXact,
int maxKBytes);
-
extern void tuplestore_set_eflags(Tuplestorestate *state, int eflags);
extern void tuplestore_puttupleslot(Tuplestorestate *state,
@@ -72,9 +71,12 @@ extern bool tuplestore_in_memory(Tuplestorestate *state);
extern bool tuplestore_gettupleslot(Tuplestorestate *state, bool forward,
bool copy, TupleTableSlot *slot);
+extern bool tuplestore_gettupleslot_heaptuple(Tuplestorestate *state, bool forward,
+ bool copy, TupleTableSlot *slot);
extern bool tuplestore_advance(Tuplestorestate *state, bool forward);
+
extern bool tuplestore_skiptuples(Tuplestorestate *state,
int64 ntuples, bool forward);
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index c18f547cbd..95f9f5f3d9 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -108,6 +108,63 @@ SELECT ''::text AS five, unique1, unique2, stringu1
| 904 | 793 | UIAAAA
(5 rows)
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 51 | 76 | ZBAAAA
+ | 52 | 985 | ACAAAA
+ | 53 | 196 | BCAAAA
+ | 54 | 356 | CCAAAA
+ | 55 | 627 | DCAAAA
+ | 56 | 54 | ECAAAA
+ | 57 | 942 | FCAAAA
+ | 58 | 114 | GCAAAA
+ | 59 | 593 | HCAAAA
+ | 60 | 483 | ICAAAA
+(10 rows)
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 61 | 560 | JCAAAA
+(1 row)
+
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+ three | unique1 | unique2 | stringu1
+-------+---------+---------+----------
+ | 121 | 700 | REAAAA
+ | 122 | 519 | SEAAAA
+ | 123 | 777 | TEAAAA
+ | 124 | 503 | UEAAAA
+ | 125 | 849 | VEAAAA
+ | 126 | 330 | WEAAAA
+ | 127 | 511 | XEAAAA
+ | 128 | 721 | YEAAAA
+ | 129 | 696 | ZEAAAA
+(9 rows)
+
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+ eleven | unique1 | unique2 | stringu1
+--------+---------+---------+----------
+ | 10 | 520 | KAAAAA
+ | 9 | 49 | JAAAAA
+ | 8 | 653 | IAAAAA
+ | 7 | 647 | HAAAAA
+ | 6 | 978 | GAAAAA
+(5 rows)
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -286,6 +343,46 @@ fetch all in c4;
----+----
(0 rows)
+declare c5 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c5;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
+fetch 1 in c5;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c5;
+ q1 | q2
+------------------+-----
+ 4567890123456789 | 123
+(1 row)
+
+fetch backward all in c5;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+ 123 | 456
+(2 rows)
+
+fetch backward 1 in c5;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c5;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
@@ -503,3 +600,19 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
45020 | 45020
(3 rows)
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
+ s1 | s2
+-------+-------
+ 45000 | 45000
+ 45010 | 45010
+ 45020 | 45020
+ 45030 | 45030
+ 45040 | 45040
+ 45050 | 45050
+ 45060 | 45060
+ 45070 | 45070
+ 45080 | 45080
+ 45090 | 45090
+(10 rows)
+
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index 2a313d80ca..051b21a099 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -31,6 +31,23 @@ SELECT ''::text AS five, unique1, unique2, stringu1
FROM onek
ORDER BY unique1 LIMIT 5 OFFSET 900;
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -38,7 +55,6 @@ select * from int8_tbl offset (case when random() < 0.5 then null::bigint end);
-- Test assorted cases involving backwards fetch from a LIMIT plan node
begin;
-
declare c1 cursor for select * from int8_tbl limit 10;
fetch all in c1;
fetch 1 in c1;
@@ -71,6 +87,14 @@ fetch backward all in c4;
fetch backward 1 in c4;
fetch all in c4;
+declare c5 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c5;
+fetch 1 in c5;
+fetch backward 1 in c5;
+fetch backward all in c5;
+fetch backward 1 in c5;
+fetch all in c5;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
@@ -141,3 +165,6 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
from tenk1 group by thousand order by thousand limit 3;
+
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
The following review has been posted through the commitfest application:
make installcheck-world: tested, passed
Implements feature: tested, passed
Spec compliant: tested, passed
Documentation: tested, passed
Hi everyone,
The v8 patch [1]/messages/by-id/attachment/103486/percent-incremental-v8.patch applies cleanly and passes tests. I reviewed the conversation to date and from what I can see, all identified bugs have been fixed and concerns either fixed or addressed. Marking as ready for committer.
[1]: /messages/by-id/attachment/103486/percent-incremental-v8.patch
Ryan Lambert
The new status of this patch is: Ready for Committer
Surafel Temesgen <surafel3000@gmail.com> writes:
[ percent-incremental-v8.patch ]
I took a quick look through this.
* Why is this adding new functionality in tuplestore.c? Especially
functionality to get out a different tuple representation than what was
put in? I can't see a reason why nodeLimit should need that, and for
sure I don't see a reason why it would need it for this feature if it
didn't before.
* The business with query.limitCount having different data types
depending on LimitOption seems entirely bletcherous and bug-inducing.
(There are live bugs of that sort in what you have now: notably, that
kluge in adjust_limit_rows_costs will crash and burn if float8 is
pass-by-reference. Breaking the Datum abstraction is bad practice.)
I wonder if we could get away with making limitCount be float8 all the
time. This would mean that you'd lose exactness for counts above 2^53
(or so, depending on whether float8 is IEEE or not), but I doubt that
anybody would ever notice.
* Names like "PERCENTAGE" seem mighty generic to be exposing as globally
known symbols; also you're disregarding a fairly widespread PG
convention that members of an enum should have names derived from the
enum type's name. So I'd be inclined to make the values of enum
LimitOption be LIMIT_OPTION_COUNT and LIMIT_OPTION_PERCENT. (I don't
especially like EXACT_NUMBER, first because it's going to be quite long
with the prefix, and second because it suggests that maybe there's an
INEXACT_NUMBER alternative.)
* The "void *limitOption" business in gram.y also seems pretty ugly;
it'd be better for that to be enum LimitOption. I gather you want
the null to indicate "no option given", but likely it'd be better
to invent a LIMIT_OPTION_DEFAULT enum alternative for that purpose.
* An alternative to the three points above is just to separate
Query.limitCount (an int8) from Query.limitPercent (a float8), with the
understanding that at most one of the two can be non-null, and use
similar representations at other steps of the processing chain. Then
you don't need enum limitOption at all. That might not scale especially
nicely to additional variants, but trying to overload limitCount even
further isn't going to be nice either.
* The proposed change in grouping_planner() seems like a complete hack.
Why would you not change the way that limit_tuples was computed earlier,
instead? That is, it seems like the part of the planner to teach about
this feature is preprocess_limit (and limit_needed), not someplace else.
It is surely not sane that only this one planner decision would react to
a percentage-based limit.
* I didn't really study the changes in nodeLimit.c, but doing
"tuplestore_rescan" in ExecReScanLimit is surely just wrong. You
probably want to delete and recreate the tuplestore, instead, since
whatever data you already collected is of no further use. Maybe, in
the case where no rescan of the child node is needed, you could re-use
the data already collected; but that would require a bunch of additional
logic. I'm inclined to think that v1 of the patch shouldn't concern
itself with that sort of optimization.
* I don't see any delta in ruleutils.c, but surely that needs to know
about this feature so that it can correctly print views using it.
regards, tom lane
Hi Tom,
On Fri, Sep 6, 2019 at 1:26 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Surafel Temesgen <surafel3000@gmail.com> writes:
[ percent-incremental-v8.patch ]
I took a quick look through this.
* Why is this adding new functionality in tuplestore.c? Especially
functionality to get out a different tuple representation than what was
put in?
If am not mistaken tuplestore store MinimalTuple(tuple without transaction
status information) for minimal memory usage. it change what we put in into
MinimalTuple so we need the same format for getting out. The other way I
can think of doing it is allocating new MinimalTuple slot every time to get
the tuple which have resource like. The operation is very similar to
RunFromStore without the ability of reusing once allocated slot.
I can't see a reason why nodeLimit should need that, and for
sure I don't see a reason why it would need it for this feature if it
didn't before.
Only this feature needs tuple storage
* The business with query.limitCount having different data types
depending on LimitOption seems entirely bletcherous and bug-inducing.
(There are live bugs of that sort in what you have now: notably, that
kluge in adjust_limit_rows_costs will crash and burn if float8 is
pass-by-reference. Breaking the Datum abstraction is bad practice.)
I wonder if we could get away with making limitCount be float8 all the
time. This would mean that you'd lose exactness for counts above 2^53
(or so, depending on whether float8 is IEEE or not), but I doubt that
anybody would ever notice.* Names like "PERCENTAGE" seem mighty generic to be exposing as globally
known symbols; also you're disregarding a fairly widespread PG
convention that members of an enum should have names derived from the
enum type's name. So I'd be inclined to make the values of enum
LimitOption be LIMIT_OPTION_COUNT and LIMIT_OPTION_PERCENT. (I don't
especially like EXACT_NUMBER, first because it's going to be quite long
with the prefix, and second because it suggests that maybe there's an
INEXACT_NUMBER alternative.)* The "void *limitOption" business in gram.y also seems pretty ugly;
it'd be better for that to be enum LimitOption. I gather you want
the null to indicate "no option given", but likely it'd be better
to invent a LIMIT_OPTION_DEFAULT enum alternative for that purpose.
i like to fix the above three more because of scaling
* An alternative to the three points above is just to separate
Query.limitCount (an int8) from Query.limitPercent (a float8), with the
understanding that at most one of the two can be non-null, and use
similar representations at other steps of the processing chain. Then
you don't need enum limitOption at all. That might not scale especially
nicely to additional variants, but trying to overload limitCount even
further isn't going to be nice either.* The proposed change in grouping_planner() seems like a complete hack.
Why would you not change the way that limit_tuples was computed earlier,
instead?
That is, it seems like the part of the planner to teach about
this feature is preprocess_limit (and limit_needed), not someplace else.
It is surely not sane that only this one planner decision would react to
a percentage-based limit.* I didn't really study the changes in nodeLimit.c, but doing
"tuplestore_rescan" in ExecReScanLimit is surely just wrong. You
probably want to delete and recreate the tuplestore, instead, since
whatever data you already collected is of no further use. Maybe, in
the case where no rescan of the child node is needed, you could re-use
the data already collected; but that would require a bunch of additional
logic. I'm inclined to think that v1 of the patch shouldn't concern
itself with that sort of optimization.
ok
* I don't see any delta in ruleutils.c, but surely that needs to know
about this feature so that it can correctly print views using it.
ok
regards
Surafel
Hi Tom,
In the attached patch i include the comments given
regards
Surafel
Attachments:
percent-incremental-v9.patchtext/x-patch; charset=US-ASCII; name=percent-incremental-v9.patchDownload
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 82d8140ba2..692d6492bd 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -3035,7 +3035,8 @@ estimate_path_cost_size(PlannerInfo *root,
if (fpextra && fpextra->has_limit)
{
adjust_limit_rows_costs(&rows, &startup_cost, &total_cost,
- fpextra->offset_est, fpextra->count_est);
+ fpextra->offset_est, fpextra->count_est,
+ LIMIT_OPTION_COUNT);
retrieved_rows = rows;
}
}
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 06d611b64c..4ef42df51d 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1430,7 +1430,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } ONLY
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } ONLY
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1440,8 +1440,10 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
- <literal>ROW</literal>
- and <literal>ROWS</literal> as well as <literal>FIRST</literal>
+ With <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage round up to the nearest integer. Using <literal>PERCENT</literal>
+ is best suited to returning single-digit percentages of the query's total row count.
+ <literal>ROW</literal> and <literal>ROWS</literal> as well as <literal>FIRST</literal>
and <literal>NEXT</literal> are noise words that don't influence
the effects of these clauses.
According to the standard, the <literal>OFFSET</literal> clause must come
diff --git a/src/backend/catalog/sql_features.txt b/src/backend/catalog/sql_features.txt
index 059ec02cd0..1adc312f7a 100644
--- a/src/backend/catalog/sql_features.txt
+++ b/src/backend/catalog/sql_features.txt
@@ -348,7 +348,7 @@ F862 <result offset clause> in subqueries YES
F863 Nested <result offset clause> in <query expression> YES
F864 Top-level <result offset clause> in views YES
F865 <offset row count> in <result offset clause> YES
-F866 FETCH FIRST clause: PERCENT option NO
+F866 FETCH FIRST clause: PERCENT option YES
F867 FETCH FIRST clause: WITH TIES option NO
R010 Row pattern recognition: FROM clause NO
R020 Row pattern recognition: WINDOW clause NO
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index baa669abe8..c03bb2c971 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -21,6 +21,8 @@
#include "postgres.h"
+#include <math.h>
+
#include "executor/executor.h"
#include "executor/nodeLimit.h"
#include "miscadmin.h"
@@ -52,6 +54,7 @@ ExecLimit(PlanState *pstate)
*/
direction = node->ps.state->es_direction;
outerPlan = outerPlanState(node);
+ slot = node->subSlot;
/*
* The main logic is a simple state machine.
@@ -81,7 +84,15 @@ ExecLimit(PlanState *pstate)
/*
* Check for empty window; if so, treat like empty subplan.
*/
- if (node->count <= 0 && !node->noCount)
+ if (node->limitOption == LIMIT_OPTION_PERCENT)
+ {
+ if (node->percent == 0.0)
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ }
+ else if (node->count <= 0 && !node->noCount)
{
node->lstate = LIMIT_EMPTY;
return NULL;
@@ -107,6 +118,16 @@ ExecLimit(PlanState *pstate)
break;
}
+ /*
+ * We may needed this tuple in subsequent scan so put it into
+ * tuplestore.
+ */
+ if (node->limitOption == LIMIT_OPTION_PERCENT)
+ {
+ tuplestore_puttupleslot(node->tupleStore, slot);
+ tuplestore_advance(node->tupleStore, true);
+ }
+
/*
* Okay, we have the first tuple of the window.
*/
@@ -124,6 +145,82 @@ ExecLimit(PlanState *pstate)
case LIMIT_INWINDOW:
if (ScanDirectionIsForward(direction))
{
+ /*
+ * In case of coming back from backward scan the tuple is
+ * already in tuple store.
+ */
+ if (node->limitOption == LIMIT_OPTION_PERCENT && node->backwardPosition > 0)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ node->backwardPosition--;
+ return slot;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+
+ /*
+ * In LIMIT_OPTION_PERCENT case no need of executing outerPlan multiple
+ * times.
+ */
+ if (node->limitOption == LIMIT_OPTION_PERCENT && node->reachEnd)
+ {
+ node->lstate = LIMIT_WINDOWEND;
+
+ /*
+ * If we know we won't need to back up, we can release
+ * resources at this point.
+ */
+ if (!(node->ps.state->es_top_eflags & EXEC_FLAG_BACKWARD))
+ (void) ExecShutdownNode(outerPlan);
+
+ return NULL;
+ }
+
+ /*
+ * When in percentage mode, we need to see if we can get any
+ * additional rows from the subplan (enough to increase the
+ * node->count value).
+ */
+ if (node->limitOption == LIMIT_OPTION_PERCENT)
+ {
+ /* loop until the node->count became greater than the number of tuple returned so far */
+ do
+ {
+ int64 cnt;
+
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->reachEnd = true;
+ node->lstate = LIMIT_SUBPLANEOF;
+
+ /*
+ * The only operation from here is backward scan
+ * but there's no API to refetch the tuple at the
+ * current position. We have to move one tuple
+ * backward, and then we will scan forward for it
+ * for the first tuple and precede as usual
+ * for the rest
+ */
+ tuplestore_advance(node->tupleStore, false);
+ return NULL;
+ }
+
+ tuplestore_puttupleslot(node->tupleStore, slot);
+
+ cnt = tuplestore_tuple_count(node->tupleStore) + node->offset;
+
+ node->count = ceil(node->percent * cnt / 100.0);
+ }while (node->position - node->offset >= node->count);
+ }
+
/*
* Forwards scan, so check for stepping off end of window. If
* we are at the end of the window, return NULL without
@@ -145,17 +242,34 @@ ExecLimit(PlanState *pstate)
return NULL;
}
- /*
- * Get next tuple from subplan, if any.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
+ if (node->limitOption == LIMIT_OPTION_PERCENT)
{
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+
+ }
+ else if (node->limitOption == LIMIT_OPTION_COUNT)
+ {
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ node->subSlot = slot;
+ node->position++;
}
- node->subSlot = slot;
- node->position++;
}
else
{
@@ -168,15 +282,29 @@ ExecLimit(PlanState *pstate)
node->lstate = LIMIT_WINDOWSTART;
return NULL;
}
-
- /*
- * Get previous tuple from subplan; there should be one!
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
+ /* In LIMIT_OPTION_PERCENT case the result is already in tuplestore */
+ if (node->limitOption == LIMIT_OPTION_PERCENT)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, false, true, slot))
+ {
+ node->subSlot = slot;
+ node->position--;
+ node->backwardPosition++;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == LIMIT_OPTION_COUNT)
+ {
+ /*
+ * Get previous tuple from subplan; there should be one!
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->position--;
+ }
}
break;
@@ -185,15 +313,32 @@ ExecLimit(PlanState *pstate)
return NULL;
/*
- * Backing up from subplan EOF, so re-fetch previous tuple; there
- * should be one! Note previous tuple must be in window.
+ * Scan forward for the first tuple
*/
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->lstate = LIMIT_INWINDOW;
- /* position does not change 'cause we didn't advance it before */
+ if (node->limitOption == LIMIT_OPTION_PERCENT)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else if (node->limitOption == LIMIT_OPTION_COUNT)
+ {
+ /*
+ * Backing up from subplan EOF, so re-fetch previous tuple;
+ * there should be one! Note previous tuple must be in
+ * window.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ /* position does not change 'cause we didn't advance it before */
+ }
break;
case LIMIT_WINDOWEND:
@@ -283,12 +428,36 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
- node->noCount = false;
+ if (node->limitOption == LIMIT_OPTION_PERCENT)
+ {
+ /*
+ * We expect to return at least one row (unless there are no
+ * rows in the subplan), and we'll update this count later as
+ * we go.
+ */
+ node->count = 0;
+ node->percent = DatumGetFloat8(val);
+
+ if (node->percent < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("PERCENT must not be negative")));
+
+ if (node->percent > 100)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("PERCENT must not be greater than 100")));
+
+ }
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+
+ }
}
}
else
@@ -301,6 +470,8 @@ recompute_limits(LimitState *node)
/* Reset position to start-of-scan */
node->position = 0;
node->subSlot = NULL;
+ node->reachEnd = false;
+ node->backwardPosition = 0;
/* Set state-machine state */
node->lstate = LIMIT_RESCAN;
@@ -309,7 +480,8 @@ recompute_limits(LimitState *node)
* Notify child node about limit. Note: think not to "optimize" by
* skipping ExecSetTupleBound if compute_tuples_needed returns < 0. We
* must update the child node anyway, in case this is a rescan and the
- * previous time we got a different result.
+ * previous time we got a different result. In LIMIT_OPTION_PERCENT option there are
+ * no bound on the number of output tuples
*/
ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
}
@@ -321,7 +493,7 @@ recompute_limits(LimitState *node)
static int64
compute_tuples_needed(LimitState *node)
{
- if (node->noCount)
+ if (node->noCount || node->limitOption == LIMIT_OPTION_PERCENT)
return -1;
/* Note: if this overflows, we'll return a negative value, which is OK */
return node->count + node->offset;
@@ -374,6 +546,7 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
(PlanState *) limitstate);
limitstate->limitCount = ExecInitExpr((Expr *) node->limitCount,
(PlanState *) limitstate);
+ limitstate->limitOption = node->limitOption;
/*
* Initialize result type.
@@ -390,6 +563,9 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
*/
limitstate->ps.ps_ProjInfo = NULL;
+ if (node->limitOption == LIMIT_OPTION_PERCENT)
+ limitstate->tupleStore = tuplestore_begin_heap(true, false, work_mem);
+
return limitstate;
}
@@ -405,6 +581,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->tupleStore != NULL)
+ tuplestore_end(node->tupleStore);
}
@@ -424,4 +602,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->tupleStore != NULL)
+ tuplestore_clear(node->tupleStore);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index a2617c7cfd..d2c6ababb3 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1147,6 +1147,7 @@ _copyLimit(const Limit *from)
*/
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
return newnode;
}
@@ -3035,6 +3036,7 @@ _copyQuery(const Query *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
@@ -3119,6 +3121,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(sortClause);
COPY_NODE_FIELD(limitOffset);
COPY_NODE_FIELD(limitCount);
+ COPY_SCALAR_FIELD(limitOption);
COPY_NODE_FIELD(lockingClause);
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 4f2ebe5118..2e14fa72b4 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -975,6 +975,7 @@ _equalQuery(const Query *a, const Query *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
@@ -1049,6 +1050,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(sortClause);
COMPARE_NODE_FIELD(limitOffset);
COMPARE_NODE_FIELD(limitCount);
+ COMPARE_SCALAR_FIELD(limitOption);
COMPARE_NODE_FIELD(lockingClause);
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e6ce8e2110..959dc79a00 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -911,6 +911,7 @@ _outLimit(StringInfo str, const Limit *node)
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2108,6 +2109,7 @@ _outLimitPath(StringInfo str, const LimitPath *node)
WRITE_NODE_FIELD(subpath);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
}
static void
@@ -2704,6 +2706,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(lockingClause);
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
@@ -2914,6 +2917,7 @@ _outQuery(StringInfo str, const Query *node)
WRITE_NODE_FIELD(sortClause);
WRITE_NODE_FIELD(limitOffset);
WRITE_NODE_FIELD(limitCount);
+ WRITE_ENUM_FIELD(limitOption, LimitOption);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 764e3bb90c..65a59d6650 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -278,6 +278,7 @@ _readQuery(void)
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
@@ -2337,6 +2338,7 @@ _readLimit(void)
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
+ READ_ENUM_FIELD(limitOption, LimitOption);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 0c036209f0..01c2590b08 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2333,7 +2333,8 @@ create_minmaxagg_plan(PlannerInfo *root, MinMaxAggPath *best_path)
plan = (Plan *) make_limit(plan,
subparse->limitOffset,
- subparse->limitCount);
+ subparse->limitCount,
+ subparse->limitOption);
/* Must apply correct cost/width data to Limit node */
plan->startup_cost = mminfo->path->startup_cost;
@@ -2646,7 +2647,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
plan = make_limit(subplan,
best_path->limitOffset,
- best_path->limitCount);
+ best_path->limitCount,
+ best_path->limitOption);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -6549,7 +6551,7 @@ make_lockrows(Plan *lefttree, List *rowMarks, int epqParam)
* Build a Limit plan node
*/
Limit *
-make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
+make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption)
{
Limit *node = makeNode(Limit);
Plan *plan = &node->plan;
@@ -6561,6 +6563,7 @@ make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount)
node->limitOffset = limitOffset;
node->limitCount = limitCount;
+ node->limitOption = limitOption;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 17c5f086fb..72452a7d5b 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2315,6 +2315,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
path = (Path *) create_limit_path(root, final_rel, path,
parse->limitOffset,
parse->limitCount,
+ parse->limitOption,
offset_est, count_est);
}
@@ -2884,12 +2885,17 @@ preprocess_limit(PlannerInfo *root, double tuple_fraction,
/*
* A LIMIT clause limits the absolute number of tuples returned.
* However, if it's not a constant LIMIT then we have to guess; for
- * lack of a better idea, assume 10% of the plan's result is wanted.
+ * lack of a better idea, assume 10% of the plan's result is wanted for
+ * LIMIT_OPTION_COUNT and 100% wanted in a case of LIMIT_OPTION_PERCENT.
+
*/
if (*count_est < 0 || *offset_est < 0)
{
/* LIMIT or OFFSET is an expression ... punt ... */
- limit_fraction = 0.10;
+ if (parse->limitOption == LIMIT_OPTION_PERCENT)
+ limit_fraction = 1.0;
+ else
+ limit_fraction = 0.10;
}
else
{
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 34acb732ee..15ad2ae6aa 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3546,6 +3546,7 @@ LimitPath *
create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est)
{
LimitPath *pathnode = makeNode(LimitPath);
@@ -3567,6 +3568,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->subpath = subpath;
pathnode->limitOffset = limitOffset;
pathnode->limitCount = limitCount;
+ pathnode->limitOption = limitOption;
/*
* Adjust the output rows count and costs according to the offset/limit.
@@ -3574,7 +3576,8 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
adjust_limit_rows_costs(&pathnode->path.rows,
&pathnode->path.startup_cost,
&pathnode->path.total_cost,
- offset_est, count_est);
+ offset_est, count_est,
+ limitOption);
return pathnode;
}
@@ -3600,7 +3603,8 @@ adjust_limit_rows_costs(double *rows, /* in/out parameter */
Cost *startup_cost, /* in/out parameter */
Cost *total_cost, /* in/out parameter */
int64 offset_est,
- int64 count_est)
+ int64 count_est,
+ LimitOption limitOption)
{
double input_rows = *rows;
Cost input_startup_cost = *startup_cost;
@@ -3633,6 +3637,17 @@ adjust_limit_rows_costs(double *rows, /* in/out parameter */
count_rows = (double) count_est;
else
count_rows = clamp_row_est(input_rows * 0.10);
+ if (limitOption == LIMIT_OPTION_PERCENT)
+ {
+ count_rows = clamp_row_est((input_rows * count_est) / 100);
+ if (rows > 0)
+ {
+ *startup_cost = count_rows *
+ input_total_cost / input_rows;
+ *total_cost = input_total_cost +
+ (count_rows * 0.1);
+ }
+ }
if (count_rows > *rows)
count_rows = *rows;
if (input_rows > 0)
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 85d7a96406..79b389009c 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1288,10 +1288,11 @@ transformSelectStmt(ParseState *pstate, SelectStmt *stmt)
}
/* transform LIMIT */
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
/* transform window clauses after we have seen all window functions */
qry->windowClause = transformWindowDefinitions(pstate,
@@ -1536,10 +1537,11 @@ transformValuesClause(ParseState *pstate, SelectStmt *stmt)
EXPR_KIND_ORDER_BY,
false /* allow SQL92 rules */ );
- qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, stmt->limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, stmt->limitCount,
+ qry->limitCount = transformLimitClause(pstate, stmt->limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
if (stmt->lockingClause)
ereport(ERROR,
@@ -1770,10 +1772,11 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
parser_errposition(pstate,
exprLocation(list_nth(qry->targetList, tllen)))));
- qry->limitOffset = transformLimitClause(pstate, limitOffset,
+ qry->limitOffset = transformLimitClause(pstate, limitOffset, stmt->limitOption,
EXPR_KIND_OFFSET, "OFFSET");
- qry->limitCount = transformLimitClause(pstate, limitCount,
+ qry->limitCount = transformLimitClause(pstate, limitCount, stmt->limitOption,
EXPR_KIND_LIMIT, "LIMIT");
+ qry->limitOption = stmt->limitOption;
qry->rtable = pstate->p_rtable;
qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index a954acf509..789aab7fe4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -127,6 +127,21 @@ typedef struct ImportQual
List *table_names;
} ImportQual;
+/* Private struct for the result of opt_select_limit production */
+typedef struct SelectLimit
+{
+ Node *limitOffset;
+ Node *limitCount;
+ LimitOption limitOption;
+} SelectLimit;
+
+/* Private struct for the result of limit_clause production */
+typedef struct LimitClause
+{
+ Node *limitCount;
+ LimitOption limitOption;
+} LimitClause;
+
/* ConstraintAttributeSpec yields an integer bitmask of these flags: */
#define CAS_NOT_DEFERRABLE 0x01
#define CAS_DEFERRABLE 0x02
@@ -165,6 +180,7 @@ static List *makeOrderedSetArgs(List *directargs, List *orderedargs,
static void insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
WithClause *withClause,
core_yyscan_t yyscanner);
static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
@@ -242,6 +258,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
PartitionSpec *partspec;
PartitionBoundSpec *partboundspec;
RoleSpec *rolespec;
+ struct SelectLimit *SelectLimit;
+ struct LimitClause *LimitClause;
}
%type <node> stmt schema_stmt
@@ -373,6 +391,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <ival> import_qualification_type
%type <importqual> import_qualification
%type <node> vacuum_relation
+%type <SelectLimit> opt_select_limit select_limit
+%type <LimitClause> limit_clause
%type <list> stmtblock stmtmulti
OptTableElementList TableElementList OptInherit definition
@@ -393,8 +413,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
target_list opt_target_list insert_column_list set_target_list
set_clause_list set_clause
def_list operator_def_list indirection opt_indirection
- reloption_list group_clause TriggerFuncArgs select_limit
- opt_select_limit opclass_item_list opclass_drop_list
+ reloption_list group_clause TriggerFuncArgs
+ opclass_item_list opclass_drop_list
opclass_purpose opt_opfamily transaction_mode_list_or_empty
OptTableFuncElementList TableFuncElementList opt_type_modifiers
prep_type_clause
@@ -455,7 +475,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
comment_type_any_name comment_type_name
security_label_type_any_name security_label_type_name
-%type <node> fetch_args limit_clause select_limit_value
+%type <node> fetch_args select_limit_value
offset_clause select_offset_value
select_fetch_first_value I_or_F_const
%type <ival> row_or_rows first_or_next
@@ -667,7 +687,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11225,14 +11245,15 @@ select_no_parens:
| select_clause sort_clause
{
insertSelectOptions((SelectStmt *) $1, $2, NIL,
- NULL, NULL, NULL,
+ NULL, NULL, LIMIT_OPTION_DEFAULT, NULL,
yyscanner);
$$ = $1;
}
| select_clause opt_sort_clause for_locking_clause opt_select_limit
{
insertSelectOptions((SelectStmt *) $1, $2, $3,
- list_nth($4, 0), list_nth($4, 1),
+ ($4)->limitOffset, ($4)->limitCount,
+ ($4)->limitOption,
NULL,
yyscanner);
$$ = $1;
@@ -11240,7 +11261,8 @@ select_no_parens:
| select_clause opt_sort_clause select_limit opt_for_locking_clause
{
insertSelectOptions((SelectStmt *) $1, $2, $4,
- list_nth($3, 0), list_nth($3, 1),
+ ($3)->limitOffset, ($3)->limitCount,
+ ($3)->limitOption,
NULL,
yyscanner);
$$ = $1;
@@ -11249,7 +11271,7 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, NULL, NIL,
NULL, NULL,
- $1,
+ LIMIT_OPTION_DEFAULT, $1,
yyscanner);
$$ = $2;
}
@@ -11257,14 +11279,15 @@ select_no_parens:
{
insertSelectOptions((SelectStmt *) $2, $3, NIL,
NULL, NULL,
- $1,
+ LIMIT_OPTION_DEFAULT, $1,
yyscanner);
$$ = $2;
}
| with_clause select_clause opt_sort_clause for_locking_clause opt_select_limit
{
insertSelectOptions((SelectStmt *) $2, $3, $4,
- list_nth($5, 0), list_nth($5, 1),
+ ($5)->limitOffset, ($5)->limitCount,
+ ($5)->limitOption,
$1,
yyscanner);
$$ = $2;
@@ -11272,7 +11295,8 @@ select_no_parens:
| with_clause select_clause opt_sort_clause select_limit opt_for_locking_clause
{
insertSelectOptions((SelectStmt *) $2, $3, $5,
- list_nth($4, 0), list_nth($4, 1),
+ ($4)->limitOffset, ($4)->limitCount,
+ ($4)->limitOption,
$1,
yyscanner);
$$ = $2;
@@ -11566,20 +11590,60 @@ sortby: a_expr USING qual_all_Op opt_nulls_order
select_limit:
- limit_clause offset_clause { $$ = list_make2($2, $1); }
- | offset_clause limit_clause { $$ = list_make2($1, $2); }
- | limit_clause { $$ = list_make2(NULL, $1); }
- | offset_clause { $$ = list_make2($1, NULL); }
+ limit_clause offset_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $2;
+ n->limitCount = ($1)->limitCount;
+ n->limitOption = ($1)->limitOption;
+ $$ = n;
+ }
+ | offset_clause limit_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $1;
+ n->limitCount = ($2)->limitCount;
+ n->limitOption = ($2)->limitOption;
+ $$ = n;
+ }
+ | limit_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = NULL;
+ n->limitCount = ($1)->limitCount;
+ n->limitOption = ($1)->limitOption;
+ $$ = n;
+ }
+ | offset_clause
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = $1;
+ n->limitCount = NULL;
+ n->limitOption = LIMIT_OPTION_DEFAULT;
+ $$ = n;
+ }
;
opt_select_limit:
select_limit { $$ = $1; }
- | /* EMPTY */ { $$ = list_make2(NULL,NULL); }
+ | /* EMPTY */
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = NULL;
+ n->limitCount = NULL;
+ n->limitOption = LIMIT_OPTION_DEFAULT;
+ $$ = n;
+ }
;
limit_clause:
LIMIT select_limit_value
- { $$ = $2; }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $2;
+ n->limitOption = LIMIT_OPTION_COUNT;
+ $$ = n;
+ }
| LIMIT select_limit_value ',' select_offset_value
{
/* Disabled because it was too confusing, bjm 2002-02-18 */
@@ -11597,9 +11661,26 @@ limit_clause:
* we can see the ONLY token in the lookahead slot.
*/
| FETCH first_or_next select_fetch_first_value row_or_rows ONLY
- { $$ = $3; }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $3;
+ n->limitOption = LIMIT_OPTION_COUNT;
+ $$ = n;
+ }
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = $3;
+ n->limitOption = LIMIT_OPTION_PERCENT;
+ $$ = n;
+ }
| FETCH first_or_next row_or_rows ONLY
- { $$ = makeIntConst(1, -1); }
+ {
+ LimitClause *n = (LimitClause *) palloc(sizeof(LimitClause));
+ n->limitCount = makeIntConst(1, -1);
+ n->limitOption = LIMIT_OPTION_COUNT;
+ $$ = n;
+ }
;
offset_clause:
@@ -15192,6 +15273,7 @@ unreserved_keyword:
| PARTITION
| PASSING
| PASSWORD
+ | PERCENT
| PLANS
| POLICY
| PRECEDING
@@ -15856,6 +15938,7 @@ static void
insertSelectOptions(SelectStmt *stmt,
List *sortClause, List *lockingClause,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
WithClause *withClause,
core_yyscan_t yyscanner)
{
@@ -15894,6 +15977,14 @@ insertSelectOptions(SelectStmt *stmt,
parser_errposition(exprLocation(limitCount))));
stmt->limitCount = limitCount;
}
+ if (limitOption && limitOption != LIMIT_OPTION_DEFAULT)
+ {
+ if (stmt->limitOption)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("multiple LIMIT options not allowed")));
+ stmt->limitOption = limitOption;
+ }
if (withClause)
{
if (stmt->withClause)
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 260ccd4d7f..884a4a01b4 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1704,7 +1704,7 @@ transformWhereClause(ParseState *pstate, Node *clause,
* constructName does not affect the semantics, but is used in error messages
*/
Node *
-transformLimitClause(ParseState *pstate, Node *clause,
+transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName)
{
Node *qual;
@@ -1713,8 +1713,10 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
-
- qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
+ if (limitOption == LIMIT_OPTION_PERCENT && strcmp(constructName, "LIMIT") == 0)
+ qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
+ else
+ qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
/* LIMIT can't refer to any variables of the current query */
checkExprIsVarFree(pstate, qual, constructName);
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 3e64390d81..bf88e276e7 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -5310,7 +5310,15 @@ get_select_query_def(Query *query, deparse_context *context,
-PRETTYINDENT_STD, PRETTYINDENT_STD, 0);
get_rule_expr(query->limitOffset, context, false);
}
- if (query->limitCount != NULL)
+ if (query->limitOption == LIMIT_OPTION_PERCENT)
+ {
+ appendContextKeyword(context, " FETCH FIRST ",
+ -PRETTYINDENT_STD, PRETTYINDENT_STD, 0);
+ get_rule_expr(query->limitCount, context, false);
+ appendContextKeyword(context, " PERCENT ROWS ONLY ",
+ -PRETTYINDENT_STD, PRETTYINDENT_STD, 0);
+ }
+ if (query->limitCount != NULL && query->limitOption != LIMIT_OPTION_PERCENT)
{
appendContextKeyword(context, " LIMIT ",
-PRETTYINDENT_STD, PRETTYINDENT_STD, 0);
diff --git a/src/backend/utils/sort/tuplestore.c b/src/backend/utils/sort/tuplestore.c
index 3fc7f92182..a03e568f1f 100644
--- a/src/backend/utils/sort/tuplestore.c
+++ b/src/backend/utils/sort/tuplestore.c
@@ -1100,6 +1100,39 @@ tuplestore_gettupleslot(Tuplestorestate *state, bool forward,
}
}
+/*
+ * tuplestore_gettupleslot_heaptuple
+ * It is similar to tuplestore_gettupleslot except it return stored HeapTuple
+ * instead of MinimalTuple
+ */
+bool
+tuplestore_gettupleslot_heaptuple(Tuplestorestate *state, bool forward,
+ bool copy, TupleTableSlot *slot)
+{
+ MinimalTuple tuple;
+ HeapTuple htuple;
+ bool should_free;
+
+ tuple = (MinimalTuple) tuplestore_gettuple(state, forward, &should_free);
+
+ if (tuple)
+ {
+ if (copy && !should_free)
+ {
+ tuple = heap_copy_minimal_tuple(tuple);
+ should_free = true;
+ }
+ htuple = heap_tuple_from_minimal_tuple(tuple);
+ ExecForceStoreHeapTuple(htuple, slot, should_free);
+ return true;
+ }
+ else
+ {
+ ExecClearTuple(slot);
+ return false;
+ }
+}
+
/*
* tuplestore_advance - exported function to adjust position without fetching
*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index f42189d2bf..b41151cd50 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2299,8 +2299,14 @@ typedef struct LimitState
PlanState ps; /* its first field is NodeTag */
ExprState *limitOffset; /* OFFSET parameter, or NULL if none */
ExprState *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ float8 percent; /* percentage */
+ int64 backwardPosition; /* the number of tuple returned in
+ * backward scan */
+ bool reachEnd; /* if true, outerPlan executed until the end */
+ Tuplestorestate *tupleStore; /* holds the returned tuple */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 3cbb08df92..08bd32867e 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -821,4 +821,17 @@ typedef enum OnConflictAction
ONCONFLICT_UPDATE /* ON CONFLICT ... DO UPDATE */
} OnConflictAction;
+/*
+ * LimitOption -
+ * LIMIT option of query
+ *
+ * This is needed in both parsenodes.h and plannodes.h, so put it here...
+ */
+typedef enum LimitOption
+{
+ LIMIT_OPTION_COUNT, /* LIMIT in exact number of rows */
+ LIMIT_OPTION_PERCENT, /* LIMIT in percentage */
+ LIMIT_OPTION_DEFAULT /* There are no LIMIT */
+}LimitOption;
+
#endif /* NODES_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 94ded3c135..b158e9eaeb 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,6 +159,7 @@ typedef struct Query
Node *limitOffset; /* # of result tuples to skip (int8 expr) */
Node *limitCount; /* # of result tuples to return (int8 expr) */
+ LimitOption limitOption; /* limit type */
List *rowMarks; /* a list of RowMarkClause's */
@@ -1595,6 +1596,7 @@ typedef struct SelectStmt
List *sortClause; /* sort clause (a list of SortBy's) */
Node *limitOffset; /* # of result tuples to skip */
Node *limitCount; /* # of result tuples to return */
+ LimitOption limitOption; /* limit type */
List *lockingClause; /* FOR UPDATE (list of LockingClause's) */
WithClause *withClause; /* WITH clause */
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 23a06d718e..b77697f45a 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1793,6 +1793,7 @@ typedef struct LimitPath
Path *subpath; /* path representing input source */
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} LimitPath;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 8e6594e355..c8c497295b 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -967,6 +967,7 @@ typedef struct Limit
Plan plan;
Node *limitOffset; /* OFFSET parameter, or NULL if none */
Node *limitCount; /* COUNT parameter, or NULL if none */
+ LimitOption limitOption; /* LIMIT in percentage or exact number */
} Limit;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index a12af54971..117ed8d761 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -262,10 +262,11 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath,
Node *limitOffset, Node *limitCount,
+ LimitOption limitOption,
int64 offset_est, int64 count_est);
extern void adjust_limit_rows_costs(double *rows,
Cost *startup_cost, Cost *total_cost,
- int64 offset_est, int64 count_est);
+ int64 offset_est, int64 count_est, LimitOption limitOption);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
Relids required_outer,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index e7aaddd50d..3395e558c4 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -56,7 +56,7 @@ extern Agg *make_agg(List *tlist, List *qual,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators, Oid *grpCollations,
List *groupingSets, List *chain,
double dNumGroups, Plan *lefttree);
-extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
+extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount, LimitOption limitOption);
/*
* prototypes for plan/initsplan.c
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 00ace8425e..e1a8d703ab 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -299,6 +299,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, UNRESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h
index 42adc63d1f..e66510e965 100644
--- a/src/include/parser/parse_clause.h
+++ b/src/include/parser/parse_clause.h
@@ -22,7 +22,7 @@ extern int setTargetTable(ParseState *pstate, RangeVar *relation,
extern Node *transformWhereClause(ParseState *pstate, Node *clause,
ParseExprKind exprKind, const char *constructName);
-extern Node *transformLimitClause(ParseState *pstate, Node *clause,
+extern Node *transformLimitClause(ParseState *pstate, Node *clause, LimitOption limitOption,
ParseExprKind exprKind, const char *constructName);
extern List *transformGroupClause(ParseState *pstate, List *grouplist,
List **groupingSets,
diff --git a/src/include/utils/tuplestore.h b/src/include/utils/tuplestore.h
index f9b6fcec29..eeefde3da4 100644
--- a/src/include/utils/tuplestore.h
+++ b/src/include/utils/tuplestore.h
@@ -47,7 +47,6 @@ typedef struct Tuplestorestate Tuplestorestate;
extern Tuplestorestate *tuplestore_begin_heap(bool randomAccess,
bool interXact,
int maxKBytes);
-
extern void tuplestore_set_eflags(Tuplestorestate *state, int eflags);
extern void tuplestore_puttupleslot(Tuplestorestate *state,
@@ -72,9 +71,12 @@ extern bool tuplestore_in_memory(Tuplestorestate *state);
extern bool tuplestore_gettupleslot(Tuplestorestate *state, bool forward,
bool copy, TupleTableSlot *slot);
+extern bool tuplestore_gettupleslot_heaptuple(Tuplestorestate *state, bool forward,
+ bool copy, TupleTableSlot *slot);
extern bool tuplestore_advance(Tuplestorestate *state, bool forward);
+
extern bool tuplestore_skiptuples(Tuplestorestate *state,
int64 ntuples, bool forward);
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index c18f547cbd..95f9f5f3d9 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -108,6 +108,63 @@ SELECT ''::text AS five, unique1, unique2, stringu1
| 904 | 793 | UIAAAA
(5 rows)
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 51 | 76 | ZBAAAA
+ | 52 | 985 | ACAAAA
+ | 53 | 196 | BCAAAA
+ | 54 | 356 | CCAAAA
+ | 55 | 627 | DCAAAA
+ | 56 | 54 | ECAAAA
+ | 57 | 942 | FCAAAA
+ | 58 | 114 | GCAAAA
+ | 59 | 593 | HCAAAA
+ | 60 | 483 | ICAAAA
+(10 rows)
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 61 | 560 | JCAAAA
+(1 row)
+
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+ three | unique1 | unique2 | stringu1
+-------+---------+---------+----------
+ | 121 | 700 | REAAAA
+ | 122 | 519 | SEAAAA
+ | 123 | 777 | TEAAAA
+ | 124 | 503 | UEAAAA
+ | 125 | 849 | VEAAAA
+ | 126 | 330 | WEAAAA
+ | 127 | 511 | XEAAAA
+ | 128 | 721 | YEAAAA
+ | 129 | 696 | ZEAAAA
+(9 rows)
+
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+ eleven | unique1 | unique2 | stringu1
+--------+---------+---------+----------
+ | 10 | 520 | KAAAAA
+ | 9 | 49 | JAAAAA
+ | 8 | 653 | IAAAAA
+ | 7 | 647 | HAAAAA
+ | 6 | 978 | GAAAAA
+(5 rows)
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -286,6 +343,46 @@ fetch all in c4;
----+----
(0 rows)
+declare c5 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c5;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
+fetch 1 in c5;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c5;
+ q1 | q2
+------------------+-----
+ 4567890123456789 | 123
+(1 row)
+
+fetch backward all in c5;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+ 123 | 456
+(2 rows)
+
+fetch backward 1 in c5;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c5;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
@@ -503,3 +600,19 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
45020 | 45020
(3 rows)
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
+ s1 | s2
+-------+-------
+ 45000 | 45000
+ 45010 | 45010
+ 45020 | 45020
+ 45030 | 45030
+ 45040 | 45040
+ 45050 | 45050
+ 45060 | 45060
+ 45070 | 45070
+ 45080 | 45080
+ 45090 | 45090
+(10 rows)
+
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index 2a313d80ca..051b21a099 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -31,6 +31,23 @@ SELECT ''::text AS five, unique1, unique2, stringu1
FROM onek
ORDER BY unique1 LIMIT 5 OFFSET 900;
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -38,7 +55,6 @@ select * from int8_tbl offset (case when random() < 0.5 then null::bigint end);
-- Test assorted cases involving backwards fetch from a LIMIT plan node
begin;
-
declare c1 cursor for select * from int8_tbl limit 10;
fetch all in c1;
fetch 1 in c1;
@@ -71,6 +87,14 @@ fetch backward all in c4;
fetch backward 1 in c4;
fetch all in c4;
+declare c5 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c5;
+fetch 1 in c5;
+fetch backward 1 in c5;
+fetch backward all in c5;
+fetch backward 1 in c5;
+fetch all in c5;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
@@ -141,3 +165,6 @@ select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
from tenk1 group by thousand order by thousand limit 3;
+
+select sum(tenthous) as s1, sum(tenthous) + random()*0 as s2
+ from tenk1 group by thousand order by thousand FETCH FIRST 1 PERCENT ROWS ONLY;
On Thu, Sep 19, 2019 at 6:52 AM Surafel Temesgen <surafel3000@gmail.com>
wrote:
Hi Tom,
In the attached patch i include the comments givenregards
Surafel
Patch v9 applies and passes make installcheck-world.
From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Date: 2019-09-05 22:26:29
* I didn't really study the changes in nodeLimit.c, but doing
"tuplestore_rescan" in ExecReScanLimit is surely just wrong. You
probably want to delete and recreate the tuplestore, instead, since
whatever data you already collected is of no further use. Maybe, in
the case where no rescan of the child node is needed, you could re-use
the data already collected; but that would require a bunch of additional
logic. I'm inclined to think that v1 of the patch shouldn't concern
itself with that sort of optimization.
I don't think this was addressed.
Ryan Lambert
Show quoted text
At Thu, 26 Sep 2019 15:13:42 -0600, Ryan Lambert <ryan@rustprooflabs.com> wrote in <CAN-V+g_cG4_R9q25nccNoMrtwA7DsCuHh-G5_ygaj8PBFTWUDw@mail.gmail.com>
On Thu, Sep 19, 2019 at 6:52 AM Surafel Temesgen <surafel3000@gmail.com>
wrote:Hi Tom,
In the attached patch i include the comments givenregards
SurafelPatch v9 applies and passes make installcheck-world.
From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Date: 2019-09-05 22:26:29* I didn't really study the changes in nodeLimit.c, but doing
"tuplestore_rescan" in ExecReScanLimit is surely just wrong. You
probably want to delete and recreate the tuplestore, instead, since
whatever data you already collected is of no further use. Maybe, in
the case where no rescan of the child node is needed, you could re-use
the data already collected; but that would require a bunch of additional
logic. I'm inclined to think that v1 of the patch shouldn't concern
itself with that sort of optimization.I don't think this was addressed.
It seems to me that Tom is suggesting to conisider that kind of
optimization after once completing the patch without it. (That in
the first patch was just wrong, but more stuff is needed to
achieve that.)
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
The following review has been posted through the commitfest application:
make installcheck-world: tested, passed
Implements feature: tested, passed
Spec compliant: not tested
Documentation: not tested
The latest patch applies cleanly and passes all tests. I believe all feedback has been addressed except for the one case that can be improved upon in later work. Updating status as ready for committer.
Ryan Lambert
The new status of this patch is: Ready for Committer
Hi,
This patch is marked as RFC since September. Since then there was no
discussion on this thread, but Andrew proposed an alternative approach
based on window functions in a separate thread [1]/messages/by-id/87o8wvz253.fsf@news-spur.riddles.org.uk (both for this and
for the WITH TIES case).
I'll set this patch back to "needs review" - at this point we need to
decide which of the approaches is the right one.
regards
[1]: /messages/by-id/87o8wvz253.fsf@news-spur.riddles.org.uk
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Thank you for the notification, Tomas.
At Thu, 16 Jan 2020 22:33:58 +0100, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote in
This patch is marked as RFC since September. Since then there was no
discussion on this thread, but Andrew proposed an alternative approach
based on window functions in a separate thread [1] (both for this and
for the WITH TIES case).I'll set this patch back to "needs review" - at this point we need to
decide which of the approaches is the right one.
Even consdiering that we have only two usage , WITH TIES and PERCENT,
of the mechanism for now, I like [1] for the generic nature,
simpleness and smaller invasiveness to executor. The fact that cursor
needs materialize to allow backward scan would not be a big problem.
It seems that PERCENT will be easily implemented on that.
However, isn't there a possibility that we allow FETCH FIRST x PERCENT
WITH TIES, though I'm not sure what SQL spec tells about that? I
don't come up right now how to do that using window functions..
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
On 17/01/2020 07:20, Kyotaro Horiguchi wrote:
Thank you for the notification, Tomas.
At Thu, 16 Jan 2020 22:33:58 +0100, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote in
This patch is marked as RFC since September. Since then there was no
discussion on this thread, but Andrew proposed an alternative approach
based on window functions in a separate thread [1] (both for this and
for the WITH TIES case).I'll set this patch back to "needs review" - at this point we need to
decide which of the approaches is the right one.Even consdiering that we have only two usage , WITH TIES and PERCENT,
of the mechanism for now, I like [1] for the generic nature,
simpleness and smaller invasiveness to executor. The fact that cursor
needs materialize to allow backward scan would not be a big problem.
It seems that PERCENT will be easily implemented on that.However, isn't there a possibility that we allow FETCH FIRST x PERCENT
WITH TIES, though I'm not sure what SQL spec tells about that? I
don't come up right now how to do that using window functions..
PERCENT and WITH TIES can play together, per spec.
--
Vik Fearing
Hi
PERCENT and WITH TIES can play together, per spec.
I also Implement PERCENT WITH TIES option. patch is attached
i don't start a new tread because the patches share common code
regards
Surafel
Attachments:
percent-incremental-v10.patchtext/x-patch; charset=US-ASCII; name=percent-incremental-v10.patchDownload
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 9fc53cad68..d512f4ddd0 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -3097,7 +3097,8 @@ estimate_path_cost_size(PlannerInfo *root,
if (fpextra && fpextra->has_limit)
{
adjust_limit_rows_costs(&rows, &startup_cost, &total_cost,
- fpextra->offset_est, fpextra->count_est);
+ LIMIT_OPTION_COUNT, fpextra->offset_est,
+ fpextra->count_est);
retrieved_rows = rows;
}
}
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index b93e4ca208..9811380ec6 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } { ONLY | WITH TIES } ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } { ONLY | WITH TIES } ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1434,7 +1434,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } { ONLY | WITH TIES }
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } { ONLY | WITH TIES }
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1444,6 +1444,9 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
+ With <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage round up to the nearest integer. Using <literal>PERCENT</literal>
+ is best suited to returning single-digit percentages of the query's total row count.
The <literal>WITH TIES</literal> option is used to return any additional
rows that tie for the last place in the result set according to
the <literal>ORDER BY</literal> clause; <literal>ORDER BY</literal>
diff --git a/src/backend/catalog/sql_features.txt b/src/backend/catalog/sql_features.txt
index b6e58e8493..68e0c5e2e1 100644
--- a/src/backend/catalog/sql_features.txt
+++ b/src/backend/catalog/sql_features.txt
@@ -344,7 +344,7 @@ F862 <result offset clause> in subqueries YES
F863 Nested <result offset clause> in <query expression> YES
F864 Top-level <result offset clause> in views YES
F865 <offset row count> in <result offset clause> YES
-F866 FETCH FIRST clause: PERCENT option NO
+F866 FETCH FIRST clause: PERCENT option YES
F867 FETCH FIRST clause: WITH TIES option YES
R010 Row pattern recognition: FROM clause NO
R020 Row pattern recognition: WINDOW clause NO
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index d85cf7d93e..cfb28a9fd4 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -21,6 +21,8 @@
#include "postgres.h"
+#include <math.h>
+
#include "executor/executor.h"
#include "executor/nodeLimit.h"
#include "miscadmin.h"
@@ -29,6 +31,8 @@
static void recompute_limits(LimitState *node);
static int64 compute_tuples_needed(LimitState *node);
+#define IsPercentOption(opt) \
+ (opt == LIMIT_OPTION_PERCENT || opt == LIMIT_OPTION_PER_WITH_TIES)
/* ----------------------------------------------------------------
* ExecLimit
@@ -53,6 +57,7 @@ ExecLimit(PlanState *pstate)
*/
direction = node->ps.state->es_direction;
outerPlan = outerPlanState(node);
+ slot = node->subSlot;
/*
* The main logic is a simple state machine.
@@ -82,7 +87,15 @@ ExecLimit(PlanState *pstate)
/*
* Check for empty window; if so, treat like empty subplan.
*/
- if (node->count <= 0 && !node->noCount)
+ if (IsPercentOption(node->limitOption))
+ {
+ if (node->percent == 0.0)
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ }
+ else if (node->count <= 0 && !node->noCount)
{
node->lstate = LIMIT_EMPTY;
return NULL;
@@ -118,6 +131,16 @@ ExecLimit(PlanState *pstate)
break;
}
+ /*
+ * We may needed this tuple in subsequent scan so put it into
+ * tuplestore.
+ */
+ if (IsPercentOption(node->limitOption))
+ {
+ tuplestore_puttupleslot(node->tupleStore, slot);
+ tuplestore_advance(node->tupleStore, true);
+ }
+
/*
* Okay, we have the first tuple of the window.
*/
@@ -135,6 +158,85 @@ ExecLimit(PlanState *pstate)
case LIMIT_INWINDOW:
if (ScanDirectionIsForward(direction))
{
+ /*
+ * In case of coming back from backward scan the tuple is
+ * already in tuple store.
+ */
+ if (IsPercentOption(node->limitOption) && node->backwardPosition > 0)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ node->backwardPosition--;
+ return slot;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+
+ /*
+ * In LIMIT_OPTION_PERCENT case no need of executing outerPlan
+ * multiple times.
+ */
+ if (IsPercentOption(node->limitOption) && node->reachEnd)
+ {
+ node->lstate = LIMIT_WINDOWEND;
+
+ /*
+ * If we know we won't need to back up, we can release
+ * resources at this point.
+ */
+ if (!(node->ps.state->es_top_eflags & EXEC_FLAG_BACKWARD))
+ (void) ExecShutdownNode(outerPlan);
+
+ return NULL;
+ }
+
+ /*
+ * When in percentage mode, we need to see if we can get any
+ * additional rows from the subplan (enough to increase the
+ * node->count value).
+ */
+ if (IsPercentOption(node->limitOption))
+ {
+ /*
+ * loop until the node->count became greater than the
+ * number of tuple returned so far
+ */
+ do
+ {
+ int64 cnt;
+
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->reachEnd = true;
+ node->lstate = LIMIT_SUBPLANEOF;
+
+ /*
+ * The only operation from here is backward scan
+ * but there's no API to refetch the tuple at the
+ * current position. We have to move one tuple
+ * backward, and then we will scan forward for it
+ * for the first tuple and precede as usual for
+ * the rest
+ */
+ tuplestore_advance(node->tupleStore, false);
+ return NULL;
+ }
+
+ tuplestore_puttupleslot(node->tupleStore, slot);
+
+ cnt = tuplestore_tuple_count(node->tupleStore) + node->offset;
+
+ node->count = ceil(node->percent * cnt / 100.0);
+ } while (node->position - node->offset >= node->count);
+ }
+
/*
* Forwards scan, so check for stepping off end of window. At
* the end of the window, the behavior depends on whether WITH
@@ -152,7 +254,8 @@ ExecLimit(PlanState *pstate)
* whether rescans are possible.
*/
if (!node->noCount &&
- node->position - node->offset >= node->count)
+ node->position - node->offset >= node->count
+ && !IsPercentOption(node->limitOption))
{
if (node->limitOption == LIMIT_OPTION_COUNT)
{
@@ -166,54 +269,88 @@ ExecLimit(PlanState *pstate)
}
}
else
+ {
+ if (IsPercentOption(node->limitOption))
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ break;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+
+ }
+ else
+ {
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+
+ /*
+ * If WITH TIES is active, and this is the last
+ * in-window tuple, save it to be used in subsequent
+ * WINDOWEND_TIES processing.
+ */
+ if (node->limitOption == LIMIT_OPTION_WITH_TIES &&
+ node->position - node->offset == node->count - 1)
+ {
+ ExecCopySlot(node->last_slot, slot);
+ }
+ node->subSlot = slot;
+ node->position++;
+ break;
+ }
+ }
+ }
+ else
+ {
+ /*
+ * Backwards scan, so check for stepping off start of window.
+ * As above, only change state-machine status if so.
+ */
+ if (node->position <= node->offset + 1)
+ {
+ node->lstate = LIMIT_WINDOWSTART;
+ return NULL;
+ }
+
+ /* In percent case the result is already in tuplestore */
+ if (IsPercentOption(node->limitOption))
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, false, true, slot))
+ {
+ node->subSlot = slot;
+ node->position--;
+ node->backwardPosition++;
+ break;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else
{
/*
- * Get next tuple from subplan, if any.
+ * Get previous tuple from subplan; there should be one!
*/
slot = ExecProcNode(outerPlan);
if (TupIsNull(slot))
- {
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
- }
-
- /*
- * If WITH TIES is active, and this is the last in-window
- * tuple, save it to be used in subsequent WINDOWEND_TIES
- * processing.
- */
- if (node->limitOption == LIMIT_OPTION_WITH_TIES &&
- node->position - node->offset == node->count - 1)
- {
- ExecCopySlot(node->last_slot, slot);
- }
+ elog(ERROR, "LIMIT subplan failed to run backwards");
node->subSlot = slot;
- node->position++;
+ node->position--;
break;
}
}
- else
- {
- /*
- * Backwards scan, so check for stepping off start of window.
- * As above, only change state-machine status if so.
- */
- if (node->position <= node->offset + 1)
- {
- node->lstate = LIMIT_WINDOWSTART;
- return NULL;
- }
-
- /*
- * Get previous tuple from subplan; there should be one!
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
- break;
- }
Assert(node->lstate == LIMIT_WINDOWEND_TIES);
/* FALL THRU */
@@ -279,15 +416,32 @@ ExecLimit(PlanState *pstate)
return NULL;
/*
- * Backing up from subplan EOF, so re-fetch previous tuple; there
- * should be one! Note previous tuple must be in window.
+ * Scan forward for the first tuple
*/
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->lstate = LIMIT_INWINDOW;
- /* position does not change 'cause we didn't advance it before */
+ if (IsPercentOption(node->limitOption))
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else
+ {
+ /*
+ * Backing up from subplan EOF, so re-fetch previous tuple;
+ * there should be one! Note previous tuple must be in
+ * window.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ /* position does not change 'cause we didn't advance it before */
+ }
break;
case LIMIT_WINDOWEND:
@@ -393,12 +547,36 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
- node->noCount = false;
+ if (IsPercentOption(node->limitOption))
+ {
+ /*
+ * We expect to return at least one row (unless there are no
+ * rows in the subplan), and we'll update this count later as
+ * we go.
+ */
+ node->count = 0;
+ node->percent = DatumGetFloat8(val);
+
+ if (node->percent < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("PERCENT must not be negative")));
+
+ if (node->percent > 100)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("PERCENT must not be greater than 100")));
+
+ }
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+
+ }
}
}
else
@@ -411,6 +589,8 @@ recompute_limits(LimitState *node)
/* Reset position to start-of-scan */
node->position = 0;
node->subSlot = NULL;
+ node->reachEnd = false;
+ node->backwardPosition = 0;
/* Set state-machine state */
node->lstate = LIMIT_RESCAN;
@@ -419,7 +599,8 @@ recompute_limits(LimitState *node)
* Notify child node about limit. Note: think not to "optimize" by
* skipping ExecSetTupleBound if compute_tuples_needed returns < 0. We
* must update the child node anyway, in case this is a rescan and the
- * previous time we got a different result.
+ * previous time we got a different result. In LIMIT_OPTION_PERCENT option
+ * there are no bound on the number of output tuples
*/
ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
}
@@ -431,7 +612,8 @@ recompute_limits(LimitState *node)
static int64
compute_tuples_needed(LimitState *node)
{
- if ((node->noCount) || (node->limitOption == LIMIT_OPTION_WITH_TIES))
+ if ((node->noCount) || (node->limitOption == LIMIT_OPTION_WITH_TIES)
+ || (IsPercentOption(node->limitOption)))
return -1;
/* Note: if this overflows, we'll return a negative value, which is OK */
return node->count + node->offset;
@@ -521,6 +703,9 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
&limitstate->ps);
}
+ if (IsPercentOption(node->limitOption))
+ limitstate->tupleStore = tuplestore_begin_heap(true, false, work_mem);
+
return limitstate;
}
@@ -536,6 +721,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->tupleStore != NULL)
+ tuplestore_end(node->tupleStore);
}
@@ -555,4 +742,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->tupleStore != NULL)
+ tuplestore_clear(node->tupleStore);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index c1fc866cbf..c4dabb02de 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3665,7 +3665,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
adjust_limit_rows_costs(&pathnode->path.rows,
&pathnode->path.startup_cost,
&pathnode->path.total_cost,
- offset_est, count_est);
+ limitOption, offset_est, count_est);
return pathnode;
}
@@ -3690,6 +3690,7 @@ void
adjust_limit_rows_costs(double *rows, /* in/out parameter */
Cost *startup_cost, /* in/out parameter */
Cost *total_cost, /* in/out parameter */
+ LimitOption limitOption,
int64 offset_est,
int64 count_est)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dbb47d4982..0168cb37f8 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -680,7 +680,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11559,6 +11559,14 @@ limit_clause:
n->limitOption = LIMIT_OPTION_COUNT;
$$ = n;
}
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = NULL;
+ n->limitCount = $3;
+ n->limitOption = LIMIT_OPTION_PERCENT;
+ $$ = n;
+ }
| FETCH first_or_next select_fetch_first_value row_or_rows WITH TIES
{
SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
@@ -15202,6 +15210,7 @@ unreserved_keyword:
| PARTITION
| PASSING
| PASSWORD
+ | PERCENT
| PLANS
| POLICY
| PRECEDING
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 6fff13479e..d3782697c8 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1754,8 +1754,10 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
-
- qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
+ if (limitOption == LIMIT_OPTION_PERCENT && strcmp(constructName, "LIMIT") == 0)
+ qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
+ else
+ qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
/* LIMIT can't refer to any variables of the current query */
checkExprIsVarFree(pstate, qual, constructName);
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 2cbcb4b85e..43e5647201 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -5243,7 +5243,15 @@ get_select_query_def(Query *query, deparse_context *context,
-PRETTYINDENT_STD, PRETTYINDENT_STD, 0);
get_rule_expr(query->limitOffset, context, false);
}
- if (query->limitCount != NULL)
+ if (query->limitOption == LIMIT_OPTION_PERCENT)
+ {
+ appendContextKeyword(context, " FETCH FIRST ",
+ -PRETTYINDENT_STD, PRETTYINDENT_STD, 0);
+ get_rule_expr(query->limitCount, context, false);
+ appendContextKeyword(context, " PERCENT ROWS ONLY ",
+ -PRETTYINDENT_STD, PRETTYINDENT_STD, 0);
+ }
+ if (query->limitCount != NULL && query->limitOption != LIMIT_OPTION_PERCENT)
{
if (query->limitOption == LIMIT_OPTION_WITH_TIES)
{
diff --git a/src/backend/utils/sort/tuplestore.c b/src/backend/utils/sort/tuplestore.c
index 452a85a423..8743a0f03d 100644
--- a/src/backend/utils/sort/tuplestore.c
+++ b/src/backend/utils/sort/tuplestore.c
@@ -1100,6 +1100,39 @@ tuplestore_gettupleslot(Tuplestorestate *state, bool forward,
}
}
+/*
+ * tuplestore_gettupleslot_heaptuple
+ * It is similar to tuplestore_gettupleslot except it return stored HeapTuple
+ * instead of MinimalTuple
+ */
+bool
+tuplestore_gettupleslot_heaptuple(Tuplestorestate *state, bool forward,
+ bool copy, TupleTableSlot *slot)
+{
+ MinimalTuple tuple;
+ HeapTuple htuple;
+ bool should_free;
+
+ tuple = (MinimalTuple) tuplestore_gettuple(state, forward, &should_free);
+
+ if (tuple)
+ {
+ if (copy && !should_free)
+ {
+ tuple = heap_copy_minimal_tuple(tuple);
+ should_free = true;
+ }
+ htuple = heap_tuple_from_minimal_tuple(tuple);
+ ExecForceStoreHeapTuple(htuple, slot, should_free);
+ return true;
+ }
+ else
+ {
+ ExecClearTuple(slot);
+ return false;
+ }
+}
+
/*
* tuplestore_advance - exported function to adjust position without fetching
*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cf832d7f90..d9c441187a 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2501,6 +2501,11 @@ typedef struct LimitState
LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ float8 percent; /* percentage */
+ int64 backwardPosition; /* the number of tuple returned in
+ * backward scan */
+ bool reachEnd; /* if true, outerPlan executed until the end */
+ Tuplestorestate *tupleStore; /* holds the returned tuple */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 381d84b4e4..9253ac734c 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -836,6 +836,8 @@ typedef enum LimitOption
{
LIMIT_OPTION_COUNT, /* FETCH FIRST... ONLY */
LIMIT_OPTION_WITH_TIES, /* FETCH FIRST... WITH TIES */
+ LIMIT_OPTION_PERCENT, /* FETCH FIRST... PERCENT ONLY */
+ LIMIT_OPTION_PER_WITH_TIES, /* FETCH FIRST... PERCENT WITH TIES */
LIMIT_OPTION_DEFAULT, /* No limit present */
} LimitOption;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 715a24ad29..ddc8f74138 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -272,7 +272,7 @@ extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
int64 offset_est, int64 count_est);
extern void adjust_limit_rows_costs(double *rows,
Cost *startup_cost, Cost *total_cost,
- int64 offset_est, int64 count_est);
+ LimitOption limitOption, int64 offset_est, int64 count_est);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
Relids required_outer,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 08f22ce211..1605a6f8ea 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -306,6 +306,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD)
+PG_KEYWORD("percent", PERCENT, UNRESERVED_KEYWORD)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD)
diff --git a/src/include/utils/tuplestore.h b/src/include/utils/tuplestore.h
index 2c6403f9e8..1d59aa0d57 100644
--- a/src/include/utils/tuplestore.h
+++ b/src/include/utils/tuplestore.h
@@ -47,7 +47,6 @@ typedef struct Tuplestorestate Tuplestorestate;
extern Tuplestorestate *tuplestore_begin_heap(bool randomAccess,
bool interXact,
int maxKBytes);
-
extern void tuplestore_set_eflags(Tuplestorestate *state, int eflags);
extern void tuplestore_puttupleslot(Tuplestorestate *state,
@@ -72,9 +71,12 @@ extern bool tuplestore_in_memory(Tuplestorestate *state);
extern bool tuplestore_gettupleslot(Tuplestorestate *state, bool forward,
bool copy, TupleTableSlot *slot);
+extern bool tuplestore_gettupleslot_heaptuple(Tuplestorestate *state, bool forward,
+ bool copy, TupleTableSlot *slot);
extern bool tuplestore_advance(Tuplestorestate *state, bool forward);
+
extern bool tuplestore_skiptuples(Tuplestorestate *state,
int64 ntuples, bool forward);
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index e6f6809fbe..46557e2dae 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -108,6 +108,63 @@ SELECT ''::text AS five, unique1, unique2, stringu1
| 904 | 793 | UIAAAA
(5 rows)
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 51 | 76 | ZBAAAA
+ | 52 | 985 | ACAAAA
+ | 53 | 196 | BCAAAA
+ | 54 | 356 | CCAAAA
+ | 55 | 627 | DCAAAA
+ | 56 | 54 | ECAAAA
+ | 57 | 942 | FCAAAA
+ | 58 | 114 | GCAAAA
+ | 59 | 593 | HCAAAA
+ | 60 | 483 | ICAAAA
+(10 rows)
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 61 | 560 | JCAAAA
+(1 row)
+
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+ three | unique1 | unique2 | stringu1
+-------+---------+---------+----------
+ | 121 | 700 | REAAAA
+ | 122 | 519 | SEAAAA
+ | 123 | 777 | TEAAAA
+ | 124 | 503 | UEAAAA
+ | 125 | 849 | VEAAAA
+ | 126 | 330 | WEAAAA
+ | 127 | 511 | XEAAAA
+ | 128 | 721 | YEAAAA
+ | 129 | 696 | ZEAAAA
+(9 rows)
+
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+ eleven | unique1 | unique2 | stringu1
+--------+---------+---------+----------
+ | 10 | 520 | KAAAAA
+ | 9 | 49 | JAAAAA
+ | 8 | 653 | IAAAAA
+ | 7 | 647 | HAAAAA
+ | 6 | 978 | GAAAAA
+(5 rows)
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -338,6 +395,46 @@ fetch backward all in c5;
123 | 456
(2 rows)
+declare c6 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c6;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
+fetch 1 in c6;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c6;
+ q1 | q2
+------------------+-----
+ 4567890123456789 | 123
+(1 row)
+
+fetch backward all in c6;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+ 123 | 456
+(2 rows)
+
+fetch backward 1 in c6;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c6;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index d2d4ef132d..c6e660913f 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -31,6 +31,23 @@ SELECT ''::text AS five, unique1, unique2, stringu1
FROM onek
ORDER BY unique1 LIMIT 5 OFFSET 900;
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -38,7 +55,6 @@ select * from int8_tbl offset (case when random() < 0.5 then null::bigint end);
-- Test assorted cases involving backwards fetch from a LIMIT plan node
begin;
-
declare c1 cursor for select * from int8_tbl limit 10;
fetch all in c1;
fetch 1 in c1;
@@ -81,6 +97,14 @@ fetch backward all in c5;
fetch all in c5;
fetch backward all in c5;
+declare c6 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c6;
+fetch 1 in c6;
+fetch backward 1 in c6;
+fetch backward all in c6;
+fetch backward 1 in c6;
+fetch all in c6;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
percent_with_ties_v1.patchtext/x-patch; charset=US-ASCII; name=percent_with_ties_v1.patchDownload
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index cfb28a9fd4..bb117c77fd 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -193,6 +193,12 @@ ExecLimit(PlanState *pstate)
if (!(node->ps.state->es_top_eflags & EXEC_FLAG_BACKWARD))
(void) ExecShutdownNode(outerPlan);
+ /*
+ * The only operation from here is backward scan We have
+ * to move one postion forward to get previous tuple
+ */
+ tuplestore_advance(node->tupleStore, true);
+
return NULL;
}
@@ -215,26 +221,51 @@ ExecLimit(PlanState *pstate)
if (TupIsNull(slot))
{
node->reachEnd = true;
- node->lstate = LIMIT_SUBPLANEOF;
-
/*
- * The only operation from here is backward scan
- * but there's no API to refetch the tuple at the
- * current position. We have to move one tuple
- * backward, and then we will scan forward for it
- * for the first tuple and precede as usual for
- * the rest
+ * If WITH TIES is active, get previous tuple,
+ * save it to be used in subsequent WINDOWEND_TIES
+ * processing.
*/
- tuplestore_advance(node->tupleStore, false);
- return NULL;
- }
+ if (node->limitOption == LIMIT_OPTION_PER_WITH_TIES)
+ {
+ slot = node->subSlot;
+ tuplestore_advance(node->tupleStore, false);
+ if (!tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ tuplestore_advance(node->tupleStore, true);
+ return NULL;
+ }
+
+ ExecCopySlot(node->last_slot, slot);
+ node->lstate = LIMIT_WINDOWEND_TIES;
+ /* we'll fall through to the next case */
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
- tuplestore_puttupleslot(node->tupleStore, slot);
+ /*
+ * The only operation from here is backward
+ * scan but there's no API to refetch the
+ * tuple at the current position. We have to
+ * move one postion backward, and then we will
+ * scan forward for it for the first tuple and
+ * precede as usual for the rest
+ */
+ tuplestore_advance(node->tupleStore, true);
+ return NULL;
+ }
+ }
+ if (node->lstate != LIMIT_WINDOWEND_TIES)
+ {
+ tuplestore_puttupleslot(node->tupleStore, slot);
- cnt = tuplestore_tuple_count(node->tupleStore) + node->offset;
+ cnt = tuplestore_tuple_count(node->tupleStore) + node->offset;
- node->count = ceil(node->percent * cnt / 100.0);
- } while (node->position - node->offset >= node->count);
+ node->count = ceil(node->percent * cnt / 100.0);
+ }
+ } while (node->position - node->offset >= node->count && node->lstate != LIMIT_WINDOWEND_TIES);
}
/*
@@ -255,7 +286,7 @@ ExecLimit(PlanState *pstate)
*/
if (!node->noCount &&
node->position - node->offset >= node->count
- && !IsPercentOption(node->limitOption))
+ && !IsPercentOption(node->limitOption) && node->lstate != LIMIT_WINDOWEND_TIES)
{
if (node->limitOption == LIMIT_OPTION_COUNT)
{
@@ -268,49 +299,45 @@ ExecLimit(PlanState *pstate)
/* we'll fall through to the next case */
}
}
- else
+ else if (IsPercentOption(node->limitOption) && node->lstate != LIMIT_WINDOWEND_TIES)
{
- if (IsPercentOption(node->limitOption))
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
{
- if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
- {
- node->subSlot = slot;
- node->position++;
- break;
- }
- else
- {
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
- }
-
- }
- else
- {
- /*
- * Get next tuple from subplan, if any.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- {
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
- }
-
- /*
- * If WITH TIES is active, and this is the last
- * in-window tuple, save it to be used in subsequent
- * WINDOWEND_TIES processing.
- */
- if (node->limitOption == LIMIT_OPTION_WITH_TIES &&
- node->position - node->offset == node->count - 1)
- {
- ExecCopySlot(node->last_slot, slot);
- }
node->subSlot = slot;
node->position++;
break;
}
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+ else if (!IsPercentOption(node->limitOption) && node->lstate != LIMIT_WINDOWEND_TIES)
+ {
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+
+ /*
+ * If WITH TIES is active, and this is the last in-window
+ * tuple, save it to be used in subsequent WINDOWEND_TIES
+ * processing.
+ */
+ if (node->limitOption == LIMIT_OPTION_WITH_TIES &&
+ node->position - node->offset == node->count - 1)
+ {
+ ExecCopySlot(node->last_slot, slot);
+ }
+ node->subSlot = slot;
+ node->position++;
+ break;
}
}
else
@@ -359,14 +386,25 @@ ExecLimit(PlanState *pstate)
if (ScanDirectionIsForward(direction))
{
/*
- * Advance the subplan until we find the first row with
- * different ORDER BY pathkeys.
+ * Advance the subplan or tuple store until we find the first
+ * row with different ORDER BY pathkeys.
*/
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
+ if (node->limitOption == LIMIT_OPTION_PER_WITH_TIES)
{
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
+ if (!tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+ else
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
}
/*
@@ -399,15 +437,30 @@ ExecLimit(PlanState *pstate)
}
/*
- * Get previous tuple from subplan; there should be one! And
- * change state-machine status.
+ * Get previous tuple from subplan or tuple store; there
+ * should be one! And change state-machine status.
*/
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
- node->lstate = LIMIT_INWINDOW;
+ if (node->limitOption == LIMIT_OPTION_PER_WITH_TIES)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, false, true, slot))
+ {
+ node->backwardPosition++;
+ node->position--;
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->position--;
+ node->lstate = LIMIT_INWINDOW;
+ }
}
break;
@@ -416,11 +469,12 @@ ExecLimit(PlanState *pstate)
return NULL;
/*
- * Scan forward for the first tuple
+ * Scan forward for the previous tuple. there should be one! Note
+ * previous tuple must be in window.
*/
if (IsPercentOption(node->limitOption))
{
- if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, false, true, slot))
{
node->subSlot = slot;
node->lstate = LIMIT_INWINDOW;
@@ -461,6 +515,16 @@ ExecLimit(PlanState *pstate)
node->subSlot = slot;
node->lstate = LIMIT_INWINDOW;
}
+ if (node->limitOption == LIMIT_OPTION_PER_WITH_TIES)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, false, true, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
else
{
/*
@@ -686,7 +750,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
/*
* Initialize the equality evaluation, to detect ties.
*/
- if (node->limitOption == LIMIT_OPTION_WITH_TIES)
+ if (node->limitOption == LIMIT_OPTION_WITH_TIES
+ || node->limitOption == LIMIT_OPTION_PER_WITH_TIES)
{
TupleDesc desc;
const TupleTableSlotOps *ops;
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 99278eed93..c4a81dd2ad 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2701,7 +2701,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
subplan = create_plan_recurse(root, best_path->subpath, flags);
/* Extract information necessary for comparing rows for WITH TIES. */
- if (best_path->limitOption == LIMIT_OPTION_WITH_TIES)
+ if (best_path->limitOption == LIMIT_OPTION_WITH_TIES ||
+ best_path->limitOption == LIMIT_OPTION_PER_WITH_TIES)
{
Query *parse = root->parse;
ListCell *l;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 0168cb37f8..d18aad8722 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -11575,6 +11575,14 @@ limit_clause:
n->limitOption = LIMIT_OPTION_WITH_TIES;
$$ = n;
}
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows WITH TIES
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = NULL;
+ n->limitCount = $3;
+ n->limitOption = LIMIT_OPTION_PER_WITH_TIES;
+ $$ = n;
+ }
| FETCH first_or_next row_or_rows ONLY
{
SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
@@ -15921,7 +15929,8 @@ insertSelectOptions(SelectStmt *stmt,
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("multiple limit options not allowed")));
- if (!stmt->sortClause && limitClause->limitOption == LIMIT_OPTION_WITH_TIES)
+ if (!stmt->sortClause && (limitClause->limitOption == LIMIT_OPTION_WITH_TIES
+ || limitClause->limitOption == LIMIT_OPTION_PER_WITH_TIES))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("WITH TIES options can not be specified without ORDER BY clause")));
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index d3782697c8..2837771d1d 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1754,7 +1754,9 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
- if (limitOption == LIMIT_OPTION_PERCENT && strcmp(constructName, "LIMIT") == 0)
+ if ((limitOption == LIMIT_OPTION_PERCENT || limitOption == LIMIT_OPTION_PER_WITH_TIES)
+ && strcmp(constructName, "LIMIT") == 0)
+
qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
else
qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index 46557e2dae..7352632b6c 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -435,6 +435,43 @@ fetch all in c6;
4567890123456789 | 123
(3 rows)
+declare c7 cursor for select * from int8_tbl order by q1 fetch first 15 percent rows with ties;
+fetch all in c7;
+ q1 | q2
+-----+------------------
+ 123 | 456
+ 123 | 4567890123456789
+(2 rows)
+
+fetch 1 in c7;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c7;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+(1 row)
+
+fetch backward all in c7;
+ q1 | q2
+-----+-----
+ 123 | 456
+(1 row)
+
+fetch backward 1 in c7;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c7;
+ q1 | q2
+-----+------------------
+ 123 | 456
+ 123 | 4567890123456789
+(2 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
@@ -716,6 +753,80 @@ SELECT thousand
0
(2 rows)
+--
+-- FETCH FIRST
+-- Check the PERCENT WITH TIES clause
+--
+SELECT thousand
+ FROM onek WHERE thousand < 5
+ ORDER BY thousand FETCH FIRST 2 PERCENT ROW ONLY;
+ thousand
+----------
+ 0
+(1 row)
+
+SELECT thousand
+ FROM onek WHERE thousand < 5
+ ORDER BY thousand FETCH FIRST 2 PERCENT ROWS WITH TIES;
+ thousand
+----------
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+(10 rows)
+
+SELECT thousand
+ FROM onek WHERE thousand < 5
+ ORDER BY thousand FETCH FIRST 21 PERCENT ROW ONLY;
+ thousand
+----------
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 1
+(11 rows)
+
+SELECT thousand
+ FROM onek WHERE thousand < 5
+ ORDER BY thousand FETCH FIRST 21 PERCENT ROW WITH TIES;
+ thousand
+----------
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 1
+ 1
+ 1
+ 1
+ 1
+ 1
+ 1
+ 1
+ 1
+ 1
+(20 rows)
+
-- should fail
SELECT ''::text AS two, unique1, unique2, stringu1
FROM onek WHERE unique1 > 50
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index c6e660913f..f9e3d24910 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -105,6 +105,14 @@ fetch backward all in c6;
fetch backward 1 in c6;
fetch all in c6;
+declare c7 cursor for select * from int8_tbl order by q1 fetch first 15 percent rows with ties;
+fetch all in c7;
+fetch 1 in c7;
+fetch backward 1 in c7;
+fetch backward all in c7;
+fetch backward 1 in c7;
+fetch all in c7;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
@@ -197,6 +205,27 @@ SELECT thousand
FROM onek WHERE thousand < 5
ORDER BY thousand FETCH FIRST 2 ROW ONLY;
+--
+-- FETCH FIRST
+-- Check the PERCENT WITH TIES clause
+--
+
+SELECT thousand
+ FROM onek WHERE thousand < 5
+ ORDER BY thousand FETCH FIRST 2 PERCENT ROW ONLY;
+
+SELECT thousand
+ FROM onek WHERE thousand < 5
+ ORDER BY thousand FETCH FIRST 2 PERCENT ROWS WITH TIES;
+
+SELECT thousand
+ FROM onek WHERE thousand < 5
+ ORDER BY thousand FETCH FIRST 21 PERCENT ROW ONLY;
+
+SELECT thousand
+ FROM onek WHERE thousand < 5
+ ORDER BY thousand FETCH FIRST 21 PERCENT ROW WITH TIES;
+
-- should fail
SELECT ''::text AS two, unique1, unique2, stringu1
FROM onek WHERE unique1 > 50
On Mon, Aug 10, 2020 at 01:23:44PM +0300, Surafel Temesgen wrote:
I also Implement PERCENT WITH TIES option. patch is attached
i don't start a new tread because the patches share common code
This fails to apply per the CF bot. Could you send a rebase?
--
Michael
Hi Michael
On Thu, Sep 24, 2020 at 6:58 AM Michael Paquier <michael@paquier.xyz> wrote:
On Mon, Aug 10, 2020 at 01:23:44PM +0300, Surafel Temesgen wrote:
I also Implement PERCENT WITH TIES option. patch is attached
i don't start a new tread because the patches share common codeThis fails to apply per the CF bot. Could you send a rebase?
--
Attaches are rebased patches
regards
Surafel
Attachments:
percent-incremental-v11.patchtext/x-patch; charset=US-ASCII; name=percent-incremental-v11.patchDownload
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index a31abce7c9..135b98da5d 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -3096,7 +3096,8 @@ estimate_path_cost_size(PlannerInfo *root,
if (fpextra && fpextra->has_limit)
{
adjust_limit_rows_costs(&rows, &startup_cost, &total_cost,
- fpextra->offset_est, fpextra->count_est);
+ LIMIT_OPTION_COUNT, fpextra->offset_est,
+ fpextra->count_est);
retrieved_rows = rows;
}
}
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index b93e4ca208..9811380ec6 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -44,7 +44,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="parameter">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="parameter">start</replaceable> [ ROW | ROWS ] ]
- [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } { ONLY | WITH TIES } ]
+ [ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } { ONLY | WITH TIES } ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF <replaceable class="parameter">table_name</replaceable> [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
<phrase>where <replaceable class="parameter">from_item</replaceable> can be one of:</phrase>
@@ -1434,7 +1434,7 @@ OFFSET <replaceable class="parameter">start</replaceable>
which <productname>PostgreSQL</productname> also supports. It is:
<synopsis>
OFFSET <replaceable class="parameter">start</replaceable> { ROW | ROWS }
-FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] { ROW | ROWS } { ONLY | WITH TIES }
+FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] [ PERCENT ] { ROW | ROWS } { ONLY | WITH TIES }
</synopsis>
In this syntax, the <replaceable class="parameter">start</replaceable>
or <replaceable class="parameter">count</replaceable> value is required by
@@ -1444,6 +1444,9 @@ FETCH { FIRST | NEXT } [ <replaceable class="parameter">count</replaceable> ] {
ambiguity.
If <replaceable class="parameter">count</replaceable> is
omitted in a <literal>FETCH</literal> clause, it defaults to 1.
+ With <literal>PERCENT</literal> count specifies the maximum number of rows to return
+ in percentage round up to the nearest integer. Using <literal>PERCENT</literal>
+ is best suited to returning single-digit percentages of the query's total row count.
The <literal>WITH TIES</literal> option is used to return any additional
rows that tie for the last place in the result set according to
the <literal>ORDER BY</literal> clause; <literal>ORDER BY</literal>
diff --git a/src/backend/catalog/sql_features.txt b/src/backend/catalog/sql_features.txt
index b6e58e8493..68e0c5e2e1 100644
--- a/src/backend/catalog/sql_features.txt
+++ b/src/backend/catalog/sql_features.txt
@@ -344,7 +344,7 @@ F862 <result offset clause> in subqueries YES
F863 Nested <result offset clause> in <query expression> YES
F864 Top-level <result offset clause> in views YES
F865 <offset row count> in <result offset clause> YES
-F866 FETCH FIRST clause: PERCENT option NO
+F866 FETCH FIRST clause: PERCENT option YES
F867 FETCH FIRST clause: WITH TIES option YES
R010 Row pattern recognition: FROM clause NO
R020 Row pattern recognition: WINDOW clause NO
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index d85cf7d93e..cfb28a9fd4 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -21,6 +21,8 @@
#include "postgres.h"
+#include <math.h>
+
#include "executor/executor.h"
#include "executor/nodeLimit.h"
#include "miscadmin.h"
@@ -29,6 +31,8 @@
static void recompute_limits(LimitState *node);
static int64 compute_tuples_needed(LimitState *node);
+#define IsPercentOption(opt) \
+ (opt == LIMIT_OPTION_PERCENT || opt == LIMIT_OPTION_PER_WITH_TIES)
/* ----------------------------------------------------------------
* ExecLimit
@@ -53,6 +57,7 @@ ExecLimit(PlanState *pstate)
*/
direction = node->ps.state->es_direction;
outerPlan = outerPlanState(node);
+ slot = node->subSlot;
/*
* The main logic is a simple state machine.
@@ -82,7 +87,15 @@ ExecLimit(PlanState *pstate)
/*
* Check for empty window; if so, treat like empty subplan.
*/
- if (node->count <= 0 && !node->noCount)
+ if (IsPercentOption(node->limitOption))
+ {
+ if (node->percent == 0.0)
+ {
+ node->lstate = LIMIT_EMPTY;
+ return NULL;
+ }
+ }
+ else if (node->count <= 0 && !node->noCount)
{
node->lstate = LIMIT_EMPTY;
return NULL;
@@ -118,6 +131,16 @@ ExecLimit(PlanState *pstate)
break;
}
+ /*
+ * We may needed this tuple in subsequent scan so put it into
+ * tuplestore.
+ */
+ if (IsPercentOption(node->limitOption))
+ {
+ tuplestore_puttupleslot(node->tupleStore, slot);
+ tuplestore_advance(node->tupleStore, true);
+ }
+
/*
* Okay, we have the first tuple of the window.
*/
@@ -135,6 +158,85 @@ ExecLimit(PlanState *pstate)
case LIMIT_INWINDOW:
if (ScanDirectionIsForward(direction))
{
+ /*
+ * In case of coming back from backward scan the tuple is
+ * already in tuple store.
+ */
+ if (IsPercentOption(node->limitOption) && node->backwardPosition > 0)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ node->backwardPosition--;
+ return slot;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+
+ /*
+ * In LIMIT_OPTION_PERCENT case no need of executing outerPlan
+ * multiple times.
+ */
+ if (IsPercentOption(node->limitOption) && node->reachEnd)
+ {
+ node->lstate = LIMIT_WINDOWEND;
+
+ /*
+ * If we know we won't need to back up, we can release
+ * resources at this point.
+ */
+ if (!(node->ps.state->es_top_eflags & EXEC_FLAG_BACKWARD))
+ (void) ExecShutdownNode(outerPlan);
+
+ return NULL;
+ }
+
+ /*
+ * When in percentage mode, we need to see if we can get any
+ * additional rows from the subplan (enough to increase the
+ * node->count value).
+ */
+ if (IsPercentOption(node->limitOption))
+ {
+ /*
+ * loop until the node->count became greater than the
+ * number of tuple returned so far
+ */
+ do
+ {
+ int64 cnt;
+
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->reachEnd = true;
+ node->lstate = LIMIT_SUBPLANEOF;
+
+ /*
+ * The only operation from here is backward scan
+ * but there's no API to refetch the tuple at the
+ * current position. We have to move one tuple
+ * backward, and then we will scan forward for it
+ * for the first tuple and precede as usual for
+ * the rest
+ */
+ tuplestore_advance(node->tupleStore, false);
+ return NULL;
+ }
+
+ tuplestore_puttupleslot(node->tupleStore, slot);
+
+ cnt = tuplestore_tuple_count(node->tupleStore) + node->offset;
+
+ node->count = ceil(node->percent * cnt / 100.0);
+ } while (node->position - node->offset >= node->count);
+ }
+
/*
* Forwards scan, so check for stepping off end of window. At
* the end of the window, the behavior depends on whether WITH
@@ -152,7 +254,8 @@ ExecLimit(PlanState *pstate)
* whether rescans are possible.
*/
if (!node->noCount &&
- node->position - node->offset >= node->count)
+ node->position - node->offset >= node->count
+ && !IsPercentOption(node->limitOption))
{
if (node->limitOption == LIMIT_OPTION_COUNT)
{
@@ -166,54 +269,88 @@ ExecLimit(PlanState *pstate)
}
}
else
+ {
+ if (IsPercentOption(node->limitOption))
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->position++;
+ break;
+ }
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+
+ }
+ else
+ {
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+
+ /*
+ * If WITH TIES is active, and this is the last
+ * in-window tuple, save it to be used in subsequent
+ * WINDOWEND_TIES processing.
+ */
+ if (node->limitOption == LIMIT_OPTION_WITH_TIES &&
+ node->position - node->offset == node->count - 1)
+ {
+ ExecCopySlot(node->last_slot, slot);
+ }
+ node->subSlot = slot;
+ node->position++;
+ break;
+ }
+ }
+ }
+ else
+ {
+ /*
+ * Backwards scan, so check for stepping off start of window.
+ * As above, only change state-machine status if so.
+ */
+ if (node->position <= node->offset + 1)
+ {
+ node->lstate = LIMIT_WINDOWSTART;
+ return NULL;
+ }
+
+ /* In percent case the result is already in tuplestore */
+ if (IsPercentOption(node->limitOption))
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, false, true, slot))
+ {
+ node->subSlot = slot;
+ node->position--;
+ node->backwardPosition++;
+ break;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else
{
/*
- * Get next tuple from subplan, if any.
+ * Get previous tuple from subplan; there should be one!
*/
slot = ExecProcNode(outerPlan);
if (TupIsNull(slot))
- {
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
- }
-
- /*
- * If WITH TIES is active, and this is the last in-window
- * tuple, save it to be used in subsequent WINDOWEND_TIES
- * processing.
- */
- if (node->limitOption == LIMIT_OPTION_WITH_TIES &&
- node->position - node->offset == node->count - 1)
- {
- ExecCopySlot(node->last_slot, slot);
- }
+ elog(ERROR, "LIMIT subplan failed to run backwards");
node->subSlot = slot;
- node->position++;
+ node->position--;
break;
}
}
- else
- {
- /*
- * Backwards scan, so check for stepping off start of window.
- * As above, only change state-machine status if so.
- */
- if (node->position <= node->offset + 1)
- {
- node->lstate = LIMIT_WINDOWSTART;
- return NULL;
- }
-
- /*
- * Get previous tuple from subplan; there should be one!
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
- break;
- }
Assert(node->lstate == LIMIT_WINDOWEND_TIES);
/* FALL THRU */
@@ -279,15 +416,32 @@ ExecLimit(PlanState *pstate)
return NULL;
/*
- * Backing up from subplan EOF, so re-fetch previous tuple; there
- * should be one! Note previous tuple must be in window.
+ * Scan forward for the first tuple
*/
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->lstate = LIMIT_INWINDOW;
- /* position does not change 'cause we didn't advance it before */
+ if (IsPercentOption(node->limitOption))
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else
+ {
+ /*
+ * Backing up from subplan EOF, so re-fetch previous tuple;
+ * there should be one! Note previous tuple must be in
+ * window.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ /* position does not change 'cause we didn't advance it before */
+ }
break;
case LIMIT_WINDOWEND:
@@ -393,12 +547,36 @@ recompute_limits(LimitState *node)
}
else
{
- node->count = DatumGetInt64(val);
- if (node->count < 0)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
- errmsg("LIMIT must not be negative")));
- node->noCount = false;
+ if (IsPercentOption(node->limitOption))
+ {
+ /*
+ * We expect to return at least one row (unless there are no
+ * rows in the subplan), and we'll update this count later as
+ * we go.
+ */
+ node->count = 0;
+ node->percent = DatumGetFloat8(val);
+
+ if (node->percent < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("PERCENT must not be negative")));
+
+ if (node->percent > 100)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("PERCENT must not be greater than 100")));
+
+ }
+ else
+ {
+ node->count = DatumGetInt64(val);
+ if (node->count < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_ROW_COUNT_IN_LIMIT_CLAUSE),
+ errmsg("LIMIT must not be negative")));
+
+ }
}
}
else
@@ -411,6 +589,8 @@ recompute_limits(LimitState *node)
/* Reset position to start-of-scan */
node->position = 0;
node->subSlot = NULL;
+ node->reachEnd = false;
+ node->backwardPosition = 0;
/* Set state-machine state */
node->lstate = LIMIT_RESCAN;
@@ -419,7 +599,8 @@ recompute_limits(LimitState *node)
* Notify child node about limit. Note: think not to "optimize" by
* skipping ExecSetTupleBound if compute_tuples_needed returns < 0. We
* must update the child node anyway, in case this is a rescan and the
- * previous time we got a different result.
+ * previous time we got a different result. In LIMIT_OPTION_PERCENT option
+ * there are no bound on the number of output tuples
*/
ExecSetTupleBound(compute_tuples_needed(node), outerPlanState(node));
}
@@ -431,7 +612,8 @@ recompute_limits(LimitState *node)
static int64
compute_tuples_needed(LimitState *node)
{
- if ((node->noCount) || (node->limitOption == LIMIT_OPTION_WITH_TIES))
+ if ((node->noCount) || (node->limitOption == LIMIT_OPTION_WITH_TIES)
+ || (IsPercentOption(node->limitOption)))
return -1;
/* Note: if this overflows, we'll return a negative value, which is OK */
return node->count + node->offset;
@@ -521,6 +703,9 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
&limitstate->ps);
}
+ if (IsPercentOption(node->limitOption))
+ limitstate->tupleStore = tuplestore_begin_heap(true, false, work_mem);
+
return limitstate;
}
@@ -536,6 +721,8 @@ ExecEndLimit(LimitState *node)
{
ExecFreeExprContext(&node->ps);
ExecEndNode(outerPlanState(node));
+ if (node->tupleStore != NULL)
+ tuplestore_end(node->tupleStore);
}
@@ -555,4 +742,6 @@ ExecReScanLimit(LimitState *node)
*/
if (node->ps.lefttree->chgParam == NULL)
ExecReScan(node->ps.lefttree);
+ if (node->tupleStore != NULL)
+ tuplestore_clear(node->tupleStore);
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index c1fc866cbf..c4dabb02de 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3665,7 +3665,7 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
adjust_limit_rows_costs(&pathnode->path.rows,
&pathnode->path.startup_cost,
&pathnode->path.total_cost,
- offset_est, count_est);
+ limitOption, offset_est, count_est);
return pathnode;
}
@@ -3690,6 +3690,7 @@ void
adjust_limit_rows_costs(double *rows, /* in/out parameter */
Cost *startup_cost, /* in/out parameter */
Cost *total_cost, /* in/out parameter */
+ LimitOption limitOption,
int64 offset_est,
int64 count_est)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 17653ef3a7..d9535f3764 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -682,7 +682,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
ORDER ORDINALITY OTHERS OUT_P OUTER_P
OVER OVERLAPS OVERLAY OVERRIDING OWNED OWNER
- PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PLACING PLANS POLICY
+ PARALLEL PARSER PARTIAL PARTITION PASSING PASSWORD PERCENT PLACING PLANS POLICY
POSITION PRECEDING PRECISION PRESERVE PREPARE PREPARED PRIMARY
PRIOR PRIVILEGES PROCEDURAL PROCEDURE PROCEDURES PROGRAM PUBLICATION
@@ -11560,6 +11560,14 @@ limit_clause:
n->limitOption = LIMIT_OPTION_COUNT;
$$ = n;
}
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows ONLY
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = NULL;
+ n->limitCount = $3;
+ n->limitOption = LIMIT_OPTION_PERCENT;
+ $$ = n;
+ }
| FETCH first_or_next select_fetch_first_value row_or_rows WITH TIES
{
SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
@@ -15209,6 +15217,7 @@ unreserved_keyword:
| PARTITION
| PASSING
| PASSWORD
+ | PERCENT
| PLANS
| POLICY
| PRECEDING
@@ -15780,6 +15789,7 @@ bare_label_keyword:
| PARTITION
| PASSING
| PASSWORD
+ | PERCENT
| PLACING
| PLANS
| POLICY
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index edcaf276c0..4f84497928 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1754,8 +1754,10 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
-
- qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
+ if (limitOption == LIMIT_OPTION_PERCENT && strcmp(constructName, "LIMIT") == 0)
+ qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
+ else
+ qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
/* LIMIT can't refer to any variables of the current query */
checkExprIsVarFree(pstate, qual, constructName);
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 15877e37a6..aa393d539a 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -5243,7 +5243,15 @@ get_select_query_def(Query *query, deparse_context *context,
-PRETTYINDENT_STD, PRETTYINDENT_STD, 0);
get_rule_expr(query->limitOffset, context, false);
}
- if (query->limitCount != NULL)
+ if (query->limitOption == LIMIT_OPTION_PERCENT)
+ {
+ appendContextKeyword(context, " FETCH FIRST ",
+ -PRETTYINDENT_STD, PRETTYINDENT_STD, 0);
+ get_rule_expr(query->limitCount, context, false);
+ appendContextKeyword(context, " PERCENT ROWS ONLY ",
+ -PRETTYINDENT_STD, PRETTYINDENT_STD, 0);
+ }
+ if (query->limitCount != NULL && query->limitOption != LIMIT_OPTION_PERCENT)
{
if (query->limitOption == LIMIT_OPTION_WITH_TIES)
{
diff --git a/src/backend/utils/sort/tuplestore.c b/src/backend/utils/sort/tuplestore.c
index 452a85a423..8743a0f03d 100644
--- a/src/backend/utils/sort/tuplestore.c
+++ b/src/backend/utils/sort/tuplestore.c
@@ -1100,6 +1100,39 @@ tuplestore_gettupleslot(Tuplestorestate *state, bool forward,
}
}
+/*
+ * tuplestore_gettupleslot_heaptuple
+ * It is similar to tuplestore_gettupleslot except it return stored HeapTuple
+ * instead of MinimalTuple
+ */
+bool
+tuplestore_gettupleslot_heaptuple(Tuplestorestate *state, bool forward,
+ bool copy, TupleTableSlot *slot)
+{
+ MinimalTuple tuple;
+ HeapTuple htuple;
+ bool should_free;
+
+ tuple = (MinimalTuple) tuplestore_gettuple(state, forward, &should_free);
+
+ if (tuple)
+ {
+ if (copy && !should_free)
+ {
+ tuple = heap_copy_minimal_tuple(tuple);
+ should_free = true;
+ }
+ htuple = heap_tuple_from_minimal_tuple(tuple);
+ ExecForceStoreHeapTuple(htuple, slot, should_free);
+ return true;
+ }
+ else
+ {
+ ExecClearTuple(slot);
+ return false;
+ }
+}
+
/*
* tuplestore_advance - exported function to adjust position without fetching
*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index a5ab1aed14..6d81e48f11 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2500,6 +2500,11 @@ typedef struct LimitState
LimitOption limitOption; /* limit specification type */
int64 offset; /* current OFFSET value */
int64 count; /* current COUNT, if any */
+ float8 percent; /* percentage */
+ int64 backwardPosition; /* the number of tuple returned in
+ * backward scan */
+ bool reachEnd; /* if true, outerPlan executed until the end */
+ Tuplestorestate *tupleStore; /* holds the returned tuple */
bool noCount; /* if true, ignore count */
LimitStateCond lstate; /* state machine status, as above */
int64 position; /* 1-based index of last tuple returned */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 381d84b4e4..9253ac734c 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -836,6 +836,8 @@ typedef enum LimitOption
{
LIMIT_OPTION_COUNT, /* FETCH FIRST... ONLY */
LIMIT_OPTION_WITH_TIES, /* FETCH FIRST... WITH TIES */
+ LIMIT_OPTION_PERCENT, /* FETCH FIRST... PERCENT ONLY */
+ LIMIT_OPTION_PER_WITH_TIES, /* FETCH FIRST... PERCENT WITH TIES */
LIMIT_OPTION_DEFAULT, /* No limit present */
} LimitOption;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 715a24ad29..ddc8f74138 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -272,7 +272,7 @@ extern LimitPath *create_limit_path(PlannerInfo *root, RelOptInfo *rel,
int64 offset_est, int64 count_est);
extern void adjust_limit_rows_costs(double *rows,
Cost *startup_cost, Cost *total_cost,
- int64 offset_est, int64 count_est);
+ LimitOption limitOption, int64 offset_est, int64 count_est);
extern Path *reparameterize_path(PlannerInfo *root, Path *path,
Relids required_outer,
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..27cff51c30 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -306,6 +306,7 @@ PG_KEYWORD("partial", PARTIAL, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("partition", PARTITION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("passing", PASSING, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("password", PASSWORD, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("percent", PERCENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("placing", PLACING, RESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("plans", PLANS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("policy", POLICY, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/utils/tuplestore.h b/src/include/utils/tuplestore.h
index 2c6403f9e8..1d59aa0d57 100644
--- a/src/include/utils/tuplestore.h
+++ b/src/include/utils/tuplestore.h
@@ -47,7 +47,6 @@ typedef struct Tuplestorestate Tuplestorestate;
extern Tuplestorestate *tuplestore_begin_heap(bool randomAccess,
bool interXact,
int maxKBytes);
-
extern void tuplestore_set_eflags(Tuplestorestate *state, int eflags);
extern void tuplestore_puttupleslot(Tuplestorestate *state,
@@ -72,9 +71,12 @@ extern bool tuplestore_in_memory(Tuplestorestate *state);
extern bool tuplestore_gettupleslot(Tuplestorestate *state, bool forward,
bool copy, TupleTableSlot *slot);
+extern bool tuplestore_gettupleslot_heaptuple(Tuplestorestate *state, bool forward,
+ bool copy, TupleTableSlot *slot);
extern bool tuplestore_advance(Tuplestorestate *state, bool forward);
+
extern bool tuplestore_skiptuples(Tuplestorestate *state,
int64 ntuples, bool forward);
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index b75afcc01a..57e4bb2b8b 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -108,6 +108,63 @@ SELECT ''::text AS five, unique1, unique2, stringu1
| 904 | 793 | UIAAAA
(5 rows)
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 51 | 76 | ZBAAAA
+ | 52 | 985 | ACAAAA
+ | 53 | 196 | BCAAAA
+ | 54 | 356 | CCAAAA
+ | 55 | 627 | DCAAAA
+ | 56 | 54 | ECAAAA
+ | 57 | 942 | FCAAAA
+ | 58 | 114 | GCAAAA
+ | 59 | 593 | HCAAAA
+ | 60 | 483 | ICAAAA
+(10 rows)
+
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+ two | unique1 | unique2 | stringu1
+-----+---------+---------+----------
+ | 61 | 560 | JCAAAA
+(1 row)
+
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+ three | unique1 | unique2 | stringu1
+-------+---------+---------+----------
+ | 121 | 700 | REAAAA
+ | 122 | 519 | SEAAAA
+ | 123 | 777 | TEAAAA
+ | 124 | 503 | UEAAAA
+ | 125 | 849 | VEAAAA
+ | 126 | 330 | WEAAAA
+ | 127 | 511 | XEAAAA
+ | 128 | 721 | YEAAAA
+ | 129 | 696 | ZEAAAA
+(9 rows)
+
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+ eleven | unique1 | unique2 | stringu1
+--------+---------+---------+----------
+ | 10 | 520 | KAAAAA
+ | 9 | 49 | JAAAAA
+ | 8 | 653 | IAAAAA
+ | 7 | 647 | HAAAAA
+ | 6 | 978 | GAAAAA
+(5 rows)
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -338,6 +395,46 @@ fetch backward all in c5;
123 | 456
(2 rows)
+declare c6 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c6;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
+fetch 1 in c6;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c6;
+ q1 | q2
+------------------+-----
+ 4567890123456789 | 123
+(1 row)
+
+fetch backward all in c6;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+ 123 | 456
+(2 rows)
+
+fetch backward 1 in c6;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c6;
+ q1 | q2
+------------------+------------------
+ 123 | 456
+ 123 | 4567890123456789
+ 4567890123456789 | 123
+(3 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index d2d4ef132d..c6e660913f 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -31,6 +31,23 @@ SELECT ''::text AS five, unique1, unique2, stringu1
FROM onek
ORDER BY unique1 LIMIT 5 OFFSET 900;
+--
+-- PERCENT
+-- Check the PERCENT option of limit clause
+--
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 50
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY;
+SELECT ''::text AS two, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 60 AND unique1 < 63
+ ORDER BY unique1 FETCH FIRST 50 PERCENT ROWS ONLY;
+SELECT ''::text AS three, unique1, unique2, stringu1
+ FROM onek WHERE unique1 > 100
+ ORDER BY unique1 FETCH FIRST 1 PERCENT ROWS ONLY OFFSET 20;
+SELECT ''::text AS eleven, unique1, unique2, stringu1
+ FROM onek WHERE unique1 < 50
+ ORDER BY unique1 DESC FETCH FIRST 10 PERCENT ROWS ONLY OFFSET 39;
+
-- Test null limit and offset. The planner would discard a simple null
-- constant, so to ensure executor is exercised, do this:
select * from int8_tbl limit (case when random() < 0.5 then null::bigint end);
@@ -38,7 +55,6 @@ select * from int8_tbl offset (case when random() < 0.5 then null::bigint end);
-- Test assorted cases involving backwards fetch from a LIMIT plan node
begin;
-
declare c1 cursor for select * from int8_tbl limit 10;
fetch all in c1;
fetch 1 in c1;
@@ -81,6 +97,14 @@ fetch backward all in c5;
fetch all in c5;
fetch backward all in c5;
+declare c6 cursor for select * from int8_tbl fetch first 50 percent rows only;
+fetch all in c6;
+fetch 1 in c6;
+fetch backward 1 in c6;
+fetch backward all in c6;
+fetch backward 1 in c6;
+fetch all in c6;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
percent_with_ties_v2.patchtext/x-patch; charset=US-ASCII; name=percent_with_ties_v2.patchDownload
diff --git a/src/backend/executor/nodeLimit.c b/src/backend/executor/nodeLimit.c
index cfb28a9fd4..47e71a221a 100644
--- a/src/backend/executor/nodeLimit.c
+++ b/src/backend/executor/nodeLimit.c
@@ -193,6 +193,12 @@ ExecLimit(PlanState *pstate)
if (!(node->ps.state->es_top_eflags & EXEC_FLAG_BACKWARD))
(void) ExecShutdownNode(outerPlan);
+ /*
+ * The only operation from here is backward scan We have
+ * to move one postion forward to get previous tuple
+ */
+ tuplestore_advance(node->tupleStore, true);
+
return NULL;
}
@@ -215,26 +221,47 @@ ExecLimit(PlanState *pstate)
if (TupIsNull(slot))
{
node->reachEnd = true;
- node->lstate = LIMIT_SUBPLANEOF;
+ if (node->limitOption == LIMIT_OPTION_PER_WITH_TIES)
+ {
+ slot = node->subSlot;
+ tuplestore_advance(node->tupleStore, false);
+ if (!tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ tuplestore_advance(node->tupleStore, true);
+ return NULL;
+ }
- /*
- * The only operation from here is backward scan
- * but there's no API to refetch the tuple at the
- * current position. We have to move one tuple
- * backward, and then we will scan forward for it
- * for the first tuple and precede as usual for
- * the rest
- */
- tuplestore_advance(node->tupleStore, false);
- return NULL;
- }
+ ExecCopySlot(node->last_slot, slot);
+ node->lstate = LIMIT_WINDOWEND_TIES;
+ /* we'll fall through to the next case */
+ }
+ else
+ {
+ node->reachEnd = true;
+ node->lstate = LIMIT_SUBPLANEOF;
- tuplestore_puttupleslot(node->tupleStore, slot);
+ /*
+ * The only operation from here is backward
+ * scan but there's no API to refetch the
+ * tuple at the current position. We have to
+ * move one postion backward, and then we will
+ * scan forward for it for the first tuple and
+ * precede as usual for the rest
+ */
+ tuplestore_advance(node->tupleStore, true);
+ return NULL;
+ }
+ }
+ if (node->lstate != LIMIT_WINDOWEND_TIES)
+ {
+ tuplestore_puttupleslot(node->tupleStore, slot);
- cnt = tuplestore_tuple_count(node->tupleStore) + node->offset;
+ cnt = tuplestore_tuple_count(node->tupleStore) + node->offset;
- node->count = ceil(node->percent * cnt / 100.0);
- } while (node->position - node->offset >= node->count);
+ node->count = ceil(node->percent * cnt / 100.0);
+ }
+ } while (node->position - node->offset >= node->count && node->lstate != LIMIT_WINDOWEND_TIES);
}
/*
@@ -255,7 +282,7 @@ ExecLimit(PlanState *pstate)
*/
if (!node->noCount &&
node->position - node->offset >= node->count
- && !IsPercentOption(node->limitOption))
+ && !IsPercentOption(node->limitOption) && node->lstate != LIMIT_WINDOWEND_TIES)
{
if (node->limitOption == LIMIT_OPTION_COUNT)
{
@@ -268,49 +295,45 @@ ExecLimit(PlanState *pstate)
/* we'll fall through to the next case */
}
}
- else
+ else if (IsPercentOption(node->limitOption) && node->lstate != LIMIT_WINDOWEND_TIES)
{
- if (IsPercentOption(node->limitOption))
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
{
- if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
- {
- node->subSlot = slot;
- node->position++;
- break;
- }
- else
- {
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
- }
-
- }
- else
- {
- /*
- * Get next tuple from subplan, if any.
- */
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- {
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
- }
-
- /*
- * If WITH TIES is active, and this is the last
- * in-window tuple, save it to be used in subsequent
- * WINDOWEND_TIES processing.
- */
- if (node->limitOption == LIMIT_OPTION_WITH_TIES &&
- node->position - node->offset == node->count - 1)
- {
- ExecCopySlot(node->last_slot, slot);
- }
node->subSlot = slot;
node->position++;
break;
}
+ else
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+ else if (!IsPercentOption(node->limitOption) && node->lstate != LIMIT_WINDOWEND_TIES)
+ {
+ /*
+ * Get next tuple from subplan, if any.
+ */
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+
+ /*
+ * If WITH TIES is active, and this is the last in-window
+ * tuple, save it to be used in subsequent WINDOWEND_TIES
+ * processing.
+ */
+ if (node->limitOption == LIMIT_OPTION_WITH_TIES &&
+ node->position - node->offset == node->count - 1)
+ {
+ ExecCopySlot(node->last_slot, slot);
+ }
+ node->subSlot = slot;
+ node->position++;
+ break;
}
}
else
@@ -359,14 +382,25 @@ ExecLimit(PlanState *pstate)
if (ScanDirectionIsForward(direction))
{
/*
- * Advance the subplan until we find the first row with
- * different ORDER BY pathkeys.
+ * Advance the subplan or tuple store until we find the first
+ * row with different ORDER BY pathkeys.
*/
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
+ if (node->limitOption == LIMIT_OPTION_PER_WITH_TIES)
{
- node->lstate = LIMIT_SUBPLANEOF;
- return NULL;
+ if (!tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
+ }
+ else
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ {
+ node->lstate = LIMIT_SUBPLANEOF;
+ return NULL;
+ }
}
/*
@@ -399,15 +433,30 @@ ExecLimit(PlanState *pstate)
}
/*
- * Get previous tuple from subplan; there should be one! And
- * change state-machine status.
+ * Get previous tuple from subplan or tuple store; there
+ * should be one! And change state-machine status.
*/
- slot = ExecProcNode(outerPlan);
- if (TupIsNull(slot))
- elog(ERROR, "LIMIT subplan failed to run backwards");
- node->subSlot = slot;
- node->position--;
- node->lstate = LIMIT_INWINDOW;
+ if (node->limitOption == LIMIT_OPTION_PER_WITH_TIES)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, false, true, slot))
+ {
+ node->backwardPosition++;
+ node->position--;
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
+ else
+ {
+ slot = ExecProcNode(outerPlan);
+ if (TupIsNull(slot))
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ node->subSlot = slot;
+ node->position--;
+ node->lstate = LIMIT_INWINDOW;
+ }
}
break;
@@ -416,11 +465,12 @@ ExecLimit(PlanState *pstate)
return NULL;
/*
- * Scan forward for the first tuple
+ * Scan forward for the previous tuple. there should be one! Note
+ * previous tuple must be in window.
*/
if (IsPercentOption(node->limitOption))
{
- if (tuplestore_gettupleslot_heaptuple(node->tupleStore, true, true, slot))
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, false, true, slot))
{
node->subSlot = slot;
node->lstate = LIMIT_INWINDOW;
@@ -461,6 +511,16 @@ ExecLimit(PlanState *pstate)
node->subSlot = slot;
node->lstate = LIMIT_INWINDOW;
}
+ if (node->limitOption == LIMIT_OPTION_PER_WITH_TIES)
+ {
+ if (tuplestore_gettupleslot_heaptuple(node->tupleStore, false, true, slot))
+ {
+ node->subSlot = slot;
+ node->lstate = LIMIT_INWINDOW;
+ }
+ else
+ elog(ERROR, "LIMIT subplan failed to run backwards");
+ }
else
{
/*
@@ -686,7 +746,8 @@ ExecInitLimit(Limit *node, EState *estate, int eflags)
/*
* Initialize the equality evaluation, to detect ties.
*/
- if (node->limitOption == LIMIT_OPTION_WITH_TIES)
+ if (node->limitOption == LIMIT_OPTION_WITH_TIES
+ || node->limitOption == LIMIT_OPTION_PER_WITH_TIES)
{
TupleDesc desc;
const TupleTableSlotOps *ops;
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 99278eed93..c4a81dd2ad 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2701,7 +2701,8 @@ create_limit_plan(PlannerInfo *root, LimitPath *best_path, int flags)
subplan = create_plan_recurse(root, best_path->subpath, flags);
/* Extract information necessary for comparing rows for WITH TIES. */
- if (best_path->limitOption == LIMIT_OPTION_WITH_TIES)
+ if (best_path->limitOption == LIMIT_OPTION_WITH_TIES ||
+ best_path->limitOption == LIMIT_OPTION_PER_WITH_TIES)
{
Query *parse = root->parse;
ListCell *l;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index d9535f3764..94c105400c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -11576,6 +11576,14 @@ limit_clause:
n->limitOption = LIMIT_OPTION_WITH_TIES;
$$ = n;
}
+ | FETCH first_or_next select_fetch_first_value PERCENT row_or_rows WITH TIES
+ {
+ SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
+ n->limitOffset = NULL;
+ n->limitCount = $3;
+ n->limitOption = LIMIT_OPTION_PER_WITH_TIES;
+ $$ = n;
+ }
| FETCH first_or_next row_or_rows ONLY
{
SelectLimit *n = (SelectLimit *) palloc(sizeof(SelectLimit));
@@ -16352,7 +16360,8 @@ insertSelectOptions(SelectStmt *stmt,
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("multiple limit options not allowed")));
- if (!stmt->sortClause && limitClause->limitOption == LIMIT_OPTION_WITH_TIES)
+ if (!stmt->sortClause && (limitClause->limitOption == LIMIT_OPTION_WITH_TIES
+ || limitClause->limitOption == LIMIT_OPTION_PER_WITH_TIES))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("WITH TIES cannot be specified without ORDER BY clause")));
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 4f84497928..3cbbad987d 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1754,7 +1754,9 @@ transformLimitClause(ParseState *pstate, Node *clause,
return NULL;
qual = transformExpr(pstate, clause, exprKind);
- if (limitOption == LIMIT_OPTION_PERCENT && strcmp(constructName, "LIMIT") == 0)
+ if ((limitOption == LIMIT_OPTION_PERCENT || limitOption == LIMIT_OPTION_PER_WITH_TIES)
+ && strcmp(constructName, "LIMIT") == 0)
+
qual = coerce_to_specific_type(pstate, qual, FLOAT8OID, constructName);
else
qual = coerce_to_specific_type(pstate, qual, INT8OID, constructName);
diff --git a/src/test/regress/expected/limit.out b/src/test/regress/expected/limit.out
index 57e4bb2b8b..4b6d3f10ed 100644
--- a/src/test/regress/expected/limit.out
+++ b/src/test/regress/expected/limit.out
@@ -435,6 +435,43 @@ fetch all in c6;
4567890123456789 | 123
(3 rows)
+declare c7 cursor for select * from int8_tbl order by q1 fetch first 15 percent rows with ties;
+fetch all in c7;
+ q1 | q2
+-----+------------------
+ 123 | 456
+ 123 | 4567890123456789
+(2 rows)
+
+fetch 1 in c7;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch backward 1 in c7;
+ q1 | q2
+-----+------------------
+ 123 | 4567890123456789
+(1 row)
+
+fetch backward all in c7;
+ q1 | q2
+-----+-----
+ 123 | 456
+(1 row)
+
+fetch backward 1 in c7;
+ q1 | q2
+----+----
+(0 rows)
+
+fetch all in c7;
+ q1 | q2
+-----+------------------
+ 123 | 456
+ 123 | 4567890123456789
+(2 rows)
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
SELECT
@@ -716,6 +753,80 @@ SELECT thousand
0
(2 rows)
+--
+-- FETCH FIRST
+-- Check the PERCENT WITH TIES clause
+--
+SELECT thousand
+ FROM onek WHERE thousand < 5
+ ORDER BY thousand FETCH FIRST 2 PERCENT ROW ONLY;
+ thousand
+----------
+ 0
+(1 row)
+
+SELECT thousand
+ FROM onek WHERE thousand < 5
+ ORDER BY thousand FETCH FIRST 2 PERCENT ROWS WITH TIES;
+ thousand
+----------
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+(10 rows)
+
+SELECT thousand
+ FROM onek WHERE thousand < 5
+ ORDER BY thousand FETCH FIRST 21 PERCENT ROW ONLY;
+ thousand
+----------
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 1
+(11 rows)
+
+SELECT thousand
+ FROM onek WHERE thousand < 5
+ ORDER BY thousand FETCH FIRST 21 PERCENT ROW WITH TIES;
+ thousand
+----------
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 0
+ 1
+ 1
+ 1
+ 1
+ 1
+ 1
+ 1
+ 1
+ 1
+ 1
+(20 rows)
+
-- should fail
SELECT ''::text AS two, unique1, unique2, stringu1
FROM onek WHERE unique1 > 50
diff --git a/src/test/regress/sql/limit.sql b/src/test/regress/sql/limit.sql
index c6e660913f..f9e3d24910 100644
--- a/src/test/regress/sql/limit.sql
+++ b/src/test/regress/sql/limit.sql
@@ -105,6 +105,14 @@ fetch backward all in c6;
fetch backward 1 in c6;
fetch all in c6;
+declare c7 cursor for select * from int8_tbl order by q1 fetch first 15 percent rows with ties;
+fetch all in c7;
+fetch 1 in c7;
+fetch backward 1 in c7;
+fetch backward all in c7;
+fetch backward 1 in c7;
+fetch all in c7;
+
rollback;
-- Stress test for variable LIMIT in conjunction with bounded-heap sorting
@@ -197,6 +205,27 @@ SELECT thousand
FROM onek WHERE thousand < 5
ORDER BY thousand FETCH FIRST 2 ROW ONLY;
+--
+-- FETCH FIRST
+-- Check the PERCENT WITH TIES clause
+--
+
+SELECT thousand
+ FROM onek WHERE thousand < 5
+ ORDER BY thousand FETCH FIRST 2 PERCENT ROW ONLY;
+
+SELECT thousand
+ FROM onek WHERE thousand < 5
+ ORDER BY thousand FETCH FIRST 2 PERCENT ROWS WITH TIES;
+
+SELECT thousand
+ FROM onek WHERE thousand < 5
+ ORDER BY thousand FETCH FIRST 21 PERCENT ROW ONLY;
+
+SELECT thousand
+ FROM onek WHERE thousand < 5
+ ORDER BY thousand FETCH FIRST 21 PERCENT ROW WITH TIES;
+
-- should fail
SELECT ''::text AS two, unique1, unique2, stringu1
FROM onek WHERE unique1 > 50
Sorry for the dealy. I started to look this.
At Fri, 25 Sep 2020 12:25:24 +0300, Surafel Temesgen <surafel3000@gmail.com> wrote in
Hi Michael
On Thu, Sep 24, 2020 at 6:58 AM Michael Paquier <michael@paquier.xyz> wrote:On Mon, Aug 10, 2020 at 01:23:44PM +0300, Surafel Temesgen wrote:
I also Implement PERCENT WITH TIES option. patch is attached
i don't start a new tread because the patches share common codeThis fails to apply per the CF bot. Could you send a rebase?
This still applies on the master HEAD.
percent-incremental-v11.patch
The existing nodeLimit passes the slot of the subnode to the
caller. but this patch changes that behavior. You added a new function
to tuplestore.c not to store a minimal tuple into the slot that passed
from subnode, but we should refrain from scribbling on the slot passed
from the subnode. Instead, the PERCENT path of the limit node should
use its own ResultTupleSlot for the purpose. See nodeSort for a
concrete example.
+ LIMIT_OPTION_PER_WITH_TIES, /* FETCH FIRST... PERCENT WITH TIES */
That name is a bit hard to read. We should spell it with complete
words.
case LIMIT_INWINDOW:
...
+ if (IsPercentOption(node->limitOption) && node->backwardPosition
...
+ if (IsPercentOption(node->limitOption) && node->reachEnd)
...
+ if (IsPercentOption(node->limitOption))
I think we can use separate lstate state for each condition above
since IsPercentOption() gives a constant result through the execution
time. For example, LIMIT_PERCENT_TUPLESLOT_NOT_FILLED and
LIMIT_PERCENT_TUPLESLOT_FILLED and some derived states similar to the
non-percent path. I *feel* that makes code simpler.
What do you think about this?
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
On Mon, Jan 25, 2021 at 6:39 AM Kyotaro Horiguchi
<horikyota.ntt@gmail.com> wrote:
Sorry for the dealy. I started to look this.
At Fri, 25 Sep 2020 12:25:24 +0300, Surafel Temesgen <surafel3000@gmail.com> wrote in
Hi Michael
On Thu, Sep 24, 2020 at 6:58 AM Michael Paquier <michael@paquier.xyz> wrote:On Mon, Aug 10, 2020 at 01:23:44PM +0300, Surafel Temesgen wrote:
I also Implement PERCENT WITH TIES option. patch is attached
i don't start a new tread because the patches share common codeThis fails to apply per the CF bot. Could you send a rebase?
This still applies on the master HEAD.
percent-incremental-v11.patch
The existing nodeLimit passes the slot of the subnode to the
caller. but this patch changes that behavior. You added a new function
to tuplestore.c not to store a minimal tuple into the slot that passed
from subnode, but we should refrain from scribbling on the slot passed
from the subnode. Instead, the PERCENT path of the limit node should
use its own ResultTupleSlot for the purpose. See nodeSort for a
concrete example.+ LIMIT_OPTION_PER_WITH_TIES, /* FETCH FIRST... PERCENT WITH TIES */
That name is a bit hard to read. We should spell it with complete
words.case LIMIT_INWINDOW: ... + if (IsPercentOption(node->limitOption) && node->backwardPosition ... + if (IsPercentOption(node->limitOption) && node->reachEnd) ... + if (IsPercentOption(node->limitOption))I think we can use separate lstate state for each condition above
since IsPercentOption() gives a constant result through the execution
time. For example, LIMIT_PERCENT_TUPLESLOT_NOT_FILLED and
LIMIT_PERCENT_TUPLESLOT_FILLED and some derived states similar to the
non-percent path. I *feel* that makes code simpler.What do you think about this?
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
Hi,
I was testing this and found that it doesn't work well with
subselects, this query make the server crash:
"""
select * from pg_type where exists (select 1 from pg_authid fetch
first 10 percent rows only);
"""
postgres was compiled with these options:
"""
CFLAGS="-ggdb -O0 -g3 -fno-omit-frame-pointer" ./configure
--prefix=/opt/var/pgdg/14dev/percent --enable-debug --enable-cassert
--with-pgport=54314 --enable-depend
"""
attached is the stack trace
--
Jaime Casanova
Director de Servicios Profesionales
SystemGuards - Consultores de PostgreSQL
Attachments:
On Mon, Jan 25, 2021 at 2:39 PM Kyotaro Horiguchi <horikyota.ntt@gmail.com>
wrote:
Sorry for the dealy. I started to look this.
Hi kyotaro,
Thanks for looking into this but did we agree to proceed
on this approach? I fear that it will be the west of effort if
Andrew comes up with the patch for his approach.
Andrew Gierth: Are you working on your approach?
regards
Surafel
On 1/26/21 11:29 AM, Surafel Temesgen wrote:
On Mon, Jan 25, 2021 at 2:39 PM Kyotaro Horiguchi
<horikyota.ntt@gmail.com <mailto:horikyota.ntt@gmail.com>> wrote:Sorry for the dealy. I started to look this.
Hi kyotaro,
Thanks for looking into this but did we agree to proceed
on this approach? I fear that it will be the west of effort if
Andrew comes up with the patch for his approach.
Andrew Gierth: Are you working on your approach?
[Added Andrew to recipients]
Andrew, would you care to comment?
Regards,
--
-David
david@pgmasters.net
On Thu, Mar 25, 2021 at 11:15:10AM -0400, David Steele wrote:
On 1/26/21 11:29 AM, Surafel Temesgen wrote:
On Mon, Jan 25, 2021 at 2:39 PM Kyotaro Horiguchi
<horikyota.ntt@gmail.com <mailto:horikyota.ntt@gmail.com>> wrote:Sorry for the dealy. I started to look this.
Hi kyotaro,
Thanks for looking into this but did we agree to proceed
on this approach? I fear that it will be the west of effort if
Andrew comes up with the patch for his approach.
Andrew Gierth: Are you working on your approach?[Added Andrew to recipients]
Andrew, would you care to comment?
Hi everyone,
This is still marked as needs review, but I think it should be marked
as "Returned with feedback" or "waiting on author".
suggestions?
--
Jaime Casanova
Director de Servicios Profesionales
SystemGuards - Consultores de PostgreSQL
On Tue, Jan 26, 2021 at 07:29:11PM +0300, Surafel Temesgen wrote:
On Mon, Jan 25, 2021 at 2:39 PM Kyotaro Horiguchi <horikyota.ntt@gmail.com>
wrote:Sorry for the dealy. I started to look this.
Hi kyotaro,
Thanks for looking into this but did we agree to proceed
on this approach? I fear that it will be the west of effort if
Andrew comes up with the patch for his approach.
Andrew Gierth: Are you working on your approach?
Hi Surafel,
I'm marking this one as "returned with feedback". Then you or Andrew can
submit a new version of the patch for the next CF.
--
Jaime Casanova
Director de Servicios Profesionales
SystemGuards - Consultores de PostgreSQL