pgsql: Support XMLTABLE query expression

Started by Alvaro Herreraover 9 years ago5 messagescomitters
Jump to latest
#1Alvaro Herrera
alvherre@2ndquadrant.com

Support XMLTABLE query expression

XMLTABLE is defined by the SQL/XML standard as a feature that allows
turning XML-formatted data into relational form, so that it can be used
as a <table primary> in the FROM clause of a query.

This new construct provides significant simplicity and performance
benefit for XML data processing; what in a client-side custom
implementation was reported to take 20 minutes can be executed in 400ms
using XMLTABLE. (The same functionality was said to take 10 seconds
using nested PostgreSQL XPath function calls, and 5 seconds using
XMLReader under PL/Python).

The implemented syntax deviates slightly from what the standard
requires. First, the standard indicates that the PASSING clause is
optional and that multiple XML input documents may be given to it; we
make it mandatory and accept a single document only. Second, we don't
currently support a default namespace to be specified.

This implementation relies on a new executor node based on a hardcoded
method table. (Because the grammar is fixed, there is no extensibility
in the current approach; further constructs can be implemented on top of
this such as JSON_TABLE, but they require changes to core code.)

Author: Pavel Stehule, Álvaro Herrera
Extensively reviewed by: Craig Ringer
Discussion: /messages/by-id/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/fcec6caafa2346b6c9d3ad5065e417733bd63cd9

Modified Files
--------------
contrib/pg_stat_statements/pg_stat_statements.c | 12 +
doc/src/sgml/func.sgml | 203 ++++++++-
src/backend/commands/explain.c | 24 +
src/backend/executor/Makefile | 3 +-
src/backend/executor/execAmi.c | 6 +
src/backend/executor/execProcnode.c | 14 +
src/backend/executor/nodeTableFuncscan.c | 502 +++++++++++++++++++++
src/backend/nodes/copyfuncs.c | 91 ++++
src/backend/nodes/equalfuncs.c | 60 +++
src/backend/nodes/makefuncs.c | 8 +-
src/backend/nodes/nodeFuncs.c | 66 +++
src/backend/nodes/outfuncs.c | 73 ++++
src/backend/nodes/print.c | 4 +
src/backend/nodes/readfuncs.c | 47 ++
src/backend/optimizer/path/allpaths.c | 37 ++
src/backend/optimizer/path/costsize.c | 81 ++++
src/backend/optimizer/plan/createplan.c | 77 +++-
src/backend/optimizer/plan/initsplan.c | 2 +
src/backend/optimizer/plan/planner.c | 37 +-
src/backend/optimizer/plan/setrefs.c | 14 +
src/backend/optimizer/plan/subselect.c | 6 +
src/backend/optimizer/prep/prepjointree.c | 6 +
src/backend/optimizer/util/pathnode.c | 26 ++
src/backend/optimizer/util/plancat.c | 6 +-
src/backend/optimizer/util/relnode.c | 5 +-
src/backend/parser/analyze.c | 9 +
src/backend/parser/gram.y | 189 +++++++-
src/backend/parser/parse_clause.c | 245 ++++++++++-
src/backend/parser/parse_coerce.c | 33 +-
src/backend/parser/parse_relation.c | 79 +++-
src/backend/parser/parse_target.c | 7 +
src/backend/rewrite/rewriteHandler.c | 4 +
src/backend/utils/adt/ruleutils.c | 133 +++++-
src/backend/utils/adt/xml.c | 558 +++++++++++++++++++++++-
src/include/catalog/catversion.h | 2 +-
src/include/executor/nodeTableFuncscan.h | 24 +
src/include/executor/tablefunc.h | 67 +++
src/include/nodes/execnodes.h | 25 ++
src/include/nodes/nodes.h | 5 +
src/include/nodes/parsenodes.h | 39 ++
src/include/nodes/plannodes.h | 10 +
src/include/nodes/primnodes.h | 22 +
src/include/optimizer/cost.h | 5 +
src/include/optimizer/pathnode.h | 4 +
src/include/parser/kwlist.h | 3 +
src/include/parser/parse_coerce.h | 4 +
src/include/parser/parse_relation.h | 5 +
src/include/utils/xml.h | 3 +
src/test/regress/expected/xml.out | 504 +++++++++++++++++++++
src/test/regress/expected/xml_1.out | 475 ++++++++++++++++++++
src/test/regress/expected/xml_2.out | 504 +++++++++++++++++++++
src/test/regress/sql/xml.sql | 288 ++++++++++++
52 files changed, 4606 insertions(+), 50 deletions(-)

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alvaro Herrera (#1)
Re: pgsql: Support XMLTABLE query expression

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

Support XMLTABLE query expression

This fails immediately on RHEL6:

$ ./configure --with-libxml [ other usual arguments ]
$ make -s
xml.c: In function 'XmlTableGetValue':
xml.c:4434: warning: implicit declaration of function 'xmlXPathSetContextNode'
utils/adt/xml.o: In function `XmlTableGetValue':
/home/postgres/pgsql/src/backend/utils/adt/xml.c:4434: undefined reference to `xmlXPathSetContextNode'
collect2: ld returned 1 exit status
make[2]: *** [postgres] Error 1
make[1]: *** [all-backend-recurse] Error 2
make: *** [all-src-recurse] Error 2

Relevant package versions are

libxml2-2.7.6-21.el6_8.1.x86_64
libxml2-devel-2.7.6-21.el6_8.1.x86_64
xml-common-0.6.3-33.el6.noarch

regards, tom lane

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

#3Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Tom Lane (#2)
Re: pgsql: Support XMLTABLE query expression

Tom Lane wrote:

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

Support XMLTABLE query expression

This fails immediately on RHEL6:

Yeah, i noticed on termite as well. Apparently xmlXPathSetContextNode
appeared in 2.9.1 only, which was released in 2013. :-(

The easiest for now is to disable XMLTABLE completely with older
libxml2.

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

#4Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alvaro Herrera (#3)
Re: pgsql: Support XMLTABLE query expression

Alvaro Herrera <alvherre@2ndquadrant.com> writes:

Tom Lane wrote:

This fails immediately on RHEL6:

Yeah, i noticed on termite as well. Apparently xmlXPathSetContextNode
appeared in 2.9.1 only, which was released in 2013. :-(

The easiest for now is to disable XMLTABLE completely with older
libxml2.

That does not sound like an acceptable solution. These aren't
obsolete platforms we're talking about.

regards, tom lane

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

#5Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Tom Lane (#4)
Re: pgsql: Support XMLTABLE query expression

Tom Lane wrote:

Alvaro Herrera <alvherre@2ndquadrant.com> writes:

Tom Lane wrote:

This fails immediately on RHEL6:

Yeah, i noticed on termite as well. Apparently xmlXPathSetContextNode
appeared in 2.9.1 only, which was released in 2013. :-(

The easiest for now is to disable XMLTABLE completely with older
libxml2.

That does not sound like an acceptable solution. These aren't
obsolete platforms we're talking about.

Apparently, it's a one line change to support that. I'm testing it now.

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers