On How To Shorten the Steep Learning Curve Towards PG Hacking...
Dear PG Hackers/Experts,
I am newbie to PG Hacking.
I have been reading the PG code base to find my space in it but without
success.
There are hundreds of Hands-on with PG Application development on the web.
Alas, there is almost none in PG hacking.
I have found PG source Code reading and hacking to be one the most
frustrating experiences in my life. I believe that PG hacking should not
be a painful
Dear PG Hacker/Experts,
I am newbie to PG Hacking.
I have been reading the PG code base to find my space in it but without
success.
There are hundreds of Hands-on with PG Application development on the web.
Alas, there is almost none in PG hacking.
I have found PG source Code reading and hacking to be one the most
frustrating experiences in my life. I believe that PG hacking should not
be a painful journey but an enjoyable one!
It is my strong believe that out of my PG hacking frustrations, there may
come insights for the PG experts on ways how to devise hands-on with PG
internals so that new comers will be great coders as quickly as possible.
I also believe that we should spend our time reading great Papers and Books
related to Data Management problems BUT not PG code base.
Here are my suggestion for the experts to devise ways to shorten the steep
learning curve towards PG Hacking.
1. Prepare Hands-on with PG internals
For example, a complete Hands-on with SELECT/INSERT SQL Standard PG
internals. The point is the experts can pick one fairly complex feature and
walk it from Parser to Executor in a hands-on manner explaining step by
step every technical detail.
2. Write a book on PG Internals.
There is one book on PG internals. Unfortunately, it's in Chinese.
Why not in English??
It is my strong believe that if there were a great book on PG Internals
with hands-on with some of the basic features of PG internals machinery, PG
hacking would be almost as easy as PG application development.
If the experts make the newbie understand the PG code base as quickly as
possible, that means more reviewers, more contributors and more users of PG
which in turn means more PG usability, more PG popularity, stronger PG
community.
This is my personal feelings and am the ready to be corrected and advised
the right way towards the PG hacking.
Regards,
Zeray
On Mon, Mar 27, 2017 at 9:00 PM, Kang Yuzhe <tiggreen87@gmail.com> wrote:
1. Prepare Hands-on with PG internals
For example, a complete Hands-on with SELECT/INSERT SQL Standard PG
internals. The point is the experts can pick one fairly complex feature and
walk it from Parser to Executor in a hands-on manner explaining step by step
every technical detail.
There are resources on the net, in English as well. Take for example
this manual explaining the internals of Postgres by Hironobu Suzuki:
http://www.interdb.jp/pg/
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
From: pgsql-hackers-owner@postgresql.org
[mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Kang Yuzhe
1. Prepare Hands-on with PG internals
For example, a complete Hands-on with SELECT/INSERT SQL Standard PG
internals. The point is the experts can pick one fairly complex feature
and walk it from Parser to Executor in a hands-on manner explaining step
by step every technical detail.
I sympathize with you. What level of detail do you have in mind? The query processing framework is described in the manual:
Chapter 50. Overview of PostgreSQL Internals
https://www.postgresql.org/docs/devel/static/overview.html
More detailed source code analysis is provided for very old PostgreSQL 7.4, but I guess it's not much different now. The document is in Japanese only:
http://ikubo.x0.com/PostgreSQL/pg_source.htm
Are you thinking of something like this?
MySQL Internals Manual
https://dev.mysql.com/doc/internals/en/
Regards
Takayuki Tsunakawa
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Thanks Tsunakawa for such an informative reply.
Almost all of the docs related to the internals of PG are of introductory
concepts only.
There is even more useful PG internals site entitled "The Internals of
PostgreSQL" in http://www.interdb.jp/pg/ translation of the Japanese PG
Internals.
The query processing framework that is described in the manual as you
mentioned is of informative and introductory nature.
In theory, the query processing framework described in the manual is
understandable.
Unfortunate, it is another story to understand how query processing
framework in PG codebase really works.
It has become a difficult task for me to walk through the PG source code
for example how SELECT/INSERT/TRUNCATE in the the different modules under
"src/..". really works.
I wish there were Hands-On with PostgreSQL Internals like
https://bkmjournal.wordpress.com/2017/01/22/hands-on-with-postgresql-internals/
for more complex PG features.
For example, MERGE SQL standard is not supported yet by PG. I wish there
were Hands-On with PostgreSQL Internals for MERGE/UPSERT. How it is
implemented in parser/executor/storage etc. modules with detailed
explanation for each code and debugging and other important concepts
related to system programming.
Zeray,
Regards
On Tue, Mar 28, 2017 at 6:04 AM, Tsunakawa, Takayuki <
tsunakawa.takay@jp.fujitsu.com> wrote:
Show quoted text
From: pgsql-hackers-owner@postgresql.org
[mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Kang Yuzhe
1. Prepare Hands-on with PG internals
For example, a complete Hands-on with SELECT/INSERT SQL Standard PG
internals. The point is the experts can pick one fairly complex feature
and walk it from Parser to Executor in a hands-on manner explaining step
by step every technical detail.I sympathize with you. What level of detail do you have in mind? The
query processing framework is described in the manual:Chapter 50. Overview of PostgreSQL Internals
https://www.postgresql.org/docs/devel/static/overview.htmlMore detailed source code analysis is provided for very old PostgreSQL
7.4, but I guess it's not much different now. The document is in Japanese
only:http://ikubo.x0.com/PostgreSQL/pg_source.htm
Are you thinking of something like this?
MySQL Internals Manual
https://dev.mysql.com/doc/internals/en/Regards
Takayuki Tsunakawa
On 03/27/2017 02:00 PM, Kang Yuzhe wrote:
1. Prepare Hands-on with PG internals
For example, a complete Hands-on with SELECT/INSERT SQL Standard PG internals.
The point is the experts can pick one fairly complex feature and walk it from
Parser to Executor in a hands-on manner explaining step by step every technical
detail.
Hi,
Bruce Momjian has made several presentations about Postgres Internal :
http://momjian.us/main/presentations/internals.html
Regards
--
Adrien NAYRAT
Hi,
On 2017/03/28 15:40, Kang Yuzhe wrote:
Thanks Tsunakawa for such an informative reply.
Almost all of the docs related to the internals of PG are of introductory
concepts only.
There is even more useful PG internals site entitled "The Internals of
PostgreSQL" in http://www.interdb.jp/pg/ translation of the Japanese PG
Internals.The query processing framework that is described in the manual as you
mentioned is of informative and introductory nature.
In theory, the query processing framework described in the manual is
understandable.Unfortunate, it is another story to understand how query processing
framework in PG codebase really works.
It has become a difficult task for me to walk through the PG source code
for example how SELECT/INSERT/TRUNCATE in the the different modules under
"src/..". really works.I wish there were Hands-On with PostgreSQL Internals like
https://bkmjournal.wordpress.com/2017/01/22/hands-on-with-postgresql-internals/
for more complex PG features.For example, MERGE SQL standard is not supported yet by PG. I wish there
were Hands-On with PostgreSQL Internals for MERGE/UPSERT. How it is
implemented in parser/executor/storage etc. modules with detailed
explanation for each code and debugging and other important concepts
related to system programming.
I am not sure if I can show you that one place where you could learn all
of that, but many people who started with PostgreSQL development at some
point started by exploring the source code itself (either for learning or
to write a feature patch), articles on PostgreSQL wiki, and many related
presentations accessible using the Internet. I liked the following among
many others:
Introduction to Hacking PostgreSQL:
http://www.neilconway.org/talks/hacking/
Inside the PostgreSQL Query Optimizer:
http://www.neilconway.org/talks/optimizer/optimizer.pdf
Postgres Internals Presentations:
http://momjian.us/main/presentations/internals.html
Thanks,
Amit
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 29 March 2017 at 10:53, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:
Hi,
On 2017/03/28 15:40, Kang Yuzhe wrote:
Thanks Tsunakawa for such an informative reply.
Almost all of the docs related to the internals of PG are of introductory
concepts only.
There is even more useful PG internals site entitled "The Internals of
PostgreSQL" in http://www.interdb.jp/pg/ translation of the Japanese PG
Internals.The query processing framework that is described in the manual as you
mentioned is of informative and introductory nature.
In theory, the query processing framework described in the manual is
understandable.Unfortunate, it is another story to understand how query processing
framework in PG codebase really works.
It has become a difficult task for me to walk through the PG source code
for example how SELECT/INSERT/TRUNCATE in the the different modules under
"src/..". really works.I wish there were Hands-On with PostgreSQL Internals like
https://bkmjournal.wordpress.com/2017/01/22/hands-on-with-postgresql-internals/
for more complex PG features.For example, MERGE SQL standard is not supported yet by PG. I wish there
were Hands-On with PostgreSQL Internals for MERGE/UPSERT. How it is
implemented in parser/executor/storage etc. modules with detailed
explanation for each code and debugging and other important concepts
related to system programming.I am not sure if I can show you that one place where you could learn all
of that, but many people who started with PostgreSQL development at some
point started by exploring the source code itself (either for learning or
to write a feature patch), articles on PostgreSQL wiki, and many related
presentations accessible using the Internet. I liked the following among
many others:
Personally I have to agree that the learning curve is very steep. Some
of the docs and presentations help, but there's a LOT to understand.
When you're getting started you're lost in a world of language you
don't know, and trying to understand one piece often gets you lost in
other pieces. In no particular order:
* Memory contexts and palloc
* Managing transactions and how that interacts with memory contexts
and the default memory context
* Snapshots, snapshot push/pop, etc
* LWLocks, memory barriers, spinlocks, latches
* Heavyweight locks (and the different APIs to them)
* GUCs, their scopes, the rules around their callbacks, etc
* dynahash
* catalogs and oids and access methods
* The heap AM like heap_open
* relcache, catcache, syscache
* genam and the systable_ calls and their limitations with indexes
* The SPI
* When to use each of the above 4!
* Heap tuples and minimal tuples
* VARLENA
* GETSTRUCT, when you can/can't use it, other attribute fetching methods
* TOAST and detoasting datums.
* forming and deforming tuples
* LSNs, WAL/xlog generation and redo. Timelines. (ARGH, timelines).
* cache invalidations, when they can happen, and how to do anything
safely around them.
* TIDs, cmin and cmax, xmin and xmax
* postmaster, vacuum, bgwriter, checkpointer, startup process,
walsender, walreceiver, all our auxillary procs and what they do
* relmapper, relfilenodes vs relation oids, filenode extents
* ondisk structure, page headers, pages
* shmem management, buffers and buffer pins
* bgworkers
* PG_TRY() and PG_CATCH() and their limitations
* elog and ereport and errcontexts, exception unwinding/longjmp and
how it interacts with memory contexts, lwlocks, etc
* The nest of macros around datum manipulation and functions, PL
handlers. How to find the macros for the data types you want to work
with.
* Everything to do with the C API for arrays (is horrible)
* The details of the parse/rewrite/plan phases with rewrite calling
back into parse, paths, the mess with inheritance_planner, reading and
understanding plantrees
* The permissions and grants model and how to interact with it
* PGPROC, PGXACT, other main shmem structures
* Resource owners (which I still don't fully "get")
* Checkpoints, pg_control and ShmemVariableCache, crash recovery
* How globals are used in Pg and how they interact with fork()ing from
postmaster
* SSI (haven't gone there yet myself)
* ....
Personally I recall finding the magic of resource owner and memory
context changing under me when I started/stopped xacts in a bgworker,
along with the need to manage snapshots and SPI state to be distinctly
confusing.
There are various READMEs, blog posts, presentation slides/videos, etc
that explain bits and pieces. But not much exists to tie it together
into a comprehensible hole with simple, minimal explanations for each
part so someone who's new to it all can begin to get a handle on it,
find resources to learn more about subsystems they need to care about,
etc.
Lots of it boils down to "read the code". But so much code! You don't
know if what you're reading is really relevant or if it's even
correct, or if it makes assumptions that differ from your situation.
There are lots of coding rules that aren't necessarily obvious unless
you read the right place, e.g. that you don't need to and shouldn't
LWLockRelease() before elog(ERROR). That SPI doesn't manage snapshots
or xacts for you (but will often silently work anyway!). etc.
I've long intended to start a blog series on postgresql innards
concepts, partly with the intent of turning it into such an overview.
I find that people are better at shouting you down when you're wrong
than they are at writing new material or reviewing proposed docs, so
it's often a good way to fact-check things ;) . Plus it's a good way
to learn. Time is always short though.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Thanks you all for pointing me to useful docs on PG kernel stuff as well as
for being sympathetic with me and the newbie question that appears to be
true and interesting but yet be addressed by PG experts.
Last but not least, *Craig Ringer*, you just nailed it!! You also made me
feel and think that my question is working asking.
Regards,
Zeray
On Wed, Mar 29, 2017 at 6:36 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
Show quoted text
On 29 March 2017 at 10:53, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>
wrote:Hi,
On 2017/03/28 15:40, Kang Yuzhe wrote:
Thanks Tsunakawa for such an informative reply.
Almost all of the docs related to the internals of PG are of
introductory
concepts only.
There is even more useful PG internals site entitled "The Internals of
PostgreSQL" in http://www.interdb.jp/pg/ translation of the Japanese PG
Internals.The query processing framework that is described in the manual as you
mentioned is of informative and introductory nature.
In theory, the query processing framework described in the manual is
understandable.Unfortunate, it is another story to understand how query processing
framework in PG codebase really works.
It has become a difficult task for me to walk through the PG source code
for example how SELECT/INSERT/TRUNCATE in the the different modulesunder
"src/..". really works.
I wish there were Hands-On with PostgreSQL Internals like
https://bkmjournal.wordpress.com/2017/01/22/hands-on-with-postgresql-internals/
for more complex PG features.
For example, MERGE SQL standard is not supported yet by PG. I wish
there
were Hands-On with PostgreSQL Internals for MERGE/UPSERT. How it is
implemented in parser/executor/storage etc. modules with detailed
explanation for each code and debugging and other important concepts
related to system programming.I am not sure if I can show you that one place where you could learn all
of that, but many people who started with PostgreSQL development at some
point started by exploring the source code itself (either for learning or
to write a feature patch), articles on PostgreSQL wiki, and many related
presentations accessible using the Internet. I liked the following among
many others:Personally I have to agree that the learning curve is very steep. Some
of the docs and presentations help, but there's a LOT to understand.When you're getting started you're lost in a world of language you
don't know, and trying to understand one piece often gets you lost in
other pieces. In no particular order:* Memory contexts and palloc
* Managing transactions and how that interacts with memory contexts
and the default memory context
* Snapshots, snapshot push/pop, etc
* LWLocks, memory barriers, spinlocks, latches
* Heavyweight locks (and the different APIs to them)
* GUCs, their scopes, the rules around their callbacks, etc
* dynahash
* catalogs and oids and access methods
* The heap AM like heap_open
* relcache, catcache, syscache
* genam and the systable_ calls and their limitations with indexes
* The SPI
* When to use each of the above 4!
* Heap tuples and minimal tuples
* VARLENA
* GETSTRUCT, when you can/can't use it, other attribute fetching methods
* TOAST and detoasting datums.
* forming and deforming tuples
* LSNs, WAL/xlog generation and redo. Timelines. (ARGH, timelines).
* cache invalidations, when they can happen, and how to do anything
safely around them.
* TIDs, cmin and cmax, xmin and xmax
* postmaster, vacuum, bgwriter, checkpointer, startup process,
walsender, walreceiver, all our auxillary procs and what they do
* relmapper, relfilenodes vs relation oids, filenode extents
* ondisk structure, page headers, pages
* shmem management, buffers and buffer pins
* bgworkers
* PG_TRY() and PG_CATCH() and their limitations
* elog and ereport and errcontexts, exception unwinding/longjmp and
how it interacts with memory contexts, lwlocks, etc
* The nest of macros around datum manipulation and functions, PL
handlers. How to find the macros for the data types you want to work
with.
* Everything to do with the C API for arrays (is horrible)
* The details of the parse/rewrite/plan phases with rewrite calling
back into parse, paths, the mess with inheritance_planner, reading and
understanding plantrees
* The permissions and grants model and how to interact with it
* PGPROC, PGXACT, other main shmem structures
* Resource owners (which I still don't fully "get")
* Checkpoints, pg_control and ShmemVariableCache, crash recovery
* How globals are used in Pg and how they interact with fork()ing from
postmaster
* SSI (haven't gone there yet myself)
* ....Personally I recall finding the magic of resource owner and memory
context changing under me when I started/stopped xacts in a bgworker,
along with the need to manage snapshots and SPI state to be distinctly
confusing.There are various READMEs, blog posts, presentation slides/videos, etc
that explain bits and pieces. But not much exists to tie it together
into a comprehensible hole with simple, minimal explanations for each
part so someone who's new to it all can begin to get a handle on it,
find resources to learn more about subsystems they need to care about,
etc.Lots of it boils down to "read the code". But so much code! You don't
know if what you're reading is really relevant or if it's even
correct, or if it makes assumptions that differ from your situation.
There are lots of coding rules that aren't necessarily obvious unless
you read the right place, e.g. that you don't need to and shouldn't
LWLockRelease() before elog(ERROR). That SPI doesn't manage snapshots
or xacts for you (but will often silently work anyway!). etc.I've long intended to start a blog series on postgresql innards
concepts, partly with the intent of turning it into such an overview.
I find that people are better at shouting you down when you're wrong
than they are at writing new material or reviewing proposed docs, so
it's often a good way to fact-check things ;) . Plus it's a good way
to learn. Time is always short though.--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On 2017/03/29 12:36, Craig Ringer wrote:
On 29 March 2017 at 10:53, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:
Hi,
On 2017/03/28 15:40, Kang Yuzhe wrote:
Thanks Tsunakawa for such an informative reply.
Almost all of the docs related to the internals of PG are of introductory
concepts only.
There is even more useful PG internals site entitled "The Internals of
PostgreSQL" in http://www.interdb.jp/pg/ translation of the Japanese PG
Internals.The query processing framework that is described in the manual as you
mentioned is of informative and introductory nature.
In theory, the query processing framework described in the manual is
understandable.Unfortunate, it is another story to understand how query processing
framework in PG codebase really works.
It has become a difficult task for me to walk through the PG source code
for example how SELECT/INSERT/TRUNCATE in the the different modules under
"src/..". really works.I wish there were Hands-On with PostgreSQL Internals like
https://bkmjournal.wordpress.com/2017/01/22/hands-on-with-postgresql-internals/
for more complex PG features.For example, MERGE SQL standard is not supported yet by PG. I wish there
were Hands-On with PostgreSQL Internals for MERGE/UPSERT. How it is
implemented in parser/executor/storage etc. modules with detailed
explanation for each code and debugging and other important concepts
related to system programming.I am not sure if I can show you that one place where you could learn all
of that, but many people who started with PostgreSQL development at some
point started by exploring the source code itself (either for learning or
to write a feature patch), articles on PostgreSQL wiki, and many related
presentations accessible using the Internet. I liked the following among
many others:Personally I have to agree that the learning curve is very steep. Some
of the docs and presentations help, but there's a LOT to understand.
I agree too. :)
When you're getting started you're lost in a world of language you
don't know, and trying to understand one piece often gets you lost in
other pieces. In no particular order:* Memory contexts and palloc
* Managing transactions and how that interacts with memory contexts
and the default memory context
* Snapshots, snapshot push/pop, etc
* LWLocks, memory barriers, spinlocks, latches
* Heavyweight locks (and the different APIs to them)
* GUCs, their scopes, the rules around their callbacks, etc
* dynahash
* catalogs and oids and access methods
* The heap AM like heap_open
* relcache, catcache, syscache
* genam and the systable_ calls and their limitations with indexes
* The SPI
* When to use each of the above 4!
* Heap tuples and minimal tuples
* VARLENA
* GETSTRUCT, when you can/can't use it, other attribute fetching methods
* TOAST and detoasting datums.
* forming and deforming tuples
* LSNs, WAL/xlog generation and redo. Timelines. (ARGH, timelines).
* cache invalidations, when they can happen, and how to do anything
safely around them.
* TIDs, cmin and cmax, xmin and xmax
* postmaster, vacuum, bgwriter, checkpointer, startup process,
walsender, walreceiver, all our auxillary procs and what they do
* relmapper, relfilenodes vs relation oids, filenode extents
* ondisk structure, page headers, pages
* shmem management, buffers and buffer pins
* bgworkers
* PG_TRY() and PG_CATCH() and their limitations
* elog and ereport and errcontexts, exception unwinding/longjmp and
how it interacts with memory contexts, lwlocks, etc
* The nest of macros around datum manipulation and functions, PL
handlers. How to find the macros for the data types you want to work
with.
* Everything to do with the C API for arrays (is horrible)
* The details of the parse/rewrite/plan phases with rewrite calling
back into parse, paths, the mess with inheritance_planner, reading and
understanding plantrees
* The permissions and grants model and how to interact with it
* PGPROC, PGXACT, other main shmem structures
* Resource owners (which I still don't fully "get")
* Checkpoints, pg_control and ShmemVariableCache, crash recovery
* How globals are used in Pg and how they interact with fork()ing from
postmaster
* SSI (haven't gone there yet myself)
* ....
That is indeed a big list of things to know and (have to) worry about. If
we indeed come up with a PG-hackers-handbook someday, things in your list
could be organized such that it's clear to someone wanting to contribute
code which of those things they need to *absolutely* worry about and which
they don't.
Personally I recall finding the magic of resource owner and memory
context changing under me when I started/stopped xacts in a bgworker,
along with the need to manage snapshots and SPI state to be distinctly
confusing.There are various READMEs, blog posts, presentation slides/videos, etc
that explain bits and pieces. But not much exists to tie it together
into a comprehensible hole with simple, minimal explanations for each
part so someone who's new to it all can begin to get a handle on it,
find resources to learn more about subsystems they need to care about,
etc.Lots of it boils down to "read the code". But so much code! You don't
know if what you're reading is really relevant or if it's even
correct, or if it makes assumptions that differ from your situation.
There are lots of coding rules that aren't necessarily obvious unless
you read the right place, e.g. that you don't need to and shouldn't
LWLockRelease() before elog(ERROR). That SPI doesn't manage snapshots
or xacts for you (but will often silently work anyway!). etc.I've long intended to start a blog series on postgresql innards
concepts, partly with the intent of turning it into such an overview.
I find that people are better at shouting you down when you're wrong
than they are at writing new material or reviewing proposed docs, so
it's often a good way to fact-check things ;) . Plus it's a good way
to learn. Time is always short though.
Agreed on all counts. Look forward to the blog. :)
Thanks,
Amit
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Thanks Amit for further confirmation on the Craig's intention.
I am looking forward to seeing your "PG internal machinery under
microscope" blog. May health, persistence and courage be with YOU.
Regards,
Zeray
On Wed, Mar 29, 2017 at 10:36 AM, Amit Langote <
Langote_Amit_f8@lab.ntt.co.jp> wrote:
Show quoted text
On 2017/03/29 12:36, Craig Ringer wrote:
On 29 March 2017 at 10:53, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>
wrote:
Hi,
On 2017/03/28 15:40, Kang Yuzhe wrote:
Thanks Tsunakawa for such an informative reply.
Almost all of the docs related to the internals of PG are of
introductory
concepts only.
There is even more useful PG internals site entitled "The Internals of
PostgreSQL" in http://www.interdb.jp/pg/ translation of the JapanesePG
Internals.
The query processing framework that is described in the manual as you
mentioned is of informative and introductory nature.
In theory, the query processing framework described in the manual is
understandable.Unfortunate, it is another story to understand how query processing
framework in PG codebase really works.
It has become a difficult task for me to walk through the PG sourcecode
for example how SELECT/INSERT/TRUNCATE in the the different modules
under
"src/..". really works.
I wish there were Hands-On with PostgreSQL Internals like
https://bkmjournal.wordpress.com/2017/01/22/hands-on-with-postgresql-internals/
for more complex PG features.
For example, MERGE SQL standard is not supported yet by PG. I wish
there
were Hands-On with PostgreSQL Internals for MERGE/UPSERT. How it is
implemented in parser/executor/storage etc. modules with detailed
explanation for each code and debugging and other important concepts
related to system programming.I am not sure if I can show you that one place where you could learn all
of that, but many people who started with PostgreSQL development at some
point started by exploring the source code itself (either for learningor
to write a feature patch), articles on PostgreSQL wiki, and many related
presentations accessible using the Internet. I liked the following among
many others:Personally I have to agree that the learning curve is very steep. Some
of the docs and presentations help, but there's a LOT to understand.I agree too. :)
When you're getting started you're lost in a world of language you
don't know, and trying to understand one piece often gets you lost in
other pieces. In no particular order:* Memory contexts and palloc
* Managing transactions and how that interacts with memory contexts
and the default memory context
* Snapshots, snapshot push/pop, etc
* LWLocks, memory barriers, spinlocks, latches
* Heavyweight locks (and the different APIs to them)
* GUCs, their scopes, the rules around their callbacks, etc
* dynahash
* catalogs and oids and access methods
* The heap AM like heap_open
* relcache, catcache, syscache
* genam and the systable_ calls and their limitations with indexes
* The SPI
* When to use each of the above 4!
* Heap tuples and minimal tuples
* VARLENA
* GETSTRUCT, when you can/can't use it, other attribute fetching methods
* TOAST and detoasting datums.
* forming and deforming tuples
* LSNs, WAL/xlog generation and redo. Timelines. (ARGH, timelines).
* cache invalidations, when they can happen, and how to do anything
safely around them.
* TIDs, cmin and cmax, xmin and xmax
* postmaster, vacuum, bgwriter, checkpointer, startup process,
walsender, walreceiver, all our auxillary procs and what they do
* relmapper, relfilenodes vs relation oids, filenode extents
* ondisk structure, page headers, pages
* shmem management, buffers and buffer pins
* bgworkers
* PG_TRY() and PG_CATCH() and their limitations
* elog and ereport and errcontexts, exception unwinding/longjmp and
how it interacts with memory contexts, lwlocks, etc
* The nest of macros around datum manipulation and functions, PL
handlers. How to find the macros for the data types you want to work
with.
* Everything to do with the C API for arrays (is horrible)
* The details of the parse/rewrite/plan phases with rewrite calling
back into parse, paths, the mess with inheritance_planner, reading and
understanding plantrees
* The permissions and grants model and how to interact with it
* PGPROC, PGXACT, other main shmem structures
* Resource owners (which I still don't fully "get")
* Checkpoints, pg_control and ShmemVariableCache, crash recovery
* How globals are used in Pg and how they interact with fork()ing from
postmaster
* SSI (haven't gone there yet myself)
* ....That is indeed a big list of things to know and (have to) worry about. If
we indeed come up with a PG-hackers-handbook someday, things in your list
could be organized such that it's clear to someone wanting to contribute
code which of those things they need to *absolutely* worry about and which
they don't.Personally I recall finding the magic of resource owner and memory
context changing under me when I started/stopped xacts in a bgworker,
along with the need to manage snapshots and SPI state to be distinctly
confusing.There are various READMEs, blog posts, presentation slides/videos, etc
that explain bits and pieces. But not much exists to tie it together
into a comprehensible hole with simple, minimal explanations for each
part so someone who's new to it all can begin to get a handle on it,
find resources to learn more about subsystems they need to care about,
etc.Lots of it boils down to "read the code". But so much code! You don't
know if what you're reading is really relevant or if it's even
correct, or if it makes assumptions that differ from your situation.
There are lots of coding rules that aren't necessarily obvious unless
you read the right place, e.g. that you don't need to and shouldn't
LWLockRelease() before elog(ERROR). That SPI doesn't manage snapshots
or xacts for you (but will often silently work anyway!). etc.I've long intended to start a blog series on postgresql innards
concepts, partly with the intent of turning it into such an overview.
I find that people are better at shouting you down when you're wrong
than they are at writing new material or reviewing proposed docs, so
it's often a good way to fact-check things ;) . Plus it's a good way
to learn. Time is always short though.Agreed on all counts. Look forward to the blog. :)
Thanks,
Amit
On Tue, Mar 28, 2017 at 10:36 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
Personally I have to agree that the learning curve is very steep. Some
of the docs and presentations help, but there's a LOT to understand.
Some small patches can be kept to a fairly narrow set of areas, and
if you can find a similar capability to can crib technique for
handling some of the more mysterious areas it might brush up
against. When I started working on my first *big* patch that was
bound to touch many areas (around the start of development for 9.1)
I counted lines of code and found over a million lines just in .c
and .h files. We're now closing in on 1.5 million lines. That's
not counting over 376,000 lines of documentation in .sgml files,
over 12,000 lines of text in README* files, over 26,000 lines of
perl code, over 103,000 lines of .sql code (60% of which is in
regression tests), over 38,000 lines of .y code (for flex/bison
parsing), about 9,000 lines of various type of code just for
generating the configure file, and over 439,000 lines of .po files
(for message translations). I'm sure I missed a lot of important
stuff there, but it gives some idea the challenge it is to get your
head around it all.
My first advice is to try to identify which areas of the code you
will need to touch, and read those over. Several times. Try to
infer the API to areas *that* code needs to reference from looking
at other code (as similar to what you want to work on as you can
find), reading code comments and README files, and asking
questions. Secondly, there is a lot that is considered to be
"coding rules" that is, as far as I've been able to tell, only
contained inside the heads of veteran PostgreSQL coders, with
occasional references in the discussion list archives. Asking
questions, proposing approaches before coding, and showing work in
progress early and often will help a lot in terms of discovering
these issues and allowing you to rearrange things to fit these
conventions. If someone with the "gift of gab" is able to capture
these and put them into a readily available form, that would be
fantastic.
* SSI (haven't gone there yet myself)
For anyone wanting to approach this area, there is a fair amount to
look at. There is some overlap, but in rough order of "practical"
to "theoretical foundation", you might want to look at:
https://www.postgresql.org/docs/current/static/transaction-iso.html
https://wiki.postgresql.org/wiki/SSI
The SQL standard
http://www.vldb.org/pvldb/vol5.html
http://hdl.handle.net/2123/5353
Papers cited in these last two. I have found papers authored by
Alan Fekete or Adul Adya particularly enlightening.
If any of the other areas that Craig listed have similar work
available, maybe we should start a Wiki page where we list areas of
code (starting with the list Craig included) as section headers, and
put links to useful reading below each?
--
Kevin Grittner
VMware vCenter Server
https://www.vmware.com/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 27 March 2017 at 13:00, Kang Yuzhe <tiggreen87@gmail.com> wrote:
I have found PG source Code reading and hacking to be one the most
frustrating experiences in my life. I believe that PG hacking should not be
a painful journey but an enjoyable one!It is my strong believe that out of my PG hacking frustrations, there may
come insights for the PG experts on ways how to devise hands-on with PG
internals so that new comers will be great coders as quickly as possible.
I'm here now because PostgreSQL has clear, well designed and
maintained code with accurate docs, great comments and a helpful team.
I'd love to see detailed cases where another project is better in a
measurable way; I am willing to learn from that.
Any journey to expertise takes 10,000 hours. There is no way to shorten that.
What aspect of your journey caused you pain?
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Craig Ringer wrote:
Personally I have to agree that the learning curve is very steep. Some
of the docs and presentations help, but there's a LOT to understand.
There is a wiki page "Developer_FAQ" which is supposed to help answer
these questions. It is currently not very useful, because people
stopped adding to it very early and is now mostly unmaintained, but
I'm sure it could become a very useful central resource for this kind of
information.
--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Thanks Alvaro for taking your time and pointing me to "Developer_FAQ". I
knew this web page and there is good stuff int it.
The most important about "Developer_FAQ" which I believe is that it lists
vital books for PG developers.
Comparing the real challenge I am facing in finding my way in the rabbit
role(the PG source code), "Developer_FAQ" is indeed less useful.
Of course, I am a beginner and I am just beginning and one day I hope with
your support I will figure out to find my space in PG development.
My question is why is that there is a lot of hands-on about PG application
development(eg. connecting to PG using JAVA/JDBC) but almost nothing about
PG hacking hands-on lessons. For example, I wanna add the keyword
"Encrypted" in CREATE TABLE t1(a int, b int encrypted) or CREATE TABLE t1(a
int, b int) encrypted. Alas, its not easy task.
Regards,
Zeray
On Mon, Apr 17, 2017 at 8:29 PM, Alvaro Herrera <alvherre@2ndquadrant.com>
wrote:
Show quoted text
Craig Ringer wrote:
Personally I have to agree that the learning curve is very steep. Some
of the docs and presentations help, but there's a LOT to understand.There is a wiki page "Developer_FAQ" which is supposed to help answer
these questions. It is currently not very useful, because people
stopped adding to it very early and is now mostly unmaintained, but
I'm sure it could become a very useful central resource for this kind of
information.--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Thanks Kevin for taking your time and justifying the real difficult of
finding ones space/way in PG development.And thanks for your genuine advice
which I have taken it AS IS.
My question is why is that there is a lot of hands-on about PG application
development(eg. connecting to PG using JAVA/JDBC) but almost nothing about
PG hacking hands-on lessons. For example, I wanna add the keyword
"Encrypted" in "CREATE TABLE t1(a int, b int encrypted)" or "CREATE TABLE
t1(a int, b int) encrypted". Alas, its not easy task.
Lastly, I have come to understand that PG community is not harsh to newbies
and thus, I am feeling at home.
Regards,
Zeray
On Mon, Apr 17, 2017 at 6:53 PM, Kevin Grittner <kgrittn@gmail.com> wrote:
Show quoted text
On Tue, Mar 28, 2017 at 10:36 PM, Craig Ringer <craig@2ndquadrant.com>
wrote:Personally I have to agree that the learning curve is very steep. Some
of the docs and presentations help, but there's a LOT to understand.Some small patches can be kept to a fairly narrow set of areas, and
if you can find a similar capability to can crib technique for
handling some of the more mysterious areas it might brush up
against. When I started working on my first *big* patch that was
bound to touch many areas (around the start of development for 9.1)
I counted lines of code and found over a million lines just in .c
and .h files. We're now closing in on 1.5 million lines. That's
not counting over 376,000 lines of documentation in .sgml files,
over 12,000 lines of text in README* files, over 26,000 lines of
perl code, over 103,000 lines of .sql code (60% of which is in
regression tests), over 38,000 lines of .y code (for flex/bison
parsing), about 9,000 lines of various type of code just for
generating the configure file, and over 439,000 lines of .po files
(for message translations). I'm sure I missed a lot of important
stuff there, but it gives some idea the challenge it is to get your
head around it all.My first advice is to try to identify which areas of the code you
will need to touch, and read those over. Several times. Try to
infer the API to areas *that* code needs to reference from looking
at other code (as similar to what you want to work on as you can
find), reading code comments and README files, and asking
questions. Secondly, there is a lot that is considered to be
"coding rules" that is, as far as I've been able to tell, only
contained inside the heads of veteran PostgreSQL coders, with
occasional references in the discussion list archives. Asking
questions, proposing approaches before coding, and showing work in
progress early and often will help a lot in terms of discovering
these issues and allowing you to rearrange things to fit these
conventions. If someone with the "gift of gab" is able to capture
these and put them into a readily available form, that would be
fantastic.* SSI (haven't gone there yet myself)
For anyone wanting to approach this area, there is a fair amount to
look at. There is some overlap, but in rough order of "practical"
to "theoretical foundation", you might want to look at:https://www.postgresql.org/docs/current/static/transaction-iso.html
https://wiki.postgresql.org/wiki/SSI
The SQL standard
https://git.postgresql.org/gitweb/?p=postgresql.git;a=
blob_plain;f=src/backend/storage/lmgr/README-SSI;hb=refs/heads/masterhttp://www.vldb.org/pvldb/vol5.html
http://hdl.handle.net/2123/5353
Papers cited in these last two. I have found papers authored by
Alan Fekete or Adul Adya particularly enlightening.If any of the other areas that Craig listed have similar work
available, maybe we should start a Wiki page where we list areas of
code (starting with the list Craig included) as section headers, and
put links to useful reading below each?--
Kevin Grittner
VMware vCenter Server
https://www.vmware.com/
On 2017/04/18 15:31, Kang Yuzhe wrote:
My question is why is that there is a lot of hands-on about PG application
development(eg. connecting to PG using JAVA/JDBC) but almost nothing about
PG hacking hands-on lessons. For example, I wanna add the keyword
"Encrypted" in CREATE TABLE t1(a int, b int encrypted) or CREATE TABLE t1(a
int, b int) encrypted. Alas, its not easy task.
Regarding this part, at one of the links shared above [1]http://www.neilconway.org/talks/hacking/, you can find
presentations with hands-on instructions about how to implement a new SQL
functionality by modifying various parts of the source code. See these:
Implementing a TABLESAMPLE clause (by Neil Conway)
http://www.neilconway.org/talks/hacking/ottawa/ottawa_slides.pdf
Add support for the WHEN clause to the CREATE TRIGGER statement (by Neil
Conway)
http://www.neilconway.org/talks/hacking/hack_slides.pdf
(by Gavin Sherry)
https://linux.org.au/conf/2007/att_data/Miniconfs(2f)PostgreSQL/attachments/hacking_intro.pdf
Handout: The Implementation of TABLESAMPLE
http://www.neilconway.org/talks/hacking/ottawa/ottawa_handout.pdf
Handout: Adding WHEN clause to CREATE TRIGGER
http://www.neilconway.org/talks/hacking/hack_handout.pdf
Some of the details might be dated, because they were written more than 10
years ago, but will definitely get you motivated to dive more into the
source code.
Thanks,
Amit
[1]: http://www.neilconway.org/talks/hacking/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Thanks Simon for taking your time and trying to tell and warn me the harsh
reality truth:there is no shortcut to expertise. One has to fail and rise
towards any journey to expertise.
Overall, you are right. But I do believe that there is a way(some
techniques) to speed up any journey to expertise. One of them is
mentorship. For example(just an example), If you show me how to design and
implement FDW to Hadoop/HBase., I believe that I will manage to design and
implement FDW to Cassandra/MengoDB.
The paths towards any journey to expertise by working alone/the hard way
and working with you using as a mentorship are completely different. I
believe that we humans have to power to imitate and get innovative
afterwords.
There are many books on PG business application development:
1.
*PostgreSQL Essential Reference/Barry Stinson2. *PostgreSQL : introduction
and concepts / Momjian,
Bruce.
3. PostgreSQL Cookbook/Over 90 hands-on recipes to effectively manage,
administer, and design solutions using PostgreSQL
4.PostgreSQL Developer's Handbook
5.PostgreSQL 9.0 High Performance
6.PostgreSQL Server Programming
7.PostgreSQL for Data Architects/Discover how to design, develop, and
maintain your
database application effectively with PostgreSQL
8.Practical PostgreSQL
9.Practical SQL Handbook, The: Using SQL Variants, Fourth Edition
10.PostgreSQL: The comprehensive guide to building, programming, and
administering PostgreSQL databases, Second Edition
11.Beginning Databases with PostgreSQL From Novice to Professional, Second
Edition
12.PostgreSQL Succinctly
13.PostgreSQL Up and Running
....
But almost nothing about The Internals of PostgreSQL:
1. The Internals of PostgreSQL:
http://www.interdb.jp/pg/index.html translated from Japanese Book
2. PostgreSQL数据库内核分析(Chinese) Book on the Internals of PostgreSQL:
3. PG Docs/site
4. some here and there which are less useful
Lastly, I have come to understand that PG community is not
harsh/intimidating to newbies and thus, I am feeling at home.
Regards,
Zeray
On Mon, Apr 17, 2017 at 7:33 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
Show quoted text
On 27 March 2017 at 13:00, Kang Yuzhe <tiggreen87@gmail.com> wrote:
I have found PG source Code reading and hacking to be one the most
frustrating experiences in my life. I believe that PG hacking shouldnot be
a painful journey but an enjoyable one!
It is my strong believe that out of my PG hacking frustrations, there may
come insights for the PG experts on ways how to devise hands-on with PG
internals so that new comers will be great coders as quickly as possible.I'm here now because PostgreSQL has clear, well designed and
maintained code with accurate docs, great comments and a helpful team.I'd love to see detailed cases where another project is better in a
measurable way; I am willing to learn from that.Any journey to expertise takes 10,000 hours. There is no way to shorten
that.What aspect of your journey caused you pain?
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 18 April 2017 at 01:29, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
Craig Ringer wrote:
Personally I have to agree that the learning curve is very steep. Some
of the docs and presentations help, but there's a LOT to understand.There is a wiki page "Developer_FAQ" which is supposed to help answer
these questions. It is currently not very useful, because people
stopped adding to it very early and is now mostly unmaintained, but
I'm sure it could become a very useful central resource for this kind of
information.
I add to it when I think of things.
But it'll become an unmaintainable useless morass if random things are
just indiscriminately added. Something more structured is needed to
cover subsystems, coding rules ("don't LWLockRelease() before
ereport(ERROR, ...)"), etc.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 18 April 2017 at 15:41, Kang Yuzhe <tiggreen87@gmail.com> wrote:
Thanks Simon for taking your time and trying to tell and warn me the harsh
reality truth:there is no shortcut to expertise. One has to fail and rise
towards any journey to expertise.
Yeah, just because Pg is hard doesn't mean it's notably bad or worse
than other things. I generally find working on code in other projects,
even smaller and simpler ones, a rather unpleasant change.
That doesn't mean we can't do things to help interested new people get
and stay engaged and grow into productive devs to grow our pool.
Overall, you are right. But I do believe that there is a way(some
techniques) to speed up any journey to expertise. One of them is mentorship.
For example(just an example), If you show me how to design and implement FDW
to Hadoop/HBase., I believe that I will manage to design and implement FDW
to Cassandra/MengoDB.
TBH, that's the sort of thing where looking at existing examples is
often the best way forward and will stay that way.
What I'd like to do is make it easier to understand those examples by
providing background and overview info on subsystems, so you can read
the code and have more idea what it does and why.
But almost nothing about The Internals of PostgreSQL:
Not surprising. They'd go out of date fast, be a huge effort to write
and maintain, and sell poorly given the small audience.
Print books probably aren't the way forward here.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Thanks Amit for taking your time and pointing to some useful stuff on the
Internals of PostgreSQL.
One thing I have learned is that PG community is not as hostile/harsh as I
imagined to newbies. Rather, its the reverse.
I am feeling at home here.
Amit, would you please help out on how to apply some patches in PG source
code. For example, there are two patches attached here: one on
CORRESPONDING CLAUSE and one on MERGE SQL Standard.
There are some errors saying Hunk failed(src/backend/parser/gram.y.rej).
postgresql-9.6.2$ patch --dry-run -p1 < corresponding_clause_v12.patch
patching file doc/src/sgml/queries.sgml
Hunk #1 succeeded at 1603 (offset 2 lines).
Hunk #2 succeeded at 1622 (offset 2 lines).
Hunk #3 succeeded at 1664 (offset 2 lines).
patching file doc/src/sgml/sql.sgml
patching file src/backend/nodes/copyfuncs.c
Hunk #1 succeeded at 2807 (offset -188 lines).
Hunk #2 succeeded at 2823 (offset -188 lines).
Hunk #3 succeeded at 4251 (offset -340 lines).
patching file src/backend/nodes/equalfuncs.c
Hunk #1 succeeded at 995 (offset -55 lines).
Hunk #2 succeeded at 1009 (offset -55 lines).
Hunk #3 succeeded at 2708 (offset -230 lines).
patching file src/backend/nodes/nodeFuncs.c
Hunk #1 succeeded at 3384 (offset -60 lines).
patching file src/backend/nodes/outfuncs.c
Hunk #1 succeeded at 2500 (offset -164 lines).
Hunk #2 succeeded at 2793 (offset -179 lines).
Hunk #3 succeeded at 2967 (offset -184 lines).
patching file src/backend/nodes/readfuncs.c
Hunk #1 succeeded at 414 (offset -2 lines).
patching file src/backend/nodes/value.c
patching file src/backend/optimizer/prep/prepunion.c
Hunk #1 succeeded at 92 (offset 1 line).
Hunk #2 succeeded at 112 (offset 1 line).
Hunk #3 succeeded at 190 (offset 1 line).
Hunk #4 succeeded at 273 (offset 1 line).
Hunk #5 succeeded at 339 (offset 1 line).
Hunk #6 succeeded at 445 (offset 1 line).
Hunk #7 succeeded at 1057 (offset 1 line).
Hunk #8 succeeded at 1080 (offset 1 line).
Hunk #9 succeeded at 2190 (offset -13 lines).
patching file src/backend/parser/analyze.c
Hunk #1 succeeded at 75 (offset -1 lines).
Hunk #2 succeeded at 1600 (offset -61 lines).
Hunk #3 succeeded at 1882 (offset -69 lines).
Hunk #4 succeeded at 1892 (offset -69 lines).
Hunk #5 succeeded at 1994 (offset -69 lines).
patching file src/backend/parser/gram.y
Hunk #1 succeeded at 158 (offset -8 lines).
Hunk #2 FAILED at 394.
Hunk #3 succeeded at 573 with fuzz 2 (offset -41 lines).
Hunk #4 succeeded at 3328 (offset -251 lines).
Hunk #5 succeeded at 10182 (offset -699 lines).
Hunk #6 succeeded at 13470 (offset -771 lines).
Hunk #7 succeeded at 13784 (offset -773 lines).
Hunk #8 succeeded at 14581 (offset -811 lines).
Hunk #9 succeeded at 14589 (offset -811 lines).
1 out of 9 hunks FAILED -- saving rejects to file
src/backend/parser/gram.y.rej
patching file src/backend/parser/parse_type.c
Hunk #1 succeeded at 736 (offset 1 line).
patching file src/backend/utils/adt/ruleutils.c
Hunk #1 succeeded at 5166 (offset -276 lines).
patching file src/include/nodes/parsenodes.h
Hunk #1 succeeded at 1285 (offset -175 lines).
Hunk #2 succeeded at 1321 (offset -175 lines).
Hunk #3 succeeded at 1350 (offset -175 lines).
patching file src/include/nodes/value.h
patching file src/include/parser/kwlist.h
Hunk #1 succeeded at 95 (offset -2 lines).
patching file src/test/regress/expected/create_view.out
Hunk #1 succeeded at 1571 (offset -55 lines).
patching file src/test/regress/expected/rules.out
Hunk #1 succeeded at 2260 (offset -85 lines).
patching file src/test/regress/expected/union.out
Hunk #1 succeeded at 59 with fuzz 2.
Hunk #3 succeeded at 479 (offset -1 lines).
Hunk #4 succeeded at 609 (offset -1 lines).
Hunk #5 succeeded at 684 (offset -1 lines).
Hunk #6 succeeded at 785 with fuzz 1 (offset -1 lines).
Hunk #7 succeeded at 838 (offset -1 lines).
patching file src/test/regress/sql/create_view.sql
Hunk #1 succeeded at 524 (offset -27 lines).
patching file src/test/regress/sql/union.sql
Hunk #1 succeeded at 20 with fuzz 2.
Hunk #2 succeeded at 69 with fuzz 2.
Hunk #3 succeeded at 149 (offset -1 lines).
Hunk #4 succeeded at 194 (offset -1 lines).
Hunk #5 succeeded at 218 (offset -1 lines).
Hunk #6 succeeded at 252 with fuzz 2 (offset -1 lines).
Hunk #7 succeeded at 281 (offset -1 lines).
..../postgresql-9.6.2$
Regards,
Zeray
On Wed, Mar 29, 2017 at 5:53 AM, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp
Show quoted text
wrote:
Hi,
On 2017/03/28 15:40, Kang Yuzhe wrote:
Thanks Tsunakawa for such an informative reply.
Almost all of the docs related to the internals of PG are of introductory
concepts only.
There is even more useful PG internals site entitled "The Internals of
PostgreSQL" in http://www.interdb.jp/pg/ translation of the Japanese PG
Internals.The query processing framework that is described in the manual as you
mentioned is of informative and introductory nature.
In theory, the query processing framework described in the manual is
understandable.Unfortunate, it is another story to understand how query processing
framework in PG codebase really works.
It has become a difficult task for me to walk through the PG source code
for example how SELECT/INSERT/TRUNCATE in the the different modules under
"src/..". really works.I wish there were Hands-On with PostgreSQL Internals like
https://bkmjournal.wordpress.com/2017/01/22/hands-on-with-postgresql-internals/
for more complex PG features.
For example, MERGE SQL standard is not supported yet by PG. I wish there
were Hands-On with PostgreSQL Internals for MERGE/UPSERT. How it is
implemented in parser/executor/storage etc. modules with detailed
explanation for each code and debugging and other important concepts
related to system programming.I am not sure if I can show you that one place where you could learn all
of that, but many people who started with PostgreSQL development at some
point started by exploring the source code itself (either for learning or
to write a feature patch), articles on PostgreSQL wiki, and many related
presentations accessible using the Internet. I liked the following among
many others:Introduction to Hacking PostgreSQL:
http://www.neilconway.org/talks/hacking/Inside the PostgreSQL Query Optimizer:
http://www.neilconway.org/talks/optimizer/optimizer.pdfPostgres Internals Presentations:
http://momjian.us/main/presentations/internals.htmlThanks,
Amit
Attachments:
corresponding_clause_v12.patchapplication/octet-stream; name=corresponding_clause_v12.patchDownload
diff --git a/doc/src/sgml/queries.sgml b/doc/src/sgml/queries.sgml
index 30792f45f1..2d60718ff1 100644
--- a/doc/src/sgml/queries.sgml
+++ b/doc/src/sgml/queries.sgml
@@ -1601,6 +1601,9 @@ SELECT DISTINCT ON (<replaceable>expression</replaceable> <optional>, <replaceab
<primary>EXCEPT</primary>
</indexterm>
<indexterm zone="queries-union">
+ <primary>CORRESPONDING</primary>
+ </indexterm>
+ <indexterm zone="queries-union">
<primary>set union</primary>
</indexterm>
<indexterm zone="queries-union">
@@ -1617,9 +1620,9 @@ SELECT DISTINCT ON (<replaceable>expression</replaceable> <optional>, <replaceab
The results of two queries can be combined using the set operations
union, intersection, and difference. The syntax is
<synopsis>
-<replaceable>query1</replaceable> UNION <optional>ALL</optional> <replaceable>query2</replaceable>
-<replaceable>query1</replaceable> INTERSECT <optional>ALL</optional> <replaceable>query2</replaceable>
-<replaceable>query1</replaceable> EXCEPT <optional>ALL</optional> <replaceable>query2</replaceable>
+<replaceable>query1</replaceable> UNION <optional>ALL</optional> <optional>CORRESPONDING</optional> <optional>BY</optional> <replaceable>query2</replaceable>
+<replaceable>query1</replaceable> INTERSECT <optional>ALL</optional> <optional>CORRESPONDING</optional> <optional>BY</optional> <replaceable>query2</replaceable>
+<replaceable>query1</replaceable> EXCEPT <optional>ALL</optional> <optional>CORRESPONDING</optional> <optional>BY</optional> <replaceable>query2</replaceable>
</synopsis>
<replaceable>query1</replaceable> and
<replaceable>query2</replaceable> are queries that can use any of
@@ -1659,14 +1662,31 @@ SELECT DISTINCT ON (<replaceable>expression</replaceable> <optional>, <replaceab
</para>
<para>
- In order to calculate the union, intersection, or difference of two
- queries, the two queries must be <quote>union compatible</quote>,
- which means that they return the same number of columns and
- the corresponding columns have compatible data types, as
- described in <xref linkend="typeconv-union-case">.
+ <literal>EXCEPT</> returns all rows that are in the result of
+ <replaceable>query1</replaceable> but not in the result of
+ <replaceable>query2</replaceable>. (This is sometimes called the
+ <firstterm>difference</> between two queries.) Again, duplicates
+ are eliminated unless <literal>EXCEPT ALL</> is used.
</para>
- </sect1>
+ <para>
+ <literal>CORRESPONDING</> returns all columns that are in both
+ <replaceable>query1</> and <replaceable>query2</> with the same name.
+ </para>
+
+ <para>
+ <literal>CORRESPONDING BY</> returns all columns in the column list
+ that are also in both <replaceable>query1</> and
+ <replaceable>query2</> with the same name. The names in column list
+ must be unique.
+ </para>
+
+ <para>
+ The names of columns in result when <literal>CORRESPONDING</> or
+ <literal>CORRESPONDING BY</> clause is used must be unique in
+ <replaceable>query1</replaceable> and <replaceable>query2</replaceable>.
+ </para>
+ </sect1>
<sect1 id="queries-order">
<title>Sorting Rows</title>
diff --git a/doc/src/sgml/sql.sgml b/doc/src/sgml/sql.sgml
index 57396d7c24..f98c22e696 100644
--- a/doc/src/sgml/sql.sgml
+++ b/doc/src/sgml/sql.sgml
@@ -859,7 +859,7 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="PARAMETER">expression</replac
[ WHERE <replaceable class="PARAMETER">condition</replaceable> ]
[ GROUP BY <replaceable class="PARAMETER">expression</replaceable> [, ...] ]
[ HAVING <replaceable class="PARAMETER">condition</replaceable> [, ...] ]
- [ { UNION | INTERSECT | EXCEPT } [ ALL ] <replaceable class="PARAMETER">select</replaceable> ]
+ [ { UNION | INTERSECT | EXCEPT } [ ALL ] [ CORRESPONDING [ BY ( <replaceable class="PARAMETER">expression</replaceable> ) ] ] <replaceable class="PARAMETER">select</replaceable> ]
[ ORDER BY <replaceable class="parameter">expression</replaceable> [ ASC | DESC | USING <replaceable class="parameter">operator</replaceable> ] [ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { <replaceable class="PARAMETER">count</replaceable> | ALL } ]
[ OFFSET <replaceable class="PARAMETER">start</replaceable> ]
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 1c88d601bd..11e0590eec 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2995,6 +2995,7 @@ _copySelectStmt(const SelectStmt *from)
COPY_NODE_FIELD(withClause);
COPY_SCALAR_FIELD(op);
COPY_SCALAR_FIELD(all);
+ COPY_NODE_FIELD(correspondingClause);
COPY_NODE_FIELD(larg);
COPY_NODE_FIELD(rarg);
@@ -3010,6 +3011,8 @@ _copySetOperationStmt(const SetOperationStmt *from)
COPY_SCALAR_FIELD(all);
COPY_NODE_FIELD(larg);
COPY_NODE_FIELD(rarg);
+ COPY_NODE_FIELD(correspondingColumns);
+ COPY_SCALAR_FIELD(hasCorrespondingBy);
COPY_NODE_FIELD(colTypes);
COPY_NODE_FIELD(colTypmods);
COPY_NODE_FIELD(colCollations);
@@ -4588,6 +4591,8 @@ _copyValue(const Value *from)
(int) from->type);
break;
}
+ COPY_LOCATION_FIELD(location);
+
return newnode;
}
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 5941b7a2bf..dd6598d85b 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1050,6 +1050,7 @@ _equalSelectStmt(const SelectStmt *a, const SelectStmt *b)
COMPARE_NODE_FIELD(withClause);
COMPARE_SCALAR_FIELD(op);
COMPARE_SCALAR_FIELD(all);
+ COMPARE_NODE_FIELD(correspondingClause);
COMPARE_NODE_FIELD(larg);
COMPARE_NODE_FIELD(rarg);
@@ -1063,6 +1064,8 @@ _equalSetOperationStmt(const SetOperationStmt *a, const SetOperationStmt *b)
COMPARE_SCALAR_FIELD(all);
COMPARE_NODE_FIELD(larg);
COMPARE_NODE_FIELD(rarg);
+ COMPARE_NODE_FIELD(correspondingColumns);
+ COMPARE_SCALAR_FIELD(hasCorrespondingBy);
COMPARE_NODE_FIELD(colTypes);
COMPARE_NODE_FIELD(colTypmods);
COMPARE_NODE_FIELD(colCollations);
@@ -2935,6 +2938,8 @@ _equalValue(const Value *a, const Value *b)
break;
}
+ COMPARE_LOCATION_FIELD(location);
+
return true;
}
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 6e52eb7231..7102ea96c2 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3444,6 +3444,8 @@ raw_expression_tree_walker(Node *node,
return true;
if (walker(stmt->lockingClause, context))
return true;
+ if (walker(stmt->correspondingClause, context))
+ return true;
if (walker(stmt->withClause, context))
return true;
if (walker(stmt->larg, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index bbb63a4bfa..09c097857d 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2664,6 +2664,7 @@ _outSelectStmt(StringInfo str, const SelectStmt *node)
WRITE_NODE_FIELD(withClause);
WRITE_ENUM_FIELD(op, SetOperation);
WRITE_BOOL_FIELD(all);
+ WRITE_NODE_FIELD(correspondingClause);
WRITE_NODE_FIELD(larg);
WRITE_NODE_FIELD(rarg);
}
@@ -2971,6 +2972,8 @@ _outSetOperationStmt(StringInfo str, const SetOperationStmt *node)
WRITE_BOOL_FIELD(all);
WRITE_NODE_FIELD(larg);
WRITE_NODE_FIELD(rarg);
+ WRITE_NODE_FIELD(correspondingColumns);
+ WRITE_BOOL_FIELD(hasCorrespondingBy);
WRITE_NODE_FIELD(colTypes);
WRITE_NODE_FIELD(colTypmods);
WRITE_NODE_FIELD(colCollations);
@@ -3148,6 +3151,7 @@ _outAExpr(StringInfo str, const A_Expr *node)
static void
_outValue(StringInfo str, const Value *value)
{
+ /* NB: this isn't a complete set of fields */
switch (value->type)
{
case T_Integer:
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 474f221a75..6e284f9ef8 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -416,6 +416,8 @@ _readSetOperationStmt(void)
READ_BOOL_FIELD(all);
READ_NODE_FIELD(larg);
READ_NODE_FIELD(rarg);
+ READ_NODE_FIELD(correspondingColumns);
+ READ_BOOL_FIELD(hasCorrespondingBy);
READ_NODE_FIELD(colTypes);
READ_NODE_FIELD(colTypmods);
READ_NODE_FIELD(colCollations);
diff --git a/src/backend/nodes/value.c b/src/backend/nodes/value.c
index 5d2f96c103..72afc172f9 100644
--- a/src/backend/nodes/value.c
+++ b/src/backend/nodes/value.c
@@ -26,6 +26,7 @@ makeInteger(long i)
v->type = T_Integer;
v->val.ival = i;
+ v->location = -1;
return v;
}
@@ -41,6 +42,7 @@ makeFloat(char *numericStr)
v->type = T_Float;
v->val.str = numericStr;
+ v->location = -1;
return v;
}
@@ -56,6 +58,7 @@ makeString(char *str)
v->type = T_String;
v->val.str = str;
+ v->location = -1;
return v;
}
@@ -71,5 +74,6 @@ makeBitString(char *str)
v->type = T_BitString;
v->val.str = str;
+ v->location = -1;
return v;
}
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
index e327e66f6b..f02066dd5b 100644
--- a/src/backend/optimizer/prep/prepunion.c
+++ b/src/backend/optimizer/prep/prepunion.c
@@ -91,7 +91,8 @@ static List *generate_setop_tlist(List *colTypes, List *colCollations,
Index varno,
bool hack_constants,
List *input_tlist,
- List *refnames_tlist);
+ List *refnames_tlist,
+ bool no_corresponding);
static List *generate_append_tlist(List *colTypes, List *colCollations,
bool flag,
List *input_tlists,
@@ -110,6 +111,7 @@ static Node *adjust_appendrel_attrs_mutator(Node *node,
static Relids adjust_relid_set(Relids relids, Index oldrelid, Index newrelid);
static List *adjust_inherited_tlist(List *tlist,
AppendRelInfo *context);
+static List *make_corresponding_target(List *corresponding_list, List *subroot_list);
/*
@@ -187,6 +189,24 @@ plan_set_operations(PlannerInfo *root)
leftmostQuery->targetList,
&top_tlist);
}
+ /*
+ * If corresponding column specified, we take column names from it.
+ */
+ else if (topop->correspondingColumns != NIL )
+ {
+ /*
+ * Recurse on setOperations tree to generate paths for set ops. The
+ * final output path should have just the column types shown as the
+ * output from the top-level node, plus possibly resjunk working
+ * columns (we can rely on upper-level nodes to deal with that).
+ */
+ path = recurse_set_operations((Node *) topop, root,
+ topop->colTypes, topop->colCollations,
+ true, -1,
+ topop->correspondingColumns,
+ &top_tlist,
+ NULL);
+ }
else
{
/*
@@ -252,6 +272,8 @@ recurse_set_operations(Node *setOp, PlannerInfo *root,
List **pTargetList,
double *pNumGroups)
{
+ SetOperationStmt *topop = (SetOperationStmt *) root->parse->setOperations;
+
if (IsA(setOp, RangeTblRef))
{
RangeTblRef *rtr = (RangeTblRef *) setOp;
@@ -316,23 +338,53 @@ recurse_set_operations(Node *setOp, PlannerInfo *root,
path = (Path *) create_subqueryscan_path(root, rel, subpath,
NIL, NULL);
- /*
- * Figure out the appropriate target list, and update the
- * SubqueryScanPath with the PathTarget form of that.
- */
- tlist = generate_setop_tlist(colTypes, colCollations,
+ if (topop->correspondingColumns != NIL )
+ {
+ List *correspondingTarget;
+
+ /*
+ * make target list that only contains corresponding column
+ * from sub-queries list ito use it for projection
+ */
+ correspondingTarget = make_corresponding_target(
+ topop->correspondingColumns,
+ subroot->processed_tlist);
+
+ /*
+ * Figure out the appropriate target list, and update the
+ * SubqueryScanPath with the PathTarget form of that.
+ */
+ tlist = generate_setop_tlist(colTypes, colCollations, flag,
+ rtr->rtindex, true,
+ correspondingTarget,
+ refnames_tlist, false);
+
+ path = apply_projection_to_path(root, rel, path,
+ create_pathtarget(root, tlist));
+
+ /* Return the fully-fledged tlist to caller, too */
+ *pTargetList = tlist;
+
+ }
+ else
+ {
+ /*
+ * Figure out the appropriate target list, and update the
+ * SubqueryScanPath with the PathTarget form of that.
+ */
+ tlist = generate_setop_tlist(colTypes, colCollations,
flag,
rtr->rtindex,
true,
subroot->processed_tlist,
- refnames_tlist);
+ refnames_tlist, true);
- path = apply_projection_to_path(root, rel, path,
+ path = apply_projection_to_path(root, rel, path,
create_pathtarget(root, tlist));
- /* Return the fully-fledged tlist to caller, too */
- *pTargetList = tlist;
-
+ /* Return the fully-fledged tlist to caller, too */
+ *pTargetList = tlist;
+ }
/*
* Estimate number of groups if caller wants it. If the subquery used
* grouping or aggregation, its output is probably mostly unique
@@ -392,7 +444,7 @@ recurse_set_operations(Node *setOp, PlannerInfo *root,
0,
false,
*pTargetList,
- refnames_tlist);
+ refnames_tlist, true);
path = apply_projection_to_path(root,
path->parent,
path,
@@ -1004,7 +1056,8 @@ generate_setop_tlist(List *colTypes, List *colCollations,
Index varno,
bool hack_constants,
List *input_tlist,
- List *refnames_tlist)
+ List *refnames_tlist,
+ bool no_corresponding)
{
List *tlist = NIL;
int resno = 1;
@@ -1026,8 +1079,8 @@ generate_setop_tlist(List *colTypes, List *colCollations,
rtlc = lnext(rtlc);
- Assert(inputtle->resno == resno);
- Assert(reftle->resno == resno);
+ Assert(!no_corresponding || inputtle->resno == resno);
+ Assert(!no_corresponding || reftle->resno == resno);
Assert(!inputtle->resjunk);
Assert(!reftle->resjunk);
@@ -2150,3 +2203,70 @@ adjust_appendrel_attrs_multilevel(PlannerInfo *root, Node *node,
/* Now translate for this child */
return adjust_appendrel_attrs(root, node, appinfo);
}
+
+/*
+ * generate target list from left target list with the order
+ * of right target list
+ */
+static List *
+make_corresponding_target(List *corresponding_list, List *subroot_list)
+{
+ Index internal = 0;
+ ListCell *ltl;
+ ListCell *rtl;
+ int size;
+ int i;
+ List *matchingColumns = NIL;
+ TargetEntry *simple_te_array;
+
+ size = list_length(corresponding_list) + 1;
+
+ /* Use array to find the order of corresponding columen */
+ simple_te_array = (TargetEntry *) palloc0(size * sizeof(TargetEntry));
+ foreach(ltl, corresponding_list)
+ {
+ foreach(rtl, subroot_list)
+ {
+ TargetEntry *ltle = (TargetEntry *) lfirst(ltl);
+ TargetEntry *rtle = (TargetEntry *) lfirst(rtl);
+
+ /* Names of the columns must be resolved before calling this method. */
+ Assert(ltle->resname != NULL);
+ Assert(rtle->resname != NULL);
+
+ /* If column names are the same, add it to array. */
+ if (strcmp(ltle->resname, rtle->resname) == 0)
+ {
+ simple_te_array[internal].xpr = rtle->xpr;
+ simple_te_array[internal].expr = rtle->expr;
+ simple_te_array[internal].resno = rtle->resno;
+ simple_te_array[internal].resname = rtle->resname;
+ simple_te_array[internal].ressortgroupref =
+ rtle->ressortgroupref;
+ simple_te_array[internal].resorigtbl = rtle->resorigtbl;
+ simple_te_array[internal].resorigcol = rtle->resorigcol;
+ simple_te_array[internal].resjunk = rtle->resjunk;
+ internal++;
+ continue;
+ }
+ }
+ }
+ /* traverse the array and make targetlist */
+ for (i = 0; i < internal; i++)
+ {
+ TargetEntry *tle = makeNode(TargetEntry);
+
+ tle->xpr = simple_te_array[i].xpr;
+ tle->expr = simple_te_array[i].expr;
+ tle->resno = simple_te_array[i].resno;
+ tle->resname = simple_te_array[i].resname;
+ tle->ressortgroupref = simple_te_array[i].ressortgroupref;
+ tle->resorigtbl = simple_te_array[i].resorigtbl;
+ tle->resorigcol = simple_te_array[i].resorigcol;
+ tle->resjunk = simple_te_array[i].resjunk;
+
+ matchingColumns = lappend(matchingColumns, tle);
+
+ }
+ return matchingColumns;
+}
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index f6025225be..65dc7d153c 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -76,10 +76,18 @@ static Query *transformCreateTableAsStmt(ParseState *pstate,
CreateTableAsStmt *stmt);
static void transformLockingClause(ParseState *pstate, Query *qry,
LockingClause *lc, bool pushedDown);
+static void makeUnionDatatype(List *ltargetlist, List *rtargetlist,
+ SetOperationStmt *op, List **targetlist, ParseState *parentParseState,
+ const char *context);
#ifdef RAW_EXPRESSION_COVERAGE_TEST
static bool test_raw_expression_coverage(Node *node, void *context);
#endif
-
+static List *CommonColumns(List *ltargetlist, List *rtargetlist, bool filtered,
+ ParseState *pstate, const char *context);
+static List *FilterColumnsByNames(List *common_columns, List *filter,
+ ParseState *pstate, const char *context);
+static List *FilterColumnsByTL(List *targetlist, List *filter, bool check_uniq,
+ ParseState *pstate, const char *context);
/*
* parse_analyze
@@ -1653,7 +1661,37 @@ transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt)
qry->targetList = NIL;
targetvars = NIL;
targetnames = NIL;
- left_tlist = list_head(leftmostQuery->targetList);
+
+ /*
+ * for corresponding clause limits top-level query targetlist to those
+ * corresponding column list only
+ */
+ if (sostmt->correspondingColumns != NIL )
+ {
+ left_tlist = list_head(sostmt->correspondingColumns);
+ /*
+ * In the case of corresponding without by clause property across
+ * the statement may differ
+ */
+ if (!sostmt->hasCorrespondingBy)
+ {
+ Node *correspodning_node;
+ correspodning_node = sostmt->larg;
+ while (correspodning_node && IsA(correspodning_node, SetOperationStmt))
+ {
+ SetOperationStmt *op = (SetOperationStmt *) correspodning_node;
+ op->correspondingColumns = sostmt->correspondingColumns;
+ op->colTypes = sostmt->colTypes;
+ op->colTypmods = sostmt->colTypmods;
+ op->colCollations = sostmt->colCollations;
+ op->groupClauses = sostmt->groupClauses;
+
+ correspodning_node = op->larg;
+ }
+ }
+ }
+ else
+ left_tlist = list_head(leftmostQuery->targetList);
forthree(lct, sostmt->colTypes,
lcm, sostmt->colTypmods,
@@ -1913,8 +1951,6 @@ transformSetOperationTree(ParseState *pstate, SelectStmt *stmt,
SetOperationStmt *op = makeNode(SetOperationStmt);
List *ltargetlist;
List *rtargetlist;
- ListCell *ltl;
- ListCell *rtl;
const char *context;
context = (stmt->op == SETOP_UNION ? "UNION" :
@@ -1925,6 +1961,84 @@ transformSetOperationTree(ParseState *pstate, SelectStmt *stmt,
op->all = stmt->all;
/*
+ * If CORRESPONDING is specified, syntax and column name validities checked,
+ * column filtering is done by a subquery later on.
+ */
+ if (stmt->correspondingClause == NIL )
+ {
+ /* No CORRESPONDING clause, no operation needed for column filtering */
+ op->correspondingColumns = stmt->correspondingClause;
+ op->hasCorrespondingBy = false;
+ }
+ else
+ {
+ /*
+ * CORRESPONDING clause, find matching column names from both tables.
+ * If there are none then it is a syntax error.
+ */
+ Query *largQuery;
+ Query *rargQuery;
+ List *matchingColumns;
+ List *rightCorrespondingColumns;
+
+ op->hasCorrespondingBy = linitial(stmt->correspondingClause) != NULL;
+
+ /* Analyze left query to resolve column names. */
+ largQuery = parse_sub_analyze((Node *) stmt->larg,
+ pstate, NULL, false, false );
+
+ /* Analyze right query to resolve column names. */
+ rargQuery = parse_sub_analyze((Node *) stmt->rarg,
+ pstate, NULL, false, false );
+
+ /* Find matching columns from both queries. */
+ matchingColumns = CommonColumns(largQuery->targetList,
+ rargQuery->targetList,
+ op->hasCorrespondingBy,
+ pstate,
+ context);
+
+ /*
+ * If matchingColumns is empty, there is an error.
+ * At least one column in the select lists must have the same name.
+ */
+ if (matchingColumns == NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("there is not any corresponding name"),
+ errhint("%s queries with a CORRESPONDING clause must have at least one column with the same name",
+ context),
+ parser_errposition(pstate,
+ exprLocation((Node *)
+ linitial(largQuery->targetList)))));
+
+ /* Use column filter when it is known */
+ if (op->hasCorrespondingBy)
+ matchingColumns = FilterColumnsByNames(matchingColumns,
+ stmt->correspondingClause,
+ pstate,
+ context);
+
+ op->correspondingColumns = matchingColumns;
+
+ /*
+ * When we know matching columns, we can quickly create
+ * corresponding target list for right target list. It is faster,
+ * than using symmetry. Ensure unique columns when hasCorrespondingBy
+ * is true - in this case, the uniq is not checked already.
+ */
+ rightCorrespondingColumns = FilterColumnsByTL(rargQuery->targetList,
+ matchingColumns,
+ op->hasCorrespondingBy,
+ pstate,
+ context);
+
+ /* make union'd datatype of output column */
+ makeUnionDatatype(matchingColumns, rightCorrespondingColumns,
+ op, targetlist, pstate, context);
+ }
+
+ /*
* Recursively transform the left child node.
*/
op->larg = transformSetOperationTree(pstate, stmt->larg,
@@ -1949,177 +2063,417 @@ transformSetOperationTree(ParseState *pstate, SelectStmt *stmt,
false,
&rtargetlist);
- /*
- * Verify that the two children have the same number of non-junk
- * columns, and determine the types of the merged output columns.
- */
- if (list_length(ltargetlist) != list_length(rtargetlist))
- ereport(ERROR,
- (errcode(ERRCODE_SYNTAX_ERROR),
- errmsg("each %s query must have the same number of columns",
- context),
- parser_errposition(pstate,
- exprLocation((Node *) rtargetlist))));
-
- if (targetlist)
- *targetlist = NIL;
- op->colTypes = NIL;
- op->colTypmods = NIL;
- op->colCollations = NIL;
- op->groupClauses = NIL;
- forboth(ltl, ltargetlist, rtl, rtargetlist)
+ if (op->correspondingColumns == NIL )
{
- TargetEntry *ltle = (TargetEntry *) lfirst(ltl);
- TargetEntry *rtle = (TargetEntry *) lfirst(rtl);
- Node *lcolnode = (Node *) ltle->expr;
- Node *rcolnode = (Node *) rtle->expr;
- Oid lcoltype = exprType(lcolnode);
- Oid rcoltype = exprType(rcolnode);
- int32 lcoltypmod = exprTypmod(lcolnode);
- int32 rcoltypmod = exprTypmod(rcolnode);
- Node *bestexpr;
- int bestlocation;
- Oid rescoltype;
- int32 rescoltypmod;
- Oid rescolcoll;
-
- /* select common type, same as CASE et al */
- rescoltype = select_common_type(pstate,
- list_make2(lcolnode, rcolnode),
- context,
- &bestexpr);
- bestlocation = exprLocation(bestexpr);
- /* if same type and same typmod, use typmod; else default */
- if (lcoltype == rcoltype && lcoltypmod == rcoltypmod)
- rescoltypmod = lcoltypmod;
- else
- rescoltypmod = -1;
-
+ makeUnionDatatype(ltargetlist, rtargetlist, op, targetlist, pstate,
+ context);
/*
- * Verify the coercions are actually possible. If not, we'd fail
- * later anyway, but we want to fail now while we have sufficient
- * context to produce an error cursor position.
- *
- * For all non-UNKNOWN-type cases, we verify coercibility but we
- * don't modify the child's expression, for fear of changing the
- * child query's semantics.
- *
- * If a child expression is an UNKNOWN-type Const or Param, we
- * want to replace it with the coerced expression. This can only
- * happen when the child is a leaf set-op node. It's safe to
- * replace the expression because if the child query's semantics
- * depended on the type of this output column, it'd have already
- * coerced the UNKNOWN to something else. We want to do this
- * because (a) we want to verify that a Const is valid for the
- * target type, or resolve the actual type of an UNKNOWN Param,
- * and (b) we want to avoid unnecessary discrepancies between the
- * output type of the child query and the resolved target type.
- * Such a discrepancy would disable optimization in the planner.
- *
- * If it's some other UNKNOWN-type node, eg a Var, we do nothing
- * (knowing that coerce_to_common_type would fail). The planner
- * is sometimes able to fold an UNKNOWN Var to a constant before
- * it has to coerce the type, so failing now would just break
- * cases that might work.
+ * Verify that the two children have the same number of non-junk
+ * columns, and determine the types of the merged output columns.
*/
- if (lcoltype != UNKNOWNOID)
- lcolnode = coerce_to_common_type(pstate, lcolnode,
- rescoltype, context);
- else if (IsA(lcolnode, Const) ||
- IsA(lcolnode, Param))
- {
- lcolnode = coerce_to_common_type(pstate, lcolnode,
- rescoltype, context);
- ltle->expr = (Expr *) lcolnode;
- }
+ if (list_length(ltargetlist) != list_length(rtargetlist))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("each %s query must have the same number of columns", context),
+ parser_errposition(pstate,
+ exprLocation((Node *) rtargetlist))));
+ }
+
+ return (Node *) op;
+ }
+}
+
+/*
+ * Processes targetlists of two queries for columns with same names to use
+ * with UNION/INTERSECT/EXCEPT CORRESPONDING. filtered is true, when
+ * CORRESPONDING BY is used. When it is false, we can check uniq names
+ * in rtargetlist here.
+ */
+static List *
+CommonColumns(List *ltargetlist, List *rtargetlist, bool filtered,
+ ParseState *pstate, const char *context)
+{
+ List *common_columns = NIL;
+ ListCell *ltlc;
+ ListCell *rtlc;
+ int resno = 1;
+
+ foreach(ltlc, ltargetlist)
+ {
+ TargetEntry *lte = (TargetEntry *) lfirst(ltlc);
+ bool found = false;
+
+ Assert(lte->resname != NULL);
- if (rcoltype != UNKNOWNOID)
- rcolnode = coerce_to_common_type(pstate, rcolnode,
- rescoltype, context);
- else if (IsA(rcolnode, Const) ||
- IsA(rcolnode, Param))
+ foreach(rtlc, rtargetlist)
+ {
+ ListCell *lc;
+ TargetEntry *rte = (TargetEntry *) lfirst(rtlc);
+
+ Assert(rte->resname != NULL);
+
+ if (strcmp(lte->resname, rte->resname) == 0)
{
- rcolnode = coerce_to_common_type(pstate, rcolnode,
- rescoltype, context);
- rtle->expr = (Expr *) rcolnode;
+ if (filtered)
+ {
+ /*
+ * We found common column, but we don't know if it
+ * is in CORRESPONDING BY list - so don't try do more
+ * work here. The column list will be modified later,
+ * so use shall copy here.
+ */
+ common_columns = lappend(common_columns, lte);
+ break;
+ }
+
+ /* If same column name mentioned more than once it is syntax error . */
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("corresponding column \"%s\" is used more times", rte->resname),
+ errhint("In %s queries with CORRESPONDING clause the corresponding column names must be unique.",
+ context),
+ parser_errposition(pstate,
+ exprLocation((Node *) rte))));
+
+ found = true;
+
+ /* In this case, common_columns must be unique */
+ foreach(lc, common_columns)
+ {
+ TargetEntry *te = (TargetEntry *) lfirst(lc);
+
+ if (strcmp(te->resname, lte->resname) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("corresponding column \"%s\" is used more times", lte->resname),
+ errhint("In %s queries with CORRESPONDING clause the corresponding column names must be unique.",
+ context),
+ parser_errposition(pstate,
+ exprLocation((Node *) lte))));
+ }
+
+ /* When is not any other filter create final te */
+ common_columns = lappend(common_columns,
+ makeTargetEntry(lte->expr,
+ (AttrNumber) resno++,
+ lte->resname,
+ false));
}
+ }
+ }
- /*
- * Select common collation. A common collation is required for
- * all set operators except UNION ALL; see SQL:2008 7.13 <query
- * expression> Syntax Rule 15c. (If we fail to identify a common
- * collation for a UNION ALL column, the curCollations element
- * will be set to InvalidOid, which may result in a runtime error
- * if something at a higher query level wants to use the column's
- * collation.)
- */
- rescolcoll = select_common_collation(pstate,
- list_make2(lcolnode, rcolnode),
- (op->op == SETOP_UNION && op->all));
+ return common_columns;
+}
- /* emit results */
- op->colTypes = lappend_oid(op->colTypes, rescoltype);
- op->colTypmods = lappend_int(op->colTypmods, rescoltypmod);
- op->colCollations = lappend_oid(op->colCollations, rescolcoll);
+/*
+ * Returns filtered common columns list - filter is based on CORRESPONDING BY
+ * list Ensure CORRESPONDING BY list is unique. Result is in CORRESPONDING BY
+ * list order. Common columns list can hold duplicate columns.
+ */
+static List *
+FilterColumnsByNames(List *common_columns, List *filter,
+ ParseState *pstate, const char *context)
+{
+ List *filtered_columns = NIL;
+ ListCell *flc;
+ int resno = 1;
- /*
- * For all cases except UNION ALL, identify the grouping operators
- * (and, if available, sorting operators) that will be used to
- * eliminate duplicates.
- */
- if (op->op != SETOP_UNION || !op->all)
+ Assert(common_columns != NIL);
+ Assert(filter != NIL);
+
+ foreach(flc, filter)
+ {
+ Value *strval = (Value *) lfirst(flc);
+ char *name = strVal(strval);
+ ListCell *tlc;
+ bool found = false;
+
+ foreach(tlc, common_columns)
+ {
+ TargetEntry *tec = (TargetEntry *) lfirst(tlc);
+
+ if (strcmp(tec->resname, name) == 0)
{
- SortGroupClause *grpcl = makeNode(SortGroupClause);
- Oid sortop;
- Oid eqop;
- bool hashable;
- ParseCallbackState pcbstate;
-
- setup_parser_errposition_callback(&pcbstate, pstate,
- bestlocation);
-
- /* determine the eqop and optional sortop */
- get_sort_group_operators(rescoltype,
- false, true, false,
- &sortop, &eqop, NULL,
- &hashable);
-
- cancel_parser_errposition_callback(&pcbstate);
-
- /* we don't have a tlist yet, so can't assign sortgrouprefs */
- grpcl->tleSortGroupRef = 0;
- grpcl->eqop = eqop;
- grpcl->sortop = sortop;
- grpcl->nulls_first = false; /* OK with or without sortop */
- grpcl->hashable = hashable;
-
- op->groupClauses = lappend(op->groupClauses, grpcl);
+ ListCell *lc;
+
+ /*
+ * When "found" is true, then common_columns contains
+ * duplicate columns. Raise exception then.
+ */
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("corresponding column \"%s\" is used more times", name),
+ errhint("In %s queries with CORRESPONDING BY clause the corresponding column names must be unique.",
+ context),
+ parser_errposition(pstate,
+ exprLocation((Node *) tec))));
+
+ found = true;
+
+ /* result list should not to contains this name */
+ foreach(lc, filtered_columns)
+ {
+ TargetEntry *te = (TargetEntry *) lfirst(lc);
+
+ /*
+ * CORRESPONDING BY clause contains a column name that is
+ * not in unique in this clause
+ */
+ if (strcmp(te->resname, name) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("column name \"%s\" is not unique in CORRESPONDING BY clause", name),
+ errhint("CORRESPONDING BY clause must contain unique column names only."),
+ parser_errposition(pstate, strval->location)));
+ }
+
+ /* create te with correct resno */
+ filtered_columns = lappend(filtered_columns,
+ makeTargetEntry(tec->expr,
+ (AttrNumber) resno++,
+ tec->resname,
+ false));
}
+ }
- /*
- * Construct a dummy tlist entry to return. We use a SetToDefault
- * node for the expression, since it carries exactly the fields
- * needed, but any other expression node type would do as well.
- */
- if (targetlist)
+ /*
+ * CORRESPONDING BY clause contains a column name that is not
+ * in common columns.
+ */
+ if (!found)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("column name \"%s\" can not be used in CORRESPONDING BY list", name),
+ errhint("%s queries with a CORRESPONDING BY clause must contain column names from both tables.",
+ context),
+ parser_errposition(pstate, strval->location)));
+ }
+
+ return filtered_columns;
+}
+
+/*
+ * Prepare target list for right query of CORRESPONDING clause.
+ * When check_uniq is true, we should to check uniq names from
+ * filter in target list. When it is false, then uniquenes was
+ * checked in CommonColumns function and should not be checked
+ * here again.
+ */
+static List *
+FilterColumnsByTL(List *targetlist, List *filter, bool check_uniq,
+ ParseState *pstate, const char *context)
+{
+ List *result = NIL;
+ ListCell *lc;
+ int resno = 1;
+
+ foreach(lc, filter)
+ {
+ TargetEntry *fte = (TargetEntry *) lfirst(lc);
+ ListCell *tle;
+ bool found = false;
+
+ foreach(tle, targetlist)
+ {
+ TargetEntry *te = (TargetEntry *) lfirst(tle);
+
+ if (strcmp(fte->resname, te->resname) == 0)
{
- SetToDefault *rescolnode = makeNode(SetToDefault);
- TargetEntry *restle;
-
- rescolnode->typeId = rescoltype;
- rescolnode->typeMod = rescoltypmod;
- rescolnode->collation = rescolcoll;
- rescolnode->location = bestlocation;
- restle = makeTargetEntry((Expr *) rescolnode,
- 0, /* no need to set resno */
- NULL,
- false);
- *targetlist = lappend(*targetlist, restle);
+ /* create te with correct resno */
+ result = lappend(result,
+ makeTargetEntry(te->expr,
+ (AttrNumber) resno++,
+ te->resname,
+ false));
+
+ if (!check_uniq)
+ break;
+
+ /*
+ * When "found" is true, then targetlist contains
+ * duplicate filtered columns. Raise exception then.
+ */
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("corresponding column \"%s\" is used more times", te->resname),
+ errhint("In %s queries with CORRESPONDING BY clause the corresponding column names must be unique.",
+ context),
+ parser_errposition(pstate,
+ exprLocation((Node *) te))));
+
+ found = true;
}
}
+ }
- return (Node *) op;
+ return result;
+}
+
+/*
+ * process right and left target list to set up union'd datatype
+ */
+static void
+makeUnionDatatype(List *ltargetlist, List *rtargetlist, SetOperationStmt *op,
+ List **targetlist, ParseState *pstate, const char *context)
+{
+ ListCell *ltl;
+ ListCell *rtl;
+
+ if (targetlist)
+ *targetlist = NIL;
+
+ op->colTypes = NIL;
+ op->colTypmods = NIL;
+ op->colCollations = NIL;
+ op->groupClauses = NIL;
+
+ forboth(ltl, ltargetlist, rtl, rtargetlist)
+ {
+ TargetEntry *ltle = (TargetEntry *) lfirst(ltl);
+ TargetEntry *rtle = (TargetEntry *) lfirst(rtl);
+ Node *lcolnode = (Node *) ltle->expr;
+ Node *rcolnode = (Node *) rtle->expr;
+ Oid lcoltype = exprType(lcolnode);
+ Oid rcoltype = exprType(rcolnode);
+ int32 lcoltypmod = exprTypmod(lcolnode);
+ int32 rcoltypmod = exprTypmod(rcolnode);
+ Node *bestexpr;
+ int bestlocation;
+ Oid rescoltype;
+ int32 rescoltypmod;
+ Oid rescolcoll;
+
+ /* select common type, same as CASE et al */
+ rescoltype = select_common_type(pstate, list_make2(lcolnode, rcolnode),
+ context, &bestexpr);
+ bestlocation = exprLocation(bestexpr);
+ /* if same type and same typmod, use typmod; else default */
+ if (lcoltype == rcoltype && lcoltypmod == rcoltypmod)
+ rescoltypmod = lcoltypmod;
+ else
+ rescoltypmod = -1;
+
+ /*
+ * Verify the coercions are actually possible. If not, we'd fail
+ * later anyway, but we want to fail now while we have sufficient
+ * context to produce an error cursor position.
+ *
+ * For all non-UNKNOWN-type cases, we verify coercibility but we
+ * don't modify the child's expression, for fear of changing the
+ * child query's semantics.
+ *
+ * If a child expression is an UNKNOWN-type Const or Param, we
+ * want to replace it with the coerced expression. This can only
+ * happen when the child is a leaf set-op node. It's safe to
+ * replace the expression because if the child query's semantics
+ * depended on the type of this output column, it'd have already
+ * coerced the UNKNOWN to something else. We want to do this
+ * because (a) we want to verify that a Const is valid for the
+ * target type, or resolve the actual type of an UNKNOWN Param,
+ * and (b) we want to avoid unnecessary discrepancies between the
+ * output type of the child query and the resolved target type.
+ * Such a discrepancy would disable optimization in the planner.
+ *
+ * If it's some other UNKNOWN-type node, eg a Var, we do nothing
+ * (knowing that coerce_to_common_type would fail). The planner
+ * is sometimes able to fold an UNKNOWN Var to a constant before
+ * it has to coerce the type, so failing now would just break
+ * cases that might work.
+ */
+ if (lcoltype != UNKNOWNOID)
+ lcolnode = coerce_to_common_type(pstate, lcolnode, rescoltype,
+ context);
+ else if (IsA(lcolnode, Const) || IsA(lcolnode, Param))
+ {
+ lcolnode = coerce_to_common_type(pstate, lcolnode, rescoltype,
+ context);
+ ltle->expr = (Expr *) lcolnode;
+ }
+
+ if (rcoltype != UNKNOWNOID)
+ rcolnode = coerce_to_common_type(pstate, rcolnode, rescoltype,
+ context);
+ else if (IsA(rcolnode, Const) || IsA(rcolnode, Param))
+ {
+ rcolnode = coerce_to_common_type(pstate, rcolnode, rescoltype,
+ context);
+ rtle->expr = (Expr *) rcolnode;
+ }
+
+ /*
+ * Select common collation. A common collation is required for
+ * all set operators except UNION ALL; see SQL:2008 7.13 <query
+ * expression> Syntax Rule 15c. (If we fail to identify a common
+ * collation for a UNION ALL column, the curCollations element
+ * will be set to InvalidOid, which may result in a runtime error
+ * if something at a higher query level wants to use the column's
+ * collation.)
+ */
+ rescolcoll = select_common_collation(pstate,
+ list_make2(lcolnode, rcolnode),
+ (op->op == SETOP_UNION && op->all));
+
+ /* emit results */
+ op->colTypes = lappend_oid(op->colTypes, rescoltype);
+ op->colTypmods = lappend_int(op->colTypmods, rescoltypmod);
+ op->colCollations = lappend_oid(op->colCollations, rescolcoll);
+
+ /*
+ * For all cases except UNION ALL, identify the grouping operators
+ * (and, if available, sorting operators) that will be used to
+ * eliminate duplicates.
+ */
+ if (op->op != SETOP_UNION || !op->all)
+ {
+ SortGroupClause *grpcl = makeNode(SortGroupClause);
+ Oid sortop;
+ Oid eqop;
+ bool hashable;
+ ParseCallbackState pcbstate;
+
+ setup_parser_errposition_callback(&pcbstate, pstate, bestlocation);
+
+ /* determine the eqop and optional sortop */
+ get_sort_group_operators(rescoltype, false, true, false, &sortop,
+ &eqop, NULL, &hashable);
+
+ cancel_parser_errposition_callback(&pcbstate);
+
+ /* we don't have a tlist yet, so can't assign sortgrouprefs */
+ grpcl->tleSortGroupRef = 0;
+ grpcl->eqop = eqop;
+ grpcl->sortop = sortop;
+ grpcl->nulls_first = false; /* OK with or without sortop */
+ grpcl->hashable = hashable;
+
+ op->groupClauses = lappend(op->groupClauses, grpcl);
+ }
+
+ /*
+ * Construct a dummy tlist entry to return. We use a SetToDefault
+ * node for the expression, since it carries exactly the fields
+ * needed, but any other expression node type would do as well.
+ */
+ if (targetlist)
+ {
+ SetToDefault *rescolnode = makeNode(SetToDefault);
+ TargetEntry *restle;
+
+ rescolnode->typeId = rescoltype;
+ rescolnode->typeMod = rescoltypmod;
+ rescolnode->collation = rescolcoll;
+ rescolnode->location = bestlocation;
+
+ /* no need to set resno */
+ restle = makeTargetEntry((Expr *) rescolnode, 0,
+ NULL, false );
+ *targetlist = lappend(*targetlist, restle);
+ }
}
+
}
/*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d53a29ad2..6e5257d30d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -166,7 +166,7 @@ static void insertSelectOptions(SelectStmt *stmt,
Node *limitOffset, Node *limitCount,
WithClause *withClause,
core_yyscan_t yyscanner);
-static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg);
+static Node *makeSetOp(SetOperation op, bool all, List *correspondingClause, Node *larg, Node *rarg);
static Node *doNegate(Node *n, int location);
static void doNegateFloat(Value *v);
static Node *makeAndExpr(Node *lexpr, Node *rexpr, int location);
@@ -394,7 +394,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
relation_expr_list dostmt_opt_list
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
- publication_name_list
+ publication_name_list opt_corresponding_clause
%type <list> group_by_list
%type <node> group_by_item empty_grouping_set rollup_clause cube_clause
@@ -614,7 +614,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
+ CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY CORRESPONDING COST CREATE
CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3579,7 +3579,10 @@ columnList:
columnElem: ColId
{
- $$ = (Node *) makeString($1);
+ Value *v = makeString($1);
+
+ v->location = @1;
+ $$ = (Node *) v;
}
;
@@ -10878,20 +10881,26 @@ simple_select:
n->fromClause = list_make1($2);
$$ = (Node *)n;
}
- | select_clause UNION all_or_distinct select_clause
+ | select_clause UNION all_or_distinct opt_corresponding_clause select_clause
{
- $$ = makeSetOp(SETOP_UNION, $3, $1, $4);
+ $$ = makeSetOp(SETOP_UNION, $3, $4, $1, $5);
}
- | select_clause INTERSECT all_or_distinct select_clause
+ | select_clause INTERSECT all_or_distinct opt_corresponding_clause select_clause
{
- $$ = makeSetOp(SETOP_INTERSECT, $3, $1, $4);
+ $$ = makeSetOp(SETOP_INTERSECT, $3, $4, $1, $5);
}
- | select_clause EXCEPT all_or_distinct select_clause
+ | select_clause EXCEPT all_or_distinct opt_corresponding_clause select_clause
{
- $$ = makeSetOp(SETOP_EXCEPT, $3, $1, $4);
+ $$ = makeSetOp(SETOP_EXCEPT, $3, $4, $1, $5);
}
;
+opt_corresponding_clause:
+ CORRESPONDING BY '(' columnList ')' { $$ = $4; }
+ | CORRESPONDING { $$ = list_make1(NIL); }
+ | /*EMPTY*/ { $$ = NIL; }
+ ;
+
/*
* SQL standard WITH clause looks like:
*
@@ -14232,7 +14241,6 @@ name_list: name
{ $$ = lappend($1, makeString($3)); }
;
-
name: ColId { $$ = $1; };
database_name:
@@ -14549,6 +14557,7 @@ unreserved_keyword:
| CONTINUE_P
| CONVERSION_P
| COPY
+ | CORRESPONDING
| COST
| CSV
| CUBE
@@ -15383,7 +15392,7 @@ insertSelectOptions(SelectStmt *stmt,
}
static Node *
-makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg)
+makeSetOp(SetOperation op, bool all, List *correspondingClause, Node *larg, Node *rarg)
{
SelectStmt *n = makeNode(SelectStmt);
@@ -15391,6 +15400,7 @@ makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg)
n->all = all;
n->larg = (SelectStmt *) larg;
n->rarg = (SelectStmt *) rarg;
+ n->correspondingClause = correspondingClause;
return (Node *) n;
}
diff --git a/src/backend/parser/parse_type.c b/src/backend/parser/parse_type.c
index 0d7a2b1e1b..b553d847d8 100644
--- a/src/backend/parser/parse_type.c
+++ b/src/backend/parser/parse_type.c
@@ -735,7 +735,8 @@ typeStringToTypeName(const char *str)
stmt->limitCount != NULL ||
stmt->lockingClause != NIL ||
stmt->withClause != NULL ||
- stmt->op != SETOP_NONE)
+ stmt->op != SETOP_NONE ||
+ stmt->correspondingClause != NULL)
goto fail;
if (list_length(stmt->targetList) != 1)
goto fail;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index c2681ced2a..adf797d600 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -5442,6 +5442,30 @@ get_setop_query(Node *setOp, Query *query, deparse_context *context,
}
if (op->all)
appendStringInfoString(buf, "ALL ");
+ if (op->correspondingColumns != NIL )
+ {
+ if (op->hasCorrespondingBy)
+ {
+ const char *sep;
+ ListCell *l;
+ appendStringInfoString(buf, "CORRESPONDING BY(");
+ sep = "";
+
+ foreach(l, op->correspondingColumns)
+ {
+ TargetEntry *tle = (TargetEntry *) lfirst(l);
+
+ appendStringInfoString(buf, sep);
+ appendStringInfo(buf, "%s", tle->resname);
+ sep = ", ";
+ }
+ appendStringInfoChar(buf, ')');
+
+ }
+ else
+
+ appendStringInfoString(buf, "CORRESPONDING ");
+ }
/* Always parenthesize if RHS is another setop */
need_paren = IsA(op->rarg, SetOperationStmt);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 3a71dd5b37..ed8ce2f2d0 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1460,6 +1460,7 @@ typedef struct SelectStmt
*/
List *distinctClause; /* NULL, list of DISTINCT ON exprs, or
* lcons(NIL,NIL) for all (SELECT DISTINCT) */
+ List *correspondingClause; /* CORRESPONDING BY clauses*/
IntoClause *intoClause; /* target for SELECT INTO */
List *targetList; /* the target list (of ResTarget) */
List *fromClause; /* the FROM clause */
@@ -1495,7 +1496,6 @@ typedef struct SelectStmt
bool all; /* ALL specified? */
struct SelectStmt *larg; /* left child */
struct SelectStmt *rarg; /* right child */
- /* Eventually add fields for CORRESPONDING spec here */
} SelectStmt;
@@ -1525,8 +1525,8 @@ typedef struct SetOperationStmt
bool all; /* ALL specified? */
Node *larg; /* left child */
Node *rarg; /* right child */
- /* Eventually add fields for CORRESPONDING spec here */
-
+ List *correspondingColumns; /* list of corresponding column names */
+ bool hasCorrespondingBy; /* has corresponding by cluase? */
/* Fields derived during parse analysis: */
List *colTypes; /* OID list of output column type OIDs */
List *colTypmods; /* integer list of output column typmods */
diff --git a/src/include/nodes/value.h b/src/include/nodes/value.h
index ede97b7bcd..bf3b6e9b68 100644
--- a/src/include/nodes/value.h
+++ b/src/include/nodes/value.h
@@ -47,6 +47,7 @@ typedef struct Value
long ival; /* machine integer */
char *str; /* string */
} val;
+ int location; /* token location, or -1 if unknown */
} Value;
#define intVal(v) (((Value *)(v))->val.ival)
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index cd21a789d5..7f1c2554e3 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -97,6 +97,7 @@ PG_KEYWORD("content", CONTENT_P, UNRESERVED_KEYWORD)
PG_KEYWORD("continue", CONTINUE_P, UNRESERVED_KEYWORD)
PG_KEYWORD("conversion", CONVERSION_P, UNRESERVED_KEYWORD)
PG_KEYWORD("copy", COPY, UNRESERVED_KEYWORD)
+PG_KEYWORD("corresponding", CORRESPONDING, UNRESERVED_KEYWORD)
PG_KEYWORD("cost", COST, UNRESERVED_KEYWORD)
PG_KEYWORD("create", CREATE, RESERVED_KEYWORD)
PG_KEYWORD("cross", CROSS, TYPE_FUNC_NAME_KEYWORD)
diff --git a/src/test/regress/expected/create_view.out b/src/test/regress/expected/create_view.out
index c719262720..3bcb89ceca 100644
--- a/src/test/regress/expected/create_view.out
+++ b/src/test/regress/expected/create_view.out
@@ -1626,3 +1626,28 @@ select pg_get_viewdef('tt19v', true);
set client_min_messages = warning;
DROP SCHEMA temp_view_test CASCADE;
DROP SCHEMA testviewschm2 CASCADE;
+-- views with corresponding clause
+create view view_corresponding_01 as select 1 as a, 2 as b union all corresponding select 3 as a, 4 as b;
+select * from view_corresponding_01;
+ a | b
+---+---
+ 1 | 2
+ 3 | 4
+(2 rows)
+
+create view view_corresponding_02 as select 1 as a, 2 as b union all corresponding by (a,b) select 3 as a, 4 as b, 5 as c;
+select * from view_corresponding_02;
+ a | b
+---+---
+ 1 | 2
+ 3 | 4
+(2 rows)
+
+create view view_corresponding_03 as select 1 as a, 2 as b union all corresponding by (b,a) select 3 as a, 4 as b, 5 as c;
+select * from view_corresponding_03;
+ b | a
+---+---
+ 2 | 1
+ 4 | 3
+(2 rows)
+
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index d706f42b2d..3e36dc58f9 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2345,6 +2345,23 @@ toyemp| SELECT emp.name,
emp.location,
(12 * emp.salary) AS annualsal
FROM emp;
+view_corresponding_01| SELECT 1 AS a,
+ 2 AS b
+UNION ALL CORRESPONDING
+ SELECT 3 AS a,
+ 4 AS b;
+view_corresponding_02| SELECT 1 AS a,
+ 2 AS b
+UNION ALL CORRESPONDING BY(a, b)
+ SELECT 3 AS a,
+ 4 AS b,
+ 5 AS c;
+view_corresponding_03| SELECT 1 AS b,
+ 2 AS a
+UNION ALL CORRESPONDING BY(b, a)
+ SELECT 3 AS b,
+ 4 AS a,
+ 5 AS c;
SELECT tablename, rulename, definition FROM pg_rules
ORDER BY tablename, rulename;
pg_settings|pg_settings_n|CREATE RULE pg_settings_n AS
diff --git a/src/test/regress/expected/union.out b/src/test/regress/expected/union.out
index 4d697bada7..59d6c001ef 100644
--- a/src/test/regress/expected/union.out
+++ b/src/test/regress/expected/union.out
@@ -59,6 +59,87 @@ SELECT 1.1 AS two UNION SELECT 2.2 ORDER BY 1;
2.2
(2 rows)
+SELECT 1 AS two UNION CORRESPONDING SELECT 2 two UNION CORRESPONDING SELECT 2 two;
+ two
+-----
+ 2
+ 1
+(2 rows)
+
+SELECT 1 a, 2 b, 3 c UNION CORRESPONDING BY(a) SELECT 4 a, 5 b, 6 c;
+ a
+---
+ 1
+ 4
+(2 rows)
+
+SELECT 1 a, 2 b, 3 c UNION CORRESPONDING BY(a, b) SELECT 4 a, 5 b, 6 c;
+ a | b
+---+---
+ 1 | 2
+ 4 | 5
+(2 rows)
+
+SELECT 1 a, 2 b, 3 c UNION CORRESPONDING BY(b, c) SELECT 4 a, 5 b, 6 c;
+ b | c
+---+---
+ 2 | 3
+ 5 | 6
+(2 rows)
+
+SELECT 1 a, 2 b, 3 c UNION CORRESPONDING BY (c,b) SELECT 4 a, 5 b, 6 c, 8 d;
+ c | b
+---+---
+ 3 | 2
+ 6 | 5
+(2 rows)
+
+-- CORRESPONDING column ordering, left clause's column ordering must be preserved.
+SELECT 1 a, 2 b, 3 c UNION CORRESPONDING SELECT 4 a, 5 b, 6 c;
+ a | b | c
+---+---+---
+ 1 | 2 | 3
+ 4 | 5 | 6
+(2 rows)
+
+SELECT 2 b, 1 a, 3 c UNION CORRESPONDING SELECT 4 a, 5 b, 6 c;
+ b | a | c
+---+---+---
+ 2 | 1 | 3
+ 5 | 4 | 6
+(2 rows)
+
+-- CORRESPONDING BY column ordering, BY clause column ordering must be preserved.
+SELECT 1 a, 2 b, 3 c UNION CORRESPONDING BY(a, b, c) SELECT 4 a, 5 b, 6 c;
+ a | b | c
+---+---+---
+ 1 | 2 | 3
+ 4 | 5 | 6
+(2 rows)
+
+SELECT 1 a, 2 b, 3 c UNION CORRESPONDING BY(b, c, a) SELECT 4 a, 5 b, 6 c;
+ b | c | a
+---+---+---
+ 2 | 3 | 1
+ 5 | 6 | 4
+(2 rows)
+
+-- should to fail
+SELECT 10 a, 20 b UNION ALL CORRESPONDING SELECT 10 c, 20 d;
+ERROR: there is not any corresponding name
+LINE 1: SELECT 10 a, 20 b UNION ALL CORRESPONDING SELECT 10 c, 20 d;
+ ^
+HINT: UNION queries with a CORRESPONDING clause must have at least one column with the same name
+SELECT 10 a, 20 b UNION ALL CORRESPONDING BY (a,b) SELECT 10 c, 20 d;
+ERROR: there is not any corresponding name
+LINE 1: SELECT 10 a, 20 b UNION ALL CORRESPONDING BY (a,b) SELECT 10...
+ ^
+HINT: UNION queries with a CORRESPONDING clause must have at least one column with the same name
+SELECT 10 a, 20 b UNION ALL CORRESPONDING BY (x) SELECT 10 c, 20 d;
+ERROR: there is not any corresponding name
+LINE 1: SELECT 10 a, 20 b UNION ALL CORRESPONDING BY (x) SELECT 10 c...
+ ^
+HINT: UNION queries with a CORRESPONDING clause must have at least one column with the same name
-- Mixed types
SELECT 1.1 AS two UNION SELECT 2 ORDER BY 1;
two
@@ -124,6 +205,147 @@ SELECT 1.1 AS two UNION (SELECT 2 UNION ALL SELECT 2) ORDER BY 1;
2
(2 rows)
+-- other corresponding clause tests,
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6;
+ a | b
+---+---
+ 1 | 2
+ 3 | 4
+(2 rows)
+
+-- when column is not in result, then the name should not be unique
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6;
+ a | b
+---+---
+ 1 | 2
+ 3 | 4
+(2 rows)
+
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6;
+ a | b
+---+---
+ 1 | 2
+ 3 | 4
+(2 rows)
+
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6;
+ a | b
+---+---
+ 1 | 2
+ 3 | 4
+(2 rows)
+
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6 UNION ALL CORRESPONDING SELECT 0 AS x8, 6 AS b, 5 AS a;
+ a | b
+---+---
+ 1 | 2
+ 3 | 4
+ 5 | 6
+(3 rows)
+
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING BY(a,b) SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x3;
+ a | b
+---+---
+ 1 | 2
+ 3 | 4
+(2 rows)
+
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x6 UNION ALL CORRESPONDING BY(a,b) SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6;
+ a | b
+---+---
+ 1 | 2
+ 3 | 4
+(2 rows)
+
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING BY(a,b) SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6 UNION ALL CORRESPONDING SELECT 0 AS x8, 6 AS b, 5 AS a;
+ a | b
+---+---
+ 1 | 2
+ 3 | 4
+ 5 | 6
+(3 rows)
+
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING BY(b) SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6 UNION ALL CORRESPONDING SELECT 0 AS x8, 6 AS b, -100 AS a;
+ b
+---
+ 2
+ 4
+ 6
+(3 rows)
+
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6 UNION ALL CORRESPONDING SELECT 0 AS x8, 6 AS a, -100 AS x9;
+ a
+---
+ 1
+ 3
+ 6
+(3 rows)
+
+SELECT 0 AS a, 1 AS b, 0 AS c UNION ALL CORRESPONDING SELECT 2 AS a, 3 AS b, 10 AS c, 20 AS d UNION ALL CORRESPONDING SELECT 11 AS c, 21 AS d;
+ c
+----
+ 0
+ 10
+ 11
+(3 rows)
+
+SELECT 0 AS a, 1 AS b UNION ALL CORRESPONDING SELECT 2 AS a, 3 AS b, 10 AS c, 20 AS d UNION ALL CORRESPONDING SELECT 11 AS c, 21 AS d, 4 AS a;
+ a
+---
+ 0
+ 2
+ 4
+(3 rows)
+
+SELECT 0 AS a, 1 AS b UNION ALL CORRESPONDING SELECT 2 AS a, 3 AS b, 10 AS c, 20 AS d UNION ALL CORRESPONDING SELECT 11 AS c, 21 AS d, 5 AS b;
+ b
+---
+ 1
+ 3
+ 5
+(3 rows)
+
+-- should fail
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -5 AS x3;
+ERROR: corresponding column "x3" is used more times
+LINE 1: SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 U...
+ ^
+HINT: In UNION queries with CORRESPONDING clause the corresponding column names must be unique.
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -5 AS x3, -10 AS x3;
+ERROR: corresponding column "x3" is used more times
+LINE 1: ...ELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -5 AS x3, -10 AS x3;
+ ^
+HINT: In UNION queries with CORRESPONDING clause the corresponding column names must be unique.
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING BY(a,b,x3) SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -5 AS x3;
+ERROR: corresponding column "x3" is used more times
+LINE 1: SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 U...
+ ^
+HINT: In UNION queries with CORRESPONDING BY clause the corresponding column names must be unique.
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3 UNION ALL CORRESPONDING BY(a,b,x3) SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -5 AS x3, -10 AS x3;
+ERROR: corresponding column "x3" is used more times
+LINE 1: ...ELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -5 AS x3, -10 AS x3;
+ ^
+HINT: In UNION queries with CORRESPONDING BY clause the corresponding column names must be unique.
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x6 UNION ALL CORRESPONDING BY(a,b,b) SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6;
+ERROR: column name "b" is not unique in CORRESPONDING BY clause
+LINE 1: ... 0 AS x3, -1 AS x6 UNION ALL CORRESPONDING BY(a,b,b) SELECT ...
+ ^
+HINT: CORRESPONDING BY clause must contain unique column names only.
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING BY(a,b) SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6 UNION ALL CORRESPONDING SELECT 0 AS x8, 6 AS bb, -100 AS aa;
+ERROR: there is not any corresponding name
+LINE 1: SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 U...
+ ^
+HINT: UNION queries with a CORRESPONDING clause must have at least one column with the same name
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6 UNION ALL CORRESPONDING SELECT 0 AS x8, 6 AS xxx, -100 AS x9;
+ERROR: there is not any corresponding name
+LINE 1: SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 U...
+ ^
+HINT: UNION queries with a CORRESPONDING clause must have at least one column with the same name
+SELECT 0 AS a, 1 AS b UNION ALL CORRESPONDING SELECT 2 AS a, 3 AS b, 10 AS c, 20 AS d UNION ALL CORRESPONDING SELECT 11 AS c, 21 AS d;
+ERROR: there is not any corresponding name
+LINE 1: SELECT 0 AS a, 1 AS b UNION ALL CORRESPONDING SELECT 2 AS a,...
+ ^
+HINT: UNION queries with a CORRESPONDING clause must have at least one column with the same name
--
-- Try testing from tables...
--
@@ -258,6 +480,74 @@ ORDER BY 1;
hi de ho neighbor
(5 rows)
+SELECT f1 AS five FROM FLOAT8_TBL
+UNION CORRESPONDING
+SELECT f1 AS five FROM FLOAT8_TBL
+ORDER BY 1;
+ five
+-----------------------
+ -1.2345678901234e+200
+ -1004.3
+ -34.84
+ -1.2345678901234e-200
+ 0
+(5 rows)
+
+SELECT f1 AS five FROM FLOAT8_TBL
+UNION CORRESPONDING BY(five)
+SELECT f1 AS five FROM FLOAT8_TBL
+ORDER BY 1;
+ five
+-----------------------
+ -1.2345678901234e+200
+ -1004.3
+ -34.84
+ -1.2345678901234e-200
+ 0
+(5 rows)
+
+SELECT f1 AS ten FROM FLOAT8_TBL
+UNION ALL CORRESPONDING
+SELECT f1 AS ten FROM FLOAT8_TBL;
+ ten
+-----------------------
+ 0
+ -34.84
+ -1004.3
+ -1.2345678901234e+200
+ -1.2345678901234e-200
+ 0
+ -34.84
+ -1004.3
+ -1.2345678901234e+200
+ -1.2345678901234e-200
+(10 rows)
+
+SELECT f1 AS five FROM FLOAT8_TBL
+ WHERE f1 BETWEEN -1e6 AND 1e6
+UNION CORRESPONDING
+SELECT f1 AS five FROM INT4_TBL
+ WHERE f1 BETWEEN 0 AND 1000000;
+ five
+-----------------------
+ -1004.3
+ -34.84
+ -1.2345678901234e-200
+ 0
+ 123456
+(5 rows)
+
+SELECT * FROM FLOAT8_TBL
+UNION corresponding SELECT * FROM FLOAT8_TBL;
+ f1
+-----------------------
+ 0
+ -1.2345678901234e-200
+ -34.84
+ -1.2345678901234e+200
+ -1004.3
+(5 rows)
+
--
-- INTERSECT and EXCEPT
--
@@ -320,6 +610,63 @@ SELECT q1 FROM int8_tbl EXCEPT ALL SELECT DISTINCT q2 FROM int8_tbl ORDER BY 1;
SELECT q1 FROM int8_tbl EXCEPT ALL SELECT q1 FROM int8_tbl FOR NO KEY UPDATE;
ERROR: FOR NO KEY UPDATE is not allowed with UNION/INTERSECT/EXCEPT
+SELECT q2 FROM int8_tbl INTERSECT CORRESPONDING SELECT q1 AS q2 FROM int8_tbl;
+ q2
+------------------
+ 123
+ 4567890123456789
+(2 rows)
+
+SELECT q2 FROM int8_tbl INTERSECT ALL CORRESPONDING SELECT q1 AS q2 FROM int8_tbl;
+ q2
+------------------
+ 123
+ 4567890123456789
+ 4567890123456789
+(3 rows)
+
+SELECT q2 FROM int8_tbl EXCEPT CORRESPONDING SELECT q1 AS q2 FROM int8_tbl ORDER BY 1;
+ q2
+-------------------
+ -4567890123456789
+ 456
+(2 rows)
+
+SELECT q2 FROM int8_tbl EXCEPT ALL CORRESPONDING SELECT q1 AS q2 FROM int8_tbl ORDER BY 1;
+ q2
+-------------------
+ -4567890123456789
+ 456
+(2 rows)
+
+SELECT q2 FROM int8_tbl EXCEPT ALL CORRESPONDING SELECT DISTINCT q1 AS q2 FROM int8_tbl ORDER BY 1;
+ q2
+-------------------
+ -4567890123456789
+ 456
+ 4567890123456789
+(3 rows)
+
+SELECT q1 FROM int8_tbl EXCEPT CORRESPONDING SELECT q2 AS q1 FROM int8_tbl;
+ q1
+----
+(0 rows)
+
+SELECT q1 FROM int8_tbl EXCEPT ALL CORRESPONDING SELECT q2 AS q1 FROM int8_tbl;
+ q1
+------------------
+ 123
+ 4567890123456789
+(2 rows)
+
+SELECT q1 FROM int8_tbl EXCEPT ALL CORRESPONDING SELECT DISTINCT q2 AS q1 FROM int8_tbl;
+ q1
+------------------
+ 123
+ 4567890123456789
+ 4567890123456789
+(3 rows)
+
--
-- Mixed types
--
@@ -338,6 +685,21 @@ SELECT f1 FROM float8_tbl EXCEPT SELECT f1 FROM int4_tbl ORDER BY 1;
-1.2345678901234e-200
(4 rows)
+SELECT f1 FROM float8_tbl INTERSECT CORRESPONDING SELECT f1 FROM int4_tbl;
+ f1
+----
+ 0
+(1 row)
+
+SELECT f1 FROM float8_tbl EXCEPT CORRESPONDING SELECT f1 FROM int4_tbl ORDER BY 1;
+ f1
+-----------------------
+ -1.2345678901234e+200
+ -1004.3
+ -34.84
+ -1.2345678901234e-200
+(4 rows)
+
--
-- Operator precedence and (((((extra))))) parentheses
--
@@ -424,6 +786,24 @@ SELECT q1 FROM int8_tbl EXCEPT (((SELECT q2 FROM int8_tbl ORDER BY q2 LIMIT 1)))
4567890123456789
(2 rows)
+SELECT q1,q2 FROM int8_tbl EXCEPT CORRESPONDING SELECT q2,q1 FROM int8_tbl
+ORDER BY q2,q1;
+ q1 | q2
+----+----
+(0 rows)
+
+SELECT q1,q2 FROM int8_tbl EXCEPT CORRESPONDING BY(q1) SELECT q2,q1 FROM int8_tbl
+ORDER BY q1;
+ q1
+----
+(0 rows)
+
+SELECT q1,q2 FROM int8_tbl EXCEPT CORRESPONDING BY(q2) SELECT q2,q1 FROM int8_tbl
+ORDER BY q2;
+ q2
+----
+(0 rows)
+
--
-- New syntaxes (7.1) permit new tests
--
@@ -459,6 +839,22 @@ SELECT '3.4'::numeric UNION SELECT 'foo';
ERROR: invalid input syntax for type numeric: "foo"
LINE 1: SELECT '3.4'::numeric UNION SELECT 'foo';
^
+SELECT '3.4'::numeric AS a UNION CORRESPONDING SELECT 'foo' AS a;
+ERROR: invalid input syntax for type numeric: "foo"
+LINE 1: ...CT '3.4'::numeric AS a UNION CORRESPONDING SELECT 'foo' AS a...
+ ^
+SELECT a.f1 FROM (SELECT 'test' AS f1 FROM varchar_tbl) a
+UNION CORRESPONDING
+SELECT b.f1 FROM (SELECT f1 FROM varchar_tbl) b
+ORDER BY 1;
+ f1
+------
+ a
+ ab
+ abcd
+ test
+(4 rows)
+
--
-- Test that expression-index constraints can be pushed down through
-- UNION or UNION ALL
diff --git a/src/test/regress/sql/create_view.sql b/src/test/regress/sql/create_view.sql
index d6f50d6105..f61a01323b 100644
--- a/src/test/regress/sql/create_view.sql
+++ b/src/test/regress/sql/create_view.sql
@@ -551,3 +551,13 @@ select pg_get_viewdef('tt19v', true);
set client_min_messages = warning;
DROP SCHEMA temp_view_test CASCADE;
DROP SCHEMA testviewschm2 CASCADE;
+
+-- views with corresponding clause
+create view view_corresponding_01 as select 1 as a, 2 as b union all corresponding select 3 as a, 4 as b;
+select * from view_corresponding_01;
+
+create view view_corresponding_02 as select 1 as a, 2 as b union all corresponding by (a,b) select 3 as a, 4 as b, 5 as c;
+select * from view_corresponding_02;
+
+create view view_corresponding_03 as select 1 as a, 2 as b union all corresponding by (b,a) select 3 as a, 4 as b, 5 as c;
+select * from view_corresponding_03;
diff --git a/src/test/regress/sql/union.sql b/src/test/regress/sql/union.sql
index 48e6850798..731e886b9a 100644
--- a/src/test/regress/sql/union.sql
+++ b/src/test/regress/sql/union.sql
@@ -20,6 +20,35 @@ SELECT 1 AS three UNION SELECT 2 UNION ALL SELECT 2 ORDER BY 1;
SELECT 1.1 AS two UNION SELECT 2.2 ORDER BY 1;
+SELECT 1 AS two UNION CORRESPONDING SELECT 2 two UNION CORRESPONDING SELECT 2 two;
+
+SELECT 1 a, 2 b, 3 c UNION CORRESPONDING BY(a) SELECT 4 a, 5 b, 6 c;
+
+SELECT 1 a, 2 b, 3 c UNION CORRESPONDING BY(a, b) SELECT 4 a, 5 b, 6 c;
+
+SELECT 1 a, 2 b, 3 c UNION CORRESPONDING BY(b, c) SELECT 4 a, 5 b, 6 c;
+
+SELECT 1 a, 2 b, 3 c UNION CORRESPONDING BY (c,b) SELECT 4 a, 5 b, 6 c, 8 d;
+
+-- CORRESPONDING column ordering, left clause's column ordering must be preserved.
+
+SELECT 1 a, 2 b, 3 c UNION CORRESPONDING SELECT 4 a, 5 b, 6 c;
+
+SELECT 2 b, 1 a, 3 c UNION CORRESPONDING SELECT 4 a, 5 b, 6 c;
+
+-- CORRESPONDING BY column ordering, BY clause column ordering must be preserved.
+
+SELECT 1 a, 2 b, 3 c UNION CORRESPONDING BY(a, b, c) SELECT 4 a, 5 b, 6 c;
+
+SELECT 1 a, 2 b, 3 c UNION CORRESPONDING BY(b, c, a) SELECT 4 a, 5 b, 6 c;
+
+-- should to fail
+SELECT 10 a, 20 b UNION ALL CORRESPONDING SELECT 10 c, 20 d;
+
+SELECT 10 a, 20 b UNION ALL CORRESPONDING BY (a,b) SELECT 10 c, 20 d;
+
+SELECT 10 a, 20 b UNION ALL CORRESPONDING BY (x) SELECT 10 c, 20 d;
+
-- Mixed types
SELECT 1.1 AS two UNION SELECT 2 ORDER BY 1;
@@ -40,6 +69,37 @@ SELECT 1.1 AS three UNION SELECT 2 UNION ALL SELECT 2 ORDER BY 1;
SELECT 1.1 AS two UNION (SELECT 2 UNION ALL SELECT 2) ORDER BY 1;
+-- other corresponding clause tests,
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6;
+
+-- when column is not in result, then the name should not be unique
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6;
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6;
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6;
+
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6 UNION ALL CORRESPONDING SELECT 0 AS x8, 6 AS b, 5 AS a;
+
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING BY(a,b) SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x3;
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x6 UNION ALL CORRESPONDING BY(a,b) SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6;
+
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING BY(a,b) SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6 UNION ALL CORRESPONDING SELECT 0 AS x8, 6 AS b, 5 AS a;
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING BY(b) SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6 UNION ALL CORRESPONDING SELECT 0 AS x8, 6 AS b, -100 AS a;
+
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6 UNION ALL CORRESPONDING SELECT 0 AS x8, 6 AS a, -100 AS x9;
+SELECT 0 AS a, 1 AS b, 0 AS c UNION ALL CORRESPONDING SELECT 2 AS a, 3 AS b, 10 AS c, 20 AS d UNION ALL CORRESPONDING SELECT 11 AS c, 21 AS d;
+SELECT 0 AS a, 1 AS b UNION ALL CORRESPONDING SELECT 2 AS a, 3 AS b, 10 AS c, 20 AS d UNION ALL CORRESPONDING SELECT 11 AS c, 21 AS d, 4 AS a;
+SELECT 0 AS a, 1 AS b UNION ALL CORRESPONDING SELECT 2 AS a, 3 AS b, 10 AS c, 20 AS d UNION ALL CORRESPONDING SELECT 11 AS c, 21 AS d, 5 AS b;
+
+-- should fail
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -5 AS x3;
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -5 AS x3, -10 AS x3;
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING BY(a,b,x3) SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -5 AS x3;
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3 UNION ALL CORRESPONDING BY(a,b,x3) SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -5 AS x3, -10 AS x3;
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x6 UNION ALL CORRESPONDING BY(a,b,b) SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6;
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING BY(a,b) SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6 UNION ALL CORRESPONDING SELECT 0 AS x8, 6 AS bb, -100 AS aa;
+SELECT 0 AS x1, 1 AS a, 0 AS x2, 2 AS b, 0 AS x3, -1 AS x3 UNION ALL CORRESPONDING SELECT 4 AS b, 0 AS x4, 3 AS a, 0 AS x6, -1 AS x6 UNION ALL CORRESPONDING SELECT 0 AS x8, 6 AS xxx, -100 AS x9;
+SELECT 0 AS a, 1 AS b UNION ALL CORRESPONDING SELECT 2 AS a, 3 AS b, 10 AS c, 20 AS d UNION ALL CORRESPONDING SELECT 11 AS c, 21 AS d;
+
--
-- Try testing from tables...
--
@@ -90,6 +150,29 @@ UNION
SELECT TRIM(TRAILING FROM f1) FROM CHAR_TBL
ORDER BY 1;
+SELECT f1 AS five FROM FLOAT8_TBL
+UNION CORRESPONDING
+SELECT f1 AS five FROM FLOAT8_TBL
+ORDER BY 1;
+
+SELECT f1 AS five FROM FLOAT8_TBL
+UNION CORRESPONDING BY(five)
+SELECT f1 AS five FROM FLOAT8_TBL
+ORDER BY 1;
+
+SELECT f1 AS ten FROM FLOAT8_TBL
+UNION ALL CORRESPONDING
+SELECT f1 AS ten FROM FLOAT8_TBL;
+
+SELECT f1 AS five FROM FLOAT8_TBL
+ WHERE f1 BETWEEN -1e6 AND 1e6
+UNION CORRESPONDING
+SELECT f1 AS five FROM INT4_TBL
+ WHERE f1 BETWEEN 0 AND 1000000;
+
+SELECT * FROM FLOAT8_TBL
+UNION corresponding SELECT * FROM FLOAT8_TBL;
+
--
-- INTERSECT and EXCEPT
--
@@ -112,6 +195,22 @@ SELECT q1 FROM int8_tbl EXCEPT ALL SELECT DISTINCT q2 FROM int8_tbl ORDER BY 1;
SELECT q1 FROM int8_tbl EXCEPT ALL SELECT q1 FROM int8_tbl FOR NO KEY UPDATE;
+SELECT q2 FROM int8_tbl INTERSECT CORRESPONDING SELECT q1 AS q2 FROM int8_tbl;
+
+SELECT q2 FROM int8_tbl INTERSECT ALL CORRESPONDING SELECT q1 AS q2 FROM int8_tbl;
+
+SELECT q2 FROM int8_tbl EXCEPT CORRESPONDING SELECT q1 AS q2 FROM int8_tbl ORDER BY 1;
+
+SELECT q2 FROM int8_tbl EXCEPT ALL CORRESPONDING SELECT q1 AS q2 FROM int8_tbl ORDER BY 1;
+
+SELECT q2 FROM int8_tbl EXCEPT ALL CORRESPONDING SELECT DISTINCT q1 AS q2 FROM int8_tbl ORDER BY 1;
+
+SELECT q1 FROM int8_tbl EXCEPT CORRESPONDING SELECT q2 AS q1 FROM int8_tbl;
+
+SELECT q1 FROM int8_tbl EXCEPT ALL CORRESPONDING SELECT q2 AS q1 FROM int8_tbl;
+
+SELECT q1 FROM int8_tbl EXCEPT ALL CORRESPONDING SELECT DISTINCT q2 AS q1 FROM int8_tbl;
+
--
-- Mixed types
--
@@ -120,6 +219,10 @@ SELECT f1 FROM float8_tbl INTERSECT SELECT f1 FROM int4_tbl ORDER BY 1;
SELECT f1 FROM float8_tbl EXCEPT SELECT f1 FROM int4_tbl ORDER BY 1;
+SELECT f1 FROM float8_tbl INTERSECT CORRESPONDING SELECT f1 FROM int4_tbl;
+
+SELECT f1 FROM float8_tbl EXCEPT CORRESPONDING SELECT f1 FROM int4_tbl ORDER BY 1;
+
--
-- Operator precedence and (((((extra))))) parentheses
--
@@ -150,6 +253,15 @@ SELECT q1 FROM int8_tbl EXCEPT SELECT q2 FROM int8_tbl ORDER BY q2 LIMIT 1;
-- But this should work:
SELECT q1 FROM int8_tbl EXCEPT (((SELECT q2 FROM int8_tbl ORDER BY q2 LIMIT 1))) ORDER BY 1;
+SELECT q1,q2 FROM int8_tbl EXCEPT CORRESPONDING SELECT q2,q1 FROM int8_tbl
+ORDER BY q2,q1;
+
+SELECT q1,q2 FROM int8_tbl EXCEPT CORRESPONDING BY(q1) SELECT q2,q1 FROM int8_tbl
+ORDER BY q1;
+
+SELECT q1,q2 FROM int8_tbl EXCEPT CORRESPONDING BY(q2) SELECT q2,q1 FROM int8_tbl
+ORDER BY q2;
+
--
-- New syntaxes (7.1) permit new tests
--
@@ -170,6 +282,13 @@ ORDER BY 1;
-- This should fail, but it should produce an error cursor
SELECT '3.4'::numeric UNION SELECT 'foo';
+SELECT '3.4'::numeric AS a UNION CORRESPONDING SELECT 'foo' AS a;
+
+SELECT a.f1 FROM (SELECT 'test' AS f1 FROM varchar_tbl) a
+UNION CORRESPONDING
+SELECT b.f1 FROM (SELECT f1 FROM varchar_tbl) b
+ORDER BY 1;
+
--
-- Test that expression-index constraints can be pushed down through
-- UNION or UNION ALL
merge_v201.patchapplication/octet-stream; name=merge_v201.patchDownload
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index 7518c84..3aa9aab 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -119,6 +119,7 @@ Complete list of usable sgml source files in this directory.
<!entity listen system "listen.sgml">
<!entity load system "load.sgml">
<!entity lock system "lock.sgml">
+<!entity merge system "merge.sgml">
<!entity move system "move.sgml">
<!entity notify system "notify.sgml">
<!entity prepare system "prepare.sgml">
diff --git a/doc/src/sgml/ref/merge.sgml b/doc/src/sgml/ref/merge.sgml
new file mode 100644
index 0000000..7c73623
--- /dev/null
+++ b/doc/src/sgml/ref/merge.sgml
@@ -0,0 +1,409 @@
+<!--
+$PostgreSQL$
+-->
+
+<refentry id="SQL-MERGE">
+ <refmeta>
+ <refentrytitle id="SQL-MERGE-TITLE">MERGE</refentrytitle>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>MERGE</refname>
+ <refpurpose>update, insert or delete rows of a table based upon source data</refpurpose>
+ </refnamediv>
+
+ <indexterm zone="sql-merge">
+ <primary>MERGE</primary>
+ </indexterm>
+
+ <refsynopsisdiv>
+<synopsis>
+MERGE INTO <replaceable class="PARAMETER">table</replaceable> [ [ AS ] <replaceable class="parameter">alias</replaceable> ]
+USING <replaceable class="PARAMETER">source-query</replaceable>
+ON <replaceable class="PARAMETER">join_condition</replaceable>
+[<replaceable class="PARAMETER">when_clause</replaceable> [...]]
+
+where <replaceable class="PARAMETER">when_clause</replaceable> is
+
+{ WHEN MATCHED [ AND <replaceable class="PARAMETER">condition</replaceable> ] THEN { <replaceable class="PARAMETER">merge_update</replaceable> | DELETE | DO NOTHING | RAISE ERROR}
+ WHEN NOT MATCHED [ AND <replaceable class="PARAMETER">condition</replaceable> ] THEN { <replaceable class="PARAMETER">merge_insert</replaceable> | DO NOTHING | RAISE ERROR} }
+
+where <replaceable class="PARAMETER">merge_update</replaceable> is
+
+UPDATE SET { <replaceable class="PARAMETER">column</replaceable> = { <replaceable class="PARAMETER">expression</replaceable> | DEFAULT } |
+ ( <replaceable class="PARAMETER">column</replaceable> [, ...] ) = ( { <replaceable class="PARAMETER">expression</replaceable> | DEFAULT } [, ...] ) } [, ...]
+
+and <replaceable class="PARAMETER">merge_insert</replaceable> is
+
+INSERT [( <replaceable class="PARAMETER">column</replaceable> [, ...] )] { VALUES ( { <replaceable class="PARAMETER">expression</replaceable> | DEFAULT } [, ...] ) | DEFAULT VALUES }
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>MERGE</command> performs at most one action on each row from
+ the target table, driven by the rows from the source query. This
+ provides a way to specify a single SQL statement that can conditionally
+ <command>UPDATE</command> or <command>INSERT</command> rows, a task
+ that would otherwise require multiple procedural language statements.
+ </para>
+
+ <para>
+ First, the <command>MERGE</command> command performs a left outer join
+ from source query to target table, producing zero or more merged rows. For
+ each merged row, <literal>WHEN</> clauses are evaluated in the
+ specified order until one of them is activated. The corresponding action
+ is then applied and processing continues for the next row.
+ </para>
+
+ <para>
+ <command>MERGE</command> actions have the same effect as
+ regular <command>UPDATE</command>, <command>INSERT</command>, or
+ <command>DELETE</command> commands of the same names, though the syntax
+ is slightly different.
+ </para>
+
+ <para>
+ If no <literal>WHEN</> clause activates then an implicit action of
+ <literal>RAISE ERROR</> is performed for that row. If that
+ implicit action is not desirable an explicit action of
+ <literal>DO NOTHING</> may be specified instead.
+ </para>
+
+ <para>
+ <command>MERGE</command> will only affect rows only in the specified table.
+ </para>
+
+ <para>
+ There is no <literal>RETURNING</> clause with <command>MERGE</command>.
+ </para>
+
+ <para>
+ There is no MERGE privilege.
+ You must have the <literal>UPDATE</literal> privilege on the table
+ if you specify an update action, the <literal>INSERT</literal> privilege if
+ you specify an insert action and/or the <literal>DELETE</literal> privilege
+ if you wish to delete. You will also require the
+ <literal>SELECT</literal> privilege to any table whose values are read
+ in the <replaceable class="parameter">expressions</replaceable> or
+ <replaceable class="parameter">condition</replaceable>.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><replaceable class="PARAMETER">table</replaceable></term>
+ <listitem>
+ <para>
+ The name (optionally schema-qualified) of the table to merge into.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">alias</replaceable></term>
+ <listitem>
+ <para>
+ A substitute name for the target table. When an alias is
+ provided, it completely hides the actual name of the table. For
+ example, given <literal>MERGE foo AS f</>, the remainder of the
+ <command>MERGE</command> statement must refer to this table as
+ <literal>f</> not <literal>foo</>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="PARAMETER">source-query</replaceable></term>
+ <listitem>
+ <para>
+ A query (<command>SELECT</command> statement or <command>VALUES</command>
+ statement) that supplies the rows to be merged into the target table.
+ Refer to the <xref linkend="sql-select">
+ statement or <xref linkend="sql-values">
+ statement for a description of the syntax.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="PARAMETER">join_condition</replaceable></term>
+ <listitem>
+ <para>
+ <replaceable class="parameter">join_condition</replaceable> is
+ an expression resulting in a value of type
+ <type>boolean</type> (similar to a <literal>WHERE</literal>
+ clause) that specifies which rows in the join are considered to
+ match. You should ensure that the join produces at most one output
+ row for each row to be modified. An attempt to modify any row of the
+ target table more than once will result in an error. This behaviour
+ requires the user to take greater care in using <command>MERGE</command>,
+ though is required explicitly by the SQL Standard.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="PARAMETER">condition</replaceable></term>
+ <listitem>
+ <para>
+ An expression that returns a value of type <type>boolean</type>.
+ If this expression returns <literal>true</> then the <literal>WHEN</>
+ clause will be activated and the corresponding action will occur for
+ that row.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="PARAMETER">merge_update</replaceable></term>
+ <listitem>
+ <para>
+ The specification of an <literal>UPDATE</> action. Do not include
+ the table name, as you would normally do with an
+ <xref linkend="sql-update"> command.
+ For example, <literal>UPDATE tab SET col = 1</> is invalid. Also,
+ do not include a <literal>WHERE</> clause, since only the current
+ can be updated. For example,
+ <literal>UPDATE SET col = 1 WHERE key = 57</> is invalid.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="PARAMETER">merge_insert</replaceable></term>
+ <listitem>
+ <para>
+ The specification of an <literal>INSERT</> action. Do not include
+ the table name, as you would normally do with an
+ <xref linkend="sql-insert"> command.
+ For example, <literal>INSERT INTO tab VALUES (1, 50)</> is invalid.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="PARAMETER">column</replaceable></term>
+ <listitem>
+ <para>
+ The name of a column in <replaceable
+ class="PARAMETER">table</replaceable>.
+ The column name can be qualified with a subfield name or array
+ subscript, if needed. Do not include the table's name in the
+ specification of a target column — for example,
+ <literal>UPDATE SET tab.col = 1</> is invalid.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="PARAMETER">expression</replaceable></term>
+ <listitem>
+ <para>
+ An expression to assign to the column. The expression can use the
+ old values of this and other columns in the table.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>DEFAULT</literal></term>
+ <listitem>
+ <para>
+ Set the column to its default value (which will be NULL if no
+ specific default expression has been assigned to it).
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </refsect1>
+
+ <refsect1>
+ <title>Outputs</title>
+
+ <para>
+ On successful completion, a <command>MERGE</> command returns a command
+ tag of the form
+<screen>
+MERGE <replaceable class="parameter">total-count</replaceable>
+</screen>
+ The <replaceable class="parameter">total-count</replaceable> is the number
+ of rows changed (either updated, inserted or deleted).
+ If <replaceable class="parameter">total-count</replaceable> is 0, no rows
+ were changed (this is not considered an error).
+ </para>
+
+ <para>
+ The number of rows updated, inserted or deleted is not available as part
+ of the command tag. An optional NOTIFY message can be generated to
+ present this information, if desired.
+<screen>
+NOTIFY: 34 rows processed: 11 updated, 5 deleted, 15 inserted, 3 default inserts, 0 no action
+</screen>
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>Notes</title>
+
+ <para>
+ What essentially happens is that the target table is left outer-joined to
+ the tables mentioned in the <replaceable>source-query</replaceable>, and
+ each output row of the join may then activate at most one when-clause.
+ The row will be matched only once per statement, so the status of
+ <literal>MATCHED</> or <literal>NOT MATCHED</> cannot change once testing
+ of <literal>WHEN</> clauses has begun. <command>MERGE</command> will not
+ invoke Rules.
+ </para>
+
+ <para>
+ The following steps take place during the execution of
+ <command>MERGE</command>.
+ <orderedlist>
+ <listitem>
+ <para>
+ Perform any BEFORE STATEMENT triggers for actions specified, whether or
+ not they actually occur.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Perform left outer join from source to target table. Then for each row:
+ <orderedlist>
+ <listitem>
+ <para>
+ Evaluate whether each row is MATCHED or NOT MATCHED.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Test each WHEN condition in the order specified until one activates.
+ Identify the action and its event type.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Perform any BEFORE ROW triggers that fire for the action's event type.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Apply the action specified.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Perform any AFTER ROW triggers that fire for the action's event type.
+ </para>
+ </listitem>
+ </orderedlist>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Perform any AFTER STATEMENT triggers for actions specified, whether or
+ not they actually occur.
+ </para>
+ </listitem>
+ </orderedlist>
+ In summary, statement triggers for an event type (say, INSERT) will
+ be fired whenever we <emphasis>specify</> an action of that kind. Row-level
+ triggers will fire only for event type <emphasis>activated</>.
+ So a <command>MERGE</command> might fire statement triggers for both
+ <literal>UPDATE</> and <literal>INSERT</>, even though only
+ <literal>UPDATE</> row triggers were fired.
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>Examples</title>
+
+ <para>
+ Attempt to insert a new stock item along with the quantity of stock. If
+ the item already exists, instead update the stock count of the existing
+ item.
+<programlisting>
+MERGE INTO wines w
+USING (VALUES('Chateau Lafite 2003', '24')) v
+ON v.column1 = w.winename
+WHEN NOT MATCHED THEN
+ INSERT VALUES(v.column1, v.column2)
+WHEN MATCHED THEN
+ UPDATE SET stock = stock + v.column2;
+</programlisting>
+ </para>
+
+ <para>
+ Perform maintenance on CustomerAccounts based upon new Transactions.
+ The following statement will fail if any accounts have had more than
+ one transaction
+
+<programlisting>
+MERGE CustomerAccount CA
+
+USING (SELECT CustomerId, TransactionValue,
+ FROM Transactions
+ WHERE TransactionId > 35345678) AS T
+
+ON T.CustomerId = CA.CustomerId
+
+WHEN MATCHED THEN
+ UPDATE SET Balance = Balance - TransactionValue
+
+WHEN NOT MATCHED THEN
+ INSERT (CustomerId, Balance)
+ VALUES (T.CustomerId, T.TransactionValue)
+;
+</programlisting>
+
+ so the right way to do this is to pre-aggregate the data
+
+<programlisting>
+MERGE CustomerAccount CA
+
+USING (SELECT CustomerId, Sum(TransactionValue) As TransactionSum
+ FROM Transactions
+ WHERE TransactionId > 35345678
+ GROUP BY CustomerId) AS T
+
+ON T.CustomerId = CA.CustomerId
+
+WHEN MATCHED THEN
+ UPDATE SET Balance = Balance - TransactionSum
+
+WHEN NOT MATCHED THEN
+ INSERT (CustomerId, Balance)
+ VALUES (T.CustomerId, T.TransactionSum)
+;
+</programlisting>
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>Compatibility</title>
+
+ <para>
+ This command conforms to the <acronym>SQL</acronym> standard, except
+ that the <literal>DELETE</literal> and <literal>DO NOTHING</> actions
+ are <productname>PostgreSQL</productname> extensions.
+ </para>
+
+ <para>
+ According to the standard, the column-list syntax for an <literal>UPDATE</>
+ action should allow a list of columns to be assigned from a single
+ row-valued expression.
+ This is not currently implemented — the source must be a list
+ of independent expressions.
+ </para>
+ </refsect1>
+</refentry>
diff --git a/doc/src/sgml/ref/update.sgml b/doc/src/sgml/ref/update.sgml
index 5268794..677390c 100644
--- a/doc/src/sgml/ref/update.sgml
+++ b/doc/src/sgml/ref/update.sgml
@@ -327,6 +327,9 @@ UPDATE wines SET stock = stock + 24 WHERE winename = 'Chateau Lafite 2003';
-- continue with other operations, and eventually
COMMIT;
</programlisting>
+
+ This operation can be executed in a single statement using
+ <xref linkend="sql-merge" endterm="sql-merge-title">.
</para>
<para>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index c33d883..5068235 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -147,6 +147,7 @@
&listen;
&load;
&lock;
+ &merge;
&move;
¬ify;
&prepare;
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index b776ad1..518ca3d 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -73,6 +73,8 @@ static void show_sort_keys(SortState *sortstate, List *ancestors,
static void show_sort_info(SortState *sortstate, ExplainState *es);
static void show_hash_info(HashState *hashstate, ExplainState *es);
static const char *explain_get_index_name(Oid indexId);
+static void ExplainMergeActions(ModifyTableState *mt_planstate,
+ List *ancestors, ExplainState *es);
static void ExplainScanTarget(Scan *plan, ExplainState *es);
static void ExplainMemberNodes(List *plans, PlanState **planstates,
List *ancestors, ExplainState *es);
@@ -636,6 +638,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
case CMD_DELETE:
pname = operation = "Delete";
break;
+ case CMD_MERGE:
+ pname = operation = "Merge";
+ break;
default:
pname = "???";
break;
@@ -1185,6 +1190,11 @@ ExplainNode(PlanState *planstate, List *ancestors,
if (innerPlanState(planstate))
ExplainNode(innerPlanState(planstate), ancestors,
"Inner", NULL, es);
+
+ if (IsA(plan, ModifyTable) &&
+ ((ModifyTable *)plan)->operation == CMD_MERGE)
+ ExplainMergeActions((ModifyTableState *) planstate,
+ ancestors, es);
/* special child plans */
switch (nodeTag(plan))
@@ -1482,6 +1492,92 @@ explain_get_index_name(Oid indexId)
return result;
}
+static void
+ExplainMergeActions(ModifyTableState *mt_planstate, List *ancestors,
+ ExplainState *es)
+{
+ ListCell *l;
+ int actno = 1;
+ StringInfo buf = makeStringInfo();
+ StringInfo acttitle = makeStringInfo();
+ MergeActionSet *actset;
+
+ if (mt_planstate->operation != CMD_MERGE ||
+ mt_planstate->mt_mergeActPstates == NIL)
+ return;
+
+ actset = mt_planstate->mt_mergeActPstates[0];
+
+ foreach(l, actset->actions)
+ {
+ ModifyTableState *mt_state = (ModifyTableState *) lfirst(l);
+ MergeActionState *act_pstate = (MergeActionState *) mt_state->mt_plans[0];
+ MergeAction *act_plan = (MergeAction *) act_pstate->ps.plan;
+
+ /*prepare the title*/
+ resetStringInfo(acttitle);
+ appendStringInfo(acttitle, "Action %d", actno);
+
+ /*prepare the string for printing*/
+ resetStringInfo(buf);
+ switch(act_pstate->operation)
+ {
+ case CMD_INSERT:
+ appendStringInfoString(buf, "Insert When ");
+ break;
+ case CMD_UPDATE:
+ appendStringInfoString(buf, "Update When ");
+ break;
+ case CMD_DELETE:
+ appendStringInfoString(buf, "Delete When ");
+ break;
+ case CMD_DONOTHING:
+ appendStringInfoString(buf, "Do Nothing When ");
+ break;
+ case CMD_RAISEERR:
+ appendStringInfoString(buf, "Raise Error When ");
+ break;
+ default:
+ elog(ERROR, "unknown merge action");
+ }
+
+ if (act_plan->matched)
+ appendStringInfoString(buf, "Matched ");
+ else
+ appendStringInfoString(buf, "Not Mactched ");
+
+ if (act_plan->flattenedqual)
+ appendStringInfoString(buf, "And ");
+
+ /*print the action type*/
+ ExplainPropertyText(acttitle->data, buf->data, es);
+
+ /*print the action qual*/
+ show_qual(act_plan->flattenedqual, "Qual",
+ &act_pstate->ps, ancestors, true, es);
+
+ /*print the target list of action*/
+ if (es->verbose &&
+ (act_plan->operation == CMD_INSERT ||
+ act_plan->operation == CMD_UPDATE))
+ {
+ List *orignialtlist;
+
+ orignialtlist = act_plan->plan.targetlist;
+ act_plan->plan.targetlist = act_plan->flattenedtlist;
+ show_plan_tlist((PlanState *) act_pstate, ancestors, es);
+ act_plan->plan.targetlist = orignialtlist;
+ }
+
+ if (act_pstate->ps.subPlan)
+ ExplainSubPlans(act_pstate->ps.subPlan, ancestors, "SubPlan", es);
+
+ actno++;
+ }
+
+ ExplainPropertyText("MainPlan", "", es);
+}
+
/*
* Show the target of a Scan node
*/
diff --git a/src/backend/commands/trigger.c b/src/backend/commands/trigger.c
index 8b017ae..f4a7b74 100644
--- a/src/backend/commands/trigger.c
+++ b/src/backend/commands/trigger.c
@@ -2342,6 +2342,87 @@ ExecASTruncateTriggers(EState *estate, ResultRelInfo *relinfo)
false, NULL, NULL, NIL, NULL);
}
+void
+ExecBSMergeTriggers(ModifyTableState *mt_state)
+{
+ ListCell *l;
+ MergeActionSet *actset;
+ bool doUpdateTriggers = false;
+ bool doInsertTriggers = false;
+ bool doDeleteTriggers = false;
+
+ /* Scan the actions to see what kind of statements there is */
+ actset = mt_state->mt_mergeActPstates[0];
+ foreach(l, actset->actions)
+ {
+ ModifyTableState *actmtstate;
+ MergeActionState *actPstate;
+ MergeAction *actplan;
+
+ actmtstate = (ModifyTableState *) lfirst(l);
+ actPstate = (MergeActionState *) actmtstate->mt_plans[0];
+ actplan = (MergeAction *) actPstate->ps.plan;
+
+ if (actplan->operation == CMD_UPDATE)
+ doUpdateTriggers = true;
+ else if (actplan->operation == CMD_INSERT)
+ doInsertTriggers = true;
+ else if (actplan->operation == CMD_DELETE)
+ doDeleteTriggers = true;
+ }
+
+ /* And fire the triggers */
+ if (doUpdateTriggers)
+ ExecBSUpdateTriggers(mt_state->ps.state,
+ mt_state->ps.state->es_result_relations);
+ if (doInsertTriggers)
+ ExecBSInsertTriggers(mt_state->ps.state,
+ mt_state->ps.state->es_result_relations);
+ if (doDeleteTriggers)
+ ExecBSDeleteTriggers(mt_state->ps.state,
+ mt_state->ps.state->es_result_relations);
+}
+
+void
+ExecASMergeTriggers(ModifyTableState *mt_state)
+{
+ ListCell *l;
+ MergeActionSet *actset;
+ bool doUpdateTriggers = false;
+ bool doInsertTriggers = false;
+ bool doDeleteTriggers = false;
+
+ /* Scan the actions to see what kind of statements there is */
+ actset = mt_state->mt_mergeActPstates[0];
+ foreach(l, actset->actions)
+ {
+ ModifyTableState *actmtstate;
+ MergeActionState *actPstate;
+ MergeAction *actplan;
+
+ actmtstate = (ModifyTableState *)lfirst(l);
+ actPstate = (MergeActionState *)actmtstate->mt_plans[0];
+ actplan = (MergeAction *)actPstate->ps.plan;
+
+ if(actplan->operation == CMD_UPDATE)
+ doUpdateTriggers = true;
+ else if(actplan->operation == CMD_INSERT)
+ doInsertTriggers = true;
+ else if(actplan->operation == CMD_DELETE)
+ doDeleteTriggers = true;
+ }
+
+ /* And fire the triggers */
+ if (doUpdateTriggers)
+ ExecASUpdateTriggers(mt_state->ps.state,
+ mt_state->ps.state->es_result_relations);
+ if (doInsertTriggers)
+ ExecASInsertTriggers(mt_state->ps.state,
+ mt_state->ps.state->es_result_relations);
+ if (doDeleteTriggers)
+ ExecASDeleteTriggers(mt_state->ps.state,
+ mt_state->ps.state->es_result_relations);
+}
static HeapTuple
GetTupleForTrigger(EState *estate,
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index b34a154..2b7ceb1 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -170,6 +170,7 @@ standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
case CMD_INSERT:
case CMD_DELETE:
case CMD_UPDATE:
+ case CMD_MERGE:
estate->es_output_cid = GetCurrentCommandId(true);
break;
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index f4cc7d9..ff691c7 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -153,6 +153,11 @@ ExecInitNode(Plan *node, EState *estate, int eflags)
estate, eflags);
break;
+ case T_MergeAction:
+ result = (PlanState *) ExecInitMergeAction((MergeAction *) node,
+ estate, eflags);
+ break;
+
case T_Append:
result = (PlanState *) ExecInitAppend((Append *) node,
estate, eflags);
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 8619ce3..e3ac758 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -582,6 +582,113 @@ lreplace:;
return NULL;
}
+static TupleTableSlot *
+MergeRaiseErr(void)
+{
+ elog(NOTICE, "one tuple is ERROR");
+ return NULL;
+}
+
+static TupleTableSlot *
+ExecMerge(ItemPointer tupleid,
+ TupleTableSlot *slot,
+ TupleTableSlot *planSlot,
+ MergeActionSet *actset,
+ EState *estate)
+{
+
+ TupleTableSlot *actslot = NULL;
+ ListCell *each;
+
+ /*
+ * Try the merge actions one by one until we have a match.
+ */
+ foreach(each, actset->actions)
+ {
+ ModifyTableState *mt_pstate;
+ MergeActionState *action_pstate;
+ ExprContext *econtext;
+ bool matched;
+
+ mt_pstate = (ModifyTableState *) lfirst(each);
+ Assert(IsA(mt_pstate, ModifyTableState));
+
+ /*
+ * mt_pstate is supposed to have only ONE mt_plans,
+ * which is a MergeActionState
+ */
+ action_pstate = (MergeActionState *) mt_pstate->mt_plans[0];
+ matched = ((MergeAction *)action_pstate->ps.plan)->matched;
+
+ /*
+ * If tupleid == NULL, it is a NOT MATCHED case,
+ * else, it is a MATCHED case,
+ */
+ if ((tupleid == NULL && matched) ||
+ (tupleid != NULL && !matched))
+ continue;
+
+ /* Setup the expression context. */
+ econtext = action_pstate->ps.ps_ExprContext;
+
+ /*
+ * Check that additional quals match, if any.
+ */
+ if (action_pstate->ps.qual)
+ {
+ ResetExprContext(econtext);
+
+ econtext->ecxt_scantuple = slot;
+ econtext->ecxt_outertuple = planSlot;
+
+ if (!ExecQual(action_pstate->ps.qual, econtext, false))
+ continue;
+ }
+
+ /* Ok, we have a match. Perform the action */
+
+ /* First project any RETURNING result tuple slot, if needed */
+ if (action_pstate->operation == CMD_INSERT ||
+ action_pstate->operation == CMD_UPDATE)
+ actslot = ExecProcessReturning(action_pstate->ps.ps_ProjInfo,
+ slot, planSlot);
+
+ switch (action_pstate->operation)
+ {
+ case CMD_INSERT:
+ return ExecInsert(actslot, planSlot, estate);
+
+ case CMD_UPDATE:
+ return ExecUpdate(tupleid,
+ actslot,
+ planSlot,
+ &mt_pstate->mt_epqstate,
+ estate);
+
+ case CMD_DELETE:
+ return ExecDelete(tupleid,
+ planSlot,
+ &mt_pstate->mt_epqstate,
+ estate);
+
+ case CMD_DONOTHING:
+ return NULL;
+
+ case CMD_RAISEERR:
+ return MergeRaiseErr();
+
+ default:
+ elog(ERROR, "unknown merge action type for excute");
+ break;
+ }
+ }
+
+ /*
+ * No matching action found. Perform the default action, which is
+ * RAISE ERROR.
+ */
+ return MergeRaiseErr();
+}
/*
* Process BEFORE EACH STATEMENT triggers
@@ -603,6 +710,9 @@ fireBSTriggers(ModifyTableState *node)
ExecBSDeleteTriggers(node->ps.state,
node->ps.state->es_result_relations);
break;
+ case CMD_MERGE:
+ ExecBSMergeTriggers(node);
+ break;
default:
elog(ERROR, "unknown operation");
break;
@@ -629,6 +739,9 @@ fireASTriggers(ModifyTableState *node)
ExecASDeleteTriggers(node->ps.state,
node->ps.state->es_result_relations);
break;
+ case CMD_MERGE:
+ ExecASMergeTriggers(node);
+ break;
default:
elog(ERROR, "unknown operation");
break;
@@ -654,6 +767,7 @@ ExecModifyTable(ModifyTableState *node)
TupleTableSlot *planSlot;
ItemPointer tupleid = NULL;
ItemPointerData tuple_ctid;
+ MergeActionSet *mergeActSet;
/*
* On first call, fire BEFORE STATEMENT triggers before proceeding.
@@ -675,6 +789,8 @@ ExecModifyTable(ModifyTableState *node)
/* Preload local variables */
subplanstate = node->mt_plans[node->mt_whichplan];
junkfilter = estate->es_result_relation_info->ri_junkFilter;
+ if(node->mt_mergeActPstates)
+ mergeActSet = node->mt_mergeActPstates[node->mt_whichplan];
/*
* Fetch rows from subplan(s), and execute the required table modification
@@ -693,6 +809,8 @@ ExecModifyTable(ModifyTableState *node)
estate->es_result_relation_info++;
subplanstate = node->mt_plans[node->mt_whichplan];
junkfilter = estate->es_result_relation_info->ri_junkFilter;
+ if(node->mt_mergeActPstates)
+ mergeActSet = node->mt_mergeActPstates[node->mt_whichplan];
EvalPlanQualSetPlan(&node->mt_epqstate, subplanstate->plan);
continue;
}
@@ -708,20 +826,32 @@ ExecModifyTable(ModifyTableState *node)
/*
* extract the 'ctid' junk attribute.
*/
- if (operation == CMD_UPDATE || operation == CMD_DELETE)
+ if (operation == CMD_UPDATE || operation == CMD_DELETE || operation == CMD_MERGE)
{
Datum datum;
bool isNull;
datum = ExecGetJunkAttribute(slot, junkfilter->jf_junkAttNo,
&isNull);
- /* shouldn't ever get a null result... */
+
if (isNull)
- elog(ERROR, "ctid is NULL");
+ {
+ /*
+ * Shouldn't ever get a null result for UPDATE or DELETE.
+ * MERGE gets a null ctid in "NOT MATCHED" case
+ */
+ if (operation != CMD_MERGE)
+ elog(ERROR, "ctid is NULL");
+ else
+ tupleid = NULL;
+ }
+ else
+ {
- tupleid = (ItemPointer) DatumGetPointer(datum);
- tuple_ctid = *tupleid; /* be sure we don't free the ctid!! */
- tupleid = &tuple_ctid;
+ tupleid = (ItemPointer) DatumGetPointer(datum);
+ tuple_ctid = *tupleid; /* be sure we don't free the ctid!! */
+ tupleid = &tuple_ctid;
+ }
}
/*
@@ -744,6 +874,10 @@ ExecModifyTable(ModifyTableState *node)
slot = ExecDelete(tupleid, planSlot,
&node->mt_epqstate, estate);
break;
+ case CMD_MERGE:
+ slot = ExecMerge(tupleid, slot, planSlot,
+ mergeActSet, estate);
+ break;
default:
elog(ERROR, "unknown operation");
break;
@@ -771,6 +905,69 @@ ExecModifyTable(ModifyTableState *node)
return NULL;
}
+/*
+ * When init a merge plan, we also need init its action plans.
+ * These action plans are "MergeAction" plans.
+ *
+ * This function mainly handles the tlist and qual in the plan.
+ * The returning result is a "MergeActionState".
+ */
+MergeActionState *
+ExecInitMergeAction(MergeAction *node, EState *estate, int eflags)
+{
+ MergeActionState *result;
+
+ /*
+ * do nothing when we get to the end of a leaf on tree.
+ */
+ if (node == NULL)
+ return NULL;
+
+ /*
+ * create state structure
+ */
+ result = makeNode(MergeActionState);
+ result->operation = node->operation;
+ result->ps.plan = (Plan *)node;
+ result->ps.state = estate;
+
+ /*
+ * tuple table initialization
+ */
+ ExecInitResultTupleSlot(estate, &result->ps);
+
+ /*
+ * initialize tuple type
+ */
+ ExecAssignResultTypeFromTL(&result->ps);
+
+ /*
+ * create expression context for node
+ */
+ ExecAssignExprContext(estate, &result->ps);
+
+ /*
+ * initialize child expressions
+ */
+ result->ps.targetlist = (List *)
+ ExecInitExpr((Expr *) node->plan.targetlist, &result->ps);
+
+ result->ps.qual = (List *)
+ ExecInitExpr((Expr *) node->plan.qual, &result->ps);
+
+ /*
+ * init the projection information
+ */
+ ExecAssignProjectionInfo(&result->ps, NULL);
+
+ /*
+ * XXX: do we need a check for the plan output here ?
+ * (by calling the ExecCheckPlanOutput() function
+ */
+
+ return result;
+}
+
/* ----------------------------------------------------------------
* ExecInitModifyTable
* ----------------------------------------------------------------
@@ -786,6 +983,7 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
Plan *subplan;
ListCell *l;
int i;
+ bool isMergeAction = false;
/* check for unsupported flags */
Assert(!(eflags & (EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK)));
@@ -826,6 +1024,16 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
foreach(l, node->plans)
{
subplan = (Plan *) lfirst(l);
+
+ /*
+ * test if this subplan node is a MergeAction.
+ * We need this information for setting the junkfilter.
+ * junkfilter is necessary for an ordinary UPDATE/DELETE plan,
+ * but not for an UPDATE/DELETE merge action
+ */
+ if (IsA(subplan, MergeAction))
+ isMergeAction = true;
+
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
estate->es_result_relation_info++;
i++;
@@ -955,7 +1163,12 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
break;
case CMD_UPDATE:
case CMD_DELETE:
- junk_filter_needed = true;
+ case CMD_MERGE:
+ if(!isMergeAction)
+ junk_filter_needed = true;
+ break;
+ case CMD_DONOTHING:
+ case CMD_RAISEERR:
break;
default:
elog(ERROR, "unknown operation");
@@ -978,9 +1191,10 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
resultRelInfo->ri_RelationDesc->rd_att->tdhasoid,
ExecInitExtraTupleSlot(estate));
- if (operation == CMD_UPDATE || operation == CMD_DELETE)
+ if (operation == CMD_UPDATE || operation == CMD_DELETE ||
+ operation == CMD_MERGE)
{
- /* For UPDATE/DELETE, find the ctid junk attr now */
+ /* For UPDATE/DELETE/MERGE, find the ctid junk attr now */
j->jf_junkAttNo = ExecFindJunkAttribute(j, "ctid");
if (!AttributeNumberIsValid(j->jf_junkAttNo))
elog(ERROR, "could not find junk ctid column");
@@ -1006,6 +1220,36 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
if (estate->es_trig_tuple_slot == NULL)
estate->es_trig_tuple_slot = ExecInitExtraTupleSlot(estate);
+ /*
+ * for the merge actions, we need to do similar things as above.
+ *Each action in each action set should be init by the ExecInitNode.
+ *The returned planstates will take the place of the orignial plan nodes.
+ *These new action set will be put in the mt_mergeActPstates array.
+ */
+ if(node->operation == CMD_MERGE)
+ {
+ /*we have one action set for each result relation (main plan)*/
+ mtstate->mt_mergeActPstates =
+ (MergeActionSet **) palloc0(sizeof(MergeActionSet*) * nplans);
+
+ estate->es_result_relation_info = estate->es_result_relations;
+ i = 0;
+ foreach(l, node->mergeActPlan)
+ {
+ ListCell *e;
+ MergeActionSet *actset = (MergeActionSet *) lfirst(l);
+
+ foreach(e, actset->actions)
+ {
+ lfirst(e) = ExecInitNode((Plan *)lfirst(e), estate, 0);
+ }
+ mtstate->mt_mergeActPstates[i] = actset;
+ estate->es_result_relation_info++;
+ i++;
+ }
+ estate->es_result_relation_info = NULL;
+ }
+
return mtstate;
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 69262d6..cd117d6 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -176,6 +176,42 @@ _copyModifyTable(ModifyTable *from)
COPY_NODE_FIELD(returningLists);
COPY_NODE_FIELD(rowMarks);
COPY_SCALAR_FIELD(epqParam);
+ COPY_NODE_FIELD(mergeActPlan);
+
+ return newnode;
+}
+
+/*
+ * _copyMergeAction
+ */
+static MergeAction *
+_copyMergeAction(MergeAction *from)
+{
+ MergeAction *newnode = makeNode(MergeAction);
+
+ /*
+ * copy node superclass fields
+ */
+ CopyPlanFields((Plan *) from, (Plan *) newnode);
+
+ COPY_SCALAR_FIELD(operation);
+ COPY_SCALAR_FIELD(matched);
+ COPY_NODE_FIELD(flattenedqual);
+ COPY_NODE_FIELD(flattenedtlist);
+
+ return newnode;
+}
+
+/*
+ * _copyMergeActionSet
+ */
+static MergeActionSet *
+_copyMergeActionSet(MergeActionSet *from)
+{
+ MergeActionSet *newnode = makeNode(MergeActionSet);
+
+ COPY_SCALAR_FIELD(result_relation);
+ COPY_NODE_FIELD(actions);
return newnode;
}
@@ -2273,6 +2309,10 @@ _copyQuery(Query *from)
COPY_NODE_FIELD(rowMarks);
COPY_NODE_FIELD(setOperations);
COPY_NODE_FIELD(constraintDeps);
+ COPY_SCALAR_FIELD(isMergeAction);
+ COPY_SCALAR_FIELD(matched);
+ COPY_SCALAR_FIELD(sourceAttrNo);
+ COPY_NODE_FIELD(mergeActQry);
return newnode;
}
@@ -2282,6 +2322,7 @@ _copyInsertStmt(InsertStmt *from)
{
InsertStmt *newnode = makeNode(InsertStmt);
+ COPY_SCALAR_FIELD(isMergeAction);
COPY_NODE_FIELD(relation);
COPY_NODE_FIELD(cols);
COPY_NODE_FIELD(selectStmt);
@@ -2295,6 +2336,7 @@ _copyDeleteStmt(DeleteStmt *from)
{
DeleteStmt *newnode = makeNode(DeleteStmt);
+ COPY_SCALAR_FIELD(isMergeAction);
COPY_NODE_FIELD(relation);
COPY_NODE_FIELD(usingClause);
COPY_NODE_FIELD(whereClause);
@@ -2308,6 +2350,7 @@ _copyUpdateStmt(UpdateStmt *from)
{
UpdateStmt *newnode = makeNode(UpdateStmt);
+ COPY_SCALAR_FIELD(isMergeAction);
COPY_NODE_FIELD(relation);
COPY_NODE_FIELD(targetList);
COPY_NODE_FIELD(whereClause);
@@ -2344,6 +2387,47 @@ _copySelectStmt(SelectStmt *from)
return newnode;
}
+static MergeStmt *
+_copyMergeStmt(MergeStmt *from)
+{
+ MergeStmt *newnode = makeNode(MergeStmt);
+
+ COPY_NODE_FIELD(relation);
+ COPY_NODE_FIELD(source);
+ COPY_NODE_FIELD(matchCondition);
+ COPY_NODE_FIELD(actions);
+
+ return newnode;
+}
+
+static MergeConditionAction *
+_copyMergeConditionAction(MergeConditionAction *from)
+{
+ MergeConditionAction *newnode = makeNode(MergeConditionAction);
+
+ COPY_SCALAR_FIELD(match);
+ COPY_NODE_FIELD(condition);
+ COPY_NODE_FIELD(action);
+
+ return newnode;
+}
+
+static MergeDoNothing *
+_copyMergeDoNothing(MergeDoNothing *from)
+{
+ MergeDoNothing *newnode = makeNode(MergeDoNothing);
+
+ return newnode;
+}
+
+static MergeError*
+_copyMergeError(MergeError *from)
+{
+ MergeError *newnode = makeNode(MergeError);
+
+ return newnode;
+}
+
static SetOperationStmt *
_copySetOperationStmt(SetOperationStmt *from)
{
@@ -3605,6 +3689,12 @@ copyObject(void *from)
case T_ModifyTable:
retval = _copyModifyTable(from);
break;
+ case T_MergeAction:
+ retval = _copyMergeAction(from);
+ break;
+ case T_MergeActionSet:
+ retval = _copyMergeActionSet(from);
+ break;
case T_Append:
retval = _copyAppend(from);
break;
@@ -3905,6 +3995,18 @@ copyObject(void *from)
case T_SelectStmt:
retval = _copySelectStmt(from);
break;
+ case T_MergeStmt:
+ retval = _copyMergeStmt(from);
+ break;
+ case T_MergeConditionAction:
+ retval = _copyMergeConditionAction(from);
+ break;
+ case T_MergeDoNothing:
+ retval = _copyMergeDoNothing(from);
+ break;
+ case T_MergeError:
+ retval = _copyMergeError(from);
+ break;
case T_SetOperationStmt:
retval = _copySetOperationStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 667057b..6337986 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -878,6 +878,10 @@ _equalQuery(Query *a, Query *b)
COMPARE_NODE_FIELD(rowMarks);
COMPARE_NODE_FIELD(setOperations);
COMPARE_NODE_FIELD(constraintDeps);
+ COMPARE_SCALAR_FIELD(isMergeAction);
+ COMPARE_SCALAR_FIELD(matched);
+ COMPARE_SCALAR_FIELD(sourceAttrNo);
+ COMPARE_NODE_FIELD(mergeActQry);
return true;
}
@@ -885,6 +889,7 @@ _equalQuery(Query *a, Query *b)
static bool
_equalInsertStmt(InsertStmt *a, InsertStmt *b)
{
+ COMPARE_SCALAR_FIELD(isMergeAction);
COMPARE_NODE_FIELD(relation);
COMPARE_NODE_FIELD(cols);
COMPARE_NODE_FIELD(selectStmt);
@@ -896,6 +901,7 @@ _equalInsertStmt(InsertStmt *a, InsertStmt *b)
static bool
_equalDeleteStmt(DeleteStmt *a, DeleteStmt *b)
{
+ COMPARE_SCALAR_FIELD(isMergeAction);
COMPARE_NODE_FIELD(relation);
COMPARE_NODE_FIELD(usingClause);
COMPARE_NODE_FIELD(whereClause);
@@ -907,6 +913,7 @@ _equalDeleteStmt(DeleteStmt *a, DeleteStmt *b)
static bool
_equalUpdateStmt(UpdateStmt *a, UpdateStmt *b)
{
+ COMPARE_SCALAR_FIELD(isMergeAction);
COMPARE_NODE_FIELD(relation);
COMPARE_NODE_FIELD(targetList);
COMPARE_NODE_FIELD(whereClause);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 04a6647..e784f08 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -331,6 +331,7 @@ _outModifyTable(StringInfo str, ModifyTable *node)
WRITE_NODE_FIELD(returningLists);
WRITE_NODE_FIELD(rowMarks);
WRITE_INT_FIELD(epqParam);
+ WRITE_NODE_FIELD(mergeActPlan);
}
static void
@@ -2021,6 +2022,53 @@ _outQuery(StringInfo str, Query *node)
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(setOperations);
WRITE_NODE_FIELD(constraintDeps);
+ WRITE_BOOL_FIELD(isMergeAction);
+ WRITE_BOOL_FIELD(matched);
+ WRITE_INT_FIELD(sourceAttrNo);
+ WRITE_NODE_FIELD(mergeActQry);
+}
+
+static void
+_outMergeConditionAction(StringInfo str, MergeConditionAction *node)
+{
+ WRITE_NODE_TYPE("MERGECONDITIONACTION");
+
+ WRITE_BOOL_FIELD(match);
+ WRITE_NODE_FIELD(condition);
+ WRITE_NODE_FIELD(action);
+}
+
+static void
+_outMergeStmt(StringInfo str, MergeStmt *node)
+{
+ WRITE_NODE_TYPE("MERGESTMT");
+
+ WRITE_NODE_FIELD(relation);
+ WRITE_NODE_FIELD(source);
+ WRITE_NODE_FIELD(matchCondition);
+ WRITE_NODE_FIELD(actions);
+}
+
+static void
+_outMergeAction(StringInfo str, MergeAction *node)
+{
+ WRITE_NODE_TYPE("MERGEACTION");
+
+ _outPlanInfo(str, (Plan *)node);
+
+ WRITE_ENUM_FIELD(operation, CmdType);
+ WRITE_BOOL_FIELD(matched);
+ WRITE_NODE_FIELD(flattenedqual);
+ WRITE_NODE_FIELD(flattenedtlist);
+}
+
+static void
+_outMergeActionSet(StringInfo str, MergeActionSet *node)
+{
+ WRITE_NODE_TYPE("MERGEACTIONSET");
+
+ WRITE_INT_FIELD(result_relation);
+ WRITE_NODE_FIELD(actions);
}
static void
@@ -2906,6 +2954,18 @@ _outNode(StringInfo str, void *obj)
case T_XmlSerialize:
_outXmlSerialize(str, obj);
break;
+ case T_MergeAction:
+ _outMergeAction(str, obj);
+ break;
+ case T_MergeActionSet:
+ _outMergeActionSet(str, obj);
+ break;
+ case T_MergeStmt:
+ _outMergeStmt(str, obj);
+ break;
+ case T_MergeConditionAction:
+ _outMergeConditionAction(str,obj);
+ break;
default:
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 0a2edcb..0aeab17 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -219,6 +219,10 @@ _readQuery(void)
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(setOperations);
READ_NODE_FIELD(constraintDeps);
+ READ_BOOL_FIELD(isMergeAction);
+ READ_BOOL_FIELD(matched);
+ READ_INT_FIELD(sourceAttrNo);
+ READ_NODE_FIELD(mergeActQry);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 2115cc0..f5df243 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -3919,7 +3919,7 @@ make_result(PlannerInfo *root,
ModifyTable *
make_modifytable(CmdType operation, List *resultRelations,
List *subplans, List *returningLists,
- List *rowMarks, int epqParam)
+ List *rowMarks, List *mergeActPlans, int epqParam)
{
ModifyTable *node = makeNode(ModifyTable);
Plan *plan = &node->plan;
@@ -3972,6 +3972,8 @@ make_modifytable(CmdType operation, List *resultRelations,
node->returningLists = returningLists;
node->rowMarks = rowMarks;
node->epqParam = epqParam;
+ if (operation == CMD_MERGE)
+ node->mergeActPlan = mergeActPlans;
return node;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 3950ab4..0b5bcad 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -102,7 +102,8 @@ static void get_column_info_for_window(PlannerInfo *root, WindowClause *wc,
int *ordNumCols,
AttrNumber **ordColIdx,
Oid **ordOperators);
-
+static ModifyTable *merge_action_planner(PlannerInfo *root, Plan *mainPlan);
+static MergeActionSet *merge_action_set_planner(PlannerInfo *root, Plan *mainPlan);
/*****************************************************************************
*
@@ -452,6 +453,27 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
}
/*
+ * for MERGE command, we need to preprocess
+ * the expressions for merge actions too.
+ */
+ if(parse->commandType == CMD_MERGE)
+ {
+ ListCell *e;
+
+ foreach(e, parse->mergeActQry)
+ {
+ Query *actqry = (Query *)lfirst(e);
+
+ actqry->targetList = (List *) preprocess_expression(root,
+ (Node *) actqry->targetList,
+ EXPRKIND_TARGET);
+ actqry->jointree->quals = preprocess_expression(root,
+ (Node *) actqry->jointree->quals,
+ EXPRKIND_QUAL);
+ }
+ }
+
+ /*
* In some cases we may want to transfer a HAVING clause into WHERE. We
* cannot do so if the HAVING clause contains aggregates (obviously) or
* volatile functions (since a HAVING clause is supposed to be executed
@@ -528,6 +550,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
{
List *returningLists;
List *rowMarks;
+ List *mergeAction;
/*
* Deal with the RETURNING clause if any. It's convenient to pass
@@ -559,12 +582,19 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
else
rowMarks = root->rowMarks;
+ /*if here is a MERGE command, we need to plan the actions too*/
+ if(parse->commandType == CMD_MERGE)
+ mergeAction = list_make1(merge_action_set_planner(root, plan));
+ else
+ mergeAction = NIL;
+
plan = (Plan *) make_modifytable(parse->commandType,
copyObject(root->resultRelations),
list_make1(plan),
returningLists,
rowMarks,
- SS_assign_special_param(root));
+ mergeAction,
+ SS_assign_special_param(root));
}
}
@@ -585,6 +615,163 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
}
/*
+to generate the merge action set for the main plan.
+Call this function after the grouping_planner()
+
+Only works for MERGE command
+*/
+static MergeActionSet *
+merge_action_set_planner(PlannerInfo *root, Plan *mainPlan)
+{
+ MergeActionSet *result;
+ Query *mainQry = root->parse;
+ ListCell *l;
+ PlannerInfo subroot;
+
+ /*for non-merge command, no need to plan the merge actions*/
+ if(mainQry->commandType != CMD_MERGE ||
+ mainQry->mergeActQry == NIL)
+ return NULL;
+
+ /*do a copy of the root info*/
+ memcpy(&subroot, root, sizeof(PlannerInfo));
+
+ /*create the result node*/
+ result = makeNode(MergeActionSet);
+ result->result_relation = mainQry->resultRelation;
+ result->actions = NIL;
+
+ /*plan the actions one by one*/
+ foreach(l, mainQry->mergeActQry)
+ {
+ ModifyTable *actplan;
+
+ /*put the action query into the subroot*/
+ subroot.parse = (Query *) lfirst(l);
+ actplan = merge_action_planner(&subroot, mainPlan);
+ result->actions = lappend(result->actions, actplan);
+ }
+ return result;
+}
+#if 0
+static void
+merge_action_list_planner(PlannerInfo *root, ModifyTable *mainplan)
+{
+ ListCell *l;
+ Plan *topplan;
+ Query *parse = root->parse;
+ PlannerInfo subroot;
+
+ /* this is a function for MERGE command only */
+ if (parse->commandType != CMD_MERGE ||
+ mainplan->operation != CMD_MERGE)
+ return;
+
+ /* if the merge actions are already there, no need to do it again */
+ if (mainplan->mergeActPlan != NIL)
+ return;
+
+ memcpy(&subroot, root, sizeof(PlannerInfo));
+
+ topplan = (Plan *) linitial(mainplan->plans);
+
+ /* plan each action query */
+ foreach(l, parse->mergeActQry)
+ {
+ Plan *actplan;
+
+ subroot.parse = (Query *) lfirst(l);
+ actplan = (Plan *) merge_action_planner(&subroot, topplan);
+ mainplan->mergeActPlan = lappend(mainplan->mergeActPlan,
+ actplan);
+ }
+}
+#endif
+/* create plan for a single merge action */
+static ModifyTable *
+merge_action_planner(PlannerInfo *root, Plan *mainPlan)
+{
+ Query *parse = root->parse;
+ MergeAction *actplan;
+ ModifyTable *result;
+
+ List *returningLists;
+ List *rowMarks;
+
+ /*
+ * no having clause in a merge action
+ */
+ Assert(parse->havingQual == NULL);
+
+ /*
+ * Create the action plan node
+ */
+ actplan = makeNode(MergeAction);
+ actplan->operation = parse->commandType;
+ actplan->matched = parse->matched;
+
+ /* copy the cost from the top_plan */
+ actplan->plan.startup_cost = mainPlan->startup_cost;
+ actplan->plan.total_cost = mainPlan->total_cost;
+ actplan->plan.plan_rows = mainPlan->plan_rows;
+ actplan->plan.plan_width = mainPlan->plan_width;
+
+ /*
+ * Here, the quals expressions are flattened, which is accepted
+ * by deparse functions in EXPLAIN.
+ * But, these expreesions will be processed by push_up_vars
+ * latterly and become not flat again.
+ * So,we need keep a copy of current quals for explaining.
+ */
+ actplan->flattenedqual = (List *) copyObject(parse->jointree->quals);
+ actplan->plan.qual = (List *)parse->jointree->quals;
+
+ /* prepare the target list */
+ if (parse->targetList)
+ {
+ actplan->plan.targetlist = preprocess_targetlist(root,
+ parse->targetList);
+ /*the target list should also be copied for EXPLAIN*/
+ actplan->flattenedtlist = (List *) copyObject(actplan->plan.targetlist);
+ }
+
+ /*
+ *In general situation, all the vars in target list and quals are flattened.
+ *But, we want them to point to the attributes of the top join plan, not to
+ *the subplans. So push them up again here.
+ */
+ push_up_merge_action_vars(actplan, parse);
+
+ if (parse->returningList)
+ {
+ List *rlist;
+
+ Assert(parse->resultRelation);
+ rlist = set_returning_clause_references(root->glob,
+ parse->returningList,
+ &actplan->plan,
+ parse->resultRelation);
+ returningLists = list_make1(rlist);
+ }
+ else
+ returningLists = NIL;
+
+ if (parse->rowMarks)
+ rowMarks = NIL;
+ else
+ rowMarks = root->rowMarks;
+
+ result = make_modifytable(parse->commandType,
+ list_make1_int(parse->resultRelation),
+ list_make1(actplan),
+ returningLists,
+ rowMarks,
+ NIL,
+ SS_assign_special_param(root));
+ return result;
+}
+
+/*
* preprocess_expression
* Do subquery_planner's preprocessing work for an expression,
* which can be a targetlist, a WHERE clause (including JOIN/ON
@@ -729,6 +916,7 @@ inheritance_planner(PlannerInfo *root)
List *returningLists = NIL;
List *rtable = NIL;
List *rowMarks;
+ List *mergeActSets = NIL;
List *tlist;
PlannerInfo subroot;
ListCell *l;
@@ -790,6 +978,66 @@ inheritance_planner(PlannerInfo *root)
appinfo->child_relid);
returningLists = lappend(returningLists, rlist);
}
+
+ /*
+ *For a merge command, we need to generate a set of action plans corresponding
+ *to current result relations.
+ *The adjust_appendrel_attrs() will not process the merge action list of
+ *the main query. And, We need to do this here for a MERGE comand.
+ *In fact, no need to do a full query tree mutator on the action queries.
+ *only adjust the target list and quals.
+ */
+ if (parse->commandType == CMD_MERGE)
+ {
+ ListCell *e;
+ MergeActionSet *maset;
+
+ /*
+ *the parse in subroot now is a copy of the main query of current result relation
+ *Here we need to generate a copy of the action queries and shift their target table
+ *to current result relation
+ */
+ subroot.parse->mergeActQry = NIL;
+
+ foreach(e, parse->mergeActQry)
+ {
+ Query *actqry = (Query *)lfirst(e);
+ Query *newactqry = makeNode(Query);
+
+ /*copy most of the common fileds from original query*/
+ *newactqry = *actqry;
+
+ /*reset the result relation to current child table*/
+ newactqry->resultRelation = subroot.parse->resultRelation;
+
+ /*make the range table to be consistent with current main query*/
+ newactqry->rtable = subroot.parse->rtable;
+
+ /*adjust the target list*/
+ newactqry->targetList = (List *) adjust_appendrel_attrs(
+ (Node *) actqry->targetList,
+ appinfo);
+ newactqry->targetList = adjust_inherited_tlist(newactqry->targetList,
+ appinfo);
+ /*and qual*/
+ newactqry->jointree = makeNode(FromExpr);
+ newactqry->jointree->fromlist = subroot.parse->jointree->fromlist;
+ newactqry->jointree->quals = adjust_appendrel_attrs(
+ actqry->jointree->quals,
+ appinfo);
+
+ /*put this new action query in to the action list of current main query*/
+ subroot.parse->mergeActQry = lappend(subroot.parse->mergeActQry, newactqry);
+ }
+
+ /*
+ * now we have a complete query (main query + action queries) that has been
+ *shifted to current result relation. Plan these actions here.
+ */
+ maset = merge_action_set_planner(&subroot, subplan);
+ Assert(maset != NULL);
+ mergeActSets = lappend(mergeActSets, maset);
+ }
}
root->resultRelations = resultRelations;
@@ -842,6 +1090,7 @@ inheritance_planner(PlannerInfo *root)
subplans,
returningLists,
rowMarks,
+ mergeActSets,
SS_assign_special_param(root));
}
diff --git a/src/backend/optimizer/prep/preptlist.c b/src/backend/optimizer/prep/preptlist.c
index 59d3518..76ea604 100644
--- a/src/backend/optimizer/prep/preptlist.c
+++ b/src/backend/optimizer/prep/preptlist.c
@@ -78,25 +78,64 @@ preprocess_targetlist(PlannerInfo *root, List *tlist)
result_relation, range_table);
/*
- * for "update" and "delete" queries, add ctid of the result relation into
+ for MERGE command, we also need to expend the target list of the main query.
+ Note that, the target list of main query is the combination of attrs
+ from Source table and target table. We only want to expend the part
+ of target table.
+
+ We do this in an aggresive way:
+ 1. Truncate the old target list, keep only the entries for source table
+ 2. expend the target list of result relation from an NIL list.
+ 3. Append this new list at the end of old target list.
+ */
+ if (command_type == CMD_MERGE)
+ {
+ ListCell *l;
+ List *TLforResultRelatoin = NIL;
+ int new_resno;
+
+ Assert(parse->sourceAttrNo > 0);
+
+ tlist = list_truncate(tlist, parse->sourceAttrNo);
+
+ TLforResultRelatoin = expand_targetlist(TLforResultRelatoin, command_type,
+ result_relation, range_table);
+ new_resno = parse->sourceAttrNo + 1;
+ foreach(l, TLforResultRelatoin)
+ {
+ TargetEntry *te = (TargetEntry *)lfirst(l);
+ te->resno = new_resno++;
+ }
+
+ tlist = list_concat(tlist, TLforResultRelatoin);
+ }
+
+ /*
+ * for "update" , "delete" and "merge" queries, add ctid of the result relation into
* the target list so that the ctid will propagate through execution and
* ExecutePlan() will be able to identify the right tuple to replace or
* delete. This extra field is marked "junk" so that it is not stored
* back into the tuple.
+ *
+ * BUT, if the query node is a merge action,
+ * we don't need to expand the ctid attribute in tlist.
+ * The tlist of the merge top level plan already contains
+ * a "ctid" junk attr of the target relation.
*/
- if (command_type == CMD_UPDATE || command_type == CMD_DELETE)
+ if (!parse->isMergeAction &&
+ (command_type == CMD_UPDATE ||
+ command_type == CMD_DELETE ||
+ command_type == CMD_MERGE))
{
TargetEntry *tle;
Var *var;
var = makeVar(result_relation, SelfItemPointerAttributeNumber,
TIDOID, -1, 0);
-
tle = makeTargetEntry((Expr *) var,
list_length(tlist) + 1,
pstrdup("ctid"),
true);
-
/*
* For an UPDATE, expand_targetlist already created a fresh tlist. For
* DELETE, better do a listCopy so that we don't destructively modify
@@ -339,6 +378,7 @@ expand_targetlist(List *tlist, int command_type,
}
break;
case CMD_UPDATE:
+ case CMD_MERGE:
if (!att_tup->attisdropped)
{
new_expr = (Node *) makeVar(result_relation,
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
index 68fe92f..ba946d2 100644
--- a/src/backend/optimizer/prep/prepunion.c
+++ b/src/backend/optimizer/prep/prepunion.c
@@ -101,8 +101,6 @@ static Bitmapset *translate_col_privs(const Bitmapset *parent_privs,
static Node *adjust_appendrel_attrs_mutator(Node *node,
AppendRelInfo *context);
static Relids adjust_relid_set(Relids relids, Index oldrelid, Index newrelid);
-static List *adjust_inherited_tlist(List *tlist,
- AppendRelInfo *context);
/*
@@ -1740,8 +1738,9 @@ adjust_relid_set(Relids relids, Index oldrelid, Index newrelid)
* scribble on.
*
* Note that this is not needed for INSERT because INSERT isn't inheritable.
+ *BUT, the INSERT action in MERGE command need to this function.
*/
-static List *
+List *
adjust_inherited_tlist(List *tlist, AppendRelInfo *context)
{
bool changed_it = false;
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
index 92c2208..2cd5488 100644
--- a/src/backend/optimizer/util/var.c
+++ b/src/backend/optimizer/util/var.c
@@ -67,6 +67,16 @@ typedef struct
bool inserted_sublink; /* have we inserted a SubLink? */
} flatten_join_alias_vars_context;
+typedef struct
+{
+ int varno_source;
+ int varno_target;
+ int varno_join;
+
+ int offset_source;
+ int offset_target;
+} push_up_merge_action_vars_context;
+
static bool pull_varnos_walker(Node *node,
pull_varnos_context *context);
static bool pull_varattnos_walker(Node *node, Bitmapset **varattnos);
@@ -83,6 +93,8 @@ static bool pull_var_clause_walker(Node *node,
static Node *flatten_join_alias_vars_mutator(Node *node,
flatten_join_alias_vars_context *context);
static Relids alias_relid_set(PlannerInfo *root, Relids relids);
+static bool push_up_merge_action_vars_walker(Node *node,
+ push_up_merge_action_vars_context *context);
/*
@@ -677,6 +689,79 @@ pull_var_clause_walker(Node *node, pull_var_clause_context *context)
(void *) context);
}
+/*
+ * When prepare for the MERGE command, we have made a
+ * left join between the Source table and target table as the
+ * main plan.
+ *
+ * In this case, the range table contains ONLY THREE range table entries:
+ * 1. the source table, which may be a subquery or a plain table
+ * 2. the entry of the targe table, which is a plain table
+ * 3. join expression with the sourse table and target table as its parameters.
+ *
+ * Each merge action of the command has its own query and
+ * plan nodes as well. And, the vars in its target list and qual
+ * expressions may refers to the attribute in any one of the above 3
+ * range table entries.
+ *
+ * However, since the result tuple slots of merge actions are
+ * projected from the returned tuple of the join, we need to
+ * mapping the vars of source table and target table to their
+ * corresponding attributes in the third range table entry.
+ *
+ * This function does the opposite of the flatten_join_alias_vars()
+ * function. It walks through the target list and qual of a
+ * MergeAction plan, changes the vars' varno and varattno to the
+ * corresponding position in the upper level join RTE.
+ */
+void
+push_up_merge_action_vars(MergeAction *actplan, Query *actqry)
+{
+ push_up_merge_action_vars_context context;
+
+ context.varno_source = Merge_SourceTableRTindex;
+ context.varno_target = actqry->resultRelation;
+ context.varno_join = Merge_TopJoinTableRTindex;
+ context.offset_source = 0;
+ context.offset_target = actqry->sourceAttrNo;
+
+ push_up_merge_action_vars_walker((Node *)actplan->plan.targetlist,
+ &context);
+ push_up_merge_action_vars_walker((Node *)actplan->plan.qual,
+ &context);
+}
+
+static bool
+push_up_merge_action_vars_walker(Node *node,
+ push_up_merge_action_vars_context *context)
+{
+ if (node == NULL)
+ return false;
+ if (IsA(node, Var))
+ {
+ Var *var = (Var *)node;
+
+ if(var->varno == context->varno_source)
+ {
+ var->varno = context->varno_join;
+ var->varattno += context->offset_source;
+ return false;
+ }
+ else if(var->varno == context->varno_target)
+ {
+ var->varno = context->varno_join;
+ var->varattno += context->offset_target;
+ return false;
+ }
+ else if(var->varno == context->varno_join)
+ return false;
+ else
+ elog(ERROR, "the vars in merge action tlist of qual should only belongs to the source table or targe table");
+ }
+
+ return expression_tree_walker(node, push_up_merge_action_vars_walker,
+ (void *) context);
+}
/*
* flatten_join_alias_vars
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 6b99a10..b66b806 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -47,6 +47,7 @@ static Query *transformDeleteStmt(ParseState *pstate, DeleteStmt *stmt);
static Query *transformInsertStmt(ParseState *pstate, InsertStmt *stmt);
static List *transformInsertRow(ParseState *pstate, List *exprlist,
List *stmtcols, List *icolumns, List *attrnos);
+static Query *transformMergeStmt(ParseState *pstate, MergeStmt *stmt);
static Query *transformSelectStmt(ParseState *pstate, SelectStmt *stmt);
static Query *transformValuesClause(ParseState *pstate, SelectStmt *stmt);
static Query *transformSetOperationStmt(ParseState *pstate, SelectStmt *stmt);
@@ -175,6 +176,10 @@ transformStmt(ParseState *pstate, Node *parseTree)
result = transformUpdateStmt(pstate, (UpdateStmt *) parseTree);
break;
+ case T_MergeStmt:
+ result = transformMergeStmt(pstate, (MergeStmt *)parseTree);
+ break;
+
case T_SelectStmt:
{
SelectStmt *n = (SelectStmt *) parseTree;
@@ -245,6 +250,7 @@ analyze_requires_snapshot(Node *parseTree)
case T_DeleteStmt:
case T_UpdateStmt:
case T_SelectStmt:
+ case T_MergeStmt:
result = true;
break;
@@ -282,21 +288,27 @@ transformDeleteStmt(ParseState *pstate, DeleteStmt *stmt)
qry->commandType = CMD_DELETE;
- /* set up range table with just the result rel */
- qry->resultRelation = setTargetTable(pstate, stmt->relation,
- interpretInhOption(stmt->relation->inhOpt),
- true,
- ACL_DELETE);
-
qry->distinctClause = NIL;
/*
- * The USING clause is non-standard SQL syntax, and is equivalent in
- * functionality to the FROM list that can be specified for UPDATE. The
- * USING keyword is used rather than FROM because FROM is already a
- * keyword in the DELETE syntax.
+ * The input stmt could be a MergeDelete node.
+ * In this case, we don't need the process on range table.
*/
- transformFromClause(pstate, stmt->usingClause);
+ if (!stmt->isMergeAction)
+ {
+ /* set up range table with just the result rel */
+ qry->resultRelation = setTargetTable(pstate, stmt->relation,
+ interpretInhOption(stmt->relation->inhOpt),
+ true,
+ ACL_DELETE);
+ /*
+ * The USING clause is non-standard SQL syntax, and is equivalent in
+ * functionality to the FROM list that can be specified for UPDATE. The
+ * USING keyword is used rather than FROM because FROM is already a
+ * keyword in the DELETE syntax.
+ */
+ transformFromClause(pstate, stmt->usingClause);
+ }
qual = transformWhereClause(pstate, stmt->whereClause, "WHERE");
@@ -347,6 +359,8 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
* VALUES list, or general SELECT input. We special-case VALUES, both for
* efficiency and so we can handle DEFAULT specifications.
*/
+
+ /* a MergeInsert statement is always a VALUES clause*/
isGeneralSelect = (selectStmt && selectStmt->valuesLists == NIL);
/*
@@ -382,7 +396,8 @@ transformInsertStmt(ParseState *pstate, InsertStmt *stmt)
* mentioned in the SELECT part. Note that the target table is not added
* to the joinlist or namespace.
*/
- qry->resultRelation = setTargetTable(pstate, stmt->relation,
+ if (!stmt->isMergeAction) /* for MergeInsert, no need to do this */
+ qry->resultRelation = setTargetTable(pstate, stmt->relation,
false, false, ACL_INSERT);
/* Validate stmt->cols list, or build default list if no list given */
@@ -1730,16 +1745,18 @@ transformUpdateStmt(ParseState *pstate, UpdateStmt *stmt)
qry->commandType = CMD_UPDATE;
pstate->p_is_update = true;
- qry->resultRelation = setTargetTable(pstate, stmt->relation,
- interpretInhOption(stmt->relation->inhOpt),
- true,
- ACL_UPDATE);
-
- /*
- * the FROM clause is non-standard SQL syntax. We used to be able to do
- * this with REPLACE in POSTQUEL so we keep the feature.
- */
- transformFromClause(pstate, stmt->fromClause);
+ if (!stmt->isMergeAction) /* for MergeUpdate, no need to do this */
+ {
+ qry->resultRelation = setTargetTable(pstate, stmt->relation,
+ interpretInhOption(stmt->relation->inhOpt),
+ true,
+ ACL_UPDATE);
+ /*
+ * the FROM clause is non-standard SQL syntax. We used to be able to do
+ * this with REPLACE in POSTQUEL so we keep the feature.
+ */
+ transformFromClause(pstate, stmt->fromClause);
+ }
qry->targetList = transformTargetList(pstate, stmt->targetList);
@@ -2241,3 +2258,348 @@ applyLockingClause(Query *qry, Index rtindex,
rc->pushedDown = pushedDown;
qry->rowMarks = lappend(qry->rowMarks, rc);
}
+
+/*
+ * transform an action of merge command into a query.
+ * No change of the pstate range table is allowed in this function.
+ */
+static Query *
+transformMergeActions(ParseState *pstate, MergeStmt *stmt,
+ MergeConditionAction *condact)
+{
+ Query *actqry;
+
+ /*
+ * First, we need to make sure that DELETE and UPDATE
+ * actions are only taken in MATCHED condition,
+ * and INSERTs are only takend when not MATCHED
+ */
+ switch(condact->action->type)
+ {
+ case T_DeleteStmt:/*a delete action*/
+ {
+ DeleteStmt *deleteact = (DeleteStmt *) condact->action;
+ Assert(deleteact->isMergeAction);
+
+ if (!condact->match)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("The DELETE action in MERGE command is not allowed when NOT MATCHED")));
+
+ /*put new right code to the result relaion.
+ This line chages the RTE in range table directly*/
+ pstate->p_target_rangetblentry->requiredPerms |= ACL_DELETE;
+
+ deleteact->relation = stmt->relation;
+ deleteact->usingClause = stmt->source;
+ deleteact->whereClause = condact->condition;
+
+ /*parse the action query*/
+ actqry = transformStmt(pstate, (Node *) deleteact);
+
+ if (!IsA(actqry, Query) ||
+ actqry->commandType != CMD_DELETE ||
+ actqry->utilityStmt != NULL)
+ elog(ERROR, "improper DELETE action in merge stmt");
+
+ actqry->isMergeAction = true;
+ actqry->matched = condact->match;
+
+ return actqry;
+ }
+ break;
+ case T_UpdateStmt:/*an update action*/
+ {
+ UpdateStmt *updateact = (UpdateStmt *) condact->action;
+ Assert(updateact->isMergeAction);
+
+ if (!condact->match)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("The UPDATE action in MERGE command is not allowed when NOT MATCHED")));
+
+ pstate->p_target_rangetblentry->requiredPerms |= ACL_UPDATE;
+
+ /*the "targetlist" of the updateact is filled in the parser */
+ updateact->relation = stmt->relation;
+ updateact->fromClause = stmt->source;
+ updateact->whereClause = condact->condition;
+
+ /*parse the action query*/
+ actqry = transformStmt(pstate, (Node *)updateact);
+
+ if (!IsA(actqry, Query) ||
+ actqry->commandType != CMD_UPDATE||
+ actqry->utilityStmt != NULL)
+ elog(ERROR, "improper UPDATE action in merge stmt");
+
+ actqry->isMergeAction = true;
+ actqry->matched = condact->match;
+
+ return actqry;
+ }
+ break;
+ case T_InsertStmt:/*an insert action*/
+ {
+ InsertStmt *insertact = (InsertStmt *) condact->action;
+ Assert(insertact->isMergeAction);
+
+ if(condact->match)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("The INSERT action in MERGE command is not allowed when MATCHED")));
+
+ pstate->p_target_rangetblentry->requiredPerms |= ACL_INSERT;
+
+ /*the "cols" and "selectStmt" of the insertact is filled in the parser */
+ insertact->relation = stmt->relation;
+
+ /*
+ the merge insert action has a strange feature.
+ In an ordinary INSERT, the VALUES list can only
+ contains constants and DEFAULT. (am I right??)
+ But in the INSERT action of MERGE command,
+ the VALUES list can have expressions with
+ variables(attributes of the targe and source tables).
+ Besides, in the ordinary INSERT, a VALUES list can
+ never be followed by a WHERE clause.
+ But in MERGE INSERT action, there are matching conditions.
+
+ Thus, the output qry of this function is an INSERT
+ query in the style of "INSERT...VALUES...",
+ except that we have other range tables and a WHERE clause.
+ Note that it is also different from the "INSERT ... SELECT..."
+ query, in which the whole SELECT is a subquery.
+ (We don't have subquery here).
+
+ We construct this novel query structure in order
+ to keep consitency with other merge action types
+ (DELETE, UPDATE). In this way, all the merge action
+ queries are in fact share the very same Range Table,
+ They only differs in their target lists and join trees
+ */
+
+ /*parse the action query, this will call
+ transformInsertStmt() which analyzes the VALUES list.*/
+ actqry = transformStmt(pstate, (Node *)insertact);
+
+ /*do the WHERE clause here, Since the
+ transformInsertStmt() function only analyzes
+ the VALUES list but not the WHERE clause*/
+ actqry->jointree = makeFromExpr(pstate->p_joinlist,
+ transformWhereClause(pstate,
+ condact->condition,
+ "WHERE"));
+ if(!IsA(actqry, Query) ||
+ actqry->commandType != CMD_INSERT||
+ actqry->utilityStmt != NULL)
+ elog(ERROR, "improper INSERT action in merge stmt");
+
+ actqry->isMergeAction = true;
+ actqry->matched = condact->match;
+
+ return actqry;
+ }
+ break;
+ case T_MergeDoNothing:
+ {
+ MergeDoNothing *nothingact = (MergeDoNothing *)(condact->action);
+
+ Assert(IsA(nothingact,MergeDoNothing));
+
+ actqry = makeNode(Query);
+
+ actqry->jointree = makeFromExpr(pstate->p_joinlist,
+ transformWhereClause(pstate,
+ condact->condition,
+ "WHERE"));
+ actqry->rtable = pstate->p_rtable;
+
+ actqry->commandType = CMD_DONOTHING;
+ actqry->isMergeAction = true;
+ actqry->matched = condact->match;
+
+ return actqry;
+ }
+ break;
+ case T_MergeError:
+ {
+ MergeError *erract = (MergeError *)(condact->action);
+ Assert(IsA(erract,MergeError));
+
+ actqry = makeNode(Query);
+
+ actqry->jointree = makeFromExpr(pstate->p_joinlist,
+ transformWhereClause(pstate,
+ condact->condition,
+ "WHERE"));
+ actqry->rtable = pstate->p_rtable;
+
+ actqry->commandType = CMD_RAISEERR;
+ actqry->isMergeAction = true;
+ actqry->matched = condact->match;
+
+ return actqry;
+ }
+ break;
+ default:
+ elog(ERROR, "unknown MERGE action type %d", condact->action->type);
+ break;
+ }
+
+ /*never comes here*/
+ return NULL;
+}
+
+static Query *
+transformMergeStmt(ParseState *pstate, MergeStmt *stmt)
+{
+ Query *qry;
+
+ ColumnRef *starRef;
+ ResTarget *starResTarget;
+ ListCell *act;
+ ListCell *l;
+ JoinExpr *joinexp;
+ int rtindex;
+ RangeTblEntry *resultRTE = NULL;
+ RangeTblEntry *topjoinRTE = NULL;
+ RangeTblEntry *sourceRTE = NULL;
+
+ /*firstly, create the output node structure*/
+ qry = makeNode(Query);
+ qry->commandType = CMD_MERGE;
+
+ /*
+ * What we are doing here is to create a query like
+ * "SELECT * FROM <source_rel> LEFT JOIN <target_rel> ON <match_condition>;"
+ *
+ * Note:
+ * 1. we set the "match condition" as the join qualification.
+ * The left join will scan both the matched and non-matched tuples.
+ *
+ * 2. a normal SELECT query has no "target relation".
+ * But here we need to set the targe relation in query,
+ * like the UPDATE/DELETE/INSERT queries.
+ * So this is a left join SELECT with a "target table" in its range table.
+ *
+ * 3. We don't have a specific ACL level for Merge, here we just use
+ * ACL_SELECT.
+ * But we will add other ACL levels when handle each merge actions.
+ */
+
+ /*
+ * Before transforming the FROM clause, acquire write lock on the
+ * target relation. We don't want to add it to the range table yet,
+ * so we use setTargetTableLock() instead of setTargetTable().
+ */
+ setTargetTableLock(pstate, stmt->relation);
+
+ /*
+ * Make the join expression on source table and target table,
+ * as the only element in FROM list
+ */
+ joinexp = makeNode(JoinExpr);
+ joinexp->jointype = JOIN_LEFT;
+ joinexp->isNatural = FALSE;
+ joinexp->larg = stmt->source;
+ joinexp->rarg = (Node *)stmt->relation;
+ joinexp->quals = stmt->matchCondition;
+
+ /*
+ * transform the FROM clause. The target relation and
+ * source relation will be added to the range table here.
+ */
+ transformFromClause(pstate, list_make1(joinexp));
+
+ /* the targetList of the main query is "*" */
+ starRef = makeNode(ColumnRef);
+ starRef->fields = list_make1(makeNode(A_Star));
+ starRef->location = 1;
+
+ starResTarget = makeNode(ResTarget);
+ starResTarget->name = NULL;
+ starResTarget->indirection = NIL;
+ starResTarget->val = (Node *)starRef;
+ starResTarget->location = 1;
+
+ qry->targetList = transformTargetList(pstate, list_make1(starResTarget));
+
+ /* we don't need a WHERE clause here. Set it null. */
+ qry->jointree = makeFromExpr(pstate->p_joinlist, NULL);
+
+ /*
+ *The range table of a MERGE commond query has only 3 RTE.
+ *RTE 1 is the source table, RTE 2 is the target table, RTE 3 is the left join
+ *between source and target table.
+ */
+ qry->rtable = pstate->p_rtable;
+
+ /*Set RTE 2 as the result relation of query*/
+ qry->resultRelation = 2;
+ resultRTE = rt_fetch(qry->resultRelation, pstate->p_rtable);
+ if(resultRTE->relid != pstate->p_target_relation->rd_id)
+ elog(ERROR, "The target relation entry should be the second one in range table");
+
+ resultRTE->requiredPerms = ACL_SELECT;
+ resultRTE->inh = interpretInhOption(stmt->relation->inhOpt);
+ pstate->p_target_rangetblentry = resultRTE;
+
+ /*
+ *we also need to find out how many attributes are there in the source table.
+ *There are many ways to do this. Here, I choose to scan the join var list of the
+ *top join table entry. And count how many vars belong to source table.
+ */
+ topjoinRTE = rt_fetch(Merge_TopJoinTableRTindex, pstate->p_rtable);
+ Assert(topjoinRTE->jointype == JOIN_LEFT);
+
+ qry->sourceAttrNo = 0;
+ foreach(l, topjoinRTE->joinaliasvars)
+ {
+ Var *jv = (Var *)lfirst(l);
+
+ if(jv->varno == Merge_SourceTableRTindex)
+ qry->sourceAttrNo++;
+ }
+ /*lets do a simple check here*/
+ sourceRTE = rt_fetch(Merge_SourceTableRTindex, pstate->p_rtable);
+ Assert(qry->sourceAttrNo == list_length(sourceRTE->eref->colnames));
+
+ /*
+ * For each action, transform it to a seperate query.
+ * The action queries share the range table with the main query.
+ *
+ * In other words, in the extra conditions of the sub actions,
+ * we don't allow involvement of new tables
+ */
+ qry->mergeActQry = NIL;
+
+ foreach(act,stmt->actions)
+ {
+ MergeConditionAction *mca = (MergeConditionAction *) lfirst(act);
+ Query *actqry;
+
+ /* transform the act (and its condition) as a single query. */
+ actqry = transformMergeActions(pstate, stmt, mca);
+
+ /*
+ * since we don't invoke setTargetTable() in transformMergeActions(),
+ * we need to set actqry->resultRelation here
+ */
+ actqry->resultRelation = qry->resultRelation;
+
+ /*record the source attr number in each action query, for latter use*/
+ actqry->sourceAttrNo = qry->sourceAttrNo;
+
+ /* put it into the list */
+ qry->mergeActQry = lappend(qry->mergeActQry, actqry);
+ }
+
+ /*
+ * set the sublink mark at the last.
+ * Thus, the sublink in actions will be counted in.
+ */
+ qry->hasSubLinks = pstate->p_hasSubLinks;
+
+ return qry;
+}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index aab7789..135d287 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -211,6 +211,11 @@ static TypeName *TableFuncTypeName(List *columns);
DeallocateStmt PrepareStmt ExecuteStmt
DropOwnedStmt ReassignOwnedStmt
AlterTSConfigurationStmt AlterTSDictionaryStmt
+ MergeStmt
+
+%type <node> opt_and_condition merge_condition_action merge_action
+%type <boolean> opt_not
+%type <list> merge_condition_action_list
%type <node> select_no_parens select_with_parens select_clause
simple_select values_clause
@@ -480,7 +485,7 @@ static TypeName *TableFuncTypeName(List *columns);
DEFERRABLE DEFERRED DEFINER DELETE_P DELIMITER DELIMITERS DESC
DICTIONARY DISABLE_P DISCARD DISTINCT DO DOCUMENT_P DOMAIN_P DOUBLE_P DROP
- EACH ELSE ENABLE_P ENCODING ENCRYPTED END_P ENUM_P ESCAPE EXCEPT
+ EACH ELSE ENABLE_P ENCODING ENCRYPTED END_P ENUM_P ERROR_P ESCAPE EXCEPT
EXCLUDE EXCLUDING EXCLUSIVE EXECUTE EXISTS EXPLAIN EXTERNAL EXTRACT
FALSE_P FAMILY FETCH FIRST_P FLOAT_P FOLLOWING FOR FORCE FOREIGN FORWARD
@@ -503,7 +508,7 @@ static TypeName *TableFuncTypeName(List *columns);
LEAST LEFT LEVEL LIKE LIMIT LISTEN LOAD LOCAL LOCALTIME LOCALTIMESTAMP
LOCATION LOCK_P LOGIN_P
- MAPPING MATCH MAXVALUE MINUTE_P MINVALUE MODE MONTH_P MOVE
+ MAPPING MATCH MATCHED MAXVALUE MERGE MINUTE_P MINVALUE MODE MONTH_P MOVE
NAME_P NAMES NATIONAL NATURAL NCHAR NEXT NO NOCREATEDB
NOCREATEROLE NOCREATEUSER NOINHERIT NOLOGIN_P NONE NOSUPERUSER
@@ -518,7 +523,7 @@ static TypeName *TableFuncTypeName(List *columns);
QUOTE
- RANGE READ REAL REASSIGN RECHECK RECURSIVE REF REFERENCES REINDEX
+ RAISE RANGE READ REAL REASSIGN RECHECK RECURSIVE REF REFERENCES REINDEX
RELATIVE_P RELEASE RENAME REPEATABLE REPLACE REPLICA RESET RESTART
RESTRICT RETURNING RETURNS REVOKE RIGHT ROLE ROLLBACK ROW ROWS RULE
@@ -726,6 +731,7 @@ stmt :
| ListenStmt
| LoadStmt
| LockStmt
+ | MergeStmt
| NotifyStmt
| PrepareStmt
| ReassignOwnedStmt
@@ -6986,6 +6992,7 @@ ExplainableStmt:
| InsertStmt
| UpdateStmt
| DeleteStmt
+ | MergeStmt
| DeclareCursorStmt
| CreateAsStmt
| ExecuteStmt /* by default all are $$=$1 */
@@ -7331,6 +7338,116 @@ set_target_list:
/*****************************************************************************
*
* QUERY:
+ * MERGE STATEMENT
+ *
+ *****************************************************************************/
+
+
+MergeStmt:
+ MERGE INTO relation_expr_opt_alias
+ USING table_ref
+ ON a_expr
+ merge_condition_action_list
+ {
+ MergeStmt *m = makeNode(MergeStmt);
+
+ m->relation = $3;
+ m->source = $5;
+ m->matchCondition = $7;
+ m->actions = $8;
+
+ $$ = (Node *)m;
+ }
+ ;
+
+merge_condition_action_list:
+ merge_condition_action
+ { $$ = list_make1($1); }
+ | merge_condition_action_list merge_condition_action
+ { $$ = lappend($1,$2); }
+ ;
+
+merge_condition_action:
+ WHEN opt_not MATCHED opt_and_condition THEN merge_action
+ {
+ MergeConditionAction *m = makeNode(MergeConditionAction);
+
+ m->match = $2;
+ m->condition = $4;
+ m->action = $6;
+
+ $$ = (Node *)m;
+ }
+ ;
+
+opt_and_condition:
+ AND a_expr { $$ = $2; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+opt_not:
+ NOT { $$ = false; }
+ | /*EMPTY*/ { $$ = true; }
+ ;
+
+merge_action:
+ DELETE_P
+ {
+ DeleteStmt *n = makeNode(DeleteStmt);
+ n->isMergeAction = true;
+ $$ = (Node *) n;
+ }
+ | UPDATE SET set_clause_list
+ {
+ UpdateStmt *n = makeNode(UpdateStmt);
+ n->targetList = $3;
+ n->isMergeAction = true;
+
+ $$ = (Node *) n;
+ }
+ | INSERT values_clause
+ {
+ InsertStmt *n = makeNode(InsertStmt);
+ n->cols = NIL;
+ n->selectStmt = $2;
+ n->isMergeAction = true;
+
+ $$ = (Node *) n;
+ }
+
+ | INSERT '(' insert_column_list ')' values_clause
+ {
+ InsertStmt *n = makeNode(InsertStmt);
+ n->cols = $3;
+ n->selectStmt = $5;
+ n->isMergeAction = true;
+
+ $$ = (Node *) n;
+ }
+ | INSERT DEFAULT VALUES
+ {
+ InsertStmt *n = makeNode(InsertStmt);
+ n->cols = NIL;
+ n->selectStmt = NULL;
+ n->isMergeAction = true;
+
+ $$ = (Node *) n;
+ }
+ | DO NOTHING
+ {
+ $$ = (Node *) makeNode(MergeDoNothing);
+ }
+ | RAISE ERROR_P
+ {
+ $$ = (Node *) makeNode(MergeError);
+ }
+ ;
+
+
+
+/*****************************************************************************
+ *
+ * QUERY:
* CURSOR STATEMENTS
*
*****************************************************************************/
@@ -10952,6 +11069,7 @@ unreserved_keyword:
| ENCODING
| ENCRYPTED
| ENUM_P
+ | ERROR_P
| ESCAPE
| EXCLUDE
| EXCLUDING
@@ -11005,7 +11123,9 @@ unreserved_keyword:
| LOGIN_P
| MAPPING
| MATCH
+ | MATCHED
| MAXVALUE
+ | MERGE
| MINUTE_P
| MINVALUE
| MODE
@@ -11048,6 +11168,7 @@ unreserved_keyword:
| PROCEDURAL
| PROCEDURE
| QUOTE
+ | RAISE
| RANGE
| READ
| REASSIGN
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 16ca583..c3e8038 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -214,6 +214,25 @@ setTargetTable(ParseState *pstate, RangeVar *relation,
return rtindex;
}
+void
+setTargetTableLock(ParseState *pstate, RangeVar *relation)
+{
+
+ /* Close old target; this could only happen for multi-action rules */
+ if (pstate->p_target_relation != NULL)
+ heap_close(pstate->p_target_relation, NoLock);
+
+ /*
+ * Open target rel and grab suitable lock (which we will hold till end of
+ * transaction).
+ *
+ * free_parsestate() will eventually do the corresponding heap_close(),
+ * but *not* release the lock.
+ */
+ pstate->p_target_relation = parserOpenTable(pstate, relation,
+ RowExclusiveLock);
+}
+
/*
* Simplify InhOption (yes/no/default) into boolean yes/no.
*
diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index 25b44dd..0d1e5ff 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1836,6 +1836,41 @@ RewriteQuery(Query *parsetree, List *rewrite_events)
return rewritten;
}
+/*if the merge action type has already been processed by rewriter*/
+#define insert_rewrite (1<<0)
+#define delete_rewrite (1<<1)
+#define update_rewrite (1<<2)
+
+/*if the merge action type is fully replace by rules.*/
+#define insert_instead (1<<3)
+#define delete_instead (1<<4)
+#define update_instead (1<<5)
+
+#define merge_action_already_rewrite(acttype, flag) \
+ ((acttype == CMD_INSERT && (flag & insert_rewrite)) || \
+ (acttype == CMD_UPDATE && (flag & update_rewrite)) || \
+ (acttype == CMD_DELETE && (flag & delete_rewrite)))
+
+#define set_action_rewrite(acttype, flag) \
+ if(acttype == CMD_INSERT) \
+ {flag |= insert_rewrite;}\
+ else if(acttype == CMD_UPDATE) \
+ {flag |= update_rewrite;}\
+ else if(acttype == CMD_DELETE) \
+ {flag |= delete_rewrite;}
+
+#define merge_action_instead(acttype, flag) \
+ ((acttype == CMD_INSERT && (flag & insert_instead)) || \
+ (acttype == CMD_UPDATE && (flag & update_instead)) || \
+ (acttype == CMD_DELETE && (flag & delete_instead)))
+
+#define set_action_instead(acttype, flag)\
+ if(acttype == CMD_INSERT) \
+ {flag |= insert_instead;}\
+ else if(acttype == CMD_UPDATE) \
+ {flag |= update_instead;}\
+ else if(acttype == CMD_DELETE) \
+ {flag |= delete_instead;}
/*
* QueryRewrite -
@@ -1861,7 +1896,149 @@ QueryRewrite(Query *parsetree)
*
* Apply all non-SELECT rules possibly getting 0 or many queries
*/
- querylist = RewriteQuery(parsetree, NIL);
+ if (parsetree->commandType == CMD_MERGE)
+ {
+ /*
+ * for MERGE, we have a set of action queries (not subquery).
+ * each of these action queries should rewritten with RewriteQuery().
+ */
+ ListCell *l;
+ int flag = 0;
+ List *pre_qry = NIL;
+ List *post_qry = NIL;
+
+ querylist = NIL;
+
+ /*rewrite the merge action queries one by one.*/
+ foreach(l, parsetree->mergeActQry)
+ {
+ List *queryList4action = NIL;
+ Query *actionqry;
+ Query *q;
+
+ actionqry = lfirst(l);
+
+ /* no rewriting for DO NOTHING or RAISE ERROR*/
+ if (actionqry->commandType == CMD_DONOTHING ||
+ actionqry->commandType == CMD_RAISEERR)
+ continue;
+
+ /*
+ * if this kind of actions are fully replaced by rules,
+ * we change it into a DO NOTHING action
+ */
+ if (merge_action_instead(actionqry->commandType, flag))
+ {
+ /*
+ * Still need to call RewriteQuery(),
+ * since we need the process on target list and so on.
+ * BUT, the returned list is discarded
+ */
+ RewriteQuery(actionqry, NIL);
+ actionqry->commandType = CMD_DONOTHING;
+ actionqry->targetList = NIL;
+ continue;
+ }
+
+ /* if this kind of actions are already processed by rewriter, skip it.*/
+ if (merge_action_already_rewrite(actionqry->commandType, flag))
+ {
+ RewriteQuery(actionqry, NIL);
+ continue;
+ }
+
+ /* ok this action has not been processed before, let's do it now. */
+ queryList4action = RewriteQuery(actionqry, NIL);
+
+ /* this kind of actions has been processed, set the flag */
+ set_action_rewrite(actionqry->commandType, flag);
+
+ /*
+ * if the returning list is empty, this merge action
+ * is replaced by a do-nothing rule
+ */
+ if (queryList4action == NIL)
+ {
+ /* set the flag for other merge actions of the same type */
+ set_action_instead(actionqry->commandType, flag);
+ actionqry->commandType = CMD_DONOTHING;
+ actionqry->targetList = NIL;
+ continue;
+ }
+
+ /*
+ * if the rewriter return a non-NIL list, the merge action query
+ * could be one element in it.
+ * if so, it must be the head (for INSERT acton)
+ * or tail (for UPDATE/DELETE action).
+ */
+
+ /* test the list head */
+ q = (Query *) linitial(queryList4action);
+ if (q->querySource == QSRC_ORIGINAL)
+ {
+ /*
+ * the merge action is the head, the remaining part
+ * of the list are the queries generated by rules
+ * we put them in the post_qry list.
+ */
+ if (querylist == NIL)
+ querylist = list_make1(parsetree);
+
+ queryList4action = list_delete_first(queryList4action);
+ post_qry = list_concat(post_qry,queryList4action);
+
+ continue;
+ }
+
+ /*test the list tail*/
+ q = (Query *) llast(queryList4action);
+ if (q->querySource == QSRC_ORIGINAL)
+ {
+ /*
+ * the merge action is the tail.
+ * Put the rule queries in pre_qry list
+ */
+ if (querylist == NIL)
+ querylist = list_make1(parsetree);
+
+ queryList4action = list_truncate(queryList4action,
+ list_length(queryList4action) - 1);
+
+ pre_qry = list_concat(pre_qry,queryList4action);
+ continue;
+ }
+
+ /*
+ * here, the merge action query is not in the rewritten query list,
+ * which means the action is replaced by INSTEAD rule(s).
+ * We need to change it into do noting action.
+ *
+ * For a INSERT action, we put the rule queries in the post list
+ * otherwise, in the pre list
+ */
+ if(actionqry->commandType == CMD_INSERT)
+ post_qry = list_concat(post_qry,queryList4action);
+ else
+ pre_qry = list_concat(pre_qry,queryList4action);
+
+ set_action_instead(actionqry->commandType, flag);
+ actionqry->commandType = CMD_DONOTHING;
+ actionqry->targetList = NIL;
+ }
+
+ /*
+ * finally, put the 3 lists into one.
+ * If all the merge actions are replaced by rules,
+ * the original merge query
+ * will not be involved in the querylist.
+ */
+ querylist = list_concat(pre_qry,querylist);
+ querylist = list_concat(querylist, post_qry);
+
+ }
+ else
+ querylist = RewriteQuery(parsetree, NIL);
/*
* Step 2
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 8ad4915..0dc3117 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -225,6 +225,10 @@ ProcessQuery(PlannedStmt *plan,
snprintf(completionTag, COMPLETION_TAG_BUFSIZE,
"DELETE %u", queryDesc->estate->es_processed);
break;
+ case CMD_MERGE:
+ snprintf(completionTag, COMPLETION_TAG_BUFSIZE,
+ "MERGE %u", queryDesc->estate->es_processed);
+ break;
default:
strcpy(completionTag, "???");
break;
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 1815539..0ee3074 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -125,6 +125,7 @@ CommandIsReadOnly(Node *parsetree)
case CMD_UPDATE:
case CMD_INSERT:
case CMD_DELETE:
+ case CMD_MERGE:
return false;
default:
elog(WARNING, "unrecognized commandType: %d",
@@ -1405,6 +1406,10 @@ CreateCommandTag(Node *parsetree)
tag = "SELECT";
break;
+ case T_MergeStmt:
+ tag = "MERGE";
+ break;
+
/* utility statements --- same whether raw or cooked */
case T_TransactionStmt:
{
@@ -2242,6 +2247,7 @@ GetCommandLogLevel(Node *parsetree)
case T_InsertStmt:
case T_DeleteStmt:
case T_UpdateStmt:
+ case T_MergeStmt:
lev = LOGSTMT_MOD;
break;
diff --git a/src/include/commands/trigger.h b/src/include/commands/trigger.h
index 267a08e..51d0f11 100644
--- a/src/include/commands/trigger.h
+++ b/src/include/commands/trigger.h
@@ -164,6 +164,8 @@ extern void ExecBSTruncateTriggers(EState *estate,
ResultRelInfo *relinfo);
extern void ExecASTruncateTriggers(EState *estate,
ResultRelInfo *relinfo);
+extern void ExecBSMergeTriggers(ModifyTableState *mt_state);
+extern void ExecASMergeTriggers(ModifyTableState *mt_state);
extern void AfterTriggerBeginXact(void);
extern void AfterTriggerBeginQuery(void);
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 67ba3e8..422e3ce 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -16,6 +16,7 @@
#include "nodes/execnodes.h"
extern ModifyTableState *ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags);
+extern MergeActionState *ExecInitMergeAction(MergeAction *node, EState *estate, int eflags);
extern TupleTableSlot *ExecModifyTable(ModifyTableState *node);
extern void ExecEndModifyTable(ModifyTableState *node);
extern void ExecReScanModifyTable(ModifyTableState *node);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 7442d2d..24a53d6 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1032,9 +1032,23 @@ typedef struct ModifyTableState
int mt_whichplan; /* which one is being executed (0..n-1) */
EPQState mt_epqstate; /* for evaluating EvalPlanQual rechecks */
bool fireBSTriggers; /* do we need to fire stmt triggers? */
+ MergeActionSet **mt_mergeActPstates; /*the list of the planstate of meger command actions.
+ NULL if this is not a merge command.
+ The elements if it are still MergeActionSet nodes.
+ But the action list in these nodes are ModifyTableState */
} ModifyTableState;
/* ----------------
+ * MergeActionState information
+ * ----------------
+ */
+typedef struct MergeActionState
+{
+ PlanState ps; /* its first field is NodeTag */
+ CmdType operation;
+} MergeActionState;
+
+/* ----------------
* AppendState information
*
* nplans how many plans are in the array
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index a5f5df5..4becf19 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -44,6 +44,8 @@ typedef enum NodeTag
T_Plan = 100,
T_Result,
T_ModifyTable,
+ T_MergeAction,
+ T_MergeActionSet,
T_Append,
T_RecursiveUnion,
T_BitmapAnd,
@@ -86,6 +88,7 @@ typedef enum NodeTag
T_PlanState = 200,
T_ResultState,
T_ModifyTableState,
+ T_MergeActionState,
T_AppendState,
T_RecursiveUnionState,
T_BitmapAndState,
@@ -347,6 +350,10 @@ typedef enum NodeTag
T_AlterUserMappingStmt,
T_DropUserMappingStmt,
T_AlterTableSpaceOptionsStmt,
+ T_MergeStmt,
+ T_MergeConditionAction,
+ T_MergeDoNothing,
+ T_MergeError,
/*
* TAGS FOR PARSE TREE NODES (parsenodes.h)
@@ -511,6 +518,9 @@ typedef enum CmdType
CMD_UPDATE, /* update stmt */
CMD_INSERT, /* insert stmt */
CMD_DELETE,
+ CMD_MERGE, /* merge stmt */
+ CMD_DONOTHING,
+ CMD_RAISEERR,
CMD_UTILITY, /* cmds like create, destroy, copy, vacuum,
* etc. */
CMD_NOTHING /* dummy command for instead nothing rules
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index d31cf6c..50fe7bb 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -149,8 +149,24 @@ typedef struct Query
List *constraintDeps; /* a list of pg_constraint OIDs that the query
* depends on to be semantically valid */
+
+ /* fields for MERGE command */
+ bool isMergeAction; /* if this query is a merge action. */
+ bool matched; /* this is a MATCHED action or NOT */
+ int sourceAttrNo; /*the number of attributes in source table*/
+ List *mergeActQry; /* the list of all the merge actions.
+ * used only for merge query statement */
} Query;
+/*
+* In MERGE command, the initial MERGE query has only three range table entry
+*The first one is the source table; The second one is the target table; and The
+*third one is the left join entry of them.
+*During the whole proecess of a MEREGE command, the rt_index for the source table
+*entry and the join table entry will never change. We set them as constant here.
+*/
+#define Merge_SourceTableRTindex 1
+#define Merge_TopJoinTableRTindex 3
/****************************************************************************
* Supporting data structures for Parse Trees
@@ -892,6 +908,7 @@ typedef struct CommonTableExpr
typedef struct InsertStmt
{
NodeTag type;
+ bool isMergeAction; /*if this is a merge insert action*/
RangeVar *relation; /* relation to insert into */
List *cols; /* optional: names of the target columns */
Node *selectStmt; /* the source SELECT/VALUES, or NULL */
@@ -905,6 +922,7 @@ typedef struct InsertStmt
typedef struct DeleteStmt
{
NodeTag type;
+ bool isMergeAction; /*if this is a merge delete action*/
RangeVar *relation; /* relation to delete from */
List *usingClause; /* optional using clause for more tables */
Node *whereClause; /* qualifications */
@@ -918,6 +936,7 @@ typedef struct DeleteStmt
typedef struct UpdateStmt
{
NodeTag type;
+ bool isMergeAction; /*if this is a merge delete action*/
RangeVar *relation; /* relation to update */
List *targetList; /* the target list (of ResTarget) */
Node *whereClause; /* qualifications */
@@ -993,6 +1012,54 @@ typedef struct SelectStmt
/* Eventually add fields for CORRESPONDING spec here */
} SelectStmt;
+/* ----------------------
+ * Merge Statement
+ * ----------------------
+ */
+typedef struct MergeStmt
+{
+ NodeTag type;
+ RangeVar *relation; /* target relation for merge */
+
+ /*
+ * source relations for the merge.
+ * Currently, we only allwo single-source merge,
+ * so the length of this list should always be 1.
+ */
+ Node *source;
+ Node *matchCondition; /* qualifications of the merge */
+
+ /* list of MergeConditionAction structure.
+ * It stores all the matched / not-matched
+ * conditions and the corresponding actions
+ * The elments of this list are MergeConditionAction
+ * nodes.
+ */
+ List *actions;
+} MergeStmt;
+
+/*
+ * the structure for the actions of MERGE command.
+ * Holds info of the clauses like
+ * "WHEN MATCHED AND ... THEN UPDATE/DELETE/INSERT"
+ */
+typedef struct MergeConditionAction
+{
+ NodeTag type;
+ bool match; /* WHEN MATCHED or WHEN NOT MATCHED? */
+ Node *condition; /* the AND condition for this action */
+ Node *action; /* the actions: delete, insert or update */
+} MergeConditionAction;
+
+typedef struct MergeDoNothing
+{
+ NodeTag type;
+} MergeDoNothing;
+
+typedef struct MergeError
+{
+ NodeTag type;
+} MergeError;
/* ----------------------
* Set Operation node for post-analysis query trees
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 037bc0b..5583f59 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -169,9 +169,38 @@ typedef struct ModifyTable
List *returningLists; /* per-target-table RETURNING tlists */
List *rowMarks; /* PlanRowMarks (non-locking only) */
int epqParam; /* ID of Param for EvalPlanQual re-eval */
+ List *mergeActPlan; /* the plans for merge actions,
+ * which are MergeActionSet nodes.
+ * one set for one target relation */
} ModifyTable;
/* ----------------
+ * MergeActionSet node -
+ * The node contains all actions of MERGE command for one specific target relation
+ * ----------------
+ */
+typedef struct MergeActionSet
+{
+ NodeTag type;
+ int result_relation;
+ List *actions;
+}MergeActionSet;
+
+/* ----------------
+ * MergeAction node -
+ * The plan node for the actions of MERGE command
+ * ----------------
+ */
+typedef struct MergeAction
+{
+ Plan plan;
+ CmdType operation; /* INSERT, UPDATE, DELETE, DO_NOTHING or RAISE_ERR */
+ bool matched; /* is this a MATCHED or NOT MATCHED rule? */
+ List *flattenedqual; /* the flattened qual expression of action */
+ List *flattenedtlist;/*the fattened target list*/
+} MergeAction;
+
+/* ----------------
* Append node -
* Generate the concatenation of the results of sub-plans.
* ----------------
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 35cdd2c..3660e57 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -80,7 +80,7 @@ extern Result *make_result(PlannerInfo *root, List *tlist,
Node *resconstantqual, Plan *subplan);
extern ModifyTable *make_modifytable(CmdType operation, List *resultRelations,
List *subplans, List *returningLists,
- List *rowMarks, int epqParam);
+ List *rowMarks, List *mergeActPlans, int epqParam);
extern bool is_projection_capable_plan(Plan *plan);
/*
diff --git a/src/include/optimizer/prep.h b/src/include/optimizer/prep.h
index 9df6390..116be42 100644
--- a/src/include/optimizer/prep.h
+++ b/src/include/optimizer/prep.h
@@ -52,4 +52,6 @@ extern void expand_inherited_tables(PlannerInfo *root);
extern Node *adjust_appendrel_attrs(Node *node, AppendRelInfo *appinfo);
+extern List *adjust_inherited_tlist(List *tlist, AppendRelInfo *context);
+
#endif /* PREP_H */
diff --git a/src/include/optimizer/var.h b/src/include/optimizer/var.h
index b0e04a0..ec773f4 100644
--- a/src/include/optimizer/var.h
+++ b/src/include/optimizer/var.h
@@ -15,6 +15,7 @@
#define VAR_H
#include "nodes/relation.h"
+#include "nodes/plannodes.h"
typedef enum
{
@@ -32,5 +33,6 @@ extern int locate_var_of_relation(Node *node, int relid, int levelsup);
extern int find_minimum_var_level(Node *node);
extern List *pull_var_clause(Node *node, PVCPlaceHolderBehavior behavior);
extern Node *flatten_join_alias_vars(PlannerInfo *root, Node *node);
+extern void push_up_merge_action_vars(MergeAction *actplan, Query *actqry);
#endif /* VAR_H */
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 974bb7a..208c3e4 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -141,6 +141,7 @@ PG_KEYWORD("encoding", ENCODING, UNRESERVED_KEYWORD)
PG_KEYWORD("encrypted", ENCRYPTED, UNRESERVED_KEYWORD)
PG_KEYWORD("end", END_P, RESERVED_KEYWORD)
PG_KEYWORD("enum", ENUM_P, UNRESERVED_KEYWORD)
+PG_KEYWORD("error", ERROR_P, UNRESERVED_KEYWORD)
PG_KEYWORD("escape", ESCAPE, UNRESERVED_KEYWORD)
PG_KEYWORD("except", EXCEPT, RESERVED_KEYWORD)
PG_KEYWORD("exclude", EXCLUDE, UNRESERVED_KEYWORD)
@@ -229,7 +230,9 @@ PG_KEYWORD("lock", LOCK_P, UNRESERVED_KEYWORD)
PG_KEYWORD("login", LOGIN_P, UNRESERVED_KEYWORD)
PG_KEYWORD("mapping", MAPPING, UNRESERVED_KEYWORD)
PG_KEYWORD("match", MATCH, UNRESERVED_KEYWORD)
+PG_KEYWORD("matched", MATCHED, UNRESERVED_KEYWORD)
PG_KEYWORD("maxvalue", MAXVALUE, UNRESERVED_KEYWORD)
+PG_KEYWORD("merge", MERGE, UNRESERVED_KEYWORD)
PG_KEYWORD("minute", MINUTE_P, UNRESERVED_KEYWORD)
PG_KEYWORD("minvalue", MINVALUE, UNRESERVED_KEYWORD)
PG_KEYWORD("mode", MODE, UNRESERVED_KEYWORD)
@@ -296,6 +299,7 @@ PG_KEYWORD("privileges", PRIVILEGES, UNRESERVED_KEYWORD)
PG_KEYWORD("procedural", PROCEDURAL, UNRESERVED_KEYWORD)
PG_KEYWORD("procedure", PROCEDURE, UNRESERVED_KEYWORD)
PG_KEYWORD("quote", QUOTE, UNRESERVED_KEYWORD)
+PG_KEYWORD("raise", RAISE, UNRESERVED_KEYWORD)
PG_KEYWORD("range", RANGE, UNRESERVED_KEYWORD)
PG_KEYWORD("read", READ, UNRESERVED_KEYWORD)
PG_KEYWORD("real", REAL, COL_NAME_KEYWORD)
diff --git a/src/include/parser/parse_clause.h b/src/include/parser/parse_clause.h
index f3d3ee9..b54f530 100644
--- a/src/include/parser/parse_clause.h
+++ b/src/include/parser/parse_clause.h
@@ -19,6 +19,7 @@
extern void transformFromClause(ParseState *pstate, List *frmList);
extern int setTargetTable(ParseState *pstate, RangeVar *relation,
bool inh, bool alsoSource, AclMode requiredPerms);
+extern void setTargetTableLock(ParseState *pstate, RangeVar *relation);
extern bool interpretInhOption(InhOption inhOpt);
extern bool interpretOidsOption(List *defList);
diff --git a/src/test/regress/expected/merge.out b/src/test/regress/expected/merge.out
new file mode 100644
index 0000000..0e30194
--- /dev/null
+++ b/src/test/regress/expected/merge.out
@@ -0,0 +1,260 @@
+--
+-- MERGE
+--
+CREATE TABLE target (id integer, balance integer);
+CREATE TABLE source (id integer, balance integer);
+INSERT INTO target VALUES (1, 10);
+INSERT INTO target VALUES (2, 20);
+INSERT INTO target VALUES (3, 30);
+SELECT * FROM target;
+ id | balance
+----+---------
+ 1 | 10
+ 2 | 20
+ 3 | 30
+(3 rows)
+
+--
+-- initial tests
+--
+-- empty source means 0 rows touched
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN MATCHED THEN
+ UPDATE SET balance = t.balance + s.balance
+;
+-- insert some source rows to work from
+INSERT INTO source VALUES (2, 5);
+INSERT INTO source VALUES (3, 20);
+INSERT INTO source VALUES (4, 40);
+SELECT * FROM source;
+ id | balance
+----+---------
+ 2 | 5
+ 3 | 20
+ 4 | 40
+(3 rows)
+
+-- do a simple equivalent of an UPDATE join
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN MATCHED THEN
+ UPDATE SET balance = t.balance + s.balance
+;
+NOTICE: one tuple is ERROR
+ROLLBACK;
+-- do a simple equivalent of an INSERT SELECT
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN NOT MATCHED THEN
+ INSERT VALUES (s.id, s.balance)
+;
+NOTICE: one tuple is ERROR
+NOTICE: one tuple is ERROR
+SELECT * FROM target;
+ id | balance
+----+---------
+ 1 | 10
+ 2 | 20
+ 3 | 30
+ 4 | 40
+(4 rows)
+
+ROLLBACK;
+-- now the classic UPSERT
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN MATCHED THEN
+ UPDATE SET balance = t.balance + s.balance
+WHEN NOT MATCHED THEN
+ INSERT VALUES (s.id, s.balance)
+;
+SELECT * FROM target;
+ id | balance
+----+---------
+ 1 | 10
+ 2 | 25
+ 3 | 50
+ 4 | 40
+(4 rows)
+
+ROLLBACK;
+--
+-- Non-standard functionality
+--
+-- do a simple equivalent of a DELETE join
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN MATCHED THEN
+ DELETE
+;
+NOTICE: one tuple is ERROR
+SELECT * FROM target;
+ id | balance
+----+---------
+ 1 | 10
+(1 row)
+
+ROLLBACK;
+-- now the classic UPSERT, with a DELETE
+-- the Standard doesn't allow the DELETE clause for some reason,
+-- though other implementations do
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN MATCHED AND s.balance > 10 THEN
+ UPDATE SET balance = t.balance + s.balance
+WHEN MATCHED THEN
+ DELETE
+WHEN NOT MATCHED THEN
+ INSERT VALUES (s.id, s.balance)
+;
+SELECT * FROM target;
+ id | balance
+----+---------
+ 1 | 10
+ 3 | 50
+ 4 | 40
+(3 rows)
+
+ROLLBACK;
+-- Prepare the test data to generate multiple matching rows for a single target
+INSERT INTO source VALUES (3, 5);
+SELECT * FROM source ORDER BY id, balance;
+ id | balance
+----+---------
+ 2 | 5
+ 3 | 5
+ 3 | 20
+ 4 | 40
+(4 rows)
+
+-- we now have a duplicate key in source, so when we join to
+-- target we will generate 2 matching rows, not one.
+-- According to standard this command should fail.
+-- But it suceeds in PostgreSQL implementation by simply ignoring the second
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN MATCHED AND s.balance > 10 THEN
+ UPDATE SET balance = t.balance + s.balance
+WHEN NOT MATCHED THEN
+ INSERT VALUES (s.id, s.balance)
+;
+NOTICE: one tuple is ERROR
+NOTICE: one tuple is ERROR
+ROLLBACK;
+-- Now lets prepare the test data to generate 2 non-matching rows
+DELETE FROM source WHERE id = 3 AND balance = 5;
+INSERT INTO source VALUES (4, 5);
+SELECT * FROM source;
+ id | balance
+----+---------
+ 2 | 5
+ 3 | 20
+ 4 | 40
+ 4 | 5
+(4 rows)
+
+-- This next SQL statement
+-- suceeds according to standard (yes, it is inconsistent)
+-- suceeds in PostgreSQL implementation, though could easily fail if
+-- there was an appropriate unique constraint
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN NOT MATCHED THEN
+ INSERT VALUES (s.id, s.balance)
+;
+NOTICE: one tuple is ERROR
+NOTICE: one tuple is ERROR
+SELECT * FROM target;
+ id | balance
+----+---------
+ 1 | 10
+ 2 | 20
+ 3 | 30
+ 4 | 40
+ 4 | 5
+(5 rows)
+
+ROLLBACK;
+-- This next SQL statement works, but since there is no WHEN clause that
+-- applies to non-matching rows, SQL standard requires us to generate
+-- rows with DEFAULT VALUES for all columns, which is why we support the
+-- syntax DO NOTHING (similar to the way Rules work) in addition
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN NOT MATCHED AND s.balance > 100 THEN
+ INSERT VALUES (s.id, s.balance)
+;
+NOTICE: one tuple is ERROR
+NOTICE: one tuple is ERROR
+NOTICE: one tuple is ERROR
+NOTICE: one tuple is ERROR
+SELECT * FROM target;
+ id | balance
+----+---------
+ 1 | 10
+ 2 | 20
+ 3 | 30
+(3 rows)
+
+ROLLBACK;
+-- This next SQL statement suceeds, but does nothing since there are
+-- only non-matching rows that do not activate a WHEN clause, so we
+-- provide syntax to just ignore them, rather than allowing data quality
+-- problems
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN NOT MATCHED AND s.balance > 100 THEN
+ INSERT VALUES (s.id, s.balance)
+WHEN NOT MATCHED THEN
+ DO NOTHING
+;
+NOTICE: one tuple is ERROR
+NOTICE: one tuple is ERROR
+SELECT * FROM target;
+ id | balance
+----+---------
+ 1 | 10
+ 2 | 20
+ 3 | 30
+(3 rows)
+
+ROLLBACK;
+--
+-- Weirdness
+--
+-- MERGE statement containing WHEN clauses that are never executable
+-- NOT an error under the standard
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN MATCHED AND s.balance > 0 THEN
+ UPDATE SET balance = t.balance + s.balance
+WHEN MATCHED THEN
+ DELETE
+WHEN NOT MATCHED THEN
+ INSERT VALUES (s.id, s.balance)
+WHEN NOT MATCHED THEN /* never executed because of order of WHEN clauses */
+ INSERT VALUES (s.id, s.balance + 10)
+WHEN MATCHED THEN /* never executed because of order of WHEN clauses */
+ UPDATE SET balance = t.balance + s.balance
+;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 191d1fe..2551b2a 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -91,7 +91,7 @@ test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combo
# NB: temp.sql does a reconnect which transiently uses 2 connections,
# so keep this parallel group to at most 19 tests
# ----------
-test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml
+test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml merge
# run stats by itself because its delay may be insufficient under heavy load
test: stats
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 80a9881..e7d7fae 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -123,4 +123,5 @@ test: returning
test: largeobject
test: with
test: xml
+test: merge
test: stats
diff --git a/src/test/regress/sql/merge.sql b/src/test/regress/sql/merge.sql
new file mode 100644
index 0000000..522868a
--- /dev/null
+++ b/src/test/regress/sql/merge.sql
@@ -0,0 +1,178 @@
+--
+-- MERGE
+--
+CREATE TABLE target (id integer, balance integer);
+CREATE TABLE source (id integer, balance integer);
+INSERT INTO target VALUES (1, 10);
+INSERT INTO target VALUES (2, 20);
+INSERT INTO target VALUES (3, 30);
+SELECT * FROM target;
+
+--
+-- initial tests
+--
+-- empty source means 0 rows touched
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN MATCHED THEN
+ UPDATE SET balance = t.balance + s.balance
+;
+-- insert some source rows to work from
+INSERT INTO source VALUES (2, 5);
+INSERT INTO source VALUES (3, 20);
+INSERT INTO source VALUES (4, 40);
+SELECT * FROM source;
+
+-- do a simple equivalent of an UPDATE join
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN MATCHED THEN
+ UPDATE SET balance = t.balance + s.balance
+;
+
+ROLLBACK;
+-- do a simple equivalent of an INSERT SELECT
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN NOT MATCHED THEN
+ INSERT VALUES (s.id, s.balance)
+;
+SELECT * FROM target;
+
+ROLLBACK;
+-- now the classic UPSERT
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN MATCHED THEN
+ UPDATE SET balance = t.balance + s.balance
+WHEN NOT MATCHED THEN
+ INSERT VALUES (s.id, s.balance)
+;
+SELECT * FROM target;
+
+ROLLBACK;
+--
+-- Non-standard functionality
+--
+-- do a simple equivalent of a DELETE join
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN MATCHED THEN
+ DELETE
+;
+SELECT * FROM target;
+
+ROLLBACK;
+-- now the classic UPSERT, with a DELETE
+-- the Standard doesn't allow the DELETE clause for some reason,
+-- though other implementations do
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN MATCHED AND s.balance > 10 THEN
+ UPDATE SET balance = t.balance + s.balance
+WHEN MATCHED THEN
+ DELETE
+WHEN NOT MATCHED THEN
+ INSERT VALUES (s.id, s.balance)
+;
+SELECT * FROM target;
+
+ROLLBACK;
+
+-- Prepare the test data to generate multiple matching rows for a single target
+INSERT INTO source VALUES (3, 5);
+SELECT * FROM source ORDER BY id, balance;
+
+-- we now have a duplicate key in source, so when we join to
+-- target we will generate 2 matching rows, not one.
+-- According to standard this command should fail.
+-- But it suceeds in PostgreSQL implementation by simply ignoring the second
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN MATCHED AND s.balance > 10 THEN
+ UPDATE SET balance = t.balance + s.balance
+WHEN NOT MATCHED THEN
+ INSERT VALUES (s.id, s.balance)
+;
+ROLLBACK;
+-- Now lets prepare the test data to generate 2 non-matching rows
+DELETE FROM source WHERE id = 3 AND balance = 5;
+INSERT INTO source VALUES (4, 5);
+SELECT * FROM source;
+
+-- This next SQL statement
+-- suceeds according to standard (yes, it is inconsistent)
+-- suceeds in PostgreSQL implementation, though could easily fail if
+-- there was an appropriate unique constraint
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN NOT MATCHED THEN
+ INSERT VALUES (s.id, s.balance)
+;
+SELECT * FROM target;
+
+ROLLBACK;
+-- This next SQL statement works, but since there is no WHEN clause that
+-- applies to non-matching rows, SQL standard requires us to generate
+-- rows with DEFAULT VALUES for all columns, which is why we support the
+-- syntax DO NOTHING (similar to the way Rules work) in addition
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN NOT MATCHED AND s.balance > 100 THEN
+ INSERT VALUES (s.id, s.balance)
+;
+SELECT * FROM target;
+
+ROLLBACK;
+-- This next SQL statement suceeds, but does nothing since there are
+-- only non-matching rows that do not activate a WHEN clause, so we
+-- provide syntax to just ignore them, rather than allowing data quality
+-- problems
+BEGIN;
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN NOT MATCHED AND s.balance > 100 THEN
+ INSERT VALUES (s.id, s.balance)
+WHEN NOT MATCHED THEN
+ DO NOTHING
+;
+SELECT * FROM target;
+
+ROLLBACK;
+--
+-- Weirdness
+--
+-- MERGE statement containing WHEN clauses that are never executable
+-- NOT an error under the standard
+MERGE into target t
+USING (select * from source) AS s
+ON t.id = s.id
+WHEN MATCHED AND s.balance > 0 THEN
+ UPDATE SET balance = t.balance + s.balance
+WHEN MATCHED THEN
+ DELETE
+WHEN NOT MATCHED THEN
+ INSERT VALUES (s.id, s.balance)
+WHEN NOT MATCHED THEN /* never executed because of order of WHEN clauses */
+ INSERT VALUES (s.id, s.balance + 10)
+WHEN MATCHED THEN /* never executed because of order of WHEN clauses */
+ UPDATE SET balance = t.balance + s.balance
+;
Hi,
On 2017/04/18 16:54, Kang Yuzhe wrote:
Thanks Amit for taking your time and pointing to some useful stuff on the
Internals of PostgreSQL.One thing I have learned is that PG community is not as hostile/harsh as I
imagined to newbies. Rather, its the reverse.
I am feeling at home here.Amit, would you please help out on how to apply some patches in PG source
code. For example, there are two patches attached here: one on
CORRESPONDING CLAUSE and one on MERGE SQL Standard.There are some errors saying Hunk failed(src/backend/parser/gram.y.rej).
postgresql-9.6.2$ patch --dry-run -p1 < corresponding_clause_v12.patch
patching file doc/src/sgml/queries.sgml
Hunk #1 succeeded at 1603 (offset 2 lines).
Hunk #2 succeeded at 1622 (offset 2 lines).
Hunk #3 succeeded at 1664 (offset 2 lines).
[ ... ]
..../postgresql-9.6.2$
Firstly, it looks like you're trying to apply the patch to the 9.6 source
tree (are you working with the PostgreSQL source git repository?). But,
since all the new feature patches are created against the master
development branch of the git repository, the patch most likely won't
apply cleanly against a source tree from the older branch.
If you're not using the git repository currently, you may have better luck
trying the development branch snapshot tarballs (see the link below):
https://www.postgresql.org/ftp/snapshot/dev/
Also, it's a good idea to reply on the email thread from where you
downloaded the patch to ask them to update the patch, so that they can
send a fresh patch that applies cleanly.
The MERGE patch looks very old (from 2010 probably), so properly applying
it to the source tree of today is going to be hard. Actually, it most
likely won't be in a working condition anymore. You can try recently
proposed patches, for example, those in the next commitfest:
https://commitfest.postgresql.org/14/
Patches listed on the above page are more likely to apply cleanly and be
in working condition. But of course, you will need to be interested in
the topics those patches are related to. There are some new SQL feature
patches, for example:
https://commitfest.postgresql.org/14/839/
Thanks,
Amit
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Thanks Craig for teaching me a lot of things. I am just learning a lot why
PG hacking/development is the way it is.
Regarding interest and enthusiasm, no problem. Whats is lacking is the
skill-sets and I believe having interest and enthusiasm and with your
support, we will expand PG hacking/devs/usage in Africa and other
continents.
People here in Africa using Oracle/SQL Server/IBM products(generally
commercial products) even for which PG is more than enough.
I want to change this scenario and trend and I hope one day in the future
there will be PG conference in Africa/Ethiopia which is my country.
Thanks,
zeray
On Tue, Apr 18, 2017 at 10:54 AM, Craig Ringer <craig@2ndquadrant.com>
wrote:
Show quoted text
On 18 April 2017 at 15:41, Kang Yuzhe <tiggreen87@gmail.com> wrote:
Thanks Simon for taking your time and trying to tell and warn me the
harsh
reality truth:there is no shortcut to expertise. One has to fail and rise
towards any journey to expertise.Yeah, just because Pg is hard doesn't mean it's notably bad or worse
than other things. I generally find working on code in other projects,
even smaller and simpler ones, a rather unpleasant change.That doesn't mean we can't do things to help interested new people get
and stay engaged and grow into productive devs to grow our pool.Overall, you are right. But I do believe that there is a way(some
techniques) to speed up any journey to expertise. One of them ismentorship.
For example(just an example), If you show me how to design and implement
FDW
to Hadoop/HBase., I believe that I will manage to design and implement
FDW
to Cassandra/MengoDB.
TBH, that's the sort of thing where looking at existing examples is
often the best way forward and will stay that way.What I'd like to do is make it easier to understand those examples by
providing background and overview info on subsystems, so you can read
the code and have more idea what it does and why.But almost nothing about The Internals of PostgreSQL:
Not surprising. They'd go out of date fast, be a huge effort to write
and maintain, and sell poorly given the small audience.Print books probably aren't the way forward here.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Hello Amit,
Thanks gain for being patient with me.
YES, I am working with the PostgreSQL source git repository but I don't
think I updated my local forked/cloned branch. I am also working on
standalone PG 9.6.2 source code as well.
I will try to fetch/pull the PG master content to my forked/cloned branch
and apply those current patches.
I will also try to reply to the email threads where I downloaded the
patches so that they can update their patches accordingly.
Thanks,
Zeray
On Tue, Apr 18, 2017 at 11:25 AM, Amit Langote <
Langote_Amit_f8@lab.ntt.co.jp> wrote:
Show quoted text
Hi,
On 2017/04/18 16:54, Kang Yuzhe wrote:
Thanks Amit for taking your time and pointing to some useful stuff on the
Internals of PostgreSQL.One thing I have learned is that PG community is not as hostile/harsh as
I
imagined to newbies. Rather, its the reverse.
I am feeling at home here.Amit, would you please help out on how to apply some patches in PG
source
code. For example, there are two patches attached here: one on
CORRESPONDING CLAUSE and one on MERGE SQL Standard.There are some errors saying Hunk failed(src/backend/parser/gram.y.rej).
postgresql-9.6.2$ patch --dry-run -p1 < corresponding_clause_v12.patch
patching file doc/src/sgml/queries.sgml
Hunk #1 succeeded at 1603 (offset 2 lines).
Hunk #2 succeeded at 1622 (offset 2 lines).
Hunk #3 succeeded at 1664 (offset 2 lines).[ ... ]
..../postgresql-9.6.2$
Firstly, it looks like you're trying to apply the patch to the 9.6 source
tree (are you working with the PostgreSQL source git repository?). But,
since all the new feature patches are created against the master
development branch of the git repository, the patch most likely won't
apply cleanly against a source tree from the older branch.If you're not using the git repository currently, you may have better luck
trying the development branch snapshot tarballs (see the link below):https://www.postgresql.org/ftp/snapshot/dev/
Also, it's a good idea to reply on the email thread from where you
downloaded the patch to ask them to update the patch, so that they can
send a fresh patch that applies cleanly.The MERGE patch looks very old (from 2010 probably), so properly applying
it to the source tree of today is going to be hard. Actually, it most
likely won't be in a working condition anymore. You can try recently
proposed patches, for example, those in the next commitfest:https://commitfest.postgresql.org/14/
Patches listed on the above page are more likely to apply cleanly and be
in working condition. But of course, you will need to be interested in
the topics those patches are related to. There are some new SQL feature
patches, for example:https://commitfest.postgresql.org/14/839/
Thanks,
Amit
On 04/18/2017 03:41 AM, Kang Yuzhe wrote:
But almost nothing about The Internals of PostgreSQL:
1. The Internals of PostgreSQL:
http://www.interdb.jp/pg/index.html translated from Japanese Book
2. PostgreSQL数据库内核分析(Chinese) Book on the Internals of PostgreSQL:
3. PG Docs/site
4. some here and there which are less useful
I agree that this is an area where more material would be very welcome,
and not only to newcomers. #1 is useful as far as it goes, but the
missing bits (esp. Query Processing) are critical.
Lastly, I have come to understand that PG community is not
harsh/intimidating to newbies and thus, I am feeling at home.
Glad you have found it so.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 04/18/2017 03:54 AM, Craig Ringer wrote:
But almost nothing about The Internals of PostgreSQL:
Not surprising. They'd go out of date fast, be a huge effort to write
and maintain, and sell poorly given the small audience.Print books probably aren't the way forward here.
Agreed, a well organized web site would work much better.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
I’ve seen the inside of quite a few commercial database products, and imho PostgreSQL’s code is cleaner and better-organized than all but one of them. That one retained PostgreSQL for parsing and semantic analysis, but replaced the optimizer, execution engine, and storage layer in C++.
…and as far as the interest and enthusiasm of the PG community, I’ve never seen anything like it. My only complaint is that pgsql-hackers keeps blowing out my Outlook file size limit. :^)
From: <pgsql-hackers-owner@postgresql.org> on behalf of Kang Yuzhe <tiggreen87@gmail.com>
Date: Tuesday, April 18, 2017 at 4:33 AM
To: Craig Ringer <craig@2ndquadrant.com>
Cc: Simon Riggs <simon@2ndquadrant.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] On How To Shorten the Steep Learning Curve Towards PG Hacking...
Thanks Craig for teaching me a lot of things. I am just learning a lot why PG hacking/development is the way it is.
Regarding interest and enthusiasm, no problem. Whats is lacking is the skill-sets and I believe having interest and enthusiasm and with your support, we will expand PG hacking/devs/usage in Africa and other continents.
People here in Africa using Oracle/SQL Server/IBM products(generally commercial products) even for which PG is more than enough.
I want to change this scenario and trend and I hope one day in the future there will be PG conference in Africa/Ethiopia which is my country.
Thanks,
zeray
On Tue, Apr 18, 2017 at 10:54 AM, Craig Ringer <craig@2ndquadrant.com<mailto:craig@2ndquadrant.com>> wrote:
On 18 April 2017 at 15:41, Kang Yuzhe <tiggreen87@gmail.com<mailto:tiggreen87@gmail.com>> wrote:
Thanks Simon for taking your time and trying to tell and warn me the harsh
reality truth:there is no shortcut to expertise. One has to fail and rise
towards any journey to expertise.
Yeah, just because Pg is hard doesn't mean it's notably bad or worse
than other things. I generally find working on code in other projects,
even smaller and simpler ones, a rather unpleasant change.
That doesn't mean we can't do things to help interested new people get
and stay engaged and grow into productive devs to grow our pool.
Overall, you are right. But I do believe that there is a way(some
techniques) to speed up any journey to expertise. One of them is mentorship.
For example(just an example), If you show me how to design and implement FDW
to Hadoop/HBase., I believe that I will manage to design and implement FDW
to Cassandra/MengoDB.
TBH, that's the sort of thing where looking at existing examples is
often the best way forward and will stay that way.
What I'd like to do is make it easier to understand those examples by
providing background and overview info on subsystems, so you can read
the code and have more idea what it does and why.
But almost nothing about The Internals of PostgreSQL:
Not surprising. They'd go out of date fast, be a huge effort to write
and maintain, and sell poorly given the small audience.
Print books probably aren't the way forward here.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Hello Simon,
The journey that caused and is causing me a lot of pain is finding my way
in PG development.
Complex Code Reading like PG. Fully understanding the science of DBMS
Engines: Query Processing, Storage stuff, Transaction Management and so
on...
Anyway as you said, the rough estimation towards any expertise seems to be
in abidance with by The 10,000 Hour Rule. I will strive based on this rule.
For now, would please tell me how to know the exact PG version to which a
specific patch was developed?
Given x patch, how do I know the specific PG version it was developed for?
Regards,
Zeray
On Mon, Apr 17, 2017 at 7:33 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
Show quoted text
On 27 March 2017 at 13:00, Kang Yuzhe <tiggreen87@gmail.com> wrote:
I have found PG source Code reading and hacking to be one the most
frustrating experiences in my life. I believe that PG hacking shouldnot be
a painful journey but an enjoyable one!
It is my strong believe that out of my PG hacking frustrations, there may
come insights for the PG experts on ways how to devise hands-on with PG
internals so that new comers will be great coders as quickly as possible.I'm here now because PostgreSQL has clear, well designed and
maintained code with accurate docs, great comments and a helpful team.I'd love to see detailed cases where another project is better in a
measurable way; I am willing to learn from that.Any journey to expertise takes 10,000 hours. There is no way to shorten
that.What aspect of your journey caused you pain?
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 28 Apr. 2017 17:04, "Kang Yuzhe" <tiggreen87@gmail.com> wrote:
Hello Simon,
The journey that caused and is causing me a lot of pain is finding my way
in PG development.
Complex Code Reading like PG. Fully understanding the science of DBMS
Engines: Query Processing, Storage stuff, Transaction Management and so
on...
Anyway as you said, the rough estimation towards any expertise seems to be
in abidance with by The 10,000 Hour Rule. I will strive based on this rule.
Start with not top-posting on the mailing list ;)
For now, would please tell me how to know the exact PG version to which a
specific patch was developed?
Given x patch, how do I know the specific PG version it was developed for?
If it a was created by git format-patch then the base git revision will be
shown. This may be a commit from postgres public tree that you can find
with 'git branch --contains'.
Otherwise look at the proposed commit message if any, in the patch header.
Or the email it was attached to. If all else fails guess based on the date.
Hi, Kang and everyone in this thread.
I'm planning to present the online course "Hacking PostgreSQL: data
access methods in action and under the hood" on edX on June 1st. It's
not announced yet, links will be available later.
This course I'm describing information that was crucial for me to
start hacking. Currently, my knowledge of technologies behind Postgres
is quite limited, though I know the border of my knowledge quite well.
Chances are that description from my point of view will not be helpful
in some cases: before starting contributing to Postgres I had already
held PhD in CS for database technology and I had already implemented 3
different commercial DBMS (all in different technologies, PLs,
paradigms, focuses, different prbolems being solved). And still,
production of minimally viable contribution took 3 months (I was
hacking for an hour a day, mostly at evenings).
That's why I decided that it worth talking about how to get there
before I'm already there. It's quite easy to forget that some concepts
are really hard before you get them.
The course will cover:
1. Major differences of Postgres from others
2. Dev tools as I use them
3. Concept of paged memory, latches and paged data structures
4. WAL, recovery, replication
5. Concurrency and locking in B-trees
6. GiST internals
7. Extensions
8. Text search and some of GIN
9. Postgres community mechanics
Every topic will consist of two parts: 1 - video lectures on YouTube
(in English and Russian, BTW my English is far from perfect) with
references to docs and other resources, 2 - practical tasks where you
change code slightly and observe differences (this part is mostly to
help the student to observe easy entry points).
Best regards, Andrey Borodin, Octonica.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sun, Apr 30, 2017 at 10:47 AM, Andrew Borodin <borodin@octonica.com> wrote:
Hi, Kang and everyone in this thread.
I'm planning to present the online course "Hacking PostgreSQL: data
access methods in action and under the hood" on edX on June 1st. It's
not announced yet, links will be available later.
This course I'm describing information that was crucial for me to
start hacking. Currently, my knowledge of technologies behind Postgres
is quite limited, though I know the border of my knowledge quite well.Chances are that description from my point of view will not be helpful
in some cases: before starting contributing to Postgres I had already
held PhD in CS for database technology and I had already implemented 3
different commercial DBMS (all in different technologies, PLs,
paradigms, focuses, different prbolems being solved). And still,
production of minimally viable contribution took 3 months (I was
hacking for an hour a day, mostly at evenings).
That's why I decided that it worth talking about how to get there
before I'm already there. It's quite easy to forget that some concepts
are really hard before you get them.The course will cover:
1. Major differences of Postgres from others
2. Dev tools as I use them
3. Concept of paged memory, latches and paged data structures
4. WAL, recovery, replication
5. Concurrency and locking in B-trees
6. GiST internals
7. Extensions
8. Text search and some of GIN
9. Postgres community mechanics
Every topic will consist of two parts: 1 - video lectures on YouTube
(in English and Russian, BTW my English is far from perfect) with
references to docs and other resources, 2 - practical tasks where you
change code slightly and observe differences (this part is mostly to
help the student to observe easy entry points).
Thanks Andrey in advance. I am looking forward to meetingyou there at Edx.
Regards,
Zeray
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers