Which file does the SELECT?

On sön, 2010-10-10 at 13:32 +0530, Vaibhav Kaushal wrote:

I have gone through the source code a bit but I wanted to know that
which file contains the code that performs the final SLECTION after
the optimizer has created the final plan? I mean which part of the
executor is responsible for the SELCT to be run?

That depends on what plan was chosen for the SELECT, since the executor
is primarily organized by plan node type, independent of which statement
caused the plan to be generated.

Thanks for the reply.

So if I am not wrong, I will have to understand the whole querying process
in detail? If it is so, then where do I start from?

-Vaibhav

On Sun, Oct 10, 2010 at 1:41 PM, Peter Eisentraut <peter_e@gmx.net> wrote:

The PostgreSQL documentation (9.0.1) has the following section in section
44.5.1:

The planner preferentially considers joins between any two relations for
which
there exist a corresponding join clause in the WHERE qualification (i.e., for
which a restriction like
where rel1.attr1=rel2.attr2 exists). *Join pairs with no join clause are
considered only when*
there is no other choice, that is, a particular relation has no available
join clauses to any other relation.
All possible plans are generated for every join pair considered by the
planner, and the one that is
(estimated to be) the cheapest is chosen.

Can someone tell me what are 'Join Pairs with no Join clause' ? I am not
able to figure that out!

-Vaibhav (*_*)

On Sun, Oct 10, 2010 at 1:58 PM, Vaibhav Kaushal <
vaibhavkaushal123@gmail.com> wrote:

2010/10/10 Vaibhav Kaushal <vaibhavkaushal123@gmail.com>:

Thanks for the reply.
So if I am not wrong, I will have to understand the whole querying process
in detail? If it is so, then where do I start from?
-Vaibhav

On Sun, Oct 10, 2010 at 1:41 PM, Peter Eisentraut <peter_e@gmx.net> wrote:

On sön, 2010-10-10 at 13:32 +0530, Vaibhav Kaushal wrote:

I have gone through the source code a bit but I wanted to know that
which file contains the code that performs the final SLECTION after
the optimizer has created the final plan? I mean which part of the
executor is responsible for the SELCT to be run?

That depends on what plan was chosen for the SELECT, since the executor
is primarily organized by plan node type, independent of which statement
caused the plan to be generated.

So if I am not wrong, I will have to understand the whole querying process
in detail? If it is so, then where do I start from?

And it depends on what you are interested in. If the executor behavior
is your interest, see execMain.c, but your question looks more
interested in nodeSeqscan.c which scans rows from a relation.

Regards,

--
Hitoshi Harada

Vaibhav Kaushal <vaibhavkaushal123@gmail.com> writes:

So if I am not wrong, I will have to understand the whole querying process
in detail? If it is so, then where do I start from?

If you haven't seen it already, this is a good place to start:
http://developer.postgresql.org/pgdocs/postgres/overview.html

There's also some stuff in src/tools/backend/ that tries to summarize
which subdirectories of the source tree do what.

Also, many subsystems have README files in the source tree.
It sounds like you might like to look at src/backend/executor/README.

But in the end there's no substitute for reading source code...

regards, tom lane

Vaibhav Kaushal <vaibhavkaushal123@gmail.com> writes:

Can someone tell me what are 'Join Pairs with no Join clause' ? I am not
able to figure that out!

Consider

select * from t1, t2, t3 where t1.a = t2.x and t1.b = t3.y;

In theory this query could be done by first joining t2 and t3, then
joining that to t1. But the planner won't investigate the possibility
because the t2/t3 join would have to be a cartesian product join:
there's no WHERE clause relating them.

On the other hand, if we have

select * from t1, t2, t3 where t1.a = t2.x and t1.a = t3.y;

then the planner is able to infer the additional join clause t2.x =
t3.y, so it will consider that join sequence.

regards, tom lane

Thanks to both hitoshi and tom for your replies.

I think I need to look into the Postgres code itself (I am better at code
than documentation). But since I have not been touch with C lately (these
days I am programming on PHP) I think I have forgot a few rules of game
(afterall PHP is so much more easy than C :P ). Moreover, postgres is the
first Open Source software whose code I am interested in. I have never
looked into other OSS codes much except correcting a few compilation errors
here and there on beta / alpha releases.

I have had the chance and success to compile my own Linux OS and it was fun
to do so... but I guess development is a tougher job. With an idea in mind,
and a thankful feeling towards postgres is what drives me to do this tougher
job.

When I was designing my database for a web app, I found so many problems in
MySQL that I could not continue (the best of all, I can't use the commands
written in my DB book to create a foreign key, it does not natively support
foreign keys, confusing storage engines and so on).. and then I got postgres
which I am a fan of.

I hope I will not be flamed when I will ask those questions (some of them
are actually very silly ones).

I will look inside the code now and will get back after i get some progress
with it.

However, I find too many references to the Data structure "datum" what is it
and where is it defined? Can someone tell me please? Also, what role does it
play?

Thanks to you all for your replies.

-Vaibhav

On Sun, Oct 10, 2010 at 9:29 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

On Sun, Oct 10, 2010 at 10:51:54PM +0530, Vaibhav Kaushal wrote:

However, I find too many references to the Data structure "datum" what is it
and where is it defined? Can someone tell me please? Also, what role does it
play?

"Datum" is the singular form of "data". It refers to a single item of
any type. So it may be an integer, text type, geometry type, anything.
A row is a list of datums. (A datum can also be a composite type).

Normally in the planner code you don't need to worry to much about what
it explicitly refers to, but if you do, you need to know the type of a
datum before you can manipulate it. The type is not stored inside the
datum.

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

On Oct 10, 2010, at 12:21 PM, Vaibhav Kaushal wrote:

Thanks to both hitoshi and tom for your replies.

I think I need to look into the Postgres code itself (I am better at code than documentation). But since I have not been touch with C lately (these days I am programming on PHP) I think I have forgot a few rules of game (afterall PHP is so much more easy than C :P ). Moreover, postgres is the first Open Source software whose code I am interested in. I have never looked into other OSS codes much except correcting a few compilation errors here and there on beta / alpha releases.

I have had the chance and success to compile my own Linux OS and it was fun to do so... but I guess development is a tougher job. With an idea in mind, and a thankful feeling towards postgres is what drives me to do this tougher job.

When I was designing my database for a web app, I found so many problems in MySQL that I could not continue (the best of all, I can't use the commands written in my DB book to create a foreign key, it does not natively support foreign keys, confusing storage engines and so on).. and then I got postgres which I am a fan of.

I hope I will not be flamed when I will ask those questions (some of them are actually very silly ones).

I will look inside the code now and will get back after i get some progress with it.

However, I find too many references to the Data structure "datum" what is it and where is it defined? Can someone tell me please? Also, what role does it play?

Thanks to you all for your replies.

-Vaibhav

Depending on your text editor, you may be able to utilize TAGS files; see src/tools/make_(e|c)tags for creating TAGS files for your editor of choice (emacs/vim, although other editors may support specific formats). This will allow you to navigate to the specific definition of the type/function/macro, and can be very enlightening and help answer some of these questions. `git grep` will also come in handy if you're working directly from a git checkout.

Regards,

David
--
David Christensen
End Point Corporation
david@endpoint.com

Is there something like that in Kdevelop? I dont use CLI editors much.

On Mon, Oct 11, 2010 at 7:55 AM, David Christensen <david@endpoint.com>wrote:

On Mon, Oct 11, 2010 at 04:14:04PM +0530, Vaibhav Kaushal wrote:

Is there something like that in Kdevelop? I dont use CLI editors much.

KDevelop is listed as one of the editors that support ctags.

http://en.wikipedia.org/wiki/Ctags

Cheers,
David.
--
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

David Fetter <david@fetter.org> writes:

On Mon, Oct 11, 2010 at 04:14:04PM +0530, Vaibhav Kaushal wrote:

Is there something like that in Kdevelop? I dont use CLI editors much.

KDevelop is listed as one of the editors that support ctags.

I've just developed some code for the backend and used cscope (from
Emacs directly) which is excellent.

http://cscope.sourceforge.net/
http://kscope.sourceforge.net/
http://www.ziplink.net/~felaco/cbrowser/

Unfortunately the KDE GUI for it seems unmaintained now.

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

---------- Forwarded message ----------
From: Vaibhav Kaushal <vaibhavkaushal123@gmail.com>
Date: Tue, Nov 9, 2010 at 8:24 AM
Subject: Re: [HACKERS] Which file does the SELECT?
To: Dimitri Fontaine <dimitri@2ndquadrant.fr>

I have started with the work and am using Eclipse and it helps quite a lot.
I can find the declarations quite easily. Thanks to open Source.

BTW, I am encountering too many (just too many) data types as I try to
understand the backend (specifically the executor). I do think that its
normal because the executor has to consider almost everything that the other
parts of the DB can possibly command it to do.

Is there some documentation available on the data types / structures which
are in use at the backend?

-Vaibhav (*_*)

On Wed, Oct 13, 2010 at 1:20 AM, Dimitri Fontaine <dimitri@2ndquadrant.fr>wrote:

I have started with the work and am using Eclipse and it helps quite a lot.
I can find the declarations quite easily. Thanks to open Source.

BTW, I am encountering too many (just too many) data types as I try to
understand the backend (specifically the executor). I do think that its
normal because the executor has to consider almost everything that the other
parts of the DB can possibly command it to do.

Is there some documentation available on the data types / structures which
are in use at the backend?

-Vaibhav (*_*)

On Wed, Oct 13, 2010 at 1:20 AM, Dimitri Fontaine <dimitri@2ndquadrant.fr>wrote:

On Mon, Nov 8, 2010 at 9:55 PM, Vaibhav Kaushal
<vaibhavkaushal123@gmail.com> wrote:

I have started with the work and am using Eclipse and it helps quite a lot.
I can find the declarations quite easily. Thanks to open Source.

BTW, I am encountering too many (just too many) data types as I try to
understand the backend (specifically the executor). I do think that its
normal because the executor has to consider almost everything that the other
parts of the DB can possibly command it to do.

Is there some documentation available on the data types / structures which
are in use at the backend?

There's less than one might hope. I think you pretty much have to
look through the README files and source code comments.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Yeah that is what seems to be the best way. The thing is that I am looking
in the PostgreSQL code for the first time and I am not fully aware of the
data structures or the methods / algos implemented in the project. This
makes the work of finding out whats and whys much more difficult. So I asked
it on the list.

Thanks anyways for the reply.

-Vaibhav (*_*)

On Wed, Nov 10, 2010 at 9:38 AM, Robert Haas <robertmhaas@gmail.com> wrote: