Parallel queries
All,
How hard would it be to get a back end to hand off parts of a query tree to
other back ends (which it created), and then collate the results, and take
it from there? Obviously, it would only do this under certain conditions,
specifically, if it was compiled in to the server, and the query tree in
question had a particular profile, i.e.: lent itself to parallel processing.
But surely it couldn't be that difficult to get a back end to hand off, say,
a sub-select sub-tree, or UNION sub-trees. Under heavy loads with the right
hardware, this could gain significant performance improvement.
Michael Ansley
**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
Nick West - Global Infrastructure Manager.
This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.
www.mimesweeper.com
**********************************************************************
How hard would it be to get a back end to hand off parts of a query
tree to other back ends (which it created), and then collate the
results, and take it from there? Obviously, it would only do this
under certain conditions, specifically, if it was compiled in to the
server, and the query tree in question had a particular profile, i.e.:
lent itself to parallel processing. But surely it couldn't be that
difficult to get a back end to hand off, say, a sub-select sub-tree,
or UNION sub-trees. Under heavy loads with the right hardware, this
could gain significant performance improvement.
Right. This would help speed up some single-client queries on SMP
machines (we already scale will with multiple clients on SMP machines).
This could perhaps be solved by threading each client backend, or, as
you suggest, by spawning entirely new processes (which would easily ??
generalize into distributed database support). Bring it up after 7.1 is
released (though there may be enough hoops to jump through that it would
not be in 7.2; not certain yet).
All imho of course, though we've had a few discussions off-list to help
get us thinking about it for planning purposes.
I would assume that it will take a few folks working together to get
this to fly, but feel free to give it a go on your own.
- Thomas