Antw: Re: Postgresql and multithreading
Marc,
not only the global variables are a problem. PostgreSQL doesn't clean
up
all the memory before a process terminates and you must deal with
signals between threads.
I've modified PostgreSQL that it is completely thread based and nearly
clean
with allocating/deallocating memory and I'm using signals between
threads.
I've written the code that it's easy to combine the changes with new
releases of
PostgreSQL but it has some problems that require a decrease of the
number of
global variables. This is very hard if you don't want to drift away
from the
main sourcecode of PostgreSQL.
Actually I have a table that holds all the globals that needs to be
addressed
and I can tell you that there are a lot of global variables.
However. I've tried to give the changes with the memory freeing back to
the
main source tree but my feeling was that they don't like it because it
doesn't
matter on UNIX if you're only process based. The same with several
tools like
PSQL. After this I wanted to look closer at the global variables.
If this may change in the future you would be very happy if you don't
have to
search for unreleased memory because it is solved.
I'm still VERY interested in giving those changes back to the
community,
but i think that my changes are still not wanted.
Another problem I have is that the build tools i'm using are not GNU
and
the project file is different in this case. The developers from
PostgreSQL
don't want this too.
Currently i'm working on a project file that uses GNU make but i'm not
very
happy with this because it is dramatically slower compared to the
Metrowerks Tools. With Codewarrior I can build PostgreSQL in around
2 Minutes with all libraires and so on and GNU make needs much much
longer.
But I still hope that I'll find a way how I can handle all this.
PostgreSQL itslef is running very good with "only threads" and the
database
itself is a VERY GOOD product.
In the future I'll make some tests to find out information about the
performance
and how threads affect the performance on Linux.
I'm also working on several additions to PostgreSQL for remote realtime
Monitoring
and configuration, LDAP authentication and more.
Ulrich
"Marc G. Fournier" <scrappy@hub.org> 18.10.2002 05:21:59 >>>
On Wed, 16 Oct 2002, Anuradha Ratnaweera wrote:
On Wed, Oct 16, 2002 at 01:25:23AM -0400, Bruce Momjian wrote:
Anuradha Ratnaweera wrote:
... what I want to know is whether multithreading is likely to
get
into in postgresql, say somewhere in 8.x, or even in 9.x?
It may be optional some day, most likely for Win32 at first, but we
see
little value to it on most other platforms; of course, we may be
wrong.
In that case, I wonder if it is worth folking a new project to add
threading support to the backend? Of course, keeping in sync with
the
original would be lot of work.
Actually, if you go through the archives, there has been talk about
what
would have to be done to the main source tree towards getting
threading
included ... as well as a lengthy discussion about the steps involved.
the first and foremost issue that needs to be addressed is cleaning up
the
global variables that are used throughout ...
---------------------------(end of
broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/users-lounge/docs/faq.html
----------------------------------
This e-mail is virus scanned
Diese e-mail ist virusgeprueft
On 18 Oct 2002 at 18:10, Ulrich Neumann wrote:
Marc,
not only the global variables are a problem. PostgreSQL doesn't clean
up
all the memory before a process terminates and you must deal with
signals between threads.
OK, first of all let me say this. I am interested in seeing postgresql
mutlithreaded but I don't think it's a good idea to provide a thread per
connection. On slick unices like freebsd/linux, threads and processes are
almost equally heavy. Besides if connection opening time for a database is an
issue, application should cache the pool of connections. Databases are not web
servers which should cater to thousands of connections per minute.
I am interested in seeing postgresql multithreaded so that it performs better
on SMP machine. Right now I am building/designing a *huge* database which is
going to use at least 4 way machines. With number of connections very small,
around 10-15 and being persistent, I am afraid that postgresql would not use
all the available horsepower.
I've modified PostgreSQL that it is completely thread based and nearly
clean
with allocating/deallocating memory and I'm using signals between
threads.
I am sure it must have been massive task.
I'm still VERY interested in giving those changes back to the
community,
but i think that my changes are still not wanted.
Well, I understand your feeling but I am sure it's not the cases. People here
have stated that there are other optimisation areas which needs to be addresses
first. Threading is not ruled out but it's not on current radar either.
I would be more than happy to assist you to make this happen. I have some
experience in threads(pthreads on linux) but absolute no experience in
postgresql code.
Can we make a patch that converts a single module/operation to multithreaded?
Let's say sequential table scan.
I propose that such a conversion should be done under conditional compilation
with default threaded behaviour as off. This would not disturb existing setup
and behaviour where threads are not good enough. Besides since we are
distributing almost exclusive work to threads, there should be little
syncronisation issues.
I request postgresql hackers to look up to this proposal favourably. This
would be a good and gradual way to get threading in postgresql.
Awaiting your feedback..
Bye
Shridhar
--
The Consultant's Curse: When the customer has beaten upon you long enough, give
him what he asks for, instead of what he needs. This is very strong medicine,
and is normally only required once.
On Sat, 19 Oct 2002, Shridhar Daithankar wrote:
On 18 Oct 2002 at 18:10, Ulrich Neumann wrote:
I'm still VERY interested in giving those changes back to the
community, but i think that my changes are still not wanted.
Okay, can you submit such changes in pieces / steps? For instance, I know
there was alot of talk about how to fix/improve the global variable
situation, but nobody appeared to have picked that up and done anything
with it ... if you have, is it something you could submit as a patch for
review?
Note that as the source matures, we've become more critical of patches,
especially those that affect the *whole* server, so if you can submit
patches in manageable chunks, it becomes easier to review and get those
changes in before the next round of changes ...