Status of 7.2

Thought you might like to know that I should be able to upload regression
test reports for:

IRIX 6.5
FreeBSD 4.4 on Intel
FreeBSD 4.4 on Alpha
VMS on Alpha

For 7.2b2 when it's available. Is Postgres supported on all these
platforms?

Chris

-----Original Message-----
From: pgsql-hackers-owner@postgresql.org
[mailto:pgsql-hackers-owner@postgresql.org]On Behalf Of Bruce Momjian
Sent: Wednesday, 7 November 2001 11:45 AM
To: PostgreSQL-development
Subject: [HACKERS] Status of 7.2

OK, 7.2 is looking _very_ good. We have very few open items. They are:

Source Code Changes
-------------------
Compile in syslog feature by default? (Peter, Tom)
AIX compile (Tatsuo)
Libpq++ compile on Solaris (Peter)

Documentation Changes
---------------------

The always-updated list is at:

ftp://candle.pha.pa.us/pub/postgresql/open_items.

I also have created a post-7.2 list of items that are either patches
that need to be applied or discussed for 7.3. That is at:

http://candle.pha.pa.us/cgi-bin/pgpatches2

This list is longer than usual. Seems we have quite a number of things
in-progress that can be worked on as soon as 7.2 is complete. If there
are things there than can be decided now, please dig in and send an
email to the hackers list.

Once we start 7.3, I will use that list to request patches to complete
these items. Because we are done development on 7.2, people can start
working on patches now. If you send them to the lists, I will load them
up on the page and apply them as soon as 7.3 starts.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

I'll be announcing v7.2b2 tomorrow afternoon ... its packaged and ready to
go, ifyou want to get a head start (ftp.postgresql.org), but am giving a
bit of time for mirrors to catch up ...

On Wed, 7 Nov 2001, Christopher Kings-Lynne wrote:

Is there any list of changes between 7.1.3 and 7.2b2 available?

--
Turbo __ _ Debian GNU Unix _IS_ user friendly - it's just
^^^^^ / /(_)_ __ _ ___ __ selective about who its friends are
/ / | | '_ \| | | \ \/ / Debian Certified Linux Developer
_ /// / /__| | | | | |_| |> < Turbo Fredriksson turbo@tripnet.se
\\\/ \____/_|_| |_|\__,_/_/\_\ Stockholm/Sweden

security counter-intelligence [Hello to all my fans in domestic
surveillance] Soviet Legion of Doom South Africa SEAL Team 6 subway
iodine $400 million in gold bullion Ft. Meade Delta Force killed
attack Waco, Texas
[See http://www.aclu.org/echelonwatch/index.html for more about this]

On Tue, 6 Nov 2001, Bruce Momjian wrote:

I also have created a post-7.2 list of items that are either patches
that need to be applied or discussed for 7.3. That is at:

http://candle.pha.pa.us/cgi-bin/pgpatches2

This list is longer than usual. Seems we have quite a number of things
in-progress that can be worked on as soon as 7.2 is complete. If there
are things there than can be decided now, please dig in and send an
email to the hackers list.

Once we start 7.3, I will use that list to request patches to complete
these items. Because we are done development on 7.2, people can start
working on patches now. If you send them to the lists, I will load them
up on the page and apply them as soon as 7.3 starts.

Sorry, I�m really unable to send patches but I have a feature request
which was addressed in the thread "Serious performance problem" on this
list. It mainly concerns the performance increase if there would be
an index scan method which doesn�t have to check the validity of data
in the table. I�m just waiting for a statement from you guys if you
think it will be doable in 7.3 (while now started to optimize my
database as you suggested ;-).) I think this would increase acceptance
of PostgreSQL for certain people here in Germany which have real influence
on decisions about database in medical diagnostics and care in Germany.

Kind regards

Andreas.

Bruce Momjian wrote:

OK, 7.2 is looking _very_ good. We have very few open items. They are:

Source Code Changes
-------------------
Compile in syslog feature by default? (Peter, Tom)
AIX compile (Tatsuo)
Libpq++ compile on Solaris (Peter)

Documentation Changes
---------------------

The always-updated list is at:

ftp://candle.pha.pa.us/pub/postgresql/open_items.

I also have created a post-7.2 list of items that are either patches
that need to be applied or discussed for 7.3. That is at:

http://candle.pha.pa.us/cgi-bin/pgpatches2

This list is longer than usual. Seems we have quite a number of things
in-progress that can be worked on as soon as 7.2 is complete. If there
are things there than can be decided now, please dig in and send an
email to the hackers list.

I would suggest to schedule my patch (the last on the list) for 7.2 since it
finishes the work I began for 7.2.
Since some patches (part of the work/redesign) are in but the last two are
yet unapplied (IIRC Michael is really busy at the moment), I'd vote for not
leaving this work half-done.

Christof

Is there any list of changes between 7.1.3 and 7.2b2 available?

Sure see /HISTORY in the source tarball.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

This list is longer than usual. Seems we have quite a number of things
in-progress that can be worked on as soon as 7.2 is complete. If there
are things there than can be decided now, please dig in and send an
email to the hackers list.

I would suggest to schedule my patch (the last on the list) for 7.2 since it
finishes the work I began for 7.2.
Since some patches (part of the work/redesign) are in but the last two are
yet unapplied (IIRC Michael is really busy at the moment), I'd vote for not
leaving this work half-done.

OK, this is for ecpg. If you can get an OK from Michael, I will be glad
to apply them.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

I would suggest to schedule my patch (the last on the list) for 7.2 since it
finishes the work I began for 7.2.
Since some patches (part of the work/redesign) are in but the last two are
yet unapplied (IIRC Michael is really busy at the moment), I'd vote for not
leaving this work half-done.

OK, this is for ecpg. If you can get an OK from Michael, I will be glad
to apply them.

More to the point, I don't think it's core's business to overrule
Michael's technical decisions about ecpg. If he thinks the patch
is okay, but hasn't time to apply it, then we can do that for him.
But we won't apply it without his review and okay.

regards, tom lane

On Wed, 7 Nov 2001, Tille, Andreas wrote:

Sorry, I�m really unable to send patches but I have a feature request
which was addressed in the thread "Serious performance problem" on this
list. It mainly concerns the performance increase if there would be
an index scan method which doesn�t have to check the validity of data
in the table. I�m just waiting for a statement from you guys if you
think it will be doable in 7.3 (while now started to optimize my
database as you suggested ;-).) I think this would increase acceptance
of PostgreSQL for certain people here in Germany which have real influence
on decisions about database in medical diagnostics and care in Germany.

Is it possible that hackers do any statement according this issue.
I want to repeat the problem. It�s hard to argue for PostgreSQL (and
I would really like to advocate for PostgreSQL) against MS SQL if we
talk about an imaginary possible dataloss if my colleague has not ever
faced dataloss and certainly know that other power users of MS SQL are
using it. It�s much more hard to argue if there are cases in which
MS SQL outperforms PostgreSQL in the order of magnitude. It�s hard
to convince somebody if I tell him that the reason is his bad database
design. He really isn�t sooo bad and he claims that MS SQL has transparent
transaction *and* fast index usage. Don�t ask me how they do this.
I repeat that my colleague is in the position to decide about software
usage of several medicine related projects in Germany.

I just want to know now if this is an issue for PostgreSQL hackers:

[ ] yes
[ ] no
[ ] we are discussing about that

In case of "no" I would be happy if you could provide me with some
technical reasons which could help me arguing.

Kind regards

Andreas.

I just want to know now if this is an issue for PostgreSQL hackers:
[X] yes
[X] no
[X] we are discussing about that
In case of "no" I would be happy if you could provide me with some
technical reasons which could help me arguing.

The hacker community has a wide range of interests.

From my POV, the overall performance of PostgreSQL is more than
competitive with other database products, including M$SQL. There is not
much point in arguing a specific query case, though we are happy to talk
about specific overall applications and to offer suggestions on how to
build databases that are generally well designed and that will perform
well on more than one product.

If you have a colleague who firmly believes that M$SQL is the best
solution, it sounds like he is not listening to all of the facts. That
certainly can be frustrating, eh? Maybe after a few more years of
crashed machines and increasing costs he will be more open to
alternatives ;)

- Thomas

I just want to know now if this is an issue for PostgreSQL hackers:

[ ] yes
[ ] no
[ ] we are discussing about that

In case of "no" I would be happy if you could provide me with some
technical reasons which could help me arguing.

My guess is that its likely to get discussed again when 7.3 development
starts if someone brings it up. I think right now alot of discussion
is towards the 7.2betas and bugs and stuff that might possibly get put off
that was already talked about earlier in this cycle.

Tille, Andreas writes:

Sorry, I�m really unable to send patches but I have a feature request
which was addressed in the thread "Serious performance problem" on this
list. It mainly concerns the performance increase if there would be
an index scan method which doesn�t have to check the validity of data
in the table.

I just want to know now if this is an issue for PostgreSQL hackers:

[ ] yes
[ ] no
[ ] we are discussing about that

We are always willing to discuss changes that improve performance,
reliability, standards compliance, etc. However, "MS SQL does it, and MS
SQL is fast" is not sufficient proof that a feature would improve average
performance in PostgreSQL. This issue has been brought up with similarly
unsatisfactory arguments in the past, so you should be able to find out
about the discussion in the archives. Some of the arguments against this
change were bigger indexes, slower write operations, non-existent proof
that it's really faster, putting the index on a different disk will mostly
obsolete the issue. Consequently, this is currently not something that
has got a chance to be implemented anytime soon.

--
Peter Eisentraut peter_e@gmx.net

We are always willing to discuss changes that improve performance,
reliability, standards compliance, etc. However, "MS SQL does it, and MS
SQL is fast" is not sufficient proof that a feature would improve average
performance in PostgreSQL. This issue has been brought up with similarly
unsatisfactory arguments in the past, so you should be able to find out
about the discussion in the archives. Some of the arguments against this
change were bigger indexes, slower write operations, non-existent proof
that it's really faster, putting the index on a different disk will mostly
obsolete the issue. Consequently, this is currently not something that
has got a chance to be implemented anytime soon.

I personally would like to have index scans that look up heap rows
record the heap expired status into the index entry via one bit of
storage. This will not _prevent_ checking the heap but it will prevent
heap lookups for index entries that have been exipred for a long time.
However, with the new vacuum, and perhaps autovacuum coming soon, may be
little need for this optimization.

The underlying problem the user is seeing is how to _know_ an index
tuple is valid without checking the heap, and I don't see how to do that
unless we start storing the transaction id in the index tuple, and that
requires extra storage.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

On Fri, 16 Nov 2001, Thomas Lockhart wrote:

The hacker community has a wide range of interests.

For sure, but there will be a raodmap with general consensus of the
hackers.

From my POV, the overall performance of PostgreSQL is more than
competitive with other database products, including M$SQL.

I never doubt you point of view, but it hardly counts as an
argument for my current problem. There is a technical reason
why MS SQL server is faster here and they claim to do it safely.
(Well personally I do not give a cent for thigs that MS claims
about but this does not help here.)

There is not much point in arguing a specific query case,

It is no specific query case. It is the speed of an index scan which
goes like N if you do it with PostgreSQL and it goes like log N if
you do not have to look back into the table like MS SQL server does.

though we are happy to talk
about specific overall applications and to offer suggestions on how to
build databases that are generally well designed and that will perform
well on more than one product.

I doubt that you could care about any database designer who does
poor database design and just does a straigtforeward index scan.
If you think that PostgreSQL is only targeted to high professional
database designers which know how to avoid index scans I doubt that
PostgreSQL will get the user base it would deserve.
I could imagine several cases like my colleague who might think about
porting their application and get into the trap as me that the first
simple question they try performs that badly. I really want to say
that we should address this issue in the documentation. If there
exists such cases we should make it clear *why* PostgreSQL fails
this performance test (and perhaps include your text in your mail
as a base of this documentation). If we ignore that we will not
attrakt users.

If you have a colleague who firmly believes that M$SQL is the best
solution, it sounds like he is not listening to all of the facts.

He is a little bit MS centric but in principle knows the advantage
of OpenSource. On the other hand he is led by pragmatism and just
asks: Which software gives the solution quickly. And he found his
answer.
On the other hand we should also listen to things he presents as
"facts" ...

That certainly can be frustrating, eh?

Yes.

Maybe after a few more years of
crashed machines and increasing costs he will be more open to
alternatives ;)

This does not help currently.
I repeat: We should at least upgrade PostgreSQL documentation to address
those issues.

Kind regards

Andreas.

PS: I prefer not to be CCed if I do not explicite ask for this service.
It seems to be common habit on PostgreSQL lists to CC users. Does
this make any sense? On many other lists such bahaviour is banned.

On Fri, 16 Nov 2001, Peter Eisentraut wrote:

We are always willing to discuss changes that improve performance,
reliability, standards compliance, etc. However, "MS SQL does it, and MS
SQL is fast" is not sufficient proof that a feature would improve average
performance in PostgreSQL. This issue has been brought up with similarly
unsatisfactory arguments in the past, so you should be able to find out
about the discussion in the archives.

Sorry, I do not see any favour for PostgreSQL if we want people who
consider switching to PostgreSQL to search the archive for useful information.
Just stating the issues and principles clearly could convince people.
If not PostgreSQL is faster removed from the list of available
alternatives of database servers than a web browser is fired up.

Kind regards

Andreas.

On Fri, 16 Nov 2001, Bruce Momjian wrote:

I personally would like to have index scans that look up heap rows
record the heap expired status into the index entry via one bit of
storage. This will not _prevent_ checking the heap but it will prevent
heap lookups for index entries that have been exipred for a long time.
However, with the new vacuum, and perhaps autovacuum coming soon, may be
little need for this optimization.

The underlying problem the user is seeing is how to _know_ an index
tuple is valid without checking the heap, and I don't see how to do that
unless we start storing the transaction id in the index tuple, and that
requires extra storage.

For my special case I think doubling main memory is about the same
price as a MS SQL server license. I can�t say which further problems
might occure.

Kind regards

Andreas.

There is not much point in arguing a specific query case,

It is no specific query case. It is the speed of an index scan which
goes like N if you do it with PostgreSQL and it goes like log N if
you do not have to look back into the table like MS SQL server does.

I cannot see why you keep saying that. It is simply not true.
MS SQL shows a behavior of O(N), it is simply, that PostgreSQL
because of well described methodology takes longer per affected row.
The speed difference is linear, no matter how many rows
are affected.

Andreas

On Mon, 19 Nov 2001, Zeugswetter Andreas SB SD wrote:

It is no specific query case. It is the speed of an index scan which
goes like N if you do it with PostgreSQL and it goes like log N if
you do not have to look back into the table like MS SQL server does.

I cannot see why you keep saying that. It is simply not true.
MS SQL shows a behavior of O(N), it is simply, that PostgreSQL
because of well described methodology takes longer per affected row.
The speed difference is linear, no matter how many rows
are affected.

I�m basing my assumption on the statement of my colleague. He
told me that consequent index usage results in O(log N) behaviour.
I�m really no expert in database theory but if you like I can foreward
your question.

Kind regards

Andreas.

Tille, Andreas wrote:

On Mon, 19 Nov 2001, Zeugswetter Andreas SB SD wrote:

It is no specific query case. It is the speed of an index scan which
goes like N if you do it with PostgreSQL and it goes like log N if
you do not have to look back into the table like MS SQL server does.

I cannot see why you keep saying that. It is simply not true.
MS SQL shows a behavior of O(N), it is simply, that PostgreSQL
because of well described methodology takes longer per affected row.
The speed difference is linear, no matter how many rows
are affected.

I�m basing my assumption on the statement of my colleague. He
told me that consequent index usage results in O(log N) behaviour.

Searching through index only vs. searching through index + looking up
each tuple in main
table can be better than linear, if the tuples are scattered throughout
main table.

Searching through index only is probably faster by roughly a factor of
2 * (size_of_heap_tuple/size_of_index_entry) in your case where you want
to count
about half of the rows in table.

----------------
Hannu