Feature freeze date for 8.1
You might remember that when we released 8.0, the plan was to have a
12-month development cycle for 8.1, unless there were Win32 problems
that required complex fixes, in which case we would have a shorter 8.1
cycle.
Well the good news is that there have been almost no Win32 problems, but
the other good news is that we are getting a lot of powerful features
for 8.1 already:
o two-phase (Heikki Linnakangas, almost done)
o multiple out function paramters (Tom, done)
o bitmappted indexes (Tom, almost done)
o shared row locks (Alvaro, almost done)
o integrated auto-vacuum (Bruce)
o buffer cache fixes for SMP (Tom, done)
It is possible all these items will be done by sometime in June. Now,
if that happens, do we want to continue with the 12-month plan or
shorten the 8.1 release cycle, perhaps targeting a release in the
September/October timeframe?
The current core proposal is to do feature freeze on July 1, with the
understanding that we will be done most of the items above by then and
have the outstanding patches applied.
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073
As a user, I would definetly prefer to see 8.1
released sooner with the feature set listed below,
than wait another 6+ months for a few other features.
Additionally, the beta may go smoother/faster if you
don't have too many huge features going in at once.
Just my opinion.
Later
Rob
--- Bruce Momjian <pgman@candle.pha.pa.us> wrote:You might remember that when we released 8.0, the
plan was to have a
12-month development cycle for 8.1, unless there
were Win32 problems
that required complex fixes, in which case we would
have a shorter 8.1
cycle.Well the good news is that there have been almost no
Win32 problems, but
the other good news is that we are getting a lot of
powerful features
for 8.1 already:o two-phase (Heikki Linnakangas, almost done)
o multiple out function paramters (Tom, done)
o bitmappted indexes (Tom, almost done)
o shared row locks (Alvaro, almost done)
o integrated auto-vacuum (Bruce)
o buffer cache fixes for SMP (Tom, done)It is possible all these items will be done by
sometime in June. Now,
if that happens, do we want to continue with the
12-month plan or
shorten the 8.1 release cycle, perhaps targeting a
release in the
September/October timeframe?The current core proposal is to do feature freeze on
July 1, with the
understanding that we will be done most of the items
above by then and
have the outstanding patches applied.-- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073---------------------------(end of
broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose
an index scan if your
joining column's datatypes do not match
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Import Notes
Reply to msg id not found: 6667 | Resolved by subject fallback
On Thu, Apr 28, 2005 at 09:02:40 -0400,
Bruce Momjian <pgman@candle.pha.pa.us> wrote:
Well the good news is that there have been almost no Win32 problems, but
the other good news is that we are getting a lot of powerful features
for 8.1 already:
You forgot to list the indexed aggregate feature for max and min. While
this isn't that important for experienced postgres users, it is a gotcha
for new users. Between this, integrated autovacuum and the cross type
index changes in 8.0 we have covered almost all of the newbie gotchas.
This should make Postgres effectively equivalent in difficulty with
getting started for new users as MySQL. That could significantly
boost usage for low end use once word gets out.
Bruce Momjian wrote:
You might remember that when we released 8.0, the plan was to have a
12-month development cycle for 8.1, unless there were Win32 problems
that required complex fixes, in which case we would have a shorter 8.1
cycle.Well the good news is that there have been almost no Win32 problems, but
the other good news is that we are getting a lot of powerful features
for 8.1 already:o two-phase (Heikki Linnakangas, almost done)
o multiple out function paramters (Tom, done)
o bitmappted indexes (Tom, almost done)
o shared row locks (Alvaro, almost done)
o integrated auto-vacuum (Bruce)
o buffer cache fixes for SMP (Tom, done)It is possible all these items will be done by sometime in June. Now,
if that happens, do we want to continue with the 12-month plan or
shorten the 8.1 release cycle, perhaps targeting a release in the
September/October timeframe?The current core proposal is to do feature freeze on July 1, with the
understanding that we will be done most of the items above by then and
have the outstanding patches applied.
It seems to be a good idea to take the chance now to make a release.
Delaying the release would mean preventing the wide usage of features
although they appear production grade. OTOH, if feature freeze is
delayed some seemingly essential features for 8.1 which might arise in
the meantime might delay the release further, or induce late feature
exclusion when some last minute issues are discovered. Integrated
autovacuum seems good enough a reason to release early.
Regards,
Andreas
In the last exciting episode, pgman@candle.pha.pa.us (Bruce Momjian) wrote:
o integrated auto-vacuum (Bruce)
If this can kick off a vacuum of a Very Large Table at an unfortunate
time, this can turn out to be a prety painful misfeature.
What I'd _really_ love to see (and alas, it's beyond my ken) is some
parallel to the FSM, namely a "Recently Updated Blocks Map," which
would enable a vacuuming approach that would not go through entire
tables, but which would rather go through only those blocks known to
be recently updated.
There continues to be trouble if you have a table that grows to 50
million rows where there are 100K rows that are being heavily
updated. In effect, only the 100K rows need facuuming.
--
(reverse (concatenate 'string "moc.liamg" "@" "enworbbc"))
http://linuxfinances.info/info/emacs.html
Group Dynamics
"Following Minsky and Schelling, consider a person as a society of
agents. A group is then a larger society of such agents. Understand
groups by examining interactions of coalitions of agents that
cross-cut their grouping into people."
-- Mark Miller
Christopher Browne <cbbrowne@acm.org> writes:
In the last exciting episode, pgman@candle.pha.pa.us (Bruce Momjian) wrote:
o integrated auto-vacuum (Bruce)
If this can kick off a vacuum of a Very Large Table at an unfortunate
time, this can turn out to be a prety painful misfeature.
[ shrug... ] You'll always be able to turn it off if you don't want it.
I'm not sure that we'll be ready to turn it on by default even in 8.1.
regards, tom lane
Tom Lane wrote:
Christopher Browne <cbbrowne@acm.org> writes:
In the last exciting episode, pgman@candle.pha.pa.us (Bruce Momjian) wrote:
o integrated auto-vacuum (Bruce)
If this can kick off a vacuum of a Very Large Table at an unfortunate
time, this can turn out to be a prety painful misfeature.[ shrug... ] You'll always be able to turn it off if you don't want it.
I'm not sure that we'll be ready to turn it on by default even in 8.1.
Agreed. It will just be there to turn on from postgresql.conf if you
want it, and we do have TODO information about keeping such an FSM for
recently expired pages.
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073
Tom Lane wrote:
Christopher Browne <cbbrowne@acm.org> writes:
If this can kick off a vacuum of a Very Large Table at an unfortunate
time, this can turn out to be a prety painful misfeature.[ shrug... ] You'll always be able to turn it off if you don't want it.
I'm not sure that we'll be ready to turn it on by default even in 8.1.
What to people think about having an optional "maintenance window" so
that autovac only takes action during an approved time. But perhaps
just using the vacuum delay settings will be enough.
On Fri, 29 Apr 2005, Bruno Wolff III wrote:
On Fri, Apr 29, 2005 at 10:09:43 -0400,
Bruce Momjian <pgman@candle.pha.pa.us> wrote:Tom Lane wrote:
Christopher Browne <cbbrowne@acm.org> writes:
In the last exciting episode, pgman@candle.pha.pa.us (Bruce Momjian) wrote:
o integrated auto-vacuum (Bruce)
If this can kick off a vacuum of a Very Large Table at an unfortunate
time, this can turn out to be a prety painful misfeature.[ shrug... ] You'll always be able to turn it off if you don't want it.
I'm not sure that we'll be ready to turn it on by default even in 8.1.Agreed. It will just be there to turn on from postgresql.conf if you
want it, and we do have TODO information about keeping such an FSM for
recently expired pages.I think if we aren't finding problems in testing that it would be better
to turn pg_autovacuum on by default. Vacuum is something the burns new
users and having it one by default is going to cut down on surprises.
Except for the surprise of peridically having the system go unresponsive
because it hit a large table, and that new user wondering what is wrong
with postgresql that it just stalls seemingly randomly :(
----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Import Notes
Reply to msg id not found: 20050429155140.GA7394@wolff.to
On Fri, Apr 29, 2005 at 10:09:43 -0400,
Bruce Momjian <pgman@candle.pha.pa.us> wrote:
Tom Lane wrote:
Christopher Browne <cbbrowne@acm.org> writes:
In the last exciting episode, pgman@candle.pha.pa.us (Bruce Momjian) wrote:
o integrated auto-vacuum (Bruce)
If this can kick off a vacuum of a Very Large Table at an unfortunate
time, this can turn out to be a prety painful misfeature.[ shrug... ] You'll always be able to turn it off if you don't want it.
I'm not sure that we'll be ready to turn it on by default even in 8.1.Agreed. It will just be there to turn on from postgresql.conf if you
want it, and we do have TODO information about keeping such an FSM for
recently expired pages.
I think if we aren't finding problems in testing that it would be better
to turn pg_autovacuum on by default. Vacuum is something the burns new
users and having it one by default is going to cut down on surprises.
Experienced users know about vacuum and will probably read the release
notes when upgrading and do something that is appropiate for them.
"Matthew T. O'Connor" <matthew@zeut.net> writes:
What to people think about having an optional "maintenance window" so
that autovac only takes action during an approved time. But perhaps
just using the vacuum delay settings will be enough.
I'm not sure autovac should go completely catatonic during the day;
what if someone does an unusual mass deletion, or something? But
it does seem pretty reasonable to have a notion of a maintenance
window where it should be more active than it is at other times.
Maybe what you want is two complete sets of autovac parameters.
Definitely at least two sets of the vacuum-delay values.
regards, tom lane
On Fri, Apr 29, 2005 at 12:43:37 -0300,
"Marc G. Fournier" <scrappy@postgresql.org> wrote:
On Fri, 29 Apr 2005, Bruno Wolff III wrote:
Except for the surprise of peridically having the system go unresponsive
because it hit a large table, and that new user wondering what is wrong
with postgresql that it just stalls seemingly randomly :(
I think most users running systems with that large of tables will know
what they are doing. And will be more careful with vacuum.
Vacuum running for a long time on large tables with few pages that need
updating is really a separate problem that could use its own solution.
On Fri, 29 Apr 2005, Tom Lane wrote:
"Matthew T. O'Connor" <matthew@zeut.net> writes:
What to people think about having an optional "maintenance window" so
that autovac only takes action during an approved time. But perhaps
just using the vacuum delay settings will be enough.I'm not sure autovac should go completely catatonic during the day;
what if someone does an unusual mass deletion, or something? But
it does seem pretty reasonable to have a notion of a maintenance
window where it should be more active than it is at other times.Maybe what you want is two complete sets of autovac parameters.
Definitely at least two sets of the vacuum-delay values.
With the introduction of the stats collector, is there not some way of
extending it so that autovac has more information to work off of? For
instance, in my environment, we have clients in every timezone hitting the
database ... our Japanese clients will be busy at a totally different time
of day then our East Coast/NA clients, so a 'maintenance window' is near
impossible to state ...
I know one person was talking about being able to target only those that
pages that have changes, instead of the whole table ... but some sort of
"load monitoring" that checks # of active connections and tries to find
'lulls'?
Basically, everything right now is being keyed to updates to the tables
themselves, but isn't looking at what the system itself is doing ... if
I'm doing a massive import of data into a table, the last time I want is a
VACUUM to cut in and slow down the loading ...
----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Martha Stewart called it a Good Thing when scrappy@postgresql.org ("Marc G. Fournier") wrote:
I know one person was talking about being able to target only those
that pages that have changes, instead of the whole table ... but some
sort of "load monitoring" that checks # of active connections and
tries to find 'lulls'?
I have some "log table purging" processes I'd like to put in place; it
would be really slick to be able to get some statistics from the
system as to how busy the DB has been in the last little while.
The nice, adaptive algorithm:
- Loop forever
- Once a minute, evaluate how busy things seem, giving some metric X
-> If X is "high" then purge 10 elderly tuples from table log_table
-> If X is "moderate" then purge 100 elderly tuples from table
log_table
-> If X is "low" then purge 1000 elderly tuples from table
log_table
The trouble is in measuring some form of "X."
Some reasonable approximations might include:
- How much disk I/O was recorded in the last 60 seconds?
- How many application transactions (e.g. - invoices or such) were
issued in the last 60 seconds (monitoring a sequence could be
good enough).
--
output = reverse("gro.mca" "@" "enworbbc")
http://linuxfinances.info/info/slony.html
?OM ERROR
I think what you're suggesting is that vacuum settings (most likely
delay) take into consideration the load on the database, which I think
is a great idea. One possibility is if vacuum tracks how many blocks
it's read/written, it can see how many blocks the database has done
overall; subtract the two and you know how much other disk IO is going
on in the system. You can then use that number to decide how long you'll
sleep before the next vacuum cycle.
On Fri, Apr 29, 2005 at 01:34:56PM -0300, Marc G. Fournier wrote:
On Fri, 29 Apr 2005, Tom Lane wrote:
"Matthew T. O'Connor" <matthew@zeut.net> writes:
What to people think about having an optional "maintenance window" so
that autovac only takes action during an approved time. But perhaps
just using the vacuum delay settings will be enough.I'm not sure autovac should go completely catatonic during the day;
what if someone does an unusual mass deletion, or something? But
it does seem pretty reasonable to have a notion of a maintenance
window where it should be more active than it is at other times.Maybe what you want is two complete sets of autovac parameters.
Definitely at least two sets of the vacuum-delay values.With the introduction of the stats collector, is there not some way of
extending it so that autovac has more information to work off of? For
instance, in my environment, we have clients in every timezone hitting the
database ... our Japanese clients will be busy at a totally different time
of day then our East Coast/NA clients, so a 'maintenance window' is near
impossible to state ...I know one person was talking about being able to target only those that
pages that have changes, instead of the whole table ... but some sort of
"load monitoring" that checks # of active connections and tries to find
'lulls'?Basically, everything right now is being keyed to updates to the tables
themselves, but isn't looking at what the system itself is doing ... if
I'm doing a massive import of data into a table, the last time I want is a
VACUUM to cut in and slow down the loading ...----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
--
Jim C. Nasby, Database Consultant decibel@decibel.org
Give your computer some brain candy! www.distributed.net Team #1828
Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"
On Fri, 29 Apr 2005, Christopher Browne wrote:
Martha Stewart called it a Good Thing when scrappy@postgresql.org ("Marc G. Fournier") wrote:
I know one person was talking about being able to target only those
that pages that have changes, instead of the whole table ... but some
sort of "load monitoring" that checks # of active connections and
tries to find 'lulls'?I have some "log table purging" processes I'd like to put in place; it
would be really slick to be able to get some statistics from the
system as to how busy the DB has been in the last little while.The nice, adaptive algorithm:
- Loop forever
- Once a minute, evaluate how busy things seem, giving some metric X
-> If X is "high" then purge 10 elderly tuples from table log_table
-> If X is "moderate" then purge 100 elderly tuples from table
log_table
-> If X is "low" then purge 1000 elderly tuples from table
log_tableThe trouble is in measuring some form of "X."
Some reasonable approximations might include:
- How much disk I/O was recorded in the last 60 seconds?
- How many application transactions (e.g. - invoices or such) were
issued in the last 60 seconds (monitoring a sequence could be
good enough).
Some way of doing a 'partial vacuum' would be nice ... where a VACUUM
could stop after it processed those '10 elderly tuples' and on the next
pass, resume from that point instead of starting from the beginning again
...
----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Marc G. Fournier wrote:
On Fri, 29 Apr 2005, Christopher Browne wrote:
Some reasonable approximations might include:
- How much disk I/O was recorded in the last 60 seconds?
- How many application transactions (e.g. - invoices or such) were
issued in the last 60 seconds (monitoring a sequence could be
good enough).Some way of doing a 'partial vacuum' would be nice ... where a VACUUM
could stop after it processed those '10 elderly tuples' and on the
next pass, resume from that point instead of starting from the
beginning again ...
That is sorta what the vacuum delay settings accomplish.
The world rejoiced as matthew@zeut.net ("Matthew T. O'Connor") wrote:
Marc G. Fournier wrote:
On Fri, 29 Apr 2005, Christopher Browne wrote:
Some reasonable approximations might include:
- How much disk I/O was recorded in the last 60 seconds?
- How many application transactions (e.g. - invoices or such) were
issued in the last 60 seconds (monitoring a sequence could be
good enough).Some way of doing a 'partial vacuum' would be nice ... where a
VACUUM could stop after it processed those '10 elderly tuples' and
on the next pass, resume from that point instead of starting from
the beginning again ...That is sorta what the vacuum delay settings accomplish.
What they do is orthogonal to that.
"Vacuum delay" prevents vacuum I/O from taking over the I/O bus.
Unfortunately, if you have a table with a very large number of _live_
tuples, there is no way to skip over those and only concentrate on the
dead ones.
In that scenario "vacuum delay" leads to the vacuum on the table
running for a Very, Very Long Time, because it sits there delaying a
lot as it walks thru pages it never modifies. The one good news is
that, for any pages where no tuples are touched, the indices are also
left untouched.
--
wm(X,Y):-write(X),write('@'),write(Y). wm('cbbrowne','gmail.com').
http://linuxdatabases.info/info/slony.html
"The Board views the endemic use of PowerPoint briefing slides instead
of technical papers as an illustration of the problematic methods of
technical communication at NASA." -- Official report on the Columbia
shuttle disaster.
Hi,
What to people think about having an optional "maintenance window" so
that autovac only takes action during an approved time.
This sounds like a realy good idea to me!
Sander.
We have talked about performance and some new features
before freeze of 8.1. Like ;
� Bitmap indexes
� Autovacuum
� GIS features
� Object-Oriented features
� PITR
� Table Partition
But there is a feature that is too important for a
database. It is availability.Now PostgreSQL doesn't have
high availability.We must discuss it here. Imagine a
database that has a lots of features that others don�t
have. I tested the PostgreSQL for that feature, i couldn't
find it enough. Here :
Process A start to update / insert some rows in a table
and then the connection of process A is lost to PostgreSQL
before it sends commit or rollback. Other processes want to
update the same rows or SELECT �..FOR UPDATE for the same
rows.Now these processes are providing SELECT WAITING� or
CANCEL QUERY if statement_timeout was set. Imagine these
processes is getting grower. What will we do now ?
Restarting backend or finding process A and kill it ?
Now, do you think that the PostgreSQL database is a high
available database ? A feature must be added to solve that
problem or PostgreSQL databases would never get a good
place among huge databases.
Best Regards
Adnan DURSUN
ASRIN Bilişim Ltd.
Ankara /TURKEY
