Re: 7.4?

On Tue, 2003-02-25 at 23:53, Ed L. wrote:

On Tuesday February 25 2003 11:52, Tom Lane wrote:

Also, there are nontrivial licensing issues involved. The PG-R
design depends on an underlying "group communication" system, which
is a nontrivial bit of software that none of the core team wants to
rewrite. But none of the available GC systems are BSD-license open
source. We had had some hopes of getting Spread to offer BSD
terms, but that seems to have fallen through. So right now, PG-R
is on the outside looking in, as far as inclusion in the core
distribution goes :-(

Is anyone aware of particular reasons why the group is pushing on a
syncronous solution? I'm sure they have good reasons, but I would've
assumed an asyncronous solution would be far more applicable for
simple redundancy as opposed to syncronicity for high-performance
clusters, not too mention being far simpler implementations.

Some business needs absolutely must have synchronus replication. You're
right that most users want async so they can have clusters serving data
out, but there are some very good reasons for sync:

1. Financial transactions MUST have off-site sync replication.

2. If you have a cluster of servers, sync guarantees that all the data
are in a consistent state. Ease of administration and reliability may
be worth the slight performance penalty.

3. For servers on a fast LAN, sync may not make much of a difference on
performance.

I personally think that sync should be used in more cases than is
generally thought. We can get very reliable LANs and even WAN links,
and if I know that the data are always consistent, that makes my life
much much easier than having to worry about cases where they are not.

On Wed, 2003-02-26 at 07:08, Tom Lane wrote:

Back when I was working for Great Bridge and got to spend a fair amount
of time at trade shows talking to potential customers, the thing we
heard over and over again was that people wanted multiple servers for
reliability/redundancy. That goal seems to me to be best served by a
symmetric multi-master configuration, which is difficult if not
impossible to do with async replication.

I'm glad to hear that PG is heading in that direction. Think of it this
way: the entire reason we use databases instead of a mess of text files
is because of the benefits we get in terms of data consistency. ACID
forms the foundation for a reliable service, whether it's financial
transactions or a chat room. It is difficult or impossible to maintain
ACID in an async situation, I believe.

Sync is actually much more useful because it lets a business have a
cluster of servers without having to worry about what state things are
in.

If it ain't broke,
don't fix it!

2/24/2003 11:36:33 AM, Dmitry Tkach <dmitry@openratings.com> wrote:

Do you guys have any tentative estimates about when 7.4 is going to get released?
What is the timeframe? Also, what is going to be there that is not in 7.3?
I am currently on 7.2, and trying to find out whether it makes sense to upgrade to 7.3 now or to

wait for 7.4 and go right there...

On Mon, Feb 24, 2003 at 14:36:33 -0500,
Dmitry Tkach <dmitry@openratings.com> wrote:

Do you guys have any tentative estimates about when 7.4 is going to get
released?
What is the timeframe? Also, what is going to be there that is not in 7.3?
I am currently on 7.2, and trying to find out whether it makes sense to
upgrade to 7.3 now or to wait for 7.4 and go right there...

My opinion is that you should go to 7.3 now.

7.4 seems to have a ways to go before it is even going to be in beta, so it
will be a minimum of several months before it is released.

The change from 7.3 to 7.4 will likely be easier than going from
7.2 to 7.3 (because schemas are a big change in 7.3).

Could someone summarize what would be the 3 changes that would have the
most impact in 7.4?

- Ericson Smith
eric@did-it.com

On Mon, 2003-02-24 at 14:05, Bruno Wolff III wrote:

On Mon, Feb 24, 2003 at 14:36:33 -0500,
Dmitry Tkach <dmitry@openratings.com> wrote:

Do you guys have any tentative estimates about when 7.4 is going to get
released?
What is the timeframe? Also, what is going to be there that is not in 7.3?
I am currently on 7.2, and trying to find out whether it makes sense to
upgrade to 7.3 now or to wait for 7.4 and go right there...

My opinion is that you should go to 7.3 now.

7.4 seems to have a ways to go before it is even going to be in beta, so it
will be a minimum of several months before it is released.

The change from 7.3 to 7.4 will likely be easier than going from
7.2 to 7.3 (because schemas are a big change in 7.3).

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

--
Ericson Smith <eric@did-it.com>

Dennis Gearon wrote:

If it ain't broke,
don't fix it!

Yeah... I know. That's exactly where I am coming from :-)
I am sitting on about a 200G database here, and I know what a pain in
the neck it is going to be to migrate it (plus, I've got some "C"
functions, that, as I figured recently do not even compile with 7.3) :-(

But there is some stuff in 7.3 that I really miss - like function
returning tuples, prepared and saved query plans, vacuum (full)
behaviour etc...
Besides, Tom Lane has said some really scarry things about possible data
corruption/loss stuff in 7.2...

So, it sounds like I am going to have to go through this upgrade
nightmare... the only question is - when it is a good time to do it...

Dima

On Mon, 2003-02-24 at 14:29, Ericson Smith wrote:

Could someone summarize what would be the 3 changes that would have the
most impact in 7.4?

Impact to who? :-)

1. Native Windows Support
2. Point In Time Recovery
3. Standard Replication Solution

(Please note that while these are all currently planned for 7.4, and are
actively being developed, I can't guarantee they will all be there.)

Robert Treat

On 24 Feb 2003, Ericson Smith wrote:

Could someone summarize what would be the 3 changes that would have the
most impact in 7.4?

I would say the biggest impact from 7.2 to 7.3 are:

- Schemas and their support
- Indexing issues related to using C locale or anything else
- The less greedy type matching changes result in some indexes being
ignored that were once used.
- Old sequences and other things won't inherit dependency. not a real
problem, just an fyi.

From 7.3 to 7.4 the biggest changes planned so far:
- windows native support <-- gartner gives it a 90% chance of making it
into the main disto... :-)
- fix for in() performance issue <-- already in CVS tip
- partitioning or tablespaces <-- who know? seriously, who?
- vacuum will be able to compact out unused space <-- Tom Lane
- possible newer settings for the default postgresql.conf could
increase load on some machines to a point they may need to be reconfigured
to come up.

Data wise I don't think there's gonna be a lot that will mess up a 7.3 to
7.4 conversion so far that's gone by.

From 7.3 to 7.4 the biggest changes planned so far:
- windows native support <-- gartner gives it a 90% chance of making it
into the main disto... :-)
- fix for in() performance issue <-- already in CVS tip
- partitioning or tablespaces <-- who know? seriously, who?
- vacuum will be able to compact out unused space <-- Tom Lane

Will we be able to run the new vacuum system without locking the tables?

Currently our database is at about 15gigs. Over the period of a month,
it grows to about 25gigs (10Gb wasted space). Every month we have a
cleanup routine which involves copying the most actively updated tables
to text, and importing the data again. We vacuum every night and analyze
4 times per day, but we're loath to do a VACUUM FULL because of the
table locking issues (locking some of the tables would break the
operation of some of our 24/7 systems), hence we prefer to stop the db
about once per month, eat the downtime as scheduled (about 1.5 hours),
and get back to business for the next 30 days again.

We're all behind Tom on this change :-)

- Ericson Smith
eric@did-it.com

--
Ericson Smith <eric@did-it.com>

On 24 Feb 2003, Ericson Smith wrote:

From 7.3 to 7.4 the biggest changes planned so far:
- windows native support <-- gartner gives it a 90% chance of making it
into the main disto... :-)
- fix for in() performance issue <-- already in CVS tip
- partitioning or tablespaces <-- who know? seriously, who?
- vacuum will be able to compact out unused space <-- Tom Lane

Will we be able to run the new vacuum system without locking the tables?

Vacuum only locks the tables now for a second or so.

You might want to look at using regular vacuums more often. In my
testing, vacuum (non full) uses <2% of the CPU on a heavily loaded server.
Between the low load of vacuum and having higher fsm settings, you might
obviate the need to regular vacuum any more than once a day or week or
whatever.

Do you guys have any tentative estimates about when 7.4 is going to get

released?

What is the timeframe? Also, what is going to be there that is not in

7.3?

I am currently on 7.2, and trying to find out whether it makes sense to

upgrade to 7.3 now or to

wait for 7.4 and go right there...

Hello
Am I wrrong if I said that with 7.3.3 you can passed up to 32 parameters to
a function and only 16 with 7.2.?

Regards
Luc

Hi all,

I am having trouble reindexing our DB.

We are running PostgreSQL 7.2.1 on i686-pc-linux-gnu, compiled by GCC gcc
(GCC) 3.2.1

backend> reindex database evolvo force
NOTICE: relation 1247 was reindexed
NOTICE: relation 1249 was reindexed
NOTICE: relation 1259 was reindexed
NOTICE: relation 1261 was reindexed
NOTICE: relation 1262 was reindexed
ERROR: cannot create pg_inherits_relid_seqno_index: File exists

Can someone help me.

Thanks

Andrew Bartley

On Mon, Feb 24, 2003 at 03:53:14PM -0500, Ericson Smith wrote:

Will we be able to run the new vacuum system without locking the tables?

I believe the new vacuum stuff is just to help on btree indexes. The
current vacuum doesn't lock the tables, either.

Currently our database is at about 15gigs. Over the period of a month,
it grows to about 25gigs (10Gb wasted space). Every month we have a
cleanup routine which involves copying the most actively updated tables
to text, and importing the data again. We vacuum every night and analyze
4 times per day, but we're loath to do a VACUUM FULL because of the
table locking issues (locking some of the tables would break the

It sounds like either your free space map is not set correctly, or
you have ever-growing indexes. Reindexing would be enough for this
problem. There are still some 24/7-operation problems with having to
reindex.

A

-- 
----
Andrew Sullivan                         204-4141 Yonge Street
Liberty RMS                           Toronto, Ontario Canada
<andrew@libertyrms.info>                              M2P 2A8
                                         +1 416 646 3304 x110

On Tue, 2003-02-25 at 08:18, Andrew Sullivan wrote:

I believe the new vacuum stuff is just to help on btree indexes. The
current vacuum doesn't lock the tables, either.

I was actually thinking about a VACUUM FULL. Currently we have not
problems doing a regular VACUUM. That said, will the new vacuum free as
much space like the current vacuum full, without the handicap of table
locking?

It sounds like either your free space map is not set correctly, or
you have ever-growing indexes. Reindexing would be enough for this
problem. There are still some 24/7-operation problems with having to
reindex.

We will definitely try the reindex as suggested.

--
Ericson Smith <eric@did-it.com>

On Tue, Feb 25, 2003 at 09:41:13AM -0500, Ericson Smith wrote:

I was actually thinking about a VACUUM FULL. Currently we have not

VACUUM FULL will always block. To make a rather nasty comparison,
it's like defragging your disk under Windows: you can't really access
a file which is being moved around.

problems doing a regular VACUUM. That said, will the new vacuum free as
much space like the current vacuum full, without the handicap of table
locking?

Yes and no. VACUUM FULL recovers space absolutely. So if you know
that the table has really shrunk, and shrunk permanently (or similar
cases, like 100% of the table was replaced), then you need VACUUM
FULL. Non-blocking VACUUM will make the freed space available to
Postgres, but not to the filesystem in general. In other words, the
regular VACUUM should mean that your table size stabilises, given
that your database is always more or less the same number of tuples;
but it will be slightly larger on disk than that number of tuples
strictly requires. (Is that clear? If not, maybe someone else can
make it clearer.)

A
--
----
Andrew Sullivan 204-4141 Yonge Street
Liberty RMS Toronto, Ontario Canada
<andrew@libertyrms.info> M2P 2A8
+1 416 646 3304 x110

"Andrew Bartley" <abartley@evolvosystems.com> writes:

We are running PostgreSQL 7.2.1 on i686-pc-linux-gnu, compiled by GCC gcc
(GCC) 3.2.1

backend> reindex database evolvo force
NOTICE: relation 1247 was reindexed
NOTICE: relation 1249 was reindexed
NOTICE: relation 1259 was reindexed
NOTICE: relation 1261 was reindexed
NOTICE: relation 1262 was reindexed
ERROR: cannot create pg_inherits_relid_seqno_index: File exists

If you've been running this database long enough to wrap around the OID
counter, then this failure is possible (though I wouldn't have thought
it probable). Try a few times and it'll probably succeed eventually.

regards, tom lane

On Mon, 2003-02-24 at 15:28, Robert Treat wrote:

3. Standard Replication Solution

I'd be quite surprised to see this in 7.4. Not that I wouldn't love to
have someone contribute it -- I just wouldn't bet on it being in 7.4 if
the beta date is around the middle of April.

Cheers,

Neil
--
Neil Conway <neilc@samurai.com> || PGP Key ID: DB3C29FC

On Sat, 2003-02-22 at 07:56, Joe Tomcat wrote:

On Mon, 2003-02-24 at 12:28, Robert Treat wrote:

On Mon, 2003-02-24 at 14:29, Ericson Smith wrote:

Could someone summarize what would be the 3 changes that would have the
most impact in 7.4?

Impact to who? :-)

1. Native Windows Support
2. Point In Time Recovery
3. Standard Replication Solution

#3 is a very big deal for business users. I am very glad to hear it is
in 7.4. Point in time recovery is also an important thing for biz
users.

For the record let me re-emphasize that none of these are guaranteed to
be in 7.4. They are all actively being worked on, and the current hopes
of the developers is that they will all be in, but it is quite possible
that one of these might not make it in (and most likely to not make
would be replication imho)

Robert Treat

Neil Conway writes:

I'd be quite surprised to see this in 7.4. Not that I wouldn't love to
have someone contribute it -- I just wouldn't bet on it being in 7.4 if
the beta date is around the middle of April.

When someone says that beta is planned April-ish, that means beta will be
in June and the final release will be in September. Mark my words.

--
Peter Eisentraut peter_e@gmx.net