PITR Dead horse?

Austin Gonyou <austin@coremetrics.com> writes:

Has this been beaten to death now? Just curious if PITR was in Dev tree
yet. Been out of the loop. TIA.

Nope... I've got some patches from Patrick Macdonald and JR Nield that I
need to integrate, but I believe those only cover some low-level changes
to the WAL log contents. There's a lot of management code yet to be
written.

regards, tom lane

Has this been beaten to death now? Just curious if PITR was in Dev tree
yet. Been out of the loop. TIA.

I and my co workers are very interested in implementing PITR. We will
tackle this for 7.5 if no one objects.
--
Tatsuo Ishii

I and some other developers are also interested in.
Do you think we can work together?

Tatsuo Ishii <t-ishii@sra.co.jp> wrote:

Has this been beaten to death now? Just curious if PITR was in Dev tree
yet. Been out of the loop. TIA.

I and my co workers are very interested in implementing PITR. We will
tackle this for 7.5 if no one objects.
--
Tatsuo Ishii

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

--
NAGAYASU Satoshi <snaga@snaga.org>

I and some other developers are also interested in.
Do you think we can work together?

Tatsuo Ishii <t-ishii@sra.co.jp> wrote:

Has this been beaten to death now? Just curious if PITR was in Dev tree
yet. Been out of the loop. TIA.

I and my co workers are very interested in implementing PITR. We will
tackle this for 7.5 if no one objects.
--
Tatsuo Ishii

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

--
NAGAYASU Satoshi <snaga@snaga.org>

Tatsuo Ishii <t-ishii@sra.co.jp> writes:

I and my co workers are very interested in implementing PITR. We will
tackle this for 7.5 if no one objects.

Sounds good. I'll try to push in the work that Patrick and JR did
within the next day or two, and then you can take it from there...

regards, tom lane

I and some other developers are also interested in.
Do you think we can work together?

Sure. Why not. I think it would be practical to decide who is the
leader of this project, though.
--
Tatsuo Ishii

Hi, This is Suzuki from NTT DATA Intellilink.

I told Bruce Momjan that I and my colleagues are interested in
implementing PITR in BOF in NY LW2004. NTT's laboratory is very
interested in this issue and I'm planning to work with them. I hope we
could cooperate.

Tatsuo Ishii wrote:

I would like to join this effort too.
I was afraid that people at RedHat are already
halfway though and were to release their work
shortly. But it does not seem to be the case.

Regards,
Nicolai

-----Original Message-----
From: pgsql-hackers-owner@postgresql.org [mailto:pgsql-hackers-
owner@postgresql.org] On Behalf Of Koichi Suzuki
Sent: Wednesday, February 04, 2004 11:25 AM
To: Tatsuo Ishii
Cc: austin@coremetrics.com; pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] PITR Dead horse?

Hi, This is Suzuki from NTT DATA Intellilink.

I told Bruce Momjan that I and my colleagues are interested in
implementing PITR in BOF in NY LW2004. NTT's laboratory is very
interested in this issue and I'm planning to work with them. I hope

we

could cooperate.

Tatsuo Ishii wrote:

Has this been beaten to death now? Just curious if PITR was in Dev

tree

yet. Been out of the loop. TIA.

I and my co workers are very interested in implementing PITR. We

will

tackle this for 7.5 if no one objects.
--
Tatsuo Ishii

---------------------------(end of

broadcast)---------------------------

TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to

majordomo@postgresql.org)

---------------------------(end of

broadcast)---------------------------

TIP 1: subscribe and unsubscribe commands go to

majordomo@postgresql.org

On Wed, 4 Feb 2004, Tatsuo Ishii wrote:

I and some other developers are also interested in.
Do you think we can work together?

Sure. Why not. I think it would be practical to decide who is the
leader of this project, though.

Is this something large enough, like the win32 stuff, that having a side
list for discussions is worth setting up?

----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664

"Marc G. Fournier" <scrappy@postgresql.org> writes:

Is this something large enough, like the win32 stuff, that having a side
list for discussions is worth setting up?

In terms of the amount of code to be written, I expect it's larger than
the win32 porting effort. And it should be mostly pretty separate from
hacking the core backend, since most of what remains to do is writing
external management utilities (I think).

I've been dissatisfied with having the separate pgsql-hackers-win32
list; I feel it just fragments the discussion, and people tend to end up
crossposting to -hackers anyway. But a separate list for PITR work
might be a good idea despite that experience, since it seems like it'd
be a more separable project.

Any other opinions out there?

regards, tom lane

Tom Lane wrote
"Marc G. Fournier" <scrappy@postgresql.org> writes:

Is this something large enough, like the win32 stuff, that having a

side

list for discussions is worth setting up?

In terms of the amount of code to be written, I expect it's larger

than

the win32 porting effort. And it should be mostly pretty separate

from

hacking the core backend, since most of what remains to do is writing
external management utilities (I think).

Yes it is! I'd like to start the discussion about PITR and try to go
through some functional requirements and how those might be implemented.
The Win32 port has a self-evident set of functional requirements; I'm
not sure that the PITR stuff is as clear - so I couldn't pass any
judgement at all (even if I did know the code well enough) on how big a
coding task that is, but I can see that the analysis and discussion is
large indeed.

I've been dissatisfied with having the separate pgsql-hackers-win32
list; I feel it just fragments the discussion, and people tend to end

up

crossposting to -hackers anyway. But a separate list for PITR work
might be a good idea despite that experience, since it seems like it'd
be a more separable project.

I'd vote for a new list dedicated to discussing "Robustness" issues,
such as PITR and the fsync/sync issues. IMHO, PostgreSQL has the
Functionality and Performance, it just needs some rock-solid analysis of
where-things-can-go-wrong with it, so that the business data centre
people will be able to use it with absolute confidence...even if the
answer is "we've got every base covered". For me, the issues about
robustness are as much to do with risk reduction and confidence building
as they are about specific features in that area. [Wow, I expect some
flames on those comments!]

The list probably would remain clearly differentiated, in the same way
[Performance] covers lots of areas not discussed in [Hackers].

Not hung up on the name either, just something that indicates
breadth-of-scope, e.g. Availability or Data Protection or Resilience
etc..; maybe the Advocates would like to name it? It might even be a
press-release: "PostgreSQL community focuses new efforts towards
Robustness features for its next major release".

Best Regards, Simon Riggs

"Simon Riggs" <simon@2ndquadrant.com> writes:

I'd vote for a new list dedicated to discussing "Robustness" issues,
such as PITR and the fsync/sync issues. IMHO, PostgreSQL has the
Functionality and Performance, it just needs some rock-solid analysis of
where-things-can-go-wrong with it, so that the business data centre
people will be able to use it with absolute confidence...even if the
answer is "we've got every base covered". For me, the issues about
robustness are as much to do with risk reduction and confidence building
as they are about specific features in that area. [Wow, I expect some
flames on those comments!]

You're right. Exactly where do you expect to find the expertise and
interest to do such an analysis? On pghackers, that's where. There's
no reason to invent a new mailing list for what should be a continuing
topic in pghackers. And to the extent that you were to move such a
discussion somewhere else, you'd just risk losing the attention of the
pair of eyeballs that might notice a hole in your analysis.

Not hung up on the name either, just something that indicates
breadth-of-scope, e.g. Availability or Data Protection or Resilience
etc..; maybe the Advocates would like to name it? It might even be a
press-release: "PostgreSQL community focuses new efforts towards
Robustness features for its next major release".

I think such a press release would be counterproductive, as it would
immediately make people question whether we have reliability problems.

regards, tom lane

Totally agree. Robustness and rock-solidness are the only
things missing for PostgreSQL to become the killer of certain
commercial enterprise databases out there. And the only thing
that is missing in this respect is PITR. PITR should be there
INGRES had it in '84 and some people as why PostgreSQL does
not have it.

I am well versed in the internals of "PITR" features of a certain
leading enterprise-class database out there. And would like to
contribute (write code) to this effort as much as I can.

Best regards,
Nicolai Tufar

-----Original Message-----
From: pgsql-hackers-owner@postgresql.org [mailto:pgsql-hackers-
owner@postgresql.org] On Behalf Of Simon Riggs
Sent: Thursday, February 05, 2004 1:33 AM
To: 'Tom Lane'; 'Marc G. Fournier'
Cc: 'Tatsuo Ishii'; snaga@snaga.org; austin@coremetrics.com; pgsql-
hackers@postgresql.org
Subject: Re: [HACKERS] PITR Dead horse?

Tom Lane wrote
"Marc G. Fournier" <scrappy@postgresql.org> writes:

Is this something large enough, like the win32 stuff, that having

a

side

list for discussions is worth setting up?

In terms of the amount of code to be written, I expect it's larger

than

the win32 porting effort. And it should be mostly pretty separate

from

hacking the core backend, since most of what remains to do is

writing

external management utilities (I think).

Yes it is! I'd like to start the discussion about PITR and try to go
through some functional requirements and how those might be

implemented.

The Win32 port has a self-evident set of functional requirements; I'm
not sure that the PITR stuff is as clear - so I couldn't pass any
judgement at all (even if I did know the code well enough) on how big

a

coding task that is, but I can see that the analysis and discussion is
large indeed.

I've been dissatisfied with having the separate pgsql-hackers-win32
list; I feel it just fragments the discussion, and people tend to

end

up

crossposting to -hackers anyway. But a separate list for PITR work
might be a good idea despite that experience, since it seems like

it'd

be a more separable project.

I'd vote for a new list dedicated to discussing "Robustness" issues,
such as PITR and the fsync/sync issues. IMHO, PostgreSQL has the
Functionality and Performance, it just needs some rock-solid analysis

of

where-things-can-go-wrong with it, so that the business data centre
people will be able to use it with absolute confidence...even if the
answer is "we've got every base covered". For me, the issues about
robustness are as much to do with risk reduction and confidence

building

as they are about specific features in that area. [Wow, I expect some
flames on those comments!]

The list probably would remain clearly differentiated, in the same way
[Performance] covers lots of areas not discussed in [Hackers].

Not hung up on the name either, just something that indicates
breadth-of-scope, e.g. Availability or Data Protection or Resilience
etc..; maybe the Advocates would like to name it? It might even be a
press-release: "PostgreSQL community focuses new efforts towards
Robustness features for its next major release".

Best Regards, Simon Riggs

---------------------------(end of

broadcast)---------------------------

TIP 9: the planner will ignore your desire to choose an index scan if

your

Simon,

I'd vote for a new list dedicated to discussing "Robustness" issues,
such as PITR and the fsync/sync issues.

<snip>

The list probably would remain clearly differentiated, in the same way
[Performance] covers lots of areas not discussed in [Hackers].

Actually, Simon, you're welcome to bring this discussion over to PERFORMANCE.
We discuss scalability and HA on Performance frequently, and I don't feel
that the discussion you refer to would be out of place.

But Tom is right that you need the feedback of a lot of the people on Hackers
once you start discussing a code solution, so there's not much point in
starting a new mailing list that all the same people need to subscribe to.
Certainly Jan had enough trouble getting meaningful feedback on the sync
issue here; on his own list he'd still be talking to himself.

As far as promoting an image of reliability, that belongs on Advocacy. The
image and the reality don't sync much; we're already about 500% more reliable
than MS SQL Server but ask any ten CIOs what they think? That's just
marketing.

--
-Josh Berkus
Aglio Database Solutions
San Francisco

-----Original Message-----
From: Nicolai Tufar [mailto:ntufar@pisem.net]
Sent: 05 February 2004 00:01
To: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] PITR Dead horse?

Totally agree. Robustness and rock-solidness are the only
things missing for PostgreSQL to become the killer of certain
commercial enterprise databases out there.

Well I've only been using PostgreSQL since 1997 and the *only* release I
ever had problems with was 6.3.2. We also use(d) Informix SE, DB2,
Unidata and SQL Server and only Informix and Unidata come close to the
robustness of PostgreSQL - and they're not the ones we need to worry
about.

Now I'm not saying we shouldn't be continually looking to improve
things, but I don't think this is quite the problem you imply.

Regards, Dave.

-----Original Message-----
From: Nicolai Tufar [mailto:ntufar@pisem.net]
Sent: 05 February 2004 08:15
To: Dave Page
Subject: RE: [HACKERS] PITR Dead horse?

-----Original Message-----
From: Dave Page [mailto:dpage@vale-housing.co.uk] Well I've

only been

using PostgreSQL since 1997 and the *only* release

I

ever had problems with was 6.3.2. We also use(d) Informix SE, DB2,
Unidata and SQL Server and only Informix and Unidata come

close to the

robustness of PostgreSQL - and they're not the ones we need

to worry

about.

Don't know. But apparently different users will have
different demands From a database.

Of course, but I would argue that my claim that PostgreSQL is reliable
is backed up by the lack of people posting messages like 'we had a
powercut and now my DB is hosed'.

Now I'm not saying we shouldn't be continually looking to improve
things, but I don't think this is quite the problem you imply.

For the customers I am dealing with it is quite a problem, believe me.

Do they have specific problems with the reliability of PostgreSQL then?
Perhaps you could post details of how things have gone wrong for them
(assuming you haven't already - I don't recall anything on -hackers
recently).

Regards, Dave

Don't know. But apparently different users will have
different demands From a database.

Of course, but I would argue that my claim that PostgreSQL is reliable
is backed up by the lack of people posting messages like 'we had a
powercut and now my DB is hosed'.

One thing we could use (and I have no idea how to do it) is a "This
hardware is not appropriate for a database" test kit.

Something to detect lying disks, battery backed write cache that isn't
so battery backed, etc.

--
Rod Taylor <rbt [at] rbt [dot] ca>

Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
PGP Key: http://www.rbt.ca/rbtpub.asc

Dave Page wrote:

-----Original Message-----
From: Nicolai Tufar [mailto:ntufar@pisem.net]
Sent: 05 February 2004 00:01
To: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] PITR Dead horse?

Totally agree. Robustness and rock-solidness are the only
things missing for PostgreSQL to become the killer of certain
commercial enterprise databases out there.

Well I've only been using PostgreSQL since 1997 and the *only* release I
ever had problems with was 6.3.2. We also use(d) Informix SE, DB2,
Unidata and SQL Server and only Informix and Unidata come close to the
robustness of PostgreSQL - and they're not the ones we need to worry
about.

Now I'm not saying we shouldn't be continually looking to improve
things, but I don't think this is quite the problem you imply.

I assume he was talking about the lack of data recovery in cases of hard
drive failure --- we now require you restore from backup or use a
replicated machine/drive setup.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Tatsuo Ishii wrote:

Has this been beaten to death now? Just curious if PITR was in Dev tree
yet. Been out of the loop. TIA.

I and my co workers are very interested in implementing PITR. We will
tackle this for 7.5 if no one objects.

I have put up a PITR project page:

http://momjian.postgresql.org/main/writings/pgsql/project

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073