What's left?

I would like very much to help any way possible in solving any last
remaining issues. Once the CVS sources are compliable, it will be
easier to make meaningful contributions. I'm really looking
forward to testing and benchmarking the win32 port. A big thanks
to all who continue to work so hard on this project.

To answer the question, "what's left?"

There is one patch I've submitted waiting for application (12/01/04), and
I've got another patch almost ready (adds some missing functions, and fixes
busted Makefiles etc) that, with Magnus's signal code, will give us a
compilable, runnable, and somewhat buggy Win32 port.

It will then be a matter of fixing things like:
* installation directory issues (/usr/local/pgsql/bin won't work too well
outside of the MingW environment :-)
* general directory handling (ie. whitespaces in directory names;
forward/backslash path canonicalization)
* sync issues
* any missing structs/items in shared memory
* generally, running the test suite, and fixing whatever is busted (I'm at
41 tests passing now :-)

I'm imagining we'll be in a position to be able to start on the above in as
little as 2 weeks...

Cheers,
Claudio

--- 
Certain disclaimers and policies apply to all email sent from Memetrics.
For the full text of these disclaimers and policies see 
<a
href="http://www.memetrics.com/emailpolicy.html">http://www.memetrics.com/em
ailpolicy.html</a>

Claudio Natoli wrote:

* installation directory issues (/usr/local/pgsql/bin won't work too well
outside of the MingW environment :-)

Clearly we will need an installer for a binary distribution. But for now
I suggest that the default prefix on Windows is
C:\Program Files\PostgreSQL

cheers

andrew

Andrew Dunstan wrote:

Claudio Natoli wrote:

* installation directory issues (/usr/local/pgsql/bin won't work too
well outside of the MingW environment :-)

Clearly we will need an installer for a binary distribution.

Yes. To be more precise, my point was that doing so will require some
changes to the code (ie. configure/compile time constants like PKGLIBDIR
just won't do us any good).

But for now I suggest that the default prefix on Windows is
C:\Program Files\PostgreSQL

For right now, I'd suggest a directory that doesn't have whitespace and
localization issues :-P

Cheers,
Claudio

--- 
Certain disclaimers and policies apply to all email sent from Memetrics.
For the full text of these disclaimers and policies see 
<a
href="http://www.memetrics.com/emailpolicy.html">http://www.memetrics.com/em
ailpolicy.html</a>

Hi all,

Might I just suggest good old "C:\PostgreSQL" ?

MS SQL server defaults to C:\MSSQL, so I don't think that a directory in the
root path is unreasonable. Further, it makes it look more important if it
installs in the root directory :)

All the best,

-David Felstead

Claudio Natoli wrote:

-----Original Message-----
From: Claudio Natoli [mailto:claudio.natoli@memetrics.com]
Sent: Thursday, January 22, 2004 4:44 PM
To: 'Andrew Dunstan '; 'pgsql-hackers-win32 ';
'PostgreSQL-development '
Subject: Re: [pgsql-hackers-win32] [HACKERS] What's left?

Andrew Dunstan wrote:

Claudio Natoli wrote:

* installation directory issues (/usr/local/pgsql/bin

won't work too

well outside of the MingW environment :-)

Clearly we will need an installer for a binary distribution.

Yes. To be more precise, my point was that doing so will
require some changes to the code (ie. configure/compile time
constants like PKGLIBDIR just won't do us any good).

But for now I suggest that the default prefix on Windows is
C:\Program Files\PostgreSQL

More properly:
%ProgramFiles%\PostgreSQL

And then, if they don't like that, let them put it wherever they darn
well please.

For right now, I'd suggest a directory that doesn't have
whitespace and localization issues :-P

It will have to be faced. If it cannot handle spaces or funny
characters, it will simply fail.

I may be able to help on the localization and path stuff. We have
solved those issues for our port of 7.1.3, and I expect the work for 7.5
to be extremely similar.

Where can I get the latest tarball for Win32 development?

Where can I get the latest tarball for Win32 development?

There isn't a specific Win32 tarball, but you can get nightly snapshots from
the usual place (ftp://ftp.postgresql.org/pub/dev/), or pull down the tip
from CVS.

Reading back through the thread though, you'll find that the code is not yet
compilable/runnable, and you might want to rein in your enthusiasm for just
two or three weeks more (which might not be easy :-) for these last couple
bits (which are nearly ready) to get sent, approved, + applied.

Cheers,
Claudio

--- 
Certain disclaimers and policies apply to all email sent from Memetrics.
For the full text of these disclaimers and policies see 
<a
href="http://www.memetrics.com/emailpolicy.html">http://www.memetrics.com/em
ailpolicy.html</a>

Some fool wrote:

It will then be a matter of fixing things like:
* installation directory issues (/usr/local/pgsql/bin won't work too
well outside of the MingW environment :-)
* general directory handling (ie. whitespaces in directory names;
forward/backslash path canonicalization)
* sync issues
* any missing structs/items in shared memory
* generally, running the test suite, and fixing whatever is busted (I'm
at 41 tests passing now :-)

One important thing I forgot, that someone could start looking at now:
* backends keeping files open when other backends are trying to
delete/rename them

The port I wrote for here at work simply modified the functions in dirmod.c,
to attempt the delete (or rename), and, on a failure identifiable as being
presumably due to another process holding the file open, simply schedules
the file for deletion at system start time using the Win32 API for doing so
(hey, it is Windows, it is going to reboot sooner or later :-). In the case
of rename, just copies the existing file and schedules the original for
deletion.

Ugly, and sometimes slow where we'd rather not be, but it gets us by.

We must do better for the official port, and whilst better solutions are
obviously conceivable, AFAICS they will require some amount of backend
changes and therefore consent from main list. Someone might want to start
looking at a nice, clean solution to this.

Cheers,
Claudio

--- 
Certain disclaimers and policies apply to all email sent from Memetrics.
For the full text of these disclaimers and policies see 
<a
href="http://www.memetrics.com/emailpolicy.html">http://www.memetrics.com/em
ailpolicy.html</a>

Might I just suggest good old "C:\PostgreSQL" ?

MS SQL server defaults to C:\MSSQL, so I don't think that a directory in

the

root path is unreasonable. Further, it makes it look more important if it
installs in the root directory :)

Don't do that. I hate software that does that. To me it immediately
screams "WE DON'T CARE ABOUT DOING THINGS RIGHT!".

my 2cents
Later
Rob

Claudio Natoli <claudio.natoli@memetrics.com> writes:

One important thing I forgot, that someone could start looking at now:
* backends keeping files open when other backends are trying to
delete/rename them

We must do better for the official port,

Why? The procedure you mentioned seems perfectly adequate to me,
seeing that it's a bit of a corner case to start with.

I cannot think of any way of "doing better" that wouldn't be far too
invasive to be acceptable.

regards, tom lane

Tom Lane wrote:

Claudio Natoli <claudio.natoli@memetrics.com> writes:

One important thing I forgot, that someone could start looking at now:
* backends keeping files open when other backends are trying to
delete/rename them

We must do better for the official port,

Why? The procedure you mentioned seems perfectly adequate to me,
seeing that it's a bit of a corner case to start with.

Because, on occasion, I end up with GBs of log files hanging around. You can
wrack up disk space real fast that way.

I cannot think of any way of "doing better" that wouldn't be far too
invasive to be acceptable.

In that case, I'm more than happy to contribute the code for this solution
along with the remaining changes I've got ahead of me, but, still, if
someone thinks they can do better without a great deal of drama (which, like
you, I think is unlikely) then I'm all for it.

Cheers,
Claudio

--- 
Certain disclaimers and policies apply to all email sent from Memetrics.
For the full text of these disclaimers and policies see 
<a
href="http://www.memetrics.com/emailpolicy.html">http://www.memetrics.com/em
ailpolicy.html</a>

Dann Corbit wrote:

But for now I suggest that the default prefix on Windows is
C:\Program Files\PostgreSQL

More properly:
%ProgramFiles%\PostgreSQL

Another suggestion: %ProgramFiles%\PGDG\PostgreSQL (or even
%ProgramFiles%\PGDG\PostgreSQL 7.5). Apache2 uses %ProgramFiles%\Apache
Group\Apache2.

Note: Many software uses the %ProgramFiles%\<VendorName>\<ProductName>
convention, but apparently Microsoft itself puts stuffs right under
%ProgramFiles% (%ProgramFiles%\Microsoft Money, \Internet Explorer,
\Windows Media Player, etc).

And then, if they don't like that, let them put it wherever they darn
well please.

--
dave

I think users would prefer %ProgramFiles%\PostgreSQL - that's what Mozilla
and some other projects do, although still other projects do
%ProgramFiles%\GNU\PostgreSQL.

I'd vote for %ProgramFiles%\PostgreSQL.

- Steve

-----Original Message-----
From: pgsql-hackers-win32-owner@postgresql.org
[mailto:pgsql-hackers-win32-owner@postgresql.org] On Behalf Of David
Garamond
Sent: January 23, 2004 2:42 AM
To: Dann Corbit
Cc: Claudio Natoli; Andrew Dunstan; pgsql-hackers-win32;
PostgreSQL-development
Subject: Re: [pgsql-hackers-win32] [HACKERS] What's left?

Dann Corbit wrote:

But for now I suggest that the default prefix on Windows is
C:\Program Files\PostgreSQL

More properly:
%ProgramFiles%\PostgreSQL

Another suggestion: %ProgramFiles%\PGDG\PostgreSQL (or even
%ProgramFiles%\PGDG\PostgreSQL 7.5). Apache2 uses %ProgramFiles%\Apache
Group\Apache2.

Note: Many software uses the %ProgramFiles%\<VendorName>\<ProductName>
convention, but apparently Microsoft itself puts stuffs right under
%ProgramFiles% (%ProgramFiles%\Microsoft Money, \Internet Explorer, \Windows
Media Player, etc).

And then, if they don't like that, let them put it wherever they darn
well please.

--
dave

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Dann Corbit wrote:

-----Original Message-----
From: Claudio Natoli [mailto:claudio.natoli@memetrics.com]
Sent: Thursday, January 22, 2004 4:44 PM
To: 'Andrew Dunstan '; 'pgsql-hackers-win32 ';
'PostgreSQL-development '
Subject: Re: [pgsql-hackers-win32] [HACKERS] What's left?

Andrew Dunstan wrote:

Claudio Natoli wrote:

* installation directory issues (/usr/local/pgsql/bin

won't work too

well outside of the MingW environment :-)

Clearly we will need an installer for a binary distribution.

Yes. To be more precise, my point was that doing so will
require some changes to the code (ie. configure/compile time
constants like PKGLIBDIR just won't do us any good).

But for now I suggest that the default prefix on Windows is
C:\Program Files\PostgreSQL

More properly:
%ProgramFiles%\PostgreSQL

And then, if they don't like that, let them put it wherever they darn
well please.

For right now, I'd suggest a directory that doesn't have
whitespace and localization issues :-P

It will have to be faced. If it cannot handle spaces or funny
characters, it will simply fail.

What about 8.3 short names for the win32 environment?
"C:\Progra~1\Postgres" Should be able to work correctly. Isn't there a
win32 command to get the short path if you need it?

Thomas Swan wrote:

More properly:
%ProgramFiles%\PostgreSQL

And then, if they don't like that, let them put it wherever they darn
well please.

For right now, I'd suggest a directory that doesn't have whitespace
and localization issues :-P

It will have to be faced. If it cannot handle spaces or funny
characters, it will simply fail.

What about 8.3 short names for the win32 environment?
"C:\Progra~1\Postgres" Should be able to work correctly. Isn't there
a win32 command to get the short path if you need it?

Please not... 8.3 is really ancient.
PostgreSQL-win32 should be as native as possible, including all
naming/path/registry conventions. The more familiar pgsql looks to
hardcore windowers, the better will be the acceptance for a newbie.

Regards,
Andreas

I have added these and your later mention of delete/rename of open files
to the Win32 project page:

http://momjian.postgresql.org/main/writings/pgsql/project

---------------------------------------------------------------------------

Claudio Natoli wrote:

I would like very much to help any way possible in solving any last
remaining issues. Once the CVS sources are compliable, it will be
easier to make meaningful contributions. I'm really looking
forward to testing and benchmarking the win32 port. A big thanks
to all who continue to work so hard on this project.

To answer the question, "what's left?"

There is one patch I've submitted waiting for application (12/01/04), and
I've got another patch almost ready (adds some missing functions, and fixes
busted Makefiles etc) that, with Magnus's signal code, will give us a
compilable, runnable, and somewhat buggy Win32 port.

It will then be a matter of fixing things like:
* installation directory issues (/usr/local/pgsql/bin won't work too well
outside of the MingW environment :-)
* general directory handling (ie. whitespaces in directory names;
forward/backslash path canonicalization)
* sync issues
* any missing structs/items in shared memory
* generally, running the test suite, and fixing whatever is busted (I'm at
41 tests passing now :-)

I'm imagining we'll be in a position to be able to start on the above in as
little as 2 weeks...

Cheers,
Claudio

--- 
Certain disclaimers and policies apply to all email sent from Memetrics.
For the full text of these disclaimers and policies see 
<a
href="http://www.memetrics.com/emailpolicy.html">http://www.memetrics.com/em
ailpolicy.html</a>

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Tom Lane wrote:

Claudio Natoli <claudio.natoli@memetrics.com> writes:

One important thing I forgot, that someone could start looking at now:
* backends keeping files open when other backends are trying to
delete/rename them

We must do better for the official port,

Why? The procedure you mentioned seems perfectly adequate to me,
seeing that it's a bit of a corner case to start with.

I don't see this as a corner case, except it being a corner case
operating system. :-)

I think it will very likely rename/unlink will fail because of the file
descriptor cache kept by each backend.

I am attaching dir.c from the PeerDirect port. It handles unlink
failures by appending the failed file name to a file that is later read
and another unlink attempted. Perhaps this is something we can do, and
have try unlinks after each checkpoint.

PeerDirect handles rename by just looping. We really can't delay a
rename. There is discussion in the Win32 TODO detail that goes over
some options, I think.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

I think it will very likely rename/unlink will fail because of the file
descriptor cache kept by each backend.

Hmm ... you're probably right. Okay, it's a more significant issue than
I thought.

I am attaching dir.c from the PeerDirect port. It handles unlink
failures by appending the failed file name to a file that is later read
and another unlink attempted. Perhaps this is something we can do, and
have try unlinks after each checkpoint.

That seems like a possibility. The open files will get closed very soon
after the delete occurs (as a byproduct of relcache flush), so we don't
need very much of a delay. Next checkpoint sounds reasonable.

PeerDirect handles rename by just looping. We really can't delay a
rename. There is discussion in the Win32 TODO detail that goes over
some options, I think.

Do we really have any problem with rename? We don't rename table files.
The renames I can think of are renaming temp files into place as
permanent ones, and there would be no open references to such a file.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

I think it will very likely rename/unlink will fail because of the file
descriptor cache kept by each backend.

Hmm ... you're probably right. Okay, it's a more significant issue than
I thought.

I am attaching dir.c from the PeerDirect port. It handles unlink
failures by appending the failed file name to a file that is later read
and another unlink attempted. Perhaps this is something we can do, and
have try unlinks after each checkpoint.

That seems like a possibility. The open files will get closed very soon
after the delete occurs (as a byproduct of relcache flush), so we don't
need very much of a delay. Next checkpoint sounds reasonable.

Good. I am glad for the recache closing because we were going to need
something like that.

PeerDirect handles rename by just looping. We really can't delay a
rename. There is discussion in the Win32 TODO detail that goes over
some options, I think.

Do we really have any problem with rename? We don't rename table files.
The renames I can think of are renaming temp files into place as
permanent ones, and there would be no open references to such a file.

We do have a problem. It is with cache files read on startup, like
pg_pwd. We can generate the file as temp, but we have to slide it in
while a backend is not reading it. On a busy system, I am not sure how
large a window we will get for the rename. The rename is all
centralized in port/dirmod.c, so we can deal with it there, whatever the
solution.

We also have to do the rename during xact close because we need to hold
locks so we are sure the files are written in the same order that they
modify pg_shadow, waiting a long time for the rename is a serious
problem.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Dann Corbit wrote:

I may be able to help on the localization and path stuff. We have
solved those issues for our port of 7.1.3, and I expect the work for 7.5
to be extremely similar.

Where can I get the latest tarball for Win32 development?

CVS HEAD now has all the Win32 work.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073