rename/unlink handling for Win32

Started by Bruce Momjianover 22 years ago4 messages
#1Bruce Momjian
pgman@candle.pha.pa.us
2 attachment(s)

Here is my approach to the use of rename/unlink on Win32. The full
discussion is at:

http://momjian.postgresql.org/cgi-bin/pgtodo?win32

Basically, rename/unlink will fail if the file is opened. You can move
the open file to another name, but that then requires open to loop in
case the file is missing.

The following patch loops over rename/unlink every 1/10th of second,
printing a warning message after 1 second, and printing a completion
message if a warning message was printed.

I looked at PeerDirect's and SRA's port, and neither provides a better
method. I looked at PeerDirect's and it actually has some conditional
code for rename. For example, it has a signal that is sent to all
backends to inform them to close their open WAL files. I am not sure if
that is required for us because there is some replication stuff in
there that we aren't using. Jan? In a few other places, it allows the
rename to fail.

I am inclined to implement it as shown, then see what delayed
rename/unlinks we get in testing.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Attachments:

/pgpatches/win32/renametext/plainDownload
Index: configure.in
===================================================================
RCS file: /cvsroot/pgsql-server/configure.in,v
retrieving revision 1.242
diff -c -c -r1.242 configure.in
*** configure.in	6 Apr 2003 22:45:22 -0000	1.242
--- configure.in	20 Apr 2003 02:11:49 -0000
***************
*** 856,863 ****
  esac
  
  # Solaris has a very slow qsort in certain cases, so we replace it.
! case $host_os in
!   solaris*) AC_LIBOBJ(qsort) ;;
  esac
  
  # On HPUX 9, rint() is not in regular libm.a but in /lib/pa1.1/libm.a;
--- 856,868 ----
  esac
  
  # Solaris has a very slow qsort in certain cases, so we replace it.
! case $host_os in solaris*) 
! AC_LIBOBJ(qsort) ;;
! esac
! 
! # Win32 can't to rename or unlink on an open file
! case $host_os in win32*) 
! AC_LIBOBJ(dirmod) ;;
  esac
  
  # On HPUX 9, rint() is not in regular libm.a but in /lib/pa1.1/libm.a;
Index: src/include/pg_config_manual.h
===================================================================
RCS file: /cvsroot/pgsql-server/src/include/pg_config_manual.h,v
retrieving revision 1.2
diff -c -c -r1.2 pg_config_manual.h
*** src/include/pg_config_manual.h	18 Apr 2003 01:03:42 -0000	1.2
--- src/include/pg_config_manual.h	20 Apr 2003 02:12:07 -0000
***************
*** 151,156 ****
--- 151,167 ----
  #endif
    
  /*
+  * Win32 doesn't have reliable rename/unlink during concurrent access
+  */
+ #ifdef WIN32
+ int pgrename(const char *from, const char *to);
+ int pgunlink(const char *path);      
+ #define rename(path)		pgrename(path)
+ #define unlink(from, to)	pgunlink(from, to)
+ #endif
+ 
+   
+ /*
   * This is the default directory in which AF_UNIX socket files are
   * placed.  Caution: changing this risks breaking your existing client
   * applications, which are likely to continue to look in the old
/pg/port/dirmod.ctext/plainDownload
#2Bruce Momjian
pgman@candle.pha.pa.us
In reply to: Bruce Momjian (#1)
Re: rename/unlink handling for Win32

Oh, one more thing. dirmod.c has Win32 source to show the actual hangs
caused by rename/unlink --- just define TEST_VERSION and compile. If
someone wants a binary, let me know.

And sorry I posted to hackers rather than patches, where it belongs.

---------------------------------------------------------------------------

Bruce Momjian wrote:

Here is my approach to the use of rename/unlink on Win32. The full
discussion is at:

http://momjian.postgresql.org/cgi-bin/pgtodo?win32

Basically, rename/unlink will fail if the file is opened. You can move
the open file to another name, but that then requires open to loop in
case the file is missing.

The following patch loops over rename/unlink every 1/10th of second,
printing a warning message after 1 second, and printing a completion
message if a warning message was printed.

I looked at PeerDirect's and SRA's port, and neither provides a better
method. I looked at PeerDirect's and it actually has some conditional
code for rename. For example, it has a signal that is sent to all
backends to inform them to close their open WAL files. I am not sure if
that is required for us because there is some replication stuff in
there that we aren't using. Jan? In a few other places, it allows the
rename to fail.

I am inclined to implement it as shown, then see what delayed
rename/unlinks we get in testing.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
#3Peter Eisentraut
peter_e@gmx.net
In reply to: Bruce Momjian (#1)
Re: rename/unlink handling for Win32

Bruce Momjian writes:

The following patch loops over rename/unlink every 1/10th of second,
printing a warning message after 1 second, and printing a completion
message if a warning message was printed.

I don't like that; it seems arbitrary. How does the need to wait relate
to other factors, such as the system load?

About the code: The code you placed into pg_config_manual.h must go into
some other header file, probably a separate one that parallels the .c
file. Also, I would prefer if the C files in src/port were named after
the function they implement, so rename.c.

It might also be cleaner if we changed the code to use remove() instead of
unlink(), since the ISO C standard uses the former whereas the latter is
Unix-ish.

--
Peter Eisentraut peter_e@gmx.net

#4Bruce Momjian
pgman@candle.pha.pa.us
In reply to: Peter Eisentraut (#3)
1 attachment(s)
Re: [HACKERS] rename/unlink handling for Win32

[ Thread moved to patches, where I should have posted it at first.]

Peter Eisentraut wrote:

Bruce Momjian writes:

The following patch loops over rename/unlink every 1/10th of second,
printing a warning message after 1 second, and printing a completion
message if a warning message was printed.

I don't like that; it seems arbitrary. How does the need to wait relate
to other factors, such as the system load?

I wasn't clear --- it basically tries 10 times, not necessarily over one
second, and it doesn't print "1 second" or anything.

The values only control whether it prints anything to the logs --- it
will continue looping until it succeeds. Do you see any other solution?

About the code: The code you placed into pg_config_manual.h must go into
some other header file, probably a separate one that parallels the .c
file. Also, I would prefer if the C files in src/port were named after

You want dirmod.h for two prototypes? If I do that, then am I including
that from pg_config_manual.h or somewhere else?

What we could do is to create a port.h file and pull the other /port
prototypes like fseeko() into that file. Is that what you want?

the function they implement, so rename.c.

But we have rename and unlink in there. I don't think we want two
files, do we? They do almost the same thing. That's why I called it
dirmod.c.

It might also be cleaner if we changed the code to use remove() instead of
unlink(), since the ISO C standard uses the former whereas the latter is
Unix-ish.

I didn't want to get into that for this patch. If someone wants to
rename them across the source code, they are welcome to do that, though
unlink() seems more common than remove() to me.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Attachments:

/pgpatches/win32/renametext/plainDownload
Index: configure.in
===================================================================
RCS file: /cvsroot/pgsql-server/configure.in,v
retrieving revision 1.242
diff -c -c -r1.242 configure.in
*** configure.in	6 Apr 2003 22:45:22 -0000	1.242
--- configure.in	21 Apr 2003 14:24:02 -0000
***************
*** 856,863 ****
  esac
  
  # Solaris has a very slow qsort in certain cases, so we replace it.
! case $host_os in
!   solaris*) AC_LIBOBJ(qsort) ;;
  esac
  
  # On HPUX 9, rint() is not in regular libm.a but in /lib/pa1.1/libm.a;
--- 856,868 ----
  esac
  
  # Solaris has a very slow qsort in certain cases, so we replace it.
! case $host_os in solaris*) 
! AC_LIBOBJ(qsort) ;;
! esac
! 
! # Win32 can't to rename or unlink on an open file
! case $host_os in win32*) 
! AC_LIBOBJ(dirmod) ;;
  esac
  
  # On HPUX 9, rint() is not in regular libm.a but in /lib/pa1.1/libm.a;
Index: src/include/c.h
===================================================================
RCS file: /cvsroot/pgsql-server/src/include/c.h,v
retrieving revision 1.138
diff -c -c -r1.138 c.h
*** src/include/c.h	18 Apr 2003 01:03:42 -0000	1.138
--- src/include/c.h	21 Apr 2003 14:24:24 -0000
***************
*** 711,716 ****
--- 711,727 ----
  off_t ftello(FILE *stream);
  #endif
  
+ /*
+  * Win32 doesn't have reliable rename/unlink during concurrent access
+  */
+ #ifdef WIN32
+ int pgrename(const char *from, const char *to);
+ int pgunlink(const char *path);      
+ #define rename(path)		pgrename(path)
+ #define unlink(from, to)	pgunlink(from, to)
+ #endif
+ 
+   
  /* These are for things that are one way on Unix and another on NT */
  #define NULL_DEV		"/dev/null"