pg_archivecleanup bug
An EDB customer reported a problem with pg_archivecleanup which I
have looked into and found a likely cause. It is, in any event, a
bug which I think should be fixed. It has to do with our use of
the readdir() function:
http://pubs.opengroup.org/onlinepubs/7908799/xsh/readdir_r.html
These are the relevant bits:
| Applications wishing to check for error situations should set
| errno to 0 before calling readdir(). If errno is set to non-zero
| on return, an error occurred.
| Upon successful completion, readdir() returns a pointer to an
| object of type struct dirent. When an error is encountered, a
| null pointer is returned and errno is set to indicate the error.
| When the end of the directory is encountered, a null pointer is
| returned and errno is not changed.
Here is our current usage:
So an error in scanning the directory will not be reported; the
cleanup will quietly terminate the WAL deletions without processing
the remainder of the directory. Attached is the simplest fix,
which would report the error, stop looking for WAL files, and
continue with other clean-ups. I'm not sure we should keep the fix
that simple. We could set a flag so that we would exit with a
non-zero code, or we could try a new directory scan as long as the
last scan found and deleted at least one WAL file. Perhaps we want
to back-patch the simple fix and do something fancier for 9.4?
I would also add a few comment lines before committing this, if we
decide to go with the simple approach -- this is for purposes of
illustration; to facilitate discussion.
Thoughts?
--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
archivecleanup-dir-error-v1.difftext/x-diff; name=archivecleanup-dir-error-v1.diffDownload
diff --git a/contrib/pg_archivecleanup/pg_archivecleanup.c b/contrib/pg_archivecleanup/pg_archivecleanup.c
index f12331a..3e47f3c 100644
--- a/contrib/pg_archivecleanup/pg_archivecleanup.c
+++ b/contrib/pg_archivecleanup/pg_archivecleanup.c
@@ -117,7 +117,7 @@ CleanupPriorWALFiles(void)
if ((xldir = opendir(archiveLocation)) != NULL)
{
- while ((xlde = readdir(xldir)) != NULL)
+ while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
strncpy(walfile, xlde->d_name, MAXPGPATH);
TrimExtension(walfile, additional_ext);
@@ -175,6 +175,11 @@ CleanupPriorWALFiles(void)
}
}
}
+
+ if (errno)
+ fprintf(stderr, "%s: ERROR: could not read next directory entry: %s\n",
+ progname, strerror(errno));
+
closedir(xldir);
}
else
On Thu, Dec 5, 2013 at 3:06 PM, Kevin Grittner <kgrittn@ymail.com> wrote:
An EDB customer reported a problem with pg_archivecleanup which I
have looked into and found a likely cause. It is, in any event, a
bug which I think should be fixed. It has to do with our use of
the readdir() function:http://pubs.opengroup.org/onlinepubs/7908799/xsh/readdir_r.html
These are the relevant bits:
| Applications wishing to check for error situations should set
| errno to 0 before calling readdir(). If errno is set to non-zero
| on return, an error occurred.| Upon successful completion, readdir() returns a pointer to an
| object of type struct dirent. When an error is encountered, a
| null pointer is returned and errno is set to indicate the error.
| When the end of the directory is encountered, a null pointer is
| returned and errno is not changed.Here is our current usage:
So an error in scanning the directory will not be reported; the
cleanup will quietly terminate the WAL deletions without processing
the remainder of the directory. Attached is the simplest fix,
which would report the error, stop looking for WAL files, and
continue with other clean-ups. I'm not sure we should keep the fix
that simple. We could set a flag so that we would exit with a
non-zero code, or we could try a new directory scan as long as the
last scan found and deleted at least one WAL file. Perhaps we want
to back-patch the simple fix and do something fancier for 9.4?
A directory that you can't read sounds like a pretty bad thing. I'd
be inclined to print an error message and exit forthwith.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Kevin Grittner <kgrittn@ymail.com> writes:
| Applications wishing to check for error situations should set
| errno to 0 before calling readdir(). If errno is set to non-zero
| on return, an error occurred.
So an error in scanning the directory will not be reported; the
cleanup will quietly terminate the WAL deletions without processing
the remainder of the directory. Attached is the simplest fix,
which would report the error, stop looking for WAL files, and
continue with other clean-ups. I'm not sure we should keep the fix
that simple. We could set a flag so that we would exit with a
non-zero code, or we could try a new directory scan as long as the
last scan found and deleted at least one WAL file. Perhaps we want
to back-patch the simple fix and do something fancier for 9.4?
A quick grep shows about ten other readdir() usages, most of which
have a similar disease.
In general, I think there is no excuse for code in the backend to use
readdir() directly; it should be using ReadDir(), which takes care of this
as well as error reporting. It appears that src/backend/storage/ipc/dsm.c
didn't get that memo; it certainly is innocent of any error checking
concerns. But the other usages seem to be in assorted utilities, which
will need to do it right for themselves. initdb.c's walkdir() seems to
have it right and might be a reasonable model to follow. Or maybe we
should invent a frontend-friendly version of ReadDir() rather than
duplicating all the error checking code in ten-and-counting places?
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Dec 5, 2013 at 6:15 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
In general, I think there is no excuse for code in the backend to use
readdir() directly; it should be using ReadDir(), which takes care of this
as well as error reporting.
My understanding is that the fd.c infrastructure can't be used in the
postmaster.
I agree that sucks.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Robert Haas <robertmhaas@gmail.com> writes:
On Thu, Dec 5, 2013 at 6:15 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
In general, I think there is no excuse for code in the backend to use
readdir() directly; it should be using ReadDir(), which takes care of this
as well as error reporting.
My understanding is that the fd.c infrastructure can't be used in the
postmaster.
Say what? See ParseConfigDirectory for code that certainly runs in the
postmaster, and uses ReadDir().
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Dec 6, 2013 at 11:10 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
On Thu, Dec 5, 2013 at 6:15 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
In general, I think there is no excuse for code in the backend to use
readdir() directly; it should be using ReadDir(), which takes care of this
as well as error reporting.My understanding is that the fd.c infrastructure can't be used in the
postmaster.Say what? See ParseConfigDirectory for code that certainly runs in the
postmaster, and uses ReadDir().
Gosh, I could have sworn that I had calls into fd.c that were crashing
and burning during development because they happened too early in
postmaster startup. But it seems to work fine now, so I've pushed a
fix for this and a few related issues. Please let me know if you
think there are remaining issues.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Dec 5, 2013 at 6:15 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
But the other usages seem to be in assorted utilities, which
will need to do it right for themselves. initdb.c's walkdir() seems to
have it right and might be a reasonable model to follow. Or maybe we
should invent a frontend-friendly version of ReadDir() rather than
duplicating all the error checking code in ten-and-counting places?
If there's enough uniformity in all of those places to make that
feasible, it certainly seems wise to do it that way. I don't know if
that's the case, though - e.g. maybe some callers want to exit and
others do not. pg_resetxlog wants to exit; pg_archivecleanup and
pg_standby most likely want to print an error and carry on.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Dec 5, 2013 at 12:06:07PM -0800, Kevin Grittner wrote:
An EDB customer reported a problem with pg_archivecleanup which I
have looked into and found a likely cause.� It is, in any event, a
bug which I think should be fixed.� It has to do with our use of
the readdir() function:http://pubs.opengroup.org/onlinepubs/7908799/xsh/readdir_r.html
These are the relevant bits:
| Applications wishing to check for error situations should set
| errno to 0 before calling readdir(). If errno is set to non-zero
| on return, an error occurred.| Upon successful completion, readdir() returns a pointer to an
| object of type struct dirent. When an error is encountered, a
| null pointer is returned and errno is set to indicate the error.
| When the end of the directory is encountered, a null pointer is
| returned and errno is not changed.
Wow, another case where errno clearing is necessary. We were just
looking this requirement for getpwuid() last week.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Dec 9, 2013 at 11:27:28AM -0500, Robert Haas wrote:
On Thu, Dec 5, 2013 at 6:15 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
But the other usages seem to be in assorted utilities, which
will need to do it right for themselves. initdb.c's walkdir() seems to
have it right and might be a reasonable model to follow. Or maybe we
should invent a frontend-friendly version of ReadDir() rather than
duplicating all the error checking code in ten-and-counting places?If there's enough uniformity in all of those places to make that
feasible, it certainly seems wise to do it that way. I don't know if
that's the case, though - e.g. maybe some callers want to exit and
others do not. pg_resetxlog wants to exit; pg_archivecleanup and
pg_standby most likely want to print an error and carry on.
I have developed the attached patch which fixes all cases where
readdir() wasn't checking for errno, and cleaned up the syntax in other
cases to be consistent.
While I am not a fan of backpatching, the fact we are ignoring errors in
some critical cases seems the non-cosmetic parts should be backpatched.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
Attachments:
readdir.difftext/x-diff; charset=us-asciiDownload
diff --git a/contrib/pg_archivecleanup/pg_archivecleanup.c b/contrib/pg_archivecleanup/pg_archivecleanup.c
new file mode 100644
index 7b5484b..fbc3e5a
*** a/contrib/pg_archivecleanup/pg_archivecleanup.c
--- b/contrib/pg_archivecleanup/pg_archivecleanup.c
*************** CleanupPriorWALFiles(void)
*** 106,112 ****
if ((xldir = opendir(archiveLocation)) != NULL)
{
! while ((xlde = readdir(xldir)) != NULL)
{
strncpy(walfile, xlde->d_name, MAXPGPATH);
TrimExtension(walfile, additional_ext);
--- 106,112 ----
if ((xldir = opendir(archiveLocation)) != NULL)
{
! while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
strncpy(walfile, xlde->d_name, MAXPGPATH);
TrimExtension(walfile, additional_ext);
*************** CleanupPriorWALFiles(void)
*** 164,169 ****
--- 164,172 ----
}
}
}
+ if (errno)
+ fprintf(stderr, "%s: could not read archive location \"%s\": %s\n",
+ progname, archiveLocation, strerror(errno));
closedir(xldir);
}
else
diff --git a/contrib/pg_standby/pg_standby.c b/contrib/pg_standby/pg_standby.c
new file mode 100644
index 8ddd486..be4d31f
*** a/contrib/pg_standby/pg_standby.c
--- b/contrib/pg_standby/pg_standby.c
*************** CustomizableCleanupPriorWALFiles(void)
*** 245,251 ****
*/
if ((xldir = opendir(archiveLocation)) != NULL)
{
! while ((xlde = readdir(xldir)) != NULL)
{
/*
* We ignore the timeline part of the XLOG segment identifiers
--- 245,251 ----
*/
if ((xldir = opendir(archiveLocation)) != NULL)
{
! while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
/*
* We ignore the timeline part of the XLOG segment identifiers
*************** CustomizableCleanupPriorWALFiles(void)
*** 283,288 ****
--- 283,291 ----
}
}
}
+ if (errno)
+ fprintf(stderr, "%s: could not read archive location \"%s\": %s\n",
+ progname, archiveLocation, strerror(errno));
if (debug)
fprintf(stderr, "\n");
}
diff --git a/src/bin/pg_basebackup/pg_receivexlog.c b/src/bin/pg_basebackup/pg_receivexlog.c
new file mode 100644
index 2478789..e252405
*** a/src/bin/pg_basebackup/pg_receivexlog.c
--- b/src/bin/pg_basebackup/pg_receivexlog.c
*************** FindStreamingStart(uint32 *tli)
*** 139,145 ****
disconnect_and_exit(1);
}
! while ((dirent = readdir(dir)) != NULL)
{
uint32 tli;
XLogSegNo segno;
--- 139,145 ----
disconnect_and_exit(1);
}
! while (errno = 0, (dirent = readdir(dir)) != NULL)
{
uint32 tli;
XLogSegNo segno;
*************** FindStreamingStart(uint32 *tli)
*** 209,214 ****
--- 209,221 ----
}
}
+ if (errno)
+ {
+ fprintf(stderr, _("%s: could not read directory \"%s\": %s\n"),
+ progname, basedir, strerror(errno));
+ disconnect_and_exit(1);
+ }
+
closedir(dir);
if (high_segno > 0)
diff --git a/src/bin/pg_dump/pg_backup_directory.c b/src/bin/pg_dump/pg_backup_directory.c
new file mode 100644
index 1bed8a9..c24f7e3
*** a/src/bin/pg_dump/pg_backup_directory.c
--- b/src/bin/pg_dump/pg_backup_directory.c
*************** InitArchiveFmt_Directory(ArchiveHandle *
*** 177,183 ****
struct dirent *d;
is_empty = true;
! while ((d = readdir(dir)))
{
if (strcmp(d->d_name, ".") != 0 && strcmp(d->d_name, "..") != 0)
{
--- 177,183 ----
struct dirent *d;
is_empty = true;
! while (errno = 0, (d = readdir(dir)))
{
if (strcmp(d->d_name, ".") != 0 && strcmp(d->d_name, "..") != 0)
{
*************** InitArchiveFmt_Directory(ArchiveHandle *
*** 185,190 ****
--- 185,193 ----
break;
}
}
+ if (errno)
+ exit_horribly(modulename, "could not read directory \"%s\": %s\n",
+ ctx->directory, strerror(errno));
closedir(dir);
}
}
diff --git a/src/bin/pg_resetxlog/pg_resetxlog.c b/src/bin/pg_resetxlog/pg_resetxlog.c
new file mode 100644
index 28a4f19..7bd8052
*** a/src/bin/pg_resetxlog/pg_resetxlog.c
--- b/src/bin/pg_resetxlog/pg_resetxlog.c
*************** FindEndOfXLOG(void)
*** 821,828 ****
exit(1);
}
! errno = 0;
! while ((xlde = readdir(xldir)) != NULL)
{
if (strlen(xlde->d_name) == 24 &&
strspn(xlde->d_name, "0123456789ABCDEF") == 24)
--- 821,827 ----
exit(1);
}
! while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
if (strlen(xlde->d_name) == 24 &&
strspn(xlde->d_name, "0123456789ABCDEF") == 24)
*************** FindEndOfXLOG(void)
*** 844,850 ****
if (segno > newXlogSegNo)
newXlogSegNo = segno;
}
- errno = 0;
}
#ifdef WIN32
--- 843,848 ----
*************** KillExistingXLOG(void)
*** 892,899 ****
exit(1);
}
! errno = 0;
! while ((xlde = readdir(xldir)) != NULL)
{
if (strlen(xlde->d_name) == 24 &&
strspn(xlde->d_name, "0123456789ABCDEF") == 24)
--- 890,896 ----
exit(1);
}
! while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
if (strlen(xlde->d_name) == 24 &&
strspn(xlde->d_name, "0123456789ABCDEF") == 24)
*************** KillExistingXLOG(void)
*** 906,912 ****
exit(1);
}
}
- errno = 0;
}
#ifdef WIN32
--- 903,908 ----
*************** KillExistingArchiveStatus(void)
*** 948,955 ****
exit(1);
}
! errno = 0;
! while ((xlde = readdir(xldir)) != NULL)
{
if (strspn(xlde->d_name, "0123456789ABCDEF") == 24 &&
(strcmp(xlde->d_name + 24, ".ready") == 0 ||
--- 944,950 ----
exit(1);
}
! while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
if (strspn(xlde->d_name, "0123456789ABCDEF") == 24 &&
(strcmp(xlde->d_name + 24, ".ready") == 0 ||
*************** KillExistingArchiveStatus(void)
*** 963,969 ****
exit(1);
}
}
- errno = 0;
}
#ifdef WIN32
--- 958,963 ----
diff --git a/src/common/pgfnames.c b/src/common/pgfnames.c
new file mode 100644
index 9a68163..0d852b1
*** a/src/common/pgfnames.c
--- b/src/common/pgfnames.c
*************** pgfnames(const char *path)
*** 50,57 ****
filenames = (char **) palloc(fnsize * sizeof(char *));
! errno = 0;
! while ((file = readdir(dir)) != NULL)
{
if (strcmp(file->d_name, ".") != 0 && strcmp(file->d_name, "..") != 0)
{
--- 50,56 ----
filenames = (char **) palloc(fnsize * sizeof(char *));
! while (errno = 0, (file = readdir(dir)) != NULL)
{
if (strcmp(file->d_name, ".") != 0 && strcmp(file->d_name, "..") != 0)
{
*************** pgfnames(const char *path)
*** 63,69 ****
}
filenames[numnames++] = pstrdup(file->d_name);
}
- errno = 0;
}
#ifdef WIN32
--- 62,67 ----
diff --git a/src/port/pgcheckdir.c b/src/port/pgcheckdir.c
new file mode 100644
index fc97f8c..6454525
*** a/src/port/pgcheckdir.c
--- b/src/port/pgcheckdir.c
*************** pg_check_dir(const char *dir)
*** 33,46 ****
struct dirent *file;
bool dot_found = false;
- errno = 0;
-
chkdir = opendir(dir);
-
if (chkdir == NULL)
return (errno == ENOENT) ? 0 : -1;
! while ((file = readdir(chkdir)) != NULL)
{
if (strcmp(".", file->d_name) == 0 ||
strcmp("..", file->d_name) == 0)
--- 33,43 ----
struct dirent *file;
bool dot_found = false;
chkdir = opendir(dir);
if (chkdir == NULL)
return (errno == ENOENT) ? 0 : -1;
! while (errno = 0, (file = readdir(chkdir)) != NULL)
{
if (strcmp(".", file->d_name) == 0 ||
strcmp("..", file->d_name) == 0)
*************** pg_check_dir(const char *dir)
*** 76,84 ****
errno = 0;
#endif
! closedir(chkdir);
!
! if (errno != 0)
result = -1; /* some kind of I/O error? */
/* We report on dot-files if we _only_ find dot files */
--- 73,79 ----
errno = 0;
#endif
! if (errno || closedir(chkdir) == -1)
result = -1; /* some kind of I/O error? */
/* We report on dot-files if we _only_ find dot files */
On Thu, Mar 13, 2014 at 1:48 AM, Bruce Momjian <bruce@momjian.us> wrote:
On Mon, Dec 9, 2013 at 11:27:28AM -0500, Robert Haas wrote:
On Thu, Dec 5, 2013 at 6:15 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
But the other usages seem to be in assorted utilities, which
will need to do it right for themselves. initdb.c's walkdir() seems to
have it right and might be a reasonable model to follow. Or maybe we
should invent a frontend-friendly version of ReadDir() rather than
duplicating all the error checking code in ten-and-counting places?If there's enough uniformity in all of those places to make that
feasible, it certainly seems wise to do it that way. I don't know if
that's the case, though - e.g. maybe some callers want to exit and
others do not. pg_resetxlog wants to exit; pg_archivecleanup and
pg_standby most likely want to print an error and carry on.I have developed the attached patch which fixes all cases where
readdir() wasn't checking for errno, and cleaned up the syntax in other
cases to be consistent.
Thanks!
While I am not a fan of backpatching, the fact we are ignoring errors in
some critical cases seems the non-cosmetic parts should be backpatched.
While I haven't read the patch, I agree that this is a back-patchable bug fix.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Mar 13, 2014 at 11:18 AM, Bruce Momjian <bruce@momjian.us> wrote:
I have developed the attached patch which fixes all cases where
readdir() wasn't checking for errno, and cleaned up the syntax in other
cases to be consistent.
1. One common thing missed wherever handling for errno is added
is below check which is present in all existing cases where errno
is used (initdb.c, pg_resetxlog.c, ReadDir, ..)
#ifdef WIN32
/*
* This fix is in mingw cvs (runtime/mingwex/dirent.c rev 1.4), but not in
* released version
*/
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
2.
! if (errno || closedir(chkdir) == -1)
result = -1; /* some kind of I/O error? */
Is there a special need to check return value of closedir in this
function, as all other uses (initdb.c, pg_resetxlog.c, pgfnames.c)
of it in similar context doesn't check the same?
One thing I think for which this code needs change is to check
errno before closedir as is done in initdb.c or pg_resetxlog.c
While I am not a fan of backpatching, the fact we are ignoring errors in
some critical cases seems the non-cosmetic parts should be backpatched.
+1
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Mar 18, 2014 at 11:25:46AM +0530, Amit Kapila wrote:
On Thu, Mar 13, 2014 at 11:18 AM, Bruce Momjian <bruce@momjian.us> wrote:
I have developed the attached patch which fixes all cases where
readdir() wasn't checking for errno, and cleaned up the syntax in other
cases to be consistent.1. One common thing missed wherever handling for errno is added
is below check which is present in all existing cases where errno
is used (initdb.c, pg_resetxlog.c, ReadDir, ..)#ifdef WIN32
/*
* This fix is in mingw cvs (runtime/mingwex/dirent.c rev 1.4), but not in
* released version
*/
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
Very good point. I have modified the patch to add this block in all
cases where it was missing. I started to wonder about the comment and
if the Mingw fix was released. Based on some research, I see this as
fixed in mingw-runtime-3.2, released 2003-10-10. That's pretty old.
(What I don't know is when that was paired with Msys in a bundled
release.) Here is the Mingw fixed code:
http://ftp.ntua.gr/mirror/mingw/OldFiles/mingw-runtime-3.2-src.tar.gz
{
/* Get the next search entry. */
if (_tfindnext (dirp->dd_handle, &(dirp->dd_dta)))
{
/* We are off the end or otherwise error.
_findnext sets errno to ENOENT if no more file
Undo this. */
DWORD winerr = GetLastError();
if (winerr == ERROR_NO_MORE_FILES)
errno = 0;
The current code has a better explanation:
http://sourceforge.net/p/mingw/mingw-org-wsl/ci/master/tree/src/libcrt/tchar/dirent.c
if( dirp->dd_private.dd_stat++ > 0 )
{
/* Otherwise...
*
* Get the next search entry. POSIX mandates that this must
* return NULL after the last entry has been read, but that it
* MUST NOT change errno in this case. MS-Windows _findnext()
* DOES change errno (to ENOENT) after the last entry has been
* read, so we must be prepared to restore it to its previous
* value, when no actual error has occurred.
*/
int prev_errno = errno;
if( DIRENT_UPDATE( dirp->dd_private ) != 0 )
{
/* May be an error, or just the case described above...
*/
if( GetLastError() == ERROR_NO_MORE_FILES )
/*
* ...which requires us to reset errno.
*/
errno = prev_errno;
but it is basically doing the same thing. I am wondering if we should
back-patch the PG code block where it was missing, and remove it from
head in all places on the logic that everyone running 9.4 will have a
post-3.1 version of Mingw. Postgres 8.4 was released in 2009 and it is
possible some people are still using pre-3.2 Mingw versions with that PG
release.
2.
! if (errno || closedir(chkdir) == -1)
result = -1; /* some kind of I/O error? */Is there a special need to check return value of closedir in this
function, as all other uses (initdb.c, pg_resetxlog.c, pgfnames.c)
of it in similar context doesn't check the same?One thing I think for which this code needs change is to check
errno before closedir as is done in initdb.c or pg_resetxlog.c
Yes, good point. Patch adjusted to add this.
While I am not a fan of backpatching, the fact we are ignoring errors in
some critical cases seems the non-cosmetic parts should be backpatched.+1
The larger the patch gets, the more worried I am about backpatching.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Bruce Momjian <bruce@momjian.us> writes:
Very good point. I have modified the patch to add this block in all
cases where it was missing. I started to wonder about the comment and
if the Mingw fix was released. Based on some research, I see this as
fixed in mingw-runtime-3.2, released 2003-10-10. That's pretty old.
Yeah. I would vote for removing that code in all branches. There is no
reason to suppose somebody is going to install 8.4.22 on a machine that
they haven't updated mingw on since 2003. Or, if you prefer, just remove
it in HEAD --- but going around and *adding* more copies seems like
make-work. The fact that we've not heard complaints about the omissions
is good evidence that nobody's using the buggy mingw versions anymore.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Mar 18, 2014 at 9:56 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Bruce Momjian <bruce@momjian.us> writes:
Very good point. I have modified the patch to add this block in all
cases where it was missing. I started to wonder about the comment and
if the Mingw fix was released. Based on some research, I see this as
fixed in mingw-runtime-3.2, released 2003-10-10. That's pretty old.Yeah. I would vote for removing that code in all branches. There is no
reason to suppose somebody is going to install 8.4.22 on a machine that
they haven't updated mingw on since 2003. Or, if you prefer, just remove
it in HEAD --- but going around and *adding* more copies seems like
make-work. The fact that we've not heard complaints about the omissions
is good evidence that nobody's using the buggy mingw versions anymore.
I don't think it is. Right now we're not checking errno *at all* in a
bunch of these places, so we're sure not going to get complaints about
doing it incorrectly in those places. Or do I need more caffeine?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Mar 18, 2014 at 10:03:46AM -0400, Robert Haas wrote:
On Tue, Mar 18, 2014 at 9:56 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Bruce Momjian <bruce@momjian.us> writes:
Very good point. I have modified the patch to add this block in all
cases where it was missing. I started to wonder about the comment and
if the Mingw fix was released. Based on some research, I see this as
fixed in mingw-runtime-3.2, released 2003-10-10. That's pretty old.Yeah. I would vote for removing that code in all branches. There is no
reason to suppose somebody is going to install 8.4.22 on a machine that
they haven't updated mingw on since 2003. Or, if you prefer, just remove
it in HEAD --- but going around and *adding* more copies seems like
make-work. The fact that we've not heard complaints about the omissions
is good evidence that nobody's using the buggy mingw versions anymore.I don't think it is. Right now we're not checking errno *at all* in a
bunch of these places, so we're sure not going to get complaints about
doing it incorrectly in those places. Or do I need more caffeine?
You are correct. This code is seriously broken and I am susprised we
have not gotten more complaints. Good thing readdir/closedir rarely
fail.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Bruce Momjian escribi�:
On Tue, Mar 18, 2014 at 10:03:46AM -0400, Robert Haas wrote:
On Tue, Mar 18, 2014 at 9:56 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Bruce Momjian <bruce@momjian.us> writes:
Very good point. I have modified the patch to add this block in all
cases where it was missing. I started to wonder about the comment and
if the Mingw fix was released. Based on some research, I see this as
fixed in mingw-runtime-3.2, released 2003-10-10. That's pretty old.Yeah. I would vote for removing that code in all branches. There is no
reason to suppose somebody is going to install 8.4.22 on a machine that
they haven't updated mingw on since 2003. Or, if you prefer, just remove
it in HEAD --- but going around and *adding* more copies seems like
make-work. The fact that we've not heard complaints about the omissions
is good evidence that nobody's using the buggy mingw versions anymore.I don't think it is. Right now we're not checking errno *at all* in a
bunch of these places, so we're sure not going to get complaints about
doing it incorrectly in those places. Or do I need more caffeine?You are correct. This code is seriously broken and I am susprised we
have not gotten more complaints. Good thing readdir/closedir rarely
fail.
I think we need to keep the check for old mingw runtime in older
branches; it seems reasonable to keep updating Postgres when new
versions come out but keep mingw the same if it doesn't break. A good
criterion here, to me, is: would we make it a runtime error if an old
mingw version is detected? If we would, then let's go and remove all
those errno checks. Then we force everyone to update to a sane mingw.
But if we're not adding such a check, then we might cause subtle trouble
just because we think running old mingw is unlikely.
On another note, please let's not make the code dissimilar in some
branches just because of source code embellishments are not back-ported
out of fear. I mean, if we want them in master, let them be in older
branches as well. Otherwise we end up with slightly different versions
that make back-patching future fixes a lot harder, for no gain.
--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 18 March 2014 14:15, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
Bruce Momjian escribió:
On Tue, Mar 18, 2014 at 10:03:46AM -0400, Robert Haas wrote:
On Tue, Mar 18, 2014 at 9:56 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Bruce Momjian <bruce@momjian.us> writes:
Very good point. I have modified the patch to add this block in all
cases where it was missing. I started to wonder about the comment and
if the Mingw fix was released. Based on some research, I see this as
fixed in mingw-runtime-3.2, released 2003-10-10. That's pretty old.Yeah. I would vote for removing that code in all branches. There is no
reason to suppose somebody is going to install 8.4.22 on a machine that
they haven't updated mingw on since 2003. Or, if you prefer, just remove
it in HEAD --- but going around and *adding* more copies seems like
make-work. The fact that we've not heard complaints about the omissions
is good evidence that nobody's using the buggy mingw versions anymore.I don't think it is. Right now we're not checking errno *at all* in a
bunch of these places, so we're sure not going to get complaints about
doing it incorrectly in those places. Or do I need more caffeine?You are correct. This code is seriously broken and I am susprised we
have not gotten more complaints. Good thing readdir/closedir rarely
fail.
back-patching
Some commentary on this...
Obviously, all errors are mine.
If pg_archivecleanup is a problem, then so is pg_standby a problem.
Given the above, this means we've run for about 7 years without a
reported issue on this. If we are going to "make this better" by
actually having it throw errors in places that didn't throw errors
before, are we sure that is going to make people happier? The archive
cleanup isn't exactly critical in most cases, so dynamic errors don't
matter much.
Also, the programs were originally written to work as standalone
program as well as an archive_cleanup_command. So we can't use
PostgreSQL infrastructure (can we?). That aspect is needed to allow
testing the program before it goes live.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Mar 18, 2014 at 11:36 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
Given the above, this means we've run for about 7 years without a
reported issue on this. If we are going to "make this better" by
actually having it throw errors in places that didn't throw errors
before, are we sure that is going to make people happier? The archive
cleanup isn't exactly critical in most cases, so dynamic errors don't
matter much.
We report errors returned by system calls in many other places. I
can't see why this place should be any different.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 18 March 2014 15:50, Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Mar 18, 2014 at 11:36 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
Given the above, this means we've run for about 7 years without a
reported issue on this. If we are going to "make this better" by
actually having it throw errors in places that didn't throw errors
before, are we sure that is going to make people happier? The archive
cleanup isn't exactly critical in most cases, so dynamic errors don't
matter much.We report errors returned by system calls in many other places. I
can't see why this place should be any different.
Sure. Just wanted to make sure it's a conscious, explicit choice to do so.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 13 March 2014 05:48, Bruce Momjian <bruce@momjian.us> wrote:
On Mon, Dec 9, 2013 at 11:27:28AM -0500, Robert Haas wrote:
On Thu, Dec 5, 2013 at 6:15 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
But the other usages seem to be in assorted utilities, which
will need to do it right for themselves. initdb.c's walkdir() seems to
have it right and might be a reasonable model to follow. Or maybe we
should invent a frontend-friendly version of ReadDir() rather than
duplicating all the error checking code in ten-and-counting places?If there's enough uniformity in all of those places to make that
feasible, it certainly seems wise to do it that way. I don't know if
that's the case, though - e.g. maybe some callers want to exit and
others do not. pg_resetxlog wants to exit; pg_archivecleanup and
pg_standby most likely want to print an error and carry on.I have developed the attached patch which fixes all cases where
readdir() wasn't checking for errno, and cleaned up the syntax in other
cases to be consistent.While I am not a fan of backpatching, the fact we are ignoring errors in
some critical cases seems the non-cosmetic parts should be backpatched.
pg_resetxlog was not an offender here; its coding was sound.
We shouldn't be discussing backpatching a patch that contains changes
to coding style.
ISTM we should change the code with missing checks to adopt the coding
style of pg_resetxlog, not the other way around.
I assume you or Kevin have this in hand and you don't want me to apply
the patch? (Since it was originally my bug)
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Mar 18, 2014 at 04:17:53PM +0000, Simon Riggs wrote:
While I am not a fan of backpatching, the fact we are ignoring errors in
some critical cases seems the non-cosmetic parts should be backpatched.pg_resetxlog was not an offender here; its coding was sound.
We shouldn't be discussing backpatching a patch that contains changes
to coding style.
I was going to remove the coding style changes to pg_resetxlog from the
backpatched portion.
ISTM we should change the code with missing checks to adopt the coding
style of pg_resetxlog, not the other way around.I assume you or Kevin have this in hand and you don't want me to apply
the patch? (Since it was originally my bug)
I know the email subject says pg_archivecleanup but the problem is all
over our code.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 18 March 2014 18:01, Bruce Momjian <bruce@momjian.us> wrote:
On Tue, Mar 18, 2014 at 04:17:53PM +0000, Simon Riggs wrote:
While I am not a fan of backpatching, the fact we are ignoring errors in
some critical cases seems the non-cosmetic parts should be backpatched.pg_resetxlog was not an offender here; its coding was sound.
We shouldn't be discussing backpatching a patch that contains changes
to coding style.I was going to remove the coding style changes to pg_resetxlog from the
backpatched portion.
Why make style changes at all? A patch with no style changes would
mean backpatch and HEAD versions would be the same.
ISTM we should change the code with missing checks to adopt the coding
style of pg_resetxlog, not the other way around.I assume you or Kevin have this in hand and you don't want me to apply
the patch? (Since it was originally my bug)I know the email subject says pg_archivecleanup but the problem is all
over our code.
Yes, understood.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Mar 18, 2014 at 06:11:30PM +0000, Simon Riggs wrote:
On 18 March 2014 18:01, Bruce Momjian <bruce@momjian.us> wrote:
On Tue, Mar 18, 2014 at 04:17:53PM +0000, Simon Riggs wrote:
While I am not a fan of backpatching, the fact we are ignoring errors in
some critical cases seems the non-cosmetic parts should be backpatched.pg_resetxlog was not an offender here; its coding was sound.
We shouldn't be discussing backpatching a patch that contains changes
to coding style.I was going to remove the coding style changes to pg_resetxlog from the
backpatched portion.Why make style changes at all? A patch with no style changes would
mean backpatch and HEAD versions would be the same.
The old style had errno set in two places in the loop, while the new
style has it set in only one place.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Bruce Momjian escribi�:
On Tue, Mar 18, 2014 at 06:11:30PM +0000, Simon Riggs wrote:
On 18 March 2014 18:01, Bruce Momjian <bruce@momjian.us> wrote:
On Tue, Mar 18, 2014 at 04:17:53PM +0000, Simon Riggs wrote:
While I am not a fan of backpatching, the fact we are ignoring errors in
some critical cases seems the non-cosmetic parts should be backpatched.pg_resetxlog was not an offender here; its coding was sound.
We shouldn't be discussing backpatching a patch that contains changes
to coding style.I was going to remove the coding style changes to pg_resetxlog from the
backpatched portion.Why make style changes at all? A patch with no style changes would
mean backpatch and HEAD versions would be the same.The old style had errno set in two places in the loop, while the new
style has it set in only one place.
I think it makes more sense to have all callsites look the same in all
supported branches.
--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 18 March 2014 18:18, Bruce Momjian <bruce@momjian.us> wrote:
On Tue, Mar 18, 2014 at 06:11:30PM +0000, Simon Riggs wrote:
On 18 March 2014 18:01, Bruce Momjian <bruce@momjian.us> wrote:
On Tue, Mar 18, 2014 at 04:17:53PM +0000, Simon Riggs wrote:
While I am not a fan of backpatching, the fact we are ignoring errors in
some critical cases seems the non-cosmetic parts should be backpatched.pg_resetxlog was not an offender here; its coding was sound.
We shouldn't be discussing backpatching a patch that contains changes
to coding style.I was going to remove the coding style changes to pg_resetxlog from the
backpatched portion.Why make style changes at all? A patch with no style changes would
mean backpatch and HEAD versions would be the same.The old style had errno set in two places in the loop, while the new
style has it set in only one place.
Seems better to leave the previously-good coding in place. ISTM to be
clearer to use simple C.
You're doing this anyway for the backpatch, so I'm not causing you any
more work. Better one patch than two.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Simon Riggs escribi�:
On 18 March 2014 18:18, Bruce Momjian <bruce@momjian.us> wrote:
On Tue, Mar 18, 2014 at 06:11:30PM +0000, Simon Riggs wrote:
Why make style changes at all? A patch with no style changes would
mean backpatch and HEAD versions would be the same.The old style had errno set in two places in the loop, while the new
style has it set in only one place.Seems better to leave the previously-good coding in place. ISTM to be
clearer to use simple C.
If you're saying we should use that style in all readdir loops, with the
errno=0 before the loop and at the bottom of it, I don't disagree.
Let's just make sure they're all safe though (i.e. watch out for
"continue" for instance).
That said, I don't find comma expression to be particularly "not
simple".
--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 18 March 2014 18:55, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
That said, I don't find comma expression to be particularly "not
simple".
Maybe, but we've not used it before anywhere in Postgres, so I don't
see a reason to start now. Especially since C is not the native
language of many people these days and people just won't understand
it.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 03/18/2014 09:04 PM, Simon Riggs wrote:
On 18 March 2014 18:55, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
That said, I don't find comma expression to be particularly "not
simple".Maybe, but we've not used it before anywhere in Postgres, so I don't
see a reason to start now. Especially since C is not the native
language of many people these days and people just won't understand
it.
Agreed. The psqlODBC code is littered with comma expressions, and the
first time I saw it, it took me a really long time to figure out what
"if (foo = malloc(...), foo) { } " meant. And I consider myself quite
experienced with C.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Mar 18, 2014 at 09:13:28PM +0200, Heikki Linnakangas wrote:
On 03/18/2014 09:04 PM, Simon Riggs wrote:
On 18 March 2014 18:55, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
That said, I don't find comma expression to be particularly "not
simple".Maybe, but we've not used it before anywhere in Postgres, so I don't
see a reason to start now. Especially since C is not the native
language of many people these days and people just won't understand
it.Agreed. The psqlODBC code is littered with comma expressions, and
the first time I saw it, it took me a really long time to figure out
what "if (foo = malloc(...), foo) { } " meant. And I consider myself
quite experienced with C.
I can see how the comma syntax would be confusing, though it does the
job well. Attached is a patch that does the double-errno. However,
some of these loops are large, and there are 'continue' calls in there,
causing the addition of many new errno locations. I am not totally
comfortable that this new coding layout will stay unbroken.
Would people accept?
for (errno = 0; (dirent = readdir(dir)) != NULL; errno = 0)
That would keep the errno's together and avoid the 'continue' additions.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
Attachments:
readdir.difftext/x-diff; charset=us-asciiDownload
diff --git a/contrib/pg_archivecleanup/pg_archivecleanup.c b/contrib/pg_archivecleanup/pg_archivecleanup.c
new file mode 100644
index 7b5484b..405ec48
*** a/contrib/pg_archivecleanup/pg_archivecleanup.c
--- b/contrib/pg_archivecleanup/pg_archivecleanup.c
*************** CleanupPriorWALFiles(void)
*** 106,111 ****
--- 106,112 ----
if ((xldir = opendir(archiveLocation)) != NULL)
{
+ errno = 0;
while ((xlde = readdir(xldir)) != NULL)
{
strncpy(walfile, xlde->d_name, MAXPGPATH);
*************** CleanupPriorWALFiles(void)
*** 148,153 ****
--- 149,155 ----
fprintf(stderr,
"%s: file \"%s\" would be removed\n",
progname, WALFilePath);
+ errno = 0;
continue;
}
*************** CleanupPriorWALFiles(void)
*** 163,170 ****
break;
}
}
}
! closedir(xldir);
}
else
fprintf(stderr, "%s: could not open archive location \"%s\": %s\n",
--- 165,185 ----
break;
}
}
+ errno = 0;
}
!
! #ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
! if (GetLastError() == ERROR_NO_MORE_FILES)
! errno = 0;
! #endif
!
! if (errno)
! fprintf(stderr, "%s: could not read archive location \"%s\": %s\n",
! progname, archiveLocation, strerror(errno));
! if (!closedir(xldir))
! fprintf(stderr, "%s: could not close archive location \"%s\": %s\n",
! progname, archiveLocation, strerror(errno));
}
else
fprintf(stderr, "%s: could not open archive location \"%s\": %s\n",
diff --git a/contrib/pg_standby/pg_standby.c b/contrib/pg_standby/pg_standby.c
new file mode 100644
index 8ddd486..d4731d7
*** a/contrib/pg_standby/pg_standby.c
--- b/contrib/pg_standby/pg_standby.c
*************** CustomizableCleanupPriorWALFiles(void)
*** 245,250 ****
--- 245,251 ----
*/
if ((xldir = opendir(archiveLocation)) != NULL)
{
+ errno = 0;
while ((xlde = readdir(xldir)) != NULL)
{
/*
*************** CustomizableCleanupPriorWALFiles(void)
*** 282,288 ****
--- 283,300 ----
break;
}
}
+ errno = 0;
}
+
+ #ifdef WIN32
+ /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
+ if (GetLastError() == ERROR_NO_MORE_FILES)
+ errno = 0;
+ #endif
+
+ if (errno)
+ fprintf(stderr, "%s: could not read archive location \"%s\": %s\n",
+ progname, archiveLocation, strerror(errno));
if (debug)
fprintf(stderr, "\n");
}
*************** CustomizableCleanupPriorWALFiles(void)
*** 290,296 ****
fprintf(stderr, "%s: could not open archive location \"%s\": %s\n",
progname, archiveLocation, strerror(errno));
! closedir(xldir);
fflush(stderr);
}
}
--- 302,311 ----
fprintf(stderr, "%s: could not open archive location \"%s\": %s\n",
progname, archiveLocation, strerror(errno));
! if (!closedir(xldir))
! fprintf(stderr, "%s: could not close archive location \"%s\": %s\n",
! progname, archiveLocation, strerror(errno));
!
fflush(stderr);
}
}
diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c
new file mode 100644
index 4dc809d..5158cfe
*** a/src/backend/storage/file/fd.c
--- b/src/backend/storage/file/fd.c
*************** ReadDir(DIR *dir, const char *dirname)
*** 1957,1966 ****
return dent;
#ifdef WIN32
! /*
! * This fix is in mingw cvs (runtime/mingwex/dirent.c rev 1.4), but not in
! * released version
! */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
--- 1957,1963 ----
return dent;
#ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
new file mode 100644
index 61bd785..7416377
*** a/src/bin/initdb/initdb.c
--- b/src/bin/initdb/initdb.c
*************** walkdir(char *path, void (*action) (char
*** 541,553 ****
exit_nicely();
}
! while (errno = 0, (direntry = readdir(dir)) != NULL)
{
struct stat fst;
if (strcmp(direntry->d_name, ".") == 0 ||
strcmp(direntry->d_name, "..") == 0)
continue;
snprintf(subpath, MAXPGPATH, "%s/%s", path, direntry->d_name);
--- 541,557 ----
exit_nicely();
}
! errno = 0;
! while ((direntry = readdir(dir)) != NULL)
{
struct stat fst;
if (strcmp(direntry->d_name, ".") == 0 ||
strcmp(direntry->d_name, "..") == 0)
+ {
+ errno = 0;
continue;
+ }
snprintf(subpath, MAXPGPATH, "%s/%s", path, direntry->d_name);
*************** walkdir(char *path, void (*action) (char
*** 562,574 ****
walkdir(subpath, action);
else if (S_ISREG(fst.st_mode))
(*action) (subpath, false);
}
#ifdef WIN32
! /*
! * This fix is in mingw cvs (runtime/mingwex/dirent.c rev 1.4), but not in
! * released version
! */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
--- 566,576 ----
walkdir(subpath, action);
else if (S_ISREG(fst.st_mode))
(*action) (subpath, false);
+ errno = 0;
}
#ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
*************** walkdir(char *path, void (*action) (char
*** 580,586 ****
exit_nicely();
}
! closedir(dir);
/*
* It's important to fsync the destination directory itself as individual
--- 582,593 ----
exit_nicely();
}
! if (!closedir(dir))
! {
! fprintf(stderr, _("%s: could not close directory \"%s\": %s\n"),
! progname, path, strerror(errno));
! exit_nicely();
! }
/*
* It's important to fsync the destination directory itself as individual
diff --git a/src/bin/pg_basebackup/pg_receivexlog.c b/src/bin/pg_basebackup/pg_receivexlog.c
new file mode 100644
index 2478789..9b62edf
*** a/src/bin/pg_basebackup/pg_receivexlog.c
--- b/src/bin/pg_basebackup/pg_receivexlog.c
*************** FindStreamingStart(uint32 *tli)
*** 139,144 ****
--- 139,145 ----
disconnect_and_exit(1);
}
+ errno = 0;
while ((dirent = readdir(dir)) != NULL)
{
uint32 tli;
*************** FindStreamingStart(uint32 *tli)
*** 153,171 ****
--- 154,184 ----
if (strlen(dirent->d_name) == 24)
{
if (strspn(dirent->d_name, "0123456789ABCDEF") != 24)
+ {
+ errno = 0;
continue;
+ }
ispartial = false;
}
else if (strlen(dirent->d_name) == 32)
{
if (strspn(dirent->d_name, "0123456789ABCDEF") != 24)
+ {
+ errno = 0;
continue;
+ }
if (strcmp(&dirent->d_name[24], ".partial") != 0)
+ {
+ errno = 0;
continue;
+ }
ispartial = true;
}
else
+ {
+ errno = 0;
continue;
+ }
/*
* Looks like an xlog file. Parse its position.
*************** FindStreamingStart(uint32 *tli)
*** 194,199 ****
--- 207,213 ----
fprintf(stderr,
_("%s: segment file \"%s\" has incorrect size %d, skipping\n"),
progname, dirent->d_name, (int) statbuf.st_size);
+ errno = 0;
continue;
}
}
*************** FindStreamingStart(uint32 *tli)
*** 207,215 ****
high_tli = tli;
high_ispartial = ispartial;
}
}
! closedir(dir);
if (high_segno > 0)
{
--- 221,248 ----
high_tli = tli;
high_ispartial = ispartial;
}
+ errno = 0;
}
! #ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
! if (GetLastError() == ERROR_NO_MORE_FILES)
! errno = 0;
! #endif
!
! if (errno)
! {
! fprintf(stderr, _("%s: could not read directory \"%s\": %s\n"),
! progname, basedir, strerror(errno));
! disconnect_and_exit(1);
! }
!
! if (!closedir(dir))
! {
! fprintf(stderr, _("%s: could not close directory \"%s\": %s\n"),
! progname, basedir, strerror(errno));
! disconnect_and_exit(1);
! }
if (high_segno > 0)
{
diff --git a/src/bin/pg_dump/pg_backup_directory.c b/src/bin/pg_dump/pg_backup_directory.c
new file mode 100644
index 1bed8a9..161bfb0
*** a/src/bin/pg_dump/pg_backup_directory.c
--- b/src/bin/pg_dump/pg_backup_directory.c
*************** InitArchiveFmt_Directory(ArchiveHandle *
*** 177,182 ****
--- 177,183 ----
struct dirent *d;
is_empty = true;
+ errno = 0;
while ((d = readdir(dir)))
{
if (strcmp(d->d_name, ".") != 0 && strcmp(d->d_name, "..") != 0)
*************** InitArchiveFmt_Directory(ArchiveHandle *
*** 184,191 ****
is_empty = false;
break;
}
}
! closedir(dir);
}
}
--- 185,205 ----
is_empty = false;
break;
}
+ errno = 0;
}
!
! #ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
! if (GetLastError() == ERROR_NO_MORE_FILES)
! errno = 0;
! #endif
!
! if (errno)
! exit_horribly(modulename, "could not read directory \"%s\": %s\n",
! ctx->directory, strerror(errno));
! if (!closedir(dir))
! exit_horribly(modulename, "could not close directory \"%s\": %s\n",
! ctx->directory, strerror(errno));
}
}
diff --git a/src/bin/pg_resetxlog/pg_resetxlog.c b/src/bin/pg_resetxlog/pg_resetxlog.c
new file mode 100644
index 28a4f19..e63a8e4
*** a/src/bin/pg_resetxlog/pg_resetxlog.c
--- b/src/bin/pg_resetxlog/pg_resetxlog.c
*************** FindEndOfXLOG(void)
*** 848,868 ****
}
#ifdef WIN32
! /*
! * This fix is in mingw cvs (runtime/mingwex/dirent.c rev 1.4), but not in
! * released version
! */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
if (errno)
{
! fprintf(stderr, _("%s: could not read from directory \"%s\": %s\n"),
progname, XLOGDIR, strerror(errno));
exit(1);
}
- closedir(xldir);
/*
* Finally, convert to new xlog seg size, and advance by one to ensure we
--- 848,870 ----
}
#ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
if (errno)
{
! fprintf(stderr, _("%s: could not read directory \"%s\": %s\n"),
! progname, XLOGDIR, strerror(errno));
! exit(1);
! }
! if (!closedir(xldir))
! {
! fprintf(stderr, _("%s: could not close directory \"%s\": %s\n"),
progname, XLOGDIR, strerror(errno));
exit(1);
}
/*
* Finally, convert to new xlog seg size, and advance by one to ensure we
*************** KillExistingXLOG(void)
*** 910,930 ****
}
#ifdef WIN32
! /*
! * This fix is in mingw cvs (runtime/mingwex/dirent.c rev 1.4), but not in
! * released version
! */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
if (errno)
{
! fprintf(stderr, _("%s: could not read from directory \"%s\": %s\n"),
progname, XLOGDIR, strerror(errno));
exit(1);
}
- closedir(xldir);
}
--- 912,934 ----
}
#ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
if (errno)
{
! fprintf(stderr, _("%s: could not read directory \"%s\": %s\n"),
! progname, XLOGDIR, strerror(errno));
! exit(1);
! }
! if (!closedir(xldir))
! {
! fprintf(stderr, _("%s: could not close directory \"%s\": %s\n"),
progname, XLOGDIR, strerror(errno));
exit(1);
}
}
*************** KillExistingArchiveStatus(void)
*** 967,987 ****
}
#ifdef WIN32
! /*
! * This fix is in mingw cvs (runtime/mingwex/dirent.c rev 1.4), but not in
! * released version
! */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
if (errno)
{
! fprintf(stderr, _("%s: could not read from directory \"%s\": %s\n"),
progname, ARCHSTATDIR, strerror(errno));
exit(1);
}
- closedir(xldir);
}
--- 971,993 ----
}
#ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
if (errno)
{
! fprintf(stderr, _("%s: could not read directory \"%s\": %s\n"),
! progname, ARCHSTATDIR, strerror(errno));
! exit(1);
! }
! if (!closedir(xldir))
! {
! fprintf(stderr, _("%s: could not close directory \"%s\": %s\n"),
progname, ARCHSTATDIR, strerror(errno));
exit(1);
}
}
diff --git a/src/common/pgfnames.c b/src/common/pgfnames.c
new file mode 100644
index 9a68163..411acfd
*** a/src/common/pgfnames.c
--- b/src/common/pgfnames.c
*************** pgfnames(const char *path)
*** 67,76 ****
}
#ifdef WIN32
! /*
! * This fix is in mingw cvs (runtime/mingwex/dirent.c rev 1.4), but not in
! * released version
! */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
--- 67,73 ----
}
#ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
*************** pgfnames(const char *path)
*** 87,93 ****
filenames[numnames] = NULL;
! closedir(dir);
return filenames;
}
--- 84,98 ----
filenames[numnames] = NULL;
! if (!closedir(dir))
! {
! #ifndef FRONTEND
! elog(WARNING, "could not close directory \"%s\": %m", path);
! #else
! fprintf(stderr, _("could not close directory \"%s\": %s\n"),
! path, strerror(errno));
! #endif
! }
return filenames;
}
diff --git a/src/port/dirent.c b/src/port/dirent.c
new file mode 100644
index f9d93ea..31121c2
*** a/src/port/dirent.c
--- b/src/port/dirent.c
*************** readdir(DIR *d)
*** 111,119 ****
int
closedir(DIR *d)
{
if (d->handle != INVALID_HANDLE_VALUE)
! FindClose(d->handle);
free(d->dirname);
free(d);
! return 0;
}
--- 111,121 ----
int
closedir(DIR *d)
{
+ int ret = 1;
+
if (d->handle != INVALID_HANDLE_VALUE)
! ret = FindClose(d->handle);
free(d->dirname);
free(d);
! return !ret;
}
diff --git a/src/port/pgcheckdir.c b/src/port/pgcheckdir.c
new file mode 100644
index fc97f8c..70d5cd1
*** a/src/port/pgcheckdir.c
--- b/src/port/pgcheckdir.c
*************** pg_check_dir(const char *dir)
*** 33,51 ****
struct dirent *file;
bool dot_found = false;
- errno = 0;
-
chkdir = opendir(dir);
-
if (chkdir == NULL)
return (errno == ENOENT) ? 0 : -1;
while ((file = readdir(chkdir)) != NULL)
{
if (strcmp(".", file->d_name) == 0 ||
strcmp("..", file->d_name) == 0)
{
/* skip this and parent directory */
continue;
}
#ifndef WIN32
--- 33,50 ----
struct dirent *file;
bool dot_found = false;
chkdir = opendir(dir);
if (chkdir == NULL)
return (errno == ENOENT) ? 0 : -1;
+ errno = 0;
while ((file = readdir(chkdir)) != NULL)
{
if (strcmp(".", file->d_name) == 0 ||
strcmp("..", file->d_name) == 0)
{
/* skip this and parent directory */
+ errno = 0;
continue;
}
#ifndef WIN32
*************** pg_check_dir(const char *dir)
*** 65,84 ****
result = 4; /* not empty */
break;
}
}
#ifdef WIN32
! /*
! * This fix is in mingw cvs (runtime/mingwex/dirent.c rev 1.4), but not in
! * released version
! */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
! closedir(chkdir);
!
! if (errno != 0)
result = -1; /* some kind of I/O error? */
/* We report on dot-files if we _only_ find dot files */
--- 64,79 ----
result = 4; /* not empty */
break;
}
+ errno = 0;
}
#ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
! if (errno || closedir(chkdir) == -1)
result = -1; /* some kind of I/O error? */
/* We report on dot-files if we _only_ find dot files */
Bruce Momjian <bruce@momjian.us> writes:
Would people accept?
for (errno = 0; (dirent = readdir(dir)) != NULL; errno = 0)
It's a bit weird looking, but I agree that there's value in only needing
the errno-zeroing in precisely one place.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 03/19/2014 02:30 AM, Bruce Momjian wrote:
On Tue, Mar 18, 2014 at 09:13:28PM +0200, Heikki Linnakangas wrote:
On 03/18/2014 09:04 PM, Simon Riggs wrote:
On 18 March 2014 18:55, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
That said, I don't find comma expression to be particularly "not
simple".Maybe, but we've not used it before anywhere in Postgres, so I don't
see a reason to start now. Especially since C is not the native
language of many people these days and people just won't understand
it.Agreed. The psqlODBC code is littered with comma expressions, and
the first time I saw it, it took me a really long time to figure out
what "if (foo = malloc(...), foo) { } " meant. And I consider myself
quite experienced with C.I can see how the comma syntax would be confusing, though it does the
job well. Attached is a patch that does the double-errno. However,
some of these loops are large, and there are 'continue' calls in there,
causing the addition of many new errno locations. I am not totally
comfortable that this new coding layout will stay unbroken.Would people accept?
for (errno = 0; (dirent = readdir(dir)) != NULL; errno = 0)
That would keep the errno's together and avoid the 'continue' additions.
That's clever. A less clever way would be:
for (;;)
{
errno = 0;
if ((dirent = readdir(dir)) != NULL)
break;
...
}
I'm fine with either, but that's how I would naturally write it.
Yet another way would be to have a wrapper function for readdir that
resets errno, and just replace the current readdir() calls with that.
And now that I look at initdb.c, walkdir is using the comma expression
for this already. So there's some precedence, and it doesn't actually
look that bad. So I withdraw my objection for that approach; I'm fine
with any of the discussed alternatives, really.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Mar 19, 2014 at 09:59:19AM +0200, Heikki Linnakangas wrote:
Would people accept?
for (errno = 0; (dirent = readdir(dir)) != NULL; errno = 0)
That would keep the errno's together and avoid the 'continue' additions.
That's clever. A less clever way would be:
for (;;)
{
errno = 0;
if ((dirent = readdir(dir)) != NULL)
break;...
}I'm fine with either, but that's how I would naturally write it.
Yet another way would be to have a wrapper function for readdir that
resets errno, and just replace the current readdir() calls with
that.And now that I look at initdb.c, walkdir is using the comma
expression for this already. So there's some precedence, and it
doesn't actually look that bad. So I withdraw my objection for that
approach; I'm fine with any of the discussed alternatives, really.
OK, let's go with the comma. Ironically, the for() loop would be an odd
way to avoid commas as 'for' uses commas often:
for (i = 0, j = 1; i < 10; i++, j++)
The attached patch is slightly updated. I will apply it to head and all
the back branches, including the stylistic change to pg_resetxlog (for
consistency) and remove the MinGW block in head.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
Attachments:
readdir.difftext/x-diff; charset=us-asciiDownload
diff --git a/contrib/pg_archivecleanup/pg_archivecleanup.c b/contrib/pg_archivecleanup/pg_archivecleanup.c
new file mode 100644
index 7b5484b..4f4d507
*** a/contrib/pg_archivecleanup/pg_archivecleanup.c
--- b/contrib/pg_archivecleanup/pg_archivecleanup.c
*************** CleanupPriorWALFiles(void)
*** 106,112 ****
if ((xldir = opendir(archiveLocation)) != NULL)
{
! while ((xlde = readdir(xldir)) != NULL)
{
strncpy(walfile, xlde->d_name, MAXPGPATH);
TrimExtension(walfile, additional_ext);
--- 106,112 ----
if ((xldir = opendir(archiveLocation)) != NULL)
{
! while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
strncpy(walfile, xlde->d_name, MAXPGPATH);
TrimExtension(walfile, additional_ext);
*************** CleanupPriorWALFiles(void)
*** 164,170 ****
}
}
}
! closedir(xldir);
}
else
fprintf(stderr, "%s: could not open archive location \"%s\": %s\n",
--- 164,182 ----
}
}
}
!
! #ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
! if (GetLastError() == ERROR_NO_MORE_FILES)
! errno = 0;
! #endif
!
! if (errno)
! fprintf(stderr, "%s: could not read archive location \"%s\": %s\n",
! progname, archiveLocation, strerror(errno));
! if (!closedir(xldir))
! fprintf(stderr, "%s: could not close archive location \"%s\": %s\n",
! progname, archiveLocation, strerror(errno));
}
else
fprintf(stderr, "%s: could not open archive location \"%s\": %s\n",
diff --git a/contrib/pg_standby/pg_standby.c b/contrib/pg_standby/pg_standby.c
new file mode 100644
index 8ddd486..70fb387
*** a/contrib/pg_standby/pg_standby.c
--- b/contrib/pg_standby/pg_standby.c
*************** CustomizableCleanupPriorWALFiles(void)
*** 245,251 ****
*/
if ((xldir = opendir(archiveLocation)) != NULL)
{
! while ((xlde = readdir(xldir)) != NULL)
{
/*
* We ignore the timeline part of the XLOG segment identifiers
--- 245,251 ----
*/
if ((xldir = opendir(archiveLocation)) != NULL)
{
! while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
/*
* We ignore the timeline part of the XLOG segment identifiers
*************** CustomizableCleanupPriorWALFiles(void)
*** 283,288 ****
--- 283,298 ----
}
}
}
+
+ #ifdef WIN32
+ /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
+ if (GetLastError() == ERROR_NO_MORE_FILES)
+ errno = 0;
+ #endif
+
+ if (errno)
+ fprintf(stderr, "%s: could not read archive location \"%s\": %s\n",
+ progname, archiveLocation, strerror(errno));
if (debug)
fprintf(stderr, "\n");
}
*************** CustomizableCleanupPriorWALFiles(void)
*** 290,296 ****
fprintf(stderr, "%s: could not open archive location \"%s\": %s\n",
progname, archiveLocation, strerror(errno));
! closedir(xldir);
fflush(stderr);
}
}
--- 300,309 ----
fprintf(stderr, "%s: could not open archive location \"%s\": %s\n",
progname, archiveLocation, strerror(errno));
! if (!closedir(xldir))
! fprintf(stderr, "%s: could not close archive location \"%s\": %s\n",
! progname, archiveLocation, strerror(errno));
!
fflush(stderr);
}
}
diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c
new file mode 100644
index 4dc809d..5158cfe
*** a/src/backend/storage/file/fd.c
--- b/src/backend/storage/file/fd.c
*************** ReadDir(DIR *dir, const char *dirname)
*** 1957,1966 ****
return dent;
#ifdef WIN32
! /*
! * This fix is in mingw cvs (runtime/mingwex/dirent.c rev 1.4), but not in
! * released version
! */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
--- 1957,1963 ----
return dent;
#ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
new file mode 100644
index 61bd785..f23d988
*** a/src/bin/initdb/initdb.c
--- b/src/bin/initdb/initdb.c
*************** walkdir(char *path, void (*action) (char
*** 565,574 ****
}
#ifdef WIN32
! /*
! * This fix is in mingw cvs (runtime/mingwex/dirent.c rev 1.4), but not in
! * released version
! */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
--- 565,571 ----
}
#ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
*************** walkdir(char *path, void (*action) (char
*** 580,586 ****
exit_nicely();
}
! closedir(dir);
/*
* It's important to fsync the destination directory itself as individual
--- 577,588 ----
exit_nicely();
}
! if (!closedir(dir))
! {
! fprintf(stderr, _("%s: could not close directory \"%s\": %s\n"),
! progname, path, strerror(errno));
! exit_nicely();
! }
/*
* It's important to fsync the destination directory itself as individual
diff --git a/src/bin/pg_basebackup/pg_receivexlog.c b/src/bin/pg_basebackup/pg_receivexlog.c
new file mode 100644
index 2478789..b742f0c
*** a/src/bin/pg_basebackup/pg_receivexlog.c
--- b/src/bin/pg_basebackup/pg_receivexlog.c
*************** FindStreamingStart(uint32 *tli)
*** 139,145 ****
disconnect_and_exit(1);
}
! while ((dirent = readdir(dir)) != NULL)
{
uint32 tli;
XLogSegNo segno;
--- 139,145 ----
disconnect_and_exit(1);
}
! while (errno = 0, (dirent = readdir(dir)) != NULL)
{
uint32 tli;
XLogSegNo segno;
*************** FindStreamingStart(uint32 *tli)
*** 209,215 ****
}
}
! closedir(dir);
if (high_segno > 0)
{
--- 209,233 ----
}
}
! #ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
! if (GetLastError() == ERROR_NO_MORE_FILES)
! errno = 0;
! #endif
!
! if (errno)
! {
! fprintf(stderr, _("%s: could not read directory \"%s\": %s\n"),
! progname, basedir, strerror(errno));
! disconnect_and_exit(1);
! }
!
! if (!closedir(dir))
! {
! fprintf(stderr, _("%s: could not close directory \"%s\": %s\n"),
! progname, basedir, strerror(errno));
! disconnect_and_exit(1);
! }
if (high_segno > 0)
{
diff --git a/src/bin/pg_dump/pg_backup_directory.c b/src/bin/pg_dump/pg_backup_directory.c
new file mode 100644
index 1bed8a9..c450cda
*** a/src/bin/pg_dump/pg_backup_directory.c
--- b/src/bin/pg_dump/pg_backup_directory.c
*************** InitArchiveFmt_Directory(ArchiveHandle *
*** 177,183 ****
struct dirent *d;
is_empty = true;
! while ((d = readdir(dir)))
{
if (strcmp(d->d_name, ".") != 0 && strcmp(d->d_name, "..") != 0)
{
--- 177,183 ----
struct dirent *d;
is_empty = true;
! while (errno = 0, (d = readdir(dir)))
{
if (strcmp(d->d_name, ".") != 0 && strcmp(d->d_name, "..") != 0)
{
*************** InitArchiveFmt_Directory(ArchiveHandle *
*** 185,191 ****
break;
}
}
! closedir(dir);
}
}
--- 185,204 ----
break;
}
}
!
! #ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
! if (GetLastError() == ERROR_NO_MORE_FILES)
! errno = 0;
! #endif
!
! if (errno)
! exit_horribly(modulename, "could not read directory \"%s\": %s\n",
! ctx->directory, strerror(errno));
!
! if (!closedir(dir))
! exit_horribly(modulename, "could not close directory \"%s\": %s\n",
! ctx->directory, strerror(errno));
}
}
diff --git a/src/bin/pg_resetxlog/pg_resetxlog.c b/src/bin/pg_resetxlog/pg_resetxlog.c
new file mode 100644
index 28a4f19..b6cd72b
*** a/src/bin/pg_resetxlog/pg_resetxlog.c
--- b/src/bin/pg_resetxlog/pg_resetxlog.c
*************** FindEndOfXLOG(void)
*** 821,828 ****
exit(1);
}
! errno = 0;
! while ((xlde = readdir(xldir)) != NULL)
{
if (strlen(xlde->d_name) == 24 &&
strspn(xlde->d_name, "0123456789ABCDEF") == 24)
--- 821,827 ----
exit(1);
}
! while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
if (strlen(xlde->d_name) == 24 &&
strspn(xlde->d_name, "0123456789ABCDEF") == 24)
*************** FindEndOfXLOG(void)
*** 844,868 ****
if (segno > newXlogSegNo)
newXlogSegNo = segno;
}
- errno = 0;
}
#ifdef WIN32
! /*
! * This fix is in mingw cvs (runtime/mingwex/dirent.c rev 1.4), but not in
! * released version
! */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
if (errno)
{
! fprintf(stderr, _("%s: could not read from directory \"%s\": %s\n"),
progname, XLOGDIR, strerror(errno));
exit(1);
}
- closedir(xldir);
/*
* Finally, convert to new xlog seg size, and advance by one to ensure we
--- 843,869 ----
if (segno > newXlogSegNo)
newXlogSegNo = segno;
}
}
#ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
if (errno)
{
! fprintf(stderr, _("%s: could not read directory \"%s\": %s\n"),
! progname, XLOGDIR, strerror(errno));
! exit(1);
! }
!
! if (!closedir(xldir))
! {
! fprintf(stderr, _("%s: could not close directory \"%s\": %s\n"),
progname, XLOGDIR, strerror(errno));
exit(1);
}
/*
* Finally, convert to new xlog seg size, and advance by one to ensure we
*************** KillExistingXLOG(void)
*** 892,899 ****
exit(1);
}
! errno = 0;
! while ((xlde = readdir(xldir)) != NULL)
{
if (strlen(xlde->d_name) == 24 &&
strspn(xlde->d_name, "0123456789ABCDEF") == 24)
--- 893,899 ----
exit(1);
}
! while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
if (strlen(xlde->d_name) == 24 &&
strspn(xlde->d_name, "0123456789ABCDEF") == 24)
*************** KillExistingXLOG(void)
*** 906,930 ****
exit(1);
}
}
- errno = 0;
}
#ifdef WIN32
! /*
! * This fix is in mingw cvs (runtime/mingwex/dirent.c rev 1.4), but not in
! * released version
! */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
if (errno)
{
! fprintf(stderr, _("%s: could not read from directory \"%s\": %s\n"),
progname, XLOGDIR, strerror(errno));
exit(1);
}
- closedir(xldir);
}
--- 906,932 ----
exit(1);
}
}
}
#ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
if (errno)
{
! fprintf(stderr, _("%s: could not read directory \"%s\": %s\n"),
! progname, XLOGDIR, strerror(errno));
! exit(1);
! }
!
! if (!closedir(xldir))
! {
! fprintf(stderr, _("%s: could not close directory \"%s\": %s\n"),
progname, XLOGDIR, strerror(errno));
exit(1);
}
}
*************** KillExistingArchiveStatus(void)
*** 948,955 ****
exit(1);
}
! errno = 0;
! while ((xlde = readdir(xldir)) != NULL)
{
if (strspn(xlde->d_name, "0123456789ABCDEF") == 24 &&
(strcmp(xlde->d_name + 24, ".ready") == 0 ||
--- 950,956 ----
exit(1);
}
! while (errno = 0, (xlde = readdir(xldir)) != NULL)
{
if (strspn(xlde->d_name, "0123456789ABCDEF") == 24 &&
(strcmp(xlde->d_name + 24, ".ready") == 0 ||
*************** KillExistingArchiveStatus(void)
*** 963,987 ****
exit(1);
}
}
- errno = 0;
}
#ifdef WIN32
! /*
! * This fix is in mingw cvs (runtime/mingwex/dirent.c rev 1.4), but not in
! * released version
! */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
if (errno)
{
! fprintf(stderr, _("%s: could not read from directory \"%s\": %s\n"),
progname, ARCHSTATDIR, strerror(errno));
exit(1);
}
- closedir(xldir);
}
--- 964,990 ----
exit(1);
}
}
}
#ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
if (errno)
{
! fprintf(stderr, _("%s: could not read directory \"%s\": %s\n"),
! progname, ARCHSTATDIR, strerror(errno));
! exit(1);
! }
!
! if (!closedir(xldir))
! {
! fprintf(stderr, _("%s: could not close directory \"%s\": %s\n"),
progname, ARCHSTATDIR, strerror(errno));
exit(1);
}
}
diff --git a/src/common/pgfnames.c b/src/common/pgfnames.c
new file mode 100644
index 9a68163..4497272
*** a/src/common/pgfnames.c
--- b/src/common/pgfnames.c
*************** pgfnames(const char *path)
*** 50,57 ****
filenames = (char **) palloc(fnsize * sizeof(char *));
! errno = 0;
! while ((file = readdir(dir)) != NULL)
{
if (strcmp(file->d_name, ".") != 0 && strcmp(file->d_name, "..") != 0)
{
--- 50,56 ----
filenames = (char **) palloc(fnsize * sizeof(char *));
! while (errno = 0, (file = readdir(dir)) != NULL)
{
if (strcmp(file->d_name, ".") != 0 && strcmp(file->d_name, "..") != 0)
{
*************** pgfnames(const char *path)
*** 63,76 ****
}
filenames[numnames++] = pstrdup(file->d_name);
}
- errno = 0;
}
#ifdef WIN32
! /*
! * This fix is in mingw cvs (runtime/mingwex/dirent.c rev 1.4), but not in
! * released version
! */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
--- 62,71 ----
}
filenames[numnames++] = pstrdup(file->d_name);
}
}
#ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
*************** pgfnames(const char *path)
*** 87,93 ****
filenames[numnames] = NULL;
! closedir(dir);
return filenames;
}
--- 82,96 ----
filenames[numnames] = NULL;
! if (!closedir(dir))
! {
! #ifndef FRONTEND
! elog(WARNING, "could not close directory \"%s\": %m", path);
! #else
! fprintf(stderr, _("could not close directory \"%s\": %s\n"),
! path, strerror(errno));
! #endif
! }
return filenames;
}
diff --git a/src/port/dirent.c b/src/port/dirent.c
new file mode 100644
index f9d93ea..ed5661c
*** a/src/port/dirent.c
--- b/src/port/dirent.c
*************** readdir(DIR *d)
*** 111,119 ****
int
closedir(DIR *d)
{
if (d->handle != INVALID_HANDLE_VALUE)
! FindClose(d->handle);
free(d->dirname);
free(d);
! return 0;
}
--- 111,121 ----
int
closedir(DIR *d)
{
+ int ret = 0;
+
if (d->handle != INVALID_HANDLE_VALUE)
! ret = !FindClose(d->handle);
free(d->dirname);
free(d);
! return ret;
}
diff --git a/src/port/pgcheckdir.c b/src/port/pgcheckdir.c
new file mode 100644
index fc97f8c..f10a84c
*** a/src/port/pgcheckdir.c
--- b/src/port/pgcheckdir.c
*************** pg_check_dir(const char *dir)
*** 33,46 ****
struct dirent *file;
bool dot_found = false;
- errno = 0;
-
chkdir = opendir(dir);
-
if (chkdir == NULL)
return (errno == ENOENT) ? 0 : -1;
! while ((file = readdir(chkdir)) != NULL)
{
if (strcmp(".", file->d_name) == 0 ||
strcmp("..", file->d_name) == 0)
--- 33,43 ----
struct dirent *file;
bool dot_found = false;
chkdir = opendir(dir);
if (chkdir == NULL)
return (errno == ENOENT) ? 0 : -1;
! while (errno = 0, (file = readdir(chkdir)) != NULL)
{
if (strcmp(".", file->d_name) == 0 ||
strcmp("..", file->d_name) == 0)
*************** pg_check_dir(const char *dir)
*** 68,84 ****
}
#ifdef WIN32
! /*
! * This fix is in mingw cvs (runtime/mingwex/dirent.c rev 1.4), but not in
! * released version
! */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
! closedir(chkdir);
!
! if (errno != 0)
result = -1; /* some kind of I/O error? */
/* We report on dot-files if we _only_ find dot files */
--- 65,76 ----
}
#ifdef WIN32
! /* Bug in old Mingw dirent.c; fixed in mingw-runtime-3.2, 2003-10-10 */
if (GetLastError() == ERROR_NO_MORE_FILES)
errno = 0;
#endif
! if (errno || closedir(chkdir) == -1)
result = -1; /* some kind of I/O error? */
/* We report on dot-files if we _only_ find dot files */
On Wed, Mar 19, 2014 at 02:02:50PM -0400, Bruce Momjian wrote:
The attached patch is slightly updated. I will apply it to head and all
the back branches, including the stylistic change to pg_resetxlog (for
consistency) and remove the MinGW block in head.
Patch applied back through 8.4. I had the closedir() tests backwards
and that was fixed. I also went over all the readdir/closedir() calls
in all back branches to make sure they were properly handled.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers