8.3 .4 + Vista + MingW + initdb = ACCESS_DENIED

Charlie Savage wrote:

I'm trying to upgrade my copy of postgresql from 8.2.x to 8.3.4 on a
Windows Vista SP1 laptop. I build postgres using mingw/msys and have had
no issues with 8.1.x and 8.2.x. However, with 8.3.4 I run into problems.

First, building fails:

c:/Development/MingW/bin/../lib/gcc/mingw32/3.4.5/../../../../include/sspi.h:60:

error: syntax error before "SECURITY_STRING"
In file included from
c:/Development/MingW/bin/../lib/gcc/mingw32/3.4.5/../../../../include/security.h:39,

from ../../../../src/include/libpq/libpq-be.h:50,

This also happens from libpq-int.h. The solution in both cases is to
add an additional header file:

#ifdef ENABLE_SSPI
#define SECURITY_WIN32
#include <ntsecapi.h> <------- Add this include
#include <security.h>
#undef SECURITY_WIN32

That fixes the build issue.

What the... This works fine on the mingw versions on the buildfarm :-(

Have you done anything special to your mingw install?

Second, once I've successfully built and installed postgres, I run into
a bigger problem. When using initdb, I get this error:

creating template1 database in c:/Data/postgres30/base/1 ... FATAL:
could not create shared memory segment: 5
DETAIL: Failed system call was CreateFileMapping(size=1802240,
name=Global\PostgreSQL:c:/Data/postgres).

A bit of googling and reading MSDN docs shows that applications that
don't run in Session 0 on Vista are not allowed to create shared memory
in the Global namespace. Since initdb is invoked from the command line,
it runs in Session 1.

Where did you find that information? I've been specifically looking for
it, but my searches didn't turn up anything conclusive.

The latest versions contain a fix for the global namespace code. Dave
noticed that this caused issues on vista and thus manually reverted the
patch in the official binary installer. But since we haven't (hadn't)
yet found documentation as to *why* it was failing, the patch has not
yet been reverted in the main source tree.

This is why it's working, probably, and it's not related to how it's built.

If you want to revert the patch in your local tree, this is the one:
http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/port/win32_shmem.c?r1=1.4&amp;r2=1.5

Specifically, you can reintroduce the old bug (that I think is hat made
it work on Vista) by removing the +18 in the lowest loop there.

//Magnus

Have you done anything special to your mingw install?

The only thing different is that I upgraded to the latest mingw and msys
packages a couple of weeks ago. The other thing I can think of is that
I installed openssl and related packages (the official ones from the
msys project). Would that force ENABLE_SSPI to be set or is it always
set (and is it set in the build farm)?

Second, once I've successfully built and installed postgres, I run into
a bigger problem. When using initdb, I get this error:

creating template1 database in c:/Data/postgres30/base/1 ... FATAL:
could not create shared memory segment: 5
DETAIL: Failed system call was CreateFileMapping(size=1802240,
name=Global\PostgreSQL:c:/Data/postgres).

A bit of googling and reading MSDN docs shows that applications that
don't run in Session 0 on Vista are not allowed to create shared memory
in the Global namespace. Since initdb is invoked from the command line,
it runs in Session 1.

Where did you find that information? I've been specifically looking for
it, but my searches didn't turn up anything conclusive.

Yeah, information is scattered and came mostly through reading MSDN
forum posts. It seems a number of people have been bitten by this
change to createFileMapping in Vista. My searches:

http://www.google.com/search?hl=en&amp;q=createfilemapping+access_denied+vista&amp;btnG=Search
http://www.google.com/search?hl=en&amp;q=createfilemapping+SeCreateGlobalPrivilege+&amp;btnG=Search

In particular, the session 0 information is documented at:

http://msdn.microsoft.com/en-us/library/aa366537(VS.85).aspx

Quoting:

The name can have a "Global\" or "Local\" prefix to explicitly create the

object in the global or session namespace...creating a file mapping object
in the global namespace from a session other than session zero requires
the SeCreateGlobalPrivilege privilege.

And more here:

http://msdn.microsoft.com/en-us/library/aa480152.aspx#appcomp_topic12

And this blog post might be helpful:

http://www.celceo.com/blogs/windows-insight/2007/09/global-createfilemapping-under.html

Towards the bottom it says:

Additionally, of course, we need to create everything with the appropriate security

attributes. Null Dacls don't grant global access in Vista, so we need
to create
a real security descriptior and add an ACE with GENERIC_ALL privileges
for the world SID.

So - clearly initdb needs the SeCreateGlobalPrivilege permission. But
I gave the user that runs initdb that permission using the Local
Security Policy Editor, so something else is going on. My first guess
was that the initdb code had to also request that permission also, but
that theory seems unlikely to be correct since the official win32
binaries seem to work fine (I'm assuming they are the exact same code??)

The latest versions contain a fix for the global namespace code. Dave
noticed that this caused issues on vista and thus manually reverted the
patch in the official binary installer. But since we haven't (hadn't)
yet found documentation as to *why* it was failing, the patch has not
yet been reverted in the main source tree.

This is why it's working, probably, and it's not related to how it's built.

Ah - so the trunk code is different than the binary release code? That
would explain it. And in that case, then maybe the solution is setting
up the security descriptor for calling createFileMapping?

If you want to revert the patch in your local tree, this is the one:
http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/port/win32_shmem.c?r1=1.4&amp;r2=1.5

Specifically, you can reintroduce the old bug (that I think is hat made
it work on Vista) by removing the +18 in the lowest loop there.

Ok, I just reverted the whole patch (so all 4 changes), did a make clean
; make; make install. And no dice, I'm still having the same issue.

Thanks,

Charlie

--
Charlie Savage
http://cfis.savagexi.com

Magnus Hagander wrote:

Second, once I've successfully built and installed postgres, I run into
a bigger problem. When using initdb, I get this error:

creating template1 database in c:/Data/postgres30/base/1 ... FATAL:
could not create shared memory segment: 5
DETAIL: Failed system call was CreateFileMapping(size=1802240,
name=Global\PostgreSQL:c:/Data/postgres).

A bit of googling and reading MSDN docs shows that applications that
don't run in Session 0 on Vista are not allowed to create shared memory
in the Global namespace. Since initdb is invoked from the command line,
it runs in Session 1.

Where did you find that information? I've been specifically looking for
it, but my searches didn't turn up anything conclusive.

The latest versions contain a fix for the global namespace code. Dave
noticed that this caused issues on vista and thus manually reverted the
patch in the official binary installer. But since we haven't (hadn't)
yet found documentation as to *why* it was failing, the patch has not
yet been reverted in the main source tree.

This is why it's working, probably, and it's not related to how it's built.

If you want to revert the patch in your local tree, this is the one:
http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/port/win32_shmem.c?r1=1.4&amp;r2=1.5

Specifically, you can reintroduce the old bug (that I think is hat made
it work on Vista) by removing the +18 in the lowest loop there.

I am getting this error on XP Pro. The buildfarm members run happily
from the scheduler, but when run by hand from the command line they
fail. This is true of both MinGW and MSVC.

This is in *URGENT* need of a fix.

cheers

andrew

On Mon, Oct 13, 2008 at 10:38 PM, Andrew Dunstan <andrew@dunslane.net> wrote:

I am getting this error on XP Pro. The buildfarm members run happily from
the scheduler, but when run by hand from the command line they fail. This is
true of both MinGW and MSVC.

It's on my list to discuss it with Magnus when we meet in Italy
tomorrow. The simple fix is to back out the change that broke it,
which leaves us in our previous broken-but-less-severely state (which
is what we did for the binary packages).

This is in *URGENT* need of a fix.

I can't see us pushing a point release for this alone (I think I've
seen two reports of people running into the problem when building from
source themselves, and one was internal here in EDB), so the urgency
is simply to ensure it's fixed before we do produce our next release
imho.

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com

It's on my list to discuss it with Magnus when we meet in Italy
tomorrow. The simple fix is to back out the change that broke it,
which leaves us in our previous broken-but-less-severely state (which
is what we did for the binary packages).

If you mean reverting the patch that Mangus mentioned, I tried that, and
it did not solve the problem. See:

http://archives.postgresql.org/pgsql-hackers/2008-09/msg01517.php

So I'm guessing it is something else.

This is in *URGENT* need of a fix.

I can't see us pushing a point release for this alone (I think I've
seen two reports of people running into the problem when building from
source themselves, and one was internal here in EDB), so the urgency
is simply to ensure it's fixed before we do produce our next release
imho.

Well - building your own Postgres 8.3 on Windows using MingW appears
broken. Not sure how many people fall into that category, but its seems
like a fairly major issue.

Charlie

On Tue, Oct 14, 2008 at 9:43 AM, Charlie Savage <cfis@savagexi.com> wrote:

It's on my list to discuss it with Magnus when we meet in Italy
tomorrow. The simple fix is to back out the change that broke it,
which leaves us in our previous broken-but-less-severely state (which
is what we did for the binary packages).

If you mean reverting the patch that Mangus mentioned, I tried that, and it
did not solve the problem. See:

http://archives.postgresql.org/pgsql-hackers/2008-09/msg01517.php

So I'm guessing it is something else.

You must be seeing an unrelated issue then - it's definitely that
change which caused us to start using the global namespace again, for
which it seems additional privileges may be required, and reverting
that change has fixed the issue on every failure machine I've seen.

It's also worth noting that that was the *only* change made to the
binary packages, and despite thousands of downloads we've seen no bug
reports that could be attributed to this.

This is in *URGENT* need of a fix.

I can't see us pushing a point release for this alone (I think I've
seen two reports of people running into the problem when building from
source themselves, and one was internal here in EDB), so the urgency
is simply to ensure it's fixed before we do produce our next release
imho.

Well - building your own Postgres 8.3 on Windows using MingW appears broken.
Not sure how many people fall into that category, but its seems like a
fairly major issue.

Very few people build their own Postgres on Windows, because it's not
exactly straightforward to do because of all the dependencies.

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com

On Tue, Oct 14, 2008 at 12:43 PM, Charlie Savage <cfis@savagexi.com> wrote:

It's on my list to discuss it with Magnus when we meet in Italy

tomorrow. The simple fix is to back out the change that broke it,
which leaves us in our previous broken-but-less-severely state (which
is what we did for the binary packages).

If you mean reverting the patch that Mangus mentioned, I tried that, and it
did not solve the problem. See:

http://archives.postgresql.org/pgsql-hackers/2008-09/msg01517.php

So I'm guessing it is something else.

I confirm that fix (actually a rollback of 4 changes) solves "access_denied"
issue in Vista+MinGW+latest snapshot of 8.4

As Microsoft says, only services are allowed (as they are the only processes
run in session0) to created objects in "Global\" namespace.

The only workaround to create globally accessible objects I saw was to
create shared memory locally, specify the correct non-null DACL (say, allow
access for owner) and access that created shared memory using
"\Session\[SID]\shared_memory_name" name. The only problem is to figure out
the session identifier where shared memory was created.
I do not see an elegant solution here. It looks quite easy to store the SID
into some file in working directory, then read it and connect to the shared
memory.

If that makes sense, I could implement it that way and test it under Vista.

Sincerely yours,
Vladimir Sitnikov

Dave Page wrote:

Well - building your own Postgres 8.3 on Windows using MingW appears broken.
Not sure how many people fall into that category, but its seems like a
fairly major issue.

Very few people build their own Postgres on Windows, because it's not
exactly straightforward to do because of all the dependencies.

IMNSHO, it is not acceptable to leave CVS for any significant period in
a state where someone can't run "make check" by hand, and you certainly
can't assume that no-one will want to do so.

I can confirm that commenting out '+ 18' from the last loop of the
offending patch allows me to run and test on XP-Pro.

What do we actually need to create this object in the global namespace?

cheers

andrew

On Tue, Oct 14, 2008 at 2:33 PM, Andrew Dunstan <andrew@dunslane.net> wrote:

Dave Page wrote:

Well - building your own Postgres 8.3 on Windows using MingW appears
broken.
Not sure how many people fall into that category, but its seems like a
fairly major issue.

Very few people build their own Postgres on Windows, because it's not
exactly straightforward to do because of all the dependencies.

IMNSHO, it is not acceptable to leave CVS for any significant period in a
state where someone can't run "make check" by hand, and you certainly can't
assume that no-one will want to do so.

It's not the case that no one can run 'make check' - it's very much
dependent on the configuration of the user account. Much as it pains
me to say it, running as Administrator or a member of the
Administrators group should normally work as you will inherit the
required privileges.

I can confirm that commenting out '+ 18' from the last loop of the offending
patch allows me to run and test on XP-Pro.

What do we actually need to create this object in the global namespace?

The user running initdb (or the postmaster) needs
SeCreateGlobalPrivilege - which is something we cannot really start
telling people they must have. My view is that we revert the change
(well, replace it with something that looks less like a broken attempt
to use the global namespace) and leave it at that. iirc, the use of
the global namespace is there to ensure things work as they should
under a non-console terminal services session - which is pretty rare
and can usually be avoided.

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com

The user running initdb (or the postmaster) needs
SeCreateGlobalPrivilege - which is something we cannot really start
telling people they must have. My view is that we revert the change
(well, replace it with something that looks less like a broken attempt
to use the global namespace) and leave it at that. iirc, the use of
the global namespace is there to ensure things work as they should
under a non-console terminal services session - which is pretty rare
and can usually be avoided.

I'm not so sure that non-console terminal service sessions should be
categorized as "pretty rare".

I use them routinely.

...Robert

On Tue, Oct 14, 2008 at 3:31 PM, Robert Haas <robertmhaas@gmail.com> wrote:

The user running initdb (or the postmaster) needs
SeCreateGlobalPrivilege - which is something we cannot really start
telling people they must have. My view is that we revert the change
(well, replace it with something that looks less like a broken attempt
to use the global namespace) and leave it at that. iirc, the use of
the global namespace is there to ensure things work as they should
under a non-console terminal services session - which is pretty rare
and can usually be avoided.

I'm not so sure that non-console terminal service sessions should be
categorized as "pretty rare".

I use them routinely.

For installing and running Postgres? Note that we're not talking about
running clients apps here, but the server itself.

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com

"Robert Haas" <robertmhaas@gmail.com> writes:

The user running initdb (or the postmaster) needs
SeCreateGlobalPrivilege - which is something we cannot really start
telling people they must have. My view is that we revert the change
(well, replace it with something that looks less like a broken attempt
to use the global namespace) and leave it at that. iirc, the use of
the global namespace is there to ensure things work as they should
under a non-console terminal services session - which is pretty rare
and can usually be avoided.

I'm not so sure that non-console terminal service sessions should be
categorized as "pretty rare".

Would there be any value in trying a global name first and falling back
to non-global if that fails?

regards, tom lane

Dave Page wrote:

On Tue, Oct 14, 2008 at 2:33 PM, Andrew Dunstan <andrew@dunslane.net> wrote:

Dave Page wrote:

Well - building your own Postgres 8.3 on Windows using MingW appears
broken.
Not sure how many people fall into that category, but its seems like a
fairly major issue.

Very few people build their own Postgres on Windows, because it's not
exactly straightforward to do because of all the dependencies.

IMNSHO, it is not acceptable to leave CVS for any significant period in a
state where someone can't run "make check" by hand, and you certainly can't
assume that no-one will want to do so.

It's not the case that no one can run 'make check' - it's very much
dependent on the configuration of the user account. Much as it pains
me to say it, running as Administrator or a member of the
Administrators group should normally work as you will inherit the
required privileges.

I can confirm that commenting out '+ 18' from the last loop of the offending
patch allows me to run and test on XP-Pro.

What do we actually need to create this object in the global namespace?

The user running initdb (or the postmaster) needs
SeCreateGlobalPrivilege - which is something we cannot really start
telling people they must have. My view is that we revert the change
(well, replace it with something that looks less like a broken attempt
to use the global namespace) and leave it at that. iirc, the use of
the global namespace is there to ensure things work as they should
under a non-console terminal services session - which is pretty rare
and can usually be avoided.

Not quite. The reason it's in the global namespace is to provide an
interlock preventing the starting of a postmaster in two different
sessions at the same time against the same data directory. We need to
figure out exactly how much this interlock is reduced, and if there is
something else we can do to make it work on Vista+...

//Magnus

Tom Lane wrote:

"Robert Haas" <robertmhaas@gmail.com> writes:

The user running initdb (or the postmaster) needs
SeCreateGlobalPrivilege - which is something we cannot really start
telling people they must have. My view is that we revert the change
(well, replace it with something that looks less like a broken attempt
to use the global namespace) and leave it at that. iirc, the use of
the global namespace is there to ensure things work as they should
under a non-console terminal services session - which is pretty rare
and can usually be avoided.

I'm not so sure that non-console terminal service sessions should be
categorized as "pretty rare".

Would there be any value in trying a global name first and falling back
to non-global if that fails?

Hmm. We could fail on the specific error that we get in this case,
perhaps. I think it should be "a required privilege is not held by the
client", which shouldn't occur otherwise. If it's just "access denied",
that could equally well be caused by a running postmaster in a different
session under a different useraccount...

//Magnus

On Tue, Oct 14, 2008 at 4:19 PM, Magnus Hagander <magnus@hagander.net> wrote:

Not quite. The reason it's in the global namespace is to provide an
interlock preventing the starting of a postmaster in two different
sessions at the same time against the same data directory. We need to
figure out exactly how much this interlock is reduced, and if there is
something else we can do to make it work on Vista+...

This isn't a Vista+ issue - it happens on XP as well.

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com

I'm not so sure that non-console terminal service sessions should be
categorized as "pretty rare".

I use them routinely.

For installing and running Postgres? Note that we're not talking about
running clients apps here, but the server itself.

Sure, why not? I mean, I've come across applications that simply fail
to install - or run - from a non-console session, but it's pretty
broken. You'd at least like to get an error message that says, hey,
you have to run this from a console session, rather than just crapping
out for no obvious reason.

...Robert

On Tue, Oct 14, 2008 at 5:12 PM, Robert Haas <robertmhaas@gmail.com> wrote:

I'm not so sure that non-console terminal service sessions should be
categorized as "pretty rare".

I use them routinely.

For installing and running Postgres? Note that we're not talking about
running clients apps here, but the server itself.

Sure, why not? I mean, I've come across applications that simply fail
to install - or run - from a non-console session, but it's pretty
broken. You'd at least like to get an error message that says, hey,
you have to run this from a console session, rather than just crapping
out for no obvious reason.

You do get such a message from the installer.

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com

The user running initdb (or the postmaster) needs
SeCreateGlobalPrivilege - which is something we cannot really start

Why not ? Doesn't the pg installer already tweak the permissions of the
installation user. On XP you can connect to session 0, so that is an alternative on XP.

telling people they must have. My view is that we revert the change
(well, replace it with something that looks less like a broken attempt
to use the global namespace) and leave it at that. iirc, the use of

yea, removing the +18 is grotesque, since it moves the shmem into the private
session but gives it a very misleading name (Global\ --> Global/).

the global namespace is there to ensure things work as they should
under a non-console terminal services session - which is pretty rare
and can usually be avoided.

Hm? non-console is the default for remote, and all you can get in Vista/2008 and up.

The reason it should be in the Global namespace is that shmem is one part of
detecting an existing postmaster. Especially in situations where the db is started
by hand, the protection against duplicate startup is important.

Andreas

On Wed, Oct 15, 2008 at 10:49 AM, Zeugswetter Andreas OSB sIT
<Andreas.Zeugswetter@s-itsolutions.at> wrote:

The user running initdb (or the postmaster) needs
SeCreateGlobalPrivilege - which is something we cannot really start

Why not ? Doesn't the pg installer already tweak the permissions of the
installation user. On XP you can connect to session 0, so that is an alternative on XP.

It does, but that's not the user running initdb.

the global namespace is there to ensure things work as they should
under a non-console terminal services session - which is pretty rare
and can usually be avoided.

Hm? non-console is the default for remote, and all you can get in Vista/2008 and up.

I didn't realise you couldn't get console on Vista/2K8. Guess I've
spent too much time on Macs in the last couple of years :-)

The reason it should be in the Global namespace is that shmem is one part of
detecting an existing postmaster. Especially in situations where the db is started
by hand, the protection against duplicate startup is important.

Yeah, as Magnus reminded me.

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com