desktop heap usage in windows

Heine Ferreira wrote:

Hi

In the latest release 8.4.3 is the desktop heap still a problem in
Windows?
I found some articles on the web that said on Windows xp the desktop
heap is rather small.
Apparantly under xp it's limited to 45 connections.

what does the desktop heap (a GUI thing used to manage window handles)
have to do with network connections? do you perhaps mean the kernel
paged and non-paged pools?

Windows desktop versions (pro, home, etc) are intentionally limited with
respect to how many network connections can be made as they are sold as
workstations and not servers. the biggest limit is on 'half open'
connections, which impacts how many connections can be initiated at the
same time.

On Mon, Apr 5, 2010 at 22:33, John R Pierce <pierce@hogranch.com> wrote:

Heine Ferreira wrote:

Hi

In the latest release 8.4.3 is the desktop heap still a problem in
Windows?
I found some articles on the web that said on Windows xp the desktop heap
is rather small.
Apparantly under xp it's limited to 45 connections.

what does the desktop heap (a GUI thing used to manage window handles) have
to do with network connections?   do you perhaps mean the kernel paged and
non-paged pools?
Windows desktop versions (pro, home, etc) are intentionally limited with
respect to how many network connections can be made as they are sold as
workstations and not servers.   the biggest limit is on 'half open'
connections, which impacts how many connections can be initiated at the same
time.

No, there's a problem with the Desktop Heap getting exhausted because
of system DLLs that allocate from it whenever they are loaded into a
process. It's in the Windows FAQ on wiki.postgresql.org - IIRC it's
right at the bottom.

And yes, we still have the problem - it's a Windows problem, really,
not a PostgreSQL problem. It simply can't deal with many processes the
way the system DLLs are built. The exact limit varies between
installations - likely due to third party DLLs being loaded, like AV
and FW software.

It may be that this is actually not a problem with the 64-bit version
of 9.0. I haven't tried that - probably should.

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

Magnus Hagander wrote:

No, there's a problem with the Desktop Heap getting exhausted because
of system DLLs that allocate from it whenever they are loaded into a
process. It's in the Windows FAQ on wiki.postgresql.org - IIRC it's
right at the bottom.

And yes, we still have the problem - it's a Windows problem, really,
not a PostgreSQL problem. It simply can't deal with many processes the
way the system DLLs are built. The exact limit varies between
installations - likely due to third party DLLs being loaded, like AV
and FW software.

It may be that this is actually not a problem with the 64-bit version
of 9.0. I haven't tried that - probably should.

k, so the FAQ entry here [1]http://wiki.postgresql.org/wiki/Running_%26_Installing_PostgreSQL_On_Native_Windows#I_cannot_run_with_more_than_about_125_connections_at_once.2C_despite_having_capable_hardware references the MSKB article [2]http://support.microsoft.com/kb/184802 which,
under Cause 2, talks about the SharedSection in the registry key [3]HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\SubSystems\Windows

Presumably, tis the 3rd value, the 512, thats used by non-interactive
sessions, thats the one that needs boosting?

fwiw, on my win7 pro 64bit desktop, that reads...

%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows
SharedSection=1024,20480,768
Windows=On SubSystemType=Windows ServerDll=basesrv,1
ServerDll=winsrv:UserServerDllInitialization,3
ServerDll=winsrv:ConServerDllInitialization,2 ServerDll=sxssrv,4
ProfileControl=Off
MaxRequestThreads=16

which is larger (768) than the value shown in the KB article [2]http://support.microsoft.com/kb/184802 (512)

[1]: http://wiki.postgresql.org/wiki/Running_%26_Installing_PostgreSQL_On_Native_Windows#I_cannot_run_with_more_than_about_125_connections_at_once.2C_despite_having_capable_hardware
http://wiki.postgresql.org/wiki/Running_%26_Installing_PostgreSQL_On_Native_Windows#I_cannot_run_with_more_than_about_125_connections_at_once.2C_despite_having_capable_hardware

[2]: http://support.microsoft.com/kb/184802

[3]: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\SubSystems\Windows
Manager\SubSystems\Windows

On Mon, Apr 5, 2010 at 23:45, John R Pierce <pierce@hogranch.com> wrote:

Magnus Hagander wrote:

No, there's a problem with the Desktop Heap getting exhausted because
of system DLLs that allocate from it whenever they are loaded into a
process. It's in the Windows FAQ on wiki.postgresql.org - IIRC it's
right at the bottom.

And yes, we still have the problem - it's a Windows problem, really,
not a PostgreSQL problem. It simply can't deal with many processes the
way the system DLLs are built. The exact limit varies between
installations - likely due to third party DLLs being loaded, like AV
and FW software.

It may be that this is actually not a problem with the 64-bit version
of 9.0. I haven't tried that - probably should.

k, so the FAQ entry here [1]  references the MSKB article [2] which, under
Cause 2, talks about the SharedSection in the registry key [3]

Presumably, tis the 3rd value, the 512, thats used by non-interactive
sessions, thats the one that needs boosting?

fwiw, on my win7 pro 64bit desktop, that reads...

  %SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows
SharedSection=1024,20480,768
      Windows=On SubSystemType=Windows ServerDll=basesrv,1
ServerDll=winsrv:UserServerDllInitialization,3
      ServerDll=winsrv:ConServerDllInitialization,2 ServerDll=sxssrv,4
ProfileControl=Off
      MaxRequestThreads=16

which is larger (768) than the value shown in the KB article [2] (512)

Hmm. I was hoping for orders-of-magniture larger though, that would
remove the problem completely. This seems to just push it off a little
bit more (assuming it makes a difference, that should probably be
tested and not just assumed)

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

Magnus Hagander wrote:

fwiw, on my win7 pro 64bit desktop, that reads...

%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows
SharedSection=1024,20480,768
Windows=On SubSystemType=Windows ServerDll=basesrv,1
ServerDll=winsrv:UserServerDllInitialization,3
ServerDll=winsrv:ConServerDllInitialization,2 ServerDll=sxssrv,4
ProfileControl=Off
MaxRequestThreads=16

which is larger (768) than the value shown in the KB article [2] (512)

Hmm. I was hoping for orders-of-magniture larger though, that would
remove the problem completely. This seems to just push it off a little
bit more (assuming it makes a difference, that should probably be
tested and not just assumed)

I realized I have access to a win7 Pro 32bit system too, the registery
setting is...

%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows
SharedSection=1024,12288,512 Windows=On SubSystemType=Windows
ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3
ServerDll=winsrv:ConServerDllInitialization,2 ServerDll=sxssrv,4
ProfileControl=Off MaxRequestThreads=16

on XP Pro 32bit, its

%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows
SharedSection=1024,3072,512 Windows=On SubSystemType=Windows
ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3
ServerDll=winsrv:ConServerDllInitialization,2 ProfileControl=Off
MaxRequestThreads=16

so the first number is 1MB (1024K) on all three systems.

the middle number goes from 3MB on XP32 to 12MB on Win7 32, to 20MB on
Win7 64bit,

the 3rd value goes from 512k, 512k, 768k

now, I do wonder. this IS the user32.dll heap. what library that a
service such as postgres invokes is using objects in the user32 (window
manager) heap?

Microsoft's explanation of these three values is...

The first SharedSection value (1024) is the shared heap size common
to all desktops.
This includes the global handle table, which holds handles to
windows, menus, icons,
cursors, and so forth, and shared system settings. It is unlikely
that you would ever
need to change this value.

The second SharedSection value (3072) is the size of the desktop
heap for each
desktop that is associated with the "interactive" window
station /WinSta0/. User objects
like hooks, menus, strings, and windows consume memory in this
desktop heap.
It is unlikely that you would ever need to change this second
SharedSection value.

The third SharedSection value (512) is the size of the desktop heap
for each
desktop that is associated with a "noninteractive" window station.
If this value
is not present, the size of the desktop heap for noninteractive
window stations
will be same as the size specified for interactive window stations
(the second
SharedSection value).

Every service process executed under a user account will receive a
new desktop
in a noninteractive window station created by the Service Control
Manager (SCM).
Thus, each service executed under a user account will consume the
number of
kilobytes of desktop heap specified in the third SharedSection
value. All services
executed under the LocalSystem account with *Allow Service to
Interact with
the Desktop*/ not/ selected share the desktop heap of the "Default"
desktop in
the noninteractive service windows station (Service-0x0-3e7$).

it goes on to say all 3 of these consume from a 48MB total heap (no idea
if this heap is bigger on 64 bit Windows 7). The 20MB number certainly
implies it must be, since I can create several desktops via remote
desktop connections with fast user switching enabled. that clause
about each service control process using a sepearate heap allocation is
a bit scary in conjunction with that 48MB total limit. this correlates
to the SVCHOST processes, windows runs all the services owned by a given
account in the same SVCHOST process, and uses a seperate SVCHOST for
each unique service account.

did you confirm that in fact its the 3rd number we care about for the
Postgres service? (certainly seems like it to me)

On Tue, Apr 6, 2010 at 09:42, John R Pierce <pierce@hogranch.com> wrote:

Magnus Hagander wrote:

fwiw, on my win7 pro 64bit desktop, that reads...

 %SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows
SharedSection=1024,20480,768
     Windows=On SubSystemType=Windows ServerDll=basesrv,1
ServerDll=winsrv:UserServerDllInitialization,3
     ServerDll=winsrv:ConServerDllInitialization,2 ServerDll=sxssrv,4
ProfileControl=Off
     MaxRequestThreads=16

which is larger (768) than the value shown in the KB article [2] (512)

Hmm. I was hoping for orders-of-magniture larger though, that would
remove the problem completely. This seems to just push it off a little
bit more (assuming it makes a difference, that should probably be
tested and not just assumed)

I realized I have access to a win7 Pro 32bit system too, the registery
setting is...

%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows
SharedSection=1024,12288,512 Windows=On SubSystemType=Windows
ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3
ServerDll=winsrv:ConServerDllInitialization,2 ServerDll=sxssrv,4
ProfileControl=Off MaxRequestThreads=16

on XP Pro 32bit, its

%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows
SharedSection=1024,3072,512 Windows=On SubSystemType=Windows
ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3
ServerDll=winsrv:ConServerDllInitialization,2 ProfileControl=Off
MaxRequestThreads=16

so the first number is 1MB (1024K) on all three systems.

the middle number goes from 3MB on XP32 to 12MB on Win7 32, to 20MB on Win7
64bit,

the 3rd value goes from 512k, 512k, 768k

now, I do wonder.  this IS the user32.dll heap.   what library that a
service such as postgres invokes is using objects in the user32 (window
manager) heap?

Microsoft's explanation of these three values is...

  The first SharedSection value (1024) is the shared heap size common to all
desktops.
  This includes the global handle table, which holds handles to windows,
menus, icons,
  cursors, and so forth, and shared system settings. It is unlikely that you
would ever
  need to change this value.

  The second SharedSection value (3072) is the size of the desktop heap for
each
  desktop that is associated with the "interactive" window station
/WinSta0/. User objects
  like hooks, menus, strings, and windows consume memory in this desktop
heap.
  It is unlikely that you would ever need to change this second
SharedSection value.

  The third SharedSection value (512) is the size of the desktop heap for
each
  desktop that is associated with a "noninteractive" window station. If this
value
  is not present, the size of the desktop heap for noninteractive window
stations
  will be same as the size specified for interactive window stations (the
second
  SharedSection value).

  Every service process executed under a user account will receive a new
desktop
  in a noninteractive window station created by the Service Control Manager
(SCM).
  Thus, each service executed under a user account will consume the number
of
  kilobytes of desktop heap specified in the third SharedSection value. All
services
  executed under the LocalSystem account with *Allow Service to Interact
with
  the Desktop*/ not/ selected share the desktop heap of the "Default"
desktop in
  the noninteractive service windows station (Service-0x0-3e7$).

it goes on to say all 3 of these consume from a 48MB total heap (no idea if
this heap is bigger on 64 bit Windows 7).   The 20MB number certainly
implies it must be, since I can create several desktops via remote desktop
connections with fast user switching enabled.   that clause about each
service control process using a sepearate heap allocation is a bit scary in
conjunction with that 48MB total limit.  this correlates to the SVCHOST
processes, windows runs all the services owned by a given account in the
same SVCHOST process, and uses a seperate SVCHOST for each unique service
account.

did you confirm that in fact its the 3rd number we care about for the
Postgres service? (certainly seems like it to me)

Yes, it is.

(It even says so in the FAQ - do we need to make that even clearer?)

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/