shared memory settings: SHMMAX and SHMALL

I've spent a whole day trying to figure this out.
Every FAQ I read, between Linux, Postgres, and Oracle,
just sends me further into confusion, so I ask:

If I have 512MB of memory in my system, excluding swap
space,
what values do I want to set for SHMMAX and SHMALL?

That depeneds on your kernel implemetaion and hardware. I found
followings in /usr/include/asm/shmparam.h on my Linux box.

#define _SHM_ID_BITS 7
#define SHM_ID_MASK ((1<<_SHM_ID_BITS)-1)

#define SHM_IDX_SHIFT (_SHM_ID_BITS)
#define _SHM_IDX_BITS 15
#define SHM_IDX_MASK ((1<<_SHM_IDX_BITS)-1)

/*
* _SHM_ID_BITS + _SHM_IDX_BITS must be <= 24 on the i386 and
* SHMMAX <= (PAGE_SIZE << _SHM_IDX_BITS).
*/

here PAGE_SIZE == 4096. So I guess we cannot have more than 128MB for
the shared memory on this platform if above comment is correct.
--
Tatsuo Ishii

Tatsuo Ishii <t-ishii@sra.co.jp> writes:

If I have 512MB of memory in my system, excluding swap
space,
what values do I want to set for SHMMAX and SHMALL?

That depeneds on your kernel implemetaion and hardware. I found
followings in /usr/include/asm/shmparam.h on my Linux box.

here PAGE_SIZE == 4096. So I guess we cannot have more than 128MB for
the shared memory on this platform if above comment is correct.

You can recompile the kernel with the new defaults or just change them on
a live system (kernel 2.2.x or higher) with something like:

echo 419430400 >/proc/sys/kernel/shmall
echo 419430400 >/proc/sys/kernel/shmmax

You pick the numbers, of course.
--
matthew rice <matt@starnix.com> starnix inc.
tollfree: 1-87-pro-linux thornhill, ontario, canada
http://www.starnix.com professional linux services & products

Matthew Rice wrote:

Tatsuo Ishii <t-ishii@sra.co.jp> writes:

If I have 512MB of memory in my system, excluding swap
space,
what values do I want to set for SHMMAX and SHMALL?

That depeneds on your kernel implemetaion and hardware. I found
followings in /usr/include/asm/shmparam.h on my Linux box.

here PAGE_SIZE == 4096. So I guess we cannot have more than 128MB for
the shared memory on this platform if above comment is correct.

You can recompile the kernel with the new defaults or just change them on
a live system (kernel 2.2.x or higher) with something like:

echo 419430400 >/proc/sys/kernel/shmall
echo 419430400 >/proc/sys/kernel/shmmax

You pick the numbers, of course.

Two questions jump to my mind.

1) You've allocated 419430400 bytes (400MB) to shared memory. If I am
planning on running, e.g., 10 postmasters, wouldn't I want to set SHMMAX to
40MB (400MB divided by 10), since SHMMAX is the max shared memory allotment
per running application?

2) I've read that SHMALL = (SHMMAX/PAGE_SIZE*SHMMNI). I haven't been able
to determine SHMMNI's value on my system. I am inferring that PAGE_SIZE = 1
and SHMMNI = 4096 based simply on other users' comments and my
/usr/include/asm/shmparam.h (reprinted below) which, frankly, I am not very
good at reading. Anyway, if this is the case, isn't it likely that
SHMALL is less than SHMMAX?

/*
* Format of a swap-entry for shared memory pages currently out in
* swap space (see also mm/swap.c).
*
* SWP_TYPE = SHM_SWP_TYPE
* SWP_OFFSET is used as follows:
*
* bits 0..6 : id of shared memory segment page belongs to (SHM_ID)
* bits 7..21: index of page within shared memory segment (SHM_IDX)
* (actually fewer bits get used since SHMMAX is so low)
*/

/*
* Keep _SHM_ID_BITS as low as possible since SHMMNI depends on it and
* there is a static array of size SHMMNI.
*/
#define _SHM_ID_BITS 9
#define SHM_ID_MASK ((1<<_SHM_ID_BITS)-1)

#define SHM_IDX_SHIFT (_SHM_ID_BITS)
#define _SHM_IDX_BITS 15
#define SHM_IDX_MASK ((1<<_SHM_IDX_BITS)-1)

/*
* _SHM_ID_BITS + _SHM_IDX_BITS must be <= 24 on the i386 and
* SHMMAX <= (PAGE_SIZE << _SHM_IDX_BITS).
*/

#define SHMMAX 0x2000000 /* max shared seg size (bytes) */
/* Try not to change the default shipped SHMMAX - people rely on it */

#define SHMMIN 1 /* really PAGE_SIZE */ /* min shared seg size (bytes) */
#define SHMMNI (1<<_SHM_ID_BITS) /* max num of segs system wide */
#define SHMALL /* max shm system wide (pages) */ \
(1<<(_SHM_IDX_BITS+_SHM_ID_BITS))
#define SHMLBA PAGE_SIZE /* attach addr a multiple of this */
#define SHMSEG SHMMNI /* max shared segs per process */

#endif /* _ASMI386_SHMPARAM_H */

My last reply was made when I was exhausted. Sorry.

Reconsidering....

According to my /usr/include/asm/shmparam.h,
SHMMAX (should be) <= (PAGE_SIZE << _SHM_IDX_BITS) = (4096 bytes << 15)
= 128MB.

I still don't see where to find the PAGE_SIZE value, although 4096 seems
to be a general default out there.

Also according to my server's /usr/include/asm/shmparam.h,
SHMALL should be = (1 << (_SHM_IDX_BITS + _SHM_ID_BITS)) = (1 <<
(9+15)) = 16.

I wonder why some people are setting SHMALL and SHMMAX to the same value
when one is a measurement in pages and the other a measurement in
bytes. Also, I thought that SHMMAX should be the max allotment in bytes
per application, rather than the max shared by all applications. If I
anticipate 10 postmasters, is it dangerous to set SHMMAX so high?

Tatsuo Ishii wrote:

Gregory Bittar <gbittar@iqa.cc> writes:

According to my /usr/include/asm/shmparam.h,
SHMMAX (should be) <= (PAGE_SIZE << _SHM_IDX_BITS) = (4096 bytes << 15)
= 128MB.

Mine is just:

#define SHMMAX 0x2000000

I still don't see where to find the PAGE_SIZE value, although 4096 seems
to be a general default out there.

man getpagesize

or

matt@sol:/usr/include[106]% egrep 'define PAGE_S(IZE|HIFT)' */* 2>/dev/null
asm/page.h:#define PAGE_SHIFT 12
asm/page.h:#define PAGE_SIZE (1UL << PAGE_SHIFT)
linux/a.out.h:#define PAGE_SIZE 0x400
sys/user.h:#define PAGE_SHIFT 12
sys/user.h:#define PAGE_SIZE (1UL << PAGE_SHIFT)

Although, 'man postmaster' says that it used 8k (or "whatever BLCKSZ
is set to in config.h") for it's shared memory buffers.

Also according to my server's /usr/include/asm/shmparam.h,
SHMALL should be = (1 << (_SHM_IDX_BITS + _SHM_ID_BITS)) = (1 <<
(9+15)) = 16.

My 2.2.16 kernel has 15 and 7, BTW.

I wonder why some people are setting SHMALL and SHMMAX to the same value
when one is a measurement in pages and the other a measurement in

Pure laziness, I'm sure.

bytes. Also, I thought that SHMMAX should be the max allotment in bytes
per application, rather than the max shared by all applications. If I
anticipate 10 postmasters, is it dangerous to set SHMMAX so high?

I'm not certain. I just used those numbers for a testing server (I actually
only told postmaster to use ~300MB).

Are you saying 10 postmasters or 10 backends?

Either way, you can set SHMMAX as high as you like. Just don't tell the
postmasters to try and use them all.

Actually, I just did this:

echo 11111111111 >/proc/sys/kernel/shmmax

and got:

cat /proc/sys/kernel/shmmax
-1773790777

Probably not good, long term :)

My kernel also wouldn't let me make SHMALL higher than 16M
--
matthew rice <matt@starnix.com> starnix inc.
tollfree: 1-87-pro-linux thornhill, ontario, canada
http://www.starnix.com professional linux services & products

My last reply was made when I was exhausted. Sorry.

Reconsidering....

According to my /usr/include/asm/shmparam.h,
SHMMAX (should be) <= (PAGE_SIZE << _SHM_IDX_BITS) = (4096 bytes << 15)
= 128MB.

I still don't see where to find the PAGE_SIZE value, although 4096 seems
to be a general default out there.

On my Linux box, I found:

#define PAGE_SHIFT 12
#define PAGE_SIZE (1UL << PAGE_SHIFT)

so PAGE_SIZE must be 4096.

Also according to my server's /usr/include/asm/shmparam.h,
SHMALL should be = (1 << (_SHM_IDX_BITS + _SHM_ID_BITS)) = (1 <<
(9+15)) = 16.

I wonder why some people are setting SHMALL and SHMMAX to the same value
when one is a measurement in pages and the other a measurement in
bytes.

Me too. Probably SHMALL in /proc/sys/kernel/shmall is treated
differently from the one defined in those header files?

Also, I thought that SHMMAX should be the max allotment in bytes
per application, rather than the max shared by all applications. If I
anticipate 10 postmasters, is it dangerous to set SHMMAX so high?

No, SHMMAX is for all applications (10 postmasters in your case?). I
guess you are using x86/Linux and from your mail I would say you could
set SHMMAX to 128MB(= 134217728 bytes).

Also please make sure that you have enough file table entries in your
kernel. You are going to run many postmasters which involves many
backends. In my case I set following in /etc/sysctl.conf.

fs.file-max = 16384

This works for recent red hat (or derived) distributions. If this
doesn't work for you, then you could set a variable in
/proc(file-max).
--
Tatsuo Ishii