7.4 shared memory error on 64-bit SPARC/Solaris 5.8

Your patch has been added to the PostgreSQL unapplied patches list at:

http://momjian.postgresql.org/cgi-bin/pgpatches

I will try to apply it within the next 48 hours.

---------------------------------------------------------------------------

Thomas Baden wrote:

Hi there. My install was failing the regression tests
due to a shared memory error.

I'm running Solaris 5.8, YASSP, and I am using the Sun
Forte 7 compiler suite generating 64-bit code.

I did some digging around, and it turns out that the
IpcMemoryKey is a 64-bit number. The source code uses
uint32. Changing this to a size_t fixes the problem
under Solaris. I don't know whether this would break
other systems. Anyway, here's the fix:

I don't have the GNU diff, so please forgive the
crudeness of the following.

in file postgresql-7.4/src/backend/port/sysv_shmem.c,
lines 42-48:
IpcMemoryKey UsedShmemSegID = 0;
void *UsedShmemSegAddr = NULL;

- static void *InternalIpcMemoryCreate(IpcMemoryKey
memKey, uint32 size);
+ static void *InternalIpcMemoryCreate(IpcMemoryKey
memKey, size_t size);
static void IpcMemoryDetach(int status, Datum
shmaddr);
static void IpcMemoryDelete(int status, Datum
shmId);
static PGShmemHeader
*PGSharedMemoryAttach(IpcMemoryKey key,

in file postgresql-7.4/src/backend/port/sysv_shmem.c,
lines 62-68:
* print out an error and abort.  Other types of
errors are not recoverable.
*/
static void *
- InternalIpcMemoryCreate(IpcMemoryKey memKey, uint32
size)
+ InternalIpcMemoryCreate(IpcMemoryKey memKey, size_t
size)
{
IpcMemoryId shmid;
void       *memAddress;

in file postgresql-7.4/src/backend/port/sysv_shmem.c,
lines 234-240:
* zero will be passed.
*/
PGShmemHeader *
- PGSharedMemoryCreate(uint32 size, bool makePrivate,
int port)
+ PGSharedMemoryCreate(size_t size, bool makePrivate,
int port)
{
IpcMemoryKey NextShmemSegID;
void       *memAddress;

in file postgresql-7.4/src/include/storage/pg_shmem.h,
lines 24-30:
#ifndef PG_SHMEM_H
#define PG_SHMEM_H

- typedef uint32 IpcMemoryKey;  /* shared memory key
passed to shmget(2) */
+ typedef size_t IpcMemoryKey;  /* shared memory key
passed to shmget(2) */

typedef struct PGShmemHeader /* standard header for
all Postgres shmem */
{

in file postgresql-7.4/src/include/storage/pg_shmem.h,
lines 41-47:
extern void *UsedShmemSegAddr;
#endif

- extern PGShmemHeader *PGSharedMemoryCreate(uint32
size, bool makePrivate,
+ extern PGShmemHeader *PGSharedMemoryCreate(size_t
size, bool makePrivate,
int port);
extern bool PGSharedMemoryIsInUse(unsigned long id1,
unsigned long id2);
extern void PGSharedMemoryDetach(void);

Cheers,
-Thomas

__________________________________
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

We have applied a variant of this to CVS. Thanks.

---------------------------------------------------------------------------

Thomas Baden wrote:

Hi there. My install was failing the regression tests
due to a shared memory error.

I'm running Solaris 5.8, YASSP, and I am using the Sun
Forte 7 compiler suite generating 64-bit code.

I did some digging around, and it turns out that the
IpcMemoryKey is a 64-bit number. The source code uses
uint32. Changing this to a size_t fixes the problem
under Solaris. I don't know whether this would break
other systems. Anyway, here's the fix:

I don't have the GNU diff, so please forgive the
crudeness of the following.

in file postgresql-7.4/src/backend/port/sysv_shmem.c,
lines 42-48:
IpcMemoryKey UsedShmemSegID = 0;
void *UsedShmemSegAddr = NULL;

- static void *InternalIpcMemoryCreate(IpcMemoryKey
memKey, uint32 size);
+ static void *InternalIpcMemoryCreate(IpcMemoryKey
memKey, size_t size);
static void IpcMemoryDetach(int status, Datum
shmaddr);
static void IpcMemoryDelete(int status, Datum
shmId);
static PGShmemHeader
*PGSharedMemoryAttach(IpcMemoryKey key,

in file postgresql-7.4/src/backend/port/sysv_shmem.c,
lines 62-68:
* print out an error and abort.  Other types of
errors are not recoverable.
*/
static void *
- InternalIpcMemoryCreate(IpcMemoryKey memKey, uint32
size)
+ InternalIpcMemoryCreate(IpcMemoryKey memKey, size_t
size)
{
IpcMemoryId shmid;
void       *memAddress;

in file postgresql-7.4/src/backend/port/sysv_shmem.c,
lines 234-240:
* zero will be passed.
*/
PGShmemHeader *
- PGSharedMemoryCreate(uint32 size, bool makePrivate,
int port)
+ PGSharedMemoryCreate(size_t size, bool makePrivate,
int port)
{
IpcMemoryKey NextShmemSegID;
void       *memAddress;

in file postgresql-7.4/src/include/storage/pg_shmem.h,
lines 24-30:
#ifndef PG_SHMEM_H
#define PG_SHMEM_H

- typedef uint32 IpcMemoryKey;  /* shared memory key
passed to shmget(2) */
+ typedef size_t IpcMemoryKey;  /* shared memory key
passed to shmget(2) */

typedef struct PGShmemHeader /* standard header for
all Postgres shmem */
{

in file postgresql-7.4/src/include/storage/pg_shmem.h,
lines 41-47:
extern void *UsedShmemSegAddr;
#endif

- extern PGShmemHeader *PGSharedMemoryCreate(uint32
size, bool makePrivate,
+ extern PGShmemHeader *PGSharedMemoryCreate(size_t
size, bool makePrivate,
int port);
extern bool PGSharedMemoryIsInUse(unsigned long id1,
unsigned long id2);
extern void PGSharedMemoryDetach(void);

Cheers,
-Thomas

__________________________________
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thomas Baden <lindysandiego@yahoo.com> writes:

Hi there. My install was failing the regression tests
due to a shared memory error.

After reviewing the proposed patch, I find it hard to believe that the
patch would have fixed any such problem --- even if key_t is 64 bits
on your system, the code should still have worked, because we'd never
have selected a key value wider than 32 bits anyway. What *exact*
misbehavior were you seeing?

regards, tom lane

On Mon, Dec 01, 2003 at 05:19:17PM -0500, Tom Lane wrote:

Thomas Baden <lindysandiego@yahoo.com> writes:

Hi there. My install was failing the regression tests
due to a shared memory error.

After reviewing the proposed patch, I find it hard to believe that the
patch would have fixed any such problem --- even if key_t is 64 bits
on your system, the code should still have worked, because we'd never
have selected a key value wider than 32 bits anyway. What *exact*
misbehavior were you seeing?

It's not the key (key_t) that is the problem, but the size, which
used to be int but got replaced by a size_t.

Kurt

Kurt Roeckx <Q@ping.be> writes:

On Mon, Dec 01, 2003 at 05:19:17PM -0500, Tom Lane wrote:

After reviewing the proposed patch, I find it hard to believe that the
patch would have fixed any such problem ---

It's not the key (key_t) that is the problem, but the size, which
used to be int but got replaced by a size_t.

I don't see a problem there either. We don't create shmem segments
larger than 2Gb (and if we wanted to do so, this patch certainly
isn't enough to get it done --- all the arithmetic for shmem sizing
is int).

regards, tom lane

On Mon, Dec 01, 2003 at 06:08:06PM -0500, Tom Lane wrote:

Kurt Roeckx <Q@ping.be> writes:

On Mon, Dec 01, 2003 at 05:19:17PM -0500, Tom Lane wrote:

After reviewing the proposed patch, I find it hard to believe that the
patch would have fixed any such problem ---

It's not the key (key_t) that is the problem, but the size, which
used to be int but got replaced by a size_t.

I don't see a problem there either. We don't create shmem segments
larger than 2Gb (and if we wanted to do so, this patch certainly
isn't enough to get it done --- all the arithmetic for shmem sizing
is int).

You're right that it shouldn't cause any problems because of the
promotion rules. But changing the size to an size_t wouldn't be
a bad thing.

He seems to have changed this too:
- typedef uint32 IpcMemoryKey;
+ typedef size_t IpcMemoryKey;

That really should be a key_t. Can't we use a key_t on systems
that have it? The oldest sources I could find for shmget() all
said key_t. And typedef key_t to an int for those that don't
have it?

Kurt

Kurt Roeckx <Q@ping.be> writes:

He seems to have changed this too:
- typedef uint32 IpcMemoryKey;
+ typedef size_t IpcMemoryKey;
That really should be a key_t.

Yeah, I made it key_t in CVS tip. I'm just wondering what this could
avoid other than possibly a compile warning.

regards, tom lane

--- Tom Lane <tgl@sss.pgh.pa.us> wrote:

Kurt Roeckx <Q@ping.be> writes:

He seems to have changed this too:
- typedef uint32 IpcMemoryKey;
+ typedef size_t IpcMemoryKey;
That really should be a key_t.

Yeah, I made it key_t in CVS tip. I'm just
wondering what this could
avoid other than possibly a compile warning.

regards, tom lane

Greetings. Sorry for the late reply. I just
re-extracted and compiled the 7.4 release, and
recreated the error.

$ ls -ld postgresql-7.4.tar.bz2
-rw------- 1 tbaden prd 9999669 Nov 18 12:22
postgresql-7.4.tar.bz2
$ md5 postgresql-7.4.tar.bz2
MD5(postgresql-7.4.tar.bz2)=
9db7432c431d1570b1f605727daf27bc

$ ./configure --with-python --with-openssl
--without-readline

Pertinent environment variables:

CC='cc -xtarget=ultra -xarch=v9 -D_XOPEN_SOURCE=500
-D__EXTENSIONS__'
CXX='CC -xtarget=ultra -xarch=v9 -D_XOPEN_SOURCE=500
-D__EXTENSIONS__'

After compilation, make check fails as follows:

make[3]: Leaving directory
`/export/home/tbaden/posttemp/postgresql-7.4/contrib/spi'
/bin/sh ./pg_regress --temp-install
--top-builddir=../../.. --schedule=./parallel_schedule
--multibyte=SQL_ASCII
============== creating temporary installation
==============
============== initializing database system
==============

pg_regress: initdb failed
Examine ./log/initdb.log for the reason.

I have attached initdb.log for your perusal.

I just went digging through the system include files,
and sure enough they do refer to key_t instead of
size_t. I don't recall why I used size_t, except that
it must have been referenced in either a different .h
file or man page.

Cheers,
-Thomas

__________________________________
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree

creating template1 database in
/export/home/tbaden/posttemp/postgresql-7.4/src/test/regress/./tmp_check/data/b+ase/1...
FATAL: could not create semaphores: No space left on device
DETAIL: Failed system call was semget(1, 17, 03600).
HINT: This error does *not* mean that you have run out of disk space.
It occurs when either the system limit for the maximum number of
semaphore sets (SEMMNI), or the system wide maximum number of semaphores
(SEMMNS), would be exceeded. You need to raise the respective kernel
parameter. Alternatively, reduce PostgreSQL's consumption of semaphores by
reducing its max_connections parameter (currently 10).

semget() is about semaphores, not shared memory.

If that really was the error you got, your patch couldn't have
fixed it since it didn't change anything that has something to do
with it.

Did you read the documentation on how to raise those limits, and
raise them? (Not sure if it's needed for 2.8)

Kurt

Kurt Roeckx <Q@ping.be> writes:

If that really was the error you got, your patch couldn't have
fixed it since it didn't change anything that has something to do
with it.

I entirely concur with Kurt's observation.

My bet is you got the "no space left on device" failure because you were
already running a postmaster that was eating all the available
semaphores. When it worked, it was because you'd stopped the other
postmaster, and not because of any code changes.

regards, tom lane