IPC Memory problem with Postmaster on BSDi 4.x

Started by Roberth Anderssonover 26 years ago23 messages
#1Roberth Andersson
roberth@jump-gate.com

I have PostgreSQL 6.4.2 installed since before and it was pretty easy to
compile it except for a few minor things. But but today I planned to
upgrade to version 6.5.1. So I made a pg_dumpall and backed up the rest and
deleted my /usr/local/pgsql to install the new version there.

The compilation and all worked great, it recognised the system as BSDi 4
which is correct and no errors. After this I logged in as postgres user and
wrote "initdb", worked great, BUT when I wrote "postmaster -i" I got a big
problem which I have never got ever before and I don't know why. Here is
the error message from postmaster ->

IpcMemoryCreate: shmget failed (Invalid argument) key=5432001,
size=1063936, permission=600
FATAL 1: ShmemCreate: cannot create region

To me it sounds like it is using some kind of wrong argument with shmget,
but I don't know how to fix this. I hope someone can help me.

This is what I get when I write ipcs ->
Message Queues:
T ID KEY MODE OWNER GROUP

Shared Memory:
T ID KEY MODE OWNER GROUP
m 196608 5432210 --rwa------ postgres user
m 196609 5432201 --rw------- postgres user
m 983042 5432207 --rw------- postgres user
m 1376259 5432010 --rwa------ postgres postgres
m 131076 5432001 --rw------- postgres user
m 786437 5432007 --rw------- postgres postgres

Semaphores:
T ID KEY MODE OWNER GROUP

If it's anything I need to do with IPC or PostgreSQL, please let me know
what and how, or maybe this is a bug ? I hope not, I love this database
engine and really wanna start using 6.5.1 asap.

Sincerely
Roberth Andersson

Roberth Andersson, Server Administrator @ Jump-Gate & Webworqs
Phone: 011-46-550-17864
Cellphone: 011-46-70-6422024
EMail: roberth@jump-gate.com / roberth@webworqs.com

#2Bruce Momjian
maillist@candle.pha.pa.us
In reply to: Roberth Andersson (#1)
Re: [HACKERS] IPC Memory problem with Postmaster on BSDi 4.x

IpcMemoryCreate: shmget failed (Invalid argument) key=5432001,
size=1063936, permission=600
FATAL 1: ShmemCreate: cannot create region

To me it sounds like it is using some kind of wrong argument with shmget,
but I don't know how to fix this. I hope someone can help me.

This is what I get when I write ipcs ->
Message Queues:
T ID KEY MODE OWNER GROUP

Shared Memory:
T ID KEY MODE OWNER GROUP
m 196608 5432210 --rwa------ postgres user
m 196609 5432201 --rw------- postgres user
m 983042 5432207 --rw------- postgres user
m 1376259 5432010 --rwa------ postgres postgres
m 131076 5432001 --rw------- postgres user
m 786437 5432007 --rw------- postgres postgres

Semaphores:
T ID KEY MODE OWNER GROUP

If it's anything I need to do with IPC or PostgreSQL, please let me know
what and how, or maybe this is a bug ? I hope not, I love this database
engine and really wanna start using 6.5.1 asap.

I am running BSDI here. Try using pgsql/bin/ipcclean to remove the
current shared memory stuff. Seems the old version did not clean up its
shared memory.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
#3Tom Lane
tgl@sss.pgh.pa.us
In reply to: Bruce Momjian (#2)
Re: [HACKERS] IPC Memory problem with Postmaster on BSDi 4.x

Roberth Andersson <roberth@jump-gate.com> writes:

BUT when I wrote "postmaster -i" I got a big
problem which I have never got ever before and I don't know why. Here is
the error message from postmaster ->

IpcMemoryCreate: shmget failed (Invalid argument) key=5432001,
size=1063936, permission=600
FATAL 1: ShmemCreate: cannot create region

The kernel error message ("Invalid argument", here) is often very
unhelpful when dealing with shared memory and semaphore operations :-(
I will bet that the real problem is that your kernel is configured
not to allow shared mem regions bigger than 1 megabyte --- but could
it say "Request too big", or some such? Nooo...

Postgres 6.5 defaults to wanting a shmem region just over a meg, whereas
6.4 was just under IIRC, so this problem will bite anyone who has the
fairly common kernel parameter setting SHMEMMAX = 1meg.

If that's the problem, you can either reconfigure your kernel with a
larger SHMEMMAX setting, or start Postgres with smaller-than-default
limits on number of buffers and backends. I'd try -N 16 for starters.

Another possibility is that you are running into a kernel limit on the
total amount of shared memory, not the size of this individual chunk.
You say:

This is what I get when I write ipcs ->
Message Queues:
T ID KEY MODE OWNER GROUP

Shared Memory:
T ID KEY MODE OWNER GROUP
m 196608 5432210 --rwa------ postgres user
m 196609 5432201 --rw------- postgres user
m 983042 5432207 --rw------- postgres user
m 1376259 5432010 --rwa------ postgres postgres
m 131076 5432001 --rw------- postgres user
m 786437 5432007 --rw------- postgres postgres

Semaphores:
T ID KEY MODE OWNER GROUP

If you do not have a postmaster running then those postgres-owned shared
memory segments should not be there; they must be left over from some
old run where the postmaster crashed without releasing 'em :-(. They
could be causing the kernel to decide it's given out too much shared
memory. Use ipcclean to get rid of them.

regards, tom lane

#4Roberth Andersson
roberth@jump-gate.com
In reply to: Bruce Momjian (#2)
Re: [HACKERS] IPC Memory problem with Postmaster on BSDi 4.x

At 00:11 1999-07-31 -0400, you wrote:

IpcMemoryCreate: shmget failed (Invalid argument) key=5432001,
size=1063936, permission=600
FATAL 1: ShmemCreate: cannot create region

To me it sounds like it is using some kind of wrong argument with shmget,
but I don't know how to fix this. I hope someone can help me.

This is what I get when I write ipcs ->
Message Queues:
T ID KEY MODE OWNER GROUP

Shared Memory:
T ID KEY MODE OWNER GROUP
m 196608 5432210 --rwa------ postgres user
m 196609 5432201 --rw------- postgres user
m 983042 5432207 --rw------- postgres user
m 1376259 5432010 --rwa------ postgres postgres
m 131076 5432001 --rw------- postgres user
m 786437 5432007 --rw------- postgres postgres

Semaphores:
T ID KEY MODE OWNER GROUP

If it's anything I need to do with IPC or PostgreSQL, please let me know
what and how, or maybe this is a bug ? I hope not, I love this database
engine and really wanna start using 6.5.1 asap.

I am running BSDI here. Try using pgsql/bin/ipcclean to remove the
current shared memory stuff. Seems the old version did not clean up its
shared memory.

Thanks Bruce

I tried to do that and it worked just fine the first time, and after that I
wrote "ipcs" to get a statistical if it really was cleaned or not, but I
got this ->

Message Queues:
T ID KEY MODE OWNER GROUP

Shared Memory:
T ID KEY MODE OWNER GROUP
m 1376259 0 --rwa------ postgres postgres
m 131076 0 --rw------- postgres user
m 786437 0 --rw------- postgres postgres

Semaphores:
T ID KEY MODE OWNER GROUP

Now I tried to use "ipcclean" once again, and I am always getting these
errors ->

ipcrm: shmid(1376259): : Invalid argument
ipcrm: shmid(131076): : Invalid argument
ipcrm: shmid(786437): : Invalid argument

I have no idea why, except maybe this could be something that is left over
since old and now the system doesn't know how to remove this Postgres stuff.

If anyone have any clue about what I can do, please let me know, I would
appreciate it a lot.

Sincerely

Roberth Andersson, Server Administrator @ Jump-Gate & Webworqs
Phone: 011-46-550-17864
Cellphone: 011-46-70-6422024
EMail: roberth@jump-gate.com / roberth@webworqs.com

#5Roberth Andersson
roberth@jump-gate.com
In reply to: Roberth Andersson (#4)
Re: [HACKERS] IPC Memory problem with Postmaster on BSDi 4.x

At 11:55 1999-07-31 -0400, you wrote:

Roberth Andersson <roberth@jump-gate.com> writes:

BUT when I wrote "postmaster -i" I got a big
problem which I have never got ever before and I don't know why. Here is
the error message from postmaster ->

IpcMemoryCreate: shmget failed (Invalid argument) key=5432001,
size=1063936, permission=600
FATAL 1: ShmemCreate: cannot create region

The kernel error message ("Invalid argument", here) is often very
unhelpful when dealing with shared memory and semaphore operations :-(
I will bet that the real problem is that your kernel is configured
not to allow shared mem regions bigger than 1 megabyte --- but could
it say "Request too big", or some such? Nooo...
Postgres 6.5 defaults to wanting a shmem region just over a meg, whereas
6.4 was just under IIRC, so this problem will bite anyone who has the
fairly common kernel parameter setting SHMEMMAX = 1meg.
If that's the problem, you can either reconfigure your kernel with a
larger SHMEMMAX setting, or start Postgres with smaller-than-default
limits on number of buffers and backends. I'd try -N 16 for starters.
Another possibility is that you are running into a kernel limit on the
total amount of shared memory, not the size of this individual chunk.
You say:

Thanks Tom

Is it possible to find how big it is right now without touching the kernel
source codes ? I tried to search also for SHMEMMAX in the source codes,
but found nothing.

I am going to try to start up Postgres later today with your suggested
parameter -N 16 and see whats happends.

This is what I get when I write ipcs ->
Message Queues:
T ID KEY MODE OWNER GROUP

Shared Memory:
T ID KEY MODE OWNER GROUP
m 196608 5432210 --rwa------ postgres user
m 196609 5432201 --rw------- postgres user
m 983042 5432207 --rw------- postgres user
m 1376259 5432010 --rwa------ postgres postgres
m 131076 5432001 --rw------- postgres user
m 786437 5432007 --rw------- postgres postgres

Semaphores:
T ID KEY MODE OWNER GROUP

If you do not have a postmaster running then those postgres-owned shared
memory segments should not be there; they must be left over from some
old run where the postmaster crashed without releasing 'em :-(. They
could be causing the kernel to decide it's given out too much shared
memory. Use ipcclean to get rid of them.

I tried to do that and it worked just fine the first time, and after that I
wrote "ipcs" to get a statistical if it really was cleaned or not, but I
got this ->

Message Queues:
T ID KEY MODE OWNER GROUP

Shared Memory:
T ID KEY MODE OWNER GROUP
m 1376259 0 --rwa------ postgres postgres
m 131076 0 --rw------- postgres user
m 786437 0 --rw------- postgres postgres

Semaphores:
T ID KEY MODE OWNER GROUP

Now I tried to use "ipcclean" once again, and I am always getting these
errors ->

ipcrm: shmid(1376259): : Invalid argument
ipcrm: shmid(131076): : Invalid argument
ipcrm: shmid(786437): : Invalid argument

I have no idea why, except maybe this could be something that is left over
since old and now the system doesn't know how to remove this Postgres stuff.

If anyone have any clue about what I can do, please let me know, I would
appreciate it a lot.

Sincerely

#6Bruce Momjian
maillist@candle.pha.pa.us
In reply to: Roberth Andersson (#4)
Re: [HACKERS] IPC Memory problem with Postmaster on BSDi 4.x

I am running BSDI here. Try using pgsql/bin/ipcclean to remove the
current shared memory stuff. Seems the old version did not clean up its
shared memory.

Thanks Bruce

I tried to do that and it worked just fine the first time, and after that I
wrote "ipcs" to get a statistical if it really was cleaned or not, but I
got this ->

Message Queues:
T ID KEY MODE OWNER GROUP

Shared Memory:
T ID KEY MODE OWNER GROUP
m 1376259 0 --rwa------ postgres postgres
m 131076 0 --rw------- postgres user
m 786437 0 --rw------- postgres postgres

Semaphores:
T ID KEY MODE OWNER GROUP

Now I tried to use "ipcclean" once again, and I am always getting these
errors ->

ipcrm: shmid(1376259): : Invalid argument
ipcrm: shmid(131076): : Invalid argument
ipcrm: shmid(786437): : Invalid argument

I have no idea why, except maybe this could be something that is left over
since old and now the system doesn't know how to remove this Postgres stuff.

If anyone have any clue about what I can do, please let me know, I would
appreciate it a lot.

Go to /tmp, and do a ls -la. There will be some file in there that are
left over that should be deleted. But now that I look, they aren't in
/tmp anymore in 4.0. Use ipcs and ipcrm to manually delete them.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
#7Mike Mascari
mascarim@yahoo.com
In reply to: Bruce Momjian (#6)
Re: [HACKERS] IPC Memory problem with Postmaster on BSDi 4.x

Just curious...why can't he just reboot?

Mike Mascari
(mascarim@yahoo.com)

--- Bruce Momjian <maillist@candle.pha.pa.us> wrote:

I am running BSDI here. Try using

pgsql/bin/ipcclean to remove the

current shared memory stuff. Seems the old

version did not clean up its

shared memory.

Thanks Bruce

I tried to do that and it worked just fine the

first time, and after that I

wrote "ipcs" to get a statistical if it really was

cleaned or not, but I

got this ->

Message Queues:
T ID KEY MODE OWNER

GROUP

Shared Memory:
T ID KEY MODE OWNER

GROUP

m 1376259 0 --rwa------ postgres

postgres

m 131076 0 --rw------- postgres

user

m 786437 0 --rw------- postgres

postgres

Semaphores:
T ID KEY MODE OWNER

GROUP

Now I tried to use "ipcclean" once again, and I am

always getting these

errors ->

ipcrm: shmid(1376259): : Invalid argument
ipcrm: shmid(131076): : Invalid argument
ipcrm: shmid(786437): : Invalid argument

I have no idea why, except maybe this could be

something that is left over

since old and now the system doesn't know how to

remove this Postgres stuff.

If anyone have any clue about what I can do,

please let me know, I would

appreciate it a lot.

Go to /tmp, and do a ls -la. There will be some
file in there that are
left over that should be deleted. But now that I
look, they aren't in
/tmp anymore in 4.0. Use ipcs and ipcrm to manually
delete them.

_____________________________________________________________
Do You Yahoo!?
Free instant messaging and more at http://messenger.yahoo.com

#8Bruce Momjian
maillist@candle.pha.pa.us
In reply to: Mike Mascari (#7)
Re: [HACKERS] IPC Memory problem with Postmaster on BSDi 4.x

Just curious...why can't he just reboot?

Mike Mascari
(mascarim@yahoo.com)

That would be the best. I assume he can't, for some reason.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
#9Roberth Andersson
roberth@jump-gate.com
In reply to: Bruce Momjian (#8)
Re: [HACKERS] IPC Memory problem with Postmaster on BSDi 4.x

At 11:17 1999-07-31 -0700, you wrote:

Just curious...why can't he just reboot?

Mike Mascari
(mascarim@yahoo.com)

Becuase this is a server with many customers/users, we rather let it stay
as long as possible online without any reboot, we only reboots if we REALLY
needs to do that like patching the BSDi kernel or something similar.

Sincerely
Roberth Andersson, Server Administrator @ Jump-Gate & Webworqs
Phone: 011-46-550-17864
Cellphone: 011-46-70-6422024
EMail: roberth@jump-gate.com / roberth@webworqs.com

#10J. Michael Roberts
mirobert@cs.indiana.edu
In reply to: Tom Lane (#3)
Re: [HACKERS] IPC Memory problem with Postmaster on BSDi 4.x

On Sat, 31 Jul 1999, Tom Lane wrote:

Roberth Andersson <roberth@jump-gate.com> writes:

IpcMemoryCreate: shmget failed (Invalid argument) key=5432001,
size=1063936, permission=600
FATAL 1: ShmemCreate: cannot create region

The kernel error message ("Invalid argument", here) is often very
unhelpful when dealing with shared memory and semaphore operations :-(

FWIW, I'm just installing v6.5.1 on Solaris 2.5 -- and lo! I had the
same problem, and sure enough, SHMEMMAX is 1meg, and -N 16 worked like a
charm! So Tom, looks like you're right. As always.

Since this was my first time compiling/installing pgsql, I've noticed a
couple of oopses (maybe mine) in the installation instructions... Who do
I talk to to update them? (Example: Instead of being able simply to type
"initdb" to get started, I had to specify a user with "initdb -u
postgres". That kind of stuff.)

Michael

#11Bruce Momjian
maillist@candle.pha.pa.us
In reply to: J. Michael Roberts (#10)
Re: [HACKERS] IPC Memory problem with Postmaster on BSDi 4.x

Since this was my first time compiling/installing pgsql, I've noticed a
couple of oopses (maybe mine) in the installation instructions... Who do
I talk to to update them? (Example: Instead of being able simply to type
"initdb" to get started, I had to specify a user with "initdb -u
postgres". That kind of stuff.)

Send it to hackers or focs. Either is good. The -u may be because of
your pg_hba.conf file.

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
#12Thomas Lockhart
lockhart@alumni.caltech.edu
In reply to: Bruce Momjian (#11)
Re: [HACKERS] IPC Memory problem with Postmaster on BSDi 4.x

Since this was my first time compiling/installing pgsql, I've noticed a
couple of oopses (maybe mine) in the installation instructions... Who do
I talk to to update them?

Send it to hackers or focs. Either is good. The -u may be because of
your pg_hba.conf file.

If you have trouble finding the "focs" list, try "docs" instead ;)

- Thomas

--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California

#13Bruce Momjian
maillist@candle.pha.pa.us
In reply to: Thomas Lockhart (#12)
Re: [HACKERS] IPC Memory problem with Postmaster on BSDi 4.x

Since this was my first time compiling/installing pgsql, I've noticed a
couple of oopses (maybe mine) in the installation instructions... Who do
I talk to to update them?

Send it to hackers or focs. Either is good. The -u may be because of
your pg_hba.conf file.

If you have trouble finding the "focs" list, try "docs" instead ;)

Shhh, don't tell everyone else about the focs list. :-)

-- 
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026
#14J. Michael Roberts
mirobert@cs.indiana.edu
In reply to: Thomas Lockhart (#12)
[HACKERS] Installation procedure.

You people are harsh.

BTW, I didn't modify the pg_hba.conf file at all since I want to allow
only local use, so ... um ... since I really haven't digested the usage of
the whole host configuration thing, I haven't the faintest clue whether
that would be the problem.

However, either way, I'd like to understand what happened and make sure
that the next hapless newbie doesn't spend as much time on it as I did.
Which, granted, wasn't that much time.

Here are a couple of surprises, then, that I encountered during this.

- initdb complained that it couldn't find a user. I gave it -u postgres.
- I needed to install flex (no surprise) -- the instructions are quite
explicit, but, well, wrong: flex depends on bison. So you have to get
and compile bison first. Also, the GNU FTP server has "redisorganized"
their file structure, so the very detailed FTP instructions for getting
flex are also outdated.
- Being only halfway a sysadmin, I was a little worried about making a
postgres "superuser". I just made a postgres user and didn't worry
about the super part, and it seems to work. Am I missing a point?
- The aforementioned shared memory problem was distressing. Thank God
somebody else had just encountered it. Is there any better way to
trap for that? Should the default number of backends be made something
less than 32 so that the "common setting" of 1 meg will be safe? Am I
being too wimpy?

That's pretty much it. Seems to be perking along happily now, but I
haven't run the regression tests yet.

On Sun, 1 Aug 1999, Thomas Lockhart wrote:

Show quoted text

Since this was my first time compiling/installing pgsql, I've noticed a
couple of oopses (maybe mine) in the installation instructions... Who do
I talk to to update them?

Send it to hackers or focs. Either is good. The -u may be because of
your pg_hba.conf file.

If you have trouble finding the "focs" list, try "docs" instead ;)

- Thomas

--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California

#15Thomas Lockhart
lockhart@alumni.caltech.edu
In reply to: J. Michael Roberts (#14)
Re: [HACKERS] Installation procedure.

You people are harsh.

Ooh, a compliment. We live for those... ;)

- initdb complained that it couldn't find a user. I gave it -u postgres.

This is the only report of this I can remember (others might remind me
otherwise, but...). The best I can tell you somehow didn't have a USER
environment variable or mucked around with accounts between building
software and trying to initdb. There are several messages from initdb
with similar wording but with different diagnostics so you would need
to send the actual text or look in the initdb source code yourself
(src/bin/initdb/initdb.sh).

- I needed to install flex (no surprise) -- the instructions are quite
explicit, but, well, wrong: flex depends on bison. So you have to get
and compile bison first. Also, the GNU FTP server has "redisorganized"
their file structure, so the very detailed FTP instructions for getting
flex are also outdated.

Thanks. Can you give a suggestion for a more helpful phrasing for
this, or a better choice of content?

- Being only halfway a sysadmin, I was a little worried about making a
postgres "superuser". I just made a postgres user and didn't worry
about the super part, and it seems to work. Am I missing a point?

Only sort of. The "postgres superuser" is a normal user as far as the
OS is concerned, but is a superuser as far as the Postgres
installation is concerned. Ya done good.

- The aforementioned shared memory problem was distressing. Thank God
somebody else had just encountered it. Is there any better way to
trap for that? Should the default number of backends be made something
less than 32 so that the "common setting" of 1 meg will be safe? Am I
being too wimpy?

This is the first release where the shared memory size was actually
being calculated correctly. The numbers used pretty much match the
theoretical (but incorrectly calculated) maximum limits in previous
releases, but the calculated number is bigger and a few OSes seem to
cough. Your OS is being wimpy imho, but the workaround is pretty easy.

Do you have a specific suggestion for a change here in the docs?
Probably no need to change the build procedure, but perhaps a warning
about possible startup problems?

Have fun with the new toy...

- Thomas

--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California

#16Tom Lane
tgl@sss.pgh.pa.us
In reply to: Bruce Momjian (#13)
Re: [HACKERS] IPC Memory problem with Postmaster on BSDi 4.x

"J. Michael Roberts" <mirobert@cs.indiana.edu> writes:

Since this was my first time compiling/installing pgsql, I've noticed a
couple of oopses (maybe mine) in the installation instructions... Who do
I talk to to update them? (Example: Instead of being able simply to type
"initdb" to get started, I had to specify a user with "initdb -u
postgres". That kind of stuff.)

FWIW, I think I know the cause of that one --- initdb, and also the
regression tests (and maybe other places?) look at the USER environment
variable by default to get the name of the postgres user. If you are
on a platform that doesn't ordinarily set USER, you lose. I've been
burnt by that myself.

I am not sure whether we ought to make the code look at LOGNAME as
a fallback if USER isn't set, or just document that you ought to set
USER. The first sounds good, but I wonder what the odds are of
picking up the wrong username. On my platform, for example, su'ing
to the postgres account does *not* change LOGNAME, which would mean
initdb would pick the wrong thing. Maybe what we need is just a
better error message ("USER environment variable is not set, please
set it or provide -u switch" ...)

regards, tom lane

#17J. Michael Roberts
mirobert@cs.indiana.edu
In reply to: Thomas Lockhart (#15)
Re: [HACKERS] Installation procedure.

You people are harsh.

Ooh, a compliment. We live for those... ;)

Heh.

- initdb complained that it couldn't find a user. I gave it -u postgres.

This is the only report of this I can remember (others might remind me
otherwise, but...). The best I can tell you somehow didn't have a USER
environment variable or mucked around with accounts between building
software and trying to initdb.

Aha. Come to think of it, I'm sure that I was actually root su'd to
postgres. That was probably the problem. I hadn't even thought of that.

- I needed to install flex (no surprise) -- the instructions are quite
explicit, but, well, wrong: flex depends on bison.

Thanks. Can you give a suggestion for a more helpful phrasing for
this, or a better choice of content?

Will do. Um, "soon." Actually, I was very heartened by the explicit
detail.

- Being only halfway a sysadmin, I was a little worried about making a
postgres "superuser". I just made a postgres user and didn't worry
about the super part, and it seems to work. Am I missing a point?

Only sort of. The "postgres superuser" is a normal user as far as the
OS is concerned, but is a superuser as far as the Postgres
installation is concerned. Ya done good.

IMHO that should probably be more explicit in INSTALL. I'll update it.

Your OS is being wimpy imho, but the workaround is pretty easy.

Do you have a specific suggestion for a change here in the docs?
Probably no need to change the build procedure, but perhaps a warning
about possible startup problems?

Yeah, that would be a good idea. Again, sounds like it's up to me to
revise INSTALL a bit.

Have fun with the new toy...

Toy, schmoy. This is going to back up several web sites. After the
recent glowing description on this mailing list of v6.5.1, I decided to
take the plunge and junk Illustra (the total lack of support and
documentation and tuning and licensing freedom and so on have finally
taken their toll on my confidence...) Wish me luck.

#18J. Michael Roberts
mirobert@cs.indiana.edu
In reply to: Tom Lane (#16)
Re: [HACKERS] IPC Memory problem with Postmaster on BSDi 4.x

If you are
on a platform that doesn't ordinarily set USER, you lose. I've been
burnt by that myself.

I am not sure whether we ought to make the code look at LOGNAME as
a fallback if USER isn't set, or just document that you ought to set
USER.

My take would be at least to change the installation documentation,
and improve the error message (you can never improve error message *too*
much). Getting clever about what username to use seems more dangerous
than helpful, and besides, it's simple enough simply to say "initdb needs
to figure out your user somehow" and leave it at that. How often do you
run initdb anyway?

The first sounds good, but I wonder what the odds are of

Show quoted text

picking up the wrong username. On my platform, for example, su'ing
to the postgres account does *not* change LOGNAME, which would mean
initdb would pick the wrong thing. Maybe what we need is just a
better error message ("USER environment variable is not set, please
set it or provide -u switch" ...)

#19Tom Lane
tgl@sss.pgh.pa.us
In reply to: J. Michael Roberts (#17)
Re: [HACKERS] Installation procedure.

Great stuff, Michael. I think by the time most of us got to the point
of contributing much to Postgres, we'd forgotten the little glitches
we hit on the first try. Cleaning up these issues is definitely
worthwhile, and I am glad to see you willing to help out.

Thomas already gave good responses about the technical issues,
but I have one point to add:

- The aforementioned shared memory problem was distressing. Thank God
somebody else had just encountered it. Is there any better way to
trap for that? Should the default number of backends be made something
less than 32 so that the "common setting" of 1 meg will be safe? Am I
being too wimpy?

This is the first release where the shared memory size was actually
being calculated correctly. The numbers used pretty much match the
theoretical (but incorrectly calculated) maximum limits in previous
releases, but the calculated number is bigger and a few OSes seem to
cough. Your OS is being wimpy imho, but the workaround is pretty easy.

Actually, when we set the default MAXBACKENDS to 32 for 6.5, it was
done specifically to ensure that the default shared mem block size would
stay under a meg. (The equivalent setting in 6.4 was 64 backends.)
But I guess various data structures changed a little bit after that
time, and we ended up on the wrong side of the breakpoint without
thinking about it.

Should we cut the default MAXBACKENDS some more, or just try to
document the issue better?

regards, tom lane

#20J. Michael Roberts
mirobert@cs.indiana.edu
In reply to: Tom Lane (#19)
Re: [HACKERS] Installation procedure.

Actually, when we set the default MAXBACKENDS to 32 for 6.5, it was
done specifically to ensure that the default shared mem block size would
stay under a meg. (The equivalent setting in 6.4 was 64 backends.)
But I guess various data structures changed a little bit after that
time, and we ended up on the wrong side of the breakpoint without
thinking about it.

Oops. :-)

Should we cut the default MAXBACKENDS some more, or just try to
document the issue better?

I've added a paragraph to the system requirements in INSTALL which
explains the situation. Now to figure out how to get the changes to you
guys.... Is the procedure simply to diff it and email it to somebody, or
what? Just because I've been lurking on this list for nearly a year now
doesn't mean I know what I'm doing.

As to whether MAXBACKENDS should be changed -- I have no idea what impact
that would actually have. What *is* a backend, precisely? In Illustra,
anyway, each active query starts a new process while it's working -- is
that a backend? I've never seen more than about 5 on my own server (not
that I hang around monitoring all the time) so 16 seems capacious.

#21Tom Lane
tgl@sss.pgh.pa.us
In reply to: J. Michael Roberts (#20)
Re: [HACKERS] Installation procedure.

"J. Michael Roberts" <mirobert@cs.indiana.edu> writes:

Now to figure out how to get the changes to you
guys.... Is the procedure simply to diff it and email it to somebody, or
what?

Standard operating procedure is to make a patch-compatible diff
(I think -c format is preferred) and post it to the pgsql-patches
mailing list. If you have a real good idea which core member is
probably going to apply the patch you could send it just to that
person, but it's more courteous to put it on the public mailing list.

As to whether MAXBACKENDS should be changed -- I have no idea what impact
that would actually have. What *is* a backend, precisely? In Illustra,
anyway, each active query starts a new process while it's working -- is
that a backend?

No. There's one backend process per client connection; it lives till
the client disconnects, and handles all queries that come through that
connection. So MAXBACKENDS really means "how many simultaneous
clients am I expecting"?

regards, tom lane

#22The Hermit Hacker
scrappy@hub.org
In reply to: Tom Lane (#19)
Re: [HACKERS] Installation procedure.

On Mon, 2 Aug 1999, Tom Lane wrote:

Great stuff, Michael. I think by the time most of us got to the point
of contributing much to Postgres, we'd forgotten the little glitches
we hit on the first try. Cleaning up these issues is definitely
worthwhile, and I am glad to see you willing to help out.

Thomas already gave good responses about the technical issues,
but I have one point to add:

- The aforementioned shared memory problem was distressing. Thank God
somebody else had just encountered it. Is there any better way to
trap for that? Should the default number of backends be made something
less than 32 so that the "common setting" of 1 meg will be safe? Am I
being too wimpy?

This is the first release where the shared memory size was actually
being calculated correctly. The numbers used pretty much match the
theoretical (but incorrectly calculated) maximum limits in previous
releases, but the calculated number is bigger and a few OSes seem to
cough. Your OS is being wimpy imho, but the workaround is pretty easy.

Actually, when we set the default MAXBACKENDS to 32 for 6.5, it was
done specifically to ensure that the default shared mem block size would
stay under a meg. (The equivalent setting in 6.4 was 64 backends.)
But I guess various data structures changed a little bit after that
time, and we ended up on the wrong side of the breakpoint without
thinking about it.

Should we cut the default MAXBACKENDS some more, or just try to
document the issue better?

My opinion...cut it back to 16 and document. Reason: those new won't hit
the problem, and those that have started to use it in "more production
environments" will have started to look into performance tuning, and most
likely have at least scan'd thruogh the postmaster man page (and will have
seen the -B option)...

Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

#23Thomas Lockhart
lockhart@alumni.caltech.edu
In reply to: J. Michael Roberts (#17)
Re: [HACKERS] Installation procedure.

IMHO that should probably be more explicit in INSTALL. I'll update it.

Oops. Sorry I took a break for dinner (I see another e-mail or two
referencing INSTALL too).

The changes need to happen in doc/src/sgml/install.sgml, which is the
source code from which INSTALL is derived. If you look at that file I
think you will see how it corresponds to the output file.

If you can make changes and preserve the sgml markup, great. If not,
just send patches with text inserted in the right places and I'll
finish up the markup.

If your changes to INSTALL are pretty isolated, then I can also accept
patches on that file. But ones on the sgml source would be easier.

Toy, schmoy. This is going to back up several web sites. After the
recent glowing description on this mailing list of v6.5.1, I decided to
take the plunge and junk Illustra (the total lack of support and
documentation and tuning and licensing freedom and so on have finally
taken their toll on my confidence...) Wish me luck.

We'll be interested in hearing how it goes. afaik Illustra was
"Postgres done right", based on Postgres as of a few years ago with
some sections rewritten. In the meantime, PostgreSQL has had
substantial improvements, and I wonder if we've caught up to or passed
where Illustra froze.

- Thomas

--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California