pgsql: Get rid of the dynamic shared memory state file.

Started by Robert Haasover 12 years ago5 messagescomitters
Jump to latest
#1Robert Haas
robertmhaas@gmail.com

Get rid of the dynamic shared memory state file.

Instead of storing the ID of the dynamic shared memory control
segment in a file within the data directory, store it in the main
control segment. This avoids a number of nasty corner cases,
most seriously that doing an online backup and then using it on
the same machine (e.g. to fire up a standby) would result in the
standby clobbering all of the master's dynamic shared memory
segments.

Per complaints from Heikki Linnakangas, Fujii Masao, and Tom
Lane.

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/11a65eed1637a05b03e174700799b024e104bfb4

Modified Files
--------------
src/backend/port/sysv_shmem.c | 12 ++-
src/backend/port/win32_shmem.c | 6 +-
src/backend/storage/ipc/dsm.c | 200 ++++++++--------------------------------
src/backend/storage/ipc/ipci.c | 6 +-
src/include/storage/dsm.h | 8 +-
src/include/storage/pg_shmem.h | 5 +-
6 files changed, 71 insertions(+), 166 deletions(-)

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

#2Andres Freund
andres@anarazel.de
In reply to: Robert Haas (#1)
Re: pgsql: Get rid of the dynamic shared memory state file.

On 2014-04-08 15:45:39 +0000, Robert Haas wrote:

Get rid of the dynamic shared memory state file.

Instead of storing the ID of the dynamic shared memory control
segment in a file within the data directory, store it in the main
control segment. This avoids a number of nasty corner cases,
most seriously that doing an online backup and then using it on
the same machine (e.g. to fire up a standby) would result in the
standby clobbering all of the master's dynamic shared memory
segments.

Per complaints from Heikki Linnakangas, Fujii Masao, and Tom
Lane.

Looks like it's not working for windows yet:
http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=bowerbird&dt=2014-04-08%2016%3A31%3A18
http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=baiji&dt=2014-04-08%2017%3A00%3A01

There's also the to_reg* commit in the ones tested by those runs, but it
sounds unlikely that those would trigger during initdb.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

#3Robert Haas
robertmhaas@gmail.com
In reply to: Andres Freund (#2)
Re: pgsql: Get rid of the dynamic shared memory state file.

On Tue, Apr 8, 2014 at 4:14 PM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2014-04-08 15:45:39 +0000, Robert Haas wrote:

Get rid of the dynamic shared memory state file.

Instead of storing the ID of the dynamic shared memory control
segment in a file within the data directory, store it in the main
control segment. This avoids a number of nasty corner cases,
most seriously that doing an online backup and then using it on
the same machine (e.g. to fire up a standby) would result in the
standby clobbering all of the master's dynamic shared memory
segments.

Per complaints from Heikki Linnakangas, Fujii Masao, and Tom
Lane.

Looks like it's not working for windows yet:
http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=bowerbird&amp;dt=2014-04-08%2016%3A31%3A18
http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=baiji&amp;dt=2014-04-08%2017%3A00%3A01

There's also the to_reg* commit in the ones tested by those runs, but it
sounds unlikely that those would trigger during initdb.

Crap. I did test that it worked with EXEC_BACKEND, but I don't have a
Windows environment set up. I can try to eyeball the code and see
what might be wrong with it, but...

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

#4Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#1)
Re: pgsql: Get rid of the dynamic shared memory state file.

Robert Haas <rhaas@postgresql.org> writes:

Get rid of the dynamic shared memory state file.

The Windows buildfarm critters don't seem to like this patch.
narwhal for example:

creating directory C:/msys/1.0/local/pgbuildfarm/buildroot/HEAD/pgsql.3944/src/test/regress/./tmp_check/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... windows
creating configuration files ... ok
creating template1 database in C:/msys/1.0/local/pgbuildfarm/buildroot/HEAD/pgsql.3944/src/test/regress/./tmp_check/data/base/1 ... LOG: dynamic shared memory control segment is corrupt
child process was terminated by exception 0xC0000005

baiji and bowerbird also crashed in initdb, though without the message
about "dynamic shared memory control segment is corrupt". Presumably
it's the same bug though.

regards, tom lane

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

#5Andres Freund
andres@anarazel.de
In reply to: Robert Haas (#3)
Re: pgsql: Get rid of the dynamic shared memory state file.

On 2014-04-08 16:17:46 -0400, Robert Haas wrote:

On Tue, Apr 8, 2014 at 4:14 PM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2014-04-08 15:45:39 +0000, Robert Haas wrote:

Get rid of the dynamic shared memory state file.

Instead of storing the ID of the dynamic shared memory control
segment in a file within the data directory, store it in the main
control segment. This avoids a number of nasty corner cases,
most seriously that doing an online backup and then using it on
the same machine (e.g. to fire up a standby) would result in the
standby clobbering all of the master's dynamic shared memory
segments.

Per complaints from Heikki Linnakangas, Fujii Masao, and Tom
Lane.

Looks like it's not working for windows yet:
http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=bowerbird&amp;dt=2014-04-08%2016%3A31%3A18
http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=baiji&amp;dt=2014-04-08%2017%3A00%3A01

There's also the to_reg* commit in the ones tested by those runs, but it
sounds unlikely that those would trigger during initdb.

Crap. I did test that it worked with EXEC_BACKEND, but I don't have a
Windows environment set up. I can try to eyeball the code and see
what might be wrong with it, but...

Not a windows guy either, but I just noticed that
http://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=narwhal&amp;dt=2014-04-08%2016%3A00%3A03&amp;stg=check
has an additional log entry, the others don't:
LOG: dynamic shared memory control segment is corrupt

since that's apparently the first crash on narwahl related to this, it
might be a hint where things are going wrong.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers