pg_upgrade ?deficiency

Started by Karsten Hilbertover 12 years ago64 messageshackersgeneral
Jump to latest
#1Karsten Hilbert
Karsten.Hilbert@gmx.net
hackersgeneral

Hello all,

I am upgrading a 8.4 cluster to 9.1 and am seeing the following:

SQL command failed

CREATE TEMPORARY TABLE info_rels (reloid) AS
SELECT c.oid
FROM
pg_catalog.pg_class c
JOIN pg_catalog.pg_namespace n ON c.relnamespace = n.oid
LEFT OUTER JOIN pg_catalog.pg_index i ON c.oid = i.indexrelid
WHERE
relkind IN ('r', 'm', 'i', 'S')
AND
i.indisvalid IS DISTINCT FROM false
AND
i.indisready IS DISTINCT FROM false
AND
((n.nspname !~ '^pg_temp_'
AND
n.nspname !~ '^ pg_toast_temp_'
AND
n.nspname NOT IN ('pg_catalog', 'information_schema', 'binary_upgrade', 'pg_toast')
AND
c.oid >= 16384
)
OR
(n.nspname = 'pg_catalog'
AND
relname IN ('pg_largeobject', 'pg_largeobject_loid_pn_index')
));

ERROR: transaction is read-only

Now, this is quite understandable since one of the databases
is set to

ALTER DATABASE ... SET DEFAULT_TRANSACTION_READ_ONLY TO ON;

However, since the above setting is something which can
be expected every so often in any odd PostgreSQL cluster
(and not some weird coincidence no one really knows how
they got into in the first place) I would think pg_upgrade
really should be able to handle.

Technically that's pretty easy - make sure transactions are
set to readwrite for the pg_upgrade run by any number of
means:

- ALTER DATABASE before/after pg_upgrade
- ALTER USER running the pg_upgrade
- SET TRANSACTION READ WRITE at the appropriate times
- ...

Or at least this limitation of pg_upgrade (requiring
DB write access) should get a mention in the docs and/or
man page.

What is the informed opinion on this ?

Thanks,
Karsten
--
GPG key ID E4071346 @ gpg-keyserver.de
E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#2Bruce Momjian
bruce@momjian.us
In reply to: Karsten Hilbert (#1)
hackersgeneral
Re: pg_upgrade ?deficiency

On Tue, Nov 19, 2013 at 11:22:47AM +0100, Karsten Hilbert wrote:

ERROR: transaction is read-only

Now, this is quite understandable since one of the databases
is set to

ALTER DATABASE ... SET DEFAULT_TRANSACTION_READ_ONLY TO ON;

However, since the above setting is something which can
be expected every so often in any odd PostgreSQL cluster
(and not some weird coincidence no one really knows how
they got into in the first place) I would think pg_upgrade
really should be able to handle.

Technically that's pretty easy - make sure transactions are
set to readwrite for the pg_upgrade run by any number of
means:

- ALTER DATABASE before/after pg_upgrade
- ALTER USER running the pg_upgrade
- SET TRANSACTION READ WRITE at the appropriate times
- ...

Or at least this limitation of pg_upgrade (requiring
DB write access) should get a mention in the docs and/or
man page.

What is the informed opinion on this ?

I think pg_upgrade did the right thing here by throwing an error. There
is no clean way to handle these cases without possibly causing more
problems.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#3Karsten Hilbert
Karsten.Hilbert@gmx.net
In reply to: Bruce Momjian (#2)
hackersgeneral
Re: pg_upgrade ?deficiency

On Tue, Nov 19, 2013 at 11:22:47AM +0100, Karsten Hilbert wrote:

ERROR: transaction is read-only

Now, this is quite understandable since one of the databases
is set to

ALTER DATABASE ... SET DEFAULT_TRANSACTION_READ_ONLY TO ON;

However, since the above setting is something which can
be expected every so often in any odd PostgreSQL cluster
(and not some weird coincidence no one really knows how
they got into in the first place) I would think pg_upgrade
really should be able to handle.

Technically that's pretty easy - make sure transactions are
set to readwrite for the pg_upgrade run by any number of
means:

- ALTER DATABASE before/after pg_upgrade
- ALTER USER running the pg_upgrade
- SET TRANSACTION READ WRITE at the appropriate times
- ...

Or at least this limitation of pg_upgrade (requiring
DB write access) should get a mention in the docs and/or
man page.

What is the informed opinion on this ?

I think pg_upgrade did the right thing here by throwing an error. There
is no clean way to handle these cases without possibly causing more
problems.

I am not sure this is the ideal way of looking at the problem (for one
thing it wasn't pg_upgrade throwing the error). Maybe I have not clearly
expressed myself.

Let me try to rephrase:

Fact: pg_upgrade can NOT properly upgrade clusters which contain
databases that are set to "default_transaction_read_only on"

Question: Is this intended ?

Thanks,
Karsten

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#4Laurenz Albe
laurenz.albe@cybertec.at
In reply to: Karsten Hilbert (#3)
hackersgeneral
Re: pg_upgrade ?deficiency

Karsten Hilbert wrote:

Let me try to rephrase:

Fact: pg_upgrade can NOT properly upgrade clusters which contain
databases that are set to "default_transaction_read_only on"

Question: Is this intended ?

I am pretty sure that this is an oversight and hence a bug.

Yours,
Laurenz Albe

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#5Karsten Hilbert
Karsten.Hilbert@gmx.net
In reply to: Laurenz Albe (#4)
hackersgeneral
Re: pg_upgrade ?deficiency

Karsten Hilbert wrote:

Let me try to rephrase:

Fact: pg_upgrade can NOT properly upgrade clusters which contain
databases that are set to "default_transaction_read_only on"

Question: Is this intended ?

I am pretty sure that this is an oversight and hence a bug.

oversight, yes ... I thought as much and was therefore a bit
cautious of calling it a bug, chose to name it "?deficiency" ;-)

Karsten

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#6Bruce Momjian
bruce@momjian.us
In reply to: Karsten Hilbert (#5)
hackersgeneral
Re: pg_upgrade ?deficiency

On Wed, Nov 20, 2013 at 02:36:08PM +0100, Karsten Hilbert wrote:

Karsten Hilbert wrote:

Let me try to rephrase:

Fact: pg_upgrade can NOT properly upgrade clusters which contain
databases that are set to "default_transaction_read_only on"

Question: Is this intended ?

I am pretty sure that this is an oversight and hence a bug.

oversight, yes ... I thought as much and was therefore a bit
cautious of calling it a bug, chose to name it "?deficiency" ;-)

Well, pg_upgrade can't handle every possible configuration. How do we
even restore into such a database? You marked the database as
read-only, and pg_upgrade is going to honor that and not modify it. I
believe a pg_dumpall restore might fail in the same way.

You need to change the default on the old cluster before upgrading. It
is overly cumbersome to set the default_transaction_read_only for every
database connection, and there are many other settings that might also
cause failures. If it was a silent failure, I would be more concerned.

What you might be able to do is to set PGOPTIONS to "-c
default_transaction_read_only=false" and run pg_upgrade. If more people
report this problem, I could document this work-around.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#7Karsten Hilbert
Karsten.Hilbert@gmx.net
In reply to: Bruce Momjian (#6)
hackersgeneral
Re: pg_upgrade ?deficiency

On Wed, Nov 20, 2013 at 02:36:08PM +0100, Karsten Hilbert wrote:

Karsten Hilbert wrote:

Let me try to rephrase:

Fact: pg_upgrade can NOT properly upgrade clusters which contain
databases that are set to "default_transaction_read_only on"

Question: Is this intended ?

I am pretty sure that this is an oversight and hence a bug.

Well, pg_upgrade can't handle every possible configuration.

Agreed. That would be a design decision: "no, pg_upgrade will
not support upgrading some of your databases, for example those
which are set to default_transaction_ready_only=on".

If I don't like that, fine, I can go and use other tools or
else submit a patch and hope for inclusion or apply a workaround.

That's why I tacitly suggested a hint in the docs might
help to become aware of the above limitation.

Of course, I should submit a patch to the docs just as well.

How do we even restore into such a database?

We read the state, remember the state, change the state,
restore the data, set the initial state. But you knew that,
I assume.

You marked the database as read-only, and pg_upgrade
is going to honor that and not modify it.

Oh, I am extremely happy for pg_upgrade to NOT modify
ANY of my databases ! All I am wondering is whether
it is by design decision (and if so, why) that it cannot
transfer some databases from one PG version to another
one. I am more than happy if it doesn't modify the
databases in the process ;-)

I believe a pg_dumpall restore might fail in the same way.

pg_dumpall works but a full pg_restore/psql from that dump
likely will not. I haven't tested that yet, though, and I
deliberately did not want to raise *that* question just
yet...

You need to change the default on the old cluster before upgrading.

I know. That wasn't my question though.

It is overly cumbersome to set the default_transaction_read_only for every
database connection,

There is no need for that (see above).

and there are many other settings that might also cause failures.

If so they warrant documentation as well as they become known.

If it was a silent failure, I would be more concerned.

Absolutely, full agreement.

What you might be able to do is to set PGOPTIONS to "-c
default_transaction_read_only=false" and run pg_upgrade.

That is a good idea. It might have occurred to me earlier
had the pg_upgrade limitation been documented ;-)

Thanks for your work on PostgreSQL,
Karsten

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#8Bruce Momjian
bruce@momjian.us
In reply to: Karsten Hilbert (#7)
hackersgeneral
Re: pg_upgrade ?deficiency

On Wed, Nov 20, 2013 at 04:07:59PM +0100, Karsten Hilbert wrote:

On Wed, Nov 20, 2013 at 02:36:08PM +0100, Karsten Hilbert wrote:

Karsten Hilbert wrote:

Let me try to rephrase:

Fact: pg_upgrade can NOT properly upgrade clusters which contain
databases that are set to "default_transaction_read_only on"

Question: Is this intended ?

I am pretty sure that this is an oversight and hence a bug.

Well, pg_upgrade can't handle every possible configuration.

Agreed. That would be a design decision: "no, pg_upgrade will
not support upgrading some of your databases, for example those
which are set to default_transaction_ready_only=on".

If I don't like that, fine, I can go and use other tools or
else submit a patch and hope for inclusion or apply a workaround.

That's why I tacitly suggested a hint in the docs might
help to become aware of the above limitation.

Of course, I should submit a patch to the docs just as well.

I think the big question is whether a generic mention that there are
some database settings, like read-only, that can prevent updates, and
you might need to use PGOPTIONS to avoid that. However, you are the
first case to report this, so I am hesistant.

How do we even restore into such a database?

We read the state, remember the state, change the state,
restore the data, set the initial state. But you knew that,
I assume.

Yep.

You marked the database as read-only, and pg_upgrade
is going to honor that and not modify it.

Oh, I am extremely happy for pg_upgrade to NOT modify
ANY of my databases ! All I am wondering is whether
it is by design decision (and if so, why) that it cannot
transfer some databases from one PG version to another
one. I am more than happy if it doesn't modify the
databases in the process ;-)

Yes, messing with status can often be problematic.

What you might be able to do is to set PGOPTIONS to "-c
default_transaction_read_only=false" and run pg_upgrade.

That is a good idea. It might have occurred to me earlier
had the pg_upgrade limitation been documented ;-)

True. Does anyone else see value in documenting this? I can do the
docs.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#9Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Bruce Momjian (#6)
hackersgeneral
Re: pg_upgrade ?deficiency

Bruce Momjian <bruce@momjian.us> wrote:

On Wed, Nov 20, 2013 at 02:36:08PM +0100, Karsten Hilbert wrote:

Karsten Hilbert wrote:

Let me try to rephrase:

Fact: pg_upgrade can NOT properly upgrade clusters which
       contain databases that are set to
       "default_transaction_read_only on"
Question: Is this intended ?

I am pretty sure that this is an oversight and hence a bug.

oversight, yes ... I thought as much and was therefore a bit
cautious of calling it a bug, chose to name it "?deficiency" ;-)

Well, pg_upgrade can't handle every possible configuration.  How
do we even restore into such a database?  You marked the database
as read-only, and pg_upgrade is going to honor that and not
modify it.

That interpretation makes no sense to me.  I know of users who have
databases where 90% of their transactions don't modify data, so
they set the *default* for transactions to read only, and override
that for transactions that are read write.  The default is not, and
never has been, a restriction on what is allowed -- it is a default
that is quite easy to override.  If we have tools that don't handle
that correctly, I consider that a bug.

I believe a pg_dumpall restore might fail in the same way.

Then it should also be fixed.

You need to change the default on the old cluster before
upgrading.  It is overly cumbersome to set the
default_transaction_read_only for every database connection

Why is this any different from other settings we cover at the front
of pg_dump output?:

| SET statement_timeout = 0;
| SET lock_timeout = 0;
| SET client_encoding = 'UTF8';
| SET standard_conforming_strings = on;
| SET check_function_bodies = false;
| SET client_min_messages = warning;

and there are many other settings that might also cause failures.

You mean, like the above?

What you might be able to do is to set PGOPTIONS to "-c
default_transaction_read_only=false" and run pg_upgrade.  If more
people report this problem, I could document this work-around.

This is most likely to bite those using serializable transactions
for data integrity, because declaring transactions read only makes
a huge difference in performance in those cases.  That is where I
have seen people set the default for read only to on; they want to
explicitly set it off only when needed.

I would be happy to supply a patch to treat
default_transaction_read_only the same as statement_timeout or
standard_conforming_strings in pg_dump and related utilities.
Since it causes backup/restore failure on perfectly valid databases
I even think this is a bug which merits back-patching.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#10Karsten Hilbert
Karsten.Hilbert@gmx.net
In reply to: Kevin Grittner (#9)
hackersgeneral
Re: pg_upgrade ?deficiency

On Thu, Nov 21, 2013 at 06:22:50AM -0800, Kevin Grittner wrote:

I would be happy to supply a patch to treat
default_transaction_read_only the same as statement_timeout or
standard_conforming_strings in pg_dump and related utilities.
Since it causes backup/restore failure

... (and pg_upgrade failures -- which may internally
just be dump/restore cycles ?) ...

on perfectly valid databases I even think this is
a bug which merits back-patching.

Thanks so much, Kevin, for offering to work
on that part. Maybe it's a small thing but
it'll make PostgreSQL once again feel
professionally consistent.

I would have needed to become proficient in C
and get acqainted with the PG source in order
to produce a patch myself.

Thanks,
Karsten
--
GPG key ID E4071346 @ gpg-keyserver.de
E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#11Bruce Momjian
bruce@momjian.us
In reply to: Kevin Grittner (#9)
hackersgeneral
Re: pg_upgrade ?deficiency

On Thu, Nov 21, 2013 at 06:22:50AM -0800, Kevin Grittner wrote:

Well, pg_upgrade can't handle every possible configuration.� How
do we even restore into such a database?� You marked the database
as read-only, and pg_upgrade is going to honor that and not
modify it.

That interpretation makes no sense to me.� I know of users who have
databases where 90% of their transactions don't modify data, so
they set the *default* for transactions to read only, and override
that for transactions that are read write.� The default is not, and
never has been, a restriction on what is allowed -- it is a default
that is quite easy to override.� If we have tools that don't handle
that correctly, I consider that a bug.

OK, this is good information to hear.

I believe a pg_dumpall restore might fail in the same way.

Then it should also be fixed.

Yes, that is easy to do.

You need to change the default on the old cluster before
upgrading.� It is overly cumbersome to set the
default_transaction_read_only for every database connection

Why is this any different from other settings we cover at the front
of pg_dump output?:

| SET statement_timeout = 0;
| SET lock_timeout = 0;
| SET client_encoding = 'UTF8';
| SET standard_conforming_strings = on;
| SET check_function_bodies = false;
| SET client_min_messages = warning;

and there are many other settings that might also cause failures.

You mean, like the above?

What you might be able to do is to set PGOPTIONS to "-c
default_transaction_read_only=false" and run pg_upgrade.� If more
people report this problem, I could document this work-around.

This is most likely to bite those using serializable transactions
for data integrity, because declaring transactions read only makes
a huge difference in performance in those cases.� That is where I
have seen people set the default for read only to on; they want to
explicitly set it off only when needed.

I would be happy to supply a patch to treat
default_transaction_read_only the same as statement_timeout or
standard_conforming_strings in pg_dump and related utilities.
Since it causes backup/restore failure on perfectly valid databases
I even think this is a bug which merits back-patching.

Not sure about backpatching. default_transaction_read_only has been
around since 7.4. Setting it to true would cause pg_dump to fail unless
you changed the database setting, and pg_dumpall would fail completely
as there is no way to turn off the database setting.

The problem is that I don't remember any report of this failing in
pg_dump, pg_dumpall, or pg_upgrade, so saying it is a major issue is
hard to accept.

However, looking forward, I think we should add it to pg_dump (and hence
pg_dumpall), and we should fix pg_upgrade so it is more reliable. I am
thinking we should either document in pg_upgrade the use of PGOPTIONS to
avoid this issue, or have pg_upgrade append to PGOPTIONS in its
environment to use some of pg_dump's resets, and that will be passed to
libpq connections, psql, and all the utilities pg_upgrade calls.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#12Bruce Momjian
bruce@momjian.us
In reply to: Bruce Momjian (#11)
hackersgeneral
Re: [GENERAL] pg_upgrade ?deficiency

Sending to hackers for comment; seems setting
default_transaction_read_only to true via ALTER database/user or
postgresql.conf can cause pg_dump, pg_dumpall, and pg_upgrade failures.
We are considering the right solution:

---------------------------------------------------------------------------

On Fri, Nov 22, 2013 at 01:32:30PM -0500, Bruce Momjian wrote:

On Thu, Nov 21, 2013 at 06:22:50AM -0800, Kevin Grittner wrote:

Well, pg_upgrade can't handle every possible configuration.� How
do we even restore into such a database?� You marked the database
as read-only, and pg_upgrade is going to honor that and not
modify it.

That interpretation makes no sense to me.� I know of users who have
databases where 90% of their transactions don't modify data, so
they set the *default* for transactions to read only, and override
that for transactions that are read write.� The default is not, and
never has been, a restriction on what is allowed -- it is a default
that is quite easy to override.� If we have tools that don't handle
that correctly, I consider that a bug.

OK, this is good information to hear.

I believe a pg_dumpall restore might fail in the same way.

Then it should also be fixed.

Yes, that is easy to do.

You need to change the default on the old cluster before
upgrading.� It is overly cumbersome to set the
default_transaction_read_only for every database connection

Why is this any different from other settings we cover at the front
of pg_dump output?:

| SET statement_timeout = 0;
| SET lock_timeout = 0;
| SET client_encoding = 'UTF8';
| SET standard_conforming_strings = on;
| SET check_function_bodies = false;
| SET client_min_messages = warning;

and there are many other settings that might also cause failures.

You mean, like the above?

What you might be able to do is to set PGOPTIONS to "-c
default_transaction_read_only=false" and run pg_upgrade.� If more
people report this problem, I could document this work-around.

This is most likely to bite those using serializable transactions
for data integrity, because declaring transactions read only makes
a huge difference in performance in those cases.� That is where I
have seen people set the default for read only to on; they want to
explicitly set it off only when needed.

I would be happy to supply a patch to treat
default_transaction_read_only the same as statement_timeout or
standard_conforming_strings in pg_dump and related utilities.
Since it causes backup/restore failure on perfectly valid databases
I even think this is a bug which merits back-patching.

Not sure about backpatching. default_transaction_read_only has been
around since 7.4. Setting it to true would cause pg_dump to fail unless
you changed the database setting, and pg_dumpall would fail completely
as there is no way to turn off the database setting.

The problem is that I don't remember any report of this failing in
pg_dump, pg_dumpall, or pg_upgrade, so saying it is a major issue is
hard to accept.

However, looking forward, I think we should add it to pg_dump (and hence
pg_dumpall), and we should fix pg_upgrade so it is more reliable. I am
thinking we should either document in pg_upgrade the use of PGOPTIONS to
avoid this issue, or have pg_upgrade append to PGOPTIONS in its
environment to use some of pg_dump's resets, and that will be passed to
libpq connections, psql, and all the utilities pg_upgrade calls.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#13Tom Lane
tgl@sss.pgh.pa.us
In reply to: Bruce Momjian (#11)
hackersgeneral
Re: pg_upgrade ?deficiency

Bruce Momjian <bruce@momjian.us> writes:

Not sure about backpatching. default_transaction_read_only has been
around since 7.4. Setting it to true would cause pg_dump to fail unless
you changed the database setting, and pg_dumpall would fail completely
as there is no way to turn off the database setting.

No, neither pg_dump nor pg_dumpall would fail. What would fail is
restoring into a database that has this option already set. It's possible
that users of this option haven't noticed it because they never attempted
a restore in such a context.

The problem is that I don't remember any report of this failing in
pg_dump, pg_dumpall, or pg_upgrade, so saying it is a major issue is
hard to accept.

Yeah, it's a minor issue at best, but perhaps worth fixing since
the solution is so easy.

The bigger picture here is that there are lots of ways to break
pg_upgrade via not-sane settings, and there always will be.
I don't think we should try to promise that there won't be.

regards, tom lane

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#14Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Bruce Momjian (#12)
hackersgeneral
Re: [GENERAL] pg_upgrade ?deficiency

Bruce Momjian <bruce@momjian.us> wrote:

Not sure about backpatching.  default_transaction_read_only has been
around since 7.4.  Setting it to true would cause pg_dump to fail unless
you changed the database setting, and pg_dumpall would fail completely
as there is no way to turn off the database setting.

See the attached patch.  It seems to fix pg_dump and pg_dumpall.  I
don't think it will cause any problem for *dumping* earlier
versions,  Backpatching would mean that if you try to restore a
dump made by 8.4 or later software to a 7.3 or earlier database,
you would get an error; but I don't think that's supported, and I
would be amazed if that were the *only* error you got if you tried
that.

The problem is that I don't remember any report of this failing in
pg_dump, pg_dumpall, or pg_upgrade, so saying it is a major issue is
hard to accept.

Any time that you can appear to successfully dump a database, and
the restore attempt fails, I consider that to be a major issue.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#15Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Kevin Grittner (#14)
hackersgeneral
Re: [GENERAL] pg_upgrade ?deficiency

Kevin Grittner <kgrittn@ymail.com> wrote:

See the attached patch.

Trying that again.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Attachments:

pg_dump-vs-default_transaction_read_only-v1.difftext/x-diff; name=pg_dump-vs-default_transaction_read_only-v1.diffDownload+3-0
#16Karsten Hilbert
Karsten.Hilbert@gmx.net
In reply to: Tom Lane (#13)
hackersgeneral
Re: pg_upgrade ?deficiency

Bruce Momjian <bruce@momjian.us> writes:

Not sure about backpatching. default_transaction_read_only has been
around since 7.4. Setting it to true would cause pg_dump to fail unless
you changed the database setting, and pg_dumpall would fail completely
as there is no way to turn off the database setting.

No, neither pg_dump nor pg_dumpall would fail. What would fail is
restoring into a database that has this option already set. It's possible
that users of this option haven't noticed it because they never attempted
a restore in such a context.

I was the original poster on -users who raised this issue. Maybe I can
clarify somewhat:

I have been attempting to upgrade an 8.4 cluster to 9.1
by means of the 9.1 pg_upgrade command.

That failed due to one of the databases in the 8.4 cluster
being "ALTER DATABASE ... SET DEFAULT_TRANSACTION_READ_ONLY TO ON".

Hence my question on that list whether that was to be considered
a bug, a deficiency, or an oversight.

I knew workarounds quite well but wondered whether that
pg_upgrade behaviour was intended to stay that way.

I suggested that if it is intended to stay it might benefit
from a hint in the documentation.

Yeah, it's a minor issue at best, but perhaps worth fixing since
the solution is so easy.

That would be really helpful.

The bigger picture here is that there are lots of ways to break
pg_upgrade via not-sane settings, and there always will be.

Would setting default_transaction_read_only to on be considered
non-sane ? If so, why ?

I don't think we should try to promise that there won't be.

That last assertion is what everyone should certainly be able
to agree with ;-)

Thanks,
Karsten

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#17Tom Lane
tgl@sss.pgh.pa.us
In reply to: Kevin Grittner (#15)
hackersgeneral
Re: [GENERAL] pg_upgrade ?deficiency

Kevin Grittner <kgrittn@ymail.com> writes:

Kevin Grittner <kgrittn@ymail.com> wrote:

See the attached patch.

Trying that again.

That looks sane for pg_dump, but I would rather have expected that
pg_dumpall would need to emit the same thing (possibly more than once
due to reconnections).

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#18Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Tom Lane (#17)
hackersgeneral
Re: [GENERAL] pg_upgrade ?deficiency

Tom Lane <tgl@sss.pgh.pa.us> wrote:

That looks sane for pg_dump, but I would rather have expected
that pg_dumpall would need to emit the same thing (possibly more
than once due to reconnections).

I was kinda surprised myself.  I changed it for pg_dump, tested
that, and then tested pg_dumpall to get a baseline, and the setting
was taken care of.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#19Andres Freund
andres@anarazel.de
In reply to: Kevin Grittner (#18)
hackersgeneral
Re: [GENERAL] pg_upgrade ?deficiency

On 2013-11-22 12:45:25 -0800, Kevin Grittner wrote:

Tom Lane <tgl@sss.pgh.pa.us> wrote:

That looks sane for pg_dump, but I would rather have expected
that pg_dumpall would need to emit the same thing (possibly more
than once due to reconnections).

I was kinda surprised myself.� I changed it for pg_dump, tested
that, and then tested pg_dumpall to get a baseline, and the setting
was taken care of.

pg_dumpall is lazy and just executes pg_dump for every database, that's
the reason... But are you sure it also unsets default_transaction_readonly for
the roles etc. it creates?

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#20Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Andres Freund (#19)
hackersgeneral
Re: [GENERAL] pg_upgrade ?deficiency

Andres Freund <andres@2ndquadrant.com> wrote:

are you sure it also unsets default_transaction_readonly for
the roles etc. it creates?

I'm not sure I understand.  Could you give an example of what
you're concerned about?

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#21Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#13)
hackersgeneral
#22Andres Freund
andres@anarazel.de
In reply to: Kevin Grittner (#20)
hackersgeneral
#23Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#22)
hackersgeneral
#24Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Andres Freund (#22)
hackersgeneral
#25Andres Freund
andres@anarazel.de
In reply to: Kevin Grittner (#24)
hackersgeneral
#26Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Kevin Grittner (#24)
hackersgeneral
#27Andres Freund
andres@anarazel.de
In reply to: Kevin Grittner (#24)
hackersgeneral
#28Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Andres Freund (#27)
hackersgeneral
#29Bruce Momjian
bruce@momjian.us
In reply to: Kevin Grittner (#26)
hackersgeneral
#30Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Bruce Momjian (#29)
hackersgeneral
#31Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Andres Freund (#25)
hackersgeneral
#32Bruce Momjian
bruce@momjian.us
In reply to: Kevin Grittner (#30)
hackersgeneral
#33Bruce Momjian
bruce@momjian.us
In reply to: Kevin Grittner (#31)
hackersgeneral
#34Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Bruce Momjian (#33)
hackersgeneral
#35Karsten Hilbert
Karsten.Hilbert@gmx.net
In reply to: Kevin Grittner (#34)
hackersgeneral
#36Tom Lane
tgl@sss.pgh.pa.us
In reply to: Kevin Grittner (#34)
hackersgeneral
#37Karsten Hilbert
Karsten.Hilbert@gmx.net
In reply to: Tom Lane (#36)
hackersgeneral
#38Sebastian Hilbert
sebastian.hilbert@gmx.net
In reply to: Kevin Grittner (#34)
hackersgeneral
#39Bruce Momjian
bruce@momjian.us
In reply to: Kevin Grittner (#34)
hackersgeneral
#40Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Bruce Momjian (#39)
hackersgeneral
#41Bruce Momjian
bruce@momjian.us
In reply to: Kevin Grittner (#40)
hackersgeneral
#42Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Bruce Momjian (#41)
hackersgeneral
#43Bruce Momjian
bruce@momjian.us
In reply to: Kevin Grittner (#42)
hackersgeneral
#44Karsten Hilbert
Karsten.Hilbert@gmx.net
In reply to: Kevin Grittner (#42)
hackersgeneral
#45Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Bruce Momjian (#43)
hackersgeneral
#46Bruce Momjian
bruce@momjian.us
In reply to: Kevin Grittner (#45)
hackersgeneral
#47Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Bruce Momjian (#46)
hackersgeneral
#48Bruce Momjian
bruce@momjian.us
In reply to: Kevin Grittner (#47)
hackersgeneral
#49Bruce Momjian
bruce@momjian.us
In reply to: Kevin Grittner (#47)
hackersgeneral
#50Karsten Hilbert
Karsten.Hilbert@gmx.net
In reply to: Bruce Momjian (#48)
hackersgeneral
#51Bruce Momjian
bruce@momjian.us
In reply to: Karsten Hilbert (#50)
hackersgeneral
#52Karsten Hilbert
Karsten.Hilbert@gmx.net
In reply to: Bruce Momjian (#51)
hackersgeneral
#53Bruce Momjian
bruce@momjian.us
In reply to: Karsten Hilbert (#52)
hackersgeneral
#54Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Bruce Momjian (#53)
hackersgeneral
#55Bruce Momjian
bruce@momjian.us
In reply to: Kevin Grittner (#26)
hackersgeneral
#56Tom Lane
tgl@sss.pgh.pa.us
In reply to: Bruce Momjian (#55)
hackersgeneral
#57Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Tom Lane (#56)
hackersgeneral
#58Tom Lane
tgl@sss.pgh.pa.us
In reply to: Kevin Grittner (#57)
hackersgeneral
#59Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#58)
hackersgeneral
#60Karsten Hilbert
Karsten.Hilbert@gmx.net
In reply to: Kevin Grittner (#57)
hackersgeneral
#61Bruce Momjian
bruce@momjian.us
In reply to: Karsten Hilbert (#60)
hackersgeneral
#62Karsten Hilbert
Karsten.Hilbert@gmx.net
In reply to: Bruce Momjian (#61)
hackersgeneral
#63Bruce Momjian
bruce@momjian.us
In reply to: Karsten Hilbert (#62)
hackersgeneral
#64Karsten Hilbert
Karsten.Hilbert@gmx.net
In reply to: Bruce Momjian (#63)
hackersgeneral