Nested xacts: looking for testers and review

On Wed, 26 May 2004, Alvaro Herrera wrote:

I'm missing one item: deferred triggers. The problem with this is that
the deftrig queue is not implemented using normal Lists, so there's no
efficient way to reassign to the parent when the subtransaction commits.
Also I'm not sure what should happen to the "immediate" pointer --- a
subtransaction should have it's own private copy, or it should inherit
the parent's? Please whoever implemented this speak up (Stephan
Szabo?), as I'm not sure of the semantics.

[I haven't had a chance to look at the patch, because I'm at work, so you
may already have throught about/coded for some of the things below,
but...]

The immediate pointer basically points at the end of the queue from the
last scanning of the trigger queue (since any "immediate" triggers from
before that would have been run at that time there's no reason to scan
from the beginning).

As a related side question I just thought of (and wish I had earlier),
what's the correct behavior of SET CONSTRAINTS when setting a constraint
to immediate mode in the subtransaction case? In the single transaction
case, we change whether we consider the constraint deferred (or set the
"all"), reset the immediate pointer to the beginning of the queue, and
allow the end of statement scan to do what should hopefully be the right
thing. If one sets a constraint to immediate in a subtransaction,
does/should it cause the immediate check of pending events from its
parent? And does that revert when the transaction ends?

Before thinking about that, I figured that each subtransaction would have
a private trigger queue and then that the immediate pointer would be
private in each queue (starting as NULL when the subtransaction started)
and that if items were copied/reassigned at subtransaction end, the parent
transaction's immediate would move to just after that point after the
copy. Now, I'm not entirely sure.

On Wed, May 26, 2004 at 04:35:52PM -0700, Stephan Szabo wrote:

On Wed, 26 May 2004, Alvaro Herrera wrote:

I'm missing one item: deferred triggers. The problem with this is that
the deftrig queue is not implemented using normal Lists, so there's no
efficient way to reassign to the parent when the subtransaction commits.
Also I'm not sure what should happen to the "immediate" pointer --- a
subtransaction should have it's own private copy, or it should inherit
the parent's? Please whoever implemented this speak up (Stephan
Szabo?), as I'm not sure of the semantics.

The immediate pointer basically points at the end of the queue from the
last scanning of the trigger queue (since any "immediate" triggers from
before that would have been run at that time there's no reason to scan
from the beginning).

Hmm. You assume correctly that a subtransaction has (or will have) a
private queue. But we do not consider a subtransaction to be somewhat a
separate entity -- the principle is that the transaction has to feel
just like the BEGIN wasn't there. So

BEGIN;
UPDATE foo ...
UPDATE bar ...
COMMIT

has to be exactly the same as

BEGIN;
BEGIN;
UPDATE foo ...
COMMIT;
BEGIN;
UPDATE bar ...
COMMIT;
COMMIT;

Now, with that in mind: is it correct that the "immediate" pointer
points to the beginning of the subtransaction's private deferred trigger
queue, at subtransaction's start?

Now, at subtransaction end, lets suppose I concatenate the list the
original transaction had with the subtransaction's private list. What
should the immediate pointer be?

When is the immediate pointer advanced? I know it's "during scanning of
the list", but when is this? At the end of each query?

If one sets a constraint to immediate in a subtransaction, does/should
it cause the immediate check of pending events from its parent? And
does that revert when the transaction ends?

Yes, I think it should fire all events, including the parent's. Good
point; it means there has to be a way of getting the whole list, from
the topmost transaction, in order :-(

I'm not sure what you mean by reverting though.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
La web junta la gente porque no importa que clase de mutante sexual seas,
tienes millones de posibles parejas. Pon "buscar gente que tengan sexo con
ciervos incendi�nse", y el computador dir� "especifique el tipo de ciervo"
(Jason Alexander)

On Thu, 27 May 2004, Alvaro Herrera wrote:

On Wed, May 26, 2004 at 04:35:52PM -0700, Stephan Szabo wrote:

On Wed, 26 May 2004, Alvaro Herrera wrote:

I'm missing one item: deferred triggers. The problem with this is that
the deftrig queue is not implemented using normal Lists, so there's no
efficient way to reassign to the parent when the subtransaction commits.
Also I'm not sure what should happen to the "immediate" pointer --- a
subtransaction should have it's own private copy, or it should inherit
the parent's? Please whoever implemented this speak up (Stephan
Szabo?), as I'm not sure of the semantics.

The immediate pointer basically points at the end of the queue from the
last scanning of the trigger queue (since any "immediate" triggers from
before that would have been run at that time there's no reason to scan
from the beginning).

Hmm. You assume correctly that a subtransaction has (or will have) a
private queue. But we do not consider a subtransaction to be somewhat a
separate entity -- the principle is that the transaction has to feel
just like the BEGIN wasn't there. So

BEGIN;
UPDATE foo ...
UPDATE bar ...
COMMIT

has to be exactly the same as

BEGIN;
BEGIN;
UPDATE foo ...
COMMIT;
BEGIN;
UPDATE bar ...
COMMIT;
COMMIT;

Now, with that in mind: is it correct that the "immediate" pointer
points to the beginning of the subtransaction's private deferred trigger
queue, at subtransaction's start?

AFAIR you can set it to NULL because that means scan the entire list.

Now, at subtransaction end, lets suppose I concatenate the list the
original transaction had with the subtransaction's private list. What
should the immediate pointer be?

I'd say pointing at the last item that was added to the queue.

When is the immediate pointer advanced? I know it's "during scanning of
the list", but when is this? At the end of each query?

At the point after triggers fire after a statement. It's the actual
scanning of the trigger queue to see what to run that changes it unless
I'm misremembering.

If one sets a constraint to immediate in a subtransaction, does/should
it cause the immediate check of pending events from its parent? And
does that revert when the transaction ends?

Yes, I think it should fire all events, including the parent's. Good
point; it means there has to be a way of getting the whole list, from
the topmost transaction, in order :-(

Yeah... Right now we don't need to do something special because resetting
the immediate pointer basically does what we want (re-scan the entire set
looking for non-run things that are now immediate).

I'm not sure what you mean by reverting though.

The state about whether a trigger is actually deferred or immediate. I
believe it basically works as:
begin;
set constraints all immediate;
-- here any deferrable constraints are treated as immediate
end;
begin;
-- here any deferrable constraints are in their default state
end;

So, if you have
begin;
-- 1
begin;
set constraints all immediate;
end;
-- 2
end;

Do 1 and 2 see the same constraint checking mode or is 2 at immediate?

I have tested the patches with May 28 16:20 JST snapshot. Here is my
first impression:

1) errors, rules regression tests are failed (I'm not sure this is due
to your patches)

2) certain behavior was different from what I expected (please correct me
if my expectation is wrong).

test=# begin;
BEGIN
test=# insert into t1 values(1);
INSERT 17216 1
test=# begin;
BEGIN
test=# aaa;
ERROR: syntax error at or near "aaa" at character 1
ERROR: syntax error at or near "aaa" at character 1
LINE 1: aaa;
^
test=# end;
COMMIT
test=# select * from t1; <-- I thought this should work since subtransaction was closed
ERROR: current transaction is aborted, commands ignored until end of transaction block
ERROR: current transaction is aborted, commands ignored until end of transaction block
test=# end;
COMMIT
test=# select * from t1;
i
---
(0 rows)

3) no docs found in the patches.

I think before this can be tested fully by a wide audience there needs to
be some basic documentation. Or is there?
What is the new syntax?
Can we see some basic examples that you have used in your testing?

Regards
Bob

|---------+---------------------------------->
| | Tatsuo Ishii |
| | <t-ishii@sra.co.jp> |
| | Sent by: |
| | pgsql-hackers-owner@pos|
| | tgresql.org |
| | |
| | |
| | 05/28/2004 02:51 AM |
| | |
|---------+---------------------------------->

------------------------------------------------------------------------------------------------------------------------------|

| |
| To: alvherre@dcc.uchile.cl |
| cc: pgsql-hackers@postgresql.org |
| Subject: Re: [HACKERS] Nested xacts: looking for testers and review |

------------------------------------------------------------------------------------------------------------------------------|

I have tested the patches with May 28 16:20 JST snapshot. Here is my
first impression:

1) errors, rules regression tests are failed (I'm not sure this is due
to your patches)

2) certain behavior was different from what I expected (please correct me
if my expectation is wrong).

test=# begin;
BEGIN
test=# insert into t1 values(1);
INSERT 17216 1
test=# begin;
BEGIN
test=# aaa;
ERROR: syntax error at or near "aaa" at character 1
ERROR: syntax error at or near "aaa" at character 1
LINE 1: aaa;
^
test=# end;
COMMIT
test=# select * from t1; <-- I thought this should work since
subtransaction was closed
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
test=# end;
COMMIT
test=# select * from t1;
i
---
(0 rows)

3) no docs found in the patches.

Hackers,

Ok, I've finally coded solutions to most problems regarding nested
transactions. This means:

- reversing for the lock manager, catcache, relcache, buffer manager,
asynchronous notifies, storage manager.

- transaction block state support, including appropiate XLog recording

- pg_subtrans subsystem (including changing state from SUBTRANS
COMMITTED to COMMITTED when appropiate). Also pg_clog XLog recovery
was handed to SLRU so pg_subtrans and pg_clog share a rmgr identity.

- visibility rules.

I'm missing one item: deferred triggers. The problem with this is that
the deftrig queue is not implemented using normal Lists, so there's no
efficient way to reassign to the parent when the subtransaction commits.
Also I'm not sure what should happen to the "immediate" pointer --- a
subtransaction should have it's own private copy, or it should inherit
the parent's? Please whoever implemented this speak up (Stephan
Szabo?), as I'm not sure of the semantics.

I have tested it and it passes all regression tests (including ones I
added), plus some more tests I threw at it mainly for concurrency.
Everything behaves as expected. At this time I'd like to have it
reviewed by the critic eye of the committers, and tested by whoever
would be using it.

I'm open for comments and suggestions and general input. Thank you.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
La web junta la gente porque no importa que clase de mutante sexual seas,
tienes millones de posibles parejas. Pon "buscar gente que tengan sexo

con

ciervos incendiánse", y el computador dirá "especifique el tipo de

ciervo"

(Jason Alexander)

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

*************************************************************************
PRIVILEGED AND CONFIDENTIAL: This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
*************************************************************************

This reply was meant to be directed to Alvaro.

|---------+---------------------------------->
| | Bob.Henkel@hartfordlife|
| | .com |
| | Sent by: |
| | pgsql-hackers-owner@pos|
| | tgresql.org |
| | |
| | |
| | 05/28/2004 07:45 AM |
| | |
|---------+---------------------------------->

------------------------------------------------------------------------------------------------------------------------------|

| |
| To: Tatsuo Ishii <t-ishii@sra.co.jp> |
| cc: alvherre@dcc.uchile.cl, pgsql-hackers@postgresql.org, pgsql-hackers-owner@postgresql.org |
| Subject: Re: [HACKERS] Nested xacts: looking for testers and review |

------------------------------------------------------------------------------------------------------------------------------|

I think before this can be tested fully by a wide audience there needs to
be some basic documentation. Or is there?
What is the new syntax?
Can we see some basic examples that you have used in your testing?

Regards
Bob

|---------+---------------------------------->
| | Tatsuo Ishii |
| | <t-ishii@sra.co.jp> |
| | Sent by: |
| | pgsql-hackers-owner@pos|
| | tgresql.org |
| | |
| | |
| | 05/28/2004 02:51 AM |
| | |
|---------+---------------------------------->

------------------------------------------------------------------------------------------------------------------------------|

|
|
| To: alvherre@dcc.uchile.cl
|
| cc: pgsql-hackers@postgresql.org
|
| Subject: Re: [HACKERS] Nested xacts: looking for testers and
review |

------------------------------------------------------------------------------------------------------------------------------|

I have tested the patches with May 28 16:20 JST snapshot. Here is my
first impression:

1) errors, rules regression tests are failed (I'm not sure this is due
to your patches)

2) certain behavior was different from what I expected (please correct me
if my expectation is wrong).

test=# begin;
BEGIN
test=# insert into t1 values(1);
INSERT 17216 1
test=# begin;
BEGIN
test=# aaa;
ERROR: syntax error at or near "aaa" at character 1
ERROR: syntax error at or near "aaa" at character 1
LINE 1: aaa;
^
test=# end;
COMMIT
test=# select * from t1; <-- I thought this should work since
subtransaction was closed
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
test=# end;
COMMIT
test=# select * from t1;
i
---
(0 rows)

3) no docs found in the patches.

Hackers,

Ok, I've finally coded solutions to most problems regarding nested
transactions. This means:

- reversing for the lock manager, catcache, relcache, buffer manager,
asynchronous notifies, storage manager.

- transaction block state support, including appropiate XLog recording

- pg_subtrans subsystem (including changing state from SUBTRANS
COMMITTED to COMMITTED when appropiate). Also pg_clog XLog recovery
was handed to SLRU so pg_subtrans and pg_clog share a rmgr identity.

- visibility rules.

I'm missing one item: deferred triggers. The problem with this is that
the deftrig queue is not implemented using normal Lists, so there's no
efficient way to reassign to the parent when the subtransaction commits.
Also I'm not sure what should happen to the "immediate" pointer --- a
subtransaction should have it's own private copy, or it should inherit
the parent's? Please whoever implemented this speak up (Stephan
Szabo?), as I'm not sure of the semantics.

I have tested it and it passes all regression tests (including ones I
added), plus some more tests I threw at it mainly for concurrency.
Everything behaves as expected. At this time I'd like to have it
reviewed by the critic eye of the committers, and tested by whoever
would be using it.

I'm open for comments and suggestions and general input. Thank you.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
La web junta la gente porque no importa que clase de mutante sexual seas,
tienes millones de posibles parejas. Pon "buscar gente que tengan sexo

con

ciervos incendiánse", y el computador dirá "especifique el tipo de

ciervo"

(Jason Alexander)

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

*************************************************************************
PRIVILEGED AND CONFIDENTIAL: This communication, including attachments, is
for the exclusive use of addressee and may contain proprietary,
confidential and/or privileged information. If you are not the intended
recipient, any use, copying, disclosure, dissemination or distribution is
strictly prohibited. If you are not the intended recipient, please notify
the sender immediately by return e-mail, delete this communication and
destroy all copies.
*************************************************************************

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

*************************************************************************
PRIVILEGED AND CONFIDENTIAL: This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies.
*************************************************************************

On Fri, May 28, 2004 at 04:51:07PM +0900, Tatsuo Ishii wrote:

2) certain behavior was different from what I expected (please correct me
if my expectation is wrong).

Yes, the expected behavior is different: if an aborted subtransaction is
closed with a COMMIT or END command, the parent transaction is aborted
too. This is to inhibit an application which blindly expects the
subtransaction to succeed to reach an invalid state. If you want to
return to non-aborted state, end the subtransaction with ROLLBACK
instead.

But Bob Henkel and you are right: there needo to be documentation.
Initially I figured I could do that later because there is no new
syntax, but it is obviously needed to explain all sorts of assumptions
and behavior like this.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"Lim�tate a mirar... y algun d�a veras"

On Fri, May 28, 2004 at 04:51:07PM +0900, Tatsuo Ishii wrote:

2) certain behavior was different from what I expected (please correct me
if my expectation is wrong).

Yes, the expected behavior is different: if an aborted subtransaction is
closed with a COMMIT or END command, the parent transaction is aborted
too. This is to inhibit an application which blindly expects the
subtransaction to succeed to reach an invalid state. If you want to
return to non-aborted state, end the subtransaction with ROLLBACK
instead.

Oh, I got it. Thanks.

But Bob Henkel and you are right: there needo to be documentation.
Initially I figured I could do that later because there is no new
syntax, but it is obviously needed to explain all sorts of assumptions
and behavior like this.

--
Tatsuo Ishii

Can I ask you one more question?

Is there any limit for nesting leveles of subtransactions?
--
Tatsuo Ishii

On Sat, May 29, 2004 at 12:27:39AM +0900, Tatsuo Ishii wrote:

Can I ask you one more question?

Is there any limit for nesting leveles of subtransactions?

In theory 2^16 I think, but I haven't tested it. It tried to 30 or so
only. Not sure if it's practical.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"Escucha y olvidar�s; ve y recordar�s; haz y entender�s" (Confucio)

Alvaro Herrera wrote:

On Fri, May 28, 2004 at 04:51:07PM +0900, Tatsuo Ishii wrote:

2) certain behavior was different from what I expected (please correct me
if my expectation is wrong).

Yes, the expected behavior is different: if an aborted subtransaction is
closed with a COMMIT or END command, the parent transaction is aborted
too. This is to inhibit an application which blindly expects the
subtransaction to succeed to reach an invalid state. If you want to
return to non-aborted state, end the subtransaction with ROLLBACK
instead.

I am interested to know if people agree with this behavior, reproduced below:

test=# begin;
BEGIN
test=# insert into t1 values(1);
INSERT 17216 1
test=# begin;
BEGIN
test=# aaa;
ERROR: syntax error at or near "aaa" at character 1
ERROR: syntax error at or near "aaa" at character 1
LINE 1: aaa;
^
test=# end;
COMMIT
test=# select * from t1; <-- I thought this should work since
subtransaction was closed
ERROR: current transaction is aborted, commands ignored until end of
transaction block
ERROR: current transaction is aborted, commands ignored until end of
transaction block
test=# end;
COMMIT

The problem I see with the behavior shown is that there is no way to use
subtransactions in scripts, where the queries can't be changed.
Consider this:

BEGIN;
DO SOME WORK...
BEGIN;
INSERT ...
COMMIT;
BEGIN;
INSERT ...
COMMIT;
BEGIN;
INSERT ...
COMMIT;
BEGIN;
INSERT ...
COMMIT;
DO MORE WORK...
COMMIT;

In this case, I want to try all of the inserts, but any of them can
fail, then I want the bottom part done.

I guess my big question is that if they issue a COMMIT for a
subtransaction that failed, do we assume they made a mistake and fail
the outer transaction, or do we just accept it and not affect the outer
transaction.

In my logic, the subtransaction COMMIT is part of the subtransaction and
should not affect the outer transaction's state.

Unfortunately, we don't have any similar behavior in our 7.4 code
because whether you issue COMMIT or ABORT, it does not affect the outer
session.

Do any other databases have nested transactions, and how to they handle
it?

I think we should issue a warning but not affect the outer transaction.

But Bob Henkel and you are right: there needs to be documentation.
Initially I figured I could do that later because there is no new
syntax, but it is obviously needed to explain all sorts of assumptions
and behavior like this.

Right now I think just posting examples will work fine. I think the
above case shows we are not ready for documentation yet. What I would
like is for folks to focus on testing so we can find any open issues
like this one and address them.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

On Fri, May 28, 2004 at 01:43:16PM -0400, Bruce Momjian wrote:

In this case, I want to try all of the inserts, but any of them can
fail, then I want the bottom part done.

I wonder where everyone eas when I asked this question a lot of time
ago. I said I thought the behavior should be like I described, and no
one objected.

Personally I think it would be a mistake to allow the COMMIT for the
subtransaction to ignore the fact that the subxact was aborted. However
I realize what you are proposing, and maybe this can be implemented
using a parameter to COMMIT (indicating to not propagate the error if
it's in aborted state, but commit normally otherwise).

However if everyone disagrees, I can take that part out, and the code
would be simpler. IMHO however, it would be less reliable.

In my logic, the subtransaction COMMIT is part of the subtransaction and
should not affect the outer transaction's state.

In some cases yes, but not all. In others, the outer transaction could
trust that the inner one worked; to make the example you posted work,
I'd use a program rather than a script, and check the return values (or
the transaction state). If the subxact is in aborted state, issue
ROLLBACK and try again; if not, commit.

Unfortunately, we don't have any similar behavior in our 7.4 code
because whether you issue COMMIT or ABORT, it does not affect the outer
session.

Of course. This is new functionality.

Right now I think just posting examples will work fine. I think the
above case shows we are not ready for documentation yet. What I would
like is for folks to focus on testing so we can find any open issues
like this one and address them.

Ok.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"In Europe they call me Niklaus Wirth; in the US they call me Nickel's worth.
That's because in Europe they call me by name, and in the US by value!"

Alvaro Herrera wrote:

On Fri, May 28, 2004 at 01:43:16PM -0400, Bruce Momjian wrote:

In this case, I want to try all of the inserts, but any of them can
fail, then I want the bottom part done.

I wonder where everyone eas when I asked this question a lot of time
ago. I said I thought the behavior should be like I described, and no
one objected.

Sorry, I didn't understand the question at the time, or wasn't paying
attention.

Personally I think it would be a mistake to allow the COMMIT for the
subtransaction to ignore the fact that the subxact was aborted. However
I realize what you are proposing, and maybe this can be implemented
using a parameter to COMMIT (indicating to not propagate the error if
it's in aborted state, but commit normally otherwise).

However if everyone disagrees, I can take that part out, and the code
would be simpler. IMHO however, it would be less reliable.

Imagine this case used in a script:

BEGIN;
DROP TABLE test;
CREATE TABLE test(x int);
COMMIT;

This will not work because the drop might fail. However you could use this:

BEGIN;
BEGIN;
DROP TABLE test;
COMMIT;
CREATE TABLE test(x int);
COMMIT;

It is done in a transaction so the table replace is an atomic operation.

One interesting idea would be for COMMIT to affect the outer
transaction, and END not affect the outer transaction. Of course that
kills the logic that COMMIT and END are the same, but it is an
interesting idea, and doesn't affect backward compatibility because
END/COMMIT behave the same in non-nested transactions.

If this is the type of issue we are dealing with for the patch, I feel
very good. Good job Alvaro.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

On Fri, May 28, 2004 at 04:05:40PM -0400, Bruce Momjian wrote:

Hm, you are right that there needs to be a more automatic way of doing
this.

One interesting idea would be for COMMIT to affect the outer
transaction, and END not affect the outer transaction. Of course that
kills the logic that COMMIT and END are the same, but it is an
interesting idea, and doesn't affect backward compatibility because
END/COMMIT behave the same in non-nested transactions.

How about "COMMIT SUB" and "END SUB"? I don't feel it's good to give
different meaning to COMMIT versus END, but this is only a gut kind of
thing and I could be convinced otherwise. It is even easier to
differentiate COMMIT/END than adding a parameter to them.

I mean, COMMIT SUB would not affect the state of the outer transaction,
while COMMIT would.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"In fact, the basic problem with Perl 5's subroutines is that they're not
crufty enough, so the cruft leaks out into user-defined code instead, by
the Conservation of Cruft Principle." (Larry Wall, Apocalypse 6)

On Wed, 26 May 2004, Alvaro Herrera wrote:

I have tested it and it passes all regression tests (including ones I
added), plus some more tests I threw at it mainly for concurrency.
Everything behaves as expected. At this time I'd like to have it
reviewed by the critic eye of the committers, and tested by whoever
would be using it.

I'm open for comments and suggestions and general input. Thank you.

I unfortunately didn't really follow the discussions in the past (sorry :(
), but are the transaction state modifying statements done in a
subtransaction supposed to live beyond subtransaction rollback?

For example,
sszabo=# begin;
BEGIN
sszabo=# begin;
BEGIN
sszabo=# set transaction read only;
SET
sszabo=# select * from a;
a
---
6
(1 row)

sszabo=# rollback;
ROLLBACK
sszabo=# update a set a=6;
ERROR: transaction is read-only

On Fri, May 28, 2004 at 05:43:41PM -0700, Stephan Szabo wrote:

On Wed, 26 May 2004, Alvaro Herrera wrote:

I have tested it and it passes all regression tests (including ones I
added), plus some more tests I threw at it mainly for concurrency.
Everything behaves as expected. At this time I'd like to have it
reviewed by the critic eye of the committers, and tested by whoever
would be using it.

I unfortunately didn't really follow the discussions in the past (sorry :(
), but are the transaction state modifying statements done in a
subtransaction supposed to live beyond subtransaction rollback?

Hmm, I suppose not.

I think this applies to all GUC variables, but I wonder if we want to
save the value of each one at subtransaction start and recover it at
abort? Things could easily get huge. Maybe only saving the ones that
are different from the default value, and from the last saved value.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"C�mo ponemos nuestros dedos en la arcilla del otro. Eso es la amistad; jugar
al alfarero y ver qu� formas se pueden sacar del otro" (C. Halloway en
La Feria de las Tinieblas, R. Bradbury)

Alvaro Herrera wrote:

On Fri, May 28, 2004 at 05:43:41PM -0700, Stephan Szabo wrote:

On Wed, 26 May 2004, Alvaro Herrera wrote:

I have tested it and it passes all regression tests (including ones I
added), plus some more tests I threw at it mainly for concurrency.
Everything behaves as expected. At this time I'd like to have it
reviewed by the critic eye of the committers, and tested by whoever
would be using it.

I unfortunately didn't really follow the discussions in the past (sorry :(
), but are the transaction state modifying statements done in a
subtransaction supposed to live beyond subtransaction rollback?

Hmm, I suppose not.

I think this applies to all GUC variables, but I wonder if we want to
save the value of each one at subtransaction start and recover it at
abort? Things could easily get huge. Maybe only saving the ones that
are different from the default value, and from the last saved value.

We have an on-commit field in the guc structures to handle
commit/rollback settings. Do we need to extend that to subtransactions?

I don't think you can save off only the defaults in an efficient manner.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

On Fri, May 28, 2004 at 11:11:27PM -0400, Bruce Momjian wrote:

Alvaro Herrera wrote:

I think this applies to all GUC variables, but I wonder if we want to
save the value of each one at subtransaction start and recover it at
abort? Things could easily get huge. Maybe only saving the ones that
are different from the default value, and from the last saved value.

We have an on-commit field in the guc structures to handle
commit/rollback settings. Do we need to extend that to subtransactions?

Yes IMHO. I'm not sure actually _how_ should this be handled. Maybe
the on-commit field should go away and be replaced by something more
general (probably a stack, like everything else). Let me look at the
code.

I don't think you can save off only the defaults in an efficient manner.

What do you mean by efficient? Space efficient? It will be much more
efficient to save only the changed values. Time efficient? We will
have to traverse the whole list anyway, whether we only save the changed
values or all of them.

Remember, we already traverse the whole list of shared buffers, the
whole CatCache, the whole Relcache, maybe do some repallocs, and lots of
other stuff. Traversing the whole GUC array does not seem all that
expensive to me ... After all, we are saving lots of I/O if subxacts
are used correctly (try, rollback, try again -- you save the XLog for
the first try.)

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"Oh, great altar of passive entertainment, bestow upon me thy discordant images
at such speed as to render linear thought impossible" (Calvin a la TV)

Alvaro Herrera wrote:

On Fri, May 28, 2004 at 11:11:27PM -0400, Bruce Momjian wrote:

Alvaro Herrera wrote:

I think this applies to all GUC variables, but I wonder if we want to
save the value of each one at subtransaction start and recover it at
abort? Things could easily get huge. Maybe only saving the ones that
are different from the default value, and from the last saved value.

We have an on-commit field in the guc structures to handle
commit/rollback settings. Do we need to extend that to subtransactions?

Yes IMHO. I'm not sure actually _how_ should this be handled. Maybe
the on-commit field should go away and be replaced by something more
general (probably a stack, like everything else). Let me look at the
code.

I don't think you can save off only the defaults in an efficient manner.

What do you mean by efficient? Space efficient? It will be much more
efficient to save only the changed values. Time efficient? We will
have to traverse the whole list anyway, whether we only save the changed
values or all of them.

Remember, we already traverse the whole list of shared buffers, the
whole CatCache, the whole Relcache, maybe do some repallocs, and lots of
other stuff. Traversing the whole GUC array does not seem all that
expensive to me ... After all, we are saving lots of I/O if subxacts
are used correctly (try, rollback, try again -- you save the XLog for
the first try.)

My comment was based on the fact that guc already does some special
_saves_ when you change a value and triggers some stuff on xact end. I
was just thinking it would be cleaner to use that infrastructure rather
than do a scan not knowing if any GUC will change or not, but if a scan
is easier, I think that would be fine.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073