XTM & parallel search
We have to add three more functions to eXtensible Transaction Manager API (XTM):
/*
* Calculate transaction state size. This method is invoked by EstimateTransactionStateSpace to copy transaction
* state to parallel workers
*/
size_t (*GetTransactionStateSize)(void);
/*
* Serialize transaction state
*/
void (*SerializeTransactionState)(void* ctx);
/*
* Deserialize transaction state
*/
void (*DeserializeTransactionState)(void* ctx);
The reason is that we find out that our multimaster is not correctly working when max_parallel_workers > 0
because multimaster transaction context is not properly shared between workers.
Unfortunately right now serialization/deserialization of transaction state is hardcoded in xact.c. and IMHO is done in quite ugly way:
XactIsoLevel = (int) tstate[0];
XactDeferrable = (bool) tstate[1];
XactTopTransactionId = tstate[2];
CurrentTransactionState->transactionId = tstate[3];
currentCommandId = tstate[4];
nParallelCurrentXids = (int) tstate[5];
ParallelCurrentXids = &tstate[6];
- there is even no declared structure with fixed part of saved context.
I wonder if not only DTM will be interested in sharing some common state between workers and should we provide some way of replicating user defined context between workers? From my point of view XTM seems to be good place for it...
--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Jun 3, 2016 at 1:34 AM, Konstantin Knizhnik <
k.knizhnik@postgrespro.ru> wrote:
We have to add three more functions to eXtensible Transaction Manager API
(XTM):/*
* Calculate transaction state size. This method is invoked by
EstimateTransactionStateSpace to copy transaction
* state to parallel workers
*/
size_t (*GetTransactionStateSize)(void);/*
* Serialize transaction state
*/
void (*SerializeTransactionState)(void* ctx);/*
* Deserialize transaction state
*/
void (*DeserializeTransactionState)(void* ctx);
In above proposal, are you suggesting to change the existing API's as well,
because the parameters of function pointers don't match with exiting API's.
I think it is better to consider this along with the overall XTM API.
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
On 03.06.2016 16:05, Amit Kapila wrote:
On Fri, Jun 3, 2016 at 1:34 AM, Konstantin Knizhnik
<k.knizhnik@postgrespro.ru <mailto:k.knizhnik@postgrespro.ru>> wrote:We have to add three more functions to eXtensible Transaction
Manager API (XTM):/*
* Calculate transaction state size. This method is invoked by
EstimateTransactionStateSpace to copy transaction
* state to parallel workers
*/
size_t (*GetTransactionStateSize)(void);/*
* Serialize transaction state
*/
void (*SerializeTransactionState)(void* ctx);/*
* Deserialize transaction state
*/
void (*DeserializeTransactionState)(void* ctx);In above proposal, are you suggesting to change the existing API's as
well, because the parameters of function pointers don't match with
exiting API's. I think it is better to consider this along with the
overall XTM API.
Sorry, but right now I have not replaced existed functions
EstimateTransactionStateSpace/SerializeTransactionState/StartParallelWorkerTransaction
with XTM indirect calls. If was my original intention, but these
functions access static variable CurrentTransactionState defined in xact.c.
So if user-defined TM wants to override this functions, it will have to
invoke original functions to save/restore CurrentTransactionState.
It is not convenient.
This is why three XTM functions above are now called by existed xact
funcations to save additional state, for example:
Size
EstimateTransactionStateSpace(void)
{
TransactionState s;
Size nxids = 6; /* iso level, deferrable, top &
current XID,
* command counter, XID count */
for (s = CurrentTransactionState; s != NULL; s = s->parent)
{
if (TransactionIdIsValid(s->transactionId))
nxids = add_size(nxids, 1);
nxids = add_size(nxids, s->nChildXids);
}
nxids = add_size(nxids, nParallelCurrentXids);
nxids = mul_size(nxids, sizeof(TransactionId));
return add_size(nxids, TM->GetTransactionStateSize());
}
--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
On Fri, Jun 3, 2016 at 6:49 PM, Konstantin Knizhnik <
k.knizhnik@postgrespro.ru> wrote:
On 03.06.2016 16:05, Amit Kapila wrote:
On Fri, Jun 3, 2016 at 1:34 AM, Konstantin Knizhnik <
<k.knizhnik@postgrespro.ru>k.knizhnik@postgrespro.ru> wrote:We have to add three more functions to eXtensible Transaction Manager API
(XTM):/*
* Calculate transaction state size. This method is invoked by
EstimateTransactionStateSpace to copy transaction
* state to parallel workers
*/
size_t (*GetTransactionStateSize)(void);/*
* Serialize transaction state
*/
void (*SerializeTransactionState)(void* ctx);/*
* Deserialize transaction state
*/
void (*DeserializeTransactionState)(void* ctx);In above proposal, are you suggesting to change the existing API's as
well, because the parameters of function pointers don't match with exiting
API's. I think it is better to consider this along with the overall XTM API.Sorry, but right now I have not replaced existed functions
EstimateTransactionStateSpace/SerializeTransactionState/StartParallelWorkerTransaction
with XTM indirect calls. If was my original intention, but these functions
access static variable CurrentTransactionState defined in xact.c.
So if user-defined TM wants to override this functions, it will have to
invoke original functions to save/restore CurrentTransactionState.
It is not convenient.
This is why three XTM functions above are now called by existed xact
funcations to save additional state, for example:
Sounds reasonable, although one might want to have a hook for
EndParallelWorkerTransaction as well. I think here larger question is
whether there is a consensus on getting XTM API in core.
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com