Add new function to convert 32-bit XID to 64-bit

Started by Fujii Masaoabout 4 years ago2 messages
#1Fujii Masao
masao.fujii@oss.nttdata.com

Hi,

When periodically collecting and accumulating statistics or status information like pg_locks, pg_stat_activity, pg_prepared_xacts, etc for future troubleshooting or some reasons, I'd like to store a transaction ID of such information as 64-bit version so that the information of specified transaction easily can be identified and picked up by transaction ID. Otherwise it's not easy to distinguish transactions with the same 32-bit XID but different epoch, only by 32-bit XID.

But since pg_locks or pg_stat_activity etc returns 32-bit XID, we could not store it as 64-bit version. To improve this situation, I'd like to propose to add new function that converts the given 32-bit XID (i.e., xid data type) to 64-bit one (xid8 data type). If we do this, for example we can easily get 64-bit XID from pg_stat_activity by "SELECT convert_xid_to_xid8(backend_xid) FROM pg_stat_activity", if necessary. Thought?

As another approach, we can change the data type of pg_stat_activity.backend_xid or pg_locks.transaction, etc from xid to xid8. But this idea looks overkill to me, and it may break the existing applications accessing pg_locks etc. So IMO it's better to just provide the convertion function.

Regards,

--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

#2Ashutosh Bapat
ashutosh.bapat.oss@gmail.com
In reply to: Fujii Masao (#1)
Re: Add new function to convert 32-bit XID to 64-bit

On Thu, Dec 23, 2021 at 8:33 PM Fujii Masao <masao.fujii@oss.nttdata.com> wrote:

Hi,

When periodically collecting and accumulating statistics or status information like pg_locks, pg_stat_activity, pg_prepared_xacts, etc for future troubleshooting or some reasons, I'd like to store a transaction ID of such information as 64-bit version so that the information of specified transaction easily can be identified and picked up by transaction ID. Otherwise it's not easy to distinguish transactions with the same 32-bit XID but different epoch, only by 32-bit XID.

But since pg_locks or pg_stat_activity etc returns 32-bit XID, we could not store it as 64-bit version. To improve this situation, I'd like to propose to add new function that converts the given 32-bit XID (i.e., xid data type) to 64-bit one (xid8 data type). If we do this, for example we can easily get 64-bit XID from pg_stat_activity by "SELECT convert_xid_to_xid8(backend_xid) FROM pg_stat_activity", if necessary. Thought?

What will this function do?

From your earlier description it looks like you want this function to
add epoch to the xid when making it a 64bit value. Is that right?

As another approach, we can change the data type of pg_stat_activity.backend_xid or pg_locks.transaction, etc from xid to xid8. But this idea looks overkill to me, and it may break the existing applications accessing pg_locks etc. So IMO it's better to just provide the convertion function.

Obviously we will break backward compatibility for applications
upgrading to a newer PostgreSQL version. Downside is applications
using 64bit xid will need to change their applications.

If we want to change the datatype anyway, better to create a new type
LongTransactionId or something like to represent 64bit transaction id
and then change these functions to use that.

--
Best Wishes,
Ashutosh Bapat