Initializing LWLock Array from Server Code

Hi Robert,

Thank you for your reply and sorry that I couldn't reply earlier.
Since I didn't get any response within a couple of days, I took the longer
route -- changed the lwlock.h and lwlock.c
for accommodating the lw locks for the shared hash table.

I'll describe what I modified in the lwlock.h and lwlock.c and it seems to
be working fine. Although I haven't got an opportunity
to test it extensively. If you could let me know if I missed out anything
that might cause problems later that would be great.

In lwlock.h, the following modifications were made:

*/* the number of partitions of the shared hash table */*
#define NUM_FILTERHASH_PARTITIONS 64

*/* Offsets for various chunks of preallocated lwlocks. */*
#define BUFFER_MAPPING_LWLOCK_OFFSET NUM_INDIVIDUAL_LWLOCKS
#define LOCK_MANAGER_LWLOCK_OFFSET \
(BUFFER_MAPPING_LWLOCK_OFFSET + NUM_BUFFER_PARTITIONS)
#define PREDICATELOCK_MANAGER_LWLOCK_OFFSET \
(LOCK_MANAGER_LWLOCK_OFFSET + NUM_LOCK_PARTITIONS)

*/* offset for shared filterhash lwlocks in the MainLWLockArray */*
#define FILTERHASH_OFFSET \
(PREDICATELOCK_MANAGER_LWLOCK_OFFSET + NUM_PREDICATELOCK_PARTITIONS)
*/* modified NUM_FIXED_LOCKS */*
#define NUM_FIXED_LWLOCKS \
(FILTERHASH_OFFSET + NUM_FILTERHASH_PARTITIONS)

*/* added LWTRANCHE_FILTERHASH in the BuiltinTrancheIds */*
typedef enum BuiltinTrancheIds
{
LWTRANCHE_CLOG_BUFFERS = NUM_INDIVIDUAL_LWLOCKS,
LWTRANCHE_COMMITTS_BUFFERS,
LWTRANCHE_SUBTRANS_BUFFERS,
LWTRANCHE_MXACTOFFSET_BUFFERS,
LWTRANCHE_MXACTMEMBER_BUFFERS,
LWTRANCHE_ASYNC_BUFFERS,
LWTRANCHE_OLDSERXID_BUFFERS,
LWTRANCHE_WAL_INSERT,
LWTRANCHE_BUFFER_CONTENT,
LWTRANCHE_BUFFER_IO_IN_PROGRESS,
LWTRANCHE_REPLICATION_ORIGIN,
LWTRANCHE_REPLICATION_SLOT_IO_IN_PROGRESS,
LWTRANCHE_PROC,
LWTRANCHE_BUFFER_MAPPING,
LWTRANCHE_LOCK_MANAGER,
LWTRANCHE_PREDICATE_LOCK_MANAGER,
*LWTRANCHE_FILTERHASH*,
LWTRANCHE_PARALLEL_HASH_JOIN,
LWTRANCHE_PARALLEL_QUERY_DSA,
LWTRANCHE_SESSION_DSA,
LWTRANCHE_SESSION_RECORD_TABLE,
LWTRANCHE_SESSION_TYPMOD_TABLE,
LWTRANCHE_SHARED_TUPLESTORE,
LWTRANCHE_TBM,
LWTRANCHE_PARALLEL_APPEND,
LWTRANCHE_FIRST_USER_DEFINED
} BuiltinTrancheIds;

In lwlock.c, the following modifications were made:

In function, static void InitializeLWLocks(void), the following lines were
added:

*/* Initialize filterhash LWLocks in main array */*
lock = MainLWLockArray + NUM_INDIVIDUAL_LWLOCKS +
NUM_BUFFER_PARTITIONS + NUM_LOCK_PARTITIONS +
NUM_FILTERHASH_PARTITIONS;
for (id = 0; id < NUM_FILTERHASH_PARTITIONS; id++, lock++)
LWLockInitialize(&lock->lock, LWTRANCHE_FILTERHASH);

In function void RegisterLWLockTranches(void), the following line was added:

LWLockRegisterTranche(LWTRANCHE_FILTERHASH, "filter_hash");

All of this was done after allocating the required of space in the shared
memory.

Thanks,
-SB

On Mon, Apr 29, 2019 at 1:59 PM Robert Haas <robertmhaas@gmail.com> wrote:

On Tue, Apr 30, 2019 at 5:52 PM Souvik Bhattacherjee <kivuosb@gmail.com> wrote:

Thank you for your reply and sorry that I couldn't reply earlier.
Since I didn't get any response within a couple of days, I took the longer route -- changed the lwlock.h and lwlock.c
for accommodating the lw locks for the shared hash table.

I'll describe what I modified in the lwlock.h and lwlock.c and it seems to be working fine. Although I haven't got an opportunity
to test it extensively. If you could let me know if I missed out anything that might cause problems later that would be great.

Well, I can't really vouch for your code on a quick look, but it
doesn't look insane.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Replying to myself to resend to the list, since my previous attempt
seems to have been eaten by a grue.

On Mon, Apr 29, 2019 at 1:59 PM Robert Haas <robertmhaas@gmail.com> wrote:

On Fri, Apr 26, 2019 at 2:58 PM Souvik Bhattacherjee <kivuosb@gmail.com> wrote:

I have created a shared hash table in partitioned mode inside the postgres server code. In order to guard the partitions, I'm trying to initialize an array of LWLocks. The code that I'm trying to use for that is

void RequestNamedLWLockTranche(const char *tranche_name, int num_lwlocks);
LWLockPadded *GetNamedLWLockTranche(const char *tranche_name);

I'm not sure where exactly should this code be called from the server code. So I had placed it in

void CreateSharedMemoryAndSemaphores(bool makePrivate, int port);

within ipic.c. However, I'm getting the following error message when starting the database:

FATAL: requested tranche is not registered

So at this point, I'm a little confused as to where the methods should be called from inside the server code. Any pointers would be appreciated.

RequestNamedLWLockTranche() changes the behavior of LWLockShmemSize()
and CreateLWLocks(), so must be called before either of those.
GetNamedLWLockTranche() must be called after CreateLWLocks().

This machinery works for pg_stat_statements, so see that as an example
of how to make this work from an extension. If you're hacking the
core server code, then look at the places where the corresponding bits
of pg_stat_statements code get called. IIRC, _PG_init() gets called
from process_shared_preload_libraries(), so you might look at the
placement of the calls to that function.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company