Using a latch between a background worker process and a thread

On Tue, Nov 1, 2016 at 12:35 PM, Abbas Butt <abbas.butt@enterprisedb.com> wrote:

Hi,
Consider this situation:
1. I have a background worker process.
2. The process creates a latch, initializes it using InitLatch & resets it.
3. It then creates a thread and passes the latch created in step 2 to it.
To pass it, the process uses the last argument of pthread_create.
4. The thread blocks by calling WaitLatch.
5. The process after some time sets the latch using SetLatch.

The thread does not notice that the latch has been set and keeps waiting.

My question is:
Are latches supposed to work between a process and a thread created by that
process?

Nothing in the entire backend is guaranteed to work if you spawn
multiple threads within the same process.

Including this.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2 November 2016 at 02:10, Robert Haas <robertmhaas@gmail.com> wrote:

On Tue, Nov 1, 2016 at 12:35 PM, Abbas Butt <abbas.butt@enterprisedb.com> wrote:

Hi,
Consider this situation:
1. I have a background worker process.
2. The process creates a latch, initializes it using InitLatch & resets it.
3. It then creates a thread and passes the latch created in step 2 to it.
To pass it, the process uses the last argument of pthread_create.
4. The thread blocks by calling WaitLatch.
5. The process after some time sets the latch using SetLatch.

The thread does not notice that the latch has been set and keeps waiting.

My question is:
Are latches supposed to work between a process and a thread created by that
process?

Nothing in the entire backend is guaranteed to work if you spawn
multiple threads within the same process.

Including this.

Yep.

You could have the main thread wait on the latch, then signal the
other threads via appropriate pthread primitives. But you must ensure
your other threads do nothing that calls into backend code. Including
things like atexit handlers. They need to coordinate with the main
thread to do everything PostgreSQL related, and you'd need to make
sure the main thread handles all signals. That's the default for Linux
- the main thread gets first chance at all signals and other threads'
sigmasks are only checked if the main thread has masked the signal,
but that means your other threads should be sure to mask all signals
used by PostgreSQL. Good luck doing that portably.

There are exceptions where you can call some backend functions and
macros from other threads. But you'd have to analyse each on a case by
case basis, and there's no guarantee they'll _stay_ safe.

I'd just avoid using threads in the backend if at all possible.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Thanks every body for the detailed advise.
Let me try replacing latches by condition variables.
I will report the results here.

On Wed, Nov 2, 2016 at 11:54 AM, Craig Ringer <craig@2ndquadrant.com> wrote:

On 2 November 2016 at 02:10, Robert Haas <robertmhaas@gmail.com> wrote:

On Tue, Nov 1, 2016 at 12:35 PM, Abbas Butt <abbas.butt@enterprisedb.com>

wrote:

Hi,
Consider this situation:
1. I have a background worker process.
2. The process creates a latch, initializes it using InitLatch & resets

it.

3. It then creates a thread and passes the latch created in step 2 to

it.

To pass it, the process uses the last argument of pthread_create.
4. The thread blocks by calling WaitLatch.
5. The process after some time sets the latch using SetLatch.

The thread does not notice that the latch has been set and keeps

waiting.

My question is:
Are latches supposed to work between a process and a thread created by

that

process?

Nothing in the entire backend is guaranteed to work if you spawn
multiple threads within the same process.

Including this.

Yep.

You could have the main thread wait on the latch, then signal the
other threads via appropriate pthread primitives. But you must ensure
your other threads do nothing that calls into backend code. Including
things like atexit handlers. They need to coordinate with the main
thread to do everything PostgreSQL related, and you'd need to make
sure the main thread handles all signals. That's the default for Linux
- the main thread gets first chance at all signals and other threads'
sigmasks are only checked if the main thread has masked the signal,
but that means your other threads should be sure to mask all signals
used by PostgreSQL. Good luck doing that portably.

There are exceptions where you can call some backend functions and
macros from other threads. But you'd have to analyse each on a case by
case basis, and there's no guarantee they'll _stay_ safe.

I'd just avoid using threads in the backend if at all possible.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
--
*Abbas*
Architect

Ph: 92.334.5100153
Skype ID: gabbasb
www.enterprisedb.co <http://www.enterprisedb.com/&gt;m
<http://www.enterprisedb.com/&gt;

*Follow us on Twitter*
@EnterpriseDB

Visit EnterpriseDB for tutorials, webinars, whitepapers
<http://www.enterprisedb.com/resources-community&gt; and more
<http://www.enterprisedb.com/resources-community&gt;