How to kill a Background worker and Its metadata

Started by Akash Agrawalover 9 years ago6 messages
#1Akash Agrawal
aagrawa6@ncsu.edu

Hi,

I've created a background worker and I am using Postgresql-9.4. This
bgworker handles the job queue dynamically and goes to sleep if there is no
job to process within the next 1 hour.

Now, I want to have a mechanism to wake the bgworker up in case if someone
adds a new job while the bgworker is in sleep mode. So to do it, I have
created a trigger which initially removes the existing background worker
and then registers a new one. I am using the following two queries inside
it:

select pg_terminate_backend(pid of bgworker);
select worker_spi1_launch(1);

I am retrieving the pid from pg_Stat_activity. The maximum number of
background worker that can run simultaneously is equal to 8. I think even
if I call pg_terminate_backend the metadata of the background worker is not
being deleted and as a result after 8 insert operations I am not able to
register a background worker. Please suggest what can be done here.

Best,
Akash

#2Michael Paquier
michael.paquier@gmail.com
In reply to: Akash Agrawal (#1)
Re: How to kill a Background worker and Its metadata

On Tue, Jun 28, 2016 at 3:27 AM, Akash Agrawal <aagrawa6@ncsu.edu> wrote:

I've created a background worker and I am using Postgresql-9.4. This
bgworker handles the job queue dynamically and goes to sleep if there is no
job to process within the next 1 hour.

Now, I want to have a mechanism to wake the bgworker up in case if someone
adds a new job while the bgworker is in sleep mode. So to do it, I have
created a trigger which initially removes the existing background worker and
then registers a new one. I am using the following two queries inside it:

Why don't you just register and use a signal in this case? You could
even do something with SIGHUP...
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Akash Agrawal
aagrawa6@ncsu.edu
In reply to: Michael Paquier (#2)
Re: How to kill a Background worker and Its metadata

I've handled SIGTERM signal. pg_terminate_backend send signals (SIGTERM) to
backend processes identified by process ID. And also, after this call I am
able to track in my logs that the background worker gets terminated.

Yet, I am only able to register first 8 background workers. I am using
select worker_spi1_launch(1) to launch it every time. This is why I guess
there is some metadata maintained which has got to be deleted.

On Mon, Jun 27, 2016 at 7:59 PM, Michael Paquier <michael.paquier@gmail.com>
wrote:

Show quoted text

On Tue, Jun 28, 2016 at 3:27 AM, Akash Agrawal <aagrawa6@ncsu.edu> wrote:

I've created a background worker and I am using Postgresql-9.4. This
bgworker handles the job queue dynamically and goes to sleep if there is

no

job to process within the next 1 hour.

Now, I want to have a mechanism to wake the bgworker up in case if

someone

adds a new job while the bgworker is in sleep mode. So to do it, I have
created a trigger which initially removes the existing background worker

and

then registers a new one. I am using the following two queries inside it:

Why don't you just register and use a signal in this case? You could
even do something with SIGHUP...
--
Michael

#4Craig Ringer
craig@2ndquadrant.com
In reply to: Akash Agrawal (#1)
Re: How to kill a Background worker and Its metadata

On 28 June 2016 at 02:27, Akash Agrawal <aagrawa6@ncsu.edu> wrote:

Hi,

I've created a background worker and I am using Postgresql-9.4. This
bgworker handles the job queue dynamically and goes to sleep if there is no
job to process within the next 1 hour.

Now, I want to have a mechanism to wake the bgworker up in case if someone
adds a new job while the bgworker is in sleep mode. So to do it, I have
created a trigger which initially removes the existing background worker
and then registers a new one.

Don't do that.

Instead, set the background worker's process latch, which you can find in
the PGPROC array. There are a variety of examples of how to set latches in
the sources.

I am retrieving the pid from pg_Stat_activity. The maximum number of
background worker that can run simultaneously is equal to 8. I think even
if I call pg_terminate_backend the metadata of the background worker is not
being deleted

Correct. Unless you register it as a dynamic bgworker with no automatic
restart, it'll get restarted when it exits uncleanly.

Have the worker call proc_exit(0) if you want it not to be restarted.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

#5Akash Agrawal
aagrawa6@ncsu.edu
In reply to: Craig Ringer (#4)
1 attachment(s)
Re: How to kill a Background worker and Its metadata

I am calling proc_exit(1) once the worker encounters SIGTERM signal. I've
attached my code here.

Here is the link to stackoverflow:
http://stackoverflow.com/questions/38058628/how-to-kill-a-background-worker-including-its-metadata-in-postgres

On Mon, Jun 27, 2016 at 8:41 PM, Craig Ringer <craig@2ndquadrant.com> wrote:

Show quoted text

On 28 June 2016 at 02:27, Akash Agrawal <aagrawa6@ncsu.edu> wrote:

Hi,

I've created a background worker and I am using Postgresql-9.4. This
bgworker handles the job queue dynamically and goes to sleep if there is no
job to process within the next 1 hour.

Now, I want to have a mechanism to wake the bgworker up in case if
someone adds a new job while the bgworker is in sleep mode. So to do it, I
have created a trigger which initially removes the existing background
worker and then registers a new one.

Don't do that.

Instead, set the background worker's process latch, which you can find in
the PGPROC array. There are a variety of examples of how to set latches in
the sources.

I am retrieving the pid from pg_Stat_activity. The maximum number of
background worker that can run simultaneously is equal to 8. I think even
if I call pg_terminate_backend the metadata of the background worker is not
being deleted

Correct. Unless you register it as a dynamic bgworker with no automatic
restart, it'll get restarted when it exits uncleanly.

Have the worker call proc_exit(0) if you want it not to be restarted.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Attachments:

bgworker.rtfapplication/rtf; name=bgworker.rtfDownload
#6Craig Ringer
craig@2ndquadrant.com
In reply to: Akash Agrawal (#3)
Re: How to kill a Background worker and Its metadata

On 28 June 2016 at 08:28, Akash Agrawal <aagrawa6@ncsu.edu> wrote:

I've handled SIGTERM signal. pg_terminate_backend send signals (SIGTERM)
to backend processes identified by process ID. And also, after this call I
am able to track in my logs that the background worker gets terminated.

Yet, I am only able to register first 8 background workers. I am using
select worker_spi1_launch(1) to launch it every time. This is why I guess
there is some metadata maintained which has got to be deleted.

(Please reply below other posts, not above)

The bgworker API currently offers no way to enumerate bgworkers or
unregister them from the outside. The only way to unregister a dynamic
bgworker is to:

* proc_exit(0) from within the worker; or

* register it with BGW_NO_RESTART so it doesn't auto-restart in the
first place.

This is a deficiency in the bgworker API, but there are workarounds in
place and other things are more important for now. Just make your your
workers proc_exit(0) on SIGTERM or don't register them as auto-restarting.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services