Feature proposal: distinguish each PostgreSQL instance in the event log

On 03/22/2011 08:22 AM, MauMau wrote:

Hello,

I have several software products which use PostgreSQL as a data
repository and embed the same PostgreSQL binaries. Currently, those
software support Linux. I'm trying to port them to Windows.

I've encountered one problem on Windows. I need to support running all
of my products on one host simultaneously. Plus, I need to log
messages in syslog/event log. On Linux, I can distinguish the messages
of one product and those of other products by setting syslog_ident in
postgresql.conf. On the other hand, I can't do that on Windows: all of
the PostgreSQL instances use the same fixed event source name
"PostgreSQL".

SO I'd like to propose a trivial feature which allows users to set
event source. As I'm a newbie and this is my first proposal, I might
behave inappropriately in the development process. Please point out if
you notice I should act better. I've read the following site.

Development information
http://wiki.postgresql.org/wiki/Developer_and_Contributor_Resources

Functional specification
========================================

[Current usage]
To enable event logging:
1. Register the event source "PostgreSQL" with the registry:
c:\> regsvr32.exe <path of pgevent.dll>
2. Add 'eventlog' to log_destination parameter.

To disable event logging:
1. Remove 'eventlog' from log_destination parameter.
2. Deregister the event source "PostgreSQL" from the registry.
c:\> regsvr32.exe /u <path of pgevent.dll>

[New usage]
To use the default event source "PostgreSQL", the usage is the same as
now. To use other event sources, add a bit of information.

To enable event logging:
1. Register the event source with the registry:
c:\> regsvr32.exe /i:<event source> <path of pgevent.dll>
ex. To register "MyApp" event source:
c:\> regsvr32.exe /i:"MyApp" pgevent.dll
2. Add 'eventlog' to log_destination parameter.
3. Name the event source with the new "event_source" parameter of
postgresql.conf.
ex.
event_source = 'MyApp'

To disable event logging:
1. Remove event_source parameter.
2. Remove 'eventlog' from log_destination parameter.
3. Deregister the event source from the registry.
c:\> regsvr32.exe /u /i:<event source> <path of pgevent.dll>
ex. To deregister "MyApp" event source:
c:\> regsvr32.exe /u /i:"MyApp" pgevent.dll

<<Considerations>>
Is it better to have one parameter for syslog program name and event
source, instead of having both syslog_ident and event_source?
If it is, what name is good for the parameter? How about "program_name"?
ex.
program_name = 'MyApp'
In this case, the traditional syslog_ident parameter is kept as an
alias for backward compatibility, like the relationship between
work_mem and sort_mem.

If we have syslog_ident and event_source parameters separately, we
will probably end up in having similar functions as follows:
guc.c: assign_syslog_ident() and assign_event_source()
elog.c: set_syslog_parameters() and set_eventlog_parameters()

Does the community prefer separate parameters for each purpose (syslog
and event log) at the expense of a bit duplicate code, or one unified
parameter (program_name) with less code?

I would appreciate your opinions and advice. I'll try making the patch
while I'm waiting for response. I would be very much pleased if I
could contribute to PostgreSQL and my proposal could be included in 9.1.

It's a good idea, but 9.1 has been closed for new features for some
time. This would have to wait for 9.2 I believe.

cheers

andrew

"MauMau" <maumau307@gmail.com> writes:

I've encountered one problem on Windows. I need to support running all of my
products on one host simultaneously. Plus, I need to log messages in
syslog/event log. On Linux, I can distinguish the messages of one product
and those of other products by setting syslog_ident in postgresql.conf. On
the other hand, I can't do that on Windows: all of the PostgreSQL instances
use the same fixed event source name "PostgreSQL".

SO I'd like to propose a trivial feature which allows users to set event
source.

I'm a bit concerned by the fact that this seems to require global
actions to be taken (registering/deregistering) when a GUC value
changes. That's going to cause headaches, not least because you cannot
assume that the value changes synchronously across all Postgres
processes. Maybe that's only an artifact of the way you worded this and
not a real requirement ... but if it is a requirement you should think
about how to avoid it. If you can't avoid it then most likely the event
name is going to have to be PGC_POSTMASTER, ie not changeable after
startup. Also, what happens if we fail to deregister because of
crashing?

Is it better to have one parameter for syslog program name and event source,
instead of having both syslog_ident and event_source?

I don't like aliases for GUC variables, and in any case I think it'd be
confusing to force both usages to share a name that's not particularly
appropriate for either one alone. So personally, -1 for unifying those.
Others might see it differently though.

I would appreciate your opinions and advice. I'll try making the patch while
I'm waiting for response. I would be very much pleased if I could contribute
to PostgreSQL and my proposal could be included in 9.1.

As Andrew said, it's way too late for 9.1. Plenty of time to work on it
for 9.2 though.

regards, tom lane

I'm sorry that I've mistakenly sent an empty mail. This is the intended
mail.

"Andrew Dunstan" <andrew@dunslane.net> wrote in message
news:4D889879.3080705@dunslane.net...

On 03/22/2011 08:22 AM, MauMau wrote:

I would appreciate your opinions and advice. I'll try making the patch
while I'm waiting for response. I would be very much pleased if I
could contribute to PostgreSQL and my proposal could be included in 9.1.

It's a good idea, but 9.1 has been closed for new features for some
time. This would have to wait for 9.2 I believe.

cheers

andrew

OK. I'll try to make a patch for 9.2, considering Tom's advice and opinion.
By that time, I will learn more about PostgreSQL design and source code.

I seem to have misunderstood the commit fest. I've re-read the development
info, and my corrected understanding related to the development cycle is as
follows:

According to the following two pages, now is the commit fest 5. The current
commit fest will end on April 15. I would be grateful if you could tell me
where I can find out that 9.1 is closed for new features.

PostgreSQL 9.1 Development Plan
http://wiki.postgresql.org/wiki/PostgreSQL_9.1_Development_Plan

CommitFest 2011-Next (Open)
https://commitfest.postgresql.org/action/commitfest_view?id=10

Now that the next alpha 5 seems to be the last alpha release for 9.1, and
considering the following description, how should I behave? I don't want to
interrupt your work for release of 9.1.

--------------------------------------------------
Submitting a Patch
http://wiki.postgresql.org/wiki/Submitting_a_Patch
Submission timing
...PostgreSQL development is also organized with periodic CommitFests, which
are periods where new development halts in order to focus on patch review
and committing. It's best if you can avoid sending in a new patch during the
occasional weeks when there is an active CommitFest; you can check the
schedule via the CommitFest application. If your schedule doesn't allow
waiting until an active CommitFest is over, you should explicitly label your
submission as intended for the next CommitFest, not the current one, so that
it's clear it's not intended to be part of the active review process.
--------------------------------------------------

I think the actions I should take are as follows. What is the order of
preference?

1. Wait until the end of current commit fest (possibly April 15), and
continue the discussion and patch submission.
2. Continue the discussion on the func spec and reach agreement as it seems
relatively trivial, but refrain from submitting the patch until the end of
the current commit fest (April 15).
3. Continue the discussion on the func spec and reach agreement as it seems
relatively trivial, and submit the patch to pgsql-hackers and register it
with the current commit fest page. However, the register patch will be
transferred to the first commit fest page for 9.2?
4. Wait until the beginning of 9.2 development and continue the discussion
and patch submission. (But when is it? How can I catch the timing
efficiently?)

Regards
MauMau

I'm sorry that I've mistakenly sent an empty mail. This is the intended
mail.

"Andrew Dunstan" <andrew@dunslane.net> wrote in message
news:4D889879.3080705@dunslane.net...

On 03/22/2011 08:22 AM, MauMau wrote:

I would appreciate your opinions and advice. I'll try making the patch
while I'm waiting for response. I would be very much pleased if I
could contribute to PostgreSQL and my proposal could be included in 9.1.

It's a good idea, but 9.1 has been closed for new features for some
time. This would have to wait for 9.2 I believe.

cheers

andrew

OK. I'll try to make a patch for 9.2, considering Tom's advice and opinion.
By that time, I will learn more about PostgreSQL design and source code.

I seem to have misunderstood the commit fest. I've re-read the development
info, and my corrected understanding related to the development cycle is as
follows:

According to the following two pages, now is the commit fest 5. The current
commit fest will end on April 15. I would be grateful if you could tell me
where I can find out that 9.1 is closed for new features.

PostgreSQL 9.1 Development Plan
http://wiki.postgresql.org/wiki/PostgreSQL_9.1_Development_Plan

CommitFest 2011-Next (Open)
https://commitfest.postgresql.org/action/commitfest_view?id=10

Now that the next alpha 5 seems to be the last alpha release for 9.1, and
considering the following description, how should I behave? I don't want to
interrupt your work for release of 9.1.

--------------------------------------------------
Submitting a Patch
http://wiki.postgresql.org/wiki/Submitting_a_Patch
Submission timing
...PostgreSQL development is also organized with periodic CommitFests, which
are periods where new development halts in order to focus on patch review
and committing. It's best if you can avoid sending in a new patch during the
occasional weeks when there is an active CommitFest; you can check the
schedule via the CommitFest application. If your schedule doesn't allow
waiting until an active CommitFest is over, you should explicitly label your
submission as intended for the next CommitFest, not the current one, so that
it's clear it's not intended to be part of the active review process.
--------------------------------------------------

I think the actions I should take are as follows. What is the order of
preference?

1. Wait until the end of current commit fest (possibly April 15), and
continue the discussion and patch submission.
2. Continue the discussion on the func spec and reach agreement as it seems
relatively trivial, but refrain from submitting the patch until the end of
the current commit fest (April 15).
3. Continue the discussion on the func spec and reach agreement as it seems
relatively trivial, and submit the patch to pgsql-hackers and register it
with the current commit fest page. However, the register patch will be
transferred to the first commit fest page for 9.2?
4. Wait until the beginning of 9.2 development and continue the discussion
and patch submission. (But when is it? How can I catch the timing
efficiently?)

Regards
MauMau

On 03/22/2011 11:35 AM, MauMau wrote:

I'm sorry that I've mistakenly sent an empty mail. This is the
intended mail.

"Andrew Dunstan" <andrew@dunslane.net> wrote in message
news:4D889879.3080705@dunslane.net...

On 03/22/2011 08:22 AM, MauMau wrote:

I would appreciate your opinions and advice. I'll try making the patch
while I'm waiting for response. I would be very much pleased if I
could contribute to PostgreSQL and my proposal could be included in
9.1.

It's a good idea, but 9.1 has been closed for new features for some
time. This would have to wait for 9.2 I believe.

cheers

andrew

OK. I'll try to make a patch for 9.2, considering Tom's advice and
opinion. By that time, I will learn more about PostgreSQL design and
source code.

I seem to have misunderstood the commit fest. I've re-read the
development info, and my corrected understanding related to the
development cycle is as follows:

According to the following two pages, now is the commit fest 5. The
current commit fest will end on April 15. I would be grateful if you
could tell me where I can find out that 9.1 is closed for new features.

PostgreSQL 9.1 Development Plan
http://wiki.postgresql.org/wiki/PostgreSQL_9.1_Development_Plan

CommitFest 2011-Next (Open)
https://commitfest.postgresql.org/action/commitfest_view?id=10

The last commitfest for this development cycle opened on January 15th,
as shown in the first URL above, so that was the last date for
submitting items for 9.1 Any items in the next commitfest are for 9.2,
and that is the place to submit your item when you have a patch. There
is no currently running commitfest.

cheers

andrew

Added to TODO list:

Allow multiple Postgres clusters running on the same machine to
distinguish themselves in the event log

http://archives.postgresql.org/pgsql-hackers/2011-03/msg01297.php

---------------------------------------------------------------------------

Andrew Dunstan wrote:

On 03/22/2011 11:35 AM, MauMau wrote:

I'm sorry that I've mistakenly sent an empty mail. This is the
intended mail.

"Andrew Dunstan" <andrew@dunslane.net> wrote in message
news:4D889879.3080705@dunslane.net...

On 03/22/2011 08:22 AM, MauMau wrote:

I would appreciate your opinions and advice. I'll try making the patch
while I'm waiting for response. I would be very much pleased if I
could contribute to PostgreSQL and my proposal could be included in
9.1.

It's a good idea, but 9.1 has been closed for new features for some
time. This would have to wait for 9.2 I believe.

cheers

andrew

OK. I'll try to make a patch for 9.2, considering Tom's advice and
opinion. By that time, I will learn more about PostgreSQL design and
source code.

I seem to have misunderstood the commit fest. I've re-read the
development info, and my corrected understanding related to the
development cycle is as follows:

According to the following two pages, now is the commit fest 5. The
current commit fest will end on April 15. I would be grateful if you
could tell me where I can find out that 9.1 is closed for new features.

PostgreSQL 9.1 Development Plan
http://wiki.postgresql.org/wiki/PostgreSQL_9.1_Development_Plan

CommitFest 2011-Next (Open)
https://commitfest.postgresql.org/action/commitfest_view?id=10

The last commitfest for this development cycle opened on January 15th,
as shown in the first URL above, so that was the last date for
submitting items for 9.1 Any items in the next commitfest are for 9.2,
and that is the place to submit your item when you have a patch. There
is no currently running commitfest.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

From: "Tom Lane" <tgl@sss.pgh.pa.us>

"MauMau" <maumau307@gmail.com> writes:

I've encountered one problem on Windows. I need to support running all of
my
products on one host simultaneously. Plus, I need to log messages in
syslog/event log. On Linux, I can distinguish the messages of one product
and those of other products by setting syslog_ident in postgresql.conf.
On
the other hand, I can't do that on Windows: all of the PostgreSQL
instances
use the same fixed event source name "PostgreSQL".

SO I'd like to propose a trivial feature which allows users to set event
source.

I'm a bit concerned by the fact that this seems to require global
actions to be taken (registering/deregistering) when a GUC value
changes. That's going to cause headaches, not least because you cannot
assume that the value changes synchronously across all Postgres
processes. Maybe that's only an artifact of the way you worded this and
not a real requirement ... but if it is a requirement you should think
about how to avoid it. If you can't avoid it then most likely the event
name is going to have to be PGC_POSTMASTER, ie not changeable after
startup. Also, what happens if we fail to deregister because of
crashing?

Is it better to have one parameter for syslog program name and event
source,
instead of having both syslog_ident and event_source?

I don't like aliases for GUC variables, and in any case I think it'd be
confusing to force both usages to share a name that's not particularly
appropriate for either one alone. So personally, -1 for unifying those.
Others might see it differently though.

Considering Tom's advice, I'll make a patch as follows:

1. Create a new GUC variable called event_source and don't touch anything
with the existing syslog_ident.
2. Make event_source PGC_POSTMASTER at first because I don't think it is not
a requirement to change the event source name while PostgreSQL server is
active.

Also, what happens if we fail to deregister because of
crashing?

The registration/deregistration is performed separately from PostgreSQL's
start/stop, so there is no concern about this. Please see "Registering
eventlog on Windows" on the page below:

Installation Procedure
http://www.postgresql.org/docs/9.0/static/install-procedure.html

In fact, I've almost finished writing the program patch and testing.
However, I'd like to ask for your opinions about a few things before
completing and submitting the patch.

1. Documentation
Currently, the event Source registration is described on the above page.
However, many of my colleagues fail to find the article. They use
PostgreSQL as one of many software and don't fully read the manual.

I wonder if it would be better to put the article on the following section,
because this is in the chapter about installation on Windows:

16.1.4. Cleaning and installing
http://www.postgresql.org/docs/9.0/static/install-windows-full.html

or:

Post-Installation Setup
http://www.postgresql.org/docs/9.0/static/install-post.html

In addition, I think the above page should be linked from the description of
log_destination parameter.

2. pg_ctl's event logging
pg_ctl also logs to event log. Should pg_ctl use the event_source setting?
According to the response to the bug #6011 report, pg_ctl does not need to
obey the postgresql.conf setting, because postgresql.conf is for the server.
However, I'm afraid that the users claim that logging with event source
"PostgreSQL" instead of the event_source setting is a bug. For reference,
pg_ctl uses "port" parameter in postgresql.conf.

Regards,
MauMau

"MauMau" <maumau307@gmail.com> writes:

"MauMau" <maumau307@gmail.com> writes:

I've encountered one problem on Windows. I need to support running all of
my
products on one host simultaneously. Plus, I need to log messages in
syslog/event log. On Linux, I can distinguish the messages of one product
and those of other products by setting syslog_ident in postgresql.conf.
On
the other hand, I can't do that on Windows: all of the PostgreSQL
instances
use the same fixed event source name "PostgreSQL".

SO I'd like to propose a trivial feature which allows users to set event
source.

BTW, what will this accomplish exactly that couldn't be accomplished by
setting log_line_prefix to include the desired identifier?

regards, tom lane

On Tue, May 10, 2011 at 11:55 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

"MauMau" <maumau307@gmail.com> writes:

"MauMau" <maumau307@gmail.com> writes:

I've encountered one problem on Windows. I need to support running all of
my
products on one host simultaneously. Plus, I need to log messages in
syslog/event log. On Linux, I can distinguish the messages of one product
and those of other products by setting syslog_ident in postgresql.conf.
On
the other hand, I can't do that on Windows: all of the PostgreSQL
instances
use the same fixed event source name "PostgreSQL".

SO I'd like to propose a trivial feature which allows users to set event
source.

BTW, what will this accomplish exactly that couldn't be accomplished by
setting log_line_prefix to include the desired identifier?

Windows uses the event source field to show where events in the logs
have come from. The Event Viewer and other network management tools
expect to be able to use the field for sorting and filtering etc, so
having a different value for different PG instances allows those tools
to distinguish between them properly. It's also useful for ISVs who
may want to make an embedded PG instance used by their apps identify
itself differently from other instances.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

From: "Dave Page" <dpage@pgadmin.org>

On Tue, May 10, 2011 at 11:55 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

BTW, what will this accomplish exactly that couldn't be accomplished by
setting log_line_prefix to include the desired identifier?

Windows uses the event source field to show where events in the logs
have come from. The Event Viewer and other network management tools
expect to be able to use the field for sorting and filtering etc, so
having a different value for different PG instances allows those tools
to distinguish between them properly. It's also useful for ISVs who
may want to make an embedded PG instance used by their apps identify
itself differently from other instances.

Thank you for help, Dave.

The Windows Event Viewer can filter event log records with event source,
computer, user, date/time, etc. but not with message body. So
log_line_prefix cannot be used for filtering. Even if filtering with message
body were possible, using event source field to filter applications is more
natural and intuitive.

I would appreciate comments from all on the following items mentioned
previously. If no comments, I'll do as follows and try to submit the patch a
week later or so if I have enough time.

1. Documentation
Move the event log registration/deregistion article to the following
section. Add a link to this article from log_destination description.

16.1.4. Cleaning and installing
http://www.postgresql.org/docs/9.0/static/install-windows-full.html

2. pg_ctl's event logging
Do not change anything - that is, use the fixed event source "PostgreSQL".
How do we explain the reason for not using event_source parameter in
postgresql.conf? "postgresql.conf is for the server. pg_ctl uses the fixed
event source. This is a specification. This is not a problem because
starting/stopping the database/application is infrequent once the system is
in steady operation." -- this may sound abrupt, though.

Regards,
MauMau

----- Original Message -----
From: "Dave Page" <dpage@pgadmin.org>
To: "Tom Lane" <tgl@sss.pgh.pa.us>
Cc: "MauMau" <maumau307@gmail.com>; <pgsql-hackers@postgresql.org>
Sent: Wednesday, May 11, 2011 4:58 PM
Subject: Re: [HACKERS] Feature proposal: distinguish each PostgreSQL
instance in the event log