split postgresql logfile

On Mon, Nov 29, 2021 at 12:12:11PM +0100, Paul van Rixel wrote:

Now the postgresql.logs are polluted with log connections/disconnection,
Agent monitoring entries and in between some entries which need some
attention but you missed them. I know the option exists to disable
log_(dis)connections but because of auditing this option must be enabled.
We don't use a tool like Grafana yet and with pgBadger we are experimenting
but skipping "unnessary" lines makes the manually reading of the logs much
more clear.

The logging collector is an all-or-nothing operation, so your options
when using it are limited when it comes to filtering, I am afraid. If
you'd wish to split the log entries, something that you could consider
is using the elog hook and redirect those logs into a dedicated file,
but that would mean being also careful about concurrency as you cannot
send those entries into the logging collector that would just write
logs in a serializable fashion in which log file it sees suited
depending on the server configuration (.csv, .log, etc.).

Another option for more filtering flexibility is syslog. While,
contrary to the logging collector that it designed to never lose
messages, it could drop messages that cannot be written :/

For auditing compliance, the latter is never a good option.

(FWIW, another thing that's been asked across the years is to have
JSON as logging format. There is a patch floating around to add this
support, but that won't apply to already-released versions.)
--
Michael