PG Log MultiLine Records

Started by Arvind Singhover 13 years ago2 messagesgeneral
Jump to latest
#1Arvind Singh
arvindps@hotmail.com

we have been working on a CSV Log parser application in csharp.

we recently discovered that certain log entries or records can span across multiple lines.

in the sense, that the same activity has more detail in subsequent lines.

For ex, a select,insert query has
A query entry
A Duration entry
A Stats entry

and that is why there was a serious coding mistake in counting select entries, because the same select statement
is displayed in Query column twice and once in Detail column

So our application counted them as three queries.

My query is, has any of us come across more such examples or
:: is there a link or page explaining type of Log entries
:: Number of log statements a logentry type occupy.
:: or other finer details that we may overlook

it will be helpfull as we wish to release this application for all pg users at no price.

thanks arvind

----------------------------------------------What charm in words, a charm no words could give?
O dying words, can Music make you live
Far--far--away?
--------- (Alfred, Lord Tennyson in Far-Far-Away)

#2Laurenz Albe
laurenz.albe@cybertec.at
In reply to: Arvind Singh (#1)
Re: PG Log MultiLine Records

Arvind Singh wrote:

we have been working on a CSV Log parser application in csharp.

we recently discovered that certain log entries or records can span

across multiple lines.

in the sense, that the same activity has more detail in subsequent

lines.

Not really, they are different log entries that belong to one
SQL statement.

For example, if you have log_statement='all' and
log_min_duration_statement=3s, then some statements will
produce one log entry and others will produce two.

For ex, a select,insert query has
A query entry
A Duration entry
A Stats entry

and that is why there was a serious coding mistake in counting select

entries, because the same select

statement
is displayed in Query column twice and once in Detail column

So our application counted them as three queries.

My query is, has any of us come across more such examples or
:: is there a link or page explaining type of Log entries

Not really, the details are in the code.

:: Number of log statements a logentry type occupy.

I don't quite understand that.

:: or other finer details that we may overlook

it will be helpfull as we wish to release this application for all pg

users at no price.

It is hard to find a solution that works no matter how
logging is configured.

It would be helpful to know what your program is trying to achieve.

What I did for pgreplay is
a) force certain configuration settings as prerequisite
(like log_statement='all').
b) search for log lines with severity "LOG" where the query
entry begins with "statement: ".

That should filter out the "duration" and "stats" lines you
mention above.

Yours,
Laurenz Albe