WIP csv logs

Andrew Dunstan wrote:

Attached is my WIP version of the CSV logs patch. It does not include
docs changes.

It fixes the CSV thinko I just posted about, and also implements the
"safe to rotate" scheme I suggested yesterday, at quite a modest cost.
As Tom rightly points out, that does not protect us against interleaving
log lines from different backends, which is an open problem.

I wonder, if we were to use an LWLock to protect writing to the stderr
pipe, would it be too contentious?

--
Alvaro Herrera Valdivia, Chile ICBM: S 39� 49' 18.1", W 73� 13' 56.4"
"El destino baraja y nosotros jugamos" (A. Schopenhauer)

Alvaro Herrera wrote:

Andrew Dunstan wrote:

Attached is my WIP version of the CSV logs patch. It does not include
docs changes.

It fixes the CSV thinko I just posted about, and also implements the
"safe to rotate" scheme I suggested yesterday, at quite a modest cost.
As Tom rightly points out, that does not protect us against interleaving
log lines from different backends, which is an open problem.

I wonder, if we were to use an LWLock to protect writing to the stderr
pipe, would it be too contentious?

That was the first thing I wanted to try. I'll code it up and then
people can test to see if we get a significant performance hit.

cheers

andrew

Alvaro Herrera <alvherre@commandprompt.com> writes:

I wonder, if we were to use an LWLock to protect writing to the stderr
pipe, would it be too contentious?

Sorry, that's a nonstarter.

1. Not all our processes are connected to shared memory. Even the ones
that are don't necessarily have PGPROCs.

2. If somebody crashes while holding the lock, the entire system
including the postmaster soon freezes up.

3. Having the postmaster take LWLocks is verboten anyway, mainly because
of problem #2.

regards, tom lane

Tom Lane wrote:

Alvaro Herrera <alvherre@commandprompt.com> writes:

I wonder, if we were to use an LWLock to protect writing to the stderr
pipe, would it be too contentious?

Sorry, that's a nonstarter.

1. Not all our processes are connected to shared memory. Even the ones
that are don't necessarily have PGPROCs.

2. If somebody crashes while holding the lock, the entire system
including the postmaster soon freezes up.

3. Having the postmaster take LWLocks is verboten anyway, mainly because
of problem #2.

Ugh. Well, until this problem is solved I'm not prepared to commit a
patch that purports to provide machine readable logs. The only
non-locking solution I can think of is to have one set of pipes per
backend rather than one set per cluster. That could get pretty ugly,
though, and have all sorts of problems with portability.

I'm starting to wonder if the CSVlog patch needs to be put on hold at
least for 8.4. If we do, I think I should at least commit the piece that
prevents rotation in the middle of a line, and possible even backpatch it.

cheers

andrew