log file rotate

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Tom, you didn't like Andreas' idea of allowing the user to rotate the
log files on demand.

Give me a use case that requires that, and is sufficiently interesting
to justify even a marginal decrease in the reliability of the log
process.

Frankly, I do not believe that database users should have anything to do
with the log rotation process. Do we have a TODO for allowing users to
force switching to a new WAL file segment?

Is this a TODO?

IMHO, no.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Tom, you didn't like Andreas' idea of allowing the user to rotate the
log files on demand.

Give me a use case that requires that, and is sufficiently interesting
to justify even a marginal decrease in the reliability of the log
process.

Frankly, I do not believe that database users should have anything to do
with the log rotation process. Do we have a TODO for allowing users to
force switching to a new WAL file segment?

I thought rotatelogs supported it so we should in cases where someone
wanted to make a new log file to delete an unusually large one, like a 1
gig log file caused by some runaway process. However, I see rotatelogs
doesn't have that capability so I guess we don't need it either.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

I thought rotatelogs supported it so we should in cases where someone
wanted to make a new log file to delete an unusually large one, like a 1
gig log file caused by some runaway process.

Hm? We have a rotate-on-size parameter, so that's not going to happen.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

I thought rotatelogs supported it so we should in cases where someone
wanted to make a new log file to delete an unusually large one, like a 1
gig log file caused by some runaway process.

Hm? We have a rotate-on-size parameter, so that's not going to happen.

Ah, OK.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Tom, you didn't like Andreas' idea of allowing the user to rotate the
log files on demand.

Give me a use case that requires that, and is sufficiently interesting
to justify even a marginal decrease in the reliability of the log
process.

- Debugging some stuff, a distinct period of work, I'd like to start
with a fresh serverlog.
- When doing a backup, I'd like to continue with a fresh serverlog so I
can see what was happening since that point in time.

MSSQL calls this sp_cycle_errorlog.

I can't see even a marginal decrease of reliability, since the logger
already accepts SIGHUP, and will reevaluate postgresql.conf to set the
rotation_requested flag accordingly; I'm just asking for setting the
flag directly.

Regards,
Andreas

Tom Lane wrote:

Give me a use case that requires that, and is sufficiently interesting
to justify even a marginal decrease in the reliability of the log
process.

Frankly, I do not believe that database users should have anything to do
with the log rotation process.

The (super)user will sometimes purge old files, so he *does* have to do
something with the rotation process' output. It's up to him what he
regards as old, maybe everything before 'now'.

YAUC (yet another use case): rotating every day at 3 am.

Regards,
Andreas

Tom Lane wrote:

Do we have a TODO for allowing users to
force switching to a new WAL file segment?

Together with PITR, this might make sense?

Regards,
Andreas

Andreas Pflug wrote:

Tom Lane wrote:

Do we have a TODO for allowing users to
force switching to a new WAL file segment?

Together with PITR, this might make sense?

I thought about this, and it would perhaps help, but it would bloat the
archive location if used repeatedly, perhaps every minute so you have a
recent version. It also might not work on server shutdown because there
might not be time to do the archive. I think we are going to need a
solution to archive the current file in a different way.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Andreas Pflug wrote:

Tom Lane wrote:

Do we have a TODO for allowing users to
force switching to a new WAL file segment?

Together with PITR, this might make sense?

I thought about this, and it would perhaps help, but it would bloat the
archive location if used repeatedly, perhaps every minute so you have a
recent version. It also might not work on server shutdown because there
might not be time to do the archive. I think we are going to need a
solution to archive the current file in a different way.

More to the point, if we did think that was worth doing then we'd just
make pg_start_backup do it automatically. There's still no need for
user control.

regards, tom lane

Andreas Pflug wrote:

Tom Lane wrote:

Do we have a TODO for allowing users to
force switching to a new WAL file segment?

Together with PITR, this might make sense?

Another idea:
Has anyone tried to put the WAL segment directory on a cluster
filesystem and use that for cold (perhaps even hot) failover?
The archive script could apply completed wal segments to the backup
node. If the primary node fails, the last (partial) segment is applied
as well and the backup node is activated.

--
Manfred