Theory question

On Tue, Nov 12, 2013 at 6:57 AM, Jayadevan M <maymala.jayadevan@gmail.com>wrote:

Hi,
What are the real differences between the bgwriter and checkpointer
process? Both of them write data from the buffer to the data files, right?
Is it just a matter of 'when' they write?
Regards,
Jayadevan

Expect some corrections
by others
on
my understanding
described below.

AFAIK, they share the load of writing dirty-buffers to disk, though they
are defined to serve different purpose. Basically, background writer
process sole function is to write "dirty" shared buffers to disk and evict
those pages from shared buffer pool. Whereas checkpoint, arrives to write
all dirty data pages in shared_buffers to disk only when checkpoint_timeout
or when all checkpoint_segments are filled, whichever comes first. However,
BG Writer (Writer Process) will be continuously
trickle out dirty pages to disk so that by the time checkpoint arrives
there will be left only with f
ew dirty pages, instead of
having
lots of dirty
pages

to carry out by i
tself
alone
and cause I/O loaded
.

---
Regards,
Raghavendra

On Tue, Nov 12, 2013 at 6:57 AM, Jayadevan M <maymala.jayadevan@gmail.com>wrote:

Hi,
What are the real differences between the bgwriter and checkpointer
process? Both of them write data from the buffer to the data files, right?
Is it just a matter of 'when' they write?

I believe, "Checkpoint" is one of the responsible duty of bgwriter process.
AFAIK from PG 9.2 onwards, we have another independent process called
"CKPTR process" which independently works. "Bgwriter" simply move the
shared_buffer's pages into local disk on the basis of LRU method. Where as
"CKPTR process" moves all the dirty buffers into local disk, and make a
transaction log sequence, at which all the data files have been updated to
reflect this information.

Best Regards,
Dinesh
manojadinesh.blogspot.com

Jayadevan M <maymala.jayadevan@gmail.com> wrote:

What are the real differences between the bgwriter and
checkpointer process? Both of them write data from the buffer to
the data files, right? Is it just a matter of 'when' they write?

The checkpointer process is responsible for creating safe points
from which a recovery can begin; the background writer tries to
keep some pages available for re-use so that processes running
queries don't need to wait for page writes  in order to have free
spots to use in shared buffers.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Kevin Grittner-5 wrote

The checkpointer process is responsible for creating safe points
from which a recovery can begin; the background writer tries to
keep some pages available for re-use so that processes running
queries don't need to wait for page writes  in order to have free
spots to use in shared buffers.

Thank you. Do both of them write to the same files? Is it that checkpoint
writes only committed data whereas background writer may also write
non-committed data if there is no space available in the buffers?

--
View this message in context: http://postgresql.1045698.n5.nabble.com/Theory-question-tp5777838p5778052.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

On Tue, Nov 12, 2013 at 7:09 PM, Jayadevan <maymala.jayadevan@gmail.com>wrote:

Kevin Grittner-5 wrote

The checkpointer process is responsible for creating safe points
from which a recovery can begin; the background writer tries to
keep some pages available for re-use so that processes running
queries don't need to wait for page writes in order to have free
spots to use in shared buffers.

Thank you. Do both of them write to the same files?

Yes, mostly. (The checkpointer also does some other housekeeping writes
beyond the ordinary data file ones)

Is it that checkpoint
writes only committed data whereas background writer may also write
non-committed data if there is no space available in the buffers?

No. The checkpointer writes all data that was dirty as of a certain time
(the start of the checkpoint) regardless of how often it was used since
dirtied, and the background writer writes data that hasn't been used
recently, regardless of when it was first dirtied. Neither knows or cares
whether the data being written was committed, rolled back, or still in
progress.

Cheers,

Jeff

Jeff Janes wrote

No. The checkpointer writes all data that was dirty as of a certain time
(the start of the checkpoint) regardless of how often it was used since
dirtied, and the background writer writes data that hasn't been used
recently, regardless of when it was first dirtied. Neither knows or cares
whether the data being written was committed, rolled back, or still in
progress.

Thank you. So checkpointer writes "all dirty data" while backgrounder writes
"all or some dirty data" depending on some (Clocksweep?) algorithm. Correct?
From this discussion
http://postgresql.1045698.n5.nabble.com/Separating-bgwriter-and-checkpointer-td4808791.html
<http://postgresql.1045698.n5.nabble.com/Separating-bgwriter-and-checkpointer-td4808791.html&gt;
the bgwrites has some 'other dutties'. Probably those involve marking the
buffers - when they were last used, how frequently etc?

--
View this message in context: http://postgresql.1045698.n5.nabble.com/Theory-question-tp5777838p5778272.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

On Thu, Nov 14, 2013 at 7:58 AM, Jayadevan <maymala.jayadevan@gmail.com>wrote:

Jeff Janes wrote

No. The checkpointer writes all data that was dirty as of a certain time
(the start of the checkpoint) regardless of how often it was used since
dirtied, and the background writer writes data that hasn't been used
recently, regardless of when it was first dirtied. Neither knows or

cares

whether the data being written was committed, rolled back, or still in
progress.

Thank you. So checkpointer writes "all dirty data" while backgrounder
writes
"all or some dirty data" depending on some (Clocksweep?) algorithm.
Correct?
From this discussion

http://postgresql.1045698.n5.nabble.com/Separating-bgwriter-and-checkpointer-td4808791.html
<
http://postgresql.1045698.n5.nabble.com/Separating-bgwriter-and-checkpointer-td4808791.html

the bgwrites has some 'other dutties'. Probably those involve marking the
buffers - when they were last used, how frequently etc?

That should have been "backgrounder writes "all or some dirty or non-dirty

data" "...