Why SyncOneBuffer does not called frequently?

Started by 高健over 13 years ago4 messagesgeneral

luckyjackgao@gmail.com

over 13 years ago

Hi all:

I am trying to understand when the bgwriter is written.

I thought that the bgwriter.c's calling turn is:

BackgroundWriterMain ->BgBufferSync-> SyncOneBuffer

And In my postgresql.conf , the bgwriter_delay=200ms.

I did the following:

postgres=# select * from testtab;
id | val
----+-------
1 | 12345
(1 row)

postgres=# update testtab set val='54321' where id=1;
UPDATE 1
postgres=# select * from testtab;
id | val
----+-------
1 | 54321
(1 row)

postgres=#

Now I can say the buffer is dirty ,right?

I wait for a few minutes, I can found bgwriter's BackgroundWriterMain
called BgBufferSync many times.

But I can't find BgBufferSync really call SyncOneBuffer to put the dirty
data todisk.

Untill I close the postgres process, I can find the SyncOneBuffer is called
for many times.

My question is:
Why even there are dirty buffer(s), the SyncOneBuffer is still not called?
Is it violating the background writer's purpose?

Or the flushing to disk will be done untill the amount of block/buffer is
satisfied? If so , what is it?

Thanks in advance for any help

jeff.janes@gmail.com

over 13 years ago

In reply to: 高健 (#1)

Re: Why SyncOneBuffer does not called frequently?

On Wed, Oct 31, 2012 at 1:53 AM, 高健 <luckyjackgao@gmail.com> wrote:

Hi all:

I am trying to understand when the bgwriter is written.

I thought that the bgwriter.c's calling turn is:

BackgroundWriterMain ->BgBufferSync-> SyncOneBuffer

SyncOneBuffer is called with skip_recently_used true, so the buffer
will not get written by the background writer if it is was recently
used.

And In my postgresql.conf , the bgwriter_delay=200ms.

I did the following:

postgres=# select * from testtab;
id | val
----+-------
1 | 12345
(1 row)

postgres=# update testtab set val='54321' where id=1;
UPDATE 1
postgres=# select * from testtab;
id | val
----+-------
1 | 54321
(1 row)

postgres=#

Now I can say the buffer is dirty ,right?

It is dirty, but it also has a positive usagecount.

Are you sure it is that SyncOneBuffer is not getting called, rather
than it is getting called but returning before doing the write?

Cheers,

Jeff

iihero@qq.com

over 13 years ago

In reply to: Jeff Janes (#2)

Re: Why SyncOneBuffer does not called frequently?

HI,

I just debug the code, find that the following stack will call the SyncOneBuffer.

postgres.exe!SyncOneBuffer(int buf_id=2, char skip_recently_used=0) Line 1640 C

postgres.exe!BufferSync(int flags=64) Line 1284 + 0xb bytes C
postgres.exe!CheckPointBuffers(int flags=64) Line 1801 + 0x9 bytes C
postgres.exe!CheckPointGuts(XLogRecPtr checkPointRedo={...}, int flags=64) Line 8129 + 0x9 bytes C
postgres.exe!CreateCheckPoint(int flags=64) Line 7977 + 0x11 bytes C
postgres.exe!CheckpointerMain() Line 505 + 0x9 bytes C
postgres.exe!AuxiliaryProcessMain(int argc=2, char * * argv=0x002f6fb8) Line 429 C
postgres.exe!SubPostmasterMain(int argc=4, char * * argv=0x002f6fb0) Line 4136 + 0x13 bytes C
postgres.exe!main(int argc=4, char * * argv=0x002f6fb0) Line 176 + 0xd bytes C
postgres.exe!__tmainCRTStartup() Line 582 + 0x19 bytes C
postgres.exe!mainCRTStartup() Line 399 C

This is the check point background process.

------------------
Thanks&Regards,
Xiong He

------------------ Original ------------------
From: "高健"<luckyjackgao@gmail.com>;
Date: Wed, Oct 31, 2012 04:53 PM
To: "pgsql-general"<pgsql-general@postgresql.org>;

Subject: [GENERAL] Why SyncOneBuffer does not called frequently?

Hi all:

I am trying to understand when the bgwriter is written.

I thought that the bgwriter.c's calling turn is:

BackgroundWriterMain ->BgBufferSync-> SyncOneBuffer

And In my postgresql.conf , the bgwriter_delay=200ms.

I did the following:

postgres=# select * from testtab;
id | val
----+-------
1 | 12345
(1 row)

postgres=# update testtab set val='54321' where id=1;
UPDATE 1
postgres=# select * from testtab;
id | val
----+-------
1 | 54321
(1 row)

postgres=#

Now I can say the buffer is dirty ,right?

I wait for a few minutes, I can found bgwriter's BackgroundWriterMain called BgBufferSync many times.

But I can't find BgBufferSync really call SyncOneBuffer to put the dirty data todisk.

Untill I close the postgres process, I can find the SyncOneBuffer is called for many times.

My question is:
Why even there are dirty buffer(s), the SyncOneBuffer is still not called?
Is it violating the background writer's purpose?

Or the flushing to disk will be done untill the amount of block/buffer is satisfied? If so , what is it?

Thanks in advance for any help

Import Notes

Resolved by subject fallback

iihero@qq.com

over 13 years ago

In reply to: Xiong He (#3)

Re: Why SyncOneBuffer does not called frequently?

This perhaps proves that the bgwriter  doesn't need to call the SyncOneBuffer again on the same buffer.<br><br><div><div style="color:#909090;font-family:Arial Narrow;font-size:12px">------------------</div><div style="font-size:14px;font-family:Verdana;color:#000;"><div><div style="font-family: 'lucida Grande', Verdana; font-size: 12px; line-height: 18px; ">Thanks&Regards,</div><div>Xiong He</div></div></div></div><div><includetail><div style="font-size: 12px;font-family: Arial Narrow;padding:2px 0 2px 0;">------------------ Original ------------------</div><div style="font-size: 12px;background:#efefef;padding:8px;"><div><b>From: </b> "Xiong He"<iihero@qq.com>;</div><div><b>Date: </b> Thu, Nov 1, 2012 07:40 AM</div><div><b>To: </b> "高健"<luckyjackgao@gmail.com>; "pgsql-general"<pgsql-general@postgresql.org>; <wbr></div><div></div><div><b>Subject: </b> Re: [GENERAL] Why SyncOneBuffer does not called frequently?</div></div><div><br></div>HI, <br><br>I just debug the code, find that the following stack will call the SyncOneBuffer.<br><br>>    postgres.exe!SyncOneBuffer(int buf_id=2, char skip_recently_used=0)  Line 1640    C<br>     postgres.exe!BufferSync(int flags=64)  Line 1284 + 0xb bytes    C<br>     postgres.exe!CheckPointBuffers(int flags=64)  Line 1801 + 0x9 bytes    C<br>     postgres.exe!CheckPointGuts(XLogRecPtr checkPointRedo={...}, int flags=64)  Line 8129 + 0x9 bytes    C<br>     postgres.exe!CreateCheckPoint(int flags=64)  Line 7977 + 0x11 bytes    C<br>     postgres.exe!CheckpointerMain()  Line 505 + 0x9 bytes    C<br>     postgres.exe!AuxiliaryProcessMain(int argc=2, char * * argv=0x002f6fb8)  Line 429    C<br>     postgres.exe!SubPostmasterMain(int argc=4, char * * argv=0x002f6fb0)  Line 4136 + 0x13 bytes    C<br>     postgres.exe!main(int argc=4, char * * argv=0x002f6fb0)  Line 176 + 0xd bytes    C<br>     postgres.exe!__tmainCRTStartup()  Line 582 + 0x19 bytes    C<br>     postgres.exe!mainCRTStartup()  Line 399    C<br><br>This is the check point background process.<br><br><br><br><div><div style="color:#909090;font-family:Arial Narrow;font-size:12px">------------------</div><div style="font-size:14px;font-family:Verdana;color:#000;"><div><div style="font-family: 'lucida Grande', Verdana; font-size: 12px; line-height: 18px; ">Thanks&Regards,</div><div style="font-family: 'lucida Grande', Verdana; font-size: 12px; line-height: 18px; ">Xiong He<br><br></div></div></div></div><div> </div><div><div><br></div><div><br></div><div style="font-size: 12px;font-family: Arial Narrow;padding:2px 0 2px 0;">------------------ Original ------------------</div><div style="font-size: 12px;background:#efefef;padding:8px;"><div><b>From: </b> "高健"<luckyjackgao@gmail.com>;</div><div><b>Date: </b> Wed, Oct 31, 2012 04:53 PM</div><div><b>To: </b> "pgsql-general"<pgsql-general@postgresql.org>; <wbr></div><div></div><div><b>Subject: </b> [GENERAL] Why SyncOneBuffer does not called frequently?</div></div><div><br></div><div>Hi all:</div><div><br></div><div>I am trying to understand when the bgwriter is written.</div><div><br></div><div>I thought that the  bgwriter.c's calling turn is:</div><div><br></div><div>BackgroundWriterMain ->BgBufferSync-> SyncOneBuffer</div>
<div><br></div><div>And In my postgresql.conf , the bgwriter_delay=200ms.</div><div><br></div><div>I did the following:</div><div><br></div><div>postgres=# select * from testtab;</div><div> id |  val  </div><div>----+-------</div>
<div>  1 | 12345</div><div>(1 row)</div><div><br></div><div>postgres=# update testtab set val='54321' where id=1;</div><div>UPDATE 1</div><div>postgres=# select * from testtab;</div><div> id |  val  </div><div>----+-------</div>
<div>  1 | 54321</div><div>(1 row)</div><div><br></div><div>postgres=# </div><div><br></div><div>Now I can say the buffer is dirty ,right?</div><div><br></div><div>I wait for a few minutes, I can found bgwriter's BackgroundWriterMain called BgBufferSync many times.</div>
<div><br></div><div>But I can't find BgBufferSync really call SyncOneBuffer to put the dirty data todisk.</div><div><br></div><div>Untill I close the postgres process, I can find the SyncOneBuffer is called for many times.</div>
<div><br></div><div>My question is: </div><div>Why even there are dirty buffer(s), the SyncOneBuffer is still not called?  </div><div>Is it violating the background writer's purpose? </div><div><br></div><div>Or the flushing to disk will be done untill  the amount of block/buffer is satisfied? If so , what is it?</div>
<div><br></div><div>Thanks in advance for any help</div></div></includetail></div>

Import Notes

Resolved by subject fallback