How to generate specific WAL records?

Started by Koichi Suzukiabout 16 years ago8 messageshackers
Jump to latest
#1Koichi Suzuki
koichi.szk@gmail.com

Hi,

Does anyone know how to generate the following WAL records from psql?

I'm now fixing pg_lesslog, which I reported a bug. Now code is
almost okay and I'd like to add test tools to show pg_lesslog can
handle all the WAL record correctly.

XLOG_MULTIXACT_ZERO_OFF_PAGE
XLOG_MULTIXACT_ZERO_MEM_PAGE
XLOG_MULTIXACT_CREATE_ID
XLOG_HEAP_LOCK
XLOG_HEAP_INIT_PAGE
XLOG_BTREE_INSERT_META
XLOG_BTREE_DELETE_PAGE
XLOG_BTREE_DELETE_PAGE_META
XLOG_BTREE_DELETE_PAGE_HALF
XLOG_GIN_CREATE_PTREE
XLOG_GIN_SPLIT
XLOG_GIN_VACUUM_PAGE
XLOG_GIN_DELETE_PAGE
XLOG_GIN_DELETE_LISTPAGE
XLOG_GIST_PAGE_DELETE

So far, I'm using conventional BTREE, btree_gin and btree_gist for
the test, as well as 2PC and savepoint.

Any information is welcome.

Thank you very much in advance;
------
Koichi Suzuki

#2Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Koichi Suzuki (#1)
Re: How to generate specific WAL records?

Koichi Suzuki escribi�:

Hi,

Does anyone know how to generate the following WAL records from psql?

I'm now fixing pg_lesslog, which I reported a bug. Now code is
almost okay and I'd like to add test tools to show pg_lesslog can
handle all the WAL record correctly.

XLOG_MULTIXACT_ZERO_OFF_PAGE
XLOG_MULTIXACT_ZERO_MEM_PAGE
XLOG_MULTIXACT_CREATE_ID

For these, you need to do SELECT FOR SHARE of a tuple in more than one
concurrent session.

I think HEAP_LOCK is SELECT FOR UPDATE.

XLOG_BTREE_DELETE_PAGE
XLOG_BTREE_DELETE_PAGE_META
XLOG_BTREE_DELETE_PAGE_HALF

These are caused during vacuum of a btree. Create an index, populate
it, then remove all items. Then vacuum twice.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

#3Koichi Suzuki
koichi.szk@gmail.com
In reply to: Alvaro Herrera (#2)
Re: How to generate specific WAL records?

Alvaro-san;

Thank you for a great advice. I successfully generated all the WAL
records listed below. By deleteing and VACUUMing a table with
btree_gist based index, I was successful to generate all the GIST WAL
records.

Still have the following WAL record to create:

CLOG_TRUNCATE
XLOG_HEAP_INIT_PAGE
XLOG_BTREE_INSERT_META
XLOG_GIN_CREATE_PTREE
XLOG_GIN_DELETE_PAGE

I have to run good amount of transactions before VACUUM FREEZE to
create CLOG_TRUNCATE. I'll try.

It's very helpful if anybody knows how to generate XLOG_HEAP_INIT_PAGE
and XLOG_BTREE_INSERT_META.

I'll ask Oleg-san how to generate GIN-related WALs.

Thank you very much;
----------
Koichi Suzuki

2010/4/14 Alvaro Herrera <alvherre@commandprompt.com>:

Show quoted text

Koichi Suzuki escribió:

Hi,

Does anyone know how to generate the following WAL records from psql?

I'm now fixing pg_lesslog, which I reported a bug.   Now code is
almost okay and I'd like to add test tools to show pg_lesslog can
handle all the WAL record correctly.

XLOG_MULTIXACT_ZERO_OFF_PAGE
XLOG_MULTIXACT_ZERO_MEM_PAGE
XLOG_MULTIXACT_CREATE_ID

For these, you need to do SELECT FOR SHARE of a tuple in more than one
concurrent session.

I think HEAP_LOCK is SELECT FOR UPDATE.

XLOG_BTREE_DELETE_PAGE
XLOG_BTREE_DELETE_PAGE_META
XLOG_BTREE_DELETE_PAGE_HALF

These are caused during vacuum of a btree.  Create an index, populate
it, then remove all items.  Then vacuum twice.

--
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

#4Simon Riggs
simon@2ndQuadrant.com
In reply to: Koichi Suzuki (#3)
Re: How to generate specific WAL records?

On Wed, 2010-04-14 at 11:13 +0900, Koichi Suzuki wrote:

Thank you for a great advice. I successfully generated all the WAL
records listed below. By deleteing and VACUUMing a table with
btree_gist based index, I was successful to generate all the GIST WAL
records.

It would be a very useful test case to publish. There are already a
number of standby related tests, this would be very useful for general
case testing, not just for pg_lesslog.

--
Simon Riggs www.2ndQuadrant.com

#5Koichi Suzuki
koichi.szk@gmail.com
In reply to: Simon Riggs (#4)
Re: How to generate specific WAL records?

Thanks for encouraging comment. I'm still struggling to generate
remaing WAL records.
----------
Koichi Suzuki

2010/4/14 Simon Riggs <simon@2ndquadrant.com>:

Show quoted text

On Wed, 2010-04-14 at 11:13 +0900, Koichi Suzuki wrote:

Thank you for a great advice.   I successfully generated all the WAL
records listed below.   By deleteing and VACUUMing a table with
btree_gist based index, I was successful to generate all the GIST WAL
records.

It would be a very useful test case to publish. There are already a
number of standby related tests, this would be very useful for general
case testing, not just for pg_lesslog.

--
 Simon Riggs           www.2ndQuadrant.com

#6Kevin Grittner
Kevin.Grittner@wicourts.gov
In reply to: Koichi Suzuki (#5)
Re: How to generate specific WAL records?

Koichi Suzuki <koichi.szk@gmail.com> wrote:

2010/4/14 Simon Riggs <simon@2ndquadrant.com>:

It would be a very useful test case to publish.

I'm still struggling to generate remaing WAL records.

Sure, but when you've got it all, please share. I'd like to see us
have a much larger set of tests than the "make check" regression
tests which would get run less frequently but test a lot more. This
sounds like a good one to include.

-Kevin

#7Bruce Momjian
bruce@momjian.us
In reply to: Kevin Grittner (#6)
Re: How to generate specific WAL records?

Kevin Grittner wrote:

Koichi Suzuki <koichi.szk@gmail.com> wrote:

2010/4/14 Simon Riggs <simon@2ndquadrant.com>:

It would be a very useful test case to publish.

I'm still struggling to generate remaing WAL records.

Sure, but when you've got it all, please share. I'd like to see us
have a much larger set of tests than the "make check" regression
tests which would get run less frequently but test a lot more. This
sounds like a good one to include.

Agreed. This test belongs in our CVS tree so we can maintain it as we
add new WAL types.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

#8Koichi Suzuki
koichi.szk@gmail.com
In reply to: Bruce Momjian (#7)
Re: How to generate specific WAL records?

Now I've tested almost all WAL record, including CLOG_TRUNCATE and all
the GIST and GIN-related WALs.

I'm still struggling to find how to have XLOG_HEAP_INIT_PAGE and
XLOG_BTREE_INSERT_META. I'm trying to find how to have these WAL
records but it's a great help if anyone suggests me how.

Thank you very much in advance;
----------
Koichi Suzuki

2010/4/15 Bruce Momjian <bruce@momjian.us>:

Show quoted text

Kevin Grittner wrote:

Koichi Suzuki <koichi.szk@gmail.com> wrote:

2010/4/14 Simon Riggs <simon@2ndquadrant.com>:

It would be a very useful test case to publish.

I'm still struggling to generate remaing WAL records.

Sure, but when you've got it all, please share.  I'd like to see us
have a much larger set of tests than the "make check" regression
tests which would get run less frequently but test a lot more.  This
sounds like a good one to include.

Agreed.  This test belongs in our CVS tree so we can maintain it as we
add new WAL types.

--
 Bruce Momjian  <bruce@momjian.us>        http://momjian.us
 EnterpriseDB                             http://enterprisedb.com