pgsql: Add amcheck extension to contrib.

Started by Andres Freundabout 9 years ago4 messageshackers
Jump to latest
#1Andres Freund
andres@anarazel.de

Add amcheck extension to contrib.

This is the beginning of a collection of SQL-callable functions to
verify the integrity of data files. For now it only contains code to
verify B-Tree indexes.

This adds two SQL-callable functions, validating B-Tree consistency to
a varying degree. Check the, extensive, docs for details.

The goal is to later extend the coverage of the module to further
access methods, possibly including the heap. Once checks for
additional access methods exist, we'll likely add some "dispatch"
functions that cover multiple access methods.

Author: Peter Geoghegan, editorialized by Andres Freund
Reviewed-By: Andres Freund, Tomas Vondra, Thomas Munro,
Anastasia Lubennikova, Robert Haas, Amit Langote
Discussion: CAM3SWZQzLMhMwmBqjzK+pRKXrNUZ4w90wYMUWfkeV8mZ3Debvw@mail.gmail.com

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/3717dc149ecf44b8be95350a68605ba7299474fd

Modified Files
--------------
contrib/Makefile | 1 +
contrib/amcheck/Makefile | 21 +
contrib/amcheck/amcheck--1.0.sql | 24 +
contrib/amcheck/amcheck.control | 5 +
contrib/amcheck/expected/check.out | 1 +
contrib/amcheck/expected/check_btree.out | 90 +++
contrib/amcheck/sql/check.sql | 1 +
contrib/amcheck/sql/check_btree.sql | 59 ++
contrib/amcheck/verify_nbtree.c | 1249 ++++++++++++++++++++++++++++++
doc/src/sgml/amcheck.sgml | 273 +++++++
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/filelist.sgml | 1 +
src/tools/pgindent/typedefs.list | 2 +
13 files changed, 1728 insertions(+)

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#1)
Re: [COMMITTERS] pgsql: Add amcheck extension to contrib.

Andres Freund <andres@anarazel.de> writes:

Add amcheck extension to contrib.

axolotl just failed on this:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=axolotl&amp;dt=2017-03-13%2017%3A49%3A24

***************
*** 78,86 ****

-- make sure we don't have any leftover locks
SELECT * FROM pg_locks WHERE relation IN ('bttest_a_idx'::regclass, 'bttest_b_idx'::regclass);
! locktype | database | relation | page | tuple | virtualxid | transactionid | classid | objid | objsubid | virtualtransaction | pid | mode | granted | fastpath
! ----------+----------+----------+------+-------+------------+---------------+---------+-------+----------+--------------------+-----+------+---------+----------
! (0 rows)

  COMMIT;
  -- cleanup
--- 78,87 ----

-- make sure we don't have any leftover locks
SELECT * FROM pg_locks WHERE relation IN ('bttest_a_idx'::regclass, 'bttest_b_idx'::regclass);
! locktype | database | relation | page | tuple | virtualxid | transactionid | classid | objid | objsubid | virtualtransaction | pid | mode | granted | fastpath
! ----------+----------+----------+------+-------+------------+---------------+---------+-------+----------+--------------------+-------+-----------------+---------+----------
! relation | 57562 | 57573 | | | | | | | | 4/29 | 20342 | AccessShareLock | t | t
! (1 row)

COMMIT;
-- cleanup

I could be wrong, but the most obvious explanation for this failure is
that autovacuum had a lock on the table or index when we looked.
Even if that isn't why axolotl failed in this particular case, I think
it's dead certain that we will see such failures from time to time
if this test script isn't tightened up. IIUC what the test is trying
to look for, I think adding "AND pid = pg_backend_pid()" to this query
would be an appropriate fix.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#2)
Re: [COMMITTERS] pgsql: Add amcheck extension to contrib.

Hi,

On 2017-03-13 15:45:01 -0400, Tom Lane wrote:

I could be wrong, but the most obvious explanation for this failure is
that autovacuum had a lock on the table or index when we looked.
Even if that isn't why axolotl failed in this particular case, I think
it's dead certain that we will see such failures from time to time
if this test script isn't tightened up. IIUC what the test is trying
to look for, I think adding "AND pid = pg_backend_pid()" to this query
would be an appropriate fix.

Yes, that sounds reasonable. Will do in a bit. Thanks for noticing.

- Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4Andres Freund
andres@anarazel.de
In reply to: Andres Freund (#3)
Re: [COMMITTERS] pgsql: Add amcheck extension to contrib.

On 2017-03-13 14:09:39 -0700, Andres Freund wrote:

Hi,

On 2017-03-13 15:45:01 -0400, Tom Lane wrote:

I could be wrong, but the most obvious explanation for this failure is
that autovacuum had a lock on the table or index when we looked.
Even if that isn't why axolotl failed in this particular case, I think
it's dead certain that we will see such failures from time to time
if this test script isn't tightened up. IIUC what the test is trying
to look for, I think adding "AND pid = pg_backend_pid()" to this query
would be an appropriate fix.

Yes, that sounds reasonable. Will do in a bit. Thanks for noticing.

Pushed.

- Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers