pageinspect patch, for showing tuple data
Started by Nikolay Shaplovover 10 years ago43 messages
Hi!
I've created a patch for pageinspect that allows to see data stored in the
tuple.
This patch has two main purposes:
1. Practical: Make manual DB recovery more simple
2. Educational: Seeing what data is actually stored in tuple, allows to get
better understanding of how does postgres actually works.
This patch adds several new functions, available from SQL queries. All these
functions are based on heap_page_items, but accept slightly different
arguments and has one additional column at the result set:
heap_page_tuples - accepts relation name, and bulkno, and returns usual
heap_page_items set with additional column that contain snapshot of tuple data
area stored in bytea.
heap_page_tuples_attributes - same as heap_page_tuples, but instead of single
tuple data bytea snapshot, it has array of bytea values, that were splitted
into attributes as they would be spitted by nocachegetattr function (I
actually reimplemented this function main algorithm to get this done)
heap_page_tuples_attrs_detoasted - same as heap_page_tuples_attrs, but all
varlen attributes values that were compressed or TOASTed, are replaced with
unTOASTed and uncompressed values.
There is also one strange function: _heap_page_items it is useless for
practical purposes. As heap_page_items it accepts page data as bytea, but also
get a relation name. It tries to apply tuple descriptor of that relation to
that page data.
This would allow you to try to read page data from one table using tuple
descriptor from anther. A strange idea, one should say. But this will allow
you: a) See how wrong data can be interpreted (educational purpose).
b) I have plenty of sanity check while reading parsing that tuple, for this
function I've changed error level from ERROR to WARNING. This function will
allow to write proper tests that all these checks work as they were designed
(I hope to write these tests sooner or later)
I've also added raw tuple data output to original heap_page_items function,
thought I am not sure if it is good idea. I just can add it there so I did it.
May be it would be better to change it back for better backward compatibility.
Attached patched is in "almost ready" state. It has some formatting issues.
I'd like to hear HACKER's opinion before finishing it and sending to
commitfest.
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
pageinspect.difftext/x-patch; charset=UTF-8; name=pageinspect.diffDownload
diff --git a/contrib/pageinspect/Makefile b/contrib/pageinspect/Makefile
index aec5258..e4bc1af 100644
--- a/contrib/pageinspect/Makefile
+++ b/contrib/pageinspect/Makefile
@@ -5,7 +5,7 @@ OBJS = rawpage.o heapfuncs.o btreefuncs.o fsmfuncs.o \
brinfuncs.o ginfuncs.o $(WIN32RES)
EXTENSION = pageinspect
-DATA = pageinspect--1.3.sql pageinspect--1.2--1.3.sql \
+DATA = pageinspect--1.4.sql pageinspect--1.3--1.4.sql pageinspect--1.2--1.3.sql \
pageinspect--1.1--1.2.sql pageinspect--1.0--1.1.sql \
pageinspect--unpackaged--1.0.sql
PGFILEDESC = "pageinspect - functions to inspect contents of database pages"
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 8d1666c..e296619 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -29,7 +29,12 @@
#include "funcapi.h"
#include "utils/builtins.h"
#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/rel.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_type.h"
+#include "rawpage.h"
/*
* bits_to_text
@@ -53,6 +58,9 @@ bits_to_text(bits8 *bits, int len)
return str;
}
+Datum heap_page_items_internal(FunctionCallInfo fcinfo, bytea *raw_page, text *relname, int error_level, bool do_detoast);
+void fill_header_data(FuncCallContext *fctx, Datum *values, bool *nulls, bool do_detoast);
+void split_tuple_data(FuncCallContext *fctx, HeapTupleHeader tuphdr, Datum *values, bool *nulls, bool do_detoast);
/*
* heap_page_items
@@ -66,38 +74,98 @@ typedef struct heap_page_items_state
TupleDesc tupd;
Page page;
uint16 offset;
+ TupleDesc page_tuple_desc;
+ int raw_page_size;
+ int error_level;
} heap_page_items_state;
Datum
heap_page_items(PG_FUNCTION_ARGS)
{
bytea *raw_page = PG_GETARG_BYTEA_P(0);
+ text *relname = PG_NARGS()==2 ? PG_GETARG_TEXT_P(1) : NULL;
+
+ /*
+ * Error level is only used while splitting tuple into array of attributes
+ * This is done only for _heap_page_items function. _heap_page_items is intended
+ * for educational and research purposes, so we should change all error checks
+ * to warnings
+ */
+ return heap_page_items_internal(fcinfo, raw_page, relname, WARNING, false);
+}
+
+PG_FUNCTION_INFO_V1(heap_page_tuples);
+Datum
+heap_page_tuples(PG_FUNCTION_ARGS)
+{
+ text *relname = PG_GETARG_TEXT_P(0);
+ uint32 blkno = PG_GETARG_UINT32(1);
+ bytea *raw_page;
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ raw_page = get_raw_page_internal(relname, MAIN_FORKNUM, blkno);
+ }
+ return heap_page_items_internal(fcinfo, raw_page, NULL, ERROR, false);
+}
+
+PG_FUNCTION_INFO_V1(heap_page_tuples_attrs);
+Datum
+heap_page_tuples_attrs(PG_FUNCTION_ARGS)
+{
+ text *relname = PG_GETARG_TEXT_P(0);
+ uint32 blkno = PG_GETARG_UINT32(1);
+ bytea *raw_page;
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ raw_page = get_raw_page_internal(relname, MAIN_FORKNUM, blkno);
+ }
+ return heap_page_items_internal(fcinfo, raw_page, relname, ERROR, false);
+}
+
+PG_FUNCTION_INFO_V1(heap_page_tuples_attrs_detoasted);
+Datum
+heap_page_tuples_attrs_detoasted(PG_FUNCTION_ARGS)
+{
+ text *relname = PG_GETARG_TEXT_P(0);
+ uint32 blkno = PG_GETARG_UINT32(1);
+ bytea *raw_page;
+
+ if (SRF_IS_FIRSTCALL())
+ {
+ raw_page = get_raw_page_internal(relname, MAIN_FORKNUM, blkno);
+ }
+ return heap_page_items_internal(fcinfo, raw_page, relname, ERROR, true);
+}
+
+Datum
+heap_page_items_internal(FunctionCallInfo fcinfo, bytea *raw_page, text *relname, int error_level, bool do_detoast)
+{
heap_page_items_state *inter_call_data = NULL;
FuncCallContext *fctx;
- int raw_page_size;
if (!superuser())
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
- (errmsg("must be superuser to use raw page functions"))));
-
- raw_page_size = VARSIZE(raw_page) - VARHDRSZ;
+ (errmsg("must be superuser to use raw page functions"))));
if (SRF_IS_FIRSTCALL())
{
TupleDesc tupdesc;
- MemoryContext mctx;
-
- if (raw_page_size < SizeOfPageHeaderData)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("input page too small (%d bytes)", raw_page_size)));
+ MemoryContext mctx;
fctx = SRF_FIRSTCALL_INIT();
mctx = MemoryContextSwitchTo(fctx->multi_call_memory_ctx);
inter_call_data = palloc(sizeof(heap_page_items_state));
+ inter_call_data->raw_page_size = VARSIZE(raw_page) - VARHDRSZ;
+ if (inter_call_data->raw_page_size < SizeOfPageHeaderData)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("input page too small (%d bytes)", inter_call_data->raw_page_size)));
+
/* Build a tuple descriptor for our result type */
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
@@ -105,7 +173,21 @@ heap_page_items(PG_FUNCTION_ARGS)
inter_call_data->tupd = tupdesc;
inter_call_data->offset = FirstOffsetNumber;
- inter_call_data->page = VARDATA(raw_page);
+
+ inter_call_data->page = palloc(BLCKSZ);
+ memcpy(inter_call_data->page, VARDATA(raw_page), BLCKSZ);
+
+ inter_call_data->page_tuple_desc = NULL;
+ inter_call_data->error_level = error_level;
+
+ if (relname)
+ {
+ RangeVar * relrv = makeRangeVarFromNameList(textToQualifiedNameList(relname));
+ RelationData * rel = relation_openrv(relrv, NoLock);
+
+ inter_call_data->page_tuple_desc = CreateTupleDescCopyConstr(rel->rd_att);
+ relation_close(rel, NoLock);
+ }
fctx->max_calls = PageGetMaxOffsetNumber(inter_call_data->page);
fctx->user_fctx = inter_call_data;
@@ -114,112 +196,255 @@ heap_page_items(PG_FUNCTION_ARGS)
}
fctx = SRF_PERCALL_SETUP();
+
inter_call_data = fctx->user_fctx;
if (fctx->call_cntr < fctx->max_calls)
{
- Page page = inter_call_data->page;
+ Datum values[14];
+ bool nulls[14];
HeapTuple resultTuple;
Datum result;
- ItemId id;
- Datum values[13];
- bool nulls[13];
- uint16 lp_offset;
- uint16 lp_flags;
- uint16 lp_len;
memset(nulls, 0, sizeof(nulls));
- /* Extract information from the line pointer */
+ fill_header_data(fctx, values, nulls, do_detoast);
+
+ /* Build and return the result tuple. */
+ resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
+ result = HeapTupleGetDatum(resultTuple);
+
+ inter_call_data->offset++;
+
+ SRF_RETURN_NEXT(fctx, result);
+ }
+ else
+ SRF_RETURN_DONE(fctx);
+}
- id = PageGetItemId(page, inter_call_data->offset);
- lp_offset = ItemIdGetOffset(id);
- lp_flags = ItemIdGetFlags(id);
- lp_len = ItemIdGetLength(id);
- values[0] = UInt16GetDatum(inter_call_data->offset);
- values[1] = UInt16GetDatum(lp_offset);
- values[2] = UInt16GetDatum(lp_flags);
- values[3] = UInt16GetDatum(lp_len);
+void
+fill_header_data(FuncCallContext *fctx, Datum *values, bool *nulls, bool do_detoast)
+{
+ Page page;
+ ItemId id;
+ uint16 lp_offset;
+ uint16 lp_flags;
+ uint16 lp_len;
+ heap_page_items_state *inter_call_data = NULL;
+ inter_call_data = fctx->user_fctx;
+ page = inter_call_data->page;
+
+ /* Extract information from the line pointer */
+
+ id = PageGetItemId(page, inter_call_data->offset);
+
+ lp_offset = ItemIdGetOffset(id);
+ lp_flags = ItemIdGetFlags(id);
+ lp_len = ItemIdGetLength(id);
+
+ values[0] = UInt16GetDatum(inter_call_data->offset);
+ values[1] = UInt16GetDatum(lp_offset);
+ values[2] = UInt16GetDatum(lp_flags);
+ values[3] = UInt16GetDatum(lp_len);
+
+ /*
+ * We do just enough validity checking to make sure we don't reference
+ * data outside the page passed to us. The page could be corrupt in
+ * many other ways, but at least we won't crash.
+ */
+ if (ItemIdHasStorage(id) &&
+ lp_len >= MinHeapTupleSize &&
+ lp_offset == MAXALIGN(lp_offset) &&
+ lp_offset + lp_len <= inter_call_data->raw_page_size)
+ {
+ HeapTupleHeader tuphdr;
+ int bits_len;
+ bytea *tuple_data_bytea;
+ int tuple_data_len;
+
+ /* Extract information from the tuple header */
+
+ tuphdr = (HeapTupleHeader) PageGetItem(page, id);
+
+ values[4] = UInt32GetDatum(HeapTupleHeaderGetRawXmin(tuphdr));
+ values[5] = UInt32GetDatum(HeapTupleHeaderGetRawXmax(tuphdr));
+ values[6] = UInt32GetDatum(HeapTupleHeaderGetRawCommandId(tuphdr)); /* shared with xvac */
+ values[7] = PointerGetDatum(&tuphdr->t_ctid);
+ values[8] = UInt32GetDatum(tuphdr->t_infomask2);
+ values[9] = UInt32GetDatum(tuphdr->t_infomask);
+ values[10] = UInt8GetDatum(tuphdr->t_hoff);
+
+ /* Copy raw tuple data into bytea attribute */
+ tuple_data_len = lp_len - tuphdr->t_hoff;
+ tuple_data_bytea = (bytea *) palloc(tuple_data_len + VARHDRSZ);
+ SET_VARSIZE(tuple_data_bytea, tuple_data_len + VARHDRSZ);
+ memcpy(VARDATA(tuple_data_bytea), (char *) tuphdr + tuphdr->t_hoff, tuple_data_len);
+ values[13] = PointerGetDatum(tuple_data_bytea);
+
+ if (inter_call_data->page_tuple_desc)
+ {
+ split_tuple_data(fctx, tuphdr, values, nulls, do_detoast);
+ }
/*
- * We do just enough validity checking to make sure we don't reference
- * data outside the page passed to us. The page could be corrupt in
- * many other ways, but at least we won't crash.
+ * We already checked that the item is completely within the raw
+ * page passed to us, with the length given in the line pointer.
+ * Let's check that t_hoff doesn't point over lp_len, before using
+ * it to access t_bits and oid.
*/
- if (ItemIdHasStorage(id) &&
- lp_len >= MinHeapTupleSize &&
- lp_offset == MAXALIGN(lp_offset) &&
- lp_offset + lp_len <= raw_page_size)
+ if (tuphdr->t_hoff >= SizeofHeapTupleHeader &&
+ tuphdr->t_hoff <= lp_len &&
+ tuphdr->t_hoff == MAXALIGN(tuphdr->t_hoff))
{
- HeapTupleHeader tuphdr;
- int bits_len;
-
- /* Extract information from the tuple header */
-
- tuphdr = (HeapTupleHeader) PageGetItem(page, id);
-
- values[4] = UInt32GetDatum(HeapTupleHeaderGetRawXmin(tuphdr));
- values[5] = UInt32GetDatum(HeapTupleHeaderGetRawXmax(tuphdr));
- values[6] = UInt32GetDatum(HeapTupleHeaderGetRawCommandId(tuphdr)); /* shared with xvac */
- values[7] = PointerGetDatum(&tuphdr->t_ctid);
- values[8] = UInt32GetDatum(tuphdr->t_infomask2);
- values[9] = UInt32GetDatum(tuphdr->t_infomask);
- values[10] = UInt8GetDatum(tuphdr->t_hoff);
-
- /*
- * We already checked that the item is completely within the raw
- * page passed to us, with the length given in the line pointer.
- * Let's check that t_hoff doesn't point over lp_len, before using
- * it to access t_bits and oid.
- */
- if (tuphdr->t_hoff >= SizeofHeapTupleHeader &&
- tuphdr->t_hoff <= lp_len &&
- tuphdr->t_hoff == MAXALIGN(tuphdr->t_hoff))
+ if (tuphdr->t_infomask & HEAP_HASNULL)
{
- if (tuphdr->t_infomask & HEAP_HASNULL)
- {
- bits_len = tuphdr->t_hoff -
- offsetof(HeapTupleHeaderData, t_bits);
-
- values[11] = CStringGetTextDatum(
- bits_to_text(tuphdr->t_bits, bits_len * 8));
- }
- else
- nulls[11] = true;
+ bits_len = tuphdr->t_hoff -
+ offsetof(HeapTupleHeaderData, t_bits);
- if (tuphdr->t_infomask & HEAP_HASOID)
- values[12] = HeapTupleHeaderGetOid(tuphdr);
- else
- nulls[12] = true;
+ values[11] = CStringGetTextDatum(
+ bits_to_text(tuphdr->t_bits, bits_len * 8));
}
else
- {
nulls[11] = true;
+
+ if (tuphdr->t_infomask & HEAP_HASOID)
+ values[12] = HeapTupleHeaderGetOid(tuphdr);
+ else
nulls[12] = true;
- }
}
else
{
- /*
- * The line pointer is not used, or it's invalid. Set the rest of
- * the fields to NULL
- */
- int i;
-
- for (i = 4; i <= 12; i++)
- nulls[i] = true;
+ nulls[11] = true;
+ nulls[12] = true;
}
+ }
+ else
+ {
+ /*
+ * The line pointer is not used, or it's invalid. Set the rest of
+ * the fields to NULL
+ */
+ int i;
- /* Build and return the result tuple. */
- resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ for (i = 4; i <= 13; i++)
+ nulls[i] = true;
+ }
+}
- inter_call_data->offset++;
- SRF_RETURN_NEXT(fctx, result);
+void
+split_tuple_data(FuncCallContext *fctx, HeapTupleHeader tuphdr, Datum *values, bool *nulls, bool do_detoast)
+{
+ TupleDesc tuple_desc;
+ ArrayBuildState *raw_attrs;
+ uint16 lp_len;
+ char *tuple_data_p;
+ int nattrs;
+ int i;
+ int off;
+ heap_page_items_state *inter_call_data;
+
+ inter_call_data = fctx->user_fctx;
+ lp_len = DatumGetUInt16(values[3]);
+
+ /*
+ * Here we reimplement the basic functionality of nocachegetattr from
+ * backend/access/common/heaptuple.c whitch is basically used for fetching
+ * attributes from tuple when it is not cached. We can not use nocachegetattr here
+ * directly, because we should ignore all cache optimisations and other stuff
+ * just get binary data as it is.
+ */
+
+ tuple_desc = inter_call_data->page_tuple_desc;
+ raw_attrs = initArrayResult(BYTEAOID,fctx->multi_call_memory_ctx,0);
+ tuple_data_p = (char *) tuphdr + tuphdr->t_hoff;
+
+ off = 0;
+ nattrs = tuple_desc->natts;
+ if (inter_call_data->error_level &&
+ nattrs < (tuphdr->t_infomask2 & HEAP_NATTS_MASK))
+ {
+ ereport(inter_call_data->error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("Data corruption: number of attributes in tuple header is greater than number of attributes in tuple descripor")));
}
- else
- SRF_RETURN_DONE(fctx);
+
+ for(i=0; i < nattrs; i++)
+ {
+ Form_pg_attribute attr;
+ Datum raw_attr;
+ bool is_null;
+
+ attr = tuple_desc->attrs[i];
+ is_null = (tuphdr->t_infomask & HEAP_HASNULL) && att_isnull(i, tuphdr->t_bits);
+ /*
+ * Tuple header can specify less attributes then tuple descriptor
+ * as ALTER TABLE ADD COLUMN without DEFAULT keyword does not
+ * actualy change tuples in pages, so attributes with numbers greater
+ * than tuphdr->t_infomask2 & HEAP_NATTS_MASK should be treated as NULL
+ */
+ if (i >= (tuphdr->t_infomask2 & HEAP_NATTS_MASK) )
+ is_null = 1;
+ if (!is_null)
+ {
+ int len;
+ bytea * attr_data;
+ if (attr->attlen == -1)
+ {
+ off = att_align_pointer(off, tuple_desc->attrs[i]->attalign, -1, tuple_data_p + off);
+ /*
+ * As VARSIZE_ANY throws an exeption if it can't properly detect
+ * type of external storage in macros VARTAG_SIZE, so we repeat
+ * this check here to preform nicer error handling
+ */
+ if ( inter_call_data->error_level &&
+ VARATT_IS_1B_E(tuple_data_p + off) &&
+ VARTAG_EXTERNAL(tuple_data_p + off) != VARTAG_INDIRECT &&
+ VARTAG_EXTERNAL(tuple_data_p + off) != VARTAG_ONDISK)
+ {
+ ereport(inter_call_data->error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("Data corruption: First byte of varlen attr seems to be corrupted")));
+ }
+ len = VARSIZE_ANY(tuple_data_p + off);
+ }
+ else
+ {
+ off = att_align_nominal(off, tuple_desc->attrs[i]->attalign);
+ len = attr->attlen;
+ }
+ if ( inter_call_data->error_level &&
+ lp_len < tuphdr->t_hoff + off + len)
+ {
+ ereport(inter_call_data->error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("Data corruption: Iterating over tuple data reached out of actual tuple size")));
+ }
+ attr_data = (bytea *) palloc(len + VARHDRSZ);
+ SET_VARSIZE(attr_data, len + VARHDRSZ);
+ memcpy(VARDATA(attr_data), tuple_data_p + off, len);
+ raw_attr = PointerGetDatum(attr_data);
+
+ if ( attr->attlen == -1 && do_detoast)
+ {
+ Datum raw_attr_copy;
+ raw_attr_copy = PointerGetDatum(PG_DETOAST_DATUM_COPY(tuple_data_p + off));
+ pfree(attr_data);
+ raw_attr = raw_attr_copy;
+ }
+ off = att_addlength_pointer(off, tuple_desc->attrs[i]->attlen, tuple_data_p + off);
+ }
+ raw_attrs = accumArrayResult(raw_attrs, raw_attr, is_null, BYTEAOID, fctx->multi_call_memory_ctx);
+ }
+ if (inter_call_data->error_level &&
+ lp_len != tuphdr->t_hoff + off)
+ {
+ ereport(inter_call_data->error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("Data corruption: Iterating over tuple data did not actualy reach tuple end")));
+ }
+ pfree(DatumGetPointer(values[13]));
+ values[13] = makeArrayResult(raw_attrs, fctx->multi_call_memory_ctx);
}
diff --git a/contrib/pageinspect/pageinspect--1.3--1.4.sql b/contrib/pageinspect/pageinspect--1.3--1.4.sql
new file mode 100644
index 0000000..6f71020
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.3--1.4.sql
@@ -0,0 +1,110 @@
+/* contrib/pageinspect/pageinspect--1.3--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via ALTER EXTENSION
+\echo Use "ALTER EXTENSION pageinspect UPDATE TO '1.4'" to load this file. \quit
+
+--
+-- heap_page_items()
+--
+DROP FUNCTION heap_page_items(bytea);
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_items()
+--
+CREATE FUNCTION _heap_page_items(IN page bytea, IN relname text,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+
+
+CREATE FUNCTION heap_page_tuples(IN relname text, blkno int,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_tuples'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION heap_page_tuples_attrs(IN relname text, blkno int,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_tuples_attrs'
+LANGUAGE C STRICT;
+
+
+CREATE FUNCTION heap_page_tuples_attrs_detoasted(IN relname text, blkno int,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_tuples_attrs_detoasted'
+LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect--1.3.sql b/contrib/pageinspect/pageinspect--1.3.sql
deleted file mode 100644
index a99e058..0000000
--- a/contrib/pageinspect/pageinspect--1.3.sql
+++ /dev/null
@@ -1,189 +0,0 @@
-/* contrib/pageinspect/pageinspect--1.3.sql */
-
--- complain if script is sourced in psql, rather than via CREATE EXTENSION
-\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
-
---
--- get_raw_page()
---
-CREATE FUNCTION get_raw_page(text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page'
-LANGUAGE C STRICT;
-
-CREATE FUNCTION get_raw_page(text, text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page_fork'
-LANGUAGE C STRICT;
-
---
--- page_header()
---
-CREATE FUNCTION page_header(IN page bytea,
- OUT lsn pg_lsn,
- OUT checksum smallint,
- OUT flags smallint,
- OUT lower smallint,
- OUT upper smallint,
- OUT special smallint,
- OUT pagesize smallint,
- OUT version smallint,
- OUT prune_xid xid)
-AS 'MODULE_PATHNAME', 'page_header'
-LANGUAGE C STRICT;
-
---
--- heap_page_items()
---
-CREATE FUNCTION heap_page_items(IN page bytea,
- OUT lp smallint,
- OUT lp_off smallint,
- OUT lp_flags smallint,
- OUT lp_len smallint,
- OUT t_xmin xid,
- OUT t_xmax xid,
- OUT t_field3 int4,
- OUT t_ctid tid,
- OUT t_infomask2 integer,
- OUT t_infomask integer,
- OUT t_hoff smallint,
- OUT t_bits text,
- OUT t_oid oid)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'heap_page_items'
-LANGUAGE C STRICT;
-
---
--- bt_metap()
---
-CREATE FUNCTION bt_metap(IN relname text,
- OUT magic int4,
- OUT version int4,
- OUT root int4,
- OUT level int4,
- OUT fastroot int4,
- OUT fastlevel int4)
-AS 'MODULE_PATHNAME', 'bt_metap'
-LANGUAGE C STRICT;
-
---
--- bt_page_stats()
---
-CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
- OUT blkno int4,
- OUT type "char",
- OUT live_items int4,
- OUT dead_items int4,
- OUT avg_item_size int4,
- OUT page_size int4,
- OUT free_size int4,
- OUT btpo_prev int4,
- OUT btpo_next int4,
- OUT btpo int4,
- OUT btpo_flags int4)
-AS 'MODULE_PATHNAME', 'bt_page_stats'
-LANGUAGE C STRICT;
-
---
--- bt_page_items()
---
-CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
- OUT itemoffset smallint,
- OUT ctid tid,
- OUT itemlen smallint,
- OUT nulls bool,
- OUT vars bool,
- OUT data text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'bt_page_items'
-LANGUAGE C STRICT;
-
---
--- brin_page_type()
---
-CREATE FUNCTION brin_page_type(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'brin_page_type'
-LANGUAGE C STRICT;
-
---
--- brin_metapage_info()
---
-CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
- OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
-AS 'MODULE_PATHNAME', 'brin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- brin_revmap_data()
---
-CREATE FUNCTION brin_revmap_data(IN page bytea,
- OUT pages tid)
-RETURNS SETOF tid
-AS 'MODULE_PATHNAME', 'brin_revmap_data'
-LANGUAGE C STRICT;
-
---
--- brin_page_items()
---
-CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
- OUT itemoffset int,
- OUT blknum int,
- OUT attnum int,
- OUT allnulls bool,
- OUT hasnulls bool,
- OUT placeholder bool,
- OUT value text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'brin_page_items'
-LANGUAGE C STRICT;
-
---
--- fsm_page_contents()
---
-CREATE FUNCTION fsm_page_contents(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'fsm_page_contents'
-LANGUAGE C STRICT;
-
---
--- GIN functions
---
-
---
--- gin_metapage_info()
---
-CREATE FUNCTION gin_metapage_info(IN page bytea,
- OUT pending_head bigint,
- OUT pending_tail bigint,
- OUT tail_free_size int4,
- OUT n_pending_pages bigint,
- OUT n_pending_tuples bigint,
- OUT n_total_pages bigint,
- OUT n_entry_pages bigint,
- OUT n_data_pages bigint,
- OUT n_entries bigint,
- OUT version int4)
-AS 'MODULE_PATHNAME', 'gin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- gin_page_opaque_info()
---
-CREATE FUNCTION gin_page_opaque_info(IN page bytea,
- OUT rightlink bigint,
- OUT maxoff int4,
- OUT flags text[])
-AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
-LANGUAGE C STRICT;
-
---
--- gin_leafpage_items()
---
-CREATE FUNCTION gin_leafpage_items(IN page bytea,
- OUT first_tid tid,
- OUT nbytes int2,
- OUT tids tid[])
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'gin_leafpage_items'
-LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect--1.4.sql b/contrib/pageinspect/pageinspect--1.4.sql
new file mode 100644
index 0000000..9bb62d3
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.4.sql
@@ -0,0 +1,271 @@
+/* contrib/pageinspect/pageinspect--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
+
+--
+-- get_raw_page()
+--
+CREATE FUNCTION get_raw_page(text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION get_raw_page(text, text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page_fork'
+LANGUAGE C STRICT;
+
+--
+-- page_header()
+--
+CREATE FUNCTION page_header(IN page bytea,
+ OUT lsn pg_lsn,
+ OUT checksum smallint,
+ OUT flags smallint,
+ OUT lower smallint,
+ OUT upper smallint,
+ OUT special smallint,
+ OUT pagesize smallint,
+ OUT version smallint,
+ OUT prune_xid xid)
+AS 'MODULE_PATHNAME', 'page_header'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_items()
+--
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_items()
+--
+CREATE FUNCTION _heap_page_items(IN page bytea, IN relname text,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION heap_page_tuples(IN relname text, blkno int,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_tuples'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION heap_page_tuples_attrs(IN relname text, blkno int,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_tuples_attrs'
+LANGUAGE C STRICT;
+
+
+CREATE FUNCTION heap_page_tuples_attrs_detoasted(IN relname text, blkno int,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_tuples_attrs_detoasted'
+LANGUAGE C STRICT;
+
+--
+-- bt_metap()
+--
+CREATE FUNCTION bt_metap(IN relname text,
+ OUT magic int4,
+ OUT version int4,
+ OUT root int4,
+ OUT level int4,
+ OUT fastroot int4,
+ OUT fastlevel int4)
+AS 'MODULE_PATHNAME', 'bt_metap'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_stats()
+--
+CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
+ OUT blkno int4,
+ OUT type "char",
+ OUT live_items int4,
+ OUT dead_items int4,
+ OUT avg_item_size int4,
+ OUT page_size int4,
+ OUT free_size int4,
+ OUT btpo_prev int4,
+ OUT btpo_next int4,
+ OUT btpo int4,
+ OUT btpo_flags int4)
+AS 'MODULE_PATHNAME', 'bt_page_stats'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_items()
+--
+CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
+ OUT itemoffset smallint,
+ OUT ctid tid,
+ OUT itemlen smallint,
+ OUT nulls bool,
+ OUT vars bool,
+ OUT data text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'bt_page_items'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_type()
+--
+CREATE FUNCTION brin_page_type(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'brin_page_type'
+LANGUAGE C STRICT;
+
+--
+-- brin_metapage_info()
+--
+CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
+ OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
+AS 'MODULE_PATHNAME', 'brin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- brin_revmap_data()
+--
+CREATE FUNCTION brin_revmap_data(IN page bytea,
+ OUT pages tid)
+RETURNS SETOF tid
+AS 'MODULE_PATHNAME', 'brin_revmap_data'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_items()
+--
+CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
+ OUT itemoffset int,
+ OUT blknum int,
+ OUT attnum int,
+ OUT allnulls bool,
+ OUT hasnulls bool,
+ OUT placeholder bool,
+ OUT value text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'brin_page_items'
+LANGUAGE C STRICT;
+
+--
+-- fsm_page_contents()
+--
+CREATE FUNCTION fsm_page_contents(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'fsm_page_contents'
+LANGUAGE C STRICT;
+
+--
+-- GIN functions
+--
+
+--
+-- gin_metapage_info()
+--
+CREATE FUNCTION gin_metapage_info(IN page bytea,
+ OUT pending_head bigint,
+ OUT pending_tail bigint,
+ OUT tail_free_size int4,
+ OUT n_pending_pages bigint,
+ OUT n_pending_tuples bigint,
+ OUT n_total_pages bigint,
+ OUT n_entry_pages bigint,
+ OUT n_data_pages bigint,
+ OUT n_entries bigint,
+ OUT version int4)
+AS 'MODULE_PATHNAME', 'gin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_page_opaque_info()
+--
+CREATE FUNCTION gin_page_opaque_info(IN page bytea,
+ OUT rightlink bigint,
+ OUT maxoff int4,
+ OUT flags text[])
+AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_leafpage_items()
+--
+CREATE FUNCTION gin_leafpage_items(IN page bytea,
+ OUT first_tid tid,
+ OUT nbytes int2,
+ OUT tids tid[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'gin_leafpage_items'
+LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect.control b/contrib/pageinspect/pageinspect.control
index a9dab33..68c7d61 100644
--- a/contrib/pageinspect/pageinspect.control
+++ b/contrib/pageinspect/pageinspect.control
@@ -1,5 +1,5 @@
# pageinspect extension
comment = 'inspect the contents of database pages at a low level'
-default_version = '1.3'
+default_version = '1.4'
module_pathname = '$libdir/pageinspect'
relocatable = true
diff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.c
index 38c136f..6659ee5 100644
--- a/contrib/pageinspect/rawpage.c
+++ b/contrib/pageinspect/rawpage.c
@@ -26,11 +26,9 @@
#include "utils/pg_lsn.h"
#include "utils/rel.h"
-PG_MODULE_MAGIC;
-
-static bytea *get_raw_page_internal(text *relname, ForkNumber forknum,
- BlockNumber blkno);
+#include "rawpage.h"
+PG_MODULE_MAGIC;
/*
* get_raw_page
@@ -87,7 +85,7 @@ get_raw_page_fork(PG_FUNCTION_ARGS)
/*
* workhorse
*/
-static bytea *
+bytea *
get_raw_page_internal(text *relname, ForkNumber forknum, BlockNumber blkno)
{
bytea *raw_page;
diff --git a/contrib/pageinspect/rawpage.h b/contrib/pageinspect/rawpage.h
new file mode 100644
index 0000000..d9f2678
--- /dev/null
+++ b/contrib/pageinspect/rawpage.h
@@ -0,0 +1,20 @@
+/*-------------------------------------------------------------------------
+ *
+ * rawpage.h
+ *
+ * Copyright (c) 2007-2015, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * contrib/pageinspect/rawpage.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef __RAWPAGE_H__
+#define __RAWPAGE_H__
+
+
+bytea *get_raw_page_internal(text *relname, ForkNumber forknum,
+ BlockNumber blkno);
+
+#endif /*__RAWPAGE_H__*/
\ No newline at end of file
Re: pageinspect patch, for showing tuple data
On Mon, Aug 3, 2015 at 1:03 AM, Nikolay Shaplov
<n.shaplov@postgrespro.ru> wrote:
Hi!
I've created a patch for pageinspect that allows to see data stored in the
tuple.This patch has two main purposes:
1. Practical: Make manual DB recovery more simple
To what are you referring to in this case? Manual manipulation of
on-disk data manually?
b) I have plenty of sanity check while reading parsing that tuple, for this
function I've changed error level from ERROR to WARNING. This function will
allow to write proper tests that all these checks work as they were designed
(I hope to write these tests sooner or later)
+ ereport(inter_call_data->error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("Data corruption: Iterating over
tuple data reached out of actual tuple size")));
I don't think that the possibility to raise a WARNING is a good thing
in those code paths. If data is corrupted this may crash, and I am not
sure that anybody would want that even for educational purposes.
--
Michael--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
В письме от 3 августа 2015 14:30:46 пользователь Michael Paquier написал:
On Mon, Aug 3, 2015 at 1:03 AM, Nikolay Shaplov
<n.shaplov@postgrespro.ru> wrote:
Hi!
I've created a patch for pageinspect that allows to see data stored in the
tuple.This patch has two main purposes:
1. Practical: Make manual DB recovery more simple
To what are you referring to in this case? Manual manipulation of
on-disk data manually?
Yes, when DB is broken for example
b) I have plenty of sanity check while reading parsing that tuple, for
this
function I've changed error level from ERROR to WARNING. This function
will
allow to write proper tests that all these checks work as they were
designed (I hope to write these tests sooner or later)+ ereport(inter_call_data->error_level, + (errcode(ERRCODE_DATA_CORRUPTED), + errmsg("Data corruption: Iterating over tuple data reached out of actual tuple size"))); I don't think that the possibility to raise a WARNING is a good thing in those code paths. If data is corrupted this may crash, and I am not sure that anybody would want that even for educational purposes.
Hm... I considered _heap_page_items really very dangerous function, with big
red "Do not call it if you not sure what are you doing" warning. Reading data
with not proper attribute descriptor is dangerous any way. But when I wrote
that code, I did not have that checks at first, and it was really interesting
to subst one data with another and see what will happen. And I thought that
may be other explorers will like to do the same. And it is really possible
only in warning mode. So I left warnings only in _heap_page_items, and set
errors for all other cases.
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
Nikolay Shaplov wrote:
This patch adds several new functions, available from SQL queries. All these
functions are based on heap_page_items, but accept slightly different
arguments and has one additional column at the result set:heap_page_tuples - accepts relation name, and bulkno, and returns usual
heap_page_items set with additional column that contain snapshot of tuple data
area stored in bytea.
I think the API around get_raw_page is more useful generally. You might
have table blocks stored in a bytea column for instance, because you
extracted from some remote server and inserted into a local table for
examination. If you only accept relname/blkno, there's no way to
examine data other than blocks directly in the database dir, which is
limiting.
There is also one strange function: _heap_page_items it is useless for
practical purposes. As heap_page_items it accepts page data as bytea, but also
get a relation name. It tries to apply tuple descriptor of that relation to
that page data.
For BRIN, I added something similar (brin_page_items), but it receives
the index OID rather than name, and constructs a tuple descriptor to
read the data. I think OID is better than name generally. (You can
cast the relation name to regclass).
It's easy to misuse, but these functions are superuser-only, so there
should be no security issue at least. The possibility of a crash
remains real, though, so maybe we should try to come up with a way to
harden that.
--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
В письме от 3 августа 2015 15:35:23 Вы написали:
Nikolay Shaplov wrote:
This patch adds several new functions, available from SQL queries. All
these functions are based on heap_page_items, but accept slightly
different arguments and has one additional column at the result set:heap_page_tuples - accepts relation name, and bulkno, and returns usual
heap_page_items set with additional column that contain snapshot of tuple
data area stored in bytea.I think the API around get_raw_page is more useful generally. You might
have table blocks stored in a bytea column for instance, because you
extracted from some remote server and inserted into a local table for
examination. If you only accept relname/blkno, there's no way to
examine data other than blocks directly in the database dir, which is
limiting.There is also one strange function: _heap_page_items it is useless for
practical purposes. As heap_page_items it accepts page data as bytea, but
also get a relation name. It tries to apply tuple descriptor of that
relation to that page data.For BRIN, I added something similar (brin_page_items), but it receives
the index OID rather than name, and constructs a tuple descriptor to
read the data. I think OID is better than name generally. (You can
cast the relation name to regclass).It's easy to misuse, but these functions are superuser-only, so there
should be no security issue at least. The possibility of a crash
remains real, though, so maybe we should try to come up with a way to
harden that.
Hmm... may be you are right. I'll try to change it would take raw page and
OID.
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
В письме от 3 августа 2015 15:35:23 пользователь Alvaro Herrera написал:
Nikolay Shaplov wrote:
This patch adds several new functions, available from SQL queries. All
these functions are based on heap_page_items, but accept slightly
different arguments and has one additional column at the result set:heap_page_tuples - accepts relation name, and bulkno, and returns usual
heap_page_items set with additional column that contain snapshot of tuple
data area stored in bytea.I think the API around get_raw_page is more useful generally. You might
have table blocks stored in a bytea column for instance, because you
extracted from some remote server and inserted into a local table for
examination. If you only accept relname/blkno, there's no way to
examine data other than blocks directly in the database dir, which is
limiting.There is also one strange function: _heap_page_items it is useless for
practical purposes. As heap_page_items it accepts page data as bytea, but
also get a relation name. It tries to apply tuple descriptor of that
relation to that page data.For BRIN, I added something similar (brin_page_items), but it receives
the index OID rather than name, and constructs a tuple descriptor to
read the data. I think OID is better than name generally. (You can
cast the relation name to regclass).It's easy to misuse, but these functions are superuser-only, so there
should be no security issue at least. The possibility of a crash
remains real, though, so maybe we should try to come up with a way to
harden that.
I've done as you've said: Now patch adds two functions heap_page_item_attrs
and heap_page_item_detoast_attrs and these function takes raw_page and
relation OID as arguments. They also have third optional parameter that allows
to change error mode from ERROR to WARNING.
Is it ok now?
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
pageinspect_show_tuple_data.difftext/x-patch; charset=UTF-8; name=pageinspect_show_tuple_data.diffDownload
diff --git a/contrib/pageinspect/Makefile b/contrib/pageinspect/Makefile
index aec5258..e4bc1af 100644
--- a/contrib/pageinspect/Makefile
+++ b/contrib/pageinspect/Makefile
@@ -5,7 +5,7 @@ OBJS = rawpage.o heapfuncs.o btreefuncs.o fsmfuncs.o \
brinfuncs.o ginfuncs.o $(WIN32RES)
EXTENSION = pageinspect
-DATA = pageinspect--1.3.sql pageinspect--1.2--1.3.sql \
+DATA = pageinspect--1.4.sql pageinspect--1.3--1.4.sql pageinspect--1.2--1.3.sql \
pageinspect--1.1--1.2.sql pageinspect--1.0--1.1.sql \
pageinspect--unpackaged--1.0.sql
PGFILEDESC = "pageinspect - functions to inspect contents of database pages"
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 8d1666c..59eae0f 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -29,7 +29,12 @@
#include "funcapi.h"
#include "utils/builtins.h"
#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/rel.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_type.h"
+#include "rawpage.h"
/*
* bits_to_text
@@ -53,6 +58,9 @@ bits_to_text(bits8 *bits, int len)
return str;
}
+Datum heap_page_items_internal(FunctionCallInfo fcinfo, bytea *raw_page, Oid rel_oid, int error_level, bool do_detoast);
+void fill_header_data(FuncCallContext *fctx, Datum *values, bool *nulls, bool do_detoast);
+void split_tuple_data(FuncCallContext *fctx, HeapTupleHeader tuphdr, Datum *values, bool *nulls, bool do_detoast);
/*
* heap_page_items
@@ -66,38 +74,80 @@ typedef struct heap_page_items_state
TupleDesc tupd;
Page page;
uint16 offset;
+ TupleDesc page_tuple_desc;
+ int raw_page_size;
+ int error_level;
} heap_page_items_state;
Datum
heap_page_items(PG_FUNCTION_ARGS)
{
bytea *raw_page = PG_GETARG_BYTEA_P(0);
+ return heap_page_items_internal(fcinfo, raw_page, InvalidOid, ERROR, false);
+}
+
+PG_FUNCTION_INFO_V1(heap_page_item_attrs);
+Datum
+heap_page_item_attrs(PG_FUNCTION_ARGS)
+{
+ bytea *raw_page = PG_GETARG_BYTEA_P(0);
+ Oid rel_oid = PG_GETARG_OID(1);
+ int error_mode = PG_NARGS()==3 ? PG_GETARG_BOOL(2) :NULL;
+ error_mode = error_mode?WARNING:ERROR;
+ if (SRF_IS_FIRSTCALL() && (error_mode == WARNING))
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ (errmsg("Runnung heap_page_item_attrs in WARNING mode."))));
+ }
+
+ return heap_page_items_internal(fcinfo, raw_page, rel_oid, error_mode, false);
+}
+
+PG_FUNCTION_INFO_V1(heap_page_item_detoast_attrs);
+Datum
+heap_page_item_detoast_attrs(PG_FUNCTION_ARGS)
+{
+ bytea *raw_page = PG_GETARG_BYTEA_P(0);
+ Oid rel_oid = PG_GETARG_OID(1);
+ int error_mode = PG_NARGS()==3 ? PG_GETARG_BOOL(2) :NULL;
+ error_mode = error_mode?WARNING:ERROR;
+ if (SRF_IS_FIRSTCALL() && (error_mode == WARNING))
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ (errmsg("Runnung heap_page_item_detoast_attrs in WARNING mode."))));
+ }
+ return heap_page_items_internal(fcinfo, raw_page, rel_oid, error_mode, true);
+}
+
+Datum
+heap_page_items_internal(FunctionCallInfo fcinfo, bytea *raw_page, Oid rel_oid, int error_level, bool do_detoast)
+{
heap_page_items_state *inter_call_data = NULL;
FuncCallContext *fctx;
- int raw_page_size;
if (!superuser())
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
- (errmsg("must be superuser to use raw page functions"))));
-
- raw_page_size = VARSIZE(raw_page) - VARHDRSZ;
+ (errmsg("must be superuser to use raw page functions"))));
if (SRF_IS_FIRSTCALL())
{
TupleDesc tupdesc;
- MemoryContext mctx;
-
- if (raw_page_size < SizeOfPageHeaderData)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("input page too small (%d bytes)", raw_page_size)));
+ MemoryContext mctx;
fctx = SRF_FIRSTCALL_INIT();
mctx = MemoryContextSwitchTo(fctx->multi_call_memory_ctx);
inter_call_data = palloc(sizeof(heap_page_items_state));
+ inter_call_data->raw_page_size = VARSIZE(raw_page) - VARHDRSZ;
+ if (inter_call_data->raw_page_size < SizeOfPageHeaderData)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("input page too small (%d bytes)", inter_call_data->raw_page_size)));
+
/* Build a tuple descriptor for our result type */
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
@@ -105,7 +155,19 @@ heap_page_items(PG_FUNCTION_ARGS)
inter_call_data->tupd = tupdesc;
inter_call_data->offset = FirstOffsetNumber;
- inter_call_data->page = VARDATA(raw_page);
+
+ inter_call_data->page = palloc(BLCKSZ);
+ memcpy(inter_call_data->page, VARDATA(raw_page), BLCKSZ);
+
+ inter_call_data->page_tuple_desc = NULL;
+ inter_call_data->error_level = error_level;
+
+ if (rel_oid != InvalidOid)
+ {
+ RelationData * rel = relation_open(rel_oid, NoLock);
+ inter_call_data->page_tuple_desc = CreateTupleDescCopyConstr(rel->rd_att);
+ relation_close(rel, NoLock);
+ }
fctx->max_calls = PageGetMaxOffsetNumber(inter_call_data->page);
fctx->user_fctx = inter_call_data;
@@ -114,112 +176,255 @@ heap_page_items(PG_FUNCTION_ARGS)
}
fctx = SRF_PERCALL_SETUP();
+
inter_call_data = fctx->user_fctx;
if (fctx->call_cntr < fctx->max_calls)
{
- Page page = inter_call_data->page;
+ Datum values[14];
+ bool nulls[14];
HeapTuple resultTuple;
Datum result;
- ItemId id;
- Datum values[13];
- bool nulls[13];
- uint16 lp_offset;
- uint16 lp_flags;
- uint16 lp_len;
memset(nulls, 0, sizeof(nulls));
- /* Extract information from the line pointer */
+ fill_header_data(fctx, values, nulls, do_detoast);
+
+ /* Build and return the result tuple. */
+ resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
+ result = HeapTupleGetDatum(resultTuple);
+
+ inter_call_data->offset++;
+
+ SRF_RETURN_NEXT(fctx, result);
+ }
+ else
+ SRF_RETURN_DONE(fctx);
+}
- id = PageGetItemId(page, inter_call_data->offset);
- lp_offset = ItemIdGetOffset(id);
- lp_flags = ItemIdGetFlags(id);
- lp_len = ItemIdGetLength(id);
- values[0] = UInt16GetDatum(inter_call_data->offset);
- values[1] = UInt16GetDatum(lp_offset);
- values[2] = UInt16GetDatum(lp_flags);
- values[3] = UInt16GetDatum(lp_len);
+void
+fill_header_data(FuncCallContext *fctx, Datum *values, bool *nulls, bool do_detoast)
+{
+ Page page;
+ ItemId id;
+ uint16 lp_offset;
+ uint16 lp_flags;
+ uint16 lp_len;
+ heap_page_items_state *inter_call_data = NULL;
+ inter_call_data = fctx->user_fctx;
+ page = inter_call_data->page;
+
+ /* Extract information from the line pointer */
+
+ id = PageGetItemId(page, inter_call_data->offset);
+
+ lp_offset = ItemIdGetOffset(id);
+ lp_flags = ItemIdGetFlags(id);
+ lp_len = ItemIdGetLength(id);
+
+ values[0] = UInt16GetDatum(inter_call_data->offset);
+ values[1] = UInt16GetDatum(lp_offset);
+ values[2] = UInt16GetDatum(lp_flags);
+ values[3] = UInt16GetDatum(lp_len);
+
+ /*
+ * We do just enough validity checking to make sure we don't reference
+ * data outside the page passed to us. The page could be corrupt in
+ * many other ways, but at least we won't crash.
+ */
+ if (ItemIdHasStorage(id) &&
+ lp_len >= MinHeapTupleSize &&
+ lp_offset == MAXALIGN(lp_offset) &&
+ lp_offset + lp_len <= inter_call_data->raw_page_size)
+ {
+ HeapTupleHeader tuphdr;
+ int bits_len;
+ bytea *tuple_data_bytea;
+ int tuple_data_len;
+
+ /* Extract information from the tuple header */
+
+ tuphdr = (HeapTupleHeader) PageGetItem(page, id);
+
+ values[4] = UInt32GetDatum(HeapTupleHeaderGetRawXmin(tuphdr));
+ values[5] = UInt32GetDatum(HeapTupleHeaderGetRawXmax(tuphdr));
+ values[6] = UInt32GetDatum(HeapTupleHeaderGetRawCommandId(tuphdr)); /* shared with xvac */
+ values[7] = PointerGetDatum(&tuphdr->t_ctid);
+ values[8] = UInt32GetDatum(tuphdr->t_infomask2);
+ values[9] = UInt32GetDatum(tuphdr->t_infomask);
+ values[10] = UInt8GetDatum(tuphdr->t_hoff);
+
+ /* Copy raw tuple data into bytea attribute */
+ tuple_data_len = lp_len - tuphdr->t_hoff;
+ tuple_data_bytea = (bytea *) palloc(tuple_data_len + VARHDRSZ);
+ SET_VARSIZE(tuple_data_bytea, tuple_data_len + VARHDRSZ);
+ memcpy(VARDATA(tuple_data_bytea), (char *) tuphdr + tuphdr->t_hoff, tuple_data_len);
+ values[13] = PointerGetDatum(tuple_data_bytea);
+
+ if (inter_call_data->page_tuple_desc)
+ {
+ split_tuple_data(fctx, tuphdr, values, nulls, do_detoast);
+ }
/*
- * We do just enough validity checking to make sure we don't reference
- * data outside the page passed to us. The page could be corrupt in
- * many other ways, but at least we won't crash.
+ * We already checked that the item is completely within the raw
+ * page passed to us, with the length given in the line pointer.
+ * Let's check that t_hoff doesn't point over lp_len, before using
+ * it to access t_bits and oid.
*/
- if (ItemIdHasStorage(id) &&
- lp_len >= MinHeapTupleSize &&
- lp_offset == MAXALIGN(lp_offset) &&
- lp_offset + lp_len <= raw_page_size)
+ if (tuphdr->t_hoff >= SizeofHeapTupleHeader &&
+ tuphdr->t_hoff <= lp_len &&
+ tuphdr->t_hoff == MAXALIGN(tuphdr->t_hoff))
{
- HeapTupleHeader tuphdr;
- int bits_len;
-
- /* Extract information from the tuple header */
-
- tuphdr = (HeapTupleHeader) PageGetItem(page, id);
-
- values[4] = UInt32GetDatum(HeapTupleHeaderGetRawXmin(tuphdr));
- values[5] = UInt32GetDatum(HeapTupleHeaderGetRawXmax(tuphdr));
- values[6] = UInt32GetDatum(HeapTupleHeaderGetRawCommandId(tuphdr)); /* shared with xvac */
- values[7] = PointerGetDatum(&tuphdr->t_ctid);
- values[8] = UInt32GetDatum(tuphdr->t_infomask2);
- values[9] = UInt32GetDatum(tuphdr->t_infomask);
- values[10] = UInt8GetDatum(tuphdr->t_hoff);
-
- /*
- * We already checked that the item is completely within the raw
- * page passed to us, with the length given in the line pointer.
- * Let's check that t_hoff doesn't point over lp_len, before using
- * it to access t_bits and oid.
- */
- if (tuphdr->t_hoff >= SizeofHeapTupleHeader &&
- tuphdr->t_hoff <= lp_len &&
- tuphdr->t_hoff == MAXALIGN(tuphdr->t_hoff))
+ if (tuphdr->t_infomask & HEAP_HASNULL)
{
- if (tuphdr->t_infomask & HEAP_HASNULL)
- {
- bits_len = tuphdr->t_hoff -
- offsetof(HeapTupleHeaderData, t_bits);
-
- values[11] = CStringGetTextDatum(
- bits_to_text(tuphdr->t_bits, bits_len * 8));
- }
- else
- nulls[11] = true;
+ bits_len = tuphdr->t_hoff -
+ offsetof(HeapTupleHeaderData, t_bits);
- if (tuphdr->t_infomask & HEAP_HASOID)
- values[12] = HeapTupleHeaderGetOid(tuphdr);
- else
- nulls[12] = true;
+ values[11] = CStringGetTextDatum(
+ bits_to_text(tuphdr->t_bits, bits_len * 8));
}
else
- {
nulls[11] = true;
+
+ if (tuphdr->t_infomask & HEAP_HASOID)
+ values[12] = HeapTupleHeaderGetOid(tuphdr);
+ else
nulls[12] = true;
- }
}
else
{
- /*
- * The line pointer is not used, or it's invalid. Set the rest of
- * the fields to NULL
- */
- int i;
-
- for (i = 4; i <= 12; i++)
- nulls[i] = true;
+ nulls[11] = true;
+ nulls[12] = true;
}
+ }
+ else
+ {
+ /*
+ * The line pointer is not used, or it's invalid. Set the rest of
+ * the fields to NULL
+ */
+ int i;
- /* Build and return the result tuple. */
- resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ for (i = 4; i <= 13; i++)
+ nulls[i] = true;
+ }
+}
- inter_call_data->offset++;
- SRF_RETURN_NEXT(fctx, result);
+void
+split_tuple_data(FuncCallContext *fctx, HeapTupleHeader tuphdr, Datum *values, bool *nulls, bool do_detoast)
+{
+ TupleDesc tuple_desc;
+ ArrayBuildState *raw_attrs;
+ uint16 lp_len;
+ char *tuple_data_p;
+ int nattrs;
+ int i;
+ int off;
+ heap_page_items_state *inter_call_data;
+
+ inter_call_data = fctx->user_fctx;
+ lp_len = DatumGetUInt16(values[3]);
+
+ /*
+ * Here we reimplement the basic functionality of nocachegetattr from
+ * backend/access/common/heaptuple.c whitch is basically used for fetching
+ * attributes from tuple when it is not cached. We can not use nocachegetattr here
+ * directly, because we should ignore all cache optimisations and other stuff
+ * just get binary data as it is.
+ */
+
+ tuple_desc = inter_call_data->page_tuple_desc;
+ raw_attrs = initArrayResult(BYTEAOID,fctx->multi_call_memory_ctx,0);
+ tuple_data_p = (char *) tuphdr + tuphdr->t_hoff;
+
+ off = 0;
+ nattrs = tuple_desc->natts;
+ if (inter_call_data->error_level &&
+ nattrs < (tuphdr->t_infomask2 & HEAP_NATTS_MASK))
+ {
+ ereport(inter_call_data->error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("Data corruption: number of attributes in tuple header is greater than number of attributes in tuple descripor")));
}
- else
- SRF_RETURN_DONE(fctx);
+
+ for(i=0; i < nattrs; i++)
+ {
+ Form_pg_attribute attr;
+ Datum raw_attr;
+ bool is_null;
+
+ attr = tuple_desc->attrs[i];
+ is_null = (tuphdr->t_infomask & HEAP_HASNULL) && att_isnull(i, tuphdr->t_bits);
+ /*
+ * Tuple header can specify less attributes then tuple descriptor
+ * as ALTER TABLE ADD COLUMN without DEFAULT keyword does not
+ * actualy change tuples in pages, so attributes with numbers greater
+ * than tuphdr->t_infomask2 & HEAP_NATTS_MASK should be treated as NULL
+ */
+ if (i >= (tuphdr->t_infomask2 & HEAP_NATTS_MASK) )
+ is_null = 1;
+ if (!is_null)
+ {
+ int len;
+ bytea * attr_data;
+ if (attr->attlen == -1)
+ {
+ off = att_align_pointer(off, tuple_desc->attrs[i]->attalign, -1, tuple_data_p + off);
+ /*
+ * As VARSIZE_ANY throws an exeption if it can't properly detect
+ * type of external storage in macros VARTAG_SIZE, so we repeat
+ * this check here to preform nicer error handling
+ */
+ if ( inter_call_data->error_level &&
+ VARATT_IS_1B_E(tuple_data_p + off) &&
+ VARTAG_EXTERNAL(tuple_data_p + off) != VARTAG_INDIRECT &&
+ VARTAG_EXTERNAL(tuple_data_p + off) != VARTAG_ONDISK)
+ {
+ ereport(inter_call_data->error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("Data corruption: First byte of varlen attr seems to be corrupted")));
+ }
+ len = VARSIZE_ANY(tuple_data_p + off);
+ }
+ else
+ {
+ off = att_align_nominal(off, tuple_desc->attrs[i]->attalign);
+ len = attr->attlen;
+ }
+ if ( inter_call_data->error_level &&
+ lp_len < tuphdr->t_hoff + off + len)
+ {
+ ereport(inter_call_data->error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("Data corruption: Iterating over tuple data reached out of actual tuple size")));
+ }
+ attr_data = (bytea *) palloc(len + VARHDRSZ);
+ SET_VARSIZE(attr_data, len + VARHDRSZ);
+ memcpy(VARDATA(attr_data), tuple_data_p + off, len);
+ raw_attr = PointerGetDatum(attr_data);
+
+ if ( attr->attlen == -1 && do_detoast)
+ {
+ Datum raw_attr_copy;
+ raw_attr_copy = PointerGetDatum(PG_DETOAST_DATUM_COPY(tuple_data_p + off));
+ pfree(attr_data);
+ raw_attr = raw_attr_copy;
+ }
+ off = att_addlength_pointer(off, tuple_desc->attrs[i]->attlen, tuple_data_p + off);
+ }
+ raw_attrs = accumArrayResult(raw_attrs, raw_attr, is_null, BYTEAOID, fctx->multi_call_memory_ctx);
+ }
+ if (inter_call_data->error_level &&
+ lp_len != tuphdr->t_hoff + off)
+ {
+ ereport(inter_call_data->error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("Data corruption: Iterating over tuple data did not actualy reach tuple end")));
+ }
+ pfree(DatumGetPointer(values[13]));
+ values[13] = makeArrayResult(raw_attrs, fctx->multi_call_memory_ctx);
}
diff --git a/contrib/pageinspect/pageinspect--1.3--1.4.sql b/contrib/pageinspect/pageinspect--1.3--1.4.sql
new file mode 100644
index 0000000..3966045
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.3--1.4.sql
@@ -0,0 +1,110 @@
+/* contrib/pageinspect/pageinspect--1.3--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via ALTER EXTENSION
+\echo Use "ALTER EXTENSION pageinspect UPDATE TO '1.4'" to load this file. \quit
+
+--
+-- heap_page_items()
+--
+DROP FUNCTION heap_page_items(bytea);
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_item_attrs'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass, IN warning_mode bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_item_attrs'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_item_detoast_attrs()
+--
+
+CREATE FUNCTION heap_page_item_detoast_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_item_detoast_attrs'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION heap_page_item_detoast_attrs(IN page bytea, IN heap_oid regclass, IN warning_mode bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_item_detoast_attrs'
+LANGUAGE C STRICT;
\ No newline at end of file
diff --git a/contrib/pageinspect/pageinspect--1.3.sql b/contrib/pageinspect/pageinspect--1.3.sql
deleted file mode 100644
index a99e058..0000000
--- a/contrib/pageinspect/pageinspect--1.3.sql
+++ /dev/null
@@ -1,189 +0,0 @@
-/* contrib/pageinspect/pageinspect--1.3.sql */
-
--- complain if script is sourced in psql, rather than via CREATE EXTENSION
-\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
-
---
--- get_raw_page()
---
-CREATE FUNCTION get_raw_page(text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page'
-LANGUAGE C STRICT;
-
-CREATE FUNCTION get_raw_page(text, text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page_fork'
-LANGUAGE C STRICT;
-
---
--- page_header()
---
-CREATE FUNCTION page_header(IN page bytea,
- OUT lsn pg_lsn,
- OUT checksum smallint,
- OUT flags smallint,
- OUT lower smallint,
- OUT upper smallint,
- OUT special smallint,
- OUT pagesize smallint,
- OUT version smallint,
- OUT prune_xid xid)
-AS 'MODULE_PATHNAME', 'page_header'
-LANGUAGE C STRICT;
-
---
--- heap_page_items()
---
-CREATE FUNCTION heap_page_items(IN page bytea,
- OUT lp smallint,
- OUT lp_off smallint,
- OUT lp_flags smallint,
- OUT lp_len smallint,
- OUT t_xmin xid,
- OUT t_xmax xid,
- OUT t_field3 int4,
- OUT t_ctid tid,
- OUT t_infomask2 integer,
- OUT t_infomask integer,
- OUT t_hoff smallint,
- OUT t_bits text,
- OUT t_oid oid)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'heap_page_items'
-LANGUAGE C STRICT;
-
---
--- bt_metap()
---
-CREATE FUNCTION bt_metap(IN relname text,
- OUT magic int4,
- OUT version int4,
- OUT root int4,
- OUT level int4,
- OUT fastroot int4,
- OUT fastlevel int4)
-AS 'MODULE_PATHNAME', 'bt_metap'
-LANGUAGE C STRICT;
-
---
--- bt_page_stats()
---
-CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
- OUT blkno int4,
- OUT type "char",
- OUT live_items int4,
- OUT dead_items int4,
- OUT avg_item_size int4,
- OUT page_size int4,
- OUT free_size int4,
- OUT btpo_prev int4,
- OUT btpo_next int4,
- OUT btpo int4,
- OUT btpo_flags int4)
-AS 'MODULE_PATHNAME', 'bt_page_stats'
-LANGUAGE C STRICT;
-
---
--- bt_page_items()
---
-CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
- OUT itemoffset smallint,
- OUT ctid tid,
- OUT itemlen smallint,
- OUT nulls bool,
- OUT vars bool,
- OUT data text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'bt_page_items'
-LANGUAGE C STRICT;
-
---
--- brin_page_type()
---
-CREATE FUNCTION brin_page_type(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'brin_page_type'
-LANGUAGE C STRICT;
-
---
--- brin_metapage_info()
---
-CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
- OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
-AS 'MODULE_PATHNAME', 'brin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- brin_revmap_data()
---
-CREATE FUNCTION brin_revmap_data(IN page bytea,
- OUT pages tid)
-RETURNS SETOF tid
-AS 'MODULE_PATHNAME', 'brin_revmap_data'
-LANGUAGE C STRICT;
-
---
--- brin_page_items()
---
-CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
- OUT itemoffset int,
- OUT blknum int,
- OUT attnum int,
- OUT allnulls bool,
- OUT hasnulls bool,
- OUT placeholder bool,
- OUT value text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'brin_page_items'
-LANGUAGE C STRICT;
-
---
--- fsm_page_contents()
---
-CREATE FUNCTION fsm_page_contents(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'fsm_page_contents'
-LANGUAGE C STRICT;
-
---
--- GIN functions
---
-
---
--- gin_metapage_info()
---
-CREATE FUNCTION gin_metapage_info(IN page bytea,
- OUT pending_head bigint,
- OUT pending_tail bigint,
- OUT tail_free_size int4,
- OUT n_pending_pages bigint,
- OUT n_pending_tuples bigint,
- OUT n_total_pages bigint,
- OUT n_entry_pages bigint,
- OUT n_data_pages bigint,
- OUT n_entries bigint,
- OUT version int4)
-AS 'MODULE_PATHNAME', 'gin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- gin_page_opaque_info()
---
-CREATE FUNCTION gin_page_opaque_info(IN page bytea,
- OUT rightlink bigint,
- OUT maxoff int4,
- OUT flags text[])
-AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
-LANGUAGE C STRICT;
-
---
--- gin_leafpage_items()
---
-CREATE FUNCTION gin_leafpage_items(IN page bytea,
- OUT first_tid tid,
- OUT nbytes int2,
- OUT tids tid[])
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'gin_leafpage_items'
-LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect--1.4.sql b/contrib/pageinspect/pageinspect--1.4.sql
new file mode 100644
index 0000000..67bf545
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.4.sql
@@ -0,0 +1,296 @@
+/* contrib/pageinspect/pageinspect--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
+
+--
+-- get_raw_page()
+--
+CREATE FUNCTION get_raw_page(text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION get_raw_page(text, text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page_fork'
+LANGUAGE C STRICT;
+
+--
+-- page_header()
+--
+CREATE FUNCTION page_header(IN page bytea,
+ OUT lsn pg_lsn,
+ OUT checksum smallint,
+ OUT flags smallint,
+ OUT lower smallint,
+ OUT upper smallint,
+ OUT special smallint,
+ OUT pagesize smallint,
+ OUT version smallint,
+ OUT prune_xid xid)
+AS 'MODULE_PATHNAME', 'page_header'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_items()
+--
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_item_attrs'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass, IN warning_mode bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_item_attrs'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_item_detoast_attrs()
+--
+
+CREATE FUNCTION heap_page_item_detoast_attrs(IN page bytea, IN heap_oid regclass, IN warning_mode bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_item_detoast_attrs'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION heap_page_item_detoast_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_item_detoast_attrs'
+LANGUAGE C STRICT;
+
+--
+-- _heap_page_items()
+--
+CREATE FUNCTION _heap_page_items(IN page bytea, IN relname text,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- bt_metap()
+--
+CREATE FUNCTION bt_metap(IN relname text,
+ OUT magic int4,
+ OUT version int4,
+ OUT root int4,
+ OUT level int4,
+ OUT fastroot int4,
+ OUT fastlevel int4)
+AS 'MODULE_PATHNAME', 'bt_metap'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_stats()
+--
+CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
+ OUT blkno int4,
+ OUT type "char",
+ OUT live_items int4,
+ OUT dead_items int4,
+ OUT avg_item_size int4,
+ OUT page_size int4,
+ OUT free_size int4,
+ OUT btpo_prev int4,
+ OUT btpo_next int4,
+ OUT btpo int4,
+ OUT btpo_flags int4)
+AS 'MODULE_PATHNAME', 'bt_page_stats'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_items()
+--
+CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
+ OUT itemoffset smallint,
+ OUT ctid tid,
+ OUT itemlen smallint,
+ OUT nulls bool,
+ OUT vars bool,
+ OUT data text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'bt_page_items'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_type()
+--
+CREATE FUNCTION brin_page_type(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'brin_page_type'
+LANGUAGE C STRICT;
+
+--
+-- brin_metapage_info()
+--
+CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
+ OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
+AS 'MODULE_PATHNAME', 'brin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- brin_revmap_data()
+--
+CREATE FUNCTION brin_revmap_data(IN page bytea,
+ OUT pages tid)
+RETURNS SETOF tid
+AS 'MODULE_PATHNAME', 'brin_revmap_data'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_items()
+--
+CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
+ OUT itemoffset int,
+ OUT blknum int,
+ OUT attnum int,
+ OUT allnulls bool,
+ OUT hasnulls bool,
+ OUT placeholder bool,
+ OUT value text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'brin_page_items'
+LANGUAGE C STRICT;
+
+--
+-- fsm_page_contents()
+--
+CREATE FUNCTION fsm_page_contents(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'fsm_page_contents'
+LANGUAGE C STRICT;
+
+--
+-- GIN functions
+--
+
+--
+-- gin_metapage_info()
+--
+CREATE FUNCTION gin_metapage_info(IN page bytea,
+ OUT pending_head bigint,
+ OUT pending_tail bigint,
+ OUT tail_free_size int4,
+ OUT n_pending_pages bigint,
+ OUT n_pending_tuples bigint,
+ OUT n_total_pages bigint,
+ OUT n_entry_pages bigint,
+ OUT n_data_pages bigint,
+ OUT n_entries bigint,
+ OUT version int4)
+AS 'MODULE_PATHNAME', 'gin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_page_opaque_info()
+--
+CREATE FUNCTION gin_page_opaque_info(IN page bytea,
+ OUT rightlink bigint,
+ OUT maxoff int4,
+ OUT flags text[])
+AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_leafpage_items()
+--
+CREATE FUNCTION gin_leafpage_items(IN page bytea,
+ OUT first_tid tid,
+ OUT nbytes int2,
+ OUT tids tid[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'gin_leafpage_items'
+LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect.control b/contrib/pageinspect/pageinspect.control
index a9dab33..68c7d61 100644
--- a/contrib/pageinspect/pageinspect.control
+++ b/contrib/pageinspect/pageinspect.control
@@ -1,5 +1,5 @@
# pageinspect extension
comment = 'inspect the contents of database pages at a low level'
-default_version = '1.3'
+default_version = '1.4'
module_pathname = '$libdir/pageinspect'
relocatable = true
diff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.c
index 38c136f..6659ee5 100644
--- a/contrib/pageinspect/rawpage.c
+++ b/contrib/pageinspect/rawpage.c
@@ -26,11 +26,9 @@
#include "utils/pg_lsn.h"
#include "utils/rel.h"
-PG_MODULE_MAGIC;
-
-static bytea *get_raw_page_internal(text *relname, ForkNumber forknum,
- BlockNumber blkno);
+#include "rawpage.h"
+PG_MODULE_MAGIC;
/*
* get_raw_page
@@ -87,7 +85,7 @@ get_raw_page_fork(PG_FUNCTION_ARGS)
/*
* workhorse
*/
-static bytea *
+bytea *
get_raw_page_internal(text *relname, ForkNumber forknum, BlockNumber blkno)
{
bytea *raw_page;
diff --git a/contrib/pageinspect/rawpage.h b/contrib/pageinspect/rawpage.h
new file mode 100644
index 0000000..d9f2678
--- /dev/null
+++ b/contrib/pageinspect/rawpage.h
@@ -0,0 +1,20 @@
+/*-------------------------------------------------------------------------
+ *
+ * rawpage.h
+ *
+ * Copyright (c) 2007-2015, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * contrib/pageinspect/rawpage.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef __RAWPAGE_H__
+#define __RAWPAGE_H__
+
+
+bytea *get_raw_page_internal(text *relname, ForkNumber forknum,
+ BlockNumber blkno);
+
+#endif /*__RAWPAGE_H__*/
\ No newline at end of file
Re: pageinspect patch, for showing tuple data
On Wed, Sep 2, 2015 at 6:58 PM, Nikolay Shaplov <n.shaplov@postgrespro.ru>
wrote:
В письме от 3 августа 2015 15:35:23 пользователь Alvaro Herrera написал:
Nikolay Shaplov wrote:
This patch adds several new functions, available from SQL queries. All
these functions are based on heap_page_items, but accept slightly
different arguments and has one additional column at the result set:heap_page_tuples - accepts relation name, and bulkno, and returns usual
heap_page_items set with additional column that contain snapshot oftuple
data area stored in bytea.
I think the API around get_raw_page is more useful generally. You might
have table blocks stored in a bytea column for instance, because you
extracted from some remote server and inserted into a local table for
examination. If you only accept relname/blkno, there's no way to
examine data other than blocks directly in the database dir, which is
limiting.There is also one strange function: _heap_page_items it is useless for
practical purposes. As heap_page_items it accepts page data as bytea,but
also get a relation name. It tries to apply tuple descriptor of that
relation to that page data.For BRIN, I added something similar (brin_page_items), but it receives
the index OID rather than name, and constructs a tuple descriptor to
read the data. I think OID is better than name generally. (You can
cast the relation name to regclass).It's easy to misuse, but these functions are superuser-only, so there
should be no security issue at least. The possibility of a crash
remains real, though, so maybe we should try to come up with a way to
harden that.I've done as you've said: Now patch adds two functions heap_page_item_attrs
and heap_page_item_detoast_attrs and these function takes raw_page and
relation OID as arguments. They also have third optional parameter that
allows
to change error mode from ERROR to WARNING.Is it ok now?
Yeah, I think that's acceptable to have a switch, defaulting to ERROR if
caller specifies nothing.
Here are some observations after looking at the code, no functional testing.
+ int error_mode = PG_NARGS()==3 ? PG_GETARG_BOOL(2)
:NULL;
When handling additional arguments, it seems to me that it is more readable
to use something like that:
if (PG_NARGS >= 3)
{
arg = PG_GETARG_XXX(2);
//etc.
}
+ error_mode = error_mode?WARNING:ERROR;
This generates warnings at compilation.
+ if (SRF_IS_FIRSTCALL() && (error_mode == WARNING))
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ (errmsg("Runnung heap_page_item_attrs in
WARNING mode."))));
+ }I am not convinced that this is necessary.
+ inter_call_data->raw_page_size = VARSIZE(raw_page) -
VARHDRSZ;
+ if (inter_call_data->raw_page_size < SizeOfPageHeaderData)
Including raw_page_size in the status data does not seem necessary to me.
The page data is already available for each loop so you could just extract
it from it.+ ereport(inter_call_data->error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("Data corruption: number of
attributes in tuple header is greater than number of attributes in tuple
descripor")));
I'd rather switch the error reports related to data corruption from ereport
to elog, which is more suited for internal errors, and it seems to me that
it is the case here.heapfuncs.c:370:7: warning: variable 'raw_attr' is used uninitialized
whenever 'if' condition is false [-Wsometimes-uninitialized]
if (!is_null)
^~~~~~~~
heapfuncs.c:419:43: note: uninitialized use occurs here
raw_attrs = accumArrayResult(raw_attrs, raw_attr, is_null,
BYTEAOID, fctx->multi_call_memory_ctx);
^~~~~~~~
heapfuncs.c:370:3: note: remove the 'if' if its condition is always true
if (!is_null)
^~~~~~~~~~~~~
heapfuncs.c:357:17: note: initialize the variable 'raw_attr' to silence
this warning
Datum raw_attr;
My compiler complains about uninitialized variables.
+--
+-- _heap_page_items()
+--
+CREATE FUNCTION _heap_page_items(IN page bytea, IN relname text,
I am not sure why this is necessary and visibly it overlaps with the
existing declaration of heap_page_items. The last argument is different
though, and I recall that we decided not to use anymore the relation name
specified as text, but its OID instead in this module.Documentation is missing, that would be good to have to understand what
each function is intended to do. Code has some whitespaces.
--
Michael
Re: pageinspect patch, for showing tuple data
В письме от 4 сентября 2015 14:58:29 пользователь Michael Paquier написал:
Yeah, I think that's acceptable to have a switch, defaulting to ERROR if
caller specifies nothing.Here are some observations after looking at the code, no functional testing.
+ int error_mode = PG_NARGS()==3 ? PG_GETARG_BOOL(2)
:NULL;
When handling additional arguments, it seems to me that it is more readable
to use something like that:
if (PG_NARGS >= 3)
{
arg = PG_GETARG_XXX(2);
//etc.
}+ error_mode = error_mode?WARNING:ERROR;
This generates warnings at compilation.
yeah, what I have done here is too complex with no reason. I've simplified this
code now.
+ if (SRF_IS_FIRSTCALL() && (error_mode == WARNING)) + { + ereport(WARNING, + (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE), + (errmsg("Runnung heap_page_item_attrs in WARNING mode.")))); + }I am not convinced that this is necessary.
I've removed it.
+ inter_call_data->raw_page_size = VARSIZE(raw_page) - VARHDRSZ; + if (inter_call_data->raw_page_size < SizeOfPageHeaderData) Including raw_page_size in the status data does not seem necessary to me. The page data is already available for each loop so you could just extract it from it.+ ereport(inter_call_data->error_level, + (errcode(ERRCODE_DATA_CORRUPTED), + errmsg("Data corruption: number of attributes in tuple header is greater than number of attributes in tuple descripor"))); I'd rather switch the error reports related to data corruption from ereport to elog, which is more suited for internal errors, and it seems to me that it is the case here.
Hm... First, since we have proper error code, do not see why not give it.
Second, this is not exactly internal error, in some cases user tries to parse
corrupted data on purpose. So for user it is not an internal error, error from
deep below, but error on the level he is currently working at. So I would not
call it internal error.
heapfuncs.c:370:7: warning: variable 'raw_attr' is used uninitialized
whenever 'if' condition is false [-Wsometimes-uninitialized]
if (!is_null)
^~~~~~~~
heapfuncs.c:419:43: note: uninitialized use occurs here
raw_attrs = accumArrayResult(raw_attrs, raw_attr, is_null,
BYTEAOID, fctx->multi_call_memory_ctx);
^~~~~~~~
heapfuncs.c:370:3: note: remove the 'if' if its condition is always true
if (!is_null)
^~~~~~~~~~~~~
heapfuncs.c:357:17: note: initialize the variable 'raw_attr' to silence
this warning
Datum raw_attr;
My compiler complains about uninitialized variables.
Fixed it
+-- +-- _heap_page_items() +-- +CREATE FUNCTION _heap_page_items(IN page bytea, IN relname text, I am not sure why this is necessary and visibly it overlaps with the existing declaration of heap_page_items. The last argument is different though, and I recall that we decided not to use anymore the relation name specified as text, but its OID instead in this module.
Oh! This should not go to the final patch :-( Sorry. Removed it.
Documentation is missing, that would be good to have to understand what
each function is intended to do.
I were going to add documentation when this patch is commited, or at least
approved for commit. So I would know for sure that function definition would
not change, so had not to rewrite it again and again.
So if it is possible I would write documentation and test when this patch is
already approved.
Code has some whitespaces.
I've found and removed some. Hope that was all of them...
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
pageinspect_show_tuple_data_v3.difftext/x-patch; charset=utf-8; name=pageinspect_show_tuple_data_v3.diffDownload
diff --git a/contrib/pageinspect/Makefile b/contrib/pageinspect/Makefile
index aec5258..e4bc1af 100644
--- a/contrib/pageinspect/Makefile
+++ b/contrib/pageinspect/Makefile
@@ -5,7 +5,7 @@ OBJS = rawpage.o heapfuncs.o btreefuncs.o fsmfuncs.o \
brinfuncs.o ginfuncs.o $(WIN32RES)
EXTENSION = pageinspect
-DATA = pageinspect--1.3.sql pageinspect--1.2--1.3.sql \
+DATA = pageinspect--1.4.sql pageinspect--1.3--1.4.sql pageinspect--1.2--1.3.sql \
pageinspect--1.1--1.2.sql pageinspect--1.0--1.1.sql \
pageinspect--unpackaged--1.0.sql
PGFILEDESC = "pageinspect - functions to inspect contents of database pages"
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 8d1666c..a1b07cb 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -29,7 +29,12 @@
#include "funcapi.h"
#include "utils/builtins.h"
#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/rel.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_type.h"
+#include "rawpage.h"
/*
* bits_to_text
@@ -53,6 +58,9 @@ bits_to_text(bits8 *bits, int len)
return str;
}
+Datum heap_page_items_internal(FunctionCallInfo fcinfo, bytea *raw_page, Oid rel_oid, int error_level, bool do_detoast);
+void fill_header_data(FuncCallContext *fctx, Datum *values, bool *nulls, bool do_detoast);
+void split_tuple_data(FuncCallContext *fctx, HeapTupleHeader tuphdr, Datum *values, bool *nulls, bool do_detoast);
/*
* heap_page_items
@@ -66,38 +74,73 @@ typedef struct heap_page_items_state
TupleDesc tupd;
Page page;
uint16 offset;
+ TupleDesc page_tuple_desc;
+ int raw_page_size;
+ int error_level;
} heap_page_items_state;
Datum
heap_page_items(PG_FUNCTION_ARGS)
{
bytea *raw_page = PG_GETARG_BYTEA_P(0);
+ return heap_page_items_internal(fcinfo, raw_page, InvalidOid, ERROR, false);
+}
+
+PG_FUNCTION_INFO_V1(heap_page_item_attrs);
+Datum
+heap_page_item_attrs(PG_FUNCTION_ARGS)
+{
+ bytea *raw_page = PG_GETARG_BYTEA_P(0);
+ Oid rel_oid = PG_GETARG_OID(1);
+ int error_mode = ERROR;
+ if (( PG_NARGS()>=3) && PG_GETARG_BOOL(2))
+ {
+ error_mode = WARNING;
+ }
+ return heap_page_items_internal(fcinfo, raw_page, rel_oid, error_mode, false);
+}
+
+PG_FUNCTION_INFO_V1(heap_page_item_detoast_attrs);
+Datum
+heap_page_item_detoast_attrs(PG_FUNCTION_ARGS)
+{
+ bytea *raw_page = PG_GETARG_BYTEA_P(0);
+ Oid rel_oid = PG_GETARG_OID(1);
+ int error_mode = ERROR;
+ if (( PG_NARGS()>=3) && PG_GETARG_BOOL(2))
+ {
+ error_mode = WARNING;
+ }
+ return heap_page_items_internal(fcinfo, raw_page, rel_oid, error_mode, true);
+}
+
+Datum
+heap_page_items_internal(FunctionCallInfo fcinfo, bytea *raw_page, Oid rel_oid, int error_level, bool do_detoast)
+{
heap_page_items_state *inter_call_data = NULL;
FuncCallContext *fctx;
- int raw_page_size;
if (!superuser())
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
- (errmsg("must be superuser to use raw page functions"))));
-
- raw_page_size = VARSIZE(raw_page) - VARHDRSZ;
+ (errmsg("must be superuser to use raw page functions"))));
if (SRF_IS_FIRSTCALL())
{
TupleDesc tupdesc;
- MemoryContext mctx;
-
- if (raw_page_size < SizeOfPageHeaderData)
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("input page too small (%d bytes)", raw_page_size)));
+ MemoryContext mctx;
fctx = SRF_FIRSTCALL_INIT();
mctx = MemoryContextSwitchTo(fctx->multi_call_memory_ctx);
inter_call_data = palloc(sizeof(heap_page_items_state));
+ inter_call_data->raw_page_size = VARSIZE(raw_page) - VARHDRSZ;
+ if (inter_call_data->raw_page_size < SizeOfPageHeaderData)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("input page too small (%d bytes)", inter_call_data->raw_page_size)));
+
/* Build a tuple descriptor for our result type */
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
@@ -105,7 +148,19 @@ heap_page_items(PG_FUNCTION_ARGS)
inter_call_data->tupd = tupdesc;
inter_call_data->offset = FirstOffsetNumber;
- inter_call_data->page = VARDATA(raw_page);
+
+ inter_call_data->page = palloc(BLCKSZ);
+ memcpy(inter_call_data->page, VARDATA(raw_page), BLCKSZ);
+
+ inter_call_data->page_tuple_desc = NULL;
+ inter_call_data->error_level = error_level;
+
+ if (rel_oid != InvalidOid)
+ {
+ RelationData * rel = relation_open(rel_oid, NoLock);
+ inter_call_data->page_tuple_desc = CreateTupleDescCopyConstr(rel->rd_att);
+ relation_close(rel, NoLock);
+ }
fctx->max_calls = PageGetMaxOffsetNumber(inter_call_data->page);
fctx->user_fctx = inter_call_data;
@@ -114,112 +169,252 @@ heap_page_items(PG_FUNCTION_ARGS)
}
fctx = SRF_PERCALL_SETUP();
+
inter_call_data = fctx->user_fctx;
if (fctx->call_cntr < fctx->max_calls)
{
- Page page = inter_call_data->page;
+ Datum values[14];
+ bool nulls[14];
HeapTuple resultTuple;
Datum result;
- ItemId id;
- Datum values[13];
- bool nulls[13];
- uint16 lp_offset;
- uint16 lp_flags;
- uint16 lp_len;
memset(nulls, 0, sizeof(nulls));
- /* Extract information from the line pointer */
+ fill_header_data(fctx, values, nulls, do_detoast);
- id = PageGetItemId(page, inter_call_data->offset);
+ /* Build and return the result tuple. */
+ resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
+ result = HeapTupleGetDatum(resultTuple);
- lp_offset = ItemIdGetOffset(id);
- lp_flags = ItemIdGetFlags(id);
- lp_len = ItemIdGetLength(id);
+ inter_call_data->offset++;
- values[0] = UInt16GetDatum(inter_call_data->offset);
- values[1] = UInt16GetDatum(lp_offset);
- values[2] = UInt16GetDatum(lp_flags);
- values[3] = UInt16GetDatum(lp_len);
+ SRF_RETURN_NEXT(fctx, result);
+ }
+ else
+ SRF_RETURN_DONE(fctx);
+}
+void
+fill_header_data(FuncCallContext *fctx, Datum *values, bool *nulls, bool do_detoast)
+{
+ Page page;
+ ItemId id;
+ uint16 lp_offset;
+ uint16 lp_flags;
+ uint16 lp_len;
+ heap_page_items_state *inter_call_data = NULL;
+
+ inter_call_data = fctx->user_fctx;
+ page = inter_call_data->page;
+
+ /* Extract information from the line pointer */
+
+ id = PageGetItemId(page, inter_call_data->offset);
+
+ lp_offset = ItemIdGetOffset(id);
+ lp_flags = ItemIdGetFlags(id);
+ lp_len = ItemIdGetLength(id);
+
+ values[0] = UInt16GetDatum(inter_call_data->offset);
+ values[1] = UInt16GetDatum(lp_offset);
+ values[2] = UInt16GetDatum(lp_flags);
+ values[3] = UInt16GetDatum(lp_len);
+
+ /*
+ * We do just enough validity checking to make sure we don't reference
+ * data outside the page passed to us. The page could be corrupt in
+ * many other ways, but at least we won't crash.
+ */
+ if (ItemIdHasStorage(id) &&
+ lp_len >= MinHeapTupleSize &&
+ lp_offset == MAXALIGN(lp_offset) &&
+ lp_offset + lp_len <= inter_call_data->raw_page_size)
+ {
+ HeapTupleHeader tuphdr;
+ int bits_len;
+ bytea *tuple_data_bytea;
+ int tuple_data_len;
+
+ /* Extract information from the tuple header */
+
+ tuphdr = (HeapTupleHeader) PageGetItem(page, id);
+
+ values[4] = UInt32GetDatum(HeapTupleHeaderGetRawXmin(tuphdr));
+ values[5] = UInt32GetDatum(HeapTupleHeaderGetRawXmax(tuphdr));
+ values[6] = UInt32GetDatum(HeapTupleHeaderGetRawCommandId(tuphdr)); /* shared with xvac */
+ values[7] = PointerGetDatum(&tuphdr->t_ctid);
+ values[8] = UInt32GetDatum(tuphdr->t_infomask2);
+ values[9] = UInt32GetDatum(tuphdr->t_infomask);
+ values[10] = UInt8GetDatum(tuphdr->t_hoff);
+
+ /* Copy raw tuple data into bytea attribute */
+ tuple_data_len = lp_len - tuphdr->t_hoff;
+ tuple_data_bytea = (bytea *) palloc(tuple_data_len + VARHDRSZ);
+ SET_VARSIZE(tuple_data_bytea, tuple_data_len + VARHDRSZ);
+ memcpy(VARDATA(tuple_data_bytea), (char *) tuphdr + tuphdr->t_hoff, tuple_data_len);
+ values[13] = PointerGetDatum(tuple_data_bytea);
+
+ if (inter_call_data->page_tuple_desc)
+ {
+ split_tuple_data(fctx, tuphdr, values, nulls, do_detoast);
+ }
/*
- * We do just enough validity checking to make sure we don't reference
- * data outside the page passed to us. The page could be corrupt in
- * many other ways, but at least we won't crash.
+ * We already checked that the item is completely within the raw
+ * page passed to us, with the length given in the line pointer.
+ * Let's check that t_hoff doesn't point over lp_len, before using
+ * it to access t_bits and oid.
*/
- if (ItemIdHasStorage(id) &&
- lp_len >= MinHeapTupleSize &&
- lp_offset == MAXALIGN(lp_offset) &&
- lp_offset + lp_len <= raw_page_size)
+ if (tuphdr->t_hoff >= SizeofHeapTupleHeader &&
+ tuphdr->t_hoff <= lp_len &&
+ tuphdr->t_hoff == MAXALIGN(tuphdr->t_hoff))
{
- HeapTupleHeader tuphdr;
- int bits_len;
-
- /* Extract information from the tuple header */
-
- tuphdr = (HeapTupleHeader) PageGetItem(page, id);
-
- values[4] = UInt32GetDatum(HeapTupleHeaderGetRawXmin(tuphdr));
- values[5] = UInt32GetDatum(HeapTupleHeaderGetRawXmax(tuphdr));
- values[6] = UInt32GetDatum(HeapTupleHeaderGetRawCommandId(tuphdr)); /* shared with xvac */
- values[7] = PointerGetDatum(&tuphdr->t_ctid);
- values[8] = UInt32GetDatum(tuphdr->t_infomask2);
- values[9] = UInt32GetDatum(tuphdr->t_infomask);
- values[10] = UInt8GetDatum(tuphdr->t_hoff);
-
- /*
- * We already checked that the item is completely within the raw
- * page passed to us, with the length given in the line pointer.
- * Let's check that t_hoff doesn't point over lp_len, before using
- * it to access t_bits and oid.
- */
- if (tuphdr->t_hoff >= SizeofHeapTupleHeader &&
- tuphdr->t_hoff <= lp_len &&
- tuphdr->t_hoff == MAXALIGN(tuphdr->t_hoff))
+ if (tuphdr->t_infomask & HEAP_HASNULL)
{
- if (tuphdr->t_infomask & HEAP_HASNULL)
- {
- bits_len = tuphdr->t_hoff -
- offsetof(HeapTupleHeaderData, t_bits);
-
- values[11] = CStringGetTextDatum(
- bits_to_text(tuphdr->t_bits, bits_len * 8));
- }
- else
- nulls[11] = true;
+ bits_len = tuphdr->t_hoff -
+ offsetof(HeapTupleHeaderData, t_bits);
- if (tuphdr->t_infomask & HEAP_HASOID)
- values[12] = HeapTupleHeaderGetOid(tuphdr);
- else
- nulls[12] = true;
+ values[11] = CStringGetTextDatum(
+ bits_to_text(tuphdr->t_bits, bits_len * 8));
}
else
- {
nulls[11] = true;
+
+ if (tuphdr->t_infomask & HEAP_HASOID)
+ values[12] = HeapTupleHeaderGetOid(tuphdr);
+ else
nulls[12] = true;
- }
}
else
{
- /*
- * The line pointer is not used, or it's invalid. Set the rest of
- * the fields to NULL
- */
- int i;
-
- for (i = 4; i <= 12; i++)
- nulls[i] = true;
+ nulls[11] = true;
+ nulls[12] = true;
}
+ }
+ else
+ {
+ /*
+ * The line pointer is not used, or it's invalid. Set the rest of
+ * the fields to NULL
+ */
+ int i;
- /* Build and return the result tuple. */
- resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ for (i = 4; i <= 13; i++)
+ nulls[i] = true;
+ }
+}
- inter_call_data->offset++;
+void
+split_tuple_data(FuncCallContext *fctx, HeapTupleHeader tuphdr, Datum *values, bool *nulls, bool do_detoast)
+{
+ TupleDesc tuple_desc;
+ ArrayBuildState *raw_attrs;
+ uint16 lp_len;
+ char *tuple_data_p;
+ int nattrs;
+ int i;
+ int off;
+ heap_page_items_state *inter_call_data;
- SRF_RETURN_NEXT(fctx, result);
+ inter_call_data = fctx->user_fctx;
+ lp_len = DatumGetUInt16(values[3]);
+
+ /*
+ * Here we reimplement the basic functionality of nocachegetattr from
+ * backend/access/common/heaptuple.c whitch is basically used for fetching
+ * attributes from tuple when it is not cached. We can not use nocachegetattr here
+ * directly, because we should ignore all cache optimisations and other stuff
+ * just get binary data as it is.
+ */
+
+ tuple_desc = inter_call_data->page_tuple_desc;
+ raw_attrs = initArrayResult(BYTEAOID,fctx->multi_call_memory_ctx,0);
+ tuple_data_p = (char *) tuphdr + tuphdr->t_hoff;
+
+ off = 0;
+ nattrs = tuple_desc->natts;
+ if (inter_call_data->error_level &&
+ nattrs < (tuphdr->t_infomask2 & HEAP_NATTS_MASK))
+ {
+ ereport(inter_call_data->error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: number of attributes in tuple header is greater than number of attributes in tuple descripor")));
}
- else
- SRF_RETURN_DONE(fctx);
+
+ for(i=0; i < nattrs; i++)
+ {
+ Form_pg_attribute attr;
+ Datum raw_attr = PointerGetDatum(NULL);
+ bool is_null;
+
+ attr = tuple_desc->attrs[i];
+ is_null = (tuphdr->t_infomask & HEAP_HASNULL) && att_isnull(i, tuphdr->t_bits);
+ /*
+ * Tuple header can specify less attributes then tuple descriptor
+ * as ALTER TABLE ADD COLUMN without DEFAULT keyword does not
+ * actualy change tuples in pages, so attributes with numbers greater
+ * than tuphdr->t_infomask2 & HEAP_NATTS_MASK should be treated as NULL
+ */
+ if (i >= (tuphdr->t_infomask2 & HEAP_NATTS_MASK) )
+ is_null = true;
+ if (!is_null)
+ {
+ int len;
+ bytea * attr_data;
+ if (attr->attlen == -1)
+ {
+ off = att_align_pointer(off, tuple_desc->attrs[i]->attalign, -1, tuple_data_p + off);
+ /*
+ * As VARSIZE_ANY throws an exeption if it can't properly detect
+ * type of external storage in macros VARTAG_SIZE, so we repeat
+ * this check here to preform nicer error handling
+ */
+ if ( inter_call_data->error_level &&
+ VARATT_IS_1B_E(tuple_data_p + off) &&
+ VARTAG_EXTERNAL(tuple_data_p + off) != VARTAG_INDIRECT &&
+ VARTAG_EXTERNAL(tuple_data_p + off) != VARTAG_ONDISK)
+ {
+ ereport(inter_call_data->error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: First byte of varlen attr seems to be corrupted")));
+ }
+ len = VARSIZE_ANY(tuple_data_p + off);
+ }
+ else
+ {
+ off = att_align_nominal(off, tuple_desc->attrs[i]->attalign);
+ len = attr->attlen;
+ }
+ if ( inter_call_data->error_level &&
+ lp_len < tuphdr->t_hoff + off + len)
+ {
+ ereport(inter_call_data->error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: Iterating over tuple data reached out of actual tuple size")));
+ }
+ attr_data = (bytea *) palloc(len + VARHDRSZ);
+ SET_VARSIZE(attr_data, len + VARHDRSZ);
+ memcpy(VARDATA(attr_data), tuple_data_p + off, len);
+ raw_attr = PointerGetDatum(attr_data);
+
+ if ( attr->attlen == -1 && do_detoast)
+ {
+ Datum raw_attr_copy;
+ raw_attr_copy = PointerGetDatum(PG_DETOAST_DATUM_COPY(tuple_data_p + off));
+ pfree(attr_data);
+ raw_attr = raw_attr_copy;
+ }
+ off = att_addlength_pointer(off, tuple_desc->attrs[i]->attlen, tuple_data_p + off);
+ }
+ raw_attrs = accumArrayResult(raw_attrs, raw_attr, is_null, BYTEAOID, fctx->multi_call_memory_ctx);
+ }
+ if (inter_call_data->error_level &&
+ lp_len != tuphdr->t_hoff + off)
+ {
+ ereport(inter_call_data->error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: Iterating over tuple data did not actualy reach tuple end")));
+ }
+ pfree(DatumGetPointer(values[13]));
+ values[13] = makeArrayResult(raw_attrs, fctx->multi_call_memory_ctx);
}
diff --git a/contrib/pageinspect/pageinspect--1.3--1.4.sql b/contrib/pageinspect/pageinspect--1.3--1.4.sql
new file mode 100644
index 0000000..3966045
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.3--1.4.sql
@@ -0,0 +1,110 @@
+/* contrib/pageinspect/pageinspect--1.3--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via ALTER EXTENSION
+\echo Use "ALTER EXTENSION pageinspect UPDATE TO '1.4'" to load this file. \quit
+
+--
+-- heap_page_items()
+--
+DROP FUNCTION heap_page_items(bytea);
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_item_attrs'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass, IN warning_mode bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_item_attrs'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_item_detoast_attrs()
+--
+
+CREATE FUNCTION heap_page_item_detoast_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_item_detoast_attrs'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION heap_page_item_detoast_attrs(IN page bytea, IN heap_oid regclass, IN warning_mode bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_item_detoast_attrs'
+LANGUAGE C STRICT;
\ No newline at end of file
diff --git a/contrib/pageinspect/pageinspect--1.3.sql b/contrib/pageinspect/pageinspect--1.3.sql
deleted file mode 100644
index a99e058..0000000
--- a/contrib/pageinspect/pageinspect--1.3.sql
+++ /dev/null
@@ -1,189 +0,0 @@
-/* contrib/pageinspect/pageinspect--1.3.sql */
-
--- complain if script is sourced in psql, rather than via CREATE EXTENSION
-\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
-
---
--- get_raw_page()
---
-CREATE FUNCTION get_raw_page(text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page'
-LANGUAGE C STRICT;
-
-CREATE FUNCTION get_raw_page(text, text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page_fork'
-LANGUAGE C STRICT;
-
---
--- page_header()
---
-CREATE FUNCTION page_header(IN page bytea,
- OUT lsn pg_lsn,
- OUT checksum smallint,
- OUT flags smallint,
- OUT lower smallint,
- OUT upper smallint,
- OUT special smallint,
- OUT pagesize smallint,
- OUT version smallint,
- OUT prune_xid xid)
-AS 'MODULE_PATHNAME', 'page_header'
-LANGUAGE C STRICT;
-
---
--- heap_page_items()
---
-CREATE FUNCTION heap_page_items(IN page bytea,
- OUT lp smallint,
- OUT lp_off smallint,
- OUT lp_flags smallint,
- OUT lp_len smallint,
- OUT t_xmin xid,
- OUT t_xmax xid,
- OUT t_field3 int4,
- OUT t_ctid tid,
- OUT t_infomask2 integer,
- OUT t_infomask integer,
- OUT t_hoff smallint,
- OUT t_bits text,
- OUT t_oid oid)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'heap_page_items'
-LANGUAGE C STRICT;
-
---
--- bt_metap()
---
-CREATE FUNCTION bt_metap(IN relname text,
- OUT magic int4,
- OUT version int4,
- OUT root int4,
- OUT level int4,
- OUT fastroot int4,
- OUT fastlevel int4)
-AS 'MODULE_PATHNAME', 'bt_metap'
-LANGUAGE C STRICT;
-
---
--- bt_page_stats()
---
-CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
- OUT blkno int4,
- OUT type "char",
- OUT live_items int4,
- OUT dead_items int4,
- OUT avg_item_size int4,
- OUT page_size int4,
- OUT free_size int4,
- OUT btpo_prev int4,
- OUT btpo_next int4,
- OUT btpo int4,
- OUT btpo_flags int4)
-AS 'MODULE_PATHNAME', 'bt_page_stats'
-LANGUAGE C STRICT;
-
---
--- bt_page_items()
---
-CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
- OUT itemoffset smallint,
- OUT ctid tid,
- OUT itemlen smallint,
- OUT nulls bool,
- OUT vars bool,
- OUT data text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'bt_page_items'
-LANGUAGE C STRICT;
-
---
--- brin_page_type()
---
-CREATE FUNCTION brin_page_type(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'brin_page_type'
-LANGUAGE C STRICT;
-
---
--- brin_metapage_info()
---
-CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
- OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
-AS 'MODULE_PATHNAME', 'brin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- brin_revmap_data()
---
-CREATE FUNCTION brin_revmap_data(IN page bytea,
- OUT pages tid)
-RETURNS SETOF tid
-AS 'MODULE_PATHNAME', 'brin_revmap_data'
-LANGUAGE C STRICT;
-
---
--- brin_page_items()
---
-CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
- OUT itemoffset int,
- OUT blknum int,
- OUT attnum int,
- OUT allnulls bool,
- OUT hasnulls bool,
- OUT placeholder bool,
- OUT value text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'brin_page_items'
-LANGUAGE C STRICT;
-
---
--- fsm_page_contents()
---
-CREATE FUNCTION fsm_page_contents(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'fsm_page_contents'
-LANGUAGE C STRICT;
-
---
--- GIN functions
---
-
---
--- gin_metapage_info()
---
-CREATE FUNCTION gin_metapage_info(IN page bytea,
- OUT pending_head bigint,
- OUT pending_tail bigint,
- OUT tail_free_size int4,
- OUT n_pending_pages bigint,
- OUT n_pending_tuples bigint,
- OUT n_total_pages bigint,
- OUT n_entry_pages bigint,
- OUT n_data_pages bigint,
- OUT n_entries bigint,
- OUT version int4)
-AS 'MODULE_PATHNAME', 'gin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- gin_page_opaque_info()
---
-CREATE FUNCTION gin_page_opaque_info(IN page bytea,
- OUT rightlink bigint,
- OUT maxoff int4,
- OUT flags text[])
-AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
-LANGUAGE C STRICT;
-
---
--- gin_leafpage_items()
---
-CREATE FUNCTION gin_leafpage_items(IN page bytea,
- OUT first_tid tid,
- OUT nbytes int2,
- OUT tids tid[])
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'gin_leafpage_items'
-LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect--1.4.sql b/contrib/pageinspect/pageinspect--1.4.sql
new file mode 100644
index 0000000..96aac98
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.4.sql
@@ -0,0 +1,273 @@
+/* contrib/pageinspect/pageinspect--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
+
+--
+-- get_raw_page()
+--
+CREATE FUNCTION get_raw_page(text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION get_raw_page(text, text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page_fork'
+LANGUAGE C STRICT;
+
+--
+-- page_header()
+--
+CREATE FUNCTION page_header(IN page bytea,
+ OUT lsn pg_lsn,
+ OUT checksum smallint,
+ OUT flags smallint,
+ OUT lower smallint,
+ OUT upper smallint,
+ OUT special smallint,
+ OUT pagesize smallint,
+ OUT version smallint,
+ OUT prune_xid xid)
+AS 'MODULE_PATHNAME', 'page_header'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_items()
+--
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_item_attrs'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass, IN warning_mode bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_item_attrs'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_item_detoast_attrs()
+--
+
+CREATE FUNCTION heap_page_item_detoast_attrs(IN page bytea, IN heap_oid regclass, IN warning_mode bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_item_detoast_attrs'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION heap_page_item_detoast_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_item_detoast_attrs'
+LANGUAGE C STRICT;
+
+--
+-- bt_metap()
+--
+CREATE FUNCTION bt_metap(IN relname text,
+ OUT magic int4,
+ OUT version int4,
+ OUT root int4,
+ OUT level int4,
+ OUT fastroot int4,
+ OUT fastlevel int4)
+AS 'MODULE_PATHNAME', 'bt_metap'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_stats()
+--
+CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
+ OUT blkno int4,
+ OUT type "char",
+ OUT live_items int4,
+ OUT dead_items int4,
+ OUT avg_item_size int4,
+ OUT page_size int4,
+ OUT free_size int4,
+ OUT btpo_prev int4,
+ OUT btpo_next int4,
+ OUT btpo int4,
+ OUT btpo_flags int4)
+AS 'MODULE_PATHNAME', 'bt_page_stats'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_items()
+--
+CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
+ OUT itemoffset smallint,
+ OUT ctid tid,
+ OUT itemlen smallint,
+ OUT nulls bool,
+ OUT vars bool,
+ OUT data text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'bt_page_items'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_type()
+--
+CREATE FUNCTION brin_page_type(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'brin_page_type'
+LANGUAGE C STRICT;
+
+--
+-- brin_metapage_info()
+--
+CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
+ OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
+AS 'MODULE_PATHNAME', 'brin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- brin_revmap_data()
+--
+CREATE FUNCTION brin_revmap_data(IN page bytea,
+ OUT pages tid)
+RETURNS SETOF tid
+AS 'MODULE_PATHNAME', 'brin_revmap_data'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_items()
+--
+CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
+ OUT itemoffset int,
+ OUT blknum int,
+ OUT attnum int,
+ OUT allnulls bool,
+ OUT hasnulls bool,
+ OUT placeholder bool,
+ OUT value text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'brin_page_items'
+LANGUAGE C STRICT;
+
+--
+-- fsm_page_contents()
+--
+CREATE FUNCTION fsm_page_contents(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'fsm_page_contents'
+LANGUAGE C STRICT;
+
+--
+-- GIN functions
+--
+
+--
+-- gin_metapage_info()
+--
+CREATE FUNCTION gin_metapage_info(IN page bytea,
+ OUT pending_head bigint,
+ OUT pending_tail bigint,
+ OUT tail_free_size int4,
+ OUT n_pending_pages bigint,
+ OUT n_pending_tuples bigint,
+ OUT n_total_pages bigint,
+ OUT n_entry_pages bigint,
+ OUT n_data_pages bigint,
+ OUT n_entries bigint,
+ OUT version int4)
+AS 'MODULE_PATHNAME', 'gin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_page_opaque_info()
+--
+CREATE FUNCTION gin_page_opaque_info(IN page bytea,
+ OUT rightlink bigint,
+ OUT maxoff int4,
+ OUT flags text[])
+AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_leafpage_items()
+--
+CREATE FUNCTION gin_leafpage_items(IN page bytea,
+ OUT first_tid tid,
+ OUT nbytes int2,
+ OUT tids tid[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'gin_leafpage_items'
+LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect.control b/contrib/pageinspect/pageinspect.control
index a9dab33..68c7d61 100644
--- a/contrib/pageinspect/pageinspect.control
+++ b/contrib/pageinspect/pageinspect.control
@@ -1,5 +1,5 @@
# pageinspect extension
comment = 'inspect the contents of database pages at a low level'
-default_version = '1.3'
+default_version = '1.4'
module_pathname = '$libdir/pageinspect'
relocatable = true
diff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.c
index 38c136f..6659ee5 100644
--- a/contrib/pageinspect/rawpage.c
+++ b/contrib/pageinspect/rawpage.c
@@ -26,11 +26,9 @@
#include "utils/pg_lsn.h"
#include "utils/rel.h"
-PG_MODULE_MAGIC;
-
-static bytea *get_raw_page_internal(text *relname, ForkNumber forknum,
- BlockNumber blkno);
+#include "rawpage.h"
+PG_MODULE_MAGIC;
/*
* get_raw_page
@@ -87,7 +85,7 @@ get_raw_page_fork(PG_FUNCTION_ARGS)
/*
* workhorse
*/
-static bytea *
+bytea *
get_raw_page_internal(text *relname, ForkNumber forknum, BlockNumber blkno)
{
bytea *raw_page;
diff --git a/contrib/pageinspect/rawpage.h b/contrib/pageinspect/rawpage.h
new file mode 100644
index 0000000..ef8c8ff
--- /dev/null
+++ b/contrib/pageinspect/rawpage.h
@@ -0,0 +1,20 @@
+/*-------------------------------------------------------------------------
+ *
+ * rawpage.h
+ *
+ * Copyright (c) 2007-2015, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * contrib/pageinspect/rawpage.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef __RAWPAGE_H__
+#define __RAWPAGE_H__
+
+
+bytea *get_raw_page_internal(text *relname, ForkNumber forknum,
+ BlockNumber blkno);
+
+#endif /*__RAWPAGE_H__*/
Re: pageinspect patch, for showing tuple data
On Sat, Sep 5, 2015 at 1:05 AM, Nikolay Shaplov wrote:
В письме от 4 сентября 2015 14:58:29 пользователь Michael Paquier написал:
Documentation is missing, that would be good to have to understand what
each function is intended to do.I were going to add documentation when this patch is commited, or at least
approved for commit. So I would know for sure that function definition
would
not change, so had not to rewrite it again and again.
I doubt that this patch would be committed without documentation, this is a
requirement for any patch.
So if it is possible I would write documentation and test when this patch
is
already approved.
I'm fine with that. Let's discuss its shape and come up with an agreement.
It would have been good to post test cases of what this stuff actually does
though, people reviewing this thread are limited to guess on what is tried
to be achieved just by reading the code. That's actually where
documentation, even a draft of documentation helps a lot in the review to
see if what is expected by the developer matches what the code actually
does.
Code has some whitespaces.
I've found and removed some. Hope that was all of them...
Yeah, it looks that you took completely rid of them.
In details, this patch introduces two independent concepts:
- add tuple data as a new bytea column of heap_page_items. This is indeed
where it should be, and note that this is where the several corruption
checks are done by checking the state of the tuple data.
- add heap_page_item_attrs and heap_page_item_detoast_attrs, which is very
similar to heap_page_items, at the difference that it can take an OID to be
able to use a TupleDesc and return a bytea array with the data of each
attribute.
Honestly, heap_page_item_attrs and heap_page_item_detoast_attrs are way too
similar to what heap_page_items does, leading to a code maze that is going
to make future extensions more difficult, which is what lead to the
refactoring your patch does.
Hence, instead of all those complications, why not simply introducing two
functions that take as input the tuple data and the OID of the relation,
returning those bytea arrays? It seems to be that this would be a more
handy interface, and as this is for educational purposes I guess that the
addition of the overhead induced by the extra function call would be
acceptable.
Regards,
--
Michael
Re: pageinspect patch, for showing tuple data
On Tue, Sep 8, 2015 at 11:53 AM, Michael Paquier wrote:
Honestly, heap_page_item_attrs and heap_page_item_detoast_attrs are way too
similar to what heap_page_items does, leading to a code maze that is going
to make future extensions more difficult, which is what lead to the
refactoring your patch does.
Hence, instead of all those complications, why not simply introducing two
functions that take as input the tuple data and the OID of the relation,
returning those bytea arrays? It seems to be that this would be a more handy
interface, and as this is for educational purposes I guess that the addition
of the overhead induced by the extra function call would be acceptable.
Actually not two functions, but just one, with an extra flag to be
able to enforce detoasting on the field values where this can be done.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
В письме от 8 сентября 2015 11:53:24 Вы написали:
On Sat, Sep 5, 2015 at 1:05 AM, Nikolay Shaplov wrote:
В письме от 4 сентября 2015 14:58:29 пользователь Michael Paquier написал:
Documentation is missing, that would be good to have to understand what
each function is intended to do.I were going to add documentation when this patch is commited, or at least
approved for commit. So I would know for sure that function definition
would
not change, so had not to rewrite it again and again.I doubt that this patch would be committed without documentation, this is a
requirement for any patch.
Ok. I can do it.
So if it is possible I would write documentation and test when this patch
is
already approved.I'm fine with that. Let's discuss its shape and come up with an agreement.
It would have been good to post test cases of what this stuff actually does
though, people reviewing this thread are limited to guess on what is tried
to be achieved just by reading the code.
To test non detoasted functions one should do:
CREATE EXTENSION pageinspect;
create table test (a int, b smallint,c varchar);
insert into test VALUES
(-1,2,'111'),
(2,null,'222'),
(3,8,'333'),
(null,16,null);
Then
# select * from heap_page_items(get_raw_page('test', 0));
lp | lp_off | lp_flags | lp_len | t_xmin | t_xmax | t_field3 | t_ctid | t_infomask2 | t_infomask | t_hoff | t_bits | t_oid | t_data
----+--------+----------+--------+--------+--------+----------+--------+-------------+------------+--------+----------+-------+------------------------
1 | 8152 | 1 | 34 | 763 | 0 | 0 | (0,1) | 3 | 2050 | 24 | | | \xffffffff020009313131
2 | 8120 | 1 | 32 | 763 | 0 | 0 | (0,2) | 3 | 2051 | 24 | 10100000 | | \x0200000009323232
3 | 8080 | 1 | 34 | 763 | 0 | 0 | (0,3) | 3 | 2050 | 24 | | | \x03000000080009333333
4 | 8048 | 1 | 26 | 763 | 0 | 0 | (0,4) | 3 | 2049 | 24 | 01000000 | | \x1000
or
# select * from heap_page_item_attrs(get_raw_page('test', 0),'test'::regclass);
lp | lp_off | lp_flags | lp_len | t_xmin | t_xmax | t_field3 | t_ctid | t_infomask2 | t_infomask | t_hoff | t_bits | t_oid | t_attrs
----+--------+----------+--------+--------+--------+----------+--------+-------------+------------+--------+----------+-------+-----------------------------------------
1 | 8152 | 1 | 34 | 763 | 0 | 0 | (0,1) | 3 | 2050 | 24 | | | {"\\xffffffff","\\x0200","\\x09313131"}
2 | 8120 | 1 | 32 | 763 | 0 | 0 | (0,2) | 3 | 2051 | 24 | 10100000 | | {"\\x02000000",NULL,"\\x09323232"}
3 | 8080 | 1 | 34 | 763 | 0 | 0 | (0,3) | 3 | 2050 | 24 | | | {"\\x03000000","\\x0800","\\x09333333"}
4 | 8048 | 1 | 26 | 763 | 0 | 0 | (0,4) | 3 | 2049 | 24 | 01000000 | | {NULL,"\\x1000",NULL}
(4 rows)
For detoasted function you should do something like this:
CREATE EXTENSION pageinspect;
create table test2 (c varchar);
insert into test2 VALUES ('++');
insert into test2 VALUES (repeat('+',2100));
Then heap_page_item_attrs will show you compressed attr data:
# select * from heap_page_item_attrs(get_raw_page('test2', 0),'test2'::regclass);
lp | lp_off | lp_flags | lp_len | t_xmin | t_xmax | t_field3 | t_ctid | t_infomask2 | t_infomask | t_hoff | t_bits | t_oid | t_attrs
----+--------+----------+--------+--------+--------+----------+--------+-------------+------------+--------+--------+-------+-------------------------------------------------------------------------------
1 | 8160 | 1 | 27 | 768 | 0 | 0 | (0,1) | 1 | 2050 | 24 | | | {"\\x072b2b"}
2 | 8096 | 1 | 59 | 769 | 0 | 0 | (0,2) | 1 | 2050 | 24 | | | {"\\x8e00000034080000fe2b0f01ff0f01ff0f01ff0f01ff0f01ff0f01ff0f01ff010f01aa"}
(2 rows)
and heap_page_item_detoast_attrs will show you original data
# select * from heap_page_item_detoast_attrs(get_raw_page('test2', 0),'test2'::regclass);
[will not paste output here it is too long, see it for yourself]
That's actually where
documentation, even a draft of documentation helps a lot in the review to
see if what is expected by the developer matches what the code actually
does.Code has some whitespaces.
I've found and removed some. Hope that was all of them...Yeah, it looks that you took completely rid of them.
In details, this patch introduces two independent concepts:
- add tuple data as a new bytea column of heap_page_items. This is indeed
where it should be, and note that this is where the several corruption
checks are done by checking the state of the tuple data.
Sorry do not understand what do you want to say in the second part of the last
sentence. About corruption checks.
- add heap_page_item_attrs and heap_page_item_detoast_attrs, which is very
similar to heap_page_items, at the difference that it can take an OID to be
able to use a TupleDesc and return a bytea array with the data of each
attribute.
That's right.
And more:
- heap_page_item_attrs and heap_page_item_detoast_attrs has third optional
attribute, set it to true if you want these functions to report about problems in
WARNING mode. This will allow to force attribute parsing even if some consistency
checks are failed. This is necessary for educational purposes, so everyone can
make fake data and see how postgres will try to parse it. Also this will be needed
for test. So we can test that pageinspect will is properly doing all checks. And it is
much easier to do this when it is done in warning mode.
Honestly, heap_page_item_attrs and heap_page_item_detoast_attrs are way too
similar to what heap_page_items does, leading to a code maze that is going
to make future extensions more difficult, which is what lead to the
refactoring your patch does.
Hence, instead of all those complications, why not simply introducing two
functions that take as input the tuple data and the OID of the relation,
returning those bytea arrays? It seems to be that this would be a more
handy interface, and as this is for educational purposes I guess that the
addition of the overhead induced by the extra function call would be
acceptable.
When I looked at this task I considered doing the same thing. Bun unfortunately it is
not possible. (Or if be more correct it is possible, but if I do so, it would be hard to us
e it)
The thing is, that to properly split tuple data into attributes, we need some values from
tuple header:
t_infomask2: where postgres store actual number of stored attributes, that may differ
from one in tuple data. That will allow to properly parse tuples after
ALTER TABLE ADD COLUMN when it was done without SET DEFAULT option
t_infomask: has bit that indicates that there is some null attributes in tuple.
t_bits: has a bit mask that shows what attributes were set to null.
So if move tuple data parsing into separate function, then we have to pass these
values alongside the tuple data. This is not convenient at all.
So I did it in a way you see in the patch.
Actually not two functions, but just one, with an extra flag to be
able to enforce detoasting on the field values where this can be done.
Yeah, I also thought about it. But did not come to any final conclusion. Should
we add forth argument, that enables detoasting, instead of adding separate
function?
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
On Wed, Sep 9, 2015 at 8:39 PM, Nikolay Shaplov
<n.shaplov@postgrespro.ru> wrote:
В письме от 8 сентября 2015 11:53:24 Вы написали:
Hence, instead of all those complications, why not simply introducing two
functions that take as input the tuple data and the OID of the relation,
returning those bytea arrays? It seems to be that this would be a more
handy interface, and as this is for educational purposes I guess that the
addition of the overhead induced by the extra function call would be
acceptable.When I looked at this task I considered doing the same thing. Bun unfortunately it is
not possible. (Or if be more correct it is possible, but if I do so, it would be hard to us
e it)
The thing is, that to properly split tuple data into attributes, we need some values from
tuple header:
t_infomask2: where postgres store actual number of stored attributes, that may differ
from one in tuple data. That will allow to properly parse tuples after
ALTER TABLE ADD COLUMN when it was done without SET DEFAULT option
t_infomask: has bit that indicates that there is some null attributes in tuple.
t_bits: has a bit mask that shows what attributes were set to null.So if move tuple data parsing into separate function, then we have to pass these
values alongside the tuple data. This is not convenient at all.
So I did it in a way you see in the patch.
Why is it not convenient at all? Yes, you have a point, we need those
fields to be able to parse the t_data properly. Still the possibility
to show individual fields of a tuple as a bytea array either with
toasted or detoasted values is a concept completely different from
simply showing the page items, which is what, it seems to me,
heap_page_items is aimed to only do. Hence, As t_infomask2, t_infomask
and t_bits are already available as return fields of heap_page_items,
we should simply add a function like that:
heap_page_item_parse(Oid relid, bytea data, t_infomask2 int,
t_infomask int, t_bits int, bool force_detoast, warning_mode bool)
returns bytea[]
Note that the data corruption checks apply only to this function as
far as I understand, so I think that things could be actually split
into two independent patches:
1) Upgrade heap_page_items to add the tuple data as bytea.
2) Add the new function able to parse those fields appropriately.
As this code, as you justly mentioned, is aimed mainly for educational
purposes to understand a page structure, we should definitely make it
as simple as possible at code level, and it seems to me that this
comes with a separate SQL interface to control tuple data parsing as a
bytea[]. We are doing no favor to our users to complicate the code of
pageinspect.c as this patch is doing in its current state, especially
if beginners want to have a look at it.
Actually not two functions, but just one, with an extra flag to be
able to enforce detoasting on the field values where this can be done.Yeah, I also thought about it. But did not come to any final conclusion. Should
we add forth argument, that enables detoasting, instead of adding separate
function?
This is definitely something you want to control with a switch.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
В письме от 10 сентября 2015 15:46:25 пользователь Michael Paquier написал:
So if move tuple data parsing into separate function, then we have to pass
these values alongside the tuple data. This is not convenient at all.
So I did it in a way you see in the patch.Why is it not convenient at all? Yes, you have a point, we need those
fields to be able to parse the t_data properly. Still the possibility
to show individual fields of a tuple as a bytea array either with
toasted or detoasted values is a concept completely different from
simply showing the page items, which is what, it seems to me,
heap_page_items is aimed to only do. Hence, As t_infomask2, t_infomask
and t_bits are already available as return fields of heap_page_items,
we should simply add a function like that:
heap_page_item_parse(Oid relid, bytea data, t_infomask2 int,
t_infomask int, t_bits int, bool force_detoast, warning_mode bool)
returns bytea[]
Just compare two expressions:
select * from heap_page_item_attrs(get_raw_page('test', 0),'test'::regclass);
and
select lp, lp_off, lp_flags, lp_len, t_xmin, t_xmax, t_field3, t_ctid,
t_infomask2, t_infomask, t_hoff, t_bits, t_oid, tuple_data_parse (
t_data, t_infomask, t_infomask2, t_bits, 'test'::regclass, true, false)
from heap_page_item_attrs(get_raw_page('test', 0));
The second variant is a total mess and almost unsable...
Though I've discussed this issue with friedns, and we came to conclusion that
it might be good to implement tuple_data_parse and then implement
easy to use heap_page_item_attrs in pure SQL, using heap_page_items and
tuple_data_parse.
That would keep usage simplicity, and make code more simple as you offer.
The only one argument against it is that in internal representation t_bist is
binary, and in sql output it is string with '0' and '1' characters. We will
have to convert it back to binary mode. This is not difficult, but just useless
convertations there and back again.
What do you think about this solution?
Note that the data corruption checks apply only to this function as
far as I understand, so I think that things could be actually split
into two independent patches:
1) Upgrade heap_page_items to add the tuple data as bytea.
2) Add the new function able to parse those fields appropriately.As this code, as you justly mentioned, is aimed mainly for educational
purposes to understand a page structure, we should definitely make it
as simple as possible at code level, and it seems to me that this
comes with a separate SQL interface to control tuple data parsing as a
bytea[]. We are doing no favor to our users to complicate the code of
pageinspect.c as this patch is doing in its current state, especially
if beginners want to have a look at it.Actually not two functions, but just one, with an extra flag to be
able to enforce detoasting on the field values where this can be done.Yeah, I also thought about it. But did not come to any final conclusion.
Should we add forth argument, that enables detoasting, instead of adding
separate function?This is definitely something you want to control with a switch.
Ok.Let's come to the final decision with tuple_data_parse, and i will add this
switch there and to pure sql heap_page_item_attrs
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
On Thu, Sep 10, 2015 at 03:46:25PM +0900, Michael Paquier wrote:
Why is it not convenient at all? Yes, you have a point, we need those
fields to be able to parse the t_data properly. Still the possibility
to show individual fields of a tuple as a bytea array either with
toasted or detoasted values is a concept completely different from
simply showing the page items, which is what, it seems to me,
heap_page_items is aimed to only do. Hence, As t_infomask2, t_infomask
and t_bits are already available as return fields of heap_page_items,
we should simply add a function like that:
heap_page_item_parse(Oid relid, bytea data, t_infomask2 int,
t_infomask int, t_bits int, bool force_detoast, warning_mode bool)
returns bytea[]
Should pageinspect create a table that contains some of the constants
used to interpret infomask?
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
On Fri, Sep 11, 2015 at 12:08 AM, Nikolay Shaplov
<n.shaplov@postgrespro.ru> wrote:
В письме от 10 сентября 2015 15:46:25 пользователь Michael Paquier написал:
So if move tuple data parsing into separate function, then we have to pass
these values alongside the tuple data. This is not convenient at all.
So I did it in a way you see in the patch.Why is it not convenient at all? Yes, you have a point, we need those
fields to be able to parse the t_data properly. Still the possibility
to show individual fields of a tuple as a bytea array either with
toasted or detoasted values is a concept completely different from
simply showing the page items, which is what, it seems to me,
heap_page_items is aimed to only do. Hence, As t_infomask2, t_infomask
and t_bits are already available as return fields of heap_page_items,
we should simply add a function like that:
heap_page_item_parse(Oid relid, bytea data, t_infomask2 int,
t_infomask int, t_bits int, bool force_detoast, warning_mode bool)
returns bytea[]Just compare two expressions:
select * from heap_page_item_attrs(get_raw_page('test', 0),'test'::regclass);
and
select lp, lp_off, lp_flags, lp_len, t_xmin, t_xmax, t_field3, t_ctid,
t_infomask2, t_infomask, t_hoff, t_bits, t_oid, tuple_data_parse (
t_data, t_infomask, t_infomask2, t_bits, 'test'::regclass, true, false)
from heap_page_item_attrs(get_raw_page('test', 0));The second variant is a total mess and almost unsable...
It is hard to believe as well that any sane application would use * as
well in a SELECT query. Users would though and we are talking about
user's education here :)
Though I've discussed this issue with friedns, and we came to conclusion that
it might be good to implement tuple_data_parse and then implement
easy to use heap_page_item_attrs in pure SQL, using heap_page_items and
tuple_data_parse.
That would keep usage simplicity, and make code more simple as you offer.
Yep. That's doable with a simple SQL function. I am not sure that it
is worth including in pageinspect though.
The only one argument against it is that in internal representation t_bits is
binary, and in sql output it is string with '0' and '1' characters. We will
have to convert it back to binary mode. This is not difficult, but just useless
convertations there and back again.
The reason why this is visibly converted from bit to text is that the
in-core bit data type has a fixed length, and in the case of
HeapTupleHeaderData there is a variable length.
What do you think about this solution?
For code simplicity's sake this seems worth the cost.
Note that the data corruption checks apply only to this function as
far as I understand, so I think that things could be actually split
into two independent patches:
1) Upgrade heap_page_items to add the tuple data as bytea.
2) Add the new function able to parse those fields appropriately.As this code, as you justly mentioned, is aimed mainly for educational
purposes to understand a page structure, we should definitely make it
as simple as possible at code level, and it seems to me that this
comes with a separate SQL interface to control tuple data parsing as a
bytea[]. We are doing no favor to our users to complicate the code of
pageinspect.c as this patch is doing in its current state, especially
if beginners want to have a look at it.Actually not two functions, but just one, with an extra flag to be
able to enforce detoasting on the field values where this can be done.Yeah, I also thought about it. But did not come to any final conclusion.
Should we add forth argument, that enables detoasting, instead of adding
separate function?This is definitely something you want to control with a switch.
Ok.Let's come to the final decision with tuple_data_parse, and i will add this
switch there and to pure sql heap_page_item_attrs
Fine for me.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
On Fri, Sep 11, 2015 at 12:12 AM, Bruce Momjian <bruce@momjian.us> wrote:
On Thu, Sep 10, 2015 at 03:46:25PM +0900, Michael Paquier wrote:
Why is it not convenient at all? Yes, you have a point, we need those
fields to be able to parse the t_data properly. Still the possibility
to show individual fields of a tuple as a bytea array either with
toasted or detoasted values is a concept completely different from
simply showing the page items, which is what, it seems to me,
heap_page_items is aimed to only do. Hence, As t_infomask2, t_infomask
and t_bits are already available as return fields of heap_page_items,
we should simply add a function like that:
heap_page_item_parse(Oid relid, bytea data, t_infomask2 int,
t_infomask int, t_bits int, bool force_detoast, warning_mode bool)
returns bytea[]Should pageinspect create a table that contains some of the constants
used to interpret infomask?
Interesting idea. It may be indeed useful to show to a user mappings
between t_infomask flags <=> textual meaning. I guess that we could
have an SRF function with a view on top of it that returns such a
list. The same can apply to t_infomask2.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
В письме от 11 сентября 2015 15:12:04 Вы написали:
Ok.Let's come to the final decision with tuple_data_parse, and i will add
this switch there and to pure sql heap_page_item_attrsFine for me.
So I've modified the code, now we have:
heap_page_items - have a column with raw tuple data
tuple_data_split - takes oid, raw tuple data, infomask, infomask2 and bits
parsed as string and returns bytea[] with attribute raw values. It also have
two optional arguments do_detoast that forces function to detoast attribute,
and warning_mode that allows to set this function to warning mode, and do not
stop working if some inconsistency were found.
there is also pure sql function heap_page_item_attrs that takes page data, and
table oid, and returns same data as heap_page_items but bytea[] of attributes
instead of one whole piece of raw data. It also has optional argument
do_detoast that allows to get bytea[] of detoasted attribute data.
I've decided that there is no real need in warning_mode in
heap_page_item_attrs so there is no such argument there.
So now it is still RFC. Final patch with documentation will come soon
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
pageinspect_show_tuple_data_v4a.difftext/x-patch; charset=utf-8; name=pageinspect_show_tuple_data_v4a.diffDownload
diff --git a/contrib/pageinspect/Makefile b/contrib/pageinspect/Makefile
index aec5258..e4bc1af 100644
--- a/contrib/pageinspect/Makefile
+++ b/contrib/pageinspect/Makefile
@@ -5,7 +5,7 @@ OBJS = rawpage.o heapfuncs.o btreefuncs.o fsmfuncs.o \
brinfuncs.o ginfuncs.o $(WIN32RES)
EXTENSION = pageinspect
-DATA = pageinspect--1.3.sql pageinspect--1.2--1.3.sql \
+DATA = pageinspect--1.4.sql pageinspect--1.3--1.4.sql pageinspect--1.2--1.3.sql \
pageinspect--1.1--1.2.sql pageinspect--1.0--1.1.sql \
pageinspect--unpackaged--1.0.sql
PGFILEDESC = "pageinspect - functions to inspect contents of database pages"
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 8d1666c..d42ca48 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -29,7 +29,11 @@
#include "funcapi.h"
#include "utils/builtins.h"
#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/rel.h"
+#include "catalog/pg_type.h"
+Datum split_tuple_data(char *tuple_data, uint16 tuple_data_len, TupleDesc tuple_desc, uint16 t_infomask, uint16 t_infomask2, bits8 *t_bits, bool do_detoast, int error_level);
/*
* bits_to_text
@@ -122,8 +126,8 @@ heap_page_items(PG_FUNCTION_ARGS)
HeapTuple resultTuple;
Datum result;
ItemId id;
- Datum values[13];
- bool nulls[13];
+ Datum values[14];
+ bool nulls[14];
uint16 lp_offset;
uint16 lp_flags;
uint16 lp_len;
@@ -155,6 +159,8 @@ heap_page_items(PG_FUNCTION_ARGS)
{
HeapTupleHeader tuphdr;
int bits_len;
+ bytea *tuple_data_bytea;
+ int tuple_data_len;
/* Extract information from the tuple header */
@@ -168,6 +174,13 @@ heap_page_items(PG_FUNCTION_ARGS)
values[9] = UInt32GetDatum(tuphdr->t_infomask);
values[10] = UInt8GetDatum(tuphdr->t_hoff);
+ /* Copy raw tuple data into bytea attribute */
+ tuple_data_len = lp_len - tuphdr->t_hoff;
+ tuple_data_bytea = (bytea *) palloc(tuple_data_len + VARHDRSZ);
+ SET_VARSIZE(tuple_data_bytea, tuple_data_len + VARHDRSZ);
+ memcpy(VARDATA(tuple_data_bytea), (char *) tuphdr + tuphdr->t_hoff, tuple_data_len);
+ values[13] = PointerGetDatum(tuple_data_bytea);
+ nulls[13] = false;
/*
* We already checked that the item is completely within the raw
* page passed to us, with the length given in the line pointer.
@@ -208,7 +221,7 @@ heap_page_items(PG_FUNCTION_ARGS)
*/
int i;
- for (i = 4; i <= 12; i++)
+ for (i = 4; i <= 13; i++)
nulls[i] = true;
}
@@ -223,3 +236,210 @@ heap_page_items(PG_FUNCTION_ARGS)
else
SRF_RETURN_DONE(fctx);
}
+
+PG_FUNCTION_INFO_V1(tuple_data_split);
+Datum
+tuple_data_split(PG_FUNCTION_ARGS)
+{
+ Oid rel_oid;
+ bytea *raw_data;
+ uint16 t_infomask;
+ uint16 t_infomask2;
+ text *t_bits_str;
+ RelationData *rel;
+ TupleDesc tuple_desc;
+ bool do_detoast = false;
+ int error_level = ERROR;
+
+ bits8 *t_bits = NULL;
+ Datum res;
+
+ rel_oid = PG_GETARG_OID(0);
+ raw_data = PG_GETARG_BYTEA_P(1);
+ t_infomask = PG_GETARG_INT16(2);
+ t_infomask2 = PG_GETARG_INT16(3);
+ t_bits_str = PG_ARGISNULL(4) ? NULL : PG_GETARG_TEXT_P(4);
+ if (PG_NARGS()>=6) do_detoast = PG_GETARG_BOOL(5);
+ if (PG_NARGS()>=7) error_level = PG_GETARG_BOOL(6)?WARNING:ERROR;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ (errmsg("must be superuser to use raw page functions"))));
+
+ /*
+ * Here we converting t_bits string back to the bits8 array,
+ * as it really is in the tuple header
+ */
+ if (t_infomask & HEAP_HASNULL)
+ {
+ int bits_str_len;
+ int bits_len;
+ char * p;
+ int off;
+ char byte = 0;
+ bits_len = (t_infomask2 & HEAP_NATTS_MASK) / 8 + 1;
+ if (! t_bits_str)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits argument is NULL, though we expect it to be NOT NULL and %i character long", bits_len * 8)));
+ }
+ bits_str_len = VARSIZE(t_bits_str) - VARHDRSZ;
+ if (bits_str_len % 8)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits argument length should be multiple of eight")));
+ }
+ if (bits_len * 8 != bits_str_len)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("wrong t_bits argument length. Expected %i, actual is %i", bits_len * 8, bits_str_len)));
+ }
+ t_bits = palloc(bits_len + 1);
+
+ p = (char *) t_bits_str + VARHDRSZ;
+ off = 0;
+ while (off<bits_str_len)
+ {
+ if (!(off % 8))
+ {
+ byte = 0;
+ }
+ if (( p[off] == '0') || (p[off] == '1'))
+ {
+ byte = byte | ( (p[off]-'0')<<off % 8);
+ } else
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("illegal character '%c' in t_bits string", p[off])));
+ }
+ if (off % 8 == 7)
+ {
+ t_bits[off / 8] = byte;
+ }
+ off++;
+ }
+ } else
+ {
+ if (t_bits_str)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits string is expected to be NULL, but instead it is %i bytes length", VARSIZE(t_bits_str) - VARHDRSZ)));
+ }
+ }
+ rel = relation_open(rel_oid, NoLock);
+ tuple_desc = CreateTupleDescCopyConstr(rel->rd_att);
+ relation_close(rel, NoLock);
+ res = split_tuple_data((char *) raw_data + VARHDRSZ, VARSIZE(raw_data) - VARHDRSZ, tuple_desc, t_infomask, t_infomask2, t_bits, do_detoast, error_level);
+ PG_RETURN_ARRAYTYPE_P(res);
+}
+
+
+Datum
+split_tuple_data(char *tuple_data, uint16 tuple_data_len, TupleDesc tuple_desc, uint16 t_infomask, uint16 t_infomask2, bits8 *t_bits, bool do_detoast, int error_level)
+{
+ ArrayBuildState *raw_attrs;
+ int nattrs;
+ int i;
+ int off;
+
+ /*
+ * Here we reimplement the basic functionality of nocachegetattr from
+ * backend/access/common/heaptuple.c whitch is basically used for fetching
+ * attributes from tuple when it is not cached. We can not use nocachegetattr here
+ * directly, because we should ignore all cache optimisations and other stuff
+ * just get binary data as it is.
+ */
+
+ raw_attrs = initArrayResult(BYTEAOID,CurrentMemoryContext,0);
+ off = 0;
+ nattrs = tuple_desc->natts;
+ if (error_level &&
+ nattrs < (t_infomask2 & HEAP_NATTS_MASK))
+ {
+ ereport(error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: number of attributes in tuple header is greater than number of attributes in tuple descripor")));
+ }
+
+ for(i=0; i < nattrs; i++)
+ {
+ Form_pg_attribute attr;
+ Datum raw_attr = PointerGetDatum(NULL);
+ bool is_null;
+
+ attr = tuple_desc->attrs[i];
+ is_null = (t_infomask & HEAP_HASNULL) && att_isnull(i, t_bits);
+ /*
+ * Tuple header can specify less attributes then tuple descriptor
+ * as ALTER TABLE ADD COLUMN without DEFAULT keyword does not
+ * actualy change tuples in pages, so attributes with numbers greater
+ * than t_infomask2 & HEAP_NATTS_MASK should be treated as NULL
+ */
+ if (i >= (t_infomask2 & HEAP_NATTS_MASK) )
+ is_null = true;
+ if (!is_null)
+ {
+ int len;
+ bytea * attr_data;
+ if (attr->attlen == -1)
+ {
+ off = att_align_pointer(off, tuple_desc->attrs[i]->attalign, -1, tuple_data + off);
+ /*
+ * As VARSIZE_ANY throws an exeption if it can't properly detect
+ * type of external storage in macros VARTAG_SIZE, so we repeat
+ * this check here to preform nicer error handling
+ */
+ if ( error_level &&
+ VARATT_IS_1B_E(tuple_data + off) &&
+ VARTAG_EXTERNAL(tuple_data + off) != VARTAG_INDIRECT &&
+ VARTAG_EXTERNAL(tuple_data + off) != VARTAG_ONDISK)
+ {
+ ereport(error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: First byte of varlen attr seems to be corrupted")));
+ }
+ len = VARSIZE_ANY(tuple_data + off);
+ }
+ else
+ {
+ off = att_align_nominal(off, tuple_desc->attrs[i]->attalign);
+ len = attr->attlen;
+ }
+ if ( error_level &&
+ tuple_data_len < off + len)
+ {
+ ereport(error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: Iterating over tuple data reached out of actual tuple size")));
+ }
+ attr_data = (bytea *) palloc(len + VARHDRSZ);
+ SET_VARSIZE(attr_data, len + VARHDRSZ);
+ memcpy(VARDATA(attr_data), tuple_data + off, len);
+ raw_attr = PointerGetDatum(attr_data);
+
+ if ( attr->attlen == -1 && do_detoast)
+ {
+ Datum raw_attr_copy;
+ raw_attr_copy = PointerGetDatum(PG_DETOAST_DATUM_COPY(tuple_data + off));
+ pfree(attr_data);
+ raw_attr = raw_attr_copy;
+ }
+ off = att_addlength_pointer(off, tuple_desc->attrs[i]->attlen, tuple_data + off);
+ }
+ raw_attrs = accumArrayResult(raw_attrs, raw_attr, is_null, BYTEAOID, CurrentMemoryContext);
+ }
+ if (error_level &&
+ tuple_data_len != off)
+ {
+ ereport(error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: Iterating over tuple data did not actualy reach tuple end")));
+ }
+ return makeArrayResult(raw_attrs, CurrentMemoryContext);
+}
diff --git a/contrib/pageinspect/pageinspect--1.3--1.4.sql b/contrib/pageinspect/pageinspect--1.3--1.4.sql
new file mode 100644
index 0000000..9c6cd8b
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.3--1.4.sql
@@ -0,0 +1,87 @@
+/* contrib/pageinspect/pageinspect--1.3--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via ALTER EXTENSION
+\echo Use "ALTER EXTENSION pageinspect UPDATE TO '1.4'" to load this file. \quit
+
+--
+-- heap_page_items()
+--
+DROP FUNCTION heap_page_items(bytea);
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(t_oid oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(t_oid oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(t_oid oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool, warning_mode bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass, IN do_detoast bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT lp, lp_off, lp_flags, lp_len, t_xmin, t_xmax, t_field3, t_ctid, t_infomask2, t_infomask, t_hoff, t_bits, t_oid, tuple_data_split(heap_oid, t_data, t_infomask, t_infomask2, t_bits, do_detoast) as t_attrs from heap_page_items(page)'
+LANGUAGE SQL;
+
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT * from heap_page_item_attrs(page, heap_oid, false)'
+LANGUAGE SQL;
diff --git a/contrib/pageinspect/pageinspect--1.3.sql b/contrib/pageinspect/pageinspect--1.3.sql
deleted file mode 100644
index a99e058..0000000
--- a/contrib/pageinspect/pageinspect--1.3.sql
+++ /dev/null
@@ -1,189 +0,0 @@
-/* contrib/pageinspect/pageinspect--1.3.sql */
-
--- complain if script is sourced in psql, rather than via CREATE EXTENSION
-\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
-
---
--- get_raw_page()
---
-CREATE FUNCTION get_raw_page(text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page'
-LANGUAGE C STRICT;
-
-CREATE FUNCTION get_raw_page(text, text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page_fork'
-LANGUAGE C STRICT;
-
---
--- page_header()
---
-CREATE FUNCTION page_header(IN page bytea,
- OUT lsn pg_lsn,
- OUT checksum smallint,
- OUT flags smallint,
- OUT lower smallint,
- OUT upper smallint,
- OUT special smallint,
- OUT pagesize smallint,
- OUT version smallint,
- OUT prune_xid xid)
-AS 'MODULE_PATHNAME', 'page_header'
-LANGUAGE C STRICT;
-
---
--- heap_page_items()
---
-CREATE FUNCTION heap_page_items(IN page bytea,
- OUT lp smallint,
- OUT lp_off smallint,
- OUT lp_flags smallint,
- OUT lp_len smallint,
- OUT t_xmin xid,
- OUT t_xmax xid,
- OUT t_field3 int4,
- OUT t_ctid tid,
- OUT t_infomask2 integer,
- OUT t_infomask integer,
- OUT t_hoff smallint,
- OUT t_bits text,
- OUT t_oid oid)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'heap_page_items'
-LANGUAGE C STRICT;
-
---
--- bt_metap()
---
-CREATE FUNCTION bt_metap(IN relname text,
- OUT magic int4,
- OUT version int4,
- OUT root int4,
- OUT level int4,
- OUT fastroot int4,
- OUT fastlevel int4)
-AS 'MODULE_PATHNAME', 'bt_metap'
-LANGUAGE C STRICT;
-
---
--- bt_page_stats()
---
-CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
- OUT blkno int4,
- OUT type "char",
- OUT live_items int4,
- OUT dead_items int4,
- OUT avg_item_size int4,
- OUT page_size int4,
- OUT free_size int4,
- OUT btpo_prev int4,
- OUT btpo_next int4,
- OUT btpo int4,
- OUT btpo_flags int4)
-AS 'MODULE_PATHNAME', 'bt_page_stats'
-LANGUAGE C STRICT;
-
---
--- bt_page_items()
---
-CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
- OUT itemoffset smallint,
- OUT ctid tid,
- OUT itemlen smallint,
- OUT nulls bool,
- OUT vars bool,
- OUT data text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'bt_page_items'
-LANGUAGE C STRICT;
-
---
--- brin_page_type()
---
-CREATE FUNCTION brin_page_type(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'brin_page_type'
-LANGUAGE C STRICT;
-
---
--- brin_metapage_info()
---
-CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
- OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
-AS 'MODULE_PATHNAME', 'brin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- brin_revmap_data()
---
-CREATE FUNCTION brin_revmap_data(IN page bytea,
- OUT pages tid)
-RETURNS SETOF tid
-AS 'MODULE_PATHNAME', 'brin_revmap_data'
-LANGUAGE C STRICT;
-
---
--- brin_page_items()
---
-CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
- OUT itemoffset int,
- OUT blknum int,
- OUT attnum int,
- OUT allnulls bool,
- OUT hasnulls bool,
- OUT placeholder bool,
- OUT value text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'brin_page_items'
-LANGUAGE C STRICT;
-
---
--- fsm_page_contents()
---
-CREATE FUNCTION fsm_page_contents(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'fsm_page_contents'
-LANGUAGE C STRICT;
-
---
--- GIN functions
---
-
---
--- gin_metapage_info()
---
-CREATE FUNCTION gin_metapage_info(IN page bytea,
- OUT pending_head bigint,
- OUT pending_tail bigint,
- OUT tail_free_size int4,
- OUT n_pending_pages bigint,
- OUT n_pending_tuples bigint,
- OUT n_total_pages bigint,
- OUT n_entry_pages bigint,
- OUT n_data_pages bigint,
- OUT n_entries bigint,
- OUT version int4)
-AS 'MODULE_PATHNAME', 'gin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- gin_page_opaque_info()
---
-CREATE FUNCTION gin_page_opaque_info(IN page bytea,
- OUT rightlink bigint,
- OUT maxoff int4,
- OUT flags text[])
-AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
-LANGUAGE C STRICT;
-
---
--- gin_leafpage_items()
---
-CREATE FUNCTION gin_leafpage_items(IN page bytea,
- OUT first_tid tid,
- OUT nbytes int2,
- OUT tids tid[])
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'gin_leafpage_items'
-LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect--1.4.sql b/contrib/pageinspect/pageinspect--1.4.sql
new file mode 100644
index 0000000..681e7dc
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.4.sql
@@ -0,0 +1,251 @@
+/* contrib/pageinspect/pageinspect--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
+
+--
+-- get_raw_page()
+--
+CREATE FUNCTION get_raw_page(text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION get_raw_page(text, text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page_fork'
+LANGUAGE C STRICT;
+
+--
+-- page_header()
+--
+CREATE FUNCTION page_header(IN page bytea,
+ OUT lsn pg_lsn,
+ OUT checksum smallint,
+ OUT flags smallint,
+ OUT lower smallint,
+ OUT upper smallint,
+ OUT special smallint,
+ OUT pagesize smallint,
+ OUT version smallint,
+ OUT prune_xid xid)
+AS 'MODULE_PATHNAME', 'page_header'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_items()
+--
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea
+ )
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(t_oid oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(t_oid oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(t_oid oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool, warning_mode bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass, IN do_detoast bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT lp, lp_off, lp_flags, lp_len, t_xmin, t_xmax, t_field3, t_ctid, t_infomask2, t_infomask, t_hoff, t_bits, t_oid, tuple_data_split(heap_oid, t_data, t_infomask, t_infomask2, t_bits, do_detoast) as t_attrs from heap_page_items(page)'
+LANGUAGE SQL;
+
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT * from heap_page_item_attrs(page, heap_oid, false)'
+LANGUAGE SQL;
+
+--
+-- bt_metap()
+--
+CREATE FUNCTION bt_metap(IN relname text,
+ OUT magic int4,
+ OUT version int4,
+ OUT root int4,
+ OUT level int4,
+ OUT fastroot int4,
+ OUT fastlevel int4)
+AS 'MODULE_PATHNAME', 'bt_metap'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_stats()
+--
+CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
+ OUT blkno int4,
+ OUT type "char",
+ OUT live_items int4,
+ OUT dead_items int4,
+ OUT avg_item_size int4,
+ OUT page_size int4,
+ OUT free_size int4,
+ OUT btpo_prev int4,
+ OUT btpo_next int4,
+ OUT btpo int4,
+ OUT btpo_flags int4)
+AS 'MODULE_PATHNAME', 'bt_page_stats'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_items()
+--
+CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
+ OUT itemoffset smallint,
+ OUT ctid tid,
+ OUT itemlen smallint,
+ OUT nulls bool,
+ OUT vars bool,
+ OUT data text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'bt_page_items'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_type()
+--
+CREATE FUNCTION brin_page_type(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'brin_page_type'
+LANGUAGE C STRICT;
+
+--
+-- brin_metapage_info()
+--
+CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
+ OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
+AS 'MODULE_PATHNAME', 'brin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- brin_revmap_data()
+--
+CREATE FUNCTION brin_revmap_data(IN page bytea,
+ OUT pages tid)
+RETURNS SETOF tid
+AS 'MODULE_PATHNAME', 'brin_revmap_data'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_items()
+--
+CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
+ OUT itemoffset int,
+ OUT blknum int,
+ OUT attnum int,
+ OUT allnulls bool,
+ OUT hasnulls bool,
+ OUT placeholder bool,
+ OUT value text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'brin_page_items'
+LANGUAGE C STRICT;
+
+--
+-- fsm_page_contents()
+--
+CREATE FUNCTION fsm_page_contents(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'fsm_page_contents'
+LANGUAGE C STRICT;
+
+--
+-- GIN functions
+--
+
+--
+-- gin_metapage_info()
+--
+CREATE FUNCTION gin_metapage_info(IN page bytea,
+ OUT pending_head bigint,
+ OUT pending_tail bigint,
+ OUT tail_free_size int4,
+ OUT n_pending_pages bigint,
+ OUT n_pending_tuples bigint,
+ OUT n_total_pages bigint,
+ OUT n_entry_pages bigint,
+ OUT n_data_pages bigint,
+ OUT n_entries bigint,
+ OUT version int4)
+AS 'MODULE_PATHNAME', 'gin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_page_opaque_info()
+--
+CREATE FUNCTION gin_page_opaque_info(IN page bytea,
+ OUT rightlink bigint,
+ OUT maxoff int4,
+ OUT flags text[])
+AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_leafpage_items()
+--
+CREATE FUNCTION gin_leafpage_items(IN page bytea,
+ OUT first_tid tid,
+ OUT nbytes int2,
+ OUT tids tid[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'gin_leafpage_items'
+LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect.control b/contrib/pageinspect/pageinspect.control
index a9dab33..68c7d61 100644
--- a/contrib/pageinspect/pageinspect.control
+++ b/contrib/pageinspect/pageinspect.control
@@ -1,5 +1,5 @@
# pageinspect extension
comment = 'inspect the contents of database pages at a low level'
-default_version = '1.3'
+default_version = '1.4'
module_pathname = '$libdir/pageinspect'
relocatable = true
Re: pageinspect patch, for showing tuple data
В письме от 11 сентября 2015 15:12:04 пользователь Michael Paquier написал:
Ok.Let's come to the final decision with tuple_data_parse, and i will add
this switch there and to pure sql heap_page_item_attrsFine for me.
Here is final version with documentation.
Hope it will be the last one. :-)
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
pageinspect_show_tuple_data_v5.difftext/x-patch; charset=UTF-8; name=pageinspect_show_tuple_data_v5.diffDownload
diff --git a/contrib/pageinspect/Makefile b/contrib/pageinspect/Makefile
index aec5258..e4bc1af 100644
--- a/contrib/pageinspect/Makefile
+++ b/contrib/pageinspect/Makefile
@@ -5,7 +5,7 @@ OBJS = rawpage.o heapfuncs.o btreefuncs.o fsmfuncs.o \
brinfuncs.o ginfuncs.o $(WIN32RES)
EXTENSION = pageinspect
-DATA = pageinspect--1.3.sql pageinspect--1.2--1.3.sql \
+DATA = pageinspect--1.4.sql pageinspect--1.3--1.4.sql pageinspect--1.2--1.3.sql \
pageinspect--1.1--1.2.sql pageinspect--1.0--1.1.sql \
pageinspect--unpackaged--1.0.sql
PGFILEDESC = "pageinspect - functions to inspect contents of database pages"
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 8d1666c..d42ca48 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -29,7 +29,11 @@
#include "funcapi.h"
#include "utils/builtins.h"
#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/rel.h"
+#include "catalog/pg_type.h"
+Datum split_tuple_data(char *tuple_data, uint16 tuple_data_len, TupleDesc tuple_desc, uint16 t_infomask, uint16 t_infomask2, bits8 *t_bits, bool do_detoast, int error_level);
/*
* bits_to_text
@@ -122,8 +126,8 @@ heap_page_items(PG_FUNCTION_ARGS)
HeapTuple resultTuple;
Datum result;
ItemId id;
- Datum values[13];
- bool nulls[13];
+ Datum values[14];
+ bool nulls[14];
uint16 lp_offset;
uint16 lp_flags;
uint16 lp_len;
@@ -155,6 +159,8 @@ heap_page_items(PG_FUNCTION_ARGS)
{
HeapTupleHeader tuphdr;
int bits_len;
+ bytea *tuple_data_bytea;
+ int tuple_data_len;
/* Extract information from the tuple header */
@@ -168,6 +174,13 @@ heap_page_items(PG_FUNCTION_ARGS)
values[9] = UInt32GetDatum(tuphdr->t_infomask);
values[10] = UInt8GetDatum(tuphdr->t_hoff);
+ /* Copy raw tuple data into bytea attribute */
+ tuple_data_len = lp_len - tuphdr->t_hoff;
+ tuple_data_bytea = (bytea *) palloc(tuple_data_len + VARHDRSZ);
+ SET_VARSIZE(tuple_data_bytea, tuple_data_len + VARHDRSZ);
+ memcpy(VARDATA(tuple_data_bytea), (char *) tuphdr + tuphdr->t_hoff, tuple_data_len);
+ values[13] = PointerGetDatum(tuple_data_bytea);
+ nulls[13] = false;
/*
* We already checked that the item is completely within the raw
* page passed to us, with the length given in the line pointer.
@@ -208,7 +221,7 @@ heap_page_items(PG_FUNCTION_ARGS)
*/
int i;
- for (i = 4; i <= 12; i++)
+ for (i = 4; i <= 13; i++)
nulls[i] = true;
}
@@ -223,3 +236,210 @@ heap_page_items(PG_FUNCTION_ARGS)
else
SRF_RETURN_DONE(fctx);
}
+
+PG_FUNCTION_INFO_V1(tuple_data_split);
+Datum
+tuple_data_split(PG_FUNCTION_ARGS)
+{
+ Oid rel_oid;
+ bytea *raw_data;
+ uint16 t_infomask;
+ uint16 t_infomask2;
+ text *t_bits_str;
+ RelationData *rel;
+ TupleDesc tuple_desc;
+ bool do_detoast = false;
+ int error_level = ERROR;
+
+ bits8 *t_bits = NULL;
+ Datum res;
+
+ rel_oid = PG_GETARG_OID(0);
+ raw_data = PG_GETARG_BYTEA_P(1);
+ t_infomask = PG_GETARG_INT16(2);
+ t_infomask2 = PG_GETARG_INT16(3);
+ t_bits_str = PG_ARGISNULL(4) ? NULL : PG_GETARG_TEXT_P(4);
+ if (PG_NARGS()>=6) do_detoast = PG_GETARG_BOOL(5);
+ if (PG_NARGS()>=7) error_level = PG_GETARG_BOOL(6)?WARNING:ERROR;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ (errmsg("must be superuser to use raw page functions"))));
+
+ /*
+ * Here we converting t_bits string back to the bits8 array,
+ * as it really is in the tuple header
+ */
+ if (t_infomask & HEAP_HASNULL)
+ {
+ int bits_str_len;
+ int bits_len;
+ char * p;
+ int off;
+ char byte = 0;
+ bits_len = (t_infomask2 & HEAP_NATTS_MASK) / 8 + 1;
+ if (! t_bits_str)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits argument is NULL, though we expect it to be NOT NULL and %i character long", bits_len * 8)));
+ }
+ bits_str_len = VARSIZE(t_bits_str) - VARHDRSZ;
+ if (bits_str_len % 8)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits argument length should be multiple of eight")));
+ }
+ if (bits_len * 8 != bits_str_len)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("wrong t_bits argument length. Expected %i, actual is %i", bits_len * 8, bits_str_len)));
+ }
+ t_bits = palloc(bits_len + 1);
+
+ p = (char *) t_bits_str + VARHDRSZ;
+ off = 0;
+ while (off<bits_str_len)
+ {
+ if (!(off % 8))
+ {
+ byte = 0;
+ }
+ if (( p[off] == '0') || (p[off] == '1'))
+ {
+ byte = byte | ( (p[off]-'0')<<off % 8);
+ } else
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("illegal character '%c' in t_bits string", p[off])));
+ }
+ if (off % 8 == 7)
+ {
+ t_bits[off / 8] = byte;
+ }
+ off++;
+ }
+ } else
+ {
+ if (t_bits_str)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits string is expected to be NULL, but instead it is %i bytes length", VARSIZE(t_bits_str) - VARHDRSZ)));
+ }
+ }
+ rel = relation_open(rel_oid, NoLock);
+ tuple_desc = CreateTupleDescCopyConstr(rel->rd_att);
+ relation_close(rel, NoLock);
+ res = split_tuple_data((char *) raw_data + VARHDRSZ, VARSIZE(raw_data) - VARHDRSZ, tuple_desc, t_infomask, t_infomask2, t_bits, do_detoast, error_level);
+ PG_RETURN_ARRAYTYPE_P(res);
+}
+
+
+Datum
+split_tuple_data(char *tuple_data, uint16 tuple_data_len, TupleDesc tuple_desc, uint16 t_infomask, uint16 t_infomask2, bits8 *t_bits, bool do_detoast, int error_level)
+{
+ ArrayBuildState *raw_attrs;
+ int nattrs;
+ int i;
+ int off;
+
+ /*
+ * Here we reimplement the basic functionality of nocachegetattr from
+ * backend/access/common/heaptuple.c whitch is basically used for fetching
+ * attributes from tuple when it is not cached. We can not use nocachegetattr here
+ * directly, because we should ignore all cache optimisations and other stuff
+ * just get binary data as it is.
+ */
+
+ raw_attrs = initArrayResult(BYTEAOID,CurrentMemoryContext,0);
+ off = 0;
+ nattrs = tuple_desc->natts;
+ if (error_level &&
+ nattrs < (t_infomask2 & HEAP_NATTS_MASK))
+ {
+ ereport(error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: number of attributes in tuple header is greater than number of attributes in tuple descripor")));
+ }
+
+ for(i=0; i < nattrs; i++)
+ {
+ Form_pg_attribute attr;
+ Datum raw_attr = PointerGetDatum(NULL);
+ bool is_null;
+
+ attr = tuple_desc->attrs[i];
+ is_null = (t_infomask & HEAP_HASNULL) && att_isnull(i, t_bits);
+ /*
+ * Tuple header can specify less attributes then tuple descriptor
+ * as ALTER TABLE ADD COLUMN without DEFAULT keyword does not
+ * actualy change tuples in pages, so attributes with numbers greater
+ * than t_infomask2 & HEAP_NATTS_MASK should be treated as NULL
+ */
+ if (i >= (t_infomask2 & HEAP_NATTS_MASK) )
+ is_null = true;
+ if (!is_null)
+ {
+ int len;
+ bytea * attr_data;
+ if (attr->attlen == -1)
+ {
+ off = att_align_pointer(off, tuple_desc->attrs[i]->attalign, -1, tuple_data + off);
+ /*
+ * As VARSIZE_ANY throws an exeption if it can't properly detect
+ * type of external storage in macros VARTAG_SIZE, so we repeat
+ * this check here to preform nicer error handling
+ */
+ if ( error_level &&
+ VARATT_IS_1B_E(tuple_data + off) &&
+ VARTAG_EXTERNAL(tuple_data + off) != VARTAG_INDIRECT &&
+ VARTAG_EXTERNAL(tuple_data + off) != VARTAG_ONDISK)
+ {
+ ereport(error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: First byte of varlen attr seems to be corrupted")));
+ }
+ len = VARSIZE_ANY(tuple_data + off);
+ }
+ else
+ {
+ off = att_align_nominal(off, tuple_desc->attrs[i]->attalign);
+ len = attr->attlen;
+ }
+ if ( error_level &&
+ tuple_data_len < off + len)
+ {
+ ereport(error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: Iterating over tuple data reached out of actual tuple size")));
+ }
+ attr_data = (bytea *) palloc(len + VARHDRSZ);
+ SET_VARSIZE(attr_data, len + VARHDRSZ);
+ memcpy(VARDATA(attr_data), tuple_data + off, len);
+ raw_attr = PointerGetDatum(attr_data);
+
+ if ( attr->attlen == -1 && do_detoast)
+ {
+ Datum raw_attr_copy;
+ raw_attr_copy = PointerGetDatum(PG_DETOAST_DATUM_COPY(tuple_data + off));
+ pfree(attr_data);
+ raw_attr = raw_attr_copy;
+ }
+ off = att_addlength_pointer(off, tuple_desc->attrs[i]->attlen, tuple_data + off);
+ }
+ raw_attrs = accumArrayResult(raw_attrs, raw_attr, is_null, BYTEAOID, CurrentMemoryContext);
+ }
+ if (error_level &&
+ tuple_data_len != off)
+ {
+ ereport(error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: Iterating over tuple data did not actualy reach tuple end")));
+ }
+ return makeArrayResult(raw_attrs, CurrentMemoryContext);
+}
diff --git a/contrib/pageinspect/pageinspect--1.3--1.4.sql b/contrib/pageinspect/pageinspect--1.3--1.4.sql
new file mode 100644
index 0000000..5a16261
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.3--1.4.sql
@@ -0,0 +1,87 @@
+/* contrib/pageinspect/pageinspect--1.3--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via ALTER EXTENSION
+\echo Use "ALTER EXTENSION pageinspect UPDATE TO '1.4'" to load this file. \quit
+
+--
+-- heap_page_items()
+--
+DROP FUNCTION heap_page_items(bytea);
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(t_oid rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(t_oid rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(t_oid rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool, warning_mode bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN rel_oid regclass, IN do_detoast bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT lp, lp_off, lp_flags, lp_len, t_xmin, t_xmax, t_field3, t_ctid, t_infomask2, t_infomask, t_hoff, t_bits, t_oid, tuple_data_split(rel_oid, t_data, t_infomask, t_infomask2, t_bits, do_detoast) as t_attrs from heap_page_items(page)'
+LANGUAGE SQL;
+
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT * from heap_page_item_attrs(page, heap_oid, false)'
+LANGUAGE SQL;
diff --git a/contrib/pageinspect/pageinspect--1.3.sql b/contrib/pageinspect/pageinspect--1.3.sql
deleted file mode 100644
index a99e058..0000000
--- a/contrib/pageinspect/pageinspect--1.3.sql
+++ /dev/null
@@ -1,189 +0,0 @@
-/* contrib/pageinspect/pageinspect--1.3.sql */
-
--- complain if script is sourced in psql, rather than via CREATE EXTENSION
-\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
-
---
--- get_raw_page()
---
-CREATE FUNCTION get_raw_page(text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page'
-LANGUAGE C STRICT;
-
-CREATE FUNCTION get_raw_page(text, text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page_fork'
-LANGUAGE C STRICT;
-
---
--- page_header()
---
-CREATE FUNCTION page_header(IN page bytea,
- OUT lsn pg_lsn,
- OUT checksum smallint,
- OUT flags smallint,
- OUT lower smallint,
- OUT upper smallint,
- OUT special smallint,
- OUT pagesize smallint,
- OUT version smallint,
- OUT prune_xid xid)
-AS 'MODULE_PATHNAME', 'page_header'
-LANGUAGE C STRICT;
-
---
--- heap_page_items()
---
-CREATE FUNCTION heap_page_items(IN page bytea,
- OUT lp smallint,
- OUT lp_off smallint,
- OUT lp_flags smallint,
- OUT lp_len smallint,
- OUT t_xmin xid,
- OUT t_xmax xid,
- OUT t_field3 int4,
- OUT t_ctid tid,
- OUT t_infomask2 integer,
- OUT t_infomask integer,
- OUT t_hoff smallint,
- OUT t_bits text,
- OUT t_oid oid)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'heap_page_items'
-LANGUAGE C STRICT;
-
---
--- bt_metap()
---
-CREATE FUNCTION bt_metap(IN relname text,
- OUT magic int4,
- OUT version int4,
- OUT root int4,
- OUT level int4,
- OUT fastroot int4,
- OUT fastlevel int4)
-AS 'MODULE_PATHNAME', 'bt_metap'
-LANGUAGE C STRICT;
-
---
--- bt_page_stats()
---
-CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
- OUT blkno int4,
- OUT type "char",
- OUT live_items int4,
- OUT dead_items int4,
- OUT avg_item_size int4,
- OUT page_size int4,
- OUT free_size int4,
- OUT btpo_prev int4,
- OUT btpo_next int4,
- OUT btpo int4,
- OUT btpo_flags int4)
-AS 'MODULE_PATHNAME', 'bt_page_stats'
-LANGUAGE C STRICT;
-
---
--- bt_page_items()
---
-CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
- OUT itemoffset smallint,
- OUT ctid tid,
- OUT itemlen smallint,
- OUT nulls bool,
- OUT vars bool,
- OUT data text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'bt_page_items'
-LANGUAGE C STRICT;
-
---
--- brin_page_type()
---
-CREATE FUNCTION brin_page_type(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'brin_page_type'
-LANGUAGE C STRICT;
-
---
--- brin_metapage_info()
---
-CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
- OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
-AS 'MODULE_PATHNAME', 'brin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- brin_revmap_data()
---
-CREATE FUNCTION brin_revmap_data(IN page bytea,
- OUT pages tid)
-RETURNS SETOF tid
-AS 'MODULE_PATHNAME', 'brin_revmap_data'
-LANGUAGE C STRICT;
-
---
--- brin_page_items()
---
-CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
- OUT itemoffset int,
- OUT blknum int,
- OUT attnum int,
- OUT allnulls bool,
- OUT hasnulls bool,
- OUT placeholder bool,
- OUT value text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'brin_page_items'
-LANGUAGE C STRICT;
-
---
--- fsm_page_contents()
---
-CREATE FUNCTION fsm_page_contents(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'fsm_page_contents'
-LANGUAGE C STRICT;
-
---
--- GIN functions
---
-
---
--- gin_metapage_info()
---
-CREATE FUNCTION gin_metapage_info(IN page bytea,
- OUT pending_head bigint,
- OUT pending_tail bigint,
- OUT tail_free_size int4,
- OUT n_pending_pages bigint,
- OUT n_pending_tuples bigint,
- OUT n_total_pages bigint,
- OUT n_entry_pages bigint,
- OUT n_data_pages bigint,
- OUT n_entries bigint,
- OUT version int4)
-AS 'MODULE_PATHNAME', 'gin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- gin_page_opaque_info()
---
-CREATE FUNCTION gin_page_opaque_info(IN page bytea,
- OUT rightlink bigint,
- OUT maxoff int4,
- OUT flags text[])
-AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
-LANGUAGE C STRICT;
-
---
--- gin_leafpage_items()
---
-CREATE FUNCTION gin_leafpage_items(IN page bytea,
- OUT first_tid tid,
- OUT nbytes int2,
- OUT tids tid[])
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'gin_leafpage_items'
-LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect--1.4.sql b/contrib/pageinspect/pageinspect--1.4.sql
new file mode 100644
index 0000000..c09657b
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.4.sql
@@ -0,0 +1,251 @@
+/* contrib/pageinspect/pageinspect--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
+
+--
+-- get_raw_page()
+--
+CREATE FUNCTION get_raw_page(text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION get_raw_page(text, text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page_fork'
+LANGUAGE C STRICT;
+
+--
+-- page_header()
+--
+CREATE FUNCTION page_header(IN page bytea,
+ OUT lsn pg_lsn,
+ OUT checksum smallint,
+ OUT flags smallint,
+ OUT lower smallint,
+ OUT upper smallint,
+ OUT special smallint,
+ OUT pagesize smallint,
+ OUT version smallint,
+ OUT prune_xid xid)
+AS 'MODULE_PATHNAME', 'page_header'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_items()
+--
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea
+ )
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(rel_oid oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(rel_oid oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(rel_oid oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool, warning_mode bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN rel_oid regclass, IN do_detoast bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT lp, lp_off, lp_flags, lp_len, t_xmin, t_xmax, t_field3, t_ctid, t_infomask2, t_infomask, t_hoff, t_bits, t_oid, tuple_data_split(rel_oid, t_data, t_infomask, t_infomask2, t_bits, do_detoast) as t_attrs from heap_page_items(page)'
+LANGUAGE SQL;
+
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT * from heap_page_item_attrs(page, heap_oid, false)'
+LANGUAGE SQL;
+
+--
+-- bt_metap()
+--
+CREATE FUNCTION bt_metap(IN relname text,
+ OUT magic int4,
+ OUT version int4,
+ OUT root int4,
+ OUT level int4,
+ OUT fastroot int4,
+ OUT fastlevel int4)
+AS 'MODULE_PATHNAME', 'bt_metap'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_stats()
+--
+CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
+ OUT blkno int4,
+ OUT type "char",
+ OUT live_items int4,
+ OUT dead_items int4,
+ OUT avg_item_size int4,
+ OUT page_size int4,
+ OUT free_size int4,
+ OUT btpo_prev int4,
+ OUT btpo_next int4,
+ OUT btpo int4,
+ OUT btpo_flags int4)
+AS 'MODULE_PATHNAME', 'bt_page_stats'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_items()
+--
+CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
+ OUT itemoffset smallint,
+ OUT ctid tid,
+ OUT itemlen smallint,
+ OUT nulls bool,
+ OUT vars bool,
+ OUT data text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'bt_page_items'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_type()
+--
+CREATE FUNCTION brin_page_type(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'brin_page_type'
+LANGUAGE C STRICT;
+
+--
+-- brin_metapage_info()
+--
+CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
+ OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
+AS 'MODULE_PATHNAME', 'brin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- brin_revmap_data()
+--
+CREATE FUNCTION brin_revmap_data(IN page bytea,
+ OUT pages tid)
+RETURNS SETOF tid
+AS 'MODULE_PATHNAME', 'brin_revmap_data'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_items()
+--
+CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
+ OUT itemoffset int,
+ OUT blknum int,
+ OUT attnum int,
+ OUT allnulls bool,
+ OUT hasnulls bool,
+ OUT placeholder bool,
+ OUT value text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'brin_page_items'
+LANGUAGE C STRICT;
+
+--
+-- fsm_page_contents()
+--
+CREATE FUNCTION fsm_page_contents(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'fsm_page_contents'
+LANGUAGE C STRICT;
+
+--
+-- GIN functions
+--
+
+--
+-- gin_metapage_info()
+--
+CREATE FUNCTION gin_metapage_info(IN page bytea,
+ OUT pending_head bigint,
+ OUT pending_tail bigint,
+ OUT tail_free_size int4,
+ OUT n_pending_pages bigint,
+ OUT n_pending_tuples bigint,
+ OUT n_total_pages bigint,
+ OUT n_entry_pages bigint,
+ OUT n_data_pages bigint,
+ OUT n_entries bigint,
+ OUT version int4)
+AS 'MODULE_PATHNAME', 'gin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_page_opaque_info()
+--
+CREATE FUNCTION gin_page_opaque_info(IN page bytea,
+ OUT rightlink bigint,
+ OUT maxoff int4,
+ OUT flags text[])
+AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_leafpage_items()
+--
+CREATE FUNCTION gin_leafpage_items(IN page bytea,
+ OUT first_tid tid,
+ OUT nbytes int2,
+ OUT tids tid[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'gin_leafpage_items'
+LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect.control b/contrib/pageinspect/pageinspect.control
index a9dab33..68c7d61 100644
--- a/contrib/pageinspect/pageinspect.control
+++ b/contrib/pageinspect/pageinspect.control
@@ -1,5 +1,5 @@
# pageinspect extension
comment = 'inspect the contents of database pages at a low level'
-default_version = '1.3'
+default_version = '1.4'
module_pathname = '$libdir/pageinspect'
relocatable = true
diff --git a/doc/src/sgml/pageinspect.sgml b/doc/src/sgml/pageinspect.sgml
index b95cc81..f877390 100644
--- a/doc/src/sgml/pageinspect.sgml
+++ b/doc/src/sgml/pageinspect.sgml
@@ -93,19 +93,355 @@ test=# SELECT * FROM page_header(get_raw_page('pg_class', 0));
<listitem>
<para>
<function>heap_page_items</function> shows all line pointers on a heap
- page. For those line pointers that are in use, tuple headers are also
- shown. All tuples are shown, whether or not the tuples were visible to
- an MVCC snapshot at the time the raw page was copied.
+ page. For those line pointers that are in use,
+ <function>heap_page_items</function> also shows tuple headers and raw
+ tuple data. All tuples are shown, whether or not the tuples were visible
+ to an MVCC snapshot at the time the raw page was copied.
</para>
<para>
A heap page image obtained with <function>get_raw_page</function> should
be passed as argument. For example:
<screen>
-test=# SELECT * FROM heap_page_items(get_raw_page('pg_class', 0));
+
+test=# select * from heap_page_items(get_raw_page('pg_range', 0));
+ lp | lp_off | lp_flags | lp_len | t_xmin | t_xmax | t_field3 | t_ctid | t_infomask2 | t_infomask | t_hoff | t_bits | t_oid | t_data
+----+--------+----------+--------+--------+--------+----------+--------+-------------+------------+--------+--------+-------+----------------------------------------------------
+ 1 | 8144 | 1 | 48 | 1 | 0 | 0 | (0,1) | 6 | 2304 | 24 | | | \x400f00001700000000000000ba0700004a0f0000520f0000
+ 2 | 8096 | 1 | 48 | 1 | 0 | 0 | (0,2) | 6 | 2304 | 24 | | | \x420f0000a406000000000000350c000000000000540f0000
+ 3 | 8048 | 1 | 48 | 1 | 0 | 0 | (0,3) | 6 | 2304 | 24 | | | \x440f00005a04000000000000380c000000000000590f0000
+ 4 | 8000 | 1 | 48 | 1 | 0 | 0 | (0,4) | 6 | 2304 | 24 | | | \x460f0000a004000000000000370c0000000000005a0f0000
+ 5 | 7952 | 1 | 48 | 1 | 0 | 0 | (0,5) | 6 | 2304 | 24 | | | \x480f00003a04000000000000320c00004b0f0000550f0000
+ 6 | 7904 | 1 | 48 | 1 | 0 | 0 | (0,6) | 6 | 2304 | 24 | | | \x560f00001400000000000000340c0000580f0000530f0000
+(6 rows)
+
+</screen>
+ The output columns are:
+
+ <informaltable>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Column</entry>
+ <entry>Type</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry><structfield>lp</structfield></entry>
+ <entry><type>smallint</type></entry>
+ <entry>line number</entry>
+ </row>
+
+ <row>
+ <entry><structfield>lp_off</structfield></entry>
+ <entry><type>smallint</type></entry>
+ <entry>offset to tuple (from start of page)</entry>
+ </row>
+
+ <row>
+ <entry><structfield>lp_flags</structfield></entry>
+ <entry><type>smallint</type></entry>
+ <entry>state of item pointer:
+<screen>
+0 - unused (should always have lp_len=0);
+1 - used (should always have lp_len>0);
+2 - HOT redirect (should have lp_len=0);
+3 - dead, may or may not have storage.
+</screen>
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>lp_len</structfield></entry>
+ <entry><type>smallint</type></entry>
+ <entry>length of tuple</entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_xmin</structfield></entry>
+ <entry><type>xid</type></entry>
+ <entry>transaction ID when this tuple was created. See
+ <xref linkend="ddl-system-columns">
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_xmax</structfield></entry>
+ <entry><type>xid</type></entry>
+ <entry>transaction ID when tuple was deleted or replaced with newer
+ version. See <xref linkend="ddl-system-columns">
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_field3</structfield></entry>
+ <entry><type>int4</type></entry>
+ <entry><filename>t_field3</filename> is a tricky a attribute, that can
+ be treated as Cmin, Cmax (command numbers inside transaction),
+ combination of Cmin and Cmax, or as Xvac - that is used by
+ oldstyle VACUUM. See <xref linkend="storage-page-layout">
+ and <xref linkend="ddl-system-columns"> for more
+ info
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_ctid</structfield></entry>
+ <entry><type>tid</type></entry>
+ <entry>location of the tuple in the heap (page number, row number). See
+ <xref linkend="ddl-system-columns">
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_infomask2</structfield></entry>
+ <entry><type>integer</type></entry>
+ <entry>stores number of attributes (11 bits of the word). The rest are used for flag bits:
+<screen>
+0x2000 - tuple was updated and key cols modified, or tuple deleted
+0x4000 - tuple was HOT-updated
+0x8000 - this is heap-only tuple
+0xE000 - visibility-related bits (so called "hint bits")
+</screen>
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_infomask</structfield></entry>
+ <entry><type>integer</type></entry>
+ <entry>consists of following flag bits:
+<screen>
+0x0001 - has null attribute(s)
+0x0002 - has variable-width attribute(s)
+0x0004 - has external stored attribute(s)
+0x0008 - has an object-id field
+0x0010 - xmax is a key-shared locker
+0x0020 - t_cid is a combo cid
+0x0040 - xmax is exclusive locker
+0x0080 - xmax, if valid, is only a locker
</screen>
- See <filename>src/include/storage/itemid.h</> and
- <filename>src/include/access/htup_details.h</> for explanations of the fields
- returned.
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_hoff</structfield></entry>
+ <entry><type>smallint</type></entry>
+ <entry>offset to user data: size of header tuple header, plus alingn
+ padding
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_bits</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry>bitmap of nulls converted into human readable text representation
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_oid</structfield></entry>
+ <entry><type>oid</type></entry>
+ <entry>object identifier. Now actually is used only in system tables.
+ See <xref linkend="ddl-system-columns">
+ </entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </informaltable>
+
+ For better understanding of these fields see
+ <xref linkend="storage-page-layout">, <xref linkend="ddl-system-columns">
+ and comments from source code <filename>src/include/storage/itemid.h</> and
+ <filename>src/include/access/htup_details.h</>.
+
+ </para>
+ </listitem>
+ </varlistentry>
+
+
+ <varlistentry>
+ <term>
+ <function>tuple_data_split(rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text) returs bytea[]</function>
+ <indexterm>
+ <primary>tuple_data_split</primary>
+ </indexterm>
+ </term>
+ <term>
+ <function>tuple_data_split(rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool) returs bytea[]</function>
+ </term>
+ <term>
+ <function>tuple_data_split(rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool, warning_mode bool) returs bytea[]</function>
+ </term>
+ <listitem>
+ <para>
+ <function>tuple_data_split</function> splits tuple data into attributes in
+ the same way as <function>nocachegetattr</function> function do it in
+ postgres internals. Returns array of bytea.
+<screen>
+test=# select tuple_data_split('pg_range'::regclass, '\x400f00001700000000000000ba0700004a0f0000520f0000'::bytea, 2304, 6, null);
+ tuple_data_split
+---------------------------------------------------------------------------------------
+ {"\\x400f0000","\\x17000000","\\x00000000","\\xba070000","\\x4a0f0000","\\x520f0000"}
+(1 row)
+</screen>
+ </para>
+
+ <para>
+ <function>tuple_data_split</function> takes following arguments:
+
+ <informaltable>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Argument</entry>
+ <entry>Type</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry><structfield>rel_oid</structfield></entry>
+ <entry><type>oid</type></entry>
+ <entry>OID of the relation, of the tuple we want to split</entry>
+ </row>
+
+ <row>
+ <entry><structfield>tuple_data</structfield></entry>
+ <entry><type>bytea</type></entry>
+ <entry>tuple raw data to split
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_infomask</structfield></entry>
+ <entry><type>integer</type></entry>
+ <entry><function>t_infomask</function> as returned by
+ <function>heap_page_items</function>
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_infomask2</structfield></entry>
+ <entry><type>integer</type></entry>
+ <entry><function>t_infomask2</function> as returned by
+ <function>heap_page_items</function>
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_bitd</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry>bitmsp of nulls as returned by
+ <function>heap_page_items</function>
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>do_detoast</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry>optional attribute, if <function>do_detoast</function> is set to
+ true, <function>tuple_data_split</function> will take attribute value
+ from TOAST and uncompress it if it was TOASTed or compressed. See
+ <xref linkend="storage-toast">
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>warning_mode</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry>optional attribute, if <function>warning_mode</function> is set to
+ true, <function>tuple_data_split</function> will not raise ERROR if
+ data consistency check fails, just send WARNING. This may lead to
+ backend crash, use it with care.
+ </entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </informaltable>
+ </para>
+ <para>
+ In most cases you will not need <function>tuple_data_split</function>
+ itself, consider using <function>heap_page_item_attrs</function> for
+ viewing page data with split attributes.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <function>heap_page_item_attrs(page bytea, rel_oid regclass) returns record</function>
+ <indexterm>
+ <primary>heap_page_item_attrs</primary>
+ </indexterm>
+ </term>
+ <term>
+ <function>heap_page_item_attrs(page bytea, rel_oid regclass, do_detoast bool) returns record</function>
+ </term>
+
+ <listitem>
+ <para>
+ <function>heap_page_item_attrs</function> is actually a clone of
+ <function>heap_page_items</function>, with one difference:
+ <function>heap_page_item_attrs</function> returns array of raw values of
+ tuple attributes instead of one peace of raw tuple data. All other return
+ columns are same as in <function>heap_page_items</function>.
+<screen>
+test=# select * from heap_page_item_attrs(get_raw_page('pg_range', 0),'pg_range'::regclass);
+ lp | lp_off | lp_flags | lp_len | t_xmin | t_xmax | t_field3 | t_ctid | t_infomask2 | t_infomask | t_hoff | t_bits | t_oid | t_attrs
+----+--------+----------+--------+--------+--------+----------+--------+-------------+------------+--------+--------+-------+--------------------------------------------------------------------------------------
+ 1 | 8144 | 1 | 48 | 1 | 0 | 0 | (0,1) | 6 | 2304 | 24 | | | {"\\x400f0000","\\x17000000","\\x00000000","\\xba070000","\\x4a0f0000","\\x520f0000"}
+ 2 | 8096 | 1 | 48 | 1 | 0 | 0 | (0,2) | 6 | 2304 | 24 | | | {"\\x420f0000","\\xa4060000","\\x00000000","\\x350c0000","\\x00000000","\\x540f0000"}
+ 3 | 8048 | 1 | 48 | 1 | 0 | 0 | (0,3) | 6 | 2304 | 24 | | | {"\\x440f0000","\\x5a040000","\\x00000000","\\x380c0000","\\x00000000","\\x590f0000"}
+ 4 | 8000 | 1 | 48 | 1 | 0 | 0 | (0,4) | 6 | 2304 | 24 | | | {"\\x460f0000","\\xa0040000","\\x00000000","\\x370c0000","\\x00000000","\\x5a0f0000"}
+ 5 | 7952 | 1 | 48 | 1 | 0 | 0 | (0,5) | 6 | 2304 | 24 | | | {"\\x480f0000","\\x3a040000","\\x00000000","\\x320c0000","\\x4b0f0000","\\x550f0000"}
+ 6 | 7904 | 1 | 48 | 1 | 0 | 0 | (0,6) | 6 | 2304 | 24 | | | {"\\x560f0000","\\x14000000","\\x00000000","\\x340c0000","\\x580f0000","\\x530f0000"}
+(6 rows)
+</screen>
+ </para>
+ <para>
+ <function>heap_page_item_attrs</function> takes following argiments:
+ <informaltable>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Argument</entry>
+ <entry>Type</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry><structfield>page</structfield></entry>
+ <entry><type>bytea</type></entry>
+ <entry>page raw data, usually returned by <function>get_raw_page</function></entry>
+ </row>
+
+ <row>
+ <entry><structfield>rel_oid</structfield></entry>
+ <entry><type>oid</type></entry>
+ <entry>OID of the relation, whose page we want to parse</entry>
+ </row>
+
+ <row>
+ <entry><structfield>do_detoast</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry>optional attribute, if <function>do_detoast</function> is set to
+ true, <function>heap_page_item_attrs</function> will not only split
+ tuple data into attributes, but also try to deTOAST and uncompress
+ attribute value if it was TOASTed or compressed.
+ See <xref linkend="storage-toast">
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </informaltable>
+
</para>
</listitem>
</varlistentry>
Re: pageinspect patch, for showing tuple data
On Fri, Sep 25, 2015 at 8:30 PM, Nikolay Shaplov wrote:
Here is final version with documentation.
Thanks! I just had a short look at it:
- I am not convinced that it is worth declaring 3 versions of tuple_data_split.
- The patch does not respect the project code style, particularly
one-line "if condition {foo;}" are not adapted, code line is limited
to 80 characters, etc. The list is basically here:
http://www.postgresql.org/docs/current/static/source.html
- Be aware of typos: s/whitch/which is one.
+ <entry><structfield>t_infomask2</structfield></entry>
+ <entry><type>integer</type></entry>
+ <entry>stores number of attributes (11 bits of the word). The
rest are used for flag bits:
+<screen>
+0x2000 - tuple was updated and key cols modified, or tuple deleted
+0x4000 - tuple was HOT-updated
+0x8000 - this is heap-only tuple
+0xE000 - visibility-related bits (so called "hint bits")
+</screen>
This large chunk of documentation is a duplicate of storage.sgml. If
that's really necessary, it looks adapted to me to have more detailed
comments at code level directly in heapfuncs.c.The example of tuple_data_split in the docs would be more interesting
if embedded with a call to heap_page_items.
Hope it will be the last one. :-)
Unfortunately not :) But this is definitely going in the right
direction thinking that this code is mainly targeted for educational
purposes.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
В письме от 25 сентября 2015 20:59:29 пользователь Michael Paquier написал:
Here is final version with documentation.
Thanks! I just had a short look at it:
- I am not convinced that it is worth declaring 3 versions of
tuple_data_split.
How which of them should we leave?
- The patch does not respect the project code style,
particularly one-line "if condition {foo;}" are not adapted, code line is
limited
to 80 characters, etc. The list is basically here:
http://www.postgresql.org/docs/current/static/source.html
I did my best. Results are attached.
- Be aware of typos: s/whitch/which is one.
I've run spellchecker on all comments. Hope that I removed most of the
mistakes. But as I am not native speaker, I will not be able to eliminate them
all. I will need help here.
+ <entry><structfield>t_infomask2</structfield></entry> + <entry><type>integer</type></entry> + <entry>stores number of attributes (11 bits of the word). The rest are used for flag bits: +<screen> +0x2000 - tuple was updated and key cols modified, or tuple deleted +0x4000 - tuple was HOT-updated +0x8000 - this is heap-only tuple +0xE000 - visibility-related bits (so called "hint bits") +</screen> This large chunk of documentation is a duplicate of storage.sgml. If that's really necessary, it looks adapted to me to have more detailed comments at code level directly in heapfuncs.c.
Hm... There is no explanation of t_infomask/t_infomask2 bits in storage.sgml.
If there is no source of information other then source code, then the
documentation is not good.
If there were information about t_infomask/t_infomask2 in storage.sgml, then I
would add "See storage.sgml for more info" into pageinspect doc, and thats
all. But since there is no such information there, I think that the best
thing is to quote comments from source code there, so you can get all
information from documentation, not looking for it in the code.
So I would consider two options: Either move t_infomask/t_infomask2 details
into storage.sgml or leave as it is.
I am lazy, and does not feel confidence about touching main documentation, so I
would prefer to leave as it is.
The example of tuple_data_split in the docs would be more interesting
if embedded with a call to heap_page_items.
This example would be almost not readable. May be I should add one more praise
explaining where did we take arguments for tuple_data_split
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
pageinspect_show_tuple_data_v6c.difftext/x-patch; charset=UTF-8; name=pageinspect_show_tuple_data_v6c.diffDownload
diff --git a/contrib/pageinspect/Makefile b/contrib/pageinspect/Makefile
index aec5258..e4bc1af 100644
--- a/contrib/pageinspect/Makefile
+++ b/contrib/pageinspect/Makefile
@@ -5,7 +5,7 @@ OBJS = rawpage.o heapfuncs.o btreefuncs.o fsmfuncs.o \
brinfuncs.o ginfuncs.o $(WIN32RES)
EXTENSION = pageinspect
-DATA = pageinspect--1.3.sql pageinspect--1.2--1.3.sql \
+DATA = pageinspect--1.4.sql pageinspect--1.3--1.4.sql pageinspect--1.2--1.3.sql \
pageinspect--1.1--1.2.sql pageinspect--1.0--1.1.sql \
pageinspect--unpackaged--1.0.sql
PGFILEDESC = "pageinspect - functions to inspect contents of database pages"
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 8d1666c..52a0663 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -29,7 +29,14 @@
#include "funcapi.h"
#include "utils/builtins.h"
#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/rel.h"
+#include "catalog/pg_type.h"
+Datum split_tuple_data(char *tuple_data, uint16 tuple_data_len,
+ TupleDesc tuple_desc, uint16 t_infomask,
+ uint16 t_infomask2, bits8 *t_bits, bool
+ do_detoast, int error_level);
/*
* bits_to_text
@@ -91,7 +98,7 @@ heap_page_items(PG_FUNCTION_ARGS)
if (raw_page_size < SizeOfPageHeaderData)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("input page too small (%d bytes)", raw_page_size)));
+ errmsg("input page too small (%d bytes)", raw_page_size)));
fctx = SRF_FIRSTCALL_INIT();
mctx = MemoryContextSwitchTo(fctx->multi_call_memory_ctx);
@@ -122,8 +129,8 @@ heap_page_items(PG_FUNCTION_ARGS)
HeapTuple resultTuple;
Datum result;
ItemId id;
- Datum values[13];
- bool nulls[13];
+ Datum values[14];
+ bool nulls[14];
uint16 lp_offset;
uint16 lp_flags;
uint16 lp_len;
@@ -155,6 +162,8 @@ heap_page_items(PG_FUNCTION_ARGS)
{
HeapTupleHeader tuphdr;
int bits_len;
+ bytea *tuple_data_bytea;
+ int tuple_data_len;
/* Extract information from the tuple header */
@@ -168,6 +177,14 @@ heap_page_items(PG_FUNCTION_ARGS)
values[9] = UInt32GetDatum(tuphdr->t_infomask);
values[10] = UInt8GetDatum(tuphdr->t_hoff);
+ /* Copy raw tuple data into bytea attribute */
+ tuple_data_len = lp_len - tuphdr->t_hoff;
+ tuple_data_bytea = (bytea *) palloc(tuple_data_len + VARHDRSZ);
+ SET_VARSIZE(tuple_data_bytea, tuple_data_len + VARHDRSZ);
+ memcpy(VARDATA(tuple_data_bytea), (char *) tuphdr + tuphdr->t_hoff,
+ tuple_data_len);
+ values[13] = PointerGetDatum(tuple_data_bytea);
+ nulls[13] = false;
/*
* We already checked that the item is completely within the raw
* page passed to us, with the length given in the line pointer.
@@ -208,7 +225,7 @@ heap_page_items(PG_FUNCTION_ARGS)
*/
int i;
- for (i = 4; i <= 12; i++)
+ for (i = 4; i <= 13; i++)
nulls[i] = true;
}
@@ -223,3 +240,223 @@ heap_page_items(PG_FUNCTION_ARGS)
else
SRF_RETURN_DONE(fctx);
}
+
+PG_FUNCTION_INFO_V1(tuple_data_split);
+Datum
+tuple_data_split(PG_FUNCTION_ARGS)
+{
+ Oid rel_oid;
+ bytea *raw_data;
+ uint16 t_infomask;
+ uint16 t_infomask2;
+ text *t_bits_str;
+ RelationData *rel;
+ TupleDesc tuple_desc;
+ bool do_detoast = false;
+ int error_level = ERROR;
+
+ bits8 *t_bits = NULL;
+ Datum res;
+
+ rel_oid = PG_GETARG_OID(0);
+ raw_data = PG_GETARG_BYTEA_P(1);
+ t_infomask = PG_GETARG_INT16(2);
+ t_infomask2 = PG_GETARG_INT16(3);
+ t_bits_str = PG_ARGISNULL(4) ? NULL : PG_GETARG_TEXT_P(4);
+ if (PG_NARGS()>=6)
+ do_detoast = PG_GETARG_BOOL(5);
+ if (PG_NARGS()>=7)
+ error_level = PG_GETARG_BOOL(6) ? WARNING : ERROR;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ (errmsg("must be superuser to use raw page functions"))));
+
+ /*
+ * Here we converting t_bits string back to the bits8 array,
+ * as it really is in the tuple header
+ */
+ if (t_infomask & HEAP_HASNULL)
+ {
+ int bits_str_len;
+ int bits_len;
+ char *p;
+ int off;
+ char byte = 0;
+
+ bits_len = (t_infomask2 & HEAP_NATTS_MASK) / 8 + 1;
+ if (!t_bits_str)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits argument is NULL, though we expect it to be NOT NULL and %i character long",
+ bits_len * 8)));
+
+ bits_str_len = VARSIZE(t_bits_str) - VARHDRSZ;
+ if (bits_str_len % 8)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits argument length should be multiple of eight")));
+
+ if (bits_len * 8 != bits_str_len)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("wrong t_bits argument length. Expected %i, actual is %i",
+ bits_len * 8, bits_str_len)));
+
+ t_bits = palloc(bits_len + 1);
+ p = (char *) t_bits_str + VARHDRSZ;
+ off = 0;
+
+ while (off<bits_str_len)
+ {
+ if (!(off % 8))
+ byte = 0;
+
+ if (( p[off] == '0') || (p[off] == '1'))
+ byte = byte | ( (p[off]-'0')<<off % 8);
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("illegal character '%c' in t_bits string", p[off])));
+
+ if (off % 8 == 7)
+ t_bits[off / 8] = byte;
+
+ off++;
+ }
+ }
+ else
+ {
+ if (t_bits_str)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits string is expected to be NULL, but instead it is %i bytes length",
+ VARSIZE(t_bits_str) - VARHDRSZ)));
+ }
+
+ /* Getting tuple descriptor from relation OID */
+ rel = relation_open(rel_oid, NoLock);
+ tuple_desc = CreateTupleDescCopyConstr(rel->rd_att);
+ relation_close(rel, NoLock);
+
+ /* Splitting tuple data */
+ res = split_tuple_data((char *) raw_data + VARHDRSZ,
+ VARSIZE(raw_data) - VARHDRSZ, tuple_desc,
+ t_infomask, t_infomask2, t_bits,
+ do_detoast, error_level);
+
+ PG_RETURN_ARRAYTYPE_P(res);
+}
+
+
+Datum
+split_tuple_data(char *tuple_data, uint16 tuple_data_len, TupleDesc tuple_desc,
+ uint16 t_infomask, uint16 t_infomask2, bits8 *t_bits,
+ bool do_detoast, int error_level)
+{
+ ArrayBuildState *raw_attrs;
+ int nattrs;
+ int i;
+ int off;
+
+ /*
+ * Here we reimplement the basic functionality of nocachegetattr from
+ * backend/access/common/heaptuple.c witch is basically used for fetching
+ * attributes from tuple when it is not cached. We can not use
+ * nocachegetattr here directly, because we should ignore all cache
+ * optimizations and other stuff just get binary data as it is.
+ */
+
+ raw_attrs = initArrayResult(BYTEAOID,CurrentMemoryContext,0);
+ off = 0;
+ nattrs = tuple_desc->natts;
+
+ if (error_level && nattrs < (t_infomask2 & HEAP_NATTS_MASK))
+ ereport(error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: number of attributes in tuple header is greater than number of attributes in tuple descripor")));
+
+ for(i=0; i < nattrs; i++)
+ {
+ Form_pg_attribute attr;
+ Datum raw_attr = PointerGetDatum(NULL);
+ bool is_null;
+
+ attr = tuple_desc->attrs[i];
+ is_null = (t_infomask & HEAP_HASNULL) && att_isnull(i, t_bits);
+ /*
+ * Tuple header can specify less attributes then tuple descriptor
+ * as ALTER TABLE ADD COLUMN without DEFAULT keyword does not
+ * actually change tuples in pages, so attributes with numbers greater
+ * than t_infomask2 & HEAP_NATTS_MASK should be treated as NULL
+ */
+ if (i >= (t_infomask2 & HEAP_NATTS_MASK) )
+ is_null = true;
+
+ if (!is_null)
+ {
+ int len;
+ bytea *attr_data;
+
+ if (attr->attlen == -1)
+ {
+ off = att_align_pointer(off, tuple_desc->attrs[i]->attalign, -1,
+ tuple_data + off);
+ /*
+ * As VARSIZE_ANY throws an exception if it can't properly detect
+ * type of external storage in macros VARTAG_SIZE, so we repeat
+ * this check here to preform nicer error handling
+ */
+ if ( error_level &&
+ VARATT_IS_1B_E(tuple_data + off) &&
+ VARTAG_EXTERNAL(tuple_data + off) != VARTAG_INDIRECT &&
+ VARTAG_EXTERNAL(tuple_data + off) != VARTAG_ONDISK)
+ {
+ ereport(error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: First byte of varlen attr seems to be corrupted")));
+ }
+
+ len = VARSIZE_ANY(tuple_data + off);
+ }
+ else
+ {
+ off = att_align_nominal(off, tuple_desc->attrs[i]->attalign);
+ len = attr->attlen;
+ }
+
+ if (error_level && tuple_data_len < off + len)
+ ereport(error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: Iterating over tuple data reached out of actual tuple size")));
+
+ attr_data = (bytea *) palloc(len + VARHDRSZ);
+ SET_VARSIZE(attr_data, len + VARHDRSZ);
+ memcpy(VARDATA(attr_data), tuple_data + off, len);
+ raw_attr = PointerGetDatum(attr_data);
+
+ if ( attr->attlen == -1 && do_detoast)
+ {
+ Datum raw_attr_copy;
+ raw_attr_copy = PointerGetDatum(
+ PG_DETOAST_DATUM_COPY(tuple_data + off));
+ pfree(attr_data);
+ raw_attr = raw_attr_copy;
+ }
+
+ off = att_addlength_pointer(off, tuple_desc->attrs[i]->attlen,
+ tuple_data + off);
+ }
+
+ raw_attrs = accumArrayResult(raw_attrs, raw_attr, is_null, BYTEAOID,
+ CurrentMemoryContext);
+ }
+
+ if (error_level && tuple_data_len != off)
+ ereport(error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: Iterating over tuple data did not actualy reach tuple end")));
+
+ return makeArrayResult(raw_attrs, CurrentMemoryContext);
+}
diff --git a/contrib/pageinspect/pageinspect--1.3--1.4.sql b/contrib/pageinspect/pageinspect--1.3--1.4.sql
new file mode 100644
index 0000000..5a16261
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.3--1.4.sql
@@ -0,0 +1,87 @@
+/* contrib/pageinspect/pageinspect--1.3--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via ALTER EXTENSION
+\echo Use "ALTER EXTENSION pageinspect UPDATE TO '1.4'" to load this file. \quit
+
+--
+-- heap_page_items()
+--
+DROP FUNCTION heap_page_items(bytea);
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(t_oid rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(t_oid rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(t_oid rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool, warning_mode bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN rel_oid regclass, IN do_detoast bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT lp, lp_off, lp_flags, lp_len, t_xmin, t_xmax, t_field3, t_ctid, t_infomask2, t_infomask, t_hoff, t_bits, t_oid, tuple_data_split(rel_oid, t_data, t_infomask, t_infomask2, t_bits, do_detoast) as t_attrs from heap_page_items(page)'
+LANGUAGE SQL;
+
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT * from heap_page_item_attrs(page, heap_oid, false)'
+LANGUAGE SQL;
diff --git a/contrib/pageinspect/pageinspect--1.3.sql b/contrib/pageinspect/pageinspect--1.3.sql
deleted file mode 100644
index a99e058..0000000
--- a/contrib/pageinspect/pageinspect--1.3.sql
+++ /dev/null
@@ -1,189 +0,0 @@
-/* contrib/pageinspect/pageinspect--1.3.sql */
-
--- complain if script is sourced in psql, rather than via CREATE EXTENSION
-\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
-
---
--- get_raw_page()
---
-CREATE FUNCTION get_raw_page(text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page'
-LANGUAGE C STRICT;
-
-CREATE FUNCTION get_raw_page(text, text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page_fork'
-LANGUAGE C STRICT;
-
---
--- page_header()
---
-CREATE FUNCTION page_header(IN page bytea,
- OUT lsn pg_lsn,
- OUT checksum smallint,
- OUT flags smallint,
- OUT lower smallint,
- OUT upper smallint,
- OUT special smallint,
- OUT pagesize smallint,
- OUT version smallint,
- OUT prune_xid xid)
-AS 'MODULE_PATHNAME', 'page_header'
-LANGUAGE C STRICT;
-
---
--- heap_page_items()
---
-CREATE FUNCTION heap_page_items(IN page bytea,
- OUT lp smallint,
- OUT lp_off smallint,
- OUT lp_flags smallint,
- OUT lp_len smallint,
- OUT t_xmin xid,
- OUT t_xmax xid,
- OUT t_field3 int4,
- OUT t_ctid tid,
- OUT t_infomask2 integer,
- OUT t_infomask integer,
- OUT t_hoff smallint,
- OUT t_bits text,
- OUT t_oid oid)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'heap_page_items'
-LANGUAGE C STRICT;
-
---
--- bt_metap()
---
-CREATE FUNCTION bt_metap(IN relname text,
- OUT magic int4,
- OUT version int4,
- OUT root int4,
- OUT level int4,
- OUT fastroot int4,
- OUT fastlevel int4)
-AS 'MODULE_PATHNAME', 'bt_metap'
-LANGUAGE C STRICT;
-
---
--- bt_page_stats()
---
-CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
- OUT blkno int4,
- OUT type "char",
- OUT live_items int4,
- OUT dead_items int4,
- OUT avg_item_size int4,
- OUT page_size int4,
- OUT free_size int4,
- OUT btpo_prev int4,
- OUT btpo_next int4,
- OUT btpo int4,
- OUT btpo_flags int4)
-AS 'MODULE_PATHNAME', 'bt_page_stats'
-LANGUAGE C STRICT;
-
---
--- bt_page_items()
---
-CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
- OUT itemoffset smallint,
- OUT ctid tid,
- OUT itemlen smallint,
- OUT nulls bool,
- OUT vars bool,
- OUT data text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'bt_page_items'
-LANGUAGE C STRICT;
-
---
--- brin_page_type()
---
-CREATE FUNCTION brin_page_type(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'brin_page_type'
-LANGUAGE C STRICT;
-
---
--- brin_metapage_info()
---
-CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
- OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
-AS 'MODULE_PATHNAME', 'brin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- brin_revmap_data()
---
-CREATE FUNCTION brin_revmap_data(IN page bytea,
- OUT pages tid)
-RETURNS SETOF tid
-AS 'MODULE_PATHNAME', 'brin_revmap_data'
-LANGUAGE C STRICT;
-
---
--- brin_page_items()
---
-CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
- OUT itemoffset int,
- OUT blknum int,
- OUT attnum int,
- OUT allnulls bool,
- OUT hasnulls bool,
- OUT placeholder bool,
- OUT value text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'brin_page_items'
-LANGUAGE C STRICT;
-
---
--- fsm_page_contents()
---
-CREATE FUNCTION fsm_page_contents(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'fsm_page_contents'
-LANGUAGE C STRICT;
-
---
--- GIN functions
---
-
---
--- gin_metapage_info()
---
-CREATE FUNCTION gin_metapage_info(IN page bytea,
- OUT pending_head bigint,
- OUT pending_tail bigint,
- OUT tail_free_size int4,
- OUT n_pending_pages bigint,
- OUT n_pending_tuples bigint,
- OUT n_total_pages bigint,
- OUT n_entry_pages bigint,
- OUT n_data_pages bigint,
- OUT n_entries bigint,
- OUT version int4)
-AS 'MODULE_PATHNAME', 'gin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- gin_page_opaque_info()
---
-CREATE FUNCTION gin_page_opaque_info(IN page bytea,
- OUT rightlink bigint,
- OUT maxoff int4,
- OUT flags text[])
-AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
-LANGUAGE C STRICT;
-
---
--- gin_leafpage_items()
---
-CREATE FUNCTION gin_leafpage_items(IN page bytea,
- OUT first_tid tid,
- OUT nbytes int2,
- OUT tids tid[])
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'gin_leafpage_items'
-LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect--1.4.sql b/contrib/pageinspect/pageinspect--1.4.sql
new file mode 100644
index 0000000..c09657b
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.4.sql
@@ -0,0 +1,251 @@
+/* contrib/pageinspect/pageinspect--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
+
+--
+-- get_raw_page()
+--
+CREATE FUNCTION get_raw_page(text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION get_raw_page(text, text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page_fork'
+LANGUAGE C STRICT;
+
+--
+-- page_header()
+--
+CREATE FUNCTION page_header(IN page bytea,
+ OUT lsn pg_lsn,
+ OUT checksum smallint,
+ OUT flags smallint,
+ OUT lower smallint,
+ OUT upper smallint,
+ OUT special smallint,
+ OUT pagesize smallint,
+ OUT version smallint,
+ OUT prune_xid xid)
+AS 'MODULE_PATHNAME', 'page_header'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_items()
+--
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea
+ )
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(rel_oid oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(rel_oid oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(rel_oid oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool, warning_mode bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN rel_oid regclass, IN do_detoast bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT lp, lp_off, lp_flags, lp_len, t_xmin, t_xmax, t_field3, t_ctid, t_infomask2, t_infomask, t_hoff, t_bits, t_oid, tuple_data_split(rel_oid, t_data, t_infomask, t_infomask2, t_bits, do_detoast) as t_attrs from heap_page_items(page)'
+LANGUAGE SQL;
+
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT * from heap_page_item_attrs(page, heap_oid, false)'
+LANGUAGE SQL;
+
+--
+-- bt_metap()
+--
+CREATE FUNCTION bt_metap(IN relname text,
+ OUT magic int4,
+ OUT version int4,
+ OUT root int4,
+ OUT level int4,
+ OUT fastroot int4,
+ OUT fastlevel int4)
+AS 'MODULE_PATHNAME', 'bt_metap'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_stats()
+--
+CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
+ OUT blkno int4,
+ OUT type "char",
+ OUT live_items int4,
+ OUT dead_items int4,
+ OUT avg_item_size int4,
+ OUT page_size int4,
+ OUT free_size int4,
+ OUT btpo_prev int4,
+ OUT btpo_next int4,
+ OUT btpo int4,
+ OUT btpo_flags int4)
+AS 'MODULE_PATHNAME', 'bt_page_stats'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_items()
+--
+CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
+ OUT itemoffset smallint,
+ OUT ctid tid,
+ OUT itemlen smallint,
+ OUT nulls bool,
+ OUT vars bool,
+ OUT data text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'bt_page_items'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_type()
+--
+CREATE FUNCTION brin_page_type(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'brin_page_type'
+LANGUAGE C STRICT;
+
+--
+-- brin_metapage_info()
+--
+CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
+ OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
+AS 'MODULE_PATHNAME', 'brin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- brin_revmap_data()
+--
+CREATE FUNCTION brin_revmap_data(IN page bytea,
+ OUT pages tid)
+RETURNS SETOF tid
+AS 'MODULE_PATHNAME', 'brin_revmap_data'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_items()
+--
+CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
+ OUT itemoffset int,
+ OUT blknum int,
+ OUT attnum int,
+ OUT allnulls bool,
+ OUT hasnulls bool,
+ OUT placeholder bool,
+ OUT value text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'brin_page_items'
+LANGUAGE C STRICT;
+
+--
+-- fsm_page_contents()
+--
+CREATE FUNCTION fsm_page_contents(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'fsm_page_contents'
+LANGUAGE C STRICT;
+
+--
+-- GIN functions
+--
+
+--
+-- gin_metapage_info()
+--
+CREATE FUNCTION gin_metapage_info(IN page bytea,
+ OUT pending_head bigint,
+ OUT pending_tail bigint,
+ OUT tail_free_size int4,
+ OUT n_pending_pages bigint,
+ OUT n_pending_tuples bigint,
+ OUT n_total_pages bigint,
+ OUT n_entry_pages bigint,
+ OUT n_data_pages bigint,
+ OUT n_entries bigint,
+ OUT version int4)
+AS 'MODULE_PATHNAME', 'gin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_page_opaque_info()
+--
+CREATE FUNCTION gin_page_opaque_info(IN page bytea,
+ OUT rightlink bigint,
+ OUT maxoff int4,
+ OUT flags text[])
+AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_leafpage_items()
+--
+CREATE FUNCTION gin_leafpage_items(IN page bytea,
+ OUT first_tid tid,
+ OUT nbytes int2,
+ OUT tids tid[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'gin_leafpage_items'
+LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect.control b/contrib/pageinspect/pageinspect.control
index a9dab33..68c7d61 100644
--- a/contrib/pageinspect/pageinspect.control
+++ b/contrib/pageinspect/pageinspect.control
@@ -1,5 +1,5 @@
# pageinspect extension
comment = 'inspect the contents of database pages at a low level'
-default_version = '1.3'
+default_version = '1.4'
module_pathname = '$libdir/pageinspect'
relocatable = true
diff --git a/doc/src/sgml/pageinspect.sgml b/doc/src/sgml/pageinspect.sgml
index b95cc81..f877390 100644
--- a/doc/src/sgml/pageinspect.sgml
+++ b/doc/src/sgml/pageinspect.sgml
@@ -93,19 +93,355 @@ test=# SELECT * FROM page_header(get_raw_page('pg_class', 0));
<listitem>
<para>
<function>heap_page_items</function> shows all line pointers on a heap
- page. For those line pointers that are in use, tuple headers are also
- shown. All tuples are shown, whether or not the tuples were visible to
- an MVCC snapshot at the time the raw page was copied.
+ page. For those line pointers that are in use,
+ <function>heap_page_items</function> also shows tuple headers and raw
+ tuple data. All tuples are shown, whether or not the tuples were visible
+ to an MVCC snapshot at the time the raw page was copied.
</para>
<para>
A heap page image obtained with <function>get_raw_page</function> should
be passed as argument. For example:
<screen>
-test=# SELECT * FROM heap_page_items(get_raw_page('pg_class', 0));
+
+test=# select * from heap_page_items(get_raw_page('pg_range', 0));
+ lp | lp_off | lp_flags | lp_len | t_xmin | t_xmax | t_field3 | t_ctid | t_infomask2 | t_infomask | t_hoff | t_bits | t_oid | t_data
+----+--------+----------+--------+--------+--------+----------+--------+-------------+------------+--------+--------+-------+----------------------------------------------------
+ 1 | 8144 | 1 | 48 | 1 | 0 | 0 | (0,1) | 6 | 2304 | 24 | | | \x400f00001700000000000000ba0700004a0f0000520f0000
+ 2 | 8096 | 1 | 48 | 1 | 0 | 0 | (0,2) | 6 | 2304 | 24 | | | \x420f0000a406000000000000350c000000000000540f0000
+ 3 | 8048 | 1 | 48 | 1 | 0 | 0 | (0,3) | 6 | 2304 | 24 | | | \x440f00005a04000000000000380c000000000000590f0000
+ 4 | 8000 | 1 | 48 | 1 | 0 | 0 | (0,4) | 6 | 2304 | 24 | | | \x460f0000a004000000000000370c0000000000005a0f0000
+ 5 | 7952 | 1 | 48 | 1 | 0 | 0 | (0,5) | 6 | 2304 | 24 | | | \x480f00003a04000000000000320c00004b0f0000550f0000
+ 6 | 7904 | 1 | 48 | 1 | 0 | 0 | (0,6) | 6 | 2304 | 24 | | | \x560f00001400000000000000340c0000580f0000530f0000
+(6 rows)
+
+</screen>
+ The output columns are:
+
+ <informaltable>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Column</entry>
+ <entry>Type</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry><structfield>lp</structfield></entry>
+ <entry><type>smallint</type></entry>
+ <entry>line number</entry>
+ </row>
+
+ <row>
+ <entry><structfield>lp_off</structfield></entry>
+ <entry><type>smallint</type></entry>
+ <entry>offset to tuple (from start of page)</entry>
+ </row>
+
+ <row>
+ <entry><structfield>lp_flags</structfield></entry>
+ <entry><type>smallint</type></entry>
+ <entry>state of item pointer:
+<screen>
+0 - unused (should always have lp_len=0);
+1 - used (should always have lp_len>0);
+2 - HOT redirect (should have lp_len=0);
+3 - dead, may or may not have storage.
+</screen>
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>lp_len</structfield></entry>
+ <entry><type>smallint</type></entry>
+ <entry>length of tuple</entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_xmin</structfield></entry>
+ <entry><type>xid</type></entry>
+ <entry>transaction ID when this tuple was created. See
+ <xref linkend="ddl-system-columns">
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_xmax</structfield></entry>
+ <entry><type>xid</type></entry>
+ <entry>transaction ID when tuple was deleted or replaced with newer
+ version. See <xref linkend="ddl-system-columns">
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_field3</structfield></entry>
+ <entry><type>int4</type></entry>
+ <entry><filename>t_field3</filename> is a tricky a attribute, that can
+ be treated as Cmin, Cmax (command numbers inside transaction),
+ combination of Cmin and Cmax, or as Xvac - that is used by
+ oldstyle VACUUM. See <xref linkend="storage-page-layout">
+ and <xref linkend="ddl-system-columns"> for more
+ info
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_ctid</structfield></entry>
+ <entry><type>tid</type></entry>
+ <entry>location of the tuple in the heap (page number, row number). See
+ <xref linkend="ddl-system-columns">
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_infomask2</structfield></entry>
+ <entry><type>integer</type></entry>
+ <entry>stores number of attributes (11 bits of the word). The rest are used for flag bits:
+<screen>
+0x2000 - tuple was updated and key cols modified, or tuple deleted
+0x4000 - tuple was HOT-updated
+0x8000 - this is heap-only tuple
+0xE000 - visibility-related bits (so called "hint bits")
+</screen>
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_infomask</structfield></entry>
+ <entry><type>integer</type></entry>
+ <entry>consists of following flag bits:
+<screen>
+0x0001 - has null attribute(s)
+0x0002 - has variable-width attribute(s)
+0x0004 - has external stored attribute(s)
+0x0008 - has an object-id field
+0x0010 - xmax is a key-shared locker
+0x0020 - t_cid is a combo cid
+0x0040 - xmax is exclusive locker
+0x0080 - xmax, if valid, is only a locker
</screen>
- See <filename>src/include/storage/itemid.h</> and
- <filename>src/include/access/htup_details.h</> for explanations of the fields
- returned.
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_hoff</structfield></entry>
+ <entry><type>smallint</type></entry>
+ <entry>offset to user data: size of header tuple header, plus alingn
+ padding
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_bits</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry>bitmap of nulls converted into human readable text representation
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_oid</structfield></entry>
+ <entry><type>oid</type></entry>
+ <entry>object identifier. Now actually is used only in system tables.
+ See <xref linkend="ddl-system-columns">
+ </entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </informaltable>
+
+ For better understanding of these fields see
+ <xref linkend="storage-page-layout">, <xref linkend="ddl-system-columns">
+ and comments from source code <filename>src/include/storage/itemid.h</> and
+ <filename>src/include/access/htup_details.h</>.
+
+ </para>
+ </listitem>
+ </varlistentry>
+
+
+ <varlistentry>
+ <term>
+ <function>tuple_data_split(rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text) returs bytea[]</function>
+ <indexterm>
+ <primary>tuple_data_split</primary>
+ </indexterm>
+ </term>
+ <term>
+ <function>tuple_data_split(rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool) returs bytea[]</function>
+ </term>
+ <term>
+ <function>tuple_data_split(rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool, warning_mode bool) returs bytea[]</function>
+ </term>
+ <listitem>
+ <para>
+ <function>tuple_data_split</function> splits tuple data into attributes in
+ the same way as <function>nocachegetattr</function> function do it in
+ postgres internals. Returns array of bytea.
+<screen>
+test=# select tuple_data_split('pg_range'::regclass, '\x400f00001700000000000000ba0700004a0f0000520f0000'::bytea, 2304, 6, null);
+ tuple_data_split
+---------------------------------------------------------------------------------------
+ {"\\x400f0000","\\x17000000","\\x00000000","\\xba070000","\\x4a0f0000","\\x520f0000"}
+(1 row)
+</screen>
+ </para>
+
+ <para>
+ <function>tuple_data_split</function> takes following arguments:
+
+ <informaltable>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Argument</entry>
+ <entry>Type</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry><structfield>rel_oid</structfield></entry>
+ <entry><type>oid</type></entry>
+ <entry>OID of the relation, of the tuple we want to split</entry>
+ </row>
+
+ <row>
+ <entry><structfield>tuple_data</structfield></entry>
+ <entry><type>bytea</type></entry>
+ <entry>tuple raw data to split
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_infomask</structfield></entry>
+ <entry><type>integer</type></entry>
+ <entry><function>t_infomask</function> as returned by
+ <function>heap_page_items</function>
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_infomask2</structfield></entry>
+ <entry><type>integer</type></entry>
+ <entry><function>t_infomask2</function> as returned by
+ <function>heap_page_items</function>
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_bitd</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry>bitmsp of nulls as returned by
+ <function>heap_page_items</function>
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>do_detoast</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry>optional attribute, if <function>do_detoast</function> is set to
+ true, <function>tuple_data_split</function> will take attribute value
+ from TOAST and uncompress it if it was TOASTed or compressed. See
+ <xref linkend="storage-toast">
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>warning_mode</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry>optional attribute, if <function>warning_mode</function> is set to
+ true, <function>tuple_data_split</function> will not raise ERROR if
+ data consistency check fails, just send WARNING. This may lead to
+ backend crash, use it with care.
+ </entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </informaltable>
+ </para>
+ <para>
+ In most cases you will not need <function>tuple_data_split</function>
+ itself, consider using <function>heap_page_item_attrs</function> for
+ viewing page data with split attributes.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <function>heap_page_item_attrs(page bytea, rel_oid regclass) returns record</function>
+ <indexterm>
+ <primary>heap_page_item_attrs</primary>
+ </indexterm>
+ </term>
+ <term>
+ <function>heap_page_item_attrs(page bytea, rel_oid regclass, do_detoast bool) returns record</function>
+ </term>
+
+ <listitem>
+ <para>
+ <function>heap_page_item_attrs</function> is actually a clone of
+ <function>heap_page_items</function>, with one difference:
+ <function>heap_page_item_attrs</function> returns array of raw values of
+ tuple attributes instead of one peace of raw tuple data. All other return
+ columns are same as in <function>heap_page_items</function>.
+<screen>
+test=# select * from heap_page_item_attrs(get_raw_page('pg_range', 0),'pg_range'::regclass);
+ lp | lp_off | lp_flags | lp_len | t_xmin | t_xmax | t_field3 | t_ctid | t_infomask2 | t_infomask | t_hoff | t_bits | t_oid | t_attrs
+----+--------+----------+--------+--------+--------+----------+--------+-------------+------------+--------+--------+-------+--------------------------------------------------------------------------------------
+ 1 | 8144 | 1 | 48 | 1 | 0 | 0 | (0,1) | 6 | 2304 | 24 | | | {"\\x400f0000","\\x17000000","\\x00000000","\\xba070000","\\x4a0f0000","\\x520f0000"}
+ 2 | 8096 | 1 | 48 | 1 | 0 | 0 | (0,2) | 6 | 2304 | 24 | | | {"\\x420f0000","\\xa4060000","\\x00000000","\\x350c0000","\\x00000000","\\x540f0000"}
+ 3 | 8048 | 1 | 48 | 1 | 0 | 0 | (0,3) | 6 | 2304 | 24 | | | {"\\x440f0000","\\x5a040000","\\x00000000","\\x380c0000","\\x00000000","\\x590f0000"}
+ 4 | 8000 | 1 | 48 | 1 | 0 | 0 | (0,4) | 6 | 2304 | 24 | | | {"\\x460f0000","\\xa0040000","\\x00000000","\\x370c0000","\\x00000000","\\x5a0f0000"}
+ 5 | 7952 | 1 | 48 | 1 | 0 | 0 | (0,5) | 6 | 2304 | 24 | | | {"\\x480f0000","\\x3a040000","\\x00000000","\\x320c0000","\\x4b0f0000","\\x550f0000"}
+ 6 | 7904 | 1 | 48 | 1 | 0 | 0 | (0,6) | 6 | 2304 | 24 | | | {"\\x560f0000","\\x14000000","\\x00000000","\\x340c0000","\\x580f0000","\\x530f0000"}
+(6 rows)
+</screen>
+ </para>
+ <para>
+ <function>heap_page_item_attrs</function> takes following argiments:
+ <informaltable>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Argument</entry>
+ <entry>Type</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry><structfield>page</structfield></entry>
+ <entry><type>bytea</type></entry>
+ <entry>page raw data, usually returned by <function>get_raw_page</function></entry>
+ </row>
+
+ <row>
+ <entry><structfield>rel_oid</structfield></entry>
+ <entry><type>oid</type></entry>
+ <entry>OID of the relation, whose page we want to parse</entry>
+ </row>
+
+ <row>
+ <entry><structfield>do_detoast</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry>optional attribute, if <function>do_detoast</function> is set to
+ true, <function>heap_page_item_attrs</function> will not only split
+ tuple data into attributes, but also try to deTOAST and uncompress
+ attribute value if it was TOASTed or compressed.
+ See <xref linkend="storage-toast">
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </informaltable>
+
</para>
</listitem>
</varlistentry>
Re: pageinspect patch, for showing tuple data
On Sat, Sep 26, 2015 at 1:46 AM, Nikolay Shaplov wrote:
В письме от 25 сентября 2015 20:59:29 пользователь Michael Paquier написал:
Thanks! I just had a short look at it:
- I am not convinced that it is worth declaring 3 versions of
tuple_data_split.How which of them should we leave?
The one with the most arguments. Now perhaps we could have as well two
of them: one which has the minimum number of arguments with default
values, and the second one with the maximum number of arguments.
- The patch does not respect the project code style,
particularly one-line "if condition {foo;}" are not adapted, code line islimited
to 80 characters, etc. The list is basically here:
http://www.postgresql.org/docs/current/static/source.htmlI did my best. Results are attached.
Thanks, it looks better.
- Be aware of typos: s/whitch/which is one.
I've run spellchecker on all comments. Hope that I removed most of the
mistakes. But as I am not native speaker, I will not be able to eliminate them
all. I will need help here.
I have not spotted new mistakes regarding that.
+ <entry><structfield>t_infomask2</structfield></entry> + <entry><type>integer</type></entry> + <entry>stores number of attributes (11 bits of the word). The rest are used for flag bits: +<screen> +0x2000 - tuple was updated and key cols modified, or tuple deleted +0x4000 - tuple was HOT-updated +0x8000 - this is heap-only tuple +0xE000 - visibility-related bits (so called "hint bits") +</screen> This large chunk of documentation is a duplicate of storage.sgml. If that's really necessary, it looks adapted to me to have more detailed comments at code level directly in heapfuncs.c.Hm... There is no explanation of t_infomask/t_infomask2 bits in storage.sgml.
If there is no source of information other then source code, then the
documentation is not good.
pageinspect is already something that works at low-level. My guess is
that users of this module are going to have a look at the code either
way to understand how tuple data is manipulated within a page.
So I would consider two options: Either move t_infomask/t_infomask2 details
into storage.sgml or leave as it is.
The documentation redirects the reader to look at htup_details.h (the
documentation is actually incorrect, I sent a separate patch), and I
see no point in duplicating such low-level descriptions, that would be
double maintenance for the same result.
I am lazy, and does not feel confidence about touching main documentation, so I
would prefer to leave as it is.
Your patch is proving the contrary :) Honestly I would just rip out
the part you have added to describe all the fields related to
HeapTupleHeaderData, and have only a simple self-contained example to
show how to use the new function tuple_data_split.
The example of tuple_data_split in the docs would be more interesting
if embedded with a call to heap_page_items.This example would be almost not readable. May be I should add one more praise
explaining where did we take arguments for tuple_data_split
I would as well just remove heap_page_item_attrs, an example in the
docs would be just enough IMO (note that I never mind being outvoted).
Btw, shouldn't the ereport messages in tuple_data_split use
error_level instead of ERROR?
Regards,
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
В письме от 26 сентября 2015 20:57:25 Вы написали:
Thanks! I just had a short look at it:
- I am not convinced that it is worth declaring 3 versions of
tuple_data_split.How which of them should we leave?
The one with the most arguments. Now perhaps we could have as well two
of them: one which has the minimum number of arguments with default
values, and the second one with the maximum number of arguments.
I've been thinking about this warning mode for a while...
I've added this argument to the heap_page_item_atts, because if somebody would
like to force it to parse fake data, it would be really difficult to provide
proper fake data if one heap to perfectly fit tuple descriptor of another heap.
So to do this you have to lover error level.
But since now we actually parse data with tuple_data_split, we can provide a
precisely formed fake information, so you are not limited with how it is
actually stored in page. You just pass any arguments you want. So you does not
need warning mode anymore.
So may be I should get rid of "warning mode" now.
Concerning do_detoast argument. I do not have any clear opinion. I thing that
tuple_data_split is a kind of internal function. So it is ok if it is not so
convenient to use.
But in heap_page_item_attrs should be optional. Because it is used from plsql
console, and in most cases user does not want deTOASing.
So if in one function do_detoast is optional, may be it would be good to have
it optional in other functions to keep similarity...
So summorising: if you want make things simpler, I would first get totally rid
of warning_mode first of all, then if you would insist, make do_detoast non-
optional.
+ <entry><structfield>t_infomask2</structfield></entry> + <entry><type>integer</type></entry> + <entry>stores number of attributes (11 bits of the word). The rest are used for flag bits: +<screen> +0x2000 - tuple was updated and key cols modified, or tuple deleted +0x4000 - tuple was HOT-updated +0x8000 - this is heap-only tuple +0xE000 - visibility-related bits (so called "hint bits") +</screen> This large chunk of documentation is a duplicate of storage.sgml. If that's really necessary, it looks adapted to me to have more detailed comments at code level directly in heapfuncs.c.Hm... There is no explanation of t_infomask/t_infomask2 bits in
storage.sgml.If there is no source of information other then source code, then the
documentation is not good.pageinspect is already something that works at low-level. My guess is
that users of this module are going to have a look at the code either
way to understand how tuple data is manipulated within a page.
So I would consider two options: Either move t_infomask/t_infomask2
details
into storage.sgml or leave as it is.The documentation redirects the reader to look at htup_details.h (the
documentation is actually incorrect, I sent a separate patch)
I do not see any redirect at
http://www.postgresql.org/docs/9.4/static/pageinspect.html
, and I
see no point in duplicating such low-level descriptions, that would be
double maintenance for the same result.I am lazy, and does not feel confidence about touching main documentation,
so I would prefer to leave as it is.Your patch is proving the contrary :) Honestly I would just rip out
the part you have added to describe all the fields related to
HeapTupleHeaderData, and have only a simple self-contained example to
show how to use the new function tuple_data_split.
May be. I would think about it a little bit more, and discuss it with my
friends
The example of tuple_data_split in the docs would be more interesting
if embedded with a call to heap_page_items.This example would be almost not readable. May be I should add one more
praise explaining where did we take arguments for tuple_data_splitI would as well just remove heap_page_item_attrs, an example in the
docs would be just enough IMO (note that I never mind being outvoted).
This is function that pageinspect user is definitely will use this function.
And as a user I would like to see function output in documentation the way I
would see it, in order to be better prepared to see it in real life. There are
limits however, get_raw_page output should not be shown in documentation ;-)
heap_page_item_attrs is close to these limits, but did not reach it, in my
opinion... So I'd prefer to keep it there.
Btw, shouldn't the ereport messages in tuple_data_split use
error_level instead of ERROR?
These errors are in the part where I parse t_bits from text back to bit
representation. Here there is not chance to get non-strict behavior, since
there is no way to guess what should we do if we met characters then '0' or
'1'. Or what to do if there is only 7 characters but we should write whole
byte.etc...
Moreover as I wrote above bay be we just do not need warning_mode at all for
current situation.
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
В письме от 26 сентября 2015 20:57:25 пользователь Michael Paquier написал:
So I would consider two options: Either move t_infomask/t_infomask2
details
into storage.sgml or leave as it is.The documentation redirects the reader to look at htup_details.h (the
documentation is actually incorrect, I sent a separate patch), and I
see no point in duplicating such low-level descriptions, that would be
double maintenance for the same result.
What do you think about this? (I've changed only heap_tuple_items
documentation there)
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
pageinspect_show_tuple_datav7.difftext/x-patch; charset=UTF-8; name=pageinspect_show_tuple_datav7.diffDownload
diff --git a/contrib/pageinspect/Makefile b/contrib/pageinspect/Makefile
index aec5258..e4bc1af 100644
--- a/contrib/pageinspect/Makefile
+++ b/contrib/pageinspect/Makefile
@@ -5,7 +5,7 @@ OBJS = rawpage.o heapfuncs.o btreefuncs.o fsmfuncs.o \
brinfuncs.o ginfuncs.o $(WIN32RES)
EXTENSION = pageinspect
-DATA = pageinspect--1.3.sql pageinspect--1.2--1.3.sql \
+DATA = pageinspect--1.4.sql pageinspect--1.3--1.4.sql pageinspect--1.2--1.3.sql \
pageinspect--1.1--1.2.sql pageinspect--1.0--1.1.sql \
pageinspect--unpackaged--1.0.sql
PGFILEDESC = "pageinspect - functions to inspect contents of database pages"
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 8d1666c..52a0663 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -29,7 +29,14 @@
#include "funcapi.h"
#include "utils/builtins.h"
#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/rel.h"
+#include "catalog/pg_type.h"
+Datum split_tuple_data(char *tuple_data, uint16 tuple_data_len,
+ TupleDesc tuple_desc, uint16 t_infomask,
+ uint16 t_infomask2, bits8 *t_bits, bool
+ do_detoast, int error_level);
/*
* bits_to_text
@@ -91,7 +98,7 @@ heap_page_items(PG_FUNCTION_ARGS)
if (raw_page_size < SizeOfPageHeaderData)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("input page too small (%d bytes)", raw_page_size)));
+ errmsg("input page too small (%d bytes)", raw_page_size)));
fctx = SRF_FIRSTCALL_INIT();
mctx = MemoryContextSwitchTo(fctx->multi_call_memory_ctx);
@@ -122,8 +129,8 @@ heap_page_items(PG_FUNCTION_ARGS)
HeapTuple resultTuple;
Datum result;
ItemId id;
- Datum values[13];
- bool nulls[13];
+ Datum values[14];
+ bool nulls[14];
uint16 lp_offset;
uint16 lp_flags;
uint16 lp_len;
@@ -155,6 +162,8 @@ heap_page_items(PG_FUNCTION_ARGS)
{
HeapTupleHeader tuphdr;
int bits_len;
+ bytea *tuple_data_bytea;
+ int tuple_data_len;
/* Extract information from the tuple header */
@@ -168,6 +177,14 @@ heap_page_items(PG_FUNCTION_ARGS)
values[9] = UInt32GetDatum(tuphdr->t_infomask);
values[10] = UInt8GetDatum(tuphdr->t_hoff);
+ /* Copy raw tuple data into bytea attribute */
+ tuple_data_len = lp_len - tuphdr->t_hoff;
+ tuple_data_bytea = (bytea *) palloc(tuple_data_len + VARHDRSZ);
+ SET_VARSIZE(tuple_data_bytea, tuple_data_len + VARHDRSZ);
+ memcpy(VARDATA(tuple_data_bytea), (char *) tuphdr + tuphdr->t_hoff,
+ tuple_data_len);
+ values[13] = PointerGetDatum(tuple_data_bytea);
+ nulls[13] = false;
/*
* We already checked that the item is completely within the raw
* page passed to us, with the length given in the line pointer.
@@ -208,7 +225,7 @@ heap_page_items(PG_FUNCTION_ARGS)
*/
int i;
- for (i = 4; i <= 12; i++)
+ for (i = 4; i <= 13; i++)
nulls[i] = true;
}
@@ -223,3 +240,223 @@ heap_page_items(PG_FUNCTION_ARGS)
else
SRF_RETURN_DONE(fctx);
}
+
+PG_FUNCTION_INFO_V1(tuple_data_split);
+Datum
+tuple_data_split(PG_FUNCTION_ARGS)
+{
+ Oid rel_oid;
+ bytea *raw_data;
+ uint16 t_infomask;
+ uint16 t_infomask2;
+ text *t_bits_str;
+ RelationData *rel;
+ TupleDesc tuple_desc;
+ bool do_detoast = false;
+ int error_level = ERROR;
+
+ bits8 *t_bits = NULL;
+ Datum res;
+
+ rel_oid = PG_GETARG_OID(0);
+ raw_data = PG_GETARG_BYTEA_P(1);
+ t_infomask = PG_GETARG_INT16(2);
+ t_infomask2 = PG_GETARG_INT16(3);
+ t_bits_str = PG_ARGISNULL(4) ? NULL : PG_GETARG_TEXT_P(4);
+ if (PG_NARGS()>=6)
+ do_detoast = PG_GETARG_BOOL(5);
+ if (PG_NARGS()>=7)
+ error_level = PG_GETARG_BOOL(6) ? WARNING : ERROR;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ (errmsg("must be superuser to use raw page functions"))));
+
+ /*
+ * Here we converting t_bits string back to the bits8 array,
+ * as it really is in the tuple header
+ */
+ if (t_infomask & HEAP_HASNULL)
+ {
+ int bits_str_len;
+ int bits_len;
+ char *p;
+ int off;
+ char byte = 0;
+
+ bits_len = (t_infomask2 & HEAP_NATTS_MASK) / 8 + 1;
+ if (!t_bits_str)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits argument is NULL, though we expect it to be NOT NULL and %i character long",
+ bits_len * 8)));
+
+ bits_str_len = VARSIZE(t_bits_str) - VARHDRSZ;
+ if (bits_str_len % 8)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits argument length should be multiple of eight")));
+
+ if (bits_len * 8 != bits_str_len)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("wrong t_bits argument length. Expected %i, actual is %i",
+ bits_len * 8, bits_str_len)));
+
+ t_bits = palloc(bits_len + 1);
+ p = (char *) t_bits_str + VARHDRSZ;
+ off = 0;
+
+ while (off<bits_str_len)
+ {
+ if (!(off % 8))
+ byte = 0;
+
+ if (( p[off] == '0') || (p[off] == '1'))
+ byte = byte | ( (p[off]-'0')<<off % 8);
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("illegal character '%c' in t_bits string", p[off])));
+
+ if (off % 8 == 7)
+ t_bits[off / 8] = byte;
+
+ off++;
+ }
+ }
+ else
+ {
+ if (t_bits_str)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits string is expected to be NULL, but instead it is %i bytes length",
+ VARSIZE(t_bits_str) - VARHDRSZ)));
+ }
+
+ /* Getting tuple descriptor from relation OID */
+ rel = relation_open(rel_oid, NoLock);
+ tuple_desc = CreateTupleDescCopyConstr(rel->rd_att);
+ relation_close(rel, NoLock);
+
+ /* Splitting tuple data */
+ res = split_tuple_data((char *) raw_data + VARHDRSZ,
+ VARSIZE(raw_data) - VARHDRSZ, tuple_desc,
+ t_infomask, t_infomask2, t_bits,
+ do_detoast, error_level);
+
+ PG_RETURN_ARRAYTYPE_P(res);
+}
+
+
+Datum
+split_tuple_data(char *tuple_data, uint16 tuple_data_len, TupleDesc tuple_desc,
+ uint16 t_infomask, uint16 t_infomask2, bits8 *t_bits,
+ bool do_detoast, int error_level)
+{
+ ArrayBuildState *raw_attrs;
+ int nattrs;
+ int i;
+ int off;
+
+ /*
+ * Here we reimplement the basic functionality of nocachegetattr from
+ * backend/access/common/heaptuple.c witch is basically used for fetching
+ * attributes from tuple when it is not cached. We can not use
+ * nocachegetattr here directly, because we should ignore all cache
+ * optimizations and other stuff just get binary data as it is.
+ */
+
+ raw_attrs = initArrayResult(BYTEAOID,CurrentMemoryContext,0);
+ off = 0;
+ nattrs = tuple_desc->natts;
+
+ if (error_level && nattrs < (t_infomask2 & HEAP_NATTS_MASK))
+ ereport(error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: number of attributes in tuple header is greater than number of attributes in tuple descripor")));
+
+ for(i=0; i < nattrs; i++)
+ {
+ Form_pg_attribute attr;
+ Datum raw_attr = PointerGetDatum(NULL);
+ bool is_null;
+
+ attr = tuple_desc->attrs[i];
+ is_null = (t_infomask & HEAP_HASNULL) && att_isnull(i, t_bits);
+ /*
+ * Tuple header can specify less attributes then tuple descriptor
+ * as ALTER TABLE ADD COLUMN without DEFAULT keyword does not
+ * actually change tuples in pages, so attributes with numbers greater
+ * than t_infomask2 & HEAP_NATTS_MASK should be treated as NULL
+ */
+ if (i >= (t_infomask2 & HEAP_NATTS_MASK) )
+ is_null = true;
+
+ if (!is_null)
+ {
+ int len;
+ bytea *attr_data;
+
+ if (attr->attlen == -1)
+ {
+ off = att_align_pointer(off, tuple_desc->attrs[i]->attalign, -1,
+ tuple_data + off);
+ /*
+ * As VARSIZE_ANY throws an exception if it can't properly detect
+ * type of external storage in macros VARTAG_SIZE, so we repeat
+ * this check here to preform nicer error handling
+ */
+ if ( error_level &&
+ VARATT_IS_1B_E(tuple_data + off) &&
+ VARTAG_EXTERNAL(tuple_data + off) != VARTAG_INDIRECT &&
+ VARTAG_EXTERNAL(tuple_data + off) != VARTAG_ONDISK)
+ {
+ ereport(error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: First byte of varlen attr seems to be corrupted")));
+ }
+
+ len = VARSIZE_ANY(tuple_data + off);
+ }
+ else
+ {
+ off = att_align_nominal(off, tuple_desc->attrs[i]->attalign);
+ len = attr->attlen;
+ }
+
+ if (error_level && tuple_data_len < off + len)
+ ereport(error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: Iterating over tuple data reached out of actual tuple size")));
+
+ attr_data = (bytea *) palloc(len + VARHDRSZ);
+ SET_VARSIZE(attr_data, len + VARHDRSZ);
+ memcpy(VARDATA(attr_data), tuple_data + off, len);
+ raw_attr = PointerGetDatum(attr_data);
+
+ if ( attr->attlen == -1 && do_detoast)
+ {
+ Datum raw_attr_copy;
+ raw_attr_copy = PointerGetDatum(
+ PG_DETOAST_DATUM_COPY(tuple_data + off));
+ pfree(attr_data);
+ raw_attr = raw_attr_copy;
+ }
+
+ off = att_addlength_pointer(off, tuple_desc->attrs[i]->attlen,
+ tuple_data + off);
+ }
+
+ raw_attrs = accumArrayResult(raw_attrs, raw_attr, is_null, BYTEAOID,
+ CurrentMemoryContext);
+ }
+
+ if (error_level && tuple_data_len != off)
+ ereport(error_level,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: Iterating over tuple data did not actualy reach tuple end")));
+
+ return makeArrayResult(raw_attrs, CurrentMemoryContext);
+}
diff --git a/contrib/pageinspect/pageinspect--1.3--1.4.sql b/contrib/pageinspect/pageinspect--1.3--1.4.sql
new file mode 100644
index 0000000..5a16261
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.3--1.4.sql
@@ -0,0 +1,87 @@
+/* contrib/pageinspect/pageinspect--1.3--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via ALTER EXTENSION
+\echo Use "ALTER EXTENSION pageinspect UPDATE TO '1.4'" to load this file. \quit
+
+--
+-- heap_page_items()
+--
+DROP FUNCTION heap_page_items(bytea);
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(t_oid rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(t_oid rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(t_oid rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool, warning_mode bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN rel_oid regclass, IN do_detoast bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT lp, lp_off, lp_flags, lp_len, t_xmin, t_xmax, t_field3, t_ctid, t_infomask2, t_infomask, t_hoff, t_bits, t_oid, tuple_data_split(rel_oid, t_data, t_infomask, t_infomask2, t_bits, do_detoast) as t_attrs from heap_page_items(page)'
+LANGUAGE SQL;
+
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT * from heap_page_item_attrs(page, heap_oid, false)'
+LANGUAGE SQL;
diff --git a/contrib/pageinspect/pageinspect--1.3.sql b/contrib/pageinspect/pageinspect--1.3.sql
deleted file mode 100644
index a99e058..0000000
--- a/contrib/pageinspect/pageinspect--1.3.sql
+++ /dev/null
@@ -1,189 +0,0 @@
-/* contrib/pageinspect/pageinspect--1.3.sql */
-
--- complain if script is sourced in psql, rather than via CREATE EXTENSION
-\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
-
---
--- get_raw_page()
---
-CREATE FUNCTION get_raw_page(text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page'
-LANGUAGE C STRICT;
-
-CREATE FUNCTION get_raw_page(text, text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page_fork'
-LANGUAGE C STRICT;
-
---
--- page_header()
---
-CREATE FUNCTION page_header(IN page bytea,
- OUT lsn pg_lsn,
- OUT checksum smallint,
- OUT flags smallint,
- OUT lower smallint,
- OUT upper smallint,
- OUT special smallint,
- OUT pagesize smallint,
- OUT version smallint,
- OUT prune_xid xid)
-AS 'MODULE_PATHNAME', 'page_header'
-LANGUAGE C STRICT;
-
---
--- heap_page_items()
---
-CREATE FUNCTION heap_page_items(IN page bytea,
- OUT lp smallint,
- OUT lp_off smallint,
- OUT lp_flags smallint,
- OUT lp_len smallint,
- OUT t_xmin xid,
- OUT t_xmax xid,
- OUT t_field3 int4,
- OUT t_ctid tid,
- OUT t_infomask2 integer,
- OUT t_infomask integer,
- OUT t_hoff smallint,
- OUT t_bits text,
- OUT t_oid oid)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'heap_page_items'
-LANGUAGE C STRICT;
-
---
--- bt_metap()
---
-CREATE FUNCTION bt_metap(IN relname text,
- OUT magic int4,
- OUT version int4,
- OUT root int4,
- OUT level int4,
- OUT fastroot int4,
- OUT fastlevel int4)
-AS 'MODULE_PATHNAME', 'bt_metap'
-LANGUAGE C STRICT;
-
---
--- bt_page_stats()
---
-CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
- OUT blkno int4,
- OUT type "char",
- OUT live_items int4,
- OUT dead_items int4,
- OUT avg_item_size int4,
- OUT page_size int4,
- OUT free_size int4,
- OUT btpo_prev int4,
- OUT btpo_next int4,
- OUT btpo int4,
- OUT btpo_flags int4)
-AS 'MODULE_PATHNAME', 'bt_page_stats'
-LANGUAGE C STRICT;
-
---
--- bt_page_items()
---
-CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
- OUT itemoffset smallint,
- OUT ctid tid,
- OUT itemlen smallint,
- OUT nulls bool,
- OUT vars bool,
- OUT data text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'bt_page_items'
-LANGUAGE C STRICT;
-
---
--- brin_page_type()
---
-CREATE FUNCTION brin_page_type(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'brin_page_type'
-LANGUAGE C STRICT;
-
---
--- brin_metapage_info()
---
-CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
- OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
-AS 'MODULE_PATHNAME', 'brin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- brin_revmap_data()
---
-CREATE FUNCTION brin_revmap_data(IN page bytea,
- OUT pages tid)
-RETURNS SETOF tid
-AS 'MODULE_PATHNAME', 'brin_revmap_data'
-LANGUAGE C STRICT;
-
---
--- brin_page_items()
---
-CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
- OUT itemoffset int,
- OUT blknum int,
- OUT attnum int,
- OUT allnulls bool,
- OUT hasnulls bool,
- OUT placeholder bool,
- OUT value text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'brin_page_items'
-LANGUAGE C STRICT;
-
---
--- fsm_page_contents()
---
-CREATE FUNCTION fsm_page_contents(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'fsm_page_contents'
-LANGUAGE C STRICT;
-
---
--- GIN functions
---
-
---
--- gin_metapage_info()
---
-CREATE FUNCTION gin_metapage_info(IN page bytea,
- OUT pending_head bigint,
- OUT pending_tail bigint,
- OUT tail_free_size int4,
- OUT n_pending_pages bigint,
- OUT n_pending_tuples bigint,
- OUT n_total_pages bigint,
- OUT n_entry_pages bigint,
- OUT n_data_pages bigint,
- OUT n_entries bigint,
- OUT version int4)
-AS 'MODULE_PATHNAME', 'gin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- gin_page_opaque_info()
---
-CREATE FUNCTION gin_page_opaque_info(IN page bytea,
- OUT rightlink bigint,
- OUT maxoff int4,
- OUT flags text[])
-AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
-LANGUAGE C STRICT;
-
---
--- gin_leafpage_items()
---
-CREATE FUNCTION gin_leafpage_items(IN page bytea,
- OUT first_tid tid,
- OUT nbytes int2,
- OUT tids tid[])
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'gin_leafpage_items'
-LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect--1.4.sql b/contrib/pageinspect/pageinspect--1.4.sql
new file mode 100644
index 0000000..c09657b
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.4.sql
@@ -0,0 +1,251 @@
+/* contrib/pageinspect/pageinspect--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
+
+--
+-- get_raw_page()
+--
+CREATE FUNCTION get_raw_page(text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION get_raw_page(text, text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page_fork'
+LANGUAGE C STRICT;
+
+--
+-- page_header()
+--
+CREATE FUNCTION page_header(IN page bytea,
+ OUT lsn pg_lsn,
+ OUT checksum smallint,
+ OUT flags smallint,
+ OUT lower smallint,
+ OUT upper smallint,
+ OUT special smallint,
+ OUT pagesize smallint,
+ OUT version smallint,
+ OUT prune_xid xid)
+AS 'MODULE_PATHNAME', 'page_header'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_items()
+--
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea
+ )
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(rel_oid oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(rel_oid oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(rel_oid oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool, warning_mode bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN rel_oid regclass, IN do_detoast bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT lp, lp_off, lp_flags, lp_len, t_xmin, t_xmax, t_field3, t_ctid, t_infomask2, t_infomask, t_hoff, t_bits, t_oid, tuple_data_split(rel_oid, t_data, t_infomask, t_infomask2, t_bits, do_detoast) as t_attrs from heap_page_items(page)'
+LANGUAGE SQL;
+
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT * from heap_page_item_attrs(page, heap_oid, false)'
+LANGUAGE SQL;
+
+--
+-- bt_metap()
+--
+CREATE FUNCTION bt_metap(IN relname text,
+ OUT magic int4,
+ OUT version int4,
+ OUT root int4,
+ OUT level int4,
+ OUT fastroot int4,
+ OUT fastlevel int4)
+AS 'MODULE_PATHNAME', 'bt_metap'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_stats()
+--
+CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
+ OUT blkno int4,
+ OUT type "char",
+ OUT live_items int4,
+ OUT dead_items int4,
+ OUT avg_item_size int4,
+ OUT page_size int4,
+ OUT free_size int4,
+ OUT btpo_prev int4,
+ OUT btpo_next int4,
+ OUT btpo int4,
+ OUT btpo_flags int4)
+AS 'MODULE_PATHNAME', 'bt_page_stats'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_items()
+--
+CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
+ OUT itemoffset smallint,
+ OUT ctid tid,
+ OUT itemlen smallint,
+ OUT nulls bool,
+ OUT vars bool,
+ OUT data text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'bt_page_items'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_type()
+--
+CREATE FUNCTION brin_page_type(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'brin_page_type'
+LANGUAGE C STRICT;
+
+--
+-- brin_metapage_info()
+--
+CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
+ OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
+AS 'MODULE_PATHNAME', 'brin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- brin_revmap_data()
+--
+CREATE FUNCTION brin_revmap_data(IN page bytea,
+ OUT pages tid)
+RETURNS SETOF tid
+AS 'MODULE_PATHNAME', 'brin_revmap_data'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_items()
+--
+CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
+ OUT itemoffset int,
+ OUT blknum int,
+ OUT attnum int,
+ OUT allnulls bool,
+ OUT hasnulls bool,
+ OUT placeholder bool,
+ OUT value text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'brin_page_items'
+LANGUAGE C STRICT;
+
+--
+-- fsm_page_contents()
+--
+CREATE FUNCTION fsm_page_contents(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'fsm_page_contents'
+LANGUAGE C STRICT;
+
+--
+-- GIN functions
+--
+
+--
+-- gin_metapage_info()
+--
+CREATE FUNCTION gin_metapage_info(IN page bytea,
+ OUT pending_head bigint,
+ OUT pending_tail bigint,
+ OUT tail_free_size int4,
+ OUT n_pending_pages bigint,
+ OUT n_pending_tuples bigint,
+ OUT n_total_pages bigint,
+ OUT n_entry_pages bigint,
+ OUT n_data_pages bigint,
+ OUT n_entries bigint,
+ OUT version int4)
+AS 'MODULE_PATHNAME', 'gin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_page_opaque_info()
+--
+CREATE FUNCTION gin_page_opaque_info(IN page bytea,
+ OUT rightlink bigint,
+ OUT maxoff int4,
+ OUT flags text[])
+AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_leafpage_items()
+--
+CREATE FUNCTION gin_leafpage_items(IN page bytea,
+ OUT first_tid tid,
+ OUT nbytes int2,
+ OUT tids tid[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'gin_leafpage_items'
+LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect.control b/contrib/pageinspect/pageinspect.control
index a9dab33..68c7d61 100644
--- a/contrib/pageinspect/pageinspect.control
+++ b/contrib/pageinspect/pageinspect.control
@@ -1,5 +1,5 @@
# pageinspect extension
comment = 'inspect the contents of database pages at a low level'
-default_version = '1.3'
+default_version = '1.4'
module_pathname = '$libdir/pageinspect'
relocatable = true
diff --git a/doc/src/sgml/pageinspect.sgml b/doc/src/sgml/pageinspect.sgml
index b95cc81..a67db64 100644
--- a/doc/src/sgml/pageinspect.sgml
+++ b/doc/src/sgml/pageinspect.sgml
@@ -93,19 +93,233 @@ test=# SELECT * FROM page_header(get_raw_page('pg_class', 0));
<listitem>
<para>
<function>heap_page_items</function> shows all line pointers on a heap
- page. For those line pointers that are in use, tuple headers are also
- shown. All tuples are shown, whether or not the tuples were visible to
- an MVCC snapshot at the time the raw page was copied.
+ page. For those line pointers that are in use,
+ <function>heap_page_items</function> also shows tuple headers and raw
+ tuple data. All tuples are shown, whether or not the tuples were visible
+ to an MVCC snapshot at the time the raw page was copied.
</para>
<para>
A heap page image obtained with <function>get_raw_page</function> should
be passed as argument. For example:
<screen>
-test=# SELECT * FROM heap_page_items(get_raw_page('pg_class', 0));
+
+test=# select * from heap_page_items(get_raw_page('pg_range', 0));
+ lp | lp_off | lp_flags | lp_len | t_xmin | t_xmax | t_field3 | t_ctid | t_infomask2 | t_infomask | t_hoff | t_bits | t_oid | t_data
+----+--------+----------+--------+--------+--------+----------+--------+-------------+------------+--------+--------+-------+----------------------------------------------------
+ 1 | 8144 | 1 | 48 | 1 | 0 | 0 | (0,1) | 6 | 2304 | 24 | | | \x400f00001700000000000000ba0700004a0f0000520f0000
+ 2 | 8096 | 1 | 48 | 1 | 0 | 0 | (0,2) | 6 | 2304 | 24 | | | \x420f0000a406000000000000350c000000000000540f0000
+ 3 | 8048 | 1 | 48 | 1 | 0 | 0 | (0,3) | 6 | 2304 | 24 | | | \x440f00005a04000000000000380c000000000000590f0000
+ 4 | 8000 | 1 | 48 | 1 | 0 | 0 | (0,4) | 6 | 2304 | 24 | | | \x460f0000a004000000000000370c0000000000005a0f0000
+ 5 | 7952 | 1 | 48 | 1 | 0 | 0 | (0,5) | 6 | 2304 | 24 | | | \x480f00003a04000000000000320c00004b0f0000550f0000
+ 6 | 7904 | 1 | 48 | 1 | 0 | 0 | (0,6) | 6 | 2304 | 24 | | | \x560f00001400000000000000340c0000580f0000530f0000
+(6 rows)
+
+</screen>
+ </para>
+ <para>
+ General idea about output columns: <function>lp_*</function> attributes
+ are about where tuple is located inside the page;
+ <function>t_xmin</function>, <function>t_xmax</function>,
+ <function>t_field3</function>, <function>t_ctid</function> are about
+ visibility of this tuplue inside or outside of the transaction;
+ <function>t_infomask2</function>, <function>t_infomask</function> has some
+ information about properties of attributes stored in tuple data.
+ <function>t_hoff</function> sais where tuple data begins and
+ <function>t_bits</function> sais which attributes are NULL and which are
+ not. Please notice that t_bits here is not an actual value that is stored
+ in tuple data, but it's text representation with '0' and '1' charactrs.
+ </para>
+
+ <para>
+ For more detailed information see documentation:
+ <xref linkend="storage-page-layout">, <xref linkend="ddl-system-columns">
+ and source code: <filename>src/include/storage/itemid.h</>,
+ <filename>src/include/access/htup_details.h</>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+
+ <varlistentry>
+ <term>
+ <function>tuple_data_split(rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text) returs bytea[]</function>
+ <indexterm>
+ <primary>tuple_data_split</primary>
+ </indexterm>
+ </term>
+ <term>
+ <function>tuple_data_split(rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool) returs bytea[]</function>
+ </term>
+ <term>
+ <function>tuple_data_split(rel_oid, tuple_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool, warning_mode bool) returs bytea[]</function>
+ </term>
+ <listitem>
+ <para>
+ <function>tuple_data_split</function> splits tuple data into attributes in
+ the same way as <function>nocachegetattr</function> function do it in
+ postgres internals. Returns array of bytea.
+<screen>
+test=# select tuple_data_split('pg_range'::regclass, '\x400f00001700000000000000ba0700004a0f0000520f0000'::bytea, 2304, 6, null);
+ tuple_data_split
+---------------------------------------------------------------------------------------
+ {"\\x400f0000","\\x17000000","\\x00000000","\\xba070000","\\x4a0f0000","\\x520f0000"}
+(1 row)
</screen>
- See <filename>src/include/storage/itemid.h</> and
- <filename>src/include/access/htup_details.h</> for explanations of the fields
- returned.
+ </para>
+
+ <para>
+ <function>tuple_data_split</function> takes following arguments:
+
+ <informaltable>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Argument</entry>
+ <entry>Type</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry><structfield>rel_oid</structfield></entry>
+ <entry><type>oid</type></entry>
+ <entry>OID of the relation, of the tuple we want to split</entry>
+ </row>
+
+ <row>
+ <entry><structfield>tuple_data</structfield></entry>
+ <entry><type>bytea</type></entry>
+ <entry>tuple raw data to split
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_infomask</structfield></entry>
+ <entry><type>integer</type></entry>
+ <entry><function>t_infomask</function> as returned by
+ <function>heap_page_items</function>
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_infomask2</structfield></entry>
+ <entry><type>integer</type></entry>
+ <entry><function>t_infomask2</function> as returned by
+ <function>heap_page_items</function>
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>t_bitd</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry>bitmsp of nulls as returned by
+ <function>heap_page_items</function>
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>do_detoast</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry>optional attribute, if <function>do_detoast</function> is set to
+ true, <function>tuple_data_split</function> will take attribute value
+ from TOAST and uncompress it if it was TOASTed or compressed. See
+ <xref linkend="storage-toast">
+ </entry>
+ </row>
+
+ <row>
+ <entry><structfield>warning_mode</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry>optional attribute, if <function>warning_mode</function> is set to
+ true, <function>tuple_data_split</function> will not raise ERROR if
+ data consistency check fails, just send WARNING. This may lead to
+ backend crash, use it with care.
+ </entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </informaltable>
+ </para>
+ <para>
+ In most cases you will not need <function>tuple_data_split</function>
+ itself, consider using <function>heap_page_item_attrs</function> for
+ viewing page data with split attributes.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <function>heap_page_item_attrs(page bytea, rel_oid regclass) returns record</function>
+ <indexterm>
+ <primary>heap_page_item_attrs</primary>
+ </indexterm>
+ </term>
+ <term>
+ <function>heap_page_item_attrs(page bytea, rel_oid regclass, do_detoast bool) returns record</function>
+ </term>
+
+ <listitem>
+ <para>
+ <function>heap_page_item_attrs</function> is actually a clone of
+ <function>heap_page_items</function>, with one difference:
+ <function>heap_page_item_attrs</function> returns array of raw values of
+ tuple attributes instead of one peace of raw tuple data. All other return
+ columns are same as in <function>heap_page_items</function>.
+<screen>
+test=# select * from heap_page_item_attrs(get_raw_page('pg_range', 0),'pg_range'::regclass);
+ lp | lp_off | lp_flags | lp_len | t_xmin | t_xmax | t_field3 | t_ctid | t_infomask2 | t_infomask | t_hoff | t_bits | t_oid | t_attrs
+----+--------+----------+--------+--------+--------+----------+--------+-------------+------------+--------+--------+-------+--------------------------------------------------------------------------------------
+ 1 | 8144 | 1 | 48 | 1 | 0 | 0 | (0,1) | 6 | 2304 | 24 | | | {"\\x400f0000","\\x17000000","\\x00000000","\\xba070000","\\x4a0f0000","\\x520f0000"}
+ 2 | 8096 | 1 | 48 | 1 | 0 | 0 | (0,2) | 6 | 2304 | 24 | | | {"\\x420f0000","\\xa4060000","\\x00000000","\\x350c0000","\\x00000000","\\x540f0000"}
+ 3 | 8048 | 1 | 48 | 1 | 0 | 0 | (0,3) | 6 | 2304 | 24 | | | {"\\x440f0000","\\x5a040000","\\x00000000","\\x380c0000","\\x00000000","\\x590f0000"}
+ 4 | 8000 | 1 | 48 | 1 | 0 | 0 | (0,4) | 6 | 2304 | 24 | | | {"\\x460f0000","\\xa0040000","\\x00000000","\\x370c0000","\\x00000000","\\x5a0f0000"}
+ 5 | 7952 | 1 | 48 | 1 | 0 | 0 | (0,5) | 6 | 2304 | 24 | | | {"\\x480f0000","\\x3a040000","\\x00000000","\\x320c0000","\\x4b0f0000","\\x550f0000"}
+ 6 | 7904 | 1 | 48 | 1 | 0 | 0 | (0,6) | 6 | 2304 | 24 | | | {"\\x560f0000","\\x14000000","\\x00000000","\\x340c0000","\\x580f0000","\\x530f0000"}
+(6 rows)
+</screen>
+ </para>
+ <para>
+ <function>heap_page_item_attrs</function> takes following argiments:
+ <informaltable>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Argument</entry>
+ <entry>Type</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry><structfield>page</structfield></entry>
+ <entry><type>bytea</type></entry>
+ <entry>page raw data, usually returned by <function>get_raw_page</function></entry>
+ </row>
+
+ <row>
+ <entry><structfield>rel_oid</structfield></entry>
+ <entry><type>oid</type></entry>
+ <entry>OID of the relation, whose page we want to parse</entry>
+ </row>
+
+ <row>
+ <entry><structfield>do_detoast</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry>optional attribute, if <function>do_detoast</function> is set to
+ true, <function>heap_page_item_attrs</function> will not only split
+ tuple data into attributes, but also try to deTOAST and uncompress
+ attribute value if it was TOASTed or compressed.
+ See <xref linkend="storage-toast">
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </informaltable>
+
</para>
</listitem>
</varlistentry>
Re: pageinspect patch, for showing tuple data
On Tue, Sep 29, 2015 at 11:39 PM, Nikolay Shaplov wrote:
But since now we actually parse data with tuple_data_split, we can
provide a
precisely formed fake information, so you are not limited with how it is
actually stored in page. You just pass any arguments you want. So you
does not
need warning mode anymore.
Yeah, I agree with you here, let's simplify it then. One could as well
catch the error in a plpgsql wrapper function if that's really necessary
and log the failed events at the same time in a custom way.
- errmsg("input page too small (%d bytes)",
raw_page_size)));
+ errmsg("input page too small (%d
bytes)", raw_page_size)));
Please be careful of spurious diffs. Those just make the life of committers
more difficult than it already is.+ <para>
+ General idea about output columns: <function>lp_*</function>
attributes
+ are about where tuple is located inside the page;
+ <function>t_xmin</function>, <function>t_xmax</function>,
+ <function>t_field3</function>, <function>t_ctid</function> are about
+ visibility of this tuplue inside or outside of the transaction;
+ <function>t_infomask2</function>, <function>t_infomask</function> has
some
+ information about properties of attributes stored in tuple data.
+ <function>t_hoff</function> sais where tuple data begins and
+ <function>t_bits</function> sais which attributes are NULL and which
are
+ not. Please notice that t_bits here is not an actual value that is
stored
+ in tuple data, but it's text representation with '0' and '1'
charactrs.
+ </para>
I would remove that as well. htup_details.h contains all this information.+ <para>
+ For more detailed information see documentation:
+ <xref linkend="storage-page-layout">, <xref
linkend="ddl-system-columns">
+ and source code: <filename>src/include/storage/itemid.h</>,
+ <filename>src/include/access/htup_details.h</>.
+ </para>
This looks cool to me though.+<screen>
+test=# select * from heap_page_item_attrs(get_raw_page('pg_range',
0),'pg_range'::regclass);
+[loooooong tuple data]
This example is too large in character per lines, I think that you should
cut a major part of the fields, why not just keeping lp and t_attrs for
example.+ <tbody>
+ <row>
+ <entry><structfield>rel_oid</structfield></entry>
+ <entry><type>oid</type></entry>
+ <entry>OID of the relation, of the tuple we want to split</entry>
+ </row>
+
+ <row>
+ <entry><structfield>tuple_data</structfield></entry>
+ <entry><type>bytea</type></entry>
+ <entry>tuple raw data to split
+ </entry>
+ </row>
In the description of tuple_data_split, I am not sure it is worth listing
all the argument of the function like that. IMO, we should just say: those
are the fields returned by "heap_page_items". tuple_data should as well be
renamed to t_data.+ tuple attributes instead of one peace of raw tuple data. All other
return
This is not that "peaceful" to me. It should be "piece" :)
+ values[13] = PointerGetDatum(tuple_data_bytea);
+ nulls[13] = false;
There is no point to set nulls[13] here.+<screen>
+test=# select tuple_data_split('pg_range'::regclass,
'\x400f00001700000000000000ba0700004a0f0000520f0000'::bytea, 2304, 6, null);
+ tuple_data_split
+---------------------------------------------------------------------------------------
+
{"\\x400f0000","\\x17000000","\\x00000000","\\xba070000","\\x4a0f0000","\\x520f0000"}
+(1 row)
This would be more demonstrative if combined with heap_page_items, like
that for example:
SELECT tuple_data_split('pg_class'::regclass, t_data, t_infomask,
t_infomask2, t_bits) FROM heap_page_items(get_raw_page('pg_class', 0));
And actually this query crashes.
--
MichaelRe: pageinspect patch, for showing tuple data
В письме от 30 сентября 2015 13:49:00 пользователь Michael Paquier написал:
- errmsg("input page too small (%d bytes)", raw_page_size))); + errmsg("input page too small (%d bytes)", raw_page_size))); Please be careful of spurious diffs. Those just make the life of committers more difficult than it already is.
Oh, that's not diff. That's I've fixed indent on the code I did not write :-)
+ <para> + General idea about output columns: <function>lp_*</function> attributes + are about where tuple is located inside the page; + <function>t_xmin</function>, <function>t_xmax</function>, + <function>t_field3</function>, <function>t_ctid</function> are about + visibility of this tuplue inside or outside of the transaction; + <function>t_infomask2</function>, <function>t_infomask</function> has some + information about properties of attributes stored in tuple data. + <function>t_hoff</function> sais where tuple data begins and + <function>t_bits</function> sais which attributes are NULL and which are + not. Please notice that t_bits here is not an actual value that is stored + in tuple data, but it's text representation with '0' and '1' charactrs. + </para> I would remove that as well. htup_details.h contains all this information.
Made it even more shorter. Just links and comments about representation of
t_bits.
+<screen> +test=# select * from heap_page_item_attrs(get_raw_page('pg_range', 0),'pg_range'::regclass); +[loooooong tuple data] This example is too large in character per lines, I think that you should cut a major part of the fields, why not just keeping lp and t_attrs for example.
Did id.
+ <tbody> + <row> + <entry><structfield>rel_oid</structfield></entry> + <entry><type>oid</type></entry> + <entry>OID of the relation, of the tuple we want to split</entry> + </row> + + <row> + <entry><structfield>tuple_data</structfield></entry> + <entry><type>bytea</type></entry> + <entry>tuple raw data to split + </entry> + </row> In the description of tuple_data_split, I am not sure it is worth listing all the argument of the function like that. IMO, we should just say: those are the fields returned by "heap_page_items". tuple_data should as well be renamed to t_data.
Did it.
+ tuple attributes instead of one peace of raw tuple data. All other
return
This is not that "peaceful" to me. It should be "piece" :)
Oops ;-)
+ values[13] = PointerGetDatum(tuple_data_bytea); + nulls[13] = false; There is no point to set nulls[13] here.
Oh, you are right!
+<screen> +test=# select tuple_data_split('pg_range'::regclass, '\x400f00001700000000000000ba0700004a0f0000520f0000'::bytea, 2304, 6, null); + tuple_data_split +--------------------------------------------------------------------------- ------------ + {"\\x400f0000","\\x17000000","\\x00000000","\\xba070000","\\x4a0f0000","\\x5 20f0000"} +(1 row) This would be more demonstrative if combined with heap_page_items, like that for example: SELECT tuple_data_split('pg_class'::regclass, t_data, t_infomask, t_infomask2, t_bits) FROM heap_page_items(get_raw_page('pg_class', 0)); And actually this query crashes.
Oh... It crached because I did not check that t_data can actually be NULL.
There also was a bug in original pageinspect, that did not get t_bits right
when there was OID in the tuple. I've fixed it too.
Here is next patch in attachment.
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
pageinspect_show_tuple_data_v8.difftext/x-patch; charset=UTF-8; name=pageinspect_show_tuple_data_v8.diffDownload
diff --git a/contrib/pageinspect/Makefile b/contrib/pageinspect/Makefile
index aec5258..e4bc1af 100644
--- a/contrib/pageinspect/Makefile
+++ b/contrib/pageinspect/Makefile
@@ -5,7 +5,7 @@ OBJS = rawpage.o heapfuncs.o btreefuncs.o fsmfuncs.o \
brinfuncs.o ginfuncs.o $(WIN32RES)
EXTENSION = pageinspect
-DATA = pageinspect--1.3.sql pageinspect--1.2--1.3.sql \
+DATA = pageinspect--1.4.sql pageinspect--1.3--1.4.sql pageinspect--1.2--1.3.sql \
pageinspect--1.1--1.2.sql pageinspect--1.0--1.1.sql \
pageinspect--unpackaged--1.0.sql
PGFILEDESC = "pageinspect - functions to inspect contents of database pages"
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 8d1666c..4fd3087 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -29,7 +29,14 @@
#include "funcapi.h"
#include "utils/builtins.h"
#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/rel.h"
+#include "catalog/pg_type.h"
+Datum split_tuple_data(char *tuple_data, uint16 tuple_data_len,
+ TupleDesc tuple_desc, uint16 t_infomask,
+ uint16 t_infomask2, bits8 *t_bits, bool
+ do_detoast);
/*
* bits_to_text
@@ -122,8 +129,8 @@ heap_page_items(PG_FUNCTION_ARGS)
HeapTuple resultTuple;
Datum result;
ItemId id;
- Datum values[13];
- bool nulls[13];
+ Datum values[14];
+ bool nulls[14];
uint16 lp_offset;
uint16 lp_flags;
uint16 lp_len;
@@ -154,7 +161,8 @@ heap_page_items(PG_FUNCTION_ARGS)
lp_offset + lp_len <= raw_page_size)
{
HeapTupleHeader tuphdr;
- int bits_len;
+ bytea *tuple_data_bytea;
+ int tuple_data_len;
/* Extract information from the tuple header */
@@ -168,6 +176,13 @@ heap_page_items(PG_FUNCTION_ARGS)
values[9] = UInt32GetDatum(tuphdr->t_infomask);
values[10] = UInt8GetDatum(tuphdr->t_hoff);
+ /* Copy raw tuple data into bytea attribute */
+ tuple_data_len = lp_len - tuphdr->t_hoff;
+ tuple_data_bytea = (bytea *) palloc(tuple_data_len + VARHDRSZ);
+ SET_VARSIZE(tuple_data_bytea, tuple_data_len + VARHDRSZ);
+ memcpy(VARDATA(tuple_data_bytea), (char *) tuphdr + tuphdr->t_hoff,
+ tuple_data_len);
+ values[13] = PointerGetDatum(tuple_data_bytea);
/*
* We already checked that the item is completely within the raw
* page passed to us, with the length given in the line pointer.
@@ -180,11 +195,11 @@ heap_page_items(PG_FUNCTION_ARGS)
{
if (tuphdr->t_infomask & HEAP_HASNULL)
{
- bits_len = tuphdr->t_hoff -
- offsetof(HeapTupleHeaderData, t_bits);
+ int bits_len =
+ ((tuphdr->t_infomask2 & HEAP_NATTS_MASK)/8 + 1)*8;
values[11] = CStringGetTextDatum(
- bits_to_text(tuphdr->t_bits, bits_len * 8));
+ bits_to_text(tuphdr->t_bits, bits_len));
}
else
nulls[11] = true;
@@ -208,7 +223,7 @@ heap_page_items(PG_FUNCTION_ARGS)
*/
int i;
- for (i = 4; i <= 12; i++)
+ for (i = 4; i <= 13; i++)
nulls[i] = true;
}
@@ -223,3 +238,222 @@ heap_page_items(PG_FUNCTION_ARGS)
else
SRF_RETURN_DONE(fctx);
}
+
+PG_FUNCTION_INFO_V1(tuple_data_split);
+Datum
+tuple_data_split(PG_FUNCTION_ARGS)
+{
+ Oid rel_oid;
+ bytea *raw_data;
+ uint16 t_infomask;
+ uint16 t_infomask2;
+ text *t_bits_str;
+ RelationData *rel;
+ TupleDesc tuple_desc;
+ bool do_detoast = false;
+
+ bits8 *t_bits = NULL;
+ Datum res;
+
+ rel_oid = PG_GETARG_OID(0);
+ raw_data = PG_ARGISNULL(1) ? NULL : PG_GETARG_BYTEA_P(1);
+ t_infomask = PG_GETARG_INT16(2);
+ t_infomask2 = PG_GETARG_INT16(3);
+ t_bits_str = PG_ARGISNULL(4) ? NULL : PG_GETARG_TEXT_P(4);
+ if (PG_NARGS()>=6)
+ do_detoast = PG_GETARG_BOOL(5);
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to use raw page functions")));
+
+ if (!raw_data)
+ PG_RETURN_NULL();
+
+ /*
+ * Here we converting t_bits string back to the bits8 array,
+ * as it really is in the tuple header
+ */
+ if (t_infomask & HEAP_HASNULL)
+ {
+ int bits_str_len;
+ int bits_len;
+ char *p;
+ int off;
+ char byte = 0;
+
+ bits_len = (t_infomask2 & HEAP_NATTS_MASK) / 8 + 1;
+ if (!t_bits_str)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits argument is NULL, though we expect it to be NOT NULL and %i character long",
+ bits_len * 8)));
+
+ bits_str_len = VARSIZE(t_bits_str) - VARHDRSZ;
+ if (bits_str_len % 8)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits argument length should be multiple of eight")));
+
+ if (bits_len * 8 != bits_str_len)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("wrong t_bits argument length. Expected %i, actual is %i",
+ bits_len * 8, bits_str_len)));
+
+ t_bits = palloc(bits_len + 1);
+ p = (char *) t_bits_str + VARHDRSZ;
+ off = 0;
+
+ while (off<bits_str_len)
+ {
+ if (!(off % 8))
+ byte = 0;
+
+ if (( p[off] == '0') || (p[off] == '1'))
+ byte = byte | ( (p[off]-'0')<<off % 8);
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("illegal character '%c' in t_bits string", p[off])));
+
+ if (off % 8 == 7)
+ t_bits[off / 8] = byte;
+
+ off++;
+ }
+ }
+ else
+ {
+ if (t_bits_str)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits string is expected to be NULL, but instead it is %i bytes length",
+ VARSIZE(t_bits_str) - VARHDRSZ)));
+ }
+
+ /* Getting tuple descriptor from relation OID */
+ rel = relation_open(rel_oid, NoLock);
+ tuple_desc = CreateTupleDescCopyConstr(rel->rd_att);
+ relation_close(rel, NoLock);
+
+ /* Splitting tuple data */
+ res = split_tuple_data((char *) raw_data + VARHDRSZ,
+ VARSIZE(raw_data) - VARHDRSZ, tuple_desc,
+ t_infomask, t_infomask2, t_bits,
+ do_detoast);
+
+ PG_RETURN_ARRAYTYPE_P(res);
+}
+
+
+Datum
+split_tuple_data(char *tuple_data, uint16 tuple_data_len, TupleDesc tuple_desc,
+ uint16 t_infomask, uint16 t_infomask2, bits8 *t_bits,
+ bool do_detoast)
+{
+ ArrayBuildState *raw_attrs;
+ int nattrs;
+ int i;
+ int off;
+
+ /*
+ * Here we reimplement the basic functionality of nocachegetattr from
+ * backend/access/common/heaptuple.c witch is basically used for fetching
+ * attributes from tuple when it is not cached. We can not use
+ * nocachegetattr here directly, because we should ignore all cache
+ * optimizations and other stuff just get binary data as it is.
+ */
+
+ raw_attrs = initArrayResult(BYTEAOID,CurrentMemoryContext,0);
+ off = 0;
+ nattrs = tuple_desc->natts;
+
+ if (nattrs < (t_infomask2 & HEAP_NATTS_MASK))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: number of attributes in tuple header is greater than number of attributes in tuple descripor")));
+
+ for(i=0; i < nattrs; i++)
+ {
+ Form_pg_attribute attr;
+ Datum raw_attr = PointerGetDatum(NULL);
+ bool is_null;
+
+ attr = tuple_desc->attrs[i];
+ is_null = (t_infomask & HEAP_HASNULL) && att_isnull(i, t_bits);
+ /*
+ * Tuple header can specify less attributes then tuple descriptor
+ * as ALTER TABLE ADD COLUMN without DEFAULT keyword does not
+ * actually change tuples in pages, so attributes with numbers greater
+ * than t_infomask2 & HEAP_NATTS_MASK should be treated as NULL
+ */
+ if (i >= (t_infomask2 & HEAP_NATTS_MASK) )
+ is_null = true;
+
+ if (!is_null)
+ {
+ int len;
+ bytea *attr_data;
+
+ if (attr->attlen == -1)
+ {
+ off = att_align_pointer(off, tuple_desc->attrs[i]->attalign, -1,
+ tuple_data + off);
+ /*
+ * As VARSIZE_ANY throws an exception if it can't properly detect
+ * type of external storage in macros VARTAG_SIZE, so we repeat
+ * this check here to preform nicer error handling
+ */
+ if (VARATT_IS_1B_E(tuple_data + off) &&
+ VARTAG_EXTERNAL(tuple_data + off) != VARTAG_INDIRECT &&
+ VARTAG_EXTERNAL(tuple_data + off) != VARTAG_ONDISK)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: First byte of varlen attr seems to be corrupted")));
+ }
+
+ len = VARSIZE_ANY(tuple_data + off);
+ }
+ else
+ {
+ off = att_align_nominal(off, tuple_desc->attrs[i]->attalign);
+ len = attr->attlen;
+ }
+
+ if (tuple_data_len < off + len)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: Iterating over tuple data reached out of actual tuple size")));
+
+ attr_data = (bytea *) palloc(len + VARHDRSZ);
+ SET_VARSIZE(attr_data, len + VARHDRSZ);
+ memcpy(VARDATA(attr_data), tuple_data + off, len);
+ raw_attr = PointerGetDatum(attr_data);
+
+ if ( attr->attlen == -1 && do_detoast)
+ {
+ Datum raw_attr_copy;
+ raw_attr_copy = PointerGetDatum(
+ PG_DETOAST_DATUM_COPY(tuple_data + off));
+ pfree(attr_data);
+ raw_attr = raw_attr_copy;
+ }
+
+ off = att_addlength_pointer(off, tuple_desc->attrs[i]->attlen,
+ tuple_data + off);
+ }
+
+ raw_attrs = accumArrayResult(raw_attrs, raw_attr, is_null, BYTEAOID,
+ CurrentMemoryContext);
+ }
+
+ if (tuple_data_len != off)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("data corruption: Iterating over tuple data did not actualy reach tuple end")));
+
+ return makeArrayResult(raw_attrs, CurrentMemoryContext);
+}
diff --git a/contrib/pageinspect/pageinspect--1.3--1.4.sql b/contrib/pageinspect/pageinspect--1.3--1.4.sql
new file mode 100644
index 0000000..4b1d140
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.3--1.4.sql
@@ -0,0 +1,82 @@
+/* contrib/pageinspect/pageinspect--1.3--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via ALTER EXTENSION
+\echo Use "ALTER EXTENSION pageinspect UPDATE TO '1.4'" to load this file. \quit
+
+--
+-- heap_page_items()
+--
+DROP FUNCTION heap_page_items(bytea);
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(t_oid rel_oid, t_data bytea, t_infomask integer, t_infomask2 integer, t_bits text) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(t_oid rel_oid, t_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN rel_oid regclass, IN do_detoast bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT lp, lp_off, lp_flags, lp_len, t_xmin, t_xmax, t_field3, t_ctid, t_infomask2, t_infomask, t_hoff, t_bits, t_oid, tuple_data_split(rel_oid, t_data, t_infomask, t_infomask2, t_bits, do_detoast) as t_attrs from heap_page_items(page)'
+LANGUAGE SQL;
+
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT * from heap_page_item_attrs(page, heap_oid, false)'
+LANGUAGE SQL;
diff --git a/contrib/pageinspect/pageinspect--1.3.sql b/contrib/pageinspect/pageinspect--1.3.sql
deleted file mode 100644
index a99e058..0000000
--- a/contrib/pageinspect/pageinspect--1.3.sql
+++ /dev/null
@@ -1,189 +0,0 @@
-/* contrib/pageinspect/pageinspect--1.3.sql */
-
--- complain if script is sourced in psql, rather than via CREATE EXTENSION
-\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
-
---
--- get_raw_page()
---
-CREATE FUNCTION get_raw_page(text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page'
-LANGUAGE C STRICT;
-
-CREATE FUNCTION get_raw_page(text, text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page_fork'
-LANGUAGE C STRICT;
-
---
--- page_header()
---
-CREATE FUNCTION page_header(IN page bytea,
- OUT lsn pg_lsn,
- OUT checksum smallint,
- OUT flags smallint,
- OUT lower smallint,
- OUT upper smallint,
- OUT special smallint,
- OUT pagesize smallint,
- OUT version smallint,
- OUT prune_xid xid)
-AS 'MODULE_PATHNAME', 'page_header'
-LANGUAGE C STRICT;
-
---
--- heap_page_items()
---
-CREATE FUNCTION heap_page_items(IN page bytea,
- OUT lp smallint,
- OUT lp_off smallint,
- OUT lp_flags smallint,
- OUT lp_len smallint,
- OUT t_xmin xid,
- OUT t_xmax xid,
- OUT t_field3 int4,
- OUT t_ctid tid,
- OUT t_infomask2 integer,
- OUT t_infomask integer,
- OUT t_hoff smallint,
- OUT t_bits text,
- OUT t_oid oid)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'heap_page_items'
-LANGUAGE C STRICT;
-
---
--- bt_metap()
---
-CREATE FUNCTION bt_metap(IN relname text,
- OUT magic int4,
- OUT version int4,
- OUT root int4,
- OUT level int4,
- OUT fastroot int4,
- OUT fastlevel int4)
-AS 'MODULE_PATHNAME', 'bt_metap'
-LANGUAGE C STRICT;
-
---
--- bt_page_stats()
---
-CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
- OUT blkno int4,
- OUT type "char",
- OUT live_items int4,
- OUT dead_items int4,
- OUT avg_item_size int4,
- OUT page_size int4,
- OUT free_size int4,
- OUT btpo_prev int4,
- OUT btpo_next int4,
- OUT btpo int4,
- OUT btpo_flags int4)
-AS 'MODULE_PATHNAME', 'bt_page_stats'
-LANGUAGE C STRICT;
-
---
--- bt_page_items()
---
-CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
- OUT itemoffset smallint,
- OUT ctid tid,
- OUT itemlen smallint,
- OUT nulls bool,
- OUT vars bool,
- OUT data text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'bt_page_items'
-LANGUAGE C STRICT;
-
---
--- brin_page_type()
---
-CREATE FUNCTION brin_page_type(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'brin_page_type'
-LANGUAGE C STRICT;
-
---
--- brin_metapage_info()
---
-CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
- OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
-AS 'MODULE_PATHNAME', 'brin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- brin_revmap_data()
---
-CREATE FUNCTION brin_revmap_data(IN page bytea,
- OUT pages tid)
-RETURNS SETOF tid
-AS 'MODULE_PATHNAME', 'brin_revmap_data'
-LANGUAGE C STRICT;
-
---
--- brin_page_items()
---
-CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
- OUT itemoffset int,
- OUT blknum int,
- OUT attnum int,
- OUT allnulls bool,
- OUT hasnulls bool,
- OUT placeholder bool,
- OUT value text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'brin_page_items'
-LANGUAGE C STRICT;
-
---
--- fsm_page_contents()
---
-CREATE FUNCTION fsm_page_contents(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'fsm_page_contents'
-LANGUAGE C STRICT;
-
---
--- GIN functions
---
-
---
--- gin_metapage_info()
---
-CREATE FUNCTION gin_metapage_info(IN page bytea,
- OUT pending_head bigint,
- OUT pending_tail bigint,
- OUT tail_free_size int4,
- OUT n_pending_pages bigint,
- OUT n_pending_tuples bigint,
- OUT n_total_pages bigint,
- OUT n_entry_pages bigint,
- OUT n_data_pages bigint,
- OUT n_entries bigint,
- OUT version int4)
-AS 'MODULE_PATHNAME', 'gin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- gin_page_opaque_info()
---
-CREATE FUNCTION gin_page_opaque_info(IN page bytea,
- OUT rightlink bigint,
- OUT maxoff int4,
- OUT flags text[])
-AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
-LANGUAGE C STRICT;
-
---
--- gin_leafpage_items()
---
-CREATE FUNCTION gin_leafpage_items(IN page bytea,
- OUT first_tid tid,
- OUT nbytes int2,
- OUT tids tid[])
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'gin_leafpage_items'
-LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect--1.4.sql b/contrib/pageinspect/pageinspect--1.4.sql
new file mode 100644
index 0000000..4599fdd
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.4.sql
@@ -0,0 +1,246 @@
+/* contrib/pageinspect/pageinspect--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
+
+--
+-- get_raw_page()
+--
+CREATE FUNCTION get_raw_page(text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION get_raw_page(text, text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page_fork'
+LANGUAGE C STRICT;
+
+--
+-- page_header()
+--
+CREATE FUNCTION page_header(IN page bytea,
+ OUT lsn pg_lsn,
+ OUT checksum smallint,
+ OUT flags smallint,
+ OUT lower smallint,
+ OUT upper smallint,
+ OUT special smallint,
+ OUT pagesize smallint,
+ OUT version smallint,
+ OUT prune_xid xid)
+AS 'MODULE_PATHNAME', 'page_header'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_items()
+--
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea
+ )
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(rel_oid oid, t_data bytea, t_infomask integer, t_infomask2 integer, t_bits text) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+
+CREATE FUNCTION tuple_data_split(rel_oid oid, t_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool) RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN rel_oid regclass, IN do_detoast bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT lp, lp_off, lp_flags, lp_len, t_xmin, t_xmax, t_field3, t_ctid, t_infomask2, t_infomask, t_hoff, t_bits, t_oid, tuple_data_split(rel_oid, t_data, t_infomask, t_infomask2, t_bits, do_detoast) as t_attrs from heap_page_items(page)'
+LANGUAGE SQL;
+
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN heap_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record
+AS 'SELECT * from heap_page_item_attrs(page, heap_oid, false)'
+LANGUAGE SQL;
+
+--
+-- bt_metap()
+--
+CREATE FUNCTION bt_metap(IN relname text,
+ OUT magic int4,
+ OUT version int4,
+ OUT root int4,
+ OUT level int4,
+ OUT fastroot int4,
+ OUT fastlevel int4)
+AS 'MODULE_PATHNAME', 'bt_metap'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_stats()
+--
+CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
+ OUT blkno int4,
+ OUT type "char",
+ OUT live_items int4,
+ OUT dead_items int4,
+ OUT avg_item_size int4,
+ OUT page_size int4,
+ OUT free_size int4,
+ OUT btpo_prev int4,
+ OUT btpo_next int4,
+ OUT btpo int4,
+ OUT btpo_flags int4)
+AS 'MODULE_PATHNAME', 'bt_page_stats'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_items()
+--
+CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
+ OUT itemoffset smallint,
+ OUT ctid tid,
+ OUT itemlen smallint,
+ OUT nulls bool,
+ OUT vars bool,
+ OUT data text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'bt_page_items'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_type()
+--
+CREATE FUNCTION brin_page_type(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'brin_page_type'
+LANGUAGE C STRICT;
+
+--
+-- brin_metapage_info()
+--
+CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
+ OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
+AS 'MODULE_PATHNAME', 'brin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- brin_revmap_data()
+--
+CREATE FUNCTION brin_revmap_data(IN page bytea,
+ OUT pages tid)
+RETURNS SETOF tid
+AS 'MODULE_PATHNAME', 'brin_revmap_data'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_items()
+--
+CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
+ OUT itemoffset int,
+ OUT blknum int,
+ OUT attnum int,
+ OUT allnulls bool,
+ OUT hasnulls bool,
+ OUT placeholder bool,
+ OUT value text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'brin_page_items'
+LANGUAGE C STRICT;
+
+--
+-- fsm_page_contents()
+--
+CREATE FUNCTION fsm_page_contents(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'fsm_page_contents'
+LANGUAGE C STRICT;
+
+--
+-- GIN functions
+--
+
+--
+-- gin_metapage_info()
+--
+CREATE FUNCTION gin_metapage_info(IN page bytea,
+ OUT pending_head bigint,
+ OUT pending_tail bigint,
+ OUT tail_free_size int4,
+ OUT n_pending_pages bigint,
+ OUT n_pending_tuples bigint,
+ OUT n_total_pages bigint,
+ OUT n_entry_pages bigint,
+ OUT n_data_pages bigint,
+ OUT n_entries bigint,
+ OUT version int4)
+AS 'MODULE_PATHNAME', 'gin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_page_opaque_info()
+--
+CREATE FUNCTION gin_page_opaque_info(IN page bytea,
+ OUT rightlink bigint,
+ OUT maxoff int4,
+ OUT flags text[])
+AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_leafpage_items()
+--
+CREATE FUNCTION gin_leafpage_items(IN page bytea,
+ OUT first_tid tid,
+ OUT nbytes int2,
+ OUT tids tid[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'gin_leafpage_items'
+LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect.control b/contrib/pageinspect/pageinspect.control
index a9dab33..68c7d61 100644
--- a/contrib/pageinspect/pageinspect.control
+++ b/contrib/pageinspect/pageinspect.control
@@ -1,5 +1,5 @@
# pageinspect extension
comment = 'inspect the contents of database pages at a low level'
-default_version = '1.3'
+default_version = '1.4'
module_pathname = '$libdir/pageinspect'
relocatable = true
diff --git a/doc/src/sgml/pageinspect.sgml b/doc/src/sgml/pageinspect.sgml
index b95cc81..7676dfa 100644
--- a/doc/src/sgml/pageinspect.sgml
+++ b/doc/src/sgml/pageinspect.sgml
@@ -93,19 +93,170 @@ test=# SELECT * FROM page_header(get_raw_page('pg_class', 0));
<listitem>
<para>
<function>heap_page_items</function> shows all line pointers on a heap
- page. For those line pointers that are in use, tuple headers are also
- shown. All tuples are shown, whether or not the tuples were visible to
- an MVCC snapshot at the time the raw page was copied.
+ page. For those line pointers that are in use,
+ <function>heap_page_items</function> also shows tuple headers and raw
+ tuple data. All tuples are shown, whether or not the tuples were visible
+ to an MVCC snapshot at the time the raw page was copied.
</para>
<para>
A heap page image obtained with <function>get_raw_page</function> should
be passed as argument. For example:
<screen>
-test=# SELECT * FROM heap_page_items(get_raw_page('pg_class', 0));
+
+test=# select * from heap_page_items(get_raw_page('pg_range', 0));
+ lp | lp_off | lp_flags | lp_len | t_xmin | t_xmax | t_field3 | t_ctid | t_infomask2 | t_infomask | t_hoff | t_bits | t_oid | t_data
+----+--------+----------+--------+--------+--------+----------+--------+-------------+------------+--------+--------+-------+----------------------------------------------------
+ 1 | 8144 | 1 | 48 | 1 | 0 | 0 | (0,1) | 6 | 2304 | 24 | | | \x400f00001700000000000000ba0700004a0f0000520f0000
+ 2 | 8096 | 1 | 48 | 1 | 0 | 0 | (0,2) | 6 | 2304 | 24 | | | \x420f0000a406000000000000350c000000000000540f0000
+ 3 | 8048 | 1 | 48 | 1 | 0 | 0 | (0,3) | 6 | 2304 | 24 | | | \x440f00005a04000000000000380c000000000000590f0000
+ 4 | 8000 | 1 | 48 | 1 | 0 | 0 | (0,4) | 6 | 2304 | 24 | | | \x460f0000a004000000000000370c0000000000005a0f0000
+ 5 | 7952 | 1 | 48 | 1 | 0 | 0 | (0,5) | 6 | 2304 | 24 | | | \x480f00003a04000000000000320c00004b0f0000550f0000
+ 6 | 7904 | 1 | 48 | 1 | 0 | 0 | (0,6) | 6 | 2304 | 24 | | | \x560f00001400000000000000340c0000580f0000530f0000
+(6 rows)
+
+</screen>
+ </para>
+ <para>
+ Attributes returned by <function>heap_page_items</function> are values
+ from tuple header, and from page item identifiers that points to the
+ tuple. Detailed description of these attributes can be found at
+ <xref linkend="storage-page-layout">, <xref linkend="ddl-system-columns">
+ and source in code: <filename>src/include/storage/itemid.h</>,
+ <filename>src/include/access/htup_details.h</>.
+ </para>
+ <para>
+ Please notice that <function>t_bits</function> in tuple header structure
+ is a binary bitmap, but <function>heap_page_items</function> returns
+ <function>t_bits</function> as human readable text representation of
+ original <function>t_bits</function> bitmap.
+ </para>
+ <para>
+ Last attribute <function>t_data</function> contains raw tuple data stored
+ as <function>bytea</function> binary.
+ </para>
+ </listitem>
+ </varlistentry>
+
+
+ <varlistentry>
+ <term>
+ <function>tuple_data_split(rel_oid, t_data bytea, t_infomask integer, t_infomask2 integer, t_bits text) returs bytea[]</function>
+ <indexterm>
+ <primary>tuple_data_split</primary>
+ </indexterm>
+ </term>
+ <term>
+ <function>tuple_data_split(rel_oid, t_data bytea, t_infomask integer, t_infomask2 integer, t_bits text, do_detoast bool) returs bytea[]</function>
+ </term>
+ <listitem>
+ <para>
+ <function>tuple_data_split</function> splits tuple data into attributes in
+ the same way as <function>nocachegetattr</function> function do it in
+ postgres internals. Returns array of bytea.
+<screen>
+test=# SELECT tuple_data_split('pg_range'::regclass, t_data, t_infomask, t_infomask2, t_bits) FROM heap_page_items(get_raw_page('pg_range', 0));
+ tuple_data_split
+---------------------------------------------------------------------------------------
+ {"\\x400f0000","\\x17000000","\\x00000000","\\xba070000","\\x4a0f0000","\\x520f0000"}
+ {"\\x420f0000","\\xa4060000","\\x00000000","\\x350c0000","\\x00000000","\\x540f0000"}
+ {"\\x440f0000","\\x5a040000","\\x00000000","\\x380c0000","\\x00000000","\\x590f0000"}
+ {"\\x460f0000","\\xa0040000","\\x00000000","\\x370c0000","\\x00000000","\\x5a0f0000"}
+ {"\\x480f0000","\\x3a040000","\\x00000000","\\x320c0000","\\x4b0f0000","\\x550f0000"}
+ {"\\x560f0000","\\x14000000","\\x00000000","\\x340c0000","\\x580f0000","\\x530f0000"}
+(6 rows)
</screen>
- See <filename>src/include/storage/itemid.h</> and
- <filename>src/include/access/htup_details.h</> for explanations of the fields
- returned.
+ </para>
+
+ <para>
+ <function>tuple_data_split</function> takes OID of the relation whose
+ tuple data we going to split as a first argument. The rest arguments
+ should contains <function>t_data</function>,
+ <function>t_infomask</function>, <function>t_infomask2</function> and
+ <function>t_bits</function> as they are returned by
+ <function>heap_page_items</function>. There is also one optional argument
+ <function>do_detoast</function>, if it is set to true,
+ <function>tuple_data_split</function> will deTOAST and uncompress
+ attribute values if they were TOASTed or compressed. See
+ <xref linkend="storage-toast">
+ </para>
+ <para>
+ In most cases you will not need <function>tuple_data_split</function>
+ itself, consider using <function>heap_page_item_attrs</function> for
+ viewing page data with split attributes.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <function>heap_page_item_attrs(page bytea, rel_oid regclass) returns record</function>
+ <indexterm>
+ <primary>heap_page_item_attrs</primary>
+ </indexterm>
+ </term>
+ <term>
+ <function>heap_page_item_attrs(page bytea, rel_oid regclass, do_detoast bool) returns record</function>
+ </term>
+
+ <listitem>
+ <para>
+ <function>heap_page_item_attrs</function> is actually a clone of
+ <function>heap_page_items</function>, with one difference:
+ <function>heap_page_item_attrs</function> returns array of raw values of
+ tuple attributes instead of one piece of raw tuple data. All other return
+ columns are same as in <function>heap_page_items</function>.
+<screen>
+test=# select lp, t_attrs from heap_page_item_attrs(get_raw_page('pg_range',0),'pg_range'::regclass);
+ lp | t_attrs
+----+---------------------------------------------------------------------------------------
+ 1 | {"\\x400f0000","\\x17000000","\\x00000000","\\xba070000","\\x4a0f0000","\\x520f0000"}
+ 2 | {"\\x420f0000","\\xa4060000","\\x00000000","\\x350c0000","\\x00000000","\\x540f0000"}
+ 3 | {"\\x440f0000","\\x5a040000","\\x00000000","\\x380c0000","\\x00000000","\\x590f0000"}
+ 4 | {"\\x460f0000","\\xa0040000","\\x00000000","\\x370c0000","\\x00000000","\\x5a0f0000"}
+ 5 | {"\\x480f0000","\\x3a040000","\\x00000000","\\x320c0000","\\x4b0f0000","\\x550f0000"}
+ 6 | {"\\x560f0000","\\x14000000","\\x00000000","\\x340c0000","\\x580f0000","\\x530f0000"}
+(6 rows)
+</screen>
+ </para>
+ <para>
+ <function>heap_page_item_attrs</function> takes following argiments:
+ <informaltable>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Argument</entry>
+ <entry>Type</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry><structfield>page</structfield></entry>
+ <entry><type>bytea</type></entry>
+ <entry>page raw data, usually returned by <function>get_raw_page</function></entry>
+ </row>
+
+ <row>
+ <entry><structfield>rel_oid</structfield></entry>
+ <entry><type>oid</type></entry>
+ <entry>OID of the relation, whose page we want to parse</entry>
+ </row>
+
+ <row>
+ <entry><structfield>do_detoast</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry>optional attribute, if <function>do_detoast</function> is set to
+ true, <function>heap_page_item_attrs</function> will not only split
+ tuple data into attributes, but also try to deTOAST and uncompress
+ attribute value if it was TOASTed or compressed.
+ See <xref linkend="storage-toast">
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </informaltable>
+
</para>
</listitem>
</varlistentry>
Re: pageinspect patch, for showing tuple data
On Thu, Oct 1, 2015 at 8:13 PM, Nikolay Shaplov wrote:
В письме от 30 сентября 2015 13:49:00 пользователь Michael Paquier
написал:
- errmsg("input page too small (%d
bytes)",
raw_page_size)));
+ errmsg("input page too small (%d
bytes)", raw_page_size)));
Please be careful of spurious diffs. Those just make the life of
committers
more difficult than it already is.
Oh, that's not diff. That's I've fixed indent on the code I did not write
:-)
pgindent would have taken care of that if needed. There is no need to add
noise in the code for this patch.
+ <para> + General idea about output columns: <function>lp_*</function> attributes + are about where tuple is located inside the page; + <function>t_xmin</function>, <function>t_xmax</function>, + <function>t_field3</function>, <function>t_ctid</function> are
about
+ visibility of this tuplue inside or outside of the transaction; + <function>t_infomask2</function>, <function>t_infomask</function>
has
some + information about properties of attributes stored in tuple data. + <function>t_hoff</function> sais where tuple data begins and + <function>t_bits</function> sais which attributes are NULL and
which
are + not. Please notice that t_bits here is not an actual value that is stored + in tuple data, but it's text representation with '0' and '1' charactrs. + </para> I would remove that as well. htup_details.h contains all this
information.
Made it even more shorter. Just links and comments about representation of
t_bits.
I would completely remove this part.
There also was a bug in original pageinspect, that did not get t_bits
right
when there was OID in the tuple. I've fixed it too.
Aha. Good catch! By looking at HeapTupleHeaderGetOid if the tuple has an
OID it will be stored at the end of HeapTupleHeader and t_hoff includes it,
so bits_len is definitely larger of 4 bytes should an OID be present.
if (tuphdr->t_infomask & HEAP_HASNULL)
{
- bits_len = tuphdr->t_hoff -
- offsetof(HeapTupleHeaderData, t_bits);
+ int bits_len =
+ ((tuphdr->t_infomask2 & HEAP_NATTS_MASK)/8 +
1)*8; values[11] = CStringGetTextDatum(
- bits_to_text(tuphdr->t_bits,
bits_len * 8));
+ bits_to_text(tuphdr->t_bits,
bits_len));
}
And here is what you are referring to in your patch. I think that we should
instead check for HEAP_HASOID and reduce bits_len by the size of Oid should
one be present. As this is a bug that applies to all the existing versions
of Postgres it would be good to extract it as a separate patch and then
apply your own patch on top of it instead of including in your feature.
Attached is a patch, this should be applied down to 9.0 I guess. Could you
rebase your patch on top of it?Here is next patch in attachment.
Cool, thanks.
-test=# SELECT * FROM heap_page_items(get_raw_page('pg_class', 0));
+
+test=# select * from heap_page_items(get_raw_page('pg_range', 0));
This example in the docs is far too long in character length... We should
get that reduced.+ Please notice that <function>t_bits</function> in tuple header
structure
+ is a binary bitmap, but <function>heap_page_items</function> returns
+ <function>t_bits</function> as human readable text representation of
+ original <function>t_bits</function> bitmap.
This had better remove the mention to "please notice". Still as this is
already described in htup_details.h there is no real point to describe it
again here: that's a bitmap of NULLs.
--
MichaelAttachments:
0001-Fix-overestimated-size-of-t_bits-in-pageinspect.patchtext/x-patch; charset=US-ASCII; name=0001-Fix-overestimated-size-of-t_bits-in-pageinspect.patchDownload
From 3a21fd817748b8001e91159c3f2a557088b4fa26 Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@otacoo.com>
Date: Fri, 2 Oct 2015 12:58:13 +0900
Subject: [PATCH] Fix overestimated size of t_bits in pageinspect
In a tuple, an object-id field is added at the end of HeapTupleHeader
and the size of he header is updated to include it. However heap_page_items
in pageinspect ignored that fact and overestimated the size of bitmap of
NULLs by 4 bytes should an OID be defined in this tuple.
Per report from Nikolay Sharplov, for a problem found while working on
a new feature for pageinspect, and patch by Michael Paquier.
---
contrib/pageinspect/heapfuncs.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 8d1666c..031d455 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -154,7 +154,6 @@ heap_page_items(PG_FUNCTION_ARGS)
lp_offset + lp_len <= raw_page_size)
{
HeapTupleHeader tuphdr;
- int bits_len;
/* Extract information from the tuple header */
@@ -180,9 +179,15 @@ heap_page_items(PG_FUNCTION_ARGS)
{
if (tuphdr->t_infomask & HEAP_HASNULL)
{
+ int bits_len;
+
bits_len = tuphdr->t_hoff -
offsetof(HeapTupleHeaderData, t_bits);
+ /* ignore OID field of tuple if present */
+ if (tuphdr->t_infomask & HEAP_HASOID)
+ bits_len -= sizeof(Oid);
+
values[11] = CStringGetTextDatum(
bits_to_text(tuphdr->t_bits, bits_len * 8));
}
--
2.6.0
Re: pageinspect patch, for showing tuple data
В письме от 2 октября 2015 13:10:22 Вы написали:
There also was a bug in original pageinspect, that did not get t_bits
right
when there was OID in the tuple. I've fixed it too.
Aha. Good catch! By looking at HeapTupleHeaderGetOid if the tuple has an OID it will be stored at the end of HeapTupleHeader and t_hoff includes it, so bits_len is definitely larger of 4 bytes should an OID be present. if (tuphdr->t_infomask & HEAP_HASNULL) { - bits_len = tuphdr->t_hoff - - offsetof(HeapTupleHeaderData, t_bits); + int bits_len = + ((tuphdr->t_infomask2 & HEAP_NATTS_MASK)/8 + 1)*8;values[11] = CStringGetTextDatum( - bits_to_text(tuphdr->t_bits, bits_len * 8)); + bits_to_text(tuphdr->t_bits, bits_len)); } And here is what you are referring to in your patch. I think that we should instead check for HEAP_HASOID and reduce bits_len by the size of Oid should one be present. As this is a bug that applies to all the existing versions of Postgres it would be good to extract it as a separate patch and then apply your own patch on top of it instead of including in your feature. Attached is a patch, this should be applied down to 9.0 I guess. Could you rebase your patch on top of it?
No we should not do it, because after t_bits there always goes padding bytes.
So OID or the top of tuple data is properly aligned. So we should not use
t_hoff for determinating t_bit's size at all.
Here is an example. I create a table with 10 columns and OID. (ten is greater
then 8, so there should be two bytes of t_bits data)
create table test3 (a1 int, a2 int, a3 int, a4 int,a5 int,a6 int, a7 int,a8 int, a9 int, a10 int) with oids;
insert into test3 VALUES
(1,2,3,4,5,6,7,8,null,10);
With your patch we get
test=# select lp, t_bits, t_data from heap_page_items(get_raw_page('test3', 0));
lp | t_bits | t_data
----+------------------------------------------+----------------------------------------------------------------------------
1 | 1111111101000000000000000000000000000000 | \x01000000020000000300000004000000050000000600000007000000080000000a000000
(1 row)
here we get 40 bits of t_bits.
With my way to calculate t_bits length we get
test=# select lp, t_bits, t_data from heap_page_items(get_raw_page('test3', 0));
lp | t_bits | t_data
----+------------------+----------------------------------------------------------------------------
1 | 1111111101000000 | \x01000000020000000300000004000000050000000600000007000000080000000a000000
(1 row)
16 bits as expected.
So I would keep my version of bits_len calculation
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
В письме от 2 октября 2015 13:10:22 пользователь Michael Paquier написал:
+ <para> + General idea about output columns: <function>lp_*</function> attributes + are about where tuple is located inside the page; + <function>t_xmin</function>, <function>t_xmax</function>, + <function>t_field3</function>, <function>t_ctid</function> areabout
+ visibility of this tuplue inside or outside of the transaction; + <function>t_infomask2</function>, <function>t_infomask</function>has
some + information about properties of attributes stored in tuple data. + <function>t_hoff</function> sais where tuple data begins and + <function>t_bits</function> sais which attributes are NULL andwhich
are + not. Please notice that t_bits here is not an actual value that is stored + in tuple data, but it's text representation with '0' and '1' charactrs. + </para> I would remove that as well. htup_details.h contains all thisinformation.
Made it even more shorter. Just links and comments about representation of
t_bits.I would completely remove this part.
Michael my hand would not do it. I've been working as a lecturer for six
years. If I want to pass an information in a comfortable way to reader, there
should go some binding phrase. It may be very vague, but it should outline
(may be in a very brief way, but still outline) an information that would be
found if he follows the links.
If we just give links "knowledge flow" will be uncomfortable for person who
reads it.
-test=# SELECT * FROM heap_page_items(get_raw_page('pg_class', 0)); + +test=# select * from heap_page_items(get_raw_page('pg_range', 0)); This example in the docs is far too long in character length... We should get that reduced.
May be should do
\x and limit 1 like this:
test=# select * from heap_page_items(get_raw_page('pg_range', 0)) limit 1;
-[ RECORD 1 ]---------------------------------------------------
lp | 1
lp_off | 8144
lp_flags | 1
lp_len | 48
t_xmin | 1
t_xmax | 0
t_field3 | 0
t_ctid | (0,1)
t_infomask2 | 6
t_infomask | 2304
t_hoff | 24
t_bits |
t_oid |
t_data | \x400f00001700000000000000ba0700004a0f0000520f0000
????
+ Please notice that <function>t_bits</function> in tuple header structure + is a binary bitmap, but <function>heap_page_items</function> returns + <function>t_bits</function> as human readable text representation of + original <function>t_bits</function> bitmap. This had better remove the mention to "please notice". Still as this is already described in htup_details.h there is no real point to describe it again here: that's a bitmap of NULLs.
heap_page_items returns text(!) as t_bits. This is unexpected. This is not
described in page layout documentation. We should tell about it here
explicitly.
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
So what's next?
We need something else to discuss?
We need somebody else's opinion to rule this out?
Or it's ready to commit, and just not marked this way?
I am going to make report based on this patch in Vienna. It would be
nice to have it committed until then :)
On 02.10.2015 07:10, Michael Paquier wrote:
On Thu, Oct 1, 2015 at 8:13 PM, Nikolay Shaplov wrote:
В письме от 30 сентября 2015 13:49:00 пользователь Michael Paquier
написал:
- errmsg("input page too small (%d
bytes)",
raw_page_size)));
+ errmsg("input page too small (%d
bytes)", raw_page_size)));
Please be careful of spurious diffs. Those just make the life ofcommitters
more difficult than it already is.
Oh, that's not diff. That's I've fixed indent on the code I did not write
:-)
pgindent would have taken care of that if needed. There is no need to add
noise in the code for this patch.+ <para> + General idea about output columns: <function>lp_*</function> attributes + are about where tuple is located inside the page; + <function>t_xmin</function>, <function>t_xmax</function>, + <function>t_field3</function>, <function>t_ctid</function> areabout
+ visibility of this tuplue inside or outside of the transaction; + <function>t_infomask2</function>, <function>t_infomask</function>has
some + information about properties of attributes stored in tuple data. + <function>t_hoff</function> sais where tuple data begins and + <function>t_bits</function> sais which attributes are NULL andwhich
are + not. Please notice that t_bits here is not an actual value that is stored + in tuple data, but it's text representation with '0' and '1' charactrs. + </para> I would remove that as well. htup_details.h contains all thisinformation.
Made it even more shorter. Just links and comments about representation of
t_bits.I would completely remove this part.
There also was a bug in original pageinspect, that did not get t_bits
right
when there was OID in the tuple. I've fixed it too.
Aha. Good catch! By looking at HeapTupleHeaderGetOid if the tuple has an OID it will be stored at the end of HeapTupleHeader and t_hoff includes it, so bits_len is definitely larger of 4 bytes should an OID be present. if (tuphdr->t_infomask & HEAP_HASNULL) { - bits_len = tuphdr->t_hoff - - offsetof(HeapTupleHeaderData, t_bits); + int bits_len = + ((tuphdr->t_infomask2 & HEAP_NATTS_MASK)/8 + 1)*8;values[11] = CStringGetTextDatum( - bits_to_text(tuphdr->t_bits, bits_len * 8)); + bits_to_text(tuphdr->t_bits, bits_len)); } And here is what you are referring to in your patch. I think that we should instead check for HEAP_HASOID and reduce bits_len by the size of Oid should one be present. As this is a bug that applies to all the existing versions of Postgres it would be good to extract it as a separate patch and then apply your own patch on top of it instead of including in your feature. Attached is a patch, this should be applied down to 9.0 I guess. Could you rebase your patch on top of it?Here is next patch in attachment.
Cool, thanks.
-test=# SELECT * FROM heap_page_items(get_raw_page('pg_class', 0)); + +test=# select * from heap_page_items(get_raw_page('pg_range', 0)); This example in the docs is far too long in character length... We should get that reduced.+ Please notice that <function>t_bits</function> in tuple header structure + is a binary bitmap, but <function>heap_page_items</function> returns + <function>t_bits</function> as human readable text representation of + original <function>t_bits</function> bitmap. This had better remove the mention to "please notice". Still as this is already described in htup_details.h there is no real point to describe it again here: that's a bitmap of NULLs.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
On Sat, Oct 17, 2015 at 5:15 AM, Nikolay Shaplov
<n.shaplov@postgrespro.ru> wrote:
So what's next?
Wait and see a bit.
We need something else to discuss?
We need somebody else's opinion to rule this out?
The spec of the patch looks clear to me.
Or it's ready to commit, and just not marked this way?
No, I don't think we have reached this state yet.
I am going to make report based on this patch in Vienna. It would be
nice to have it committed until then :)
This is registered in this November's CF and the current one is not
completely wrapped up, so I haven't rushed into looking at it.
Regards,
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
On Sat, Oct 17, 2015 at 1:48 AM, Michael Paquier wrote:
On Sat, Oct 17, 2015 at 5:15 AM, Nikolay Shaplov wrote:
Or it's ready to commit, and just not marked this way?
No, I don't think we have reached this state yet.
I am going to make report based on this patch in Vienna. It would be
nice to have it committed until then :)This is registered in this November's CF and the current one is not
completely wrapped up, so I haven't rushed into looking at it.
So, I have finally been able to look at this patch in more details,
resulting in the attached. I noticed a couple of things that should be
addressed, mainly:
- addition of a new routine text_to_bits to perform the reverse
operation of bits_to_text. This was previously part of
tuple_data_split, I think that it deserves its own function.
- split_tuple_data should be static
- t_bits_str should not be a text, rather a char* fetched using
text_to_cstring(PG_GETARG_TEXT_PP(4)). This way there is no need to
perform extra calculations with VARSIZE and VARHDRSZ
- split_tuple_data can directly use the relation OID instead of the
tuple descriptor of the relation
- t_bits was leaking memory. For correctness I think that it is better
to free it after calling split_tuple_data.
- PG_DETOAST_DATUM_COPY allocates some memory, this was leaking as
well in raw_attr actually. I refactored the code such as a bytea* is
used and always freed when allocated to avoid leaks. Removing raw_attr
actually simplified the code a bit.
- I simplified the docs, that was largely too verbose in my opinion.
- Instead of using VARATT_IS_1B_E and VARTAG_EXTERNAL, using
VARATT_IS_EXTERNAL and VARATT_IS_EXTERNAL_ONDISK seems more adapted to
me, those other ones are much more low-level and not really spread in
the backend code.
- Found some typos in the code, the docs and some comments. I reworked
the error messages as well.
- The code format was not really in line with the project guidelines.
I fixed that as well.
- I removed heap_page_item_attrs for now to get this patch in a
bare-bone state. Though I would not mind if this is re-added (I
personally don't think that's much necessary in the module
actually...).
- The calculation of the length of t_bits using HEAP_NATTS_MASK is
more correct as you mentioned earlier, so I let it in its state.
That's actually more accurate for error handling as well.
That's everything I recall I have. How does this look?
--
Michael
Attachments:
20151028_pageinspect_1_4.patchapplication/x-patch; name=20151028_pageinspect_1_4.patchDownload
diff --git a/contrib/pageinspect/Makefile b/contrib/pageinspect/Makefile
index aec5258..91ab119 100644
--- a/contrib/pageinspect/Makefile
+++ b/contrib/pageinspect/Makefile
@@ -5,9 +5,9 @@ OBJS = rawpage.o heapfuncs.o btreefuncs.o fsmfuncs.o \
brinfuncs.o ginfuncs.o $(WIN32RES)
EXTENSION = pageinspect
-DATA = pageinspect--1.3.sql pageinspect--1.2--1.3.sql \
- pageinspect--1.1--1.2.sql pageinspect--1.0--1.1.sql \
- pageinspect--unpackaged--1.0.sql
+DATA = pageinspect--1.4.sql pageinspect--1.3--1.4.sql \
+ pageinspect--1.2--1.3.sql pageinspect--1.1--1.2.sql \
+ pageinspect--1.0--1.1.sql pageinspect--unpackaged--1.0.sql
PGFILEDESC = "pageinspect - functions to inspect contents of database pages"
ifdef USE_PGXS
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 8d1666c..15fd7f1 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -27,8 +27,11 @@
#include "access/htup_details.h"
#include "funcapi.h"
-#include "utils/builtins.h"
+#include "catalog/pg_type.h"
#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
+#include "utils/rel.h"
/*
@@ -55,6 +58,42 @@ bits_to_text(bits8 *bits, int len)
/*
+ * text_to_bits
+ *
+ * Converts a c-string representation of bits into a bits8-array. This is
+ * the reverse operation of previous routine.
+ */
+static bits8 *
+text_to_bits(char *str, int len)
+{
+ bits8 *bits;
+ int off = 0;
+ char byte = 0;
+
+ bits = palloc(len + 1);
+
+ while (off < len)
+ {
+ if (off % 8 == 0)
+ byte = 0;
+
+ if ((str[off] == '0') || (str[off] == '1'))
+ byte = byte | ((str[off] - '0') << off % 8);
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("illegal character '%c' in t_bits string", str[off])));
+
+ if (off % 8 == 7)
+ bits[off / 8] = byte;
+
+ off++;
+ }
+
+ return bits;
+}
+
+/*
* heap_page_items
*
* Allows inspection of line pointers and tuple headers of a heap page.
@@ -122,8 +161,8 @@ heap_page_items(PG_FUNCTION_ARGS)
HeapTuple resultTuple;
Datum result;
ItemId id;
- Datum values[13];
- bool nulls[13];
+ Datum values[14];
+ bool nulls[14];
uint16 lp_offset;
uint16 lp_flags;
uint16 lp_len;
@@ -154,7 +193,8 @@ heap_page_items(PG_FUNCTION_ARGS)
lp_offset + lp_len <= raw_page_size)
{
HeapTupleHeader tuphdr;
- int bits_len;
+ bytea *tuple_data_bytea;
+ int tuple_data_len;
/* Extract information from the tuple header */
@@ -168,6 +208,14 @@ heap_page_items(PG_FUNCTION_ARGS)
values[9] = UInt32GetDatum(tuphdr->t_infomask);
values[10] = UInt8GetDatum(tuphdr->t_hoff);
+ /* Copy raw tuple data into bytea attribute */
+ tuple_data_len = lp_len - tuphdr->t_hoff;
+ tuple_data_bytea = (bytea *) palloc(tuple_data_len + VARHDRSZ);
+ SET_VARSIZE(tuple_data_bytea, tuple_data_len + VARHDRSZ);
+ memcpy(VARDATA(tuple_data_bytea), (char *) tuphdr + tuphdr->t_hoff,
+ tuple_data_len);
+ values[13] = PointerGetDatum(tuple_data_bytea);
+
/*
* We already checked that the item is completely within the raw
* page passed to us, with the length given in the line pointer.
@@ -180,11 +228,11 @@ heap_page_items(PG_FUNCTION_ARGS)
{
if (tuphdr->t_infomask & HEAP_HASNULL)
{
- bits_len = tuphdr->t_hoff -
- offsetof(HeapTupleHeaderData, t_bits);
+ int bits_len =
+ ((tuphdr->t_infomask2 & HEAP_NATTS_MASK) / 8 + 1) * 8;
values[11] = CStringGetTextDatum(
- bits_to_text(tuphdr->t_bits, bits_len * 8));
+ bits_to_text(tuphdr->t_bits, bits_len));
}
else
nulls[11] = true;
@@ -208,7 +256,7 @@ heap_page_items(PG_FUNCTION_ARGS)
*/
int i;
- for (i = 4; i <= 12; i++)
+ for (i = 4; i <= 13; i++)
nulls[i] = true;
}
@@ -223,3 +271,205 @@ heap_page_items(PG_FUNCTION_ARGS)
else
SRF_RETURN_DONE(fctx);
}
+
+/*
+ * split_tuple_data
+ *
+ * Split raw tuple data taken directly from a page into an array of bytea
+ * elements. This routine does a lookup on NULL balues and creates array
+ * elements accordindly. This is a simplified reimplementation of
+ * nocachegetattr() in heaptuple.c simplified for educational purposes.
+ */
+static Datum
+split_tuple_data(Oid relid, char *tupdata,
+ uint16 tupdata_len, uint16 t_infomask,
+ uint16 t_infomask2, bits8 *t_bits,
+ bool do_detoast)
+{
+ ArrayBuildState *raw_attrs;
+ int nattrs;
+ int i;
+ int off = 0;
+ Relation rel;
+ TupleDesc tupdesc;
+
+ /* Get tuple descriptor from relation OID */
+ rel = relation_open(relid, NoLock);
+ tupdesc = CreateTupleDescCopyConstr(rel->rd_att);
+ relation_close(rel, NoLock);
+
+ raw_attrs = initArrayResult(BYTEAOID, CurrentMemoryContext, false);
+ nattrs = tupdesc->natts;
+
+ if (nattrs < (t_infomask2 & HEAP_NATTS_MASK))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("number of attributes in tuple header is greater than number of attributes in tuple descriptor")));
+
+ for (i = 0; i < nattrs; i++)
+ {
+ Form_pg_attribute attr;
+ bool is_null;
+ bytea *attr_data = NULL;
+
+ attr = tupdesc->attrs[i];
+ is_null = (t_infomask & HEAP_HASNULL) && att_isnull(i, t_bits);
+
+ /*
+ * Tuple header can specify less attributes than tuple descriptor
+ * as ALTER TABLE ADD COLUMN without DEFAULT keyword does not
+ * actually change tuples in pages, so attributes with numbers greater
+ * than (t_infomask2 & HEAP_NATTS_MASK) should be treated as NULL.
+ */
+ if (i >= (t_infomask2 & HEAP_NATTS_MASK))
+ is_null = true;
+
+ if (!is_null)
+ {
+ int len;
+
+ if (attr->attlen == -1)
+ {
+ off = att_align_pointer(off, tupdesc->attrs[i]->attalign, -1,
+ tupdata + off);
+ /*
+ * As VARSIZE_ANY throws an exception if it can't properly detect
+ * the type of external storage in macros VARTAG_SIZE, this check
+ * is repeated to have a nicer error handling.
+ */
+ if (VARATT_IS_EXTERNAL(tupdata + off) &&
+ !VARATT_IS_EXTERNAL_ONDISK(tupdata + off) &&
+ !VARATT_IS_EXTERNAL_INDIRECT(tupdata + off))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("first byte of varlen attribute is incorrect for attribute %d", i)));
+
+ len = VARSIZE_ANY(tupdata + off);
+ }
+ else
+ {
+ off = att_align_nominal(off, tupdesc->attrs[i]->attalign);
+ len = attr->attlen;
+ }
+
+ if (tupdata_len < off + len)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("unexpected end of tuple data")));
+
+ if (attr->attlen == -1 && do_detoast)
+ attr_data = DatumGetByteaPCopy(tupdata + off);
+ else
+ {
+ attr_data = (bytea *) palloc(len + VARHDRSZ);
+ SET_VARSIZE(attr_data, len + VARHDRSZ);
+ memcpy(VARDATA(attr_data), tupdata + off, len);
+ }
+
+ off = att_addlength_pointer(off, tupdesc->attrs[i]->attlen,
+ tupdata + off);
+ }
+
+ raw_attrs = accumArrayResult(raw_attrs, PointerGetDatum(attr_data),
+ is_null, BYTEAOID, CurrentMemoryContext);
+ if (attr_data)
+ pfree(attr_data);
+ }
+
+ if (tupdata_len != off)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("end of tuple reached without looking at all its data")));
+
+ return makeArrayResult(raw_attrs, CurrentMemoryContext);
+}
+
+/*
+ * tuple_data_split
+ *
+ * Split raw tuple data taken directly from page into distinct elements
+ * taking into account null values.
+ */
+PG_FUNCTION_INFO_V1(tuple_data_split);
+
+Datum
+tuple_data_split(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ bytea *raw_data;
+ uint16 t_infomask;
+ uint16 t_infomask2;
+ char *t_bits_str;
+ bool do_detoast = false;
+ bits8 *t_bits = NULL;
+ Datum res;
+
+ relid = PG_GETARG_OID(0);
+ raw_data = PG_ARGISNULL(1) ? NULL : PG_GETARG_BYTEA_P(1);
+ t_infomask = PG_GETARG_INT16(2);
+ t_infomask2 = PG_GETARG_INT16(3);
+ t_bits_str = PG_ARGISNULL(4) ? NULL :
+ text_to_cstring(PG_GETARG_TEXT_PP(4));
+
+ if (PG_NARGS() >= 6)
+ do_detoast = PG_GETARG_BOOL(5);
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to use raw page functions")));
+
+ if (!raw_data)
+ PG_RETURN_NULL();
+
+ /*
+ * Convert t_bits string back to the bits8 array as represented in the
+ * tuple header.
+ */
+ if (t_infomask & HEAP_HASNULL)
+ {
+ int bits_str_len;
+ int bits_len;
+
+ bits_len = (t_infomask2 & HEAP_NATTS_MASK) / 8 + 1;
+ if (!t_bits_str)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("argument of t_bits is null, but it is expected to be null and %i character long",
+ bits_len * 8)));
+
+ bits_str_len = strlen(t_bits_str);
+ if ((bits_str_len % 8) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("length of t_bits is not a multiple of eight")));
+
+ if (bits_len * 8 != bits_str_len)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("unexpected length of t_bits %u, expected %i",
+ bits_str_len, bits_len * 8)));
+
+ /* do the conversion */
+ t_bits = text_to_bits(t_bits_str, bits_str_len);
+ }
+ else
+ {
+ if (t_bits_str)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits string is expected to be NULL, but instead it is %lu bytes length",
+ strlen(t_bits_str))));
+ }
+
+ /* Split tuple data */
+ res = split_tuple_data(relid, (char *) raw_data + VARHDRSZ,
+ VARSIZE(raw_data) - VARHDRSZ,
+ t_infomask, t_infomask2, t_bits,
+ do_detoast);
+
+ if (t_bits)
+ pfree(t_bits);
+
+ PG_RETURN_ARRAYTYPE_P(res);
+}
diff --git a/contrib/pageinspect/pageinspect--1.3--1.4.sql b/contrib/pageinspect/pageinspect--1.3--1.4.sql
new file mode 100644
index 0000000..ea8f356
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.3--1.4.sql
@@ -0,0 +1,49 @@
+/* contrib/pageinspect/pageinspect--1.3--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via ALTER EXTENSION
+\echo Use "ALTER EXTENSION pageinspect UPDATE TO '1.4'" to load this file. \quit
+
+--
+-- heap_page_items()
+--
+DROP FUNCTION heap_page_items(bytea);
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(t_oid rel_oid,
+ t_data bytea,
+ t_infomask integer,
+ t_infomask2 integer,
+ t_bits text)
+RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+CREATE FUNCTION tuple_data_split(t_oid rel_oid,
+ t_data bytea,
+ t_infomask integer,
+ t_infomask2 integer,
+ t_bits text,
+ do_detoast bool)
+RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
diff --git a/contrib/pageinspect/pageinspect--1.3.sql b/contrib/pageinspect/pageinspect--1.4.sql
similarity index 88%
rename from contrib/pageinspect/pageinspect--1.3.sql
rename to contrib/pageinspect/pageinspect--1.4.sql
index a99e058..47864f4 100644
--- a/contrib/pageinspect/pageinspect--1.3.sql
+++ b/contrib/pageinspect/pageinspect--1.4.sql
@@ -1,4 +1,4 @@
-/* contrib/pageinspect/pageinspect--1.3.sql */
+/* contrib/pageinspect/pageinspect--1.4.sql */
-- complain if script is sourced in psql, rather than via CREATE EXTENSION
\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
@@ -48,12 +48,35 @@ CREATE FUNCTION heap_page_items(IN page bytea,
OUT t_infomask integer,
OUT t_hoff smallint,
OUT t_bits text,
- OUT t_oid oid)
+ OUT t_oid oid,
+ OUT t_data bytea)
RETURNS SETOF record
AS 'MODULE_PATHNAME', 'heap_page_items'
LANGUAGE C STRICT;
--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(rel_oid oid,
+ t_data bytea,
+ t_infomask integer,
+ t_infomask2 integer,
+ t_bits text)
+RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+CREATE FUNCTION tuple_data_split(rel_oid oid,
+ t_data bytea,
+ t_infomask integer,
+ t_infomask2 integer,
+ t_bits text,
+ do_detoast bool)
+RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+--
-- bt_metap()
--
CREATE FUNCTION bt_metap(IN relname text,
diff --git a/contrib/pageinspect/pageinspect.control b/contrib/pageinspect/pageinspect.control
index a9dab33..68c7d61 100644
--- a/contrib/pageinspect/pageinspect.control
+++ b/contrib/pageinspect/pageinspect.control
@@ -1,5 +1,5 @@
# pageinspect extension
comment = 'inspect the contents of database pages at a low level'
-default_version = '1.3'
+default_version = '1.4'
module_pathname = '$libdir/pageinspect'
relocatable = true
diff --git a/doc/src/sgml/pageinspect.sgml b/doc/src/sgml/pageinspect.sgml
index b95cc81..5e528c9 100644
--- a/doc/src/sgml/pageinspect.sgml
+++ b/doc/src/sgml/pageinspect.sgml
@@ -93,9 +93,10 @@ test=# SELECT * FROM page_header(get_raw_page('pg_class', 0));
<listitem>
<para>
<function>heap_page_items</function> shows all line pointers on a heap
- page. For those line pointers that are in use, tuple headers are also
- shown. All tuples are shown, whether or not the tuples were visible to
- an MVCC snapshot at the time the raw page was copied.
+ page. For those line pointers that are in use, tuple headers as well
+ as tuple raw data are also shown. All tuples are shown, whether or not
+ the tuples were visible to an MVCC snapshot at the time the raw page
+ was copied.
</para>
<para>
A heap page image obtained with <function>get_raw_page</function> should
@@ -112,6 +113,31 @@ test=# SELECT * FROM heap_page_items(get_raw_page('pg_class', 0));
<varlistentry>
<term>
+ <function>tuple_data_split(rel_oid, t_data bytea, t_infomask integer, t_infomask2 integer, t_bits text [, do_detoast bool]) returs bytea[]</function>
+ <indexterm>
+ <primary>tuple_data_split</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ <function>tuple_data_split</function> splits tuple data into attributes
+ in the same way as backend internals.
+<screen>
+test=# SELECT tuple_data_split('pg_class'::regclass, t_data, t_infomask, t_infomask2, t_bits) FROM heap_page_items(get_raw_page('pg_class', 0));
+</screen>
+ This function should be called with the same arguments as the return
+ attributes of <function>heap_page_items</function>.
+ </para>
+ <para>
+ If <parameter>do_detoast</parameter> is <literal>true</literal>,
+ attribute values that will be detoasted if needed. Default value is
+ <literal>false</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
<function>bt_metap(relname text) returns record</function>
<indexterm>
<primary>bt_metap</primary>
Re: pageinspect patch, for showing tuple data
В письме от 28 октября 2015 16:57:36 пользователь Michael Paquier написал:
On Sat, Oct 17, 2015 at 1:48 AM, Michael Paquier wrote:
On Sat, Oct 17, 2015 at 5:15 AM, Nikolay Shaplov wrote:
Or it's ready to commit, and just not marked this way?
No, I don't think we have reached this state yet.
I am going to make report based on this patch in Vienna. It would be
nice to have it committed until then :)This is registered in this November's CF and the current one is not
completely wrapped up, so I haven't rushed into looking at it.So, I have finally been able to look at this patch in more details,
resulting in the attached. I noticed a couple of things that should be
addressed, mainly:
- addition of a new routine text_to_bits to perform the reverse
operation of bits_to_text. This was previously part of
tuple_data_split, I think that it deserves its own function.
- split_tuple_data should be static
- t_bits_str should not be a text, rather a char* fetched using
text_to_cstring(PG_GETARG_TEXT_PP(4)). This way there is no need to
perform extra calculations with VARSIZE and VARHDRSZ
- split_tuple_data can directly use the relation OID instead of the
tuple descriptor of the relation
- t_bits was leaking memory. For correctness I think that it is better
to free it after calling split_tuple_data.
- PG_DETOAST_DATUM_COPY allocates some memory, this was leaking as
well in raw_attr actually. I refactored the code such as a bytea* is
used and always freed when allocated to avoid leaks. Removing raw_attr
actually simplified the code a bit.
- I simplified the docs, that was largely too verbose in my opinion.
- Instead of using VARATT_IS_1B_E and VARTAG_EXTERNAL, using
VARATT_IS_EXTERNAL and VARATT_IS_EXTERNAL_ONDISK seems more adapted to
me, those other ones are much more low-level and not really spread in
the backend code.
- Found some typos in the code, the docs and some comments. I reworked
the error messages as well.
- The code format was not really in line with the project guidelines.
I fixed that as well.
- I removed heap_page_item_attrs for now to get this patch in a
bare-bone state. Though I would not mind if this is re-added (I
personally don't think that's much necessary in the module
actually...).
- The calculation of the length of t_bits using HEAP_NATTS_MASK is
more correct as you mentioned earlier, so I let it in its state.
That's actually more accurate for error handling as well.
That's everything I recall I have. How does this look?
You've completely rewrite everything ;-)
Let everything be the way you wrote. This code is better than mine.
With one exception. I really need heap_page_item_attrs function. I used it in
my Tuples Internals presentation
https://github.com/dhyannataraj/tuple-internals-presentation
and I am 100% sure that this function is needed for educational purposes, and
this function should be as simple as possible, so students cat use it without
extra efforts.
I still have an opinion that documentation should be more verbose, than your
version, but I can accept your version.
Who is going to add heap_page_item_attrs to your patch? me or you?
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
On Thu, Nov 12, 2015 at 12:41 AM, Nikolay Shaplov
<n.shaplov@postgrespro.ru> wrote:
В письме от 28 октября 2015 16:57:36 пользователь Michael Paquier написал:
On Sat, Oct 17, 2015 at 1:48 AM, Michael Paquier wrote:
On Sat, Oct 17, 2015 at 5:15 AM, Nikolay Shaplov wrote:
Or it's ready to commit, and just not marked this way?
No, I don't think we have reached this state yet.
I am going to make report based on this patch in Vienna. It would be
nice to have it committed until then :)This is registered in this November's CF and the current one is not
completely wrapped up, so I haven't rushed into looking at it.So, I have finally been able to look at this patch in more details,
resulting in the attached. I noticed a couple of things that should be
addressed, mainly:
- addition of a new routine text_to_bits to perform the reverse
operation of bits_to_text. This was previously part of
tuple_data_split, I think that it deserves its own function.
- split_tuple_data should be static
- t_bits_str should not be a text, rather a char* fetched using
text_to_cstring(PG_GETARG_TEXT_PP(4)). This way there is no need to
perform extra calculations with VARSIZE and VARHDRSZ
- split_tuple_data can directly use the relation OID instead of the
tuple descriptor of the relation
- t_bits was leaking memory. For correctness I think that it is better
to free it after calling split_tuple_data.
- PG_DETOAST_DATUM_COPY allocates some memory, this was leaking as
well in raw_attr actually. I refactored the code such as a bytea* is
used and always freed when allocated to avoid leaks. Removing raw_attr
actually simplified the code a bit.
- I simplified the docs, that was largely too verbose in my opinion.
- Instead of using VARATT_IS_1B_E and VARTAG_EXTERNAL, using
VARATT_IS_EXTERNAL and VARATT_IS_EXTERNAL_ONDISK seems more adapted to
me, those other ones are much more low-level and not really spread in
the backend code.
- Found some typos in the code, the docs and some comments. I reworked
the error messages as well.
- The code format was not really in line with the project guidelines.
I fixed that as well.
- I removed heap_page_item_attrs for now to get this patch in a
bare-bone state. Though I would not mind if this is re-added (I
personally don't think that's much necessary in the module
actually...).
- The calculation of the length of t_bits using HEAP_NATTS_MASK is
more correct as you mentioned earlier, so I let it in its state.
That's actually more accurate for error handling as well.
That's everything I recall I have. How does this look?You've completely rewrite everything ;-)
Let everything be the way you wrote. This code is better than mine.
With one exception. I really need heap_page_item_attrs function. I used it in
my Tuples Internals presentation
https://github.com/dhyannataraj/tuple-internals-presentation
and I am 100% sure that this function is needed for educational purposes, and
this function should be as simple as possible, so students can use it without
extra efforts.
Fine. That's your patch after all.
I still have an opinion that documentation should be more verbose, than your
version, but I can accept your version.
I am not sure that's necessary, pageinspect is for developers.
Who is going to add heap_page_item_attrs to your patch? me or you?
I recall some parts of the code I still did not like much :) I'll grab
some room to have an extra look at it.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
On Thu, Nov 12, 2015 at 9:04 AM, Michael Paquier wrote:
On Thu, Nov 12, 2015 at 12:41 AM, Nikolay Shaplov wrote:
I still have an opinion that documentation should be more verbose, than your
version, but I can accept your version.I am not sure that's necessary, pageinspect is for developers.
Who is going to add heap_page_item_attrs to your patch? me or you?
I recall some parts of the code I still did not like much :) I'll grab
some room to have an extra look at it.
I have added back heap_page_item_attrs the patch attached, fixing at
the same time some typos found while looking again at the code. If you
are fine with this version, let's have a committer look at it.
--
Michael
Attachments:
20151112_pageinspect_v5.patchtext/x-patch; charset=US-ASCII; name=20151112_pageinspect_v5.patchDownload
diff --git a/contrib/pageinspect/Makefile b/contrib/pageinspect/Makefile
index aec5258..91ab119 100644
--- a/contrib/pageinspect/Makefile
+++ b/contrib/pageinspect/Makefile
@@ -5,9 +5,9 @@ OBJS = rawpage.o heapfuncs.o btreefuncs.o fsmfuncs.o \
brinfuncs.o ginfuncs.o $(WIN32RES)
EXTENSION = pageinspect
-DATA = pageinspect--1.3.sql pageinspect--1.2--1.3.sql \
- pageinspect--1.1--1.2.sql pageinspect--1.0--1.1.sql \
- pageinspect--unpackaged--1.0.sql
+DATA = pageinspect--1.4.sql pageinspect--1.3--1.4.sql \
+ pageinspect--1.2--1.3.sql pageinspect--1.1--1.2.sql \
+ pageinspect--1.0--1.1.sql pageinspect--unpackaged--1.0.sql
PGFILEDESC = "pageinspect - functions to inspect contents of database pages"
ifdef USE_PGXS
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 8d1666c..fdbb66c 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -27,8 +27,11 @@
#include "access/htup_details.h"
#include "funcapi.h"
-#include "utils/builtins.h"
+#include "catalog/pg_type.h"
#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
+#include "utils/rel.h"
/*
@@ -55,6 +58,42 @@ bits_to_text(bits8 *bits, int len)
/*
+ * text_to_bits
+ *
+ * Converts a c-string representation of bits into a bits8-array. This is
+ * the reverse operation of previous routine.
+ */
+static bits8 *
+text_to_bits(char *str, int len)
+{
+ bits8 *bits;
+ int off = 0;
+ char byte = 0;
+
+ bits = palloc(len + 1);
+
+ while (off < len)
+ {
+ if (off % 8 == 0)
+ byte = 0;
+
+ if ((str[off] == '0') || (str[off] == '1'))
+ byte = byte | ((str[off] - '0') << off % 8);
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("illegal character '%c' in t_bits string", str[off])));
+
+ if (off % 8 == 7)
+ bits[off / 8] = byte;
+
+ off++;
+ }
+
+ return bits;
+}
+
+/*
* heap_page_items
*
* Allows inspection of line pointers and tuple headers of a heap page.
@@ -122,8 +161,8 @@ heap_page_items(PG_FUNCTION_ARGS)
HeapTuple resultTuple;
Datum result;
ItemId id;
- Datum values[13];
- bool nulls[13];
+ Datum values[14];
+ bool nulls[14];
uint16 lp_offset;
uint16 lp_flags;
uint16 lp_len;
@@ -154,7 +193,8 @@ heap_page_items(PG_FUNCTION_ARGS)
lp_offset + lp_len <= raw_page_size)
{
HeapTupleHeader tuphdr;
- int bits_len;
+ bytea *tuple_data_bytea;
+ int tuple_data_len;
/* Extract information from the tuple header */
@@ -168,6 +208,14 @@ heap_page_items(PG_FUNCTION_ARGS)
values[9] = UInt32GetDatum(tuphdr->t_infomask);
values[10] = UInt8GetDatum(tuphdr->t_hoff);
+ /* Copy raw tuple data into bytea attribute */
+ tuple_data_len = lp_len - tuphdr->t_hoff;
+ tuple_data_bytea = (bytea *) palloc(tuple_data_len + VARHDRSZ);
+ SET_VARSIZE(tuple_data_bytea, tuple_data_len + VARHDRSZ);
+ memcpy(VARDATA(tuple_data_bytea), (char *) tuphdr + tuphdr->t_hoff,
+ tuple_data_len);
+ values[13] = PointerGetDatum(tuple_data_bytea);
+
/*
* We already checked that the item is completely within the raw
* page passed to us, with the length given in the line pointer.
@@ -180,11 +228,11 @@ heap_page_items(PG_FUNCTION_ARGS)
{
if (tuphdr->t_infomask & HEAP_HASNULL)
{
- bits_len = tuphdr->t_hoff -
- offsetof(HeapTupleHeaderData, t_bits);
+ int bits_len =
+ ((tuphdr->t_infomask2 & HEAP_NATTS_MASK) / 8 + 1) * 8;
values[11] = CStringGetTextDatum(
- bits_to_text(tuphdr->t_bits, bits_len * 8));
+ bits_to_text(tuphdr->t_bits, bits_len));
}
else
nulls[11] = true;
@@ -208,7 +256,7 @@ heap_page_items(PG_FUNCTION_ARGS)
*/
int i;
- for (i = 4; i <= 12; i++)
+ for (i = 4; i <= 13; i++)
nulls[i] = true;
}
@@ -223,3 +271,205 @@ heap_page_items(PG_FUNCTION_ARGS)
else
SRF_RETURN_DONE(fctx);
}
+
+/*
+ * split_tuple_data
+ *
+ * Split raw tuple data taken directly from a page into an array of bytea
+ * elements. This routine does a lookup on NULL values and creates array
+ * elements accordindly. This is a reimplementation of nocachegetattr()
+ * in heaptuple.c simplified for educational purposes.
+ */
+static Datum
+split_tuple_data(Oid relid, char *tupdata,
+ uint16 tupdata_len, uint16 t_infomask,
+ uint16 t_infomask2, bits8 *t_bits,
+ bool do_detoast)
+{
+ ArrayBuildState *raw_attrs;
+ int nattrs;
+ int i;
+ int off = 0;
+ Relation rel;
+ TupleDesc tupdesc;
+
+ /* Get tuple descriptor from relation OID */
+ rel = relation_open(relid, NoLock);
+ tupdesc = CreateTupleDescCopyConstr(rel->rd_att);
+ relation_close(rel, NoLock);
+
+ raw_attrs = initArrayResult(BYTEAOID, CurrentMemoryContext, false);
+ nattrs = tupdesc->natts;
+
+ if (nattrs < (t_infomask2 & HEAP_NATTS_MASK))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("number of attributes in tuple header is greater than number of attributes in tuple descriptor")));
+
+ for (i = 0; i < nattrs; i++)
+ {
+ Form_pg_attribute attr;
+ bool is_null;
+ bytea *attr_data = NULL;
+
+ attr = tupdesc->attrs[i];
+ is_null = (t_infomask & HEAP_HASNULL) && att_isnull(i, t_bits);
+
+ /*
+ * Tuple header can specify less attributes than tuple descriptor
+ * as ALTER TABLE ADD COLUMN without DEFAULT keyword does not
+ * actually change tuples in pages, so attributes with numbers greater
+ * than (t_infomask2 & HEAP_NATTS_MASK) should be treated as NULL.
+ */
+ if (i >= (t_infomask2 & HEAP_NATTS_MASK))
+ is_null = true;
+
+ if (!is_null)
+ {
+ int len;
+
+ if (attr->attlen == -1)
+ {
+ off = att_align_pointer(off, tupdesc->attrs[i]->attalign, -1,
+ tupdata + off);
+ /*
+ * As VARSIZE_ANY throws an exception if it can't properly detect
+ * the type of external storage in macros VARTAG_SIZE, this check
+ * is repeated to have a nicer error handling.
+ */
+ if (VARATT_IS_EXTERNAL(tupdata + off) &&
+ !VARATT_IS_EXTERNAL_ONDISK(tupdata + off) &&
+ !VARATT_IS_EXTERNAL_INDIRECT(tupdata + off))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("first byte of varlena attribute is incorrect for attribute %d", i)));
+
+ len = VARSIZE_ANY(tupdata + off);
+ }
+ else
+ {
+ off = att_align_nominal(off, tupdesc->attrs[i]->attalign);
+ len = attr->attlen;
+ }
+
+ if (tupdata_len < off + len)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("unexpected end of tuple data")));
+
+ if (attr->attlen == -1 && do_detoast)
+ attr_data = DatumGetByteaPCopy(tupdata + off);
+ else
+ {
+ attr_data = (bytea *) palloc(len + VARHDRSZ);
+ SET_VARSIZE(attr_data, len + VARHDRSZ);
+ memcpy(VARDATA(attr_data), tupdata + off, len);
+ }
+
+ off = att_addlength_pointer(off, tupdesc->attrs[i]->attlen,
+ tupdata + off);
+ }
+
+ raw_attrs = accumArrayResult(raw_attrs, PointerGetDatum(attr_data),
+ is_null, BYTEAOID, CurrentMemoryContext);
+ if (attr_data)
+ pfree(attr_data);
+ }
+
+ if (tupdata_len != off)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("end of tuple reached without looking at all its data")));
+
+ return makeArrayResult(raw_attrs, CurrentMemoryContext);
+}
+
+/*
+ * tuple_data_split
+ *
+ * Split raw tuple data taken directly from page into distinct elements
+ * taking into account null values.
+ */
+PG_FUNCTION_INFO_V1(tuple_data_split);
+
+Datum
+tuple_data_split(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ bytea *raw_data;
+ uint16 t_infomask;
+ uint16 t_infomask2;
+ char *t_bits_str;
+ bool do_detoast = false;
+ bits8 *t_bits = NULL;
+ Datum res;
+
+ relid = PG_GETARG_OID(0);
+ raw_data = PG_ARGISNULL(1) ? NULL : PG_GETARG_BYTEA_P(1);
+ t_infomask = PG_GETARG_INT16(2);
+ t_infomask2 = PG_GETARG_INT16(3);
+ t_bits_str = PG_ARGISNULL(4) ? NULL :
+ text_to_cstring(PG_GETARG_TEXT_PP(4));
+
+ if (PG_NARGS() >= 6)
+ do_detoast = PG_GETARG_BOOL(5);
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to use raw page functions")));
+
+ if (!raw_data)
+ PG_RETURN_NULL();
+
+ /*
+ * Convert t_bits string back to the bits8 array as represented in the
+ * tuple header.
+ */
+ if (t_infomask & HEAP_HASNULL)
+ {
+ int bits_str_len;
+ int bits_len;
+
+ bits_len = (t_infomask2 & HEAP_NATTS_MASK) / 8 + 1;
+ if (!t_bits_str)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("argument of t_bits is null, but it is expected to be null and %i character long",
+ bits_len * 8)));
+
+ bits_str_len = strlen(t_bits_str);
+ if ((bits_str_len % 8) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("length of t_bits is not a multiple of eight")));
+
+ if (bits_len * 8 != bits_str_len)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("unexpected length of t_bits %u, expected %i",
+ bits_str_len, bits_len * 8)));
+
+ /* do the conversion */
+ t_bits = text_to_bits(t_bits_str, bits_str_len);
+ }
+ else
+ {
+ if (t_bits_str)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits string is expected to be NULL, but instead it is %lu bytes length",
+ strlen(t_bits_str))));
+ }
+
+ /* Split tuple data */
+ res = split_tuple_data(relid, (char *) raw_data + VARHDRSZ,
+ VARSIZE(raw_data) - VARHDRSZ,
+ t_infomask, t_infomask2, t_bits,
+ do_detoast);
+
+ if (t_bits)
+ pfree(t_bits);
+
+ PG_RETURN_ARRAYTYPE_P(res);
+}
diff --git a/contrib/pageinspect/pageinspect--1.3--1.4.sql b/contrib/pageinspect/pageinspect--1.3--1.4.sql
new file mode 100644
index 0000000..41fadf2
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.3--1.4.sql
@@ -0,0 +1,118 @@
+/* contrib/pageinspect/pageinspect--1.3--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via ALTER EXTENSION
+\echo Use "ALTER EXTENSION pageinspect UPDATE TO '1.4'" to load this file. \quit
+
+--
+-- heap_page_items()
+--
+DROP FUNCTION heap_page_items(bytea);
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(
+ IN page bytea,
+ IN rel_oid regclass,
+ IN do_detoast bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record AS $$
+SELECT lp,
+ lp_off,
+ lp_flags,
+ lp_len,
+ t_xmin,
+ t_xmax,
+ t_field3,
+ t_ctid,
+ t_infomask2,
+ t_infomask,
+ t_hoff,
+ t_bits,
+ t_oid,
+ tuple_data_split(
+ rel_oid,
+ t_data,
+ t_infomask,
+ t_infomask2,
+ t_bits,
+ do_detoast)
+ AS t_attrs
+ FROM heap_page_items(page);
+$$ LANGUAGE SQL;
+
+CREATE FUNCTION heap_page_item_attrs(
+ IN page bytea,
+ IN rel_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record AS $$
+SELECT * from heap_page_item_attrs(page, rel_oid, false);
+$$ LANGUAGE SQL;
+
+--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(t_oid rel_oid,
+ t_data bytea,
+ t_infomask integer,
+ t_infomask2 integer,
+ t_bits text)
+RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+CREATE FUNCTION tuple_data_split(t_oid rel_oid,
+ t_data bytea,
+ t_infomask integer,
+ t_infomask2 integer,
+ t_bits text,
+ do_detoast bool)
+RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
diff --git a/contrib/pageinspect/pageinspect--1.3.sql b/contrib/pageinspect/pageinspect--1.4.sql
similarity index 67%
rename from contrib/pageinspect/pageinspect--1.3.sql
rename to contrib/pageinspect/pageinspect--1.4.sql
index a99e058..f75aa0b 100644
--- a/contrib/pageinspect/pageinspect--1.3.sql
+++ b/contrib/pageinspect/pageinspect--1.4.sql
@@ -1,4 +1,4 @@
-/* contrib/pageinspect/pageinspect--1.3.sql */
+/* contrib/pageinspect/pageinspect--1.4.sql */
-- complain if script is sourced in psql, rather than via CREATE EXTENSION
\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
@@ -48,12 +48,102 @@ CREATE FUNCTION heap_page_items(IN page bytea,
OUT t_infomask integer,
OUT t_hoff smallint,
OUT t_bits text,
- OUT t_oid oid)
+ OUT t_oid oid,
+ OUT t_data bytea)
RETURNS SETOF record
AS 'MODULE_PATHNAME', 'heap_page_items'
LANGUAGE C STRICT;
--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(rel_oid oid,
+ t_data bytea,
+ t_infomask integer,
+ t_infomask2 integer,
+ t_bits text)
+RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+CREATE FUNCTION tuple_data_split(rel_oid oid,
+ t_data bytea,
+ t_infomask integer,
+ t_infomask2 integer,
+ t_bits text,
+ do_detoast bool)
+RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(
+ IN page bytea,
+ IN rel_oid regclass,
+ IN do_detoast bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record AS $$
+SELECT lp,
+ lp_off,
+ lp_flags,
+ lp_len,
+ t_xmin,
+ t_xmax,
+ t_field3,
+ t_ctid,
+ t_infomask2,
+ t_infomask,
+ t_hoff,
+ t_bits,
+ t_oid,
+ tuple_data_split(
+ rel_oid,
+ t_data,
+ t_infomask,
+ t_infomask2,
+ t_bits,
+ do_detoast)
+ AS t_attrs
+ FROM heap_page_items(page);
+$$ LANGUAGE SQL;
+
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN rel_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record AS $$
+SELECT * from heap_page_item_attrs(page, rel_oid, false);
+$$ LANGUAGE SQL;
+
+--
-- bt_metap()
--
CREATE FUNCTION bt_metap(IN relname text,
diff --git a/contrib/pageinspect/pageinspect.control b/contrib/pageinspect/pageinspect.control
index a9dab33..68c7d61 100644
--- a/contrib/pageinspect/pageinspect.control
+++ b/contrib/pageinspect/pageinspect.control
@@ -1,5 +1,5 @@
# pageinspect extension
comment = 'inspect the contents of database pages at a low level'
-default_version = '1.3'
+default_version = '1.4'
module_pathname = '$libdir/pageinspect'
relocatable = true
diff --git a/doc/src/sgml/pageinspect.sgml b/doc/src/sgml/pageinspect.sgml
index b95cc81..5d187ed 100644
--- a/doc/src/sgml/pageinspect.sgml
+++ b/doc/src/sgml/pageinspect.sgml
@@ -93,9 +93,10 @@ test=# SELECT * FROM page_header(get_raw_page('pg_class', 0));
<listitem>
<para>
<function>heap_page_items</function> shows all line pointers on a heap
- page. For those line pointers that are in use, tuple headers are also
- shown. All tuples are shown, whether or not the tuples were visible to
- an MVCC snapshot at the time the raw page was copied.
+ page. For those line pointers that are in use, tuple headers as well
+ as tuple raw data are also shown. All tuples are shown, whether or not
+ the tuples were visible to an MVCC snapshot at the time the raw page
+ was copied.
</para>
<para>
A heap page image obtained with <function>get_raw_page</function> should
@@ -112,6 +113,56 @@ test=# SELECT * FROM heap_page_items(get_raw_page('pg_class', 0));
<varlistentry>
<term>
+ <function>tuple_data_split(rel_oid, t_data bytea, t_infomask integer, t_infomask2 integer, t_bits text [, do_detoast bool]) returns bytea[]</function>
+ <indexterm>
+ <primary>tuple_data_split</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ <function>tuple_data_split</function> splits tuple data into attributes
+ in the same way as backend internals.
+<screen>
+test=# SELECT tuple_data_split('pg_class'::regclass, t_data, t_infomask, t_infomask2, t_bits) FROM heap_page_items(get_raw_page('pg_class', 0));
+</screen>
+ This function should be called with the same arguments as the return
+ attributes of <function>heap_page_items</function>.
+ </para>
+ <para>
+ If <parameter>do_detoast</parameter> is <literal>true</literal>,
+ attribute that will be detoasted as needed. Default value is
+ <literal>false</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <function>heap_page_item_attrs(rel_oid, t_data bytea, [, do_detoast bool]) returns bytea[]</function>
+ <indexterm>
+ <primary>heap_page_item_attrs</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ <function>heap_page_item_attrs</function> is equivalent to
+ <function>heap_page_items</function> except that it returns
+ tuple raw data as an array of attributes that can optionally
+ be detoasted by <parameter>do_detoast</parameter> which is
+ <literal>false</literal> by default.
+ </para>
+ <para>
+ A heap page image obtained with <function>get_raw_page</function> should
+ be passed as argument. For example:
+<screen>
+test=# SELECT * FROM heap_page_item_attrs(get_raw_page('pg_class', 0), 'pg_class'::regclass);
+</screen>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
<function>bt_metap(relname text) returns record</function>
<indexterm>
<primary>bt_metap</primary>
Re: pageinspect patch, for showing tuple data
I still have an opinion that documentation should be more verbose, than
your version, but I can accept your version.I am not sure that's necessary, pageinspect is for developers.
Who is going to add heap_page_item_attrs to your patch? me or you?
I recall some parts of the code I still did not like much :) I'll grab
some room to have an extra look at it.I have added back heap_page_item_attrs the patch attached, fixing at
the same time some typos found while looking again at the code. If you
are fine with this version, let's have a committer look at it.
Everything seems to be ok. I've changed only one thing in your version
of the patch. I've renamed split_tuple_data to
tuple_data_split_internal, because when there are split_tuple_data and
tuple_data_split in the same file, nobody will guess what the difference
between these function and why are they named in such a strange way.
If it is ok, set proper status, and let commiter do his job :-)
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
pageinspect_paque_v2_mod1.difftext/x-patch; name=pageinspect_paque_v2_mod1.diffDownload
diff --git a/contrib/pageinspect/Makefile b/contrib/pageinspect/Makefile
index aec5258..91ab119 100644
--- a/contrib/pageinspect/Makefile
+++ b/contrib/pageinspect/Makefile
@@ -5,9 +5,9 @@ OBJS = rawpage.o heapfuncs.o btreefuncs.o fsmfuncs.o \
brinfuncs.o ginfuncs.o $(WIN32RES)
EXTENSION = pageinspect
-DATA = pageinspect--1.3.sql pageinspect--1.2--1.3.sql \
- pageinspect--1.1--1.2.sql pageinspect--1.0--1.1.sql \
- pageinspect--unpackaged--1.0.sql
+DATA = pageinspect--1.4.sql pageinspect--1.3--1.4.sql \
+ pageinspect--1.2--1.3.sql pageinspect--1.1--1.2.sql \
+ pageinspect--1.0--1.1.sql pageinspect--unpackaged--1.0.sql
PGFILEDESC = "pageinspect - functions to inspect contents of database pages"
ifdef USE_PGXS
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 8d1666c..546a051 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -27,8 +27,11 @@
#include "access/htup_details.h"
#include "funcapi.h"
-#include "utils/builtins.h"
+#include "catalog/pg_type.h"
#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
+#include "utils/rel.h"
/*
@@ -55,6 +58,42 @@ bits_to_text(bits8 *bits, int len)
/*
+ * text_to_bits
+ *
+ * Converts a c-string representation of bits into a bits8-array. This is
+ * the reverse operation of previous routine.
+ */
+static bits8 *
+text_to_bits(char *str, int len)
+{
+ bits8 *bits;
+ int off = 0;
+ char byte = 0;
+
+ bits = palloc(len + 1);
+
+ while (off < len)
+ {
+ if (off % 8 == 0)
+ byte = 0;
+
+ if ((str[off] == '0') || (str[off] == '1'))
+ byte = byte | ((str[off] - '0') << off % 8);
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("illegal character '%c' in t_bits string", str[off])));
+
+ if (off % 8 == 7)
+ bits[off / 8] = byte;
+
+ off++;
+ }
+
+ return bits;
+}
+
+/*
* heap_page_items
*
* Allows inspection of line pointers and tuple headers of a heap page.
@@ -122,8 +161,8 @@ heap_page_items(PG_FUNCTION_ARGS)
HeapTuple resultTuple;
Datum result;
ItemId id;
- Datum values[13];
- bool nulls[13];
+ Datum values[14];
+ bool nulls[14];
uint16 lp_offset;
uint16 lp_flags;
uint16 lp_len;
@@ -154,7 +193,8 @@ heap_page_items(PG_FUNCTION_ARGS)
lp_offset + lp_len <= raw_page_size)
{
HeapTupleHeader tuphdr;
- int bits_len;
+ bytea *tuple_data_bytea;
+ int tuple_data_len;
/* Extract information from the tuple header */
@@ -168,6 +208,14 @@ heap_page_items(PG_FUNCTION_ARGS)
values[9] = UInt32GetDatum(tuphdr->t_infomask);
values[10] = UInt8GetDatum(tuphdr->t_hoff);
+ /* Copy raw tuple data into bytea attribute */
+ tuple_data_len = lp_len - tuphdr->t_hoff;
+ tuple_data_bytea = (bytea *) palloc(tuple_data_len + VARHDRSZ);
+ SET_VARSIZE(tuple_data_bytea, tuple_data_len + VARHDRSZ);
+ memcpy(VARDATA(tuple_data_bytea), (char *) tuphdr + tuphdr->t_hoff,
+ tuple_data_len);
+ values[13] = PointerGetDatum(tuple_data_bytea);
+
/*
* We already checked that the item is completely within the raw
* page passed to us, with the length given in the line pointer.
@@ -180,11 +228,11 @@ heap_page_items(PG_FUNCTION_ARGS)
{
if (tuphdr->t_infomask & HEAP_HASNULL)
{
- bits_len = tuphdr->t_hoff -
- offsetof(HeapTupleHeaderData, t_bits);
+ int bits_len =
+ ((tuphdr->t_infomask2 & HEAP_NATTS_MASK) / 8 + 1) * 8;
values[11] = CStringGetTextDatum(
- bits_to_text(tuphdr->t_bits, bits_len * 8));
+ bits_to_text(tuphdr->t_bits, bits_len));
}
else
nulls[11] = true;
@@ -208,7 +256,7 @@ heap_page_items(PG_FUNCTION_ARGS)
*/
int i;
- for (i = 4; i <= 12; i++)
+ for (i = 4; i <= 13; i++)
nulls[i] = true;
}
@@ -223,3 +271,205 @@ heap_page_items(PG_FUNCTION_ARGS)
else
SRF_RETURN_DONE(fctx);
}
+
+/*
+ * tuple_data_split_internal
+ *
+ * Split raw tuple data taken directly from a page into an array of bytea
+ * elements. This routine does a lookup on NULL values and creates array
+ * elements accordindly. This is a reimplementation of nocachegetattr()
+ * in heaptuple.c simplified for educational purposes.
+ */
+static Datum
+tuple_data_split_internal(Oid relid, char *tupdata,
+ uint16 tupdata_len, uint16 t_infomask,
+ uint16 t_infomask2, bits8 *t_bits,
+ bool do_detoast)
+{
+ ArrayBuildState *raw_attrs;
+ int nattrs;
+ int i;
+ int off = 0;
+ Relation rel;
+ TupleDesc tupdesc;
+
+ /* Get tuple descriptor from relation OID */
+ rel = relation_open(relid, NoLock);
+ tupdesc = CreateTupleDescCopyConstr(rel->rd_att);
+ relation_close(rel, NoLock);
+
+ raw_attrs = initArrayResult(BYTEAOID, CurrentMemoryContext, false);
+ nattrs = tupdesc->natts;
+
+ if (nattrs < (t_infomask2 & HEAP_NATTS_MASK))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("number of attributes in tuple header is greater than number of attributes in tuple descriptor")));
+
+ for (i = 0; i < nattrs; i++)
+ {
+ Form_pg_attribute attr;
+ bool is_null;
+ bytea *attr_data = NULL;
+
+ attr = tupdesc->attrs[i];
+ is_null = (t_infomask & HEAP_HASNULL) && att_isnull(i, t_bits);
+
+ /*
+ * Tuple header can specify less attributes than tuple descriptor
+ * as ALTER TABLE ADD COLUMN without DEFAULT keyword does not
+ * actually change tuples in pages, so attributes with numbers greater
+ * than (t_infomask2 & HEAP_NATTS_MASK) should be treated as NULL.
+ */
+ if (i >= (t_infomask2 & HEAP_NATTS_MASK))
+ is_null = true;
+
+ if (!is_null)
+ {
+ int len;
+
+ if (attr->attlen == -1)
+ {
+ off = att_align_pointer(off, tupdesc->attrs[i]->attalign, -1,
+ tupdata + off);
+ /*
+ * As VARSIZE_ANY throws an exception if it can't properly detect
+ * the type of external storage in macros VARTAG_SIZE, this check
+ * is repeated to have a nicer error handling.
+ */
+ if (VARATT_IS_EXTERNAL(tupdata + off) &&
+ !VARATT_IS_EXTERNAL_ONDISK(tupdata + off) &&
+ !VARATT_IS_EXTERNAL_INDIRECT(tupdata + off))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("first byte of varlena attribute is incorrect for attribute %d", i)));
+
+ len = VARSIZE_ANY(tupdata + off);
+ }
+ else
+ {
+ off = att_align_nominal(off, tupdesc->attrs[i]->attalign);
+ len = attr->attlen;
+ }
+
+ if (tupdata_len < off + len)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("unexpected end of tuple data")));
+
+ if (attr->attlen == -1 && do_detoast)
+ attr_data = DatumGetByteaPCopy(tupdata + off);
+ else
+ {
+ attr_data = (bytea *) palloc(len + VARHDRSZ);
+ SET_VARSIZE(attr_data, len + VARHDRSZ);
+ memcpy(VARDATA(attr_data), tupdata + off, len);
+ }
+
+ off = att_addlength_pointer(off, tupdesc->attrs[i]->attlen,
+ tupdata + off);
+ }
+
+ raw_attrs = accumArrayResult(raw_attrs, PointerGetDatum(attr_data),
+ is_null, BYTEAOID, CurrentMemoryContext);
+ if (attr_data)
+ pfree(attr_data);
+ }
+
+ if (tupdata_len != off)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("end of tuple reached without looking at all its data")));
+
+ return makeArrayResult(raw_attrs, CurrentMemoryContext);
+}
+
+/*
+ * tuple_data_split
+ *
+ * Split raw tuple data taken directly from page into distinct elements
+ * taking into account null values.
+ */
+PG_FUNCTION_INFO_V1(tuple_data_split);
+
+Datum
+tuple_data_split(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ bytea *raw_data;
+ uint16 t_infomask;
+ uint16 t_infomask2;
+ char *t_bits_str;
+ bool do_detoast = false;
+ bits8 *t_bits = NULL;
+ Datum res;
+
+ relid = PG_GETARG_OID(0);
+ raw_data = PG_ARGISNULL(1) ? NULL : PG_GETARG_BYTEA_P(1);
+ t_infomask = PG_GETARG_INT16(2);
+ t_infomask2 = PG_GETARG_INT16(3);
+ t_bits_str = PG_ARGISNULL(4) ? NULL :
+ text_to_cstring(PG_GETARG_TEXT_PP(4));
+
+ if (PG_NARGS() >= 6)
+ do_detoast = PG_GETARG_BOOL(5);
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to use raw page functions")));
+
+ if (!raw_data)
+ PG_RETURN_NULL();
+
+ /*
+ * Convert t_bits string back to the bits8 array as represented in the
+ * tuple header.
+ */
+ if (t_infomask & HEAP_HASNULL)
+ {
+ int bits_str_len;
+ int bits_len;
+
+ bits_len = (t_infomask2 & HEAP_NATTS_MASK) / 8 + 1;
+ if (!t_bits_str)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("argument of t_bits is null, but it is expected to be null and %i character long",
+ bits_len * 8)));
+
+ bits_str_len = strlen(t_bits_str);
+ if ((bits_str_len % 8) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("length of t_bits is not a multiple of eight")));
+
+ if (bits_len * 8 != bits_str_len)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("unexpected length of t_bits %u, expected %i",
+ bits_str_len, bits_len * 8)));
+
+ /* do the conversion */
+ t_bits = text_to_bits(t_bits_str, bits_str_len);
+ }
+ else
+ {
+ if (t_bits_str)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits string is expected to be NULL, but instead it is %lu bytes length",
+ strlen(t_bits_str))));
+ }
+
+ /* Split tuple data */
+ res = tuple_data_split_internal(relid, (char *) raw_data + VARHDRSZ,
+ VARSIZE(raw_data) - VARHDRSZ,
+ t_infomask, t_infomask2, t_bits,
+ do_detoast);
+
+ if (t_bits)
+ pfree(t_bits);
+
+ PG_RETURN_ARRAYTYPE_P(res);
+}
diff --git a/contrib/pageinspect/pageinspect--1.3--1.4.sql b/contrib/pageinspect/pageinspect--1.3--1.4.sql
new file mode 100644
index 0000000..41fadf2
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.3--1.4.sql
@@ -0,0 +1,118 @@
+/* contrib/pageinspect/pageinspect--1.3--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via ALTER EXTENSION
+\echo Use "ALTER EXTENSION pageinspect UPDATE TO '1.4'" to load this file. \quit
+
+--
+-- heap_page_items()
+--
+DROP FUNCTION heap_page_items(bytea);
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(
+ IN page bytea,
+ IN rel_oid regclass,
+ IN do_detoast bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record AS $$
+SELECT lp,
+ lp_off,
+ lp_flags,
+ lp_len,
+ t_xmin,
+ t_xmax,
+ t_field3,
+ t_ctid,
+ t_infomask2,
+ t_infomask,
+ t_hoff,
+ t_bits,
+ t_oid,
+ tuple_data_split(
+ rel_oid,
+ t_data,
+ t_infomask,
+ t_infomask2,
+ t_bits,
+ do_detoast)
+ AS t_attrs
+ FROM heap_page_items(page);
+$$ LANGUAGE SQL;
+
+CREATE FUNCTION heap_page_item_attrs(
+ IN page bytea,
+ IN rel_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record AS $$
+SELECT * from heap_page_item_attrs(page, rel_oid, false);
+$$ LANGUAGE SQL;
+
+--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(t_oid rel_oid,
+ t_data bytea,
+ t_infomask integer,
+ t_infomask2 integer,
+ t_bits text)
+RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+CREATE FUNCTION tuple_data_split(t_oid rel_oid,
+ t_data bytea,
+ t_infomask integer,
+ t_infomask2 integer,
+ t_bits text,
+ do_detoast bool)
+RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
diff --git a/contrib/pageinspect/pageinspect--1.3.sql b/contrib/pageinspect/pageinspect--1.3.sql
deleted file mode 100644
index a99e058..0000000
--- a/contrib/pageinspect/pageinspect--1.3.sql
+++ /dev/null
@@ -1,189 +0,0 @@
-/* contrib/pageinspect/pageinspect--1.3.sql */
-
--- complain if script is sourced in psql, rather than via CREATE EXTENSION
-\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
-
---
--- get_raw_page()
---
-CREATE FUNCTION get_raw_page(text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page'
-LANGUAGE C STRICT;
-
-CREATE FUNCTION get_raw_page(text, text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page_fork'
-LANGUAGE C STRICT;
-
---
--- page_header()
---
-CREATE FUNCTION page_header(IN page bytea,
- OUT lsn pg_lsn,
- OUT checksum smallint,
- OUT flags smallint,
- OUT lower smallint,
- OUT upper smallint,
- OUT special smallint,
- OUT pagesize smallint,
- OUT version smallint,
- OUT prune_xid xid)
-AS 'MODULE_PATHNAME', 'page_header'
-LANGUAGE C STRICT;
-
---
--- heap_page_items()
---
-CREATE FUNCTION heap_page_items(IN page bytea,
- OUT lp smallint,
- OUT lp_off smallint,
- OUT lp_flags smallint,
- OUT lp_len smallint,
- OUT t_xmin xid,
- OUT t_xmax xid,
- OUT t_field3 int4,
- OUT t_ctid tid,
- OUT t_infomask2 integer,
- OUT t_infomask integer,
- OUT t_hoff smallint,
- OUT t_bits text,
- OUT t_oid oid)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'heap_page_items'
-LANGUAGE C STRICT;
-
---
--- bt_metap()
---
-CREATE FUNCTION bt_metap(IN relname text,
- OUT magic int4,
- OUT version int4,
- OUT root int4,
- OUT level int4,
- OUT fastroot int4,
- OUT fastlevel int4)
-AS 'MODULE_PATHNAME', 'bt_metap'
-LANGUAGE C STRICT;
-
---
--- bt_page_stats()
---
-CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
- OUT blkno int4,
- OUT type "char",
- OUT live_items int4,
- OUT dead_items int4,
- OUT avg_item_size int4,
- OUT page_size int4,
- OUT free_size int4,
- OUT btpo_prev int4,
- OUT btpo_next int4,
- OUT btpo int4,
- OUT btpo_flags int4)
-AS 'MODULE_PATHNAME', 'bt_page_stats'
-LANGUAGE C STRICT;
-
---
--- bt_page_items()
---
-CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
- OUT itemoffset smallint,
- OUT ctid tid,
- OUT itemlen smallint,
- OUT nulls bool,
- OUT vars bool,
- OUT data text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'bt_page_items'
-LANGUAGE C STRICT;
-
---
--- brin_page_type()
---
-CREATE FUNCTION brin_page_type(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'brin_page_type'
-LANGUAGE C STRICT;
-
---
--- brin_metapage_info()
---
-CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
- OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
-AS 'MODULE_PATHNAME', 'brin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- brin_revmap_data()
---
-CREATE FUNCTION brin_revmap_data(IN page bytea,
- OUT pages tid)
-RETURNS SETOF tid
-AS 'MODULE_PATHNAME', 'brin_revmap_data'
-LANGUAGE C STRICT;
-
---
--- brin_page_items()
---
-CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
- OUT itemoffset int,
- OUT blknum int,
- OUT attnum int,
- OUT allnulls bool,
- OUT hasnulls bool,
- OUT placeholder bool,
- OUT value text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'brin_page_items'
-LANGUAGE C STRICT;
-
---
--- fsm_page_contents()
---
-CREATE FUNCTION fsm_page_contents(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'fsm_page_contents'
-LANGUAGE C STRICT;
-
---
--- GIN functions
---
-
---
--- gin_metapage_info()
---
-CREATE FUNCTION gin_metapage_info(IN page bytea,
- OUT pending_head bigint,
- OUT pending_tail bigint,
- OUT tail_free_size int4,
- OUT n_pending_pages bigint,
- OUT n_pending_tuples bigint,
- OUT n_total_pages bigint,
- OUT n_entry_pages bigint,
- OUT n_data_pages bigint,
- OUT n_entries bigint,
- OUT version int4)
-AS 'MODULE_PATHNAME', 'gin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- gin_page_opaque_info()
---
-CREATE FUNCTION gin_page_opaque_info(IN page bytea,
- OUT rightlink bigint,
- OUT maxoff int4,
- OUT flags text[])
-AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
-LANGUAGE C STRICT;
-
---
--- gin_leafpage_items()
---
-CREATE FUNCTION gin_leafpage_items(IN page bytea,
- OUT first_tid tid,
- OUT nbytes int2,
- OUT tids tid[])
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'gin_leafpage_items'
-LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect--1.4.sql b/contrib/pageinspect/pageinspect--1.4.sql
new file mode 100644
index 0000000..f75aa0b
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.4.sql
@@ -0,0 +1,279 @@
+/* contrib/pageinspect/pageinspect--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
+
+--
+-- get_raw_page()
+--
+CREATE FUNCTION get_raw_page(text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION get_raw_page(text, text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page_fork'
+LANGUAGE C STRICT;
+
+--
+-- page_header()
+--
+CREATE FUNCTION page_header(IN page bytea,
+ OUT lsn pg_lsn,
+ OUT checksum smallint,
+ OUT flags smallint,
+ OUT lower smallint,
+ OUT upper smallint,
+ OUT special smallint,
+ OUT pagesize smallint,
+ OUT version smallint,
+ OUT prune_xid xid)
+AS 'MODULE_PATHNAME', 'page_header'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_items()
+--
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(rel_oid oid,
+ t_data bytea,
+ t_infomask integer,
+ t_infomask2 integer,
+ t_bits text)
+RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+CREATE FUNCTION tuple_data_split(rel_oid oid,
+ t_data bytea,
+ t_infomask integer,
+ t_infomask2 integer,
+ t_bits text,
+ do_detoast bool)
+RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(
+ IN page bytea,
+ IN rel_oid regclass,
+ IN do_detoast bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record AS $$
+SELECT lp,
+ lp_off,
+ lp_flags,
+ lp_len,
+ t_xmin,
+ t_xmax,
+ t_field3,
+ t_ctid,
+ t_infomask2,
+ t_infomask,
+ t_hoff,
+ t_bits,
+ t_oid,
+ tuple_data_split(
+ rel_oid,
+ t_data,
+ t_infomask,
+ t_infomask2,
+ t_bits,
+ do_detoast)
+ AS t_attrs
+ FROM heap_page_items(page);
+$$ LANGUAGE SQL;
+
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN rel_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record AS $$
+SELECT * from heap_page_item_attrs(page, rel_oid, false);
+$$ LANGUAGE SQL;
+
+--
+-- bt_metap()
+--
+CREATE FUNCTION bt_metap(IN relname text,
+ OUT magic int4,
+ OUT version int4,
+ OUT root int4,
+ OUT level int4,
+ OUT fastroot int4,
+ OUT fastlevel int4)
+AS 'MODULE_PATHNAME', 'bt_metap'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_stats()
+--
+CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
+ OUT blkno int4,
+ OUT type "char",
+ OUT live_items int4,
+ OUT dead_items int4,
+ OUT avg_item_size int4,
+ OUT page_size int4,
+ OUT free_size int4,
+ OUT btpo_prev int4,
+ OUT btpo_next int4,
+ OUT btpo int4,
+ OUT btpo_flags int4)
+AS 'MODULE_PATHNAME', 'bt_page_stats'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_items()
+--
+CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
+ OUT itemoffset smallint,
+ OUT ctid tid,
+ OUT itemlen smallint,
+ OUT nulls bool,
+ OUT vars bool,
+ OUT data text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'bt_page_items'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_type()
+--
+CREATE FUNCTION brin_page_type(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'brin_page_type'
+LANGUAGE C STRICT;
+
+--
+-- brin_metapage_info()
+--
+CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
+ OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
+AS 'MODULE_PATHNAME', 'brin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- brin_revmap_data()
+--
+CREATE FUNCTION brin_revmap_data(IN page bytea,
+ OUT pages tid)
+RETURNS SETOF tid
+AS 'MODULE_PATHNAME', 'brin_revmap_data'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_items()
+--
+CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
+ OUT itemoffset int,
+ OUT blknum int,
+ OUT attnum int,
+ OUT allnulls bool,
+ OUT hasnulls bool,
+ OUT placeholder bool,
+ OUT value text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'brin_page_items'
+LANGUAGE C STRICT;
+
+--
+-- fsm_page_contents()
+--
+CREATE FUNCTION fsm_page_contents(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'fsm_page_contents'
+LANGUAGE C STRICT;
+
+--
+-- GIN functions
+--
+
+--
+-- gin_metapage_info()
+--
+CREATE FUNCTION gin_metapage_info(IN page bytea,
+ OUT pending_head bigint,
+ OUT pending_tail bigint,
+ OUT tail_free_size int4,
+ OUT n_pending_pages bigint,
+ OUT n_pending_tuples bigint,
+ OUT n_total_pages bigint,
+ OUT n_entry_pages bigint,
+ OUT n_data_pages bigint,
+ OUT n_entries bigint,
+ OUT version int4)
+AS 'MODULE_PATHNAME', 'gin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_page_opaque_info()
+--
+CREATE FUNCTION gin_page_opaque_info(IN page bytea,
+ OUT rightlink bigint,
+ OUT maxoff int4,
+ OUT flags text[])
+AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_leafpage_items()
+--
+CREATE FUNCTION gin_leafpage_items(IN page bytea,
+ OUT first_tid tid,
+ OUT nbytes int2,
+ OUT tids tid[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'gin_leafpage_items'
+LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect.control b/contrib/pageinspect/pageinspect.control
index a9dab33..68c7d61 100644
--- a/contrib/pageinspect/pageinspect.control
+++ b/contrib/pageinspect/pageinspect.control
@@ -1,5 +1,5 @@
# pageinspect extension
comment = 'inspect the contents of database pages at a low level'
-default_version = '1.3'
+default_version = '1.4'
module_pathname = '$libdir/pageinspect'
relocatable = true
diff --git a/doc/src/sgml/pageinspect.sgml b/doc/src/sgml/pageinspect.sgml
index b95cc81..5d187ed 100644
--- a/doc/src/sgml/pageinspect.sgml
+++ b/doc/src/sgml/pageinspect.sgml
@@ -93,9 +93,10 @@ test=# SELECT * FROM page_header(get_raw_page('pg_class', 0));
<listitem>
<para>
<function>heap_page_items</function> shows all line pointers on a heap
- page. For those line pointers that are in use, tuple headers are also
- shown. All tuples are shown, whether or not the tuples were visible to
- an MVCC snapshot at the time the raw page was copied.
+ page. For those line pointers that are in use, tuple headers as well
+ as tuple raw data are also shown. All tuples are shown, whether or not
+ the tuples were visible to an MVCC snapshot at the time the raw page
+ was copied.
</para>
<para>
A heap page image obtained with <function>get_raw_page</function> should
@@ -112,6 +113,56 @@ test=# SELECT * FROM heap_page_items(get_raw_page('pg_class', 0));
<varlistentry>
<term>
+ <function>tuple_data_split(rel_oid, t_data bytea, t_infomask integer, t_infomask2 integer, t_bits text [, do_detoast bool]) returns bytea[]</function>
+ <indexterm>
+ <primary>tuple_data_split</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ <function>tuple_data_split</function> splits tuple data into attributes
+ in the same way as backend internals.
+<screen>
+test=# SELECT tuple_data_split('pg_class'::regclass, t_data, t_infomask, t_infomask2, t_bits) FROM heap_page_items(get_raw_page('pg_class', 0));
+</screen>
+ This function should be called with the same arguments as the return
+ attributes of <function>heap_page_items</function>.
+ </para>
+ <para>
+ If <parameter>do_detoast</parameter> is <literal>true</literal>,
+ attribute that will be detoasted as needed. Default value is
+ <literal>false</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <function>heap_page_item_attrs(rel_oid, t_data bytea, [, do_detoast bool]) returns bytea[]</function>
+ <indexterm>
+ <primary>heap_page_item_attrs</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ <function>heap_page_item_attrs</function> is equivalent to
+ <function>heap_page_items</function> except that it returns
+ tuple raw data as an array of attributes that can optionally
+ be detoasted by <parameter>do_detoast</parameter> which is
+ <literal>false</literal> by default.
+ </para>
+ <para>
+ A heap page image obtained with <function>get_raw_page</function> should
+ be passed as argument. For example:
+<screen>
+test=# SELECT * FROM heap_page_item_attrs(get_raw_page('pg_class', 0), 'pg_class'::regclass);
+</screen>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
<function>bt_metap(relname text) returns record</function>
<indexterm>
<primary>bt_metap</primary>
Re: pageinspect patch, for showing tuple data
On Wed, Nov 18, 2015 at 3:10 AM, Nikolay Shaplov wrote:
Everything seems to be ok. I've changed only one thing in your version
of the patch. I've renamed split_tuple_data to
tuple_data_split_internal, because when there are split_tuple_data and
tuple_data_split in the same file, nobody will guess what the difference
between these function and why are they named in such a strange way.
Yep, that sounds better this way.
If it is ok, set proper status, and let commiter do his job :-)
OK. I have switched the status of this patch to "Ready for committer"
(please, committer-san, double-check the area around
tuple_data_split_internal when fetching data for each attribute, I
think that we got that right but I may be missing something as well).
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
Hi,
Le 12 nov. 2015 1:05 AM, "Michael Paquier" <michael.paquier@gmail.com> a
écrit :
On Thu, Nov 12, 2015 at 12:41 AM, Nikolay Shaplov
<n.shaplov@postgrespro.ru> wrote:В письме от 28 октября 2015 16:57:36 пользователь Michael Paquier
написал:
On Sat, Oct 17, 2015 at 1:48 AM, Michael Paquier wrote:
On Sat, Oct 17, 2015 at 5:15 AM, Nikolay Shaplov wrote:
Or it's ready to commit, and just not marked this way?
No, I don't think we have reached this state yet.
I am going to make report based on this patch in Vienna. It would be
nice to have it committed until then :)This is registered in this November's CF and the current one is not
completely wrapped up, so I haven't rushed into looking at it.So, I have finally been able to look at this patch in more details,
resulting in the attached. I noticed a couple of things that should be
addressed, mainly:
- addition of a new routine text_to_bits to perform the reverse
operation of bits_to_text. This was previously part of
tuple_data_split, I think that it deserves its own function.
- split_tuple_data should be static
- t_bits_str should not be a text, rather a char* fetched using
text_to_cstring(PG_GETARG_TEXT_PP(4)). This way there is no need to
perform extra calculations with VARSIZE and VARHDRSZ
- split_tuple_data can directly use the relation OID instead of the
tuple descriptor of the relation
- t_bits was leaking memory. For correctness I think that it is better
to free it after calling split_tuple_data.
- PG_DETOAST_DATUM_COPY allocates some memory, this was leaking as
well in raw_attr actually. I refactored the code such as a bytea* is
used and always freed when allocated to avoid leaks. Removing raw_attr
actually simplified the code a bit.
- I simplified the docs, that was largely too verbose in my opinion.
- Instead of using VARATT_IS_1B_E and VARTAG_EXTERNAL, using
VARATT_IS_EXTERNAL and VARATT_IS_EXTERNAL_ONDISK seems more adapted to
me, those other ones are much more low-level and not really spread in
the backend code.
- Found some typos in the code, the docs and some comments. I reworked
the error messages as well.
- The code format was not really in line with the project guidelines.
I fixed that as well.
- I removed heap_page_item_attrs for now to get this patch in a
bare-bone state. Though I would not mind if this is re-added (I
personally don't think that's much necessary in the module
actually...).
- The calculation of the length of t_bits using HEAP_NATTS_MASK is
more correct as you mentioned earlier, so I let it in its state.
That's actually more accurate for error handling as well.
That's everything I recall I have. How does this look?You've completely rewrite everything ;-)
Let everything be the way you wrote. This code is better than mine.
With one exception. I really need heap_page_item_attrs function. I
used it in
my Tuples Internals presentation
https://github.com/dhyannataraj/tuple-internals-presentation
and I am 100% sure that this function is needed for educational
purposes, and
this function should be as simple as possible, so students can use it
without
extra efforts.
Fine. That's your patch after all.
I still have an opinion that documentation should be more verbose, than
your
version, but I can accept your version.
I am not sure that's necessary, pageinspect is for developers.
FWIW, I agree that pageinspect is mostly for devs. Still, as i said to
Nikolay after his talk at pgconf.eu, it's a nice tool for people who like
to know what's going on deep inside PostgreSQL.
So +1 for that nice feature.
Show quoted text
Who is going to add heap_page_item_attrs to your patch? me or you?
I recall some parts of the code I still did not like much :) I'll grab
some room to have an extra look at it.
Re: pageinspect patch, for showing tuple data
- bits_len = tuphdr->t_hoff -
- offsetof(HeapTupleHeaderData, t_bits);
+ int bits_len =
+ ((tuphdr->t_infomask2 & HEAP_NATTS_MASK) / 8 + 1) * 8;As I understand offline comments of Nikolay, current version of page inspect
contains an mistake here. Should we backpatch this?
OK. I have switched the status of this patch to "Ready for committer"
(please, committer-san, double-check the area around
tuple_data_split_internal when fetching data for each attribute, I
think that we got that right but I may be missing something as well).
Looks good for a first glance
--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
On Wed, Nov 25, 2015 at 10:06 PM, Teodor Sigaev <teodor@sigaev.ru> wrote:
- bits_len = tuphdr->t_hoff - - offsetof(HeapTupleHeaderData, t_bits); + int bits_len = + ((tuphdr->t_infomask2 & HEAP_NATTS_MASK) / 8 + 1) * 8;As I understand offline comments of Nikolay, current version of page
inspect contains an mistake here. Should we backpatch this?
As far as I understood from the code, the current code in pageinspect is
overestimating the length of t_bits. So that's actually harmless. For the
sake of this patch though this is needed to perform the sanity checks in
place when scanning raw page entries.
--
Michael
Re: pageinspect patch, for showing tuple data
В письме от 18 ноября 2015 15:52:54 пользователь Michael Paquier написал:
Teodor Sigaev asked a very good question: does it properly do upgrade from 1.3
to 1.4
I've rechecked and fixed
here is a patch.
On Wed, Nov 18, 2015 at 3:10 AM, Nikolay Shaplov wrote:
Everything seems to be ok. I've changed only one thing in your version
of the patch. I've renamed split_tuple_data to
tuple_data_split_internal, because when there are split_tuple_data and
tuple_data_split in the same file, nobody will guess what the difference
between these function and why are they named in such a strange way.Yep, that sounds better this way.
If it is ok, set proper status, and let commiter do his job :-)
OK. I have switched the status of this patch to "Ready for committer"
(please, committer-san, double-check the area around
tuple_data_split_internal when fetching data for each attribute, I
think that we got that right but I may be missing something as well).
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
pageinspect_paque_v3.difftext/x-patch; charset=UTF-8; name=pageinspect_paque_v3.diffDownload
diff --git a/contrib/pageinspect/Makefile b/contrib/pageinspect/Makefile
index aec5258..91ab119 100644
--- a/contrib/pageinspect/Makefile
+++ b/contrib/pageinspect/Makefile
@@ -5,9 +5,9 @@ OBJS = rawpage.o heapfuncs.o btreefuncs.o fsmfuncs.o \
brinfuncs.o ginfuncs.o $(WIN32RES)
EXTENSION = pageinspect
-DATA = pageinspect--1.3.sql pageinspect--1.2--1.3.sql \
- pageinspect--1.1--1.2.sql pageinspect--1.0--1.1.sql \
- pageinspect--unpackaged--1.0.sql
+DATA = pageinspect--1.4.sql pageinspect--1.3--1.4.sql \
+ pageinspect--1.2--1.3.sql pageinspect--1.1--1.2.sql \
+ pageinspect--1.0--1.1.sql pageinspect--unpackaged--1.0.sql
PGFILEDESC = "pageinspect - functions to inspect contents of database pages"
ifdef USE_PGXS
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 8d1666c..546a051 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -27,8 +27,11 @@
#include "access/htup_details.h"
#include "funcapi.h"
-#include "utils/builtins.h"
+#include "catalog/pg_type.h"
#include "miscadmin.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
+#include "utils/rel.h"
/*
@@ -55,6 +58,42 @@ bits_to_text(bits8 *bits, int len)
/*
+ * text_to_bits
+ *
+ * Converts a c-string representation of bits into a bits8-array. This is
+ * the reverse operation of previous routine.
+ */
+static bits8 *
+text_to_bits(char *str, int len)
+{
+ bits8 *bits;
+ int off = 0;
+ char byte = 0;
+
+ bits = palloc(len + 1);
+
+ while (off < len)
+ {
+ if (off % 8 == 0)
+ byte = 0;
+
+ if ((str[off] == '0') || (str[off] == '1'))
+ byte = byte | ((str[off] - '0') << off % 8);
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("illegal character '%c' in t_bits string", str[off])));
+
+ if (off % 8 == 7)
+ bits[off / 8] = byte;
+
+ off++;
+ }
+
+ return bits;
+}
+
+/*
* heap_page_items
*
* Allows inspection of line pointers and tuple headers of a heap page.
@@ -122,8 +161,8 @@ heap_page_items(PG_FUNCTION_ARGS)
HeapTuple resultTuple;
Datum result;
ItemId id;
- Datum values[13];
- bool nulls[13];
+ Datum values[14];
+ bool nulls[14];
uint16 lp_offset;
uint16 lp_flags;
uint16 lp_len;
@@ -154,7 +193,8 @@ heap_page_items(PG_FUNCTION_ARGS)
lp_offset + lp_len <= raw_page_size)
{
HeapTupleHeader tuphdr;
- int bits_len;
+ bytea *tuple_data_bytea;
+ int tuple_data_len;
/* Extract information from the tuple header */
@@ -168,6 +208,14 @@ heap_page_items(PG_FUNCTION_ARGS)
values[9] = UInt32GetDatum(tuphdr->t_infomask);
values[10] = UInt8GetDatum(tuphdr->t_hoff);
+ /* Copy raw tuple data into bytea attribute */
+ tuple_data_len = lp_len - tuphdr->t_hoff;
+ tuple_data_bytea = (bytea *) palloc(tuple_data_len + VARHDRSZ);
+ SET_VARSIZE(tuple_data_bytea, tuple_data_len + VARHDRSZ);
+ memcpy(VARDATA(tuple_data_bytea), (char *) tuphdr + tuphdr->t_hoff,
+ tuple_data_len);
+ values[13] = PointerGetDatum(tuple_data_bytea);
+
/*
* We already checked that the item is completely within the raw
* page passed to us, with the length given in the line pointer.
@@ -180,11 +228,11 @@ heap_page_items(PG_FUNCTION_ARGS)
{
if (tuphdr->t_infomask & HEAP_HASNULL)
{
- bits_len = tuphdr->t_hoff -
- offsetof(HeapTupleHeaderData, t_bits);
+ int bits_len =
+ ((tuphdr->t_infomask2 & HEAP_NATTS_MASK) / 8 + 1) * 8;
values[11] = CStringGetTextDatum(
- bits_to_text(tuphdr->t_bits, bits_len * 8));
+ bits_to_text(tuphdr->t_bits, bits_len));
}
else
nulls[11] = true;
@@ -208,7 +256,7 @@ heap_page_items(PG_FUNCTION_ARGS)
*/
int i;
- for (i = 4; i <= 12; i++)
+ for (i = 4; i <= 13; i++)
nulls[i] = true;
}
@@ -223,3 +271,205 @@ heap_page_items(PG_FUNCTION_ARGS)
else
SRF_RETURN_DONE(fctx);
}
+
+/*
+ * tuple_data_split_internal
+ *
+ * Split raw tuple data taken directly from a page into an array of bytea
+ * elements. This routine does a lookup on NULL values and creates array
+ * elements accordindly. This is a reimplementation of nocachegetattr()
+ * in heaptuple.c simplified for educational purposes.
+ */
+static Datum
+tuple_data_split_internal(Oid relid, char *tupdata,
+ uint16 tupdata_len, uint16 t_infomask,
+ uint16 t_infomask2, bits8 *t_bits,
+ bool do_detoast)
+{
+ ArrayBuildState *raw_attrs;
+ int nattrs;
+ int i;
+ int off = 0;
+ Relation rel;
+ TupleDesc tupdesc;
+
+ /* Get tuple descriptor from relation OID */
+ rel = relation_open(relid, NoLock);
+ tupdesc = CreateTupleDescCopyConstr(rel->rd_att);
+ relation_close(rel, NoLock);
+
+ raw_attrs = initArrayResult(BYTEAOID, CurrentMemoryContext, false);
+ nattrs = tupdesc->natts;
+
+ if (nattrs < (t_infomask2 & HEAP_NATTS_MASK))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("number of attributes in tuple header is greater than number of attributes in tuple descriptor")));
+
+ for (i = 0; i < nattrs; i++)
+ {
+ Form_pg_attribute attr;
+ bool is_null;
+ bytea *attr_data = NULL;
+
+ attr = tupdesc->attrs[i];
+ is_null = (t_infomask & HEAP_HASNULL) && att_isnull(i, t_bits);
+
+ /*
+ * Tuple header can specify less attributes than tuple descriptor
+ * as ALTER TABLE ADD COLUMN without DEFAULT keyword does not
+ * actually change tuples in pages, so attributes with numbers greater
+ * than (t_infomask2 & HEAP_NATTS_MASK) should be treated as NULL.
+ */
+ if (i >= (t_infomask2 & HEAP_NATTS_MASK))
+ is_null = true;
+
+ if (!is_null)
+ {
+ int len;
+
+ if (attr->attlen == -1)
+ {
+ off = att_align_pointer(off, tupdesc->attrs[i]->attalign, -1,
+ tupdata + off);
+ /*
+ * As VARSIZE_ANY throws an exception if it can't properly detect
+ * the type of external storage in macros VARTAG_SIZE, this check
+ * is repeated to have a nicer error handling.
+ */
+ if (VARATT_IS_EXTERNAL(tupdata + off) &&
+ !VARATT_IS_EXTERNAL_ONDISK(tupdata + off) &&
+ !VARATT_IS_EXTERNAL_INDIRECT(tupdata + off))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("first byte of varlena attribute is incorrect for attribute %d", i)));
+
+ len = VARSIZE_ANY(tupdata + off);
+ }
+ else
+ {
+ off = att_align_nominal(off, tupdesc->attrs[i]->attalign);
+ len = attr->attlen;
+ }
+
+ if (tupdata_len < off + len)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("unexpected end of tuple data")));
+
+ if (attr->attlen == -1 && do_detoast)
+ attr_data = DatumGetByteaPCopy(tupdata + off);
+ else
+ {
+ attr_data = (bytea *) palloc(len + VARHDRSZ);
+ SET_VARSIZE(attr_data, len + VARHDRSZ);
+ memcpy(VARDATA(attr_data), tupdata + off, len);
+ }
+
+ off = att_addlength_pointer(off, tupdesc->attrs[i]->attlen,
+ tupdata + off);
+ }
+
+ raw_attrs = accumArrayResult(raw_attrs, PointerGetDatum(attr_data),
+ is_null, BYTEAOID, CurrentMemoryContext);
+ if (attr_data)
+ pfree(attr_data);
+ }
+
+ if (tupdata_len != off)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("end of tuple reached without looking at all its data")));
+
+ return makeArrayResult(raw_attrs, CurrentMemoryContext);
+}
+
+/*
+ * tuple_data_split
+ *
+ * Split raw tuple data taken directly from page into distinct elements
+ * taking into account null values.
+ */
+PG_FUNCTION_INFO_V1(tuple_data_split);
+
+Datum
+tuple_data_split(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ bytea *raw_data;
+ uint16 t_infomask;
+ uint16 t_infomask2;
+ char *t_bits_str;
+ bool do_detoast = false;
+ bits8 *t_bits = NULL;
+ Datum res;
+
+ relid = PG_GETARG_OID(0);
+ raw_data = PG_ARGISNULL(1) ? NULL : PG_GETARG_BYTEA_P(1);
+ t_infomask = PG_GETARG_INT16(2);
+ t_infomask2 = PG_GETARG_INT16(3);
+ t_bits_str = PG_ARGISNULL(4) ? NULL :
+ text_to_cstring(PG_GETARG_TEXT_PP(4));
+
+ if (PG_NARGS() >= 6)
+ do_detoast = PG_GETARG_BOOL(5);
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to use raw page functions")));
+
+ if (!raw_data)
+ PG_RETURN_NULL();
+
+ /*
+ * Convert t_bits string back to the bits8 array as represented in the
+ * tuple header.
+ */
+ if (t_infomask & HEAP_HASNULL)
+ {
+ int bits_str_len;
+ int bits_len;
+
+ bits_len = (t_infomask2 & HEAP_NATTS_MASK) / 8 + 1;
+ if (!t_bits_str)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("argument of t_bits is null, but it is expected to be null and %i character long",
+ bits_len * 8)));
+
+ bits_str_len = strlen(t_bits_str);
+ if ((bits_str_len % 8) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("length of t_bits is not a multiple of eight")));
+
+ if (bits_len * 8 != bits_str_len)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("unexpected length of t_bits %u, expected %i",
+ bits_str_len, bits_len * 8)));
+
+ /* do the conversion */
+ t_bits = text_to_bits(t_bits_str, bits_str_len);
+ }
+ else
+ {
+ if (t_bits_str)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("t_bits string is expected to be NULL, but instead it is %lu bytes length",
+ strlen(t_bits_str))));
+ }
+
+ /* Split tuple data */
+ res = tuple_data_split_internal(relid, (char *) raw_data + VARHDRSZ,
+ VARSIZE(raw_data) - VARHDRSZ,
+ t_infomask, t_infomask2, t_bits,
+ do_detoast);
+
+ if (t_bits)
+ pfree(t_bits);
+
+ PG_RETURN_ARRAYTYPE_P(res);
+}
diff --git a/contrib/pageinspect/pageinspect--1.3--1.4.sql b/contrib/pageinspect/pageinspect--1.3--1.4.sql
new file mode 100644
index 0000000..86e0dfa
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.3--1.4.sql
@@ -0,0 +1,118 @@
+/* contrib/pageinspect/pageinspect--1.3--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via ALTER EXTENSION
+\echo Use "ALTER EXTENSION pageinspect UPDATE TO '1.4'" to load this file. \quit
+
+--
+-- heap_page_items()
+--
+DROP FUNCTION heap_page_items(bytea);
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(rel_oid oid,
+ t_data bytea,
+ t_infomask integer,
+ t_infomask2 integer,
+ t_bits text)
+RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+CREATE FUNCTION tuple_data_split(rel_oid oid,
+ t_data bytea,
+ t_infomask integer,
+ t_infomask2 integer,
+ t_bits text,
+ do_detoast bool)
+RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(
+ IN page bytea,
+ IN rel_oid regclass,
+ IN do_detoast bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record AS $$
+SELECT lp,
+ lp_off,
+ lp_flags,
+ lp_len,
+ t_xmin,
+ t_xmax,
+ t_field3,
+ t_ctid,
+ t_infomask2,
+ t_infomask,
+ t_hoff,
+ t_bits,
+ t_oid,
+ tuple_data_split(
+ rel_oid,
+ t_data,
+ t_infomask,
+ t_infomask2,
+ t_bits,
+ do_detoast)
+ AS t_attrs
+ FROM heap_page_items(page);
+$$ LANGUAGE SQL;
+
+CREATE FUNCTION heap_page_item_attrs(
+ IN page bytea,
+ IN rel_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record AS $$
+SELECT * from heap_page_item_attrs(page, rel_oid, false);
+$$ LANGUAGE SQL;
diff --git a/contrib/pageinspect/pageinspect--1.3.sql b/contrib/pageinspect/pageinspect--1.3.sql
deleted file mode 100644
index a99e058..0000000
--- a/contrib/pageinspect/pageinspect--1.3.sql
+++ /dev/null
@@ -1,189 +0,0 @@
-/* contrib/pageinspect/pageinspect--1.3.sql */
-
--- complain if script is sourced in psql, rather than via CREATE EXTENSION
-\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
-
---
--- get_raw_page()
---
-CREATE FUNCTION get_raw_page(text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page'
-LANGUAGE C STRICT;
-
-CREATE FUNCTION get_raw_page(text, text, int4)
-RETURNS bytea
-AS 'MODULE_PATHNAME', 'get_raw_page_fork'
-LANGUAGE C STRICT;
-
---
--- page_header()
---
-CREATE FUNCTION page_header(IN page bytea,
- OUT lsn pg_lsn,
- OUT checksum smallint,
- OUT flags smallint,
- OUT lower smallint,
- OUT upper smallint,
- OUT special smallint,
- OUT pagesize smallint,
- OUT version smallint,
- OUT prune_xid xid)
-AS 'MODULE_PATHNAME', 'page_header'
-LANGUAGE C STRICT;
-
---
--- heap_page_items()
---
-CREATE FUNCTION heap_page_items(IN page bytea,
- OUT lp smallint,
- OUT lp_off smallint,
- OUT lp_flags smallint,
- OUT lp_len smallint,
- OUT t_xmin xid,
- OUT t_xmax xid,
- OUT t_field3 int4,
- OUT t_ctid tid,
- OUT t_infomask2 integer,
- OUT t_infomask integer,
- OUT t_hoff smallint,
- OUT t_bits text,
- OUT t_oid oid)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'heap_page_items'
-LANGUAGE C STRICT;
-
---
--- bt_metap()
---
-CREATE FUNCTION bt_metap(IN relname text,
- OUT magic int4,
- OUT version int4,
- OUT root int4,
- OUT level int4,
- OUT fastroot int4,
- OUT fastlevel int4)
-AS 'MODULE_PATHNAME', 'bt_metap'
-LANGUAGE C STRICT;
-
---
--- bt_page_stats()
---
-CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
- OUT blkno int4,
- OUT type "char",
- OUT live_items int4,
- OUT dead_items int4,
- OUT avg_item_size int4,
- OUT page_size int4,
- OUT free_size int4,
- OUT btpo_prev int4,
- OUT btpo_next int4,
- OUT btpo int4,
- OUT btpo_flags int4)
-AS 'MODULE_PATHNAME', 'bt_page_stats'
-LANGUAGE C STRICT;
-
---
--- bt_page_items()
---
-CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
- OUT itemoffset smallint,
- OUT ctid tid,
- OUT itemlen smallint,
- OUT nulls bool,
- OUT vars bool,
- OUT data text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'bt_page_items'
-LANGUAGE C STRICT;
-
---
--- brin_page_type()
---
-CREATE FUNCTION brin_page_type(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'brin_page_type'
-LANGUAGE C STRICT;
-
---
--- brin_metapage_info()
---
-CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
- OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
-AS 'MODULE_PATHNAME', 'brin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- brin_revmap_data()
---
-CREATE FUNCTION brin_revmap_data(IN page bytea,
- OUT pages tid)
-RETURNS SETOF tid
-AS 'MODULE_PATHNAME', 'brin_revmap_data'
-LANGUAGE C STRICT;
-
---
--- brin_page_items()
---
-CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
- OUT itemoffset int,
- OUT blknum int,
- OUT attnum int,
- OUT allnulls bool,
- OUT hasnulls bool,
- OUT placeholder bool,
- OUT value text)
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'brin_page_items'
-LANGUAGE C STRICT;
-
---
--- fsm_page_contents()
---
-CREATE FUNCTION fsm_page_contents(IN page bytea)
-RETURNS text
-AS 'MODULE_PATHNAME', 'fsm_page_contents'
-LANGUAGE C STRICT;
-
---
--- GIN functions
---
-
---
--- gin_metapage_info()
---
-CREATE FUNCTION gin_metapage_info(IN page bytea,
- OUT pending_head bigint,
- OUT pending_tail bigint,
- OUT tail_free_size int4,
- OUT n_pending_pages bigint,
- OUT n_pending_tuples bigint,
- OUT n_total_pages bigint,
- OUT n_entry_pages bigint,
- OUT n_data_pages bigint,
- OUT n_entries bigint,
- OUT version int4)
-AS 'MODULE_PATHNAME', 'gin_metapage_info'
-LANGUAGE C STRICT;
-
---
--- gin_page_opaque_info()
---
-CREATE FUNCTION gin_page_opaque_info(IN page bytea,
- OUT rightlink bigint,
- OUT maxoff int4,
- OUT flags text[])
-AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
-LANGUAGE C STRICT;
-
---
--- gin_leafpage_items()
---
-CREATE FUNCTION gin_leafpage_items(IN page bytea,
- OUT first_tid tid,
- OUT nbytes int2,
- OUT tids tid[])
-RETURNS SETOF record
-AS 'MODULE_PATHNAME', 'gin_leafpage_items'
-LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect--1.4.sql b/contrib/pageinspect/pageinspect--1.4.sql
new file mode 100644
index 0000000..f75aa0b
--- /dev/null
+++ b/contrib/pageinspect/pageinspect--1.4.sql
@@ -0,0 +1,279 @@
+/* contrib/pageinspect/pageinspect--1.4.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pageinspect" to load this file. \quit
+
+--
+-- get_raw_page()
+--
+CREATE FUNCTION get_raw_page(text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page'
+LANGUAGE C STRICT;
+
+CREATE FUNCTION get_raw_page(text, text, int4)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'get_raw_page_fork'
+LANGUAGE C STRICT;
+
+--
+-- page_header()
+--
+CREATE FUNCTION page_header(IN page bytea,
+ OUT lsn pg_lsn,
+ OUT checksum smallint,
+ OUT flags smallint,
+ OUT lower smallint,
+ OUT upper smallint,
+ OUT special smallint,
+ OUT pagesize smallint,
+ OUT version smallint,
+ OUT prune_xid xid)
+AS 'MODULE_PATHNAME', 'page_header'
+LANGUAGE C STRICT;
+
+--
+-- heap_page_items()
+--
+CREATE FUNCTION heap_page_items(IN page bytea,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_data bytea)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'heap_page_items'
+LANGUAGE C STRICT;
+
+--
+-- tuple_data_split()
+--
+CREATE FUNCTION tuple_data_split(rel_oid oid,
+ t_data bytea,
+ t_infomask integer,
+ t_infomask2 integer,
+ t_bits text)
+RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+CREATE FUNCTION tuple_data_split(rel_oid oid,
+ t_data bytea,
+ t_infomask integer,
+ t_infomask2 integer,
+ t_bits text,
+ do_detoast bool)
+RETURNS bytea[]
+AS 'MODULE_PATHNAME','tuple_data_split'
+LANGUAGE C;
+
+--
+-- heap_page_item_attrs()
+--
+CREATE FUNCTION heap_page_item_attrs(
+ IN page bytea,
+ IN rel_oid regclass,
+ IN do_detoast bool,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record AS $$
+SELECT lp,
+ lp_off,
+ lp_flags,
+ lp_len,
+ t_xmin,
+ t_xmax,
+ t_field3,
+ t_ctid,
+ t_infomask2,
+ t_infomask,
+ t_hoff,
+ t_bits,
+ t_oid,
+ tuple_data_split(
+ rel_oid,
+ t_data,
+ t_infomask,
+ t_infomask2,
+ t_bits,
+ do_detoast)
+ AS t_attrs
+ FROM heap_page_items(page);
+$$ LANGUAGE SQL;
+
+CREATE FUNCTION heap_page_item_attrs(IN page bytea, IN rel_oid regclass,
+ OUT lp smallint,
+ OUT lp_off smallint,
+ OUT lp_flags smallint,
+ OUT lp_len smallint,
+ OUT t_xmin xid,
+ OUT t_xmax xid,
+ OUT t_field3 int4,
+ OUT t_ctid tid,
+ OUT t_infomask2 integer,
+ OUT t_infomask integer,
+ OUT t_hoff smallint,
+ OUT t_bits text,
+ OUT t_oid oid,
+ OUT t_attrs bytea[]
+ )
+RETURNS SETOF record AS $$
+SELECT * from heap_page_item_attrs(page, rel_oid, false);
+$$ LANGUAGE SQL;
+
+--
+-- bt_metap()
+--
+CREATE FUNCTION bt_metap(IN relname text,
+ OUT magic int4,
+ OUT version int4,
+ OUT root int4,
+ OUT level int4,
+ OUT fastroot int4,
+ OUT fastlevel int4)
+AS 'MODULE_PATHNAME', 'bt_metap'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_stats()
+--
+CREATE FUNCTION bt_page_stats(IN relname text, IN blkno int4,
+ OUT blkno int4,
+ OUT type "char",
+ OUT live_items int4,
+ OUT dead_items int4,
+ OUT avg_item_size int4,
+ OUT page_size int4,
+ OUT free_size int4,
+ OUT btpo_prev int4,
+ OUT btpo_next int4,
+ OUT btpo int4,
+ OUT btpo_flags int4)
+AS 'MODULE_PATHNAME', 'bt_page_stats'
+LANGUAGE C STRICT;
+
+--
+-- bt_page_items()
+--
+CREATE FUNCTION bt_page_items(IN relname text, IN blkno int4,
+ OUT itemoffset smallint,
+ OUT ctid tid,
+ OUT itemlen smallint,
+ OUT nulls bool,
+ OUT vars bool,
+ OUT data text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'bt_page_items'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_type()
+--
+CREATE FUNCTION brin_page_type(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'brin_page_type'
+LANGUAGE C STRICT;
+
+--
+-- brin_metapage_info()
+--
+CREATE FUNCTION brin_metapage_info(IN page bytea, OUT magic text,
+ OUT version integer, OUT pagesperrange integer, OUT lastrevmappage bigint)
+AS 'MODULE_PATHNAME', 'brin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- brin_revmap_data()
+--
+CREATE FUNCTION brin_revmap_data(IN page bytea,
+ OUT pages tid)
+RETURNS SETOF tid
+AS 'MODULE_PATHNAME', 'brin_revmap_data'
+LANGUAGE C STRICT;
+
+--
+-- brin_page_items()
+--
+CREATE FUNCTION brin_page_items(IN page bytea, IN index_oid regclass,
+ OUT itemoffset int,
+ OUT blknum int,
+ OUT attnum int,
+ OUT allnulls bool,
+ OUT hasnulls bool,
+ OUT placeholder bool,
+ OUT value text)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'brin_page_items'
+LANGUAGE C STRICT;
+
+--
+-- fsm_page_contents()
+--
+CREATE FUNCTION fsm_page_contents(IN page bytea)
+RETURNS text
+AS 'MODULE_PATHNAME', 'fsm_page_contents'
+LANGUAGE C STRICT;
+
+--
+-- GIN functions
+--
+
+--
+-- gin_metapage_info()
+--
+CREATE FUNCTION gin_metapage_info(IN page bytea,
+ OUT pending_head bigint,
+ OUT pending_tail bigint,
+ OUT tail_free_size int4,
+ OUT n_pending_pages bigint,
+ OUT n_pending_tuples bigint,
+ OUT n_total_pages bigint,
+ OUT n_entry_pages bigint,
+ OUT n_data_pages bigint,
+ OUT n_entries bigint,
+ OUT version int4)
+AS 'MODULE_PATHNAME', 'gin_metapage_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_page_opaque_info()
+--
+CREATE FUNCTION gin_page_opaque_info(IN page bytea,
+ OUT rightlink bigint,
+ OUT maxoff int4,
+ OUT flags text[])
+AS 'MODULE_PATHNAME', 'gin_page_opaque_info'
+LANGUAGE C STRICT;
+
+--
+-- gin_leafpage_items()
+--
+CREATE FUNCTION gin_leafpage_items(IN page bytea,
+ OUT first_tid tid,
+ OUT nbytes int2,
+ OUT tids tid[])
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'gin_leafpage_items'
+LANGUAGE C STRICT;
diff --git a/contrib/pageinspect/pageinspect.control b/contrib/pageinspect/pageinspect.control
index a9dab33..68c7d61 100644
--- a/contrib/pageinspect/pageinspect.control
+++ b/contrib/pageinspect/pageinspect.control
@@ -1,5 +1,5 @@
# pageinspect extension
comment = 'inspect the contents of database pages at a low level'
-default_version = '1.3'
+default_version = '1.4'
module_pathname = '$libdir/pageinspect'
relocatable = true
diff --git a/doc/src/sgml/pageinspect.sgml b/doc/src/sgml/pageinspect.sgml
index b95cc81..5d187ed 100644
--- a/doc/src/sgml/pageinspect.sgml
+++ b/doc/src/sgml/pageinspect.sgml
@@ -93,9 +93,10 @@ test=# SELECT * FROM page_header(get_raw_page('pg_class', 0));
<listitem>
<para>
<function>heap_page_items</function> shows all line pointers on a heap
- page. For those line pointers that are in use, tuple headers are also
- shown. All tuples are shown, whether or not the tuples were visible to
- an MVCC snapshot at the time the raw page was copied.
+ page. For those line pointers that are in use, tuple headers as well
+ as tuple raw data are also shown. All tuples are shown, whether or not
+ the tuples were visible to an MVCC snapshot at the time the raw page
+ was copied.
</para>
<para>
A heap page image obtained with <function>get_raw_page</function> should
@@ -112,6 +113,56 @@ test=# SELECT * FROM heap_page_items(get_raw_page('pg_class', 0));
<varlistentry>
<term>
+ <function>tuple_data_split(rel_oid, t_data bytea, t_infomask integer, t_infomask2 integer, t_bits text [, do_detoast bool]) returns bytea[]</function>
+ <indexterm>
+ <primary>tuple_data_split</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ <function>tuple_data_split</function> splits tuple data into attributes
+ in the same way as backend internals.
+<screen>
+test=# SELECT tuple_data_split('pg_class'::regclass, t_data, t_infomask, t_infomask2, t_bits) FROM heap_page_items(get_raw_page('pg_class', 0));
+</screen>
+ This function should be called with the same arguments as the return
+ attributes of <function>heap_page_items</function>.
+ </para>
+ <para>
+ If <parameter>do_detoast</parameter> is <literal>true</literal>,
+ attribute that will be detoasted as needed. Default value is
+ <literal>false</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <function>heap_page_item_attrs(rel_oid, t_data bytea, [, do_detoast bool]) returns bytea[]</function>
+ <indexterm>
+ <primary>heap_page_item_attrs</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ <function>heap_page_item_attrs</function> is equivalent to
+ <function>heap_page_items</function> except that it returns
+ tuple raw data as an array of attributes that can optionally
+ be detoasted by <parameter>do_detoast</parameter> which is
+ <literal>false</literal> by default.
+ </para>
+ <para>
+ A heap page image obtained with <function>get_raw_page</function> should
+ be passed as argument. For example:
+<screen>
+test=# SELECT * FROM heap_page_item_attrs(get_raw_page('pg_class', 0), 'pg_class'::regclass);
+</screen>
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
<function>bt_metap(relname text) returns record</function>
<indexterm>
<primary>bt_metap</primary>
Re: pageinspect patch, for showing tuple data
On Wed, Nov 25, 2015 at 10:16 PM, Nikolay Shaplov <n.shaplov@postgrespro.ru>
wrote:
В письме от 18 ноября 2015 15:52:54 пользователь Michael Paquier написал:
Teodor Sigaev asked a very good question: does it properly do upgrade from
1.3
to 1.4I've rechecked and fixed
What was actually the problem? I have to admit that I forgot to test that
directly, and did not spot anything obvious on the 1.3--1.4.sql file.
--
Michael
Re: pageinspect patch, for showing tuple data
Teodor Sigaev asked a very good question: does it properly do upgrade from 1.3
to 1.4I've rechecked and fixed
here is a patch.
Committed, thank you.
--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: pageinspect patch, for showing tuple data
В письме от 25 ноября 2015 22:27:57 пользователь Michael Paquier написал:
On Wed, Nov 25, 2015 at 10:16 PM, Nikolay Shaplov <n.shaplov@postgrespro.ru>
wrote:В письме от 18 ноября 2015 15:52:54 пользователь Michael Paquier написал:
Teodor Sigaev asked a very good question: does it properly do upgrade from
1.3
to 1.4I've rechecked and fixed
What was actually the problem?
tuple_data_split should be defined before heap_page_item_attrs.
And there were also error in the first argument of tuple_data_split there. It
should be "rel_oid oid" instead of "t_oid rel_oid"
I have to admit that I forgot to test that
directly, and did not spot anything obvious on the 1.3--1.4.sql file.
yes. but each of us added a non-obvious mistake there :-)
--
Nikolay Shaplov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers