Vaccum and analyze counters in pgstat

Started by Magnus Haganderover 15 years ago3 messages
#1Magnus Hagander
magnus@hagander.net
1 attachment(s)

Attached is a patch that adds columns to pg_stat_*_tables for number
of [auto]vacuum and [auto]analyze runs on a table, completing the
current one that just had the last time these ran. It's particularly
useful to see how much autovac is doing on the tables, but I included
the counts of regular vacuum and analyze as well for completeness.

Comments?

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Attachments:

stat_vacuum_counters.patchapplication/octet-stream; name=stat_vacuum_counters.patchDownload
*** a/doc/src/sgml/monitoring.sgml
--- b/doc/src/sgml/monitoring.sgml
***************
*** 117,125 **** postgres: <replaceable>user</> <replaceable>database</> <replaceable>host</> <re
     is a subsystem that supports collection and reporting of information about
     server activity.  Presently, the collector can count accesses to tables
     and indexes in both disk-block and individual-row terms.  It also tracks
!    the total number of rows in each table, and the last vacuum and analyze times
!    for each table.  It can also count calls to user-defined functions and
!    the total time spent in each one.
    </para>
  
    <para>
--- 117,125 ----
     is a subsystem that supports collection and reporting of information about
     server activity.  Presently, the collector can count accesses to tables
     and indexes in both disk-block and individual-row terms.  It also tracks
!    the total number of rows in each table, and information about vacuum and
!    analyze for each table.  It can also count calls to user-defined functions
!    and the total time spent in each one.
    </para>
  
    <para>
***************
*** 293,299 **** postgres: <replaceable>user</> <replaceable>database</> <replaceable>host</> <re
        the last time the table was vacuumed manually,
        the last time it was vacuumed by the autovacuum daemon,
        the last time it was analyzed manually,
!       and the last time it was analyzed by the autovacuum daemon.
        </entry>
       </row>
  
--- 293,303 ----
        the last time the table was vacuumed manually,
        the last time it was vacuumed by the autovacuum daemon,
        the last time it was analyzed manually,
!       the last time it was analyzed by the autovacuum daemon,
!       number of times it has been vacuumed manually,
!       number of times it has been vacuumed by the autovacuum daemon,
!       number of times it has been analyzed manually,
!       and the number of times it has been analyzed by the autovacuum daemon.
        </entry>
       </row>
  
***************
*** 314,321 **** postgres: <replaceable>user</> <replaceable>database</> <replaceable>host</> <re
        <entry>Similar to <structname>pg_stat_all_tables</>, but counts actions
        taken so far within the current transaction (which are <emphasis>not</>
        yet included in <structname>pg_stat_all_tables</> and related views).
!       The columns for numbers of live and dead rows and last-vacuum and
!       last-analyze times are not present in this view.</entry>
       </row>
  
       <row>
--- 318,325 ----
        <entry>Similar to <structname>pg_stat_all_tables</>, but counts actions
        taken so far within the current transaction (which are <emphasis>not</>
        yet included in <structname>pg_stat_all_tables</> and related views).
!       The columns for numbers of live and dead rows and vacuum and
!       analyze values are not present in this view.</entry>
       </row>
  
       <row>
***************
*** 719,724 **** postgres: <replaceable>user</> <replaceable>database</> <replaceable>host</> <re
--- 723,760 ----
       </row>
  
       <row>
+       <entry><literal><function>pg_stat_get_vacuum_count</function>(<type>oid</type>)</literal></entry>
+       <entry><type>bigint</type></entry>
+       <entry>
+        The number of times this table has been vacuumed manually
+       </entry>
+      </row>
+ 
+      <row>
+       <entry><literal><function>pg_stat_get_autovacuum_count</function>(<type>oid</type>)</literal></entry>
+       <entry><type>bigint</type></entry>
+       <entry>
+        The number of times this table has been vacuumed by the autovacuum daemon
+       </entry>
+      </row>
+ 
+      <row>
+       <entry><literal><function>pg_stat_get_analyze_count</function>(<type>oid</type>)</literal></entry>
+       <entry><type>bigint</type></entry>
+       <entry>
+        The number of times this table has been analyzed manually
+       </entry>
+      </row>
+ 
+      <row>
+       <entry><literal><function>pg_stat_get_autoanalyze_count</function>(<type>oid</type>)</literal></entry>
+       <entry><type>bigint</type></entry>
+       <entry>
+        The number of times this table has been analyzed by the autovacuum daemon
+       </entry>
+      </row>
+ 
+      <row>
        <entry><literal><function>pg_stat_get_xact_numscans</function>(<type>oid</type>)</literal></entry>
        <entry><type>bigint</type></entry>
        <entry>
*** a/src/backend/catalog/system_views.sql
--- b/src/backend/catalog/system_views.sql
***************
*** 201,207 **** CREATE VIEW pg_stat_all_tables AS
              pg_stat_get_last_vacuum_time(C.oid) as last_vacuum,
              pg_stat_get_last_autovacuum_time(C.oid) as last_autovacuum,
              pg_stat_get_last_analyze_time(C.oid) as last_analyze,
!             pg_stat_get_last_autoanalyze_time(C.oid) as last_autoanalyze
      FROM pg_class C LEFT JOIN 
           pg_index I ON C.oid = I.indrelid 
           LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace) 
--- 201,211 ----
              pg_stat_get_last_vacuum_time(C.oid) as last_vacuum,
              pg_stat_get_last_autovacuum_time(C.oid) as last_autovacuum,
              pg_stat_get_last_analyze_time(C.oid) as last_analyze,
!             pg_stat_get_last_autoanalyze_time(C.oid) as last_autoanalyze,
!             pg_stat_get_vacuum_count(C.oid) AS vacuum_count,
!             pg_stat_get_autovacuum_count(C.oid) AS autovacuum_count,
!             pg_stat_get_analyze_count(C.oid) AS analyze_count,
!             pg_stat_get_autoanalyze_count(C.oid) AS autoanalyze_count
      FROM pg_class C LEFT JOIN 
           pg_index I ON C.oid = I.indrelid 
           LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace) 
*** a/src/backend/postmaster/pgstat.c
--- b/src/backend/postmaster/pgstat.c
***************
*** 3192,3197 **** pgstat_get_tab_entry(PgStat_StatDBEntry *dbentry, Oid tableoid, bool create)
--- 3192,3201 ----
  		result->autovac_vacuum_timestamp = 0;
  		result->analyze_timestamp = 0;
  		result->autovac_analyze_timestamp = 0;
+ 		result->vacuum_count = 0;
+ 		result->autovac_vacuum_count = 0;
+ 		result->analyze_count = 0;
+ 		result->autovac_analyze_count = 0;
  	}
  
  	return result;
***************
*** 4114,4122 **** pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len)
--- 4118,4132 ----
  	tabentry->n_dead_tuples = 0;
  
  	if (msg->m_autovacuum)
+ 	{
  		tabentry->autovac_vacuum_timestamp = msg->m_vacuumtime;
+ 		tabentry->autovac_vacuum_count++;
+ 	}
  	else
+ 	{
  		tabentry->vacuum_timestamp = msg->m_vacuumtime;
+ 		tabentry->vacuum_count++;
+ 	}
  }
  
  /* ----------
***************
*** 4151,4159 **** pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len)
--- 4161,4175 ----
  	tabentry->changes_since_analyze = 0;
  
  	if (msg->m_autovacuum)
+ 	{
  		tabentry->autovac_analyze_timestamp = msg->m_analyzetime;
+ 		tabentry->autovac_analyze_count++;
+ 	}
  	else
+ 	{
  		tabentry->analyze_timestamp = msg->m_analyzetime;
+ 		tabentry->analyze_count++;
+ 	}
  }
  
  
*** a/src/backend/utils/adt/pgstatfuncs.c
--- b/src/backend/utils/adt/pgstatfuncs.c
***************
*** 38,43 **** extern Datum pg_stat_get_last_vacuum_time(PG_FUNCTION_ARGS);
--- 38,47 ----
  extern Datum pg_stat_get_last_autovacuum_time(PG_FUNCTION_ARGS);
  extern Datum pg_stat_get_last_analyze_time(PG_FUNCTION_ARGS);
  extern Datum pg_stat_get_last_autoanalyze_time(PG_FUNCTION_ARGS);
+ extern Datum pg_stat_get_vacuum_count(PG_FUNCTION_ARGS);
+ extern Datum pg_stat_get_autovacuum_count(PG_FUNCTION_ARGS);
+ extern Datum pg_stat_get_analyze_count(PG_FUNCTION_ARGS);
+ extern Datum pg_stat_get_autoanalyze_count(PG_FUNCTION_ARGS);
  
  extern Datum pg_stat_get_function_calls(PG_FUNCTION_ARGS);
  extern Datum pg_stat_get_function_time(PG_FUNCTION_ARGS);
***************
*** 347,352 **** pg_stat_get_last_autoanalyze_time(PG_FUNCTION_ARGS)
--- 351,404 ----
  }
  
  Datum
+ pg_stat_get_vacuum_count(PG_FUNCTION_ARGS)
+ {
+ 	Oid			relid = PG_GETARG_OID(0);
+ 	PgStat_StatTabEntry	*tabentry;
+ 
+ 	if ((tabentry = pgstat_fetch_stat_tabentry(relid)) == NULL)
+ 		PG_RETURN_NULL();
+ 	else
+ 		PG_RETURN_INT64(tabentry->vacuum_count);
+ }
+ 
+ Datum
+ pg_stat_get_autovacuum_count(PG_FUNCTION_ARGS)
+ {
+ 	Oid			relid = PG_GETARG_OID(0);
+ 	PgStat_StatTabEntry	*tabentry;
+ 
+ 	if ((tabentry = pgstat_fetch_stat_tabentry(relid)) == NULL)
+ 		PG_RETURN_NULL();
+ 	else
+ 		PG_RETURN_INT64(tabentry->autovac_vacuum_count);
+ }
+ 
+ Datum
+ pg_stat_get_analyze_count(PG_FUNCTION_ARGS)
+ {
+ 	Oid			relid = PG_GETARG_OID(0);
+ 	PgStat_StatTabEntry	*tabentry;
+ 
+ 	if ((tabentry = pgstat_fetch_stat_tabentry(relid)) == NULL)
+ 		PG_RETURN_NULL();
+ 	else
+ 		PG_RETURN_INT64(tabentry->analyze_count);
+ }
+ 
+ Datum
+ pg_stat_get_autoanalyze_count(PG_FUNCTION_ARGS)
+ {
+ 	Oid			relid = PG_GETARG_OID(0);
+ 	PgStat_StatTabEntry	*tabentry;
+ 
+ 	if ((tabentry = pgstat_fetch_stat_tabentry(relid)) == NULL)
+ 		PG_RETURN_NULL();
+ 	else
+ 		PG_RETURN_INT64(tabentry->autovac_analyze_count);
+ }
+ 
+ Datum
  pg_stat_get_function_calls(PG_FUNCTION_ARGS)
  {
  	Oid			funcid = PG_GETARG_OID(0);
*** a/src/include/catalog/pg_proc.h
--- b/src/include/catalog/pg_proc.h
***************
*** 3029,3034 **** DATA(insert OID = 2783 (  pg_stat_get_last_analyze_time PGNSP PGUID 12 1 0 0 f f
--- 3029,3042 ----
  DESCR("statistics: last manual analyze time for a table");
  DATA(insert OID = 2784 (  pg_stat_get_last_autoanalyze_time PGNSP PGUID 12 1 0 0 f f f t f s 1 0 1184 "26" _null_ _null_ _null_ _null_	pg_stat_get_last_autoanalyze_time _null_ _null_ _null_ ));
  DESCR("statistics: last auto analyze time for a table");
+ DATA(insert OID = 3054 ( pg_stat_get_vacuum_count PGNSP PGUID 12 1 0 0 f f f t f s 1 0 20 "26" _null_ _null_ _null_ _null_ pg_stat_get_vacuum_count _null_ _null_ _null_ ));
+ DESCR("statistics: number of manual vacuums for a table");
+ DATA(insert OID = 3055 ( pg_stat_get_autovacuum_count PGNSP PGUID 12 1 0 0 f f f t f s 1 0 20 "26" _null_ _null_ _null_ _null_ pg_stat_get_autovacuum_count _null_ _null_ _null_ ));
+ DESCR("statistics: number of auto vacuums for a table");
+ DATA(insert OID = 3056 ( pg_stat_get_analyze_count PGNSP PGUID 12 1 0 0 f f f t f s 1 0 20 "26" _null_ _null_ _null_ _null_ pg_stat_get_analyze_count _null_ _null_ _null_ ));
+ DESCR("statistics: number of manual analyzes for a table");
+ DATA(insert OID = 3057 ( pg_stat_get_autoanalyze_count PGNSP PGUID 12 1 0 0 f f f t f s 1 0 20 "26" _null_ _null_ _null_ _null_ pg_stat_get_autoanalyze_count _null_ _null_ _null_ ));
+ DESCR("statistics: number of auto analyzes for a table");
  DATA(insert OID = 1936 (  pg_stat_get_backend_idset		PGNSP PGUID 12 1 100 0 f f f t t s 0 0 23 "" _null_ _null_ _null_ _null_ pg_stat_get_backend_idset _null_ _null_ _null_ ));
  DESCR("statistics: currently active backend IDs");
  DATA(insert OID = 2022 (  pg_stat_get_activity			PGNSP PGUID 12 1 100 0 f f f f t s 1 0 2249 "23" "{23,26,23,26,25,25,16,1184,1184,1184,869,23}" "{i,o,o,o,o,o,o,o,o,o,o,o}" "{pid,datid,procpid,usesysid,application_name,current_query,waiting,xact_start,query_start,backend_start,client_addr,client_port}" _null_ pg_stat_get_activity _null_ _null_ _null_ ));
*** a/src/include/pgstat.h
--- b/src/include/pgstat.h
***************
*** 525,533 **** typedef struct PgStat_StatTabEntry
--- 525,537 ----
  	PgStat_Counter blocks_hit;
  
  	TimestampTz vacuum_timestamp;		/* user initiated vacuum */
+ 	PgStat_Counter vacuum_count;
  	TimestampTz autovac_vacuum_timestamp;		/* autovacuum initiated */
+ 	PgStat_Counter autovac_vacuum_count;
  	TimestampTz analyze_timestamp;		/* user initiated */
+ 	PgStat_Counter analyze_count;
  	TimestampTz autovac_analyze_timestamp;		/* autovacuum initiated */
+ 	PgStat_Counter autovac_analyze_count;
  } PgStat_StatTabEntry;
  
  
#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Magnus Hagander (#1)
Re: Vaccum and analyze counters in pgstat

Magnus Hagander <magnus@hagander.net> writes:

Attached is a patch that adds columns to pg_stat_*_tables for number
of [auto]vacuum and [auto]analyze runs on a table, completing the
current one that just had the last time these ran. It's particularly
useful to see how much autovac is doing on the tables, but I included
the counts of regular vacuum and analyze as well for completeness.

Comments?

Looks reasonably sane in a quick read-through.

--- 117,125 ----
is a subsystem that supports collection and reporting of information about
server activity.  Presently, the collector can count accesses to tables
and indexes in both disk-block and individual-row terms.  It also tracks
!    the total number of rows in each table, and information about vacuum and
!    analyze for each table.  It can also count calls to user-defined functions
!    and the total time spent in each one.
</para>

"information about vacuum and analyze actions" might read better.

--- 318,325 ----
<entry>Similar to <structname>pg_stat_all_tables</>, but counts actions
taken so far within the current transaction (which are <emphasis>not</>
yet included in <structname>pg_stat_all_tables</> and related views).
!       The columns for numbers of live and dead rows and vacuum and
!       analyze values are not present in this view.</entry>
</row>

Likewise values -> actions here.

regards, tom lane

#3Magnus Hagander
magnus@hagander.net
In reply to: Tom Lane (#2)
Re: Vaccum and analyze counters in pgstat

On Fri, Aug 20, 2010 at 15:49, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Magnus Hagander <magnus@hagander.net> writes:

Attached is a patch that adds columns to pg_stat_*_tables for number
of [auto]vacuum and [auto]analyze runs on a table, completing the
current one that just had the last time these ran. It's particularly
useful to see how much autovac is doing on the tables, but I included
the counts of regular vacuum and analyze as well for completeness.

Comments?

Looks reasonably sane in a quick read-through.

Applied with suggested documentation changes.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/