[FEATURE PATCH] pg_stat_statements with plans

Started by Julian Markwortalmost 9 years ago25 messages
#1Julian Markwort
julian.markwort@uni-muenster.de
1 attachment(s)

Hello psql-hackers!

TL:DR;
We extended the functionality of pg_stat_statements so it can track
worst and best case execution plans.

Based on a suggestion of my colleague Arne Scheffer, Marius Timmer and I
extended pg_stat_statements so it can also record execution plans,
whenever the execution time is exceeded (or deceeded) by a definable
factor.
We were largely inspired by the pg_stat_plans extension by Peter
Geoghegan and Simon Riggs - we don't claim any originality on this part
- which is unfortunately not available on newer postgresql versions.
There are a few differences which will become apparent in the following
lines.

By default, the modified pg_stat_statements extension will now track
good plans and bad plans for each entry in pg_stat_statements.
The plans are not normalized or hashed (as opposed to pg_stat_plans),
they represent discreet statements.
A good plan is saved, whenever this sort of query has been used for the
first time or the time of the previously recorded good plan has been
deceeded by a smaller factor than 0.9 .
Analogous to this, a bad_plan is saved, when the time has been exceeded
by a factor greater than 1.1 .
There are GUCs available so these parameters can be tuned to your
liking. Tracking can be disabled for both plans individually.
A plan_format can be defined to enable better readability or
processability through other tools.

You can reset your good and bad plans by using a
select on pg_stat_statements_good_plan_reset([queryid]);
resetting bad plans uses pg_stat_statements_bad_plan_reset, obviously.
In case of a reset, the execution time, timestamp and plan itself are
just set to 0 respective NULL.

The pg_stat_statements view now provides six extra columns:
good_plan, good_plan_time, good_plan_timestamp, bad_plan, bad_plan_time
and bad_plan_timestamp.

Plans are only displayed if the showtext argument is true and the user
is the superuser or the user who has been associated with that entry.

Furthermore, we implemented a GUC that allows you to control the maximum
refresh frequency to avoid performance impacts on restarts or resets.
A plan is only updated when tracking is enabled and more time than
"plan_min_interval" has passed (default: 5 seconds) and the previously
mentioned conditions for the execution time have been met.

The major selling point of this feature?
Beeing able to find plans that need optimization (e.g. by creating
indexes). As pg_stat_statements tracks normalized queries, there might
be certain values or even daytimes that result in very bad plans, while
others result in perfectly fine plans.
Of course, the GUC log_min_duration_statement can also detect long
runners, but the advantage of pg_stat_statements is that we count the
total calls of normalized queries, which enables us to find plans, that
don't count as long runners, while their aggregated time might show
shortcomings regarding their plans.

We've found this sort of tool really useful when dealing with queries
produced by ORM libraries, where optimization is not intuitive.

Various tests using pg_bench suggest that this extension does not worsen
the performance of the database.

We're really looking forward to your opinions and feedback on this
feature patch
Julian, Marius and Arne

Attachments:

pgss_plans_v01.patchtext/x-patch; name=pgss_plans_v01.patchDownload
diff --git a/contrib/pg_stat_statements/Makefile b/contrib/pg_stat_statements/Makefile
index 298951a..2a22eb5 100644
--- a/contrib/pg_stat_statements/Makefile
+++ b/contrib/pg_stat_statements/Makefile
@@ -4,7 +4,7 @@ MODULE_big = pg_stat_statements
 OBJS = pg_stat_statements.o $(WIN32RES)
 
 EXTENSION = pg_stat_statements
-DATA = pg_stat_statements--1.4.sql pg_stat_statements--1.3--1.4.sql \
+DATA = pg_stat_statements--1.5.sql pg_stat_statements--1.4--1.5.sql pg_stat_statements--1.3--1.4.sql \
 	pg_stat_statements--1.2--1.3.sql pg_stat_statements--1.1--1.2.sql \
 	pg_stat_statements--1.0--1.1.sql pg_stat_statements--unpackaged--1.0.sql
 PGFILEDESC = "pg_stat_statements - execution statistics of SQL statements"
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 6edc3d9..a3cfe6d 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -61,7 +61,9 @@
 #include <sys/stat.h>
 #include <unistd.h>
 
+#include "utils/timestamp.h"
 #include "access/hash.h"
+#include "commands/explain.h"
 #include "executor/instrument.h"
 #include "funcapi.h"
 #include "mb/pg_wchar.h"
@@ -118,7 +120,8 @@ typedef enum pgssVersion
 	PGSS_V1_0 = 0,
 	PGSS_V1_1,
 	PGSS_V1_2,
-	PGSS_V1_3
+	PGSS_V1_3,
+	PGSS_V1_5
 } pgssVersion;
 
 /*
@@ -159,6 +162,14 @@ typedef struct Counters
 	double		usage;			/* usage factor */
 } Counters;
 
+typedef struct pgssPlan
+{
+	Size offset;
+	int len;
+	double		time;	/* execution time in msec when the latest plan was updated */
+	TimestampTz timestamp;
+} pgssPlan;
+
 /*
  * Statistics per statement
  *
@@ -172,6 +183,8 @@ typedef struct pgssEntry
 	Counters	counters;		/* the statistics for this query */
 	Size		query_offset;	/* query text offset in external file */
 	int			query_len;		/* # of valid bytes in query string, or -1 */
+	pgssPlan 	good_plan;
+	pgssPlan 	bad_plan;
 	int			encoding;		/* query text encoding */
 	slock_t		mutex;			/* protects the counters only */
 } pgssEntry;
@@ -183,7 +196,7 @@ typedef struct pgssSharedState
 {
 	LWLock	   *lock;			/* protects hashtable search/modification */
 	double		cur_median_usage;		/* current median usage in hashtable */
-	Size		mean_query_len; /* current mean entry text length */
+	Size		mean_text_len; /* current mean entry text length */
 	slock_t		mutex;			/* protects following fields only: */
 	Size		extent;			/* current extent of query file */
 	int			n_writers;		/* number of active writers to query file */
@@ -256,10 +269,25 @@ static const struct config_enum_entry track_options[] =
 	{NULL, 0, false}
 };
 
-static int	pgss_max;			/* max # statements to track */
-static int	pgss_track;			/* tracking level */
-static bool pgss_track_utility; /* whether to track utility commands */
-static bool pgss_save;			/* whether to save stats across shutdown */
+static const struct config_enum_entry format_options[] =
+{
+	{"text", EXPLAIN_FORMAT_TEXT, false},
+	{"xml", EXPLAIN_FORMAT_XML, false},
+	{"json", EXPLAIN_FORMAT_JSON, false},
+	{"yaml", EXPLAIN_FORMAT_YAML, false},
+	{NULL, 0, false}
+};
+
+static int	pgss_max;				/* max # statements to track */
+static int	pgss_track;				/* tracking level */
+static bool pgss_track_utility; 	/* whether to track utility commands */
+static bool pgss_save;				/* whether to save stats across shutdown */
+static bool pgss_good_plan_enable; 	/* whether to save good_plans */
+static bool pgss_bad_plan_enable; 	/* whether to save good_plans */
+static double pgss_good_plan_delta; /* min derivation to save a new good_plan */
+static double pgss_bad_plan_delta; 	/* min derivation to save a new bad_plan */
+static int pgss_plan_min_interval;		/* minimum time in seconds that must pass so a new plan could be saved */
+static int  pgss_plan_format;   	/* id which sets the output format */
 
 
 #define pgss_enabled() \
@@ -282,7 +310,11 @@ void		_PG_fini(void);
 PG_FUNCTION_INFO_V1(pg_stat_statements_reset);
 PG_FUNCTION_INFO_V1(pg_stat_statements_1_2);
 PG_FUNCTION_INFO_V1(pg_stat_statements_1_3);
+PG_FUNCTION_INFO_V1(pg_stat_statements_1_5);
 PG_FUNCTION_INFO_V1(pg_stat_statements);
+PG_FUNCTION_INFO_V1(pg_stat_statements_good_plan_reset);
+PG_FUNCTION_INFO_V1(pg_stat_statements_bad_plan_reset);
+
 
 static void pgss_shmem_startup(void);
 static void pgss_shmem_shutdown(int code, Datum arg);
@@ -301,12 +333,17 @@ static int	pgss_match_fn(const void *key1, const void *key2, Size keysize);
 static uint32 pgss_hash_string(const char *str, int len);
 static void pgss_store(const char *query, uint32 queryId,
 		   int query_location, int query_len,
+		   const char *plan,
 		   double total_time, uint64 rows,
 		   const BufferUsage *bufusage,
 		   pgssJumbleState *jstate);
+Datum pg_stat_statements_plan_reset(uint32 query_id, uint8 plan_type);
 static void pg_stat_statements_internal(FunctionCallInfo fcinfo,
 							pgssVersion api_version,
 							bool showtext);
+static int fill_plan_times(Datum values[], bool nulls[], int i, pgssPlan *plan);
+static int fill_plan_str(Datum values[], bool nulls[], int i, pgssPlan *plan,
+							pgssEntry *entry, char *qbuffer, Size qbuffer_size);
 static Size pgss_memsize(void);
 static pgssEntry *entry_alloc(pgssHashKey *key, Size query_offset, int query_len,
 			int encoding, bool sticky);
@@ -331,7 +368,6 @@ static void fill_in_constant_lengths(pgssJumbleState *jstate, const char *query,
 						 int query_loc);
 static int	comp_location(const void *a, const void *b);
 
-
 /*
  * Module load callback
  */
@@ -399,6 +435,80 @@ _PG_init(void)
 							 NULL,
 							 NULL);
 
+	DefineCustomBoolVariable("pg_stat_statements.good_plan_enable",
+							 "Enable bad plan detection",
+							 NULL,
+							 &pgss_good_plan_enable,
+							 true,
+							 PGC_SUSET,
+							 0,
+							 NULL,
+							 NULL,
+							 NULL);
+
+	DefineCustomBoolVariable("pg_stat_statements.bad_plan_enable",
+							 "Enable bad plan detection",
+							 NULL,
+							 &pgss_bad_plan_enable,
+							 true,
+							 PGC_SUSET,
+							 0,
+							 NULL,
+							 NULL,
+							 NULL);
+
+	DefineCustomRealVariable("pg_stat_statements.good_plan_delta",
+							"If execution time deceeds last good_plan_time by a factor smaller than this, the plan will be stored.",
+							NULL,
+							&pgss_good_plan_delta,
+							0.9,
+							0.0,
+							1.0,
+							PGC_SUSET,
+							0,
+							NULL,
+							NULL,
+							NULL);
+
+	DefineCustomRealVariable("pg_stat_statements.bad_plan_delta",
+							"If execution time exceeds last bad_plan_time by a factor greater than this, the plan will be stored.",
+							NULL,
+							&pgss_bad_plan_delta,
+							1.1,
+							1.01,
+							10.0,
+							PGC_SUSET,
+							0,
+							NULL,
+							NULL,
+							NULL);
+
+	DefineCustomIntVariable("pg_stat_statements.plan_min_interval",
+							"Sets the time in seconds that has to pass before a new plan could be saved.",
+							NULL,
+							&pgss_plan_min_interval,
+							5,
+							0,
+							INT_MAX,
+							PGC_POSTMASTER,
+							0,
+							NULL,
+							NULL,
+							NULL);
+
+	DefineCustomEnumVariable("pg_stat_statements.plan_format",
+							"Sets the output format for the plans.",
+							"Notice that the plan format can not be changed after a plan is saved. Valid values are text, json, xml and yaml.",
+							&pgss_plan_format,
+							EXPLAIN_FORMAT_TEXT,
+							format_options,
+							PGC_SUSET,
+							0,
+							NULL,
+							NULL,
+							NULL);
+
+
 	EmitWarningsOnPlaceholders("pg_stat_statements");
 
 	/*
@@ -462,7 +572,11 @@ pgss_shmem_startup(void)
 	int32		pgver;
 	int32		i;
 	int			buffer_size;
-	char	   *buffer = NULL;
+	int			good_plan_buffer_size;
+	int			bad_plan_buffer_size;
+	char	   *query_buffer = NULL;
+	char	   *good_plan_buffer = NULL;
+	char	   *bad_plan_buffer = NULL;
 
 	if (prev_shmem_startup_hook)
 		prev_shmem_startup_hook();
@@ -485,7 +599,7 @@ pgss_shmem_startup(void)
 		/* First time through ... */
 		pgss->lock = &(GetNamedLWLockTranche("pg_stat_statements"))->lock;
 		pgss->cur_median_usage = ASSUMED_MEDIAN_INIT;
-		pgss->mean_query_len = ASSUMED_LENGTH_INIT;
+		pgss->mean_text_len = ASSUMED_LENGTH_INIT;
 		SpinLockInit(&pgss->mutex);
 		pgss->extent = 0;
 		pgss->n_writers = 0;
@@ -555,7 +669,11 @@ pgss_shmem_startup(void)
 	}
 
 	buffer_size = 2048;
-	buffer = (char *) palloc(buffer_size);
+	good_plan_buffer_size = 2048;
+	bad_plan_buffer_size = 2048;
+	query_buffer = (char *) palloc(buffer_size);
+	good_plan_buffer  = (char *) palloc(good_plan_buffer_size);
+	bad_plan_buffer = (char *) palloc(bad_plan_buffer_size);
 
 	if (fread(&header, sizeof(uint32), 1, file) != 1 ||
 		fread(&pgver, sizeof(uint32), 1, file) != 1 ||
@@ -583,14 +701,42 @@ pgss_shmem_startup(void)
 		if (temp.query_len >= buffer_size)
 		{
 			buffer_size = Max(buffer_size * 2, temp.query_len + 1);
-			buffer = repalloc(buffer, buffer_size);
+			query_buffer = repalloc(query_buffer, buffer_size);
 		}
 
-		if (fread(buffer, 1, temp.query_len + 1, file) != temp.query_len + 1)
+		if (fread(query_buffer, 1, temp.query_len + 1, file)
+				!= temp.query_len + 1)
 			goto read_error;
 
+		/* Resize good plan_buffer and read into it */
+		if (temp.good_plan.len >= good_plan_buffer_size)
+		{
+			good_plan_buffer_size =
+				Max(good_plan_buffer_size * 2, temp.good_plan.len + 1);
+			good_plan_buffer= repalloc(good_plan_buffer, good_plan_buffer_size);
+		}
+
+		if (fread(good_plan_buffer, 1, temp.good_plan.len + 1, file)
+				!= temp.good_plan.len + 1)
+			goto read_error;
+
+		/* Resize bad plan_buffer and read into it */
+		if (temp.bad_plan.len >= bad_plan_buffer_size)
+		{
+			bad_plan_buffer_size =
+				Max(bad_plan_buffer_size * 2, temp.bad_plan.len + 1);
+			bad_plan_buffer= repalloc(bad_plan_buffer, bad_plan_buffer_size);
+		}
+
+		if (fread(bad_plan_buffer, 1, temp.bad_plan.len + 1, file)
+				!= temp.bad_plan.len + 1)
+			goto read_error;
+
+
 		/* Should have a trailing null, but let's make sure */
-		buffer[temp.query_len] = '\0';
+		query_buffer[temp.query_len] = '\0';
+		good_plan_buffer[temp.good_plan.len] = '\0';
+		bad_plan_buffer[temp.bad_plan.len] = '\0';
 
 		/* Skip loading "sticky" entries */
 		if (temp.counters.calls == 0)
@@ -598,10 +744,25 @@ pgss_shmem_startup(void)
 
 		/* Store the query text */
 		query_offset = pgss->extent;
-		if (fwrite(buffer, 1, temp.query_len + 1, qfile) != temp.query_len + 1)
+		if (fwrite(query_buffer, 1, temp.query_len + 1, qfile)
+				!= temp.query_len + 1)
 			goto write_error;
 		pgss->extent += temp.query_len + 1;
 
+		/* Store the good plan text*/
+		temp.good_plan.offset = pgss->extent;
+		if (fwrite(good_plan_buffer, 1, temp.good_plan.len + 1, qfile)
+				!= temp.good_plan.len + 1)
+			goto write_error;
+		pgss->extent += temp.good_plan.len + 1;
+
+		/* Store the bad plan text*/
+		temp.bad_plan.offset = pgss->extent;
+		if (fwrite(bad_plan_buffer, 1, temp.bad_plan.len + 1, qfile)
+				!= temp.bad_plan.len + 1)
+			goto write_error;
+		pgss->extent += temp.bad_plan.len + 1;
+
 		/* make the hashtable entry (discards old entries if too many) */
 		entry = entry_alloc(&temp.key, query_offset, temp.query_len,
 							temp.encoding,
@@ -609,9 +770,15 @@ pgss_shmem_startup(void)
 
 		/* copy in the actual stats */
 		entry->counters = temp.counters;
+		/* copy in the plans */
+		entry->good_plan = temp.good_plan;
+		entry->bad_plan = temp.bad_plan;
+
 	}
 
-	pfree(buffer);
+	pfree(query_buffer);
+	pfree(good_plan_buffer);
+	pfree(bad_plan_buffer);
 	FreeFile(file);
 	FreeFile(qfile);
 
@@ -650,8 +817,12 @@ write_error:
 			 errmsg("could not write pg_stat_statement file \"%s\": %m",
 					PGSS_TEXT_FILE)));
 fail:
-	if (buffer)
-		pfree(buffer);
+	if (query_buffer)
+		pfree(query_buffer);
+	if (good_plan_buffer)
+		pfree(good_plan_buffer);
+	if (bad_plan_buffer)
+		pfree(bad_plan_buffer);
 	if (file)
 		FreeFile(file);
 	if (qfile)
@@ -716,15 +887,52 @@ pgss_shmem_shutdown(int code, Datum arg)
 	hash_seq_init(&hash_seq, pgss_hash);
 	while ((entry = hash_seq_search(&hash_seq)) != NULL)
 	{
-		int			len = entry->query_len;
-		char	   *qstr = qtext_fetch(entry->query_offset, len,
+		int			query_len = entry->query_len;
+		int			good_plan_len = entry->good_plan.len;
+		int			bad_plan_len = entry->bad_plan.len;
+		char	   *query_string = qtext_fetch(entry->query_offset, query_len,
 									   qbuffer, qbuffer_size);
+		char       *good_plan_str;
+		char       *bad_plan_str;
+		if (good_plan_len > 0)
+		{
+			/* A good plan is available */
+			good_plan_str = qtext_fetch(entry->good_plan.offset, good_plan_len,
+												qbuffer, qbuffer_size);
+		} else {
+			/*
+			 * There is no good plan available. This can be caused by a query
+			 * which has no plan (for example insert or delete queries) or
+			 * because good plans are disabled. In this case we have to store
+			 * an empty string instead of null.
+			 */
+			good_plan_str = palloc(1 * sizeof(char));
+			*good_plan_str = '\0';
+		}
+		if (bad_plan_len > 0)
+		{
+			/* A bad plan is available */
+			bad_plan_str = qtext_fetch(entry->bad_plan.offset, bad_plan_len,
+											   qbuffer, qbuffer_size);
+		} else {
+			/*
+			 * There is no bad plan available. This can be caused by a query
+			 * which has no plan (for example insert or delete queries) or
+			 * because bad plans are disabled. In this case we have to store
+			 * an empty string instead of null.
+			 */
+			bad_plan_str = palloc(1 * sizeof(char));
+			*bad_plan_str = '\0';
+		}
 
-		if (qstr == NULL)
+		if (query_string == NULL)
 			continue;			/* Ignore any entries with bogus texts */
 
+		/* Write entrys, querys and plans serialized to the dump file */
 		if (fwrite(entry, sizeof(pgssEntry), 1, file) != 1 ||
-			fwrite(qstr, 1, len + 1, file) != len + 1)
+			fwrite(query_string, 1, query_len + 1, file) != query_len + 1 ||
+			fwrite(good_plan_str, 1, good_plan_len + 1, file) != good_plan_len + 1 ||
+			fwrite(bad_plan_str, 1, bad_plan_len + 1, file) != bad_plan_len + 1)
 		{
 			/* note: we assume hash_seq_term won't change errno */
 			hash_seq_term(&hash_seq);
@@ -827,6 +1035,7 @@ pgss_post_parse_analyze(ParseState *pstate, Query *query)
 				   query->queryId,
 				   query->stmt_location,
 				   query->stmt_len,
+				   NULL,
 				   0,
 				   0,
 				   NULL,
@@ -923,16 +1132,40 @@ pgss_ExecutorEnd(QueryDesc *queryDesc)
 
 	if (queryId != 0 && queryDesc->totaltime && pgss_enabled())
 	{
+		ExplainState *es = NewExplainState();
 		/*
 		 * Make sure stats accumulation is done.  (Note: it's okay if several
 		 * levels of hook all do this.)
 		 */
 		InstrEndLoop(queryDesc->totaltime);
 
+		es->analyze = (queryDesc->instrument_options && false);
+		es->verbose = false;
+		es->buffers = (es->analyze && false);
+		es->timing = (es->analyze && true);
+		es->format = pgss_plan_format;
+
+		ExplainBeginOutput(es);
+		ExplainQueryText(es, queryDesc);
+		ExplainPrintPlan(es, queryDesc);
+		ExplainEndOutput(es);
+
+		/* Remove last line break */
+		if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+			es->str->data[--es->str->len] = '\0';
+
+		/* Fix JSON to output an object */
+		if (pgss_plan_format == EXPLAIN_FORMAT_JSON)
+		{
+			es->str->data[0] = '{';
+			es->str->data[es->str->len - 1] = '}';
+		}
+
 		pgss_store(queryDesc->sourceText,
 				   queryId,
 				   queryDesc->plannedstmt->stmt_location,
 				   queryDesc->plannedstmt->stmt_len,
+				   es->str->data,								/* plan */
 				   queryDesc->totaltime->total * 1000.0,		/* convert to msec */
 				   queryDesc->estate->es_processed,
 				   &queryDesc->totaltime->bufusage,
@@ -1043,6 +1276,7 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
 				   0,			/* signal that it's a utility stmt */
 				   pstmt->stmt_location,
 				   pstmt->stmt_len,
+				   NULL,
 				   INSTR_TIME_GET_MILLISEC(duration),
 				   rows,
 				   &bufusage,
@@ -1102,6 +1336,28 @@ pgss_hash_string(const char *str, int len)
 	return hash_any((const unsigned char *) str, len);
 }
 
+static void
+update_plan(volatile pgssPlan* pgssPlan_ptr, const char* plan_str, int plan_len,
+			double total_time)
+{
+	if (plan_len > 0)
+	{
+		long secs;
+		int msecs;
+		TimestampDifference(pgssPlan_ptr->timestamp, GetCurrentTimestamp(), &secs, &msecs);
+		if(secs > pgss_plan_min_interval)
+		{
+			Size plan_offset;
+			qtext_store(plan_str, plan_len, &plan_offset, NULL);
+			pgssPlan_ptr->offset 	= plan_offset;
+			pgssPlan_ptr->len 		= plan_len;
+			pgssPlan_ptr->time 		= total_time;
+			pgssPlan_ptr->timestamp 	= GetCurrentTimestamp();
+		}
+	}
+	return;
+}
+
 /*
  * Store some statistics for a statement.
  *
@@ -1115,6 +1371,7 @@ pgss_hash_string(const char *str, int len)
 static void
 pgss_store(const char *query, uint32 queryId,
 		   int query_location, int query_len,
+		   const char *plan,
 		   double total_time, uint64 rows,
 		   const BufferUsage *bufusage,
 		   pgssJumbleState *jstate)
@@ -1123,6 +1380,7 @@ pgss_store(const char *query, uint32 queryId,
 	pgssEntry  *entry;
 	char	   *norm_query = NULL;
 	int			encoding = GetDatabaseEncoding();
+	int 		plan_len = 0;
 
 	Assert(query != NULL);
 
@@ -1153,6 +1411,9 @@ pgss_store(const char *query, uint32 queryId,
 		query_len = strlen(query);
 	}
 
+	if(plan)
+		plan_len = strlen(plan);
+
 	/*
 	 * Discard leading and trailing whitespace, too.  Use scanner_isspace()
 	 * not libc's isspace(), because we want to match the lexer's behavior.
@@ -1206,7 +1467,6 @@ pgss_store(const char *query, uint32 queryId,
 		/* Append new query text to file with only shared lock held */
 		stored = qtext_store(norm_query ? norm_query : query, query_len,
 							 &query_offset, &gc_count);
-
 		/*
 		 * Determine whether we need to garbage collect external query texts
 		 * while the shared lock is still held.  This micro-optimization
@@ -1233,7 +1493,12 @@ pgss_store(const char *query, uint32 queryId,
 		if (!stored)
 			goto done;
 
-		/* OK to create a new hashtable entry */
+		/*
+		 * OK to create a new hashtable entry.
+		 * We have to set the good- and the bad planlength to zero. Otherwise the
+		 * pgss_shmem_startup method will expect this number of chars which crashes
+		 * if there is no plan yet.
+		 */
 		entry = entry_alloc(&key, query_offset, query_len, encoding,
 							jstate != NULL);
 
@@ -1253,6 +1518,21 @@ pgss_store(const char *query, uint32 queryId,
 
 		SpinLockAcquire(&e->mutex);
 
+		/*
+		 * In case of a fast or initial plan we store a new good plan. We
+		 * are doing the same procedur for bad plans. So we store a empty
+		 * query, one empty and one full plan in the text file. After a
+		 * restart the new generated textfile will be "defragmented".
+		 */
+		if (pgss_good_plan_enable &&
+			((total_time < (e->good_plan.time * pgss_good_plan_delta)) ||
+				e->good_plan.time == 0))
+			update_plan(&e->good_plan, plan, plan_len, total_time);
+		if (pgss_bad_plan_enable &&
+			((total_time > (e->bad_plan.time * pgss_bad_plan_delta)) ||
+				e->bad_plan.time == 0))
+			update_plan(&e->bad_plan, plan, plan_len, total_time);
+
 		/* "Unstick" entry if it was previously sticky */
 		if (e->counters.calls == 0)
 			e->counters.usage = USAGE_INIT;
@@ -1324,12 +1604,89 @@ pg_stat_statements_reset(PG_FUNCTION_ARGS)
 	PG_RETURN_VOID();
 }
 
+Datum
+pg_stat_statements_plan_reset(uint32 query_id, uint8 plan_type)
+{
+	pgssEntry  *entry;
+
+	pgssHashKey key;
+
+	if (!pgss || !pgss_hash)
+		ereport(ERROR,
+				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+				 errmsg("pg_stat_statements must be loaded via shared_preload_libraries")));
+
+	/* Set up key for hashtable search */
+	key.userid = GetUserId();
+	key.dbid = MyDatabaseId;
+	key.queryid = query_id;
+
+	/* Lookup the hash table entry with shared lock. */
+	LWLockAcquire(pgss->lock, LW_SHARED);
+
+	entry = (pgssEntry *) hash_search(pgss_hash, &key, HASH_FIND, NULL);
+
+	if (entry)
+	{
+		if(superuser() || entry->key.userid == key.userid)
+		{
+			pgssPlan *pgssPlan_ptr;
+			switch (plan_type)
+			{
+				case 1:
+					pgssPlan_ptr = &entry->good_plan;
+					break;
+				case 2:
+					pgssPlan_ptr = &entry->bad_plan;
+					break;
+				default:
+					ereport(ERROR,
+						(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+							errmsg_internal("unrecognized plan type: %d",
+											plan_type)));
+					break;
+			}
+			if(pgssPlan_ptr)
+			{
+				pgssPlan_ptr->offset = -1;
+				pgssPlan_ptr->len = 0;
+				pgssPlan_ptr->time = 0;
+				pgssPlan_ptr->timestamp = 0;
+			}
+		}else{
+			elog(ERROR, "insufficient permission to reset plan");
+		}
+	}
+	LWLockRelease(pgss->lock);
+	PG_RETURN_VOID();
+}
+
+/*
+ * Reset the good_plan of the entry with queryid.
+ */
+ Datum
+ pg_stat_statements_good_plan_reset(PG_FUNCTION_ARGS)
+ {
+	return pg_stat_statements_plan_reset(PG_GETARG_INT64(0), 1);
+ }
+
+/*
+ * Reset the bad_plan of the entry with queryid.
+ */
+ Datum
+ pg_stat_statements_bad_plan_reset(PG_FUNCTION_ARGS)
+ {
+	return pg_stat_statements_plan_reset(PG_GETARG_INT64(0), 2);
+
+ }
+
 /* Number of output arguments (columns) for various API versions */
 #define PG_STAT_STATEMENTS_COLS_V1_0	14
 #define PG_STAT_STATEMENTS_COLS_V1_1	18
 #define PG_STAT_STATEMENTS_COLS_V1_2	19
 #define PG_STAT_STATEMENTS_COLS_V1_3	23
-#define PG_STAT_STATEMENTS_COLS			23		/* maximum of above */
+#define PG_STAT_STATEMENTS_COLS_V1_5	29
+#define PG_STAT_STATEMENTS_COLS			29		/* maximum of above */
 
 /*
  * Retrieve statement statistics.
@@ -1342,6 +1699,16 @@ pg_stat_statements_reset(PG_FUNCTION_ARGS)
  * function.  Unfortunately we weren't bright enough to do that for 1.1.
  */
 Datum
+pg_stat_statements_1_5(PG_FUNCTION_ARGS)
+{
+ bool		showtext = PG_GETARG_BOOL(0);
+
+ pg_stat_statements_internal(fcinfo, PGSS_V1_5, showtext);
+
+ return (Datum) 0;
+}
+
+Datum
 pg_stat_statements_1_3(PG_FUNCTION_ARGS)
 {
 	bool		showtext = PG_GETARG_BOOL(0);
@@ -1444,6 +1811,10 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 			if (api_version != PGSS_V1_3)
 				elog(ERROR, "incorrect number of output arguments");
 			break;
+		case PG_STAT_STATEMENTS_COLS_V1_5:
+			if (api_version != PGSS_V1_5)
+				elog(ERROR, "incorrect number of output arguments");
+			break;
 		default:
 			elog(ERROR, "incorrect number of output arguments");
 	}
@@ -1527,6 +1898,8 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 		bool		nulls[PG_STAT_STATEMENTS_COLS];
 		int			i = 0;
 		Counters	tmp;
+		pgssPlan 	tmp_good_plan;
+		pgssPlan 	tmp_bad_plan;
 		double		stddev;
 		int64		queryid = entry->key.queryid;
 
@@ -1543,7 +1916,8 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 
 			if (showtext)
 			{
-				char	   *qstr = qtext_fetch(entry->query_offset,
+				char	   *qstr = qtext_fetch
+				(entry->query_offset,
 											   entry->query_len,
 											   qbuffer,
 											   qbuffer_size);
@@ -1566,11 +1940,29 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 					/* Just return a null if we fail to find the text */
 					nulls[i++] = true;
 				}
+
+				if (api_version >= PGSS_V1_5)
+				{
+					/*
+					 * This version requires to print out the good
+					 * and the bad plan if they are enabled.
+					 */
+
+					i = fill_plan_str(values, nulls, i, &entry->good_plan,
+										entry, qbuffer, qbuffer_size);
+					i = fill_plan_str(values, nulls, i, &entry->bad_plan,
+										entry, qbuffer, qbuffer_size);
+				}
 			}
 			else
 			{
-				/* Query text not requested */
-				nulls[i++] = true;
+				/*
+				 * Query text not requested
+				*/
+				for (int j = 0; j < (api_version >= PGSS_V1_5 ? 3 : 1) ; j++)
+				{
+					nulls[i++] = true;
+				}
 			}
 		}
 		else
@@ -1595,6 +1987,8 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 
 			SpinLockAcquire(&e->mutex);
 			tmp = e->counters;
+			tmp_good_plan = e->good_plan;
+			tmp_bad_plan = e->bad_plan;
 			SpinLockRelease(&e->mutex);
 		}
 
@@ -1622,6 +2016,14 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 				stddev = 0.0;
 			values[i++] = Float8GetDatumFast(stddev);
 		}
+
+		if (api_version >= PGSS_V1_5)
+		{
+			/* Set plan times and timestamps (to avoid code duplication this iterator has been added*/
+			i = fill_plan_times(values, nulls, i, &tmp_good_plan);
+			i = fill_plan_times(values, nulls, i, &tmp_bad_plan);
+		}
+
 		values[i++] = Int64GetDatumFast(tmp.rows);
 		values[i++] = Int64GetDatumFast(tmp.shared_blks_hit);
 		values[i++] = Int64GetDatumFast(tmp.shared_blks_read);
@@ -1645,6 +2047,7 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 					 api_version == PGSS_V1_1 ? PG_STAT_STATEMENTS_COLS_V1_1 :
 					 api_version == PGSS_V1_2 ? PG_STAT_STATEMENTS_COLS_V1_2 :
 					 api_version == PGSS_V1_3 ? PG_STAT_STATEMENTS_COLS_V1_3 :
+					 api_version == PGSS_V1_5 ? PG_STAT_STATEMENTS_COLS_V1_5 :
 					 -1 /* fail if you forget to update this assert */ ));
 
 		tuplestore_putvalues(tupstore, tupdesc, values, nulls);
@@ -1659,6 +2062,53 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 	tuplestore_donestoring(tupstore);
 }
 
+/* This is only a helper-function to pg_stat_statements_internal */
+static int
+fill_plan_times(Datum values[], bool nulls[], int i, pgssPlan *plan)
+{
+	values[i++] = Float8GetDatumFast(plan->time);
+
+	/* if there is noting in the timestamp field, we are not interested in it */
+	if (plan->timestamp)
+		values[i++] = TimestampTzGetDatum(plan->timestamp);
+	else
+		nulls[i++] = true;
+	return i;
+}
+
+static int
+fill_plan_str(Datum values[], bool nulls[], int i, pgssPlan *plan,
+				pgssEntry *entry, char *qbuffer, Size qbuffer_size)
+{
+	if (plan && plan->len > 0)
+	{
+		char 	*pstr = qtext_fetch(plan->offset,
+								plan->len,
+								qbuffer,
+								qbuffer_size);
+		if (pstr)
+		{
+			char	   *enc;
+			enc = pg_any_to_server(pstr,
+								   plan->len,
+								   entry->encoding);
+			values[i++] = CStringGetTextDatum(enc);
+
+			if (enc != pstr)
+				pfree(enc);
+		} else {
+			/* failed to get the string of the plan */
+			nulls[i++] = true;
+		}
+	} else {
+		/* no plan available or plan_len not greater than 0 */
+		nulls[i++] = true;
+	}
+	return i;
+}
+
+
+
 /*
  * Estimate shared memory space needed.
  */
@@ -1800,9 +2250,9 @@ entry_dealloc(void)
 		pgss->cur_median_usage = entries[i / 2]->counters.usage;
 	/* Record the mean query length */
 	if (nvalidtexts > 0)
-		pgss->mean_query_len = tottextlen / nvalidtexts;
+		pgss->mean_text_len = tottextlen / nvalidtexts;
 	else
-		pgss->mean_query_len = ASSUMED_LENGTH_INIT;
+		pgss->mean_text_len = ASSUMED_LENGTH_INIT;
 
 	/* Now zap an appropriate fraction of lowest-usage entries */
 	nvictims = Max(10, i * USAGE_DEALLOC_PERCENT / 100);
@@ -2030,8 +2480,8 @@ need_gc_qtexts(void)
 		SpinLockRelease(&s->mutex);
 	}
 
-	/* Don't proceed if file does not exceed 512 bytes per possible entry */
-	if (extent < 512 * pgss_max)
+	/* Don't proceed if file does not exceed 2048 bytes per possible entry */
+	if (extent < 2048 * pgss_max)
 		return false;
 
 	/*
@@ -2041,7 +2491,7 @@ need_gc_qtexts(void)
 	 * query length in order to prevent garbage collection from thrashing
 	 * uselessly.
 	 */
-	if (extent < pgss->mean_query_len * pgss_max * 2)
+	if (extent < pgss->mean_text_len * pgss_max * 2)
 		return false;
 
 	return true;
@@ -2121,11 +2571,29 @@ gc_qtexts(void)
 									  qbuffer,
 									  qbuffer_size);
 
-		if (qry == NULL)
+		int			good_plan_len = entry->good_plan.len;
+		char	   *good_plan = qtext_fetch(entry->good_plan.offset,
+									   good_plan_len,
+									   qbuffer,
+									   qbuffer_size);
+
+		int			bad_plan_len = entry->bad_plan.len;
+		char	   *bad_plan = qtext_fetch(entry->bad_plan.offset,
+									   bad_plan_len,
+									   qbuffer,
+									   qbuffer_size);
+		/* an "empty" string */
+		char empty_str[1] = "\0";
+
+		if (qry == NULL || good_plan == NULL || bad_plan == NULL)
 		{
-			/* Trouble ... drop the text */
+			/* Trouble occured; Drop the query and the two plan texts */
 			entry->query_offset = 0;
 			entry->query_len = -1;
+			entry->good_plan.offset = 0;
+			entry->good_plan.len = -1;
+			entry->bad_plan.offset = 0;
+			entry->bad_plan.len = -1;
 			/* entry will not be counted in mean query length computation */
 			continue;
 		}
@@ -2142,6 +2610,43 @@ gc_qtexts(void)
 
 		entry->query_offset = extent;
 		extent += query_len + 1;
+
+		if(good_plan_len <= 0)
+		{
+			good_plan = empty_str;
+			good_plan_len = 0;
+		}
+		/* Save the good plan */
+		if (fwrite(good_plan, 1, good_plan_len + 1, qfile) != good_plan_len + 1)
+		{
+			ereport(LOG,
+					(errcode_for_file_access(),
+				  errmsg("could not write best plan pg_stat_statement file \"%s\": %m",
+						 PGSS_TEXT_FILE)));
+			hash_seq_term(&hash_seq);
+			goto gc_fail;
+		}
+		entry->good_plan.offset = extent;
+		extent += good_plan_len + 1;
+
+		if(bad_plan_len <= 0)
+		{
+			bad_plan = empty_str;
+			bad_plan_len = 0;
+		}
+		/* Save the bad plan */
+		if (fwrite(bad_plan, 1, bad_plan_len + 1, qfile) != bad_plan_len + 1)
+		{
+			ereport(LOG,
+					(errcode_for_file_access(),
+				  errmsg("could not write worst plan pg_stat_statement file \"%s\": %m",
+						 PGSS_TEXT_FILE)));
+			hash_seq_term(&hash_seq);
+			goto gc_fail;
+		}
+		entry->bad_plan.offset = extent;
+		extent += bad_plan_len + 1;
+
 		nentries++;
 	}
 
@@ -2176,9 +2681,9 @@ gc_qtexts(void)
 	 * won't still think we have a problem.
 	 */
 	if (nentries > 0)
-		pgss->mean_query_len = extent / nentries;
+		pgss->mean_text_len = extent / nentries;
 	else
-		pgss->mean_query_len = ASSUMED_LENGTH_INIT;
+		pgss->mean_text_len = ASSUMED_LENGTH_INIT;
 
 	free(qbuffer);
 
@@ -2228,7 +2733,7 @@ gc_fail:
 	pgss->extent = 0;
 
 	/* Reset mean_query_len to match the new state */
-	pgss->mean_query_len = ASSUMED_LENGTH_INIT;
+	pgss->mean_text_len = ASSUMED_LENGTH_INIT;
 
 	/*
 	 * Bump the GC count even though we failed.
diff --git a/contrib/pg_stat_statements/pg_stat_statements.control b/contrib/pg_stat_statements/pg_stat_statements.control
index 24038f5..193fcdf 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.control
+++ b/contrib/pg_stat_statements/pg_stat_statements.control
@@ -1,5 +1,5 @@
 # pg_stat_statements extension
 comment = 'track execution statistics of all SQL statements executed'
-default_version = '1.4'
+default_version = '1.5'
 module_pathname = '$libdir/pg_stat_statements'
 relocatable = true
diff --git a/doc/src/sgml/pgstatstatements.sgml b/doc/src/sgml/pgstatstatements.sgml
index d4f09fc..6207045 100644
--- a/doc/src/sgml/pgstatstatements.sgml
+++ b/doc/src/sgml/pgstatstatements.sgml
@@ -82,6 +82,20 @@
       <entry>Text of a representative statement</entry>
      </row>
 
+      <row>
+      <entry><structfield>good_plan</structfield></entry>
+      <entry><type>text</type></entry>
+      <entry></entry>
+      <entry>The execution plan of the statement with the shortest execution time, in milliseconds</entry>
+     </row>
+
+      <row>
+      <entry><structfield>bad_plan</structfield></entry>
+      <entry><type>text</type></entry>
+      <entry></entry>
+      <entry>The execution plan of the statement with the longest execution time, in milliseconds</entry>
+     </row>
+
      <row>
       <entry><structfield>calls</structfield></entry>
       <entry><type>bigint</type></entry>
@@ -124,6 +138,34 @@
       <entry>Population standard deviation of time spent in the statement, in milliseconds</entry>
      </row>
 
+      <row>
+      <entry><structfield>good_plan_time</structfield></entry>
+      <entry><type>double precision</type></entry>
+      <entry></entry>
+      <entry>The execution time belonging to the good_plan, in milliseconds</entry>
+     </row>
+
+      <row>
+      <entry><structfield>good_plan_timestamp</structfield></entry>
+      <entry><type>timestamp</type></entry>
+      <entry></entry>
+      <entry>Timestamp (with timezone) belonging to the good_plan</entry>
+     </row>
+
+      <row>
+      <entry><structfield>bad_plan_time</structfield></entry>
+      <entry><type>double precision</type></entry>
+      <entry></entry>
+      <entry>The execution time belonging to the bad_plan, in milliseconds</entry>
+     </row>
+
+      <row>
+      <entry><structfield>bad_plan_timestamp</structfield></entry>
+      <entry><type>timestamp</type></entry>
+      <entry></entry>
+      <entry>Timestamp (with timezone) belonging to the bad_plan</entry>
+     </row>
+
      <row>
       <entry><structfield>rows</structfield></entry>
       <entry><type>bigint</type></entry>
@@ -339,6 +381,40 @@
 
    <varlistentry>
     <term>
+     <function>pg_stat_statements_good_plan_reset(queryid bigint) returns void</function>
+     <indexterm>
+      <primary>pg_stat_statements_good_plan_reset</primary>
+     </indexterm>
+    </term>
+
+    <listitem>
+     <para>
+      resets <parameter>good_plan, good_plan_time</parameter> and <parameter> 
+      good_plan_timestamp</parameter> of the entry matching
+      <parameter>queryid</parameter>.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry>
+    <term>
+     <function>pg_stat_statements_bad_plan_reset(queryid bigint) returns void</function>
+     <indexterm>
+      <primary>pg_stat_statements_bad_plan_reset</primary>
+     </indexterm>
+    </term>
+
+    <listitem>
+     <para>
+      resets <parameter>bad_plan, bad_plan_time</parameter> and <parameter> 
+      bad_plan_timestamp</parameter> of the entry matching
+      <parameter>queryid</parameter>.
+     </para>
+    </listitem>
+   </varlistentry>      
+
+   <varlistentry>
+    <term>
      <function>pg_stat_statements(showtext boolean) returns setof record</function>
      <indexterm>
       <primary>pg_stat_statements</primary>
@@ -444,6 +520,96 @@
      </para>
     </listitem>
    </varlistentry>
+
+   <varlistentry>
+    <term>
+     <varname>pg_stat_statements.good_plan_enable</varname> (<type>boolean</type>)
+    </term>
+
+    <listitem>
+     <para>
+      <varname>pg_stat_statements.good_plan_enable</varname> specifies if good plans will be saved.
+      The default value is <literal>on</>.
+      This parameter can only be set in the <filename>postgresql.conf</>
+      file or on the server command line.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  <varlistentry>
+    <term>
+     <varname>pg_stat_statements.bad_plan_enable</varname> (<type>boolean</type>)
+    </term>
+
+    <listitem>
+     <para>
+      <varname>pg_stat_statements.bad_plan_enable</varname> specifies if bad plans will be saved.
+      The default value is <literal>on</>.
+      This parameter can only be set in the <filename>postgresql.conf</>
+      file or on the server command line.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  <varlistentry>
+    <term>
+     <varname>pg_stat_statements.good_plan_delta</varname> (<type>real</type>)
+    </term>
+
+    <listitem>
+     <para>
+      If the execution time deceeds the previous good_plan_time by a factor smaller than <varname>pg_stat_statements.good_plan_delta</varname>, the current plan will be stored. 
+      The default value is <literal>0.9</>. Must be within <literal>0.0</> (never save new plans) and <literal>1.0</>.
+      This parameter can only be set in the <filename>postgresql.conf</>
+      file or on the server command line.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  <varlistentry>
+    <term>
+     <varname>pg_stat_statements.bad_plan_delta</varname> (<type>real</type>)
+    </term>
+
+    <listitem>
+     <para>
+      If the execution time exceeds the previous bad_plan_time by a factor larger than <varname>pg_stat_statements.bad_plan_delta</varname>, the current plan will be stored. 
+      The default value is <literal>1.1</>. Must be within <literal>1.01</>and <literal>10.0</>.
+      This parameter can only be set in the <filename>postgresql.conf</>
+      file or on the server command line.
+     </para>
+    </listitem>
+   </varlistentry>
+  
+  <varlistentry>
+    <term>
+     <varname>pg_stat_statements.plan_min_interval</varname> (<type>integer</type>)
+    </term>
+
+    <listitem>
+     <para>
+      <varname>pg_stat_statements.plan_min_interval</varname> is the amount of seconds that must pass before a newer plan can be stored. 
+      The default value is <literal>5</>.
+      This parameter can only be set in the <filename>postgresql.conf</>
+      file or on the server command line.
+     </para>
+    </listitem>
+   </varlistentry>
+
+  <varlistentry>
+    <term>
+     <varname>pg_stat_statements.plan_format</varname> (<type>enum</type>)
+    </term>
+
+    <listitem>
+     <para>
+      <varname>pg_stat_statements.bad_plan_delta</varname> specifies the format which is used when saving plans.
+      The default value is <literal>text</>. Can be <literal>xml</>, <literal>json</> and <literal>yaml</>.
+      This parameter can only be set in the <filename>postgresql.conf</>
+      file or on the server command line.
+     </para>
+    </listitem>
+   </varlistentry>
   </variablelist>
 
   <para>
#2Simon Riggs
simon@2ndquadrant.com
In reply to: Julian Markwort (#1)
Re: [FEATURE PATCH] pg_stat_statements with plans

On 25 January 2017 at 17:34, Julian Markwort
<julian.markwort@uni-muenster.de> wrote:

Analogous to this, a bad_plan is saved, when the time has been exceeded by a
factor greater than 1.1 .

...and the plan differs?

Probably best to use some stat math to calculate deviation, rather than fixed %.

Sounds good.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3David Steele
david@pgmasters.net
In reply to: Julian Markwort (#1)
Re: [FEATURE PATCH] pg_stat_statements with plans

Hi Julian,

On 1/25/17 12:34 PM, Julian Markwort wrote:

TL:DR;
We extended the functionality of pg_stat_statements so it can track
worst and best case execution plans.

pg_stat_statements is an important tool and perhaps one of the most used
core extensions. Any improvements would be greatly welcome by the admin
community, I'm sure.

However, this is a rather large change which could be destabilizing to
the many users of this extension. Even though the patch was posted well
in advance of the last CF it has received little discussion so is
essentially new.

I recommend moving this patch to the 2017-07 CF.

--
-David
david@pgmasters.net

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4Peter Eisentraut
peter.eisentraut@2ndquadrant.com
In reply to: Simon Riggs (#2)
Re: [FEATURE PATCH] pg_stat_statements with plans

On 1/25/17 12:43, Simon Riggs wrote:

On 25 January 2017 at 17:34, Julian Markwort
<julian.markwort@uni-muenster.de> wrote:

Analogous to this, a bad_plan is saved, when the time has been exceeded by a
factor greater than 1.1 .

...and the plan differs?

Probably best to use some stat math to calculate deviation, rather than fixed %.

Yeah, it seems to me too that this needs a bit more deeper analysis. I
don't see offhand why a 10% deviation in execution time would be a
reasonable threshold for "good" or "bad". A deviation approach like you
allude to would be better.

The other problem is that this measures execution time, which can vary
for reasons other than plan. I would have expected that the cost
numbers are tracked somehow.

There is also the issue of generic vs specific plans, which this
approach might be papering over.

Needs more thought.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#5Julian Markwort
julian.markwort@uni-muenster.de
In reply to: David Steele (#3)
Re: [FEATURE PATCH] pg_stat_statements with plans

Any improvements would be greatly welcome by the admin
community, I'm sure.

That's good to hear - on the other hand, I really appreciate the opinion
of admins on this idea!

However, this is a rather large change which could be destabilizing to
the many users of this extension.

I'm fully aware of that, which is why there are already several switches
in place so you can keep using the existing functionality without
compromises or added complexity.
At the same time, I'm always open to suggestions regarding the reduction
of complexity and probably more importantly the reduction of disk IO.

I recommend moving this patch to the 2017-07 CF.

Since I do not have very much time for this at the moment I'll be
looking forward to discussions on this patch in the next commitfest!

kind regards
Julian

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6David Steele
david@pgmasters.net
In reply to: Julian Markwort (#5)
Re: [FEATURE PATCH] pg_stat_statements with plans

Hi Julian,

On 3/4/17 8:41 AM, Julian Markwort wrote:

Any improvements would be greatly welcome by the admin
community, I'm sure.

That's good to hear - on the other hand, I really appreciate the opinion
of admins on this idea!

However, this is a rather large change which could be destabilizing to
the many users of this extension.

I'm fully aware of that, which is why there are already several switches
in place so you can keep using the existing functionality without
compromises or added complexity.
At the same time, I'm always open to suggestions regarding the reduction
of complexity and probably more importantly the reduction of disk IO.

I recommend moving this patch to the 2017-07 CF.

Since I do not have very much time for this at the moment I'll be
looking forward to discussions on this patch in the next commitfest!

Since some concerns were raised about the implementation, I have instead
marked this "Returned with Feedback". I encourage you to continue
developing the patch with the community and resubmit into the
appropriate CF when it is ready.

--
-David
david@pgmasters.net

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#7Julian Markwort
julian.markwort@uni-muenster.de
In reply to: Peter Eisentraut (#4)
Re: [FEATURE PATCH] pg_stat_statements with plans

Alright, for the next version of this patch I'll look into standard
deviation (an implementation of Welfords' algorithm already exists in
pg_stat_statements).

On 3/4/17 14:18, Peter Eisentraut wrote:

The other problem is that this measures execution time, which can vary
for reasons other than plan. I would have expected that the cost
numbers are tracked somehow.

I've already thought of tracking specific parts of the explanation, like
the cost numbers, instead of the whole string, I'll think of something,
but if anybody has any bright ideas in the meantime, I'd gladly listen
to them.

There is also the issue of generic vs specific plans, which this
approach might be papering over.

Would you be so kind and elaborate a little bit on this? I'm not sure if
I understand this correctly. This patch only tracks specific plans, yes.
The inital idea was that there might be some edge-cases that are not
apparent when looking at generalized plans or queries.

kind regards
Julian

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#8Julian Markwort
julian.markwort@uni-muenster.de
In reply to: Julian Markwort (#7)
2 attachment(s)
Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

Hello hackers,

I'd like to follow up to my previous proposition of tracking (some) best
and worst plans for different queries in the pg_stat_statements extension.

Based on the comments and suggestions made towards my last endeavour,
I've taken the path of computing the interquartile distance (by means of
an adapted z-test, under the assumption of normal distribution, based on
the mean_time and stddev_time already used by the extension).

A bad plan is recorded, if there is no previously recorded plan, or if
the current execution time is greater than the maximum of the previously
recorded plan's time and the query's mean+1.5*interquartile_distance.
A good plan is recorded on a similar condition; The execution time needs
to be shorter than the minimum of the previously recorded good plan's
time and the query's mean-1.5*interquartile_distance.

The boundaries are chosen to resemble the boundaries for whiskers in
boxplots.
Using these boundaries, plans will be updated very seldomly, as long as
they are more or less normally distributed.
Changes in the plans (for example the use of indices) used for each kind
of query will most likely result in execution times exceeding these
boundaries, so such changes are (very probably) recorded.

The ideal solution would be to compare the current plan with the last
plan and only update when there is a difference between them, however I
think this is unreasonably complex and a rather expensive task to
compute on the completion of every query.

The intent of this patch is to provide a quick insight into the plans
currently used by the database for the execution of certain queries. The
tracked plans only represent instances of queries with very good or very
poor performance.

I've (re)submitted this patch for the next commitfest as well.

Kind regards
Julian

Show quoted text

On 03/04/2017 02:56 PM, Julian Markwort wrote:

Alright, for the next version of this patch I'll look into standard
deviation (an implementation of Welfords' algorithm already exists in
pg_stat_statements).

On 3/4/17 14:18, Peter Eisentraut wrote:

The other problem is that this measures execution time, which can vary
for reasons other than plan.  I would have expected that the cost
numbers are tracked somehow.

I've already thought of tracking specific parts of the explanation,
like the cost numbers, instead of the whole string, I'll think of
something, but if anybody has any bright ideas in the meantime, I'd
gladly listen to them.

There is also the issue of generic vs specific plans, which this
approach might be papering over.

Would you be so kind and elaborate a little bit on this? I'm not sure
if I understand this correctly. This patch only tracks specific plans,
yes. The inital idea was that there might be some edge-cases that are
not apparent when looking at generalized plans or queries.

kind regards
Julian

Attachments:

pgss_plans_v02.patchtext/x-patch; name=pgss_plans_v02.patchDownload
diff --git a/contrib/pg_stat_statements/Makefile b/contrib/pg_stat_statements/Makefile
index 39b368b70e..4d658d0ec7 100644
--- a/contrib/pg_stat_statements/Makefile
+++ b/contrib/pg_stat_statements/Makefile
@@ -4,7 +4,7 @@ MODULE_big = pg_stat_statements
 OBJS = pg_stat_statements.o $(WIN32RES)
 
 EXTENSION = pg_stat_statements
-DATA = pg_stat_statements--1.4.sql pg_stat_statements--1.4--1.5.sql \
+DATA = pg_stat_statements--1.5.sql pg_stat_statements--1.4--1.5.sql \
 	pg_stat_statements--1.3--1.4.sql pg_stat_statements--1.2--1.3.sql \
 	pg_stat_statements--1.1--1.2.sql pg_stat_statements--1.0--1.1.sql \
 	pg_stat_statements--unpackaged--1.0.sql
diff --git a/contrib/pg_stat_statements/expected/pg_stat_statements.out b/contrib/pg_stat_statements/expected/pg_stat_statements.out
index 5318c3550c..3e79890d50 100644
--- a/contrib/pg_stat_statements/expected/pg_stat_statements.out
+++ b/contrib/pg_stat_statements/expected/pg_stat_statements.out
@@ -395,4 +395,40 @@ SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
  SELECT pg_stat_statements_reset()         |     1 |    1
 (8 rows)
 
+-- test to see if any plans have been recorded.
+SELECT
+  CASE WHEN good_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN bad_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN good_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END,
+  CASE WHEN good_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END
+FROM pg_stat_statements ORDER BY query COLLATE "C";
+ case | case | case | case 
+------+------+------+------
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    1 |    1 |    1 |    1
+    1 |    1 |    1 |    1
+    1 |    1 |    1 |    1
+(9 rows)
+
+-- test if there is some text in the recorded plans.
+select substr(good_plan, 0, 11), substr(bad_plan, 0, 11) from pg_stat_statements ORDER BY query COLLATE "C";
+   substr   |   substr   
+------------+------------
+            | 
+            | 
+            | 
+            | 
+            | 
+            | 
+ Query Text | Query Text
+ Query Text | Query Text
+ Query Text | Query Text
+ Query Text | Query Text
+(10 rows)
+
 DROP EXTENSION pg_stat_statements;
diff --git a/contrib/pg_stat_statements/pg_stat_statements--1.4--1.5.sql b/contrib/pg_stat_statements/pg_stat_statements--1.4--1.5.sql
index 9c76122a2b..27a26c1c26 100644
--- a/contrib/pg_stat_statements/pg_stat_statements--1.4--1.5.sql
+++ b/contrib/pg_stat_statements/pg_stat_statements--1.4--1.5.sql
@@ -3,4 +3,69 @@
 -- complain if script is sourced in psql, rather than via ALTER EXTENSION
 \echo Use "ALTER EXTENSION pg_stat_statements UPDATE TO '1.5'" to load this file. \quit
 
+-- Register new functions.
+CREATE FUNCTION pg_stat_statements_good_plan_reset(IN queryid bigint)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_statements_bad_plan_reset(IN queryid bigint)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+/* First we have to remove them from the extension */
+ALTER EXTENSION pg_stat_statements DROP VIEW pg_stat_statements;
+ALTER EXTENSION pg_stat_statements DROP FUNCTION pg_stat_statements(boolean);
+
+/* Then we can drop them */
+DROP VIEW pg_stat_statements;
+DROP FUNCTION pg_stat_statements(boolean);
+
+/* Now redefine */
+CREATE FUNCTION pg_stat_statements(IN showtext boolean,
+    OUT userid oid,
+    OUT dbid oid,
+    OUT queryid bigint,
+    OUT query text,
+    OUT good_plan text,
+    OUT bad_plan text,
+    OUT calls int8,
+    OUT total_time float8,
+    OUT min_time float8,
+    OUT max_time float8,
+    OUT mean_time float8,
+    OUT stddev_time float8,
+    OUT good_plan_time float8,
+    OUT good_plan_timestamp timestamp,
+    OUT bad_plan_time float8,
+    OUT bad_plan_timestamp timestamp,
+    OUT rows int8,
+    OUT shared_blks_hit int8,
+    OUT shared_blks_read int8,
+    OUT shared_blks_dirtied int8,
+    OUT shared_blks_written int8,
+    OUT local_blks_hit int8,
+    OUT local_blks_read int8,
+    OUT local_blks_dirtied int8,
+    OUT local_blks_written int8,
+    OUT temp_blks_read int8,
+    OUT temp_blks_written int8,
+    OUT blk_read_time float8,
+    OUT blk_write_time float8
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stat_statements_1_5'
+LANGUAGE C STRICT VOLATILE PARALLEL SAFE;
+
+CREATE VIEW pg_stat_statements AS
+  SELECT * FROM pg_stat_statements(true);
+
+GRANT SELECT ON pg_stat_statements TO PUBLIC;
+
+-- Don't want this to be available to non-superusers.
+REVOKE ALL ON FUNCTION pg_stat_statements_reset() FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_stat_statements_good_plan_reset FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_stat_statements_bad_plan_reset FROM PUBLIC;
+
 GRANT EXECUTE ON FUNCTION pg_stat_statements_reset() TO pg_read_all_stats;
diff --git a/contrib/pg_stat_statements/pg_stat_statements--1.5.sql b/contrib/pg_stat_statements/pg_stat_statements--1.5.sql
new file mode 100644
index 0000000000..49aa65e6e3
--- /dev/null
+++ b/contrib/pg_stat_statements/pg_stat_statements--1.5.sql
@@ -0,0 +1,68 @@
+/* contrib/pg_stat_statements/pg_stat_statements--1.5.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_stat_statements" to load this file. \quit
+
+-- Register functions.
+CREATE FUNCTION pg_stat_statements_reset()
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_statements_good_plan_reset(IN queryid bigint)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_statements_bad_plan_reset(IN queryid bigint)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_statements(IN showtext boolean,
+    OUT userid oid,
+    OUT dbid oid,
+    OUT queryid bigint,
+    OUT query text,
+    OUT good_plan text,
+    OUT bad_plan text,
+    OUT calls int8,
+    OUT total_time float8,
+    OUT min_time float8,
+    OUT max_time float8,
+    OUT mean_time float8,
+    OUT stddev_time float8,
+    OUT good_plan_time float8,
+    OUT good_plan_timestamp timestamp,
+    OUT bad_plan_time float8,
+    OUT bad_plan_timestamp timestamp,
+    OUT rows int8,
+    OUT shared_blks_hit int8,
+    OUT shared_blks_read int8,
+    OUT shared_blks_dirtied int8,
+    OUT shared_blks_written int8,
+    OUT local_blks_hit int8,
+    OUT local_blks_read int8,
+    OUT local_blks_dirtied int8,
+    OUT local_blks_written int8,
+    OUT temp_blks_read int8,
+    OUT temp_blks_written int8,
+    OUT blk_read_time float8,
+    OUT blk_write_time float8
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stat_statements_1_5'
+LANGUAGE C STRICT VOLATILE PARALLEL SAFE;
+
+-- Register a view on the function for ease of use.
+CREATE VIEW pg_stat_statements AS
+  SELECT * FROM pg_stat_statements(true);
+
+GRANT SELECT ON pg_stat_statements TO PUBLIC;
+
+-- Don't want this to be available to non-superusers.
+REVOKE ALL ON FUNCTION pg_stat_statements_reset() FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_stat_statements_good_plan_reset(bigint) FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_stat_statements_bad_plan_reset(bigint) FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_stat_statements_reset() TO pg_read_all_stats;
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 928673498a..be68d68329 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -63,6 +63,7 @@
 
 #include "access/hash.h"
 #include "catalog/pg_authid.h"
+#include "commands/explain.h"
 #include "executor/instrument.h"
 #include "funcapi.h"
 #include "mb/pg_wchar.h"
@@ -78,6 +79,7 @@
 #include "tcop/utility.h"
 #include "utils/builtins.h"
 #include "utils/memutils.h"
+#include "utils/timestamp.h"
 
 PG_MODULE_MAGIC;
 
@@ -119,7 +121,8 @@ typedef enum pgssVersion
 	PGSS_V1_0 = 0,
 	PGSS_V1_1,
 	PGSS_V1_2,
-	PGSS_V1_3
+	PGSS_V1_3,
+	PGSS_V1_5
 } pgssVersion;
 
 /*
@@ -165,6 +168,14 @@ typedef struct Counters
 	double		usage;			/* usage factor */
 } Counters;
 
+typedef struct pgssPlan
+{
+	Size offset;
+	int len;
+	double		time;	/* execution time in msec when the latest plan was updated */
+	TimestampTz timestamp;
+} pgssPlan;
+
 /*
  * Statistics per statement
  *
@@ -178,6 +189,8 @@ typedef struct pgssEntry
 	Counters	counters;		/* the statistics for this query */
 	Size		query_offset;	/* query text offset in external file */
 	int			query_len;		/* # of valid bytes in query string, or -1 */
+	pgssPlan 	good_plan;
+	pgssPlan 	bad_plan;
 	int			encoding;		/* query text encoding */
 	slock_t		mutex;			/* protects the counters only */
 } pgssEntry;
@@ -265,11 +278,22 @@ static const struct config_enum_entry track_options[] =
 	{NULL, 0, false}
 };
 
+static const struct config_enum_entry format_options[] =
+{
+	{"text", EXPLAIN_FORMAT_TEXT, false},
+	{"xml", EXPLAIN_FORMAT_XML, false},
+	{"json", EXPLAIN_FORMAT_JSON, false},
+	{"yaml", EXPLAIN_FORMAT_YAML, false},
+	{NULL, 0, false}
+};
+
 static int	pgss_max;			/* max # statements to track */
 static int	pgss_track;			/* tracking level */
 static bool pgss_track_utility; /* whether to track utility commands */
 static bool pgss_save;			/* whether to save stats across shutdown */
-
+static bool pgss_good_plan_enable; 	/* whether to save good_plans */
+static bool pgss_bad_plan_enable; 	/* whether to save good_plans */
+static int  pgss_plan_format;   	/* id which sets the output format */
 
 #define pgss_enabled() \
 	(pgss_track == PGSS_TRACK_ALL || \
@@ -291,7 +315,11 @@ void		_PG_fini(void);
 PG_FUNCTION_INFO_V1(pg_stat_statements_reset);
 PG_FUNCTION_INFO_V1(pg_stat_statements_1_2);
 PG_FUNCTION_INFO_V1(pg_stat_statements_1_3);
+PG_FUNCTION_INFO_V1(pg_stat_statements_1_5);
 PG_FUNCTION_INFO_V1(pg_stat_statements);
+PG_FUNCTION_INFO_V1(pg_stat_statements_good_plan_reset);
+PG_FUNCTION_INFO_V1(pg_stat_statements_bad_plan_reset);
+
 
 static void pgss_shmem_startup(void);
 static void pgss_shmem_shutdown(int code, Datum arg);
@@ -309,12 +337,17 @@ static void pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
 static uint64 pgss_hash_string(const char *str, int len);
 static void pgss_store(const char *query, uint64 queryId,
 		   int query_location, int query_len,
+		   const char *plan,
 		   double total_time, uint64 rows,
 		   const BufferUsage *bufusage,
 		   pgssJumbleState *jstate);
+Datum pg_stat_statements_plan_reset(uint64 query_id, uint8 plan_type);
 static void pg_stat_statements_internal(FunctionCallInfo fcinfo,
 							pgssVersion api_version,
 							bool showtext);
+static int fill_plan_times(Datum values[], bool nulls[], int i, pgssPlan *plan);
+static int fill_plan_str(Datum values[], bool nulls[], int i, pgssPlan *plan,
+							pgssEntry *entry, char *qbuffer, Size qbuffer_size);
 static Size pgss_memsize(void);
 static pgssEntry *entry_alloc(pgssHashKey *key, Size query_offset, int query_len,
 			int encoding, bool sticky);
@@ -407,6 +440,40 @@ _PG_init(void)
 							 NULL,
 							 NULL);
 
+	DefineCustomBoolVariable("pg_stat_statements.good_plan_enable",
+							 "Enable bad plan detection",
+							 NULL,
+							 &pgss_good_plan_enable,
+							 true,
+							 PGC_SUSET,
+							 0,
+							 NULL,
+							 NULL,
+							 NULL);
+
+	DefineCustomBoolVariable("pg_stat_statements.bad_plan_enable",
+							 "Enable bad plan detection",
+							 NULL,
+							 &pgss_bad_plan_enable,
+							 true,
+							 PGC_SUSET,
+							 0,
+							 NULL,
+							 NULL,
+							 NULL);
+
+	DefineCustomEnumVariable("pg_stat_statements.plan_format",
+							"Sets the output format for the plans.",
+							"Notice that the plan format can not be changed after a plan is saved. Valid values are text, json, xml and yaml.",
+							&pgss_plan_format,
+							EXPLAIN_FORMAT_TEXT,
+							format_options,
+							PGC_SUSET,
+							0,
+							NULL,
+							NULL,
+							NULL);
+
 	EmitWarningsOnPlaceholders("pg_stat_statements");
 
 	/*
@@ -470,7 +537,11 @@ pgss_shmem_startup(void)
 	int32		pgver;
 	int32		i;
 	int			buffer_size;
+	int			good_plan_buffer_size;
+	int			bad_plan_buffer_size;
 	char	   *buffer = NULL;
+	char	   *good_plan_buffer = NULL;
+	char	   *bad_plan_buffer = NULL;
 
 	if (prev_shmem_startup_hook)
 		prev_shmem_startup_hook();
@@ -561,7 +632,11 @@ pgss_shmem_startup(void)
 	}
 
 	buffer_size = 2048;
+	good_plan_buffer_size = 2048;
+	bad_plan_buffer_size = 2048;
 	buffer = (char *) palloc(buffer_size);
+	good_plan_buffer  = (char *) palloc(good_plan_buffer_size);
+	bad_plan_buffer = (char *) palloc(bad_plan_buffer_size);
 
 	if (fread(&header, sizeof(uint32), 1, file) != 1 ||
 		fread(&pgver, sizeof(uint32), 1, file) != 1 ||
@@ -595,8 +670,34 @@ pgss_shmem_startup(void)
 		if (fread(buffer, 1, temp.query_len + 1, file) != temp.query_len + 1)
 			goto read_error;
 
+		/* Resize good plan_buffer and read into it */
+		if (temp.good_plan.len >= good_plan_buffer_size)
+		{
+			good_plan_buffer_size =
+				Max(good_plan_buffer_size * 2, temp.good_plan.len + 1);
+			good_plan_buffer= repalloc(good_plan_buffer, good_plan_buffer_size);
+		}
+
+		if (fread(good_plan_buffer, 1, temp.good_plan.len + 1, file)
+				!= temp.good_plan.len + 1)
+			goto read_error;
+
+		/* Resize bad plan_buffer and read into it */
+		if (temp.bad_plan.len >= bad_plan_buffer_size)
+		{
+			bad_plan_buffer_size =
+				Max(bad_plan_buffer_size * 2, temp.bad_plan.len + 1);
+			bad_plan_buffer= repalloc(bad_plan_buffer, bad_plan_buffer_size);
+		}
+
+		if (fread(bad_plan_buffer, 1, temp.bad_plan.len + 1, file)
+				!= temp.bad_plan.len + 1)
+			goto read_error;
+
 		/* Should have a trailing null, but let's make sure */
 		buffer[temp.query_len] = '\0';
+		good_plan_buffer[temp.good_plan.len] = '\0';
+		bad_plan_buffer[temp.bad_plan.len] = '\0';
 
 		/* Skip loading "sticky" entries */
 		if (temp.counters.calls == 0)
@@ -608,6 +709,20 @@ pgss_shmem_startup(void)
 			goto write_error;
 		pgss->extent += temp.query_len + 1;
 
+		/* Store the good plan text*/
+		temp.good_plan.offset = pgss->extent;
+		if (fwrite(good_plan_buffer, 1, temp.good_plan.len + 1, qfile)
+				!= temp.good_plan.len + 1)
+			goto write_error;
+		pgss->extent += temp.good_plan.len + 1;
+
+		/* Store the bad plan text*/
+		temp.bad_plan.offset = pgss->extent;
+		if (fwrite(bad_plan_buffer, 1, temp.bad_plan.len + 1, qfile)
+				!= temp.bad_plan.len + 1)
+			goto write_error;
+		pgss->extent += temp.bad_plan.len + 1;
+
 		/* make the hashtable entry (discards old entries if too many) */
 		entry = entry_alloc(&temp.key, query_offset, temp.query_len,
 							temp.encoding,
@@ -615,9 +730,14 @@ pgss_shmem_startup(void)
 
 		/* copy in the actual stats */
 		entry->counters = temp.counters;
+				/* copy in the plans */
+		entry->good_plan = temp.good_plan;
+		entry->bad_plan = temp.bad_plan;
 	}
 
 	pfree(buffer);
+	pfree(good_plan_buffer);
+	pfree(bad_plan_buffer);
 	FreeFile(file);
 	FreeFile(qfile);
 
@@ -658,6 +778,10 @@ write_error:
 fail:
 	if (buffer)
 		pfree(buffer);
+	if (good_plan_buffer)
+		pfree(good_plan_buffer);
+	if (bad_plan_buffer)
+		pfree(bad_plan_buffer);
 	if (file)
 		FreeFile(file);
 	if (qfile)
@@ -680,6 +804,11 @@ fail:
 static void
 pgss_shmem_shutdown(int code, Datum arg)
 {
+
+	//DEBUG
+//	FILE *debugfile;
+
+
 	FILE	   *file;
 	char	   *qbuffer = NULL;
 	Size		qbuffer_size = 0;
@@ -687,6 +816,11 @@ pgss_shmem_shutdown(int code, Datum arg)
 	int32		num_entries;
 	pgssEntry  *entry;
 
+	//DEBUG
+//	debugfile = fopen("/tmp/shutdown.log", "w");
+//	fprintf(debugfile, "entered pgss_shmem_shutdown\n");
+//	fflush(debugfile);
+
 	/* Don't try to dump during a crash. */
 	if (code)
 		return;
@@ -715,6 +849,10 @@ pgss_shmem_shutdown(int code, Datum arg)
 	if (qbuffer == NULL)
 		goto error;
 
+	//DEBUG
+//	fprintf(debugfile, "loaded qtexts\n");
+//	fflush(debugfile);
+
 	/*
 	 * When serializing to disk, we store query texts immediately after their
 	 * entry data.  Any orphaned query texts are thereby excluded.
@@ -723,21 +861,78 @@ pgss_shmem_shutdown(int code, Datum arg)
 	while ((entry = hash_seq_search(&hash_seq)) != NULL)
 	{
 		int			len = entry->query_len;
+		int			good_plan_len = entry->good_plan.len;
+		int			bad_plan_len = entry->bad_plan.len;
 		char	   *qstr = qtext_fetch(entry->query_offset, len,
 									   qbuffer, qbuffer_size);
+		char       *good_plan_str;
+		char       *bad_plan_str;
+		if (good_plan_len > 0)
+		{
+			/* A good plan is available */
+			good_plan_str = qtext_fetch(entry->good_plan.offset, good_plan_len,
+												qbuffer, qbuffer_size);
+		} else {
+			/*
+			 * There is no good plan available. This can be caused by a query
+			 * which has no plan (for example insert or delete queries) or
+			 * because good plans are disabled. In this case we have to store
+			 * an empty string instead of null.
+			 */
+			good_plan_str = palloc(1 * sizeof(char));
+			*good_plan_str = '\0';
+			//good_plan_str="\0";
+		}
+		if (bad_plan_len > 0)
+		{
+			/* A bad plan is available */
+			bad_plan_str = qtext_fetch(entry->bad_plan.offset, bad_plan_len,
+											   qbuffer, qbuffer_size);
+		} else {
+			/*
+			 * There is no bad plan available. This can be caused by a query
+			 * which has no plan (for example insert or delete queries) or
+			 * because bad plans are disabled. In this case we have to store
+			 * an empty string instead of null.
+			 */
+			bad_plan_str = palloc(1 * sizeof(char));
+			*bad_plan_str = '\0';
+			//bad_plan_str="\0";
+		}
 
 		if (qstr == NULL)
 			continue;			/* Ignore any entries with bogus texts */
 
+		//DEBUG
+//		fprintf(debugfile, "trying to write entry, query, plans...\n");
+//		fprintf(debugfile, "query_len: %d \t good_plan_len: %d \t bad_plan_len: %d \n", len, good_plan_len, bad_plan_len);
+//		fflush(debugfile);
+//		fprintf(debugfile, "query: %s \t good_plan: %s \t bad_plan %s \n", qstr, good_plan_str, bad_plan_str);
+//		fflush(debugfile);
+
+		/* Write entries, queries and plans serialized to the dump file */
 		if (fwrite(entry, sizeof(pgssEntry), 1, file) != 1 ||
-			fwrite(qstr, 1, len + 1, file) != len + 1)
+			fwrite(qstr, 1, len + 1, file) != len + 1 ||
+			fwrite(good_plan_str, 1, good_plan_len + 1, file) != good_plan_len + 1 ||
+			fwrite(bad_plan_str, 1, bad_plan_len + 1, file) != bad_plan_len + 1)
 		{
 			/* note: we assume hash_seq_term won't change errno */
 			hash_seq_term(&hash_seq);
 			goto error;
 		}
+
+		//DEBUG
+//		fprintf(debugfile, "wrote entry, qstr, plans to dump file.\n");
+//		fflush(debugfile);
+
 	}
 
+	//DEBUG
+//	fprintf(debugfile, ">>>>>wrote all the things to dump file.\n");
+//	fflush(debugfile);
+//	if(debugfile)
+//		fclose(debugfile);
+
 	free(qbuffer);
 	qbuffer = NULL;
 
@@ -835,6 +1030,7 @@ pgss_post_parse_analyze(ParseState *pstate, Query *query)
 				   query->queryId,
 				   query->stmt_location,
 				   query->stmt_len,
+				   NULL,
 				   0,
 				   0,
 				   NULL,
@@ -928,6 +1124,8 @@ pgss_ExecutorFinish(QueryDesc *queryDesc)
 static void
 pgss_ExecutorEnd(QueryDesc *queryDesc)
 {
+	/* *es is used to retrieve the plan associated with the statement*/
+	ExplainState *es = NewExplainState();
 	uint64		queryId = queryDesc->plannedstmt->queryId;
 
 	if (queryId != UINT64CONST(0) && queryDesc->totaltime && pgss_enabled())
@@ -938,15 +1136,47 @@ pgss_ExecutorEnd(QueryDesc *queryDesc)
 		 */
 		InstrEndLoop(queryDesc->totaltime);
 
+		/* The plan is retrieved whenever any plan tracking is enabled,
+		 * as we do not know at this point, wether the plan was better
+		 * or worse than before.
+		 */
+		if(pgss_good_plan_enable || pgss_bad_plan_enable){
+			es->analyze = (queryDesc->instrument_options && false);
+			es->verbose = false;
+			es->buffers = (es->analyze && false);
+			es->timing = (es->analyze && true);
+			es->format = pgss_plan_format;
+
+			ExplainBeginOutput(es);
+			ExplainQueryText(es, queryDesc);
+			ExplainPrintPlan(es, queryDesc);
+			ExplainEndOutput(es);
+
+			/* Remove last line break */
+			if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+				es->str->data[--es->str->len] = '\0';
+
+			//TODO:wieso kann hier data[0] gleich '{' gesetzt werden? Was stand da vorher?
+			/* Fix JSON to output an object */
+			if (pgss_plan_format == EXPLAIN_FORMAT_JSON)
+			{
+				es->str->data[0] = '{';
+				es->str->data[es->str->len - 1] = '}';
+			}
+		}
+
 		pgss_store(queryDesc->sourceText,
 				   queryId,
 				   queryDesc->plannedstmt->stmt_location,
 				   queryDesc->plannedstmt->stmt_len,
+				   es->str->data,							/* plan */
 				   queryDesc->totaltime->total * 1000.0,	/* convert to msec */
 				   queryDesc->estate->es_processed,
 				   &queryDesc->totaltime->bufusage,
 				   NULL);
 	}
+	pfree(es->str->data);
+	pfree(es);
 
 	if (prev_ExecutorEnd)
 		prev_ExecutorEnd(queryDesc);
@@ -1053,6 +1283,7 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
 				   0,			/* signal that it's a utility stmt */
 				   pstmt->stmt_location,
 				   pstmt->stmt_len,
+				   NULL,
 				   INSTR_TIME_GET_MILLISEC(duration),
 				   rows,
 				   &bufusage,
@@ -1083,6 +1314,26 @@ pgss_hash_string(const char *str, int len)
 											len, 0));
 }
 
+/*
+ * Update a plan in the text file.
+ * We can re-use qtext_store to save the plan.
+ */
+static void
+update_plan(volatile pgssPlan* pgssPlan_ptr, const char* plan_str, int plan_len,
+			double total_time)
+{
+	if (plan_len > 0)
+	{
+		Size plan_offset;
+		qtext_store(plan_str, plan_len, &plan_offset, NULL);
+		pgssPlan_ptr->offset 	= plan_offset;
+		pgssPlan_ptr->len 		= plan_len;
+		pgssPlan_ptr->time 		= total_time;
+		pgssPlan_ptr->timestamp = GetCurrentTimestamp();
+	}
+	return;
+}
+
 /*
  * Store some statistics for a statement.
  *
@@ -1096,6 +1347,7 @@ pgss_hash_string(const char *str, int len)
 static void
 pgss_store(const char *query, uint64 queryId,
 		   int query_location, int query_len,
+		   const char *plan,
 		   double total_time, uint64 rows,
 		   const BufferUsage *bufusage,
 		   pgssJumbleState *jstate)
@@ -1104,6 +1356,7 @@ pgss_store(const char *query, uint64 queryId,
 	pgssEntry  *entry;
 	char	   *norm_query = NULL;
 	int			encoding = GetDatabaseEncoding();
+	int 		plan_len = 0;
 
 	Assert(query != NULL);
 
@@ -1134,6 +1387,9 @@ pgss_store(const char *query, uint64 queryId,
 		query_len = strlen(query);
 	}
 
+	if(plan)
+		plan_len = strlen(plan);
+
 	/*
 	 * Discard leading and trailing whitespace, too.  Use scanner_isspace()
 	 * not libc's isspace(), because we want to match the lexer's behavior.
@@ -1214,7 +1470,11 @@ pgss_store(const char *query, uint64 queryId,
 		if (!stored)
 			goto done;
 
-		/* OK to create a new hashtable entry */
+		/* OK to create a new hashtable entry
+		 * We have to set the good- and the bad planlength to zero. Otherwise the
+		 * pgss_shmem_startup method will expect this number of chars which crashes
+		 * if there is no plan yet.
+		 */
 		entry = entry_alloc(&key, query_offset, query_len, encoding,
 							jstate != NULL);
 
@@ -1230,10 +1490,56 @@ pgss_store(const char *query, uint64 queryId,
 		 * Grab the spinlock while updating the counters (see comment about
 		 * locking rules at the head of the file)
 		 */
-		volatile pgssEntry *e = (volatile pgssEntry *) entry;
+		volatile 	pgssEntry *e = (volatile pgssEntry *) entry;
+		double 		interquartile_dist = 0.0;
 
 		SpinLockAcquire(&e->mutex);
 
+		/*
+		 * Both good and bad plans are updated, if no previous plan has beed recorded.
+		 */
+
+		if (pgss_good_plan_enable && e->good_plan.time == 0)
+		{
+			update_plan(&e->good_plan, plan, plan_len, total_time);
+		}
+
+		if (pgss_bad_plan_enable && e->bad_plan.time == 0)
+		{
+			update_plan(&e->bad_plan, plan, plan_len, total_time);
+		}
+
+		/*
+		 * The Z-scores allows us to estimate the values at any quantiles
+		 * in a list of values with (assumed) normal distribution, known mean,
+		 * and known standard deviation. The value at any quantile is equal to
+		 * z*stddev + mean. The distance to the mean is thus z*stddev,
+		 * which we'll use to calculate the interquartile distance.
+		 */
+		interquartile_dist = 2.0*(0.6745 * sqrt(e->counters.sum_var_time / e->counters.calls));
+		if(interquartile_dist > 0.0)
+		{
+			/*
+			 * If the execution time is smaller than the minimum of mean - 1.5*IQD,
+			 * and the previous good_plan's time, the good_plan is updated.
+			 */
+			if (pgss_good_plan_enable && total_time < e->good_plan.time
+				&& total_time < (e->counters.mean_time - 1.5 * interquartile_dist))
+			{
+				update_plan(&e->good_plan, plan, plan_len, total_time);
+			}
+
+			/*
+			 * If the execution time is greater than the maximum of mean + 1.5*IQD,
+			 * and the previous bad_plan's time, the bad_plan is updated.
+			 */
+			if (pgss_bad_plan_enable && total_time > e->bad_plan.time
+				&& total_time > (e->counters.mean_time + 1.5 * interquartile_dist))
+			{
+				update_plan(&e->bad_plan, plan, plan_len, total_time);
+			}
+		}
+
 		/* "Unstick" entry if it was previously sticky */
 		if (e->counters.calls == 0)
 			e->counters.usage = USAGE_INIT;
@@ -1305,12 +1611,89 @@ pg_stat_statements_reset(PG_FUNCTION_ARGS)
 	PG_RETURN_VOID();
 }
 
+Datum
+pg_stat_statements_plan_reset(uint64 query_id, uint8 plan_type)
+{
+	pgssEntry  *entry;
+
+	pgssHashKey key;
+
+	if (!pgss || !pgss_hash)
+		ereport(ERROR,
+				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+				 errmsg("pg_stat_statements must be loaded via shared_preload_libraries")));
+
+	/* Set up key for hashtable search */
+	key.userid = GetUserId();
+	key.dbid = MyDatabaseId;
+	key.queryid = query_id;
+
+	/* Lookup the hash table entry with shared lock. */
+	LWLockAcquire(pgss->lock, LW_SHARED);
+
+	entry = (pgssEntry *) hash_search(pgss_hash, &key, HASH_FIND, NULL);
+
+	if (entry)
+	{
+		if(superuser() || entry->key.userid == key.userid)
+		{
+			pgssPlan *pgssPlan_ptr;
+			switch (plan_type)
+			{
+				case 1:
+					pgssPlan_ptr = &entry->good_plan;
+					break;
+				case 2:
+					pgssPlan_ptr = &entry->bad_plan;
+					break;
+				default:
+					ereport(ERROR,
+						(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+							errmsg_internal("unrecognized plan type: %d",
+											plan_type)));
+					break;
+			}
+			if(pgssPlan_ptr)
+			{
+				pgssPlan_ptr->offset = -1;
+				pgssPlan_ptr->len = 0;
+				pgssPlan_ptr->time = 0;
+				pgssPlan_ptr->timestamp = 0;
+			}
+		}else{
+			elog(ERROR, "insufficient permission to reset plan");
+		}
+	}
+	LWLockRelease(pgss->lock);
+	PG_RETURN_VOID();
+}
+
+/*
+ * Reset the good_plan of the entry with queryid.
+ */
+Datum
+pg_stat_statements_good_plan_reset(PG_FUNCTION_ARGS)
+{
+	return pg_stat_statements_plan_reset(PG_GETARG_INT64(0), 1);
+}
+
+/*
+ * Reset the bad_plan of the entry with queryid.
+ */
+Datum
+pg_stat_statements_bad_plan_reset(PG_FUNCTION_ARGS)
+{
+	return pg_stat_statements_plan_reset(PG_GETARG_INT64(0), 2);
+}
+
+
 /* Number of output arguments (columns) for various API versions */
 #define PG_STAT_STATEMENTS_COLS_V1_0	14
 #define PG_STAT_STATEMENTS_COLS_V1_1	18
 #define PG_STAT_STATEMENTS_COLS_V1_2	19
 #define PG_STAT_STATEMENTS_COLS_V1_3	23
-#define PG_STAT_STATEMENTS_COLS			23	/* maximum of above */
+#define PG_STAT_STATEMENTS_COLS_V1_5	29
+#define PG_STAT_STATEMENTS_COLS			29	/* maximum of above */
 
 /*
  * Retrieve statement statistics.
@@ -1322,6 +1705,16 @@ pg_stat_statements_reset(PG_FUNCTION_ARGS)
  * expected API version is identified by embedding it in the C name of the
  * function.  Unfortunately we weren't bright enough to do that for 1.1.
  */
+Datum
+pg_stat_statements_1_5(PG_FUNCTION_ARGS)
+{
+ bool		showtext = PG_GETARG_BOOL(0);
+
+ pg_stat_statements_internal(fcinfo, PGSS_V1_5, showtext);
+
+ return (Datum) 0;
+}
+
 Datum
 pg_stat_statements_1_3(PG_FUNCTION_ARGS)
 {
@@ -1428,6 +1821,10 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 			if (api_version != PGSS_V1_3)
 				elog(ERROR, "incorrect number of output arguments");
 			break;
+		case PG_STAT_STATEMENTS_COLS_V1_5:
+			if (api_version != PGSS_V1_5)
+				elog(ERROR, "incorrect number of output arguments");
+			break;
 		default:
 			elog(ERROR, "incorrect number of output arguments");
 	}
@@ -1511,6 +1908,8 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 		bool		nulls[PG_STAT_STATEMENTS_COLS];
 		int			i = 0;
 		Counters	tmp;
+		pgssPlan 	tmp_good_plan;
+		pgssPlan 	tmp_bad_plan;
 		double		stddev;
 		int64		queryid = entry->key.queryid;
 
@@ -1550,6 +1949,18 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 					/* Just return a null if we fail to find the text */
 					nulls[i++] = true;
 				}
+
+				if (api_version >= PGSS_V1_5)
+				{
+					/*
+					 * This version requires to print out the good
+					 * and the bad plan if they are enabled.
+					 */
+					i = fill_plan_str(values, nulls, i, &entry->good_plan,
+										entry, qbuffer, qbuffer_size);
+					i = fill_plan_str(values, nulls, i, &entry->bad_plan,
+										entry, qbuffer, qbuffer_size);
+				}
 			}
 			else
 			{
@@ -1579,6 +1990,8 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 
 			SpinLockAcquire(&e->mutex);
 			tmp = e->counters;
+			tmp_good_plan = e->good_plan;
+			tmp_bad_plan = e->bad_plan;
 			SpinLockRelease(&e->mutex);
 		}
 
@@ -1606,6 +2019,14 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 				stddev = 0.0;
 			values[i++] = Float8GetDatumFast(stddev);
 		}
+
+		if (api_version >= PGSS_V1_5)
+		{
+			/* Set plan times and timestamps */
+			i = fill_plan_times(values, nulls, i, &tmp_good_plan);
+			i = fill_plan_times(values, nulls, i, &tmp_bad_plan);
+		}
+
 		values[i++] = Int64GetDatumFast(tmp.rows);
 		values[i++] = Int64GetDatumFast(tmp.shared_blks_hit);
 		values[i++] = Int64GetDatumFast(tmp.shared_blks_read);
@@ -1629,6 +2050,7 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 					 api_version == PGSS_V1_1 ? PG_STAT_STATEMENTS_COLS_V1_1 :
 					 api_version == PGSS_V1_2 ? PG_STAT_STATEMENTS_COLS_V1_2 :
 					 api_version == PGSS_V1_3 ? PG_STAT_STATEMENTS_COLS_V1_3 :
+					 api_version == PGSS_V1_5 ? PG_STAT_STATEMENTS_COLS_V1_5 :
 					 -1 /* fail if you forget to update this assert */ ));
 
 		tuplestore_putvalues(tupstore, tupdesc, values, nulls);
@@ -1643,6 +2065,53 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 	tuplestore_donestoring(tupstore);
 }
 
+/* Fill the plan time and timestamp into the values array. */
+static int
+fill_plan_times(Datum values[], bool nulls[], int i, pgssPlan *plan)
+{
+	values[i++] = Float8GetDatumFast(plan->time);
+
+	/* if there is noting in the timestamp field, we are not interested in it */
+	if (plan->timestamp)
+		values[i++] = TimestampTzGetDatum(plan->timestamp);
+	else
+		nulls[i++] = true;
+	return i;
+}
+
+/* Fill the plan string into the values array. */
+static int
+fill_plan_str(Datum values[], bool nulls[], int i, pgssPlan *plan,
+				pgssEntry *entry, char *qbuffer, Size qbuffer_size)
+{
+	if (plan && plan->len > 0)
+	{
+		char 	*pstr = qtext_fetch(plan->offset,
+								plan->len,
+								qbuffer,
+								qbuffer_size);
+		if (pstr)
+		{
+			char	   *enc;
+			enc = pg_any_to_server(pstr,
+								   plan->len,
+								   entry->encoding);
+			values[i++] = CStringGetTextDatum(enc);
+
+			if (enc != pstr)
+				pfree(enc);
+		} else {
+			/* failed to get the string of the plan */
+			nulls[i++] = true;
+		}
+	} else {
+		/* no plan available or plan_len not greater than 0 */
+		nulls[i++] = true;
+	}
+	return i;
+}
+
+
 /*
  * Estimate shared memory space needed.
  */
@@ -1694,6 +2163,8 @@ entry_alloc(pgssHashKey *key, Size query_offset, int query_len, int encoding,
 
 		/* reset the statistics */
 		memset(&entry->counters, 0, sizeof(Counters));
+		memset(&entry->good_plan, 0, sizeof(pgssPlan));
+		memset(&entry->bad_plan, 0, sizeof(pgssPlan));
 		/* set the appropriate initial usage count */
 		entry->counters.usage = sticky ? pgss->cur_median_usage : USAGE_INIT;
 		/* re-initialize the mutex each time ... we assume no one using it */
@@ -2104,11 +2575,27 @@ gc_qtexts(void)
 									  qbuffer,
 									  qbuffer_size);
 
-		if (qry == NULL)
+		int			good_plan_len = entry->good_plan.len;
+		char	   *good_plan = qtext_fetch(entry->good_plan.offset,
+									   good_plan_len,
+									   qbuffer,
+									   qbuffer_size);
+
+		int			bad_plan_len = entry->bad_plan.len;
+		char	   *bad_plan = qtext_fetch(entry->bad_plan.offset,
+									   bad_plan_len,
+									   qbuffer,
+									   qbuffer_size);
+
+		if (qry == NULL || good_plan == NULL || bad_plan == NULL)
 		{
-			/* Trouble ... drop the text */
+			/* Trouble ... drop the texts */
 			entry->query_offset = 0;
 			entry->query_len = -1;
+			entry->good_plan.offset = 0;
+			entry->good_plan.len = -1;
+			entry->bad_plan.offset = 0;
+			entry->bad_plan.len = -1;
 			/* entry will not be counted in mean query length computation */
 			continue;
 		}
@@ -2125,6 +2612,43 @@ gc_qtexts(void)
 
 		entry->query_offset = extent;
 		extent += query_len + 1;
+
+		if(good_plan_len <= 0)
+		{
+			good_plan = "\0";
+			good_plan_len = 0;
+		}
+		/* Save the good plan */
+		if (fwrite(good_plan, 1, good_plan_len + 1, qfile) != good_plan_len + 1)
+		{
+			ereport(LOG,
+					(errcode_for_file_access(),
+				  errmsg("could not write best plan pg_stat_statement file \"%s\": %m",
+						 PGSS_TEXT_FILE)));
+			hash_seq_term(&hash_seq);
+			goto gc_fail;
+		}
+		entry->good_plan.offset = extent;
+		extent += good_plan_len + 1;
+
+		if(bad_plan_len <= 0)
+		{
+			bad_plan = "\0";
+			bad_plan_len = 0;
+		}
+		/* Save the bad plan */
+		if (fwrite(bad_plan, 1, bad_plan_len + 1, qfile) != bad_plan_len + 1)
+		{
+			ereport(LOG,
+					(errcode_for_file_access(),
+				  errmsg("could not write worst plan pg_stat_statement file \"%s\": %m",
+						 PGSS_TEXT_FILE)));
+			hash_seq_term(&hash_seq);
+			goto gc_fail;
+		}
+		entry->bad_plan.offset = extent;
+		extent += bad_plan_len + 1;
+
 		nentries++;
 	}
 
diff --git a/contrib/pg_stat_statements/sql/pg_stat_statements.sql b/contrib/pg_stat_statements/sql/pg_stat_statements.sql
index a8361fd1bf..9e02470dfd 100644
--- a/contrib/pg_stat_statements/sql/pg_stat_statements.sql
+++ b/contrib/pg_stat_statements/sql/pg_stat_statements.sql
@@ -195,4 +195,15 @@ DROP FUNCTION PLUS_TWO(INTEGER);
 
 SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
 
+-- test to see if any plans have been recorded.
+SELECT
+  CASE WHEN good_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN bad_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN good_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END,
+  CASE WHEN good_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END
+FROM pg_stat_statements ORDER BY query COLLATE "C";
+
+-- test if there is some text in the recorded plans.
+select substr(good_plan, 0, 11), substr(bad_plan, 0, 11) from pg_stat_statements ORDER BY query COLLATE "C";
+
 DROP EXTENSION pg_stat_statements;
smime.p7sapplication/pkcs7-signature; name=smime.p7sDownload
#9David Fetter
david@fetter.org
In reply to: Julian Markwort (#8)
Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

On Wed, Jan 10, 2018 at 03:05:39PM +0100, Julian Markwort wrote:

Hello hackers,

I'd like to follow up to my previous proposition of tracking (some) best and
worst plans for different queries in the pg_stat_statements extension.

Based on the comments and suggestions made towards my last endeavour, I've
taken the path of computing the interquartile distance (by means of an
adapted z-test, under the assumption of normal distribution, based on the
mean_time and stddev_time already used by the extension).

Is the assumption of a normal distribution reasonable for outlier
plans as you've seen them?

Best,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

#10Julian Markwort
julian.markwort@uni-muenster.de
In reply to: David Fetter (#9)
1 attachment(s)
Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

On 01/11/2018 03:43 PM, David Fetter wrote:

Is the assumption of a normal distribution reasonable for outlier
plans as you've seen them?

This is a difficult but fair question.
First of all, I'd like to clarify that the normal distribution is
assumed for the set of all execution times matching a queryid; No
assumptions are made about the distribution of the outliers themselves.
The primary goal of this approach was the limitation of plan updates, to
avoid unnecessary IO operations.
When query performance does not vary much, no updates of the plans
should be necessary, but as soon as query performance varies too much,
the new plan should be stored.
For the purpose of distinguishing reasonable variance between execution
times and great variance due to changing conditions which ultimately
might result in a different plan, the assumption of a normal
distribution for all execution times suits well.

Based on some early testing, this results in only a few percent of
updates (1-3%) in relation to the total number of calls, when running
some short pgbench tests.
As the sample size grows, the assumption of a normal distribution
becomes increasingly accurate and the (unnecessary) sampling of plans
decreases.
In a different test, I ran several queries with identical table sizes,
the queries were fairly simple,  and the statistical evaluation led to
few updates during these tests. When I increased the table size
significantly, the database switched to a different plan. Because the
execution time differed significantly, this new bad plan was stored.
Similarly, after running a certain query a couple of times, I created an
index on my test data, which resulted in a speedup which was significant
enough to result in an update of the good plan.

Now, if a change to the data or the index situation only resulted in an
insignificant performance increase or decrease (one that falls into the
interval defined as [mean - 1.5*IQD, mean + 1-5*IQD] ), I think it might
be possible to assume that we are not interested in an updated plan for
this scenario.

Attachments:

smime.p7sapplication/pkcs7-signature; name=smime.p7sDownload
#11Andres Freund
andres@anarazel.de
In reply to: Julian Markwort (#8)
Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

Hi,

On 2018-01-10 15:05:39 +0100, Julian Markwort wrote:

I'd like to follow up to my previous proposition of tracking (some) best and
worst plans for different queries in the pg_stat_statements extension.

Cool feature.

I think the patch probably doesn't apply anymore, due to other changes
to pg_stat_statements since its posting. Could you refresh?

I've not done any sort of review. Scrolling through I noticed //
comments which aren't pg coding style.

I'd like to see a small benchmark showing the overhead of the feature.
Both in runtime and storage size.

Greetings,

Andres Freund

#12Julian Markwort
julian.markwort@uni-muenster.de
In reply to: Andres Freund (#11)
1 attachment(s)
Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

Andres Freund wrote on 2018-03-01:

I think the patch probably doesn't apply anymore, due to other changes
to pg_stat_statements since its posting. Could you refresh?

pgss_plans_v02.patch applies cleanly to master, there were no changes to pg_stat_statements since the copyright updates at the beginning of January.
(pgss_plans_v02.patch is attached to message 1bd396a9-4573-55ad-7ce8-fe7adffa1bd9@uni-muenster.de and can be found in the current commitfest as well.)

I've not done any sort of review. Scrolling through I noticed //
comments which aren't pg coding style.

I'll fix that along with any other problems that might be found in a review.

I'd like to see a small benchmark showing the overhead of the feature.
Both in runtime and storage size.

I've tried to gather some meaningful results, however either my testing methodology was flawed (as variance between all my passes of pgbench was rather high) or the takeaway is that the feature only generates little overhead.
This is what I've run on my workstation using a Ryzen 1700 and 16GB of RAM and an old Samsung 840 Evo as boot drive, which also held the database:
The database used for the tests was dropped and pgbench initialized anew for each test (pgss off, pgss on, pgss on with plan collection) using a scaling of 16437704*0.003~=50 (roughly what the phoronix test suite uses for a buffer test).
Also similar to the phoronix test suite, I used 8 jobs and 32 connections for a normal multithreaded load.

I then ran 10 passes, each for 60 seconds, with a 30 second pause between them, as well as another test which ran for 10 minutes.

With pg_stat_statements on, the latter test (10 minutes) resulted in 1833 tps, while the patched version resulted in 1700 tps, so a little over 7% overhead? Well, the "control run", without pg_stat_statements delivered only 1806 tps, so variance seems to be quite high.

The results of the ten successive tests, each running 60 seconds and then waiting for 30 seconds, are displayed in the attached plot.
I've tinkered with different settings with pgbench for quite some time now and all I can come up with are runs with high variance between them.

If anybody has any recommendations for a setup that generates less variance, I'll try this again.

Finally, the more interesting metric regarding this patch is the size of the pg_stat_statements.stat file, which stores all the metrics while the database is shut down. I reckon that the size of pgss_query_texts.stat (which holds only the query strings and plan strings while the database is running) will be similar, however it might fluctuate more as new strings are simply appended to the file until the garbagecollector decides that it has to be cleaned up.
After running the aforementioned tests, the file was 8566 bytes in size for pgss in it's unmodified form, while the tests resulted in 32607 bytes for the pgss that collects plans as well. This seems reasonable as plans strings are usually longer than the statements from which they result. Worst case, the pg_stat_statements.stat holds two plans for each type of statement.
I've not tested the length of the file with different encodings, such as JSON, YAML, or XML, however I do not expect any hugely different results.

Greetings
Julian

Attachments:

pgss_plans_pgbench.pdfapplication/pdfDownload
#13Andres Freund
andres@anarazel.de
In reply to: Julian Markwort (#12)
Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

On 2018-03-02 18:07:32 +0100, Julian Markwort wrote:

Andres Freund wrote on 2018-03-01:

I think the patch probably doesn't apply anymore, due to other changes
to pg_stat_statements since its posting. Could you refresh?

pgss_plans_v02.patch applies cleanly to master, there were no changes to pg_stat_statements since the copyright updates at the beginning of January.
(pgss_plans_v02.patch is attached to message 1bd396a9-4573-55ad-7ce8-fe7adffa1bd9@uni-muenster.de and can be found in the current commitfest as well.)

Yea, I misread the diff to think you added a conflicting version. Due
to:
-DATA =3D pg_stat_statements--1.4.sql pg_stat_statements--1.4--1.5.sql \
+DATA =3D pg_stat_statements--1.5.sql pg_stat_statements--1.4--1.5.sql \

and I'd checked that 1.5 already exists. But you just renamed the file,
presumably because it's essentially rewriting the whole file? I'm not
sure I'm a big fan of doing so, because that makes testing the upgrade
path more work.

I've tried to gather some meaningful results, however either my testing methodology was flawed (as variance between all my passes of pgbench was rather high) or the takeaway is that the feature only generates little overhead.
This is what I've run on my workstation using a Ryzen 1700 and 16GB of RAM and an old Samsung 840 Evo as boot drive, which also held the database:
The database used for the tests was dropped and pgbench initialized anew for each test (pgss off, pgss on, pgss on with plan collection) using a scaling of 16437704*0.003~=50 (roughly what the phoronix test suite uses for a buffer test).
Also similar to the phoronix test suite, I used 8 jobs and 32 connections for a normal multithreaded load.

I then ran 10 passes, each for 60 seconds, with a 30 second pause between them, as well as another test which ran for 10 minutes.

What workload did you run? read/write or readonly? This seems like a
feature were readonly makes a lot more sense. But ~1800 tps strongly
suggests that's not what you did?

With pg_stat_statements on, the latter test (10 minutes) resulted in 1833 tps, while the patched version resulted in 1700 tps, so a little over 7% overhead? Well, the "control run", without pg_stat_statements delivered only 1806 tps, so variance seems to be quite high.

That's quite some overhead, I'd say.

If anybody has any recommendations for a setup that generates less variance, I'll try this again.

I'd suggest disabling turboost, in my experience that makes tests
painful to repeat, because it'll strongly depend on the current HW
temperature.

Greetings,

Andres Freund

#14Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#13)
Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

Andres Freund <andres@anarazel.de> writes:

Yea, I misread the diff to think you added a conflicting version. Due
to:
-DATA =3D pg_stat_statements--1.4.sql pg_stat_statements--1.4--1.5.sql \
+DATA =3D pg_stat_statements--1.5.sql pg_stat_statements--1.4--1.5.sql \

and I'd checked that 1.5 already exists. But you just renamed the file,
presumably because it's essentially rewriting the whole file? I'm not
sure I'm a big fan of doing so, because that makes testing the upgrade
path more work.

Yeah, that is not project style anymore. Just add a delta file and leave
it at that. See e.g. de1d042f5979bc1388e9a6d52a4d445342b04932 for an
example.

(We might from time to time roll up the deltas and replace the base
file with a newer version, but I think that should happen in separate
housekeeping commits.)

regards, tom lane

#15Julian Markwort
julian.markwort@uni-muenster.de
In reply to: Andres Freund (#13)
Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

Andres Freund wrote on 2018-03-02:

Yea, I misread the diff to think you added a conflicting version. Due
to:
-DATA =3D pg_stat_statements--1.4.sql pg_stat_statements--1.4--1.5.sql \
+DATA =3D pg_stat_statements--1.5.sql pg_stat_statements--1.4--1.5.sql \

and I'd checked that 1.5 already exists. But you just renamed the file,
presumably because it's essentially rewriting the whole file? I'm not
sure I'm a big fan of doing so, because that makes testing the upgrade
path more work.

You're right about 1.5 already existing. I wasn't sure about the versioning policy for extensions and seeing as version 1.5 only changed a few grants I quasi reused 1.5 for my intentions.

What workload did you run? read/write or readonly? This seems like a
feature were readonly makes a lot more sense. But ~1800 tps strongly
suggests that's not what you did?

I'm sorry I forgot to mention this; I ran all tests as read-write.

With pg_stat_statements on, the latter test (10 minutes) resulted in 1833 tps, while the patched version resulted in 1700 tps, so a little over 7% overhead? Well, the "control run", without pg_stat_statements delivered only 1806 tps, so variance seems to be quite high.

That's quite some overhead, I'd say.

Yes, but I wouldn't give a warranty that it is neither more nor less overhead than 7%, seeing as for my testing, the tps were higher for (unmodified) pgss enabled vs no pgss at all.

If anybody has any recommendations for a setup that generates less variance, I'll try this again.

I'd suggest disabling turboost, in my experience that makes tests
painful to repeat, because it'll strongly depend on the current HW
temperature.

This might be a problem for average systems but I'm fairly certain this isn't the issue here.

I might try some more benchmarks and will in particular look into running read-only tests, as the aforementioned 840 EVO SSD ist -comparatively speaking- pretty slow.
Do you have any recommendations as to what constitutes adequate testing times regarding pgbench?

Kind regards
Julian

#16Tom Lane
tgl@sss.pgh.pa.us
In reply to: Julian Markwort (#15)
Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

Julian Markwort <julian.markwort@uni-muenster.de> writes:

Andres Freund wrote on 2018-03-02:

and I'd checked that 1.5 already exists. But you just renamed the file,
presumably because it's essentially rewriting the whole file? I'm not
sure I'm a big fan of doing so, because that makes testing the upgrade
path more work.

You're right about 1.5 already existing. I wasn't sure about the versioning policy for extensions and seeing as version 1.5 only changed a few grants I quasi reused 1.5 for my intentions.

Nope, that's totally wrong. You can get away with that if we've not
already shipped a 1.5 release --- but we did ship it in v10, so that
version is frozen now. You need to make your changes in a 1.5--1.6
upgrade file. Otherwise there's no clean path for existing installations
to upgrade to the new version.

regards, tom lane

#17Julian Markwort
julian.markwort@uni-muenster.de
In reply to: Tom Lane (#16)
Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

Tom Lane wrote on 2018-03-02:

You need to make your changes in a 1.5--1.6
upgrade file. Otherwise there's no clean path for existing
installations
to upgrade to the new version.

I've adressed all the issues that were brought up so far:
1. there is now only an added 1.5--1.6.sql file.
2. the overhead, as previously discussed (as much as a 12% decrease in
TPS during read-only tests), is now gone, the problem was that I was
collecting the plan String before checking if it needed to be stored at
all.
The patched version is now 99.95% as fast as unmodified
pg_stat_statements.
3. I've cleaned up my own code and made sure it adheres to GNU C coding
style, I was guilty of some // comments and curly brackets were
sometimes in the same line as my control structures.

I'd love to hear more feedback, here are two ideas to polish this
patch:
a) Right now, good_plan and bad_plan collection can be activated and
deactivated with separate GUCs. I think it would be sensible to collect
either both or none. (This would result in fewer convoluted
conditionals.)
b) Would you like to be able to tune the percentiles yourself, to
adjust for the point at which a new plan is stored?

Greetings
Julian

#18Julian Markwort
julian.markwort@uni-muenster.de
In reply to: Julian Markwort (#17)
1 attachment(s)
Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

I've goofed up now, sorry for failing to attach my updated patch.

Am Donnerstag, den 08.03.2018, 14:55 +0100 schrieb Julian Markwort:

Show quoted text

Tom Lane wrote on 2018-03-02:

You need to make your changes in a 1.5--1.6
upgrade file. Otherwise there's no clean path for existing
installations
to upgrade to the new version.

I've adressed all the issues that were brought up so far:
1. there is now only an added 1.5--1.6.sql file.
2. the overhead, as previously discussed (as much as a 12% decrease
in
TPS during read-only tests), is now gone, the problem was that I was
collecting the plan String before checking if it needed to be stored
at
all.
The patched version is now 99.95% as fast as unmodified
pg_stat_statements.
3. I've cleaned up my own code and made sure it adheres to GNU C
coding
style, I was guilty of some // comments and curly brackets were
sometimes in the same line as my control structures.

I'd love to hear more feedback, here are two ideas to polish this
patch:
a) Right now, good_plan and bad_plan collection can be activated and
deactivated with separate GUCs. I think it would be sensible to
collect
either both or none. (This would result in fewer convoluted
conditionals.)
b) Would you like to be able to tune the percentiles yourself, to
adjust for the point at which a new plan is stored?

Greetings
Julian

Attachments:

pgss_plans_v03.patchtext/x-patch; charset=UTF-8; name=pgss_plans_v03.patchDownload
diff --git a/contrib/pg_stat_statements/Makefile b/contrib/pg_stat_statements/Makefile
index 39b368b..49bb462 100644
--- a/contrib/pg_stat_statements/Makefile
+++ b/contrib/pg_stat_statements/Makefile
@@ -4,7 +4,8 @@ MODULE_big = pg_stat_statements
 OBJS = pg_stat_statements.o $(WIN32RES)
 
 EXTENSION = pg_stat_statements
-DATA = pg_stat_statements--1.4.sql pg_stat_statements--1.4--1.5.sql \
+DATA = pg_stat_statements--1.4.sql \
+	pg_stat_statements--1.5--1.6.sql pg_stat_statements--1.4--1.5.sql \
 	pg_stat_statements--1.3--1.4.sql pg_stat_statements--1.2--1.3.sql \
 	pg_stat_statements--1.1--1.2.sql pg_stat_statements--1.0--1.1.sql \
 	pg_stat_statements--unpackaged--1.0.sql
diff --git a/contrib/pg_stat_statements/expected/pg_stat_statements.out b/contrib/pg_stat_statements/expected/pg_stat_statements.out
index 5318c35..3e79890 100644
--- a/contrib/pg_stat_statements/expected/pg_stat_statements.out
+++ b/contrib/pg_stat_statements/expected/pg_stat_statements.out
@@ -395,4 +395,40 @@ SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
  SELECT pg_stat_statements_reset()         |     1 |    1
 (8 rows)
 
+-- test to see if any plans have been recorded.
+SELECT
+  CASE WHEN good_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN bad_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN good_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END,
+  CASE WHEN good_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END
+FROM pg_stat_statements ORDER BY query COLLATE "C";
+ case | case | case | case 
+------+------+------+------
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    1 |    1 |    1 |    1
+    1 |    1 |    1 |    1
+    1 |    1 |    1 |    1
+(9 rows)
+
+-- test if there is some text in the recorded plans.
+select substr(good_plan, 0, 11), substr(bad_plan, 0, 11) from pg_stat_statements ORDER BY query COLLATE "C";
+   substr   |   substr   
+------------+------------
+            | 
+            | 
+            | 
+            | 
+            | 
+            | 
+ Query Text | Query Text
+ Query Text | Query Text
+ Query Text | Query Text
+ Query Text | Query Text
+(10 rows)
+
 DROP EXTENSION pg_stat_statements;
diff --git a/contrib/pg_stat_statements/pg_stat_statements--1.5--1.6.sql b/contrib/pg_stat_statements/pg_stat_statements--1.5--1.6.sql
new file mode 100644
index 0000000..5302d01
--- /dev/null
+++ b/contrib/pg_stat_statements/pg_stat_statements--1.5--1.6.sql
@@ -0,0 +1,78 @@
+/* contrib/pg_stat_statements/pg_stat_statements--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "ALTER EXTENSION pg_stat_statements UPDATE TO '1.6'" to load this file. \quit
+
+/* First we have to remove them from the extension */
+ALTER EXTENSION pg_stat_statements DROP VIEW pg_stat_statements;
+ALTER EXTENSION pg_stat_statements DROP FUNCTION pg_stat_statements(boolean);
+ALTER EXTENSION pg_stat_statements DROP FUNCTION pg_stat_statements_reset();
+
+/* Then we can drop them */
+DROP VIEW pg_stat_statements;
+DROP FUNCTION pg_stat_statements(boolean);
+DROP FUNCTION pg_stat_statements_reset();
+
+-- Register functions.
+CREATE FUNCTION pg_stat_statements_reset()
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_statements_good_plan_reset(IN queryid bigint)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_statements_bad_plan_reset(IN queryid bigint)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_statements(IN showtext boolean,
+    OUT userid oid,
+    OUT dbid oid,
+    OUT queryid bigint,
+    OUT query text,
+    OUT good_plan text,
+    OUT bad_plan text,
+    OUT calls int8,
+    OUT total_time float8,
+    OUT min_time float8,
+    OUT max_time float8,
+    OUT mean_time float8,
+    OUT stddev_time float8,
+    OUT good_plan_time float8,
+    OUT good_plan_timestamp timestamp,
+    OUT bad_plan_time float8,
+    OUT bad_plan_timestamp timestamp,
+    OUT rows int8,
+    OUT shared_blks_hit int8,
+    OUT shared_blks_read int8,
+    OUT shared_blks_dirtied int8,
+    OUT shared_blks_written int8,
+    OUT local_blks_hit int8,
+    OUT local_blks_read int8,
+    OUT local_blks_dirtied int8,
+    OUT local_blks_written int8,
+    OUT temp_blks_read int8,
+    OUT temp_blks_written int8,
+    OUT blk_read_time float8,
+    OUT blk_write_time float8
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stat_statements_1_6'
+LANGUAGE C STRICT VOLATILE PARALLEL SAFE;
+
+-- Register a view on the function for ease of use.
+CREATE VIEW pg_stat_statements AS
+  SELECT * FROM pg_stat_statements(true);
+
+GRANT SELECT ON pg_stat_statements TO PUBLIC;
+
+-- Don't want this to be available to non-superusers.
+REVOKE ALL ON FUNCTION pg_stat_statements_reset() FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_stat_statements_good_plan_reset(bigint) FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_stat_statements_bad_plan_reset(bigint) FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_stat_statements_reset() TO pg_read_all_stats;
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 9286734..66b54ff 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -63,6 +63,7 @@
 
 #include "access/hash.h"
 #include "catalog/pg_authid.h"
+#include "commands/explain.h"
 #include "executor/instrument.h"
 #include "funcapi.h"
 #include "mb/pg_wchar.h"
@@ -78,6 +79,7 @@
 #include "tcop/utility.h"
 #include "utils/builtins.h"
 #include "utils/memutils.h"
+#include "utils/timestamp.h"
 
 PG_MODULE_MAGIC;
 
@@ -119,7 +121,8 @@ typedef enum pgssVersion
 	PGSS_V1_0 = 0,
 	PGSS_V1_1,
 	PGSS_V1_2,
-	PGSS_V1_3
+	PGSS_V1_3,
+	PGSS_V1_6
 } pgssVersion;
 
 /*
@@ -165,6 +168,14 @@ typedef struct Counters
 	double		usage;			/* usage factor */
 } Counters;
 
+typedef struct pgssPlan
+{
+	Size	offset;
+	int		len;
+	double		time;	/* execution time in msec when the latest plan was updated */
+	TimestampTz timestamp;
+} pgssPlan;
+
 /*
  * Statistics per statement
  *
@@ -178,6 +189,8 @@ typedef struct pgssEntry
 	Counters	counters;		/* the statistics for this query */
 	Size		query_offset;	/* query text offset in external file */
 	int			query_len;		/* # of valid bytes in query string, or -1 */
+	pgssPlan	good_plan;
+	pgssPlan	bad_plan;
 	int			encoding;		/* query text encoding */
 	slock_t		mutex;			/* protects the counters only */
 } pgssEntry;
@@ -265,11 +278,22 @@ static const struct config_enum_entry track_options[] =
 	{NULL, 0, false}
 };
 
+static const struct config_enum_entry format_options[] =
+{
+	{"text", EXPLAIN_FORMAT_TEXT, false},
+	{"xml", EXPLAIN_FORMAT_XML, false},
+	{"json", EXPLAIN_FORMAT_JSON, false},
+	{"yaml", EXPLAIN_FORMAT_YAML, false},
+	{NULL, 0, false}
+};
+
 static int	pgss_max;			/* max # statements to track */
 static int	pgss_track;			/* tracking level */
 static bool pgss_track_utility; /* whether to track utility commands */
 static bool pgss_save;			/* whether to save stats across shutdown */
-
+static bool pgss_good_plan_enable; 	/* whether to save good_plans */
+static bool pgss_bad_plan_enable; 	/* whether to save good_plans */
+static int	pgss_plan_format;   	/* id which sets the output format */
 
 #define pgss_enabled() \
 	(pgss_track == PGSS_TRACK_ALL || \
@@ -291,7 +315,11 @@ void		_PG_fini(void);
 PG_FUNCTION_INFO_V1(pg_stat_statements_reset);
 PG_FUNCTION_INFO_V1(pg_stat_statements_1_2);
 PG_FUNCTION_INFO_V1(pg_stat_statements_1_3);
+PG_FUNCTION_INFO_V1(pg_stat_statements_1_6);
 PG_FUNCTION_INFO_V1(pg_stat_statements);
+PG_FUNCTION_INFO_V1(pg_stat_statements_good_plan_reset);
+PG_FUNCTION_INFO_V1(pg_stat_statements_bad_plan_reset);
+
 
 static void pgss_shmem_startup(void);
 static void pgss_shmem_shutdown(int code, Datum arg);
@@ -309,12 +337,17 @@ static void pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
 static uint64 pgss_hash_string(const char *str, int len);
 static void pgss_store(const char *query, uint64 queryId,
 		   int query_location, int query_len,
+		   QueryDesc *queryDesc,
 		   double total_time, uint64 rows,
 		   const BufferUsage *bufusage,
 		   pgssJumbleState *jstate);
+Datum pg_stat_statements_plan_reset(uint64 query_id, uint8 plan_type);
 static void pg_stat_statements_internal(FunctionCallInfo fcinfo,
 							pgssVersion api_version,
 							bool showtext);
+static int fill_plan_times(Datum values[], bool nulls[], int i, pgssPlan *plan);
+static int fill_plan_str(Datum values[], bool nulls[], int i, pgssPlan *plan,
+							pgssEntry *entry, char *qbuffer, Size qbuffer_size);
 static Size pgss_memsize(void);
 static pgssEntry *entry_alloc(pgssHashKey *key, Size query_offset, int query_len,
 			int encoding, bool sticky);
@@ -407,6 +440,40 @@ _PG_init(void)
 							 NULL,
 							 NULL);
 
+	DefineCustomBoolVariable("pg_stat_statements.good_plan_enable",
+							 "Enable bad plan detection",
+							 NULL,
+							 &pgss_good_plan_enable,
+							 true,
+							 PGC_SUSET,
+							 0,
+							 NULL,
+							 NULL,
+							 NULL);
+
+	DefineCustomBoolVariable("pg_stat_statements.bad_plan_enable",
+							 "Enable bad plan detection",
+							 NULL,
+							 &pgss_bad_plan_enable,
+							 true,
+							 PGC_SUSET,
+							 0,
+							 NULL,
+							 NULL,
+							 NULL);
+
+	DefineCustomEnumVariable("pg_stat_statements.plan_format",
+							 "Sets the output format for the plans.",
+							 "Notice that the plan format can not be changed after a plan is saved. Valid values are text, json, xml and yaml.",
+							 &pgss_plan_format,
+							 EXPLAIN_FORMAT_TEXT,
+							 format_options,
+							 PGC_SUSET,
+							 0,
+							 NULL,
+							 NULL,
+							 NULL);
+
 	EmitWarningsOnPlaceholders("pg_stat_statements");
 
 	/*
@@ -470,7 +537,11 @@ pgss_shmem_startup(void)
 	int32		pgver;
 	int32		i;
 	int			buffer_size;
+	int			good_plan_buffer_size;
+	int			bad_plan_buffer_size;
 	char	   *buffer = NULL;
+	char	   *good_plan_buffer = NULL;
+	char	   *bad_plan_buffer = NULL;
 
 	if (prev_shmem_startup_hook)
 		prev_shmem_startup_hook();
@@ -561,7 +632,11 @@ pgss_shmem_startup(void)
 	}
 
 	buffer_size = 2048;
+	good_plan_buffer_size = 2048;
+	bad_plan_buffer_size = 2048;
 	buffer = (char *) palloc(buffer_size);
+	good_plan_buffer = (char *) palloc(good_plan_buffer_size);
+	bad_plan_buffer = (char *) palloc(bad_plan_buffer_size);
 
 	if (fread(&header, sizeof(uint32), 1, file) != 1 ||
 		fread(&pgver, sizeof(uint32), 1, file) != 1 ||
@@ -595,8 +670,34 @@ pgss_shmem_startup(void)
 		if (fread(buffer, 1, temp.query_len + 1, file) != temp.query_len + 1)
 			goto read_error;
 
+		/* Resize good plan_buffer and read into it */
+		if (temp.good_plan.len >= good_plan_buffer_size)
+		{
+			good_plan_buffer_size =
+				Max(good_plan_buffer_size * 2, temp.good_plan.len + 1);
+			good_plan_buffer= repalloc(good_plan_buffer, good_plan_buffer_size);
+		}
+
+		if (fread(good_plan_buffer, 1, temp.good_plan.len + 1, file)
+				!= temp.good_plan.len + 1)
+			goto read_error;
+
+		/* Resize bad plan_buffer and read into it */
+		if (temp.bad_plan.len >= bad_plan_buffer_size)
+		{
+			bad_plan_buffer_size =
+				Max(bad_plan_buffer_size * 2, temp.bad_plan.len + 1);
+			bad_plan_buffer= repalloc(bad_plan_buffer, bad_plan_buffer_size);
+		}
+
+		if (fread(bad_plan_buffer, 1, temp.bad_plan.len + 1, file)
+				!= temp.bad_plan.len + 1)
+			goto read_error;
+
 		/* Should have a trailing null, but let's make sure */
 		buffer[temp.query_len] = '\0';
+		good_plan_buffer[temp.good_plan.len] = '\0';
+		bad_plan_buffer[temp.bad_plan.len] = '\0';
 
 		/* Skip loading "sticky" entries */
 		if (temp.counters.calls == 0)
@@ -608,6 +709,20 @@ pgss_shmem_startup(void)
 			goto write_error;
 		pgss->extent += temp.query_len + 1;
 
+		/* Store the good plan text*/
+		temp.good_plan.offset = pgss->extent;
+		if (fwrite(good_plan_buffer, 1, temp.good_plan.len + 1, qfile)
+				!= temp.good_plan.len + 1)
+			goto write_error;
+		pgss->extent += temp.good_plan.len + 1;
+
+		/* Store the bad plan text*/
+		temp.bad_plan.offset = pgss->extent;
+		if (fwrite(bad_plan_buffer, 1, temp.bad_plan.len + 1, qfile)
+				!= temp.bad_plan.len + 1)
+			goto write_error;
+		pgss->extent += temp.bad_plan.len + 1;
+
 		/* make the hashtable entry (discards old entries if too many) */
 		entry = entry_alloc(&temp.key, query_offset, temp.query_len,
 							temp.encoding,
@@ -615,9 +730,14 @@ pgss_shmem_startup(void)
 
 		/* copy in the actual stats */
 		entry->counters = temp.counters;
+				/* copy in the plans */
+		entry->good_plan = temp.good_plan;
+		entry->bad_plan = temp.bad_plan;
 	}
 
 	pfree(buffer);
+	pfree(good_plan_buffer);
+	pfree(bad_plan_buffer);
 	FreeFile(file);
 	FreeFile(qfile);
 
@@ -658,6 +778,10 @@ write_error:
 fail:
 	if (buffer)
 		pfree(buffer);
+	if (good_plan_buffer)
+		pfree(good_plan_buffer);
+	if (bad_plan_buffer)
+		pfree(bad_plan_buffer);
 	if (file)
 		FreeFile(file);
 	if (qfile)
@@ -723,14 +847,54 @@ pgss_shmem_shutdown(int code, Datum arg)
 	while ((entry = hash_seq_search(&hash_seq)) != NULL)
 	{
 		int			len = entry->query_len;
+		int			good_plan_len = entry->good_plan.len;
+		int			bad_plan_len = entry->bad_plan.len;
 		char	   *qstr = qtext_fetch(entry->query_offset, len,
 									   qbuffer, qbuffer_size);
+		char	   *good_plan_str;
+		char	   *bad_plan_str;
+		if (good_plan_len > 0)
+		{
+			/* A good plan is available */
+			good_plan_str = qtext_fetch(entry->good_plan.offset, good_plan_len,
+												qbuffer, qbuffer_size);
+		}
+		else
+		{
+			/*
+			 * There is no good plan available. This can be caused by a query
+			 * which has no plan (for example insert or delete queries) or
+			 * because good plans are disabled. In this case we have to store
+			 * an empty string instead of null.
+			 */
+			good_plan_str = palloc(1 * sizeof(char));
+			*good_plan_str = '\0';
+		}
+		if (bad_plan_len > 0)
+		{
+			/* A bad plan is available */
+			bad_plan_str = qtext_fetch(entry->bad_plan.offset, bad_plan_len,
+												qbuffer, qbuffer_size);
+		}
+		else
+		{
+			/*
+			 * There is no bad plan available. This can be caused by a query
+			 * which has no plan (for example insert or delete queries) or
+			 * because bad plans are disabled. In this case we have to store
+			 * an empty string instead of null.
+			 */
+			bad_plan_str = palloc(1 * sizeof(char));
+			*bad_plan_str = '\0';
+		}
 
 		if (qstr == NULL)
 			continue;			/* Ignore any entries with bogus texts */
-
+		/* Write entries, queries and plans serialized to the dump file */
 		if (fwrite(entry, sizeof(pgssEntry), 1, file) != 1 ||
-			fwrite(qstr, 1, len + 1, file) != len + 1)
+			fwrite(qstr, 1, len + 1, file) != len + 1 ||
+			fwrite(good_plan_str, 1, good_plan_len + 1, file) != good_plan_len + 1 ||
+			fwrite(bad_plan_str, 1, bad_plan_len + 1, file) != bad_plan_len + 1)
 		{
 			/* note: we assume hash_seq_term won't change errno */
 			hash_seq_term(&hash_seq);
@@ -835,6 +999,7 @@ pgss_post_parse_analyze(ParseState *pstate, Query *query)
 				   query->queryId,
 				   query->stmt_location,
 				   query->stmt_len,
+				   NULL,
 				   0,
 				   0,
 				   NULL,
@@ -942,6 +1107,7 @@ pgss_ExecutorEnd(QueryDesc *queryDesc)
 				   queryId,
 				   queryDesc->plannedstmt->stmt_location,
 				   queryDesc->plannedstmt->stmt_len,
+				   queryDesc,								/* queryDesc itself is needed to retrieve the plan */
 				   queryDesc->totaltime->total * 1000.0,	/* convert to msec */
 				   queryDesc->estate->es_processed,
 				   &queryDesc->totaltime->bufusage,
@@ -1053,6 +1219,7 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
 				   0,			/* signal that it's a utility stmt */
 				   pstmt->stmt_location,
 				   pstmt->stmt_len,
+				   NULL,
 				   INSTR_TIME_GET_MILLISEC(duration),
 				   rows,
 				   &bufusage,
@@ -1083,6 +1250,26 @@ pgss_hash_string(const char *str, int len)
 											len, 0));
 }
 
+/*
+ * Update a plan in the text file.
+ * We can re-use qtext_store to save the plan.
+ */
+static void
+update_plan(volatile pgssPlan* pgssPlan_ptr, const char* plan_str, int plan_len,
+			double total_time)
+{
+	if (plan_len > 0)
+	{
+		Size plan_offset;
+		qtext_store(plan_str, plan_len, &plan_offset, NULL);
+		pgssPlan_ptr->offset	= plan_offset;
+		pgssPlan_ptr->len		= plan_len;
+		pgssPlan_ptr->time		= total_time;
+		pgssPlan_ptr->timestamp = GetCurrentTimestamp();
+	}
+	return;
+}
+
 /*
  * Store some statistics for a statement.
  *
@@ -1096,6 +1283,7 @@ pgss_hash_string(const char *str, int len)
 static void
 pgss_store(const char *query, uint64 queryId,
 		   int query_location, int query_len,
+		   QueryDesc *queryDesc,
 		   double total_time, uint64 rows,
 		   const BufferUsage *bufusage,
 		   pgssJumbleState *jstate)
@@ -1134,6 +1322,7 @@ pgss_store(const char *query, uint64 queryId,
 		query_len = strlen(query);
 	}
 
+
 	/*
 	 * Discard leading and trailing whitespace, too.  Use scanner_isspace()
 	 * not libc's isspace(), because we want to match the lexer's behavior.
@@ -1231,9 +1420,106 @@ pgss_store(const char *query, uint64 queryId,
 		 * locking rules at the head of the file)
 		 */
 		volatile pgssEntry *e = (volatile pgssEntry *) entry;
+		double	interquartile_dist = 0.0;
+		bool	good_plan_needs_updating = false;
+		bool	bad_plan_needs_updating = false;
+		int		plan_len = 0;
+		char*	plan;
 
 		SpinLockAcquire(&e->mutex);
 
+		if (queryDesc && (pgss_good_plan_enable || pgss_bad_plan_enable)){
+			/*
+			 * The Z-scores allows us to estimate the values at any quantiles
+			 * in a list of values with (assumed) normal distribution, known mean,
+			 * and known standard deviation. The value at any quantile is equal to
+			 * z*stddev + mean. The distance to the mean is thus z*stddev,
+			 * which we'll use to calculate the interquartile distance.
+			 */
+			interquartile_dist = 2.0*(0.6745 * sqrt(e->counters.sum_var_time / e->counters.calls));
+
+			if (pgss_good_plan_enable)
+			{
+				/*
+				 * Good plan needs to be updated if no previous good_plan has beed recorded.
+				 * Else, if the execution time is smaller than the minimum of mean - 1.5*IQD,
+				 * and the previous good_plan's time, the good_plan needs to be updated.
+				 */
+				if (e->good_plan.time == 0)
+				{
+					good_plan_needs_updating = true;
+				}
+				else if (interquartile_dist > 0.0 && total_time < e->good_plan.time
+							&& total_time < (e->counters.mean_time - 1.5 * interquartile_dist))
+				{
+					good_plan_needs_updating = true;
+				}
+
+			}
+
+			if (pgss_bad_plan_enable)
+			{
+				/*
+				 * Bad plan needs to be updated if no previous bad_plan has beed recorded.
+				 * Else, if the execution time is greater than the maximum of mean + 1.5*IQD,
+				 * and the previous bad_plan's time, the bad_plan needs to be updated.
+				 */
+
+				if (e->bad_plan.time == 0)
+				{
+					bad_plan_needs_updating = true;
+				}
+				else if (interquartile_dist > 0.0 && total_time > e->bad_plan.time
+							&& total_time > (e->counters.mean_time + 1.5 * interquartile_dist))
+				{
+					bad_plan_needs_updating = true;
+				}
+			}
+
+			if (good_plan_needs_updating || bad_plan_needs_updating){
+				/* *es is used to retrieve the plan associated with the statement*/
+				ExplainState *es = NewExplainState();
+
+				es->analyze = (queryDesc->instrument_options && false);
+				es->verbose = false;
+				es->buffers = (es->analyze && false);
+				es->timing = (es->analyze && true);
+				es->format = pgss_plan_format;
+
+				ExplainBeginOutput(es);
+				ExplainQueryText(es, queryDesc);
+				ExplainPrintPlan(es, queryDesc);
+				ExplainEndOutput(es);
+
+				/* Remove last line break */
+				if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+					es->str->data[--es->str->len] = '\0';
+
+				/* Fix JSON to output an object */
+				if (pgss_plan_format == EXPLAIN_FORMAT_JSON)
+				{
+					es->str->data[0] = '{';
+					es->str->data[es->str->len - 1] = '}';
+				}
+
+				plan = es->str->data;
+				plan_len = strlen(plan);
+
+				if (good_plan_needs_updating)
+				{
+					update_plan(&e->good_plan, plan, plan_len, total_time);
+				}
+
+				if (bad_plan_needs_updating)
+				{
+					update_plan(&e->bad_plan, plan, plan_len, total_time);
+				}
+
+				pfree(es->str->data);
+				pfree(es);
+			}
+		}
+
 		/* "Unstick" entry if it was previously sticky */
 		if (e->counters.calls == 0)
 			e->counters.usage = USAGE_INIT;
@@ -1305,12 +1591,89 @@ pg_stat_statements_reset(PG_FUNCTION_ARGS)
 	PG_RETURN_VOID();
 }
 
+Datum
+pg_stat_statements_plan_reset(uint64 query_id, uint8 plan_type)
+{
+	pgssEntry   *entry;
+
+	pgssHashKey	key;
+
+	if (!pgss || !pgss_hash)
+		ereport(ERROR,
+				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+				 errmsg("pg_stat_statements must be loaded via shared_preload_libraries")));
+
+	/* Set up key for hashtable search */
+	key.userid = GetUserId();
+	key.dbid = MyDatabaseId;
+	key.queryid = query_id;
+
+	/* Lookup the hash table entry with shared lock. */
+	LWLockAcquire(pgss->lock, LW_SHARED);
+
+	entry = (pgssEntry *) hash_search(pgss_hash, &key, HASH_FIND, NULL);
+
+	if (entry)
+	{
+		if (superuser() || entry->key.userid == key.userid)
+		{
+			pgssPlan *pgssPlan_ptr;
+			switch (plan_type)
+			{
+				case 1:
+					pgssPlan_ptr = &entry->good_plan;
+					break;
+				case 2:
+					pgssPlan_ptr = &entry->bad_plan;
+					break;
+				default:
+					ereport(ERROR,
+						(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+							errmsg_internal("unrecognized plan type: %d",
+											plan_type)));
+					break;
+			}
+			if (pgssPlan_ptr)
+			{
+				pgssPlan_ptr->offset = -1;
+				pgssPlan_ptr->len = 0;
+				pgssPlan_ptr->time = 0;
+				pgssPlan_ptr->timestamp = 0;
+			}
+		}else{
+			elog(ERROR, "insufficient permission to reset plan");
+		}
+	}
+	LWLockRelease(pgss->lock);
+	PG_RETURN_VOID();
+}
+
+/*
+ * Reset the good_plan of the entry with queryid.
+ */
+Datum
+pg_stat_statements_good_plan_reset(PG_FUNCTION_ARGS)
+{
+	return pg_stat_statements_plan_reset(PG_GETARG_INT64(0), 1);
+}
+
+/*
+ * Reset the bad_plan of the entry with queryid.
+ */
+Datum
+pg_stat_statements_bad_plan_reset(PG_FUNCTION_ARGS)
+{
+	return pg_stat_statements_plan_reset(PG_GETARG_INT64(0), 2);
+}
+
+
 /* Number of output arguments (columns) for various API versions */
 #define PG_STAT_STATEMENTS_COLS_V1_0	14
 #define PG_STAT_STATEMENTS_COLS_V1_1	18
 #define PG_STAT_STATEMENTS_COLS_V1_2	19
 #define PG_STAT_STATEMENTS_COLS_V1_3	23
-#define PG_STAT_STATEMENTS_COLS			23	/* maximum of above */
+#define PG_STAT_STATEMENTS_COLS_V1_6	29
+#define PG_STAT_STATEMENTS_COLS			29	/* maximum of above */
 
 /*
  * Retrieve statement statistics.
@@ -1322,6 +1685,16 @@ pg_stat_statements_reset(PG_FUNCTION_ARGS)
  * expected API version is identified by embedding it in the C name of the
  * function.  Unfortunately we weren't bright enough to do that for 1.1.
  */
+Datum
+pg_stat_statements_1_6(PG_FUNCTION_ARGS)
+{
+ bool		showtext = PG_GETARG_BOOL(0);
+
+ pg_stat_statements_internal(fcinfo, PGSS_V1_6, showtext);
+
+ return (Datum) 0;
+}
+
 Datum
 pg_stat_statements_1_3(PG_FUNCTION_ARGS)
 {
@@ -1428,6 +1801,10 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 			if (api_version != PGSS_V1_3)
 				elog(ERROR, "incorrect number of output arguments");
 			break;
+		case PG_STAT_STATEMENTS_COLS_V1_6:
+			if (api_version != PGSS_V1_6)
+				elog(ERROR, "incorrect number of output arguments");
+			break;
 		default:
 			elog(ERROR, "incorrect number of output arguments");
 	}
@@ -1511,6 +1888,8 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 		bool		nulls[PG_STAT_STATEMENTS_COLS];
 		int			i = 0;
 		Counters	tmp;
+		pgssPlan 	tmp_good_plan;
+		pgssPlan 	tmp_bad_plan;
 		double		stddev;
 		int64		queryid = entry->key.queryid;
 
@@ -1550,6 +1929,18 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 					/* Just return a null if we fail to find the text */
 					nulls[i++] = true;
 				}
+
+				if (api_version >= PGSS_V1_6)
+				{
+					/*
+					 * This version requires to print out the good
+					 * and the bad plan if they are enabled.
+					 */
+					i = fill_plan_str(values, nulls, i, &entry->good_plan,
+										entry, qbuffer, qbuffer_size);
+					i = fill_plan_str(values, nulls, i, &entry->bad_plan,
+										entry, qbuffer, qbuffer_size);
+				}
 			}
 			else
 			{
@@ -1579,6 +1970,8 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 
 			SpinLockAcquire(&e->mutex);
 			tmp = e->counters;
+			tmp_good_plan = e->good_plan;
+			tmp_bad_plan = e->bad_plan;
 			SpinLockRelease(&e->mutex);
 		}
 
@@ -1606,6 +1999,14 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 				stddev = 0.0;
 			values[i++] = Float8GetDatumFast(stddev);
 		}
+
+		if (api_version >= PGSS_V1_6)
+		{
+			/* Set plan times and timestamps */
+			i = fill_plan_times(values, nulls, i, &tmp_good_plan);
+			i = fill_plan_times(values, nulls, i, &tmp_bad_plan);
+		}
+
 		values[i++] = Int64GetDatumFast(tmp.rows);
 		values[i++] = Int64GetDatumFast(tmp.shared_blks_hit);
 		values[i++] = Int64GetDatumFast(tmp.shared_blks_read);
@@ -1629,6 +2030,7 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 					 api_version == PGSS_V1_1 ? PG_STAT_STATEMENTS_COLS_V1_1 :
 					 api_version == PGSS_V1_2 ? PG_STAT_STATEMENTS_COLS_V1_2 :
 					 api_version == PGSS_V1_3 ? PG_STAT_STATEMENTS_COLS_V1_3 :
+					 api_version == PGSS_V1_6 ? PG_STAT_STATEMENTS_COLS_V1_6 :
 					 -1 /* fail if you forget to update this assert */ ));
 
 		tuplestore_putvalues(tupstore, tupdesc, values, nulls);
@@ -1643,6 +2045,52 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 	tuplestore_donestoring(tupstore);
 }
 
+/* Fill the plan time and timestamp into the values array. */
+static int
+fill_plan_times(Datum values[], bool nulls[], int i, pgssPlan *plan)
+{
+	values[i++] = Float8GetDatumFast(plan->time);
+
+	/* if there is noting in the timestamp field, we are not interested in it */
+	if (plan->timestamp)
+		values[i++] = TimestampTzGetDatum(plan->timestamp);
+	else
+		nulls[i++] = true;
+	return i;
+}
+
+/* Fill the plan string into the values array. */
+static int
+fill_plan_str(Datum values[], bool nulls[], int i, pgssPlan *plan,
+				pgssEntry *entry, char *qbuffer, Size qbuffer_size)
+{
+	if (plan && plan->len > 0)
+	{
+		char *pstr = qtext_fetch(plan->offset, plan->len, qbuffer, qbuffer_size);
+		if (pstr)
+		{
+			char *enc;
+			enc = pg_any_to_server(pstr, plan->len, entry->encoding);
+			values[i++] = CStringGetTextDatum(enc);
+
+			if (enc != pstr)
+				pfree(enc);
+		}
+		else
+		{
+			/* failed to get the string of the plan */
+			nulls[i++] = true;
+		}
+	}
+	else
+	{
+		/* no plan available or plan_len not greater than 0 */
+		nulls[i++] = true;
+	}
+	return i;
+}
+
+
 /*
  * Estimate shared memory space needed.
  */
@@ -1694,6 +2142,8 @@ entry_alloc(pgssHashKey *key, Size query_offset, int query_len, int encoding,
 
 		/* reset the statistics */
 		memset(&entry->counters, 0, sizeof(Counters));
+		memset(&entry->good_plan, 0, sizeof(pgssPlan));
+		memset(&entry->bad_plan, 0, sizeof(pgssPlan));
 		/* set the appropriate initial usage count */
 		entry->counters.usage = sticky ? pgss->cur_median_usage : USAGE_INIT;
 		/* re-initialize the mutex each time ... we assume no one using it */
@@ -2104,6 +2554,18 @@ gc_qtexts(void)
 									  qbuffer,
 									  qbuffer_size);
 
+		int			good_plan_len = entry->good_plan.len;
+		char	   *good_plan = qtext_fetch(entry->good_plan.offset,
+									   good_plan_len,
+									   qbuffer,
+									   qbuffer_size);
+
+		int			bad_plan_len = entry->bad_plan.len;
+		char	   *bad_plan = qtext_fetch(entry->bad_plan.offset,
+									   bad_plan_len,
+									   qbuffer,
+									   qbuffer_size);
+
 		if (qry == NULL)
 		{
 			/* Trouble ... drop the text */
@@ -2125,6 +2587,53 @@ gc_qtexts(void)
 
 		entry->query_offset = extent;
 		extent += query_len + 1;
+
+		if (good_plan == NULL || good_plan_len <= 0)
+		{
+			/* There was an error while loading the good_plan or there was simply never a
+			 * good_plan recorded, so we make sure that this entry knows this. */
+			entry->good_plan.offset = 0;
+			entry->good_plan.len = -1;
+		}
+		else
+		{
+			/* Save the good plan */
+			if (fwrite(good_plan, 1, good_plan_len + 1, qfile) != good_plan_len + 1)
+			{
+				ereport(LOG,
+						(errcode_for_file_access(),
+					  errmsg("could not write pg_stat_statement file \"%s\": %m",
+							 PGSS_TEXT_FILE)));
+				hash_seq_term(&hash_seq);
+				goto gc_fail;
+			}
+			entry->good_plan.offset = extent;
+			extent += good_plan_len + 1;
+		}
+
+		if (bad_plan == NULL || bad_plan_len <= 0)
+		{
+			/* There was an error while loading the bad_plan or there was simply never a
+			 * bad_plan recorded, so we make sure that this entry knows this. */
+			entry->bad_plan.offset = 0;
+			entry->bad_plan.len = -1;
+		}
+		else
+		{
+			/* Save the bad plan */
+			if (fwrite(bad_plan, 1, bad_plan_len + 1, qfile) != bad_plan_len + 1)
+			{
+				ereport(LOG,
+						(errcode_for_file_access(),
+					  errmsg("could not write pg_stat_statement file \"%s\": %m",
+							 PGSS_TEXT_FILE)));
+				hash_seq_term(&hash_seq);
+				goto gc_fail;
+			}
+			entry->bad_plan.offset = extent;
+			extent += bad_plan_len + 1;
+		}
+
 		nentries++;
 	}
 
diff --git a/contrib/pg_stat_statements/pg_stat_statements.control b/contrib/pg_stat_statements/pg_stat_statements.control
index 193fcdf..617038b 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.control
+++ b/contrib/pg_stat_statements/pg_stat_statements.control
@@ -1,5 +1,5 @@
 # pg_stat_statements extension
 comment = 'track execution statistics of all SQL statements executed'
-default_version = '1.5'
+default_version = '1.6'
 module_pathname = '$libdir/pg_stat_statements'
 relocatable = true
diff --git a/contrib/pg_stat_statements/sql/pg_stat_statements.sql b/contrib/pg_stat_statements/sql/pg_stat_statements.sql
index a8361fd..9e02470 100644
--- a/contrib/pg_stat_statements/sql/pg_stat_statements.sql
+++ b/contrib/pg_stat_statements/sql/pg_stat_statements.sql
@@ -195,4 +195,15 @@ DROP FUNCTION PLUS_TWO(INTEGER);
 
 SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
 
+-- test to see if any plans have been recorded.
+SELECT
+  CASE WHEN good_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN bad_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN good_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END,
+  CASE WHEN good_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END
+FROM pg_stat_statements ORDER BY query COLLATE "C";
+
+-- test if there is some text in the recorded plans.
+select substr(good_plan, 0, 11), substr(bad_plan, 0, 11) from pg_stat_statements ORDER BY query COLLATE "C";
+
 DROP EXTENSION pg_stat_statements;
#19Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Julian Markwort (#18)
Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

Greetings,

On Thu, Mar 08, 2018 at 02:58:34PM +0100, Julian Markwort wrote:

I'd love to hear more feedback, here are two ideas to polish this
patch:
a) Right now, good_plan and bad_plan collection can be activated and
deactivated with separate GUCs. I think it would be sensible to
collect
either both or none. (This would result in fewer convoluted
conditionals.)
b) Would you like to be able to tune the percentiles yourself, to
adjust for the point at which a new plan is stored?

I'd like to review the patch and leave a feedback. I tested it with
different indexes on same table and with same queries and it works fine.

First of all, GUC variables and functions. How about union
'pg_stat_statements.good_plan_enable' and
'pg_stat_statements.bad_plan_enable' into
'pg_stat_statements.track_plan' with GUC_LIST_INPUT flag? It may accept
comma separated values 'good' and 'bad'. It lets to add another tracking
type in future without adding new variable.

In same manner pg_stat_statements_good_plan_reset() and
pg_stat_statements_bad_plan_reset() functions can be combined in
function:

pg_stat_statements_plan_reset(in queryid bigint, in good boolean, in bad
boolean)

Further comments on the code.

+-- test to see if any plans have been recorded.
+SELECT
+  CASE WHEN good_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN bad_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN good_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END,
+  CASE WHEN good_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END

I think here is a typo. Last case should be bad_plan_timestamp.

+			good_plan_str = palloc(1 * sizeof(char));
+			*good_plan_str = '\0';
...
+			bad_plan_str = palloc(1 * sizeof(char));
+			*bad_plan_str = '\0';

Here we can use empty string literals instead of pallocs. For example:

const char *good_plan_str;
const char *bad_plan_str;
...
good_plan_str = "";
...
bad_plan_str = "";

+ interquartile_dist = 2.0*(0.6745 * sqrt(e->counters.sum_var_time / e->counters.calls));

It is worth to check e->counters.calls for zero here. Because the entry
can be sticky.

--
Arthur Zakirov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company

#20Julian Markwort
julian.markwort@uni-muenster.de
In reply to: Arthur Zakirov (#19)
Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

Arthur Zakirov wrote on 2018-03-09:

I'd like to review the patch and leave a feedback. I tested it with
different indexes on same table and with same queries and it works fine.

Thanks for taking some time to review my patch, Arthur!

First of all, GUC variables and functions. How about union
'pg_stat_statements.good_plan_enable' and
'pg_stat_statements.bad_plan_enable' into
'pg_stat_statements.track_plan' with GUC_LIST_INPUT flag? It may accept
comma separated values 'good' and 'bad'. It lets to add another tracking
type in future without adding new variable.

This sounds like a good idea to me; Somebody already suggested that tracking an "average plan" would be interesting as well, however I don't have any clever ideas on how to identify such a plan.

In same manner pg_stat_statements_good_plan_reset() and
pg_stat_statements_bad_plan_reset() functions can be combined in
function:

pg_stat_statements_plan_reset(in queryid bigint, in good boolean, in bad
boolean)

This is also sensible, however if any more kinds of plans would be added in the future, there would be a lot of flags in this function. I think having varying amounts of flags (between extension versions) as arguments to the function also makes any upgrade paths difficult. Thinking more about this, we could also user function overloading to avoid this.
An alternative would be to have the caller pass the type of plan he wants to reset. Omitting the type would result in the deletion of all plans, maybe?
pg_stat_statements_plan_reset(in queryid bigint, in plan_type text)

I'm not sure if there are any better ways to do this?

Further comments on the code.

You're right about all of those, thanks!

#21Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Julian Markwort (#20)
Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

On Mon, Mar 12, 2018 at 02:11:42PM +0100, Julian Markwort wrote:

In same manner pg_stat_statements_good_plan_reset() and
pg_stat_statements_bad_plan_reset() functions can be combined in
function:

pg_stat_statements_plan_reset(in queryid bigint, in good boolean, in bad
boolean)

This is also sensible, however if any more kinds of plans would be added in the future, there would be a lot of flags in this function. I think having varying amounts of flags (between extension versions) as arguments to the function also makes any upgrade paths difficult. Thinking more about this, we could also user function overloading to avoid this.
An alternative would be to have the caller pass the type of plan he wants to reset. Omitting the type would result in the deletion of all plans, maybe?
pg_stat_statements_plan_reset(in queryid bigint, in plan_type text)

Yes, it is a good idea. But maybe instead of passing an empty string
into plans type we should overload the function with only queryid
argument. I think it is a common practice in PostgreSQL. Otherwise
allowance to pass empty string as plan type may confuse users. So
functions declaration may be the following:

pg_stat_statements_plan_reset(in queryid bigint, in plan_type text)
pg_stat_statements_plan_reset(in queryid bigint)

+ interquartile_dist = 2.0*(0.6745 * sqrt(e->counters.sum_var_time / e->counters.calls));

I think it would be better to have defines for 2.0 and 0.6745 values for
the sake of readability.

--
Arthur Zakirov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company

#22Julian Markwort
julian.markwort@uni-muenster.de
In reply to: Arthur Zakirov (#21)
1 attachment(s)
Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

To anyone who followed along with this for so long, I'd like to present my newest version of this patch.

As suggested by Arthur Zakirov, there is now only a single GUC ( pg_stat_statements.plan_track ) responsible for the selection of the plans that should be tracked. Possible values are: all, none, good, or bad.
I've mostly copied functionality from pl_handler.c . This resulted in the need to include varlena.h so I could use the SplitIdentifierString() function to parse the values, of which several (e.g. pg_stat_statements.plan_track='good, bad') could be used.

I've also added a new GUC:
pg_stat_statements.plan_fence_factor
This GUC can be used to scale the fences of the interval, outside of which a plan might be updated.
Right now, it is set to 1.5 (common factor for the definition of outliers in boxplots) and you can see through additional colums in the pg_stat_statements view, how often these fences are surpassed by execution times and how often the plans are updated. (The colums are: good_plan_outliers, good_plan_updates, bad_plan_outliers, bad_plan_updates and are primarily here for testing and review purposes and are not essential to this patch, they probably don't add any value for the average user)

Similarly to the first suggestion by Arthur, I've also changed the plan_reset functionality - there is now only one function, pg_stat_statements_plan_reset(queryid bigint), overloaded with (queryid bigint, plantype cstring) args, that can be used to remove both plans (when omitting the cstring) or either of them. The cstring argument accepts 'good' or 'bad'.

I also added more comments to the estimations of the quartiles and the calculation of the fences.

The performance impact lies now at 139312 vs 141841 tps, so roughly 1.78% slower than default pg_stat_statements.
The fact that these results are a little worse than the previous iteration is due to some changes in the definition of the fences which mistakenly calculated by adding the scaled interquartile distance to the mean, instead of adding it to the respective quartiles, which means that plan updates are triggered a little more often.
For 4259631 transactions however, only 11 updates for the bad plans where triggered.

I'm looking forward to your opinions!
Julian

Attachments:

pgss_plans_v04.patchtext/x-patchDownload
diff --git a/contrib/pg_stat_statements/Makefile b/contrib/pg_stat_statements/Makefile
index 39b368b70e..49bb462d10 100644
--- a/contrib/pg_stat_statements/Makefile
+++ b/contrib/pg_stat_statements/Makefile
@@ -4,7 +4,8 @@ MODULE_big = pg_stat_statements
 OBJS = pg_stat_statements.o $(WIN32RES)
 
 EXTENSION = pg_stat_statements
-DATA = pg_stat_statements--1.4.sql pg_stat_statements--1.4--1.5.sql \
+DATA = pg_stat_statements--1.4.sql \
+	pg_stat_statements--1.5--1.6.sql pg_stat_statements--1.4--1.5.sql \
 	pg_stat_statements--1.3--1.4.sql pg_stat_statements--1.2--1.3.sql \
 	pg_stat_statements--1.1--1.2.sql pg_stat_statements--1.0--1.1.sql \
 	pg_stat_statements--unpackaged--1.0.sql
diff --git a/contrib/pg_stat_statements/expected/pg_stat_statements.out b/contrib/pg_stat_statements/expected/pg_stat_statements.out
index 5318c3550c..2ca549686f 100644
--- a/contrib/pg_stat_statements/expected/pg_stat_statements.out
+++ b/contrib/pg_stat_statements/expected/pg_stat_statements.out
@@ -395,4 +395,40 @@ SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
  SELECT pg_stat_statements_reset()         |     1 |    1
 (8 rows)
 
+-- test to see if any plans have been recorded.
+SELECT
+  CASE WHEN good_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN bad_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN good_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END,
+  CASE WHEN bad_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END
+FROM pg_stat_statements ORDER BY query COLLATE "C";
+ case | case | case | case 
+------+------+------+------
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    1 |    1 |    1 |    1
+    1 |    1 |    1 |    1
+    1 |    1 |    1 |    1
+(9 rows)
+
+-- test if there is some text in the recorded plans.
+SELECT substr(good_plan, 0, 11), substr(bad_plan, 0, 11) FROM pg_stat_statements ORDER BY query COLLATE "C";
+   substr   |   substr   
+------------+------------
+            | 
+            | 
+            | 
+            | 
+            | 
+            | 
+ Query Text | Query Text
+ Query Text | Query Text
+ Query Text | Query Text
+ Query Text | Query Text
+(10 rows)
+
 DROP EXTENSION pg_stat_statements;
diff --git a/contrib/pg_stat_statements/pg_stat_statements--1.5--1.6.sql b/contrib/pg_stat_statements/pg_stat_statements--1.5--1.6.sql
new file mode 100644
index 0000000000..6c8f743ee5
--- /dev/null
+++ b/contrib/pg_stat_statements/pg_stat_statements--1.5--1.6.sql
@@ -0,0 +1,82 @@
+/* contrib/pg_stat_statements/pg_stat_statements--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "ALTER EXTENSION pg_stat_statements UPDATE TO '1.6'" to load this file. \quit
+
+/* First we have to remove them from the extension */
+ALTER EXTENSION pg_stat_statements DROP VIEW pg_stat_statements;
+ALTER EXTENSION pg_stat_statements DROP FUNCTION pg_stat_statements(boolean);
+ALTER EXTENSION pg_stat_statements DROP FUNCTION pg_stat_statements_reset();
+
+/* Then we can drop them */
+DROP VIEW pg_stat_statements;
+DROP FUNCTION pg_stat_statements(boolean);
+DROP FUNCTION pg_stat_statements_reset();
+
+-- Register functions.
+CREATE FUNCTION pg_stat_statements_reset()
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_statements_plan_reset(IN queryid bigint)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_statements_plan_reset(IN queryid bigint, IN plantype cstring)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_statements(IN showtext boolean,
+    OUT userid oid,
+    OUT dbid oid,
+    OUT queryid bigint,
+    OUT query text,
+    OUT good_plan text,
+    OUT bad_plan text,
+    OUT calls int8,
+    OUT total_time float8,
+    OUT min_time float8,
+    OUT max_time float8,
+    OUT mean_time float8,
+    OUT stddev_time float8,
+    OUT good_plan_time float8,
+    OUT good_plan_outliers int8,
+    OUT good_plan_updates int8,
+    OUT good_plan_timestamp timestamp,
+    OUT bad_plan_time float8,
+    OUT bad_plan_outliers int8,
+    OUT bad_plan_updates int8,
+    OUT bad_plan_timestamp timestamp,
+    OUT rows int8,
+    OUT shared_blks_hit int8,
+    OUT shared_blks_read int8,
+    OUT shared_blks_dirtied int8,
+    OUT shared_blks_written int8,
+    OUT local_blks_hit int8,
+    OUT local_blks_read int8,
+    OUT local_blks_dirtied int8,
+    OUT local_blks_written int8,
+    OUT temp_blks_read int8,
+    OUT temp_blks_written int8,
+    OUT blk_read_time float8,
+    OUT blk_write_time float8
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stat_statements_1_6'
+LANGUAGE C STRICT VOLATILE PARALLEL SAFE;
+
+-- Register a view on the function for ease of use.
+CREATE VIEW pg_stat_statements AS
+  SELECT * FROM pg_stat_statements(true);
+
+GRANT SELECT ON pg_stat_statements TO PUBLIC;
+
+-- Don't want this to be available to non-superusers.
+REVOKE ALL ON FUNCTION pg_stat_statements_reset() FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_stat_statements_plan_reset(bigint) FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_stat_statements_plan_reset(bigint, cstring) FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_stat_statements_reset() TO pg_read_all_stats;
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 928673498a..c1a6fc7b7c 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -63,6 +63,7 @@
 
 #include "access/hash.h"
 #include "catalog/pg_authid.h"
+#include "commands/explain.h"
 #include "executor/instrument.h"
 #include "funcapi.h"
 #include "mb/pg_wchar.h"
@@ -78,6 +79,8 @@
 #include "tcop/utility.h"
 #include "utils/builtins.h"
 #include "utils/memutils.h"
+#include "utils/timestamp.h"
+#include "utils/varlena.h"
 
 PG_MODULE_MAGIC;
 
@@ -119,7 +122,8 @@ typedef enum pgssVersion
 	PGSS_V1_0 = 0,
 	PGSS_V1_1,
 	PGSS_V1_2,
-	PGSS_V1_3
+	PGSS_V1_3,
+	PGSS_V1_6
 } pgssVersion;
 
 /*
@@ -165,6 +169,16 @@ typedef struct Counters
 	double		usage;			/* usage factor */
 } Counters;
 
+typedef struct pgssPlan
+{
+	Size	offset;
+	int		len;
+	double		time;	/* execution time in msec when the latest plan was updated */
+	TimestampTz timestamp;
+	int64 outliers;
+	int64 updates;
+} pgssPlan;
+
 /*
  * Statistics per statement
  *
@@ -178,6 +192,8 @@ typedef struct pgssEntry
 	Counters	counters;		/* the statistics for this query */
 	Size		query_offset;	/* query text offset in external file */
 	int			query_len;		/* # of valid bytes in query string, or -1 */
+	pgssPlan	good_plan;
+	pgssPlan	bad_plan;
 	int			encoding;		/* query text encoding */
 	slock_t		mutex;			/* protects the counters only */
 } pgssEntry;
@@ -265,11 +281,26 @@ static const struct config_enum_entry track_options[] =
 	{NULL, 0, false}
 };
 
+static const struct config_enum_entry format_options[] =
+{
+	{"text", EXPLAIN_FORMAT_TEXT, false},
+	{"xml", EXPLAIN_FORMAT_XML, false},
+	{"json", EXPLAIN_FORMAT_JSON, false},
+	{"yaml", EXPLAIN_FORMAT_YAML, false},
+	{NULL, 0, false}
+};
+
 static int	pgss_max;			/* max # statements to track */
 static int	pgss_track;			/* tracking level */
 static bool pgss_track_utility; /* whether to track utility commands */
 static bool pgss_save;			/* whether to save stats across shutdown */
-
+static bool pgss_good_plan_enable; 	/* whether to save good_plans */
+static bool pgss_bad_plan_enable; 	/* whether to save bad_plans */
+static bool pgss_good_plan_enable_temp; /* used to store state between plan_track_check and _assign */
+static bool pgss_bad_plan_enable_temp; 	/* used to store state between plan_track_check and _assign */
+static int	pgss_plan_format;   	/* id which sets the output format */
+static double pgss_plan_fence_factor; /* can be used to scale the fences for plan updates */
+static char *pgss_plan_track_str; /* String holding the arguments provided to the pgss_plan_track GUC */
 
 #define pgss_enabled() \
 	(pgss_track == PGSS_TRACK_ALL || \
@@ -291,7 +322,9 @@ void		_PG_fini(void);
 PG_FUNCTION_INFO_V1(pg_stat_statements_reset);
 PG_FUNCTION_INFO_V1(pg_stat_statements_1_2);
 PG_FUNCTION_INFO_V1(pg_stat_statements_1_3);
+PG_FUNCTION_INFO_V1(pg_stat_statements_1_6);
 PG_FUNCTION_INFO_V1(pg_stat_statements);
+PG_FUNCTION_INFO_V1(pg_stat_statements_plan_reset);
 
 static void pgss_shmem_startup(void);
 static void pgss_shmem_shutdown(int code, Datum arg);
@@ -309,12 +342,17 @@ static void pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
 static uint64 pgss_hash_string(const char *str, int len);
 static void pgss_store(const char *query, uint64 queryId,
 		   int query_location, int query_len,
+		   QueryDesc *queryDesc,
 		   double total_time, uint64 rows,
 		   const BufferUsage *bufusage,
 		   pgssJumbleState *jstate);
+Datum pg_stat_statements_plan_reset(PG_FUNCTION_ARGS);
 static void pg_stat_statements_internal(FunctionCallInfo fcinfo,
 							pgssVersion api_version,
 							bool showtext);
+static int fill_plan_times(Datum values[], bool nulls[], int i, pgssPlan *plan);
+static int fill_plan_str(Datum values[], bool nulls[], int i, pgssPlan *plan,
+							pgssEntry *entry, char *qbuffer, Size qbuffer_size);
 static Size pgss_memsize(void);
 static pgssEntry *entry_alloc(pgssHashKey *key, Size query_offset, int query_len,
 			int encoding, bool sticky);
@@ -338,7 +376,8 @@ static char *generate_normalized_query(pgssJumbleState *jstate, const char *quer
 static void fill_in_constant_lengths(pgssJumbleState *jstate, const char *query,
 						 int query_loc);
 static int	comp_location(const void *a, const void *b);
-
+static bool plan_track_check_hook(char **newvalue, void **extra, GucSource source);
+static void plan_track_assign_hook(const char *newvalue, void *extra);
 
 /*
  * Module load callback
@@ -407,6 +446,42 @@ _PG_init(void)
 							 NULL,
 							 NULL);
 
+	DefineCustomStringVariable("pg_stat_statements.plan_track",
+							   "Enable plan tracking",
+							   "Specify the types of plans that shall be tracked: all, none, good, or bad. Support for multiple individual types (curently only good and bad) is available.",
+							   &pgss_plan_track_str,
+							   "all",
+							   PGC_SUSET,
+							   GUC_LIST_INPUT,
+							   plan_track_check_hook,
+							   plan_track_assign_hook,
+							   NULL);
+
+	DefineCustomEnumVariable("pg_stat_statements.plan_format",
+							 "Sets the output format for the plans.",
+							 "Notice that the plan format can not be changed after a plan is saved. Valid values are text, json, xml and yaml.",
+							 &pgss_plan_format,
+							 EXPLAIN_FORMAT_TEXT,
+							 format_options,
+							 PGC_SUSET,
+							 0,
+							 NULL,
+							 NULL,
+							 NULL);
+
+	DefineCustomRealVariable("pg_stat_statements.plan_fence_factor",
+							 "Sets the factor that scales the fences that are used to determine whether a plan should be updated.",
+							 NULL,
+							 &pgss_plan_fence_factor,
+							 1.5,
+							 0.0,
+							 +INFINITY,
+							 PGC_SUSET,
+							 0,
+							 NULL,
+							 NULL,
+							 NULL);
+
 	EmitWarningsOnPlaceholders("pg_stat_statements");
 
 	/*
@@ -470,7 +545,11 @@ pgss_shmem_startup(void)
 	int32		pgver;
 	int32		i;
 	int			buffer_size;
+	int			good_plan_buffer_size;
+	int			bad_plan_buffer_size;
 	char	   *buffer = NULL;
+	char	   *good_plan_buffer = NULL;
+	char	   *bad_plan_buffer = NULL;
 
 	if (prev_shmem_startup_hook)
 		prev_shmem_startup_hook();
@@ -561,7 +640,11 @@ pgss_shmem_startup(void)
 	}
 
 	buffer_size = 2048;
+	good_plan_buffer_size = 2048;
+	bad_plan_buffer_size = 2048;
 	buffer = (char *) palloc(buffer_size);
+	good_plan_buffer = (char *) palloc(good_plan_buffer_size);
+	bad_plan_buffer = (char *) palloc(bad_plan_buffer_size);
 
 	if (fread(&header, sizeof(uint32), 1, file) != 1 ||
 		fread(&pgver, sizeof(uint32), 1, file) != 1 ||
@@ -595,8 +678,34 @@ pgss_shmem_startup(void)
 		if (fread(buffer, 1, temp.query_len + 1, file) != temp.query_len + 1)
 			goto read_error;
 
+		/* Resize good plan_buffer and read into it */
+		if (temp.good_plan.len >= good_plan_buffer_size)
+		{
+			good_plan_buffer_size =
+				Max(good_plan_buffer_size * 2, temp.good_plan.len + 1);
+			good_plan_buffer= repalloc(good_plan_buffer, good_plan_buffer_size);
+		}
+
+		if (fread(good_plan_buffer, 1, temp.good_plan.len + 1, file)
+				!= temp.good_plan.len + 1)
+			goto read_error;
+
+		/* Resize bad plan_buffer and read into it */
+		if (temp.bad_plan.len >= bad_plan_buffer_size)
+		{
+			bad_plan_buffer_size =
+				Max(bad_plan_buffer_size * 2, temp.bad_plan.len + 1);
+			bad_plan_buffer= repalloc(bad_plan_buffer, bad_plan_buffer_size);
+		}
+
+		if (fread(bad_plan_buffer, 1, temp.bad_plan.len + 1, file)
+				!= temp.bad_plan.len + 1)
+			goto read_error;
+
 		/* Should have a trailing null, but let's make sure */
 		buffer[temp.query_len] = '\0';
+		good_plan_buffer[temp.good_plan.len] = '\0';
+		bad_plan_buffer[temp.bad_plan.len] = '\0';
 
 		/* Skip loading "sticky" entries */
 		if (temp.counters.calls == 0)
@@ -608,6 +717,20 @@ pgss_shmem_startup(void)
 			goto write_error;
 		pgss->extent += temp.query_len + 1;
 
+		/* Store the good plan text*/
+		temp.good_plan.offset = pgss->extent;
+		if (fwrite(good_plan_buffer, 1, temp.good_plan.len + 1, qfile)
+				!= temp.good_plan.len + 1)
+			goto write_error;
+		pgss->extent += temp.good_plan.len + 1;
+
+		/* Store the bad plan text*/
+		temp.bad_plan.offset = pgss->extent;
+		if (fwrite(bad_plan_buffer, 1, temp.bad_plan.len + 1, qfile)
+				!= temp.bad_plan.len + 1)
+			goto write_error;
+		pgss->extent += temp.bad_plan.len + 1;
+
 		/* make the hashtable entry (discards old entries if too many) */
 		entry = entry_alloc(&temp.key, query_offset, temp.query_len,
 							temp.encoding,
@@ -615,9 +738,14 @@ pgss_shmem_startup(void)
 
 		/* copy in the actual stats */
 		entry->counters = temp.counters;
+				/* copy in the plans */
+		entry->good_plan = temp.good_plan;
+		entry->bad_plan = temp.bad_plan;
 	}
 
 	pfree(buffer);
+	pfree(good_plan_buffer);
+	pfree(bad_plan_buffer);
 	FreeFile(file);
 	FreeFile(qfile);
 
@@ -658,6 +786,10 @@ write_error:
 fail:
 	if (buffer)
 		pfree(buffer);
+	if (good_plan_buffer)
+		pfree(good_plan_buffer);
+	if (bad_plan_buffer)
+		pfree(bad_plan_buffer);
 	if (file)
 		FreeFile(file);
 	if (qfile)
@@ -723,14 +855,52 @@ pgss_shmem_shutdown(int code, Datum arg)
 	while ((entry = hash_seq_search(&hash_seq)) != NULL)
 	{
 		int			len = entry->query_len;
+		int			good_plan_len = entry->good_plan.len;
+		int			bad_plan_len = entry->bad_plan.len;
 		char	   *qstr = qtext_fetch(entry->query_offset, len,
 									   qbuffer, qbuffer_size);
+		char	   *good_plan_str;
+		char	   *bad_plan_str;
+		if (good_plan_len > 0)
+		{
+			/* A good plan is available */
+			good_plan_str = qtext_fetch(entry->good_plan.offset, good_plan_len,
+												qbuffer, qbuffer_size);
+		}
+		else
+		{
+			/*
+			 * There is no good plan available. This can be caused by a query
+			 * which has no plan (for example insert or delete queries) or
+			 * because good plans are disabled. In this case we have to store
+			 * an empty string instead of null.
+			 */
+			good_plan_str = "";
+		}
+		if (bad_plan_len > 0)
+		{
+			/* A bad plan is available */
+			bad_plan_str = qtext_fetch(entry->bad_plan.offset, bad_plan_len,
+												qbuffer, qbuffer_size);
+		}
+		else
+		{
+			/*
+			 * There is no bad plan available. This can be caused by a query
+			 * which has no plan (for example insert or delete queries) or
+			 * because bad plans are disabled. In this case we have to store
+			 * an empty string instead of null.
+			 */
+			bad_plan_str = "";
+		}
 
 		if (qstr == NULL)
 			continue;			/* Ignore any entries with bogus texts */
-
+		/* Write entries, queries and plans serialized to the dump file */
 		if (fwrite(entry, sizeof(pgssEntry), 1, file) != 1 ||
-			fwrite(qstr, 1, len + 1, file) != len + 1)
+			fwrite(qstr, 1, len + 1, file) != len + 1 ||
+			fwrite(good_plan_str, 1, good_plan_len + 1, file) != good_plan_len + 1 ||
+			fwrite(bad_plan_str, 1, bad_plan_len + 1, file) != bad_plan_len + 1)
 		{
 			/* note: we assume hash_seq_term won't change errno */
 			hash_seq_term(&hash_seq);
@@ -835,6 +1005,7 @@ pgss_post_parse_analyze(ParseState *pstate, Query *query)
 				   query->queryId,
 				   query->stmt_location,
 				   query->stmt_len,
+				   NULL,
 				   0,
 				   0,
 				   NULL,
@@ -942,6 +1113,7 @@ pgss_ExecutorEnd(QueryDesc *queryDesc)
 				   queryId,
 				   queryDesc->plannedstmt->stmt_location,
 				   queryDesc->plannedstmt->stmt_len,
+				   queryDesc,								/* queryDesc itself is needed to retrieve the plan */
 				   queryDesc->totaltime->total * 1000.0,	/* convert to msec */
 				   queryDesc->estate->es_processed,
 				   &queryDesc->totaltime->bufusage,
@@ -1053,6 +1225,7 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
 				   0,			/* signal that it's a utility stmt */
 				   pstmt->stmt_location,
 				   pstmt->stmt_len,
+				   NULL,
 				   INSTR_TIME_GET_MILLISEC(duration),
 				   rows,
 				   &bufusage,
@@ -1083,6 +1256,26 @@ pgss_hash_string(const char *str, int len)
 											len, 0));
 }
 
+/*
+ * Update a plan in the text file.
+ * We can re-use qtext_store to save the plan.
+ */
+static void
+update_plan(volatile pgssPlan* pgssPlan_ptr, const char* plan_str, int plan_len,
+			double total_time)
+{
+	if (plan_len > 0)
+	{
+		Size plan_offset;
+		qtext_store(plan_str, plan_len, &plan_offset, NULL);
+		pgssPlan_ptr->offset	= plan_offset;
+		pgssPlan_ptr->len		= plan_len;
+		pgssPlan_ptr->time		= total_time;
+		pgssPlan_ptr->timestamp = GetCurrentTimestamp();
+	}
+	return;
+}
+
 /*
  * Store some statistics for a statement.
  *
@@ -1096,6 +1289,7 @@ pgss_hash_string(const char *str, int len)
 static void
 pgss_store(const char *query, uint64 queryId,
 		   int query_location, int query_len,
+		   QueryDesc *queryDesc,
 		   double total_time, uint64 rows,
 		   const BufferUsage *bufusage,
 		   pgssJumbleState *jstate)
@@ -1134,6 +1328,7 @@ pgss_store(const char *query, uint64 queryId,
 		query_len = strlen(query);
 	}
 
+
 	/*
 	 * Discard leading and trailing whitespace, too.  Use scanner_isspace()
 	 * not libc's isspace(), because we want to match the lexer's behavior.
@@ -1231,9 +1426,136 @@ pgss_store(const char *query, uint64 queryId,
 		 * locking rules at the head of the file)
 		 */
 		volatile pgssEntry *e = (volatile pgssEntry *) entry;
+		bool	good_plan_needs_updating = false;
+		bool	bad_plan_needs_updating = false;
+		int		plan_len = 0;
+		char*	plan;
 
 		SpinLockAcquire(&e->mutex);
 
+		if (queryDesc && (pgss_good_plan_enable || pgss_bad_plan_enable))
+		{
+			/* Avoid division by zero errors. */
+			if (e->counters.calls > 0)
+			{
+				/*
+				 * The quantiles for a normal distributions can be estimated through
+				 * X = expected value + standard deviation * Z-score
+				 * For the 25% and 75% quantiles (=lower and upper quartile),
+				 * the z-scores -0.6754 and 0.6745 are used.
+				 */
+				double z_score = 0.6745;
+				/*
+				 * Since we're already making a bunch of statistical assumptions,
+				 * let's use the fact that the expected value can be approximated
+				 * by the mean of a dataset (see "law of large numbers").
+				 */
+
+				double std_dev = sqrt(e->counters.sum_var_time / e->counters.calls);
+				double exp_val = e->counters.mean_time;
+
+				double lower_quartile = exp_val - std_dev * z_score;
+				double upper_quartile = exp_val + std_dev * z_score;
+
+				double interquartile_dist = upper_quartile - lower_quartile;
+
+				/*
+				 * We calculate a set of inner and outer fences, similar to the
+				 * definition of whiskers in boxplots.
+				 * By default, pgss_plan_fence_factor is set to 1.5 .
+				 */
+				double lower_fence = lower_quartile - pgss_plan_fence_factor * interquartile_dist;
+				double upper_fence = upper_quartile + pgss_plan_fence_factor * interquartile_dist;
+
+				if (pgss_good_plan_enable)
+				{
+					/* 
+					 * Only update bad_plan if execution time was slower than the lower fence.
+					 * The use of statistics to estimate wether the change in execution time
+					 * was big enough to warrant a plan update is debatable, however it is
+					 * probably the fastest way to identify changes in the planner
+					 * without actually comparing the plans step by step.
+					 */
+					if (total_time < lower_fence)
+					{
+						e->good_plan.outliers++;
+						/* Only update plan if execution time was slower than that of the old plan*/
+						if (total_time < e->good_plan.time)
+						{
+							good_plan_needs_updating = true;
+							e->good_plan.updates++;
+						}
+					}
+				}
+
+				if (pgss_bad_plan_enable)
+				{
+					/* Only update good_plan if execution time was faster than the upperer fence.*/
+					if (total_time > upper_fence)
+					{
+						e->bad_plan.outliers++;
+						if (total_time > e->bad_plan.time)
+						{
+							bad_plan_needs_updating = true;
+							e->bad_plan.updates++;
+						}
+					}
+				}
+			}
+
+			/*
+			 * Plans should be updated whenever no previous plan was stored.
+			 */
+			if (pgss_good_plan_enable && e->good_plan.time==0)
+				good_plan_needs_updating = true;
+			if (pgss_bad_plan_enable && e->bad_plan.time==0)
+				bad_plan_needs_updating = true;
+
+			if (good_plan_needs_updating || bad_plan_needs_updating)
+			{
+				/* *es is used to retrieve the plan associated with the statement*/
+				ExplainState *es = NewExplainState();
+
+				es->analyze = (queryDesc->instrument_options && false);
+				es->verbose = false;
+				es->buffers = (es->analyze && false);
+				es->timing = (es->analyze && true);
+				es->format = pgss_plan_format;
+
+				ExplainBeginOutput(es);
+				ExplainQueryText(es, queryDesc);
+				ExplainPrintPlan(es, queryDesc);
+				ExplainEndOutput(es);
+
+				/* Remove last line break */
+				if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+					es->str->data[--es->str->len] = '\0';
+
+				/* Fix JSON to output an object */
+				if (pgss_plan_format == EXPLAIN_FORMAT_JSON)
+				{
+					es->str->data[0] = '{';
+					es->str->data[es->str->len - 1] = '}';
+				}
+
+				plan = es->str->data;
+				plan_len = strlen(plan);
+
+				if (good_plan_needs_updating)
+				{
+					update_plan(&e->good_plan, plan, plan_len, total_time);
+				}
+
+				if (bad_plan_needs_updating)
+				{
+					update_plan(&e->bad_plan, plan, plan_len, total_time);
+				}
+
+				pfree(es->str->data);
+				pfree(es);
+			}
+		}
+
 		/* "Unstick" entry if it was previously sticky */
 		if (e->counters.calls == 0)
 			e->counters.usage = USAGE_INIT;
@@ -1305,12 +1627,76 @@ pg_stat_statements_reset(PG_FUNCTION_ARGS)
 	PG_RETURN_VOID();
 }
 
+Datum
+pg_stat_statements_plan_reset(PG_FUNCTION_ARGS)
+{
+	pgssEntry   *entry;
+	pgssHashKey	key;
+
+	uint64 query_id = PG_GETARG_INT64(0);
+
+	bool good_plan_reset = false;
+	bool bad_plan_reset = false;
+
+	if (PG_NARGS() == 2)
+	{
+		char *plan_type_str = PG_GETARG_CSTRING(1);
+		if (pg_strcasecmp(plan_type_str, "good") == 0)
+			good_plan_reset = true;
+		else if (pg_strcasecmp(plan_type_str, "bad") == 0)
+			bad_plan_reset = true;
+		else
+			elog(LOG, "If a second argument is provided to pg_stat_statements_plan_reset, it must be of a valid plan type ('good' or 'bad').");
+	}
+	else
+	{
+		good_plan_reset = true;
+		bad_plan_reset = true;
+	}
+
+	if (!pgss || !pgss_hash)
+		ereport(ERROR,
+				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+				 errmsg("pg_stat_statements must be loaded via shared_preload_libraries")));
+
+	/* Set up key for hashtable search */
+	key.userid = GetUserId();
+	key.dbid = MyDatabaseId;
+	key.queryid = query_id;
+
+	/* Lookup the hash table entry with shared lock. */
+	LWLockAcquire(pgss->lock, LW_SHARED);
+
+	entry = (pgssEntry *) hash_search(pgss_hash, &key, HASH_FIND, NULL);
+
+	if (entry)
+	{
+		if (good_plan_reset)
+		{
+			entry->good_plan.offset = -1;
+			entry->good_plan.len = 0;
+			entry->good_plan.time = 0;
+			entry->good_plan.timestamp = 0;
+		}
+		if (bad_plan_reset)
+		{
+			entry->bad_plan.offset = -1;
+			entry->bad_plan.len = 0;
+			entry->bad_plan.time = 0;
+			entry->bad_plan.timestamp = 0;
+		}
+	}
+	LWLockRelease(pgss->lock);
+	PG_RETURN_VOID();
+}
+
 /* Number of output arguments (columns) for various API versions */
 #define PG_STAT_STATEMENTS_COLS_V1_0	14
 #define PG_STAT_STATEMENTS_COLS_V1_1	18
 #define PG_STAT_STATEMENTS_COLS_V1_2	19
 #define PG_STAT_STATEMENTS_COLS_V1_3	23
-#define PG_STAT_STATEMENTS_COLS			23	/* maximum of above */
+#define PG_STAT_STATEMENTS_COLS_V1_6	33
+#define PG_STAT_STATEMENTS_COLS			33	/* maximum of above */
 
 /*
  * Retrieve statement statistics.
@@ -1322,6 +1708,16 @@ pg_stat_statements_reset(PG_FUNCTION_ARGS)
  * expected API version is identified by embedding it in the C name of the
  * function.  Unfortunately we weren't bright enough to do that for 1.1.
  */
+Datum
+pg_stat_statements_1_6(PG_FUNCTION_ARGS)
+{
+ bool		showtext = PG_GETARG_BOOL(0);
+
+ pg_stat_statements_internal(fcinfo, PGSS_V1_6, showtext);
+
+ return (Datum) 0;
+}
+
 Datum
 pg_stat_statements_1_3(PG_FUNCTION_ARGS)
 {
@@ -1428,6 +1824,10 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 			if (api_version != PGSS_V1_3)
 				elog(ERROR, "incorrect number of output arguments");
 			break;
+		case PG_STAT_STATEMENTS_COLS_V1_6:
+			if (api_version != PGSS_V1_6)
+				elog(ERROR, "incorrect number of output arguments");
+			break;
 		default:
 			elog(ERROR, "incorrect number of output arguments");
 	}
@@ -1511,6 +1911,8 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 		bool		nulls[PG_STAT_STATEMENTS_COLS];
 		int			i = 0;
 		Counters	tmp;
+		pgssPlan 	tmp_good_plan;
+		pgssPlan 	tmp_bad_plan;
 		double		stddev;
 		int64		queryid = entry->key.queryid;
 
@@ -1550,6 +1952,18 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 					/* Just return a null if we fail to find the text */
 					nulls[i++] = true;
 				}
+
+				if (api_version >= PGSS_V1_6)
+				{
+					/*
+					 * This version requires to print out the good
+					 * and the bad plan if they are enabled.
+					 */
+					i = fill_plan_str(values, nulls, i, &entry->good_plan,
+										entry, qbuffer, qbuffer_size);
+					i = fill_plan_str(values, nulls, i, &entry->bad_plan,
+										entry, qbuffer, qbuffer_size);
+				}
 			}
 			else
 			{
@@ -1579,6 +1993,8 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 
 			SpinLockAcquire(&e->mutex);
 			tmp = e->counters;
+			tmp_good_plan = e->good_plan;
+			tmp_bad_plan = e->bad_plan;
 			SpinLockRelease(&e->mutex);
 		}
 
@@ -1606,6 +2022,14 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 				stddev = 0.0;
 			values[i++] = Float8GetDatumFast(stddev);
 		}
+
+		if (api_version >= PGSS_V1_6)
+		{
+			/* Set plan times and timestamps */
+			i = fill_plan_times(values, nulls, i, &tmp_good_plan);
+			i = fill_plan_times(values, nulls, i, &tmp_bad_plan);
+		}
+
 		values[i++] = Int64GetDatumFast(tmp.rows);
 		values[i++] = Int64GetDatumFast(tmp.shared_blks_hit);
 		values[i++] = Int64GetDatumFast(tmp.shared_blks_read);
@@ -1629,6 +2053,7 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 					 api_version == PGSS_V1_1 ? PG_STAT_STATEMENTS_COLS_V1_1 :
 					 api_version == PGSS_V1_2 ? PG_STAT_STATEMENTS_COLS_V1_2 :
 					 api_version == PGSS_V1_3 ? PG_STAT_STATEMENTS_COLS_V1_3 :
+					 api_version == PGSS_V1_6 ? PG_STAT_STATEMENTS_COLS_V1_6 :
 					 -1 /* fail if you forget to update this assert */ ));
 
 		tuplestore_putvalues(tupstore, tupdesc, values, nulls);
@@ -1643,6 +2068,54 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 	tuplestore_donestoring(tupstore);
 }
 
+/* Fill the plan time and timestamp into the values array. */
+static int
+fill_plan_times(Datum values[], bool nulls[], int i, pgssPlan *plan)
+{
+	values[i++] = Float8GetDatumFast(plan->time);
+	values[i++] = Int64GetDatumFast(plan->outliers);
+	values[i++] = Int64GetDatumFast(plan->updates);
+
+	/* if there is noting in the timestamp field, we are not interested in it */
+	if (plan->timestamp)
+		values[i++] = TimestampTzGetDatum(plan->timestamp);
+	else
+		nulls[i++] = true;
+	return i;
+}
+
+/* Fill the plan string into the values array. */
+static int
+fill_plan_str(Datum values[], bool nulls[], int i, pgssPlan *plan,
+				pgssEntry *entry, char *qbuffer, Size qbuffer_size)
+{
+	if (plan && plan->len > 0)
+	{
+		char *pstr = qtext_fetch(plan->offset, plan->len, qbuffer, qbuffer_size);
+		if (pstr)
+		{
+			char *enc;
+			enc = pg_any_to_server(pstr, plan->len, entry->encoding);
+			values[i++] = CStringGetTextDatum(enc);
+
+			if (enc != pstr)
+				pfree(enc);
+		}
+		else
+		{
+			/* failed to get the string of the plan */
+			nulls[i++] = true;
+		}
+	}
+	else
+	{
+		/* no plan available or plan_len not greater than 0 */
+		nulls[i++] = true;
+	}
+	return i;
+}
+
+
 /*
  * Estimate shared memory space needed.
  */
@@ -1694,6 +2167,8 @@ entry_alloc(pgssHashKey *key, Size query_offset, int query_len, int encoding,
 
 		/* reset the statistics */
 		memset(&entry->counters, 0, sizeof(Counters));
+		memset(&entry->good_plan, 0, sizeof(pgssPlan));
+		memset(&entry->bad_plan, 0, sizeof(pgssPlan));
 		/* set the appropriate initial usage count */
 		entry->counters.usage = sticky ? pgss->cur_median_usage : USAGE_INIT;
 		/* re-initialize the mutex each time ... we assume no one using it */
@@ -2104,6 +2579,18 @@ gc_qtexts(void)
 									  qbuffer,
 									  qbuffer_size);
 
+		int			good_plan_len = entry->good_plan.len;
+		char	   *good_plan = qtext_fetch(entry->good_plan.offset,
+									   good_plan_len,
+									   qbuffer,
+									   qbuffer_size);
+
+		int			bad_plan_len = entry->bad_plan.len;
+		char	   *bad_plan = qtext_fetch(entry->bad_plan.offset,
+									   bad_plan_len,
+									   qbuffer,
+									   qbuffer_size);
+
 		if (qry == NULL)
 		{
 			/* Trouble ... drop the text */
@@ -2125,6 +2612,53 @@ gc_qtexts(void)
 
 		entry->query_offset = extent;
 		extent += query_len + 1;
+
+		if (good_plan == NULL || good_plan_len <= 0)
+		{
+			/* There was an error while loading the good_plan or there was simply never a
+			 * good_plan recorded, so we make sure that this entry knows this. */
+			entry->good_plan.offset = 0;
+			entry->good_plan.len = -1;
+		}
+		else
+		{
+			/* Save the good plan */
+			if (fwrite(good_plan, 1, good_plan_len + 1, qfile) != good_plan_len + 1)
+			{
+				ereport(LOG,
+						(errcode_for_file_access(),
+					  errmsg("could not write pg_stat_statement file \"%s\": %m",
+							 PGSS_TEXT_FILE)));
+				hash_seq_term(&hash_seq);
+				goto gc_fail;
+			}
+			entry->good_plan.offset = extent;
+			extent += good_plan_len + 1;
+		}
+
+		if (bad_plan == NULL || bad_plan_len <= 0)
+		{
+			/* There was an error while loading the bad_plan or there was simply never a
+			 * bad_plan recorded, so we make sure that this entry knows this. */
+			entry->bad_plan.offset = 0;
+			entry->bad_plan.len = -1;
+		}
+		else
+		{
+			/* Save the bad plan */
+			if (fwrite(bad_plan, 1, bad_plan_len + 1, qfile) != bad_plan_len + 1)
+			{
+				ereport(LOG,
+						(errcode_for_file_access(),
+					  errmsg("could not write pg_stat_statement file \"%s\": %m",
+							 PGSS_TEXT_FILE)));
+				hash_seq_term(&hash_seq);
+				goto gc_fail;
+			}
+			entry->bad_plan.offset = extent;
+			extent += bad_plan_len + 1;
+		}
+
 		nentries++;
 	}
 
@@ -3163,3 +3697,81 @@ comp_location(const void *a, const void *b)
 	else
 		return 0;
 }
+
+/*
+ * Check the list of plans that shall be tracked and set the
+ * appropriate temporary bools.
+ */
+static bool
+plan_track_check_hook(char **newvalue, void **extra, GucSource source)
+{
+	char	   *rawstring;
+	List	   *elemlist;
+	ListCell   *l;
+
+	/* Set all temp bools to false */
+	pgss_good_plan_enable_temp = false;
+	pgss_bad_plan_enable_temp = false;
+
+	if (pg_strcasecmp(*newvalue, "all") == 0)
+	{
+		pgss_good_plan_enable_temp = true;
+		pgss_bad_plan_enable_temp = true;
+	}
+	else if (pg_strcasecmp(*newvalue, "none") == 0)
+		return true;
+	else
+	{
+		/* Need a modifiable copy of string */
+		rawstring = pstrdup(*newvalue);
+
+		/* Parse string into list of identifiers */
+		if (!SplitIdentifierString(rawstring, ',', &elemlist))
+		{
+			/* syntax error in list */
+			GUC_check_errdetail("List syntax is invalid.");
+			pfree(rawstring);
+			list_free(elemlist);
+			return false;
+		}
+
+		foreach(l, elemlist)
+		{
+			char	   *tok = (char *) lfirst(l);
+
+			if (pg_strcasecmp(tok, "good") == 0)
+				pgss_good_plan_enable_temp = true;
+			else if (pg_strcasecmp(tok, "bad") == 0)
+				pgss_bad_plan_enable_temp = true;
+			else if (pg_strcasecmp(tok, "all") == 0 || pg_strcasecmp(tok, "none") == 0)
+			{
+				GUC_check_errdetail("Key word \"%s\" cannot be combined with other key words.", tok);
+				pfree(rawstring);
+				list_free(elemlist);
+				return false;
+			}
+			else
+			{
+				GUC_check_errdetail("Unrecognized key word: \"%s\".", tok);
+				pfree(rawstring);
+				list_free(elemlist);
+				return false;
+			}
+		}
+		pfree(rawstring);
+		list_free(elemlist);
+	}
+
+	return true;
+}
+
+/*
+ * The bools enabling and disabling tracking of certain plans
+ * are set to the values previously set in the temporary variables.
+ */
+static void
+plan_track_assign_hook(const char *newvalue, void *extra)
+{
+	pgss_good_plan_enable = pgss_good_plan_enable_temp;
+	pgss_bad_plan_enable = pgss_bad_plan_enable_temp;
+}
\ No newline at end of file
diff --git a/contrib/pg_stat_statements/pg_stat_statements.control b/contrib/pg_stat_statements/pg_stat_statements.control
index 193fcdfafa..617038b4c0 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.control
+++ b/contrib/pg_stat_statements/pg_stat_statements.control
@@ -1,5 +1,5 @@
 # pg_stat_statements extension
 comment = 'track execution statistics of all SQL statements executed'
-default_version = '1.5'
+default_version = '1.6'
 module_pathname = '$libdir/pg_stat_statements'
 relocatable = true
diff --git a/contrib/pg_stat_statements/sql/pg_stat_statements.sql b/contrib/pg_stat_statements/sql/pg_stat_statements.sql
index a8361fd1bf..4eb49f84c6 100644
--- a/contrib/pg_stat_statements/sql/pg_stat_statements.sql
+++ b/contrib/pg_stat_statements/sql/pg_stat_statements.sql
@@ -195,4 +195,15 @@ DROP FUNCTION PLUS_TWO(INTEGER);
 
 SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
 
+-- test to see if any plans have been recorded.
+SELECT
+  CASE WHEN good_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN bad_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN good_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END,
+  CASE WHEN bad_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END
+FROM pg_stat_statements ORDER BY query COLLATE "C";
+
+-- test if there is some text in the recorded plans.
+SELECT substr(good_plan, 0, 11), substr(bad_plan, 0, 11) FROM pg_stat_statements ORDER BY query COLLATE "C";
+
 DROP EXTENSION pg_stat_statements;
#23Julian Markwort
julian.markwort@uni-muenster.de
In reply to: Julian Markwort (#22)
1 attachment(s)
Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

I just realized I made a whitespace error when I modified the comments.
I hope I don't make a habit of sending erroneous mails.

Please accept my apologies, as well as a guaranteed whitespace-error-free patch (updated to version 5 for clarity).

Julian

Julian Markwort schrieb am 2018-03-20:

Show quoted text

To anyone who followed along with this for so long, I'd like to present my newest version of this patch.

As suggested by Arthur Zakirov, there is now only a single GUC ( pg_stat_statements.plan_track ) responsible for the selection of the plans that should be tracked. Possible values are: all, none, good, or bad.
I've mostly copied functionality from pl_handler.c . This resulted in the need to include varlena.h so I could use the SplitIdentifierString() function to parse the values, of which several (e.g. pg_stat_statements.plan_track='good, bad') could be used.

I've also added a new GUC:
pg_stat_statements.plan_fence_factor
This GUC can be used to scale the fences of the interval, outside of which a plan might be updated.
Right now, it is set to 1.5 (common factor for the definition of outliers in boxplots) and you can see through additional colums in the pg_stat_statements view, how often these fences are surpassed by execution times and how often the plans are updated. (The colums are: good_plan_outliers, good_plan_updates, bad_plan_outliers, bad_plan_updates and are primarily here for testing and review purposes and are not essential to this patch, they probably don't add any value for the average user)

Similarly to the first suggestion by Arthur, I've also changed the plan_reset functionality - there is now only one function, pg_stat_statements_plan_reset(queryid bigint), overloaded with (queryid bigint, plantype cstring) args, that can be used to remove both plans (when omitting the cstring) or either of them. The cstring argument accepts 'good' or 'bad'.

I also added more comments to the estimations of the quartiles and the calculation of the fences.

The performance impact lies now at 139312 vs 141841 tps, so roughly 1.78% slower than default pg_stat_statements.
The fact that these results are a little worse than the previous iteration is due to some changes in the definition of the fences which mistakenly calculated by adding the scaled interquartile distance to the mean, instead of adding it to the respective quartiles, which means that plan updates are triggered a little more often.
For 4259631 transactions however, only 11 updates for the bad plans where triggered.

I'm looking forward to your opinions!
Julian

Attachments:

pgss_plans_v05.patchtext/x-patchDownload
diff --git a/contrib/pg_stat_statements/Makefile b/contrib/pg_stat_statements/Makefile
index 39b368b70e..49bb462d10 100644
--- a/contrib/pg_stat_statements/Makefile
+++ b/contrib/pg_stat_statements/Makefile
@@ -4,7 +4,8 @@ MODULE_big = pg_stat_statements
 OBJS = pg_stat_statements.o $(WIN32RES)
 
 EXTENSION = pg_stat_statements
-DATA = pg_stat_statements--1.4.sql pg_stat_statements--1.4--1.5.sql \
+DATA = pg_stat_statements--1.4.sql \
+	pg_stat_statements--1.5--1.6.sql pg_stat_statements--1.4--1.5.sql \
 	pg_stat_statements--1.3--1.4.sql pg_stat_statements--1.2--1.3.sql \
 	pg_stat_statements--1.1--1.2.sql pg_stat_statements--1.0--1.1.sql \
 	pg_stat_statements--unpackaged--1.0.sql
diff --git a/contrib/pg_stat_statements/expected/pg_stat_statements.out b/contrib/pg_stat_statements/expected/pg_stat_statements.out
index 5318c3550c..2ca549686f 100644
--- a/contrib/pg_stat_statements/expected/pg_stat_statements.out
+++ b/contrib/pg_stat_statements/expected/pg_stat_statements.out
@@ -395,4 +395,40 @@ SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
  SELECT pg_stat_statements_reset()         |     1 |    1
 (8 rows)
 
+-- test to see if any plans have been recorded.
+SELECT
+  CASE WHEN good_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN bad_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN good_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END,
+  CASE WHEN bad_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END
+FROM pg_stat_statements ORDER BY query COLLATE "C";
+ case | case | case | case 
+------+------+------+------
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    0 |    0 |    0 |    0
+    1 |    1 |    1 |    1
+    1 |    1 |    1 |    1
+    1 |    1 |    1 |    1
+(9 rows)
+
+-- test if there is some text in the recorded plans.
+SELECT substr(good_plan, 0, 11), substr(bad_plan, 0, 11) FROM pg_stat_statements ORDER BY query COLLATE "C";
+   substr   |   substr   
+------------+------------
+            | 
+            | 
+            | 
+            | 
+            | 
+            | 
+ Query Text | Query Text
+ Query Text | Query Text
+ Query Text | Query Text
+ Query Text | Query Text
+(10 rows)
+
 DROP EXTENSION pg_stat_statements;
diff --git a/contrib/pg_stat_statements/pg_stat_statements--1.5--1.6.sql b/contrib/pg_stat_statements/pg_stat_statements--1.5--1.6.sql
new file mode 100644
index 0000000000..6c8f743ee5
--- /dev/null
+++ b/contrib/pg_stat_statements/pg_stat_statements--1.5--1.6.sql
@@ -0,0 +1,82 @@
+/* contrib/pg_stat_statements/pg_stat_statements--1.5--1.6.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "ALTER EXTENSION pg_stat_statements UPDATE TO '1.6'" to load this file. \quit
+
+/* First we have to remove them from the extension */
+ALTER EXTENSION pg_stat_statements DROP VIEW pg_stat_statements;
+ALTER EXTENSION pg_stat_statements DROP FUNCTION pg_stat_statements(boolean);
+ALTER EXTENSION pg_stat_statements DROP FUNCTION pg_stat_statements_reset();
+
+/* Then we can drop them */
+DROP VIEW pg_stat_statements;
+DROP FUNCTION pg_stat_statements(boolean);
+DROP FUNCTION pg_stat_statements_reset();
+
+-- Register functions.
+CREATE FUNCTION pg_stat_statements_reset()
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_statements_plan_reset(IN queryid bigint)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_statements_plan_reset(IN queryid bigint, IN plantype cstring)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_statements(IN showtext boolean,
+    OUT userid oid,
+    OUT dbid oid,
+    OUT queryid bigint,
+    OUT query text,
+    OUT good_plan text,
+    OUT bad_plan text,
+    OUT calls int8,
+    OUT total_time float8,
+    OUT min_time float8,
+    OUT max_time float8,
+    OUT mean_time float8,
+    OUT stddev_time float8,
+    OUT good_plan_time float8,
+    OUT good_plan_outliers int8,
+    OUT good_plan_updates int8,
+    OUT good_plan_timestamp timestamp,
+    OUT bad_plan_time float8,
+    OUT bad_plan_outliers int8,
+    OUT bad_plan_updates int8,
+    OUT bad_plan_timestamp timestamp,
+    OUT rows int8,
+    OUT shared_blks_hit int8,
+    OUT shared_blks_read int8,
+    OUT shared_blks_dirtied int8,
+    OUT shared_blks_written int8,
+    OUT local_blks_hit int8,
+    OUT local_blks_read int8,
+    OUT local_blks_dirtied int8,
+    OUT local_blks_written int8,
+    OUT temp_blks_read int8,
+    OUT temp_blks_written int8,
+    OUT blk_read_time float8,
+    OUT blk_write_time float8
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stat_statements_1_6'
+LANGUAGE C STRICT VOLATILE PARALLEL SAFE;
+
+-- Register a view on the function for ease of use.
+CREATE VIEW pg_stat_statements AS
+  SELECT * FROM pg_stat_statements(true);
+
+GRANT SELECT ON pg_stat_statements TO PUBLIC;
+
+-- Don't want this to be available to non-superusers.
+REVOKE ALL ON FUNCTION pg_stat_statements_reset() FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_stat_statements_plan_reset(bigint) FROM PUBLIC;
+REVOKE ALL ON FUNCTION pg_stat_statements_plan_reset(bigint, cstring) FROM PUBLIC;
+
+GRANT EXECUTE ON FUNCTION pg_stat_statements_reset() TO pg_read_all_stats;
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 928673498a..c6f5e97b5c 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -63,6 +63,7 @@
 
 #include "access/hash.h"
 #include "catalog/pg_authid.h"
+#include "commands/explain.h"
 #include "executor/instrument.h"
 #include "funcapi.h"
 #include "mb/pg_wchar.h"
@@ -78,6 +79,8 @@
 #include "tcop/utility.h"
 #include "utils/builtins.h"
 #include "utils/memutils.h"
+#include "utils/timestamp.h"
+#include "utils/varlena.h"
 
 PG_MODULE_MAGIC;
 
@@ -119,7 +122,8 @@ typedef enum pgssVersion
 	PGSS_V1_0 = 0,
 	PGSS_V1_1,
 	PGSS_V1_2,
-	PGSS_V1_3
+	PGSS_V1_3,
+	PGSS_V1_6
 } pgssVersion;
 
 /*
@@ -165,6 +169,16 @@ typedef struct Counters
 	double		usage;			/* usage factor */
 } Counters;
 
+typedef struct pgssPlan
+{
+	Size	offset;
+	int		len;
+	double		time;	/* execution time in msec when the latest plan was updated */
+	TimestampTz timestamp;
+	int64 outliers;
+	int64 updates;
+} pgssPlan;
+
 /*
  * Statistics per statement
  *
@@ -178,6 +192,8 @@ typedef struct pgssEntry
 	Counters	counters;		/* the statistics for this query */
 	Size		query_offset;	/* query text offset in external file */
 	int			query_len;		/* # of valid bytes in query string, or -1 */
+	pgssPlan	good_plan;
+	pgssPlan	bad_plan;
 	int			encoding;		/* query text encoding */
 	slock_t		mutex;			/* protects the counters only */
 } pgssEntry;
@@ -265,11 +281,26 @@ static const struct config_enum_entry track_options[] =
 	{NULL, 0, false}
 };
 
+static const struct config_enum_entry format_options[] =
+{
+	{"text", EXPLAIN_FORMAT_TEXT, false},
+	{"xml", EXPLAIN_FORMAT_XML, false},
+	{"json", EXPLAIN_FORMAT_JSON, false},
+	{"yaml", EXPLAIN_FORMAT_YAML, false},
+	{NULL, 0, false}
+};
+
 static int	pgss_max;			/* max # statements to track */
 static int	pgss_track;			/* tracking level */
 static bool pgss_track_utility; /* whether to track utility commands */
 static bool pgss_save;			/* whether to save stats across shutdown */
-
+static bool pgss_good_plan_enable; 	/* whether to save good_plans */
+static bool pgss_bad_plan_enable; 	/* whether to save bad_plans */
+static bool pgss_good_plan_enable_temp; /* used to store state between plan_track_check and _assign */
+static bool pgss_bad_plan_enable_temp; 	/* used to store state between plan_track_check and _assign */
+static int	pgss_plan_format;   	/* id which sets the output format */
+static double pgss_plan_fence_factor; /* can be used to scale the fences for plan updates */
+static char *pgss_plan_track_str; /* String holding the arguments provided to the pgss_plan_track GUC */
 
 #define pgss_enabled() \
 	(pgss_track == PGSS_TRACK_ALL || \
@@ -291,7 +322,9 @@ void		_PG_fini(void);
 PG_FUNCTION_INFO_V1(pg_stat_statements_reset);
 PG_FUNCTION_INFO_V1(pg_stat_statements_1_2);
 PG_FUNCTION_INFO_V1(pg_stat_statements_1_3);
+PG_FUNCTION_INFO_V1(pg_stat_statements_1_6);
 PG_FUNCTION_INFO_V1(pg_stat_statements);
+PG_FUNCTION_INFO_V1(pg_stat_statements_plan_reset);
 
 static void pgss_shmem_startup(void);
 static void pgss_shmem_shutdown(int code, Datum arg);
@@ -309,12 +342,17 @@ static void pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
 static uint64 pgss_hash_string(const char *str, int len);
 static void pgss_store(const char *query, uint64 queryId,
 		   int query_location, int query_len,
+		   QueryDesc *queryDesc,
 		   double total_time, uint64 rows,
 		   const BufferUsage *bufusage,
 		   pgssJumbleState *jstate);
+Datum pg_stat_statements_plan_reset(PG_FUNCTION_ARGS);
 static void pg_stat_statements_internal(FunctionCallInfo fcinfo,
 							pgssVersion api_version,
 							bool showtext);
+static int fill_plan_times(Datum values[], bool nulls[], int i, pgssPlan *plan);
+static int fill_plan_str(Datum values[], bool nulls[], int i, pgssPlan *plan,
+							pgssEntry *entry, char *qbuffer, Size qbuffer_size);
 static Size pgss_memsize(void);
 static pgssEntry *entry_alloc(pgssHashKey *key, Size query_offset, int query_len,
 			int encoding, bool sticky);
@@ -338,7 +376,8 @@ static char *generate_normalized_query(pgssJumbleState *jstate, const char *quer
 static void fill_in_constant_lengths(pgssJumbleState *jstate, const char *query,
 						 int query_loc);
 static int	comp_location(const void *a, const void *b);
-
+static bool plan_track_check_hook(char **newvalue, void **extra, GucSource source);
+static void plan_track_assign_hook(const char *newvalue, void *extra);
 
 /*
  * Module load callback
@@ -407,6 +446,42 @@ _PG_init(void)
 							 NULL,
 							 NULL);
 
+	DefineCustomStringVariable("pg_stat_statements.plan_track",
+							   "Enable plan tracking",
+							   "Specify the types of plans that shall be tracked: all, none, good, or bad. Support for multiple individual types (curently only good and bad) is available.",
+							   &pgss_plan_track_str,
+							   "all",
+							   PGC_SUSET,
+							   GUC_LIST_INPUT,
+							   plan_track_check_hook,
+							   plan_track_assign_hook,
+							   NULL);
+
+	DefineCustomEnumVariable("pg_stat_statements.plan_format",
+							 "Sets the output format for the plans.",
+							 "Notice that the plan format can not be changed after a plan is saved. Valid values are text, json, xml and yaml.",
+							 &pgss_plan_format,
+							 EXPLAIN_FORMAT_TEXT,
+							 format_options,
+							 PGC_SUSET,
+							 0,
+							 NULL,
+							 NULL,
+							 NULL);
+
+	DefineCustomRealVariable("pg_stat_statements.plan_fence_factor",
+							 "Sets the factor that scales the fences that are used to determine whether a plan should be updated.",
+							 NULL,
+							 &pgss_plan_fence_factor,
+							 1.5,
+							 0.0,
+							 +INFINITY,
+							 PGC_SUSET,
+							 0,
+							 NULL,
+							 NULL,
+							 NULL);
+
 	EmitWarningsOnPlaceholders("pg_stat_statements");
 
 	/*
@@ -470,7 +545,11 @@ pgss_shmem_startup(void)
 	int32		pgver;
 	int32		i;
 	int			buffer_size;
+	int			good_plan_buffer_size;
+	int			bad_plan_buffer_size;
 	char	   *buffer = NULL;
+	char	   *good_plan_buffer = NULL;
+	char	   *bad_plan_buffer = NULL;
 
 	if (prev_shmem_startup_hook)
 		prev_shmem_startup_hook();
@@ -561,7 +640,11 @@ pgss_shmem_startup(void)
 	}
 
 	buffer_size = 2048;
+	good_plan_buffer_size = 2048;
+	bad_plan_buffer_size = 2048;
 	buffer = (char *) palloc(buffer_size);
+	good_plan_buffer = (char *) palloc(good_plan_buffer_size);
+	bad_plan_buffer = (char *) palloc(bad_plan_buffer_size);
 
 	if (fread(&header, sizeof(uint32), 1, file) != 1 ||
 		fread(&pgver, sizeof(uint32), 1, file) != 1 ||
@@ -595,8 +678,34 @@ pgss_shmem_startup(void)
 		if (fread(buffer, 1, temp.query_len + 1, file) != temp.query_len + 1)
 			goto read_error;
 
+		/* Resize good plan_buffer and read into it */
+		if (temp.good_plan.len >= good_plan_buffer_size)
+		{
+			good_plan_buffer_size =
+				Max(good_plan_buffer_size * 2, temp.good_plan.len + 1);
+			good_plan_buffer= repalloc(good_plan_buffer, good_plan_buffer_size);
+		}
+
+		if (fread(good_plan_buffer, 1, temp.good_plan.len + 1, file)
+				!= temp.good_plan.len + 1)
+			goto read_error;
+
+		/* Resize bad plan_buffer and read into it */
+		if (temp.bad_plan.len >= bad_plan_buffer_size)
+		{
+			bad_plan_buffer_size =
+				Max(bad_plan_buffer_size * 2, temp.bad_plan.len + 1);
+			bad_plan_buffer= repalloc(bad_plan_buffer, bad_plan_buffer_size);
+		}
+
+		if (fread(bad_plan_buffer, 1, temp.bad_plan.len + 1, file)
+				!= temp.bad_plan.len + 1)
+			goto read_error;
+
 		/* Should have a trailing null, but let's make sure */
 		buffer[temp.query_len] = '\0';
+		good_plan_buffer[temp.good_plan.len] = '\0';
+		bad_plan_buffer[temp.bad_plan.len] = '\0';
 
 		/* Skip loading "sticky" entries */
 		if (temp.counters.calls == 0)
@@ -608,6 +717,20 @@ pgss_shmem_startup(void)
 			goto write_error;
 		pgss->extent += temp.query_len + 1;
 
+		/* Store the good plan text*/
+		temp.good_plan.offset = pgss->extent;
+		if (fwrite(good_plan_buffer, 1, temp.good_plan.len + 1, qfile)
+				!= temp.good_plan.len + 1)
+			goto write_error;
+		pgss->extent += temp.good_plan.len + 1;
+
+		/* Store the bad plan text*/
+		temp.bad_plan.offset = pgss->extent;
+		if (fwrite(bad_plan_buffer, 1, temp.bad_plan.len + 1, qfile)
+				!= temp.bad_plan.len + 1)
+			goto write_error;
+		pgss->extent += temp.bad_plan.len + 1;
+
 		/* make the hashtable entry (discards old entries if too many) */
 		entry = entry_alloc(&temp.key, query_offset, temp.query_len,
 							temp.encoding,
@@ -615,9 +738,14 @@ pgss_shmem_startup(void)
 
 		/* copy in the actual stats */
 		entry->counters = temp.counters;
+				/* copy in the plans */
+		entry->good_plan = temp.good_plan;
+		entry->bad_plan = temp.bad_plan;
 	}
 
 	pfree(buffer);
+	pfree(good_plan_buffer);
+	pfree(bad_plan_buffer);
 	FreeFile(file);
 	FreeFile(qfile);
 
@@ -658,6 +786,10 @@ write_error:
 fail:
 	if (buffer)
 		pfree(buffer);
+	if (good_plan_buffer)
+		pfree(good_plan_buffer);
+	if (bad_plan_buffer)
+		pfree(bad_plan_buffer);
 	if (file)
 		FreeFile(file);
 	if (qfile)
@@ -723,14 +855,52 @@ pgss_shmem_shutdown(int code, Datum arg)
 	while ((entry = hash_seq_search(&hash_seq)) != NULL)
 	{
 		int			len = entry->query_len;
+		int			good_plan_len = entry->good_plan.len;
+		int			bad_plan_len = entry->bad_plan.len;
 		char	   *qstr = qtext_fetch(entry->query_offset, len,
 									   qbuffer, qbuffer_size);
+		char	   *good_plan_str;
+		char	   *bad_plan_str;
+		if (good_plan_len > 0)
+		{
+			/* A good plan is available */
+			good_plan_str = qtext_fetch(entry->good_plan.offset, good_plan_len,
+												qbuffer, qbuffer_size);
+		}
+		else
+		{
+			/*
+			 * There is no good plan available. This can be caused by a query
+			 * which has no plan (for example insert or delete queries) or
+			 * because good plans are disabled. In this case we have to store
+			 * an empty string instead of null.
+			 */
+			good_plan_str = "";
+		}
+		if (bad_plan_len > 0)
+		{
+			/* A bad plan is available */
+			bad_plan_str = qtext_fetch(entry->bad_plan.offset, bad_plan_len,
+												qbuffer, qbuffer_size);
+		}
+		else
+		{
+			/*
+			 * There is no bad plan available. This can be caused by a query
+			 * which has no plan (for example insert or delete queries) or
+			 * because bad plans are disabled. In this case we have to store
+			 * an empty string instead of null.
+			 */
+			bad_plan_str = "";
+		}
 
 		if (qstr == NULL)
 			continue;			/* Ignore any entries with bogus texts */
-
+		/* Write entries, queries and plans serialized to the dump file */
 		if (fwrite(entry, sizeof(pgssEntry), 1, file) != 1 ||
-			fwrite(qstr, 1, len + 1, file) != len + 1)
+			fwrite(qstr, 1, len + 1, file) != len + 1 ||
+			fwrite(good_plan_str, 1, good_plan_len + 1, file) != good_plan_len + 1 ||
+			fwrite(bad_plan_str, 1, bad_plan_len + 1, file) != bad_plan_len + 1)
 		{
 			/* note: we assume hash_seq_term won't change errno */
 			hash_seq_term(&hash_seq);
@@ -835,6 +1005,7 @@ pgss_post_parse_analyze(ParseState *pstate, Query *query)
 				   query->queryId,
 				   query->stmt_location,
 				   query->stmt_len,
+				   NULL,
 				   0,
 				   0,
 				   NULL,
@@ -942,6 +1113,7 @@ pgss_ExecutorEnd(QueryDesc *queryDesc)
 				   queryId,
 				   queryDesc->plannedstmt->stmt_location,
 				   queryDesc->plannedstmt->stmt_len,
+				   queryDesc,								/* queryDesc itself is needed to retrieve the plan */
 				   queryDesc->totaltime->total * 1000.0,	/* convert to msec */
 				   queryDesc->estate->es_processed,
 				   &queryDesc->totaltime->bufusage,
@@ -1053,6 +1225,7 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
 				   0,			/* signal that it's a utility stmt */
 				   pstmt->stmt_location,
 				   pstmt->stmt_len,
+				   NULL,
 				   INSTR_TIME_GET_MILLISEC(duration),
 				   rows,
 				   &bufusage,
@@ -1083,6 +1256,26 @@ pgss_hash_string(const char *str, int len)
 											len, 0));
 }
 
+/*
+ * Update a plan in the text file.
+ * We can re-use qtext_store to save the plan.
+ */
+static void
+update_plan(volatile pgssPlan* pgssPlan_ptr, const char* plan_str, int plan_len,
+			double total_time)
+{
+	if (plan_len > 0)
+	{
+		Size plan_offset;
+		qtext_store(plan_str, plan_len, &plan_offset, NULL);
+		pgssPlan_ptr->offset	= plan_offset;
+		pgssPlan_ptr->len		= plan_len;
+		pgssPlan_ptr->time		= total_time;
+		pgssPlan_ptr->timestamp = GetCurrentTimestamp();
+	}
+	return;
+}
+
 /*
  * Store some statistics for a statement.
  *
@@ -1096,6 +1289,7 @@ pgss_hash_string(const char *str, int len)
 static void
 pgss_store(const char *query, uint64 queryId,
 		   int query_location, int query_len,
+		   QueryDesc *queryDesc,
 		   double total_time, uint64 rows,
 		   const BufferUsage *bufusage,
 		   pgssJumbleState *jstate)
@@ -1134,6 +1328,7 @@ pgss_store(const char *query, uint64 queryId,
 		query_len = strlen(query);
 	}
 
+
 	/*
 	 * Discard leading and trailing whitespace, too.  Use scanner_isspace()
 	 * not libc's isspace(), because we want to match the lexer's behavior.
@@ -1231,9 +1426,136 @@ pgss_store(const char *query, uint64 queryId,
 		 * locking rules at the head of the file)
 		 */
 		volatile pgssEntry *e = (volatile pgssEntry *) entry;
+		bool	good_plan_needs_updating = false;
+		bool	bad_plan_needs_updating = false;
+		int		plan_len = 0;
+		char*	plan;
 
 		SpinLockAcquire(&e->mutex);
 
+		if (queryDesc && (pgss_good_plan_enable || pgss_bad_plan_enable))
+		{
+			/* Avoid division by zero errors. */
+			if (e->counters.calls > 0)
+			{
+				/*
+				 * The quantiles for a normal distributions can be estimated through
+				 * X = expected value + standard deviation * Z-score
+				 * For the 25% and 75% quantiles (=lower and upper quartile),
+				 * the z-scores -0.6754 and 0.6745 are used.
+				 */
+				double z_score = 0.6745;
+				/*
+				 * Since we're already making a bunch of statistical assumptions,
+				 * let's use the fact that the expected value can be approximated
+				 * by the mean of a dataset (see "law of large numbers").
+				 */
+
+				double std_dev = sqrt(e->counters.sum_var_time / e->counters.calls);
+				double exp_val = e->counters.mean_time;
+
+				double lower_quartile = exp_val - std_dev * z_score;
+				double upper_quartile = exp_val + std_dev * z_score;
+
+				double interquartile_dist = upper_quartile - lower_quartile;
+
+				/*
+				 * We calculate a set of inner and outer fences, similar to the
+				 * definition of whiskers in boxplots.
+				 * By default, pgss_plan_fence_factor is set to 1.5 .
+				 */
+				double lower_fence = lower_quartile - pgss_plan_fence_factor * interquartile_dist;
+				double upper_fence = upper_quartile + pgss_plan_fence_factor * interquartile_dist;
+
+				if (pgss_good_plan_enable)
+				{
+					/*
+					 * Only update bad_plan if execution time was slower than the lower fence.
+					 * The use of statistics to estimate wether the change in execution time
+					 * was big enough to warrant a plan update is debatable, however it is
+					 * probably the fastest way to identify changes in the planner
+					 * without actually comparing the plans step by step.
+					 */
+					if (total_time < lower_fence)
+					{
+						e->good_plan.outliers++;
+						/* Only update plan if execution time was slower than that of the old plan*/
+						if (total_time < e->good_plan.time)
+						{
+							good_plan_needs_updating = true;
+							e->good_plan.updates++;
+						}
+					}
+				}
+
+				if (pgss_bad_plan_enable)
+				{
+					/* Only update good_plan if execution time was faster than the upperer fence.*/
+					if (total_time > upper_fence)
+					{
+						e->bad_plan.outliers++;
+						if (total_time > e->bad_plan.time)
+						{
+							bad_plan_needs_updating = true;
+							e->bad_plan.updates++;
+						}
+					}
+				}
+			}
+
+			/*
+			 * Plans should be updated whenever no previous plan was stored.
+			 */
+			if (pgss_good_plan_enable && e->good_plan.time==0)
+				good_plan_needs_updating = true;
+			if (pgss_bad_plan_enable && e->bad_plan.time==0)
+				bad_plan_needs_updating = true;
+
+			if (good_plan_needs_updating || bad_plan_needs_updating)
+			{
+				/* *es is used to retrieve the plan associated with the statement*/
+				ExplainState *es = NewExplainState();
+
+				es->analyze = (queryDesc->instrument_options && false);
+				es->verbose = false;
+				es->buffers = (es->analyze && false);
+				es->timing = (es->analyze && true);
+				es->format = pgss_plan_format;
+
+				ExplainBeginOutput(es);
+				ExplainQueryText(es, queryDesc);
+				ExplainPrintPlan(es, queryDesc);
+				ExplainEndOutput(es);
+
+				/* Remove last line break */
+				if (es->str->len > 0 && es->str->data[es->str->len - 1] == '\n')
+					es->str->data[--es->str->len] = '\0';
+
+				/* Fix JSON to output an object */
+				if (pgss_plan_format == EXPLAIN_FORMAT_JSON)
+				{
+					es->str->data[0] = '{';
+					es->str->data[es->str->len - 1] = '}';
+				}
+
+				plan = es->str->data;
+				plan_len = strlen(plan);
+
+				if (good_plan_needs_updating)
+				{
+					update_plan(&e->good_plan, plan, plan_len, total_time);
+				}
+
+				if (bad_plan_needs_updating)
+				{
+					update_plan(&e->bad_plan, plan, plan_len, total_time);
+				}
+
+				pfree(es->str->data);
+				pfree(es);
+			}
+		}
+
 		/* "Unstick" entry if it was previously sticky */
 		if (e->counters.calls == 0)
 			e->counters.usage = USAGE_INIT;
@@ -1305,12 +1627,76 @@ pg_stat_statements_reset(PG_FUNCTION_ARGS)
 	PG_RETURN_VOID();
 }
 
+Datum
+pg_stat_statements_plan_reset(PG_FUNCTION_ARGS)
+{
+	pgssEntry   *entry;
+	pgssHashKey	key;
+
+	uint64 query_id = PG_GETARG_INT64(0);
+
+	bool good_plan_reset = false;
+	bool bad_plan_reset = false;
+
+	if (PG_NARGS() == 2)
+	{
+		char *plan_type_str = PG_GETARG_CSTRING(1);
+		if (pg_strcasecmp(plan_type_str, "good") == 0)
+			good_plan_reset = true;
+		else if (pg_strcasecmp(plan_type_str, "bad") == 0)
+			bad_plan_reset = true;
+		else
+			elog(LOG, "If a second argument is provided to pg_stat_statements_plan_reset, it must be of a valid plan type ('good' or 'bad').");
+	}
+	else
+	{
+		good_plan_reset = true;
+		bad_plan_reset = true;
+	}
+
+	if (!pgss || !pgss_hash)
+		ereport(ERROR,
+				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+				 errmsg("pg_stat_statements must be loaded via shared_preload_libraries")));
+
+	/* Set up key for hashtable search */
+	key.userid = GetUserId();
+	key.dbid = MyDatabaseId;
+	key.queryid = query_id;
+
+	/* Lookup the hash table entry with shared lock. */
+	LWLockAcquire(pgss->lock, LW_SHARED);
+
+	entry = (pgssEntry *) hash_search(pgss_hash, &key, HASH_FIND, NULL);
+
+	if (entry)
+	{
+		if (good_plan_reset)
+		{
+			entry->good_plan.offset = -1;
+			entry->good_plan.len = 0;
+			entry->good_plan.time = 0;
+			entry->good_plan.timestamp = 0;
+		}
+		if (bad_plan_reset)
+		{
+			entry->bad_plan.offset = -1;
+			entry->bad_plan.len = 0;
+			entry->bad_plan.time = 0;
+			entry->bad_plan.timestamp = 0;
+		}
+	}
+	LWLockRelease(pgss->lock);
+	PG_RETURN_VOID();
+}
+
 /* Number of output arguments (columns) for various API versions */
 #define PG_STAT_STATEMENTS_COLS_V1_0	14
 #define PG_STAT_STATEMENTS_COLS_V1_1	18
 #define PG_STAT_STATEMENTS_COLS_V1_2	19
 #define PG_STAT_STATEMENTS_COLS_V1_3	23
-#define PG_STAT_STATEMENTS_COLS			23	/* maximum of above */
+#define PG_STAT_STATEMENTS_COLS_V1_6	33
+#define PG_STAT_STATEMENTS_COLS			33	/* maximum of above */
 
 /*
  * Retrieve statement statistics.
@@ -1322,6 +1708,16 @@ pg_stat_statements_reset(PG_FUNCTION_ARGS)
  * expected API version is identified by embedding it in the C name of the
  * function.  Unfortunately we weren't bright enough to do that for 1.1.
  */
+Datum
+pg_stat_statements_1_6(PG_FUNCTION_ARGS)
+{
+ bool		showtext = PG_GETARG_BOOL(0);
+
+ pg_stat_statements_internal(fcinfo, PGSS_V1_6, showtext);
+
+ return (Datum) 0;
+}
+
 Datum
 pg_stat_statements_1_3(PG_FUNCTION_ARGS)
 {
@@ -1428,6 +1824,10 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 			if (api_version != PGSS_V1_3)
 				elog(ERROR, "incorrect number of output arguments");
 			break;
+		case PG_STAT_STATEMENTS_COLS_V1_6:
+			if (api_version != PGSS_V1_6)
+				elog(ERROR, "incorrect number of output arguments");
+			break;
 		default:
 			elog(ERROR, "incorrect number of output arguments");
 	}
@@ -1511,6 +1911,8 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 		bool		nulls[PG_STAT_STATEMENTS_COLS];
 		int			i = 0;
 		Counters	tmp;
+		pgssPlan 	tmp_good_plan;
+		pgssPlan 	tmp_bad_plan;
 		double		stddev;
 		int64		queryid = entry->key.queryid;
 
@@ -1550,6 +1952,18 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 					/* Just return a null if we fail to find the text */
 					nulls[i++] = true;
 				}
+
+				if (api_version >= PGSS_V1_6)
+				{
+					/*
+					 * This version requires to print out the good
+					 * and the bad plan if they are enabled.
+					 */
+					i = fill_plan_str(values, nulls, i, &entry->good_plan,
+										entry, qbuffer, qbuffer_size);
+					i = fill_plan_str(values, nulls, i, &entry->bad_plan,
+										entry, qbuffer, qbuffer_size);
+				}
 			}
 			else
 			{
@@ -1579,6 +1993,8 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 
 			SpinLockAcquire(&e->mutex);
 			tmp = e->counters;
+			tmp_good_plan = e->good_plan;
+			tmp_bad_plan = e->bad_plan;
 			SpinLockRelease(&e->mutex);
 		}
 
@@ -1606,6 +2022,14 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 				stddev = 0.0;
 			values[i++] = Float8GetDatumFast(stddev);
 		}
+
+		if (api_version >= PGSS_V1_6)
+		{
+			/* Set plan times and timestamps */
+			i = fill_plan_times(values, nulls, i, &tmp_good_plan);
+			i = fill_plan_times(values, nulls, i, &tmp_bad_plan);
+		}
+
 		values[i++] = Int64GetDatumFast(tmp.rows);
 		values[i++] = Int64GetDatumFast(tmp.shared_blks_hit);
 		values[i++] = Int64GetDatumFast(tmp.shared_blks_read);
@@ -1629,6 +2053,7 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 					 api_version == PGSS_V1_1 ? PG_STAT_STATEMENTS_COLS_V1_1 :
 					 api_version == PGSS_V1_2 ? PG_STAT_STATEMENTS_COLS_V1_2 :
 					 api_version == PGSS_V1_3 ? PG_STAT_STATEMENTS_COLS_V1_3 :
+					 api_version == PGSS_V1_6 ? PG_STAT_STATEMENTS_COLS_V1_6 :
 					 -1 /* fail if you forget to update this assert */ ));
 
 		tuplestore_putvalues(tupstore, tupdesc, values, nulls);
@@ -1643,6 +2068,54 @@ pg_stat_statements_internal(FunctionCallInfo fcinfo,
 	tuplestore_donestoring(tupstore);
 }
 
+/* Fill the plan time and timestamp into the values array. */
+static int
+fill_plan_times(Datum values[], bool nulls[], int i, pgssPlan *plan)
+{
+	values[i++] = Float8GetDatumFast(plan->time);
+	values[i++] = Int64GetDatumFast(plan->outliers);
+	values[i++] = Int64GetDatumFast(plan->updates);
+
+	/* if there is noting in the timestamp field, we are not interested in it */
+	if (plan->timestamp)
+		values[i++] = TimestampTzGetDatum(plan->timestamp);
+	else
+		nulls[i++] = true;
+	return i;
+}
+
+/* Fill the plan string into the values array. */
+static int
+fill_plan_str(Datum values[], bool nulls[], int i, pgssPlan *plan,
+				pgssEntry *entry, char *qbuffer, Size qbuffer_size)
+{
+	if (plan && plan->len > 0)
+	{
+		char *pstr = qtext_fetch(plan->offset, plan->len, qbuffer, qbuffer_size);
+		if (pstr)
+		{
+			char *enc;
+			enc = pg_any_to_server(pstr, plan->len, entry->encoding);
+			values[i++] = CStringGetTextDatum(enc);
+
+			if (enc != pstr)
+				pfree(enc);
+		}
+		else
+		{
+			/* failed to get the string of the plan */
+			nulls[i++] = true;
+		}
+	}
+	else
+	{
+		/* no plan available or plan_len not greater than 0 */
+		nulls[i++] = true;
+	}
+	return i;
+}
+
+
 /*
  * Estimate shared memory space needed.
  */
@@ -1694,6 +2167,8 @@ entry_alloc(pgssHashKey *key, Size query_offset, int query_len, int encoding,
 
 		/* reset the statistics */
 		memset(&entry->counters, 0, sizeof(Counters));
+		memset(&entry->good_plan, 0, sizeof(pgssPlan));
+		memset(&entry->bad_plan, 0, sizeof(pgssPlan));
 		/* set the appropriate initial usage count */
 		entry->counters.usage = sticky ? pgss->cur_median_usage : USAGE_INIT;
 		/* re-initialize the mutex each time ... we assume no one using it */
@@ -2104,6 +2579,18 @@ gc_qtexts(void)
 									  qbuffer,
 									  qbuffer_size);
 
+		int			good_plan_len = entry->good_plan.len;
+		char	   *good_plan = qtext_fetch(entry->good_plan.offset,
+									   good_plan_len,
+									   qbuffer,
+									   qbuffer_size);
+
+		int			bad_plan_len = entry->bad_plan.len;
+		char	   *bad_plan = qtext_fetch(entry->bad_plan.offset,
+									   bad_plan_len,
+									   qbuffer,
+									   qbuffer_size);
+
 		if (qry == NULL)
 		{
 			/* Trouble ... drop the text */
@@ -2125,6 +2612,53 @@ gc_qtexts(void)
 
 		entry->query_offset = extent;
 		extent += query_len + 1;
+
+		if (good_plan == NULL || good_plan_len <= 0)
+		{
+			/* There was an error while loading the good_plan or there was simply never a
+			 * good_plan recorded, so we make sure that this entry knows this. */
+			entry->good_plan.offset = 0;
+			entry->good_plan.len = -1;
+		}
+		else
+		{
+			/* Save the good plan */
+			if (fwrite(good_plan, 1, good_plan_len + 1, qfile) != good_plan_len + 1)
+			{
+				ereport(LOG,
+						(errcode_for_file_access(),
+					  errmsg("could not write pg_stat_statement file \"%s\": %m",
+							 PGSS_TEXT_FILE)));
+				hash_seq_term(&hash_seq);
+				goto gc_fail;
+			}
+			entry->good_plan.offset = extent;
+			extent += good_plan_len + 1;
+		}
+
+		if (bad_plan == NULL || bad_plan_len <= 0)
+		{
+			/* There was an error while loading the bad_plan or there was simply never a
+			 * bad_plan recorded, so we make sure that this entry knows this. */
+			entry->bad_plan.offset = 0;
+			entry->bad_plan.len = -1;
+		}
+		else
+		{
+			/* Save the bad plan */
+			if (fwrite(bad_plan, 1, bad_plan_len + 1, qfile) != bad_plan_len + 1)
+			{
+				ereport(LOG,
+						(errcode_for_file_access(),
+					  errmsg("could not write pg_stat_statement file \"%s\": %m",
+							 PGSS_TEXT_FILE)));
+				hash_seq_term(&hash_seq);
+				goto gc_fail;
+			}
+			entry->bad_plan.offset = extent;
+			extent += bad_plan_len + 1;
+		}
+
 		nentries++;
 	}
 
@@ -3163,3 +3697,81 @@ comp_location(const void *a, const void *b)
 	else
 		return 0;
 }
+
+/*
+ * Check the list of plans that shall be tracked and set the
+ * appropriate temporary bools.
+ */
+static bool
+plan_track_check_hook(char **newvalue, void **extra, GucSource source)
+{
+	char	   *rawstring;
+	List	   *elemlist;
+	ListCell   *l;
+
+	/* Set all temp bools to false */
+	pgss_good_plan_enable_temp = false;
+	pgss_bad_plan_enable_temp = false;
+
+	if (pg_strcasecmp(*newvalue, "all") == 0)
+	{
+		pgss_good_plan_enable_temp = true;
+		pgss_bad_plan_enable_temp = true;
+	}
+	else if (pg_strcasecmp(*newvalue, "none") == 0)
+		return true;
+	else
+	{
+		/* Need a modifiable copy of string */
+		rawstring = pstrdup(*newvalue);
+
+		/* Parse string into list of identifiers */
+		if (!SplitIdentifierString(rawstring, ',', &elemlist))
+		{
+			/* syntax error in list */
+			GUC_check_errdetail("List syntax is invalid.");
+			pfree(rawstring);
+			list_free(elemlist);
+			return false;
+		}
+
+		foreach(l, elemlist)
+		{
+			char	   *tok = (char *) lfirst(l);
+
+			if (pg_strcasecmp(tok, "good") == 0)
+				pgss_good_plan_enable_temp = true;
+			else if (pg_strcasecmp(tok, "bad") == 0)
+				pgss_bad_plan_enable_temp = true;
+			else if (pg_strcasecmp(tok, "all") == 0 || pg_strcasecmp(tok, "none") == 0)
+			{
+				GUC_check_errdetail("Key word \"%s\" cannot be combined with other key words.", tok);
+				pfree(rawstring);
+				list_free(elemlist);
+				return false;
+			}
+			else
+			{
+				GUC_check_errdetail("Unrecognized key word: \"%s\".", tok);
+				pfree(rawstring);
+				list_free(elemlist);
+				return false;
+			}
+		}
+		pfree(rawstring);
+		list_free(elemlist);
+	}
+
+	return true;
+}
+
+/*
+ * The bools enabling and disabling tracking of certain plans
+ * are set to the values previously set in the temporary variables.
+ */
+static void
+plan_track_assign_hook(const char *newvalue, void *extra)
+{
+	pgss_good_plan_enable = pgss_good_plan_enable_temp;
+	pgss_bad_plan_enable = pgss_bad_plan_enable_temp;
+}
\ No newline at end of file
diff --git a/contrib/pg_stat_statements/pg_stat_statements.control b/contrib/pg_stat_statements/pg_stat_statements.control
index 193fcdfafa..617038b4c0 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.control
+++ b/contrib/pg_stat_statements/pg_stat_statements.control
@@ -1,5 +1,5 @@
 # pg_stat_statements extension
 comment = 'track execution statistics of all SQL statements executed'
-default_version = '1.5'
+default_version = '1.6'
 module_pathname = '$libdir/pg_stat_statements'
 relocatable = true
diff --git a/contrib/pg_stat_statements/sql/pg_stat_statements.sql b/contrib/pg_stat_statements/sql/pg_stat_statements.sql
index a8361fd1bf..4eb49f84c6 100644
--- a/contrib/pg_stat_statements/sql/pg_stat_statements.sql
+++ b/contrib/pg_stat_statements/sql/pg_stat_statements.sql
@@ -195,4 +195,15 @@ DROP FUNCTION PLUS_TWO(INTEGER);
 
 SELECT query, calls, rows FROM pg_stat_statements ORDER BY query COLLATE "C";
 
+-- test to see if any plans have been recorded.
+SELECT
+  CASE WHEN good_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN bad_plan_time > 0 THEN 1 ELSE 0 END,
+  CASE WHEN good_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END,
+  CASE WHEN bad_plan_timestamp >= timestamp '1970-01-01 00:00:00' THEN 1 ELSE 0 END
+FROM pg_stat_statements ORDER BY query COLLATE "C";
+
+-- test if there is some text in the recorded plans.
+SELECT substr(good_plan, 0, 11), substr(bad_plan, 0, 11) FROM pg_stat_statements ORDER BY query COLLATE "C";
+
 DROP EXTENSION pg_stat_statements;
#24legrand legrand
legrand_legrand@hotmail.com
In reply to: Julian Markwort (#20)
Re: [FEATURE PATCH] pg_stat_statements with plans (v02)

Hello,
I'm very interested in pg_stat_statements usage, and I'm very happy to see
you adding plans to it.

Reading other pg_stat_statements threads on this forum, there are also activ
developments to add:
- planing duration,
- first date,
- last_update date,
- parameters for normalized queries,
- ...
as described in
http://www.postgresql-archive.org/pg-stat-statements-HLD-for-futur-developments-td6012381.html

I was wondering about how would your dev behave with all those new features.
It seems to me that bad and good plans will not have any of thoses
informations.

What would you think about displaying good, current, bad plans results in 3
lines inspite of only one line ?

queryid, plantype, query, ...
aaa good ...
aaa current ...
aaa bad ...

This would permit to get planing duration, first capture time, last capture,
executions, query parameters for all plans (good or bad inclusive).

Last question, didn't you think about a model to store all the different
plans using a planid like

queryid, planid, query, ...
aaa plan1 ...
aaa plan2 ...
aaa plan3 ...
...

I can not imagine that there would be so many of them ;o)
Regards
PAscal

--
Sent from: http://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html

#25Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: legrand legrand (#24)
Re: [FEATURE PATCH] pg_stat_statements with plans (v02)

On Thu, Mar 22, 2018 at 11:16:30AM -0700, legrand legrand wrote:

Reading other pg_stat_statements threads on this forum, there are also activ
developments to add:
- planing duration,
- first date,
- last_update date,
- parameters for normalized queries,
- ...
as described in
http://www.postgresql-archive.org/pg-stat-statements-HLD-for-futur-developments-td6012381.html

I was wondering about how would your dev behave with all those new features.
It seems to me that bad and good plans will not have any of thoses
informations.

Indeed. I think those developments require cooperation.
Also as you wrote an additional view can be added (pg_stat_statements_plans or pg_stat_plans). But I'm not sure about the name for now and its use cases.

--
Arthur Zakirov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company