Override compile time log levels of specific messages/modules

Started by Pavan Deolaseeover 9 years ago6 messages
#1Pavan Deolasee
pavan.deolasee@gmail.com
1 attachment(s)

While reviewing Jeff's notice_lock_waits patch, I came across his comment
about having a general facility for promoting selected LOG messages. So I
thought I should post it here, even though the patch is probably far from
being accepted in Postgres.

I recently wrote a patch for Postgres-XL to do exactly this and I found it
very useful, especially while debugging race conditions and problems with
ongoing sessions. I'm attaching a patch rebased on PG-master. It works, but
TBH I don't think it's anywhere close for being acceptable in PG. But I
hope this is good enough to show how this can be done with minimal changes
and spark ideas.

The patch uses some preprocessing and scripting magic to assign distinct
identifiers to each module (a subdir in the source code), to each file and
to each elog message. It then provides a set of functions by which an user
can increase/decrease/set log levels for either individual messages or all
messages within a source file or source module. The log levels can be
changed only for specific backends or all current or future backends. If
you configure with --enable-genmsgids switch, a MSGMODULES and MSGIDS file
is created in $srcdir, which can later be used to know ids assigned to
various modules/messages.

Now there are several problems with the patch:

- I don't know if it will work for anything other than clang and gcc (I've
tested those two, but only specific versions)
- The shared memory representation for msgs is not at all optimal and with
a large value of max_connections, it can quickly go out of control. But I
believe we can find a better representation without losing runtime
performance.
- The APIs themselves can be significantly improved.
- MSGMODULES and MSGIDS should probably be some sort of a view that user
can readily access without requiring access to the environment where it was
built.
- Right now it only supports changing levels for DEBUG[1-5] and LOG
messages. But that could be easily extended to cover INFO or NOTICE.
- The patch should probably have many more comments
- Finally, I don't know if better and more elegant ways exist to
automatically assigning module/file/msgids. I couldn't think of any without
making excessive changes to all source files and hence did what I did. That
does not mean better ways don't exists.

Thanks,
Pavan

--
Pavan Deolasee http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Attachments:

pg_msgids.patchapplication/octet-stream; name=pg_msgids.patchDownload
diff --git a/GNUmakefile.in b/GNUmakefile.in
index 15fba9f..1a5dbfc 100644
--- a/GNUmakefile.in
+++ b/GNUmakefile.in
@@ -50,6 +50,8 @@ clean:
 	rm -rf tmp_install/
 # Garbage from autoconf:
 	@rm -rf autom4te.cache/
+# Remove MSGIDS file too
+	rm -f MSGIDS
 
 # Important: distclean `src' last, otherwise Makefile.global
 # will be gone too soon.
@@ -62,6 +64,7 @@ distclean maintainer-clean:
 # Garbage from autoconf:
 	@rm -rf autom4te.cache/
 	rm -f config.cache config.log config.status GNUmakefile
+	rm -f MSGIDS MSGMODULES
 
 check check-tests installcheck installcheck-parallel installcheck-tests:
 	$(MAKE) -C src/test/regress $@
diff --git a/config/create_msgids.sh b/config/create_msgids.sh
new file mode 100755
index 0000000..eb3391b
--- /dev/null
+++ b/config/create_msgids.sh
@@ -0,0 +1,55 @@
+#/usr/bin/env sh
+
+#
+# Run this script when configuring with --enable-genmsgids
+#
+# Recurse through all subdiectories, collecting information about all subdirs
+# which has a Makefile/GNUmakefile with "subdir = <subdir_name>" entry and
+# assign a module_id for all such subdirs. The Makefile.global then looks up
+# this catalog and uses the module_id configured.
+#
+# The script assumes that every subdir's Makefile has a specific pattern of
+# "^subdir = .*", which is thankfully true for subdirs that we care for
+#
+# We could be a lot smarter than what we are doing, especially avoiding
+# module_id assignment for subdirs which do not directly compile any files with
+# elog() messages.
+#
+MSG_MODULE=0
+handle_dir()
+{
+	MSG_MODULE=`expr $2 + 1`
+	for subdir in `ls $1`; do
+		if [ -d $1/$subdir ]; then
+			makefile1=$1/$subdir/Makefile
+			makefile2=$1/$subdir/GNUmakefile
+			if [ -f $makefile1 ]; then
+				cat $makefile1 | grep -E "^subdir = " > /dev/null
+				if [ $? -ne 0 ]; then
+					if [ -f $makefile2 ]; then
+						cat $makefile2 | grep -E "^subdir = " > /dev/null
+						if [ $? -eq 0 ]; then
+							makefile=$makefile2
+						else
+							continue
+						fi
+					else
+						continue
+					fi
+				else
+					makefile=$makefile1
+				fi
+			else
+				continue
+			fi
+			cat $makefile | grep -E "^subdir = " > /dev/null
+			if [ $? -eq 0 ]; then
+				moduledir=`cat $makefile | grep -E '^subdir = '`
+				echo $moduledir:${MSG_MODULE}
+			fi
+			handle_dir "$1/$subdir" $MSG_MODULE
+		fi
+	done
+}
+
+handle_dir "." $MSG_MODULE
diff --git a/configure b/configure
index 45c8eef..f5c2562 100755
--- a/configure
+++ b/configure
@@ -719,6 +719,7 @@ with_tcl
 enable_thread_safety
 INCLUDES
 autodepend
+genmsgids
 TAS
 GCC
 CPP
@@ -818,6 +819,7 @@ with_wal_blocksize
 with_wal_segsize
 with_CC
 enable_depend
+enable_genmsgids
 enable_cassert
 enable_thread_safety
 with_tcl
@@ -5234,6 +5236,35 @@ else
 
 fi
 
+#
+# Automatic msgids generation
+#
+
+
+# Check whether --enable-genmsgids was given.
+if test "${enable_genmsgids+set}" = set; then :
+  enableval=$enable_genmsgids;
+  case $enableval in
+    yes)
+
+$as_echo "#define USE_MODULE_MSGIDS 1" >>confdefs.h
+     
+      genmsgids=yes
+	  $srcdir/config/create_msgids.sh > MSGMODULES 2>&1
+
+      ;;
+    no)
+      :
+      ;;
+    *)
+      as_fn_error $? "no argument expected for --enable-genmsgids option" "$LINENO" 5
+      ;;
+  esac
+
+else
+  enable_genmsgids=no
+
+fi
 
 
 
diff --git a/configure.in b/configure.in
index c878b4e..f06bef9 100644
--- a/configure.in
+++ b/configure.in
@@ -555,7 +555,6 @@ PGAC_ARG_BOOL(enable, depend, no, [turn on automatic dependency tracking],
               [autodepend=yes])
 AC_SUBST(autodepend)
 
-
 #
 # Enable assert checks
 #
@@ -565,6 +564,14 @@ PGAC_ARG_BOOL(enable, cassert, no, [enable assertion checks (for debugging)],
 
 
 #
+# Enable module msgids
+#
+PGAC_ARG_BOOL(enable, genmsgids, no, [enable module msgids (for debugging)],
+              [AC_DEFINE([USE_MODULE_MSGIDS], 1,
+                         [Define to 1 to build with module msgids.  (--enable-genmsgids)])])
+AC_SUBST(genmsgids)
+
+#
 # Include directories
 #
 ac_save_IFS=$IFS
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index c211a2d..660431b 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -756,7 +756,6 @@ recurse = $(foreach target,$(if $1,$1,$(standard_targets)),$(foreach subdir,$(if
 # $3: target to run in subdir (defaults to current element of $1)
 recurse_always = $(foreach target,$(if $1,$1,$(standard_always_targets)),$(foreach subdir,$(if $2,$2,$(ALWAYS_SUBDIRS)),$(eval $(call _create_recursive_target,$(target),$(subdir),$(if $3,$3,$(target))))))
 
-
 ##########################################################################
 #
 # Automatic dependency generation
@@ -873,3 +872,41 @@ coverage-clean:
 	rm -f `find . -name '*.gcda' -print`
 
 endif # enable_coverage
+
+genmsgids = @genmsgids@
+
+ifeq ($(genmsgids), yes)
+PREPROCESS.c = $(CC) $(CFLAGS) $(CPPFLAGS) -E -P
+
+PGXL_MSG_FILEID := 1
+PGXL_MSG_MODULE := $(shell cat $(top_builddir)/MSGMODULES | grep -E "^subdir = $(subdir):" | cut -d ':' -f 2)
+
+ifeq ($(PGXL_MSG_MODULE)no, no)
+PGXL_MSG_MODULE := 255
+endif
+
+ifeq ($(autodepend), yes)
+ifeq ($(GCC), yes)
+
+# GCC allows us to create object and dependency file in one invocation.
+%.o : %.c
+	@if test ! -d $(DEPDIR); then mkdir -p $(DEPDIR); fi
+	$(PREPROCESS.c) -DPGXL_MSG_MODULE=$(PGXL_MSG_MODULE) -DPGXL_MSG_FILEID=$(PGXL_MSG_FILEID)  -o $@.E $< -MMD -MP -MF $(DEPDIR)/$(*F).Po
+	-cat $@.E | grep -E "do .*errstart|do .*elog_start" >> $(top_srcdir)/MSGIDS
+	-rm -f $@.E
+	$(COMPILE.c) -DPGXL_MSG_MODULE=$(PGXL_MSG_MODULE) -DPGXL_MSG_FILEID=$(PGXL_MSG_FILEID) -o $@ $< -MMD -MP -MF $(DEPDIR)/$(*F).Po
+	$(eval PGXL_MSG_FILEID := $(shell expr $(PGXL_MSG_FILEID) + 1))
+endif # GCC
+else
+ifeq ($(GCC), yes)
+%.o : %.c
+	$(PREPROCESS.c) -DPGXL_MSG_MODULE=$(PGXL_MSG_MODULE) -DPGXL_MSG_FILEID=$(PGXL_MSG_FILEID)  -o $@.E $<
+	-cat $@.E | grep -E "do .*errstart|do .*elog_start" >> $(top_srcdir)/MSGIDS
+	-rm -f $@.E
+	$(COMPILE.c) -DPGXL_MSG_MODULE=$(PGXL_MSG_MODULE) -DPGXL_MSG_FILEID=$(PGXL_MSG_FILEID) -o $@ $<
+	$(eval PGXL_MSG_FILEID := $(shell expr $(PGXL_MSG_FILEID) + 1))
+endif # GCC
+
+endif # autodepend
+
+endif # enable_genmsgids
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index c04b17f..2b7459c 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -145,6 +145,10 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
 		size = add_size(size, ShmemBackendArraySize());
 #endif
 
+#ifdef USE_MODULE_MSGIDS
+		size = add_size(size, MsgModuleShmemSize());
+#endif
+
 		/* freeze the addin request size and include it */
 		addin_request_allowed = false;
 		size = add_size(size, total_addin_request);
@@ -263,6 +267,10 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
 		ShmemBackendArrayAllocation();
 #endif
 
+#ifdef USE_MODULE_MSGIDS
+	MsgModuleShmemInit();
+#endif
+
 	/* Initialize dynamic shared memory facilities. */
 	if (!IsUnderPostmaster)
 		dsm_postmaster_startup(shim);
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index b185c1b..3b9b055 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3804,6 +3804,10 @@ PostgresMain(int argc, char *argv[],
 	if (!IsUnderPostmaster)
 		PgStartTime = GetCurrentTimestamp();
 
+#ifdef USE_MODULE_MSGIDS
+	AtProcStart_MsgModule();
+#endif
+
 	/*
 	 * POSTGRES main processing loop begins here
 	 *
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 78d441d..84ffabf 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -73,7 +73,9 @@
 #include "postmaster/syslogger.h"
 #include "storage/ipc.h"
 #include "storage/proc.h"
+#include "storage/procarray.h"
 #include "tcop/tcopprot.h"
+#include "utils/builtins.h"
 #include "utils/guc.h"
 #include "utils/memutils.h"
 #include "utils/ps_status.h"
@@ -84,6 +86,10 @@
 
 static const char *err_gettext(const char *str) pg_attribute_format_arg(1);
 static void set_errdata_field(MemoryContextData *cxt, char **ptr, const char *str);
+#ifdef USE_MODULE_MSGIDS
+static void AtProcExit_MsgModule(int code, Datum arg);
+static bool pg_msgmodule_enable_disable(int32 pid, bool enable);
+#endif
 
 /* Global variables */
 ErrorContextCallback *error_context_stack = NULL;
@@ -178,12 +184,41 @@ static const char *useful_strerror(int errnum);
 static const char *get_errno_symbol(int errnum);
 static const char *error_severity(int elevel);
 static void append_with_tabs(StringInfo buf, const char *str);
-static bool is_log_level_output(int elevel, int log_min_level);
+static bool is_log_level_output(int elevel,
+#ifdef USE_MODULE_MSGIDS
+		int moduleid,
+		int fileid,
+		int msgid,
+#endif
+		int log_min_level);
 static void write_pipe_chunks(char *data, int len, int dest);
 static void write_csvlog(ErrorData *edata);
 static void setup_formatted_log_time(void);
 static void setup_formatted_start_time(void);
 
+#ifdef USE_MODULE_MSGIDS
+typedef struct MsgModuleCtlStruct
+{
+	bool	mm_enabled;
+	bool	mm_persistent;
+	char	mm_flags[FLEXIBLE_ARRAY_MEMBER];
+} MsgModuleCtlStruct;
+
+#define StartOfBackendFlags 	\
+	( \
+	  PGXL_MSG_MAX_MODULES * \
+	  PGXL_MSG_MAX_FILEIDS_PER_MODULE * \
+	  PGXL_MSG_MAX_MSGIDS_PER_FILE \
+	)
+
+#define SizeOfMsgModuleCtlStruct	\
+	( \
+	  offsetof(MsgModuleCtlStruct, mm_flags) + \
+	  StartOfBackendFlags + \
+	  MaxBackends \
+	)
+static MsgModuleCtlStruct *MsgModuleCtl;
+#endif
 
 /*
  * in_error_recursion_trouble --- are we at risk of infinite error recursion?
@@ -231,6 +266,9 @@ err_gettext(const char *str)
  */
 bool
 errstart(int elevel, const char *filename, int lineno,
+#ifdef USE_MODULE_MSGIDS
+		int moduleid, int fileid, int msgid,
+#endif
 		 const char *funcname, const char *domain)
 {
 	ErrorData  *edata;
@@ -290,7 +328,13 @@ errstart(int elevel, const char *filename, int lineno,
 	 */
 
 	/* Determine whether message is enabled for server log output */
-	output_to_server = is_log_level_output(elevel, log_min_messages);
+	output_to_server = is_log_level_output(elevel,
+#ifdef USE_MODULE_MSGIDS
+			moduleid,
+			fileid,
+			msgid,
+#endif
+			log_min_messages);
 
 	/* Determine whether message is enabled for client output */
 	if (whereToSendOutput == DestRemote && elevel != LOG_SERVER_ONLY)
@@ -1295,7 +1339,11 @@ getinternalerrposition(void)
  * evaluating the format arguments if we do that.)
  */
 void
-elog_start(const char *filename, int lineno, const char *funcname)
+elog_start(const char *filename, int lineno,
+#ifdef USE_MODULE_MSGIDS
+		int moduleid, int fileid, int msgid,
+#endif
+		const char *funcname)
 {
 	ErrorData  *edata;
 
@@ -1334,6 +1382,11 @@ elog_start(const char *filename, int lineno, const char *funcname)
 	edata->filename = filename;
 	edata->lineno = lineno;
 	edata->funcname = funcname;
+#ifdef USE_MODULE_MSGIDS
+	edata->moduleid = moduleid;
+	edata->fileid = fileid;
+	edata->msgid = msgid;
+#endif
 	/* errno is saved now so that error parameter eval can't change it */
 	edata->saved_errno = errno;
 
@@ -1357,7 +1410,12 @@ elog_finish(int elevel, const char *fmt,...)
 	 */
 	errordata_stack_depth--;
 	errno = edata->saved_errno;
-	if (!errstart(elevel, edata->filename, edata->lineno, edata->funcname, NULL))
+	if (!errstart(elevel, edata->filename, edata->lineno,
+#ifdef USE_MODULE_MSGIDS
+				edata->moduleid,
+				edata->fileid, edata->msgid,
+#endif
+				edata->funcname, NULL))
 		return;					/* nothing to do */
 
 	/*
@@ -1612,6 +1670,10 @@ ThrowErrorData(ErrorData *edata)
 	MemoryContext oldcontext;
 
 	if (!errstart(edata->elevel, edata->filename, edata->lineno,
+#ifdef USE_MODULE_MSGIDS
+				edata->moduleid,
+				edata->fileid, edata->msgid,
+#endif
 				  edata->funcname, NULL))
 		return;
 
@@ -1745,7 +1807,12 @@ pg_re_throw(void)
 		 */
 		if (IsPostmasterEnvironment)
 			edata->output_to_server = is_log_level_output(FATAL,
-														  log_min_messages);
+#ifdef USE_MODULE_MSGIDS
+					0,
+					0,
+					0,
+#endif
+					log_min_messages);
 		else
 			edata->output_to_server = (FATAL >= log_min_messages);
 		if (whereToSendOutput == DestRemote)
@@ -2781,7 +2848,13 @@ write_csvlog(ErrorData *edata)
 	appendStringInfoChar(&buf, ',');
 
 	/* user query --- only reported if not disabled by the caller */
-	if (is_log_level_output(edata->elevel, log_min_error_statement) &&
+	if (is_log_level_output(edata->elevel,
+#ifdef USE_MODULE_MSGIDS
+				edata->moduleid,
+				edata->fileid,
+				edata->msgid,
+#endif
+				log_min_error_statement) &&
 		debug_query_string != NULL &&
 		!edata->hide_stmt)
 		print_stmt = true;
@@ -2939,7 +3012,13 @@ send_message_to_server_log(ErrorData *edata)
 	/*
 	 * If the user wants the query that generated this error logged, do it.
 	 */
-	if (is_log_level_output(edata->elevel, log_min_error_statement) &&
+	if (is_log_level_output(edata->elevel,
+#ifdef USE_MODULE_MSGIDS
+				edata->moduleid,
+				edata->fileid,
+				edata->msgid,
+#endif
+				log_min_error_statement) &&
 		debug_query_string != NULL &&
 		!edata->hide_stmt)
 	{
@@ -3687,6 +3766,70 @@ write_stderr(const char *fmt,...)
 	va_end(ap);
 }
 
+#ifdef USE_MODULE_MSGIDS
+static int
+get_overridden_log_level(int moduleid, int fileid, int msgid, int origlevel)
+{
+	uint32 position;
+	int value, relative, change, elevel;
+
+	/*
+	 * The shared memory may not set during init processing or in a stand alone
+	 * backend.
+	 */
+	if (!IsPostmasterEnvironment || IsInitProcessingMode())
+		return origlevel;
+
+	if (!MsgModuleCtl->mm_enabled)
+		return origlevel;
+
+	/*
+	 * Reject invalid bounds
+	 */
+	if ((moduleid <= 0 || moduleid >= PGXL_MSG_MAX_MODULES) ||
+		(fileid <= 0 || fileid >= PGXL_MSG_MAX_FILEIDS_PER_MODULE) ||
+		(msgid <= 0 || msgid >= PGXL_MSG_MAX_MSGIDS_PER_FILE))
+		return origlevel;
+
+	if (origlevel < DEBUG5 || origlevel >= LOG)
+		return origlevel;
+
+	if (!(MsgModuleCtl->mm_flags[StartOfBackendFlags + MyBackendId]))
+		return origlevel;
+
+	elevel = origlevel;
+
+	/*
+	 * Get the overridden log level and return it back
+	 */
+	position = (moduleid - 1) * PGXL_MSG_MAX_FILEIDS_PER_MODULE *
+			PGXL_MSG_MAX_MSGIDS_PER_FILE +
+			(fileid - 1) * PGXL_MSG_MAX_MSGIDS_PER_FILE +
+			(msgid - 1);
+	
+	/* Read once */
+	value = MsgModuleCtl->mm_flags[position];
+
+	relative = value & 0x80;
+	change = value & 0x7f;
+
+	if (value)
+	{
+		if (relative)
+		{
+			elevel = elevel + change;
+			if (elevel < DEBUG5)
+				elevel = DEBUG5;
+			else if (elevel >= LOG)
+				elevel = LOG;
+		}
+		else
+			elevel = change;
+		return elevel;
+	}
+	return origlevel;
+}
+#endif
 
 /*
  * is_log_level_output -- is elevel logically >= log_min_level?
@@ -3697,9 +3840,34 @@ write_stderr(const char *fmt,...)
  * test is correct for testing whether the message should go to the client.
  */
 static bool
-is_log_level_output(int elevel, int log_min_level)
+is_log_level_output(int elevel,
+#ifdef USE_MODULE_MSGIDS
+		int moduleid,
+		int fileid,
+		int msgid,
+#endif
+		int log_min_level)
 {
-	if (elevel == LOG || elevel == LOG_SERVER_ONLY)
+#ifdef USE_MODULE_MSGIDS
+	/* 
+	 * Check if the message's compile time value has been changed during the
+	 * run time.
+	 *
+	 * Currently, we only support increasing the log level of messages and that
+	 * too only for deciding whether the message should go to the server log or
+	 * not. A message which would otherwise not qualify to go to the server
+	 * log, thus can be forced to be logged. 
+	 *
+	 * In future, we may also want to go otherway round i.e. supressing a log
+	 * message or also change severity of log messages. The latter may
+	 * especially be useful to turn some specific ERROR messages into FATAL or
+	 * PANIC to be able to get a core dump for analysis.
+	 */
+	elevel = get_overridden_log_level(moduleid, fileid, msgid,
+			elevel);
+#endif
+
+	if (elevel == LOG || elevel == COMMERROR)
 	{
 		if (log_min_level == LOG || log_min_level <= ERROR)
 			return true;
@@ -3741,3 +3909,362 @@ trace_recovery(int trace_level)
 
 	return trace_level;
 }
+
+#ifdef USE_MODULE_MSGIDS
+Size
+MsgModuleShmemSize(void)
+{
+	/*
+	 * One byte per message to store overridden log level.
+	 * !!TODO We don't really need a byte and a few bits would be enough. So
+	 * look for improving this/
+	 *
+	 * What we have done is a very simplisitic representation of msg-ids. The
+	 * overall memory requirement of this representation is too large as
+	 * compared to the actual number of msgs. For example, both
+	 * PGXL_MSG_MAX_MSGIDS_PER_FILE and PGXL_MSG_MAX_FILEIDS_PER_MODULE are set
+	 * to the largest value that any one module uses, but the actual values are
+	 * much smaller
+	 *
+	 * Also have a separate area for MaxBackends so that caller can selectively
+	 * change logging for individual backends. Note that we don't support
+	 * having different logging levels for different backends. So either a
+	 * backend honours the module/file/msg level overrides or it does not.
+	 */
+	return SizeOfMsgModuleCtlStruct;
+}
+
+void
+MsgModuleShmemInit(void)
+{
+	bool found;
+
+	MsgModuleCtl = ShmemInitStruct("Message Module Struct",
+								   SizeOfMsgModuleCtlStruct,
+								   &found);
+}
+
+void
+AtProcStart_MsgModule(void)
+{
+	if (MsgModuleCtl->mm_persistent)
+		MsgModuleCtl->mm_flags[StartOfBackendFlags + MyBackendId] = true;
+	else
+		MsgModuleCtl->mm_flags[StartOfBackendFlags + MyBackendId] = false;
+	before_shmem_exit(AtProcExit_MsgModule, 0);
+}
+
+static bool
+pg_msgmodule_internal(int32 moduleid, int32 fileid, int32 msgid, char value)
+{
+	uint32 start_position;
+	uint32 len;
+
+	if (!superuser())
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+			 (errmsg("must be superuser to change elog message level"))));
+
+
+	if (moduleid <= 0 || moduleid >= PGXL_MSG_MAX_MODULES)
+		ereport(ERROR, (errmsg_internal("Invalid module id %d, allowed values 1-%d",
+						moduleid, PGXL_MSG_MAX_MODULES)));
+
+	if (fileid == -1)
+	{
+		/*
+		 * All messages in the given module to be overridden with the given
+		 * level
+		 */
+		len = PGXL_MSG_MAX_FILEIDS_PER_MODULE * PGXL_MSG_MAX_MSGIDS_PER_FILE;
+		start_position = (moduleid - 1) * len;
+		memset(MsgModuleCtl->mm_flags + start_position, value, len);
+		goto success;
+	}
+	else
+	{
+		if (fileid <= 0 || fileid >= PGXL_MSG_MAX_FILEIDS_PER_MODULE)
+			ereport(ERROR, (errmsg_internal("Invalid file id %d, allowed values 1-%d",
+							fileid, PGXL_MSG_MAX_FILEIDS_PER_MODULE)));
+		
+		/*
+		 * All messages in the given <module, file> to be overridden with the
+		 * given level
+		 */
+		if (msgid == -1)
+		{
+			len = PGXL_MSG_MAX_MSGIDS_PER_FILE;
+			start_position = ((moduleid - 1) * PGXL_MSG_MAX_FILEIDS_PER_MODULE * PGXL_MSG_MAX_MSGIDS_PER_FILE) +
+						(fileid - 1) * PGXL_MSG_MAX_MSGIDS_PER_FILE;
+			memset(MsgModuleCtl->mm_flags + start_position, value, len);
+			goto success;
+		}
+
+		if (msgid <= 0 || msgid >= PGXL_MSG_MAX_MSGIDS_PER_FILE)
+			ereport(ERROR, (errmsg_internal("Invalid msg id %d, allowed values 1-%d",
+							fileid, PGXL_MSG_MAX_MSGIDS_PER_FILE)));
+
+		/*
+		 * Deal with a specific <module, file, msg>
+		 */
+		len = sizeof (char);
+		start_position = ((moduleid - 1) * PGXL_MSG_MAX_FILEIDS_PER_MODULE * PGXL_MSG_MAX_MSGIDS_PER_FILE) +
+			((fileid - 1) * PGXL_MSG_MAX_MSGIDS_PER_FILE) +
+			(msgid - 1);
+		memset(MsgModuleCtl->mm_flags + start_position, value, len);
+		goto success;
+	}
+
+success:
+	return true;
+}
+
+Datum
+pg_msgmodule_set(PG_FUNCTION_ARGS)
+{
+	int32 moduleid = PG_GETARG_INT32(0);
+	int32 fileid = PG_GETARG_INT32(1);
+	int32 msgid = PG_GETARG_INT32(2);
+	const char *levelstr = PG_GETARG_CSTRING(3);
+	int32 level;
+
+	if (!superuser())
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+			 (errmsg("must be superuser to change elog message level"))));
+
+	/*
+	 * The only accepted values for the log levels are - LOG, DEBUG[1-5] and
+	 * DEFAULT
+	 */
+	if (strcasecmp(levelstr, "LOG") == 0)
+		level = LOG;
+	else if (strcasecmp(levelstr, "DEFAULT") == 0)
+		level = 0;
+	else if (strcasecmp(levelstr, "DEBUG1") == 0)
+		level = DEBUG1;
+	else if (strcasecmp(levelstr, "DEBUG2") == 0)
+		level = DEBUG2;
+	else if (strcasecmp(levelstr, "DEBUG3") == 0)
+		level = DEBUG3;
+	else if (strcasecmp(levelstr, "DEBUG4") == 0)
+		level = DEBUG4;
+	else if (strcasecmp(levelstr, "DEBUG5") == 0)
+		level = DEBUG5;
+	else
+		ereport(ERROR,
+				(errcode(ERRCODE_INTERNAL_ERROR),
+				 (errmsg("Invalid value \"%s\" for log level", levelstr))));
+
+	PG_RETURN_BOOL(pg_msgmodule_internal(moduleid, fileid, msgid, level));
+}
+
+Datum
+pg_msgmodule_change(PG_FUNCTION_ARGS)
+{
+	int32 moduleid = PG_GETARG_INT32(0);
+	int32 fileid = PG_GETARG_INT32(1);
+	int32 msgid = PG_GETARG_INT32(2);
+	int32 change = PG_GETARG_INT32(3);
+	int level;
+
+	if (!superuser())
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+			 (errmsg("must be superuser to change elog message level"))));
+	
+	if ((change < (DEBUG5 - LOG)) || (change > (LOG - DEBUG5)))
+		ereport(ERROR,
+				(errcode(ERRCODE_INTERNAL_ERROR),
+			 (errmsg("accepted values are between %d and +%d", DEBUG5 - LOG, LOG -
+					 DEBUG5))));
+
+	level = 0x80 | change;
+
+	PG_RETURN_BOOL(pg_msgmodule_internal(moduleid, fileid, msgid, level));
+}
+
+static bool
+pg_msgmodule_enable_disable(int32 pid, bool enable)
+{
+	BackendId backendId;
+	/*
+	 * pid == -1 implies the action is applied for all backends
+	 */
+	if (pid != -1)
+	{
+		volatile PGPROC *proc = BackendPidGetProc(pid);
+		if (proc == NULL)
+			ereport(ERROR,
+					(errcode(ERRCODE_INTERNAL_ERROR),
+					 (errmsg("PID %d is not a PostgreSQL server process", pid))));
+		backendId = proc->backendId;
+		MsgModuleCtl->mm_flags[StartOfBackendFlags + backendId] = enable ? true :
+			false;
+	}
+	else
+	{
+		/*
+		 * All backends should not honour the current settings
+		 */
+		memset(MsgModuleCtl->mm_flags + StartOfBackendFlags, true,
+				MaxBackends);
+	}
+
+	/*
+	 * If we are disabling the last backend and no new backends are going to
+	 * participate in the facility, then just disable the entire facility so
+	 * that subsequent backends can quickly bail out
+	 *
+	 * XXX There is a race possible here if another call to turn on the logging
+	 * comes just after we had passed over tje slot. We should possibly protect
+	 * access to the shared memory, but doesn't seem like worth the effort
+	 * right now
+	 */
+	if (!enable && !MsgModuleCtl->mm_persistent)
+	{
+		int i;
+		for (i = 0; i < MaxBackends; i++)
+			if (MsgModuleCtl->mm_flags[StartOfBackendFlags + i])
+				break;
+		if (i == MaxBackends)
+			MsgModuleCtl->mm_enabled = false;
+	}
+	else
+		MsgModuleCtl->mm_enabled = true;
+
+	return true;
+}
+
+Datum
+pg_msgmodule_enable(PG_FUNCTION_ARGS)
+{
+	int32 pid = PG_GETARG_INT32(0);
+
+	if (!superuser())
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+			 (errmsg("must be superuser to change elog message level"))));
+
+	PG_RETURN_BOOL(pg_msgmodule_enable_disable(pid, true));
+}
+
+Datum
+pg_msgmodule_disable(PG_FUNCTION_ARGS)
+{
+	int32 pid = PG_GETARG_INT32(0);
+
+	if (!superuser())
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+			 (errmsg("must be superuser to change elog message level"))));
+
+	PG_RETURN_BOOL(pg_msgmodule_enable_disable(pid, false));
+}
+
+/*
+ * pg_msgmodule_enable_all(bool persistent)
+ *
+ * All processes to start honouring the current settings of overridden log
+ * levels. If "persistent" is set to true, then all future processes will also
+ * honour the settings.
+ */
+Datum
+pg_msgmodule_enable_all(PG_FUNCTION_ARGS)
+{
+	bool persistent = PG_GETARG_BOOL(0);
+
+	if (!superuser())
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+			 (errmsg("must be superuser to change elog message level"))));
+
+	MsgModuleCtl->mm_persistent = persistent;
+	if (persistent)
+	{
+		MsgModuleCtl->mm_enabled = true;
+		memset(MsgModuleCtl->mm_flags + StartOfBackendFlags, true,
+				MaxBackends);
+		PG_RETURN_BOOL(true);
+	}
+	else
+		PG_RETURN_BOOL(pg_msgmodule_enable_disable(-1, true));
+}
+
+/*
+ * Disable all current and future processes from honouring the overridden log
+ * levels
+ */
+Datum
+pg_msgmodule_disable_all(PG_FUNCTION_ARGS)
+{
+	if (!superuser())
+		ereport(ERROR,
+				(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+			 (errmsg("must be superuser to change elog message level"))));
+
+	MsgModuleCtl->mm_enabled = false;
+	MsgModuleCtl->mm_persistent = false;
+	memset(MsgModuleCtl->mm_flags + StartOfBackendFlags, false,
+			MaxBackends);
+	PG_RETURN_BOOL(true);
+}
+
+/*
+ * Handle proc exit. If persistent flag is not set then also check if we are
+ * the last process using the facility and if so, disable it. The current log
+ * settings are retained and will be used when newer processes are enabled for
+ * the facility
+ */
+static void
+AtProcExit_MsgModule(int code, Datum arg)
+{
+	MsgModuleCtl->mm_flags[StartOfBackendFlags + MyBackendId] = false;
+	if (!MsgModuleCtl->mm_persistent)
+	{
+		int i;
+		for (i = 0; i < MaxBackends; i++)
+			if (MsgModuleCtl->mm_flags[StartOfBackendFlags + i])
+				break;
+		if (i == MaxBackends)
+			MsgModuleCtl->mm_enabled = false;
+	}
+}
+#else
+Datum
+pg_msgmodule_set(PG_FUNCTION_ARGS)
+{
+	ereport(ERROR, (errmsg_internal("Module msgid support not available. "
+					"Please recompile with --enable-genmsgids")));
+}
+Datum
+pg_msgmodule_change(PG_FUNCTION_ARGS)
+{
+	ereport(ERROR, (errmsg_internal("Module msgid support not available. "
+					"Please recompile with --enable-genmsgids")));
+}
+Datum
+pg_msgmodule_enable(PG_FUNCTION_ARGS)
+{
+	ereport(ERROR, (errmsg_internal("Module msgid support not available. "
+					"Please recompile with --enable-genmsgids")));
+}
+Datum
+pg_msgmodule_disable(PG_FUNCTION_ARGS)
+{
+	ereport(ERROR, (errmsg_internal("Module msgid support not available. "
+					"Please recompile with --enable-genmsgids")));
+}
+Datum
+pg_msgmodule_enable_all(PG_FUNCTION_ARGS)
+{
+	ereport(ERROR, (errmsg_internal("Module msgid support not available. "
+					"Please recompile with --enable-genmsgids")));
+}
+Datum
+pg_msgmodule_disable_all(PG_FUNCTION_ARGS)
+{
+	ereport(ERROR, (errmsg_internal("Module msgid support not available. "
+					"Please recompile with --enable-genmsgids")));
+}
+#endif
diff --git a/src/common/Makefile b/src/common/Makefile
index 72b7369..84735f3 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -74,8 +74,15 @@ libpgcommon_srv.a: $(OBJS_SRV)
 # their *.o siblings as well, which do have proper dependencies.  It's
 # a hack that might fail someday if there is a *_srv.o without a
 # corresponding *.o, but it works for now.
+ifeq ($(genmsgids), yes)
+PGXL_MSG_FILEID := 1
+%_srv.o: %.c %.o
+	$(CC) $(CFLAGS) -DPGXL_MSG_MODULE=$(PGXL_MSG_MODULE) -DPGXL_MSG_FILEID=$(PGXL_MSG_FILEID) $(subst -DFRONTEND,, $(CPPFLAGS)) -c $< -o $@
+	$(eval PGXL_MSG_FILEID := $(shell expr $(PGXL_MSG_FILEID) + 1))
+else
 %_srv.o: %.c %.o
 	$(CC) $(CFLAGS) $(subst -DFRONTEND ,, $(CPPFLAGS)) -c $< -o $@
+endif
 
 $(OBJS_SRV): | submake-errcodes
 
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index e2d08ba..e41f1fe 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -5344,6 +5344,21 @@ DESCR("pg_controldata recovery state information as a function");
 DATA(insert OID = 3444 ( pg_control_init PGNSP PGUID 12 1 0 0 0 f f f f t f v s 0 0 2249 "" "{23,23,23,23,23,23,23,23,23,16,16,16,23}" "{o,o,o,o,o,o,o,o,o,o,o,o,o}" "{max_data_alignment,database_block_size,blocks_per_segment,wal_block_size,bytes_per_wal_segment,max_identifier_length,max_index_columns,max_toast_chunk_size,large_object_chunk_size,bigint_timestamps,float4_pass_by_value,float8_pass_by_value,data_page_checksum_version}" _null_ _null_ pg_control_init _null_ _null_ _null_ ));
 DESCR("pg_controldata init state information as a function");
 
+#ifdef USE_MODULE_MSGIDS
+DATA(insert OID = 6015 ( pg_msgmodule_set PGNSP PGUID 12 1 1 0 0 f f f f t t i s 4 0 16 "20 20 20 2275" _null_ _null_ _null_ _null_ _null_ pg_msgmodule_set _null_ _null_ _null_ ));
+DESCR("set debugging level for module/file/msg");
+DATA(insert OID = 6016 ( pg_msgmodule_change PGNSP PGUID 12 1 1 0 0 f f f f t t i s 4 0 16 "20 20 20 20" _null_ _null_ _null_ _null_ _null_ pg_msgmodule_change _null_ _null_ _null_ ));
+DESCR("change debugging level for module/file/msg");
+DATA(insert OID = 6017 ( pg_msgmodule_enable PGNSP PGUID 12 1 1 0 0 f f f f t t i s 1 0 16 "20" _null_ _null_ _null_ _null_ _null_ pg_msgmodule_enable _null_ _null_ _null_ ));
+DESCR("pid to honour overriden log levels");
+DATA(insert OID = 6018 ( pg_msgmodule_disable PGNSP PGUID 12 1 1 0 0 f f f f t t i s 1 0 16 "20" _null_ _null_ _null_ _null_ _null_ pg_msgmodule_disable _null_ _null_ _null_ ));
+DESCR("pid to ignore overriden log levels");
+DATA(insert OID = 6019 ( pg_msgmodule_enable_all PGNSP PGUID 12 1 1 0 0 f f f f t t i s 1 0 16 "16" _null_ _null_ _null_ _null_ _null_ pg_msgmodule_enable_all _null_ _null_ _null_ ));
+DESCR("all current/future processes to honour overriden log levels");
+DATA(insert OID = 6020 ( pg_msgmodule_disable_all PGNSP PGUID 12 1 1 0 0 f f f f t t i s 0 0 16 "" _null_ _null_ _null_ _null_ _null_ pg_msgmodule_disable_all _null_ _null_ _null_ ));
+DESCR("all processes to ignore overriden log levels");
+#endif
+
 /*
  * Symbolic values for provolatile column: these indicate whether the result
  * of a function is dependent *only* on the values of its explicit arguments,
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index b621ff2..b5f88db 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -790,6 +790,9 @@
 /* Define to 1 to build with assertion checks. (--enable-cassert) */
 #undef USE_ASSERT_CHECKING
 
+/* Define to 1 to build with module msgids. (--enable-genmsgids) */
+#undef USE_MODULE_MSGIDS
+
 /* Define to 1 to build with Bonjour support. (--with-bonjour) */
 #undef USE_BONJOUR
 
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 2ae212a..50bc8db 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -1332,4 +1332,12 @@ extern Datum pg_prepared_statement(PG_FUNCTION_ARGS);
 /* utils/mmgr/portalmem.c */
 extern Datum pg_cursor(PG_FUNCTION_ARGS);
 
+#ifdef USE_MODULE_MSGIDS
+extern Datum pg_msgmodule_set(PG_FUNCTION_ARGS);
+extern Datum pg_msgmodule_change(PG_FUNCTION_ARGS);
+extern Datum pg_msgmodule_enable(PG_FUNCTION_ARGS);
+extern Datum pg_msgmodule_disable(PG_FUNCTION_ARGS);
+extern Datum pg_msgmodule_enable_all(PG_FUNCTION_ARGS);
+extern Datum pg_msgmodule_disable_all(PG_FUNCTION_ARGS);
+#endif
 #endif   /* BUILTINS_H */
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index f4ff03e..fe2dd50 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -100,6 +100,26 @@
  * prevents gcc from making the unreachability deduction at optlevel -O0.
  *----------
  */
+#ifdef USE_MODULE_MSGIDS
+#ifdef HAVE__BUILTIN_CONSTANT_P
+#define ereport_domain(elevel, domain, rest)	\
+	do { \
+		if (errstart(elevel, __FILE__, __LINE__, PGXL_MSG_MODULE, PGXL_MSG_FILEID, __COUNTER__, PG_FUNCNAME_MACRO, domain)) \
+			errfinish rest; \
+		if (__builtin_constant_p(elevel) && (elevel) >= ERROR) \
+			pg_unreachable(); \
+	} while(0)
+#else							/* !HAVE__BUILTIN_CONSTANT_P */
+#define ereport_domain(elevel, domain, rest)	\
+	do { \
+		const int elevel_ = (elevel); \
+		if (errstart(elevel, __FILE__, __LINE__, PGXL_MSG_MODULE, PGXL_MSG_FILEID, __COUNTER__, PG_FUNCNAME_MACRO, domain)) \
+			errfinish rest; \
+		if (elevel_ >= ERROR) \
+			pg_unreachable(); \
+	} while(0)
+#endif   /* HAVE__BUILTIN_CONSTANT_P */
+#else
 #ifdef HAVE__BUILTIN_CONSTANT_P
 #define ereport_domain(elevel, domain, rest)	\
 	do { \
@@ -112,12 +132,13 @@
 #define ereport_domain(elevel, domain, rest)	\
 	do { \
 		const int elevel_ = (elevel); \
-		if (errstart(elevel_, __FILE__, __LINE__, PG_FUNCNAME_MACRO, domain)) \
+		if (errstart(elevel, __FILE__, __LINE__, PG_FUNCNAME_MACRO, domain)) \
 			errfinish rest; \
 		if (elevel_ >= ERROR) \
 			pg_unreachable(); \
 	} while(0)
 #endif   /* HAVE__BUILTIN_CONSTANT_P */
+#endif
 
 #define ereport(elevel, rest)	\
 	ereport_domain(elevel, TEXTDOMAIN, rest)
@@ -125,7 +146,11 @@
 #define TEXTDOMAIN NULL
 
 extern bool errstart(int elevel, const char *filename, int lineno,
-		 const char *funcname, const char *domain);
+#ifdef USE_MODULE_MSGIDS
+		 int moduleid, int fileid, int msgid,
+#endif
+		 const char *funcname, const char *domain
+		 );
 extern void errfinish(int dummy,...);
 
 extern int	errcode(int sqlerrcode);
@@ -188,6 +213,7 @@ extern int	getinternalerrposition(void);
  *		elog(ERROR, "portal \"%s\" not found", stmt->portalname);
  *----------
  */
+#ifdef USE_MODULE_MSGIDS
 #ifdef HAVE__VA_ARGS
 /*
  * If we have variadic macros, we can give the compiler a hint about the
@@ -198,6 +224,34 @@ extern int	getinternalerrposition(void);
 #ifdef HAVE__BUILTIN_CONSTANT_P
 #define elog(elevel, ...)  \
 	do { \
+		elog_start(__FILE__, __LINE__, PGXL_MSG_MODULE, PGXL_MSG_FILEID, __COUNTER__, PG_FUNCNAME_MACRO); \
+		elog_finish(elevel, __VA_ARGS__); \
+		if (__builtin_constant_p(elevel) && (elevel) >= ERROR) \
+			pg_unreachable(); \
+	} while(0)
+#else							/* !HAVE__BUILTIN_CONSTANT_P */
+#define elog(elevel, ...)  \
+	do { \
+		int		elevel_; \
+		elog_start(__FILE__, __LINE__, PGXL_MSG_MODULE, PGXL_MSG_FILEID, __COUNTER__, PG_FUNCNAME_MACRO); \
+		elevel_ = (elevel); \
+		elog_finish(elevel_, __VA_ARGS__); \
+		if (elevel_ >= ERROR) \
+			pg_unreachable(); \
+	} while(0)
+#endif   /* HAVE__BUILTIN_CONSTANT_P */
+#else							/* !HAVE__VA_ARGS */
+#define elog  \
+	do { \
+		elog_start(__FILE__, __LINE__, PGXL_MSG_MODULE, PGXL_MSG_FILEID, __COUNTER__, PG_FUNCNAME_MACRO); \
+	} while (0); \
+	elog_finish
+#endif   /* HAVE__VA_ARGS */
+#else
+#ifdef HAVE__VA_ARGS
+#ifdef HAVE__BUILTIN_CONSTANT_P
+#define elog(elevel, ...)  \
+	do { \
 		elog_start(__FILE__, __LINE__, PG_FUNCNAME_MACRO); \
 		elog_finish(elevel, __VA_ARGS__); \
 		if (__builtin_constant_p(elevel) && (elevel) >= ERROR) \
@@ -216,11 +270,19 @@ extern int	getinternalerrposition(void);
 #endif   /* HAVE__BUILTIN_CONSTANT_P */
 #else							/* !HAVE__VA_ARGS */
 #define elog  \
-	elog_start(__FILE__, __LINE__, PG_FUNCNAME_MACRO), \
+	do { \
+		elog_start(__FILE__, __LINE__, PG_FUNCNAME_MACRO); \
+	} while (0); \
 	elog_finish
 #endif   /* HAVE__VA_ARGS */
+#endif
 
-extern void elog_start(const char *filename, int lineno, const char *funcname);
+extern void elog_start(const char *filename, int lineno,
+#ifdef USE_MODULE_MSGIDS
+		int moduleid, int flieid, int msgid,
+#endif
+		const char *funcname
+		);
 extern void elog_finish(int elevel, const char *fmt,...) pg_attribute_printf(2, 3);
 
 
@@ -354,11 +416,26 @@ typedef struct ErrorData
 	int			internalpos;	/* cursor index into internalquery */
 	char	   *internalquery;	/* text of internally-generated query */
 	int			saved_errno;	/* errno at entry */
+#ifdef USE_MODULE_MSGIDS
+	int			moduleid;
+	int			fileid;
+	int			msgid;			/* msgid */
+#endif
 
 	/* context containing associated non-constant strings */
 	struct MemoryContextData *assoc_context;
 } ErrorData;
 
+#ifdef USE_MODULE_MSGIDS
+#define PGXL_MSG_MAX_MODULES			256
+#define PGXL_MSG_MAX_FILEIDS_PER_MODULE	100
+#define PGXL_MSG_MAX_MSGIDS_PER_FILE	300
+
+extern Size MsgModuleShmemSize(void);
+extern void MsgModuleShmemInit(void);
+extern void AtProcStart_MsgModule(void);
+#endif
+
 extern void EmitErrorReport(void);
 extern ErrorData *CopyErrorData(void);
 extern void FreeErrorData(ErrorData *edata);
diff --git a/src/port/Makefile b/src/port/Makefile
index bc9b63a..2aca390 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -76,8 +76,15 @@ libpgport_srv.a: $(OBJS_SRV)
 # a hack that might fail someday if there is a *_srv.o without a
 # corresponding *.o, but it works for now (and those would probably go
 # into src/backend/port/ anyway).
+ifeq ($(genmsgids), yes)
+PGXL_MSG_FILEID := 1
+%_srv.o: %.c %.o
+	$(CC) $(CFLAGS) -DPGXL_MSG_MODULE=$(PGXL_MSG_MODULE) -DPGXL_MSG_FILEID=$(PGXL_MSG_FILEID) $(subst -DFRONTEND,, $(CPPFLAGS)) -c $< -o $@
+	$(eval PGXL_MSG_FILEID := $(shell expr $(PGXL_MSG_FILEID) + 1))
+else
 %_srv.o: %.c %.o
 	$(CC) $(CFLAGS) $(subst -DFRONTEND,, $(CPPFLAGS)) -c $< -o $@
+endif
 
 $(OBJS_SRV): | submake-errcodes
 
#2Craig Ringer
craig.ringer@2ndquadrant.com
In reply to: Pavan Deolasee (#1)
Re: Override compile time log levels of specific messages/modules

On 6 Sep. 2016 17:28, "Pavan Deolasee" <pavan.deolasee@gmail.com> wrote:

The patch uses some preprocessing and scripting magic to assign distinct

identifiers to each module (a subdir in the source code), to each file and
to each elog message. It then provides a set of functions by which an user
can increase/decrease/set log levels for either individual messages or all
messages within a source file or source module. The log levels can be
changed only for specific backends or all current or future backends. If
you configure with --enable-genmsgids switch, a MSGMODULES and MSGIDS file
is created in $srcdir, which can later be used to know ids assigned to
various modules/messages.

I think it's worth looking at how Java handles logging. We can't achieve an
exact parallel in C as we don't really have a class hierarchy ... but we do
have subsystems roughly grouped by file and directory structure.

Being able to promote/demote these or selective lower the log level
threshold on a directory, file, function and line level would be
exceedingly handy. Pretty much all of that can be magic'd up from macro
output so hopefully no postprocessing should be needed. (Though only gcc
has _FUNC_ or _FUNCTION_ I think so we'd have selective support there.)

#3Pavan Deolasee
pavan.deolasee@gmail.com
In reply to: Craig Ringer (#2)
Re: Override compile time log levels of specific messages/modules

On Tue, Sep 6, 2016 at 3:06 PM, Craig Ringer <craig.ringer@2ndquadrant.com>
wrote:

I think it's worth looking at how Java handles logging. We can't achieve
an exact parallel in C as we don't really have a class hierarchy ... but we
do have subsystems roughly grouped by file and directory structure.

Sure. In some large, popular enterprise softwares I've worked with many

years ago, we used to define modules within each source file and then
manually assign distinct integer IDs to each message and then provide
various utilities to turn on/off specific messages or range of messages
within a module.

Being able to promote/demote these or selective lower the log level
threshold on a directory, file, function and line level would be
exceedingly handy. Pretty much all of that can be magic'd up from macro
output so hopefully no postprocessing should be needed. (Though only gcc
has _FUNC_ or _FUNCTION_ I think so we'd have selective support there.)

Well, __FUNCTION__ expands to function name and lookup based on string
identifiers will always be costly, especially if you want to sprinkle the
code with lot many debug messages. Same with __FILE__. I believe we need an
unique integer constant to make it a fast, O(1) lookup. I couldn't find any
other method to do that when I wrote the facility and hence did what I did.

Thanks,
Pavan
--
Pavan Deolasee http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

#4Craig Ringer
craig.ringer@2ndquadrant.com
In reply to: Pavan Deolasee (#1)
Re: Override compile time log levels of specific messages/modules

On 6 Sep. 2016 17:57, "Pavan Deolasee" <pavan.deolasee@gmail.com> wrote:

On Tue, Sep 6, 2016 at 3:06 PM, Craig Ringer <craig.ringer@2ndquadrant.com>

wrote:

I think it's worth looking at how Java handles logging. We can't achieve

an exact parallel in C as we don't really have a class hierarchy ... but we
do have subsystems roughly grouped by file and directory structure.

Sure. In some large, popular enterprise softwares I've worked with many

years ago, we used to define modules within each source file and then
manually assign distinct integer IDs to each message and then provide
various utilities to turn on/off specific messages or range of messages
within a module.

Yeah, there same has been discussed for Pg many times and firmly shot down
each time. I certainly see the argument against making specific elog calls
part of a sort of api people might expect to be stable ish. That's what we
have ERRCODE / SQLSTATE for right? Concerns about backpatch pain were also
raised. Probably more. I'm not especially for or against it myself, just
saying that based on past discussion such a proposal isn't likely to fly.

I think something automatic that we clearly define as unstable and not to
be relied upon would be preferable. Plus we already have much of the
infrastructure in elog.c as used by errcontext etc.

Well, __FUNCTION__ expands to function name and lookup based on string

identifiers will always be costly, especially if you want to sprinkle the
code with lot many debug messages. Same with __FILE__. I believe we need an
unique integer constant to make it a fast, O(1) lookup. I couldn't find any
other method to do that when I wrote the facility and hence did what I did.

Yeah. High performance logging and filtering isnt easy. Looking at existing
C logging libraries with dynamic filtering might be informative. Though I
can't imagine there being agreement to adopt one, this must be a reasonably
solved problem...

Show quoted text

Thanks,
Pavan
--
Pavan Deolasee http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

#5Jim Nasby
Jim.Nasby@BlueTreble.com
In reply to: Craig Ringer (#4)
Re: Override compile time log levels of specific messages/modules

On 9/6/16 5:18 AM, Craig Ringer wrote:

I think something automatic that we clearly define as unstable and not
to be relied upon would be preferable. Plus we already have much of the
infrastructure in elog.c as used by errcontext etc.

Actually, I wish this was a straight-up logging level feature, because I
need it all the time when debugging complicated user-level code.
Specifically, I wish there was a GUC that would alter
(client|log)_min_messages upon entering a specific function, either for
just that function or for anything that function subsequently called.
Bonus points if you could also specify a regex that the message text had
to match.

I realize that code-wise that's completely incompatible with what you're
discussing here, but I think the same API could be used (maybe as a
different GUC).
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532) mobile: 512-569-9461

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6Craig Ringer
craig.ringer@2ndquadrant.com
In reply to: Jim Nasby (#5)
Re: Override compile time log levels of specific messages/modules

On 11 Sep. 2016 11:31, "Jim Nasby" <Jim.Nasby@bluetreble.com> wrote:

Actually, I wish this was a straight-up logging level feature, because I

need it all the time when debugging complicated user-level code.
Specifically, I wish there was a GUC that would alter
(client|log)_min_messages upon entering a specific function, either for
just that function or for anything that function subsequently called.

We have that or close to it, but it's restricted for security. IIRC you
can't SET log_min_messages as non superuser including as part of CREATE
FUNCTION ... SET. Presumably because lowering it would let the user hide
activity admins expect to be recorded. Raising it would let them possibly
DoS via log spam - though that argument is rather undermined by the ability
to write plpgsql that does RAISE in a tight loop.

I'd like to be able to let users make logging more detailed but not less,
so they can only SET it to something equal to or more detailed to what
their session has art the start. Should actually be pretty trivial to
implement too.

You can then SET at session level, function level, SET LOCAL, etc. If you
want to control it dynamically via GUC you add calls to your functions to
check a custom user GUC and SET log level accordingly. Using whatever logic
you like.

Bonus points if you could also specify a regex that the message text had

to match.

An errfinish() hook would be nice for that.